From patchwork Mon Dec 18 17:45:44 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 122311 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp3191347qgn; Mon, 18 Dec 2017 10:48:36 -0800 (PST) X-Google-Smtp-Source: ACJfBosnXCBQLRTiCz8LCCM9eqHf61SyamYWRMUlvQehbHZhHgliTEoQtYaR5PmujuB+65rESnTK X-Received: by 10.129.136.135 with SMTP id y129mr607480ywf.418.1513622916115; Mon, 18 Dec 2017 10:48:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1513622916; cv=none; d=google.com; s=arc-20160816; b=Xwy9voAz4niHoovutUvTPcMSNhEQWsAuWldnvIuoGq8vWNMUpQA9iWRsoIKA+mzPBE LfxV+0ErieHNfmNTJpc6IOVhxouL9nx8h0Lik9pGH+L/woBGF6cz5b7fqT8Wj1viOwYD Yu/1qT/VED7EXs+XcXXeNAUwMhCo6wCr8RY96iZiZRZ19T6Jmh4BrRGvmfT8WVr61DB6 iLOX4kWtZyj1kmwpvrgQrnsVPyJiqz4esik52h7oiY1AHgaBkFjKTQp5Ns5sLhqEuUKj MZI5qtfM4aB0VKzCImKuwtBUDzaJKfjnulWuxS9QIC6xGuAqb9IYOWsFu8qQNXBB7JIf QeHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=3KZMEM2yXkoIROfRl/TBnbTATpSIAJgB7Mut5/GGdwo=; b=zQQ4MuF91iZs+mWG0S6qglRca2mxGU+wT9Bn/ONmS+AsOxvaaaW26L4flnwSC5u8tQ 00cEYxbmz9yjmp0q2QKPpLwWH4pi3t73tkgo4Gu6HJju6lm6IkQGnhiIjXk7T44KPYHB k1W6u4qHEXcWhhdL8TZn0jyaimY+bFnCqzh5msN/7QED1RElwOXLF2z6NA/U5E+47wzh RSGL164aNKWtRXcxzoCYqNNampZWfJ5VeHWEBHR0hNbsJVdY+eI+HJRL08d4JLsoRf/g Tv1bM1E/AkEdANShl3KvlKOTB+gr7uB35SCN9xYRGP29tJkpN+SoVtYeaWvKHsO8GUgC GUlA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=WGGZXLZl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id a7si2505463ywb.141.2017.12.18.10.48.36 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 18 Dec 2017 10:48:36 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=WGGZXLZl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53246 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eR0Sp-0005nS-GC for patch@linaro.org; Mon, 18 Dec 2017 13:48:35 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55875) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eQzUb-0008Kt-3p for qemu-devel@nongnu.org; Mon, 18 Dec 2017 12:46:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eQzUZ-0002Ao-Mc for qemu-devel@nongnu.org; Mon, 18 Dec 2017 12:46:21 -0500 Received: from mail-pg0-x243.google.com ([2607:f8b0:400e:c05::243]:37484) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eQzUZ-0002AB-F3 for qemu-devel@nongnu.org; Mon, 18 Dec 2017 12:46:19 -0500 Received: by mail-pg0-x243.google.com with SMTP id y6so9433270pgp.4 for ; Mon, 18 Dec 2017 09:46:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3KZMEM2yXkoIROfRl/TBnbTATpSIAJgB7Mut5/GGdwo=; b=WGGZXLZl+4zA+Sdcu7UlHwIGS/4ae2brUC5BFgwnbQlz2D0CINtF94qL6CrHLWZcmQ fk7GWgrlELjQrsRcgBXSMWoYeil8N8657z9vVNjblG0rdraB3kt+iMRscny4kyaTDSXB 7OFPhzVZZnYpnNZak4kGYR+GywcvaRQjklDFs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3KZMEM2yXkoIROfRl/TBnbTATpSIAJgB7Mut5/GGdwo=; b=szdJCWI1BKJ8gaw2Da67jAK07yuTDb9Y1HqTznMvyJrg4Ye+FRAC5nAzDDv44MyDOI vusVEZ/UPI1KoW+f7lJXDhvAuDe4iC43r5OnxGFBvOu6oWp4IrLREkhcAeJIZFsRe/mu 57EPhX5P3ZoVLRrRu8xDG41q+iMQHrKhfo32D+USyDsCV6r8GQj9N+jhwdFkmiHWYwyQ mkCV/TtE/StozLBFvwLXoSd08LsKukKoz6cKM2pbt5El7AJedd0QGmUVQg2ClQ699m0z d/m8hnqvJiY7kHdolavOlhtCcVy9UZXCG/AjHWTbOJafbPiCAVTHa6szMOIvjU5BVgcc sWFw== X-Gm-Message-State: AKGB3mIO8hoCCibckExJPZGqWm5NgW3H06FZFRSZZA6VGZ938HWf3D77 5a2M7+rdLNm32Wrh+HHcwK3HQS3RvbI= X-Received: by 10.98.64.153 with SMTP id f25mr453439pfd.213.1513619178078; Mon, 18 Dec 2017 09:46:18 -0800 (PST) Received: from cloudburst.twiddle.net (174-21-7-63.tukw.qwest.net. [174.21.7.63]) by smtp.gmail.com with ESMTPSA id t84sm26209657pfe.160.2017.12.18.09.46.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 18 Dec 2017 09:46:16 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 18 Dec 2017 09:45:44 -0800 Message-Id: <20171218174552.18871-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20171218174552.18871-1-richard.henderson@linaro.org> References: <20171218174552.18871-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::243 Subject: [Qemu-devel] [PATCH 15/23] target/arm: Implement SVE Integer Multiply-Add Group X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 18 ++++++++++++++ target/arm/sve_helper.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 27 +++++++++++++++++++++ target/arm/sve.def | 15 ++++++++++++ 4 files changed, 118 insertions(+) -- 2.14.3 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index e9382de300..abed625123 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -339,6 +339,24 @@ DEF_HELPER_FLAGS_4(sve_neg_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_neg_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_neg_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mla_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mla_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mla_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mla_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve_mls_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mls_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mls_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve_mls_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_and_pred, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_bic_pred, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_eor_pred, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 481b3bdefe..8235784a82 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -855,6 +855,64 @@ DO_ZPZI_D(sve_asrd_d, int64_t, DO_ASRD) #undef DO_SHL #undef DO_ASRD +/* Fully general four-operand expander, controlled by a predicate. + */ +#define DO_ZPZZZ(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *va, void *vn, void *vm, \ + void *vg, uint32_t desc) \ +{ \ + intptr_t iv = 0, ib = 0, opr_sz = simd_oprsz(desc); \ + for (iv = ib = 0; iv < opr_sz; iv += 16, ib += 2) { \ + uint16_t pg = *(uint16_t *)(vg + H2(ib)); \ + intptr_t i = 0; \ + do { \ + if (pg & 1) { \ + TYPE nn = *(TYPE *)(vn + iv + H(i)); \ + TYPE mm = *(TYPE *)(vm + iv + H(i)); \ + TYPE aa = *(TYPE *)(va + iv + H(i)); \ + *(TYPE *)(vd + iv + H(i)) = OP(aa, nn, mm); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (pg); \ + } \ +} + +/* Similarly, specialized for 64-bit operands. */ +#define DO_ZPZZZ_D(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *va, void *vn, void *vm, \ + void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; \ + TYPE *d = vd, *a = va, *n = vn, *m = vm; \ + uint8_t *pg = vg; \ + for (i = 0; i < opr_sz; i += 1) { \ + if (pg[H1(i)] & 1) { \ + TYPE aa = a[i], nn = n[i], mm = m[i]; \ + d[i] = OP(aa, nn, mm); \ + } \ + } \ +} + +#define DO_MLA(A, N, M) (A + N * M) +#define DO_MLS(A, N, M) (A - N * M) + +DO_ZPZZZ(sve_mla_b, uint8_t, H1, DO_MLA) +DO_ZPZZZ(sve_mls_b, uint8_t, H1, DO_MLS) + +DO_ZPZZZ(sve_mla_h, uint16_t, H1_2, DO_MLA) +DO_ZPZZZ(sve_mls_h, uint16_t, H1_2, DO_MLS) + +DO_ZPZZZ(sve_mla_s, uint32_t, H1_4, DO_MLA) +DO_ZPZZZ(sve_mls_s, uint32_t, H1_4, DO_MLS) + +DO_ZPZZZ_D(sve_mla_d, uint64_t, DO_MLA) +DO_ZPZZZ_D(sve_mls_d, uint64_t, DO_MLS) + +#undef DO_MLA +#undef DO_MLS +#undef DO_ZPZZZ +#undef DO_ZPZZZ_D + void HELPER(sve_ldr)(CPUARMState *env, void *d, target_ulong addr, uint32_t len) { intptr_t i, len_align = QEMU_ALIGN_DOWN(len, 8); diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 7dbc43fb6e..83793ab169 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -520,6 +520,33 @@ void trans_ASRD(DisasContext *s, arg_rpri_esz *a, uint32_t insn) } } +static void do_zpzzz_ool(DisasContext *s, arg_rprrr_esz *a, + gen_helper_gvec_5 *fn) +{ + unsigned vsz = size_for_gvec(vec_full_reg_size(s)); + tcg_gen_gvec_5_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->ra), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + pred_full_reg_offset(s, a->pg), + vsz, vsz, 0, fn); +} + +#define DO_ZPZZZ(NAME, name) \ +void trans_##NAME(DisasContext *s, arg_rprrr_esz *a, uint32_t insn) \ +{ \ + static gen_helper_gvec_5 * const fns[4] = { \ + gen_helper_sve_##name##_b, gen_helper_sve_##name##_h, \ + gen_helper_sve_##name##_s, gen_helper_sve_##name##_d, \ + }; \ + do_zpzzz_ool(s, a, fns[a->esz]); \ +} + +DO_ZPZZZ(MLA, mla) +DO_ZPZZZ(MLS, mls) + +#undef DO_ZPZZZ + static uint64_t pred_esz_mask[4] = { 0xffffffffffffffffull, 0x5555555555555555ull, 0x1111111111111111ull, 0x0101010101010101ull diff --git a/target/arm/sve.def b/target/arm/sve.def index 955a0275a1..3ae871394c 100644 --- a/target/arm/sve.def +++ b/target/arm/sve.def @@ -45,6 +45,7 @@ &rrr_esz rd rn rm esz &rpr_esz rd pg rn esz &rprr_esz rd pg rn rm esz +&rprrr_esz rd pg rn rm ra esz &rpri_esz rd pg rn imm esz &pred_set rd pat esz i s @@ -62,6 +63,10 @@ @rdn_pg_rm_esz ........ esz:2 ... ... ... pg:3 rm:5 rd:5 &rprr_esz rn=%reg_movprfx @rdm_pg_rn_esz ........ esz:2 ... ... ... pg:3 rn:5 rd:5 &rprr_esz rm=%reg_movprfx +# Three register operand, with governing predicate, vector element size +@rda_pg_rn_rm_esz ........ esz:2 . rm:5 ... pg:3 rn:5 rd:5 &rprrr_esz ra=%reg_movprfx +@rdn_pg_ra_rm_esz ........ esz:2 . rm:5 ... pg:3 ra:5 rd:5 &rprrr_esz rn=%reg_movprfx + # One register operand, with governing predicate, vector element size @rd_pg_rn_esz ........ esz:2 ... ... ... pg:3 rn:5 rd:5 &rpr_esz @@ -168,6 +173,16 @@ UXTH 00000100 .. 010 011 101 ... ..... ..... @rd_pg_rn_esz # Note size > 1 SXTW 00000100 .. 010 100 101 ... ..... ..... @rd_pg_rn_esz # Note size == 3 UXTW 00000100 .. 010 101 101 ... ..... ..... @rd_pg_rn_esz # Note size == 3 +### SVE Integer Multiply-Add Group + +# SVE integer multiply-add writing addend (predicated) +MLA 00000100 .. 0 ..... 010 ... ..... ..... @rda_pg_rn_rm_esz +MLS 00000100 .. 0 ..... 011 ... ..... ..... @rda_pg_rn_rm_esz + +# SVE integer multiply-add writing multiplicand (predicated) +MLA 00000100 .. 0 ..... 110 ... ..... ..... @rdn_pg_ra_rm_esz # MAD +MLS 00000100 .. 0 ..... 111 ... ..... ..... @rdn_pg_ra_rm_esz # MSB + ### SVE Logical - Unpredicated Group # SVE bitwise logical operations (unpredicated)