From patchwork Thu May 14 21:28:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 186788 Delivered-To: patch@linaro.org Received: by 2002:a92:5b0a:0:0:0:0:0 with SMTP id p10csp1770560ilb; Thu, 14 May 2020 14:29:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxkzK6C6VaMaogAe59CoHOxyn6e5kgC+j7oaym5WH1ZKKNHaE9o5bhLfC6JisAvb6dQxSoE X-Received: by 2002:a50:ec8d:: with SMTP id e13mr122927edr.8.1589491755552; Thu, 14 May 2020 14:29:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1589491755; cv=none; d=google.com; s=arc-20160816; b=UlwjtB30uzpQ6JUVm+tqdsqmOcHEcvKnSDe3kS8PmM/GYwyRbkhDDQcDsbkCqKvYfX YywF0I5Yq+gY6HdDq1L/RoKy7okyKuPS0rvdu7DHDu2cwFotF6qGH/HcNA7pvCocMzG0 DKba96fx/5inFFLG0SRr2I63H1tpnr1JRFZsROhv60ZzLlICHJk/PbDAsHezVR6hjMRK BO/OXDPVEaXPXfng6S2+RJKXQhGh7dFqQXyv277yOMUvUYCf2VPpwnWIxQrUVTL8MqxD Xt0SaUJVUFrdxE0HxD4NaIJIJ/VGa7ES20ku1IhPNlHpSw4S40Fzzw/KkRg5SQASstxl m0ZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=gQ13Nokc2QqY1T6dZahuQFzqk90ggUWz3QNTj8ciFBs=; b=Gc9iYXpvjqLVXFvrjW+3rbYhkh76fAqbAH2qz/A/70qBgzdWDcm0WKQ0XyqqWGlEsQ 0TWCprHFsfuCuASQTnnjOzyG5PlnjyRiwpgyzGGdiAUnJYyKLH7gd0hnuzoZ818ld13j F/Wd+KqpNikrG9rNr8OQAXzaiSBlEX5wz34rrIfcs4MskXPWb0kF4yNhS3Iy+QCPsjCb buPgPqL2LtswjgKQAi/XBnqybfbEjSv26c1gQGX60gADt9yEhQyVKFlVvjZBolzD/TIu SLRo2xYVE1DDXk7mIjRYBv6i6oKtyMfMrU/lLtGXgrsaVAUdl46/bdywGmo0bwAiHmi3 wHfQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=sp8QJF6U; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l11si55041eji.477.2020.05.14.14.29.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 May 2020 14:29:15 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=sp8QJF6U; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59400 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jZLPm-000454-B8 for patch@linaro.org; Thu, 14 May 2020 17:29:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49148) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jZLPC-00041Y-2v for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:38 -0400 Received: from mail-pj1-x1042.google.com ([2607:f8b0:4864:20::1042]:35439) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jZLPA-0004bN-Jy for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:37 -0400 Received: by mail-pj1-x1042.google.com with SMTP id ms17so31394pjb.0 for ; Thu, 14 May 2020 14:28:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gQ13Nokc2QqY1T6dZahuQFzqk90ggUWz3QNTj8ciFBs=; b=sp8QJF6UJc8C8nzSrdwlq92S8lxC8kOzkY0LUKd2U0GL2ZUJOlmeGWS7yJ+7geNOel xhoJJH0lY4Lbiwr7IRv6AEnpID7JV7YLH3AtZ5+D+8Ary391OcH2A+3PSYkmaQFvXge6 qV6welVO3PWIrKGk9uKI+/78QT3M+G81bCQP/YZL0rEp0bR9uq6HcFHo6dBuNt3DQwZp oJy4kdyuiY4NGhDYiwOtG326aGphaKD91hMyERz5ZO4FlX0Cp9JICV42ZaNPL0cIjyj9 eg3QTI8ekqa6smr+5SntSj27WaQ9DNTxmph37PGBkUwC6vnr5+eCsYRFMHfDjO5yLgX3 Dt0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gQ13Nokc2QqY1T6dZahuQFzqk90ggUWz3QNTj8ciFBs=; b=faccT1WMnLMhv6NcYWedMJlFdWeU4G5HUuc0TzChc74meMAohuPPlToVgZ5bHbaZvJ iOCsDXZ+6tzQosqfCULQmsImpqcoZXSGkykhpgp0o+k8TFJ9WX0YoKW9QdJCPVzYD2RG iGx3T9R/QT0JZnsEJ3AM7axxp5IStNPNxvZkcOIR7C470uTXL8JN80aJbxs11/hyEr3T lPBw6isiyxiP4JioHFsq0zBnhrKWzALlI1uJ2SzQWAKYpysQh2ZT42FjD6dUKn8KPyyV RZHB0pR5hwGhEc/Afc4j+9bqM3JSjH0hTkOL1O+WTf1Xxw4ZcARMFF26EOxKX+XpWFKm mgSQ== X-Gm-Message-State: AGi0Pubojacns7jlFzccSiKpmp4PBYYaoCNb9czzeSP6I8x1ALyN990X UIRNv7Np1nFnvWbyyBY+Fnt1jX75uDY= X-Received: by 2002:a17:90a:ea07:: with SMTP id w7mr42606511pjy.172.1589491714567; Thu, 14 May 2020 14:28:34 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id i7sm4986pjj.33.2020.05.14.14.28.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2020 14:28:33 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH 1/6] target/arm: Convert aes and sm4 to gvec helpers Date: Thu, 14 May 2020 14:28:26 -0700 Message-Id: <20200514212831.31248-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200514212831.31248-1-richard.henderson@linaro.org> References: <20200514212831.31248-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1042; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1042.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" With this conversion, we will be able to use the same helpers with sve. In particular, pass 3 vector parameters for the 3-operand operations; for advsimd the destination register is also an input. This also fixes a bug in which we failed to clear the high bits of the SVE register after an AdvSIMD operation. Signed-off-by: Richard Henderson --- target/arm/helper.h | 6 ++-- target/arm/vec_internal.h | 33 +++++++++++++++++ target/arm/crypto_helper.c | 72 +++++++++++++++++++++++++++----------- target/arm/translate-a64.c | 55 ++++++++++++++++++----------- target/arm/translate.c | 27 +++++++------- target/arm/vec_helper.c | 12 +------ 6 files changed, 138 insertions(+), 67 deletions(-) create mode 100644 target/arm/vec_internal.h -- 2.20.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 49336dc432..42759f82aa 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -510,7 +510,7 @@ DEF_HELPER_FLAGS_2(neon_qzip8, TCG_CALL_NO_RWG, void, ptr, ptr) DEF_HELPER_FLAGS_2(neon_qzip16, TCG_CALL_NO_RWG, void, ptr, ptr) DEF_HELPER_FLAGS_2(neon_qzip32, TCG_CALL_NO_RWG, void, ptr, ptr) -DEF_HELPER_FLAGS_3(crypto_aese, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_aese, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_3(crypto_aesmc, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_4(crypto_sha1_3reg, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) @@ -531,8 +531,8 @@ DEF_HELPER_FLAGS_5(crypto_sm3tt, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32, i32) DEF_HELPER_FLAGS_3(crypto_sm3partw1, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) DEF_HELPER_FLAGS_3(crypto_sm3partw2, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) -DEF_HELPER_FLAGS_2(crypto_sm4e, TCG_CALL_NO_RWG, void, ptr, ptr) -DEF_HELPER_FLAGS_3(crypto_sm4ekey, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) +DEF_HELPER_FLAGS_4(crypto_sm4e, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sm4ekey, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_3(crc32, TCG_CALL_NO_RWG_SE, i32, i32, i32, i32) DEF_HELPER_FLAGS_3(crc32c, TCG_CALL_NO_RWG_SE, i32, i32, i32, i32) diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h new file mode 100644 index 0000000000..00a8277765 --- /dev/null +++ b/target/arm/vec_internal.h @@ -0,0 +1,33 @@ +/* + * ARM AdvSIMD / SVE Vector Helpers + * + * Copyright (c) 2020 Linaro + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see . + */ + +#ifndef TARGET_ARM_VEC_INTERNALS_H +#define TARGET_ARM_VEC_INTERNALS_H + +static inline void clear_tail(void *vd, uintptr_t opr_sz, uintptr_t max_sz) +{ + uint64_t *d = vd + opr_sz; + uintptr_t i; + + for (i = opr_sz; i < max_sz; i += 8) { + *d++ = 0; + } +} + +#endif /* TARGET_ARM_VEC_INTERNALS_H */ diff --git a/target/arm/crypto_helper.c b/target/arm/crypto_helper.c index f800266727..6bd5a3d2d0 100644 --- a/target/arm/crypto_helper.c +++ b/target/arm/crypto_helper.c @@ -13,7 +13,9 @@ #include "cpu.h" #include "exec/helper-proto.h" +#include "tcg/tcg-gvec-desc.h" #include "crypto/aes.h" +#include "vec_internal.h" union CRYPTO_STATE { uint8_t bytes[16]; @@ -29,18 +31,15 @@ union CRYPTO_STATE { #define CR_ST_WORD(state, i) (state.words[i]) #endif -void HELPER(crypto_aese)(void *vd, void *vm, uint32_t decrypt) +static void do_crypto_aese(uint64_t *rd, uint64_t *rn, + uint64_t *rm, bool decrypt) { static uint8_t const * const sbox[2] = { AES_sbox, AES_isbox }; static uint8_t const * const shift[2] = { AES_shifts, AES_ishifts }; - uint64_t *rd = vd; - uint64_t *rm = vm; union CRYPTO_STATE rk = { .l = { rm[0], rm[1] } }; - union CRYPTO_STATE st = { .l = { rd[0], rd[1] } }; + union CRYPTO_STATE st = { .l = { rn[0], rn[1] } }; int i; - assert(decrypt < 2); - /* xor state vector with round key */ rk.l[0] ^= st.l[0]; rk.l[1] ^= st.l[1]; @@ -54,7 +53,18 @@ void HELPER(crypto_aese)(void *vd, void *vm, uint32_t decrypt) rd[1] = st.l[1]; } -void HELPER(crypto_aesmc)(void *vd, void *vm, uint32_t decrypt) +void HELPER(crypto_aese)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + bool decrypt = simd_data(desc); + + for (i = 0; i < opr_sz; i += 16) { + do_crypto_aese(vd + i, vn + i, vm + i, decrypt); + } + clear_tail(vd, opr_sz, simd_maxsz(desc)); +} + +static void do_crypto_aesmc(uint64_t *rd, uint64_t *rm, bool decrypt) { static uint32_t const mc[][256] = { { /* MixColumns lookup table */ @@ -190,13 +200,9 @@ void HELPER(crypto_aesmc)(void *vd, void *vm, uint32_t decrypt) 0xbe805d9f, 0xb58d5491, 0xa89a4f83, 0xa397468d, } }; - uint64_t *rd = vd; - uint64_t *rm = vm; union CRYPTO_STATE st = { .l = { rm[0], rm[1] } }; int i; - assert(decrypt < 2); - for (i = 0; i < 16; i += 4) { CR_ST_WORD(st, i >> 2) = mc[decrypt][CR_ST_BYTE(st, i)] ^ @@ -209,6 +215,17 @@ void HELPER(crypto_aesmc)(void *vd, void *vm, uint32_t decrypt) rd[1] = st.l[1]; } +void HELPER(crypto_aesmc)(void *vd, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + bool decrypt = simd_data(desc); + + for (i = 0; i < opr_sz; i += 16) { + do_crypto_aesmc(vd + i, vm + i, decrypt); + } + clear_tail(vd, opr_sz, simd_maxsz(desc)); +} + /* * SHA-1 logical functions */ @@ -638,12 +655,10 @@ static uint8_t const sm4_sbox[] = { 0x79, 0xee, 0x5f, 0x3e, 0xd7, 0xcb, 0x39, 0x48, }; -void HELPER(crypto_sm4e)(void *vd, void *vn) +static void do_crypto_sm4e(uint64_t *rd, uint64_t *rn, uint64_t *rm) { - uint64_t *rd = vd; - uint64_t *rn = vn; - union CRYPTO_STATE d = { .l = { rd[0], rd[1] } }; - union CRYPTO_STATE n = { .l = { rn[0], rn[1] } }; + union CRYPTO_STATE d = { .l = { rn[0], rn[1] } }; + union CRYPTO_STATE n = { .l = { rm[0], rm[1] } }; uint32_t t, i; for (i = 0; i < 4; i++) { @@ -665,11 +680,18 @@ void HELPER(crypto_sm4e)(void *vd, void *vn) rd[1] = d.l[1]; } -void HELPER(crypto_sm4ekey)(void *vd, void *vn, void* vm) +void HELPER(crypto_sm4e)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + + for (i = 0; i < opr_sz; i += 16) { + do_crypto_sm4e(vd + i, vn + i, vm + i); + } + clear_tail(vd, opr_sz, simd_maxsz(desc)); +} + +static void do_crypto_sm4ekey(uint64_t *rd, uint64_t *rn, uint64_t *rm) { - uint64_t *rd = vd; - uint64_t *rn = vn; - uint64_t *rm = vm; union CRYPTO_STATE d; union CRYPTO_STATE n = { .l = { rn[0], rn[1] } }; union CRYPTO_STATE m = { .l = { rm[0], rm[1] } }; @@ -693,3 +715,13 @@ void HELPER(crypto_sm4ekey)(void *vd, void *vn, void* vm) rd[0] = d.l[0]; rd[1] = d.l[1]; } + +void HELPER(crypto_sm4ekey)(void *vd, void *vn, void* vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + + for (i = 0; i < opr_sz; i += 16) { + do_crypto_sm4ekey(vd + i, vn + i, vm + i); + } + clear_tail(vd, opr_sz, simd_maxsz(desc)); +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 991e451644..1e511529b8 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -577,6 +577,15 @@ static void gen_gvec_fn4(DisasContext *s, bool is_q, int rd, int rn, int rm, is_q ? 16 : 8, vec_full_reg_size(s)); } +/* Expand a 2-operand operation using an out-of-line helper. */ +static void gen_gvec_op2_ool(DisasContext *s, bool is_q, int rd, + int rn, int data, gen_helper_gvec_2 *fn) +{ + tcg_gen_gvec_2_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + is_q ? 16 : 8, vec_full_reg_size(s), data, fn); +} + /* Expand a 3-operand operation using an out-of-line helper. */ static void gen_gvec_op3_ool(DisasContext *s, bool is_q, int rd, int rn, int rm, int data, gen_helper_gvec_3 *fn) @@ -13398,9 +13407,8 @@ static void disas_crypto_aes(DisasContext *s, uint32_t insn) int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); int decrypt; - TCGv_ptr tcg_rd_ptr, tcg_rn_ptr; - TCGv_i32 tcg_decrypt; - CryptoThreeOpIntFn *genfn; + gen_helper_gvec_2 *genfn2 = NULL; + gen_helper_gvec_3 *genfn3 = NULL; if (!dc_isar_feature(aa64_aes, s) || size != 0) { unallocated_encoding(s); @@ -13410,19 +13418,19 @@ static void disas_crypto_aes(DisasContext *s, uint32_t insn) switch (opcode) { case 0x4: /* AESE */ decrypt = 0; - genfn = gen_helper_crypto_aese; + genfn3 = gen_helper_crypto_aese; break; case 0x6: /* AESMC */ decrypt = 0; - genfn = gen_helper_crypto_aesmc; + genfn2 = gen_helper_crypto_aesmc; break; case 0x5: /* AESD */ decrypt = 1; - genfn = gen_helper_crypto_aese; + genfn3 = gen_helper_crypto_aese; break; case 0x7: /* AESIMC */ decrypt = 1; - genfn = gen_helper_crypto_aesmc; + genfn2 = gen_helper_crypto_aesmc; break; default: unallocated_encoding(s); @@ -13432,16 +13440,11 @@ static void disas_crypto_aes(DisasContext *s, uint32_t insn) if (!fp_access_check(s)) { return; } - - tcg_rd_ptr = vec_full_reg_ptr(s, rd); - tcg_rn_ptr = vec_full_reg_ptr(s, rn); - tcg_decrypt = tcg_const_i32(decrypt); - - genfn(tcg_rd_ptr, tcg_rn_ptr, tcg_decrypt); - - tcg_temp_free_ptr(tcg_rd_ptr); - tcg_temp_free_ptr(tcg_rn_ptr); - tcg_temp_free_i32(tcg_decrypt); + if (genfn2) { + gen_gvec_op2_ool(s, true, rd, rn, decrypt, genfn2); + } else { + gen_gvec_op3_ool(s, true, rd, rd, rn, decrypt, genfn3); + } } /* Crypto three-reg SHA @@ -13590,7 +13593,8 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); bool feature; - CryptoThreeOpFn *genfn; + CryptoThreeOpFn *genfn = NULL; + gen_helper_gvec_3 *oolfn = NULL; if (o == 0) { switch (opcode) { @@ -13625,7 +13629,7 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) break; case 2: /* SM4EKEY */ feature = dc_isar_feature(aa64_sm4, s); - genfn = gen_helper_crypto_sm4ekey; + oolfn = gen_helper_crypto_sm4ekey; break; default: unallocated_encoding(s); @@ -13642,6 +13646,11 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) return; } + if (oolfn) { + gen_gvec_op3_ool(s, true, rd, rn, rm, 0, oolfn); + return; + } + if (genfn) { TCGv_ptr tcg_rd_ptr, tcg_rn_ptr, tcg_rm_ptr; @@ -13694,6 +13703,7 @@ static void disas_crypto_two_reg_sha512(DisasContext *s, uint32_t insn) TCGv_ptr tcg_rd_ptr, tcg_rn_ptr; bool feature; CryptoTwoOpFn *genfn; + gen_helper_gvec_3 *oolfn = NULL; switch (opcode) { case 0: /* SHA512SU0 */ @@ -13702,7 +13712,7 @@ static void disas_crypto_two_reg_sha512(DisasContext *s, uint32_t insn) break; case 1: /* SM4E */ feature = dc_isar_feature(aa64_sm4, s); - genfn = gen_helper_crypto_sm4e; + oolfn = gen_helper_crypto_sm4e; break; default: unallocated_encoding(s); @@ -13718,6 +13728,11 @@ static void disas_crypto_two_reg_sha512(DisasContext *s, uint32_t insn) return; } + if (oolfn) { + gen_gvec_op3_ool(s, true, rd, rd, rn, 0, oolfn); + return; + } + tcg_rd_ptr = vec_full_reg_ptr(s, rd); tcg_rn_ptr = vec_full_reg_ptr(s, rn); diff --git a/target/arm/translate.c b/target/arm/translate.c index 4c9bb8b5ac..921359dfd4 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -6373,22 +6373,23 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) if (!dc_isar_feature(aa32_aes, s) || ((rm | rd) & 1)) { return 1; } - ptr1 = vfp_reg_ptr(true, rd); - ptr2 = vfp_reg_ptr(true, rm); - - /* Bit 6 is the lowest opcode bit; it distinguishes between - * encryption (AESE/AESMC) and decryption (AESD/AESIMC) - */ - tmp3 = tcg_const_i32(extract32(insn, 6, 1)); - + /* + * Bit 6 is the lowest opcode bit; it distinguishes + * between encryption (AESE/AESMC) and decryption + * (AESD/AESIMC). + */ if (op == NEON_2RM_AESE) { - gen_helper_crypto_aese(ptr1, ptr2, tmp3); + tcg_gen_gvec_3_ool(vfp_reg_offset(true, rd), + vfp_reg_offset(true, rd), + vfp_reg_offset(true, rm), + 16, 16, extract32(insn, 6, 1), + gen_helper_crypto_aese); } else { - gen_helper_crypto_aesmc(ptr1, ptr2, tmp3); + tcg_gen_gvec_2_ool(vfp_reg_offset(true, rd), + vfp_reg_offset(true, rm), + 16, 16, extract32(insn, 6, 1), + gen_helper_crypto_aesmc); } - tcg_temp_free_ptr(ptr1); - tcg_temp_free_ptr(ptr2); - tcg_temp_free_i32(tmp3); break; case NEON_2RM_SHA1H: if (!dc_isar_feature(aa32_sha1, s) || ((rm | rd) & 1)) { diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 50a499299f..7d76412ee0 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -22,7 +22,7 @@ #include "exec/helper-proto.h" #include "tcg/tcg-gvec-desc.h" #include "fpu/softfloat.h" - +#include "vec_internal.h" /* Note that vector data is stored in host-endian 64-bit chunks, so addressing units smaller than that needs a host-endian fixup. */ @@ -36,16 +36,6 @@ #define H4(x) (x) #endif -static void clear_tail(void *vd, uintptr_t opr_sz, uintptr_t max_sz) -{ - uint64_t *d = vd + opr_sz; - uintptr_t i; - - for (i = opr_sz; i < max_sz; i += 8) { - *d++ = 0; - } -} - /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */ static int16_t inl_qrdmlah_s16(int16_t src1, int16_t src2, int16_t src3, uint32_t *sat) From patchwork Thu May 14 21:28:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 186789 Delivered-To: patch@linaro.org Received: by 2002:a92:5b0a:0:0:0:0:0 with SMTP id p10csp1770577ilb; Thu, 14 May 2020 14:29:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz0Q9tyjlXbfnhB3kScpQTWy9MmIAXsGrNea2WvCjbRAy2nz1AVUfQEsI6IPAmlXYl0ZpVy X-Received: by 2002:ac8:544e:: with SMTP id d14mr201060qtq.283.1589491757173; Thu, 14 May 2020 14:29:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1589491757; cv=none; d=google.com; s=arc-20160816; b=BVUwx7DXOkOcnBmyk5pZoGh2FoZyIS+SB7ekGNQWs8HV7p2qZ+kBVd/4m69F15usAb YpppMyRi4vaYG3MF7qB7j/2lM6vecJHGX6/jaY/NXoA2Sqj+TvWrY3NhoHxPLAo1bWw8 cnvZ0aePnfjMoMe/mWYnwqpKsWLRfPuHOOVj7Xd1Hipz4zeNQO1e9cIvTCFkVXprcn+4 s/b5x2qN9/CpS/OXIHyzlZatAyHpnkOUcyQbZQ2GmaeKWiNChVqUmkLdYD25848rpyTo 6vVR7r1LzboXgTO/euALcKHKdkxFUgQDLRqrdUD6kkwULM/xRaybpqe0IqdPmrYzwtnR nQCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=QtavOgtJbqCDtFoMyh2yfPlecYMuCnzasezS0T2jFho=; b=xYByenWOKbr978dtfojJEIx6PFG8a54jJmyOzUN5WW1wSxStPEaC/4tDgI95O3DR81 3aakkk0EgeJIKAZYElBnNlkXflLlbt6BUewzNYwzUPXHO0PQVegDphgO/jRXS7GdH43/ nkEUAgwIQpuuR1GGdduugreNz/DMv2smKtdkprYvqbiQ7jUth2EeP4UtsLEMJ4JMcyRP UHaFh0svpYW2dFMg8S7/Zl55aQ+iZawcQvsRu/J5Dxlzm1R/zcVqstzGHckI/TkFvLrn uj8Qfg47N4Hrlk8QpZA8Y/7nChsQnzfw/O473ILeMWu+kSUbNFN4AozvbGyQDZuBjgZ5 drxw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Nrm1EtpZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t51si237064qtj.31.2020.05.14.14.29.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 May 2020 14:29:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Nrm1EtpZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59512 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jZLPm-00049P-UV for patch@linaro.org; Thu, 14 May 2020 17:29:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49150) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jZLPC-000439-Sv for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:38 -0400 Received: from mail-pf1-x441.google.com ([2607:f8b0:4864:20::441]:35678) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jZLPB-0004bU-VR for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:38 -0400 Received: by mail-pf1-x441.google.com with SMTP id n18so1958023pfa.2 for ; Thu, 14 May 2020 14:28:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QtavOgtJbqCDtFoMyh2yfPlecYMuCnzasezS0T2jFho=; b=Nrm1EtpZVTExEUl3ZJBZibMZF9mY+G7rJeCITqTr1OD6vv+8KVGcgFHxmPQnSVWjbJ vBCCweGupVA5Oqy2OlgU+zDforbVZE4ElCoORNLbeLyUG146wSxo/vJfScRvdYUl3uCy b8zMS5fVXXh9wnFa97u+lZ+AZIvZZ9WNggv6SXCNnZeikEZ2dlT4qQrD9bvJjW81yzcm Vi5Dk19qefQBMRQM/LJY3Tp14wwU7xvxlx0+mtdlOX47JNiPSCEuOWp3rgtaBxM9b6wc RqVK6ne74Sz58zJhhuaAToZFFpd8oOvsCmCWDw5ArZp0AAxnJn638ctNQFyZ4hGJ+EwV PyNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QtavOgtJbqCDtFoMyh2yfPlecYMuCnzasezS0T2jFho=; b=uEc8gsPHrSZkA/G5yZGXtq2LdDZ+jT16EKIeoMuqFA1LDCmUGwjCDMa+OX31G7XSPZ /VW4S1JcgwRH1LixgJlDpOKANOS7tuZYB4MveRI2L5oqxKBad0x3eMRxoG24KAtI5XoU wCru6L/H9Op252K9O/J/E0BXTsjJoS1hfqzbDMf5sGL2mssZQdE6C8CbHGeQ84VRCh0c LCbysQ26bwhtq1DfIbwyMeg3oy19C0t8ry4HuP2lSdmavLE/BwHaGxlgcE47SuyNvBAE hh4UylPCmQXlFER5ZiK+6auQd8psny6vBvbHYvVACQs0maoCX/MDZLPFBXtGz58Wp+/4 hP+Q== X-Gm-Message-State: AOAM531We2SSSkFLyLo8vjpMHDt3CUgo35EpYWuXIL0YJV65e5oaZxp5 wuhyfWzyN6NTmGvmmDaP+JyXadF1bNQ= X-Received: by 2002:aa7:9096:: with SMTP id i22mr555981pfa.250.1589491716065; Thu, 14 May 2020 14:28:36 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id i7sm4986pjj.33.2020.05.14.14.28.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2020 14:28:35 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH 2/6] target/arm: Convert rax1 to gvec helpers Date: Thu, 14 May 2020 14:28:27 -0700 Message-Id: <20200514212831.31248-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200514212831.31248-1-richard.henderson@linaro.org> References: <20200514212831.31248-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::441; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x441.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" With this conversion, we will be able to use the same helpers with sve. This also fixes a bug in which we failed to clear the high bits of the SVE register after an AdvSIMD operation. Signed-off-by: Richard Henderson --- target/arm/helper.h | 2 ++ target/arm/translate-a64.h | 3 ++ target/arm/crypto_helper.c | 11 +++++++ target/arm/translate-a64.c | 59 ++++++++++++++++++++------------------ 4 files changed, 47 insertions(+), 28 deletions(-) -- 2.20.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 42759f82aa..6c4eb9befb 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -534,6 +534,8 @@ DEF_HELPER_FLAGS_3(crypto_sm3partw2, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) DEF_HELPER_FLAGS_4(crypto_sm4e, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(crypto_sm4ekey, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_rax1, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(crc32, TCG_CALL_NO_RWG_SE, i32, i32, i32, i32) DEF_HELPER_FLAGS_3(crc32c, TCG_CALL_NO_RWG_SE, i32, i32, i32, i32) diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h index f02fbb63a4..da0f59a2ce 100644 --- a/target/arm/translate-a64.h +++ b/target/arm/translate-a64.h @@ -115,4 +115,7 @@ static inline int vec_full_reg_size(DisasContext *s) bool disas_sve(DisasContext *, uint32_t); +void gen_gvec_rax1(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz); + #endif /* TARGET_ARM_TRANSLATE_A64_H */ diff --git a/target/arm/crypto_helper.c b/target/arm/crypto_helper.c index 6bd5a3d2d0..372d8350e4 100644 --- a/target/arm/crypto_helper.c +++ b/target/arm/crypto_helper.c @@ -725,3 +725,14 @@ void HELPER(crypto_sm4ekey)(void *vd, void *vn, void* vm, uint32_t desc) } clear_tail(vd, opr_sz, simd_maxsz(desc)); } + +void HELPER(crypto_rax1)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = n[i] ^ rol64(m[i], 1); + } + clear_tail(vd, opr_sz, simd_maxsz(desc)); +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 1e511529b8..4d7a8fd2bb 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -13579,6 +13579,32 @@ static void disas_crypto_two_reg_sha(DisasContext *s, uint32_t insn) tcg_temp_free_ptr(tcg_rn_ptr); } +static void gen_rax1_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m) +{ + tcg_gen_rotli_i64(d, m, 1); + tcg_gen_xor_i64(d, d, n); +} + +static void gen_rax1_vec(unsigned vece, TCGv_vec d, TCGv_vec n, TCGv_vec m) +{ + tcg_gen_rotli_vec(vece, d, m, 1); + tcg_gen_xor_vec(vece, d, d, n); +} + +void gen_gvec_rax1(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop_list[] = { INDEX_op_rotli_vec, 0 }; + static const GVecGen3 op = { + .fni8 = gen_rax1_i64, + .fniv = gen_rax1_vec, + .opt_opc = vecop_list, + .fno = gen_helper_crypto_rax1, + .vece = MO_64, + }; + tcg_gen_gvec_3(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, &op); +} + /* Crypto three-reg SHA512 * 31 21 20 16 15 14 13 12 11 10 9 5 4 0 * +-----------------------+------+---+---+-----+--------+------+------+ @@ -13595,6 +13621,7 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) bool feature; CryptoThreeOpFn *genfn = NULL; gen_helper_gvec_3 *oolfn = NULL; + GVecGen3Fn *gvecfn = NULL; if (o == 0) { switch (opcode) { @@ -13612,7 +13639,7 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) break; case 3: /* RAX1 */ feature = dc_isar_feature(aa64_sha3, s); - genfn = NULL; + gvecfn = gen_gvec_rax1; break; default: g_assert_not_reached(); @@ -13648,10 +13675,9 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) if (oolfn) { gen_gvec_op3_ool(s, true, rd, rn, rm, 0, oolfn); - return; - } - - if (genfn) { + } else if (gvecfn) { + gen_gvec_fn3(s, true, rd, rn, rm, gvecfn, MO_64); + } else { TCGv_ptr tcg_rd_ptr, tcg_rn_ptr, tcg_rm_ptr; tcg_rd_ptr = vec_full_reg_ptr(s, rd); @@ -13663,29 +13689,6 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) tcg_temp_free_ptr(tcg_rd_ptr); tcg_temp_free_ptr(tcg_rn_ptr); tcg_temp_free_ptr(tcg_rm_ptr); - } else { - TCGv_i64 tcg_op1, tcg_op2, tcg_res[2]; - int pass; - - tcg_op1 = tcg_temp_new_i64(); - tcg_op2 = tcg_temp_new_i64(); - tcg_res[0] = tcg_temp_new_i64(); - tcg_res[1] = tcg_temp_new_i64(); - - for (pass = 0; pass < 2; pass++) { - read_vec_element(s, tcg_op1, rn, pass, MO_64); - read_vec_element(s, tcg_op2, rm, pass, MO_64); - - tcg_gen_rotli_i64(tcg_res[pass], tcg_op2, 1); - tcg_gen_xor_i64(tcg_res[pass], tcg_res[pass], tcg_op1); - } - write_vec_element(s, tcg_res[0], rd, 0, MO_64); - write_vec_element(s, tcg_res[1], rd, 1, MO_64); - - tcg_temp_free_i64(tcg_op1); - tcg_temp_free_i64(tcg_op2); - tcg_temp_free_i64(tcg_res[0]); - tcg_temp_free_i64(tcg_res[1]); } } From patchwork Thu May 14 21:28:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 186790 Delivered-To: patch@linaro.org Received: by 2002:a92:5b0a:0:0:0:0:0 with SMTP id p10csp1771863ilb; Thu, 14 May 2020 14:31:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyj/JMrL8E7Ll5gHSpkPy77bDsma/fPsdaPHmLdi+T0qtc0LK9FBBEutyuUbykt2PUS5cLQ X-Received: by 2002:a37:dc05:: with SMTP id v5mr372207qki.315.1589491860507; Thu, 14 May 2020 14:31:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1589491860; cv=none; d=google.com; s=arc-20160816; b=XoeB/N3Lj6+MOCZG4X0x6FoVisuRqzcaXeqySUjsSMoWP3sNAEheYsGmfwuOCVCrGs cS7R0xw+CaB40UVA5/zyLSAFI1MOSgo1YkbwzFVlLasScvFKwJyfczUKyQOPiL4J9Arm /IZnsN+fugBVh6oSHj6TBfDzjbumRljRAN/nxsQixAc1PuQ0/l365yfn83tU1wUk2TwG ekL1z+NxVXY6IV9upJVCw504RWujSYV6LJrhfv9EDjG09fZFVShtSXa+Md4B44fmIi2H GCslmoAB+2tvZ1MJ4TrdjJno1YMkn2S7rizEDXY+eAPBx22MEkTOjbaT+ePxwdKwn7VP CrUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=6Qx4JXdZERj9fc65yHo5SmGKN/7PZDlsjtZIgn+V0Bc=; b=eu5DohcVfmUgYl/dfJ2AuB8U32uWxsoa3E2oK0i4C4G2s9Oc3REaj6HsFfc1IfeqfZ +c9wD+o6VYWN8NNbQT/FG7w9pkADbkEHwXRhYDzbcfjXP3V8nDT1Kcrv7jImgeslvxIm HIvph8TLfgQ6l/tw73LYU7N6SfhUTYgA/6qu//xwPzvStavm/eJvfJsdjpzOIm+LOB7C Qxbtm7yDnKyH5M2HlxToiG0S6HoO6RiSmMF1CzJm9Uv5KbLbrtdth9gt0Zak3rtJ2zHl hCKpNtBfGgCbmqDKE/9O1ebJ93w0ZiXP9fEAeb6O35UQyxTuUxJBE0eMIcbA92uB2zK1 5LCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="SdO2r/xn"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v22si122948qka.27.2020.05.14.14.31.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 May 2020 14:31:00 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="SdO2r/xn"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37322 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jZLRS-0006qo-AF for patch@linaro.org; Thu, 14 May 2020 17:30:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49162) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jZLPE-00046u-HZ for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:40 -0400 Received: from mail-pg1-x543.google.com ([2607:f8b0:4864:20::543]:34330) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jZLPD-0004bb-Ak for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:40 -0400 Received: by mail-pg1-x543.google.com with SMTP id f6so1835837pgm.1 for ; Thu, 14 May 2020 14:28:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6Qx4JXdZERj9fc65yHo5SmGKN/7PZDlsjtZIgn+V0Bc=; b=SdO2r/xn78eH+N8w09IwodBE4fomQ8wOWWnvNso7668iF5CUznlVnoceyp+eQkZeDv NidEEWfLGWVhnLgDh9o0F/WMbH4i4My9BQ17mq5fnSk+FdbpPGPL8bNe/Vxg7R18cwaY FbMN/TiJHfTgqFssBcsv/pxltN4tzEDHDAmQabVshRSinSAJamKmuXfL3JlT9R9wyYCO lmCzUQeaPrdYTXKPELcryTWg53BIHDTGUUYDhu4I4ux0JSG3PAuqexjcaRAa2sV06xdj h+12BAtmy5wNb4bNeVnDiXwoYYDBOrTZNdY90XBlyXFt3F9NCe5D7aVSI4lF+iND0L2O kF0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6Qx4JXdZERj9fc65yHo5SmGKN/7PZDlsjtZIgn+V0Bc=; b=dD8odQDh3AJVfEWG0RzXIIt381RIfPZfFdn94jSmoJIjfP3ao0c+Rxn+Fy4i26LyJ5 qNAcYSdsU3zYStBw0HsjYklnPiIXNYiEYOuwBBhYELq8Q8sVYUXqmYXB302gOQ2VqgPd 9dMwuSREhsM3VW3eFi+BhfnZ35EZddmW5q/FgezZnk3XsyGQRYqc4faRu1+y9fYCyNET PlLysi2d9ZZdeIPEtU8l5VW2UtiX4CIdxOPcWvJmvq/n/XgM2r+uI/acTBvoCAaKxqem HbWAFBSlvTObPy++UXMoc9bnPCiGkmjW+QcvHQNAInlAuOckPZB6tc3bTcHPw0KhpFKQ paow== X-Gm-Message-State: AOAM533+5WHn810gr84lgNGWuVBa/sh9jqX3bsQpyNA6kmpMuX9hlkIb BzTiJc/7338PQrPVUUzafAQrZoa77+k= X-Received: by 2002:a62:e80e:: with SMTP id c14mr508182pfi.83.1589491717396; Thu, 14 May 2020 14:28:37 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id i7sm4986pjj.33.2020.05.14.14.28.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2020 14:28:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH 3/6] target/arm: Convert sha512 and sm3 to gvec helpers Date: Thu, 14 May 2020 14:28:28 -0700 Message-Id: <20200514212831.31248-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200514212831.31248-1-richard.henderson@linaro.org> References: <20200514212831.31248-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::543; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x543.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Do not yet convert the helpers to loop over opr_sz, but the descriptor allows the vector tail to be cleared. Which fixes an existing bug vs SVE. Signed-off-by: Richard Henderson --- target/arm/helper.h | 15 +++++++----- target/arm/crypto_helper.c | 37 +++++++++++++++++++++++----- target/arm/translate-a64.c | 50 ++++++++++++-------------------------- 3 files changed, 55 insertions(+), 47 deletions(-) -- 2.20.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 6c4eb9befb..784dc29ce2 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -522,14 +522,17 @@ DEF_HELPER_FLAGS_3(crypto_sha256h2, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) DEF_HELPER_FLAGS_2(crypto_sha256su0, TCG_CALL_NO_RWG, void, ptr, ptr) DEF_HELPER_FLAGS_3(crypto_sha256su1, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) -DEF_HELPER_FLAGS_3(crypto_sha512h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) -DEF_HELPER_FLAGS_3(crypto_sha512h2, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) -DEF_HELPER_FLAGS_2(crypto_sha512su0, TCG_CALL_NO_RWG, void, ptr, ptr) -DEF_HELPER_FLAGS_3(crypto_sha512su1, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) +DEF_HELPER_FLAGS_4(crypto_sha512h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sha512h2, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(crypto_sha512su0, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sha512su1, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(crypto_sm3tt, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32, i32) -DEF_HELPER_FLAGS_3(crypto_sm3partw1, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) -DEF_HELPER_FLAGS_3(crypto_sm3partw2, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) +DEF_HELPER_FLAGS_4(crypto_sm3partw1, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sm3partw2, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(crypto_sm4e, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(crypto_sm4ekey, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/crypto_helper.c b/target/arm/crypto_helper.c index 372d8350e4..637e4c00bb 100644 --- a/target/arm/crypto_helper.c +++ b/target/arm/crypto_helper.c @@ -31,6 +31,19 @@ union CRYPTO_STATE { #define CR_ST_WORD(state, i) (state.words[i]) #endif +/* + * The caller has not been converted to full gvec, and so only + * modifies the low 16 bytes of the vector register. + */ +static void clear_tail_16(void *vd, uint32_t desc) +{ + int opr_sz = simd_oprsz(desc); + int max_sz = simd_maxsz(desc); + + assert(opr_sz == 16); + clear_tail(vd, opr_sz, max_sz); +} + static void do_crypto_aese(uint64_t *rd, uint64_t *rn, uint64_t *rm, bool decrypt) { @@ -470,7 +483,7 @@ static uint64_t s1_512(uint64_t x) return ror64(x, 19) ^ ror64(x, 61) ^ (x >> 6); } -void HELPER(crypto_sha512h)(void *vd, void *vn, void *vm) +void HELPER(crypto_sha512h)(void *vd, void *vn, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rn = vn; @@ -483,9 +496,11 @@ void HELPER(crypto_sha512h)(void *vd, void *vn, void *vm) rd[0] = d0; rd[1] = d1; + + clear_tail_16(vd, desc); } -void HELPER(crypto_sha512h2)(void *vd, void *vn, void *vm) +void HELPER(crypto_sha512h2)(void *vd, void *vn, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rn = vn; @@ -498,9 +513,11 @@ void HELPER(crypto_sha512h2)(void *vd, void *vn, void *vm) rd[0] = d0; rd[1] = d1; + + clear_tail_16(vd, desc); } -void HELPER(crypto_sha512su0)(void *vd, void *vn) +void HELPER(crypto_sha512su0)(void *vd, void *vn, uint32_t desc) { uint64_t *rd = vd; uint64_t *rn = vn; @@ -512,9 +529,11 @@ void HELPER(crypto_sha512su0)(void *vd, void *vn) rd[0] = d0; rd[1] = d1; + + clear_tail_16(vd, desc); } -void HELPER(crypto_sha512su1)(void *vd, void *vn, void *vm) +void HELPER(crypto_sha512su1)(void *vd, void *vn, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rn = vn; @@ -522,9 +541,11 @@ void HELPER(crypto_sha512su1)(void *vd, void *vn, void *vm) rd[0] += s1_512(rn[0]) + rm[0]; rd[1] += s1_512(rn[1]) + rm[1]; + + clear_tail_16(vd, desc); } -void HELPER(crypto_sm3partw1)(void *vd, void *vn, void *vm) +void HELPER(crypto_sm3partw1)(void *vd, void *vn, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rn = vn; @@ -548,9 +569,11 @@ void HELPER(crypto_sm3partw1)(void *vd, void *vn, void *vm) rd[0] = d.l[0]; rd[1] = d.l[1]; + + clear_tail_16(vd, desc); } -void HELPER(crypto_sm3partw2)(void *vd, void *vn, void *vm) +void HELPER(crypto_sm3partw2)(void *vd, void *vn, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rn = vn; @@ -568,6 +591,8 @@ void HELPER(crypto_sm3partw2)(void *vd, void *vn, void *vm) rd[0] = d.l[0]; rd[1] = d.l[1]; + + clear_tail_16(vd, desc); } void HELPER(crypto_sm3tt)(void *vd, void *vn, void *vm, uint32_t imm2, diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 4d7a8fd2bb..96e20fa401 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -13619,7 +13619,6 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); bool feature; - CryptoThreeOpFn *genfn = NULL; gen_helper_gvec_3 *oolfn = NULL; GVecGen3Fn *gvecfn = NULL; @@ -13627,15 +13626,15 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) switch (opcode) { case 0: /* SHA512H */ feature = dc_isar_feature(aa64_sha512, s); - genfn = gen_helper_crypto_sha512h; + oolfn = gen_helper_crypto_sha512h; break; case 1: /* SHA512H2 */ feature = dc_isar_feature(aa64_sha512, s); - genfn = gen_helper_crypto_sha512h2; + oolfn = gen_helper_crypto_sha512h2; break; case 2: /* SHA512SU1 */ feature = dc_isar_feature(aa64_sha512, s); - genfn = gen_helper_crypto_sha512su1; + oolfn = gen_helper_crypto_sha512su1; break; case 3: /* RAX1 */ feature = dc_isar_feature(aa64_sha3, s); @@ -13648,11 +13647,11 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) switch (opcode) { case 0: /* SM3PARTW1 */ feature = dc_isar_feature(aa64_sm3, s); - genfn = gen_helper_crypto_sm3partw1; + oolfn = gen_helper_crypto_sm3partw1; break; case 1: /* SM3PARTW2 */ feature = dc_isar_feature(aa64_sm3, s); - genfn = gen_helper_crypto_sm3partw2; + oolfn = gen_helper_crypto_sm3partw2; break; case 2: /* SM4EKEY */ feature = dc_isar_feature(aa64_sm4, s); @@ -13675,20 +13674,8 @@ static void disas_crypto_three_reg_sha512(DisasContext *s, uint32_t insn) if (oolfn) { gen_gvec_op3_ool(s, true, rd, rn, rm, 0, oolfn); - } else if (gvecfn) { - gen_gvec_fn3(s, true, rd, rn, rm, gvecfn, MO_64); } else { - TCGv_ptr tcg_rd_ptr, tcg_rn_ptr, tcg_rm_ptr; - - tcg_rd_ptr = vec_full_reg_ptr(s, rd); - tcg_rn_ptr = vec_full_reg_ptr(s, rn); - tcg_rm_ptr = vec_full_reg_ptr(s, rm); - - genfn(tcg_rd_ptr, tcg_rn_ptr, tcg_rm_ptr); - - tcg_temp_free_ptr(tcg_rd_ptr); - tcg_temp_free_ptr(tcg_rn_ptr); - tcg_temp_free_ptr(tcg_rm_ptr); + gen_gvec_fn3(s, true, rd, rn, rm, gvecfn, MO_64); } } @@ -13703,19 +13690,14 @@ static void disas_crypto_two_reg_sha512(DisasContext *s, uint32_t insn) int opcode = extract32(insn, 10, 2); int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); - TCGv_ptr tcg_rd_ptr, tcg_rn_ptr; bool feature; - CryptoTwoOpFn *genfn; - gen_helper_gvec_3 *oolfn = NULL; switch (opcode) { case 0: /* SHA512SU0 */ feature = dc_isar_feature(aa64_sha512, s); - genfn = gen_helper_crypto_sha512su0; break; case 1: /* SM4E */ feature = dc_isar_feature(aa64_sm4, s); - oolfn = gen_helper_crypto_sm4e; break; default: unallocated_encoding(s); @@ -13731,18 +13713,16 @@ static void disas_crypto_two_reg_sha512(DisasContext *s, uint32_t insn) return; } - if (oolfn) { - gen_gvec_op3_ool(s, true, rd, rd, rn, 0, oolfn); - return; + switch (opcode) { + case 0: /* SHA512SU0 */ + gen_gvec_op2_ool(s, true, rd, rn, 0, gen_helper_crypto_sha512su0); + break; + case 1: /* SM4E */ + gen_gvec_op3_ool(s, true, rd, rd, rn, 0, gen_helper_crypto_sm4e); + break; + default: + g_assert_not_reached(); } - - tcg_rd_ptr = vec_full_reg_ptr(s, rd); - tcg_rn_ptr = vec_full_reg_ptr(s, rn); - - genfn(tcg_rd_ptr, tcg_rn_ptr); - - tcg_temp_free_ptr(tcg_rd_ptr); - tcg_temp_free_ptr(tcg_rn_ptr); } /* Crypto four-register From patchwork Thu May 14 21:28:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 186792 Delivered-To: patch@linaro.org Received: by 2002:a92:5b0a:0:0:0:0:0 with SMTP id p10csp1773429ilb; Thu, 14 May 2020 14:33:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyMgtQL/NErS83bCYCgCUGJm3r62CIdg9YkAXEkacMn2M56/YfHyJaNqGZUqp+VfR3Pl7oj X-Received: by 2002:a37:90c5:: with SMTP id s188mr405109qkd.275.1589491983254; Thu, 14 May 2020 14:33:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1589491983; cv=none; d=google.com; s=arc-20160816; b=ig+soSZnh/ZPYl+j8T3KfozfWOAN2zaFLtqVcaEIGKEx7cR/FC07P1aIrJqFMOcrJj +NugWijrDurz6/xtGkruVX9MHRjR9u0WD8NlJ4Lf/Lx83g73rBOd545CJsN7yylcIfvQ th2N8xac/oYxoAubs+F3jV83Al88XN+MEj3o7aacCfz8MFKj7gkp9Q2xJ3nY2tEOFjDQ 9HPGQNMYcn0v9mavTf5On4GqR9WYyGwjeQ22CaVdz+uFlifhUKbuhIXfBCeJS+piXquL N3BJ74A05+zeIqXuQLsRUiUEjyxxuhe+/jC9d25Gqrv+c6TJ+a/+qy0gexan/3mTStLL Qz0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=2x4agYGnPEqFMq4egWYtqq0ZroKFQlNuTMT0x7gawa4=; b=ugI/SS2xxOjxSdpCR1swPQMTGLlAGy+9JYiswRUSJ6kRhOHBpiW21z2mPUZWcz2OUx KBH9F+06CGLmjUMg96cUUK/pxUGCj9CnM8ZZBtXtWWmQni+fUUAcMshOzbgc1V6extGf zHfPMd2rvKp+TW9YX8++5v15M5bknPfao08K6cMD13o9UGCxiNezwTfffOPN6xFdwodi 5nYgnBp0PCSD6cPDTbIc5tPIBMTt8D6qXR5YnlBvZwHklxiNqho4jfLT82Ftj0cMUr0O 0NmYYzNpU3wx2iJ1UCdWIja3227LQLheqKwWG1mLWR8zS+q2htqb4JOp4v+PGG50zWBR 74eg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="UCf/jntX"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d18si114127qka.110.2020.05.14.14.33.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 May 2020 14:33:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="UCf/jntX"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42124 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jZLTS-00011p-H7 for patch@linaro.org; Thu, 14 May 2020 17:33:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49172) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jZLPG-0004B3-5T for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:42 -0400 Received: from mail-pj1-x1043.google.com ([2607:f8b0:4864:20::1043]:51020) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jZLPE-0004bp-OZ for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:41 -0400 Received: by mail-pj1-x1043.google.com with SMTP id t9so30486pjw.0 for ; Thu, 14 May 2020 14:28:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2x4agYGnPEqFMq4egWYtqq0ZroKFQlNuTMT0x7gawa4=; b=UCf/jntX/F6aG64oKGZpSWYqJ9e/1Yl5X2898TlpmqFCxnFT4JGGrhJluwG8NlW+bt h/7KHVWO6R10PHynPcwUIc2wvGO4cymZNWg8Ytg7WM0ANCOPwX4674CB1lCJzhQUPLWi u7aGnl9hchEEo8JpUdfN1tmcvpqYu2oImgPj/L1kRjJcB5dm0RBQAsLc3+u9LyXHsiVF 5Np52QMNG2pTW0jp6iw/8mOQZAYDgi/ALLC4r1/drHgOqMO7Y0Q9P8aM/4oliFSH+Kjk akhknRcuyNLAVSCFNiP+zS61UbcDZNo7+sp274oUaLTzQalNMO8LdPR+W35o0sN/gMOT Wxvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2x4agYGnPEqFMq4egWYtqq0ZroKFQlNuTMT0x7gawa4=; b=N/O4U/7Gsl4z0u7ZARa/z9Iz5RNRQ5cPQ+Ek3F5HevLnrPQ3fv6J+v+sKgTRSNZaX6 QSmgHahIZQBuaZsjf2SwW1Z0rekgv+zyi/8n4T5X8ikNEWZHRMY145kte9XfVbrFg3rc EYw9sQPeDBI0smKAfUSsce4VnMy8aBHczHYck51UYwDPPQt900e8p0ZTgZFPCtlpZRjB coMMQBa8U9BURwbSCRGOEhSPVRGwDxAARanz9UdAGu0b6qKuwHKXpJRqR5Dh3xuZBdJx i89eBq3RbIeytg/ESpdTCS+FkgCiE+yKrHeS5ISED5F8VjwKWRUSfPAjx3xRUmjm3pci 0ujg== X-Gm-Message-State: AOAM533T0ySs6oYu+3IdPYaXLzKIs/HeKeNrieNWnmWFJ9VM2v6wipwK jxT+zZLnjFryBFhMpB87GUVvg40TuSU= X-Received: by 2002:a17:90a:2305:: with SMTP id f5mr5414004pje.57.1589491718636; Thu, 14 May 2020 14:28:38 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id i7sm4986pjj.33.2020.05.14.14.28.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2020 14:28:37 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH 4/6] target/arm: Convert sha1 and sha256 to gvec helpers Date: Thu, 14 May 2020 14:28:29 -0700 Message-Id: <20200514212831.31248-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200514212831.31248-1-richard.henderson@linaro.org> References: <20200514212831.31248-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1043; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1043.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Do not yet convert the helpers to loop over opr_sz, but the descriptor allows the vector tail to be cleared. Which fixes an existing bug vs SVE. Signed-off-by: Richard Henderson --- target/arm/helper.h | 12 ++-- target/arm/neon-dp.decode | 12 ++-- target/arm/crypto_helper.c | 24 +++++-- target/arm/translate-a64.c | 34 ++++----- target/arm/translate-neon.inc.c | 124 +++++--------------------------- target/arm/translate.c | 24 ++----- 6 files changed, 67 insertions(+), 163 deletions(-) -- 2.20.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 784dc29ce2..cee23adbfc 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -514,13 +514,13 @@ DEF_HELPER_FLAGS_4(crypto_aese, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_3(crypto_aesmc, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_4(crypto_sha1_3reg, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_2(crypto_sha1h, TCG_CALL_NO_RWG, void, ptr, ptr) -DEF_HELPER_FLAGS_2(crypto_sha1su1, TCG_CALL_NO_RWG, void, ptr, ptr) +DEF_HELPER_FLAGS_3(crypto_sha1h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(crypto_sha1su1, TCG_CALL_NO_RWG, void, ptr, ptr, i32) -DEF_HELPER_FLAGS_3(crypto_sha256h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) -DEF_HELPER_FLAGS_3(crypto_sha256h2, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) -DEF_HELPER_FLAGS_2(crypto_sha256su0, TCG_CALL_NO_RWG, void, ptr, ptr) -DEF_HELPER_FLAGS_3(crypto_sha256su1, TCG_CALL_NO_RWG, void, ptr, ptr, ptr) +DEF_HELPER_FLAGS_4(crypto_sha256h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sha256h2, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(crypto_sha256su0, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sha256su1, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(crypto_sha512h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(crypto_sha512h2, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode index 8beb1db768..5b2fc65d72 100644 --- a/target/arm/neon-dp.decode +++ b/target/arm/neon-dp.decode @@ -165,14 +165,14 @@ VPADD_3s 1111 001 0 0 . .. .... .... 1011 . . . 1 .... @3same_q0 VQRDMLAH_3s 1111 001 1 0 . .. .... .... 1011 ... 1 .... @3same +@3same_crypto .... .... .... .... .... .... .... .... \ + &3same vm=%vm_dp vn=%vn_dp vd=%vd_dp size=0 q=1 + SHA1_3s 1111 001 0 0 . optype:2 .... .... 1100 . 1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp -SHA256H_3s 1111 001 1 0 . 00 .... .... 1100 . 1 . 0 .... \ - vm=%vm_dp vn=%vn_dp vd=%vd_dp -SHA256H2_3s 1111 001 1 0 . 01 .... .... 1100 . 1 . 0 .... \ - vm=%vm_dp vn=%vn_dp vd=%vd_dp -SHA256SU1_3s 1111 001 1 0 . 10 .... .... 1100 . 1 . 0 .... \ - vm=%vm_dp vn=%vn_dp vd=%vd_dp +SHA256H_3s 1111 001 1 0 . 00 .... .... 1100 . 1 . 0 .... @3same_crypto +SHA256H2_3s 1111 001 1 0 . 01 .... .... 1100 . 1 . 0 .... @3same_crypto +SHA256SU1_3s 1111 001 1 0 . 10 .... .... 1100 . 1 . 0 .... @3same_crypto VFMA_fp_3s 1111 001 0 0 . 0 . .... .... 1100 ... 1 .... @3same_fp VFMS_fp_3s 1111 001 0 0 . 1 . .... .... 1100 ... 1 .... @3same_fp diff --git a/target/arm/crypto_helper.c b/target/arm/crypto_helper.c index 637e4c00bb..7124745c32 100644 --- a/target/arm/crypto_helper.c +++ b/target/arm/crypto_helper.c @@ -303,7 +303,7 @@ void HELPER(crypto_sha1_3reg)(void *vd, void *vn, void *vm, uint32_t op) rd[1] = d.l[1]; } -void HELPER(crypto_sha1h)(void *vd, void *vm) +void HELPER(crypto_sha1h)(void *vd, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rm = vm; @@ -314,9 +314,11 @@ void HELPER(crypto_sha1h)(void *vd, void *vm) rd[0] = m.l[0]; rd[1] = m.l[1]; + + clear_tail_16(vd, desc); } -void HELPER(crypto_sha1su1)(void *vd, void *vm) +void HELPER(crypto_sha1su1)(void *vd, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rm = vm; @@ -330,6 +332,8 @@ void HELPER(crypto_sha1su1)(void *vd, void *vm) rd[0] = d.l[0]; rd[1] = d.l[1]; + + clear_tail_16(vd, desc); } /* @@ -357,7 +361,7 @@ static uint32_t s1(uint32_t x) return ror32(x, 17) ^ ror32(x, 19) ^ (x >> 10); } -void HELPER(crypto_sha256h)(void *vd, void *vn, void *vm) +void HELPER(crypto_sha256h)(void *vd, void *vn, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rn = vn; @@ -388,9 +392,11 @@ void HELPER(crypto_sha256h)(void *vd, void *vn, void *vm) rd[0] = d.l[0]; rd[1] = d.l[1]; + + clear_tail_16(vd, desc); } -void HELPER(crypto_sha256h2)(void *vd, void *vn, void *vm) +void HELPER(crypto_sha256h2)(void *vd, void *vn, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rn = vn; @@ -413,9 +419,11 @@ void HELPER(crypto_sha256h2)(void *vd, void *vn, void *vm) rd[0] = d.l[0]; rd[1] = d.l[1]; + + clear_tail_16(vd, desc); } -void HELPER(crypto_sha256su0)(void *vd, void *vm) +void HELPER(crypto_sha256su0)(void *vd, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rm = vm; @@ -429,9 +437,11 @@ void HELPER(crypto_sha256su0)(void *vd, void *vm) rd[0] = d.l[0]; rd[1] = d.l[1]; + + clear_tail_16(vd, desc); } -void HELPER(crypto_sha256su1)(void *vd, void *vn, void *vm) +void HELPER(crypto_sha256su1)(void *vd, void *vn, void *vm, uint32_t desc) { uint64_t *rd = vd; uint64_t *rn = vn; @@ -447,6 +457,8 @@ void HELPER(crypto_sha256su1)(void *vd, void *vn, void *vm) rd[0] = d.l[0]; rd[1] = d.l[1]; + + clear_tail_16(vd, desc); } /* diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 96e20fa401..d3094d5dfd 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -13460,8 +13460,7 @@ static void disas_crypto_three_reg_sha(DisasContext *s, uint32_t insn) int rm = extract32(insn, 16, 5); int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); - CryptoThreeOpFn *genfn; - TCGv_ptr tcg_rd_ptr, tcg_rn_ptr, tcg_rm_ptr; + gen_helper_gvec_3 *genfn; bool feature; if (size != 0) { @@ -13503,23 +13502,22 @@ static void disas_crypto_three_reg_sha(DisasContext *s, uint32_t insn) return; } - tcg_rd_ptr = vec_full_reg_ptr(s, rd); - tcg_rn_ptr = vec_full_reg_ptr(s, rn); - tcg_rm_ptr = vec_full_reg_ptr(s, rm); - if (genfn) { - genfn(tcg_rd_ptr, tcg_rn_ptr, tcg_rm_ptr); + gen_gvec_op3_ool(s, true, rd, rn, rm, 0, genfn); } else { TCGv_i32 tcg_opcode = tcg_const_i32(opcode); + TCGv_ptr tcg_rd_ptr = vec_full_reg_ptr(s, rd); + TCGv_ptr tcg_rn_ptr = vec_full_reg_ptr(s, rn); + TCGv_ptr tcg_rm_ptr = vec_full_reg_ptr(s, rm); gen_helper_crypto_sha1_3reg(tcg_rd_ptr, tcg_rn_ptr, tcg_rm_ptr, tcg_opcode); - tcg_temp_free_i32(tcg_opcode); - } - tcg_temp_free_ptr(tcg_rd_ptr); - tcg_temp_free_ptr(tcg_rn_ptr); - tcg_temp_free_ptr(tcg_rm_ptr); + tcg_temp_free_i32(tcg_opcode); + tcg_temp_free_ptr(tcg_rd_ptr); + tcg_temp_free_ptr(tcg_rn_ptr); + tcg_temp_free_ptr(tcg_rm_ptr); + } } /* Crypto two-reg SHA @@ -13534,9 +13532,8 @@ static void disas_crypto_two_reg_sha(DisasContext *s, uint32_t insn) int opcode = extract32(insn, 12, 5); int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); - CryptoTwoOpFn *genfn; + gen_helper_gvec_2 *genfn; bool feature; - TCGv_ptr tcg_rd_ptr, tcg_rn_ptr; if (size != 0) { unallocated_encoding(s); @@ -13569,14 +13566,7 @@ static void disas_crypto_two_reg_sha(DisasContext *s, uint32_t insn) if (!fp_access_check(s)) { return; } - - tcg_rd_ptr = vec_full_reg_ptr(s, rd); - tcg_rn_ptr = vec_full_reg_ptr(s, rn); - - genfn(tcg_rd_ptr, tcg_rn_ptr); - - tcg_temp_free_ptr(tcg_rd_ptr); - tcg_temp_free_ptr(tcg_rn_ptr); + gen_gvec_op2_ool(s, true, rd, rn, 0, genfn); } static void gen_rax1_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m) diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index 3fe65a0b08..205877ca48 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -661,12 +661,14 @@ DO_3SAME_CMP(VCGE_S, TCG_COND_GE) DO_3SAME_CMP(VCGE_U, TCG_COND_GEU) DO_3SAME_CMP(VCEQ, TCG_COND_EQ) -static void gen_VMUL_p_3s(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, - uint32_t rm_ofs, uint32_t oprsz, uint32_t maxsz) -{ - tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz, - 0, gen_helper_gvec_pmul_b); -} +#define WRAP_OOL_FN(WRAPNAME, FUNC) \ + static void WRAPNAME(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, \ + uint32_t rm_ofs, uint32_t oprsz, uint32_t maxsz) \ + { \ + tcg_gen_gvec_3_ool(rd_ofs, rn_ofs, rm_ofs, oprsz, maxsz, 0, FUNC); \ + } + +WRAP_OOL_FN(gen_VMUL_p_3s, gen_helper_gvec_pmul_b) static bool trans_VMUL_p_3s(DisasContext *s, arg_3same *a) { @@ -728,107 +730,19 @@ static bool trans_SHA1_3s(DisasContext *s, arg_SHA1_3s *a) return true; } -static bool trans_SHA256H_3s(DisasContext *s, arg_SHA256H_3s *a) -{ - TCGv_ptr ptr1, ptr2, ptr3; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON) || - !dc_isar_feature(aa32_sha2, s)) { - return false; +#define DO_SHA2(NAME, FUNC) \ + WRAP_OOL_FN(gen_##NAME##_3s, FUNC) \ + static bool trans_##NAME##_3s(DisasContext *s, arg_3same *a) \ + { \ + if (!dc_isar_feature(aa32_sha2, s)) { \ + return false; \ + } \ + return do_3same(s, a, gen_##NAME##_3s); \ } - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - - if ((a->vn | a->vm | a->vd) & 1) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - ptr1 = vfp_reg_ptr(true, a->vd); - ptr2 = vfp_reg_ptr(true, a->vn); - ptr3 = vfp_reg_ptr(true, a->vm); - gen_helper_crypto_sha256h(ptr1, ptr2, ptr3); - tcg_temp_free_ptr(ptr1); - tcg_temp_free_ptr(ptr2); - tcg_temp_free_ptr(ptr3); - - return true; -} - -static bool trans_SHA256H2_3s(DisasContext *s, arg_SHA256H2_3s *a) -{ - TCGv_ptr ptr1, ptr2, ptr3; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON) || - !dc_isar_feature(aa32_sha2, s)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - - if ((a->vn | a->vm | a->vd) & 1) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - ptr1 = vfp_reg_ptr(true, a->vd); - ptr2 = vfp_reg_ptr(true, a->vn); - ptr3 = vfp_reg_ptr(true, a->vm); - gen_helper_crypto_sha256h2(ptr1, ptr2, ptr3); - tcg_temp_free_ptr(ptr1); - tcg_temp_free_ptr(ptr2); - tcg_temp_free_ptr(ptr3); - - return true; -} - -static bool trans_SHA256SU1_3s(DisasContext *s, arg_SHA256SU1_3s *a) -{ - TCGv_ptr ptr1, ptr2, ptr3; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON) || - !dc_isar_feature(aa32_sha2, s)) { - return false; - } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - - if ((a->vn | a->vm | a->vd) & 1) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - ptr1 = vfp_reg_ptr(true, a->vd); - ptr2 = vfp_reg_ptr(true, a->vn); - ptr3 = vfp_reg_ptr(true, a->vm); - gen_helper_crypto_sha256su1(ptr1, ptr2, ptr3); - tcg_temp_free_ptr(ptr1); - tcg_temp_free_ptr(ptr2); - tcg_temp_free_ptr(ptr3); - - return true; -} +DO_SHA2(SHA256H, gen_helper_crypto_sha256h) +DO_SHA2(SHA256H2, gen_helper_crypto_sha256h2) +DO_SHA2(SHA256SU1, gen_helper_crypto_sha256su1) #define DO_3SAME_64(INSN, FUNC) \ static void gen_##INSN##_3s(unsigned vece, uint32_t rd_ofs, \ diff --git a/target/arm/translate.c b/target/arm/translate.c index 921359dfd4..d5c97a8e3c 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -5280,7 +5280,7 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) int vec_size; uint32_t imm; TCGv_i32 tmp, tmp2, tmp3, tmp4, tmp5; - TCGv_ptr ptr1, ptr2; + TCGv_ptr ptr1; TCGv_i64 tmp64; if (!arm_dc_feature(s, ARM_FEATURE_NEON)) { @@ -6395,13 +6395,8 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) if (!dc_isar_feature(aa32_sha1, s) || ((rm | rd) & 1)) { return 1; } - ptr1 = vfp_reg_ptr(true, rd); - ptr2 = vfp_reg_ptr(true, rm); - - gen_helper_crypto_sha1h(ptr1, ptr2); - - tcg_temp_free_ptr(ptr1); - tcg_temp_free_ptr(ptr2); + tcg_gen_gvec_2_ool(rd_ofs, rm_ofs, 16, 16, 0, + gen_helper_crypto_sha1h); break; case NEON_2RM_SHA1SU1: if ((rm | rd) & 1) { @@ -6415,17 +6410,10 @@ static int disas_neon_data_insn(DisasContext *s, uint32_t insn) } else if (!dc_isar_feature(aa32_sha1, s)) { return 1; } - ptr1 = vfp_reg_ptr(true, rd); - ptr2 = vfp_reg_ptr(true, rm); - if (q) { - gen_helper_crypto_sha256su0(ptr1, ptr2); - } else { - gen_helper_crypto_sha1su1(ptr1, ptr2); - } - tcg_temp_free_ptr(ptr1); - tcg_temp_free_ptr(ptr2); + tcg_gen_gvec_2_ool(rd_ofs, rm_ofs, 16, 16, 0, + q ? gen_helper_crypto_sha256su0 + : gen_helper_crypto_sha1su1); break; - case NEON_2RM_VMVN: tcg_gen_gvec_not(0, rd_ofs, rm_ofs, vec_size, vec_size); break; From patchwork Thu May 14 21:28:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 186793 Delivered-To: patch@linaro.org Received: by 2002:a92:5b0a:0:0:0:0:0 with SMTP id p10csp1774910ilb; Thu, 14 May 2020 14:34:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzwljWB8mDkXoXflZo5AiTk1hw1sNJo6V8u5qX6WCEre5bNn004I5haAwxKt2vzJM312IjH X-Received: by 2002:a0c:b524:: with SMTP id d36mr547282qve.12.1589492097352; Thu, 14 May 2020 14:34:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1589492097; cv=none; d=google.com; s=arc-20160816; b=PZXSbJ0d6o0zv6sdm8JTVZ2FSal5upgWqtS4yjL/ngme2mp9rM4d3JQmQ5sMWo0erx bz3rGx69d16EpcL5jbJzEy5tDt9jSpw4Vz62Fu9N43OiSQypuwfOGwz2CzIjVzc2zTYp 5kJNykBjiBzqWfpQUTCHSdRoGFfLyuQhFSkSVYsdRpROtI1n1nfRfKCk6Sc1H4MC6A2D HYIcL2TeTgs8hGCS1bQwHtvrtxnxl6lh1555wKNvE8z+Y5iHlxAWFyOXpcLSt9zQQ+F6 gTcFjtMONZb+Hq5Rxrt3Khso3BHNaa5af8SZaSTvFRxhGM84XINzD/tQNEOgQS44vkec sAsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=eFKOT9NBLVT8m1oyDedlD+LWCS/9nf6l8OhFQLOWEa4=; b=gKn9cxk87lR03yGClbBUKaKRO3t4WyCk1p/3Q78j40rAM7amwd7hcIsAlpbSHaLDTl rjR9lkdDy205dzfFHCL7ezgsuKoAuXNjMGdJMAwJo7+yN2MfCg6qwRfvq6C9jbij0hme ZA36GAw1tr0i3jqgmudkDOxdmEWVJgb9GQD13G8zoM+2rcZD9xW0Nf5pwKL3SuC0LXbI uKa7BQJ4zuthoaUtHi28o+ObZ832qlPingwO/ydngT3EjxSKBxPCJQ88lImPl984YgBk ZC9ZbeYTfTiiv9MO7jaz9X+y/+PRtBnJuJnk+Fk6Wikhs2r6/mkhr71wNUxilgXFDgqw yD2w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=krucVGRF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b13si213437qte.171.2020.05.14.14.34.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 May 2020 14:34:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=krucVGRF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47518 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jZLVH-0004KC-4z for patch@linaro.org; Thu, 14 May 2020 17:34:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49174) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jZLPH-0004ES-7p for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:43 -0400 Received: from mail-pj1-x1042.google.com ([2607:f8b0:4864:20::1042]:53903) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jZLPG-0004bx-0v for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:42 -0400 Received: by mail-pj1-x1042.google.com with SMTP id hi11so23346pjb.3 for ; Thu, 14 May 2020 14:28:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eFKOT9NBLVT8m1oyDedlD+LWCS/9nf6l8OhFQLOWEa4=; b=krucVGRF0glUTFFSPDYGzufm0exr2jnYcheom56wcdCqfsA0G3DmzA3MCkOTOlVOZi MXtcTI4F/gfMfMzYo9vr2zXBbH4eMHgLFVFx4/8c4X4vANtdAkYVruzg2GBvXSzOEdu/ khTJWm5sU+l1k0u/SGWxhYIwqn+1FdBUNCzOKnS8Cay50TIhf4Hr5o0sv2LMm0GNUR41 RceojQuQyWeOCHFZYWXXRG8Qu/iddSf3uslAuWz8OhU7xs6v7ul4vrR1LKmGksDyUH1U KpWkSrZSd9bAm+4BHdkjYk7uLTk5LIag4ZZh5h0iTx8I37U2hetaFH9Gr1n9Ani3v8KS e0RQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eFKOT9NBLVT8m1oyDedlD+LWCS/9nf6l8OhFQLOWEa4=; b=nHmJON7t8aULsDPo9/Od8iVu+FMbIjX95ZGLXDWO9N480NbTYoCdkJlVhR/bm8XY5z 32xdtAx4St6dOIYfSxHXlXkXHmqQax8qvXX5TIseBVjWRu5pnW55d9VPFSqGhbYhyhlZ Y/m9sEx7pzlaGAAFeDWPIvp7TiKrvYcSUnnxDADUrK/wGSw30ew9fPIMkBabpKNjfJ3j aQYKf67kagqSHBsYHhPR83fc7Ut9zypRvhkaj4S0GmlZCp/AmOcPCnFgTo5vI5GqGym/ MVLpkJc7uVVTzswS2EJUbNnVVg8sHqyfRe1L4pfOjy/ewOICKW2xV8Qj3IOeX/WI7Xnf fI4A== X-Gm-Message-State: AOAM5328cGunrJWfeW4KBa0orV0XvM65K63/eg/Kbw7Pm6qNOtrCwW5N EKYE7hP2GTVcrKiaRA217ADQPL43Hxc= X-Received: by 2002:a17:90a:28e5:: with SMTP id f92mr9662pjd.38.1589491720183; Thu, 14 May 2020 14:28:40 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id i7sm4986pjj.33.2020.05.14.14.28.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2020 14:28:39 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH 5/6] target/arm: Split helper_crypto_sha1_3reg Date: Thu, 14 May 2020 14:28:30 -0700 Message-Id: <20200514212831.31248-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200514212831.31248-1-richard.henderson@linaro.org> References: <20200514212831.31248-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1042; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1042.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Rather than passing an opcode to a helper, fully decode the operation at translate time. Use clear_tail_16 to zap the balance of the SVE register with the AdvSIMD write. Signed-off-by: Richard Henderson --- target/arm/helper.h | 5 +- target/arm/neon-dp.decode | 6 +- target/arm/crypto_helper.c | 99 +++++++++++++++++++++------------ target/arm/translate-a64.c | 29 ++++------ target/arm/translate-neon.inc.c | 46 ++++----------- 5 files changed, 93 insertions(+), 92 deletions(-) -- 2.20.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index cee23adbfc..13475ecf81 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -513,7 +513,10 @@ DEF_HELPER_FLAGS_2(neon_qzip32, TCG_CALL_NO_RWG, void, ptr, ptr) DEF_HELPER_FLAGS_4(crypto_aese, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_3(crypto_aesmc, TCG_CALL_NO_RWG, void, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(crypto_sha1_3reg, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sha1su0, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sha1c, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sha1p, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sha1m, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_3(crypto_sha1h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(crypto_sha1su1, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/neon-dp.decode b/target/arm/neon-dp.decode index 5b2fc65d72..8af7c53d8b 100644 --- a/target/arm/neon-dp.decode +++ b/target/arm/neon-dp.decode @@ -168,8 +168,10 @@ VQRDMLAH_3s 1111 001 1 0 . .. .... .... 1011 ... 1 .... @3same @3same_crypto .... .... .... .... .... .... .... .... \ &3same vm=%vm_dp vn=%vn_dp vd=%vd_dp size=0 q=1 -SHA1_3s 1111 001 0 0 . optype:2 .... .... 1100 . 1 . 0 .... \ - vm=%vm_dp vn=%vn_dp vd=%vd_dp +SHA1C_3s 1111 001 0 0 . 00 .... .... 1100 . 1 . 0 .... @3same_crypto +SHA1P_3s 1111 001 0 0 . 01 .... .... 1100 . 1 . 0 .... @3same_crypto +SHA1M_3s 1111 001 0 0 . 10 .... .... 1100 . 1 . 0 .... @3same_crypto +SHA1SU0_3s 1111 001 0 0 . 11 .... .... 1100 . 1 . 0 .... @3same_crypto SHA256H_3s 1111 001 1 0 . 00 .... .... 1100 . 1 . 0 .... @3same_crypto SHA256H2_3s 1111 001 1 0 . 01 .... .... 1100 . 1 . 0 .... @3same_crypto SHA256SU1_3s 1111 001 1 0 . 10 .... .... 1100 . 1 . 0 .... @3same_crypto diff --git a/target/arm/crypto_helper.c b/target/arm/crypto_helper.c index 7124745c32..636683d0f1 100644 --- a/target/arm/crypto_helper.c +++ b/target/arm/crypto_helper.c @@ -24,11 +24,11 @@ union CRYPTO_STATE { }; #ifdef HOST_WORDS_BIGENDIAN -#define CR_ST_BYTE(state, i) (state.bytes[(15 - (i)) ^ 8]) -#define CR_ST_WORD(state, i) (state.words[(3 - (i)) ^ 2]) +#define CR_ST_BYTE(state, i) ((state).bytes[(15 - (i)) ^ 8]) +#define CR_ST_WORD(state, i) ((state).words[(3 - (i)) ^ 2]) #else -#define CR_ST_BYTE(state, i) (state.bytes[i]) -#define CR_ST_WORD(state, i) (state.words[i]) +#define CR_ST_BYTE(state, i) ((state).bytes[i]) +#define CR_ST_WORD(state, i) ((state).words[i]) #endif /* @@ -258,49 +258,74 @@ static uint32_t maj(uint32_t x, uint32_t y, uint32_t z) return (x & y) | ((x | y) & z); } -void HELPER(crypto_sha1_3reg)(void *vd, void *vn, void *vm, uint32_t op) +void HELPER(crypto_sha1su0)(void *vd, void *vn, void *vm, uint32_t desc) +{ + uint64_t *d = vd, *n = vn, *m = vm; + uint64_t d0, d1; + + d0 = d[1] ^ d[0] ^ m[0]; + d1 = n[0] ^ d[1] ^ m[1]; + d[0] = d0; + d[1] = d1; + + clear_tail_16(vd, desc); +} + +static inline void crypto_sha1_3reg(uint64_t *rd, uint64_t *rn, + uint64_t *rm, uint32_t desc, + uint32_t (*fn)(union CRYPTO_STATE *d)) { - uint64_t *rd = vd; - uint64_t *rn = vn; - uint64_t *rm = vm; union CRYPTO_STATE d = { .l = { rd[0], rd[1] } }; union CRYPTO_STATE n = { .l = { rn[0], rn[1] } }; union CRYPTO_STATE m = { .l = { rm[0], rm[1] } }; + int i; - if (op == 3) { /* sha1su0 */ - d.l[0] ^= d.l[1] ^ m.l[0]; - d.l[1] ^= n.l[0] ^ m.l[1]; - } else { - int i; + for (i = 0; i < 4; i++) { + uint32_t t = fn(&d); - for (i = 0; i < 4; i++) { - uint32_t t; + t += rol32(CR_ST_WORD(d, 0), 5) + CR_ST_WORD(n, 0) + + CR_ST_WORD(m, i); - switch (op) { - case 0: /* sha1c */ - t = cho(CR_ST_WORD(d, 1), CR_ST_WORD(d, 2), CR_ST_WORD(d, 3)); - break; - case 1: /* sha1p */ - t = par(CR_ST_WORD(d, 1), CR_ST_WORD(d, 2), CR_ST_WORD(d, 3)); - break; - case 2: /* sha1m */ - t = maj(CR_ST_WORD(d, 1), CR_ST_WORD(d, 2), CR_ST_WORD(d, 3)); - break; - default: - g_assert_not_reached(); - } - t += rol32(CR_ST_WORD(d, 0), 5) + CR_ST_WORD(n, 0) - + CR_ST_WORD(m, i); - - CR_ST_WORD(n, 0) = CR_ST_WORD(d, 3); - CR_ST_WORD(d, 3) = CR_ST_WORD(d, 2); - CR_ST_WORD(d, 2) = ror32(CR_ST_WORD(d, 1), 2); - CR_ST_WORD(d, 1) = CR_ST_WORD(d, 0); - CR_ST_WORD(d, 0) = t; - } + CR_ST_WORD(n, 0) = CR_ST_WORD(d, 3); + CR_ST_WORD(d, 3) = CR_ST_WORD(d, 2); + CR_ST_WORD(d, 2) = ror32(CR_ST_WORD(d, 1), 2); + CR_ST_WORD(d, 1) = CR_ST_WORD(d, 0); + CR_ST_WORD(d, 0) = t; } rd[0] = d.l[0]; rd[1] = d.l[1]; + + clear_tail_16(rd, desc); +} + +static uint32_t do_sha1c(union CRYPTO_STATE *d) +{ + return cho(CR_ST_WORD(*d, 1), CR_ST_WORD(*d, 2), CR_ST_WORD(*d, 3)); +} + +void HELPER(crypto_sha1c)(void *vd, void *vn, void *vm, uint32_t desc) +{ + crypto_sha1_3reg(vd, vn, vm, desc, do_sha1c); +} + +static uint32_t do_sha1p(union CRYPTO_STATE *d) +{ + return par(CR_ST_WORD(*d, 1), CR_ST_WORD(*d, 2), CR_ST_WORD(*d, 3)); +} + +void HELPER(crypto_sha1p)(void *vd, void *vn, void *vm, uint32_t desc) +{ + crypto_sha1_3reg(vd, vn, vm, desc, do_sha1p); +} + +static uint32_t do_sha1m(union CRYPTO_STATE *d) +{ + return maj(CR_ST_WORD(*d, 1), CR_ST_WORD(*d, 2), CR_ST_WORD(*d, 3)); +} + +void HELPER(crypto_sha1m)(void *vd, void *vn, void *vm, uint32_t desc) +{ + crypto_sha1_3reg(vd, vn, vm, desc, do_sha1m); } void HELPER(crypto_sha1h)(void *vd, void *vm, uint32_t desc) diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index d3094d5dfd..49ca7ac76e 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -13470,10 +13470,19 @@ static void disas_crypto_three_reg_sha(DisasContext *s, uint32_t insn) switch (opcode) { case 0: /* SHA1C */ + genfn = gen_helper_crypto_sha1c; + feature = dc_isar_feature(aa64_sha1, s); + break; case 1: /* SHA1P */ + genfn = gen_helper_crypto_sha1p; + feature = dc_isar_feature(aa64_sha1, s); + break; case 2: /* SHA1M */ + genfn = gen_helper_crypto_sha1m; + feature = dc_isar_feature(aa64_sha1, s); + break; case 3: /* SHA1SU0 */ - genfn = NULL; + genfn = gen_helper_crypto_sha1su0; feature = dc_isar_feature(aa64_sha1, s); break; case 4: /* SHA256H */ @@ -13501,23 +13510,7 @@ static void disas_crypto_three_reg_sha(DisasContext *s, uint32_t insn) if (!fp_access_check(s)) { return; } - - if (genfn) { - gen_gvec_op3_ool(s, true, rd, rn, rm, 0, genfn); - } else { - TCGv_i32 tcg_opcode = tcg_const_i32(opcode); - TCGv_ptr tcg_rd_ptr = vec_full_reg_ptr(s, rd); - TCGv_ptr tcg_rn_ptr = vec_full_reg_ptr(s, rn); - TCGv_ptr tcg_rm_ptr = vec_full_reg_ptr(s, rm); - - gen_helper_crypto_sha1_3reg(tcg_rd_ptr, tcg_rn_ptr, - tcg_rm_ptr, tcg_opcode); - - tcg_temp_free_i32(tcg_opcode); - tcg_temp_free_ptr(tcg_rd_ptr); - tcg_temp_free_ptr(tcg_rn_ptr); - tcg_temp_free_ptr(tcg_rm_ptr); - } + gen_gvec_op3_ool(s, true, rd, rn, rm, 0, genfn); } /* Crypto two-reg SHA diff --git a/target/arm/translate-neon.inc.c b/target/arm/translate-neon.inc.c index 205877ca48..7b19753c8c 100644 --- a/target/arm/translate-neon.inc.c +++ b/target/arm/translate-neon.inc.c @@ -693,42 +693,20 @@ static bool trans_VMUL_p_3s(DisasContext *s, arg_3same *a) DO_VQRDMLAH(VQRDMLAH, gen_gvec_sqrdmlah_qc) DO_VQRDMLAH(VQRDMLSH, gen_gvec_sqrdmlsh_qc) -static bool trans_SHA1_3s(DisasContext *s, arg_SHA1_3s *a) -{ - TCGv_ptr ptr1, ptr2, ptr3; - TCGv_i32 tmp; - - if (!arm_dc_feature(s, ARM_FEATURE_NEON) || - !dc_isar_feature(aa32_sha1, s)) { - return false; +#define DO_SHA1(NAME, FUNC) \ + WRAP_OOL_FN(gen_##NAME##_3s, FUNC) \ + static bool trans_##NAME##_3s(DisasContext *s, arg_3same *a) \ + { \ + if (!dc_isar_feature(aa32_sha1, s)) { \ + return false; \ + } \ + return do_3same(s, a, gen_##NAME##_3s); \ } - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - - if ((a->vn | a->vm | a->vd) & 1) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - ptr1 = vfp_reg_ptr(true, a->vd); - ptr2 = vfp_reg_ptr(true, a->vn); - ptr3 = vfp_reg_ptr(true, a->vm); - tmp = tcg_const_i32(a->optype); - gen_helper_crypto_sha1_3reg(ptr1, ptr2, ptr3, tmp); - tcg_temp_free_i32(tmp); - tcg_temp_free_ptr(ptr1); - tcg_temp_free_ptr(ptr2); - tcg_temp_free_ptr(ptr3); - - return true; -} +DO_SHA1(SHA1C, gen_helper_crypto_sha1c) +DO_SHA1(SHA1P, gen_helper_crypto_sha1p) +DO_SHA1(SHA1M, gen_helper_crypto_sha1m) +DO_SHA1(SHA1SU0, gen_helper_crypto_sha1su0) #define DO_SHA2(NAME, FUNC) \ WRAP_OOL_FN(gen_##NAME##_3s, FUNC) \ From patchwork Thu May 14 21:28:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 186791 Delivered-To: patch@linaro.org Received: by 2002:a92:5b0a:0:0:0:0:0 with SMTP id p10csp1772424ilb; Thu, 14 May 2020 14:31:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwpVQvP15skNwynLbDX0fp4eReqZdqE/E4Z+9uGl6SgU8NS0HDmg3LjPlUNRYAIx4publa7 X-Received: by 2002:ac8:2f50:: with SMTP id k16mr236361qta.392.1589491904798; Thu, 14 May 2020 14:31:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1589491904; cv=none; d=google.com; s=arc-20160816; b=IRqxcRtv8FUWwgz+W3Pref7IsVXHnEANCT2JsTxgSOVJNmr6YsJEZnn7b0LdB4RGxE M25KKgc/nbSoYIabKOoIhfwUMChVlTHIdREmv+DYY+lZlaOQ3tBR5kEz6FF+kLhduE8g lmbyYDaKdtnu6Y/II4kkQcmz+yeAD/B1keAYQQh7+TmZFv5Q2F/zkidAjm2cTU4ryYlb XaMH36EN4GXl1ytst8j+WTZDvv1sJLNZzSXU9M98iM4RD9aKIrNr8Gh4BR/wlkTFBoLo 8OUGLb9qkw1svwRKicP3SrKerCN5YkfjHOPlGV26gU3lrZe/3ug7yHeIXkSKRVIKouSR /wEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=FUfGRxc6GwZY+hIGXeIydpcbMLBkJyGMOcDJ5KEiVOE=; b=j8G7CPf2bIUfK1XpcpZ3ht2G9ge9K92ghPc1bQ3uWJRe0jDfLMytXCrCf3c2Zd+olK GfpyygmsS201GL88zklJWr0vFaoRaRmopgcs3iyxtWg+HAWffBuZYlCNX6UDat/y2iGA PYIR5I4Xb6t5IFNk7dpr0jxzBHVZb73JgZEHQHiFpv1A2W9asIdqITViVpx9RbtVub0H MT+Aqi2e+SUxzTnCgB5z6utEAJA4R/PjPUrfAzl9huZl/P3jm0maOZl/c9At+KMFCK6f e0vkLd/yynq2mWRxP6niVdPcUR213aRWmK61wLDw2q4tEq5+IVGlVRXIYOZSRy4SFjDx yFAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iNm9eIGI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f17si193690qto.282.2020.05.14.14.31.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 May 2020 14:31:44 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iNm9eIGI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:39246 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jZLSC-0007s2-9L for patch@linaro.org; Thu, 14 May 2020 17:31:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49180) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jZLPJ-0004Jz-3g for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:45 -0400 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]:37822) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jZLPH-0004c8-Du for qemu-devel@nongnu.org; Thu, 14 May 2020 17:28:44 -0400 Received: by mail-pl1-x642.google.com with SMTP id x10so56789plr.4 for ; Thu, 14 May 2020 14:28:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FUfGRxc6GwZY+hIGXeIydpcbMLBkJyGMOcDJ5KEiVOE=; b=iNm9eIGIhykwfZUWzXVnpf9OxzuCyk9lMFV45qMF8hn68oR6yPPoPE3KWq+vWuMzZN IgltZqzwhicx1VcghyOA1qreVmD62CcDr2ZTZSrmEWJwEG1ydn2eRg57QVJs1rrRaiSF Au4AS7mSQCqS7uzvV6Iybcg2bGkYi9BgMM2klfLgT63L6exPoAs5PB+Vse5m3a4TwCXM wSAP8ZIEdDMU0BhMzJnYFFr6ks4XEPhIOTYFpXDEej/3dMDKYV4zoQ2VztOJ5RYp6j6O 6SDxSXkRSy2RxBR+CVaIsGIVV+5oI1upL9GlEONA47RLrEf9qBYCZLnOxOTM2baqX1m4 12aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FUfGRxc6GwZY+hIGXeIydpcbMLBkJyGMOcDJ5KEiVOE=; b=lNWBKEtVw8JPPqOPMatYWvijT5+xCsXtOc/WlwqmTuTbl/R3iv5baCvr3o2Ro01Z/A QGBg6jpYnmC8Vd8ieqa5QFsU2oFI7RzcwyrK00SuA3po6CbGWDx+5343xH2O+kOc+Rfe dooUS+2Xv37udzpDUHbWSU2fJorg1HvtXnnZo9ixRIGFVw/MRtlm0bfufJ5wYhhjpHM8 eAMEg76/KmYAX+E1Zakt/PbNwH/6ID+iblquG46N5uCJTjHZ08VKR/xhja/4WpKyBRyz y3E0VUpkebce4VILvcQzyxOcrRsWpAAPdnKdYJfighZ9jYS9MUZ1edQjokX/JU1pZpFA q4og== X-Gm-Message-State: AOAM5312q6DRaL0Ti9LjgPRNa8Cerqysof6UoHhXCN9jJxhH+zlWDV4h lKRMxFCHl9LA5p6DWid8gfiKYyT/ySo= X-Received: by 2002:a17:90a:d153:: with SMTP id t19mr10050pjw.42.1589491721560; Thu, 14 May 2020 14:28:41 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id i7sm4986pjj.33.2020.05.14.14.28.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2020 14:28:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH 6/6] target/arm: Split helper_crypto_sm3tt Date: Thu, 14 May 2020 14:28:31 -0700 Message-Id: <20200514212831.31248-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200514212831.31248-1-richard.henderson@linaro.org> References: <20200514212831.31248-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::642; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x642.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Rather than passing an opcode to a helper, fully decode the operation at translate time. Use clear_tail_16 to zap the balance of the SVE register with the AdvSIMD write. Signed-off-by: Richard Henderson --- target/arm/helper.h | 5 ++++- target/arm/crypto_helper.c | 24 ++++++++++++++++++------ target/arm/translate-a64.c | 21 +++++---------------- 3 files changed, 27 insertions(+), 23 deletions(-) -- 2.20.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 13475ecf81..2a20c8174c 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -531,7 +531,10 @@ DEF_HELPER_FLAGS_3(crypto_sha512su0, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_4(crypto_sha512su1, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(crypto_sm3tt, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32, i32) +DEF_HELPER_FLAGS_4(crypto_sm3tt1a, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sm3tt1b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sm3tt2a, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(crypto_sm3tt2b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(crypto_sm3partw1, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(crypto_sm3partw2, TCG_CALL_NO_RWG, diff --git a/target/arm/crypto_helper.c b/target/arm/crypto_helper.c index 636683d0f1..c76806dc8d 100644 --- a/target/arm/crypto_helper.c +++ b/target/arm/crypto_helper.c @@ -632,15 +632,14 @@ void HELPER(crypto_sm3partw2)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail_16(vd, desc); } -void HELPER(crypto_sm3tt)(void *vd, void *vn, void *vm, uint32_t imm2, - uint32_t opcode) +static inline void QEMU_ALWAYS_INLINE +crypto_sm3tt(uint64_t *rd, uint64_t *rn, uint64_t *rm, + uint32_t desc, uint32_t opcode) { - uint64_t *rd = vd; - uint64_t *rn = vn; - uint64_t *rm = vm; union CRYPTO_STATE d = { .l = { rd[0], rd[1] } }; union CRYPTO_STATE n = { .l = { rn[0], rn[1] } }; union CRYPTO_STATE m = { .l = { rm[0], rm[1] } }; + uint32_t imm2 = simd_data(desc); uint32_t t; assert(imm2 < 4); @@ -655,7 +654,7 @@ void HELPER(crypto_sm3tt)(void *vd, void *vn, void *vm, uint32_t imm2, /* SM3TT2B */ t = cho(CR_ST_WORD(d, 3), CR_ST_WORD(d, 2), CR_ST_WORD(d, 1)); } else { - g_assert_not_reached(); + qemu_build_not_reached(); } t += CR_ST_WORD(d, 0) + CR_ST_WORD(m, imm2); @@ -680,8 +679,21 @@ void HELPER(crypto_sm3tt)(void *vd, void *vn, void *vm, uint32_t imm2, rd[0] = d.l[0]; rd[1] = d.l[1]; + + clear_tail_16(rd, desc); } +#define DO_SM3TT(NAME, OPCODE) \ + void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ + { crypto_sm3tt(vd, vn, vm, desc, OPCODE); } + +DO_SM3TT(crypto_sm3tt1a, 0) +DO_SM3TT(crypto_sm3tt1b, 1) +DO_SM3TT(crypto_sm3tt2a, 2) +DO_SM3TT(crypto_sm3tt2b, 3) + +#undef DO_SM3TT + static uint8_t const sm4_sbox[] = { 0xd6, 0x90, 0xe9, 0xfe, 0xcc, 0xe1, 0x3d, 0xb7, 0x16, 0xb6, 0x14, 0xc2, 0x28, 0xfb, 0x2c, 0x05, diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 49ca7ac76e..9c1ebcc8e3 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -13861,13 +13861,15 @@ static void disas_crypto_xar(DisasContext *s, uint32_t insn) */ static void disas_crypto_three_reg_imm2(DisasContext *s, uint32_t insn) { + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_crypto_sm3tt1a, gen_helper_crypto_sm3tt1b, + gen_helper_crypto_sm3tt2a, gen_helper_crypto_sm3tt2b, + }; int opcode = extract32(insn, 10, 2); int imm2 = extract32(insn, 12, 2); int rm = extract32(insn, 16, 5); int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); - TCGv_ptr tcg_rd_ptr, tcg_rn_ptr, tcg_rm_ptr; - TCGv_i32 tcg_imm2, tcg_opcode; if (!dc_isar_feature(aa64_sm3, s)) { unallocated_encoding(s); @@ -13878,20 +13880,7 @@ static void disas_crypto_three_reg_imm2(DisasContext *s, uint32_t insn) return; } - tcg_rd_ptr = vec_full_reg_ptr(s, rd); - tcg_rn_ptr = vec_full_reg_ptr(s, rn); - tcg_rm_ptr = vec_full_reg_ptr(s, rm); - tcg_imm2 = tcg_const_i32(imm2); - tcg_opcode = tcg_const_i32(opcode); - - gen_helper_crypto_sm3tt(tcg_rd_ptr, tcg_rn_ptr, tcg_rm_ptr, tcg_imm2, - tcg_opcode); - - tcg_temp_free_ptr(tcg_rd_ptr); - tcg_temp_free_ptr(tcg_rn_ptr); - tcg_temp_free_ptr(tcg_rm_ptr); - tcg_temp_free_i32(tcg_imm2); - tcg_temp_free_i32(tcg_opcode); + gen_gvec_op3_ool(s, true, rd, rn, rm, imm2, fns[opcode]); } /* C3.6 Data processing - SIMD, inc Crypto