From patchwork Fri Apr 16 21:01:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422589 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp717362jaf; Fri, 16 Apr 2021 14:03:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx75+vAC/fCxbK5AXG8gFluYClkHb+xrdPaTzJgjQMJnfpgMfJdv5rbh8389vxvhGkhhpbf X-Received: by 2002:a54:488c:: with SMTP id r12mr8017567oic.160.1618606995708; Fri, 16 Apr 2021 14:03:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618606995; cv=none; d=google.com; s=arc-20160816; b=pHireSlyVtqb+Kba91d6Bpuu8GWE0LztLklSZXQT8pbPqY8NgcYn4EkzSifn5+rK27 milpyKSjz6tn03hnkXn/ZvN1GLT4aCEOBiNG+yGV0Sd+057LXg2+vWaLKWg5R0mphqYi VdWtBUBC2dGBgriuqDs84ui57Zfo00XnACfvxyUigMGLXHrMksz0pMZUtoIu8ZXblW/b Cqn7zt2WQcLKZY93g8S+10Vuzq+VKwgKPXed8KXSTFdgAe2H7Iy44ta0TMjAIxgDjUtG cbbCyv2jfLtn4JsvkLTaqqf0cyirEaBBkxA+cNjx3upaCqDDhDVqW6gAgRCdCgVMpY81 d+Sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=q1Vt8UmYtOZ5Zzksw1iaCjMqU/oXmihF2abTUk6CpYQ=; b=nEwpSpkT3G4CAgKGbBUb4bysEO30G1mQHTegW6Kkk+xGvBQ2ljbyaSUg2aMMm60DSi ND117rHoDH6yNcD5gH8Ttx/u9JXKH91dbcwU/6JMgaerfVkkue4bCzlMu63NuwxMT3jk jYSU41bK4mKadt5iAnQfh9/1Oa4OIeN61NSpJmOcdWaLcHfeHuGyBqN6Qh1ByfN8duOc uEGskBkGSRVYY5TXAUUi0vDZj+iSKo8avuEcekxTXHRMhwSJTtUyLHO2CnnOwULH9Cy9 UWdba5J/SjOScJxKwqKqXvBubMuCOlA6BhSVJvS4D9p370pVpBpSWfqgrjrxSifRNLu1 oCbw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="WTZX/2ij"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d13si5751281ote.18.2021.04.16.14.03.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:03:15 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="WTZX/2ij"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41682 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVcR-0003sG-0z for patch@linaro.org; Fri, 16 Apr 2021 17:03:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43274) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVc0-0003pz-KQ for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:48 -0400 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]:34738) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVbx-0000xu-Fx for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:48 -0400 Received: by mail-pl1-x633.google.com with SMTP id t22so14309078ply.1 for ; Fri, 16 Apr 2021 14:02:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q1Vt8UmYtOZ5Zzksw1iaCjMqU/oXmihF2abTUk6CpYQ=; b=WTZX/2ijU82WfMVX/lh+aXZDtZSe2ttLeK4Uy1TpcYW1pruMrGF/bU5FkoxAA4dDk7 sIW1mb+IR5JvdM+EFfojpuNwA3e0va5YXntX/c491fv2T0n00K8FExBA2AHwD7gcys9v WsRvG7VKarqhW4vPHV+1/73XHqB+gmnroNv8WJIRKEWr/gDS3yBxxQOFTcEc6uaclb7M Sd7e5U6JJGA3yi3vtliZ+ek9hyyDcncqYCLaTSj0qcYQlsgMh6MG4g/c7BydRuDKzNQ/ e9TSApZEKV8JIE0rFeKhR0W3psV06O+J2+LgEav85tWKwatJ8HWamQkZQMKepluw5ISi 0HIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q1Vt8UmYtOZ5Zzksw1iaCjMqU/oXmihF2abTUk6CpYQ=; b=h0Fo2eJwW3q/b2AY7hnvjf9p6in1B3FU/D1+4ndFPNDqKoh2BWsrASHuq7EiX9uZrX 2Ori5dtzZ58gbBGCoDUp2e3/jMavI8hEX0CmTNmm+k4aorpWZxMzF/x77mAO9tKwaVXQ cgBSRy8rBfQsNB+SVirtuNEFso2S/7hPICJZeMobXeb29ZEoJJ7a5Wgt97O5skM534Ea zy9Dljm4pLPSN7fTkBEcF938Hp+RJGkfTQCC967ft/pO0C3s/QZEORwE7e3GDHGIdqeN 3f+UkVCMyHKe8Pb4P2BQsXtGhLazQgK4CEn5bo+EgV4/63je6KPpeh9ZKHs/cvIPv6kG 1w0w== X-Gm-Message-State: AOAM531e6jFGHhdpOjIhNsnlV3u5EzXsRkSgvZSpvVOxn8Sh5Unhtse6 4flPh/cBIST82L8x7MDiY5/7GKIcnB+RTA== X-Received: by 2002:a17:90a:bb94:: with SMTP id v20mr11421140pjr.144.1618606964240; Fri, 16 Apr 2021 14:02:44 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:43 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 01/81] target/arm: Add ID_AA64ZFR0 fields and isar_feature_aa64_sve2 Date: Fri, 16 Apr 2021 14:01:20 -0700 Message-Id: <20210416210240.1591291-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::633; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x633.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, =?utf-8?q?Alex_Benn=C3=A9e?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Will be used for SVE2 isa subset enablement. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- v2: Do not read zfr0 from kvm unless sve is available. --- target/arm/cpu.h | 16 ++++++++++++++++ target/arm/helper.c | 3 +-- target/arm/kvm64.c | 11 +++++++++++ 3 files changed, 28 insertions(+), 2 deletions(-) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 616b393253..a6e1fa6333 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -947,6 +947,7 @@ struct ARMCPU { uint64_t id_aa64mmfr2; uint64_t id_aa64dfr0; uint64_t id_aa64dfr1; + uint64_t id_aa64zfr0; } isar; uint64_t midr; uint32_t revidr; @@ -2034,6 +2035,16 @@ FIELD(ID_AA64DFR0, DOUBLELOCK, 36, 4) FIELD(ID_AA64DFR0, TRACEFILT, 40, 4) FIELD(ID_AA64DFR0, MTPMU, 48, 4) +FIELD(ID_AA64ZFR0, SVEVER, 0, 4) +FIELD(ID_AA64ZFR0, AES, 4, 4) +FIELD(ID_AA64ZFR0, BITPERM, 16, 4) +FIELD(ID_AA64ZFR0, BFLOAT16, 20, 4) +FIELD(ID_AA64ZFR0, SHA3, 32, 4) +FIELD(ID_AA64ZFR0, SM4, 40, 4) +FIELD(ID_AA64ZFR0, I8MM, 44, 4) +FIELD(ID_AA64ZFR0, F32MM, 52, 4) +FIELD(ID_AA64ZFR0, F64MM, 56, 4) + FIELD(ID_DFR0, COPDBG, 0, 4) FIELD(ID_DFR0, COPSDBG, 4, 4) FIELD(ID_DFR0, MMAPDBG, 8, 4) @@ -4215,6 +4226,11 @@ static inline bool isar_feature_aa64_ssbs(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, SSBS) != 0; } +static inline bool isar_feature_aa64_sve2(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SVEVER) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper.c b/target/arm/helper.c index cd92a22689..ec8d477516 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -7561,8 +7561,7 @@ void register_cp_regs_for_features(ARMCPU *cpu) .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 4, .access = PL1_R, .type = ARM_CP_CONST, .accessfn = access_aa64_tid3, - /* At present, only SVEver == 0 is defined anyway. */ - .resetvalue = 0 }, + .resetvalue = cpu->isar.id_aa64zfr0 }, { .name = "ID_AA64PFR5_EL1_RESERVED", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 5, .access = PL1_R, .type = ARM_CP_CONST, diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index dff85f6db9..e8729b53fd 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -567,6 +567,17 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf) err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64mmfr2, ARM64_SYS_REG(3, 0, 0, 7, 2)); + /* + * Before v5.1, KVM did not support SVE and did not expose + * ID_AA64ZFR0_EL1 even as RAZ. After v5.1, KVM still does + * not expose the register to "user" requests like this + * unless the host supports SVE. + */ + if (isar_feature_aa64_sve(&ahcf->isar)) { + err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64zfr0, + ARM64_SYS_REG(3, 0, 0, 4, 4)); + } + /* * Note that if AArch32 support is not present in the host, * the AArch32 sysregs are present to be read, but will From patchwork Fri Apr 16 21:01:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422591 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp718358jaf; Fri, 16 Apr 2021 14:04:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzNhas/wxsX9866wH+Wip6uS6digzF+ZYB9cCYsP/qG+egSDM3NXGKqM2GTaFYY4uyUBHWX X-Received: by 2002:a9d:1ea9:: with SMTP id n38mr399840otn.233.1618607079911; Fri, 16 Apr 2021 14:04:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607079; cv=none; d=google.com; s=arc-20160816; b=hnvGcAyS/tvr7m6ywnDfLGq/n8Gzg9ujrOmkJJrheWCoDXxAwxpIDW1S3dLOC9lZia UQr4NORnXA2UpaNZoo9wxdHmRALi7Tbk+1i5H9OgWiiF8acuwzXF0kv/MfNf8dGbqkP8 f2yZgkhElP9WIFY/zNAifQ3XpOHhk/D206S7ESV5h0Hk4Ku1hUcE+6fEyRWmUc+teWNO NsltXnJqM2TqEF/j6OF0B8n9fGcL3stXd79LvWDJF64mNm9xs7kZ7nByvfzI3486Q6Xn bVGyg4ZPp9oD1r9gIUGMYtL4YhExZ4YOtplANZgaIbIB8RqajUZl81Xo4ksEUtcnK2yC wtwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Y5trnwoiyeOfl0r4vLyT45eBorm8d2BCBnFoodmdZyM=; b=SO/lssAIm4GsusYyHvATzcSxEIO5hposws6jVJPiKFCaE4SdGBoCP9nUCnR5iM+a8T HYg9Y+UZeo2EYYnnkSlpnZIhHs/s+gTkNn3w0J9bomHlrxtCuji3NqK9sefkPDigvq9G GPL8O4wymsl5yjHCyU4NHzpr1PlLCHNPVIbXh3w9K3+BXdj6N+UY/oqk5GBFVxo3pOB/ XiRrlgmTSJKEoko2q0jjj1TaVy2JPAj00IgWxrulgLxP/OblCR9L9Jmg1awqPZd5mk6h Ntsz0algJ4WdFqWSwBo0eZaI4b/unnq47iSlnvohTANC3tQDobwlRHND+bOQu8vaDeGn ZzZw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RjdsBh9w; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 185si7915890oig.193.2021.04.16.14.04.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:04:39 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RjdsBh9w; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42820 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVdn-0004Pf-58 for patch@linaro.org; Fri, 16 Apr 2021 17:04:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43286) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVc1-0003qf-5h for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:49 -0400 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]:41618) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVby-0000yB-U5 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:48 -0400 Received: by mail-pg1-x52e.google.com with SMTP id f29so20001253pgm.8 for ; Fri, 16 Apr 2021 14:02:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Y5trnwoiyeOfl0r4vLyT45eBorm8d2BCBnFoodmdZyM=; b=RjdsBh9w9wOlB9+Cdd0kMt6dXluOCMJf2ScEvjslmbrVSD6Qu6XPM4qPNqr66ABJtM r9qqqDp/ZuXNakhLjOJTIjK6ShJ3tHahZbkW0x4oWkaM8YLCJzr3/aSYKFpzSZ9g5w+P V8qoONpWDIOKetwXSsg2l/0G/CX4a39TAigBHCbord+5sc5RF1kcjF3LsyAqVu/Ud5eT oJS+aTpXuIIBkkv8puI+EYuhM2m8e998jUsjxEpehKDq3RB8nYvtvnFTW8IAricXSFrV BETaFnUwhL539Y7Ir5oz/4mZ4sDwFTtLUUMu47nUSRqlngCV6wwdt0SPpaNIdSMg2NcP KNsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y5trnwoiyeOfl0r4vLyT45eBorm8d2BCBnFoodmdZyM=; b=T43QcG3hzrfZRYQ5pyu5fo9PjgP2A3IawdKRHczCOTtzbe3CUx2vQExL5BsKBWHXcF 4hqLl4OW8oW/RvuiIH5LadGZkbBI2f7xbiP9LjEuKbhWgyhZ3easZrXmgcA2ApW/dtUx p0IIG5oXw0+imJytAVPdmg9Ml0mOsU7cfniO0Sw+VeoOBR2eI4oN8CtWJ09iv41NgL95 LRJd/6rZ1abe9K8gW7CyCYTbhOypOb5p3LAAaCnvo/z/U2o6dzay/Q9dFbEuX09VFpOG lIMebEvNsM5lRaYlZ9gufnXXsqPAWaNMcC71PBnQa0P9sTWFeMnzKnD94eE6rfUZ9T9w EtLw== X-Gm-Message-State: AOAM532xEdFbGQbruFCKb/P5YfnnjDGhFnkijPOLZsl8/BBOKau8YbDu 62tK5ihHqX7gzNEPUjuP/iVeLmqaYnafWw== X-Received: by 2002:a65:5b85:: with SMTP id i5mr782301pgr.269.1618606965464; Fri, 16 Apr 2021 14:02:45 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:45 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 02/81] target/arm: Implement SVE2 Integer Multiply - Unpredicated Date: Fri, 16 Apr 2021 14:01:21 -0700 Message-Id: <20210416210240.1591291-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52e; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For MUL, we can rely on generic support. For SMULH and UMULH, create some trivial helpers. For PMUL, back in a21bb78e5817, we organized helper_gvec_pmul_b in preparation for this use. Signed-off-by: Richard Henderson --- target/arm/helper.h | 10 ++++ target/arm/sve.decode | 10 ++++ target/arm/translate-sve.c | 50 ++++++++++++++++++++ target/arm/vec_helper.c | 96 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 166 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index ff8148ddc6..2c412ffd3b 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -828,6 +828,16 @@ DEF_HELPER_FLAGS_3(gvec_cgt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(gvec_cge0_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(gvec_cge0_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_umulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_sshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_sshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_ushl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 5c90603358..557706cacb 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1090,3 +1090,13 @@ ST1_zprz 1110010 .. 00 ..... 100 ... ..... ..... \ @rprr_scatter_store xs=0 esz=3 scale=0 ST1_zprz 1110010 .. 00 ..... 110 ... ..... ..... \ @rprr_scatter_store xs=1 esz=3 scale=0 + +#### SVE2 Support + +### SVE2 Integer Multiply - Unpredicated + +# SVE2 integer multiply vectors (unpredicated) +MUL_zzz 00000100 .. 1 ..... 0110 00 ..... ..... @rd_rn_rm +SMULH_zzz 00000100 .. 1 ..... 0110 10 ..... ..... @rd_rn_rm +UMULH_zzz 00000100 .. 1 ..... 0110 11 ..... ..... @rd_rn_rm +PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 864ed669c4..f82d7d96f6 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5795,3 +5795,53 @@ static bool trans_MOVPRFX_z(DisasContext *s, arg_rpr_esz *a) { return do_movz_zpz(s, a->rd, a->rn, a->pg, a->esz, false); } + +/* + * SVE2 Integer Multiply - Unpredicated + */ + +static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzz(s, tcg_gen_gvec_mul, a->esz, a->rd, a->rn, a->rm); + } + return true; +} + +static bool do_sve2_zzz_ool(DisasContext *s, arg_rrr_esz *a, + gen_helper_gvec_3 *fn) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); + } + return true; +} + +static bool trans_SMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_gvec_smulh_b, gen_helper_gvec_smulh_h, + gen_helper_gvec_smulh_s, gen_helper_gvec_smulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + +static bool trans_UMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_gvec_umulh_b, gen_helper_gvec_umulh_h, + gen_helper_gvec_umulh_s, gen_helper_gvec_umulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + +static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a) +{ + return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 3fbeae87cb..40b92100bf 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -1985,3 +1985,99 @@ void HELPER(simd_tblx)(void *vd, void *vm, void *venv, uint32_t desc) clear_tail(vd, oprsz, simd_maxsz(desc)); } #endif + +/* + * NxN -> N highpart multiply + * + * TODO: expose this as a generic vector operation. + */ + +void HELPER(gvec_smulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ((int32_t)n[i] * m[i]) >> 8; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_smulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = ((int32_t)n[i] * m[i]) >> 16; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_smulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = ((int64_t)n[i] * m[i]) >> 32; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_smulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn, *m = vm; + uint64_t discard; + + for (i = 0; i < opr_sz / 8; ++i) { + muls64(&discard, &d[i], n[i], m[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ((uint32_t)n[i] * m[i]) >> 8; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint16_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = ((uint32_t)n[i] * m[i]) >> 16; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint32_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = ((uint64_t)n[i] * m[i]) >> 32; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn, *m = vm; + uint64_t discard; + + for (i = 0; i < opr_sz / 8; ++i) { + mulu64(&discard, &d[i], n[i], m[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} From patchwork Fri Apr 16 21:01:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422594 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp720150jaf; Fri, 16 Apr 2021 14:07:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzFoAARIqxusXR+cdnY/fnyttZRZECXMeAyHTprVDx2MczL07s7bX8oDgUuFrCva5EVULDj X-Received: by 2002:a9f:3728:: with SMTP id z37mr1155200uad.89.1618607239102; Fri, 16 Apr 2021 14:07:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607239; cv=none; d=google.com; s=arc-20160816; b=AduDSL6jbDgudPaHmnV+62yAlVSlWkRemMQmwx6L8A6Rm5Dns8xRjPB0u8A21sOqzd S0xvhVks9I9ZT+TInH63064vT8TgqnPDQZK9trq5bm79Bm8x/Gi2hD/Prvzw5lu0sv6G F0q/73yuel8HiDosYHvyIlAWH9mshd7XK9m6M9Ly/LoDyRzSlcyYFNREpl4CcsHmyaN5 Iq7yznVShdR6WOil6cdWrOxWGDEGBn7M/ZPcDj5ln3KAsfleHk1K5FLDGoaV9ubxkD7e fyeUPtyc3E+B3J+0t2edDrfomvRIttYuNLpWQK8wi6f7Zfv0QPXRzR4gcePJUcSRmtOT NbxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=CQ6txFH7Dtop6I+Akap+iOzHMzqCTj0Z29F/hci/3rc=; b=FLwEeD/GM3TK+hIwcwZGhYmeKGKLQAbtJlL74T5H1g+qD+oIxLpzY7huS/NB57zrGJ pnJi9piQCEjc4lwUSRxdoEPpkEXbQ4uBw/CBVcPGQa73PbIJ1VpdMIEvM3T6WyvtJ0vn W+84z07goTPCEx6OhYzVV7ZdRS5tJ2RnhO0YhsIrMFxc89p1gRSiez5IevHsolf/KYfl 0pr5CHglNxSX/7P806CuEQmPWPTY1WitpgdxM4lH/1vpEp3TKPCuG1Py4wWBM49RPdPR sHea5DEl2N3Bwmp8KpNWTX1shh+MEXc/+ooJNKODwfIL2SsEfdMrnyoT5SJZAhImErbu a2Dg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qgPh8iLV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id u18si3319267vsj.352.2021.04.16.14.07.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:07:19 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qgPh8iLV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51076 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVgM-0008PW-8r for patch@linaro.org; Fri, 16 Apr 2021 17:07:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43312) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVc2-0003sQ-2T for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:50 -0400 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]:34722) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVbz-0000yh-M9 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:49 -0400 Received: by mail-pj1-x1030.google.com with SMTP id em21-20020a17090b0155b029014e204a81e6so6949981pjb.1 for ; Fri, 16 Apr 2021 14:02:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CQ6txFH7Dtop6I+Akap+iOzHMzqCTj0Z29F/hci/3rc=; b=qgPh8iLVtnRWwesx6ySNQerMYz4pBHPue3TwVKQ5YeWHDRQpnNaHoFDMfISP/vFwcQ u7dPfGkoHXOwb+MihXshDKLRISTP90GQLy8ypShUPv1E2uRVdAOdVeuGp2TIPMDsCFrc kw3y4aslHnm5fC899Wl9YPcXHWncRGFY8y44+FLIGUos+kNNBxS5A9nvq3hrXbi/aiNB aF9ILlseACukR9HL7EVzPCBmFY3yspPrxHzgfvbK9oUg9bw+dCs0vd5j5WU4dEgwShFe 3l7lDJ9EeJaUnBLcOQzLhHVJcux0YbJOEhndI53pbz2sdyZx/LqxVaYxm47Gw0vSmMEE P4ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CQ6txFH7Dtop6I+Akap+iOzHMzqCTj0Z29F/hci/3rc=; b=uXbytoQbMQo9Cb9DACOVea9ZIOnkqm3m0xXFfUZauc8wFvjianuHApnT8zXBHqd9yS dFpQeq/6Bsciw3fXG/kP9BeAvenQLWcvAxH/haFl7h8UIaXZwcj/Wx54ryuV/zm4ulGY GCPbKUu7QJ79Po9pHc9TydfSGI6NlQjS7Eq8CWqXwhhoXEYex6KEwztRrdncYKcBb16c XhYxcIpUP724HqElqmf5gfpfsR/m4tJy3wKq5fzDIWaV7TVWnOB7uyAWqbVmrYnVoL/k N+YUN8Qx5pOr25eTrWJI7Jygv15VA83aASqs7ZNY4Relnv9wvGfXPiaftWd+vtkcSEhG MT/w== X-Gm-Message-State: AOAM5305kL3gPCNXWmi8aWISstUyWeTLObTWv6K5lK1Qzy2Ww7AZFYGl UbLC+Onbhdbx4cV7ok3EyHWu5JG4DFKqDg== X-Received: by 2002:a17:90a:6b84:: with SMTP id w4mr11570813pjj.134.1618606966319; Fri, 16 Apr 2021 14:02:46 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:45 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 03/81] target/arm: Implement SVE2 integer pairwise add and accumulate long Date: Fri, 16 Apr 2021 14:01:22 -0700 Message-Id: <20210416210240.1591291-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1030.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 14 ++++++++++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 44 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 39 +++++++++++++++++++++++++++++++++ 4 files changed, 102 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index e4cadd2a65..b2a274b40b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -158,6 +158,20 @@ DEF_HELPER_FLAGS_5(sve_umulh_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve_umulh_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sadalp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sadalp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sadalp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_d, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 557706cacb..0524c01fcf 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1100,3 +1100,8 @@ MUL_zzz 00000100 .. 1 ..... 0110 00 ..... ..... @rd_rn_rm SMULH_zzz 00000100 .. 1 ..... 0110 10 ..... ..... @rd_rn_rm UMULH_zzz 00000100 .. 1 ..... 0110 11 ..... ..... @rd_rn_rm PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 + +### SVE2 Integer - Predicated + +SADALP_zpzz 01000100 .. 000 100 101 ... ..... ..... @rdm_pg_rn +UADALP_zpzz 01000100 .. 000 101 101 ... ..... ..... @rdm_pg_rn diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index c068dfa0d5..42fe315485 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -517,6 +517,50 @@ DO_ZPZZ_D(sve_asr_zpzz_d, int64_t, DO_ASR) DO_ZPZZ_D(sve_lsr_zpzz_d, uint64_t, DO_LSR) DO_ZPZZ_D(sve_lsl_zpzz_d, uint64_t, DO_LSL) +static inline uint16_t do_sadalp_h(uint16_t n, uint16_t m) +{ + int8_t n1 = n, n2 = n >> 8; + return m + n1 + n2; +} + +static inline uint32_t do_sadalp_s(uint32_t n, uint32_t m) +{ + int16_t n1 = n, n2 = n >> 16; + return m + n1 + n2; +} + +static inline uint64_t do_sadalp_d(uint64_t n, uint64_t m) +{ + int32_t n1 = n, n2 = n >> 32; + return m + n1 + n2; +} + +DO_ZPZZ(sve2_sadalp_zpzz_h, int16_t, H1_2, do_sadalp_h) +DO_ZPZZ(sve2_sadalp_zpzz_s, int32_t, H1_4, do_sadalp_s) +DO_ZPZZ_D(sve2_sadalp_zpzz_d, uint64_t, do_sadalp_d) + +static inline uint16_t do_uadalp_h(uint16_t n, uint16_t m) +{ + uint8_t n1 = n, n2 = n >> 8; + return m + n1 + n2; +} + +static inline uint32_t do_uadalp_s(uint32_t n, uint32_t m) +{ + uint16_t n1 = n, n2 = n >> 16; + return m + n1 + n2; +} + +static inline uint64_t do_uadalp_d(uint64_t n, uint64_t m) +{ + uint32_t n1 = n, n2 = n >> 32; + return m + n1 + n2; +} + +DO_ZPZZ(sve2_uadalp_zpzz_h, int16_t, H1_2, do_uadalp_h) +DO_ZPZZ(sve2_uadalp_zpzz_s, int32_t, H1_4, do_uadalp_s) +DO_ZPZZ_D(sve2_uadalp_zpzz_d, uint64_t, do_uadalp_d) + #undef DO_ZPZZ #undef DO_ZPZZ_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index f82d7d96f6..208d9ea7e0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5845,3 +5845,42 @@ static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a) { return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b); } + +/* + * SVE2 Integer - Predicated + */ + +static bool do_sve2_zpzz_ool(DisasContext *s, arg_rprr_esz *a, + gen_helper_gvec_4 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzz_ool(s, a, fn); +} + +static bool trans_SADALP_zpzz(DisasContext *s, arg_rprr_esz *a) +{ + static gen_helper_gvec_4 * const fns[3] = { + gen_helper_sve2_sadalp_zpzz_h, + gen_helper_sve2_sadalp_zpzz_s, + gen_helper_sve2_sadalp_zpzz_d, + }; + if (a->esz == 0) { + return false; + } + return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); +} + +static bool trans_UADALP_zpzz(DisasContext *s, arg_rprr_esz *a) +{ + static gen_helper_gvec_4 * const fns[3] = { + gen_helper_sve2_uadalp_zpzz_h, + gen_helper_sve2_uadalp_zpzz_s, + gen_helper_sve2_uadalp_zpzz_d, + }; + if (a->esz == 0) { + return false; + } + return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); +} From patchwork Fri Apr 16 21:01:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422593 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp719460jaf; Fri, 16 Apr 2021 14:06:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy984Qm3sNikewOfi3vCDquvBHF4GZlVqtQcTc+5xCeUH8yWO7SLAdRCKwLt6ORMPkeyWpN X-Received: by 2002:ab0:39d9:: with SMTP id g25mr1157445uaw.77.1618607178926; Fri, 16 Apr 2021 14:06:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607178; cv=none; d=google.com; s=arc-20160816; b=ZfPBKG2OflT/MNVjBkpGKiQ+MAbS6PIWUvpCVJnpav0bCXvFWXIPNecJd0EzxoQRfT A59frwrsKMUa1TJpZwHmHv+gRNsbDv3Cjsvv00flc4tOXiowSl+CPdubb7H51mpH1IW6 0g7BUUx6+ZepNuPYLZw+lFE9yojs2qSfxehCZKtnWiVMyOTMLwdFPcsYLQM7JXFGF+Ze CG2vbobnueyoLqVWwPt/+Hff96zrcDavEA6cALHYtB+bVV8XP1icCn1Gmob4I70FYk15 hejHO6IVKXXttOQRuoCOEg5Skz91093swQwzaedOM77EudrOOUA5NlQy0KfgeTJcn8td 3VTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=FXd4wOc5+bYMVSLk2jKG60M24P25fU16HFUkpUXY4d4=; b=wmzEFMpHMZuNYeJ4arXoWnDwwg7TymwcIvDzL4gEkSuyx2DP5Sj8tXBkTPODwznGr7 KJ3+ozs7S8owIFAqH7dHQzarfQyGeZV2kseMb5Iv6MUcZjGeRfangmzPWmJhwlIoEqri 9sbpYWMp1qLu5oRJo+CJx2BF4/7GW80/D0PPo9ePiwSt5j6lRGRAt8uqIvQHEc6v1ppC HpkayExvlkUDRavKwPbFJDBigkn1A38s7A6SqiVCcXkeDtOgFE6CkABAlLfN5SDVMIJp Hn62rFE5WkasWmoC6cAJAD7w/5G0+8WzH3AsHPl1aIPTWT+f3TEEccYBxXzcqBYgRUOI WV5g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wqHcwLOb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a29si3264967vsl.318.2021.04.16.14.06.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:06:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wqHcwLOb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51036 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVfO-0008Oo-3A for patch@linaro.org; Fri, 16 Apr 2021 17:06:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43318) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVc2-0003tu-NJ for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:50 -0400 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]:45760) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVc0-0000zo-Fk for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:50 -0400 Received: by mail-pl1-x629.google.com with SMTP id p16so10675660plf.12 for ; Fri, 16 Apr 2021 14:02:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FXd4wOc5+bYMVSLk2jKG60M24P25fU16HFUkpUXY4d4=; b=wqHcwLObTYqc+hsjlSTgEixMBoi38HkymluxDacv7vUW4U9LQfvwNEhe1zbYvpJlbg aUuau2hu/KkF85cfG7uZuBPxPLLrcR1Kq4D2CbO3aGNScQnZSQh+P27PTKXMX4HKyrgW 0m0Xhyluzmbsb9FCRHas/UYcdayO7ZnBnuRd/au1Ac3U5sEywZAjm0Jg5U2Y5WZ5qtuP iuaSKgKvPeiqrQMjbp3pKVVD4nifnempjvyMQJqqMInyIRVbzTVhLTlga3At2hlJmhpz ZGxWH6ZJNLM93577IozetAWXqArj5CRrhN0cMmsWZeRIcW5lBUzMap5ilH7neWK13xdh 78HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FXd4wOc5+bYMVSLk2jKG60M24P25fU16HFUkpUXY4d4=; b=GsNTq/3o5fsXTnSlyNGmeuQURClFeMJ3Xvar6ZRok9eD2dpYHwjq6gUMrOkTtV7t8L uBsVQIk1bXGR+Zk4EdUwhuDLul+tPhNjODkgrVoaCzzEa5yPs3X6zAfk7/zG4d7YvmDU UWoJswSAjfKH9xGVkyh3uPJCq7pqBr00we3PyeXm30DQtwyhoks5RpVD/pEYTtL5sNcH 5uSZyj1B/VfPBegmHkunRR+GnhjFSrzRqNY/JZdgo1LkYWRP7j0mRzhd7Ct5p/KYVIBm W2rK0wtGsRkGU4Hzu3dmJT/Ujn8GuW3uUHD6ta5hZD0CmwxPLsLKVooBcauxPYmobpAx IOnw== X-Gm-Message-State: AOAM5330pxOVWx19PWQdyKnVQ8fHa/nSMV4ffll9jagaTLCrCfNdtjSz 2MsNDeBy4978+PdyoC8fKPTY/bBkoNaBYA== X-Received: by 2002:a17:90a:420d:: with SMTP id o13mr11780832pjg.61.1618606967311; Fri, 16 Apr 2021 14:02:47 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:46 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 04/81] target/arm: Implement SVE2 integer unary operations (predicated) Date: Fri, 16 Apr 2021 14:01:23 -0700 Message-Id: <20210416210240.1591291-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x629.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix sqabs, sqneg (laurent desnogues) --- target/arm/helper-sve.h | 13 +++++++++++ target/arm/sve.decode | 7 ++++++ target/arm/sve_helper.c | 29 +++++++++++++++++++---- target/arm/translate-sve.c | 47 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 92 insertions(+), 4 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index b2a274b40b..9992e93e2b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -502,6 +502,19 @@ DEF_HELPER_FLAGS_4(sve_rbit_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_rbit_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_rbit_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqneg_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqneg_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqneg_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqneg_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_urecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ursqrte_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_splice, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_cmpeq_ppzz_b, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 0524c01fcf..5ba542969b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1105,3 +1105,10 @@ PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 SADALP_zpzz 01000100 .. 000 100 101 ... ..... ..... @rdm_pg_rn UADALP_zpzz 01000100 .. 000 101 101 ... ..... ..... @rdm_pg_rn + +### SVE2 integer unary operations (predicated) + +URECPE 01000100 .. 000 000 101 ... ..... ..... @rd_pg_rn +URSQRTE 01000100 .. 000 001 101 ... ..... ..... @rd_pg_rn +SQABS 01000100 .. 001 000 101 ... ..... ..... @rd_pg_rn +SQNEG 01000100 .. 001 001 101 ... ..... ..... @rd_pg_rn diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 42fe315485..bbab84e81d 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -535,8 +535,8 @@ static inline uint64_t do_sadalp_d(uint64_t n, uint64_t m) return m + n1 + n2; } -DO_ZPZZ(sve2_sadalp_zpzz_h, int16_t, H1_2, do_sadalp_h) -DO_ZPZZ(sve2_sadalp_zpzz_s, int32_t, H1_4, do_sadalp_s) +DO_ZPZZ(sve2_sadalp_zpzz_h, uint16_t, H1_2, do_sadalp_h) +DO_ZPZZ(sve2_sadalp_zpzz_s, uint32_t, H1_4, do_sadalp_s) DO_ZPZZ_D(sve2_sadalp_zpzz_d, uint64_t, do_sadalp_d) static inline uint16_t do_uadalp_h(uint16_t n, uint16_t m) @@ -557,8 +557,8 @@ static inline uint64_t do_uadalp_d(uint64_t n, uint64_t m) return m + n1 + n2; } -DO_ZPZZ(sve2_uadalp_zpzz_h, int16_t, H1_2, do_uadalp_h) -DO_ZPZZ(sve2_uadalp_zpzz_s, int32_t, H1_4, do_uadalp_s) +DO_ZPZZ(sve2_uadalp_zpzz_h, uint16_t, H1_2, do_uadalp_h) +DO_ZPZZ(sve2_uadalp_zpzz_s, uint32_t, H1_4, do_uadalp_s) DO_ZPZZ_D(sve2_uadalp_zpzz_d, uint64_t, do_uadalp_d) #undef DO_ZPZZ @@ -728,6 +728,27 @@ DO_ZPZ(sve_rbit_h, uint16_t, H1_2, revbit16) DO_ZPZ(sve_rbit_s, uint32_t, H1_4, revbit32) DO_ZPZ_D(sve_rbit_d, uint64_t, revbit64) +#define DO_SQABS(X) \ + ({ __typeof(X) x_ = (X), min_ = 1ull << (sizeof(X) * 8 - 1); \ + x_ >= 0 ? x_ : x_ == min_ ? -min_ - 1 : -x_; }) + +DO_ZPZ(sve2_sqabs_b, int8_t, H1, DO_SQABS) +DO_ZPZ(sve2_sqabs_h, int16_t, H1_2, DO_SQABS) +DO_ZPZ(sve2_sqabs_s, int32_t, H1_4, DO_SQABS) +DO_ZPZ_D(sve2_sqabs_d, int64_t, DO_SQABS) + +#define DO_SQNEG(X) \ + ({ __typeof(X) x_ = (X), min_ = 1ull << (sizeof(X) * 8 - 1); \ + x_ == min_ ? -min_ - 1 : -x_; }) + +DO_ZPZ(sve2_sqneg_b, uint8_t, H1, DO_SQNEG) +DO_ZPZ(sve2_sqneg_h, uint16_t, H1_2, DO_SQNEG) +DO_ZPZ(sve2_sqneg_s, uint32_t, H1_4, DO_SQNEG) +DO_ZPZ_D(sve2_sqneg_d, uint64_t, DO_SQNEG) + +DO_ZPZ(sve2_urecpe_s, uint32_t, H1_4, helper_recpe_u32) +DO_ZPZ(sve2_ursqrte_s, uint32_t, H1_4, helper_rsqrte_u32) + /* Three-operand expander, unpredicated, in which the third operand is "wide". */ #define DO_ZZW(NAME, TYPE, TYPEW, H, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 208d9ea7e0..c30b3c476e 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5884,3 +5884,50 @@ static bool trans_UADALP_zpzz(DisasContext *s, arg_rprr_esz *a) } return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); } + +/* + * SVE2 integer unary operations (predicated) + */ + +static bool do_sve2_zpz_ool(DisasContext *s, arg_rpr_esz *a, + gen_helper_gvec_3 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ool(s, a, fn); +} + +static bool trans_URECPE(DisasContext *s, arg_rpr_esz *a) +{ + if (a->esz != 2) { + return false; + } + return do_sve2_zpz_ool(s, a, gen_helper_sve2_urecpe_s); +} + +static bool trans_URSQRTE(DisasContext *s, arg_rpr_esz *a) +{ + if (a->esz != 2) { + return false; + } + return do_sve2_zpz_ool(s, a, gen_helper_sve2_ursqrte_s); +} + +static bool trans_SQABS(DisasContext *s, arg_rpr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqabs_b, gen_helper_sve2_sqabs_h, + gen_helper_sve2_sqabs_s, gen_helper_sve2_sqabs_d, + }; + return do_sve2_zpz_ool(s, a, fns[a->esz]); +} + +static bool trans_SQNEG(DisasContext *s, arg_rpr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqneg_b, gen_helper_sve2_sqneg_h, + gen_helper_sve2_sqneg_s, gen_helper_sve2_sqneg_d, + }; + return do_sve2_zpz_ool(s, a, fns[a->esz]); +} From patchwork Fri Apr 16 21:01:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422595 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp720365jaf; Fri, 16 Apr 2021 14:07:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx76AET3MBC3/9nzZbpOEAFCaanuoo9Z9dXTxK8R1QAmWLQ0ZZ9Jg8sfQi+fIIZZOxWT8Ub X-Received: by 2002:a67:8c07:: with SMTP id o7mr8942891vsd.43.1618607256368; Fri, 16 Apr 2021 14:07:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607256; cv=none; d=google.com; s=arc-20160816; b=0/2A2qMf79O2eShWCWH6dJmFPTtWKg1LBZXVOVasfTAyVDeZwhLrqKU5o/wWmgT6k9 cUwDwliKFHIitTXhwF7gNDI/FhZPaypkLWICquTPk2MF5j/HzHDjfcU98XTQoT3X6LFz HEoI/E+66R3OkqANgaAM85pmsiRGxL7eePtkkzoEjnFTLm7zNYGr/hOQ6AlE0yqwEy6g XfrTK+8GXO4navxmmJ/EHIm0Ub/VTEO8AnvhtlQrlNr5pBrPO3edaYIjqVDGSw66CBxR DD1ySeH6QhCpk3RlOBWo+eYDtLO4HhNGDz9EhTaSyMg/ZKUJNOsW2Xps/rYhXC1bCvZa hsww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=DS4qOR8a0wrDRRaz52YdBuOsYQMMqkUfZ7NchqJV82A=; b=msKMx6o0LR+aKp0C42fYOEQvjw7VEyemyaJWWhHVhjjGYOKC7aH74A1WxfVJtDu7lX xtb7hDqMNSshX0tePxzRjKBt4+22vaQT9z9ALo58DDRPss/pWjydMx4gsrZL1Ltj7F5A BAmOUs0hcmhC2pGjuJdet/PGLONUg8mHwBpf6JHzD7M+d0SPVhwdAfUZmE3mnbnNSrOs RUgDgOKSMSRDaW/dttwD4IJG/myUF2Sr5DIj70cdQGS8EsCtlSSOUQtaQ7GrislDBi1n IdDd/7E1fq/PNUjkwfHxEypfvxC4vXd9ijb2wFfOKHzLcv6IvtxTc84CCr8QTqDmS3lh gWjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="GXc/IAxw"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r2si3119556uaw.228.2021.04.16.14.07.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:07:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="GXc/IAxw"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVgd-0001N6-LS for patch@linaro.org; Fri, 16 Apr 2021 17:07:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43436) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVc7-00044W-22 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:55 -0400 Received: from mail-pg1-x536.google.com ([2607:f8b0:4864:20::536]:39511) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVc1-00010f-O1 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:54 -0400 Received: by mail-pg1-x536.google.com with SMTP id j32so5229801pgm.6 for ; Fri, 16 Apr 2021 14:02:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DS4qOR8a0wrDRRaz52YdBuOsYQMMqkUfZ7NchqJV82A=; b=GXc/IAxwa5Z8TbxaXgqgfyRpngBhac6+o82eLdsd5IuDAMjgZQXyzwLWjFR1XrXnFj KGJLREzHxYZTocr59sND6Z2G5K83KM7vQ+dw9VYpojhIwTEKcKoQiE436C1CnhX3wP1J 3z1Njwd/cVrXlD/76q5AMQJIl9o0FUCp1t672yqlJvOi1lzOeG2CRxsmkF9aBOMX0dCC busmYVs3BVVovfcYJFqlKqccDnkRswdN7GJd3UfNajrkBEAqDouGDYRUt8/ndWjvf+ib D9tByUmXkKCo4TEh4sI2qAS4uzxzMXcjRGs0oEw6LtUwaZsstsbBgkK2kKZuv8+6nEc5 kmDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DS4qOR8a0wrDRRaz52YdBuOsYQMMqkUfZ7NchqJV82A=; b=oF0pdVyXTfQ+ac5JPzLYKtHqNCL8GpbU2WHIgfG2ZQouXrmzDNQVTL4ow2e1Jns+Hl uQgNY3lyT9NIkqZEmw4Mn2mtcHyq+L1Wl7PjKdAI9CqT5zAzuPnqn8lhHVHKaQyBZfSg sxldmJodWzqGJiwfl5myRzSOPdvFsnHabfFq6nmUkCZxAxXefhvezBcmiGUpmwnmqrcC z+oDKr0k6ynzMvUgEz/REynyV/SPiHXgTcgl2OU/h2l7kJfrVLk3pQ3QLPnKaEoLav4d onoEaj8I9NFM998NRTpKLgPE/oAwxI3nUTj6p6B7IvGE0Z+vWUjoJ13SGq3OJylnXFCQ WJRg== X-Gm-Message-State: AOAM530Ymsqi/xncHW8wDuusEWL7cfh0mfHOGXoN+XqW8WMdpBUqkxUN SM5dbj/uh+T16k5idoQ0x/eumU6mL9NFbA== X-Received: by 2002:a63:d24c:: with SMTP id t12mr806405pgi.111.1618606968168; Fri, 16 Apr 2021 14:02:48 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:47 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 05/81] target/arm: Split out saturating/rounding shifts from neon Date: Fri, 16 Apr 2021 14:01:24 -0700 Message-Id: <20210416210240.1591291-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::536; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x536.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Split these operations out into a header that can be shared between neon and sve. The "sat" pointer acts both as a boolean for control of saturating behavior and controls the difference in behavior between neon and sve -- QC bit or no QC bit. Widen the shift operand in the new helpers, as the SVE2 insns treat the whole input element as significant. For the neon uses, truncate the shift to int8_t while passing the parameter. Implement right-shift rounding as tmp = src >> (shift - 1); dst = (tmp >> 1) + (tmp & 1); This is the same number of instructions as the current tmp = 1 << (shift - 1); dst = (src + tmp) >> shift; without any possibility of intermediate overflow. Signed-off-by: Richard Henderson --- v2: Widen the shift operand (laurent desnouges) --- target/arm/vec_internal.h | 138 +++++++++++ target/arm/neon_helper.c | 507 +++++++------------------------------- 2 files changed, 221 insertions(+), 424 deletions(-) -- 2.25.1 diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h index e3eb3e7a6b..0102547a10 100644 --- a/target/arm/vec_internal.h +++ b/target/arm/vec_internal.h @@ -30,4 +30,142 @@ static inline void clear_tail(void *vd, uintptr_t opr_sz, uintptr_t max_sz) } } +static inline int32_t do_sqrshl_bhs(int32_t src, int32_t shift, int bits, + bool round, uint32_t *sat) +{ + if (shift <= -bits) { + /* Rounding the sign bit always produces 0. */ + if (round) { + return 0; + } + return src >> 31; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < bits) { + int32_t val = src << shift; + if (bits == 32) { + if (!sat || val >> shift == src) { + return val; + } + } else { + int32_t extval = sextract32(val, 0, bits); + if (!sat || val == extval) { + return extval; + } + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return (1u << (bits - 1)) - (src >= 0); +} + +static inline uint32_t do_uqrshl_bhs(uint32_t src, int32_t shift, int bits, + bool round, uint32_t *sat) +{ + if (shift <= -(bits + round)) { + return 0; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < bits) { + uint32_t val = src << shift; + if (bits == 32) { + if (!sat || val >> shift == src) { + return val; + } + } else { + uint32_t extval = extract32(val, 0, bits); + if (!sat || val == extval) { + return extval; + } + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return MAKE_64BIT_MASK(0, bits); +} + +static inline int32_t do_suqrshl_bhs(int32_t src, int32_t shift, int bits, + bool round, uint32_t *sat) +{ + if (src < 0) { + *sat = 1; + return 0; + } + return do_uqrshl_bhs(src, shift, bits, round, sat); +} + +static inline int64_t do_sqrshl_d(int64_t src, int64_t shift, + bool round, uint32_t *sat) +{ + if (shift <= -64) { + /* Rounding the sign bit always produces 0. */ + if (round) { + return 0; + } + return src >> 63; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < 64) { + int64_t val = src << shift; + if (!sat || val >> shift == src) { + return val; + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return src < 0 ? INT64_MIN : INT64_MAX; +} + +static inline uint64_t do_uqrshl_d(uint64_t src, int64_t shift, + bool round, uint32_t *sat) +{ + if (shift <= -(64 + round)) { + return 0; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < 64) { + uint64_t val = src << shift; + if (!sat || val >> shift == src) { + return val; + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return UINT64_MAX; +} + +static inline int64_t do_suqrshl_d(int64_t src, int64_t shift, + bool round, uint32_t *sat) +{ + if (src < 0) { + *sat = 1; + return 0; + } + return do_uqrshl_d(src, shift, round, sat); +} + #endif /* TARGET_ARM_VEC_INTERNALS_H */ diff --git a/target/arm/neon_helper.c b/target/arm/neon_helper.c index b637265691..338b9189d5 100644 --- a/target/arm/neon_helper.c +++ b/target/arm/neon_helper.c @@ -11,6 +11,7 @@ #include "cpu.h" #include "exec/helper-proto.h" #include "fpu/softfloat.h" +#include "vec_internal.h" #define SIGNBIT (uint32_t)0x80000000 #define SIGNBIT64 ((uint64_t)1 << 63) @@ -576,496 +577,154 @@ NEON_POP(pmax_s16, neon_s16, 2) NEON_POP(pmax_u16, neon_u16, 2) #undef NEON_FN -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8 || \ - tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, false, NULL)) NEON_VOP(shl_u16, neon_u16, 2) #undef NEON_FN -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> (sizeof(src1) * 8 - 1); \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, false, NULL)) NEON_VOP(shl_s16, neon_s16, 2) #undef NEON_FN -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if ((tmp >= (ssize_t)sizeof(src1) * 8) \ - || (tmp <= -(ssize_t)sizeof(src1) * 8)) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 8, true, NULL)) NEON_VOP(rshl_s8, neon_s8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, true, NULL)) NEON_VOP(rshl_s16, neon_s16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_rshl_s32)(uint32_t valop, uint32_t shiftop) +uint32_t HELPER(neon_rshl_s32)(uint32_t val, uint32_t shift) { - int32_t dest; - int32_t val = (int32_t)valop; - int8_t shift = (int8_t)shiftop; - if ((shift >= 32) || (shift <= -32)) { - dest = 0; - } else if (shift < 0) { - int64_t big_dest = ((int64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - } - return dest; + return do_sqrshl_bhs(val, (int8_t)shift, 32, true, NULL); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_rshl_s64)(uint64_t valop, uint64_t shiftop) +uint64_t HELPER(neon_rshl_s64)(uint64_t val, uint64_t shift) { - int8_t shift = (int8_t)shiftop; - int64_t val = valop; - if ((shift >= 64) || (shift <= -64)) { - val = 0; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == INT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x4000000000000000LL; - } else { - val++; - val >>= 1; - } - } else { - val <<= shift; - } - return val; + return do_sqrshl_d(val, (int8_t)shift, true, NULL); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8 || \ - tmp < -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp == -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> (-tmp - 1); \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 8, true, NULL)) NEON_VOP(rshl_u8, neon_u8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, true, NULL)) NEON_VOP(rshl_u16, neon_u16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_rshl_u32)(uint32_t val, uint32_t shiftop) +uint32_t HELPER(neon_rshl_u32)(uint32_t val, uint32_t shift) { - uint32_t dest; - int8_t shift = (int8_t)shiftop; - if (shift >= 32 || shift < -32) { - dest = 0; - } else if (shift == -32) { - dest = val >> 31; - } else if (shift < 0) { - uint64_t big_dest = ((uint64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - } - return dest; + return do_uqrshl_bhs(val, (int8_t)shift, 32, true, NULL); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_rshl_u64)(uint64_t val, uint64_t shiftop) +uint64_t HELPER(neon_rshl_u64)(uint64_t val, uint64_t shift) { - int8_t shift = (uint8_t)shiftop; - if (shift >= 64 || shift < -64) { - val = 0; - } else if (shift == -64) { - /* Rounding a 1-bit result just preserves that bit. */ - val >>= 63; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == UINT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x8000000000000000ULL; - } else { - val++; - val >>= 1; - } - } else { - val <<= shift; - } - return val; + return do_uqrshl_d(val, (int8_t)shift, true, NULL); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = ~0; \ - } else { \ - dest = 0; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = ~0; \ - } \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 8, false, env->vfp.qc)) NEON_VOP_ENV(qshl_u8, neon_u8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, false, env->vfp.qc)) NEON_VOP_ENV(qshl_u16, neon_u16, 2) -NEON_VOP_ENV(qshl_u32, neon_u32, 1) #undef NEON_FN -uint64_t HELPER(neon_qshl_u64)(CPUARMState *env, uint64_t val, uint64_t shiftop) +uint32_t HELPER(neon_qshl_u32)(CPUARMState *env, uint32_t val, uint32_t shift) { - int8_t shift = (int8_t)shiftop; - if (shift >= 64) { - if (val) { - val = ~(uint64_t)0; - SET_QC(); - } - } else if (shift <= -64) { - val = 0; - } else if (shift < 0) { - val >>= -shift; - } else { - uint64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = ~(uint64_t)0; - } - } - return val; + return do_uqrshl_bhs(val, (int8_t)shift, 32, false, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = (uint32_t)(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } else { \ - dest = src1; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> 31; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = (uint32_t)(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } \ - }} while (0) +uint64_t HELPER(neon_qshl_u64)(CPUARMState *env, uint64_t val, uint64_t shift) +{ + return do_uqrshl_d(val, (int8_t)shift, false, env->vfp.qc); +} + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 8, false, env->vfp.qc)) NEON_VOP_ENV(qshl_s8, neon_s8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, false, env->vfp.qc)) NEON_VOP_ENV(qshl_s16, neon_s16, 2) -NEON_VOP_ENV(qshl_s32, neon_s32, 1) #undef NEON_FN -uint64_t HELPER(neon_qshl_s64)(CPUARMState *env, uint64_t valop, uint64_t shiftop) +uint32_t HELPER(neon_qshl_s32)(CPUARMState *env, uint32_t val, uint32_t shift) { - int8_t shift = (uint8_t)shiftop; - int64_t val = valop; - if (shift >= 64) { - if (val) { - SET_QC(); - val = (val >> 63) ^ ~SIGNBIT64; - } - } else if (shift <= -64) { - val >>= 63; - } else if (shift < 0) { - val >>= -shift; - } else { - int64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = (tmp >> 63) ^ ~SIGNBIT64; - } - } - return val; + return do_sqrshl_bhs(val, (int8_t)shift, 32, false, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - if (src1 & (1 << (sizeof(src1) * 8 - 1))) { \ - SET_QC(); \ - dest = 0; \ - } else { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = ~0; \ - } else { \ - dest = 0; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = ~0; \ - } \ - } \ - }} while (0) -NEON_VOP_ENV(qshlu_s8, neon_u8, 4) -NEON_VOP_ENV(qshlu_s16, neon_u16, 2) +uint64_t HELPER(neon_qshl_s64)(CPUARMState *env, uint64_t val, uint64_t shift) +{ + return do_sqrshl_d(val, (int8_t)shift, false, env->vfp.qc); +} + +#define NEON_FN(dest, src1, src2) \ + (dest = do_suqrshl_bhs(src1, (int8_t)src2, 8, false, env->vfp.qc)) +NEON_VOP_ENV(qshlu_s8, neon_s8, 4) #undef NEON_FN -uint32_t HELPER(neon_qshlu_s32)(CPUARMState *env, uint32_t valop, uint32_t shiftop) +#define NEON_FN(dest, src1, src2) \ + (dest = do_suqrshl_bhs(src1, (int8_t)src2, 16, false, env->vfp.qc)) +NEON_VOP_ENV(qshlu_s16, neon_s16, 2) +#undef NEON_FN + +uint32_t HELPER(neon_qshlu_s32)(CPUARMState *env, uint32_t val, uint32_t shift) { - if ((int32_t)valop < 0) { - SET_QC(); - return 0; - } - return helper_neon_qshl_u32(env, valop, shiftop); + return do_suqrshl_bhs(val, (int8_t)shift, 32, false, env->vfp.qc); } -uint64_t HELPER(neon_qshlu_s64)(CPUARMState *env, uint64_t valop, uint64_t shiftop) +uint64_t HELPER(neon_qshlu_s64)(CPUARMState *env, uint64_t val, uint64_t shift) { - if ((int64_t)valop < 0) { - SET_QC(); - return 0; - } - return helper_neon_qshl_u64(env, valop, shiftop); + return do_suqrshl_d(val, (int8_t)shift, false, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = ~0; \ - } else { \ - dest = 0; \ - } \ - } else if (tmp < -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp == -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> (sizeof(src1) * 8 - 1); \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = ~0; \ - } \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 8, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_u8, neon_u8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_u16, neon_u16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_qrshl_u32)(CPUARMState *env, uint32_t val, uint32_t shiftop) +uint32_t HELPER(neon_qrshl_u32)(CPUARMState *env, uint32_t val, uint32_t shift) { - uint32_t dest; - int8_t shift = (int8_t)shiftop; - if (shift >= 32) { - if (val) { - SET_QC(); - dest = ~0; - } else { - dest = 0; - } - } else if (shift < -32) { - dest = 0; - } else if (shift == -32) { - dest = val >> 31; - } else if (shift < 0) { - uint64_t big_dest = ((uint64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - if ((dest >> shift) != val) { - SET_QC(); - dest = ~0; - } - } - return dest; + return do_uqrshl_bhs(val, (int8_t)shift, 32, true, env->vfp.qc); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_qrshl_u64)(CPUARMState *env, uint64_t val, uint64_t shiftop) +uint64_t HELPER(neon_qrshl_u64)(CPUARMState *env, uint64_t val, uint64_t shift) { - int8_t shift = (int8_t)shiftop; - if (shift >= 64) { - if (val) { - SET_QC(); - val = ~0; - } - } else if (shift < -64) { - val = 0; - } else if (shift == -64) { - val >>= 63; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == UINT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x8000000000000000ULL; - } else { - val++; - val >>= 1; - } - } else { \ - uint64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = ~0; - } - } - return val; + return do_uqrshl_d(val, (int8_t)shift, true, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = (typeof(dest))(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } else { \ - dest = 0; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = (uint32_t)(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 8, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_s8, neon_s8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_s16, neon_s16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_qrshl_s32)(CPUARMState *env, uint32_t valop, uint32_t shiftop) +uint32_t HELPER(neon_qrshl_s32)(CPUARMState *env, uint32_t val, uint32_t shift) { - int32_t dest; - int32_t val = (int32_t)valop; - int8_t shift = (int8_t)shiftop; - if (shift >= 32) { - if (val) { - SET_QC(); - dest = (val >> 31) ^ ~SIGNBIT; - } else { - dest = 0; - } - } else if (shift <= -32) { - dest = 0; - } else if (shift < 0) { - int64_t big_dest = ((int64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - if ((dest >> shift) != val) { - SET_QC(); - dest = (val >> 31) ^ ~SIGNBIT; - } - } - return dest; + return do_sqrshl_bhs(val, (int8_t)shift, 32, true, env->vfp.qc); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_qrshl_s64)(CPUARMState *env, uint64_t valop, uint64_t shiftop) +uint64_t HELPER(neon_qrshl_s64)(CPUARMState *env, uint64_t val, uint64_t shift) { - int8_t shift = (uint8_t)shiftop; - int64_t val = valop; - - if (shift >= 64) { - if (val) { - SET_QC(); - val = (val >> 63) ^ ~SIGNBIT64; - } - } else if (shift <= -64) { - val = 0; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == INT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x4000000000000000ULL; - } else { - val++; - val >>= 1; - } - } else { - int64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = (tmp >> 63) ^ ~SIGNBIT64; - } - } - return val; + return do_sqrshl_d(val, (int8_t)shift, true, env->vfp.qc); } uint32_t HELPER(neon_add_u8)(uint32_t a, uint32_t b) From patchwork Fri Apr 16 21:01:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422596 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp721970jaf; Fri, 16 Apr 2021 14:09:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwTyTwBez/98oXji/22CjSwFJuROe1NfooonDRHZmT1QnMaEPXJgYIR/sb5eSAONDPw4ZMf X-Received: by 2002:a67:e34e:: with SMTP id s14mr9074624vsm.50.1618607388698; Fri, 16 Apr 2021 14:09:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607388; cv=none; d=google.com; s=arc-20160816; b=iy2M6zQynFaHOSfg2VUasJA/nkEIeRY+PezkTR9VZDgc1AQpwnGM4epDueH+wQaDQ4 KDlpzUbu02wbvuVX4Pgn9iDC5fL2bWQQw+BNs38wjjwOqlB3US1mbRVrVbOqA1FR7k3b 2h9/ltLibaguVxi/IXIfiSEwrpOZD6ioxcwMxKyOl3BSRpvFGODITwdRmO1FB/IEhu/i EoalvjftEBGRrBDl7FLEjqmVV1Ql68jNGArzO/TF5V1MFvLu4ECbqg/nmawVfUh4pnxS GVHpLl1cl0uRqCe2PTM9PYqfK1NTCpGoU5bdFXVprDHg6CCi/eVd6GLPXysMl0BODGag 5MzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=VTuzzLDpebdoMraQ+WOsKsSEBj+Qw3Yc0aepQpiGbJo=; b=Z7fgDjaBeU+KZNju49W6QPMxMwmhTWKhpPSwjW4ErAxY5ywYUaU54ZY71VZGN88AN8 lPOGc4e6vluY9CzSjkyEjjH/sQfbpEMnnQF9qKm/4Rq60SybosfDpgLn+ggIU53JmuX2 ouDn05UTx65HcdasEjTGleoW+i3brQ4VDMtvkey81xa9Agig68W/9i+1yjuCdID5Mp/S M8EXterqR9ArWpebBf4pR8dKCfVVJMIFYPAvJsYXEra/+avfNilUwqrRMbMoQvRSDT9Z /LphziXmCU9sseERvp25GgznNMsGscRm/sbNpcT8c7TjKctFJ5ZbWYHbDB//mB9Tprfq sEkg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EwV+0ItM; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e25si3273300uar.22.2021.04.16.14.09.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:09:48 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EwV+0ItM; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33688 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVim-0004M4-2w for patch@linaro.org; Fri, 16 Apr 2021 17:09:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43416) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVc6-00042l-Gl for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:54 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]:46856) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVc2-00010y-Kk for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:54 -0400 Received: by mail-pf1-x430.google.com with SMTP id d124so19103700pfa.13 for ; Fri, 16 Apr 2021 14:02:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VTuzzLDpebdoMraQ+WOsKsSEBj+Qw3Yc0aepQpiGbJo=; b=EwV+0ItMW7ZHEpMWqPSDwON/vy5HBsO7db6dnPnZ6XP9pyoAV6D4SH/l0xRs00eZy5 SsEeKm3k+cxkDm9CQ2cUSaEmRKs+SePyTlDoQYx1Srf960j4FDw9FTdqQHTHEb2nhAHk r3+PZ/ELX/Mjadzk3EwaLQ3xnT1dtDtN9vOkqDLCoehFduDENrXlJgWSbM11P3UBjsbl caA0cdx6py23227rSFtw9GrChYgNi0+ty/mxsGg/KIawxtAwFryZAwVDYu+TYHivsL56 Pnrle1xAMc5t0BmB31OViZGYNVlZF3U2haY58jIFnjdJxf0M9NhL/nzwwUbLsEHXksS5 DlDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VTuzzLDpebdoMraQ+WOsKsSEBj+Qw3Yc0aepQpiGbJo=; b=p1y0N3NPYR8R6Gb3DbjgtbjKBkqD0jMA32sUUEx2AjfZYwSlByzWfa9dSF7M1KruBr zpdOnWaP+jFCcJjzmAVKQqMoK0HdXZTpwBdU1kj2hhzUJsyn/jDmzppe7We/bMs70H41 bQ4GK2+DOUIGIWbYhkCYNgKkukissUDoj9vs5oLPanWJ4U94l17eOQPTBlTnqNQAMZB5 WAzftMN0jch4IQ53YXdZqS/+papeeqWZlaUzvCMpUIAdt9fFsF5Hw8q13GH/RHSlVkt2 T5TbgAxhm4IcZuL2qpU0xjs2zujf/MLeMUnRbtX8l4KISLcbhfkzGyDsIsnbHKtqZghZ Jr3w== X-Gm-Message-State: AOAM531eKoZdakb26GGWUSexJJYMm0s452nafnhEw8Vj9xAPrXnhJDb5 sX2F2WdHnDmgEAcNosAc8jiM+nHfmNiVQA== X-Received: by 2002:a62:ed0f:0:b029:257:7278:e72b with SMTP id u15-20020a62ed0f0000b02902577278e72bmr9136809pfh.17.1618606969150; Fri, 16 Apr 2021 14:02:49 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:48 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 06/81] target/arm: Implement SVE2 saturating/rounding bitwise shift left (predicated) Date: Fri, 16 Apr 2021 14:01:25 -0700 Message-Id: <20210416210240.1591291-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Shift values are always signed (laurent desnogues). --- target/arm/helper-sve.h | 54 ++++++++++++++++++++++++++ target/arm/sve.decode | 17 +++++++++ target/arm/sve_helper.c | 78 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 18 +++++++++ 4 files changed, 167 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 9992e93e2b..62106c74be 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -172,6 +172,60 @@ DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_d, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 5ba542969b..93f2479693 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1112,3 +1112,20 @@ URECPE 01000100 .. 000 000 101 ... ..... ..... @rd_pg_rn URSQRTE 01000100 .. 000 001 101 ... ..... ..... @rd_pg_rn SQABS 01000100 .. 001 000 101 ... ..... ..... @rd_pg_rn SQNEG 01000100 .. 001 001 101 ... ..... ..... @rd_pg_rn + +### SVE2 saturating/rounding bitwise shift left (predicated) + +SRSHL 01000100 .. 000 010 100 ... ..... ..... @rdn_pg_rm +URSHL 01000100 .. 000 011 100 ... ..... ..... @rdn_pg_rm +SRSHL 01000100 .. 000 110 100 ... ..... ..... @rdm_pg_rn # SRSHLR +URSHL 01000100 .. 000 111 100 ... ..... ..... @rdm_pg_rn # URSHLR + +SQSHL 01000100 .. 001 000 100 ... ..... ..... @rdn_pg_rm +UQSHL 01000100 .. 001 001 100 ... ..... ..... @rdn_pg_rm +SQSHL 01000100 .. 001 100 100 ... ..... ..... @rdm_pg_rn # SQSHLR +UQSHL 01000100 .. 001 101 100 ... ..... ..... @rdm_pg_rn # UQSHLR + +SQRSHL 01000100 .. 001 010 100 ... ..... ..... @rdn_pg_rm +UQRSHL 01000100 .. 001 011 100 ... ..... ..... @rdn_pg_rm +SQRSHL 01000100 .. 001 110 100 ... ..... ..... @rdm_pg_rn # SQRSHLR +UQRSHL 01000100 .. 001 111 100 ... ..... ..... @rdm_pg_rn # UQRSHLR diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index bbab84e81d..7eff204c3b 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -26,6 +26,7 @@ #include "tcg/tcg-gvec-desc.h" #include "fpu/softfloat.h" #include "tcg/tcg.h" +#include "vec_internal.h" /* Note that vector data is stored in host-endian 64-bit chunks, @@ -561,6 +562,83 @@ DO_ZPZZ(sve2_uadalp_zpzz_h, uint16_t, H1_2, do_uadalp_h) DO_ZPZZ(sve2_uadalp_zpzz_s, uint32_t, H1_4, do_uadalp_s) DO_ZPZZ_D(sve2_uadalp_zpzz_d, uint64_t, do_uadalp_d) +#define do_srshl_b(n, m) do_sqrshl_bhs(n, m, 8, true, NULL) +#define do_srshl_h(n, m) do_sqrshl_bhs(n, m, 16, true, NULL) +#define do_srshl_s(n, m) do_sqrshl_bhs(n, m, 32, true, NULL) +#define do_srshl_d(n, m) do_sqrshl_d(n, m, true, NULL) + +DO_ZPZZ(sve2_srshl_zpzz_b, int8_t, H1_2, do_srshl_b) +DO_ZPZZ(sve2_srshl_zpzz_h, int16_t, H1_2, do_srshl_h) +DO_ZPZZ(sve2_srshl_zpzz_s, int32_t, H1_4, do_srshl_s) +DO_ZPZZ_D(sve2_srshl_zpzz_d, int64_t, do_srshl_d) + +#define do_urshl_b(n, m) do_uqrshl_bhs(n, (int8_t)m, 8, true, NULL) +#define do_urshl_h(n, m) do_uqrshl_bhs(n, (int16_t)m, 16, true, NULL) +#define do_urshl_s(n, m) do_uqrshl_bhs(n, m, 32, true, NULL) +#define do_urshl_d(n, m) do_uqrshl_d(n, m, true, NULL) + +DO_ZPZZ(sve2_urshl_zpzz_b, uint8_t, H1_2, do_urshl_b) +DO_ZPZZ(sve2_urshl_zpzz_h, uint16_t, H1_2, do_urshl_h) +DO_ZPZZ(sve2_urshl_zpzz_s, uint32_t, H1_4, do_urshl_s) +DO_ZPZZ_D(sve2_urshl_zpzz_d, uint64_t, do_urshl_d) + +/* Unlike the NEON and AdvSIMD versions, there is no QC bit to set. */ +#define do_sqshl_b(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 8, false, &discard); }) +#define do_sqshl_h(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 16, false, &discard); }) +#define do_sqshl_s(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 32, false, &discard); }) +#define do_sqshl_d(n, m) \ + ({ uint32_t discard; do_sqrshl_d(n, m, false, &discard); }) + +DO_ZPZZ(sve2_sqshl_zpzz_b, int8_t, H1_2, do_sqshl_b) +DO_ZPZZ(sve2_sqshl_zpzz_h, int16_t, H1_2, do_sqshl_h) +DO_ZPZZ(sve2_sqshl_zpzz_s, int32_t, H1_4, do_sqshl_s) +DO_ZPZZ_D(sve2_sqshl_zpzz_d, int64_t, do_sqshl_d) + +#define do_uqshl_b(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int8_t)m, 8, false, &discard); }) +#define do_uqshl_h(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int16_t)m, 16, false, &discard); }) +#define do_uqshl_s(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, m, 32, false, &discard); }) +#define do_uqshl_d(n, m) \ + ({ uint32_t discard; do_uqrshl_d(n, m, false, &discard); }) + +DO_ZPZZ(sve2_uqshl_zpzz_b, uint8_t, H1_2, do_uqshl_b) +DO_ZPZZ(sve2_uqshl_zpzz_h, uint16_t, H1_2, do_uqshl_h) +DO_ZPZZ(sve2_uqshl_zpzz_s, uint32_t, H1_4, do_uqshl_s) +DO_ZPZZ_D(sve2_uqshl_zpzz_d, uint64_t, do_uqshl_d) + +#define do_sqrshl_b(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 8, true, &discard); }) +#define do_sqrshl_h(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 16, true, &discard); }) +#define do_sqrshl_s(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 32, true, &discard); }) +#define do_sqrshl_d(n, m) \ + ({ uint32_t discard; do_sqrshl_d(n, m, true, &discard); }) + +DO_ZPZZ(sve2_sqrshl_zpzz_b, int8_t, H1_2, do_sqrshl_b) +DO_ZPZZ(sve2_sqrshl_zpzz_h, int16_t, H1_2, do_sqrshl_h) +DO_ZPZZ(sve2_sqrshl_zpzz_s, int32_t, H1_4, do_sqrshl_s) +DO_ZPZZ_D(sve2_sqrshl_zpzz_d, int64_t, do_sqrshl_d) + +#define do_uqrshl_b(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int8_t)m, 8, true, &discard); }) +#define do_uqrshl_h(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int16_t)m, 16, true, &discard); }) +#define do_uqrshl_s(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, m, 32, true, &discard); }) +#define do_uqrshl_d(n, m) \ + ({ uint32_t discard; do_uqrshl_d(n, m, true, &discard); }) + +DO_ZPZZ(sve2_uqrshl_zpzz_b, uint8_t, H1_2, do_uqrshl_b) +DO_ZPZZ(sve2_uqrshl_zpzz_h, uint16_t, H1_2, do_uqrshl_h) +DO_ZPZZ(sve2_uqrshl_zpzz_s, uint32_t, H1_4, do_uqrshl_s) +DO_ZPZZ_D(sve2_uqrshl_zpzz_d, uint64_t, do_uqrshl_d) + #undef DO_ZPZZ #undef DO_ZPZZ_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c30b3c476e..6c1561d897 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5931,3 +5931,21 @@ static bool trans_SQNEG(DisasContext *s, arg_rpr_esz *a) }; return do_sve2_zpz_ool(s, a, fns[a->esz]); } + +#define DO_SVE2_ZPZZ(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ +{ \ + static gen_helper_gvec_4 * const fns[4] = { \ + gen_helper_sve2_##name##_zpzz_b, gen_helper_sve2_##name##_zpzz_h, \ + gen_helper_sve2_##name##_zpzz_s, gen_helper_sve2_##name##_zpzz_d, \ + }; \ + return do_sve2_zpzz_ool(s, a, fns[a->esz]); \ +} + +DO_SVE2_ZPZZ(SQSHL, sqshl) +DO_SVE2_ZPZZ(SQRSHL, sqrshl) +DO_SVE2_ZPZZ(SRSHL, srshl) + +DO_SVE2_ZPZZ(UQSHL, uqshl) +DO_SVE2_ZPZZ(UQRSHL, uqrshl) +DO_SVE2_ZPZZ(URSHL, urshl) From patchwork Fri Apr 16 21:01:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422592 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp718379jaf; Fri, 16 Apr 2021 14:04:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzOOU6KoBHNvA6AWx7EB127dkqNo2bSGGc4d8pBGJW7nz8Fxy4WC428OHwPtdxgjZKxFkdG X-Received: by 2002:aca:cf44:: with SMTP id f65mr7822114oig.13.1618607081827; Fri, 16 Apr 2021 14:04:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607081; cv=none; d=google.com; s=arc-20160816; b=espxkCFrl17rESaaxKDTmLsU50CftdT21vKIqAdHX6qD0iBI2a/JISszaJmdPVmzwV gZPpZjJ3j7Q9ML8B7h6C+LsFjPlHtjEx1uwI4W72FBjzBY3uw+mWzJbaS6dPBkexOKuq 6npv5AmjNfITBe6Wxp0o2CAHaPG93WD1lKFNeYFfTmyWBjL66ptmeO+/5s7L2qSu3gBe 1HYzV6XNr6nYIh4V4uBrQIHTY8V2fK8iM57OKKlbHAXxZ69NfEdgzsZn/t7/fUth2MWK gcQcWyBDkBgkunAtuaq5pZduKYsH8rXQdcw6pRc7gDOa5QWXGReTipsLTu3rUbTyRc1l pvNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9fGS2Fo4Z7uDA+2XxvFopAIhWZ1dSlmsnRxCQI2Tn/s=; b=OGQBl8wwfDUuVvdhFPcRCpYEWL0/YWdWGbpBgGu7nzY3SFMU6L+53IWb+ycv1NGqQ4 ns3xaFRojFD1xecglO8LG29d/W0++POZbfT4I1Ry1qfhSym9aEJy1EE2DA8D2BfsFKlN QzT6DphgmGClq0ZYO7XyyhVGZHvBUxWoCHwWURyrJh9NuLkBjiSgxpI/0g00CFlN21xA Z0LED8Fn5E4/3lTXb42prJPYu825BDRAzsbE3ostSC3P1OJCf0Ux+sJ6DJ9o90z935+6 HuKTbGq+KaK7+ap3KZg6xN6CO2PmTGEOjsUlyd04MdPvL9Q5vVs7ky0IoIiFR9qPUgKP Dp8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bQ22ZAXl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t11si8046044ots.184.2021.04.16.14.04.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:04:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bQ22ZAXl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45732 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVdp-0005sA-2Y for patch@linaro.org; Fri, 16 Apr 2021 17:04:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43392) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVc5-000411-Lb for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:53 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]:45762) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVc3-00011p-GL for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:53 -0400 Received: by mail-pl1-x62b.google.com with SMTP id p16so10675707plf.12 for ; Fri, 16 Apr 2021 14:02:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9fGS2Fo4Z7uDA+2XxvFopAIhWZ1dSlmsnRxCQI2Tn/s=; b=bQ22ZAXlxu/9fBUjflK/vEVfy+B6fe/9iYlNWLx4nsP/HUhqpqcxOu+i7wk1b7TBn4 MQ7XFAqnG1yZ/uF4W0yD77YKhYAzzXpWwc1Lf8Fq+bHJdds7y78+xmSIdbQboSn/HNWl 4LSqLvtiAWIiaB1oWyXH/z8PAMwDrmYqea1VzJgESd6gylWO5biN+gc6eAhC64RueVxN lucG47Pl+pUA11a73n8cYonSlBanTdymeK//Qrm5xyjswmKC/UbnTsmbTJrF/enxWqyt kBfH1mxYVK14G5baxLfbskpqxiLT06uy8CbO2vtbCFMicOdcjeDqMEJK6Rnz+y9UYQfc 2LIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9fGS2Fo4Z7uDA+2XxvFopAIhWZ1dSlmsnRxCQI2Tn/s=; b=P0kaJlW2bZKETKDhSBoZL9284+SWZTlFWX4krNeLfhSfLLnC6dSvw8yfeR1rl4Xt+D lML2rdlR/xEyHIgoamMxtQsharMXF8u0ecFhxG1wkoJLGBD57I6MsN8MQaXItq3YXBTt mHlBG3KvDCnfgOrs4L81qV+0QWOAvh/kWyCX7kKiT/3D5GPqpQ5b3i1SEFI9BTKHy8OA JAU2r1TMvC6OG3B9MgOTfTbGCr+n75FZNMtP5kt7Tj2Dr3X18GfRq8UKTimYKzFpQejb 9bpfgdi/duZaj9JQu92QgGEpTt4XZkBHZhypwmDNwADrC2BoVsCaQJmx8W5lFW34DcWr 6S+Q== X-Gm-Message-State: AOAM532Xa4JEe8ke7BtBBSTchgPLDJJuQqKg+FJ3vPF6UfYz+J8gCB+4 vqdPYzZAdUDK+62AtsXmtROuu9A7hlNHyQ== X-Received: by 2002:a17:90a:670a:: with SMTP id n10mr11555363pjj.176.1618606970165; Fri, 16 Apr 2021 14:02:50 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:49 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 07/81] target/arm: Implement SVE2 integer halving add/subtract (predicated) Date: Fri, 16 Apr 2021 14:01:26 -0700 Message-Id: <20210416210240.1591291-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 54 ++++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 11 ++++++++ target/arm/sve_helper.c | 39 +++++++++++++++++++++++++++ target/arm/translate-sve.c | 8 ++++++ 4 files changed, 112 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 62106c74be..5fdc0d223a 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -226,6 +226,60 @@ DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_d, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 93f2479693..58c3f7ede4 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1129,3 +1129,14 @@ SQRSHL 01000100 .. 001 010 100 ... ..... ..... @rdn_pg_rm UQRSHL 01000100 .. 001 011 100 ... ..... ..... @rdn_pg_rm SQRSHL 01000100 .. 001 110 100 ... ..... ..... @rdm_pg_rn # SQRSHLR UQRSHL 01000100 .. 001 111 100 ... ..... ..... @rdm_pg_rn # UQRSHLR + +### SVE2 integer halving add/subtract (predicated) + +SHADD 01000100 .. 010 000 100 ... ..... ..... @rdn_pg_rm +UHADD 01000100 .. 010 001 100 ... ..... ..... @rdn_pg_rm +SHSUB 01000100 .. 010 010 100 ... ..... ..... @rdn_pg_rm +UHSUB 01000100 .. 010 011 100 ... ..... ..... @rdn_pg_rm +SRHADD 01000100 .. 010 100 100 ... ..... ..... @rdn_pg_rm +URHADD 01000100 .. 010 101 100 ... ..... ..... @rdn_pg_rm +SHSUB 01000100 .. 010 110 100 ... ..... ..... @rdm_pg_rn # SHSUBR +UHSUB 01000100 .. 010 111 100 ... ..... ..... @rdm_pg_rn # UHSUBR diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 7eff204c3b..3703b96eb4 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -639,6 +639,45 @@ DO_ZPZZ(sve2_uqrshl_zpzz_h, uint16_t, H1_2, do_uqrshl_h) DO_ZPZZ(sve2_uqrshl_zpzz_s, uint32_t, H1_4, do_uqrshl_s) DO_ZPZZ_D(sve2_uqrshl_zpzz_d, uint64_t, do_uqrshl_d) +#define DO_HADD_BHS(n, m) (((int64_t)n + m) >> 1) +#define DO_HADD_D(n, m) ((n >> 1) + (m >> 1) + (n & m & 1)) + +DO_ZPZZ(sve2_shadd_zpzz_b, int8_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_shadd_zpzz_h, int16_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_shadd_zpzz_s, int32_t, H1_4, DO_HADD_BHS) +DO_ZPZZ_D(sve2_shadd_zpzz_d, int64_t, DO_HADD_D) + +DO_ZPZZ(sve2_uhadd_zpzz_b, uint8_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_uhadd_zpzz_h, uint16_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_uhadd_zpzz_s, uint32_t, H1_4, DO_HADD_BHS) +DO_ZPZZ_D(sve2_uhadd_zpzz_d, uint64_t, DO_HADD_D) + +#define DO_RHADD_BHS(n, m) (((int64_t)n + m + 1) >> 1) +#define DO_RHADD_D(n, m) ((n >> 1) + (m >> 1) + ((n | m) & 1)) + +DO_ZPZZ(sve2_srhadd_zpzz_b, int8_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_srhadd_zpzz_h, int16_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_srhadd_zpzz_s, int32_t, H1_4, DO_RHADD_BHS) +DO_ZPZZ_D(sve2_srhadd_zpzz_d, int64_t, DO_RHADD_D) + +DO_ZPZZ(sve2_urhadd_zpzz_b, uint8_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_urhadd_zpzz_h, uint16_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_urhadd_zpzz_s, uint32_t, H1_4, DO_RHADD_BHS) +DO_ZPZZ_D(sve2_urhadd_zpzz_d, uint64_t, DO_RHADD_D) + +#define DO_HSUB_BHS(n, m) (((int64_t)n - m) >> 1) +#define DO_HSUB_D(n, m) ((n >> 1) - (m >> 1) - (~n & m & 1)) + +DO_ZPZZ(sve2_shsub_zpzz_b, int8_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_shsub_zpzz_h, int16_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_shsub_zpzz_s, int32_t, H1_4, DO_HSUB_BHS) +DO_ZPZZ_D(sve2_shsub_zpzz_d, int64_t, DO_HSUB_D) + +DO_ZPZZ(sve2_uhsub_zpzz_b, uint8_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_uhsub_zpzz_h, uint16_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_uhsub_zpzz_s, uint32_t, H1_4, DO_HSUB_BHS) +DO_ZPZZ_D(sve2_uhsub_zpzz_d, uint64_t, DO_HSUB_D) + #undef DO_ZPZZ #undef DO_ZPZZ_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6c1561d897..43690999ab 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5949,3 +5949,11 @@ DO_SVE2_ZPZZ(SRSHL, srshl) DO_SVE2_ZPZZ(UQSHL, uqshl) DO_SVE2_ZPZZ(UQRSHL, uqrshl) DO_SVE2_ZPZZ(URSHL, urshl) + +DO_SVE2_ZPZZ(SHADD, shadd) +DO_SVE2_ZPZZ(SRHADD, srhadd) +DO_SVE2_ZPZZ(SHSUB, shsub) + +DO_SVE2_ZPZZ(UHADD, uhadd) +DO_SVE2_ZPZZ(URHADD, urhadd) +DO_SVE2_ZPZZ(UHSUB, uhsub) From patchwork Fri Apr 16 21:01:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422599 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp722415jaf; Fri, 16 Apr 2021 14:10:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzVB5xHcWzAGp+MMRArT3OnF2+e9u4YIfd55ILpy0tGO60lEsGWtsZnTXPmj6CdgsTgTzFL X-Received: by 2002:a05:6830:1d74:: with SMTP id l20mr5197980oti.211.1618607436464; Fri, 16 Apr 2021 14:10:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607436; cv=none; d=google.com; s=arc-20160816; b=sqYX80k/7iFEN08b8VsXBokS0ERrTh6nsno943N7KzV3FfxWrfo/A2T99UQznkuyYq glHjLzBoYugcVrye9puIPPhwEllCpjy9+2EoIY21PzrH9az5iTVZuncc2ANj3vz92n0U JfwobnUvRO417W3ttqXd0BT9z1eD5w3cy8wZDhJCBnpDxhWx0mWpAcoeZ5LpPNkHgr7O UAbpsvmFwT+YIeLsA5ZMRlIk4htYmOjv/WvJG2bG7W/gG+/YdfY+nHymh9UMhq+Zn8o1 FntL6RO6mKFvlqhwRxApwL/pa6JKZYsQMRyVhy18XdPzQmvYxkIx7Gg7OwPEUtvIawkO BVGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=/Nra4WlEPuo4eXKLzm+7jMvvLTFMwTtBD/ofu/WPZv8=; b=UUhFdfGXPyq0nIX3SZpLUGMF/GoxY8tOYl7rHt/OAdpBU7kCeHc5uh2dcgYb+YdXUz GXfMKRT7C5WmISvbr00NKqeWJTmuUEojzG3O0xD1XfA2Vtjcrb+VMN9CN8/sp+2ViOLj 3quURcjHneDZFdvbuBH3Ahqj4Cl/n+6oVeq8qn0p2NPGHrKwj1Sx9ZmeFn0vUKCSVFa3 BEavwYk3JU3CXPK81R7DLdyAe5Da+/0fXjijxoTi6tMmqB7ly+iu8qmrjIQ+Iglrk1Q+ NgBjCw6SK90Dc+l99qz5v5kBv7jkfrDw0GF3K6P7E2fV2+COn7a6YeYSSHQDET6EbyNM 2pbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vvzBfGJX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n198si7127916oig.45.2021.04.16.14.10.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:10:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vvzBfGJX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36302 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVjX-0005Ro-OU for patch@linaro.org; Fri, 16 Apr 2021 17:10:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43460) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVc8-00046q-1o for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:56 -0400 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]:55208) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVc4-00012u-N5 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:55 -0400 Received: by mail-pj1-x102a.google.com with SMTP id cu16so12664970pjb.4 for ; Fri, 16 Apr 2021 14:02:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/Nra4WlEPuo4eXKLzm+7jMvvLTFMwTtBD/ofu/WPZv8=; b=vvzBfGJX8y8JN+38B4ANF032uMOhiBH8HqFcqfIlvdnqOdGW3f3VEYyMfyqL8RGHFZ 1vqUP/7UAMgG/JdZEG6LWhLcKLsICIvRlvF61ocUL0e32UuysHl4eEbDDTjy81W6yi8y 3rNhJQd7GU+1pxJdrXrNboMC4kwuUqhmpYWHuE8PaUi6IczsB1MUZ+moush4rNoy21sI ti7ylGPD+cMTNUWrL4zFSssSZid6AE9gyf/vhzZTRrNNHH0VL9enTmxLR23EZtYhze72 r8wJnoHkRn+LvZu8RcfqO2atXIMDxGg2m68wUqFURU6GbCF6/R43YxnwJeX1pIACvqVR 7few== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/Nra4WlEPuo4eXKLzm+7jMvvLTFMwTtBD/ofu/WPZv8=; b=mo/okUgWZlX0qgQRUKe3RV0HHfzqNgiR63UzRBysGp1nkSzv/K1BRzIDn7dJvuGr9D mIWpZL9tM5xIAZ85Ens8/FekFnR9jYNJ0nwBS7VJvds336mnR3mvMImPSKoUVLoDnKBs d6Zk1rzO8hnVCF24N2o6rte7qUkU1CUyqZGl9XprVCSEZxGAkp41ftspJqnnfQFdSCdc ajIcMNLk6pAYl0YXAy1L8k2AzzVd/jFK50G/3vqj/bO7yXVmwGRAkJ5VqfNzQ7FAYISm VFfVXTm0m98VxKZqEXkTwvVLnKzTjFHPfEuoor0GS4yuhv/7YEXQhxMWiB2JKULT2rCB Zk4A== X-Gm-Message-State: AOAM531iTgvTqOza7q+FAtgejm2FdllrJxhhIQfwN7yCl2rMVA5qVJoo SfeQXaGiqXi9ZsQXKykY7TMEU+pt5cTNag== X-Received: by 2002:a17:90a:ee95:: with SMTP id i21mr12157907pjz.160.1618606971234; Fri, 16 Apr 2021 14:02:51 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:50 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 08/81] target/arm: Implement SVE2 integer pairwise arithmetic Date: Fri, 16 Apr 2021 14:01:27 -0700 Message-Id: <20210416210240.1591291-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Load all inputs before writing any output (laurent desnogues) --- target/arm/helper-sve.h | 45 ++++++++++++++++++++++ target/arm/sve.decode | 8 ++++ target/arm/sve_helper.c | 76 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 6 +++ 4 files changed, 135 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 5fdc0d223a..09bc067dd4 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -326,6 +326,51 @@ DEF_HELPER_FLAGS_5(sve_sel_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve_sel_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_asr_zpzw_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_asr_zpzw_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 58c3f7ede4..61a3321325 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1140,3 +1140,11 @@ SRHADD 01000100 .. 010 100 100 ... ..... ..... @rdn_pg_rm URHADD 01000100 .. 010 101 100 ... ..... ..... @rdn_pg_rm SHSUB 01000100 .. 010 110 100 ... ..... ..... @rdm_pg_rn # SHSUBR UHSUB 01000100 .. 010 111 100 ... ..... ..... @rdm_pg_rn # UHSUBR + +### SVE2 integer pairwise arithmetic + +ADDP 01000100 .. 010 001 101 ... ..... ..... @rdn_pg_rm +SMAXP 01000100 .. 010 100 101 ... ..... ..... @rdn_pg_rm +UMAXP 01000100 .. 010 101 101 ... ..... ..... @rdn_pg_rm +SMINP 01000100 .. 010 110 101 ... ..... ..... @rdn_pg_rm +UMINP 01000100 .. 010 111 101 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 3703b96eb4..7cc559d950 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -681,6 +681,82 @@ DO_ZPZZ_D(sve2_uhsub_zpzz_d, uint64_t, DO_HSUB_D) #undef DO_ZPZZ #undef DO_ZPZZ_D +/* + * Three operand expander, operating on element pairs. + * If the slot I is even, the elements from from VN {I, I+1}. + * If the slot I is odd, the elements from from VM {I-1, I}. + * Load all of the input elements in each pair before overwriting output. + */ +#define DO_ZPZZ_PAIR(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + TYPE n0 = *(TYPE *)(vn + H(i)); \ + TYPE m0 = *(TYPE *)(vm + H(i)); \ + TYPE n1 = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE m1 = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(n0, n1); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(m0, m1); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ +} + +/* Similarly, specialized for 64-bit operands. */ +#define DO_ZPZZ_PAIR_D(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; \ + TYPE *d = vd, *n = vn, *m = vm; \ + uint8_t *pg = vg; \ + for (i = 0; i < opr_sz; i += 2) { \ + TYPE n0 = n[i], n1 = n[i + 1]; \ + TYPE m0 = m[i], m1 = m[i + 1]; \ + if (pg[H1(i)] & 1) { \ + d[i] = OP(n0, n1); \ + } \ + if (pg[H1(i + 1)] & 1) { \ + d[i + 1] = OP(m0, m1); \ + } \ + } \ +} + +DO_ZPZZ_PAIR(sve2_addp_zpzz_b, uint8_t, H1_2, DO_ADD) +DO_ZPZZ_PAIR(sve2_addp_zpzz_h, uint16_t, H1_2, DO_ADD) +DO_ZPZZ_PAIR(sve2_addp_zpzz_s, uint32_t, H1_4, DO_ADD) +DO_ZPZZ_PAIR_D(sve2_addp_zpzz_d, uint64_t, DO_ADD) + +DO_ZPZZ_PAIR(sve2_umaxp_zpzz_b, uint8_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_umaxp_zpzz_h, uint16_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_umaxp_zpzz_s, uint32_t, H1_4, DO_MAX) +DO_ZPZZ_PAIR_D(sve2_umaxp_zpzz_d, uint64_t, DO_MAX) + +DO_ZPZZ_PAIR(sve2_uminp_zpzz_b, uint8_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_uminp_zpzz_h, uint16_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_uminp_zpzz_s, uint32_t, H1_4, DO_MIN) +DO_ZPZZ_PAIR_D(sve2_uminp_zpzz_d, uint64_t, DO_MIN) + +DO_ZPZZ_PAIR(sve2_smaxp_zpzz_b, int8_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_smaxp_zpzz_h, int16_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_smaxp_zpzz_s, int32_t, H1_4, DO_MAX) +DO_ZPZZ_PAIR_D(sve2_smaxp_zpzz_d, int64_t, DO_MAX) + +DO_ZPZZ_PAIR(sve2_sminp_zpzz_b, int8_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_sminp_zpzz_h, int16_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_sminp_zpzz_s, int32_t, H1_4, DO_MIN) +DO_ZPZZ_PAIR_D(sve2_sminp_zpzz_d, int64_t, DO_MIN) + +#undef DO_ZPZZ_PAIR +#undef DO_ZPZZ_PAIR_D + /* Three-operand expander, controlled by a predicate, in which the * third operand is "wide". That is, for D = N op M, the same 64-bit * value of M is used with all of the narrower values of N. diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 43690999ab..2d449c9b57 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5957,3 +5957,9 @@ DO_SVE2_ZPZZ(SHSUB, shsub) DO_SVE2_ZPZZ(UHADD, uhadd) DO_SVE2_ZPZZ(URHADD, urhadd) DO_SVE2_ZPZZ(UHSUB, uhsub) + +DO_SVE2_ZPZZ(ADDP, addp) +DO_SVE2_ZPZZ(SMAXP, smaxp) +DO_SVE2_ZPZZ(UMAXP, umaxp) +DO_SVE2_ZPZZ(SMINP, sminp) +DO_SVE2_ZPZZ(UMINP, uminp) From patchwork Fri Apr 16 21:01:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422606 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp725283jaf; Fri, 16 Apr 2021 14:15:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyv1GQ4V5KLl9LelSOMmLOBEk/sM2OaJJNJfma3YV+HpJ8R8GcHubwZdsrI71Mv2VpC+NNa X-Received: by 2002:a6b:5104:: with SMTP id f4mr5243984iob.12.1618607744319; Fri, 16 Apr 2021 14:15:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607744; cv=none; d=google.com; s=arc-20160816; b=oecuADu98rmEFU0xhOBQkt8qPgyBEibjVj3O8+NQ64cF9N6ypv/mmHb5OMIhYr7vIG o+Pykm1Ka5hg5xT0duk37f9BZc0vdGWKSY/5AODBpyk1YQFk6DhZOJEl0iZ58vnQyozO hhMcEvXU590IFHtkslDqqbyAcpC4wrljSQOEykTvpH945DsgOUMMKZWTbbh4bwYi+FOe iRLu0aMjl4LSXi+HjEVBlPGRCrQLGfaTy2j9y+G0E0p9WtVz+RrB0f3eKvREPh6Wd1UE TFvgxC7ZLrYyDaYtaOqWLyUl6eAXzbNof4hbeCR3TNuSk48Blfacfsr2AwW8SVA5KtaB 7qpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=nkSSxMkXeoKQp4t8tpIye6M1xfm+ZviLl0PQ26Xf9wU=; b=hM3y4fJjRe1qugHqn97MYIiHO8FR81h6MBX6RmqCh6EN07TwVCSBw3aTyHABgXtjtt S1c+yMSwMD60tvKCdq9Vda0DU8p3L3ysGzQhF9Cqgq3Y8Rh4mRNCb4JZZcpehTzk2mbQ vcqBwX3PICyvectubYWk+P3yqvspatlSsMuN+/wwC/SQwTHqcuj9eGcg7Y2Y11rfbywE WDSmaSwVlE1JroRPvuJ6601Q4k+IjQ+Rpg4OKqoxkfo7vEiWgsTpQQCyvGMfBRLVdk+x KPqsFR+Go/nGFClnLHiPr5N7K4XDRPkdfC+ZZp5uQtKb0pUxtqvVNd4ebPWLrzkms+pl LkxQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y0UcVowD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 15si7239416ilt.42.2021.04.16.14.15.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:15:44 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y0UcVowD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51114 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVoV-0003LJ-8a for patch@linaro.org; Fri, 16 Apr 2021 17:15:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43514) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVc9-000496-Ry for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:58 -0400 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]:37681) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVc5-00013X-Lb for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:57 -0400 Received: by mail-pj1-x1034.google.com with SMTP id e8-20020a17090a7288b029014e51f5a6baso10050157pjg.2 for ; Fri, 16 Apr 2021 14:02:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nkSSxMkXeoKQp4t8tpIye6M1xfm+ZviLl0PQ26Xf9wU=; b=Y0UcVowDlNdXri7y+xydfZi5Hm6ch+5uVmwCNL4k34yX47IpQYPMJpIVDjt3jdV6Fc tPvYC/6Vv4xZf+fgp21Rfw7OdKSE5iWfY/kkBVXTVNd9PV+F8hWLhgs63EFta1Iup9f3 7BOHqqJVw/TGeA2YAZa0R2i8nzFvyP2kdhZhLRekO6WcbrSKJNQiUHjokmrTO4nYwc/M DTeA87A68YfSe4ajX75Tg8w27vmUTXdhBLYOSwRVn8IsiQGy1+3GsxlGmUxLda2dmKQM p2vtpanhoXW1RSZuNFtGVUqXZ14zH090rRyHTop63hgFqQCut4ftPIvRfEb0g0QhDUUb vs1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nkSSxMkXeoKQp4t8tpIye6M1xfm+ZviLl0PQ26Xf9wU=; b=HwT/FtspARBC0nAd+xcbrkNpTKeSSZISVG62zscMquS1OTiGNIfiadyMo5s0HeeUuk L4rIV226DWi91Aq/fuHzobgj0ANmYnPUh+Ut/Y9Pt7MxlrhY4k/IhMgiiS4thDoi12xM illYEx9Cfj2btVhtbnoXM/jIT1M3HAnUcCbfeFU/r8m10MrhENYZjtqb43QwDoGV0s5k TMIqutG14iybDe1pS/XIn535P9piYZ4M7Z4IdgkZxgm91d049QC7jSGc0Hc3NhL36OYd DGGhZ74CG2J2dCtIvBvSG1jrB0yh4i8GTXT9JMiBb2FZX0Gw3M3XMYbd/hzC5b2yeKVG 8FUQ== X-Gm-Message-State: AOAM531m8Jd5PIf/Tbpc7hCPHbHS1SbLaHqAmR3E3c5hQakZvMnGjWIQ i0xhrVUlSUFpgX0VASFl2jCWhYxwP/i/Gg== X-Received: by 2002:a17:90b:1118:: with SMTP id gi24mr11315423pjb.30.1618606972262; Fri, 16 Apr 2021 14:02:52 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:51 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 09/81] target/arm: Implement SVE2 saturating add/subtract (predicated) Date: Fri, 16 Apr 2021 14:01:28 -0700 Message-Id: <20210416210240.1591291-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1034.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 54 +++++++++++ target/arm/sve.decode | 11 +++ target/arm/sve_helper.c | 194 ++++++++++++++++++++++++++----------- target/arm/translate-sve.c | 7 ++ 4 files changed, 210 insertions(+), 56 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 09bc067dd4..37461c9927 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -371,6 +371,60 @@ DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_asr_zpzw_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_asr_zpzw_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 61a3321325..cd4f73265f 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1148,3 +1148,14 @@ SMAXP 01000100 .. 010 100 101 ... ..... ..... @rdn_pg_rm UMAXP 01000100 .. 010 101 101 ... ..... ..... @rdn_pg_rm SMINP 01000100 .. 010 110 101 ... ..... ..... @rdn_pg_rm UMINP 01000100 .. 010 111 101 ... ..... ..... @rdn_pg_rm + +### SVE2 saturating add/subtract (predicated) + +SQADD_zpzz 01000100 .. 011 000 100 ... ..... ..... @rdn_pg_rm +UQADD_zpzz 01000100 .. 011 001 100 ... ..... ..... @rdn_pg_rm +SQSUB_zpzz 01000100 .. 011 010 100 ... ..... ..... @rdn_pg_rm +UQSUB_zpzz 01000100 .. 011 011 100 ... ..... ..... @rdn_pg_rm +SUQADD 01000100 .. 011 100 100 ... ..... ..... @rdn_pg_rm +USQADD 01000100 .. 011 101 100 ... ..... ..... @rdn_pg_rm +SQSUB_zpzz 01000100 .. 011 110 100 ... ..... ..... @rdm_pg_rn # SQSUBR +UQSUB_zpzz 01000100 .. 011 111 100 ... ..... ..... @rdm_pg_rn # UQSUBR diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 7cc559d950..12a2078edb 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -678,6 +678,135 @@ DO_ZPZZ(sve2_uhsub_zpzz_h, uint16_t, H1_2, DO_HSUB_BHS) DO_ZPZZ(sve2_uhsub_zpzz_s, uint32_t, H1_4, DO_HSUB_BHS) DO_ZPZZ_D(sve2_uhsub_zpzz_d, uint64_t, DO_HSUB_D) +static inline int32_t do_sat_bhs(int64_t val, int64_t min, int64_t max) +{ + return val >= max ? max : val <= min ? min : val; +} + +#define DO_SQADD_B(n, m) do_sat_bhs((int64_t)n + m, INT8_MIN, INT8_MAX) +#define DO_SQADD_H(n, m) do_sat_bhs((int64_t)n + m, INT16_MIN, INT16_MAX) +#define DO_SQADD_S(n, m) do_sat_bhs((int64_t)n + m, INT32_MIN, INT32_MAX) + +static inline int64_t do_sqadd_d(int64_t n, int64_t m) +{ + int64_t r = n + m; + if (((r ^ n) & ~(n ^ m)) < 0) { + /* Signed overflow. */ + return r < 0 ? INT64_MAX : INT64_MIN; + } + return r; +} + +DO_ZPZZ(sve2_sqadd_zpzz_b, int8_t, H1_2, DO_SQADD_B) +DO_ZPZZ(sve2_sqadd_zpzz_h, int16_t, H1_2, DO_SQADD_H) +DO_ZPZZ(sve2_sqadd_zpzz_s, int32_t, H1_4, DO_SQADD_S) +DO_ZPZZ_D(sve2_sqadd_zpzz_d, int64_t, do_sqadd_d) + +#define DO_UQADD_B(n, m) do_sat_bhs((int64_t)n + m, 0, UINT8_MAX) +#define DO_UQADD_H(n, m) do_sat_bhs((int64_t)n + m, 0, UINT16_MAX) +#define DO_UQADD_S(n, m) do_sat_bhs((int64_t)n + m, 0, UINT32_MAX) + +static inline uint64_t do_uqadd_d(uint64_t n, uint64_t m) +{ + uint64_t r = n + m; + return r < n ? UINT64_MAX : r; +} + +DO_ZPZZ(sve2_uqadd_zpzz_b, uint8_t, H1_2, DO_UQADD_B) +DO_ZPZZ(sve2_uqadd_zpzz_h, uint16_t, H1_2, DO_UQADD_H) +DO_ZPZZ(sve2_uqadd_zpzz_s, uint32_t, H1_4, DO_UQADD_S) +DO_ZPZZ_D(sve2_uqadd_zpzz_d, uint64_t, do_uqadd_d) + +#define DO_SQSUB_B(n, m) do_sat_bhs((int64_t)n - m, INT8_MIN, INT8_MAX) +#define DO_SQSUB_H(n, m) do_sat_bhs((int64_t)n - m, INT16_MIN, INT16_MAX) +#define DO_SQSUB_S(n, m) do_sat_bhs((int64_t)n - m, INT32_MIN, INT32_MAX) + +static inline int64_t do_sqsub_d(int64_t n, int64_t m) +{ + int64_t r = n - m; + if (((r ^ n) & (n ^ m)) < 0) { + /* Signed overflow. */ + return r < 0 ? INT64_MAX : INT64_MIN; + } + return r; +} + +DO_ZPZZ(sve2_sqsub_zpzz_b, int8_t, H1_2, DO_SQSUB_B) +DO_ZPZZ(sve2_sqsub_zpzz_h, int16_t, H1_2, DO_SQSUB_H) +DO_ZPZZ(sve2_sqsub_zpzz_s, int32_t, H1_4, DO_SQSUB_S) +DO_ZPZZ_D(sve2_sqsub_zpzz_d, int64_t, do_sqsub_d) + +#define DO_UQSUB_B(n, m) do_sat_bhs((int64_t)n - m, 0, UINT8_MAX) +#define DO_UQSUB_H(n, m) do_sat_bhs((int64_t)n - m, 0, UINT16_MAX) +#define DO_UQSUB_S(n, m) do_sat_bhs((int64_t)n - m, 0, UINT32_MAX) + +static inline uint64_t do_uqsub_d(uint64_t n, uint64_t m) +{ + return n > m ? n - m : 0; +} + +DO_ZPZZ(sve2_uqsub_zpzz_b, uint8_t, H1_2, DO_UQSUB_B) +DO_ZPZZ(sve2_uqsub_zpzz_h, uint16_t, H1_2, DO_UQSUB_H) +DO_ZPZZ(sve2_uqsub_zpzz_s, uint32_t, H1_4, DO_UQSUB_S) +DO_ZPZZ_D(sve2_uqsub_zpzz_d, uint64_t, do_uqsub_d) + +#define DO_SUQADD_B(n, m) \ + do_sat_bhs((int64_t)(int8_t)n + m, INT8_MIN, INT8_MAX) +#define DO_SUQADD_H(n, m) \ + do_sat_bhs((int64_t)(int16_t)n + m, INT16_MIN, INT16_MAX) +#define DO_SUQADD_S(n, m) \ + do_sat_bhs((int64_t)(int32_t)n + m, INT32_MIN, INT32_MAX) + +static inline int64_t do_suqadd_d(int64_t n, uint64_t m) +{ + uint64_t r = n + m; + + if (n < 0) { + /* Note that m - abs(n) cannot underflow. */ + if (r > INT64_MAX) { + /* Result is either very large positive or negative. */ + if (m > -n) { + /* m > abs(n), so r is a very large positive. */ + return INT64_MAX; + } + /* Result is negative. */ + } + } else { + /* Both inputs are positive: check for overflow. */ + if (r < m || r > INT64_MAX) { + return INT64_MAX; + } + } + return r; +} + +DO_ZPZZ(sve2_suqadd_zpzz_b, uint8_t, H1_2, DO_SUQADD_B) +DO_ZPZZ(sve2_suqadd_zpzz_h, uint16_t, H1_2, DO_SUQADD_H) +DO_ZPZZ(sve2_suqadd_zpzz_s, uint32_t, H1_4, DO_SUQADD_S) +DO_ZPZZ_D(sve2_suqadd_zpzz_d, uint64_t, do_suqadd_d) + +#define DO_USQADD_B(n, m) \ + do_sat_bhs((int64_t)n + (int8_t)m, 0, UINT8_MAX) +#define DO_USQADD_H(n, m) \ + do_sat_bhs((int64_t)n + (int16_t)m, 0, UINT16_MAX) +#define DO_USQADD_S(n, m) \ + do_sat_bhs((int64_t)n + (int32_t)m, 0, UINT32_MAX) + +static inline uint64_t do_usqadd_d(uint64_t n, int64_t m) +{ + uint64_t r = n + m; + + if (m < 0) { + return n < -m ? 0 : r; + } + return r < n ? UINT64_MAX : r; +} + +DO_ZPZZ(sve2_usqadd_zpzz_b, uint8_t, H1_2, DO_USQADD_B) +DO_ZPZZ(sve2_usqadd_zpzz_h, uint16_t, H1_2, DO_USQADD_H) +DO_ZPZZ(sve2_usqadd_zpzz_s, uint32_t, H1_4, DO_USQADD_S) +DO_ZPZZ_D(sve2_usqadd_zpzz_d, uint64_t, do_usqadd_d) + #undef DO_ZPZZ #undef DO_ZPZZ_D @@ -1614,13 +1743,7 @@ void HELPER(sve_sqaddi_b)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int8_t)) { - int r = *(int8_t *)(a + i) + b; - if (r > INT8_MAX) { - r = INT8_MAX; - } else if (r < INT8_MIN) { - r = INT8_MIN; - } - *(int8_t *)(d + i) = r; + *(int8_t *)(d + i) = DO_SQADD_B(b, *(int8_t *)(a + i)); } } @@ -1629,13 +1752,7 @@ void HELPER(sve_sqaddi_h)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int16_t)) { - int r = *(int16_t *)(a + i) + b; - if (r > INT16_MAX) { - r = INT16_MAX; - } else if (r < INT16_MIN) { - r = INT16_MIN; - } - *(int16_t *)(d + i) = r; + *(int16_t *)(d + i) = DO_SQADD_H(b, *(int16_t *)(a + i)); } } @@ -1644,13 +1761,7 @@ void HELPER(sve_sqaddi_s)(void *d, void *a, int64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int32_t)) { - int64_t r = *(int32_t *)(a + i) + b; - if (r > INT32_MAX) { - r = INT32_MAX; - } else if (r < INT32_MIN) { - r = INT32_MIN; - } - *(int32_t *)(d + i) = r; + *(int32_t *)(d + i) = DO_SQADD_S(b, *(int32_t *)(a + i)); } } @@ -1659,13 +1770,7 @@ void HELPER(sve_sqaddi_d)(void *d, void *a, int64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int64_t)) { - int64_t ai = *(int64_t *)(a + i); - int64_t r = ai + b; - if (((r ^ ai) & ~(ai ^ b)) < 0) { - /* Signed overflow. */ - r = (r < 0 ? INT64_MAX : INT64_MIN); - } - *(int64_t *)(d + i) = r; + *(int64_t *)(d + i) = do_sqadd_d(b, *(int64_t *)(a + i)); } } @@ -1678,13 +1783,7 @@ void HELPER(sve_uqaddi_b)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint8_t)) { - int r = *(uint8_t *)(a + i) + b; - if (r > UINT8_MAX) { - r = UINT8_MAX; - } else if (r < 0) { - r = 0; - } - *(uint8_t *)(d + i) = r; + *(uint8_t *)(d + i) = DO_UQADD_B(b, *(uint8_t *)(a + i)); } } @@ -1693,13 +1792,7 @@ void HELPER(sve_uqaddi_h)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint16_t)) { - int r = *(uint16_t *)(a + i) + b; - if (r > UINT16_MAX) { - r = UINT16_MAX; - } else if (r < 0) { - r = 0; - } - *(uint16_t *)(d + i) = r; + *(uint16_t *)(d + i) = DO_UQADD_H(b, *(uint16_t *)(a + i)); } } @@ -1708,13 +1801,7 @@ void HELPER(sve_uqaddi_s)(void *d, void *a, int64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint32_t)) { - int64_t r = *(uint32_t *)(a + i) + b; - if (r > UINT32_MAX) { - r = UINT32_MAX; - } else if (r < 0) { - r = 0; - } - *(uint32_t *)(d + i) = r; + *(uint32_t *)(d + i) = DO_UQADD_S(b, *(uint32_t *)(a + i)); } } @@ -1723,11 +1810,7 @@ void HELPER(sve_uqaddi_d)(void *d, void *a, uint64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint64_t)) { - uint64_t r = *(uint64_t *)(a + i) + b; - if (r < b) { - r = UINT64_MAX; - } - *(uint64_t *)(d + i) = r; + *(uint64_t *)(d + i) = do_uqadd_d(b, *(uint64_t *)(a + i)); } } @@ -1736,8 +1819,7 @@ void HELPER(sve_uqsubi_d)(void *d, void *a, uint64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint64_t)) { - uint64_t ai = *(uint64_t *)(a + i); - *(uint64_t *)(d + i) = (ai < b ? 0 : ai - b); + *(uint64_t *)(d + i) = do_uqsub_d(*(uint64_t *)(a + i), b); } } diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 2d449c9b57..609d5ae7b7 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5963,3 +5963,10 @@ DO_SVE2_ZPZZ(SMAXP, smaxp) DO_SVE2_ZPZZ(UMAXP, umaxp) DO_SVE2_ZPZZ(SMINP, sminp) DO_SVE2_ZPZZ(UMINP, uminp) + +DO_SVE2_ZPZZ(SQADD_zpzz, sqadd) +DO_SVE2_ZPZZ(UQADD_zpzz, uqadd) +DO_SVE2_ZPZZ(SQSUB_zpzz, sqsub) +DO_SVE2_ZPZZ(UQSUB_zpzz, uqsub) +DO_SVE2_ZPZZ(SUQADD, suqadd) +DO_SVE2_ZPZZ(USQADD, usqadd) From patchwork Fri Apr 16 21:01:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422602 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp723824jaf; Fri, 16 Apr 2021 14:13:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJywBQw7tTsOItKrUmjVTsouIw25f0OXRqLypSJMXMTyUsAfcv8sNlmmBi5khPgyrCg+7+L4 X-Received: by 2002:a05:6602:3146:: with SMTP id m6mr5216001ioy.158.1618607582314; Fri, 16 Apr 2021 14:13:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607582; cv=none; d=google.com; s=arc-20160816; b=uP8IaJF8W/gHEmac3EUqs0T5uodBzgR9o4Ska6dITyA9yFk0nSNe+xRl85HvxHdtG/ 23a4sccdaPBPNq0XAnxkBscGjFUhECfZyxe3uTg7Pua4G3x51zyZznpPNJCCEE3/UIGR d+Q4g7SWY4QcHjtsHxwz4/67pskyqTMeIH/vciI8KhjsmJIGKGq6AXhxNbxDVfQRf/wq fd0+iDLQYSkHUkXJ3TUnunxwQ5aFMQFk6qmNZL2tbFpIvSC1DV/mHpWFGSArWeM7KZ8Y 7Lvb1bkNjBVLNHz5lpWPg/cIUMTbLZnzpMJbwC07D21vH2Czbo9XTVrKV39dnfLKxccK OFDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Lt6mYuE55+/GmyazdMW6oFsb1RA74ewEaDqJsdcvUwc=; b=SQjPXIBwUqE5ZuuVP2gwiLnXPGYYH6MK0GikMI96r1xDkA7XXzFCAZrhS0TyjiV9G9 IWJ79BN9F+waEJaZQvg7/NoAD4SNjFI6VYoRw/cHCXkLgG96SY1molvGNjWGAaH9/VVQ JjspucdlImTexp1lkfF4F24dpdxh8FsOKW8wd2WyyK+NzeJUAyHHn6rnBVpv9z/Evt3p my0gmMiUFcRNr+yOiXhJSBr1kAQwESDBfYTDgtocWFvN1G4e+vTp49n+eLFCmenYupXT Xz+JzPkJLr5IyB//cgNknXod5GJCIYFAesyS2OgqMVrNpVu5v3z7BM4D4b24zOWKfz1y LjiQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iA9OgqXz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l11si7857541ils.61.2021.04.16.14.13.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:13:02 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iA9OgqXz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42414 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVlt-0008Bh-Lu for patch@linaro.org; Fri, 16 Apr 2021 17:13:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43518) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcA-00049Q-2C for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:58 -0400 Received: from mail-pg1-x534.google.com ([2607:f8b0:4864:20::534]:44999) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVc6-00013s-G8 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:57 -0400 Received: by mail-pg1-x534.google.com with SMTP id y32so19997159pga.11 for ; Fri, 16 Apr 2021 14:02:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Lt6mYuE55+/GmyazdMW6oFsb1RA74ewEaDqJsdcvUwc=; b=iA9OgqXzJNeqB9IA3uAalVc7jIFHY9dbjHuYUQKfTfvONMiwvO2ls442gqr7pS/pRa crmzeLI0PLaKLAffxEcqwVQqyoWkSAnnIbBpxkTtwnUC9e4raeebPM8Ly6ongtWH36Aq SAvJC/lHZ5Sw0WAdJlUuTaxClbQPDczibZVtGXmANHnCw9TKRtIpwCOugKplWWmBhkkS AP+VgP4yUZLGc0cDrkGHb+6rHA5H4vJERklJGJPyQXUw5uCLotb+kTfBiQ3mHEEIojVV z+IIQri12b0BKyJ/YsN6amPfgDRqqNdQnE7T9VEPp7KhcQG1lX3PYN2YDBmdm5nSFVSA zi5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Lt6mYuE55+/GmyazdMW6oFsb1RA74ewEaDqJsdcvUwc=; b=rUmNoCffNJlXE2YA2T/F5t+hywkU7oaBJehxlmliuzSnQNjLLs58n0lLh2bksLfQ03 bCBJChD0dxORbV3YN9ub1BlvRl1HYQi41V34/+ULMXyb+5L5XI1Ey2g+vB8T7T+txSMb sKHTS6EuQc7w3qIfuzIJQKLt7qHtKPfPPVnHGvapyE01RVclpDFK+TzjyI7Qd2tRRj2h yLcogcdruSsc4apNLDsSta3gxUKlgKuMLgAV27PDs1PT3E64jwx6KAAlFdtdrYjp942g QX76vob7ft/y+TAf/siZK6WpQBXVGbajGXCL1DlBvaZgEbTVSKjNgCgf9eXDnyCbZL+3 Xb6A== X-Gm-Message-State: AOAM532QZUN9m6a4KF3EJT4SSv2d9H1bQAWLF/VDkc+gJoIlS/7aimXg HWSbanQKkhITy1d86WXxCbWzMaclsf45Ug== X-Received: by 2002:aa7:80c9:0:b029:249:cac5:e368 with SMTP id a9-20020aa780c90000b0290249cac5e368mr9611291pfn.12.1618606973121; Fri, 16 Apr 2021 14:02:53 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 10/81] target/arm: Implement SVE2 integer add/subtract long Date: Fri, 16 Apr 2021 14:01:29 -0700 Message-Id: <20210416210240.1591291-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::534; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x534.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix select offsets (laurent desnogues). --- target/arm/helper-sve.h | 24 ++++++++++++++++++++ target/arm/sve.decode | 19 ++++++++++++++++ target/arm/sve_helper.c | 43 +++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 46 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 132 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 37461c9927..a81297b387 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1367,6 +1367,30 @@ DEF_HELPER_FLAGS_5(sve_ftmad_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_ftmad_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_ftmad_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_ssubl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sabdl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sabdl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sabdl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uaddl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_usubl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uabdl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uabdl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uabdl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(sve_ld1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld2bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld3bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index cd4f73265f..fbfd57b23a 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1159,3 +1159,22 @@ SUQADD 01000100 .. 011 100 100 ... ..... ..... @rdn_pg_rm USQADD 01000100 .. 011 101 100 ... ..... ..... @rdn_pg_rm SQSUB_zpzz 01000100 .. 011 110 100 ... ..... ..... @rdm_pg_rn # SQSUBR UQSUB_zpzz 01000100 .. 011 111 100 ... ..... ..... @rdm_pg_rn # UQSUBR + +#### SVE2 Widening Integer Arithmetic + +## SVE2 integer add/subtract long + +SADDLB 01000101 .. 0 ..... 00 0000 ..... ..... @rd_rn_rm +SADDLT 01000101 .. 0 ..... 00 0001 ..... ..... @rd_rn_rm +UADDLB 01000101 .. 0 ..... 00 0010 ..... ..... @rd_rn_rm +UADDLT 01000101 .. 0 ..... 00 0011 ..... ..... @rd_rn_rm + +SSUBLB 01000101 .. 0 ..... 00 0100 ..... ..... @rd_rn_rm +SSUBLT 01000101 .. 0 ..... 00 0101 ..... ..... @rd_rn_rm +USUBLB 01000101 .. 0 ..... 00 0110 ..... ..... @rd_rn_rm +USUBLT 01000101 .. 0 ..... 00 0111 ..... ..... @rd_rn_rm + +SABDLB 01000101 .. 0 ..... 00 1100 ..... ..... @rd_rn_rm +SABDLT 01000101 .. 0 ..... 00 1101 ..... ..... @rd_rn_rm +UABDLB 01000101 .. 0 ..... 00 1110 ..... ..... @rd_rn_rm +UABDLT 01000101 .. 0 ..... 00 1111 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 12a2078edb..3d0ee76411 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1113,6 +1113,49 @@ DO_ZZW(sve_lsl_zzw_s, uint32_t, uint64_t, H1_4, DO_LSL) #undef DO_ZPZ #undef DO_ZPZ_D +/* + * Three-operand expander, unpredicated, in which the two inputs are + * selected from the top or bottom half of the wide column. + */ +#define DO_ZZZ_TB(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sel1 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + int sel2 = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel1)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel2)); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, mm); \ + } \ +} + +DO_ZZZ_TB(sve2_saddl_h, int16_t, int8_t, H1_2, H1, DO_ADD) +DO_ZZZ_TB(sve2_saddl_s, int32_t, int16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_TB(sve2_saddl_d, int64_t, int32_t, , H1_4, DO_ADD) + +DO_ZZZ_TB(sve2_ssubl_h, int16_t, int8_t, H1_2, H1, DO_SUB) +DO_ZZZ_TB(sve2_ssubl_s, int32_t, int16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_TB(sve2_ssubl_d, int64_t, int32_t, , H1_4, DO_SUB) + +DO_ZZZ_TB(sve2_sabdl_h, int16_t, int8_t, H1_2, H1, DO_ABD) +DO_ZZZ_TB(sve2_sabdl_s, int32_t, int16_t, H1_4, H1_2, DO_ABD) +DO_ZZZ_TB(sve2_sabdl_d, int64_t, int32_t, , H1_4, DO_ABD) + +DO_ZZZ_TB(sve2_uaddl_h, uint16_t, uint8_t, H1_2, H1, DO_ADD) +DO_ZZZ_TB(sve2_uaddl_s, uint32_t, uint16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_TB(sve2_uaddl_d, uint64_t, uint32_t, , H1_4, DO_ADD) + +DO_ZZZ_TB(sve2_usubl_h, uint16_t, uint8_t, H1_2, H1, DO_SUB) +DO_ZZZ_TB(sve2_usubl_s, uint32_t, uint16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_TB(sve2_usubl_d, uint64_t, uint32_t, , H1_4, DO_SUB) + +DO_ZZZ_TB(sve2_uabdl_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) +DO_ZZZ_TB(sve2_uabdl_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) +DO_ZZZ_TB(sve2_uabdl_d, uint64_t, uint32_t, , H1_4, DO_ABD) + +#undef DO_ZZZ_TB + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 609d5ae7b7..22983b3b85 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5970,3 +5970,49 @@ DO_SVE2_ZPZZ(SQSUB_zpzz, sqsub) DO_SVE2_ZPZZ(UQSUB_zpzz, uqsub) DO_SVE2_ZPZZ(SUQADD, suqadd) DO_SVE2_ZPZZ(USQADD, usqadd) + +/* + * SVE2 Widening Integer Arithmetic + */ + +static bool do_sve2_zzw_ool(DisasContext *s, arg_rrr_esz *a, + gen_helper_gvec_3 *fn, int data) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, data, fn); + } + return true; +} + +#define DO_SVE2_ZZZ_TB(NAME, name, SEL1, SEL2) \ +static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_h, \ + gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ + }; \ + return do_sve2_zzw_ool(s, a, fns[a->esz], (SEL2 << 1) | SEL1); \ +} + +DO_SVE2_ZZZ_TB(SADDLB, saddl, false, false) +DO_SVE2_ZZZ_TB(SSUBLB, ssubl, false, false) +DO_SVE2_ZZZ_TB(SABDLB, sabdl, false, false) + +DO_SVE2_ZZZ_TB(UADDLB, uaddl, false, false) +DO_SVE2_ZZZ_TB(USUBLB, usubl, false, false) +DO_SVE2_ZZZ_TB(UABDLB, uabdl, false, false) + +DO_SVE2_ZZZ_TB(SADDLT, saddl, true, true) +DO_SVE2_ZZZ_TB(SSUBLT, ssubl, true, true) +DO_SVE2_ZZZ_TB(SABDLT, sabdl, true, true) + +DO_SVE2_ZZZ_TB(UADDLT, uaddl, true, true) +DO_SVE2_ZZZ_TB(USUBLT, usubl, true, true) +DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true) From patchwork Fri Apr 16 21:01:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422603 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp723826jaf; Fri, 16 Apr 2021 14:13:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxx0nQTqrbmGSbyjMWTGWPwukBGWCdoVT96STMXrT/GUDqfCPpW4lg79JviW7rPhneUtllt X-Received: by 2002:a5d:80d5:: with SMTP id h21mr5113695ior.11.1618607582455; Fri, 16 Apr 2021 14:13:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607582; cv=none; d=google.com; s=arc-20160816; b=GVyxr6Iu1/4YT3PqB31/2Qdkqp3FDGeRbo19ps8PmNGF2QJwczyil/5owgpoFBQSUP aznV05D3o9pjAzFZPbNA29feXCED8Tdz3sX1RTEriZCjL6bU4riEnkpkR35Lp0D+RAUb CtASsgWPM/MOowKk9f83AXdXQPqMnoQJFsRrPziTSV7Ex8vpeqMQgg8rabRmhcNJep2P YTiyATKr3LTGURZA8j7Fj4+8UKj/h6MXYvJQhuLL3PNSl+0w9qLpLPumtiuPKwcpoa45 dH92D6Wc8qOjGRTNgVgQAajb+O0dk3K7tB+9bZVRWoFp+3bYTL0y/B6JcRMukKkVb30O uUTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=br78Q7AmWJ+XfOQwE2rDFXsz+3JbpInND1mUnQKHlAY=; b=Ts1bAl0ylKf8UANB+49HSM7i/sLQWeD9A5vsFb3WbF1WJ/PmlGdaxK05ZZKSq0vhnC SGNVCgrwFLoXyUgdi2nrSlgYf+QuFd9GsGHxkbdtQw1ZJYlWJD2PIzyGgow+A7eYDYrR mdA8SGnMdIhot28G2llizwyBeGSez6nbnivPzrd313/16i0xUjTuRJ1AJMpHf3y7gQW0 8Idrv75QfEh2sHpaBJDLRGgE+b3vtbVMAiiD1i6qm5Byxnsv8KsOgExEYborclkNHPLq ZNb0ZJCEn5RnPW4SrEvxOrYgDai5ResnzwrrXAbLWw1FxuV3mDkw9DO2BQV50h3yhLzY LilA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mqQMd7jf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v15si3709826ion.84.2021.04.16.14.13.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:13:02 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mqQMd7jf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44948 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVlt-0000oG-OO for patch@linaro.org; Fri, 16 Apr 2021 17:13:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43544) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcA-0004A7-VL for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:59 -0400 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]:35441) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVc7-00014B-A4 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:02:58 -0400 Received: by mail-pl1-x630.google.com with SMTP id q11so1531507plx.2 for ; Fri, 16 Apr 2021 14:02:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=br78Q7AmWJ+XfOQwE2rDFXsz+3JbpInND1mUnQKHlAY=; b=mqQMd7jflyN2jMIWlZ+e6+21MTMwYDP+7pZaC8LqTqaErK87zNB8vct8oAP9KnreIS dyRwQwDx+5wRUd6u9TCWFYxRcGJNvor5zsawAIM4V7zHOnLmB/dXZTNDmTfK2xYWO+Za AlqOsaJlOGCiWdypuxRJkFGPqry4WgREtOLIgYGJn0r18REvYjA2UairINmtavM/sQnu QI5buX+frdkUCT526QOb82hWl0hh6ezZfmvVCVmGwns58mL7Nieg4C56cofj3NjCNiLr GL23lB6M8Zmdbt8dmEpYkekEwFp+KKm6/i/Fo1gGhtVNL73kjyTOUFSjJwonZKrpC4/V +hhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=br78Q7AmWJ+XfOQwE2rDFXsz+3JbpInND1mUnQKHlAY=; b=ouPnymRJ2P+BFJl5gTTbAksPD3PW0h26o0vBG8DOl3Gk3P7JyH7+xqdAeKop5Rys08 RNRdDDkfDdQErmFxMDlLCEBYTnHLXZMBCzFXieqW+++N7q9wkKl7/PO3CDhY9qXzJ2HC TgI+UnMuaxczGBHyR1j5KIO968AX6bPbAlGXlgQqhbu1/CM9x+LfEHC6cSlyhQvpLgeb dvXWodXPnsErSZcVg3gx/rHtzY6V+epPVKIwG5ZMCWLyGR5i32cJiz8JqpKyf+MUkIew WzboxAC7Vy27FH8Qx4JFHcOIB33hn1DdN3C40w4SkZoa+nRgTTy5dnyce4tSExzny5kr sNqw== X-Gm-Message-State: AOAM530oE2MZCDT5/l7J2IgHhnwZPp5zTHfIVV0gjPh1Q2Mrf68xR5Xu B9jG1R/QPK7ZHnf/YhoPvU/fnUHXo7bT+g== X-Received: by 2002:a17:90a:cc0d:: with SMTP id b13mr11553493pju.219.1618606973994; Fri, 16 Apr 2021 14:02:53 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 11/81] target/arm: Implement SVE2 integer add/subtract interleaved long Date: Fri, 16 Apr 2021 14:01:30 -0700 Message-Id: <20210416210240.1591291-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 6 ++++++ target/arm/translate-sve.c | 4 ++++ 2 files changed, 10 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index fbfd57b23a..12be0584a8 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1178,3 +1178,9 @@ SABDLB 01000101 .. 0 ..... 00 1100 ..... ..... @rd_rn_rm SABDLT 01000101 .. 0 ..... 00 1101 ..... ..... @rd_rn_rm UABDLB 01000101 .. 0 ..... 00 1110 ..... ..... @rd_rn_rm UABDLT 01000101 .. 0 ..... 00 1111 ..... ..... @rd_rn_rm + +## SVE2 integer add/subtract interleaved long + +SADDLBT 01000101 .. 0 ..... 1000 00 ..... ..... @rd_rn_rm +SSUBLBT 01000101 .. 0 ..... 1000 10 ..... ..... @rd_rn_rm +SSUBLTB 01000101 .. 0 ..... 1000 11 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 22983b3b85..ae8323adb7 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6016,3 +6016,7 @@ DO_SVE2_ZZZ_TB(SABDLT, sabdl, true, true) DO_SVE2_ZZZ_TB(UADDLT, uaddl, true, true) DO_SVE2_ZZZ_TB(USUBLT, usubl, true, true) DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true) + +DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true) +DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true) +DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false) From patchwork Fri Apr 16 21:01:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422597 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp722071jaf; Fri, 16 Apr 2021 14:09:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyjhIu2nW2dDbn06WezHGWHtb8jifPHH7PdBEof1FkCSlejs6vdeSKaZpnaEzSE8grZ6gAd X-Received: by 2002:a67:db18:: with SMTP id z24mr4473226vsj.44.1618607396955; Fri, 16 Apr 2021 14:09:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607396; cv=none; d=google.com; s=arc-20160816; b=gb2v+pNm1IfEWhWWMJPznvgnb06CI4pUmJdwu1Mf/8VfZDwQ/R800dp5ecwTDKnA2r OjIa8f1V3RG9JJcqkyrqCSo3AVwhO7M2XjxpV+1GSpoZWu+ZP1OgfsSwYEHpy3OqcxNw K3AjrRfh9ldaVj8hy0ink0OIHEPrxUj2elT3z7IisWm91B1JlPMcuX+qV6BXEJzQDe1B JqShaijPhtZTx9K14zQFJngto+ClWkM/uSYIbtVgsW3bjsSMQs7kW46vPwZbjxSYe3Nr fvJlqCBDwf18L55hGZS+8E14+XVB5FLJfixk+iOZN7zPkcctH5Q9lCdGc1k7lXY210Fm 3zxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ZS2f7fEpsE5PZphfky/IIrJKblfMHybmN+wC38aYIj8=; b=jwDSGHEaD2vRL7PQ0cTSGU0NIRzZU0gvxvfVYuWw6067uUdffIvp/YcAqoSbCJBWQ/ P0wnGa2TUbpA1hwZ3w1tm28qWnbajHEThVbahRbQYsR6rtFCPqooq6MQzK2zqAm2nWnj C14djtbmUVjAPEsMbtEPtgAaRNGO7cFiqCsS7LLbHQ7tfpNikWavt9ubszcs71DOHJjg iDilrDuj085/pQBJ/CuIlwPVQnr/Df1r9iQFtVo5+hSj6BBK9ZJFgecZDLt2LqD2IU1V tEFIi4UyeNRvFzZ/ZSwD5AVuUIL41ofxEDduESJFDfOU/lc1D2PwXhRlYNMuDu7nJd1i XZ1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K6E4UTWX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q18si3810250vkn.70.2021.04.16.14.09.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:09:56 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K6E4UTWX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60048 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXViu-0003cd-8t for patch@linaro.org; Fri, 16 Apr 2021 17:09:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43646) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcF-0004EH-8N for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:03 -0400 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]:51048) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVc8-00015e-7i for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:03 -0400 Received: by mail-pj1-x1035.google.com with SMTP id u11so11009731pjr.0 for ; Fri, 16 Apr 2021 14:02:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZS2f7fEpsE5PZphfky/IIrJKblfMHybmN+wC38aYIj8=; b=K6E4UTWXwdUMhcrZFkKR/Zg64hhEALveJT7XAyvHGxQsAz7ngWjzXaNU+W4YSLL1Ac oVeNCNO7+6E1a5v8vcNyqmapS6uLflfihYYfVGn+EjZaCCsKwFOBWZE0JbpHjyoAfqI8 LVxhRz+FxJMn3UsaULqXSplama7ur+tMxvF5ahzXKCEs34RfwX8eF1vX+bh8H3qobYjF xl4dNfC9p21+Sy2ImLMWwm5UVD1DyXY6/AD85J7dkFNMxhjw6wovFSN4ddHqGv5A/8CH 3u8XQI3p43ZFoueoUsKNDI2vR2J+FB4Ty8vCMK9uhjuYzCkofQnV3T+rVg/juf7OkdWr MLbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZS2f7fEpsE5PZphfky/IIrJKblfMHybmN+wC38aYIj8=; b=Cu1ki6jdpyiTOKF7Aw+MrkQuv5S1xC1O/mdQzox+NE2ujaFSEfP7pFo2L/mE+nj8CY 6d1tY6v9T89VRLFomEJVHdenoKQMjzD5/hMzlQyukoYi7oYHb10I9SqSNxj4xjMgxBYA RQ5mjlffuihB291ORGzy65e2MQAZDbWv62Drm8WRoppEezX39O9TeEG1cMA4g/D2x7f5 8J0xdRZWxDVjEunES9YHo5/AYW54nMt//4jUedWcSzrCucwvdsqxqyUVFhJkE9HGDSqE WD/aFjDI2Sw0icRB05TsiJJtxzcqwgLSzJaI1UBm+G7HqkXC5XSprSNuE2e+mV6Ie0Xh xOtw== X-Gm-Message-State: AOAM533YBGNCpid1uUSIt5wmpn90DqkbILPrCPOKnemTWc7xt1gaAXQo e5bWBN0tmXdBqWOe9grhR65dshuxBY0/rw== X-Received: by 2002:a17:90a:df96:: with SMTP id p22mr11552259pjv.55.1618606974903; Fri, 16 Apr 2021 14:02:54 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 12/81] target/arm: Implement SVE2 integer add/subtract wide Date: Fri, 16 Apr 2021 14:01:31 -0700 Message-Id: <20210416210240.1591291-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1035.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix select offsets (laurent desnogues). --- target/arm/helper-sve.h | 16 ++++++++++++++++ target/arm/sve.decode | 12 ++++++++++++ target/arm/sve_helper.c | 30 ++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 20 ++++++++++++++++++++ 4 files changed, 78 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index a81297b387..3286a9c205 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1391,6 +1391,22 @@ DEF_HELPER_FLAGS_4(sve2_uabdl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_uabdl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_uabdl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_ssubw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uaddw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_usubw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(sve_ld1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld2bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld3bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 12be0584a8..f6f21426ef 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1184,3 +1184,15 @@ UABDLT 01000101 .. 0 ..... 00 1111 ..... ..... @rd_rn_rm SADDLBT 01000101 .. 0 ..... 1000 00 ..... ..... @rd_rn_rm SSUBLBT 01000101 .. 0 ..... 1000 10 ..... ..... @rd_rn_rm SSUBLTB 01000101 .. 0 ..... 1000 11 ..... ..... @rd_rn_rm + +## SVE2 integer add/subtract wide + +SADDWB 01000101 .. 0 ..... 010 000 ..... ..... @rd_rn_rm +SADDWT 01000101 .. 0 ..... 010 001 ..... ..... @rd_rn_rm +UADDWB 01000101 .. 0 ..... 010 010 ..... ..... @rd_rn_rm +UADDWT 01000101 .. 0 ..... 010 011 ..... ..... @rd_rn_rm + +SSUBWB 01000101 .. 0 ..... 010 100 ..... ..... @rd_rn_rm +SSUBWT 01000101 .. 0 ..... 010 101 ..... ..... @rd_rn_rm +USUBWB 01000101 .. 0 ..... 010 110 ..... ..... @rd_rn_rm +USUBWT 01000101 .. 0 ..... 010 111 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 3d0ee76411..cf0dbb3987 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1156,6 +1156,36 @@ DO_ZZZ_TB(sve2_uabdl_d, uint64_t, uint32_t, , H1_4, DO_ABD) #undef DO_ZZZ_TB +#define DO_ZZZ_WTB(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sel2 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel2)); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, mm); \ + } \ +} + +DO_ZZZ_WTB(sve2_saddw_h, int16_t, int8_t, H1_2, H1, DO_ADD) +DO_ZZZ_WTB(sve2_saddw_s, int32_t, int16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_WTB(sve2_saddw_d, int64_t, int32_t, , H1_4, DO_ADD) + +DO_ZZZ_WTB(sve2_ssubw_h, int16_t, int8_t, H1_2, H1, DO_SUB) +DO_ZZZ_WTB(sve2_ssubw_s, int32_t, int16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_WTB(sve2_ssubw_d, int64_t, int32_t, , H1_4, DO_SUB) + +DO_ZZZ_WTB(sve2_uaddw_h, uint16_t, uint8_t, H1_2, H1, DO_ADD) +DO_ZZZ_WTB(sve2_uaddw_s, uint32_t, uint16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_WTB(sve2_uaddw_d, uint64_t, uint32_t, , H1_4, DO_ADD) + +DO_ZZZ_WTB(sve2_usubw_h, uint16_t, uint8_t, H1_2, H1, DO_SUB) +DO_ZZZ_WTB(sve2_usubw_s, uint32_t, uint16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_WTB(sve2_usubw_d, uint64_t, uint32_t, , H1_4, DO_SUB) + +#undef DO_ZZZ_WTB + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ae8323adb7..70900c122f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6020,3 +6020,23 @@ DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true) DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true) DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true) DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false) + +#define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \ +static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_h, \ + gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ + }; \ + return do_sve2_zzw_ool(s, a, fns[a->esz], SEL2); \ +} + +DO_SVE2_ZZZ_WTB(SADDWB, saddw, false) +DO_SVE2_ZZZ_WTB(SADDWT, saddw, true) +DO_SVE2_ZZZ_WTB(SSUBWB, ssubw, false) +DO_SVE2_ZZZ_WTB(SSUBWT, ssubw, true) + +DO_SVE2_ZZZ_WTB(UADDWB, uaddw, false) +DO_SVE2_ZZZ_WTB(UADDWT, uaddw, true) +DO_SVE2_ZZZ_WTB(USUBWB, usubw, false) +DO_SVE2_ZZZ_WTB(USUBWT, usubw, true) From patchwork Fri Apr 16 21:01:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422598 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp722239jaf; Fri, 16 Apr 2021 14:10:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzFeZRiaUaOXMtKfKz8vrWuqaqsGUP43f+70oPHSixSGyk6H/N9+UFkEUoghxVTYhoWG84p X-Received: by 2002:a05:6638:2605:: with SMTP id m5mr5685567jat.97.1618607417248; Fri, 16 Apr 2021 14:10:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607417; cv=none; d=google.com; s=arc-20160816; b=fD83u5LUNBFvkXz6hXiejFFC/6dm4H7XXu6xhM3vJ1u1CmqqcVWrjBDcMrq7txla2M CNpyHtTXBEJJd9zMOZ6oTQw0SD/voK5Fd1Rx4GPi29dQslNNwQESjdAMcH0eIgasCTl0 ja5z8Cw0Bv9iupVk40e5U7vABDVY1e/eelNxr0AEimg4D3q6pSTa0i0xA/l7Q9ODEmDg IntIdAwDOLT1BQXSva97kp5YHussLSn+pgrjnYO7xZevamtafRUhUnGsyyU/rm5DByIZ Ha3c126/86fGBRs32JYC1+YqUdvUJAwaAARWFo6isKSUWC03e3YxZyB2f3gUg2RdMZ28 Dopw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9m49cTBF+uBd2parMcxDIMwzzkCg+x47RKDDIpkpN1g=; b=gXERLhv6v4K1vgkQ2VddpS3vzk6qFC/ssO8W/JM8arvxI1wIlvvvEMyR8+2SuXLszp ZztZHDGvw88Y1Ahppz26ULk6JgllJOkN/BgW9uQYi4R/w4o7PKbMM9Xr8OnKSnDc+FJx pSfmajTn7QfW5Gmv4YoMr3Z2HO67vV8W0jsoBekA6iXnDqcsbFK5jKRTYFSBULZSNCGD cZe8pMNK5F/Q4DpvkJ5kJt8xo0y4E3VWm5r6aOcUi7KiNdJf/C/9/rpraqp5CV/oFAL2 4CTvr9Ur709+Gi0fYaTX6jgX5Wjzq3RsHLunS9se/qvqk+8Me1q4uis40Ng/MTAn0nsQ CH+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xi4NwC50; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n14si3628589ilt.105.2021.04.16.14.10.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:10:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xi4NwC50; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33040 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVjE-00046D-Mf for patch@linaro.org; Fri, 16 Apr 2021 17:10:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43678) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcG-0004EX-0z for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:04 -0400 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]:33440) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVc9-00016O-I8 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:03 -0400 Received: by mail-pl1-x629.google.com with SMTP id n10so3491850plc.0 for ; Fri, 16 Apr 2021 14:02:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9m49cTBF+uBd2parMcxDIMwzzkCg+x47RKDDIpkpN1g=; b=xi4NwC50Lb79a1r63NrPsrhBzjr4a4VBz4Tv8hhJXZMc5wQYgU5ie0PDEy0uKqV0+1 tVmeP+SVx0a7opzdI7o6PNth8aGdJhN6iTgokD9CduPYzJJVbO2oJRq4Jx3lcOMH5jS4 nx5xXIpUuNQ0HJb5rw+TY4lyPWq0slQMKrOZ9mauVR/8JNRJkV67IgcAGVPuODMPFomb 3dPZ8JemwEosIszcs4SvnAsZiE+K/+0yOtJNw1v+/mroPxOO+4wzg+qoJAkAkx8mdqiY 5RgGOpz19p+xlWE1O6CELnwgNkBx3BAKHonMoS3MnmImcOCim31CLlAWJEaNhyji4tWM O3SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9m49cTBF+uBd2parMcxDIMwzzkCg+x47RKDDIpkpN1g=; b=av72WSrEgCQiw6qGKVEw5SnWM6T5MQ6d4VCLa//n7kGknXMh9VusP7nU2q7UIlYT8/ IumPuMiHRXi1cX0/PCklzmWCTnw7twHQeMhPoFTK2kiGRY6bWH5gNSNlZIofACaNCOpb mw94iuFK1mSbB31hLt18ZjgZFW3Pe8Zg7KdMkz5/L219abx0vuTbjAMbSDLg3mGfDCjp KRaUsWO0IQAKJtGx1KzJvhPCSnkbbKQmUlt1Ro7/ML/zbPQRk/Nrn1aL0On6eZAllBrf G01fErfdEt3l8FK6GNOdQvz59Wzu8ef7P5q1biaq0rT0nOmV/JqlozPyOrtc/strwN8D 8Wxg== X-Gm-Message-State: AOAM533BrBiZEusDVKYW9c7ZcsDYFF+apXAAATVT/2PHR425uAk2Bc82 r1j3fGB3c4ZOKuKWU350yBDcW49Zqm4CBA== X-Received: by 2002:a17:90b:4a07:: with SMTP id kk7mr11473953pjb.165.1618606975971; Fri, 16 Apr 2021 14:02:55 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 13/81] target/arm: Implement SVE2 integer multiply long Date: Fri, 16 Apr 2021 14:01:32 -0700 Message-Id: <20210416210240.1591291-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x629.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Exclude PMULL from this category for the moment. Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 15 +++++++++++++++ target/arm/sve.decode | 9 +++++++++ target/arm/sve_helper.c | 31 +++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 9 +++++++++ 4 files changed, 64 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 3286a9c205..ad8121eec6 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2347,4 +2347,19 @@ DEF_HELPER_FLAGS_6(sve_stdd_le_zd_mte, TCG_CALL_NO_WG, DEF_HELPER_FLAGS_6(sve_stdd_be_zd_mte, TCG_CALL_NO_WG, void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_zzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_zzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_zzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_smull_zzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_smull_zzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_smull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_umull_zzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_umull_zzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_umull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(sve2_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index f6f21426ef..d9a72b7661 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1196,3 +1196,12 @@ SSUBWB 01000101 .. 0 ..... 010 100 ..... ..... @rd_rn_rm SSUBWT 01000101 .. 0 ..... 010 101 ..... ..... @rd_rn_rm USUBWB 01000101 .. 0 ..... 010 110 ..... ..... @rd_rn_rm USUBWT 01000101 .. 0 ..... 010 111 ..... ..... @rd_rn_rm + +## SVE2 integer multiply long + +SQDMULLB_zzz 01000101 .. 0 ..... 011 000 ..... ..... @rd_rn_rm +SQDMULLT_zzz 01000101 .. 0 ..... 011 001 ..... ..... @rd_rn_rm +SMULLB_zzz 01000101 .. 0 ..... 011 100 ..... ..... @rd_rn_rm +SMULLT_zzz 01000101 .. 0 ..... 011 101 ..... ..... @rd_rn_rm +UMULLB_zzz 01000101 .. 0 ..... 011 110 ..... ..... @rd_rn_rm +UMULLT_zzz 01000101 .. 0 ..... 011 111 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index cf0dbb3987..f30f3722af 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1154,6 +1154,37 @@ DO_ZZZ_TB(sve2_uabdl_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) DO_ZZZ_TB(sve2_uabdl_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) DO_ZZZ_TB(sve2_uabdl_d, uint64_t, uint32_t, , H1_4, DO_ABD) +DO_ZZZ_TB(sve2_smull_zzz_h, int16_t, int8_t, H1_2, H1, DO_MUL) +DO_ZZZ_TB(sve2_smull_zzz_s, int32_t, int16_t, H1_4, H1_2, DO_MUL) +DO_ZZZ_TB(sve2_smull_zzz_d, int64_t, int32_t, , H1_4, DO_MUL) + +DO_ZZZ_TB(sve2_umull_zzz_h, uint16_t, uint8_t, H1_2, H1, DO_MUL) +DO_ZZZ_TB(sve2_umull_zzz_s, uint32_t, uint16_t, H1_4, H1_2, DO_MUL) +DO_ZZZ_TB(sve2_umull_zzz_d, uint64_t, uint32_t, , H1_4, DO_MUL) + +/* Note that the multiply cannot overflow, but the doubling can. */ +static inline int16_t do_sqdmull_h(int16_t n, int16_t m) +{ + int16_t val = n * m; + return DO_SQADD_H(val, val); +} + +static inline int32_t do_sqdmull_s(int32_t n, int32_t m) +{ + int32_t val = n * m; + return DO_SQADD_S(val, val); +} + +static inline int64_t do_sqdmull_d(int64_t n, int64_t m) +{ + int64_t val = n * m; + return do_sqadd_d(val, val); +} + +DO_ZZZ_TB(sve2_sqdmull_zzz_h, int16_t, int8_t, H1_2, H1, do_sqdmull_h) +DO_ZZZ_TB(sve2_sqdmull_zzz_s, int32_t, int16_t, H1_4, H1_2, do_sqdmull_s) +DO_ZZZ_TB(sve2_sqdmull_zzz_d, int64_t, int32_t, , H1_4, do_sqdmull_d) + #undef DO_ZZZ_TB #define DO_ZZZ_WTB(NAME, TYPEW, TYPEN, HW, HN, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 70900c122f..19a1f289d8 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6021,6 +6021,15 @@ DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true) DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true) DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false) +DO_SVE2_ZZZ_TB(SQDMULLB_zzz, sqdmull_zzz, false, false) +DO_SVE2_ZZZ_TB(SQDMULLT_zzz, sqdmull_zzz, true, true) + +DO_SVE2_ZZZ_TB(SMULLB_zzz, smull_zzz, false, false) +DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true) + +DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false) +DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true) + #define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ { \ From patchwork Fri Apr 16 21:01:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422608 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp726915jaf; Fri, 16 Apr 2021 14:18:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzxTFax1KtSu5vWhHxS6uA562dsJc+hx+O3sskfbM6L1+ROcYVLicn/FInYnQTHblxc2DZN X-Received: by 2002:a05:6830:318a:: with SMTP id p10mr5319420ots.116.1618607930992; Fri, 16 Apr 2021 14:18:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607930; cv=none; d=google.com; s=arc-20160816; b=dwbuVjg4kVBGDckvpJ+V8yySlnFnT3QJzMdS7qtDmM2Go64Eo4NzZc4RC1W4tGLbkf PPIrf9ogDABX7orMkhYYG559i9MvEXWyocTUZhA2pCGNQPH/emsEYJ9FZ5F9G1cZxrFl v3SEb1nG4hmJDuU5zNQ7FtibKMPZrYP44ZugHQ69GmPraYvEM0aYToTWtTulttSkA3jt +t5yjoKrCcR3x9vDanbf4JqfYPBSZ1e7NljF2dIctZY6cGGS9Dup8MYeBJy0irWDkyuX vQ2fVdqiOU5cqzoTmA2jQ6lljwDwRJy2CnhYEQ4u50+/brB7Ot3F7djI6X51zDJgT5Mi Wt2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=TFy/ivv1KJ7f/BHQOtAUnbXQPt7AVG5AmrGqJKBkDA4=; b=PdXGSQJj5/ZU+HY08yrL9LVTLtxNqm2uJ/ZNtOB3NkeXk5rPkgPtcrsenNmycjuBL9 RPwJYn6sCan9reiP1OPR4jZISsjX+19uKS1Zeyzp49RqVRo7ib9X+pb2xRlOsEWtTjTj oJ6G1IrDs3xsjv5nKi/CJ+8KBY1FUWGtdhTtu/yirS+YE5NIogDBTW0dSFN06NLT+Ox2 1k/gVjewieUrBC7db4ZIgc7cS/yt0TINQF94OskVFugH7XeOAwaMdP9pl33Eu6irfeU8 RY5nVHgbTDZTQ+NtjK+A84lDUvxbmx5U2XgndecXmt391VKHvvLe8HmTXY2zJBjrduwi tlsA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WKDyHlxc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s23si6675248otr.50.2021.04.16.14.18.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:18:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WKDyHlxc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:32838 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVrW-0007l3-BA for patch@linaro.org; Fri, 16 Apr 2021 17:18:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43638) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcE-0004Dj-VB for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:03 -0400 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]:44600) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcA-00016a-1u for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:02 -0400 Received: by mail-pj1-x102a.google.com with SMTP id q14-20020a17090a430eb02901503aaee02bso4204092pjg.3 for ; Fri, 16 Apr 2021 14:02:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TFy/ivv1KJ7f/BHQOtAUnbXQPt7AVG5AmrGqJKBkDA4=; b=WKDyHlxceEpqJV88s4LmQXhW6Lxl7VktxaEznJyfZL6FPpoi58PZ5Ox9xTD4NSdCwE Y5gTaUpek+OpVRro48GkxxcX8bDwe+yZMJpRaw30HdkL3Cis7jWcMJiNpmzlhVmldmbn YC1/bdfKry5NJHeq/5N9RX9WDsG49hPIL9QBCMAwkVN0nPTnO2238KESaztfT9UVnZJ3 EVjg+imp/my0Ez+1U7gtLCGJe9ZKgdzAHoSCWb7+j2YO7MMb0eezcO3j3S83EpchQXrn a0NbUc0UlkymOt10bJlqySsjAaFB5jO9tRkU6VdKZ92Dpmednl5ocPiZr+3GbhseDxDP n2Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TFy/ivv1KJ7f/BHQOtAUnbXQPt7AVG5AmrGqJKBkDA4=; b=IL0z+K8E6vbHTvmIA2yIhG4AWETukvEcT1jCuTLzj0dLJQKdP7ShUpnILWwGko3gyw s/mPkt73Ft0MKQNfCrDTMIblCNsHnCcH7CWxxTUP7YLLbM8aDIAzqomGgcfot5d2CjFY DToI8aG81lm4OVpU5dwkA4mtJUD90vF3bgoBgMG1LoWwn4vY1cfHAW/skt3SiJLmJqBf XKp/Vgp/nwagcMiraG2ZYnau8LIrx6VjRcblV9LBLeHnYxr3le3j47OOn0PMhB/O+ach RFLuUk3j4xKbIycXJDDwqcUfdL4HbT8TxWtpS0SHh967JpPtvMZzuVIIFsle6yKnDSdJ nk1A== X-Gm-Message-State: AOAM5327GOb5IUzf+w5Zf4dWPIkJgp/xrFrYU4HpYVbpekDOiFQoCeJA /bfqT1ZFO2T0prvxBgqQJLW5kbnqPJUpFA== X-Received: by 2002:a17:90a:c08f:: with SMTP id o15mr11696060pjs.22.1618606976923; Fri, 16 Apr 2021 14:02:56 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 14/81] target/arm: Implement PMULLB and PMULLT Date: Fri, 16 Apr 2021 14:01:33 -0700 Message-Id: <20210416210240.1591291-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 10 ++++++++++ target/arm/helper-sve.h | 1 + target/arm/sve.decode | 2 ++ target/arm/translate-sve.c | 22 ++++++++++++++++++++++ target/arm/vec_helper.c | 24 ++++++++++++++++++++++++ 5 files changed, 59 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index a6e1fa6333..902579d24b 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4231,6 +4231,16 @@ static inline bool isar_feature_aa64_sve2(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SVEVER) != 0; } +static inline bool isar_feature_aa64_sve2_aes(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, AES) != 0; +} + +static inline bool isar_feature_aa64_sve2_pmull128(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, AES) >= 2; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index ad8121eec6..bf3e533eb4 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2363,3 +2363,4 @@ DEF_HELPER_FLAGS_4(sve2_umull_zzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_umull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_pmull_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index d9a72b7661..016c15ebb6 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1201,6 +1201,8 @@ USUBWT 01000101 .. 0 ..... 010 111 ..... ..... @rd_rn_rm SQDMULLB_zzz 01000101 .. 0 ..... 011 000 ..... ..... @rd_rn_rm SQDMULLT_zzz 01000101 .. 0 ..... 011 001 ..... ..... @rd_rn_rm +PMULLB 01000101 .. 0 ..... 011 010 ..... ..... @rd_rn_rm +PMULLT 01000101 .. 0 ..... 011 011 ..... ..... @rd_rn_rm SMULLB_zzz 01000101 .. 0 ..... 011 100 ..... ..... @rd_rn_rm SMULLT_zzz 01000101 .. 0 ..... 011 101 ..... ..... @rd_rn_rm UMULLB_zzz 01000101 .. 0 ..... 011 110 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 19a1f289d8..fbdccc1c68 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6030,6 +6030,28 @@ DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true) DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false) DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true) +static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_gvec_pmull_q, gen_helper_sve2_pmull_h, + NULL, gen_helper_sve2_pmull_d, + }; + if (a->esz == 0 && !dc_isar_feature(aa64_sve2_pmull128, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], sel); +} + +static bool trans_PMULLB(DisasContext *s, arg_rrr_esz *a) +{ + return do_trans_pmull(s, a, false); +} + +static bool trans_PMULLT(DisasContext *s, arg_rrr_esz *a) +{ + return do_trans_pmull(s, a, true); +} + #define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ { \ diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 40b92100bf..b0ce597060 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -1750,6 +1750,30 @@ void HELPER(sve2_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) d[i] = pmull_h(nn, mm); } } + +static uint64_t pmull_d(uint64_t op1, uint64_t op2) +{ + uint64_t result = 0; + int i; + + for (i = 0; i < 32; ++i) { + uint64_t mask = -((op1 >> i) & 1); + result ^= (op2 << i) & mask; + } + return result; +} + +void HELPER(sve2_pmull_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t sel = H4(simd_data(desc)); + intptr_t i, opr_sz = simd_oprsz(desc); + uint32_t *n = vn, *m = vm; + uint64_t *d = vd; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = pmull_d(n[2 * i + sel], m[2 * i + sel]); + } +} #endif #define DO_CMP0(NAME, TYPE, OP) \ From patchwork Fri Apr 16 21:01:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422601 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp723768jaf; Fri, 16 Apr 2021 14:12:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwYoPLJuThgqTakmve6rl7CDxtQJKuNxWVbPAWHiG7IANOvZx2kr96qEHehCWkfBQInvKG4 X-Received: by 2002:a6b:b746:: with SMTP id h67mr5048891iof.195.1618607578301; Fri, 16 Apr 2021 14:12:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607578; cv=none; d=google.com; s=arc-20160816; b=sBTPhBA02uWaGtnqQIVFmJvA7ZUNgBlTHpk6bgRJ7y//LLoGGPCqIKesA4IR87s9a3 KkQLJKPesX2nmz7XeRSDBl7gEJQTQXD09Z0UDNEHBYk9GdtBH45qUrRKk7yAiFBdzvWw ezpWbx9c1P1CLCF3WsmEP2rQSPH+dCb9po0xVRRCSfBkZAy23GBfSSz9ypjGNyU9sBeR 4PAByw/wwJN7UDrBirgxZTlXs1A5o/tcOx+1miUq1oEFjySpAVuPyLA8PNTKrUrA4Hjv Ij9ojA/rCSIn1WoetQBg/8+vjlMeAAUmt/Qgi/n2lSSSAO1bVxr42wuaaRHiaxyB3fof e2Tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=RWwVG9bQnvXe28WlqK7SF172m5+Drw7KJ7BBkuXVIG8=; b=u0RWhENTNXqNKhwaHHNk+bbKpnAl/12ChCxQjjWmM7Vm/idJZt//1aOeJkS7zXCKxr 7sBn6ZHP/LUeTjOEZPJ2lemJKcduc2z6Mx21mEXktxd76kCg6RpOtZvepyLEy4q7pyGd A1PvCcIHnCSPh1R9C/21s8AEMeZM5vlHdt3kkFoMgwHh/EmkPwmD3J742l4DvJvzRCBg aHKQrBLZQpx1MnaMBbAzRMVcnDqcnX/QCbugjbu03UPV+v40STWLT57l94GhlSMTNoCH 8oBZnm3yvM2RSmJvX7BvP90iudS2mwt++BKl/AXkXflfRKN9SAty7giKbabp+ZeG0+O6 Mnmw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MrLAFMoA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v20si6641406ilo.64.2021.04.16.14.12.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:12:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MrLAFMoA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41954 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVlp-0007zh-Lz for patch@linaro.org; Fri, 16 Apr 2021 17:12:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43700) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcH-0004Fs-93 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:05 -0400 Received: from mail-pj1-x102d.google.com ([2607:f8b0:4864:20::102d]:41938) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcB-00016n-AA for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:04 -0400 Received: by mail-pj1-x102d.google.com with SMTP id z22-20020a17090a0156b029014d4056663fso15283592pje.0 for ; Fri, 16 Apr 2021 14:02:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RWwVG9bQnvXe28WlqK7SF172m5+Drw7KJ7BBkuXVIG8=; b=MrLAFMoA+cP0uolRk2l6ymp6B5pgQV1IzoC0QWbWezDUg3cFQ2J4Md3CB3KFIFE6V3 fPpWeq/O+Cq+xo07/OtjLMlSOcxpj7e2rNFdvwd577BMR3qNoBtyZIUJt3S8TPSiIaIl m78T/Z57xKxh9bg+6Z8Zq1isg9dJ9a7aY3KqQlQr8g9D8hbnt59oW7FpVEhe9Ob9g74i fkAiCmPm5frQ0XXsAaueqzz/na/2fc5soScPYY5coxhkGRL+/E6qvbVR1s1Hegjv4n7G IjYxD2x9fnQWBZ8bD7n0VCJuY4SCMskVKdgwlJx/t2x6Oi5SIOglL0SjhKKlKnl0EliR LCzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RWwVG9bQnvXe28WlqK7SF172m5+Drw7KJ7BBkuXVIG8=; b=hXcQoZoILISh5zP3AoF8qqT5dhDdOoBsQiMKFbc5AGxR3c2BTyS4fJWJGMuam0nIyQ DZ5tnOIWSP+2tuNwIc8KQfQkcEMbEQcUavbPhhWRP/Oq7CZTd7RGKghJOFo8rdX+EAUv c0u7sKr5+TONRKdBDC1v4BFRLccVzwHdJ4ZXs22QPMlKxxZqDCZCmSI3mGsQxoQQTj8E fqbL2XYKnh0Evg0Koai2jS41rkmSDSQaffl0Tvi5l1NZvGUaKm64vQF3KIDFItzixf29 I+xbPadtah6eqy16Onk70uOaWYeMLMHN0wrNvc2xYYxOqngWU4WRqj+dT2Oz3phlP/48 Et1g== X-Gm-Message-State: AOAM530XMtBmNqiZGG8vTM19bTE4zJBYZ5xfS7/XmEKb3VeIikT+XyGE fdLY2U39L2JOgE+iHqmIxxQw+NmqkF/+Cg== X-Received: by 2002:a17:90a:d191:: with SMTP id fu17mr11392751pjb.137.1618606977963; Fri, 16 Apr 2021 14:02:57 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 15/81] target/arm: Implement SVE2 bitwise shift left long Date: Fri, 16 Apr 2021 14:01:34 -0700 Message-Id: <20210416210240.1591291-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102d; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 8 ++ target/arm/sve.decode | 8 ++ target/arm/sve_helper.c | 26 ++++++ target/arm/translate-sve.c | 159 +++++++++++++++++++++++++++++++++++++ 4 files changed, 201 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index bf3e533eb4..740939e7a8 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2364,3 +2364,11 @@ DEF_HELPER_FLAGS_4(sve2_umull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_pmull_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sshll_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sshll_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sshll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_ushll_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_ushll_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_ushll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 016c15ebb6..a3191eba7b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1207,3 +1207,11 @@ SMULLB_zzz 01000101 .. 0 ..... 011 100 ..... ..... @rd_rn_rm SMULLT_zzz 01000101 .. 0 ..... 011 101 ..... ..... @rd_rn_rm UMULLB_zzz 01000101 .. 0 ..... 011 110 ..... ..... @rd_rn_rm UMULLT_zzz 01000101 .. 0 ..... 011 111 ..... ..... @rd_rn_rm + +## SVE2 bitwise shift left long + +# Note bit23 == 0 is handled by esz > 0 in do_sve2_shll_tb. +SSHLLB 01000101 .. 0 ..... 1010 00 ..... ..... @rd_rn_tszimm_shl +SSHLLT 01000101 .. 0 ..... 1010 01 ..... ..... @rd_rn_tszimm_shl +USHLLB 01000101 .. 0 ..... 1010 10 ..... ..... @rd_rn_tszimm_shl +USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index f30f3722af..73aa670a77 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -625,6 +625,8 @@ DO_ZPZZ(sve2_sqrshl_zpzz_h, int16_t, H1_2, do_sqrshl_h) DO_ZPZZ(sve2_sqrshl_zpzz_s, int32_t, H1_4, do_sqrshl_s) DO_ZPZZ_D(sve2_sqrshl_zpzz_d, int64_t, do_sqrshl_d) +#undef do_sqrshl_d + #define do_uqrshl_b(n, m) \ ({ uint32_t discard; do_uqrshl_bhs(n, (int8_t)m, 8, true, &discard); }) #define do_uqrshl_h(n, m) \ @@ -639,6 +641,8 @@ DO_ZPZZ(sve2_uqrshl_zpzz_h, uint16_t, H1_2, do_uqrshl_h) DO_ZPZZ(sve2_uqrshl_zpzz_s, uint32_t, H1_4, do_uqrshl_s) DO_ZPZZ_D(sve2_uqrshl_zpzz_d, uint64_t, do_uqrshl_d) +#undef do_uqrshl_d + #define DO_HADD_BHS(n, m) (((int64_t)n + m) >> 1) #define DO_HADD_D(n, m) ((n >> 1) + (m >> 1) + (n & m & 1)) @@ -1217,6 +1221,28 @@ DO_ZZZ_WTB(sve2_usubw_d, uint64_t, uint32_t, , H1_4, DO_SUB) #undef DO_ZZZ_WTB +#define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + intptr_t sel = (simd_data(desc) & 1) * sizeof(TYPEN); \ + int shift = simd_data(desc) >> 1; \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel)); \ + *(TYPEW *)(vd + HW(i)) = nn << shift; \ + } \ +} + +DO_ZZI_SHLL(sve2_sshll_h, int16_t, int8_t, H1_2, H1) +DO_ZZI_SHLL(sve2_sshll_s, int32_t, int16_t, H1_4, H1_2) +DO_ZZI_SHLL(sve2_sshll_d, int64_t, int32_t, , H1_4) + +DO_ZZI_SHLL(sve2_ushll_h, uint16_t, uint8_t, H1_2, H1) +DO_ZZI_SHLL(sve2_ushll_s, uint32_t, uint16_t, H1_4, H1_2) +DO_ZZI_SHLL(sve2_ushll_d, uint64_t, uint32_t, , H1_4) + +#undef DO_ZZI_SHLL + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index fbdccc1c68..afd208212b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6071,3 +6071,162 @@ DO_SVE2_ZZZ_WTB(UADDWB, uaddw, false) DO_SVE2_ZZZ_WTB(UADDWT, uaddw, true) DO_SVE2_ZZZ_WTB(USUBWB, usubw, false) DO_SVE2_ZZZ_WTB(USUBWT, usubw, true) + +static void gen_sshll_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t imm) +{ + int top = imm & 1; + int shl = imm >> 1; + int halfbits = 4 << vece; + + if (top) { + if (shl == halfbits) { + TCGv_vec t = tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(halfbits, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); + } else { + tcg_gen_sari_vec(vece, d, n, halfbits); + tcg_gen_shli_vec(vece, d, d, shl); + } + } else { + tcg_gen_shli_vec(vece, d, n, halfbits); + tcg_gen_sari_vec(vece, d, d, halfbits - shl); + } +} + +static void gen_ushll_i64(unsigned vece, TCGv_i64 d, TCGv_i64 n, int imm) +{ + int halfbits = 4 << vece; + int top = imm & 1; + int shl = (imm >> 1); + int shift; + uint64_t mask; + + mask = MAKE_64BIT_MASK(0, halfbits); + mask <<= shl; + mask = dup_const(vece, mask); + + shift = shl - top * halfbits; + if (shift < 0) { + tcg_gen_shri_i64(d, n, -shift); + } else { + tcg_gen_shri_i64(d, n, shift); + } + tcg_gen_andi_i64(d, d, mask); +} + +static void gen_ushll16_i64(TCGv_i64 d, TCGv_i64 n, int64_t imm) +{ + gen_ushll_i64(MO_16, d, n, imm); +} + +static void gen_ushll32_i64(TCGv_i64 d, TCGv_i64 n, int64_t imm) +{ + gen_ushll_i64(MO_32, d, n, imm); +} + +static void gen_ushll64_i64(TCGv_i64 d, TCGv_i64 n, int64_t imm) +{ + gen_ushll_i64(MO_64, d, n, imm); +} + +static void gen_ushll_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t imm) +{ + int halfbits = 4 << vece; + int top = imm & 1; + int shl = imm >> 1; + + if (top) { + if (shl == halfbits) { + TCGv_vec t = tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(halfbits, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); + } else { + tcg_gen_shri_vec(vece, d, n, halfbits); + tcg_gen_shli_vec(vece, d, d, shl); + } + } else { + if (shl == 0) { + TCGv_vec t = tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); + } else { + tcg_gen_shli_vec(vece, d, n, halfbits); + tcg_gen_shri_vec(vece, d, d, halfbits - shl); + } + } +} + +static bool do_sve2_shll_tb(DisasContext *s, arg_rri_esz *a, + bool sel, bool uns) +{ + static const TCGOpcode sshll_list[] = { + INDEX_op_shli_vec, INDEX_op_sari_vec, 0 + }; + static const TCGOpcode ushll_list[] = { + INDEX_op_shli_vec, INDEX_op_shri_vec, 0 + }; + static const GVecGen2i ops[2][3] = { + { { .fniv = gen_sshll_vec, + .opt_opc = sshll_list, + .fno = gen_helper_sve2_sshll_h, + .vece = MO_16 }, + { .fniv = gen_sshll_vec, + .opt_opc = sshll_list, + .fno = gen_helper_sve2_sshll_s, + .vece = MO_32 }, + { .fniv = gen_sshll_vec, + .opt_opc = sshll_list, + .fno = gen_helper_sve2_sshll_d, + .vece = MO_64 } }, + { { .fni8 = gen_ushll16_i64, + .fniv = gen_ushll_vec, + .opt_opc = ushll_list, + .fno = gen_helper_sve2_ushll_h, + .vece = MO_16 }, + { .fni8 = gen_ushll32_i64, + .fniv = gen_ushll_vec, + .opt_opc = ushll_list, + .fno = gen_helper_sve2_ushll_s, + .vece = MO_32 }, + { .fni8 = gen_ushll64_i64, + .fniv = gen_ushll_vec, + .opt_opc = ushll_list, + .fno = gen_helper_sve2_ushll_d, + .vece = MO_64 } }, + }; + + if (a->esz < 0 || a->esz > 2 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2i(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vsz, vsz, (a->imm << 1) | sel, + &ops[uns][a->esz]); + } + return true; +} + +static bool trans_SSHLLB(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, false, false); +} + +static bool trans_SSHLLT(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, true, false); +} + +static bool trans_USHLLB(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, false, true); +} + +static bool trans_USHLLT(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, true, true); +} From patchwork Fri Apr 16 21:01:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422607 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp725540jaf; Fri, 16 Apr 2021 14:16:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwi7fZXHQMyjLsPXnbonskBGcznxLJ0JyXf+PvKGXJcOCNFpOIe01lBdZLnXFKgIT/Wsbv0 X-Received: by 2002:a05:6102:358:: with SMTP id e24mr8960295vsa.53.1618607772012; Fri, 16 Apr 2021 14:16:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607772; cv=none; d=google.com; s=arc-20160816; b=AHhkNJIA61I55duWeA0Ajyi+WGKZAPQb2e9Ji/8YsGrRrnJnKS7EUcGqTVK5ELkiwT wHo06cV3vh3RAiII/daMni55RRn3C6Y2JlPBH/KvCXuePyEJ0ZgK4lYNTNGRtCpTSOYV 9aqo9gh1hxa8kRT5u4jjdMTxwvarpDtH509a62n//uCW52fotjCiwzEwPgBx14ZVrjbE a0URRvwblc9KqolLMwRKVzbQjMCJa90R7g/t8J6HHdnuVk1xQDtoAoGxJzt2TKg4qNvY w/P4kksmmJTiuXsCmIquOgopHTA+czGzmGUo0ymIF3ClTp60vrP4ZtSrL3oozUf9gTwO pLrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=iA5YdMPlGGiyhjKxfBQbY6+WZzBPtERwoPFsnw8+g+g=; b=HO2Eupls0sxdejIV2Os20Bn3EeZGwgJTiWnd/sY25pjHtZsqafhgGV5IJ+BWKaEjLT pWjF1CTtVw2OpNDBKW7WoBJGtky54mZMume+isBMW1metbsKN6uhh5U6xIQ3iGs/wNkI j+PYtAPzahcRFDEmyJ5TNwZzuLV4f/FWeydye7QTzhtLMNkk0rVxyH8b9zqO/Lwk2l+l hOR3xC9g55++WevID/NEhwUknyBAsZp3oYaPt4TRdRpx4esb1+d5wwoeBr839HCxKg0j F/MquwVhwGTCGrRH1BM1fgTs8Zm+iHYgXhTJa0Owv8f1xw0kucj7OJgiycob2nd0rtcs 8e+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qVtSCigk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 126si1218414vkt.78.2021.04.16.14.16.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:16:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qVtSCigk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVox-0004vV-AM for patch@linaro.org; Fri, 16 Apr 2021 17:16:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43696) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcG-0004Fi-Ok for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:05 -0400 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]:46910) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcC-00017v-TU for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:04 -0400 Received: by mail-pl1-x635.google.com with SMTP id m18so12422458plc.13 for ; Fri, 16 Apr 2021 14:02:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iA5YdMPlGGiyhjKxfBQbY6+WZzBPtERwoPFsnw8+g+g=; b=qVtSCigkmfUUIWMlUv/xWy5wyGybOVkvin/qQGn0C+X/33q0Q6GBZVT3AINiixpkpj 3ad5VSuUdR/1Xm5iZPfOJu5D3kfcwkSeniXut+iBwuSzLT2JeUOyjP0/DvSaDzkzUu8r hfeeDalO5Qtoj5fxf2Ny9WEunql/l01BI0gphd+1LO26RZ48ff7NtPVAgYc5w3CEqFlr aCngRWOYJzaUUVbLDC9T3r5uiLPv6F2Tpk8IBtsXMWYBbxI7MKJODouponvlyebIh8KV 5+WDsrdzbl/9hJs1Y7oWd++hXxubdZefH8D2l0ZJFVlWlrD/CJf8eUtq6qPpABq1EXQC Rbqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iA5YdMPlGGiyhjKxfBQbY6+WZzBPtERwoPFsnw8+g+g=; b=fFe9fiSs2ZSrLTszGYmftTDMABN4lguicf/AJFTh1+sLfJedM/8r2rN4n+Mv0IVEzk BqvzG9sGuZrb90GjWgEZGfuBNKea3nepBQf6kpH7oLXgiQhcC0w33o2+zbRGT8ye+Myv FqOJnFW+Y0onppI0Y2r+WHrlZoc555GhFBoHsW4bzU2ECun8DX+NKVlUyGvCgQ7PDy8G 8xzDWmVezv02Lv0I7BkFAncZtEgw0s19z5nsJtP4NQSvvLSwvdboHRiH3bPjkbazZOaU 1BVpoBWUYqDmf5UhVHI+Xbj0tGh0v5rcJkyrms0GHcU3MJ+7rM7N07TVUAViPoAWk8I8 642w== X-Gm-Message-State: AOAM530YrHW25pjWnkUsYc/zM/+8njoCn8FnngFDFzPP+Jg13ndpnGGB m81kRsJwP8KRcVPxqLgfK3dls1KDb5zh1g== X-Received: by 2002:a17:90a:ec03:: with SMTP id l3mr11261635pjy.61.1618606979004; Fri, 16 Apr 2021 14:02:59 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 16/81] target/arm: Implement SVE2 bitwise exclusive-or interleaved Date: Fri, 16 Apr 2021 14:01:35 -0700 Message-Id: <20210416210240.1591291-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::635; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 20 ++++++++++++++++++++ target/arm/translate-sve.c | 19 +++++++++++++++++++ 4 files changed, 49 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 740939e7a8..f65818da05 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2372,3 +2372,8 @@ DEF_HELPER_FLAGS_3(sve2_sshll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_ushll_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_ushll_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_ushll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_eoril_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_eoril_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_eoril_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_eoril_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index a3191eba7b..0922a44829 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1215,3 +1215,8 @@ SSHLLB 01000101 .. 0 ..... 1010 00 ..... ..... @rd_rn_tszimm_shl SSHLLT 01000101 .. 0 ..... 1010 01 ..... ..... @rd_rn_tszimm_shl USHLLB 01000101 .. 0 ..... 1010 10 ..... ..... @rd_rn_tszimm_shl USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl + +## SVE2 bitwise exclusive-or interleaved + +EORBT 01000101 .. 0 ..... 10010 0 ..... ..... @rd_rn_rm +EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 73aa670a77..1de0a9bdc3 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1221,6 +1221,26 @@ DO_ZZZ_WTB(sve2_usubw_d, uint64_t, uint32_t, , H1_4, DO_SUB) #undef DO_ZZZ_WTB +#define DO_ZZZ_NTB(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + intptr_t sel1 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPE); \ + intptr_t sel2 = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(TYPE); \ + for (i = 0; i < opr_sz; i += 2 * sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + H(i + sel1)); \ + TYPE mm = *(TYPE *)(vm + H(i + sel2)); \ + *(TYPE *)(vd + H(i + sel1)) = OP(nn, mm); \ + } \ +} + +DO_ZZZ_NTB(sve2_eoril_b, uint8_t, H1, DO_EOR) +DO_ZZZ_NTB(sve2_eoril_h, uint16_t, H1_2, DO_EOR) +DO_ZZZ_NTB(sve2_eoril_s, uint32_t, H1_4, DO_EOR) +DO_ZZZ_NTB(sve2_eoril_d, uint64_t, , DO_EOR) + +#undef DO_ZZZ_NTB + #define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index afd208212b..509b3bc68c 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6030,6 +6030,25 @@ DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true) DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false) DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true) +static bool do_eor_tb(DisasContext *s, arg_rrr_esz *a, bool sel1) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_eoril_b, gen_helper_sve2_eoril_h, + gen_helper_sve2_eoril_s, gen_helper_sve2_eoril_d, + }; + return do_sve2_zzw_ool(s, a, fns[a->esz], (!sel1 << 1) | sel1); +} + +static bool trans_EORBT(DisasContext *s, arg_rrr_esz *a) +{ + return do_eor_tb(s, a, false); +} + +static bool trans_EORTB(DisasContext *s, arg_rrr_esz *a) +{ + return do_eor_tb(s, a, true); +} + static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel) { static gen_helper_gvec_3 * const fns[4] = { From patchwork Fri Apr 16 21:01:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422604 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp724760jaf; Fri, 16 Apr 2021 14:14:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwWQ42YyokLKwiRMIJ96E96KHnE4UGmRFueRUw+f2IJnQl5Ax15EChxoQ53cY1F3+m/Vkxa X-Received: by 2002:aca:d07:: with SMTP id 7mr7944993oin.1.1618607694113; Fri, 16 Apr 2021 14:14:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607694; cv=none; d=google.com; s=arc-20160816; b=yFjZAqvV6RCYQ07T1Llh3ilPkbtIpfQfHrq4zSfSD+aYSMHcy4rJb4pdX/53tIPIyZ Rl6DEl8z0SlBMoGGHVwdFgez82pT2JS0OXriOYrG8Rr40zRmMOevY1GvX+4m58fLZoYt EUEtroaPMg4yakfD5kpqMHap8uVzCRwOy48M/Rfc7Fkx/RdrL6BDWdsmEh/jv6WDKYWD L5phH8dYSV9gvqctS5fI00NJxioBZV2szRtjx+mD/DdpF5+bhXjChICWGcPWUugpjGON gD7aGFE1aAquDQbwtDN0BRlcLj5EG80l4wjWQPkmzOhJXpkRRDSIBoBXDgMxZO1ZDbXV YvtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=BNoITf8uwrxbb0GnHYmFR2vb1nVzSKvKz1i47CoRknE=; b=nsbwSR2y2y5bVs8qdV+RhWft3TENHBAuZ4yXGNkvWN5SzKMaU4fbVMwtiD/6W9tjdx PgKThSbk7XjXAr/mgWtEKVOX2I4G3HSHMHFR76wNESdERqCtSDNSo9w8IGd1KZx3zOr2 d/weKav3isbiOV+4iMXlWS6qLsoUaGXd3akCGMHLMtcJlgpmFeYeg5i5TwzjgAAoOwLp pg2IW11FOrBUOs4goaCozopIZb665RcPomgiRAey5E2lBeG51MjxA/aWN7q5aDMPqdVz EvLOH4JQvRhobuPsdIxut4ap39JJKWWNmWs1OAz/z5DyLf+pqw/hvBvNSCTqNHaqFhGu 6TRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=oG+55xaB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c5si3931639otl.92.2021.04.16.14.14.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:14:54 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=oG+55xaB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51820 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVnh-0003eQ-H7 for patch@linaro.org; Fri, 16 Apr 2021 17:14:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43802) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcK-0004Hr-0t for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:08 -0400 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]:51038) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcD-000193-AI for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:07 -0400 Received: by mail-pj1-x102a.google.com with SMTP id u11so11009857pjr.0 for ; Fri, 16 Apr 2021 14:03:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BNoITf8uwrxbb0GnHYmFR2vb1nVzSKvKz1i47CoRknE=; b=oG+55xaByEhF6knoiyegAEpQ+TVn9B1yP/p/a4vBLeZmt1bfLdGygXIDnZUNhYwxQW qBAE883t4l9Qw1c1DgS08HNsHZSlGVjOmMmlN1bh8jKAcEo4KNd1i6DnaID4ftvra/3Q 1ra1flChBAwUuQmAuSwPzN+MMql2uxdHRVMTmdI1TkDNajgq15t7nCZ4IynNEHYc75/j Y0eI6kv0BrvhL4efcs99qazXR5mqRiVuVgO/shhFJ/JhdDrs7b+cJ8ADkXowffzVCba5 XaPnU3h+EXtNP34lT0p7b+fC6P7vV81x1M+02/AxGIcO178A6dAuzOddURfRzbaHjXSW sneg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BNoITf8uwrxbb0GnHYmFR2vb1nVzSKvKz1i47CoRknE=; b=CioZbV+XLHAyFmMxoO96ZRswLobCoEuhfRwo6wnvp9lrE7jAqCn0J0g1Oo4F6WzDAJ I1FlTvaSrlsY04OG927L/tx5n6iyDvjnJloshhYEjVCoirVhnWiufDqcMxc5JLAn9FH2 NS64lXShxk4jrzStpumCbsJLdLqOZ6TgIfQxM8xbMX5qXKfv4ys1tf0S+dw37eQwXw8Q 3VIIIKgEI5NqhFccvjAqMfVveZowioWSxeKX3VhnHc4XUwJKsDkpgOdmQvHs5k8grWHF Bbr2GgxEfAM8Ey7wPjr+WeFmzjfVga4UQPJVS82P9IheywuAmYg+MpzkV7lNgsHJe348 WGUg== X-Gm-Message-State: AOAM532XQHh6DuerE9+mEfcKnrwwTzaA1AfX+tzUzg0w3gtVQMODPjo4 H/l3m2zKJv/Nuj/hCixRDKxvXsT0XsMidw== X-Received: by 2002:a17:90a:9281:: with SMTP id n1mr11573410pjo.146.1618606979795; Fri, 16 Apr 2021 14:02:59 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.02.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:02:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 17/81] target/arm: Implement SVE2 bitwise permute Date: Fri, 16 Apr 2021 14:01:36 -0700 Message-Id: <20210416210240.1591291-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++ target/arm/helper-sve.h | 15 ++++++++ target/arm/sve.decode | 6 ++++ target/arm/sve_helper.c | 73 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 36 +++++++++++++++++++ 5 files changed, 135 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 902579d24b..ae787fac8a 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4241,6 +4241,11 @@ static inline bool isar_feature_aa64_sve2_pmull128(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, AES) >= 2; } +static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index f65818da05..4861481fe0 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2377,3 +2377,18 @@ DEF_HELPER_FLAGS_4(sve2_eoril_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_eoril_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_eoril_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_eoril_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_bext_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bext_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bext_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bext_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_bdep_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bdep_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bdep_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bdep_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_bgrp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bgrp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bgrp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bgrp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 0922a44829..7cb89a0d47 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1220,3 +1220,9 @@ USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl EORBT 01000101 .. 0 ..... 10010 0 ..... ..... @rd_rn_rm EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm + +## SVE2 bitwise permute + +BEXT 01000101 .. 0 ..... 1011 00 ..... ..... @rd_rn_rm +BDEP 01000101 .. 0 ..... 1011 01 ..... ..... @rd_rn_rm +BGRP 01000101 .. 0 ..... 1011 10 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 1de0a9bdc3..d692d2fe3d 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1241,6 +1241,79 @@ DO_ZZZ_NTB(sve2_eoril_d, uint64_t, , DO_EOR) #undef DO_ZZZ_NTB +#define DO_BITPERM(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + i); \ + TYPE mm = *(TYPE *)(vm + i); \ + *(TYPE *)(vd + i) = OP(nn, mm, sizeof(TYPE) * 8); \ + } \ +} + +static uint64_t bitextract(uint64_t data, uint64_t mask, int n) +{ + uint64_t res = 0; + int db, rb = 0; + + for (db = 0; db < n; ++db) { + if ((mask >> db) & 1) { + res |= ((data >> db) & 1) << rb; + ++rb; + } + } + return res; +} + +DO_BITPERM(sve2_bext_b, uint8_t, bitextract) +DO_BITPERM(sve2_bext_h, uint16_t, bitextract) +DO_BITPERM(sve2_bext_s, uint32_t, bitextract) +DO_BITPERM(sve2_bext_d, uint64_t, bitextract) + +static uint64_t bitdeposit(uint64_t data, uint64_t mask, int n) +{ + uint64_t res = 0; + int rb, db = 0; + + for (rb = 0; rb < n; ++rb) { + if ((mask >> rb) & 1) { + res |= ((data >> db) & 1) << rb; + ++db; + } + } + return res; +} + +DO_BITPERM(sve2_bdep_b, uint8_t, bitdeposit) +DO_BITPERM(sve2_bdep_h, uint16_t, bitdeposit) +DO_BITPERM(sve2_bdep_s, uint32_t, bitdeposit) +DO_BITPERM(sve2_bdep_d, uint64_t, bitdeposit) + +static uint64_t bitgroup(uint64_t data, uint64_t mask, int n) +{ + uint64_t resm = 0, resu = 0; + int db, rbm = 0, rbu = 0; + + for (db = 0; db < n; ++db) { + uint64_t val = (data >> db) & 1; + if ((mask >> db) & 1) { + resm |= val << rbm++; + } else { + resu |= val << rbu++; + } + } + + return resm | (resu << rbm); +} + +DO_BITPERM(sve2_bgrp_b, uint8_t, bitgroup) +DO_BITPERM(sve2_bgrp_h, uint16_t, bitgroup) +DO_BITPERM(sve2_bgrp_s, uint32_t, bitgroup) +DO_BITPERM(sve2_bgrp_d, uint64_t, bitgroup) + +#undef DO_BITPERM + #define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 509b3bc68c..da2f67fb67 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6249,3 +6249,39 @@ static bool trans_USHLLT(DisasContext *s, arg_rri_esz *a) { return do_sve2_shll_tb(s, a, true, true); } + +static bool trans_BEXT(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_bext_b, gen_helper_sve2_bext_h, + gen_helper_sve2_bext_s, gen_helper_sve2_bext_d, + }; + if (!dc_isar_feature(aa64_sve2_bitperm, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], 0); +} + +static bool trans_BDEP(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_bdep_b, gen_helper_sve2_bdep_h, + gen_helper_sve2_bdep_s, gen_helper_sve2_bdep_d, + }; + if (!dc_isar_feature(aa64_sve2_bitperm, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], 0); +} + +static bool trans_BGRP(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_bgrp_b, gen_helper_sve2_bgrp_h, + gen_helper_sve2_bgrp_s, gen_helper_sve2_bgrp_d, + }; + if (!dc_isar_feature(aa64_sve2_bitperm, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], 0); +} From patchwork Fri Apr 16 21:01:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422615 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp729120jaf; Fri, 16 Apr 2021 14:22:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwF26/EWu2ENu0FHjy+7cWzi9SbpeH+p+VoxG0OJeizW2zvMiqYSVf0+jaPTsrQc7hXPGqn X-Received: by 2002:a92:320f:: with SMTP id z15mr6107734ile.231.1618608161628; Fri, 16 Apr 2021 14:22:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608161; cv=none; d=google.com; s=arc-20160816; b=c7lAuMx04oraQqc1+QFKLLZZrn1ZDrFxz0AP00ETiwyqlpMGrDq5gBs+cNPlAhNlbO 66qI8VDY13xSi9VaZ61MtWgi5iMP+iw9rV/CWEwgXeI7CU4faCwz3aAt3uiXwyHGZH3Q pCuiC+EEH4+Dy+F8Jsx43bCXft+pl351P+cAaoYsKJvCf2tMaoiFQ8iNR4Yhu7WvS086 oEnsY7FZ9TzDoFCTa3XQlLcYrhkYhV4+xhy+FWttEZ15WhyM6FrmWrRqFGMnfEneNcn0 /ArJK+4ib3lG77t/fKz1Z9DN+YTH1jm/2VnkPiHpCjGUnsxBRod37IzsUBGmb17Lj4A+ fmYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=xiy3OFu7y9n0lQLi1Ms0ijwf8zujYC8QTzkyT7iwC1M=; b=lUR2TF83la60PN4emLHxgZ2qLSvK7JNW9vC63zyCG3/orzVog0FvG7PaHs/VodTvgm lDcNTY4XNHjgN0AdtaJEHEm/p+ORyMVADHQLHPDxlJGg1bIhuHNw8TWsCl2pq5p4pZTM 4WJ6wtzYUDuxXeZB1pNZWBJIzXIAhDMvi4khZL6J65l5/5axXWhDCFMDm++28h3qysEM nP885cMenraA2df7jAFDXQQghWrvjIjiSIQXHv/2qSZ/MVbBW9kts4FJbZyjMoiIwG78 n0KydyUu4ExEvLpmpTyLGZr/UnN4qrQvQZT5ixxbSl23ZUJgHqDPL1ZC2lk+GHMMZj0i eADA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kNmsoTcA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j11si3871747ilu.123.2021.04.16.14.22.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:22:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kNmsoTcA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44948 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVvF-0004Yx-0N for patch@linaro.org; Fri, 16 Apr 2021 17:22:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43762) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcJ-0004Hc-4Q for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:08 -0400 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]:34738) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcD-00019F-RP for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:06 -0400 Received: by mail-pl1-x632.google.com with SMTP id t22so14309373ply.1 for ; Fri, 16 Apr 2021 14:03:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xiy3OFu7y9n0lQLi1Ms0ijwf8zujYC8QTzkyT7iwC1M=; b=kNmsoTcAACDKjdsKAZ6gNm2GZaIdOj0TUsAHTRFeEK2x2HVyg3VGHaurlQyQTghL3W g0FQw8KqyDt9vndbllchthlrAECn10frhsVQKBNpGosAvZ66qq3dh26b/ral7NBurP4O z+uBfuhz1hPvI04CEfPKO2cDB+3TFb2gRMPIduoIvxXosqaV9RcQLB40Donr9cmmni2Z IjwsDUKYMBF7n7JweM9NLP9HHJtxLAURnht+stqbJMhovqd93W6c+K2srK63LuYuPAEE ea0P7p8Nlrk9+ZXOzulIFw9ENt7yaV5hKMD8t38xmBCeyscrZT7lUL1EdLeOhMB2ysqh yFUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xiy3OFu7y9n0lQLi1Ms0ijwf8zujYC8QTzkyT7iwC1M=; b=t7i3ZJsMl1KLEnbZHk32F1lleTE/mRaHkodFCS9vzEuQ7F41vqG4cgZU6+74w/xc8A PE5dAI4xyX0FPa/zhEqyCIpkt4g9fqZvhi7FvyLZ4tSlln+u18n2zA+2JtRVz60709S1 T1SgPKNWIQlLHJWxJs4DWuxRRT+mfsQppJpjnk92nVc7qMxaShkeBjvtOWi+FfAm6Mza Yur2cRxZz+eYcRjaV3WwLrhGA/5t5t8Y0r4evs//Cp9tuHSdVo8VKWb+qAEOM4bB6/eB BDG7FAadIonu34UTfBk3NeMYo+hdP0tDzNxOoYX3nsRvbfcItuyEJwpVxTlLdjNL3lvU i4dg== X-Gm-Message-State: AOAM533uOn4L3HmeoO5bJ88Pl+FoI39u5mscW093s6BC5S4g9vucqE6y uNysCEXlyylxvKz7EuY79i6krPS93Evcgw== X-Received: by 2002:a17:90b:30d2:: with SMTP id hi18mr11381556pjb.94.1618606980667; Fri, 16 Apr 2021 14:03:00 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 18/81] target/arm: Implement SVE2 complex integer add Date: Fri, 16 Apr 2021 14:01:37 -0700 Message-Id: <20210416210240.1591291-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::632; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x632.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix subtraction ordering (laurent desnogues). --- target/arm/helper-sve.h | 10 +++++++++ target/arm/sve.decode | 9 ++++++++ target/arm/sve_helper.c | 42 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 31 ++++++++++++++++++++++++++++ 4 files changed, 92 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 4861481fe0..c2155cc544 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2392,3 +2392,13 @@ DEF_HELPER_FLAGS_4(sve2_bgrp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_bgrp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_bgrp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_bgrp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_cadd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_cadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_cadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_cadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqcadd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqcadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqcadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqcadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7cb89a0d47..7508b901d0 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1226,3 +1226,12 @@ EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm BEXT 01000101 .. 0 ..... 1011 00 ..... ..... @rd_rn_rm BDEP 01000101 .. 0 ..... 1011 01 ..... ..... @rd_rn_rm BGRP 01000101 .. 0 ..... 1011 10 ..... ..... @rd_rn_rm + +#### SVE2 Accumulate + +## SVE2 complex integer add + +CADD_rot90 01000101 .. 00000 0 11011 0 ..... ..... @rdn_rm +CADD_rot270 01000101 .. 00000 0 11011 1 ..... ..... @rdn_rm +SQCADD_rot90 01000101 .. 00000 1 11011 0 ..... ..... @rdn_rm +SQCADD_rot270 01000101 .. 00000 1 11011 1 ..... ..... @rdn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index d692d2fe3d..2e09c3e55b 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1314,6 +1314,48 @@ DO_BITPERM(sve2_bgrp_d, uint64_t, bitgroup) #undef DO_BITPERM +#define DO_CADD(NAME, TYPE, H, ADD_OP, SUB_OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sub_r = simd_data(desc); \ + if (sub_r) { \ + for (i = 0; i < opr_sz; i += 2 * sizeof(TYPE)) { \ + TYPE acc_r = *(TYPE *)(vn + H(i)); \ + TYPE acc_i = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE el2_r = *(TYPE *)(vm + H(i)); \ + TYPE el2_i = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + acc_r = ADD_OP(acc_r, el2_i); \ + acc_i = SUB_OP(acc_i, el2_r); \ + *(TYPE *)(vd + H(i)) = acc_r; \ + *(TYPE *)(vd + H(i + sizeof(TYPE))) = acc_i; \ + } \ + } else { \ + for (i = 0; i < opr_sz; i += 2 * sizeof(TYPE)) { \ + TYPE acc_r = *(TYPE *)(vn + H(i)); \ + TYPE acc_i = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE el2_r = *(TYPE *)(vm + H(i)); \ + TYPE el2_i = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + acc_r = SUB_OP(acc_r, el2_i); \ + acc_i = ADD_OP(acc_i, el2_r); \ + *(TYPE *)(vd + H(i)) = acc_r; \ + *(TYPE *)(vd + H(i + sizeof(TYPE))) = acc_i; \ + } \ + } \ +} + +DO_CADD(sve2_cadd_b, int8_t, H1, DO_ADD, DO_SUB) +DO_CADD(sve2_cadd_h, int16_t, H1_2, DO_ADD, DO_SUB) +DO_CADD(sve2_cadd_s, int32_t, H1_4, DO_ADD, DO_SUB) +DO_CADD(sve2_cadd_d, int64_t, , DO_ADD, DO_SUB) + +DO_CADD(sve2_sqcadd_b, int8_t, H1, DO_SQADD_B, DO_SQSUB_B) +DO_CADD(sve2_sqcadd_h, int16_t, H1_2, DO_SQADD_H, DO_SQSUB_H) +DO_CADD(sve2_sqcadd_s, int32_t, H1_4, DO_SQADD_S, DO_SQSUB_S) +DO_CADD(sve2_sqcadd_d, int64_t, , do_sqadd_d, do_sqsub_d) + +#undef DO_CADD + #define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index da2f67fb67..c594c59954 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6285,3 +6285,34 @@ static bool trans_BGRP(DisasContext *s, arg_rrr_esz *a) } return do_sve2_zzw_ool(s, a, fns[a->esz], 0); } + +static bool do_cadd(DisasContext *s, arg_rrr_esz *a, bool sq, bool rot) +{ + static gen_helper_gvec_3 * const fns[2][4] = { + { gen_helper_sve2_cadd_b, gen_helper_sve2_cadd_h, + gen_helper_sve2_cadd_s, gen_helper_sve2_cadd_d }, + { gen_helper_sve2_sqcadd_b, gen_helper_sve2_sqcadd_h, + gen_helper_sve2_sqcadd_s, gen_helper_sve2_sqcadd_d }, + }; + return do_sve2_zzw_ool(s, a, fns[sq][a->esz], rot); +} + +static bool trans_CADD_rot90(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, false, false); +} + +static bool trans_CADD_rot270(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, false, true); +} + +static bool trans_SQCADD_rot90(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, true, false); +} + +static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, true, true); +} From patchwork Fri Apr 16 21:01:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422612 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp727419jaf; Fri, 16 Apr 2021 14:19:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJztX+zS9lHWfUQzX41QHraGBO6pBvxoqaI3JrP9iaipSZKPIMUMNSxzuRyu0g9SrhuH7Hkv X-Received: by 2002:a6b:da11:: with SMTP id x17mr5310627iob.63.1618607991153; Fri, 16 Apr 2021 14:19:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607991; cv=none; d=google.com; s=arc-20160816; b=kXR6hAp7JFevWnpv/QLcSrenMdn1SP75F2mXzUIg9WNZ6nOTmcFhHGcYQaRjjSfyu5 57D4ZZjHZGN97+S7jgp3wDGrC3KeXNaVpClS/jBeOepouvFRHNteGQblqalLCi7olaDk br5uIg9EQewcH5/exDbrK4CHnaU+k0FUIPN7HQNrykWDFPtv/sxqnV3lq5TiJbrrJkOj MQ05mnFLLAuCWMb7EzAthhOVvVKUXRvkN9PPJi0J4DKg85CMnYqFg6chSyCooEWdcMOb Sl4R74attghlKqxaIKFqqSamMQ6ck5RyM7Ks+B4F0NLCpetnzblflya9FmqqO1nMDzFh 8A1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=85QbQRD/kjbvj7H0EZGYuJDct9yXvao73Ho7HNCSiAg=; b=EJ4M0m3F6Q+Mq/tmEUnfyRuwE9pvIHwETmg0/OqcFvXl6T94YewoVifBA86JWpZLC6 PlFQZspcZDJze1tdsLnJ6Ll8H8fS7YOqqwShfiOw7CCC24IkBLrOaoe5yqL5nz0zzrsA JOgyUDKLDZ/rfEu2JpxvvsQ/7kK5Ct39QLpGO/aNJqvWcXwPFyIT7f8ZHV5HY6gdGt7s XherwApp5wBAjwbSp7zic+yg0kxoklKPLJYiOHsGUJ9w1Ej4aH50gLedZNjgd/0QZ5Rj qarTSyxb/IM1pG7Tmh2F2992gRHZpFn6Qpmf4dydV2/MHARtSDCgfODHdS7F9/Ga3yQt 6+WQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dB2nrpjp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d18si6675059ilo.12.2021.04.16.14.19.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:19:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dB2nrpjp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37260 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVsU-0001Fm-Ij for patch@linaro.org; Fri, 16 Apr 2021 17:19:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43814) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcK-0004Hs-JP for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:08 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]:42525) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcE-00019T-Q5 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:08 -0400 Received: by mail-pl1-x62c.google.com with SMTP id v13so1041330ple.9 for ; Fri, 16 Apr 2021 14:03:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=85QbQRD/kjbvj7H0EZGYuJDct9yXvao73Ho7HNCSiAg=; b=dB2nrpjpzc8xfJX6KiZBECsWMbsMCTeNbShf1Sc+Dla851M7HXlK9rwET6ng9eONAU Ax7smG49YOZPw9eg3d7CDzJpD4zWXOFXBxiAtGEDqHEwZkBL8sWXXXOQF5GXKyUNIxvb f5QCy3xSYxT4CJARQEI4/Puw6X/GUL9gYG6X+Ov8ZxoUGAb920Asx3kZ5D7j1D9EutfS R2BKsJoGEw4WWLdHOPM+xUKorCK6jWkZSKEIhL9b5FpEzbt4DuOaFcW/H2XVHoUrvdYh aPTSy2rfaIzzZRUmsP3v9lsnuMwGyGDRyIGIo9wtAOdwsQEmrfn+mxpd3wjcZHgd2EMJ rWmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=85QbQRD/kjbvj7H0EZGYuJDct9yXvao73Ho7HNCSiAg=; b=Im7b6X3/wAnFHa+jVH+GKXrG0MOC1ulmcNWV1oJUp2Alf4rSZayGZ1CukiDm21Uz1Y k5esZ+8m7OsAxt22D+mXDIeQ9Q44BLA2/HvHPoRxD6H6SGRZhfX8cubT2fsaTV+WSYfz CNc3WpBxTMd8BwSXZKNoxNLm46nCFr80YXTzkyigYAz1LHsnVcsDZmM+q/eTeeu5zKfm l164/ISQbPKWYbeivWyQrIjGk7bBGf/gWVdgQJWWaIveNCLPQKmNFqBluGe598Z0veNS RWqLE3wu8RNwkYiNAimGahPhc2K+NVconEOP7UvMeLPPxZEwAG2KSyjHzl7mX+cbh52a hQaA== X-Gm-Message-State: AOAM531GzlzuyMLfvBcI6eDztqXF1ZMKY9GAV6AN/970vvkTezQ36APV OB7tCqHBg5rGOiGJ6Uex2CfolQe66JxfnA== X-Received: by 2002:a17:90a:fb89:: with SMTP id cp9mr11358074pjb.47.1618606981511; Fri, 16 Apr 2021 14:03:01 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 19/81] target/arm: Implement SVE2 integer absolute difference and accumulate long Date: Fri, 16 Apr 2021 14:01:38 -0700 Message-Id: <20210416210240.1591291-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix select offsetting and argument order (laurent desnogues). --- target/arm/helper-sve.h | 14 ++++++++++ target/arm/sve.decode | 12 +++++++++ target/arm/sve_helper.c | 23 ++++++++++++++++ target/arm/translate-sve.c | 55 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 104 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index c2155cc544..229fb396b2 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2402,3 +2402,17 @@ DEF_HELPER_FLAGS_4(sve2_sqcadd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqcadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqcadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqcadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sabal_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sabal_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sabal_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uabal_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uabal_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uabal_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7508b901d0..56b7353bfa 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -70,6 +70,7 @@ &rpr_s rd pg rn s &rprr_s rd pg rn rm s &rprr_esz rd pg rn rm esz +&rrrr_esz rd ra rn rm esz &rprrr_esz rd pg rn rm ra esz &rpri_esz rd pg rn imm esz &ptrue rd esz pat s @@ -119,6 +120,10 @@ @rdn_i8s ........ esz:2 ...... ... imm:s8 rd:5 \ &rri_esz rn=%reg_movprfx +# Four operand, vector element size +@rda_rn_rm ........ esz:2 . rm:5 ... ... rn:5 rd:5 \ + &rrrr_esz ra=%reg_movprfx + # Three operand with "memory" size, aka immediate left shift @rd_rn_msz_rm ........ ... rm:5 .... imm:2 rn:5 rd:5 &rrri @@ -1235,3 +1240,10 @@ CADD_rot90 01000101 .. 00000 0 11011 0 ..... ..... @rdn_rm CADD_rot270 01000101 .. 00000 0 11011 1 ..... ..... @rdn_rm SQCADD_rot90 01000101 .. 00000 1 11011 0 ..... ..... @rdn_rm SQCADD_rot270 01000101 .. 00000 1 11011 1 ..... ..... @rdn_rm + +## SVE2 integer absolute difference and accumulate long + +SABALB 01000101 .. 0 ..... 1100 00 ..... ..... @rda_rn_rm +SABALT 01000101 .. 0 ..... 1100 01 ..... ..... @rda_rn_rm +UABALB 01000101 .. 0 ..... 1100 10 ..... ..... @rda_rn_rm +UABALT 01000101 .. 0 ..... 1100 11 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 2e09c3e55b..4871e90d9b 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1241,6 +1241,29 @@ DO_ZZZ_NTB(sve2_eoril_d, uint64_t, , DO_EOR) #undef DO_ZZZ_NTB +#define DO_ZZZW_ACC(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + intptr_t sel1 = simd_data(desc) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel1)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel1)); \ + TYPEW aa = *(TYPEW *)(va + HW(i)); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, mm) + aa; \ + } \ +} + +DO_ZZZW_ACC(sve2_sabal_h, int16_t, int8_t, H1_2, H1, DO_ABD) +DO_ZZZW_ACC(sve2_sabal_s, int32_t, int16_t, H1_4, H1_2, DO_ABD) +DO_ZZZW_ACC(sve2_sabal_d, int64_t, int32_t, , H1_4, DO_ABD) + +DO_ZZZW_ACC(sve2_uabal_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) +DO_ZZZW_ACC(sve2_uabal_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) +DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) + +#undef DO_ZZZW_ACC + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c594c59954..6ac50fd61f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -163,6 +163,18 @@ static void gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn, vsz, vsz, data, fn); } +/* Invoke an out-of-line helper on 4 Zregs. */ +static void gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn, + int rd, int rn, int rm, int ra, int data) +{ + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), + vsz, vsz, data, fn); +} + /* Invoke an out-of-line helper on 2 Zregs and a predicate. */ static void gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn, int rd, int rn, int pg, int data) @@ -6316,3 +6328,46 @@ static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a) { return do_cadd(s, a, true, true); } + +static bool do_sve2_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, + gen_helper_gvec_4 *fn, int data) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); + } + return true; +} + +static bool do_abal(DisasContext *s, arg_rrrr_esz *a, bool uns, bool sel) +{ + static gen_helper_gvec_4 * const fns[2][4] = { + { NULL, gen_helper_sve2_sabal_h, + gen_helper_sve2_sabal_s, gen_helper_sve2_sabal_d }, + { NULL, gen_helper_sve2_uabal_h, + gen_helper_sve2_uabal_s, gen_helper_sve2_uabal_d }, + }; + return do_sve2_zzzz_ool(s, a, fns[uns][a->esz], sel); +} + +static bool trans_SABALB(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, false, false); +} + +static bool trans_SABALT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, false, true); +} + +static bool trans_UABALB(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, true, false); +} + +static bool trans_UABALT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, true, true); +} From patchwork Fri Apr 16 21:01:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422610 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp727078jaf; Fri, 16 Apr 2021 14:19:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy7HvBdh1pMDmtTvpZVa1PtdaxMP4kfFFZnoTUKaMBLt4o0pJXRQ++/m4HBHhykOydpHjNe X-Received: by 2002:a5d:93da:: with SMTP id j26mr4965417ioo.6.1618607950673; Fri, 16 Apr 2021 14:19:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607950; cv=none; d=google.com; s=arc-20160816; b=Fxd3S5er7uLiUS2bWy99D+SRz/4vqs99uRnY8FAUiNVlOWv2027jHT5lcPLZX3WMlT FwJLFiFfHruvLeJ3JjwzR7HN91gKMDURpdKoVoPzE7sJhxxhVDJeV6I0LEtFc5O7T2OT PK5PIPGDE7rGKL80w6Y31PhlchlWz5v3rPYB5tuosmhU+SWPhhcrthkFEjAE5AWcxPE6 rYvC/bJWQ0KqsrxdrsSL/DaQcHodT8dTYboxHjRuA6QhV49KTLsbgUUo95d+ij0GK2jz lWoxjWTWCIL9WjptLyfvtY3BguQfnIXL9YPn9CD36LjjuJHeUPO8Hn4ffFEdxdMLmulk 8qiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=1T/TF/O8cuPuJru/dQDvvvNN7AjjLNborpcatweuNPQ=; b=lCtogVYeJpGc0nDtRHlb9AnjQJug7jbsGN+AhNldV+cHQWNpqru3n9idiDec3D1+n2 dntz6mm0WrYhd8rlMm52brHv0/FgyECvEVqFMB4tYQ5gQG0Svj6r9fBd0SSZPXEA7UJS NtBAw2cpfHIrG0iVKmm3EGYz/RKpt2s/NHFrZ1TeKc6eBCaeU4jKHGQ1KJ3kYOxN9oWn 4uAorRRAXGrqqiqkUNZfqyJ/6QP9yyu9p0H9AJEGGP5dS5ZUE+6egxiI7ny1+T1krMsp KIwqxoVAKayfqZUko5YK9DXUeNYH7PsJZd02vRjyiIK8o/wKp1FAjkFl7WkiH2lcESe0 fyTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JThQJHw3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d18si6673320ilo.12.2021.04.16.14.19.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:19:10 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JThQJHw3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60528 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVrq-0007WM-1d for patch@linaro.org; Fri, 16 Apr 2021 17:19:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43856) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcL-0004KO-NS for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:09 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]:41942) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcF-00019v-Gi for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:09 -0400 Received: by mail-pj1-x1031.google.com with SMTP id z22-20020a17090a0156b029014d4056663fso15283660pje.0 for ; Fri, 16 Apr 2021 14:03:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1T/TF/O8cuPuJru/dQDvvvNN7AjjLNborpcatweuNPQ=; b=JThQJHw3wCliuBpXdlhpX5L1AOTApBlNGYhRutedNhaFVjOBGBn1uN3iV4aOTIms1M Z8NCJjOs8uYK+cLmJJt/H7+jXVU9g4pCXZ5m844FUXbfy7omf2ZkL/a8w/WqjKvQ1TYL /cWwZH530MWkk8foCqHGm1F1QCzrY4XZkUPOhuQ3wFjAZ/CB6cWvRPos4H6pmzCdtGhF HjMj2jnwWlrGVKzV8eyUi+mzifQjxTpr/O1/qt3NO8Y61IdH4U2fbbNaiaLj/FJ2P52M pXVVWPdx0VKv0BZs3SVa91Sxix/vQb0NOIzUUAE97BVZZ9+SRnmFDIpARYIVGglwTrlF TPMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1T/TF/O8cuPuJru/dQDvvvNN7AjjLNborpcatweuNPQ=; b=lMriHvbAgDvwHRi/EQG4XEVRi8sco1Tx9rY4P4GK7r7X5nQoDlERMNsRiSgx2ZVk+4 cvhXVs8DNoxjS8P54SZcp+p9p3xviTjsdeEAHGqTwtnOKTf0k/b9eGOQ38+p4CP5iIv5 49OIEchmqOFJWVzN/6VOPC0zlaJGkpqgM7aVat3ji0J6VhCInRvQizlO1K4fKDN6RCCY i6cbxuxoewr7/YXVlp1TY1HH4FesBHM3Ze/bobNM31p+Vd9P2xjr3XpQM4v707JABv1V ef2aa4VEHj659OgXbHKBDDzcIqprbXlIW65RmsTxGBQg7GXzJhBpXH2yu7Em2Gh+SeC5 Aw0g== X-Gm-Message-State: AOAM532kwopZLr/5nS6MDU7LlxWROjTZlPVy3inTCQlD/3R88srMlmkg 6DVldQYHcrWigurFpvIwNTBj5Lqp+JREmw== X-Received: by 2002:a17:902:7581:b029:e7:3780:3c5f with SMTP id j1-20020a1709027581b02900e737803c5fmr11426665pll.59.1618606982237; Fri, 16 Apr 2021 14:03:02 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 20/81] target/arm: Implement SVE2 integer add/subtract long with carry Date: Fri, 16 Apr 2021 14:01:39 -0700 Message-Id: <20210416210240.1591291-21-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix sel indexing and argument order (laurent desnogues). --- target/arm/helper-sve.h | 3 +++ target/arm/sve.decode | 6 ++++++ target/arm/sve_helper.c | 34 ++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 23 +++++++++++++++++++++++ 4 files changed, 66 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 229fb396b2..4a62012850 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2416,3 +2416,6 @@ DEF_HELPER_FLAGS_5(sve2_uabal_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_uabal_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_adcl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_adcl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 56b7353bfa..79046d81e3 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1247,3 +1247,9 @@ SABALB 01000101 .. 0 ..... 1100 00 ..... ..... @rda_rn_rm SABALT 01000101 .. 0 ..... 1100 01 ..... ..... @rda_rn_rm UABALB 01000101 .. 0 ..... 1100 10 ..... ..... @rda_rn_rm UABALT 01000101 .. 0 ..... 1100 11 ..... ..... @rda_rn_rm + +## SVE2 integer add/subtract long with carry + +# ADC and SBC decoded via size in helper dispatch. +ADCLB 01000101 .. 0 ..... 11010 0 ..... ..... @rda_rn_rm +ADCLT 01000101 .. 0 ..... 11010 1 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 4871e90d9b..0049ad861f 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1264,6 +1264,40 @@ DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) #undef DO_ZZZW_ACC +void HELPER(sve2_adcl_s)(void *vd, void *vn, void *vm, void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int sel = H4(extract32(desc, SIMD_DATA_SHIFT, 1)); + uint32_t inv = -extract32(desc, SIMD_DATA_SHIFT + 1, 1); + uint32_t *a = va, *n = vn; + uint64_t *d = vd, *m = vm; + + for (i = 0; i < opr_sz / 8; ++i) { + uint32_t e1 = a[2 * i + H4(0)]; + uint32_t e2 = n[2 * i + sel] ^ inv; + uint64_t c = extract64(m[i], 32, 1); + /* Compute and store the entire 33-bit result at once. */ + d[i] = c + e1 + e2; + } +} + +void HELPER(sve2_adcl_d)(void *vd, void *vn, void *vm, void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int sel = extract32(desc, SIMD_DATA_SHIFT, 1); + uint64_t inv = -(uint64_t)extract32(desc, SIMD_DATA_SHIFT + 1, 1); + uint64_t *d = vd, *a = va, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 8; i += 2) { + Int128 e1 = int128_make64(a[i]); + Int128 e2 = int128_make64(n[i + sel] ^ inv); + Int128 c = int128_make64(m[i + 1] & 1); + Int128 r = int128_add(int128_add(e1, e2), c); + d[i + 0] = int128_getlo(r); + d[i + 1] = int128_gethi(r); + } +} + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6ac50fd61f..6f5e39b741 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6371,3 +6371,26 @@ static bool trans_UABALT(DisasContext *s, arg_rrrr_esz *a) { return do_abal(s, a, true, true); } + +static bool do_adcl(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[2] = { + gen_helper_sve2_adcl_s, + gen_helper_sve2_adcl_d, + }; + /* + * Note that in this case the ESZ field encodes both size and sign. + * Split out 'subtract' into bit 1 of the data field for the helper. + */ + return do_sve2_zzzz_ool(s, a, fns[a->esz & 1], (a->esz & 2) | sel); +} + +static bool trans_ADCLB(DisasContext *s, arg_rrrr_esz *a) +{ + return do_adcl(s, a, false); +} + +static bool trans_ADCLT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_adcl(s, a, true); +} From patchwork Fri Apr 16 21:01:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422600 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp723348jaf; Fri, 16 Apr 2021 14:12:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzqraAxiGdTM6Nx8a02QLluVP0h5tdlH9qHfJuNUZJOd0czGhPyQ7R+SpbBpFiHxH4cl00J X-Received: by 2002:aca:4e56:: with SMTP id c83mr7800792oib.166.1618607534141; Fri, 16 Apr 2021 14:12:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607534; cv=none; d=google.com; s=arc-20160816; b=rwP/5Ba6tQI9v05jIVvAk9pXDKM0t2UnulbAisz/vD3vCT7DNPvcvjmrHV37gOI/a6 zAhYDGsayRBuqZ8QJ5BTAf8BMZLT6cgIUhvude28eNi1MX5aWNsgET0NUVJkJIiLlmRF GRcY7W6Qapx6NmSqyxEYfVQ9HIjVbmz8W3gRDiJQw38oZe7nFCWxJvuHE3HLHcv+FXI8 kgpmSnSHk/zAdky3smE9MAbywzz1cYWi9dmfKnp6Re+7/izDHdkpC0oPyDrnd6VXirV4 VXgnV40UjugHaiyyLzQ7uKvwUgoy3eFBWY5VSIq6B95VtZrFRK64+l6jIKiG1mht7Hbp o/xQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=KjnryLyEOHY0qpMOVIlDWjkao3wWlEhJ3Rqkl3sc8kg=; b=a0TqCtBWiPKYv/sOotfz1Z21S0LWmZbxZJIlwVEF3q8IY/PFm3KffwSWlyGooxT1sF rTCqvbp4Z7m9PpZz/a9hv9RdpZpYil8Q3FYmJ6+AGOBTAqlSK+YnYmvbw0unk/7rkyZ4 xqSswkLPpD+wawag6pQeQpGVRCdeyCX35jblxOO4HEhU8x+zyE7okUJFbHrH/OQGC1Ls NlnACyVb/+Y4FwhSThZ8bK0CKzNg/lb0qBcEg2Lo5DA52t7CUrLG0RY+XQhrb9cxDBJW cMLmOUzCfIyGsSkPAuWtkf6TE3zZbVxt4XSgO4P+u0RlW6G2XHkz5t5Sw9PnD9GxZkDS PXLw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="kOd/CTO5"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i21si8618613otj.220.2021.04.16.14.12.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:12:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="kOd/CTO5"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41962 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVl7-0007zm-HL for patch@linaro.org; Fri, 16 Apr 2021 17:12:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43872) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcM-0004LM-1u for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:10 -0400 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]:51042) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcG-0001AW-Jz for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:09 -0400 Received: by mail-pj1-x102e.google.com with SMTP id u11so11009943pjr.0 for ; Fri, 16 Apr 2021 14:03:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KjnryLyEOHY0qpMOVIlDWjkao3wWlEhJ3Rqkl3sc8kg=; b=kOd/CTO5zXQ4xCyhMCXXbMP32T39BI/fF9VisOt/gxNDXGVIqPcVXSiRXyvTU3Xwkz g3yzSU3QpNTRXWKUPV6b6IXk9UPvlyrF4MuTw4bfdojODoD7+BqQQ0F6Y0QeBx3tlo8Y lroNMS1cZpOkCH3iGD2PrsvOU4jscIiSsTVpcxPWeoBDVyEVS5DYB2KpXGSvTtjqrBHo u76AUkcvZoR5BN6FRTObPy34fCNfTPW5l5VNgAYfgr6pb1VmSh/P1Vm5/kcUlmyxX18e 6jzvrlOHOc6SiLzbPCapQWRI1Q6RXtvVX1uCHHtm5YRQRei/vB0JOAOCcgeyIJlnZaDE BS+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KjnryLyEOHY0qpMOVIlDWjkao3wWlEhJ3Rqkl3sc8kg=; b=IPHbeNNoliW/4rx101t9qQIbV6waAEh4We/NGHEBDYXRAxKUMAbKIeeHyfqbd1eXwj 0tO/Rsm0xitRK0qe7x7D6L797gznuaj3Fw6QN/jSFd4cWne91p7nhK/skms4Ewi2izHY HnjMmHXOC99lrI/JjjgIiWj+i573gE79L71+lkK9l7YqcryfZo1gTA0akoOSqW3VW2cA Dgf7HKH8/doUGvExBz3WCFHwl5BaIbhmaJwsFQqECkwJKFEm97JonrATh8A4DlmnGgHR RgUdz+dEjpYuuH2tFeJe9IqVn6IW6+WhSRFbhTU0i8OsDqVdoEpdkHZO8aZdVwS+mNBN z23Q== X-Gm-Message-State: AOAM530ikU9GGDOjUVKQDH+8qUJ3DOXI3YRCI3GysoCVnNy/cFo0xIM7 3F8Sr6pdX3Ubi+tuOqxE7AifmL8PUhsKOA== X-Received: by 2002:a17:902:b60d:b029:e6:7a9:7f4 with SMTP id b13-20020a170902b60db02900e607a907f4mr11498703pls.3.1618606983272; Fri, 16 Apr 2021 14:03:03 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:02 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 21/81] target/arm: Implement SVE2 bitwise shift right and accumulate Date: Fri, 16 Apr 2021 14:01:40 -0700 Message-Id: <20210416210240.1591291-22-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102e; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 8 ++++++++ target/arm/translate-sve.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 79046d81e3..d3c4ec6dd1 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1253,3 +1253,11 @@ UABALT 01000101 .. 0 ..... 1100 11 ..... ..... @rda_rn_rm # ADC and SBC decoded via size in helper dispatch. ADCLB 01000101 .. 0 ..... 11010 0 ..... ..... @rda_rn_rm ADCLT 01000101 .. 0 ..... 11010 1 ..... ..... @rda_rn_rm + +## SVE2 bitwise shift right and accumulate + +# TODO: Use @rda and %reg_movprfx here. +SSRA 01000101 .. 0 ..... 1110 00 ..... ..... @rd_rn_tszimm_shr +USRA 01000101 .. 0 ..... 1110 01 ..... ..... @rd_rn_tszimm_shr +SRSRA 01000101 .. 0 ..... 1110 10 ..... ..... @rd_rn_tszimm_shr +URSRA 01000101 .. 0 ..... 1110 11 ..... ..... @rd_rn_tszimm_shr diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6f5e39b741..c11074cccc 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6394,3 +6394,37 @@ static bool trans_ADCLT(DisasContext *s, arg_rrrr_esz *a) { return do_adcl(s, a, true); } + +static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn) +{ + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + unsigned rd_ofs = vec_full_reg_offset(s, a->rd); + unsigned rn_ofs = vec_full_reg_offset(s, a->rn); + fn(a->esz, rd_ofs, rn_ofs, a->imm, vsz, vsz); + } + return true; +} + +static bool trans_SSRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_ssra); +} + +static bool trans_USRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_usra); +} + +static bool trans_SRSRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_srsra); +} + +static bool trans_URSRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_ursra); +} From patchwork Fri Apr 16 21:01:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422618 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp729954jaf; Fri, 16 Apr 2021 14:24:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyAYlnxCAo7/bSkZ2ifd++TEjIvXNisJ9zVyXV7Fx0ImRJQZgwwOSX8czRzpLKjNrwiF4zR X-Received: by 2002:a05:6e02:1351:: with SMTP id k17mr8552620ilr.37.1618608246162; Fri, 16 Apr 2021 14:24:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608246; cv=none; d=google.com; s=arc-20160816; b=xz7jUGqIRdutB0cUJ5B5bNwhXiwqn81H/2V/+sJvQVd7QwiucaGwdC4vmZ57gsUQ24 PAAAn2kuMfWwkmPxf4Rsx6dDsQRgbXCJuee5YpLEwUglih2JY6WiKcUpiwFDzCJnOuRk 5tglLuOPtkFVqXpntZZGx64qvuITi3rg6cWwqo4P5oSg0+BOm/9S05f/L6PjxDQpXbJW JuMtCHntDOw5BhhAvtpsllLitcP8y5cgPnry3IVykvLgQ6UQ/gJrZo09vi9iRjQdQtLH t6Iq/dGceXuJb4KapmO3xFvkcwyA5IIAxdTPJY35PbiCcaz0wDmNV4iYmMAZHB1TjL0W 1xwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=XWdhkzqHe8iJijTRujXmdCXfYmUKYPpZlGIVkZOqBWQ=; b=AzaS/evOGPwdGphlcIGveAg2eXBhi9tyYt1EPq7xT0Io5LlOnjW0tWY5+4dQvl1Qnj M65BylF29mW6/AaUT59LLCSVV3br2uFO5d2cP0SVugn/HLSUUf2zb5IY5zJQRlMZ6WOW WbfUM1vHaw9XEynxs2T0wWJ86PYgh6CHXOrXUSdGmUmYj6tnBqwRtmn/Fw9YmhFBUFcX sH0fYupGDCbWYUFPzjeYLQZ2IK8lUfxDacLBhlWZ5ZXzn1khBvo5AipP0qo946Itxbag bVUB64xot2GkUtx8RCZz+MszFn0xU1on7AQYQS96eaCgD/dFfsNUxDBGTnMrLdRRFjKh zFBw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=s1IFdpLl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j6si7144047ilu.44.2021.04.16.14.24.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:24:06 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=s1IFdpLl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53924 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVwb-00089t-Js for patch@linaro.org; Fri, 16 Apr 2021 17:24:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43902) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcN-0004OG-4a for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:11 -0400 Received: from mail-pg1-x534.google.com ([2607:f8b0:4864:20::534]:45000) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcI-0001Bt-3U for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:10 -0400 Received: by mail-pg1-x534.google.com with SMTP id y32so19997445pga.11 for ; Fri, 16 Apr 2021 14:03:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XWdhkzqHe8iJijTRujXmdCXfYmUKYPpZlGIVkZOqBWQ=; b=s1IFdpLlqQ1L5fv0FTDrAIpbnnxa6QiLP2RYTVXO0aGQK7AL53Gsn/7oc9NWL7EAYK wfaFpkqhVwtpaDE6Ln7pG367e6S5EBUWs9LRiSxaFVeyxo/hQEgsx2rxFGLMDfuaC678 +iZiSK1Hc6wWZrsx5VH6nA2OkDytIx+ewOgAag6RDrY+6DKd6fnIrguOUPVQx4RnVXHf ec57JUryNlheRWbMciUOFrR7LaS557vgapX+X8fk9L75f8YliAiDc35sKWJdt+EOqRRv Ji2gw0R6QmOaYD3+c9eCc0of/U3kGRi6zLfNg3lb+fA6WTx7AtkXHs2JElsZkXByp7Yt +2xQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XWdhkzqHe8iJijTRujXmdCXfYmUKYPpZlGIVkZOqBWQ=; b=B3nqaOeA3I50SnmobFz5XQlQiPg9UuDtV6XPYQ3XKuc4FTa0anRRhGrYb0lTByPrBA X5gVwajD5x7pIqdbX57RvfLzFxEDYH7vWnAbsLIu1NJtp7YOlQ5pPIHRfbJsugH2X2sr vIWHkRDZbxDL0844jkVZpcfRXh9JvZffVpRdlAu/IPhmWaTDsKAdVlVpyKeOtCUKbf82 vAMipPmlLDRJaf0R0JMmepz5cn8pJTzEshMzZsYzg57XxwTNqTX1mPMw2DHeBSSGCSSJ 1sOLJGc+6YDf89azcBqn7FI7h+lfCdaDiQ3HbsqybtkemOXXqqg/olFvGOHiv/4SePy7 M90A== X-Gm-Message-State: AOAM533B04KjOtImCHk3wnbc/Ge3n5wu9gJYtdgngFz8E40MKqTfrgDV L265cJF47vdtrq15b9mA8JMriFrbQzVP/Q== X-Received: by 2002:a63:2c92:: with SMTP id s140mr819968pgs.39.1618606984307; Fri, 16 Apr 2021 14:03:04 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:03 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 22/81] target/arm: Implement SVE2 bitwise shift and insert Date: Fri, 16 Apr 2021 14:01:41 -0700 Message-Id: <20210416210240.1591291-23-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::534; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x534.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 5 +++++ target/arm/translate-sve.c | 10 ++++++++++ 2 files changed, 15 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index d3c4ec6dd1..695a16551e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1261,3 +1261,8 @@ SSRA 01000101 .. 0 ..... 1110 00 ..... ..... @rd_rn_tszimm_shr USRA 01000101 .. 0 ..... 1110 01 ..... ..... @rd_rn_tszimm_shr SRSRA 01000101 .. 0 ..... 1110 10 ..... ..... @rd_rn_tszimm_shr URSRA 01000101 .. 0 ..... 1110 11 ..... ..... @rd_rn_tszimm_shr + +## SVE2 bitwise shift and insert + +SRI 01000101 .. 0 ..... 11110 0 ..... ..... @rd_rn_tszimm_shr +SLI 01000101 .. 0 ..... 11110 1 ..... ..... @rd_rn_tszimm_shl diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c11074cccc..d74a15d8b8 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6428,3 +6428,13 @@ static bool trans_URSRA(DisasContext *s, arg_rri_esz *a) { return do_sve2_fn2i(s, a, gen_gvec_ursra); } + +static bool trans_SRI(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_sri); +} + +static bool trans_SLI(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_sli); +} From patchwork Fri Apr 16 21:01:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422613 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp728501jaf; Fri, 16 Apr 2021 14:21:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyqW2Q5nrGT3YzouEMLrSzhOoOPhwvdI4451/Y1fykT9pthUZQaVIywrOaJU+aS7imFvutR X-Received: by 2002:a1f:9e8c:: with SMTP id h134mr9007442vke.10.1618608089319; Fri, 16 Apr 2021 14:21:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608089; cv=none; d=google.com; s=arc-20160816; b=Qkyqqk4uA+DUVVMqrN7qjkhgJWMQE5iusiUw3hAsNnwafOV9Wxtx/2C3Q+4tx8saXa jvS/wrE/BrV1PAGwQ9xYIgFx5t1o4Jp20M5q93ASheAMkhI6fzXU3sWDv30In+FWJ5tn cntkL6nrueq0L4aeMsv9OWoRcES/i98azDJqz/wcOq+bIOb4ALgn5jnQxsyrPvo5sUR7 CnuA9P27tRPvHxavpYoSBFWfYI2ugRq9g+l5Q2DTqQp01A+3eQEsIJAu5zPZAQibru/8 kR2YEVTbcK54req1vfAJSrJW7uRxMkPWo5A8ZCWomp5u9AGD4EKsU1nhTUVeJEW1OYqg KOxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=QaR3WAAAd20RRoi+x5aKUmW3qXGunVbXS8iCJuTuVr8=; b=RNDFOGGrWH7pMM//yBmT4sMtpJZPWmK5eXgrASGiRlC0JT8WbIcjOPACfhIgO/Y3SW Wxgmf/2IDbaGiFCYSiekxb0TyWg7778pxYwt3fiEzUfF33InwHxo/GeX61gaLPnuLmcn L9LzhCdYWh76YwqwEG+jOI/qwC4GjuEiQzgUl3RzkVoHKsJV7lPUcOANZ7DoVUmpKwrA xBXF6J1aC5nw5/zYCWcSbmnUFPoD3EjN7As+5zyWrqqW3seHs3GMVny56H6tL3SFNcgx l+U1ortwPrX0rXPSCruP0IJPVTGjyvOrBa+OIQa4cV4vOI7Svki8whNVvfjqLusE0MwI M8xQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="blfEhj+/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i25si3271773ual.62.2021.04.16.14.21.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:21:29 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="blfEhj+/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41112 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVu4-0002s7-Me for patch@linaro.org; Fri, 16 Apr 2021 17:21:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43964) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcO-0004T8-Tm for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:12 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]:44984) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcI-0001CJ-Im for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:12 -0400 Received: by mail-pl1-x62d.google.com with SMTP id d8so14625287plh.11 for ; Fri, 16 Apr 2021 14:03:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QaR3WAAAd20RRoi+x5aKUmW3qXGunVbXS8iCJuTuVr8=; b=blfEhj+/e9n82/0dey7ShIuhnTV+UC50wKoAi0lNs7RYAMDxofQ+WacQJD/iJ8LtM3 s6EyXwuzeNRl7WIrgA9FYDXhX1iO9PaPFFh8OtSLTQXgic74vUtcn9qyMJknYAWpWMCm ojNIdiSrvecwHXxSq+4U338FcIVFIr4R9M51zDlYm3B00QXSxHlwQMdjp5KuU8BZ2h9Y 8wZXkYqQspQvvwUoYwzxHCFHBa0KQr6yLB147O67wn+gC/XoLzo1DzW2Vod6UQLGYTr9 05xQcjA3UGY0fX0sVVRUtytUpermy4DoFK7irTw9btuJBHrc5n8jfUWfd1bnz8fgpZVl Pe3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QaR3WAAAd20RRoi+x5aKUmW3qXGunVbXS8iCJuTuVr8=; b=lIXdZ0+zo7Gi4Fyqr2n1OZzKjNHFW8T480jaV9aaxmBG9YUEe8vFeubFAVa52M3jJ8 aO2NjTUWKlxhEkIuL7DG5NdJvXVD6sbWv0Ly0LkD7bHeqJgCuMuN8Rhb7tMA7ZQFWu3E sMQxh5EmdWd4c5Khhz9y2cRWJ34ZOUR2VyvQbGHu44lYCHHXr/8A+5y+soQQ6bcHDLMi 5AhmjvuY5GMNexcvq/MQvkaBTkdt2so3xoOzXp345ZHDmPwCDOlpoL3WpLIX5IIGnD0m V7me2VIwGJXqfPeOxZFl5Ak7D3wFx4t8KFPyiU18rviwUOrgK7IIVoOSbQ+BoeDMrfY/ PakQ== X-Gm-Message-State: AOAM5315MDItdisBthrbN3pLWy4jIw1aLxcYkHQDpxPgqKdx+0fsnqin Sj+ddiqZmmyQIIQobdBxChmYwvdem2B6Nw== X-Received: by 2002:a17:902:8d83:b029:e6:508a:7b8d with SMTP id v3-20020a1709028d83b02900e6508a7b8dmr11623499plo.18.1618606985315; Fri, 16 Apr 2021 14:03:05 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 23/81] target/arm: Implement SVE2 integer absolute difference and accumulate Date: Fri, 16 Apr 2021 14:01:42 -0700 Message-Id: <20210416210240.1591291-24-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 6 ++++++ target/arm/translate-sve.c | 21 +++++++++++++++++++++ 2 files changed, 27 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 695a16551e..32b15e4192 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1266,3 +1266,9 @@ URSRA 01000101 .. 0 ..... 1110 11 ..... ..... @rd_rn_tszimm_shr SRI 01000101 .. 0 ..... 11110 0 ..... ..... @rd_rn_tszimm_shr SLI 01000101 .. 0 ..... 11110 1 ..... ..... @rd_rn_tszimm_shl + +## SVE2 integer absolute difference and accumulate + +# TODO: Use @rda and %reg_movprfx here. +SABA 01000101 .. 0 ..... 11111 0 ..... ..... @rd_rn_rm +UABA 01000101 .. 0 ..... 11111 1 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index d74a15d8b8..ba1953118b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6438,3 +6438,24 @@ static bool trans_SLI(DisasContext *s, arg_rri_esz *a) { return do_sve2_fn2i(s, a, gen_gvec_sli); } + +static bool do_sve2_fn_zzz(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm); + } + return true; +} + +static bool trans_SABA(DisasContext *s, arg_rrr_esz *a) +{ + return do_sve2_fn_zzz(s, a, gen_gvec_saba); +} + +static bool trans_UABA(DisasContext *s, arg_rrr_esz *a) +{ + return do_sve2_fn_zzz(s, a, gen_gvec_uaba); +} From patchwork Fri Apr 16 21:01:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422605 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp724771jaf; Fri, 16 Apr 2021 14:14:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzVVNv9tWdL2gw60Cpd8ytsG05LHzIWiBVyVnSLR8uIA5ksOvu5hQ0kagIDafM9LWc9uKAZ X-Received: by 2002:a5e:930d:: with SMTP id k13mr5085988iom.61.1618607695011; Fri, 16 Apr 2021 14:14:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607695; cv=none; d=google.com; s=arc-20160816; b=A1YukPBb7RHMyUCiNT0zteymKkwehM4hrElc1p0QhUPO+PXI1M9X9txGDkMjAkKeZV AVGyRHm6I3JE3+v+D1ZXVVJsnuAuxIPSJ+uPyzLKQVGTdJwMZfc364wpUW8j1h6ka5+Z 5XvZvZiBTCNHAryol9BT4fw5EHB0BFfaOYaHjB2fLgfXJ06Ohe2LzV5rBvAjuoiAv84j rPl3bPb8ssq4cfzLeuu8nJyFqRUVrcBS+OGBZNYYMbDv1tLRXYSAjERtRPPhl8O3/ZbG g7OF6GLvo6NnfHWiGW0ysYdgb9A4wuwRNvYK50/U+nzHrtGQrPzz+OWQaoB2SIJiIUFS TuaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=uNqnL76ppa/JptnkZAzi8U8bmGMNrJo5oaqSMO/4t5I=; b=fXGWC/KJccwzEr3c6sEV7bZbu7O6wuMKzPh/FxhDAPw4+g6agJmPJcqr1/PTkv1FxS IQaULEgQJuXss0FjwVdqy3tLwiZQQLQ+MrU07yq07z7NtsLfhqFKAA0PYPeZeX1Ml56a QNvQchUgbA+2aW72ZI/tCpsGtxirw0xSFGiQXA6TBH5JD5fIC+E82PVAQK4+Y43FifCs iWH47gKL0uOApHzzWj8T4WVa2w8dArrC7NZYubnCOxq08S5yVUL1Z0xMCe/KpmngjZbT Q5J2UIkvunuOWaglKWc6mO2jhkfn2vBCyxiMkUd14YtPGWX8tkaz16pdYiFSYkUkfnEB fHTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EXMZJWzS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l15si2628960iln.114.2021.04.16.14.14.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:14:55 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EXMZJWzS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51762 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVni-0003co-Cl for patch@linaro.org; Fri, 16 Apr 2021 17:14:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43988) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcP-0004Vj-NK for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:13 -0400 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]:33419) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcJ-0001D4-Ms for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:13 -0400 Received: by mail-pg1-x52e.google.com with SMTP id t22so20042437pgu.0 for ; Fri, 16 Apr 2021 14:03:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uNqnL76ppa/JptnkZAzi8U8bmGMNrJo5oaqSMO/4t5I=; b=EXMZJWzSXR3/oIfnvdiwd75H0wNZGKbZtKa+IKozvT4jLOMVnBH6mtxgl2zBrMlUeF rBwH7u56GGJ8X2e0karLjFFlmL+UeLD3srVPZ1NM/DxFUMo2OopL+mlMCoQhFfXeo3HR n15i7zzbwMUcX/hGnLAba5QbSkykZuEY2ABaNWoTeQ8WW2TphNdNH00aFupwM3WJCt03 oiQx6Dvblhji1vQeaEoBut2D8DpyrMXQG9uacmFLXGCSNo2wVPS37qIH9imjVQQMUesx cCB9Sy1jeTCFQoC/7fcbejmJ8/GwWXVODdfWaJr8H1PnQnEaDuNNaZ+46UhmRFfxr7Df yMIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uNqnL76ppa/JptnkZAzi8U8bmGMNrJo5oaqSMO/4t5I=; b=miZ4iOcJD2pNh01sUOAhokjRvDl5cVXwtydRmH6i7OvVxPrL+UoP/at6MWJwWKU6ti HFotITt0nA/zLV9Kp7QwkGC1GkwDJLmQK3V6BRinrlqOo7kQlEK4987VL8YnGF6ayKHb qMKLJRco3zUPefmtZ/MLEWMkewCMTTsspT/C1Rv+hYna7lSaufSsmd6K453D/Gw57pwi Bxb36UAUaAI4XMYIX1PdL0BmDwfZpP2HCWJ9eQemg4ayekaZV3uT1P7TALxZ9lmVCsiU NOCHzA0Zq+XiGCyeevgmqdy/zNKv9Px6ZkI3DFrMjFmZlm3BPC8RhZheqfcNYkW88xj9 nW0w== X-Gm-Message-State: AOAM533KFdVDjkvJcZ5TboSvVkWRcqdF3mdCCYStaraFBqvh6RbJ4I86 AN1muQz/Xu2qzDFj2/gZy048Uc5KE6ZHIQ== X-Received: by 2002:a65:48c9:: with SMTP id o9mr843072pgs.184.1618606986383; Fri, 16 Apr 2021 14:03:06 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:06 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 24/81] target/arm: Implement SVE2 saturating extract narrow Date: Fri, 16 Apr 2021 14:01:43 -0700 Message-Id: <20210416210240.1591291-25-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52e; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 24 ++++ target/arm/sve.decode | 12 ++ target/arm/sve_helper.c | 56 +++++++++ target/arm/translate-sve.c | 238 +++++++++++++++++++++++++++++++++++++ 4 files changed, 330 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 4a62012850..b302203ce8 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2419,3 +2419,27 @@ DEF_HELPER_FLAGS_5(sve2_uabal_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_adcl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_adcl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqxtnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtunb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqxtnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 32b15e4192..19866ec4c6 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1272,3 +1272,15 @@ SLI 01000101 .. 0 ..... 11110 1 ..... ..... @rd_rn_tszimm_shl # TODO: Use @rda and %reg_movprfx here. SABA 01000101 .. 0 ..... 11111 0 ..... ..... @rd_rn_rm UABA 01000101 .. 0 ..... 11111 1 ..... ..... @rd_rn_rm + +#### SVE2 Narrowing + +## SVE2 saturating extract narrow + +# Bits 23, 18-16 are zero, limited in the translator via esz < 3 & imm == 0. +SQXTNB 01000101 .. 1 ..... 010 000 ..... ..... @rd_rn_tszimm_shl +SQXTNT 01000101 .. 1 ..... 010 001 ..... ..... @rd_rn_tszimm_shl +UQXTNB 01000101 .. 1 ..... 010 010 ..... ..... @rd_rn_tszimm_shl +UQXTNT 01000101 .. 1 ..... 010 011 ..... ..... @rd_rn_tszimm_shl +SQXTUNB 01000101 .. 1 ..... 010 100 ..... ..... @rd_rn_tszimm_shl +SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 0049ad861f..7dca67785a 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1264,6 +1264,62 @@ DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) #undef DO_ZZZW_ACC +#define DO_XTNB(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + i); \ + nn = OP(nn) & MAKE_64BIT_MASK(0, sizeof(TYPE) * 4); \ + *(TYPE *)(vd + i) = nn; \ + } \ +} + +#define DO_XTNT(NAME, TYPE, TYPEN, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc), odd = H(sizeof(TYPEN)); \ + for (i = 0; i < opr_sz; i += sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + i); \ + *(TYPEN *)(vd + i + odd) = OP(nn); \ + } \ +} + +#define DO_SQXTN_H(n) do_sat_bhs(n, INT8_MIN, INT8_MAX) +#define DO_SQXTN_S(n) do_sat_bhs(n, INT16_MIN, INT16_MAX) +#define DO_SQXTN_D(n) do_sat_bhs(n, INT32_MIN, INT32_MAX) + +DO_XTNB(sve2_sqxtnb_h, int16_t, DO_SQXTN_H) +DO_XTNB(sve2_sqxtnb_s, int32_t, DO_SQXTN_S) +DO_XTNB(sve2_sqxtnb_d, int64_t, DO_SQXTN_D) + +DO_XTNT(sve2_sqxtnt_h, int16_t, int8_t, H1, DO_SQXTN_H) +DO_XTNT(sve2_sqxtnt_s, int32_t, int16_t, H1_2, DO_SQXTN_S) +DO_XTNT(sve2_sqxtnt_d, int64_t, int32_t, H1_4, DO_SQXTN_D) + +#define DO_UQXTN_H(n) do_sat_bhs(n, 0, UINT8_MAX) +#define DO_UQXTN_S(n) do_sat_bhs(n, 0, UINT16_MAX) +#define DO_UQXTN_D(n) do_sat_bhs(n, 0, UINT32_MAX) + +DO_XTNB(sve2_uqxtnb_h, uint16_t, DO_UQXTN_H) +DO_XTNB(sve2_uqxtnb_s, uint32_t, DO_UQXTN_S) +DO_XTNB(sve2_uqxtnb_d, uint64_t, DO_UQXTN_D) + +DO_XTNT(sve2_uqxtnt_h, uint16_t, uint8_t, H1, DO_UQXTN_H) +DO_XTNT(sve2_uqxtnt_s, uint32_t, uint16_t, H1_2, DO_UQXTN_S) +DO_XTNT(sve2_uqxtnt_d, uint64_t, uint32_t, H1_4, DO_UQXTN_D) + +DO_XTNB(sve2_sqxtunb_h, int16_t, DO_UQXTN_H) +DO_XTNB(sve2_sqxtunb_s, int32_t, DO_UQXTN_S) +DO_XTNB(sve2_sqxtunb_d, int64_t, DO_UQXTN_D) + +DO_XTNT(sve2_sqxtunt_h, int16_t, int8_t, H1, DO_UQXTN_H) +DO_XTNT(sve2_sqxtunt_s, int32_t, int16_t, H1_2, DO_UQXTN_S) +DO_XTNT(sve2_sqxtunt_d, int64_t, int32_t, H1_4, DO_UQXTN_D) + +#undef DO_XTNB +#undef DO_XTNT + void HELPER(sve2_adcl_s)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ba1953118b..b8ced82e4e 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6459,3 +6459,241 @@ static bool trans_UABA(DisasContext *s, arg_rrr_esz *a) { return do_sve2_fn_zzz(s, a, gen_gvec_uaba); } + +static bool do_sve2_narrow_extract(DisasContext *s, arg_rri_esz *a, + const GVecGen2 ops[3]) +{ + if (a->esz < 0 || a->esz > MO_32 || a->imm != 0 || + !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vsz, vsz, &ops[a->esz]); + } + return true; +} + +static const TCGOpcode sqxtn_list[] = { + INDEX_op_shli_vec, INDEX_op_smin_vec, INDEX_op_smax_vec, 0 +}; + +static void gen_sqxtnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t mask = (1ull << halfbits) - 1; + int64_t min = -1ull << (halfbits - 1); + int64_t max = -min - 1; + + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, d, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, d, d, t); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_and_vec(vece, d, d, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtnb_vec, + .opt_opc = sqxtn_list, + .fno = gen_helper_sve2_sqxtnb_h, + .vece = MO_16 }, + { .fniv = gen_sqxtnb_vec, + .opt_opc = sqxtn_list, + .fno = gen_helper_sve2_sqxtnb_s, + .vece = MO_32 }, + { .fniv = gen_sqxtnb_vec, + .opt_opc = sqxtn_list, + .fno = gen_helper_sve2_sqxtnb_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static void gen_sqxtnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t mask = (1ull << halfbits) - 1; + int64_t min = -1ull << (halfbits - 1); + int64_t max = -min - 1; + + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtnt_vec, + .opt_opc = sqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtnt_h, + .vece = MO_16 }, + { .fniv = gen_sqxtnt_vec, + .opt_opc = sqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtnt_s, + .vece = MO_32 }, + { .fniv = gen_sqxtnt_vec, + .opt_opc = sqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtnt_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static const TCGOpcode uqxtn_list[] = { + INDEX_op_shli_vec, INDEX_op_umin_vec, 0 +}; + +static void gen_uqxtnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_UQXTNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_uqxtnb_vec, + .opt_opc = uqxtn_list, + .fno = gen_helper_sve2_uqxtnb_h, + .vece = MO_16 }, + { .fniv = gen_uqxtnb_vec, + .opt_opc = uqxtn_list, + .fno = gen_helper_sve2_uqxtnb_s, + .vece = MO_32 }, + { .fniv = gen_uqxtnb_vec, + .opt_opc = uqxtn_list, + .fno = gen_helper_sve2_uqxtnb_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static void gen_uqxtnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_UQXTNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_uqxtnt_vec, + .opt_opc = uqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_uqxtnt_h, + .vece = MO_16 }, + { .fniv = gen_uqxtnt_vec, + .opt_opc = uqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_uqxtnt_s, + .vece = MO_32 }, + { .fniv = gen_uqxtnt_vec, + .opt_opc = uqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_uqxtnt_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static const TCGOpcode sqxtun_list[] = { + INDEX_op_shli_vec, INDEX_op_umin_vec, INDEX_op_smax_vec, 0 +}; + +static void gen_sqxtunb_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, d, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, d, d, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTUNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtunb_vec, + .opt_opc = sqxtun_list, + .fno = gen_helper_sve2_sqxtunb_h, + .vece = MO_16 }, + { .fniv = gen_sqxtunb_vec, + .opt_opc = sqxtun_list, + .fno = gen_helper_sve2_sqxtunb_s, + .vece = MO_32 }, + { .fniv = gen_sqxtunb_vec, + .opt_opc = sqxtun_list, + .fno = gen_helper_sve2_sqxtunb_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static void gen_sqxtunt_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTUNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtunt_vec, + .opt_opc = sqxtun_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtunt_h, + .vece = MO_16 }, + { .fniv = gen_sqxtunt_vec, + .opt_opc = sqxtun_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtunt_s, + .vece = MO_32 }, + { .fniv = gen_sqxtunt_vec, + .opt_opc = sqxtun_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtunt_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} From patchwork Fri Apr 16 21:01:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422617 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp729650jaf; Fri, 16 Apr 2021 14:23:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyTTBA1RYTOUPHr7sBr5A7XbCpkq5tcoiJ6i0xjloz9Mooda/+qq9ivZdE9O0bQTqheUBds X-Received: by 2002:a92:d806:: with SMTP id y6mr8740940ilm.130.1618608215998; Fri, 16 Apr 2021 14:23:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608215; cv=none; d=google.com; s=arc-20160816; b=kp+k8SSDe3L8ocbeJVxmyd9CbNdnP2xkUPuchC6beCsjOBTrfx0g/nPvth1CoUqsKB 8xWOnu/ILXJ52aVUzM3lWvwjFKvXX0SDRiY6Qf/7mZTYGu4ToKh+NIizP2Yl/qcs7EZ8 XnNXPIm1RpfBgHjR+/q3OkuRy5fqMIm6bUdJHfbV4LUpcOcrBY3MFWCvMV1HZsYWAMi5 CWcy0vpE4K9Y+93Q7y1YYO57xSS7eJIksyk/gEFWQnR/UTHv4DEzt0sIEFf2F8/pAehI J+KCqs0B0y1J2M4oPz0aXk99xTk6f5XSkXIRpwMmOTUEm8JH0My+UyUdJEaQ8eEVj9lO hGLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=7dY9xjD1cw8e6w0AiGk+0Ct4FwwuYvpBMTDUl5kU++c=; b=ObXKFNWFlu0SAbSgP2M/aiW0MS7AdYKlIGbC/IMHYRqRUxGr4HAW1TGTjBzKDHUbzL aIJ/kZE8uwEQgj/ILBl9ieVztkZqyZn4NgBz889alq/o11C19mLDGZxaW6VqnfyBy1Pg cPSRmcwTL1sHoZEnhkpOjXHfh0JBUy4Ufv8k0JFKVM4K7CRtDNZghFODYuIpYNuohwY1 3SQuZcU099lqVfodn2rl9PP/CFROOydYIpBj9wxESZ8vhWsZUBL9ucVQ7Zp7cI+08lhm K2/RmF2OzEYX2/3aSBhiQJyJ9cvuwiJ3Invz8Snj9UIc//hczh8zeokrd442zZfB0ACI fLNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BogFVc8r; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j16si7801603jak.24.2021.04.16.14.23.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:23:35 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BogFVc8r; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51996 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVw7-0007Nz-D0 for patch@linaro.org; Fri, 16 Apr 2021 17:23:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44014) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcQ-0004Y5-LF for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:14 -0400 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]:45762) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcK-0001EW-KD for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:14 -0400 Received: by mail-pl1-x62a.google.com with SMTP id p16so10676050plf.12 for ; Fri, 16 Apr 2021 14:03:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7dY9xjD1cw8e6w0AiGk+0Ct4FwwuYvpBMTDUl5kU++c=; b=BogFVc8rrE5CqTm/0u7T+9fjcPv1sFr1EDfur6G3HaaqCDcWARobcUAxHEPwnT04VU xwxhDX/1ZqoasC2adc9YGb72tkxeq4AUCzMz8jY3fmSE1hAM3wwwpK0mJlpOisdm7zog /eZ3pGlvuF5HTWFQZ2uKWvgnhrvB7iO/Rjn+8mDhvfXinkoTSElikKmXrW2uqHNYu0Zb 9oXtsCdv0uboBAibC0pKlEnVS8Fn+pNmsosbUPnHK6U6jcsJEFabnA74Fqgh1hjSLiIT QSCOFpAM9kvbgxDv7Nn4E17pz18GfP5Y90ebobj69/JOhJk/AGVlItQkfaqCYgTu8D8n dpTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7dY9xjD1cw8e6w0AiGk+0Ct4FwwuYvpBMTDUl5kU++c=; b=T2oHiIznTwV9PwkwfLGfzwsijCCm6y625JdlkUZ+PChEUkJKKkdWY8fV+5gfn6JLrl qytEL8Ljbp0Hdnyer8Axmhwjxxw6H4am7/yk/2iTdwicbMR7k+cvpOO1SqI1BljtlUob Ao9DE+yuPlLB3YtA0NGInfJrLDZh6TDQA66mLu+ymV3RqLO776A/raoYVys7QY9K4WhB YbS++wJR0caGl4eO/+WMmNL7zg/uqmUdI23wPjdyjCL3h8WdJ/4rSM8nB5oEiHKmcVqr 27d2PWiA5kJzFqPIZi8uJGsY+YMFOrQSXbcu1omjXrU3t/gyX+SG3gtDDk/bhYuCY2Hx DuFQ== X-Gm-Message-State: AOAM5332+MrkcbY4cp5Gooj1AQg4oustvhAXuH4oKGE0zJVoBt0UPLXC mx8W3odbJqjUB+jUAg7Uey+2ZsdAZCV5uA== X-Received: by 2002:a17:902:ed06:b029:ec:7b39:9739 with SMTP id b6-20020a170902ed06b02900ec7b399739mr5167531pld.5.1618606987385; Fri, 16 Apr 2021 14:03:07 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:07 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 25/81] target/arm: Implement SVE2 floating-point pairwise Date: Fri, 16 Apr 2021 14:01:44 -0700 Message-Id: <20210416210240.1591291-26-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Reviewed-by: Richard Henderson Signed-off-by: Richard Henderson --- v2: Load all inputs before writing any output (laurent desnogues) --- target/arm/helper-sve.h | 35 +++++++++++++++++++++++++++++ target/arm/sve.decode | 8 +++++++ target/arm/sve_helper.c | 46 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 25 +++++++++++++++++++++ 4 files changed, 114 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index b302203ce8..a033b5f6b2 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2443,3 +2443,38 @@ DEF_HELPER_FLAGS_3(sve2_uqxtnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 19866ec4c6..9c75ac94c0 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1284,3 +1284,11 @@ UQXTNB 01000101 .. 1 ..... 010 010 ..... ..... @rd_rn_tszimm_shl UQXTNT 01000101 .. 1 ..... 010 011 ..... ..... @rd_rn_tszimm_shl SQXTUNB 01000101 .. 1 ..... 010 100 ..... ..... @rd_rn_tszimm_shl SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl + +## SVE2 floating-point pairwise operations + +FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm +FMAXNMP 01100100 .. 010 10 0 100 ... ..... ..... @rdn_pg_rm +FMINNMP 01100100 .. 010 10 1 100 ... ..... ..... @rdn_pg_rm +FMAXP 01100100 .. 010 11 0 100 ... ..... ..... @rdn_pg_rm +FMINP 01100100 .. 010 11 1 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 7dca67785a..11f228144c 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -890,6 +890,52 @@ DO_ZPZZ_PAIR_D(sve2_sminp_zpzz_d, int64_t, DO_MIN) #undef DO_ZPZZ_PAIR #undef DO_ZPZZ_PAIR_D +#define DO_ZPZZ_PAIR_FP(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, \ + void *status, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + TYPE n0 = *(TYPE *)(vn + H(i)); \ + TYPE m0 = *(TYPE *)(vm + H(i)); \ + TYPE n1 = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE m1 = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(n0, n1, status); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(m0, m1, status); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ +} + +DO_ZPZZ_PAIR_FP(sve2_faddp_zpzz_h, float16, H1_2, float16_add) +DO_ZPZZ_PAIR_FP(sve2_faddp_zpzz_s, float32, H1_4, float32_add) +DO_ZPZZ_PAIR_FP(sve2_faddp_zpzz_d, float64, , float64_add) + +DO_ZPZZ_PAIR_FP(sve2_fmaxnmp_zpzz_h, float16, H1_2, float16_maxnum) +DO_ZPZZ_PAIR_FP(sve2_fmaxnmp_zpzz_s, float32, H1_4, float32_maxnum) +DO_ZPZZ_PAIR_FP(sve2_fmaxnmp_zpzz_d, float64, , float64_maxnum) + +DO_ZPZZ_PAIR_FP(sve2_fminnmp_zpzz_h, float16, H1_2, float16_minnum) +DO_ZPZZ_PAIR_FP(sve2_fminnmp_zpzz_s, float32, H1_4, float32_minnum) +DO_ZPZZ_PAIR_FP(sve2_fminnmp_zpzz_d, float64, , float64_minnum) + +DO_ZPZZ_PAIR_FP(sve2_fmaxp_zpzz_h, float16, H1_2, float16_max) +DO_ZPZZ_PAIR_FP(sve2_fmaxp_zpzz_s, float32, H1_4, float32_max) +DO_ZPZZ_PAIR_FP(sve2_fmaxp_zpzz_d, float64, , float64_max) + +DO_ZPZZ_PAIR_FP(sve2_fminp_zpzz_h, float16, H1_2, float16_min) +DO_ZPZZ_PAIR_FP(sve2_fminp_zpzz_s, float32, H1_4, float32_min) +DO_ZPZZ_PAIR_FP(sve2_fminp_zpzz_d, float64, , float64_min) + +#undef DO_ZPZZ_PAIR_FP + /* Three-operand expander, controlled by a predicate, in which the * third operand is "wide". That is, for D = N op M, the same 64-bit * value of M is used with all of the narrower values of N. diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index b8ced82e4e..1670bcadb1 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6697,3 +6697,28 @@ static bool trans_SQXTUNT(DisasContext *s, arg_rri_esz *a) }; return do_sve2_narrow_extract(s, a, ops); } + +static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, + gen_helper_gvec_4_ptr *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzz_fp(s, a, fn); +} + +#define DO_SVE2_ZPZZ_FP(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ +{ \ + static gen_helper_gvec_4_ptr * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_zpzz_h, \ + gen_helper_sve2_##name##_zpzz_s, gen_helper_sve2_##name##_zpzz_d \ + }; \ + return do_sve2_zpzz_fp(s, a, fns[a->esz]); \ +} + +DO_SVE2_ZPZZ_FP(FADDP, faddp) +DO_SVE2_ZPZZ_FP(FMAXNMP, fmaxnmp) +DO_SVE2_ZPZZ_FP(FMINNMP, fminnmp) +DO_SVE2_ZPZZ_FP(FMAXP, fmaxp) +DO_SVE2_ZPZZ_FP(FMINP, fminp) From patchwork Fri Apr 16 21:01:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422616 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp729621jaf; Fri, 16 Apr 2021 14:23:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwZnFy0gd8xXKK70rnIdH4T8kyUqfl8hjfy41O0k05XbMGQpwSYdaxAlclt1So/TYtE96Ha X-Received: by 2002:a92:7f0c:: with SMTP id a12mr8405020ild.54.1618608213424; Fri, 16 Apr 2021 14:23:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608213; cv=none; d=google.com; s=arc-20160816; b=r4URTvIDHHgnRW0vED/lNIRgrlEl0UnQdoB/yUNaxWgNYTsUqzt+b8frCROltAiauR VjZbCS473lVK6oOxrjEipmeeMSRIE1q6p30JqnuG8B1E9ymKYVY00CA5KjaMVvJo9L3q 0k6qkJoRGbSHIMvoOi7Ar0kqMMK8RRtvxepSFr4r2QUHkqxsJHld9WKgKDlMyuVAqHR8 0ce9awGbWB1//GXpugTxPRwFnSLJC0tPOPtsmzJc5Ho8mEI75NcmflgK3sW82Qmp8SE9 Tip4eKMuDWjMAsh+Rwr4BKtBnhO69AeFhoDvfSloNneeufWeww6ffTBRpm11Ye8Y9kWm hx6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=WwOjRgyqW+QscwVMflkQnp8uVzX1HAgfjZDST1BGuos=; b=SNElGOi/V+5uSfmtlsYnZ4Feh3F2vBHf5oCk9hR82LK+D9yQr/l4gdrkjuWC9EQrUi 2e51obOBm8Ei1/UAVCcf7Rtz+brOfjMq/nnmFgvutyQ6+p/jYC2f0UTMeWe3VDMBupuI Sq89yPufSKegO/Dn44IIKPzKFINvGgkkhrRhh3LGXef4D2KkI+HJpq+IZ3DcO/NECkxE ZW/yckNIkjHVIlj9oafIfKoAS8Slsj6Kb/kZBhJEL5k+E6ptk5RIRcfmWslWKUgOH4hJ POpoRuwrKWeUE17DfLB6oC2j2lyWNXnK5Kd4OVDmSogjAu6ebSClO/cidFgIK9Ule56x kXnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="p3PYj5w/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b11si3536814ilj.146.2021.04.16.14.23.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:23:33 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="p3PYj5w/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51774 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVw4-0007Ia-Sn for patch@linaro.org; Fri, 16 Apr 2021 17:23:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44064) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcR-0004bP-O0 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:15 -0400 Received: from mail-pg1-x52b.google.com ([2607:f8b0:4864:20::52b]:35345) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcL-0001Eq-Qg for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:15 -0400 Received: by mail-pg1-x52b.google.com with SMTP id q10so20002591pgj.2 for ; Fri, 16 Apr 2021 14:03:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WwOjRgyqW+QscwVMflkQnp8uVzX1HAgfjZDST1BGuos=; b=p3PYj5w/yRtiVSzrEoyRkiKOX24fwi1kNUYOvsk0VZhkgob+qkX24xGmGUIdiqVMTX kH9M8BeNew0IR/QbWJqyvFub6GVy3Uq6FH1BWEEgkHQ0PmKvSoGK9CfLYDlhXlY+vXsY UBWEg3T9z5mOblLENgkhI1SEYMfshfAi3DcPgSqApGj5E+KUsO9yTt0+q4ptAC5uorXm /7X5PE7LCVPB9WK0LFkgiJU/7C0lg3pGsQY9uy0u/XUepoMqLtZlPQx/YMSmrdoF9Afc uXp1C4zuhY08ks5MODiDJ4OehenZjGAazT26TSZnlqqRPqdIZ7xj5eej40ubVIctRyyY PgtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WwOjRgyqW+QscwVMflkQnp8uVzX1HAgfjZDST1BGuos=; b=ufX0n2EvvnyO1AN53VMPPvR5ZXtvo0ON1b7kt5vuBRiV01Q91nxY+/Qw5G8xCzbN56 0Muz7B5QgdeHdtVHPrK/luwZM8KaXXyFo3Qy8kzSk1gf8OHhtV3mksrXtrtiHKrLIDWR 1VTNONfhtvEVWWlYheXfg3z+S2vLYfaQu4EGIscLsS3lWAKJKfdhHvhbWeed7mcNtbh9 NIqW+/93bimP8pLD2/cBbRfNTpBOin6z8lXUmeGFRey0vYP767ygglvX2BIyzAHmbAkh MFANjvR371BF/FD57NRui2woprwqn+oBUbJJekACNK7jK2S/66nM5CmVtxpvq64Q1owT xIDw== X-Gm-Message-State: AOAM5335PYloUpgavD1QC+X+aKFtGf7vGh2BCkBkenNXi6NDNe9nTfGf Y/ohjobRvWdqsHE2Vk55t/Tlxj+p4+qQWw== X-Received: by 2002:a63:ef4e:: with SMTP id c14mr844141pgk.166.1618606988243; Fri, 16 Apr 2021 14:03:08 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:07 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 26/81] target/arm: Implement SVE2 SHRN, RSHRN Date: Fri, 16 Apr 2021 14:01:45 -0700 Message-Id: <20210416210240.1591291-27-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52b; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix typo in gen_shrnb_vec (laurent desnogues) v3: Replace DO_RSHR with an inline function --- target/arm/helper-sve.h | 16 ++++ target/arm/sve.decode | 8 ++ target/arm/sve_helper.c | 54 ++++++++++++- target/arm/translate-sve.c | 160 +++++++++++++++++++++++++++++++++++++ 4 files changed, 236 insertions(+), 2 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index a033b5f6b2..2b2ebea631 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2444,6 +2444,22 @@ DEF_HELPER_FLAGS_3(sve2_sqxtunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_shrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_rshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_rshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 9c75ac94c0..169486ecb2 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1285,6 +1285,14 @@ UQXTNT 01000101 .. 1 ..... 010 011 ..... ..... @rd_rn_tszimm_shl SQXTUNB 01000101 .. 1 ..... 010 100 ..... ..... @rd_rn_tszimm_shl SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl +## SVE2 bitwise shift right narrow + +# Bit 23 == 0 is handled by esz > 0 in the translator. +SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr +SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr +RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr +RSHRNT 01000101 .. 1 ..... 00 0111 ..... ..... @rd_rn_tszimm_shr + ## SVE2 floating-point pairwise operations FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 11f228144c..3f864da3ab 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1863,6 +1863,17 @@ void HELPER(NAME)(void *vd, void *vn, void *vg, uint32_t desc) \ when N is negative, add 2**M-1. */ #define DO_ASRD(N, M) ((N + (N < 0 ? ((__typeof(N))1 << M) - 1 : 0)) >> M) +static inline uint64_t do_urshr(uint64_t x, unsigned sh) +{ + if (likely(sh < 64)) { + return (x >> sh) + ((x >> (sh - 1)) & 1); + } else if (sh == 64) { + return x >> 63; + } else { + return 0; + } +} + DO_ZPZI(sve_asr_zpzi_b, int8_t, H1, DO_SHR) DO_ZPZI(sve_asr_zpzi_h, int16_t, H1_2, DO_SHR) DO_ZPZI(sve_asr_zpzi_s, int32_t, H1_4, DO_SHR) @@ -1883,12 +1894,51 @@ DO_ZPZI(sve_asrd_h, int16_t, H1_2, DO_ASRD) DO_ZPZI(sve_asrd_s, int32_t, H1_4, DO_ASRD) DO_ZPZI_D(sve_asrd_d, int64_t, DO_ASRD) -#undef DO_SHR -#undef DO_SHL #undef DO_ASRD #undef DO_ZPZI #undef DO_ZPZI_D +#define DO_SHRNB(NAME, TYPEW, TYPEN, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int shift = simd_data(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + i); \ + *(TYPEW *)(vd + i) = (TYPEN)OP(nn, shift); \ + } \ +} + +#define DO_SHRNT(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int shift = simd_data(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + *(TYPEN *)(vd + HN(i + sizeof(TYPEN))) = OP(nn, shift); \ + } \ +} + +DO_SHRNB(sve2_shrnb_h, uint16_t, uint8_t, DO_SHR) +DO_SHRNB(sve2_shrnb_s, uint32_t, uint16_t, DO_SHR) +DO_SHRNB(sve2_shrnb_d, uint64_t, uint32_t, DO_SHR) + +DO_SHRNT(sve2_shrnt_h, uint16_t, uint8_t, H1_2, H1, DO_SHR) +DO_SHRNT(sve2_shrnt_s, uint32_t, uint16_t, H1_4, H1_2, DO_SHR) +DO_SHRNT(sve2_shrnt_d, uint64_t, uint32_t, , H1_4, DO_SHR) + +DO_SHRNB(sve2_rshrnb_h, uint16_t, uint8_t, do_urshr) +DO_SHRNB(sve2_rshrnb_s, uint32_t, uint16_t, do_urshr) +DO_SHRNB(sve2_rshrnb_d, uint64_t, uint32_t, do_urshr) + +DO_SHRNT(sve2_rshrnt_h, uint16_t, uint8_t, H1_2, H1, do_urshr) +DO_SHRNT(sve2_rshrnt_s, uint32_t, uint16_t, H1_4, H1_2, do_urshr) +DO_SHRNT(sve2_rshrnt_d, uint64_t, uint32_t, , H1_4, do_urshr) + +#undef DO_SHRNB +#undef DO_SHRNT + /* Fully general four-operand expander, controlled by a predicate. */ #define DO_ZPZZZ(NAME, TYPE, H, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 1670bcadb1..c1a081acaa 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6698,6 +6698,166 @@ static bool trans_SQXTUNT(DisasContext *s, arg_rri_esz *a) return do_sve2_narrow_extract(s, a, ops); } +static bool do_sve2_shr_narrow(DisasContext *s, arg_rri_esz *a, + const GVecGen2i ops[3]) +{ + if (a->esz < 0 || a->esz > MO_32 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + assert(a->imm > 0 && a->imm <= (8 << a->esz)); + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2i(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vsz, vsz, a->imm, &ops[a->esz]); + } + return true; +} + +static void gen_shrnb_i64(unsigned vece, TCGv_i64 d, TCGv_i64 n, int shr) +{ + int halfbits = 4 << vece; + uint64_t mask = dup_const(vece, MAKE_64BIT_MASK(0, halfbits)); + + tcg_gen_shri_i64(d, n, shr); + tcg_gen_andi_i64(d, d, mask); +} + +static void gen_shrnb16_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnb_i64(MO_16, d, n, shr); +} + +static void gen_shrnb32_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnb_i64(MO_32, d, n, shr); +} + +static void gen_shrnb64_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnb_i64(MO_64, d, n, shr); +} + +static void gen_shrnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + uint64_t mask = MAKE_64BIT_MASK(0, halfbits); + + tcg_gen_shri_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_SHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { INDEX_op_shri_vec, 0 }; + static const GVecGen2i ops[3] = { + { .fni8 = gen_shrnb16_i64, + .fniv = gen_shrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_shrnb_h, + .vece = MO_16 }, + { .fni8 = gen_shrnb32_i64, + .fniv = gen_shrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_shrnb_s, + .vece = MO_32 }, + { .fni8 = gen_shrnb64_i64, + .fniv = gen_shrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_shrnb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_shrnt_i64(unsigned vece, TCGv_i64 d, TCGv_i64 n, int shr) +{ + int halfbits = 4 << vece; + uint64_t mask = dup_const(vece, MAKE_64BIT_MASK(0, halfbits)); + + tcg_gen_shli_i64(n, n, halfbits - shr); + tcg_gen_andi_i64(n, n, ~mask); + tcg_gen_andi_i64(d, d, mask); + tcg_gen_or_i64(d, d, n); +} + +static void gen_shrnt16_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnt_i64(MO_16, d, n, shr); +} + +static void gen_shrnt32_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnt_i64(MO_32, d, n, shr); +} + +static void gen_shrnt64_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + tcg_gen_shri_i64(n, n, shr); + tcg_gen_deposit_i64(d, d, n, 32, 32); +} + +static void gen_shrnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + uint64_t mask = MAKE_64BIT_MASK(0, halfbits); + + tcg_gen_shli_vec(vece, n, n, halfbits - shr); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { INDEX_op_shli_vec, 0 }; + static const GVecGen2i ops[3] = { + { .fni8 = gen_shrnt16_i64, + .fniv = gen_shrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_shrnt_h, + .vece = MO_16 }, + { .fni8 = gen_shrnt32_i64, + .fniv = gen_shrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_shrnt_s, + .vece = MO_32 }, + { .fni8 = gen_shrnt64_i64, + .fniv = gen_shrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_shrnt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_RSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_rshrnb_h }, + { .fno = gen_helper_sve2_rshrnb_s }, + { .fno = gen_helper_sve2_rshrnb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_RSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_rshrnt_h }, + { .fno = gen_helper_sve2_rshrnt_s }, + { .fno = gen_helper_sve2_rshrnt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Fri Apr 16 21:01:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422625 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp733944jaf; Fri, 16 Apr 2021 14:31:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx0pY/28m/HoN0C047VFxXx+hv84iLAzExoL6+ChreU6qHL07I76NumqYAy6ikdkbFVGSmb X-Received: by 2002:a05:6830:2415:: with SMTP id j21mr5282885ots.207.1618608702615; Fri, 16 Apr 2021 14:31:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608702; cv=none; d=google.com; s=arc-20160816; b=FEpioQqNkd4PfyeSXOciOjj0txFeROE158d8rkGW43dlyYFNcoQcedBIT41x7oktrL lV3ezwa/kmq++Cmp/OGyqZeyUeAelT7Nz25Iyaq7xt1RLzJUb9377mLNPP3AZE8W8Txz uxSZAXgNVMAOHJijazSS0cdOb4tAV31TtBVfxZRpg5FvyJffJt+0t/0hxTEiZcLOW95N mkJJ4n6HS8UmUuxCQSKo5DismSLVPMn8q2l29RD6UxVBKS4hLWwMkz3/Fb/gxBlZLySk /sn+X5bmoUrVNXxKawF+VzcUrGWInMvtOEvTb2CCRZAHqfROr9oX/sHewKeoBPF12L73 4muQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=3st5GDCoYANtSqf9tcWs4eqMv4/zg0MwHali4f+3cK4=; b=LKc4sfLZX5o6klCXD6K775ghMXX8t8snVmyepysAEnF2PTm8oj+nUt4W25oGRcWTJp LauR7S4CC0eZH5s8IO7XfmoFDp/jfMFmWcgHSl07l1p4dsHdtEmugSXlnlbXDZCVqTds Or1yIZDLJXkU2p7ObQwc8cxk+RPF1h6j5hKniXIjLrR3kGg9+ZJeh79m3Dw6xUK2UbdH yBEHMA2JheR7be/XE0FuCwgAsZsiQhv+DCgpEz0sSMBbIHAN0hgC233VUO3+3Hff1Fwc z4GUmDiqBWulZSc+I2Y7H/Mj32rC+EarnTkK7MzVMn0KfJiwUUXb0qIAB9gR40R2RFSn fa8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=V47HKQzs; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x13si6705162oto.247.2021.04.16.14.31.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:31:42 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=V47HKQzs; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42324 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW3x-0006Zc-VX for patch@linaro.org; Fri, 16 Apr 2021 17:31:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44118) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcT-0004gV-G3 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:17 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]:34726) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcM-0001Fb-Eo for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:17 -0400 Received: by mail-pj1-x1033.google.com with SMTP id em21-20020a17090b0155b029014e204a81e6so6950344pjb.1 for ; Fri, 16 Apr 2021 14:03:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3st5GDCoYANtSqf9tcWs4eqMv4/zg0MwHali4f+3cK4=; b=V47HKQzs8Km0jNhcWVjCf5tTda+VDI8AnJHKSSGm1kbFPHHqjNrlca07opQjLQXrlR HDNC66122Hb9e2dJgDbUdtX0aNu4FL3h6ZPOuS7PMztFt2ugP2fjivDc/dpmDEkihJM7 YGRnYXjSTBceMBHsXWvINRrBlzrFCp2qz/EwPfolUvBkAcP4V1WxrW8JuVYAofdCKdbp zWIRSZV11X7ZDN3Ix1BpBebI53h8KH1Ws3Nu0IGxrqzmat9tJP5f5dMxjcyjyaHKg8Qb USMvUyTAlngwaylk/OUqPbxi2MbLBRQLJ6vq1Gk0xYOskwyfwWEv7KAXeOu+7Fq9mRPC 35vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3st5GDCoYANtSqf9tcWs4eqMv4/zg0MwHali4f+3cK4=; b=sFLHMRJON4HMp+HDRaLfqzE27R7FbkRCqW1Jz19vkwCo+563F7xJqtMcgtBECF055U ZDu/L8Tzgkc/V9NihI1FIcbi01we0o0oJT1YTHsL7PaVdF7Ecv5xm74sZTE5mufOLVus 2ej00pQDmSvvKLf57BIb/iFgjwM3NvyD0KM65w4WaFXcbBgvq3fWWEuYz3k5gwaFBTw9 fazzA8lPyv+TwKWju88nKWOqWo64UMhgvIb2KminQcQmFR0c4WmRZE/7PDeagzo/3wqW SjsakF9smBjfCs7kwSjJmV7631o+Edjnme5bw7dwKxb1lit0KifbiAffGquy2cBstRNM LAPw== X-Gm-Message-State: AOAM531U9P0iTp4NCM4FGdXdfeUpw/3A8RvNBUbONnkrrekbFDiCJ0rS Yob9VKRmBSUO7lpHZ/mfSVK+VyaX0/KvBw== X-Received: by 2002:a17:90b:344a:: with SMTP id lj10mr11989461pjb.101.1618606989083; Fri, 16 Apr 2021 14:03:09 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:08 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 27/81] target/arm: Implement SVE2 SQSHRUN, SQRSHRUN Date: Fri, 16 Apr 2021 14:01:46 -0700 Message-Id: <20210416210240.1591291-28-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1033.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 16 +++++++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 35 ++++++++++++++ target/arm/translate-sve.c | 98 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 153 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 2b2ebea631..2e80d9d27b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2460,6 +2460,22 @@ DEF_HELPER_FLAGS_3(sve2_rshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_rshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_rshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrunb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 169486ecb2..18faa900ca 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1288,6 +1288,10 @@ SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl ## SVE2 bitwise shift right narrow # Bit 23 == 0 is handled by esz > 0 in the translator. +SQSHRUNB 01000101 .. 1 ..... 00 0000 ..... ..... @rd_rn_tszimm_shr +SQSHRUNT 01000101 .. 1 ..... 00 0001 ..... ..... @rd_rn_tszimm_shr +SQRSHRUNB 01000101 .. 1 ..... 00 0010 ..... ..... @rd_rn_tszimm_shr +SQRSHRUNT 01000101 .. 1 ..... 00 0011 ..... ..... @rd_rn_tszimm_shr SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 3f864da3ab..d6b6293ab0 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1874,6 +1874,16 @@ static inline uint64_t do_urshr(uint64_t x, unsigned sh) } } +static inline int64_t do_srshr(int64_t x, unsigned sh) +{ + if (likely(sh < 64)) { + return (x >> sh) + ((x >> (sh - 1)) & 1); + } else { + /* Rounding the sign bit always produces 0. */ + return 0; + } +} + DO_ZPZI(sve_asr_zpzi_b, int8_t, H1, DO_SHR) DO_ZPZI(sve_asr_zpzi_h, int16_t, H1_2, DO_SHR) DO_ZPZI(sve_asr_zpzi_s, int32_t, H1_4, DO_SHR) @@ -1936,6 +1946,31 @@ DO_SHRNT(sve2_rshrnt_h, uint16_t, uint8_t, H1_2, H1, do_urshr) DO_SHRNT(sve2_rshrnt_s, uint32_t, uint16_t, H1_4, H1_2, do_urshr) DO_SHRNT(sve2_rshrnt_d, uint64_t, uint32_t, , H1_4, do_urshr) +#define DO_SQSHRUN_H(x, sh) do_sat_bhs((int64_t)(x) >> sh, 0, UINT8_MAX) +#define DO_SQSHRUN_S(x, sh) do_sat_bhs((int64_t)(x) >> sh, 0, UINT16_MAX) +#define DO_SQSHRUN_D(x, sh) \ + do_sat_bhs((int64_t)(x) >> (sh < 64 ? sh : 63), 0, UINT32_MAX) + +DO_SHRNB(sve2_sqshrunb_h, int16_t, uint8_t, DO_SQSHRUN_H) +DO_SHRNB(sve2_sqshrunb_s, int32_t, uint16_t, DO_SQSHRUN_S) +DO_SHRNB(sve2_sqshrunb_d, int64_t, uint32_t, DO_SQSHRUN_D) + +DO_SHRNT(sve2_sqshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQSHRUN_H) +DO_SHRNT(sve2_sqshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQSHRUN_S) +DO_SHRNT(sve2_sqshrunt_d, int64_t, uint32_t, , H1_4, DO_SQSHRUN_D) + +#define DO_SQRSHRUN_H(x, sh) do_sat_bhs(do_srshr(x, sh), 0, UINT8_MAX) +#define DO_SQRSHRUN_S(x, sh) do_sat_bhs(do_srshr(x, sh), 0, UINT16_MAX) +#define DO_SQRSHRUN_D(x, sh) do_sat_bhs(do_srshr(x, sh), 0, UINT32_MAX) + +DO_SHRNB(sve2_sqrshrunb_h, int16_t, uint8_t, DO_SQRSHRUN_H) +DO_SHRNB(sve2_sqrshrunb_s, int32_t, uint16_t, DO_SQRSHRUN_S) +DO_SHRNB(sve2_sqrshrunb_d, int64_t, uint32_t, DO_SQRSHRUN_D) + +DO_SHRNT(sve2_sqrshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRUN_H) +DO_SHRNT(sve2_sqrshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRUN_S) +DO_SHRNT(sve2_sqrshrunt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRUN_D) + #undef DO_SHRNB #undef DO_SHRNT diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c1a081acaa..5ff6b8ffb6 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6858,6 +6858,104 @@ static bool trans_RSHRNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static void gen_sqshrunb_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRUNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_sari_vec, INDEX_op_smax_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrunb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrunb_h, + .vece = MO_16 }, + { .fniv = gen_sqshrunb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrunb_s, + .vece = MO_32 }, + { .fniv = gen_sqshrunb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrunb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_sqshrunt_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRUNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shli_vec, INDEX_op_sari_vec, + INDEX_op_smax_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrunt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrunt_h, + .vece = MO_16 }, + { .fniv = gen_sqshrunt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrunt_s, + .vece = MO_32 }, + { .fniv = gen_sqshrunt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrunt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRUNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrunb_h }, + { .fno = gen_helper_sve2_sqrshrunb_s }, + { .fno = gen_helper_sve2_sqrshrunb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRUNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrunt_h }, + { .fno = gen_helper_sve2_sqrshrunt_s }, + { .fno = gen_helper_sve2_sqrshrunt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Fri Apr 16 21:01:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422622 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp731905jaf; Fri, 16 Apr 2021 14:28:04 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxpIroO+XCdf03cYrWFa03ibV8bNRj5ELWFYup43uYlhqNpx+y4ro3gXpxmRiZPmmoYOe4w X-Received: by 2002:a92:6a0c:: with SMTP id f12mr8791526ilc.117.1618608484641; Fri, 16 Apr 2021 14:28:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608484; cv=none; d=google.com; s=arc-20160816; b=lUyw/E5T8hLK/4oj+rE+M1O5j8sTfskxuhNGTXCNEPURG176pjEpK9udFU9f/FPHTs c3TXgl6e7hGu6WPYJdnCFPSE38vIzk/ysSx+eii6H+Yh3JbpM+W8xbKFrxbVa30FX1M8 D72j7XtcbpstZIERmbmPMZm10iYzGGpPz2NIVdu4lyZlRydhl8ejjh+OYyUB4yM5Nzy5 CxATlo/zdLIzN4T9rOsxi2Rs20fTrIYuPGNJSyeyQcTbm1SghwkAkfN8jBnxrHRZ+Dxd wtyg/d4KbJtk7xJWBEamXGd2WCt+TY7tu8FCjjCSPtPvUluiJy3tUBKKlsjvuuQjHqxU OPhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=TYIH46Yz5nuJnlEKWiLaTEG5vm1zqOPy6lL/fTv4hvo=; b=tq/Z9IUmG0mEmJnPZMPXGgTwI9pTJQifJ+p2XDjzPQHFKq198huy79kwOhbyCEE/p/ 2cy6JlJHR3LJ3qf0qqqnZVYr90gJHy9xPHubpPohBpOt1JN5LwIbxKVWiZFr/H6h3ZV9 cPMdg+SNXRZxvKSxWY6PcmYJ//arvBoSYbLwqo+LMmHck5fVc/lxIW/hHzvgQ0NaoPvv 1PtumUyHBMXg6kgkaTbQUWTGZ+rm8LniYC3CVbPtTXhX2iOTcaoTylbyKl9DdNBdtooh n87VxM8pbWnoLKK6WDm8zFAh9oeqjVJDYz3TJvnO4cEFHArZUebULZ+3nw76S9SO1NtR B7FQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=q050C6Dw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n14si8852610jaj.25.2021.04.16.14.28.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:28:04 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=q050C6Dw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33586 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW0R-00031P-TS for patch@linaro.org; Fri, 16 Apr 2021 17:28:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44084) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcS-0004e0-Jr for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:16 -0400 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]:41937) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcN-0001Fu-9F for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:16 -0400 Received: by mail-pj1-x102b.google.com with SMTP id z22-20020a17090a0156b029014d4056663fso15283821pje.0 for ; Fri, 16 Apr 2021 14:03:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TYIH46Yz5nuJnlEKWiLaTEG5vm1zqOPy6lL/fTv4hvo=; b=q050C6DwyRnv72p744+Uj84/FMfR+HMV20VjFDDmyScZ3TbWS/E9EqK5ksTKOgRjix ke0d5vDNGCdXJzVmbfLP7dzy4rVu+so7H7UzeJT0KgE5DWEH3EeMVrjzVOGeUbCvxt/D 5mu3q/9Emv8YQ0UO/pVd5Unp/GKkn3DVNbV52xtnVreDjFl9wDjq2ynf7DAY1t60zGGW xfU2FhdgrNec47/06hw8EMKl4JpnYjqyE9l7ReORobOHFG3nq3Yq74lRJNwVNeo/GYe1 YUcttvUOU2+UT5yGndnHnn7lAgs8lLEFH87tFMIKnLWav+C+XrePV+/D7fCGLxthIksR P2IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TYIH46Yz5nuJnlEKWiLaTEG5vm1zqOPy6lL/fTv4hvo=; b=pUER0Ky4hO5oXLhCwwQ15XUvL3BNlHMmFIgkwd/e2UtDox6DhC3pjaCTN9YwDNePWY MI1D1bh+Qm5z+dFxoG/AogbVgS9H+AO9t4tW4wlKNevhm2N8YNlsWsei+l/uv/bxAESl d2KsFSDRLufTpB2HoE1iPPRbUPYvfTyjnKEzwgETSkhfPjijnaVXdxBlYpChdPcbOKbT o533Z811z3SMZwG/1XePlN9qg+I2rMJrJQoc5tN664E9kFpBCQdvAj5wBfWMHrRGyQJp DU9tplX1/KB47MLDqluYwih+XI9pUknKuBzqV1hsr4Of9infW1E1OQ5Ro0ykGJPEM1YD hZgw== X-Gm-Message-State: AOAM533wC1hmEG71hL9pTxtho5v1NryHdY2FJE4PpVd832r01tEIxsRQ 6T+xdWwVgXIE6sQjNHJsvuwsXAsRsK8dXw== X-Received: by 2002:a17:90a:6b84:: with SMTP id w4mr11572266pjj.134.1618606990028; Fri, 16 Apr 2021 14:03:10 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:09 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 28/81] target/arm: Implement SVE2 UQSHRN, UQRSHRN Date: Fri, 16 Apr 2021 14:01:47 -0700 Message-Id: <20210416210240.1591291-29-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102b; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 16 +++++++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 24 ++++++++++ target/arm/translate-sve.c | 93 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 137 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 2e80d9d27b..ba6a24fc8b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2476,6 +2476,22 @@ DEF_HELPER_FLAGS_3(sve2_sqrshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqrshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 18faa900ca..13b5da0856 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1296,6 +1296,10 @@ SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr RSHRNT 01000101 .. 1 ..... 00 0111 ..... ..... @rd_rn_tszimm_shr +UQSHRNB 01000101 .. 1 ..... 00 1100 ..... ..... @rd_rn_tszimm_shr +UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr +UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr +UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr ## SVE2 floating-point pairwise operations diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index d6b6293ab0..175f3de08f 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1971,6 +1971,30 @@ DO_SHRNT(sve2_sqrshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRUN_H) DO_SHRNT(sve2_sqrshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRUN_S) DO_SHRNT(sve2_sqrshrunt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRUN_D) +#define DO_UQSHRN_H(x, sh) MIN(x >> sh, UINT8_MAX) +#define DO_UQSHRN_S(x, sh) MIN(x >> sh, UINT16_MAX) +#define DO_UQSHRN_D(x, sh) MIN(x >> sh, UINT32_MAX) + +DO_SHRNB(sve2_uqshrnb_h, uint16_t, uint8_t, DO_UQSHRN_H) +DO_SHRNB(sve2_uqshrnb_s, uint32_t, uint16_t, DO_UQSHRN_S) +DO_SHRNB(sve2_uqshrnb_d, uint64_t, uint32_t, DO_UQSHRN_D) + +DO_SHRNT(sve2_uqshrnt_h, uint16_t, uint8_t, H1_2, H1, DO_UQSHRN_H) +DO_SHRNT(sve2_uqshrnt_s, uint32_t, uint16_t, H1_4, H1_2, DO_UQSHRN_S) +DO_SHRNT(sve2_uqshrnt_d, uint64_t, uint32_t, , H1_4, DO_UQSHRN_D) + +#define DO_UQRSHRN_H(x, sh) MIN(do_urshr(x, sh), UINT8_MAX) +#define DO_UQRSHRN_S(x, sh) MIN(do_urshr(x, sh), UINT16_MAX) +#define DO_UQRSHRN_D(x, sh) MIN(do_urshr(x, sh), UINT32_MAX) + +DO_SHRNB(sve2_uqrshrnb_h, uint16_t, uint8_t, DO_UQRSHRN_H) +DO_SHRNB(sve2_uqrshrnb_s, uint32_t, uint16_t, DO_UQRSHRN_S) +DO_SHRNB(sve2_uqrshrnb_d, uint64_t, uint32_t, DO_UQRSHRN_D) + +DO_SHRNT(sve2_uqrshrnt_h, uint16_t, uint8_t, H1_2, H1, DO_UQRSHRN_H) +DO_SHRNT(sve2_uqrshrnt_s, uint32_t, uint16_t, H1_4, H1_2, DO_UQRSHRN_S) +DO_SHRNT(sve2_uqrshrnt_d, uint64_t, uint32_t, , H1_4, DO_UQRSHRN_D) + #undef DO_SHRNB #undef DO_SHRNT diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 5ff6b8ffb6..12733d9b4f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6956,6 +6956,99 @@ static bool trans_SQRSHRUNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static void gen_uqshrnb_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_shri_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_UQSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shri_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_uqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_uqshrnb_h, + .vece = MO_16 }, + { .fniv = gen_uqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_uqshrnb_s, + .vece = MO_32 }, + { .fniv = gen_uqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_uqshrnb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_uqshrnt_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_shri_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_UQSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shli_vec, INDEX_op_shri_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_uqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_uqshrnt_h, + .vece = MO_16 }, + { .fniv = gen_uqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_uqshrnt_s, + .vece = MO_32 }, + { .fniv = gen_uqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_uqshrnt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_UQRSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_uqrshrnb_h }, + { .fno = gen_helper_sve2_uqrshrnb_s }, + { .fno = gen_helper_sve2_uqrshrnb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_uqrshrnt_h }, + { .fno = gen_helper_sve2_uqrshrnt_s }, + { .fno = gen_helper_sve2_uqrshrnt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Fri Apr 16 21:01:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422611 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp727102jaf; Fri, 16 Apr 2021 14:19:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwY4B82w/ILOKlPoFNRv5JdLISuFu6feMCjFnZ3y8CSN/LHigHdg+lMviiqXgP6dtj1b8xc X-Received: by 2002:a6b:4e01:: with SMTP id c1mr5096016iob.107.1618607954012; Fri, 16 Apr 2021 14:19:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618607954; cv=none; d=google.com; s=arc-20160816; b=AfmS8ZhsK2QkdGnGY2NbUeudyfTHEfRj4XX6lR+9m66gDFrj34WIYpFgl6NAujHvy5 3wvhZ9gIQYqZjqCI9qiN3y+iHdr2pFPhoxyCkis+4EsNydtWuO7fY+YXwsGSJfIGtSIT kI6SCKJ1XbTV69FS+JK9Sc/6rgR1TI5KXGfbzXJ2jqKOuCXgGcJ79rtnXV4jwf2UvqU8 QE1XoY2fD7i+x5JJ6HDRpfnqmdWU6swK7R1sNKLE7GoFzGiQAmALtF1NTBKi1GBauvOa XChlop6vkUwT8Rd7CoW/ez56EnzWZa6WCNQbASBoN70pUj7Vkkd1gSeAyn4aHw9LzDaT 5IKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=feb7AjyBXXqqzvqsPRxefar5HpBH8rI+phXQIUCl9Ao=; b=LvMSgPkhhVTj/G7w+DxDBBCH5bWR11NgqSTYyw8XIoM9Z1XHMryNKYekdHlAIMLUHh 4ckRbR9tdM+d8T16dopTwoqa2kkHZ7J43yjuqdeE5fzyWpoLJ873p6iQk+7IyDBhMqm5 ZPH8CQoO17JRMy0taCBEi2dBr1d1Ro+zIAYu9GX6MKsQ/A2i2sxUBGJrnIwf0sSiBhsp 7QhoCOBkKpXuf0UgdXN9Cq4RIXHiwosnx6cTdRQ60oGnsBF+8/jCIUXcb2REzXwuuD4U oFHeFFFcuLUNIwW6tQ1RXa92tLKcu02tgcZ7A2P4vpVPyn2iCzX46pUWopDWvWEy3c+6 dhbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NQNd7eXz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t13si2477342ilu.145.2021.04.16.14.19.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:19:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NQNd7eXz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60796 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVrt-0007cy-Am for patch@linaro.org; Fri, 16 Apr 2021 17:19:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44126) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcT-0004hM-U1 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:17 -0400 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]:34737) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcO-0001HJ-CR for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:17 -0400 Received: by mail-pl1-x630.google.com with SMTP id t22so14309599ply.1 for ; Fri, 16 Apr 2021 14:03:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=feb7AjyBXXqqzvqsPRxefar5HpBH8rI+phXQIUCl9Ao=; b=NQNd7eXzmQVt5EOEoPB5tRaIVLZZfzUjWBwdSf19+AT07omOIiHeUWBMh/iTGgSNaE 598qqqGgx4dhdQd4KXNZBOmiNeFLM3vIAmYUIX9UQOkmcEQDJKZSoJjP2A7Z0Q/rbGx4 beFbTHPEXoLDTgeVNoQJBDx+3XFLMv/61QMQIHMYyDZB7p2WjWKBSQpQceicN2SA9V2/ T1GH3g6MkAyDFkRnaxJWP6JSPGlMHMVXnq9tHrm+fg6sI88Ztrh8Xv/RaHiQJdFTHK6S AQXdYSJbB9LF+l8oMY5flJ5L7gq6I9O3opaYaq5/B+A5HbjpL2WgJwW6WTIfcvtKAvuv 3/Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=feb7AjyBXXqqzvqsPRxefar5HpBH8rI+phXQIUCl9Ao=; b=qJB3+31LrJF/IKBCioAY/oiXeWjOr0hpWnzG9nPyBpqOTEjPVVTU+R7gT112sJVKB2 qCCi+GjtdKHhSesRpEeONg+RoqYssRSf4Hd9tD7ItwP+bWu+lijE2SryJlkkPwKwh/6X 5Ar12YjIFID29FUsmVPHqyUAOTi/bkfjufVhrt8FrC8JKg48ZyOjUXhOKTWn47uBfbHn pfXmCO3CeGdeh/dKzhsaISetTCCQl3mY4gweUSdG0J4l1e450kp2bA2+U1oDvlENm0g6 oMMxy0FzHqmsl6ZXGf8gD3XVLMrD67T2I19a4YyX+B43iK/nhh7W5Y3v6RjUg4pPamtW ZhhQ== X-Gm-Message-State: AOAM531OLgo+rLCBZBf1qE+0i2QGHK+v0cfzuyOxmPMXZJrTBsIfBFTa ng8s8uPcKDDF2tvdR0MLaVESH7Qcei89Xw== X-Received: by 2002:a17:90a:420d:: with SMTP id o13mr11782384pjg.61.1618606991103; Fri, 16 Apr 2021 14:03:11 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:10 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 29/81] target/arm: Implement SVE2 SQSHRN, SQRSHRN Date: Fri, 16 Apr 2021 14:01:48 -0700 Message-Id: <20210416210240.1591291-30-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This completes the section "SVE2 bitwise shift right narrow". Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 16 ++++++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 24 +++++++++ target/arm/translate-sve.c | 105 +++++++++++++++++++++++++++++++++++++ 4 files changed, 149 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index ba6a24fc8b..1c7fe8e417 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2476,6 +2476,22 @@ DEF_HELPER_FLAGS_3(sve2_sqrshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(sve2_uqshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 13b5da0856..0674464695 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1296,6 +1296,10 @@ SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr RSHRNT 01000101 .. 1 ..... 00 0111 ..... ..... @rd_rn_tszimm_shr +SQSHRNB 01000101 .. 1 ..... 00 1000 ..... ..... @rd_rn_tszimm_shr +SQSHRNT 01000101 .. 1 ..... 00 1001 ..... ..... @rd_rn_tszimm_shr +SQRSHRNB 01000101 .. 1 ..... 00 1010 ..... ..... @rd_rn_tszimm_shr +SQRSHRNT 01000101 .. 1 ..... 00 1011 ..... ..... @rd_rn_tszimm_shr UQSHRNB 01000101 .. 1 ..... 00 1100 ..... ..... @rd_rn_tszimm_shr UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 175f3de08f..02e87c535d 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1971,6 +1971,30 @@ DO_SHRNT(sve2_sqrshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRUN_H) DO_SHRNT(sve2_sqrshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRUN_S) DO_SHRNT(sve2_sqrshrunt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRUN_D) +#define DO_SQSHRN_H(x, sh) do_sat_bhs(x >> sh, INT8_MIN, INT8_MAX) +#define DO_SQSHRN_S(x, sh) do_sat_bhs(x >> sh, INT16_MIN, INT16_MAX) +#define DO_SQSHRN_D(x, sh) do_sat_bhs(x >> sh, INT32_MIN, INT32_MAX) + +DO_SHRNB(sve2_sqshrnb_h, int16_t, uint8_t, DO_SQSHRN_H) +DO_SHRNB(sve2_sqshrnb_s, int32_t, uint16_t, DO_SQSHRN_S) +DO_SHRNB(sve2_sqshrnb_d, int64_t, uint32_t, DO_SQSHRN_D) + +DO_SHRNT(sve2_sqshrnt_h, int16_t, uint8_t, H1_2, H1, DO_SQSHRN_H) +DO_SHRNT(sve2_sqshrnt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQSHRN_S) +DO_SHRNT(sve2_sqshrnt_d, int64_t, uint32_t, , H1_4, DO_SQSHRN_D) + +#define DO_SQRSHRN_H(x, sh) do_sat_bhs(do_srshr(x, sh), INT8_MIN, INT8_MAX) +#define DO_SQRSHRN_S(x, sh) do_sat_bhs(do_srshr(x, sh), INT16_MIN, INT16_MAX) +#define DO_SQRSHRN_D(x, sh) do_sat_bhs(do_srshr(x, sh), INT32_MIN, INT32_MAX) + +DO_SHRNB(sve2_sqrshrnb_h, int16_t, uint8_t, DO_SQRSHRN_H) +DO_SHRNB(sve2_sqrshrnb_s, int32_t, uint16_t, DO_SQRSHRN_S) +DO_SHRNB(sve2_sqrshrnb_d, int64_t, uint32_t, DO_SQRSHRN_D) + +DO_SHRNT(sve2_sqrshrnt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRN_H) +DO_SHRNT(sve2_sqrshrnt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRN_S) +DO_SHRNT(sve2_sqrshrnt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRN_D) + #define DO_UQSHRN_H(x, sh) MIN(x >> sh, UINT8_MAX) #define DO_UQSHRN_S(x, sh) MIN(x >> sh, UINT16_MAX) #define DO_UQSHRN_D(x, sh) MIN(x >> sh, UINT32_MAX) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 12733d9b4f..218f1ca5ce 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6956,6 +6956,111 @@ static bool trans_SQRSHRUNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static void gen_sqshrnb_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = MAKE_64BIT_MASK(0, halfbits - 1); + int64_t min = -max - 1; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_sari_vec, INDEX_op_smax_vec, INDEX_op_smin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrnb_h, + .vece = MO_16 }, + { .fniv = gen_sqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrnb_s, + .vece = MO_32 }, + { .fniv = gen_sqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrnb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_sqshrnt_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = MAKE_64BIT_MASK(0, halfbits - 1); + int64_t min = -max - 1; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shli_vec, INDEX_op_sari_vec, + INDEX_op_smax_vec, INDEX_op_smin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrnt_h, + .vece = MO_16 }, + { .fniv = gen_sqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrnt_s, + .vece = MO_32 }, + { .fniv = gen_sqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrnt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrnb_h }, + { .fno = gen_helper_sve2_sqrshrnb_s }, + { .fno = gen_helper_sve2_sqrshrnb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrnt_h }, + { .fno = gen_helper_sve2_sqrshrnt_s }, + { .fno = gen_helper_sve2_sqrshrnt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static void gen_uqshrnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr) { From patchwork Fri Apr 16 21:01:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422614 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp728793jaf; Fri, 16 Apr 2021 14:22:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz4gzVufBss84YsUKSKvgcXLfX5xykh4EcX49rcybW9pzq6ceINzmnNbkFnJU6Xh3a0lqO/ X-Received: by 2002:a92:d2cb:: with SMTP id w11mr8134243ilg.225.1618608120270; Fri, 16 Apr 2021 14:22:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608120; cv=none; d=google.com; s=arc-20160816; b=Z56/3j4eDhD3y8hJ8SoCXSaShNS4NPConjkCs1Q/LYMc3sYxzhaMmDmLUpY9Tko7sI 6OEqR/5JOMmSA/jpnunn0KxDWmjclfAiWgQDenPOZx1GyaOlaB7QHmCLmEDnMUP/Xbrs 4xQZXur0Pqhz+QDrEMm4hlS4edR/g4hFYXWTGNt+/uM1gxj5xnVX/7PsqoGccdqgZc1j BVVL6A3WA3OVYb6bYpLX7ZS9Vw3bXS7x4a8/ph3ZOKafyHBr83snomUhzM5l3D39jDe/ gxowff/RX5wvJ4NpmGXI/l+0cINQEAwwbYZbk3gaJBfRj5yripRgOyOveKzsjG1pu7bn 1A4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=4NnADjclEJcrO+T+qGPsu3IX/25Q5Wz2cI8VcHKA8p0=; b=vbjZYaOY74IkGdZ0oBOXgmcePeqres8gMjOVw6+1WNaDIgQTVe7sdSU/CMFqjokriU oP0n5fuVh57w1m8lr6VFZoBmsgB9y398SLntR74L2ZpI1Pf0GoFdBwXTR3NCLee8UQ3F 8fkv9pLbmfYVz8VETCjMS5mVrKgFpucafPtXT+xdUGCw+nhReNW8nWLjTBK2IIWVHRIC V3BSdnV/HAKLBpKBJg8FwRD4a1PBl7+1gzM3/DpQNdwwEBt7KB80A6NwJGGzofzlh/l1 jbBl3QyweofNEe6AWxuTCQIiuZ1ReRftBPkc9AFUdyOs70eb7DFnK8noZnDJsRcjnr4G Ytlw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=P3CInz08; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i24si7482713jav.54.2021.04.16.14.22.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:22:00 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=P3CInz08; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42560 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVuZ-0003Wu-Mj for patch@linaro.org; Fri, 16 Apr 2021 17:21:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44228) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcX-0004qG-F4 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:21 -0400 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]:33738) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcP-0001Hg-He for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:21 -0400 Received: by mail-pj1-x1034.google.com with SMTP id kb13-20020a17090ae7cdb02901503d67f0beso3639629pjb.0 for ; Fri, 16 Apr 2021 14:03:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4NnADjclEJcrO+T+qGPsu3IX/25Q5Wz2cI8VcHKA8p0=; b=P3CInz08lWNB/QhCwsz2Du7ufQW6JVpfEUr4zIPylD4DLB8YfQXTT18MvJ4REyBciB 6uaCJbCaRHt2myoOZq7prkGs8sLBO8NsLCHWQZSqSoPv8mhiKjtqfHtgkR8SSbfRFttQ TIBcyZLRV9eEDjb2lwVi1lAbyTxnwKrUVvwgK2VUcvBdqwM0J3tmojtk3kFecqOB50cB rGaJ6OaoSM5aBmnKgEPi3UO0mUyKGDnphpgBvkdTyHnqpHJJ1xKTmn563xdvg8BNMG5F nscPDLMDLEluoC+OAafA6yyV+LaLI9YcAy+cg33ztavt+Oo0tMhLVIYjbWTlpGFRVrFv jEwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4NnADjclEJcrO+T+qGPsu3IX/25Q5Wz2cI8VcHKA8p0=; b=Ed/eDZVtGvo7xR7gyUtnXduUcwc0c+ckNJqZ6khUxBY7hPSbqpMyFztvD0Jpp3toKN jdt+DQBkuidAPuRP9DgpsBoc4DO3W8LUEIrwKpSR1YzRBlTVRno+7djCD+eSOX1/wHIy kqHxMiN2Eh/BEgxjXiHDnS9Mo/rR34yZVeyRWbyycSCrJPQxjEualLqqw8gSyuNcmCy8 Ubt8dB+xPugV9aGJeZ7SZLlAbw7dDE2iagcn087XRYvS8fATm5zS0lZI4p0vpQdTN8Es lz+NYNcC3h0oQhepAug69Q6zTfXmTIaHm7eSpHIEJ6EMvEoKojpxdbveEbghgb8uZZw1 K6ug== X-Gm-Message-State: AOAM5331MZteWIqHGse5ISJLJ/dE/z2KNTR93KBvEyIygyrRfStnKYzl 5dmaRl/PfG/zBYLr6BCe8er1lB7bbwS7iQ== X-Received: by 2002:a17:90a:4d8a:: with SMTP id m10mr11186681pjh.42.1618606992100; Fri, 16 Apr 2021 14:03:12 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:11 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 30/81] target/arm: Implement SVE2 WHILEGT, WHILEGE, WHILEHI, WHILEHS Date: Fri, 16 Apr 2021 14:01:49 -0700 Message-Id: <20210416210240.1591291-31-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1034.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Rename the existing sve_while (less-than) helper to sve_whilel to make room for a new sve_whileg helper for greater-than. Signed-off-by: Richard Henderson --- v2: Use a new helper function to implement this. v4: Update for PREDDESC. --- target/arm/helper-sve.h | 3 +- target/arm/sve.decode | 2 +- target/arm/sve_helper.c | 38 +++++++++++++++++++++++++- target/arm/translate-sve.c | 56 ++++++++++++++++++++++++++++---------- 4 files changed, 82 insertions(+), 17 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 1c7fe8e417..5bf9fdc7a3 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -913,7 +913,8 @@ DEF_HELPER_FLAGS_4(sve_brkns, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_cntp, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) -DEF_HELPER_FLAGS_3(sve_while, TCG_CALL_NO_RWG, i32, ptr, i32, i32) +DEF_HELPER_FLAGS_3(sve_whilel, TCG_CALL_NO_RWG, i32, ptr, i32, i32) +DEF_HELPER_FLAGS_3(sve_whileg, TCG_CALL_NO_RWG, i32, ptr, i32, i32) DEF_HELPER_FLAGS_4(sve_subri_b, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(sve_subri_h, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 0674464695..ae853d21f2 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -700,7 +700,7 @@ SINCDECP_z 00100101 .. 1010 d:1 u:1 10000 00 .... ..... @incdec2_pred CTERM 00100101 1 sf:1 1 rm:5 001000 rn:5 ne:1 0000 # SVE integer compare scalar count and limit -WHILE 00100101 esz:2 1 rm:5 000 sf:1 u:1 1 rn:5 eq:1 rd:4 +WHILE 00100101 esz:2 1 rm:5 000 sf:1 u:1 lt:1 rn:5 eq:1 rd:4 ### SVE Integer Wide Immediate - Unpredicated Group diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 02e87c535d..fb38f2c57e 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3745,7 +3745,7 @@ uint64_t HELPER(sve_cntp)(void *vn, void *vg, uint32_t pred_desc) return sum; } -uint32_t HELPER(sve_while)(void *vd, uint32_t count, uint32_t pred_desc) +uint32_t HELPER(sve_whilel)(void *vd, uint32_t count, uint32_t pred_desc) { intptr_t oprsz = FIELD_EX32(pred_desc, PREDDESC, OPRSZ); intptr_t esz = FIELD_EX32(pred_desc, PREDDESC, ESZ); @@ -3771,6 +3771,42 @@ uint32_t HELPER(sve_while)(void *vd, uint32_t count, uint32_t pred_desc) return predtest_ones(d, oprsz, esz_mask); } +uint32_t HELPER(sve_whileg)(void *vd, uint32_t count, uint32_t pred_desc) +{ + intptr_t oprsz = FIELD_EX32(pred_desc, PREDDESC, OPRSZ); + intptr_t esz = FIELD_EX32(pred_desc, PREDDESC, ESZ); + uint64_t esz_mask = pred_esz_masks[esz]; + ARMPredicateReg *d = vd; + intptr_t i, invcount, oprbits; + uint64_t bits; + + if (count == 0) { + return do_zero(d, oprsz); + } + + oprbits = oprsz * 8; + tcg_debug_assert(count <= oprbits); + + bits = esz_mask; + if (oprbits & 63) { + bits &= MAKE_64BIT_MASK(0, oprbits & 63); + } + + invcount = oprbits - count; + for (i = (oprsz - 1) / 8; i > invcount / 64; --i) { + d->p[i] = bits; + bits = esz_mask; + } + + d->p[i] = bits & MAKE_64BIT_MASK(invcount & 63, 64); + + while (--i >= 0) { + d->p[i] = 0; + } + + return predtest_ones(d, oprsz, esz_mask); +} + /* Recursive reduction on a function; * C.f. the ARM ARM function ReducePredicated. * diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 218f1ca5ce..aff85b0220 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3112,7 +3112,14 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) unsigned vsz = vec_full_reg_size(s); unsigned desc = 0; TCGCond cond; + uint64_t maxval; + /* Note that GE/HS has a->eq == 0 and GT/HI has a->eq == 1. */ + bool eq = a->eq == a->lt; + /* The greater-than conditions are all SVE2. */ + if (!a->lt && !dc_isar_feature(aa64_sve2, s)) { + return false; + } if (!sve_access_check(s)) { return true; } @@ -3135,22 +3142,42 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) */ t0 = tcg_temp_new_i64(); t1 = tcg_temp_new_i64(); - tcg_gen_sub_i64(t0, op1, op0); + + if (a->lt) { + tcg_gen_sub_i64(t0, op1, op0); + if (a->u) { + maxval = a->sf ? UINT64_MAX : UINT32_MAX; + cond = eq ? TCG_COND_LEU : TCG_COND_LTU; + } else { + maxval = a->sf ? INT64_MAX : INT32_MAX; + cond = eq ? TCG_COND_LE : TCG_COND_LT; + } + } else { + tcg_gen_sub_i64(t0, op0, op1); + if (a->u) { + maxval = 0; + cond = eq ? TCG_COND_GEU : TCG_COND_GTU; + } else { + maxval = a->sf ? INT64_MIN : INT32_MIN; + cond = eq ? TCG_COND_GE : TCG_COND_GT; + } + } tmax = tcg_const_i64(vsz >> a->esz); - if (a->eq) { + if (eq) { /* Equality means one more iteration. */ tcg_gen_addi_i64(t0, t0, 1); - /* If op1 is max (un)signed integer (and the only time the addition - * above could overflow), then we produce an all-true predicate by - * setting the count to the vector length. This is because the - * pseudocode is described as an increment + compare loop, and the - * max integer would always compare true. + /* + * For the less-than while, if op1 is maxval (and the only time + * the addition above could overflow), then we produce an all-true + * predicate by setting the count to the vector length. This is + * because the pseudocode is described as an increment + compare + * loop, and the maximum integer would always compare true. + * Similarly, the greater-than while has the same issue with the + * minimum integer due to the decrement + compare loop. */ - tcg_gen_movi_i64(t1, (a->sf - ? (a->u ? UINT64_MAX : INT64_MAX) - : (a->u ? UINT32_MAX : INT32_MAX))); + tcg_gen_movi_i64(t1, maxval); tcg_gen_movcond_i64(TCG_COND_EQ, t0, op1, t1, tmax, t0); } @@ -3159,9 +3186,6 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) tcg_temp_free_i64(tmax); /* Set the count to zero if the condition is false. */ - cond = (a->u - ? (a->eq ? TCG_COND_LEU : TCG_COND_LTU) - : (a->eq ? TCG_COND_LE : TCG_COND_LT)); tcg_gen_movi_i64(t1, 0); tcg_gen_movcond_i64(cond, t0, op0, op1, t0, t1); tcg_temp_free_i64(t1); @@ -3181,7 +3205,11 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) ptr = tcg_temp_new_ptr(); tcg_gen_addi_ptr(ptr, cpu_env, pred_full_reg_offset(s, a->rd)); - gen_helper_sve_while(t2, ptr, t2, t3); + if (a->lt) { + gen_helper_sve_whilel(t2, ptr, t2, t3); + } else { + gen_helper_sve_whileg(t2, ptr, t2, t3); + } do_pred_flags(t2); tcg_temp_free_ptr(ptr); From patchwork Fri Apr 16 21:01:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422621 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp731842jaf; Fri, 16 Apr 2021 14:27:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxVQyz2UKUS+defLWTFoLn7GFaPQIeHxlETAbKweGWOVqICdVGQ0Z6gVrQsWmoOAsr8zaIW X-Received: by 2002:a05:6638:2044:: with SMTP id t4mr6104860jaj.12.1618608476146; Fri, 16 Apr 2021 14:27:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608476; cv=none; d=google.com; s=arc-20160816; b=s81TVHLFHnc0D/1qhwiRT6IQCAEDsNO+b937mz6iprW232FpcG0NWolYUZ0JFpVy0J I8q8sGNZAnM9GFD66eXpECnAPlDgwzEyx/ybHg2pNzTWybiJciUCG3wd8s7ZaRHv5vA2 GB/w/scJVxU8iEb77ESh6KeoNM8MHWTLF9+CWO04tDJ6hlO7H8hkfF8Gjgp2Lm9uHav0 rMbZmnEStBjMaUdYjZMGy0kZJOQ62QEpmgvwPM0o8WpH4M9kn0BFXa+g7z0oRmUa4OCA jkO5XcX/dePCOQpiJHi7F8Tqw8szXRBiSBXZCWQz+X9oLOOuh3ChGvfcuNg8anIso2xL Rwqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Zvb5+uXwo3ITZTOH3/3hYKLwYLR4lwPy53f9vTvelhk=; b=rLisWjkHNx3r9YRnSn45iqdIqxT7TCMiSc1mh7nPk9DFZtM23kMcNEGgI5O7eO3ruy risoMQR2FHa/XDG93BoQm8aBK+0RCjy1gwjUozwZs/+vdK7Na8XwcV+h4eLe9tUJbZ70 zw797yrrYDFCFGGdIczmc9uxBfyQumBt029/5eiToX0ii5LyccE4cs3fqEOXCjhMHUZq Z+CDIqH9fxtksVJ9DJtr3aHCWqyuJb4XA3BrLrwwTNruPQUaATrPLmiituf5QELhx2G+ c5t8fBb7pq3d16YzTkCCcuh2os15ZNOumJh6WJoOVJwp6ybg77lUQuo4pT/YDCjWtX9i +5Xw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jhw63yBa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k12si2316177jad.30.2021.04.16.14.27.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:27:56 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jhw63yBa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35440 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW0J-0003lf-JK for patch@linaro.org; Fri, 16 Apr 2021 17:27:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44238) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcX-0004rG-Tw for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:21 -0400 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]:38564) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcQ-0001I4-KM for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:21 -0400 Received: by mail-pf1-x431.google.com with SMTP id g16so5843191pfq.5 for ; Fri, 16 Apr 2021 14:03:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Zvb5+uXwo3ITZTOH3/3hYKLwYLR4lwPy53f9vTvelhk=; b=jhw63yBayYiKghKp3gMfOw3Ldn8PqEk4eqqiI84wZt2bKSSkvB0dSiC/09XiSao81o Jkm+tgnhyaFBsrHk7JdUH3rwFRKHatK2Bd9MczVMNTt4Z0zGfCDxG+V50aLUgPy85z/o BM8sHBGwUouUgmD25Hl5+Ql+5pAW6dpHSGg/B0W+3AltZgAioZ47/mzHyFAInHV/glet 0VPzdWnzfNNCwJlcRbCWgfzIHKWFT+NNoHeUdL+m3ANQzHZmnlp8Eh2oe9+R0G0tIIvg KsrRlitM7kJ5iigAyzaT0m5U0vzlYR5uGymPSYizIbox+er8nijlykPV8InHfturKaLJ EJ8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Zvb5+uXwo3ITZTOH3/3hYKLwYLR4lwPy53f9vTvelhk=; b=USqLtboNdqjaRGnvrLIgMT7+69+XPjFMsT4YMD1FajjlUKycW7uvnoThwRIPnc2ORy cCwnKjxtXpT2Ae4Bx4qTs73oPzV0POtZ+FNTtboUM0gP5Q1loJB80sooOci1XD1+JAug GQmJ/73MaeIp/9+pW7exyfNir58MO4UgUtbe4GLfNHIudbf0dCiWp1z+BwDiqHvg8ykV 2Z3fLxjGQVzihvSjE8yJXEqGnAX/9YprZxsLS2BweHp5Gy7E4AlRNNQk7nyknJuLKdJQ KKwdlDabNGzYa1ORLZeZQV+6pRmsUKB3uWZImLAk9feHTfrN2r4m4V7tJJQB7yMwcQjk pXWg== X-Gm-Message-State: AOAM531+hnieuUFTfSD3h+cw7wFCmr3GCow2+zbu0fHrRJ5E7XShGNgR iey+hBziLAPfqm/I1X4Q4RzxM4TviqIYow== X-Received: by 2002:a62:7e53:0:b029:254:44f:23da with SMTP id z80-20020a627e530000b0290254044f23damr9644081pfc.38.1618606992959; Fri, 16 Apr 2021 14:03:12 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:12 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 31/81] target/arm: Implement SVE2 WHILERW, WHILEWR Date: Fri, 16 Apr 2021 14:01:50 -0700 Message-Id: <20210416210240.1591291-32-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix decodetree typo v3: Fix iteration counts (zhiwei). v4: Update for PREDDESC. --- target/arm/sve.decode | 3 ++ target/arm/translate-sve.c | 67 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index ae853d21f2..f365907518 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -702,6 +702,9 @@ CTERM 00100101 1 sf:1 1 rm:5 001000 rn:5 ne:1 0000 # SVE integer compare scalar count and limit WHILE 00100101 esz:2 1 rm:5 000 sf:1 u:1 lt:1 rn:5 eq:1 rd:4 +# SVE2 pointer conflict compare +WHILE_ptr 00100101 esz:2 1 rm:5 001 100 rn:5 rw:1 rd:4 + ### SVE Integer Wide Immediate - Unpredicated Group # SVE broadcast floating-point immediate (unpredicated) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index aff85b0220..97e113ceec 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3218,6 +3218,73 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) return true; } +static bool trans_WHILE_ptr(DisasContext *s, arg_WHILE_ptr *a) +{ + TCGv_i64 op0, op1, diff, t1, tmax; + TCGv_i32 t2, t3; + TCGv_ptr ptr; + unsigned vsz = vec_full_reg_size(s); + unsigned desc = 0; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (!sve_access_check(s)) { + return true; + } + + op0 = read_cpu_reg(s, a->rn, 1); + op1 = read_cpu_reg(s, a->rm, 1); + + tmax = tcg_const_i64(vsz); + diff = tcg_temp_new_i64(); + + if (a->rw) { + /* WHILERW */ + /* diff = abs(op1 - op0), noting that op0/1 are unsigned. */ + t1 = tcg_temp_new_i64(); + tcg_gen_sub_i64(diff, op0, op1); + tcg_gen_sub_i64(t1, op1, op0); + tcg_gen_movcond_i64(TCG_COND_GEU, diff, op0, op1, diff, t1); + tcg_temp_free_i64(t1); + /* Round down to a multiple of ESIZE. */ + tcg_gen_andi_i64(diff, diff, -1 << a->esz); + /* If op1 == op0, diff == 0, and the condition is always true. */ + tcg_gen_movcond_i64(TCG_COND_EQ, diff, op0, op1, tmax, diff); + } else { + /* WHILEWR */ + tcg_gen_sub_i64(diff, op1, op0); + /* Round down to a multiple of ESIZE. */ + tcg_gen_andi_i64(diff, diff, -1 << a->esz); + /* If op0 >= op1, diff <= 0, the condition is always true. */ + tcg_gen_movcond_i64(TCG_COND_GEU, diff, op0, op1, tmax, diff); + } + + /* Bound to the maximum. */ + tcg_gen_umin_i64(diff, diff, tmax); + tcg_temp_free_i64(tmax); + + /* Since we're bounded, pass as a 32-bit type. */ + t2 = tcg_temp_new_i32(); + tcg_gen_extrl_i64_i32(t2, diff); + tcg_temp_free_i64(diff); + + desc = FIELD_DP32(desc, PREDDESC, OPRSZ, vsz / 8); + desc = FIELD_DP32(desc, PREDDESC, ESZ, a->esz); + t3 = tcg_const_i32(desc); + + ptr = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(ptr, cpu_env, pred_full_reg_offset(s, a->rd)); + + gen_helper_sve_whilel(t2, ptr, t2, t3); + do_pred_flags(t2); + + tcg_temp_free_ptr(ptr); + tcg_temp_free_i32(t2); + tcg_temp_free_i32(t3); + return true; +} + /* *** SVE Integer Wide Immediate - Unpredicated Group */ From patchwork Fri Apr 16 21:01:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422630 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp737032jaf; Fri, 16 Apr 2021 14:37:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzmE0uJZo5EXABJETc5q+4A1hmfJJ6HqJ/RrAuQ3qHgNndganxh+dwzL4OSs8YAs/YQBTcY X-Received: by 2002:a92:d70c:: with SMTP id m12mr8915894iln.216.1618609031898; Fri, 16 Apr 2021 14:37:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609031; cv=none; d=google.com; s=arc-20160816; b=luHu0EDPNOzolZPq0WqDakINOVETeyK5nmkrDLjIhGHqxlhNDeF1d3hbvCUKPEQsut e+mhapJP9vf2BRJCb3zCHiQm2+Lvu5PGRwjnXGOOEMuIH1xkTt9TskeXmtzCq/xUgpQ7 /TenU2CqLKZ3O6q+3nPXHuFETFt4ywifLJZq7/5MYr7w9k1QGRqOhECD8acEyx3djz4n 6z7cmtvVuJUo4xrI1IQOxIQF31+SCs+/wvkl1YOBGPItWQ+pd3mtYQxCLGrVCaJR2MUH RIEMo8HrH1mMOFbZqfAIodB+FRaBOh5N+/OfoGfyStzZpBE7dRMxPsvhPiVYYmCB9j6s Ghzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=VnUeaFstfauSS4JTjM/wMyZDfm9bVsWwY847CCsEpAM=; b=R5mUMaYIlwAKaX3CQdszC1pdNPqCu58QMPrP6b3HPQBKAwyvn40/lgKoEACVs28RYY JykYMz9omdGKHql60WoG56YxPGCEXTzzwEpZgIbsywz4rQpbjCb9wZNQhufkW2L1dvBe vdV3qnmG5YHYUJ3AVsJY+G0kM8DiJgNHWsjRKQXmQ3YiiBQbyju39RUTeXgVKU7W5bSd gQuQOxUQEsE/QHcZZ+p+E0bTH85OjCtIcHnys9bL73jpJYmc/1NUPBkdJyXTWIOf+9Vo 3wWZJqobACGTCRzr3IH72mwVzlqeS8JhNLGlrQuy0i0ExbC6JdpkRO6AF2efM1UfAZcT 7Y+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="ed/ttoiq"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m17si8601635jaj.76.2021.04.16.14.37.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:37:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="ed/ttoiq"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56038 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW9H-00049w-7W for patch@linaro.org; Fri, 16 Apr 2021 17:37:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44320) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVch-0004st-Hr for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:33 -0400 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]:43941) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcR-0001IR-6f for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:30 -0400 Received: by mail-pj1-x1036.google.com with SMTP id x21-20020a17090a5315b029012c4a622e4aso15260422pjh.2 for ; Fri, 16 Apr 2021 14:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VnUeaFstfauSS4JTjM/wMyZDfm9bVsWwY847CCsEpAM=; b=ed/ttoiqFSyz12vMVfDNeHnFje2bcfafpdVe/gbuzxmW98wiU/nqPUouzMQqeMieZX w09GRsMieu4tBOGvPcbVEUPI4yg3hRU5MeiRljvvoobQJ/KzIE+MlubtJKKfUZ0hEzmw Um4hLqrn80JhpKKTtLPqkiboU00C302StTDOy6uRXmPmMVjOt8ebquRo+s+sLuFElWxU YhMM8WoGmKrLbfoFpcuMfByhorpiqbQ5CgDfby/3tBguSES7QpXpfVOqoyVl3j0A8n6F Za1DVcwJ+j49lGaXUGuMFLxsy6Gc4QnX4hEJ16H3ZZHthfwsIcjggctyzFS+Sao2ruZ5 Qx+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VnUeaFstfauSS4JTjM/wMyZDfm9bVsWwY847CCsEpAM=; b=Lxq4eS570Mp9VIdRti+NnlDuL5s+6f7tPcjTFlQ4oB9ANIFv3k2FiumN4NjraA/70/ noPhtscC2HKLVOrhiDl5NcnjBHRDDwxalyHVpD7uB1qkRvIiqQoeFrTMVhl0fDZDSeNP cCswiA/62K2PNAZdaTOK/zva3UmgsQHvDe4nIqkFrHMnIhDpp8U1Jhd14lljq2GsZL+w jmiwTP1g5Ku9ddi5vNFFjKgFUIWz/7EZAKNebk4+oKlurOR2165L9j6ceJmXBK8IzTLt mLnymNlXfUZcQNCAhLlXGXcYumIzVnj9Ph1DcKaCYCZNgk/w+X8QkP509jc43B085m1o 95bA== X-Gm-Message-State: AOAM531xQbyr4gyifs0H58wijTJ8xLxCsCw9nfFqMmJo1aHtRGVsQ3Ra MNtzX9ZY565jtXkrWFdES6upLp213EmNcQ== X-Received: by 2002:a17:902:f212:b029:e9:56f2:18ba with SMTP id m18-20020a170902f212b02900e956f218bamr11369724plc.75.1618606993928; Fri, 16 Apr 2021 14:03:13 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:13 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 32/81] target/arm: Implement SVE2 bitwise ternary operations Date: Fri, 16 Apr 2021 14:01:51 -0700 Message-Id: <20210416210240.1591291-33-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1036.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 6 ++ target/arm/sve.decode | 12 +++ target/arm/sve_helper.c | 50 +++++++++ target/arm/translate-sve.c | 213 +++++++++++++++++++++++++++++++++++++ 4 files changed, 281 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 5bf9fdc7a3..df617e3351 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2543,3 +2543,9 @@ DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_eor3, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_bcax, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_bsl1n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_bsl2n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_nbsl, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index f365907518..bf673e2f16 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -124,6 +124,10 @@ @rda_rn_rm ........ esz:2 . rm:5 ... ... rn:5 rd:5 \ &rrrr_esz ra=%reg_movprfx +# Four operand with unused vector element size +@rdn_ra_rm_e0 ........ ... rm:5 ... ... ra:5 rd:5 \ + &rrrr_esz esz=0 rn=%reg_movprfx + # Three operand with "memory" size, aka immediate left shift @rd_rn_msz_rm ........ ... rm:5 .... imm:2 rn:5 rd:5 &rrri @@ -379,6 +383,14 @@ ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +# SVE2 bitwise ternary operations +EOR3 00000100 00 1 ..... 001 110 ..... ..... @rdn_ra_rm_e0 +BSL 00000100 00 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 +BCAX 00000100 01 1 ..... 001 110 ..... ..... @rdn_ra_rm_e0 +BSL1N 00000100 01 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 +BSL2N 00000100 10 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 +NBSL 00000100 11 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 + ### SVE Index Generation Group # SVE index generation (immediate start, immediate increment) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index fb38f2c57e..b0598f9097 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -6792,3 +6792,53 @@ DO_ST1_ZPZ_D(dd_be, zd, MO_64) #undef DO_ST1_ZPZ_S #undef DO_ST1_ZPZ_D + +void HELPER(sve2_eor3)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = n[i] ^ m[i] ^ k[i]; + } +} + +void HELPER(sve2_bcax)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = n[i] ^ (m[i] & ~k[i]); + } +} + +void HELPER(sve2_bsl1n)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = (~n[i] & k[i]) | (m[i] & ~k[i]); + } +} + +void HELPER(sve2_bsl2n)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = (n[i] & k[i]) | (~m[i] & ~k[i]); + } +} + +void HELPER(sve2_nbsl)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ~((n[i] & k[i]) | (m[i] & ~k[i])); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 97e113ceec..ab290b9025 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -217,6 +217,17 @@ static void gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn, vec_full_reg_offset(s, rm), vsz, vsz); } +/* Invoke a vector expander on four Zregs. */ +static void gen_gvec_fn_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn, + int esz, int rd, int rn, int rm, int ra) +{ + unsigned vsz = vec_full_reg_size(s); + gvec_fn(esz, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), vsz, vsz); +} + /* Invoke a vector move on two Zregs. */ static bool do_mov_z(DisasContext *s, int rd, int rn) { @@ -329,6 +340,208 @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a) return do_zzz_fn(s, a, tcg_gen_gvec_andc); } +static bool do_sve2_zzzz_fn(DisasContext *s, arg_rrrr_esz *a, GVecGen4Fn *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzzz(s, fn, a->esz, a->rd, a->rn, a->rm, a->ra); + } + return true; +} + +static void gen_eor3_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_xor_i64(d, n, m); + tcg_gen_xor_i64(d, d, k); +} + +static void gen_eor3_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + tcg_gen_xor_vec(vece, d, n, m); + tcg_gen_xor_vec(vece, d, d, k); +} + +static void gen_eor3(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_eor3_i64, + .fniv = gen_eor3_vec, + .fno = gen_helper_sve2_eor3, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_EOR3(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_eor3); +} + +static void gen_bcax_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_andc_i64(d, m, k); + tcg_gen_xor_i64(d, d, n); +} + +static void gen_bcax_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + tcg_gen_andc_vec(vece, d, m, k); + tcg_gen_xor_vec(vece, d, d, n); +} + +static void gen_bcax(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_bcax_i64, + .fniv = gen_bcax_vec, + .fno = gen_helper_sve2_bcax, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_BCAX(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bcax); +} + +static void gen_bsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + /* BSL differs from the generic bitsel in argument ordering. */ + tcg_gen_gvec_bitsel(vece, d, a, n, m, oprsz, maxsz); +} + +static bool trans_BSL(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bsl); +} + +static void gen_bsl1n_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_andc_i64(n, k, n); + tcg_gen_andc_i64(m, m, k); + tcg_gen_or_i64(d, n, m); +} + +static void gen_bsl1n_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + if (TCG_TARGET_HAS_bitsel_vec) { + tcg_gen_not_vec(vece, n, n); + tcg_gen_bitsel_vec(vece, d, k, n, m); + } else { + tcg_gen_andc_vec(vece, n, k, n); + tcg_gen_andc_vec(vece, m, m, k); + tcg_gen_or_vec(vece, d, n, m); + } +} + +static void gen_bsl1n(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_bsl1n_i64, + .fniv = gen_bsl1n_vec, + .fno = gen_helper_sve2_bsl1n, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_BSL1N(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bsl1n); +} + +static void gen_bsl2n_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + /* + * Z[dn] = (n & k) | (~m & ~k) + * = | ~(m | k) + */ + tcg_gen_and_i64(n, n, k); + if (TCG_TARGET_HAS_orc_i64) { + tcg_gen_or_i64(m, m, k); + tcg_gen_orc_i64(d, n, m); + } else { + tcg_gen_nor_i64(m, m, k); + tcg_gen_or_i64(d, n, m); + } +} + +static void gen_bsl2n_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + if (TCG_TARGET_HAS_bitsel_vec) { + tcg_gen_not_vec(vece, m, m); + tcg_gen_bitsel_vec(vece, d, k, n, m); + } else { + tcg_gen_and_vec(vece, n, n, k); + tcg_gen_or_vec(vece, m, m, k); + tcg_gen_orc_vec(vece, d, n, m); + } +} + +static void gen_bsl2n(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_bsl2n_i64, + .fniv = gen_bsl2n_vec, + .fno = gen_helper_sve2_bsl2n, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_BSL2N(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bsl2n); +} + +static void gen_nbsl_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_and_i64(n, n, k); + tcg_gen_andc_i64(m, m, k); + tcg_gen_nor_i64(d, n, m); +} + +static void gen_nbsl_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + tcg_gen_bitsel_vec(vece, d, k, n, m); + tcg_gen_not_vec(vece, d, d); +} + +static void gen_nbsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_nbsl_i64, + .fniv = gen_nbsl_vec, + .fno = gen_helper_sve2_nbsl, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_NBSL(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_nbsl); +} + /* *** SVE Integer Arithmetic - Unpredicated Group */ From patchwork Fri Apr 16 21:01:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422633 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp738022jaf; Fri, 16 Apr 2021 14:39:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwR9GvosxuO06+0X9qpIz8ePkkzuwa0Y6faSMDqQSAOug52RU6Ha9RYkLcao0fWXgDxFx68 X-Received: by 2002:a6b:3b08:: with SMTP id i8mr5137912ioa.36.1618609151697; Fri, 16 Apr 2021 14:39:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609151; cv=none; d=google.com; s=arc-20160816; b=zcAb4E/nIthaq2M+pW7McnGtAsAF4RF7DCvf52Bxy6kNgjwg9Ev3JdUUKZU+OMF0Xl DXptb7vqnTIbvM2+rj5DBrxVPuR4q7A+dEjRtKPUUAwIX98ViBZHr+NL1e7Dlpq64eoK HmIA69aHCuwMILHJPO2FkX/FZ93J7QWsFuLALUfYqcUMeA2AwM9hmX11JcqYrDVpK7lv M39lJWPpW/N09bjEaTxFu3dRxFlyMw1P0FYbR6KwzrqlEs+VfaRodBkbpy/tx5XN3wjo upNkX04SCreMo6JiPeI2A0KSH5cd1CsaMsqQwUQ3mAsixsvHLlacsV43Ac7/dp0MrpFE qPIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=tfgjxN5v8ydM/Pn6KFDCWVXjiic+OopJ/aa+kcSpENo=; b=zz+qpf/Z1qXLl9uvno9JFQD+AIlFAObzmvcbAXRxmDSo1rMClKiBerBzh+g6L6rKnM FVEaljt6axhvtEDT7899Rj/NcPesICDs91lwrjJCJiLFCdWWieU5UVP5VfBSuzLfIEBJ 8jXqdOD+8z8sVeUXXIyUZxGLq+Ey8BJXmaUcGwSSOJLjsGWpw3G1jlOHoFwM9NcWCqob 0oFLH9mYAlND7O7oLrHltMXQD3g5neFXmneFLe/L+p9WYTc006jYDPw1TuieLRHIcF1t sl7//uzBXXihs2gfHQCrg2vuz+rG8N2tAJRSjbIHkykUzrH6fIk+bs5LsmLYUBAk4G5o qtVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fLVA8DEY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k12si2347263jad.30.2021.04.16.14.39.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:39:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fLVA8DEY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36640 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWBD-000897-43 for patch@linaro.org; Fri, 16 Apr 2021 17:39:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44318) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVch-0004ss-Hk for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:33 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]:40650) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcS-0001Ig-2J for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:30 -0400 Received: by mail-pj1-x1031.google.com with SMTP id b8-20020a17090a5508b029014d0fbe9b64so17051146pji.5 for ; Fri, 16 Apr 2021 14:03:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tfgjxN5v8ydM/Pn6KFDCWVXjiic+OopJ/aa+kcSpENo=; b=fLVA8DEY/ShW/mkiKk3EeKD1sIEdOjy+SGrKnK3eDefyAEY1wQ6Snm7c9XFVo3/dgE PoSEV7YSesUZOjcLxXFxC2luDFHuq21qu12z5/R0QICZFbzlvIe9l+W/+KrSC5HjpY7u wb7o1oehqYIZt7q/ZYxOSKja4BWVuTJld5jGaxnPs/OlxADJZn/yPPwRY+QT/jnkR13B RbjBJ3liCVZxvFvUBwJNpO/K/8NzPYgOgBLUxlae/SulnIr/P6OUiYK2G+Hd3gW6r6jB HwUq8wV6HWnZYy15T0fj9OY8o+M2u8oyBmShux/bBsBpOoBU4aOb/4UcPlX4sONy33I6 34Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tfgjxN5v8ydM/Pn6KFDCWVXjiic+OopJ/aa+kcSpENo=; b=S9z2/b/oRGpupXY7ZERoJ7eIFbDdGbIM2JpGmXfDYgi5Hlcp0MaL6ad3p6G+lajRgJ FnviO9H5zFwklzjxn15kh+U0VgVuQtsFc1ogmISflz9/Q3sYmyXl3Qf/gW9/cwJGyC3Q ayJMboovA7O1Z7hZ3+fnXhIyt5QvZaXUwmlNlDIGUntMUu2Xwg78ZfxI6oH0/QpbsYvt wpgQyq7RALgSqPu0CkGX15KOQfrR0oiTR+BUxHGxLRxcHDSXOgBX1ayDHe4QI18u9FEp VkdhTR8dNP5uZaHCri4A2Qyg2jj+0P4Wz4taKTP6DP9UethgBNwHlxQQmDfQ8UlTfIz4 d4Zw== X-Gm-Message-State: AOAM533MKhtBftw/Wt1/YUJ4OTROPyqWE8c8NF5lUcrv0KmMwV7Eyy43 jscIxqIhMbSlXrsPSY9eqR2Se2MA2mAKcQ== X-Received: by 2002:a17:902:e812:b029:eb:59d9:e550 with SMTP id u18-20020a170902e812b02900eb59d9e550mr11235959plg.26.1618606994775; Fri, 16 Apr 2021 14:03:14 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:14 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 33/81] target/arm: Implement SVE2 MATCH, NMATCH Date: Fri, 16 Apr 2021 14:01:52 -0700 Message-Id: <20210416210240.1591291-34-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Reviewed-by: Richard Henderson Signed-off-by: Stephen Long Message-Id: <20200415145915.2859-1-steplong@quicinc.com> [rth: Expanded comment for do_match2] Signed-off-by: Richard Henderson --- v2: Apply esz_mask to input pg to fix output flags. --- target/arm/helper-sve.h | 10 ++++++ target/arm/sve.decode | 5 +++ target/arm/sve_helper.c | 64 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 22 +++++++++++++ 4 files changed, 101 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index df617e3351..11dc6870de 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2509,6 +2509,16 @@ DEF_HELPER_FLAGS_3(sve2_uqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_b, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_h, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index bf673e2f16..47fca5e12d 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1320,6 +1320,11 @@ UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr +### SVE2 Character Match + +MATCH 01000101 .. 1 ..... 100 ... ..... 0 .... @pd_pg_rn_rm +NMATCH 01000101 .. 1 ..... 100 ... ..... 1 .... @pd_pg_rn_rm + ## SVE2 floating-point pairwise operations FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index b0598f9097..010d8b260a 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -6842,3 +6842,67 @@ void HELPER(sve2_nbsl)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) d[i] = ~((n[i] & k[i]) | (m[i] & ~k[i])); } } + +/* + * Returns true if m0 or m1 contains the low uint8_t/uint16_t in n. + * See hasless(v,1) from + * https://graphics.stanford.edu/~seander/bithacks.html#ZeroInWord + */ +static inline bool do_match2(uint64_t n, uint64_t m0, uint64_t m1, int esz) +{ + int bits = 8 << esz; + uint64_t ones = dup_const(esz, 1); + uint64_t signs = ones << (bits - 1); + uint64_t cmp0, cmp1; + + cmp1 = dup_const(esz, n); + cmp0 = cmp1 ^ m0; + cmp1 = cmp1 ^ m1; + cmp0 = (cmp0 - ones) & ~cmp0; + cmp1 = (cmp1 - ones) & ~cmp1; + return (cmp0 | cmp1) & signs; +} + +static inline uint32_t do_match(void *vd, void *vn, void *vm, void *vg, + uint32_t desc, int esz, bool nmatch) +{ + uint16_t esz_mask = pred_esz_masks[esz]; + intptr_t opr_sz = simd_oprsz(desc); + uint32_t flags = PREDTEST_INIT; + intptr_t i, j, k; + + for (i = 0; i < opr_sz; i += 16) { + uint64_t m0 = *(uint64_t *)(vm + i); + uint64_t m1 = *(uint64_t *)(vm + i + 8); + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)) & esz_mask; + uint16_t out = 0; + + for (j = 0; j < 16; j += 8) { + uint64_t n = *(uint64_t *)(vn + i + j); + + for (k = 0; k < 8; k += 1 << esz) { + if (pg & (1 << (j + k))) { + bool o = do_match2(n >> (k * 8), m0, m1, esz); + out |= (o ^ nmatch) << (j + k); + } + } + } + *(uint16_t *)(vd + H1_2(i >> 3)) = out; + flags = iter_predtest_fwd(out, pg, flags); + } + return flags; +} + +#define DO_PPZZ_MATCH(NAME, ESZ, INV) \ +uint32_t HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + return do_match(vd, vn, vm, vg, desc, ESZ, INV); \ +} + +DO_PPZZ_MATCH(sve2_match_ppzz_b, MO_8, false) +DO_PPZZ_MATCH(sve2_match_ppzz_h, MO_16, false) + +DO_PPZZ_MATCH(sve2_nmatch_ppzz_b, MO_8, true) +DO_PPZZ_MATCH(sve2_nmatch_ppzz_h, MO_16, true) + +#undef DO_PPZZ_MATCH diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ab290b9025..bdf1da8424 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7462,6 +7462,28 @@ static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, + gen_helper_gvec_flags_4 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_ppzz_flags(s, a, fn); +} + +#define DO_SVE2_PPZZ_MATCH(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ +{ \ + static gen_helper_gvec_flags_4 * const fns[4] = { \ + gen_helper_sve2_##name##_ppzz_b, gen_helper_sve2_##name##_ppzz_h, \ + NULL, NULL \ + }; \ + return do_sve2_ppzz_flags(s, a, fns[a->esz]); \ +} + +DO_SVE2_PPZZ_MATCH(MATCH, match) +DO_SVE2_PPZZ_MATCH(NMATCH, nmatch) + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Fri Apr 16 21:01:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422638 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp739722jaf; Fri, 16 Apr 2021 14:42:39 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwj5sOExJUT3ynOFGOzBNaEzm39tRmEZyI6Xa0VoG4iUPzc8zDWVSF0pAnCWVvoN0kdSz8e X-Received: by 2002:a25:9a46:: with SMTP id r6mr1726054ybo.431.1618609359465; Fri, 16 Apr 2021 14:42:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609359; cv=none; d=google.com; s=arc-20160816; b=pNZJMskGbQey8FWmLXEw/fHFYj7cx/mFPzttIWvKa+V7mAAVPsyh9EZXXLFbkqQqzL ZcyofoiutZp3DqyQ/e7IWMcwiS8mRBZKExh/9BRSVelgi61caYbt08kSzgEWH1CTDRhn zpH0C2yJb5QZume4Do8h2/lo8bK2BxM85fMXoyRYNKD5e08aOBg11gfIqFJPgUpMYbNF dnJJ9NKk8YXAz51IR/HmfKUSvekhNTgb915QqznLKzBiD4nG5KP6m6bitM25PiA9tIMG bsS8TAc0zAUVKSlCbEs1DtdQNLCZKCfJrpwPvN2SQECHfkJZ8+uQpgI72VTiGGJOQNh/ dvsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=wuE0clopIn5KrUkuJnJm6l7c50sb3LTx2vbdW9i3iBQ=; b=g28DZvqd+trg8fVfQ7ZLfhSXeTznAdIm1W+A4lTvo3Ip9PS5hq3pWh8Ww4LkI8Vo2M 278ZZ3yuYmoEszERjYAMNat2VcS2fVJFXiH6+ggiC7M0P0SGlGSgu3D7YVvCLtC5LKx2 DfV8aKxUucyLEpHCPj29HS2UJV/ZZqWMirB8gd+R1VgTsYaMi2N+3mqr9xJ+9zeJ5Zim dKws6TI5chqJM11uH8uCeshMhA8+Zj5f4dPjmqNLxuSEJK96obdCAZmjsR5V2vSIz8DJ mFf1rpAEFJ0LZ4sye/Ivahh7j/spz5q+pGgi2ohgys4nvVjb6yG27JOFWrWoKMBFv//H 3dQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=IWppJfrb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g6si6979429ybe.111.2021.04.16.14.42.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:42:39 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=IWppJfrb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45354 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWEY-0003TA-L4 for patch@linaro.org; Fri, 16 Apr 2021 17:42:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44284) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVce-0004so-0P for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:32 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]:34405) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcS-0001JJ-V5 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:27 -0400 Received: by mail-pf1-x42b.google.com with SMTP id 10so10256805pfl.1 for ; Fri, 16 Apr 2021 14:03:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wuE0clopIn5KrUkuJnJm6l7c50sb3LTx2vbdW9i3iBQ=; b=IWppJfrbZ6eft9zY0AaK7NnTrVbJtOr2yODwmfhfUfo1CGVyismlP6aE/Bxk2+JDdn s4MARANTEiZjLvyYJeMQuiHceHRAnlaGElHEuPZGF0Jlr/ckKXq2Py+S9i6jXuwNca6U 7yCyvN0UpPQLtsgkQ3vgM51aXAmIGx2DE61JLcegKRUZ5QNLeAfsKZPKss0ElIw7puL3 Rm2QzX2yJYvCA51JnHBh5EFFLM2P/OuiS+nSjyP7qnVwlzr5v9QgB88AAMET/a+WcRlC cd6Wl47z20W4WripQGXa6K7QixQUULLYrVGNnTb3XWQPNRymkQWZlal3wYmSol6lw1S3 yIUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wuE0clopIn5KrUkuJnJm6l7c50sb3LTx2vbdW9i3iBQ=; b=FpqjUobyzex42W8Gfhq4suMsuQvlBGmlcdJgrWKVtHwKZKiprPRHLwfOguNqP51Uho 1xTrMP01R9C5Z5pQJFrJNhgAgm8WFujF+WMvP3G9utvwepRsu3Aqkncf7UIInwMZnZrX cDqUbD3v3jjbpWrrRxnrAJhna/cDPM25DFSmYGs2BlURFg9FdR+9GwFUiUgvnIPPYf+u 3aaRZgosX4A2U6TV0fIUUJEHZ1kAcEyso2DstP4SzWmSMvMcOtunUBaCzBQ3II0xKPlY 1E5L23A6Fmow5kt1M5Ch5z3wVtlEfjGELWO/ShdtcXFCN0cfRuoVOBpRZ23cUHe3+S34 9Xhw== X-Gm-Message-State: AOAM5334MqqAlf9nd8x+SfP9bdY1OB0UseO+IHRUvLlWU7KeWMRajqsV MjkkV6yZwmuO61O6vm/BeExPlZ4bwjH0sw== X-Received: by 2002:a63:c111:: with SMTP id w17mr801619pgf.127.1618606995679; Fri, 16 Apr 2021 14:03:15 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:15 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 34/81] target/arm: Implement SVE2 saturating multiply-add long Date: Fri, 16 Apr 2021 14:01:53 -0700 Message-Id: <20210416210240.1591291-35-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 14 ++++++++++ target/arm/sve.decode | 14 ++++++++++ target/arm/sve_helper.c | 30 +++++++++++++++++++++ target/arm/translate-sve.c | 54 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 112 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 11dc6870de..d8f390617c 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2559,3 +2559,17 @@ DEF_HELPER_FLAGS_5(sve2_bcax, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_bsl1n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_bsl2n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_nbsl, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqdmlal_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlal_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 47fca5e12d..52f615b39e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1332,3 +1332,17 @@ FMAXNMP 01100100 .. 010 10 0 100 ... ..... ..... @rdn_pg_rm FMINNMP 01100100 .. 010 10 1 100 ... ..... ..... @rdn_pg_rm FMAXP 01100100 .. 010 11 0 100 ... ..... ..... @rdn_pg_rm FMINP 01100100 .. 010 11 1 100 ... ..... ..... @rdn_pg_rm + +#### SVE Integer Multiply-Add (unpredicated) + +## SVE2 saturating multiply-add long + +SQDMLALB_zzzw 01000100 .. 0 ..... 0110 00 ..... ..... @rda_rn_rm +SQDMLALT_zzzw 01000100 .. 0 ..... 0110 01 ..... ..... @rda_rn_rm +SQDMLSLB_zzzw 01000100 .. 0 ..... 0110 10 ..... ..... @rda_rn_rm +SQDMLSLT_zzzw 01000100 .. 0 ..... 0110 11 ..... ..... @rda_rn_rm + +## SVE2 saturating multiply-add interleaved long + +SQDMLALBT 01000100 .. 0 ..... 00001 0 ..... ..... @rda_rn_rm +SQDMLSLBT 01000100 .. 0 ..... 00001 1 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 010d8b260a..859091b7cf 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1400,6 +1400,36 @@ void HELPER(sve2_adcl_d)(void *vd, void *vn, void *vm, void *va, uint32_t desc) } } +#define DO_SQDMLAL(NAME, TYPEW, TYPEN, HW, HN, DMUL_OP, SUM_OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sel1 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + int sel2 = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel1)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel2)); \ + TYPEW aa = *(TYPEW *)(va + HW(i)); \ + *(TYPEW *)(vd + HW(i)) = SUM_OP(aa, DMUL_OP(nn, mm)); \ + } \ +} + +DO_SQDMLAL(sve2_sqdmlal_zzzw_h, int16_t, int8_t, H1_2, H1, + do_sqdmull_h, DO_SQADD_H) +DO_SQDMLAL(sve2_sqdmlal_zzzw_s, int32_t, int16_t, H1_4, H1_2, + do_sqdmull_s, DO_SQADD_S) +DO_SQDMLAL(sve2_sqdmlal_zzzw_d, int64_t, int32_t, , H1_4, + do_sqdmull_d, do_sqadd_d) + +DO_SQDMLAL(sve2_sqdmlsl_zzzw_h, int16_t, int8_t, H1_2, H1, + do_sqdmull_h, DO_SQSUB_H) +DO_SQDMLAL(sve2_sqdmlsl_zzzw_s, int32_t, int16_t, H1_4, H1_2, + do_sqdmull_s, DO_SQSUB_S) +DO_SQDMLAL(sve2_sqdmlsl_zzzw_d, int64_t, int32_t, , H1_4, + do_sqdmull_d, do_sqsub_d) + +#undef DO_SQDMLAL + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index bdf1da8424..27f9cdb891 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7508,3 +7508,57 @@ DO_SVE2_ZPZZ_FP(FMAXNMP, fmaxnmp) DO_SVE2_ZPZZ_FP(FMINNMP, fminnmp) DO_SVE2_ZPZZ_FP(FMAXP, fmaxp) DO_SVE2_ZPZZ_FP(FMINP, fminp) + +/* + * SVE Integer Multiply-Add (unpredicated) + */ + +static bool do_sqdmlal_zzzw(DisasContext *s, arg_rrrr_esz *a, + bool sel1, bool sel2) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_sqdmlal_zzzw_h, + gen_helper_sve2_sqdmlal_zzzw_s, gen_helper_sve2_sqdmlal_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], (sel2 << 1) | sel1); +} + +static bool do_sqdmlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, + bool sel1, bool sel2) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_sqdmlsl_zzzw_h, + gen_helper_sve2_sqdmlsl_zzzw_s, gen_helper_sve2_sqdmlsl_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], (sel2 << 1) | sel1); +} + +static bool trans_SQDMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlal_zzzw(s, a, false, false); +} + +static bool trans_SQDMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlal_zzzw(s, a, true, true); +} + +static bool trans_SQDMLALBT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlal_zzzw(s, a, false, true); +} + +static bool trans_SQDMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlsl_zzzw(s, a, false, false); +} + +static bool trans_SQDMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlsl_zzzw(s, a, true, true); +} + +static bool trans_SQDMLSLBT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlsl_zzzw(s, a, false, true); +} From patchwork Fri Apr 16 21:01:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422624 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp733822jaf; Fri, 16 Apr 2021 14:31:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwTXo4V6UCeBRLQM8A3mmKtIWnOy0H1gYq+oCQ1nwZwjpl67x7ia3jL22eGmyxQgSAVI8jT X-Received: by 2002:a9d:7512:: with SMTP id r18mr5426965otk.360.1618608689892; Fri, 16 Apr 2021 14:31:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608689; cv=none; d=google.com; s=arc-20160816; b=BospERrvJNOQWnQSb18XnMd2zECrOkmGflwKbL0UNFUQGykw0gfMTeaHf18Mi2a+9+ YGCBfM/9GiDMvCJWQTdZjMDPTYbZ5XU5lTv1/BX9Hi1Xyx6+4nB1PjR0fNUZDFW0qULA HDXsBt7F0qZK4d7dgDnr9BpI32OFQefCZ4RDFjB6rzhDiC+YIn8JtJhnncHlngXZrvzS yYYxmdjezsMfpVVodTm/bEO+851y9MzEi1Jt0PGNcjf4wl+Mq37KQVp06fiS1DWN9Pco Vcjxg5WKGAkBy9RNBW7iY3K1bushUzLseF29/knuwvJZfVn7/Mu5srbqmwwqdqEqdV+T /8mA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=HrscdamrL4xnD14xZwWaM1FKNhEcxGneRJZ//b9WX/E=; b=S4H+BfkP/nKW4DVkXjj4NPnB4YD46DoqmwtNkwprTyBa4W6Yi7enMX370avYCASpJ5 t3OF59SOOaMSuihptnxpmVWwze1iSMhs0rBy/QnOl0S373yPgXgyYPP9bvQUNIpaOPRZ Nnj/NPLrV03mIUlwRSsv362j4i1jldXBSMT0yji3oeAB8P750owAzsJZoToxQHRzhN6l 8bG4fa8YgfDtGrpS1PT9kmaKfcM1YXFZw4lm2QNlNAt+T5Ni6UNCE7ZK4kqVTs/dPRvJ P+RhLDbdESwEWn4UOGZCoay1pO3cnMFNQoof1SCfIrO9OjBityRAPlaFbtPYxI+hUeEg i8Bw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=v7GywaSK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 3si6528799oij.168.2021.04.16.14.31.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:31:29 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=v7GywaSK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44042 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW3l-0007MZ-95 for patch@linaro.org; Fri, 16 Apr 2021 17:31:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44380) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVco-0004yX-HF for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:38 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]:33451) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcT-0001KP-U7 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:36 -0400 Received: by mail-pl1-x634.google.com with SMTP id n10so3492259plc.0 for ; Fri, 16 Apr 2021 14:03:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HrscdamrL4xnD14xZwWaM1FKNhEcxGneRJZ//b9WX/E=; b=v7GywaSKzaeWtafFyRSU/qKsasVzPMf550Gr/rPU4Rn/zRQKYL3xTv2E+7Fatez8g1 gSiTO8rv0MthjUHSHA1QyA8Z/D39PEu84MyU30eGqwre6d/J1Rrsxk3kol0FrOcOqZfe 0XbSXwgaTpjNzdcj8NnPLfn+eK52Ei4p2Fi4MGbUAEUuyNAAMcxRaf2gLydv+WVk8SL6 7YeRKmGaByO4M5o22Ij0mzy6K616iC9y946MFutz2Yj0k9+v2hRyD0c6ghW4/UDKK02d ScF66qT6cnu7Mlm/rAq1HefOyv6sT06H+f1Cjgz3XlbVuiyXFMUm/r98UxDFYhFcNMei Q9uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HrscdamrL4xnD14xZwWaM1FKNhEcxGneRJZ//b9WX/E=; b=S7lFh1V4r73unZg3idH68x8ffvW4OtkVRJTx1EzQOVJbWmqU8Df01BwdzHuvpmlt5Y 6r4NnzRMyWeVZDa+1zCua1QTS5ucz/WgBNiptPo/tEYBHnT2bPW0FUAgFZOR3eXiBLRL upkIhxm8Luc7QFCdoY6AIeeLpgzzFH1PfRdnBs20K7qzK+Bpp+ZOnxSuaoXBgoLAo9Dc fU2zYtHYoh+pxYTsLHuMPxIbSVbI76ZWH9oHDBeD2qPYBTkNUBQGO8adPvBgJvl0wwHg LdlS9vlJtG/FVQkJ8dYnkA38n11zLNGTjwGnWtjAI+Naioj/9wa8Xg70Fxmxj9q51AXc /d8A== X-Gm-Message-State: AOAM531iyb3Bq4pm6UX6TkLzW6CzOhXmJJS6Y8+pY73YHVcS7ftFNXQ8 TVVYMmBUgHBBJeFmnE+Zb9X87dyToDsvBw== X-Received: by 2002:a17:902:106:b029:e8:de25:4a1d with SMTP id 6-20020a1709020106b02900e8de254a1dmr11460689plb.7.1618606996570; Fri, 16 Apr 2021 14:03:16 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:16 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 35/81] target/arm: Implement SVE2 saturating multiply-add high Date: Fri, 16 Apr 2021 14:01:54 -0700 Message-Id: <20210416210240.1591291-36-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" SVE2 has two additional sizes of the operation and unlike NEON, there is no saturation flag. Create new entry points for SVE2 that do not set QC. Signed-off-by: Richard Henderson --- target/arm/helper.h | 17 ++++ target/arm/sve.decode | 5 ++ target/arm/translate-sve.c | 18 +++++ target/arm/vec_helper.c | 161 +++++++++++++++++++++++++++++++++++-- 4 files changed, 195 insertions(+), 6 deletions(-) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 2c412ffd3b..6bb0b0ddc0 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -591,6 +591,23 @@ DEF_HELPER_FLAGS_5(gvec_qrdmlah_s32, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_qrdmlsh_s32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 52f615b39e..8308c9238a 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1346,3 +1346,8 @@ SQDMLSLT_zzzw 01000100 .. 0 ..... 0110 11 ..... ..... @rda_rn_rm SQDMLALBT 01000100 .. 0 ..... 00001 0 ..... ..... @rda_rn_rm SQDMLSLBT 01000100 .. 0 ..... 00001 1 ..... ..... @rda_rn_rm + +## SVE2 saturating multiply-add high + +SQRDMLAH_zzzz 01000100 .. 0 ..... 01110 0 ..... ..... @rda_rn_rm +SQRDMLSH_zzzz 01000100 .. 0 ..... 01110 1 ..... ..... @rda_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 27f9cdb891..4326b597e6 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7562,3 +7562,21 @@ static bool trans_SQDMLSLBT(DisasContext *s, arg_rrrr_esz *a) { return do_sqdmlsl_zzzw(s, a, false, true); } + +static bool trans_SQRDMLAH_zzzz(DisasContext *s, arg_rrrr_esz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_sqrdmlah_b, gen_helper_sve2_sqrdmlah_h, + gen_helper_sve2_sqrdmlah_s, gen_helper_sve2_sqrdmlah_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); +} + +static bool trans_SQRDMLSH_zzzz(DisasContext *s, arg_rrrr_esz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_sqrdmlsh_b, gen_helper_sve2_sqrdmlsh_h, + gen_helper_sve2_sqrdmlsh_s, gen_helper_sve2_sqrdmlsh_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index b0ce597060..c56337e724 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -22,6 +22,7 @@ #include "exec/helper-proto.h" #include "tcg/tcg-gvec-desc.h" #include "fpu/softfloat.h" +#include "qemu/int128.h" #include "vec_internal.h" /* Note that vector data is stored in host-endian 64-bit chunks, @@ -36,15 +37,55 @@ #define H4(x) (x) #endif +/* Signed saturating rounding doubling multiply-accumulate high half, 8-bit */ +static int8_t do_sqrdmlah_b(int8_t src1, int8_t src2, int8_t src3, + bool neg, bool round) +{ + /* + * Simplify: + * = ((a3 << 8) + ((e1 * e2) << 1) + (round << 7)) >> 8 + * = ((a3 << 7) + (e1 * e2) + (round << 6)) >> 7 + */ + int32_t ret = (int32_t)src1 * src2; + if (neg) { + ret = -ret; + } + ret += ((int32_t)src3 << 7) + (round << 6); + ret >>= 7; + + if (ret != (int8_t)ret) { + ret = (ret < 0 ? INT8_MIN : INT8_MAX); + } + return ret; +} + +void HELPER(sve2_sqrdmlah_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], a[i], false, true); + } +} + +void HELPER(sve2_sqrdmlsh_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], a[i], true, true); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */ static int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, bool neg, bool round, uint32_t *sat) { - /* - * Simplify: - * = ((a3 << 16) + ((e1 * e2) << 1) + (1 << 15)) >> 16 - * = ((a3 << 15) + (e1 * e2) + (1 << 14)) >> 15 - */ + /* Simplify similarly to do_sqrdmlah_b above. */ int32_t ret = (int32_t)src1 * src2; if (neg) { ret = -ret; @@ -133,11 +174,35 @@ void HELPER(neon_sqrdmulh_h)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(sve2_sqrdmlah_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], a[i], false, true, &discard); + } +} + +void HELPER(sve2_sqrdmlsh_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], a[i], true, true, &discard); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ static int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, bool neg, bool round, uint32_t *sat) { - /* Simplify similarly to int_qrdmlah_s16 above. */ + /* Simplify similarly to do_sqrdmlah_b above. */ int64_t ret = (int64_t)src1 * src2; if (neg) { ret = -ret; @@ -220,6 +285,90 @@ void HELPER(neon_sqrdmulh_s)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(sve2_sqrdmlah_s)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], a[i], false, true, &discard); + } +} + +void HELPER(sve2_sqrdmlsh_s)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], a[i], true, true, &discard); + } +} + +/* Signed saturating rounding doubling multiply-accumulate high half, 64-bit */ +static int64_t do_sat128_d(Int128 r) +{ + int64_t ls = int128_getlo(r); + int64_t hs = int128_gethi(r); + + if (unlikely(hs != (ls >> 63))) { + return hs < 0 ? INT64_MIN : INT64_MAX; + } + return ls; +} + +static int64_t do_sqrdmlah_d(int64_t n, int64_t m, int64_t a, + bool neg, bool round) +{ + uint64_t l, h; + Int128 r, t; + + /* As in do_sqrdmlah_b, but with 128-bit arithmetic. */ + muls64(&l, &h, m, n); + r = int128_make128(l, h); + if (neg) { + r = int128_neg(r); + } + if (a) { + t = int128_exts64(a); + t = int128_lshift(t, 63); + r = int128_add(r, t); + } + if (round) { + t = int128_exts64(1ll << 62); + r = int128_add(r, t); + } + r = int128_rshift(r, 63); + + return do_sat128_d(r); +} + +void HELPER(sve2_sqrdmlah_d)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], a[i], false, true); + } +} + +void HELPER(sve2_sqrdmlsh_d)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], a[i], true, true); + } +} + /* Integer 8 and 16-bit dot-product. * * Note that for the loops herein, host endianness does not matter From patchwork Fri Apr 16 21:01:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422641 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp741258jaf; Fri, 16 Apr 2021 14:45:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwyK2GbM/+TKYZC/lGHrrZY2L+sV/eAYaFrb2jIw/a5SUzVpVmwJrfb/HTrjC0R8Emh3OU/ X-Received: by 2002:a5d:89d2:: with SMTP id a18mr5300417iot.35.1618609541924; Fri, 16 Apr 2021 14:45:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609541; cv=none; d=google.com; s=arc-20160816; b=dyGrTa0oId/zFJqFI2qFxV8KNYuKRMqPCGlqQpd0S8xo8HyI/hx/AOkBZ6RGaPYoBD PTASKNo852mYAonMeo2oKs0YXRr0AHxxjCpcaDjR4+c5UJHVAUPV7ecLzoGC6shC54mC C4l5hkP1R+IAz0cZTd6Gzio8sV6LtvM7DXPVZ+y7fIXYUIOqzmWfl2r+JeUzTDCfklib U/oVwuDqB5Xf57PPeRZ7xjA+SzHtWPbxlH6WT8NdC0iZwaEMNq2RQpZlLq4twEa7er7v WehR24prfby5LPoXFCIOdQqH94xSrUL0MV5HAsRj2K3A8/L+FUl1go44vk50hliPzu7c bsRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=nJ0tUraTJVmXKzlkMts/F1yRCYj8V5rL50FUHOT1wnQ=; b=kFnmkNOfav/eLhNLhIBW//O02sQdmvNsiQrwnsHD2Nbe3nAVL1m+GSOSYKSUM9k5wm 7kLeS2au5o5gi0DFINBQ11BRVLCmtVU4Sd3B0oIvt5uZYuv6s7vDNfdn9O/uTLhisupO CYSw0JGnEvAZtW2BkzQuGeE7R7CQCtMREaN6n1nZ0RPlVN/QChAyZ3dj1PmIct9X40uu W50N9sTEHwHb173FJf3wA4KxFhzKR6Jev2Ydxw4CcPAAvGC15IagEX22TD1ATT43NtdY fVyZWtfd7/zyYC513w0zmLOVRCws0mcbEBrpB+DeUnziwvwThzbRIzE5VzkN+5Mn3XXy e1zg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=htqssexT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n13si4103310ioo.99.2021.04.16.14.45.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:45:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=htqssexT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55030 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWHV-0007Nh-A6 for patch@linaro.org; Fri, 16 Apr 2021 17:45:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44354) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcm-0004wl-Uh for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:37 -0400 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]:44602) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcV-0001Ko-6P for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:33 -0400 Received: by mail-pj1-x102b.google.com with SMTP id q14-20020a17090a430eb02901503aaee02bso4204519pjg.3 for ; Fri, 16 Apr 2021 14:03:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nJ0tUraTJVmXKzlkMts/F1yRCYj8V5rL50FUHOT1wnQ=; b=htqssexTKoC13nx8EVJXjOQbQbIfLR0RxK3Af+UynlXWByHg+MocJDMq/FUrrbv3PF v2e0bDs9Q27r7V3Na8B5mlPavJe8wHk/zkcoy1AIGLKROdwCxtP87w9tFby2/LvAf9mU jjANbCFPXfDHCKKgopIRPtSlkyofbZmmD67dfVz4ZOmHYd3A+1E8N4mqzkHzfDdHnEQ2 reS8IO5HmkUXLndUVNqKGWk9HEJtgngfwDye/xnxKHdgPO2SnFcHgjaNoJBX3Cs82eke vfRhGOTjsn6WS60tAZzFh8lZQMJvzbUrs3C5bYo0x1KsBf67qxtC2UjC2BhYM7tFIgWx xHmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nJ0tUraTJVmXKzlkMts/F1yRCYj8V5rL50FUHOT1wnQ=; b=Lyt6yubtqI1o5QhUbvHtc7lPSgFwM7EQPWVGBBZ7RdAMZKAxRgdxcurKy0FfpHbVqk EtcIQP6hs2sw8kJ5EXCAtxwTEgPzrsv6AIanvq4tfwrg574DOEYa7gxGRz8lZycVb7dk iysTkjjxKRAX7AvurAD221PuPqghXmnjCHHCFvSOkqZera0AOOQAipM3yjxm5Sj9wsiu LVUSzMiRYq33+9paYvTPRzXzb4cYUpa8W9Ovze3wESxETo17xGXFJea2qMNv7kXAJnW1 3es/nsnJiPi+O2jlZyTZ1H7gAUTxwil6bzx4/aPASrsIJg1hEzfrNw2iDC8vEj6veq+q u6tg== X-Gm-Message-State: AOAM533QKYJ/MZr+FvcQpirm12qz4cLFmwJt1ezhMyFi9kCyuxxrMhWR pU2k89+iullYl0cSxBuJMuBVmkqHh3hNjQ== X-Received: by 2002:a17:902:cec3:b029:eb:5441:9897 with SMTP id d3-20020a170902cec3b02900eb54419897mr11173730plg.48.1618606997816; Fri, 16 Apr 2021 14:03:17 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:17 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 36/81] target/arm: Implement SVE2 integer multiply-add long Date: Fri, 16 Apr 2021 14:01:55 -0700 Message-Id: <20210416210240.1591291-37-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102b; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 28 ++++++++++++++ target/arm/sve.decode | 11 ++++++ target/arm/sve_helper.c | 18 +++++++++ target/arm/translate-sve.c | 76 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 133 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index d8f390617c..457a421455 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2573,3 +2573,31 @@ DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_smlal_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlal_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_umlal_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlal_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_smlsl_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlsl_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlsl_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 8308c9238a..b28b50e05c 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1351,3 +1351,14 @@ SQDMLSLBT 01000100 .. 0 ..... 00001 1 ..... ..... @rda_rn_rm SQRDMLAH_zzzz 01000100 .. 0 ..... 01110 0 ..... ..... @rda_rn_rm SQRDMLSH_zzzz 01000100 .. 0 ..... 01110 1 ..... ..... @rda_rn_rm + +## SVE2 integer multiply-add long + +SMLALB_zzzw 01000100 .. 0 ..... 010 000 ..... ..... @rda_rn_rm +SMLALT_zzzw 01000100 .. 0 ..... 010 001 ..... ..... @rda_rn_rm +UMLALB_zzzw 01000100 .. 0 ..... 010 010 ..... ..... @rda_rn_rm +UMLALT_zzzw 01000100 .. 0 ..... 010 011 ..... ..... @rda_rn_rm +SMLSLB_zzzw 01000100 .. 0 ..... 010 100 ..... ..... @rda_rn_rm +SMLSLT_zzzw 01000100 .. 0 ..... 010 101 ..... ..... @rda_rn_rm +UMLSLB_zzzw 01000100 .. 0 ..... 010 110 ..... ..... @rda_rn_rm +UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 859091b7cf..8b86e7ecd6 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1308,6 +1308,24 @@ DO_ZZZW_ACC(sve2_uabal_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) DO_ZZZW_ACC(sve2_uabal_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) +DO_ZZZW_ACC(sve2_smlal_zzzw_h, int16_t, int8_t, H1_2, H1, DO_MUL) +DO_ZZZW_ACC(sve2_smlal_zzzw_s, int32_t, int16_t, H1_4, H1_2, DO_MUL) +DO_ZZZW_ACC(sve2_smlal_zzzw_d, int64_t, int32_t, , H1_4, DO_MUL) + +DO_ZZZW_ACC(sve2_umlal_zzzw_h, uint16_t, uint8_t, H1_2, H1, DO_MUL) +DO_ZZZW_ACC(sve2_umlal_zzzw_s, uint32_t, uint16_t, H1_4, H1_2, DO_MUL) +DO_ZZZW_ACC(sve2_umlal_zzzw_d, uint64_t, uint32_t, , H1_4, DO_MUL) + +#define DO_NMUL(N, M) -(N * M) + +DO_ZZZW_ACC(sve2_smlsl_zzzw_h, int16_t, int8_t, H1_2, H1, DO_NMUL) +DO_ZZZW_ACC(sve2_smlsl_zzzw_s, int32_t, int16_t, H1_4, H1_2, DO_NMUL) +DO_ZZZW_ACC(sve2_smlsl_zzzw_d, int64_t, int32_t, , H1_4, DO_NMUL) + +DO_ZZZW_ACC(sve2_umlsl_zzzw_h, uint16_t, uint8_t, H1_2, H1, DO_NMUL) +DO_ZZZW_ACC(sve2_umlsl_zzzw_s, uint32_t, uint16_t, H1_4, H1_2, DO_NMUL) +DO_ZZZW_ACC(sve2_umlsl_zzzw_d, uint64_t, uint32_t, , H1_4, DO_NMUL) + #undef DO_ZZZW_ACC #define DO_XTNB(NAME, TYPE, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 4326b597e6..0fdfd1e9e0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7580,3 +7580,79 @@ static bool trans_SQRDMLSH_zzzz(DisasContext *s, arg_rrrr_esz *a) }; return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); } + +static bool do_smlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_smlal_zzzw_h, + gen_helper_sve2_smlal_zzzw_s, gen_helper_sve2_smlal_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_SMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlal_zzzw(s, a, false); +} + +static bool trans_SMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlal_zzzw(s, a, true); +} + +static bool do_umlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_umlal_zzzw_h, + gen_helper_sve2_umlal_zzzw_s, gen_helper_sve2_umlal_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_UMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlal_zzzw(s, a, false); +} + +static bool trans_UMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlal_zzzw(s, a, true); +} + +static bool do_smlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_smlsl_zzzw_h, + gen_helper_sve2_smlsl_zzzw_s, gen_helper_sve2_smlsl_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_SMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlsl_zzzw(s, a, false); +} + +static bool trans_SMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlsl_zzzw(s, a, true); +} + +static bool do_umlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_umlsl_zzzw_h, + gen_helper_sve2_umlsl_zzzw_s, gen_helper_sve2_umlsl_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_UMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlsl_zzzw(s, a, false); +} + +static bool trans_UMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlsl_zzzw(s, a, true); +} From patchwork Fri Apr 16 21:01:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422645 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp743550jaf; Fri, 16 Apr 2021 14:50:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz9FpWOZavlPfTaGGAsHkfOiu3wL2MudB0v42DjlGhUo39rv3VW67QitxgoWO8n/0eobkvY X-Received: by 2002:a5e:a515:: with SMTP id 21mr5256950iog.164.1618609831877; Fri, 16 Apr 2021 14:50:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609831; cv=none; d=google.com; s=arc-20160816; b=07Vqn3Ngt5RCUwDFnlC6HcBGPR5EIDe/sdaotlSD1EEkXMm0AukLUiMXHEXkWt9Hk+ DEJtk06H6RFM3tqq1ROk9UYoywJhnUmm3UxC5zTsjvmsQsxtph0FbnFrCxXAJVT69eFs HEirIq2NqyYDNv76ELT3I0POOCouXgGBrUmQyIItK66c9AAtkZ4XrdMKjk2DEhFi0SVc llF9RYKk3AlxG3pchs4L4wLT88tnvv+bU+2P3cInCgdCnbOh75fdAMXFSORxuu8CUcfQ V1d7lKufINGSywqfiGGApU8DAEtzIGM0zx/jvLKPoXqC6NpA74Yi537Dn0Ncp2ipSK9o VdlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=W45GPMZZVBfCUkRw+wO8T3FyCeab3ADQB7NYg/wbXMs=; b=Quob+FWnocfwOCcGnSDlpmg3G086EwF6qMRm+nnv6dRTHJ/rVYOohFHQ/6GENw+2QC 4LhVWIdHfbP0xrBwedPeTZioarW6oZhnlaoa3E4jaCmfayfsuRPQANO/CshOfeaOc/lF 9MNVaKbrml5nS/qjlkvMuKxUehdvgsU3GLiNIoHwyVm8sw493Yyf+RMACjfQjk9oyU0q LqFuUGsh0u/dH+C7eK3qIkvZgqxK0Bi5fYkRQhIknD3AHu+k+kc904hA/03hp6+Ior8D DCv1hM+4dw/mEZKXTeIzZGTYwfJSlkBAEG20UdpvXrq05AX++ykWePpsV2Gyeb5Pk9Pg 60ag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="X//tLQZp"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q5si7199302jap.93.2021.04.16.14.50.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:50:31 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="X//tLQZp"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:39946 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWMB-0004Mx-9j for patch@linaro.org; Fri, 16 Apr 2021 17:50:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44392) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcp-0004zQ-RL for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:39 -0400 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]:54213) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcW-0001Lh-52 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:39 -0400 Received: by mail-pj1-x102f.google.com with SMTP id nk8so1510844pjb.3 for ; Fri, 16 Apr 2021 14:03:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=W45GPMZZVBfCUkRw+wO8T3FyCeab3ADQB7NYg/wbXMs=; b=X//tLQZph5I8XN8zp2pUcW9bQ87S2TuVEtp4GuEOOhsD5uGlZ47AxPDNuurBvDdKSW PZoGAZrr89D+biUy3/3ahXFI2Dg3R73o4xbCX0tsc8W+767h7OzbUbffZg1CoDHbbL8s BoV7962avGjiFjm/KMg5vfRSJHRFpFVJ11QAM1a6j9BAjJ8jNuEW2v4Vork78Wl9TBRl i74YS4eZdSJVdKHv86yKuDHymzN9y1kSDo2JlMtFxZTxUpUDG3TfRqAWwto7Q3Rekrpr DxLknScU1W/+t3tjCFyt/RHjltOa+nlfji3VaY450t/InNLkRIpF7w4YhZCfh7OXtCRQ MReA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=W45GPMZZVBfCUkRw+wO8T3FyCeab3ADQB7NYg/wbXMs=; b=XzHHY15jXJD7a8rm6nXUmQ7gWcOlcIJSaIpk1K92gB5m2mR7VRy7iTj0DNS9DCQnru Ezay2edqvxZsBqwGUT8ywyrwC/T8J2v/+KmviPeKkT6xGzWvFgrM+bOAG9sCGe59HcG1 yIpooQaOfhENAcfSQEZ5GbMYvIUQaA+/2nmBj5ZmJ2+vkbCTV9yidE5NzskIcXqrcgZK Q1ZU0kcROWq2CdBXQlgGzmutczGviw2Mv5upyNuKwH4NR8cndl42T7yI6yYFMqeb2RK2 hrfj17jFy1ppmo7IBeKkgXByjHUDI1Cb8RGvL+xd07jpkK5Gh2CzO5BWOn7pETWxyRW0 cYYg== X-Gm-Message-State: AOAM531tIREyW/vv9qBK4ZG4XputBr103mBtFldFyEhbn3bNVBMYzOnA N5BGBk1X1+zcFUAQXUzXdzw/44ciI+CdNQ== X-Received: by 2002:a17:90a:ee95:: with SMTP id i21mr12159726pjz.160.1618606998856; Fri, 16 Apr 2021 14:03:18 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:18 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 37/81] target/arm: Implement SVE2 complex integer multiply-add Date: Fri, 16 Apr 2021 14:01:56 -0700 Message-Id: <20210416210240.1591291-38-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix do_sqrdmlah_d (laurent desnogues) --- target/arm/helper-sve.h | 18 ++++++++++++++++ target/arm/vec_internal.h | 5 +++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 42 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 32 +++++++++++++++++++++++++++++ target/arm/vec_helper.c | 15 +++++++------- 6 files changed, 109 insertions(+), 8 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 457a421455..d154218452 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2601,3 +2601,21 @@ DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h index 0102547a10..200fb55909 100644 --- a/target/arm/vec_internal.h +++ b/target/arm/vec_internal.h @@ -168,4 +168,9 @@ static inline int64_t do_suqrshl_d(int64_t src, int64_t shift, return do_uqrshl_d(src, shift, round, sat); } +int8_t do_sqrdmlah_b(int8_t, int8_t, int8_t, bool, bool); +int16_t do_sqrdmlah_h(int16_t, int16_t, int16_t, bool, bool, uint32_t *); +int32_t do_sqrdmlah_s(int32_t, int32_t, int32_t, bool, bool, uint32_t *); +int64_t do_sqrdmlah_d(int64_t, int64_t, int64_t, bool, bool); + #endif /* TARGET_ARM_VEC_INTERNALS_H */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index b28b50e05c..936977eacb 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1362,3 +1362,8 @@ SMLSLB_zzzw 01000100 .. 0 ..... 010 100 ..... ..... @rda_rn_rm SMLSLT_zzzw 01000100 .. 0 ..... 010 101 ..... ..... @rda_rn_rm UMLSLB_zzzw 01000100 .. 0 ..... 010 110 ..... ..... @rda_rn_rm UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm + +## SVE2 complex integer multiply-add + +CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx +SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 8b86e7ecd6..572d41a26c 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1448,6 +1448,48 @@ DO_SQDMLAL(sve2_sqdmlsl_zzzw_d, int64_t, int32_t, , H1_4, #undef DO_SQDMLAL +#define DO_CMLA(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / sizeof(TYPE); \ + int rot = simd_data(desc); \ + int sel_a = rot & 1, sel_b = sel_a ^ 1; \ + bool sub_r = rot == 1 || rot == 2; \ + bool sub_i = rot >= 2; \ + TYPE *d = vd, *n = vn, *m = vm, *a = va; \ + for (i = 0; i < opr_sz; i += 2) { \ + TYPE elt1_a = n[H(i + sel_a)]; \ + TYPE elt2_a = m[H(i + sel_a)]; \ + TYPE elt2_b = m[H(i + sel_b)]; \ + d[H(i)] = OP(elt1_a, elt2_a, a[H(i)], sub_r); \ + d[H(i + 1)] = OP(elt1_a, elt2_b, a[H(i + 1)], sub_i); \ + } \ +} + +#define do_cmla(N, M, A, S) (A + (N * M) * (S ? -1 : 1)) + +DO_CMLA(sve2_cmla_zzzz_b, uint8_t, H1, do_cmla) +DO_CMLA(sve2_cmla_zzzz_h, uint16_t, H2, do_cmla) +DO_CMLA(sve2_cmla_zzzz_s, uint32_t, H4, do_cmla) +DO_CMLA(sve2_cmla_zzzz_d, uint64_t, , do_cmla) + +#define DO_SQRDMLAH_B(N, M, A, S) \ + do_sqrdmlah_b(N, M, A, S, true) +#define DO_SQRDMLAH_H(N, M, A, S) \ + ({ uint32_t discard; do_sqrdmlah_h(N, M, A, S, true, &discard); }) +#define DO_SQRDMLAH_S(N, M, A, S) \ + ({ uint32_t discard; do_sqrdmlah_s(N, M, A, S, true, &discard); }) +#define DO_SQRDMLAH_D(N, M, A, S) \ + do_sqrdmlah_d(N, M, A, S, true) + +DO_CMLA(sve2_sqrdcmlah_zzzz_b, int8_t, H1, DO_SQRDMLAH_B) +DO_CMLA(sve2_sqrdcmlah_zzzz_h, int16_t, H2, DO_SQRDMLAH_H) +DO_CMLA(sve2_sqrdcmlah_zzzz_s, int32_t, H4, DO_SQRDMLAH_S) +DO_CMLA(sve2_sqrdcmlah_zzzz_d, int64_t, , DO_SQRDMLAH_D) + +#undef do_cmla +#undef DO_CMLA + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0fdfd1e9e0..6e92abbd8f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7656,3 +7656,35 @@ static bool trans_UMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) { return do_umlsl_zzzw(s, a, true); } + +static bool trans_CMLA_zzzz(DisasContext *s, arg_CMLA_zzzz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_cmla_zzzz_b, gen_helper_sve2_cmla_zzzz_h, + gen_helper_sve2_cmla_zzzz_s, gen_helper_sve2_cmla_zzzz_d, + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot); + } + return true; +} + +static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_sqrdcmlah_zzzz_b, gen_helper_sve2_sqrdcmlah_zzzz_h, + gen_helper_sve2_sqrdcmlah_zzzz_s, gen_helper_sve2_sqrdcmlah_zzzz_d, + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot); + } + return true; +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index c56337e724..19006f50f7 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -38,8 +38,8 @@ #endif /* Signed saturating rounding doubling multiply-accumulate high half, 8-bit */ -static int8_t do_sqrdmlah_b(int8_t src1, int8_t src2, int8_t src3, - bool neg, bool round) +int8_t do_sqrdmlah_b(int8_t src1, int8_t src2, int8_t src3, + bool neg, bool round) { /* * Simplify: @@ -82,8 +82,8 @@ void HELPER(sve2_sqrdmlsh_b)(void *vd, void *vn, void *vm, } /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */ -static int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, - bool neg, bool round, uint32_t *sat) +int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, + bool neg, bool round, uint32_t *sat) { /* Simplify similarly to do_sqrdmlah_b above. */ int32_t ret = (int32_t)src1 * src2; @@ -199,8 +199,8 @@ void HELPER(sve2_sqrdmlsh_h)(void *vd, void *vn, void *vm, } /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ -static int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, - bool neg, bool round, uint32_t *sat) +int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, + bool neg, bool round, uint32_t *sat) { /* Simplify similarly to do_sqrdmlah_b above. */ int64_t ret = (int64_t)src1 * src2; @@ -321,8 +321,7 @@ static int64_t do_sat128_d(Int128 r) return ls; } -static int64_t do_sqrdmlah_d(int64_t n, int64_t m, int64_t a, - bool neg, bool round) +int64_t do_sqrdmlah_d(int64_t n, int64_t m, int64_t a, bool neg, bool round) { uint64_t l, h; Int128 r, t; From patchwork Fri Apr 16 21:01:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422620 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp731516jaf; Fri, 16 Apr 2021 14:27:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx6Ssk7k2Id+47E8ES/BRxRYIz59axyGZIanlX+cK6SwAaIEPtn/aOY3fxfH9Ngh5NX2foV X-Received: by 2002:a25:aa8d:: with SMTP id t13mr1550562ybi.114.1618608434931; Fri, 16 Apr 2021 14:27:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608434; cv=none; d=google.com; s=arc-20160816; b=CAUTcYkOcAS/HDk/b5OJaCup43UtiqmWxGwZMd9gewaOt0Y1lmPDEzSs4+WSzCN8tS 2w3Y0XmV2MWxIdjP4CUzofuMg9XmgCKSeQ2mhNpmy8jrcmr8XZ2/w8q7L9s25WjKNFzR 8la7qUrtr1b8iE8FGVd2a7leeRhK7l5VCqjmDaR39TCKFTbHHJTmOAOcsdjqk9PO5j0w WDerAeMot4N/+q3Ida8t6ge4USi6Cx3wSPF5drlZ/x8za61jc652cIB7GEY0DAQZSzKw GgMzNB6OmRMQlE81LYbGW5NQNZzOBRGWlWS3DlnmroNlT3LYl85ek0xrDCmZ4GfNI4yh KP7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=jc6bISAZEf9j9mwCzGsWa+M8kmkHcY8KolYL9qX1dMs=; b=GX8/gsG24HiqT6R0IO6aFGsE/uVYIQexQqpbWjoQEsfGi3MHZIvGWOEWIPLUyLpCOU y0RksIH+3s7H+dBx9olI/+5TIDbreAAz4RG7LVdTyd4w3ZvUKs4EEybAj94dNIfhSprE PXlBsZRmqtqGLZrQC3KpAtwp7iEv9TYvCIsbUT/7432tGa2n7yAWxfe+AS+SmL70t49R /6cRHPq7vhbYEK4o7UXvVkKGQ8amR242xjpA0XNmjzutUxeYGEiqO4JwAfl9RxsRMSSv 8vqGxBEMjQrR65aCNC1oVRbyI+a+DMctao24L6qBaFn7VYDsOYnrUCg/9G3tEleeEQFv yHqQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YNVkii9d; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k20si6185864ybf.60.2021.04.16.14.27.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:27:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YNVkii9d; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33906 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVze-00039A-Bo for patch@linaro.org; Fri, 16 Apr 2021 17:27:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44408) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVcr-00051U-0H for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:41 -0400 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]:43544) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcX-0001Ly-52 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:40 -0400 Received: by mail-pl1-x633.google.com with SMTP id u15so6083669plf.10 for ; Fri, 16 Apr 2021 14:03:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jc6bISAZEf9j9mwCzGsWa+M8kmkHcY8KolYL9qX1dMs=; b=YNVkii9d1xslxbLWPnwNd9vnPDI/EiCRV0bUGtO/DfdVTsxGvv/9/9pS9U+KfWEQiN hUPUcBwIR8waNXM6nfIpm7ayUrxl5lXZYxYTo0t64ukSS3CANcUcNrB7C+xqADMEdXZJ X31SsqRgnW1Nt40LW7dWA2/59N3ePhmsEMCftIrx4o+KnxoNVppI632Jtov/PKAZbAxx b+sa3Wm3XmxAHfaEalD6STN+EtgDfYti2IMJ0qw52004ngkBtJsqxyFqR8pUaj/I0IHD jm3CZySUwD5489wqwx6qKoTdIVE3lgBmEhxcgLW3reBeZPvWryXHjWcfPxdfsz5giu7u O3+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jc6bISAZEf9j9mwCzGsWa+M8kmkHcY8KolYL9qX1dMs=; b=PzZKKOUXi09l0Zx/V0vyBizd/DAsMixgnRyuBFtab4qVHtsKAf2N3dWBYLPWF9ze6M ricr0/WXoZRRN1nrFbJWhBVbGatFLYsa9OxrO1pArcbIA50hfSPCDc5e93GWiyZkUvPT xgbdHhLOhVoHsaaFmAe+DYL0Ee/JbBp82xyaOQqBreY2/8guSOD7dwf/lTYnuWoANEF7 nBCvITXvks3592KagVzgsI1nfdiHSE4yqTHOtb9LQ5aBTtw6Eue2to3OKn7fIHRsyn0O ulXEb3ScJHKCsaTzYL0GX/SRSTIVk7puXB2rT4YsXQL/lDsMvLlnQg1mhoyi9Z6Gcuwq 1xmQ== X-Gm-Message-State: AOAM5310zFW0SmEd1P020xgtf0B8ROcbUNUpz60Zds1YTjVat0Isy3NU IJHqG0gTSi7YKkbgt0iGlMCKVpmYnq3ZmQ== X-Received: by 2002:a17:90a:950c:: with SMTP id t12mr11384929pjo.135.1618606999866; Fri, 16 Apr 2021 14:03:19 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:19 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 38/81] target/arm: Implement SVE2 ADDHNB, ADDHNT Date: Fri, 16 Apr 2021 14:01:57 -0700 Message-Id: <20210416210240.1591291-39-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::633; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x633.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-2-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 36 ++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 13 +++++++++++++ 4 files changed, 62 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index d154218452..a369fd2391 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2509,6 +2509,14 @@ DEF_HELPER_FLAGS_3(sve2_uqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_addhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 936977eacb..72dd36a5c8 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1320,6 +1320,11 @@ UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr +## SVE2 integer add/subtract narrow high part + +ADDHNB 01000101 .. 1 ..... 011 000 ..... ..... @rd_rn_rm +ADDHNT 01000101 .. 1 ..... 011 001 ..... ..... @rd_rn_rm + ### SVE2 Character Match MATCH 01000101 .. 1 ..... 100 ... ..... 0 .... @pd_pg_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 572d41a26c..2dead1f056 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2112,6 +2112,42 @@ DO_SHRNT(sve2_uqrshrnt_d, uint64_t, uint32_t, , H1_4, DO_UQRSHRN_D) #undef DO_SHRNB #undef DO_SHRNT +#define DO_BINOPNB(NAME, TYPEW, TYPEN, SHIFT, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + i); \ + TYPEW mm = *(TYPEW *)(vm + i); \ + *(TYPEW *)(vd + i) = (TYPEN)OP(nn, mm, SHIFT); \ + } \ +} + +#define DO_BINOPNT(NAME, TYPEW, TYPEN, SHIFT, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + TYPEW mm = *(TYPEW *)(vm + HW(i)); \ + *(TYPEN *)(vd + HN(i + sizeof(TYPEN))) = OP(nn, mm, SHIFT); \ + } \ +} + +#define DO_ADDHN(N, M, SH) ((N + M) >> SH) + +DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) +DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) +DO_BINOPNB(sve2_addhnb_d, uint64_t, uint32_t, 32, DO_ADDHN) + +DO_BINOPNT(sve2_addhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_ADDHN) +DO_BINOPNT(sve2_addhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_ADDHN) +DO_BINOPNT(sve2_addhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_ADDHN) + +#undef DO_ADDHN + +#undef DO_BINOPNB + /* Fully general four-operand expander, controlled by a predicate. */ #define DO_ZPZZZ(NAME, TYPE, H, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6e92abbd8f..86f8a24b5b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7462,6 +7462,19 @@ static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +#define DO_SVE2_ZZZ_NARROW(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_h, \ + gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ + }; \ + return do_sve2_zzz_ool(s, a, fns[a->esz]); \ +} + +DO_SVE2_ZZZ_NARROW(ADDHNB, addhnb) +DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt) + static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) { From patchwork Fri Apr 16 21:01:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422619 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp730360jaf; Fri, 16 Apr 2021 14:24:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy0pNe9X7zliAni2KaS++zoBplmwAaUtJOFh6TQiKGp7uwuigEV+wtY+VExsuzR08qd7w9b X-Received: by 2002:a05:6638:2044:: with SMTP id t4mr6095870jaj.12.1618608295351; Fri, 16 Apr 2021 14:24:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608295; cv=none; d=google.com; s=arc-20160816; b=Jm2fO861KiGF55x9/n7z83vzfHP67a39ScevSZrPus6VbxjSrDg+OOouRFgOoIFrO2 1qcZLa6cjixNC+Ap3sZhbyYj6+Q+K1e8lGJlEZ/KXPcW48UeZMTybYp8phAkHtKbBCyQ TELNft5VVSirBM01yvLfNM2k83IEE71RWifc5kdI+w0smIlEwULiu/Y52MnqVmMts2lY 4xAFGWiuH4v+9Mpbr7AdOzeqBWKFvamyL+JcG7d2AyO7K1BJ8rGKHVceidaQthOpaMqx GkhAbRYN/gf3jqcBv6PZCv8RbchiCz5SsQpDw0V9lE+HSNVDokcWHzCUzZ8CfvnkkZxs mtjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=JohW9Zep0eTrt4XneckLJzEv6J8bwztIac2YTHSBK88=; b=zNpxxyPuaWIYxzxH4Dy0+sMES2Ehai6KDIb6oKUCDyO4Q7Bfs/jYKSSPpOve++jaOF B3UdeK+G5b0J1M1gN4nnWcnnJjnxpVoLBWdvisMdYs1lp8y28eBGv5+6ZNZnOIUqk/0w uyaY3q5HSxhnGqROKXfWdAFeDVaSYLiTx8TSeOEAzTc4at59UvKtX+5u1M7j9SLmOvmE tKNNRb3nnBbLX/kRvJl6f6wNwmoCv4YMLluzljFDfZyyb6mERknF3U7BSXWq/zF/BqhB eSjxm0fW72fGYv208DD8aOsTGaGJgfFpUyjXo2XPDLIYkGXjvYH/S9rSuhs1jNlgJOsk 9NQQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y6CsIIna; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m4si6858049jap.7.2021.04.16.14.24.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:24:55 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y6CsIIna; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53896 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXVxO-00089G-M3 for patch@linaro.org; Fri, 16 Apr 2021 17:24:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44430) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVct-00057D-21 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:43 -0400 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]:40501) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVcY-0001N2-CM for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:42 -0400 Received: by mail-pl1-x62e.google.com with SMTP id 20so10681877pll.7 for ; Fri, 16 Apr 2021 14:03:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JohW9Zep0eTrt4XneckLJzEv6J8bwztIac2YTHSBK88=; b=Y6CsIInaIc6UaX+riQKIDV0KoWgZ+svJNUkMxqvpVwOXnFvuU7cu06IH16a06teARG /LYuoRIzhyXsHXOARCvo9ojodyRIMpucYLr2VzWdmUuCrC64J/4pqRq15VgEIN5b3ibj oH9ffk1oi0mCy/UObrKPDHkKOYylCpMZAj/Uxt5r2wUC7g5k1TvKxyRdE78cEEEl25Hs jfrKFrtZBemILxRyicIcRRTHJLzhFe4CRRMxZEKnFxyMnwPijyH+gv47unUsfR7QMNPe 7WsWNO/VatxA5ypUhGmWTlsfu6m8+xuAVad2vi4oHZV3X8PECsdiKyfGGDWC3ssyETnU 1eNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JohW9Zep0eTrt4XneckLJzEv6J8bwztIac2YTHSBK88=; b=Y1IvzQCx/B1eHmg+lUNAw+o1fR57/rHeoB7JdZHUPEXJKVU9EXIeT7xT2QKrs1PhLj qKv9Yw4CCxqJ8l7+r27I8tcQdqsWkf7ZOuJs8rfjS1wxePbyKipjJA63rW/PFk4sL9Ha zIZVxRfDb+B2fSgKh0tt/nFxSbeLktMHm5NAsWEjbfDKHTMThMRdyYNBXQuAPUtNi+eo KTKcrKCmGUcx84v+G0ALEgkFqtnqMt2j7GSjSDRY+D9KuDcb37l5QYCgDVyhjhSsuJ1H AKA2U05lIT4NDzDt9owoLnuU7KM/WTfaP8p3xNSEuPHhoeeU/7DNQM95GFQpyFUVc8y3 ysHw== X-Gm-Message-State: AOAM530/b9tJum3xAti3tEmJ8IFonXPN/CKelKOci4QlMaYupiboAhix i6WD7DmgtmbK4bLpIb9Oj+yitar+9gOJKg== X-Received: by 2002:a17:90a:cc0d:: with SMTP id b13mr11555262pju.219.1618607000878; Fri, 16 Apr 2021 14:03:20 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m15sm5920640pjz.36.2021.04.16.14.03.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:20 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 39/81] target/arm: Implement SVE2 RADDHNB, RADDHNT Date: Fri, 16 Apr 2021 14:01:58 -0700 Message-Id: <20210416210240.1591291-40-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-3-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fix round bit type (laurent desnogues) --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 10 ++++++++++ target/arm/translate-sve.c | 2 ++ 4 files changed, 22 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index a369fd2391..8d95c87694 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2517,6 +2517,14 @@ DEF_HELPER_FLAGS_4(sve2_addhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_addhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_addhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_raddhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 72dd36a5c8..dfcfab4bc0 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1324,6 +1324,8 @@ UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr ADDHNB 01000101 .. 1 ..... 011 000 ..... ..... @rd_rn_rm ADDHNT 01000101 .. 1 ..... 011 001 ..... ..... @rd_rn_rm +RADDHNB 01000101 .. 1 ..... 011 010 ..... ..... @rd_rn_rm +RADDHNT 01000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm ### SVE2 Character Match diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 2dead1f056..e6f6e3d5fa 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2135,6 +2135,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ } #define DO_ADDHN(N, M, SH) ((N + M) >> SH) +#define DO_RADDHN(N, M, SH) ((N + M + ((__typeof(N))1 << (SH - 1))) >> SH) DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) @@ -2144,6 +2145,15 @@ DO_BINOPNT(sve2_addhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_ADDHN) DO_BINOPNT(sve2_addhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_ADDHN) DO_BINOPNT(sve2_addhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_ADDHN) +DO_BINOPNB(sve2_raddhnb_h, uint16_t, uint8_t, 8, DO_RADDHN) +DO_BINOPNB(sve2_raddhnb_s, uint32_t, uint16_t, 16, DO_RADDHN) +DO_BINOPNB(sve2_raddhnb_d, uint64_t, uint32_t, 32, DO_RADDHN) + +DO_BINOPNT(sve2_raddhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_RADDHN) +DO_BINOPNT(sve2_raddhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_RADDHN) +DO_BINOPNT(sve2_raddhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_RADDHN) + +#undef DO_RADDHN #undef DO_ADDHN #undef DO_BINOPNB diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 86f8a24b5b..af0d0ab279 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7474,6 +7474,8 @@ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ DO_SVE2_ZZZ_NARROW(ADDHNB, addhnb) DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt) +DO_SVE2_ZZZ_NARROW(RADDHNB, raddhnb) +DO_SVE2_ZZZ_NARROW(RADDHNT, raddhnt) static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) From patchwork Fri Apr 16 21:01:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422629 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp736240jaf; Fri, 16 Apr 2021 14:35:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxHW0lZ5rET0c9gWP2RiKHKUcE/8zgo4nc3W6qslAZ/qwrVJHhpJRGfbcmLXQQFyTWhXcP2 X-Received: by 2002:a92:c26a:: with SMTP id h10mr8440980ild.294.1618608936856; Fri, 16 Apr 2021 14:35:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608936; cv=none; d=google.com; s=arc-20160816; b=KE0XuLJGwyo9XeH/qsv5S5A/7kpAzVZdDgHsmbrw6z5Pkr76n/o7Y3yc7XWSHLzzx+ MDcnvVKUv7SwQVjbMLxKIyj+oO0d0eI6wVpF9WpyLxsvQ/hTB/QWH5ONmgaojg/x9e9Y S1gtDIlu5nsRdym2fdIAUv4BrdiKzdMjEINcQNf5/V7M9QUdkL25TjyJ68a5wQVaat0C loprQmd14g+lT9f0skzu7nZ1CEe6Pw5Z/wI4qk5lyvhL8y14h6G2NUDPo/G+hgZSNZ8I NneFrenMWR67srQx7PBSbvHLuXI0SgubycIzI07C+nQtOefchuNjtmsR/9dBlZJ0qLzk EdCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ieCAX8JfYanxGz+tnZF3xTMiEcjdmHvb76wqgRW4ePI=; b=ZRSCbV3tfHVdKfZcXWT7CSM3f0Su3eu4T1slW31yNhwsYXpKiIicPsekVlpZImtbjV gzFI1mdn7e4tlSFYr+ppDNRujOj6P0i7E1U8qCIXb0TIOKO6tRLwh2LLAVl0l2+pzwbJ cKR2MFroIn5TFeSFMjxRQQCMO2EC0Z/DfLx2inEzLjwvkMeFYwzO6aJSHnlTY066l25a /acLOMhiXOjRZ2t03CFjEY9QPIZF+A4rcUMAhPIAk1PsYl9Pm8yNer1bZaA6kh613LLx iRXBcnBC0jqAW6sz7PreBx9m5xBmHdRJeSnbc2YMEw9yRIJXboIPI2vBtQ9jovy/HcXZ uFHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xdZMRusX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m22si6345978jat.1.2021.04.16.14.35.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:35:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xdZMRusX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53830 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW7k-0003BD-8K for patch@linaro.org; Fri, 16 Apr 2021 17:35:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44512) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVd5-0005ZG-FI for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:55 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]:34735) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVd3-0001bD-GJ for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:55 -0400 Received: by mail-pl1-x62c.google.com with SMTP id t22so14310420ply.1 for ; Fri, 16 Apr 2021 14:03:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ieCAX8JfYanxGz+tnZF3xTMiEcjdmHvb76wqgRW4ePI=; b=xdZMRusX1PtjF6oKymwNVTB257At55rxFGyN13xcQ69I1W36FiptLUZbYCJK05R+gA x8YIqmX9rNbPsugHAdnu2LzHkghHh0wtFWUsof3DXZ6/29tXzG/5ii7mQ7gtFfYwtO+K mVnCC1/+pxbY5j9XCnttvMVv9IZHbGFJ90iK1zE/4ivvrB9v7V9/qB0r59LOZoxde0ki 6/GgK2Snv8NKDlirckCPU4B9mv9zkxhod3HDcDmdG0+tYQSsIQWgg4a7HiJh+2Yfx5Mg dfWv5acGMpJuB7zv+dsat5o99O0r/HgolQHn4RdJ5N7Geso/b3UWGMe8BK+imubGrOay sr1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ieCAX8JfYanxGz+tnZF3xTMiEcjdmHvb76wqgRW4ePI=; b=YugrCKYxubP8ZkY9nkYb7f33Yz47JGxghhBC/QhlAbiXDuujP8HijSIaM7k1z4vynd Qqwz1GzUYLoupX00yyWhyJw7tT08tfRcqAxftfkRwBq0aw8bzfdYtfBHiG1E3BgZwrPu uT3hBtJw6wOrWc1Vo8j6h5gJM73lu2COEh5o5PD+I036jMxrYnL74wkojrqSruW8FURJ mWY9vjNCAtEDtwop5oo6kT/O+5QblXMaMVs+zOlDd9QZAGbRrpxfojaelp/ACahR2w4I 6mqVaFu1SsWjn2quk2fnjFUiLANkCVg4ri+uyOzdeEgmZcraoZuwmqunJCLS/XictlVP a46g== X-Gm-Message-State: AOAM531vXeOVDd2WxbtBbdQKsij/g5pQnrxS7ZrfT23oU9nXub8SCCKq 0FIV4hyBlDf9HhfP/nSnj488Z7IJjVkhfQ== X-Received: by 2002:a17:902:d4c2:b029:ea:deb1:dfd5 with SMTP id o2-20020a170902d4c2b02900eadeb1dfd5mr11429096plg.79.1618607032221; Fri, 16 Apr 2021 14:03:52 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.03.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:51 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 40/81] target/arm: Implement SVE2 SUBHNB, SUBHNT Date: Fri, 16 Apr 2021 14:01:59 -0700 Message-Id: <20210416210240.1591291-41-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-4-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 10 ++++++++++ target/arm/translate-sve.c | 3 +++ 4 files changed, 23 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 8d95c87694..3642e7c820 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2525,6 +2525,14 @@ DEF_HELPER_FLAGS_4(sve2_raddhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_raddhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_raddhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_subhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index dfcfab4bc0..c68bfcf6ed 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1326,6 +1326,8 @@ ADDHNB 01000101 .. 1 ..... 011 000 ..... ..... @rd_rn_rm ADDHNT 01000101 .. 1 ..... 011 001 ..... ..... @rd_rn_rm RADDHNB 01000101 .. 1 ..... 011 010 ..... ..... @rd_rn_rm RADDHNT 01000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm +SUBHNB 01000101 .. 1 ..... 011 100 ..... ..... @rd_rn_rm +SUBHNT 01000101 .. 1 ..... 011 101 ..... ..... @rd_rn_rm ### SVE2 Character Match diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index e6f6e3d5fa..0df70effe3 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2136,6 +2136,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ #define DO_ADDHN(N, M, SH) ((N + M) >> SH) #define DO_RADDHN(N, M, SH) ((N + M + ((__typeof(N))1 << (SH - 1))) >> SH) +#define DO_SUBHN(N, M, SH) ((N - M) >> SH) DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) @@ -2153,6 +2154,15 @@ DO_BINOPNT(sve2_raddhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_RADDHN) DO_BINOPNT(sve2_raddhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_RADDHN) DO_BINOPNT(sve2_raddhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_RADDHN) +DO_BINOPNB(sve2_subhnb_h, uint16_t, uint8_t, 8, DO_SUBHN) +DO_BINOPNB(sve2_subhnb_s, uint32_t, uint16_t, 16, DO_SUBHN) +DO_BINOPNB(sve2_subhnb_d, uint64_t, uint32_t, 32, DO_SUBHN) + +DO_BINOPNT(sve2_subhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_SUBHN) +DO_BINOPNT(sve2_subhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_SUBHN) +DO_BINOPNT(sve2_subhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_SUBHN) + +#undef DO_SUBHN #undef DO_RADDHN #undef DO_ADDHN diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index af0d0ab279..55303ba41d 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7477,6 +7477,9 @@ DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt) DO_SVE2_ZZZ_NARROW(RADDHNB, raddhnb) DO_SVE2_ZZZ_NARROW(RADDHNT, raddhnt) +DO_SVE2_ZZZ_NARROW(SUBHNB, subhnb) +DO_SVE2_ZZZ_NARROW(SUBHNT, subhnt) + static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) { From patchwork Fri Apr 16 21:02:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422632 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp737313jaf; Fri, 16 Apr 2021 14:37:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxugFIuX7x6wSpi1wvItqCagfY/AXtCY9OUhA1FNHm3qJ82aHrI5R9t7fhQmysv5yu9Uvak X-Received: by 2002:a92:c203:: with SMTP id j3mr7985336ilo.224.1618609071564; Fri, 16 Apr 2021 14:37:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609071; cv=none; d=google.com; s=arc-20160816; b=FDYRvkNorszLrVpfa95gU86z6Zmx0AoAN62Jzukfw+KKjZ3k9UviyLp+CRa3ppa+/r 3PtEPk2JAYsbt/ciLYXIbikRa9/cBMWU4EkALTojsGdMdYJgaxQxkqiGavLkcxjOYf2s KHp8r6pfe8BQECE5OeW69nDM3JwBOwq/ohg/LUwmnWbtVr3Wkp9jnNWHuYaL6fIDPjLp y26BJjHGOAhvQxRJTf/40Js3dsfF9RsGlx784Pu2WmP6OC4JiHjejXiM51R2NBldynDX px5JfB1Hkmwn0km2sF1NZhZrp16V9xjiCU5FG+byUyg+MbWI6qtvq8lwhyDFhl5ZKAna NHvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=L9CT6IY88WuROz45EJLGs6Vb8eBbIgK6UvA8l1eIoaI=; b=0lQn5XgVtedIKm9rpC9G2ZXNzztTSD35m3KEcDZEMND9+Aa/zPugSdso6vrmno/nkT j9789dbd//FMRLM70ftAYdwnMCoI6vr8PrHVeHLcTXqfTm8trXha4QOgoPO4aDeZnmuw AB1NqQ21M6bs0od2+b0L7dR9tadU9p2oryrCXLPq8SSxA/QOIh7heApRIDbaHdC4WLTC 5C6Rc0XX0AG2fQiP4DfWFK5YxMg7mCSuPlcYxcVG0JpfbfMmeUtzNZfdtKfQ81vzJepn p5PBBZfmXxh8d8EbVp6BGDs0C46FsRs/OohJhUn1x+9+tg0pRHiBKAlAMa7k/xgisA/8 01nQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Nv/6aq/J"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id u4si7017078jad.126.2021.04.16.14.37.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:37:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Nv/6aq/J"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34116 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW9v-0006yS-21 for patch@linaro.org; Fri, 16 Apr 2021 17:37:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44548) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVd6-0005ch-Rg for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:56 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]:44973) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVd4-0001bY-Kn for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:56 -0400 Received: by mail-pf1-x430.google.com with SMTP id m11so19141317pfc.11 for ; Fri, 16 Apr 2021 14:03:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L9CT6IY88WuROz45EJLGs6Vb8eBbIgK6UvA8l1eIoaI=; b=Nv/6aq/JsBX2uU6uzk7lJU/N0iH5/b8rtewahxQoRiWylo+OXbl/Ogzt5+NHNvMTJU YEJS+kwhWuoNhikD/spfsEVdPVFeg0e6SQDuGty6Y0vlTp5B8b5cBSZApoupehyfiHhW 4jIq2YVzow2UNKxxHfREAIxdnV68GdBdC1d58GhvloVUk6286YKuNfsWn2y/Lw/5Tx+Z I8M9Q6cZ3MJWh+FkZtCbG0yzMph4op657IYwi40bOMtT0haDTzeZTcmOkSU2IE6ZpqRB tgMkkbTd34KBTa8dbqTYNg6EbwPxHjWk9Ug9cdcSwMiqcFmJCQaTPCQSIiksNIk/clW+ zmQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L9CT6IY88WuROz45EJLGs6Vb8eBbIgK6UvA8l1eIoaI=; b=cqliaVmHy9xHws4SJXkBKptkSO0oSC9xHZtUi9WpnKTLMHyBUczbMX6FbXYOnJ2Ze+ msMYuep18D8ROITU/fuS0D0o/2TRKrhIdrymo7RDf8o/1Q4MWCMDIt7YnbT40xMEC3Xo tMordeMu5sDa9Qmd+kHkLuQx87Yoy9nDHJh5G39ztiv+jLU+DJcoD296j020JmfpL46J qwmVjiC8XrQ5JqQjCPXMm9bpECBu1prrXaF94VYmx3rCc0S1A/3akICSEC5eZKwdEeEd 4111gU0vm+I3nrI06TSHHeFE6PUUaag+JkdCMg1/aNRew8t12m/H7oQvgGy5SB7bGIRJ MxNA== X-Gm-Message-State: AOAM532GXqkLBzAECFQ6UitzTo4RHE/9bKRxrE2RosIr0/KIkqWo59C1 QDfn4zgtf9N4yLHYjfu0BWa/GvAQ7dy2TA== X-Received: by 2002:a62:52c7:0:b029:255:e78e:5069 with SMTP id g190-20020a6252c70000b0290255e78e5069mr9181267pfb.45.1618607033307; Fri, 16 Apr 2021 14:03:53 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.03.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 41/81] target/arm: Implement SVE2 RSUBHNB, RSUBHNT Date: Fri, 16 Apr 2021 14:02:00 -0700 Message-Id: <20210416210240.1591291-42-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long This completes the section 'SVE2 integer add/subtract narrow high part' Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-5-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fix round bit type (laurent desnogues) --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 10 ++++++++++ target/arm/translate-sve.c | 2 ++ 4 files changed, 22 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 3642e7c820..98e6b57e38 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2533,6 +2533,14 @@ DEF_HELPER_FLAGS_4(sve2_subhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_subhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_subhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_rsubhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index c68bfcf6ed..388bf92acf 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1328,6 +1328,8 @@ RADDHNB 01000101 .. 1 ..... 011 010 ..... ..... @rd_rn_rm RADDHNT 01000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm SUBHNB 01000101 .. 1 ..... 011 100 ..... ..... @rd_rn_rm SUBHNT 01000101 .. 1 ..... 011 101 ..... ..... @rd_rn_rm +RSUBHNB 01000101 .. 1 ..... 011 110 ..... ..... @rd_rn_rm +RSUBHNT 01000101 .. 1 ..... 011 111 ..... ..... @rd_rn_rm ### SVE2 Character Match diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 0df70effe3..12acc4fb0b 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2137,6 +2137,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ #define DO_ADDHN(N, M, SH) ((N + M) >> SH) #define DO_RADDHN(N, M, SH) ((N + M + ((__typeof(N))1 << (SH - 1))) >> SH) #define DO_SUBHN(N, M, SH) ((N - M) >> SH) +#define DO_RSUBHN(N, M, SH) ((N - M + ((__typeof(N))1 << (SH - 1))) >> SH) DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) @@ -2162,6 +2163,15 @@ DO_BINOPNT(sve2_subhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_SUBHN) DO_BINOPNT(sve2_subhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_SUBHN) DO_BINOPNT(sve2_subhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_SUBHN) +DO_BINOPNB(sve2_rsubhnb_h, uint16_t, uint8_t, 8, DO_RSUBHN) +DO_BINOPNB(sve2_rsubhnb_s, uint32_t, uint16_t, 16, DO_RSUBHN) +DO_BINOPNB(sve2_rsubhnb_d, uint64_t, uint32_t, 32, DO_RSUBHN) + +DO_BINOPNT(sve2_rsubhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_RSUBHN) +DO_BINOPNT(sve2_rsubhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_RSUBHN) +DO_BINOPNT(sve2_rsubhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_RSUBHN) + +#undef DO_RSUBHN #undef DO_SUBHN #undef DO_RADDHN #undef DO_ADDHN diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 55303ba41d..49d7a45a50 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7479,6 +7479,8 @@ DO_SVE2_ZZZ_NARROW(RADDHNT, raddhnt) DO_SVE2_ZZZ_NARROW(SUBHNB, subhnb) DO_SVE2_ZZZ_NARROW(SUBHNT, subhnt) +DO_SVE2_ZZZ_NARROW(RSUBHNB, rsubhnb) +DO_SVE2_ZZZ_NARROW(RSUBHNT, rsubhnt) static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) From patchwork Fri Apr 16 21:02:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422626 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp733946jaf; Fri, 16 Apr 2021 14:31:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxDVyYEt+aHWUdr5hL6BHTL6UFJBdEfFUBem5w/HPuKAASvGylPOCS+46evy54y2FCBL8oK X-Received: by 2002:a9d:5a9b:: with SMTP id w27mr5458465oth.146.1618608702661; Fri, 16 Apr 2021 14:31:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608702; cv=none; d=google.com; s=arc-20160816; b=IHqO7toWExqlPuZdBIGfhmupEO8LE52QDXTMihpPk8YnArrlwBAooVH1wUym9dLOy0 gz30V1OkkjIE4f74lMDz/CTTW5/tpJklFPk2krm+cnQ4x8mIUcKjeqJibN0HPkQ15ahg vFiEfn1NPF9C0Ik18wJP1UJvv0gCcQutrrb5AAi0pMCZgXZWGH+txYSS99XT16cxIlUM mP4/8neVd9zsHmFeX5Ctrf5YVQNuYmk0jIL0QiP3gdEEP59Gsy7SF0rAeWOd2eJIl+uO LU0o04YKN5VrUuYVhpwYgLIAtXvh3q8GVg0d7fsQwvDD3TIr6ZQzOoy/Z10LiG5K//vL X36Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=SGOOa0zT0FE1S29kkqgncHYFmVqbSwpz80PVFtDGE1o=; b=1Eius51/BZDh0svSNMH91RJ+EhKkdYciMASXLjc0n6dHTOBStpUmL+n0VvYd5H590c ps8eh0vEmdHXpuowvh71GPnTCwaIAJ++HjONKvH3a9IT6UlrwmTzAHoWkeds5GdBQsxY M3JUxwWyFTTgw+XSqP4aniwxjrazHmPO0EDZUlLK0OZMJNpBH4hmqmcjZS+jeYPqfVI0 KkRUCI//1Tpc8dGAA7G4knSCt2BVgA6uKCG9yxgwmu7VZ2gRLiDw33uY2DZAJ59KbRnT 9dfQF4cK2WovSpzKnzly0TzV0xlu4hdX88G8MJgKYVtotchY/tlky+ym8sm8fOwqFNR6 mmwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CDZPdi8B; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o16si7820110oik.151.2021.04.16.14.31.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:31:42 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CDZPdi8B; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44954 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW3y-0007k2-0e for patch@linaro.org; Fri, 16 Apr 2021 17:31:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44620) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVd9-0005f1-MR for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:59 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]:34415) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVd5-0001cj-Pw for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:03:59 -0400 Received: by mail-pf1-x432.google.com with SMTP id 10so10257834pfl.1 for ; Fri, 16 Apr 2021 14:03:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SGOOa0zT0FE1S29kkqgncHYFmVqbSwpz80PVFtDGE1o=; b=CDZPdi8BdgOl/Hzv8n2SJ0mxWsZgKydcpMIFMV6Uc6dm+blpdfBJtzNdBdzhLf+QPQ mEniCCsIHezCKXBCvhOuZPXDVmU5uaQkB8dO95q6Pn+vVKugKngx8set92/tkZgAyWdb k73dH1BqFzjVEkeosk52TohEnDIdQvDcQ9p90i7c/VGOxFKXZ/e+hvxhfNmCLersUYUE ObVKqokTgWUIZzraYck5M2KQ0mEFJfnEDLU5E3uqZeSofXbQSLRnpSjppPn0rEsxZWht mqRRFbMKkLNZjlMQTEv0/m9RrWIY8MH5JA0Aady/MabARXklzWfiw+Jc9XjuqwUMXgad xvpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SGOOa0zT0FE1S29kkqgncHYFmVqbSwpz80PVFtDGE1o=; b=rf1N8mLFO8UpSb+gpb5dp3YdULLNy5az65VxGxHiVXL000bPsRnhphuwcoqYoqfY8v SzjBn7HknlmeXRiOmP/TUFzED4T5pOrlJAKMJkML4ZWo2p4ROS/Dsmt36//KFey0Yx05 2Pf23P+8jYS+zv8lPuD58WRZJoKFBhOlaGYFy7Gjr42Cty1Sh9fTbP2d1H9ZsZmdmTyh Wj4qRWN+wUOUNYaTZaTU/jdGzOWZc+zTwsJwmc2Q7aJxDo7Wp/P5K+hG0yvp25/Je+E7 ZAkvP6dwzbqR3xwY6pIxukm1caGbPmUHJ+uI6QCJdNrMZO72hbwDd0fOqcNl+6b9EXQn lTJA== X-Gm-Message-State: AOAM532QmUP+j7tU7ATRk6hh2FwInWCVTh7ZkORWtT2mxtqdi7DR91vb 8E0Jr4/aFuMWKqi1tOwflpDEKEcdY+CZLA== X-Received: by 2002:aa7:84c7:0:b029:259:d7ed:a8dd with SMTP id x7-20020aa784c70000b0290259d7eda8ddmr5030310pfn.20.1618607034253; Fri, 16 Apr 2021 14:03:54 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.03.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 42/81] target/arm: Implement SVE2 HISTCNT, HISTSEG Date: Fri, 16 Apr 2021 14:02:01 -0700 Message-Id: <20210416210240.1591291-43-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200416173109.8856-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fix overlap between output and input vectors. v4: Fix histseg counting (zhiwei). --- target/arm/helper-sve.h | 7 ++ target/arm/sve.decode | 6 ++ target/arm/sve_helper.c | 131 +++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 19 ++++++ 4 files changed, 163 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 98e6b57e38..507a2fea8e 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2551,6 +2551,13 @@ DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_b, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_h, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_histcnt_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_histcnt_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_histseg, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 388bf92acf..8f501a083c 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -146,6 +146,7 @@ &rprrr_esz rn=%reg_movprfx @rdn_pg_rm_ra ........ esz:2 . ra:5 ... pg:3 rm:5 rd:5 \ &rprrr_esz rn=%reg_movprfx +@rd_pg_rn_rm ........ esz:2 . rm:5 ... pg:3 rn:5 rd:5 &rprr_esz # One register operand, with governing predicate, vector element size @rd_pg_rn ........ esz:2 ... ... ... pg:3 rn:5 rd:5 &rpr_esz @@ -1336,6 +1337,11 @@ RSUBHNT 01000101 .. 1 ..... 011 111 ..... ..... @rd_rn_rm MATCH 01000101 .. 1 ..... 100 ... ..... 0 .... @pd_pg_rn_rm NMATCH 01000101 .. 1 ..... 100 ... ..... 1 .... @pd_pg_rn_rm +### SVE2 Histogram Computation + +HISTCNT 01000101 .. 1 ..... 110 ... ..... ..... @rd_pg_rn_rm +HISTSEG 01000101 .. 1 ..... 101 000 ..... ..... @rd_rn_rm + ## SVE2 floating-point pairwise operations FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 12acc4fb0b..8d002fdb65 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7062,3 +7062,134 @@ DO_PPZZ_MATCH(sve2_nmatch_ppzz_b, MO_8, true) DO_PPZZ_MATCH(sve2_nmatch_ppzz_h, MO_16, true) #undef DO_PPZZ_MATCH + +void HELPER(sve2_histcnt_s)(void *vd, void *vn, void *vm, void *vg, + uint32_t desc) +{ + ARMVectorReg scratch; + intptr_t i, j; + intptr_t opr_sz = simd_oprsz(desc); + uint32_t *d = vd, *n = vn, *m = vm; + uint8_t *pg = vg; + + if (d == n) { + n = memcpy(&scratch, n, opr_sz); + if (d == m) { + m = n; + } + } else if (d == m) { + m = memcpy(&scratch, m, opr_sz); + } + + for (i = 0; i < opr_sz; i += 4) { + uint64_t count = 0; + uint8_t pred; + + pred = pg[H1(i >> 3)] >> (i & 7); + if (pred & 1) { + uint32_t nn = n[H4(i >> 2)]; + + for (j = 0; j <= i; j += 4) { + pred = pg[H1(j >> 3)] >> (j & 7); + if ((pred & 1) && nn == m[H4(j >> 2)]) { + ++count; + } + } + } + d[H4(i >> 2)] = count; + } +} + +void HELPER(sve2_histcnt_d)(void *vd, void *vn, void *vm, void *vg, + uint32_t desc) +{ + ARMVectorReg scratch; + intptr_t i, j; + intptr_t opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn, *m = vm; + uint8_t *pg = vg; + + if (d == n) { + n = memcpy(&scratch, n, opr_sz); + if (d == m) { + m = n; + } + } else if (d == m) { + m = memcpy(&scratch, m, opr_sz); + } + + for (i = 0; i < opr_sz / 8; ++i) { + uint64_t count = 0; + if (pg[H1(i)] & 1) { + uint64_t nn = n[i]; + for (j = 0; j <= i; ++j) { + if ((pg[H1(j)] & 1) && nn == m[j]) { + ++count; + } + } + } + d[i] = count; + } +} + +/* + * Returns the number of bytes in m0 and m1 that match n. + * Unlike do_match2 we don't just need true/false, we need an exact count. + * This requires two extra logical operations. + */ +static inline uint64_t do_histseg_cnt(uint8_t n, uint64_t m0, uint64_t m1) +{ + const uint64_t mask = dup_const(MO_8, 0x7f); + uint64_t cmp0, cmp1; + + cmp1 = dup_const(MO_8, n); + cmp0 = cmp1 ^ m0; + cmp1 = cmp1 ^ m1; + + /* + * 1: clear msb of each byte to avoid carry to next byte (& mask) + * 2: carry in to msb if byte != 0 (+ mask) + * 3: set msb if cmp has msb set (| cmp) + * 4: set ~msb to ignore them (| mask) + * We now have 0xff for byte != 0 or 0x7f for byte == 0. + * 5: invert, resulting in 0x80 if and only if byte == 0. + */ + cmp0 = ~(((cmp0 & mask) + mask) | cmp0 | mask); + cmp1 = ~(((cmp1 & mask) + mask) | cmp1 | mask); + + /* + * Combine the two compares in a way that the bits do + * not overlap, and so preserves the count of set bits. + * If the host has an efficient instruction for ctpop, + * then ctpop(x) + ctpop(y) has the same number of + * operations as ctpop(x | (y >> 1)). If the host does + * not have an efficient ctpop, then we only want to + * use it once. + */ + return ctpop64(cmp0 | (cmp1 >> 1)); +} + +void HELPER(sve2_histseg)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j; + intptr_t opr_sz = simd_oprsz(desc); + + for (i = 0; i < opr_sz; i += 16) { + uint64_t n0 = *(uint64_t *)(vn + i); + uint64_t m0 = *(uint64_t *)(vm + i); + uint64_t n1 = *(uint64_t *)(vn + i + 8); + uint64_t m1 = *(uint64_t *)(vm + i + 8); + uint64_t out0 = 0; + uint64_t out1 = 0; + + for (j = 0; j < 64; j += 8) { + uint64_t cnt0 = do_histseg_cnt(n0 >> j, m0, m1); + uint64_t cnt1 = do_histseg_cnt(n1 >> j, m0, m1); + out0 |= cnt0 << j; + out1 |= cnt1 << j; + } + + *(uint64_t *)(vd + i) = out0; + *(uint64_t *)(vd + i + 8) = out1; + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 49d7a45a50..0cbca8d715 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7504,6 +7504,25 @@ static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ DO_SVE2_PPZZ_MATCH(MATCH, match) DO_SVE2_PPZZ_MATCH(NMATCH, nmatch) +static bool trans_HISTCNT(DisasContext *s, arg_rprr_esz *a) +{ + static gen_helper_gvec_4 * const fns[2] = { + gen_helper_sve2_histcnt_s, gen_helper_sve2_histcnt_d + }; + if (a->esz < 2) { + return false; + } + return do_sve2_zpzz_ool(s, a, fns[a->esz - 2]); +} + +static bool trans_HISTSEG(DisasContext *s, arg_rrr_esz *a) +{ + if (a->esz != 0) { + return false; + } + return do_sve2_zzz_ool(s, a, gen_helper_sve2_histseg); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Fri Apr 16 21:02:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422652 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp745094jaf; Fri, 16 Apr 2021 14:54:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyFRVStU33cP00ZJemqRjTxefhru3vahP4hqcqGYZk2lx8q39PSaHAxRS+08RuHTsyt5E8A X-Received: by 2002:a25:790d:: with SMTP id u13mr1724495ybc.427.1618610051652; Fri, 16 Apr 2021 14:54:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610051; cv=none; d=google.com; s=arc-20160816; b=BNtHM/QB6uZcnlza2YTypb0T1462sc6wI4fDnsbrH/E+lU7gYqgepadUu3+I5sqkuU qp7B1QjpVYLz4GXN6evCtOS2XJSPHpH3VGASd0RH1cJOzggf7vJXmWRlxm12Iosb4ele ADzIOi3io2X0zZfSXUfHDSrPxaQ5sWmISmfAzdX0bNGQiJ2aDX1btCwbUD/is8Vz0wEf EEXirC8kYxRtEfSn6A6fs3H+gDFYkF3pW4npPm7KKlbNcyOA/jXMkL1V8jvgk2vNyn+V pkn/ESIaPVonA1i/75nMKin9mNLv49zq+ymnVpAjw3PT9xpJH/iXXNgtft04ruvkvy4g T0cQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=XIEcP0v1PbaDwLB9Hxg+FIEYQexZwgPZLsPGPjMPLII=; b=VnPJSWrhjILxq4+HSjIEHmgOWAVEtQ8L0CrWqYaNj3t7Cuu7WN6/++n4dghQVkQrLt A0iOqD5QvbqCZAP3d+Rd9JQJP/Tc1LQ27E6ER0L3YDBexuaTIqtPz8q/AXZgvjikBkfR Qhqm8mm1F+sQJUdLO/R1SsaxTqkNsUkRYFSHxklaSOxDRwKD25aZGOeqioPdKFrLhdBf Mp0/EtoZBeqPra2lLxEpSjPdHWY10SUICAqTjGaAHUUXRSgCZqAPAwzGXF0NW5/LP8gk Vdj9bFKUgNbiWPw3DtR/CKZTLZlz8VaSetvrgi910ydOQKwT4Rp8kGTHBXvRacmpxFv5 px9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iIMOHsaH; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x1si7010429ybj.345.2021.04.16.14.54.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:54:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iIMOHsaH; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50994 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWPj-0000SW-0d for patch@linaro.org; Fri, 16 Apr 2021 17:54:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44774) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdL-0005hn-42 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:12 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]:37684) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVd6-0001dP-RO for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:07 -0400 Received: by mail-pj1-x1033.google.com with SMTP id e8-20020a17090a7288b029014e51f5a6baso10051713pjg.2 for ; Fri, 16 Apr 2021 14:03:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XIEcP0v1PbaDwLB9Hxg+FIEYQexZwgPZLsPGPjMPLII=; b=iIMOHsaH8aO9qyypSQxWW3AkMjS624ifoFXhGC/Qge3V4tKyS8ZjsIxMbD2W1jWO7l s9X9JyolQ15Vk1AmUyqG7k5efkUEtuA5zRaSqXuqk6RfkUT1G4Nbb9pSJztAxnJkixY6 sU7oSKmKU1NtNNqbCZYGhZlcWV/+g1W8QEQfUOBIHDBhU007ReX0kYYDFfBDPwmCXM0l 4dW1kv3Dz9Y5WgCfFlYUk1KyIzwYV0LbmJcMpfDWzoBp7hEs6q4E8yovTbV84Eg0bkry 9uwzfvxPVwd1rg+oRw2YTBER15g09BMCk5Iyn4rcUWaOmpgDc9Q+k5DYONDE+1TlyQ8a RRWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XIEcP0v1PbaDwLB9Hxg+FIEYQexZwgPZLsPGPjMPLII=; b=nu12LQPcwdVWp8ck6PPNwK0wKk1FM2rDb5wV3A8YyoKbIi4gzR+XC7yjbzhXy6mbPA oyLuwkRg8CDVDdzyiM7/s6N64XF3pB6jxite9iDLYaRGGmiktlTmkOLrJkkK/dEU4Wxa 0M+PS9pJtXxqAUK36S9TefnbOi1Pi9T/gpsesDHKWZ5Pf42PJqLrAyio+AN/w/LoWmSf GZlZIgQwLqF5rqZ4YhX62PsO/cYVi0KOHNrKp6srDQUzlj27rWrlmAjQQsOXNNzPWCWB UldetB6d6giotnMiuNr+mRysb7MtOedQOoU148NiBWwUB7sIcsw1WUB9zd2O+8KHZKoI 2SuQ== X-Gm-Message-State: AOAM532cXhINEGLkoi6rFcm7OjV1gJPD1rn5S+PdORuHGDxYvzsKp+HE xNmmZ5/H7Jq54ngOB+OT0buLUuKGYAHHNQ== X-Received: by 2002:a17:90a:9f8c:: with SMTP id o12mr11439384pjp.215.1618607035511; Fri, 16 Apr 2021 14:03:55 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.03.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 43/81] target/arm: Implement SVE2 XAR Date: Fri, 16 Apr 2021 14:02:02 -0700 Message-Id: <20210416210240.1591291-44-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1033.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" In addition, use the same vector generator interface for AdvSIMD. This fixes a bug in which the AdvSIMD insn failed to clear the high bits of the SVE register. Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 4 ++ target/arm/helper.h | 2 + target/arm/translate-a64.h | 3 ++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 39 ++++++++++++++ target/arm/translate-a64.c | 25 ++------- target/arm/translate-sve.c | 104 +++++++++++++++++++++++++++++++++++++ target/arm/vec_helper.c | 12 +++++ 8 files changed, 172 insertions(+), 21 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 507a2fea8e..28b8f00201 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2558,6 +2558,10 @@ DEF_HELPER_FLAGS_5(sve2_histcnt_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_4(sve2_histseg, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_xar_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_xar_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_xar_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/helper.h b/target/arm/helper.h index 6bb0b0ddc0..23a7ec5638 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -953,6 +953,8 @@ DEF_HELPER_FLAGS_5(neon_sqrdmulh_h, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(neon_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + #ifdef TARGET_AARCH64 #include "helper-a64.h" #include "helper-sve.h" diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h index 868d355048..b4d391207f 100644 --- a/target/arm/translate-a64.h +++ b/target/arm/translate-a64.h @@ -122,5 +122,8 @@ bool disas_sve(DisasContext *, uint32_t); void gen_gvec_rax1(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_xar(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t rm_ofs, int64_t shift, + uint32_t opr_sz, uint32_t max_sz); #endif /* TARGET_ARM_TRANSLATE_A64_H */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 8f501a083c..7645587469 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -65,6 +65,7 @@ &rr_dbm rd rn dbm &rrri rd rn rm imm &rri_esz rd rn imm esz +&rrri_esz rd rn rm imm esz &rrr_esz rd rn rm esz &rpr_esz rd pg rn esz &rpr_s rd pg rn s @@ -384,6 +385,9 @@ ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +XAR 00000100 .. 1 ..... 001 101 rm:5 rd:5 &rrri_esz \ + rn=%reg_movprfx esz=%tszimm16_esz imm=%tszimm16_shr + # SVE2 bitwise ternary operations EOR3 00000100 00 1 ..... 001 110 ..... ..... @rdn_ra_rm_e0 BSL 00000100 00 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 8d002fdb65..c77003217e 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7193,3 +7193,42 @@ void HELPER(sve2_histseg)(void *vd, void *vn, void *vm, uint32_t desc) *(uint64_t *)(vd + i + 8) = out1; } } + +void HELPER(sve2_xar_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + int shr = simd_data(desc); + int shl = 8 - shr; + uint64_t mask = dup_const(MO_8, 0xff >> shr); + uint64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + uint64_t t = n[i] ^ m[i]; + d[i] = ((t >> shr) & mask) | ((t << shl) & ~mask); + } +} + +void HELPER(sve2_xar_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + int shr = simd_data(desc); + int shl = 16 - shr; + uint64_t mask = dup_const(MO_16, 0xffff >> shr); + uint64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + uint64_t t = n[i] ^ m[i]; + d[i] = ((t >> shr) & mask) | ((t << shl) & ~mask); + } +} + +void HELPER(sve2_xar_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 4; + int shr = simd_data(desc); + uint32_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ror32(n[i] ^ m[i], shr); + } +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 95897e63af..cd8408e84c 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -14364,8 +14364,6 @@ static void disas_crypto_xar(DisasContext *s, uint32_t insn) int imm6 = extract32(insn, 10, 6); int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); - TCGv_i64 tcg_op1, tcg_op2, tcg_res[2]; - int pass; if (!dc_isar_feature(aa64_sha3, s)) { unallocated_encoding(s); @@ -14376,25 +14374,10 @@ static void disas_crypto_xar(DisasContext *s, uint32_t insn) return; } - tcg_op1 = tcg_temp_new_i64(); - tcg_op2 = tcg_temp_new_i64(); - tcg_res[0] = tcg_temp_new_i64(); - tcg_res[1] = tcg_temp_new_i64(); - - for (pass = 0; pass < 2; pass++) { - read_vec_element(s, tcg_op1, rn, pass, MO_64); - read_vec_element(s, tcg_op2, rm, pass, MO_64); - - tcg_gen_xor_i64(tcg_res[pass], tcg_op1, tcg_op2); - tcg_gen_rotri_i64(tcg_res[pass], tcg_res[pass], imm6); - } - write_vec_element(s, tcg_res[0], rd, 0, MO_64); - write_vec_element(s, tcg_res[1], rd, 1, MO_64); - - tcg_temp_free_i64(tcg_op1); - tcg_temp_free_i64(tcg_op2); - tcg_temp_free_i64(tcg_res[0]); - tcg_temp_free_i64(tcg_res[1]); + gen_gvec_xar(MO_64, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), imm6, 16, + vec_full_reg_size(s)); } /* Crypto three-reg imm2 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0cbca8d715..eea8b6f1d0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -340,6 +340,110 @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a) return do_zzz_fn(s, a, tcg_gen_gvec_andc); } +static void gen_xar8_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) +{ + TCGv_i64 t = tcg_temp_new_i64(); + uint64_t mask = dup_const(MO_8, 0xff >> sh); + + tcg_gen_xor_i64(t, n, m); + tcg_gen_shri_i64(d, t, sh); + tcg_gen_shli_i64(t, t, 8 - sh); + tcg_gen_andi_i64(d, d, mask); + tcg_gen_andi_i64(t, t, ~mask); + tcg_gen_or_i64(d, d, t); + tcg_temp_free_i64(t); +} + +static void gen_xar16_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) +{ + TCGv_i64 t = tcg_temp_new_i64(); + uint64_t mask = dup_const(MO_16, 0xffff >> sh); + + tcg_gen_xor_i64(t, n, m); + tcg_gen_shri_i64(d, t, sh); + tcg_gen_shli_i64(t, t, 16 - sh); + tcg_gen_andi_i64(d, d, mask); + tcg_gen_andi_i64(t, t, ~mask); + tcg_gen_or_i64(d, d, t); + tcg_temp_free_i64(t); +} + +static void gen_xar_i32(TCGv_i32 d, TCGv_i32 n, TCGv_i32 m, int32_t sh) +{ + tcg_gen_xor_i32(d, n, m); + tcg_gen_rotri_i32(d, d, sh); +} + +static void gen_xar_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) +{ + tcg_gen_xor_i64(d, n, m); + tcg_gen_rotri_i64(d, d, sh); +} + +static void gen_xar_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, int64_t sh) +{ + tcg_gen_xor_vec(vece, d, n, m); + tcg_gen_rotri_vec(vece, d, d, sh); +} + +void gen_gvec_xar(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t rm_ofs, int64_t shift, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop[] = { INDEX_op_rotli_vec, 0 }; + static const GVecGen3i ops[4] = { + { .fni8 = gen_xar8_i64, + .fniv = gen_xar_vec, + .fno = gen_helper_sve2_xar_b, + .opt_opc = vecop, + .vece = MO_8 }, + { .fni8 = gen_xar16_i64, + .fniv = gen_xar_vec, + .fno = gen_helper_sve2_xar_h, + .opt_opc = vecop, + .vece = MO_16 }, + { .fni4 = gen_xar_i32, + .fniv = gen_xar_vec, + .fno = gen_helper_sve2_xar_s, + .opt_opc = vecop, + .vece = MO_32 }, + { .fni8 = gen_xar_i64, + .fniv = gen_xar_vec, + .fno = gen_helper_gvec_xar_d, + .opt_opc = vecop, + .vece = MO_64 } + }; + int esize = 8 << vece; + + /* The SVE2 range is 1 .. esize; the AdvSIMD range is 0 .. esize-1. */ + tcg_debug_assert(shift >= 0); + tcg_debug_assert(shift <= esize); + shift &= esize - 1; + + if (shift == 0) { + /* xar with no rotate devolves to xor. */ + tcg_gen_gvec_xor(vece, rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz); + } else { + tcg_gen_gvec_3i(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, + shift, &ops[vece]); + } +} + +static bool trans_XAR(DisasContext *s, arg_rrri_esz *a) +{ + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + gen_gvec_xar(a->esz, vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), a->imm, vsz, vsz); + } + return true; +} + static bool do_sve2_zzzz_fn(DisasContext *s, arg_rrrr_esz *a, GVecGen4Fn *fn) { if (!dc_isar_feature(aa64_sve2, s)) { diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 19006f50f7..a3d80ecad0 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -2253,3 +2253,15 @@ void HELPER(gvec_umulh_d)(void *vd, void *vn, void *vm, uint32_t desc) } clear_tail(d, opr_sz, simd_maxsz(desc)); } + +void HELPER(gvec_xar_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + int shr = simd_data(desc); + uint64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ror64(n[i] ^ m[i], shr); + } + clear_tail(d, opr_sz * 8, simd_maxsz(desc)); +} From patchwork Fri Apr 16 21:02:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422628 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp736215jaf; Fri, 16 Apr 2021 14:35:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzCXavN4UTseDeg9AzH1+DLAWqb0uyBXqhgzRiA4Ja3zXlry+6NGNlKq/G9xL+CiVviwROn X-Received: by 2002:a5d:9449:: with SMTP id x9mr5351866ior.157.1618608935158; Fri, 16 Apr 2021 14:35:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608935; cv=none; d=google.com; s=arc-20160816; b=CEspov7EgLn+nGT5ovw085pJdQtuZavYtTu0FZLnoHXWBUmRWhdaAUUCRSyBkke+bs DmUP/Xsh6MAZfOiyabAyi5kC8O+F42vK7/gaXmG58MUp6NK63i6DEc2WFsL/iws5vFkY qChRavPulkumRaSJw9GbMp4SX15DBhtJfn0zDKRMMD6Ys6k7ykTJ8ZCm7+tqxuy7xL9a yy1iRraVKyKy+9cSi/CkPnz0G80fH83lbWDhZIg2y3Rgewf1OWnKHbrf6sEYvKJqE2jq SplRfJAZ7klCXiAltiS/pRnyjkyYEF+vNTHticGMq+QAyEu50/yJsc2LAMSJnQa0vreQ nVIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=bYBvAAL6DC7Hi2Mjo18sNLpM8Gq57yT/LKmwPXamUwE=; b=vShbl6A7aZUnt3Q9B4O35GKX5x1+wFwg/drPvHVp98h/7vZMDvoh/WI192iBH8K2Jg v15doh+6T7n6CooyqTzQ9e10IBxYw4BsMYl4aHCgaCGZ0hI1vYVqZBdkJdlSptuPCvn0 ll22TDUF3+Lznh0ExCReKjXcAXcaXNivp8bcodcFQVmEWH4N2ZUMI392JsGjEDfeen9P 3OV+H/4afiVCo9m98j4fpAZIzTSp1dZQSvAA9J3iYxTahiOQTSZ33NyrCQjfHYvJOICw oYjSY6G/8TDRc9Wj/7Dyr6pITOp2zD93Wjg0iObe5zzrIxtoIU6Nl69rMoE0jXOOpJiF vfZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RxYxRwwI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g4si2383533jas.78.2021.04.16.14.35.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:35:35 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RxYxRwwI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53702 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW7i-00038C-Jo for patch@linaro.org; Fri, 16 Apr 2021 17:35:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44676) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdB-0005gg-GG for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:05 -0400 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]:37608) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVd9-0001eE-5s for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:00 -0400 Received: by mail-pg1-x52a.google.com with SMTP id p2so4545552pgh.4 for ; Fri, 16 Apr 2021 14:03:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bYBvAAL6DC7Hi2Mjo18sNLpM8Gq57yT/LKmwPXamUwE=; b=RxYxRwwINCDWwbj/euXXFTmjR/uiBAP90lqdHqCSB+tydAcaQKht261yRa2AUio0/R r+yBfCJsxVuHXv1G1Uvyrk4+/+qvHoNZH5ggG03lLW0KfKe8n+1U6E4r+MmgEtOqHHtl Q+vL0fpC4zuaBm66kYaJXS3PyuM5FgGJAMoQyIx9h6Qr4MqnLyOgtfU+L6aOBGfVCHbX RjILcn1LwBBSb9DGbelxw25r1Jw9Rjucu+XPwddscCZ2WR6kndX5+NUnq0K684zk8oqc RvCLmK40lN60M7MD6zXg7gIp8fv79uqzy9Y5VfYf5cSZw2LXmm64CtpnQz2PTcVvzYwQ /XDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bYBvAAL6DC7Hi2Mjo18sNLpM8Gq57yT/LKmwPXamUwE=; b=U2zNtzsn+VKhJ6TzTLaLwPFtjVGtqwozRmQ9xHXSfTaQRkFnj3gaQ1gWDe8k9g64mS PlYZOlr4mLiUa1VK/X3XTF6hI0CvhTcEOtJcsx3wr/+HQMgtGBAPCluXR8C4H+w43WgD 1VmhKHe919/AJ8heG6Xt9jmjBih5UNbp7Sao962V0H7T6vcqdSNBzQq5M+tgAy9LCNTd nOPn+4BHc26o68A8fqeSaVnUgAoZBJDJz2xgHvHKRbqb4K3bVVanS4GFEJUwbBUE50nP u8OR+K51oe+muLmV7x07YcKRtnpR5iuII5koGEupvkONwt7X8ng1IlhRs08qVdffEZfh 6dkw== X-Gm-Message-State: AOAM533SaL2pRKyDiDu7JBMzegQgJajf9oybP48Q5uSOsAznB9pyPXAn N8D2MS/WYVySj4c9fW7lRvFHgc9cnvEJKg== X-Received: by 2002:a05:6a00:24c3:b029:253:e598:2c62 with SMTP id d3-20020a056a0024c3b0290253e5982c62mr9686369pfv.30.1618607036385; Fri, 16 Apr 2021 14:03:56 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.03.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 44/81] target/arm: Implement SVE2 scatter store insns Date: Fri, 16 Apr 2021 14:02:03 -0700 Message-Id: <20210416210240.1591291-45-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52a; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Add decoding logic for SVE2 64-bit/32-bit scatter non-temporal store insns. 64-bit * STNT1B (vector plus scalar) * STNT1H (vector plus scalar) * STNT1W (vector plus scalar) * STNT1D (vector plus scalar) 32-bit * STNT1B (vector plus scalar) * STNT1H (vector plus scalar) * STNT1W (vector plus scalar) Signed-off-by: Stephen Long Message-Id: <20200422141553.8037-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/sve.decode | 10 ++++++++++ target/arm/translate-sve.c | 8 ++++++++ 2 files changed, 18 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7645587469..5cfe6df0d2 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1388,3 +1388,13 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx + +### SVE2 Memory Store Group + +# SVE2 64-bit scatter non-temporal store (vector plus scalar) +STNT1_zprz 1110010 .. 00 ..... 001 ... ..... ..... \ + @rprr_scatter_store xs=2 esz=3 scale=0 + +# SVE2 32-bit scatter non-temporal store (vector plus scalar) +STNT1_zprz 1110010 .. 10 ..... 001 ... ..... ..... \ + @rprr_scatter_store xs=0 esz=2 scale=0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index eea8b6f1d0..0356b6a124 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6167,6 +6167,14 @@ static bool trans_ST1_zpiz(DisasContext *s, arg_ST1_zpiz *a) return true; } +static bool trans_STNT1_zprz(DisasContext *s, arg_ST1_zprz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return trans_ST1_zprz(s, a); +} + /* * Prefetches */ From patchwork Fri Apr 16 21:02:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422623 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp732086jaf; Fri, 16 Apr 2021 14:28:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzZwhKJuoOlYO0UvfQC4Py9KEFSO/h8Gb6WsjHD4HxGf39vKO170y4h6Y29iyAOG+eISYiB X-Received: by 2002:a05:6e02:610:: with SMTP id t16mr3591765ils.196.1618608509316; Fri, 16 Apr 2021 14:28:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608509; cv=none; d=google.com; s=arc-20160816; b=F75sn/SBrYg8pQXbbiKqYWSpB/HyCLlB20DcLyk3jIYqEf19bFwc6gdOioWruED4Sa KTlJTYzg8H1MKJHHpST/j43nrspNKwKOmDdf3D7gPjRTiXisrz+BLvG63UoVBV3GehVE uliWvrtXN3DG33N6nJSpGrEhFUAnqqghRlFFQZNPL4XMXcsoCZHAhTCOWMJbVvE0m4YP 4j3xDvLZsxUSFi9rRKh6rVEx0zuusPXfjd+PMOdAEZaPPE0hFWPsxBl8htERGAikaj7o bLkiLSY5EgbkWy+R6N2Y4wnhtIsYSXVnbBRkUFLFcmHwDsnjq2Ybyfq6HeAMz07rYTVP UPbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=bn7hUPo4Fn4gj8R8dHIFe5YgNO7lh3cxv0iB6D6VY7Q=; b=hK+wQ2XO78CU+7JwTmV6BbVuFp1Lq2F8EiIeFvNLCt1qnWPXWcykFflNLCuNd7APRJ Ps8ozE6qHm1+62P2d2Yfe1CAfXn08AtNq41VOeP1gxpGYJFJObqzsvPu9rv/3u/zA+De qPfuOGZi9nDQWf1o/Gb5SlRh0811kwxr534K4SuJmk+WMyXXY1EQ229CI7b2m0aP0qFK lX3EaMxIF2w6khLyl4GQ/8mqr9d9ntQbjwjYK5AGPOAlzPoNQz8yVTeRsyRS75llIi2q k2QglUrspbYR/W0IM4QQQHGHXlTdD9Cbkp8HPgPa54Q9dKFkcn+Bw1zW5EWCuEJhXd3n AJ2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=X4soKN45; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i13si3859271ilm.161.2021.04.16.14.28.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:28:29 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=X4soKN45; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35456 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW0q-0003lz-QD for patch@linaro.org; Fri, 16 Apr 2021 17:28:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44772) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdK-0005hm-RP for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:12 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]:46912) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVd9-0001en-7A for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:05 -0400 Received: by mail-pl1-x634.google.com with SMTP id m18so12423559plc.13 for ; Fri, 16 Apr 2021 14:03:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bn7hUPo4Fn4gj8R8dHIFe5YgNO7lh3cxv0iB6D6VY7Q=; b=X4soKN45ZDO88UrGTj2gJ3dXwlu8xzzaBMveQlzjQtSBKMRYmN6tZfZTWdAsqMJOuO 3+O968i9OB8s+1ApqEPKEbygJ9NFwIvbJzmLB3IqfXtqSGo3RCLPyswZ3XjsSqc0tG6a 6Kv3l4CIEI3mvjaHwukCYaD0Sb8i/BSv0mPP9ZiXqbpNytdoq6Ig/g09FuwBXLI+6fYw xIAXE6P6Wolcm5DNF3Ci7B5N3ssf0/i+vRkxH2fmzUCETUHfxjATjyNl3y3dcL+ff1Ue krxtYVgtY7IaLX178ywEvYg2Eu08Ovd+1dRLsMryLIKdeSdO99gYa+AFZIxAqvjNJS+J hSPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bn7hUPo4Fn4gj8R8dHIFe5YgNO7lh3cxv0iB6D6VY7Q=; b=uHm9AFiLdKW5IDJ1U6OOzEivN8DebzMnfm5cDAu0PDvyRYc14bHiU3kKAx3g9u+7P6 VUVVECD1Gcgf6fW3rvcIPvNCJUX0HhvXQNcC42s/4+50oD/4X26PX//Z2TN3x5MtJYbJ JUemXwqlOdb/VxppKsjC+MGw1R9AZ3ETE4j6oChLizouf7w3EZljs6+usJooz6Ljh38J KQnop1lmyT3XRHjMqhP/qahmMyAU7CtdQcCNIZyG13NM9gj7SSJxhzYpDCu5fzpjtJmc JzyeZoBg5kLTRCFfLHDwcv0sGooKBrRB/ZGn/lrNFlbGeh3yb8cOdnjcoToBNLXCF5HW tLZg== X-Gm-Message-State: AOAM531TzQ/RXGrpecy+VUoMkAZeokKrqXZLwWHmAsnROpJgrvJlYuX4 xg9HGcqx5UP5N/JxdRCL6AOlhyQ0+5A7OQ== X-Received: by 2002:a17:90b:812:: with SMTP id bk18mr11902958pjb.145.1618607037536; Fri, 16 Apr 2021 14:03:57 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.03.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 45/81] target/arm: Implement SVE2 gather load insns Date: Fri, 16 Apr 2021 14:02:04 -0700 Message-Id: <20210416210240.1591291-46-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Add decoding logic for SVE2 64-bit/32-bit gather non-temporal load insns. 64-bit * LDNT1SB * LDNT1B (vector plus scalar) * LDNT1SH * LDNT1H (vector plus scalar) * LDNT1SW * LDNT1W (vector plus scalar) * LDNT1D (vector plus scalar) 32-bit * LDNT1SB * LDNT1B (vector plus scalar) * LDNT1SH * LDNT1H (vector plus scalar) * LDNT1W (vector plus scalar) Signed-off-by: Stephen Long Message-Id: <20200422152343.12493-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/sve.decode | 11 +++++++++++ target/arm/translate-sve.c | 8 ++++++++ 2 files changed, 19 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 5cfe6df0d2..c3958bed6a 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1389,6 +1389,17 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx +### SVE2 Memory Gather Load Group + +# SVE2 64-bit gather non-temporal load +# (scalar plus unpacked 32-bit unscaled offsets) +LDNT1_zprz 1100010 msz:2 00 rm:5 1 u:1 0 pg:3 rn:5 rd:5 \ + &rprr_gather_load xs=0 esz=3 scale=0 ff=0 + +# SVE2 32-bit gather non-temporal load (scalar plus 32-bit unscaled offsets) +LDNT1_zprz 1000010 msz:2 00 rm:5 10 u:1 pg:3 rn:5 rd:5 \ + &rprr_gather_load xs=0 esz=2 scale=0 ff=0 + ### SVE2 Memory Store Group # SVE2 64-bit scatter non-temporal store (vector plus scalar) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0356b6a124..a74c15b23f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6015,6 +6015,14 @@ static bool trans_LD1_zpiz(DisasContext *s, arg_LD1_zpiz *a) return true; } +static bool trans_LDNT1_zprz(DisasContext *s, arg_LD1_zprz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return trans_LD1_zprz(s, a); +} + /* Indexed by [mte][be][xs][msz]. */ static gen_helper_gvec_mem_scatter * const scatter_store_fn32[2][2][2][3] = { { /* MTE Inactive */ From patchwork Fri Apr 16 21:02:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422654 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp745982jaf; Fri, 16 Apr 2021 14:56:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxQcNOvmsGjBbGWwxp8YS94ZiNsYMqr5nLe4Gv4SxV2jWcHK1EIvh4TnzJdfmr0rYiSlUKO X-Received: by 2002:a25:ac13:: with SMTP id w19mr1670544ybi.134.1618610168212; Fri, 16 Apr 2021 14:56:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610168; cv=none; d=google.com; s=arc-20160816; b=ovws9BRBtGJig66LMCOlRtyfcWXKPR97tH54B8eMATXRiSdbBNnwUHrdXQbiYcsh7k qXdXOsxsiLIaBRGlflZuQlIqeUh7H1GaCeD1VMoiUEtyYMHKcw6d9bbML5K945OPiarP kZAHP7Lu1Wccn5FNKTw0bC8X5kAvyLpOHiEBeDrbLJA1uzavh1xnS8qVDjRlR05e5oUa LIWYsoKKK3LN+NXiQoSipYOPRk2escvN5WZd9CERZvlt/HSR50AJnEGMXHwX7dSJ3vkb 2gXqriF4Xc+QJQ0Qnocl3f8Nk/pqqxozBM+dMaIO0hC23QdXVL5HfVZdFNJpUJrtHVYd Ni8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=C4KQNDQjXRv8XyhUPnjGu99QIb+SfiePBH9Ienjk6A4=; b=bZ6tffxlMpImet6F2UlmXim+KkhSAOdvm03+bu9Tjr/XYZO2OD48yg2y0IFbGxk6VR IFquZYl3Z0xyTQ7YPyODtDrH/ICt6TF8OfeBCLTTamP76T4Vmr0Ym6Yt7ILTLPlAvwGb rJtggdwZGjJLEajtw3qXtBUwH/64OIgePJ0tByUHHGgA9cgLSMnP7e3RB8xLwv948v1d yptpgczVVghlT6ntQVR5vu3rS22DGXfiTN3WSUqycybmpi6Kkj6Tf6HnEjcCz+uSuwl7 3Mh3QTngfQNhAo+elg28SZ8ru/J6T1Qhe2q5z5/7pSqusD5fNFL0O8UUaOVkKcLfQY6a Kktw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vXpIzOWT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v133si6938914ybe.267.2021.04.16.14.56.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:56:08 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vXpIzOWT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59770 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWRb-0003zu-Jc for patch@linaro.org; Fri, 16 Apr 2021 17:56:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44820) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdO-0005in-VL for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:18 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]:35681) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdA-0001fJ-5J for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:10 -0400 Received: by mail-pf1-x42c.google.com with SMTP id h15so1450552pfv.2 for ; Fri, 16 Apr 2021 14:03:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=C4KQNDQjXRv8XyhUPnjGu99QIb+SfiePBH9Ienjk6A4=; b=vXpIzOWT9uLZmUjVcGaOQitwzT2vG28+K45a3lGnqKQmTII1vn9qN845l3XEKzIOC2 59lEdN0kkwO3Kj3hGTzLnDkKJSU572BO6NjrFmM32LDLQkDugGY1T9dw+bejVCvuS82R M5l0XGoiLDrKIqzFa7z1eWQmvvZmM3YAXr4865XQz6c+uS8HieeWKe0FdEmWMRxvqDPU VWbVocOA64IlGhxSs3a6739hHULD+lZUWZE/WHAIRt6KqY7aUTrWcnIRoz9dHV+sCrSV /HevUIzi52u9NETKXKO8n8XURmsk+t6LnGV7h8Fde2kjpUz10oNlVKpB0QvbV+Nou+RP vR0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=C4KQNDQjXRv8XyhUPnjGu99QIb+SfiePBH9Ienjk6A4=; b=W1Fg1vedxKuBeJ5B7CTSy+gAFznrV1uh7fCAXDe2Li+OPlhaNukIFvbdap1e1/36Vl mGpZQmrsfgctsksp5aQFDS2Vb+dZl8SoqiLJId38od85qvHS6eKemzHxGYyAk91etg4q 7XTdfWxRJ+5deIGvAs9GU0kk88wTBsAkXjlp0cPWmUUJqNG2/wvzGomY/U8itfSe+6xq fCuuZGJqz/bFlkZip+6mFMKXAfL5Sfbzpc4+r1NisvcOngC+cz9HygXHkaWqvc2C6Dpr wXP8Wf/UMh7yXa4JJA+qmwqBCZ7CzB+KDWyHnOWseM+EKFtLJlIwVgTYIJFPsvq5JFOp c3DA== X-Gm-Message-State: AOAM530C5oE0svNUfDL+l3Lr3SCddk0TTcRI3uHitN1n/D7GR/aMfw/A ZGjDfmK35vW8aUXX8nsPk/Fnk9j/7aMhcA== X-Received: by 2002:a05:6a00:8c7:b029:20f:1cf4:d02 with SMTP id s7-20020a056a0008c7b029020f1cf40d02mr9555973pfu.49.1618607038515; Fri, 16 Apr 2021 14:03:58 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.03.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 46/81] target/arm: Implement SVE2 FMMLA Date: Fri, 16 Apr 2021 14:02:05 -0700 Message-Id: <20210416210240.1591291-47-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200422165503.13511-1-steplong@quicinc.com> [rth: Fix indexing in helpers, expand macro to straight functions.] Signed-off-by: Richard Henderson --- target/arm/cpu.h | 10 ++++++ target/arm/helper-sve.h | 3 ++ target/arm/sve.decode | 4 +++ target/arm/sve_helper.c | 74 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 34 ++++++++++++++++++ 5 files changed, 125 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index ae787fac8a..e44bb8973a 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4246,6 +4246,16 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve2_f32mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, F32MM) != 0; +} + +static inline bool isar_feature_aa64_sve2_f64mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, F64MM) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 28b8f00201..7e99dcd119 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2662,3 +2662,6 @@ DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(fmmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(fmmla_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index c3958bed6a..cb2ee86228 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1389,6 +1389,10 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx +### SVE2 floating point matrix multiply accumulate + +FMMLA 01100100 .. 1 ..... 111001 ..... ..... @rda_rn_rm + ### SVE2 Memory Gather Load Group # SVE2 64-bit gather non-temporal load diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index c77003217e..f285c90b70 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7232,3 +7232,77 @@ void HELPER(sve2_xar_s)(void *vd, void *vn, void *vm, uint32_t desc) d[i] = ror32(n[i] ^ m[i], shr); } } + +void HELPER(fmmla_s)(void *vd, void *vn, void *vm, void *va, + void *status, uint32_t desc) +{ + intptr_t s, opr_sz = simd_oprsz(desc) / (sizeof(float32) * 4); + + for (s = 0; s < opr_sz; ++s) { + float32 *n = vn + s * sizeof(float32) * 4; + float32 *m = vm + s * sizeof(float32) * 4; + float32 *a = va + s * sizeof(float32) * 4; + float32 *d = vd + s * sizeof(float32) * 4; + float32 n00 = n[H4(0)], n01 = n[H4(1)]; + float32 n10 = n[H4(2)], n11 = n[H4(3)]; + float32 m00 = m[H4(0)], m01 = m[H4(1)]; + float32 m10 = m[H4(2)], m11 = m[H4(3)]; + float32 p0, p1; + + /* i = 0, j = 0 */ + p0 = float32_mul(n00, m00, status); + p1 = float32_mul(n01, m01, status); + d[H4(0)] = float32_add(a[H4(0)], float32_add(p0, p1, status), status); + + /* i = 0, j = 1 */ + p0 = float32_mul(n00, m10, status); + p1 = float32_mul(n01, m11, status); + d[H4(1)] = float32_add(a[H4(1)], float32_add(p0, p1, status), status); + + /* i = 1, j = 0 */ + p0 = float32_mul(n10, m00, status); + p1 = float32_mul(n11, m01, status); + d[H4(2)] = float32_add(a[H4(2)], float32_add(p0, p1, status), status); + + /* i = 1, j = 1 */ + p0 = float32_mul(n10, m10, status); + p1 = float32_mul(n11, m11, status); + d[H4(3)] = float32_add(a[H4(3)], float32_add(p0, p1, status), status); + } +} + +void HELPER(fmmla_d)(void *vd, void *vn, void *vm, void *va, + void *status, uint32_t desc) +{ + intptr_t s, opr_sz = simd_oprsz(desc) / (sizeof(float64) * 4); + + for (s = 0; s < opr_sz; ++s) { + float64 *n = vn + s * sizeof(float64) * 4; + float64 *m = vm + s * sizeof(float64) * 4; + float64 *a = va + s * sizeof(float64) * 4; + float64 *d = vd + s * sizeof(float64) * 4; + float64 n00 = n[0], n01 = n[1], n10 = n[2], n11 = n[3]; + float64 m00 = m[0], m01 = m[1], m10 = m[2], m11 = m[3]; + float64 p0, p1; + + /* i = 0, j = 0 */ + p0 = float64_mul(n00, m00, status); + p1 = float64_mul(n01, m01, status); + d[0] = float64_add(a[0], float64_add(p0, p1, status), status); + + /* i = 0, j = 1 */ + p0 = float64_mul(n00, m10, status); + p1 = float64_mul(n01, m11, status); + d[1] = float64_add(a[1], float64_add(p0, p1, status), status); + + /* i = 1, j = 0 */ + p0 = float64_mul(n10, m00, status); + p1 = float64_mul(n11, m01, status); + d[2] = float64_add(a[2], float64_add(p0, p1, status), status); + + /* i = 1, j = 1 */ + p0 = float64_mul(n10, m10, status); + p1 = float64_mul(n11, m11, status); + d[3] = float64_add(a[3], float64_add(p0, p1, status), status); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index a74c15b23f..0afae9646f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7672,6 +7672,40 @@ DO_SVE2_ZPZZ_FP(FMINP, fminp) * SVE Integer Multiply-Add (unpredicated) */ +static bool trans_FMMLA(DisasContext *s, arg_rrrr_esz *a) +{ + gen_helper_gvec_4_ptr *fn; + + switch (a->esz) { + case MO_32: + if (!dc_isar_feature(aa64_sve2_f32mm, s)) { + return false; + } + fn = gen_helper_fmmla_s; + break; + case MO_64: + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + fn = gen_helper_fmmla_d; + break; + default: + return false; + } + + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + status, vsz, vsz, 0, fn); + tcg_temp_free_ptr(status); + } + return true; +} + static bool do_sqdmlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel1, bool sel2) { From patchwork Fri Apr 16 21:02:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422627 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp734477jaf; Fri, 16 Apr 2021 14:32:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxXFjcBM9ruq4+W34MEnJsDbsUDQWtqXLOfT/WCc4arOmd1HHu830PFYMRJzDkk4gXCI02F X-Received: by 2002:ae9:f205:: with SMTP id m5mr1288453qkg.101.1618608753698; Fri, 16 Apr 2021 14:32:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618608753; cv=none; d=google.com; s=arc-20160816; b=vYgx1Ax2XKajUcYuPOWmcPoW3P2SkZ77wCASDcHFSwboYAWPfVIQpvO9irJDQj+MVf ZONGI3aiKca3yN2ZKXjQo3/4p1JFprHn30vwvH4GVrgm4aLyKtp0ffiCN7w+brqlpSVd SjJwXxOgeXAPxujBr68L/XVIoDZ1qrfTwLnTvFShG+bgUd8NYSR5ZU8WBGdKxqcBNyyq AZ/0C1qkAzGrnhYE/ZJ4F2o5LdInoS4y8vEEUSuxqEBfUKJZkhw5z8OuegQIBqDBBy15 1W69sSsI+9gdneJk2cKIHbLeHhnviu5kiy+Oy3T9fPZD+pCMh44otalwVLnkyYiwCdZK KXwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=YeuSP/ptzfHqZ3An7U1oQpcltldWGviyBgnte7gB4CU=; b=UH4byNKYoef1rLp/c3FJjoV8+DKLuyoJG2/kim/EXHXup/R/Xo1rmla1PhVPsVRuQY ngW+Rcu7IZnNYQ2FIVNVhmCZhsastemOeP2912jIDf4P4uWZXQCEXQBcqurBj4OoUwmu euznsOf0BmwWoKCjialWIyM73ZHDyISXOpOC7IGKux6HfgmNmpZAaSjp087Cj+1bppGx nMbkqu5FnqM2o94VRoztOJ2VPx/21S25KBf2Wu5BNX2p6QfThzpwaNfukV+kaF5jUAxv 5QnHSstWmMLgkvmhe2bJM33yjKQW3aAPLNYRyWV8px9KWRcDyO2PsqNw0WFBvXmOkyPQ aVzQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=oxaAt5DN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g9si4593401qts.78.2021.04.16.14.32.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:32:33 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=oxaAt5DN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45182 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW4m-0007pV-WC for patch@linaro.org; Fri, 16 Apr 2021 17:32:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44856) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdQ-0005iz-Qm for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:18 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]:35354) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdA-0001fb-HA for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:14 -0400 Received: by mail-pg1-x530.google.com with SMTP id q10so20003735pgj.2 for ; Fri, 16 Apr 2021 14:04:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YeuSP/ptzfHqZ3An7U1oQpcltldWGviyBgnte7gB4CU=; b=oxaAt5DNDxc78gDOzI35zRmEU7g/oip8KPTQVVvugOBSabmRDiuMCh3G55Wg10V3al wnAtiAFK/RBceJGk0CUECybhuXUHeJAbLoercVAJWG/npAYd1GF3qoj1YyagZz9xgO+9 GrHyG02eSTJST556rLl424hg5tPy0k3T2+pOvoFtIDpc2FbZTG/0ZmWhjt8H3xhtupbM Er2uj412Uxk7vqQPGtLVvS2rLFMX+HZjthZSH2wvn2MHAzvmtl9y/lLuAt4jFJcpXYT9 77fkdQqvgmmqRmwd6qkUDVZDCLIeYt3i2P3wcp9rebBg1adIjvi/Iq0ckET07hNZEk3C xj9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YeuSP/ptzfHqZ3An7U1oQpcltldWGviyBgnte7gB4CU=; b=CnvEYIodxI8xwq9/jn38g+lzgeoZGxF2g9/jvzr3KAQ0CJIn1O8zw5bDMLleXCxhCd VUKNjkkQlpwO3SBPMAW7hQ5RVfhRaKoe/EDPqtIasF/jp2pffgEkYPtriwe/liuvWduc 3Nf2B9dXAGnAeMDzuCMRAGXLbxrSE4lVYdIqHVdR0CPTOqZVT6fjzmN9BTlOIIFXm0fQ SrQdQFhBugegRdawEc+d+Q8Et/4EGbuWqSxgQhvJED1dfvUtWJsjlzPgPRRumu/PLQJg 0H5nvjd9B/0R8eCZNT9ht1lxbpI0tFBnatBKk/4VjDXcasERaGgPRsImE23ebWXkXYqY 7Vag== X-Gm-Message-State: AOAM531NG+BO9TdPD1M6dUQ2zv/znFVWSA6O+dqwXR1/f+nt7+QDO7lc AQDPkRd2R0LTwy5YqwP4ay1gwHrhye/Ujg== X-Received: by 2002:a62:7907:0:b029:23e:9010:3844 with SMTP id u7-20020a6279070000b029023e90103844mr9620148pfc.58.1618607039232; Fri, 16 Apr 2021 14:03:59 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.03.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 47/81] target/arm: Implement SVE2 SPLICE, EXT Date: Fri, 16 Apr 2021 14:02:06 -0700 Message-Id: <20210416210240.1591291-48-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200423180347.9403-1-steplong@quicinc.com> [rth: Rename the trans_* functions to *_sve2.] Signed-off-by: Richard Henderson --- target/arm/sve.decode | 11 +++++++++-- target/arm/translate-sve.c | 35 ++++++++++++++++++++++++++++++----- 2 files changed, 39 insertions(+), 7 deletions(-) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index cb2ee86228..67b6466a1e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -494,10 +494,14 @@ CPY_z_i 00000101 .. 01 .... 00 . ........ ..... @rdn_pg4 imm=%sh8_i8s ### SVE Permute - Extract Group -# SVE extract vector (immediate offset) +# SVE extract vector (destructive) EXT 00000101 001 ..... 000 ... rm:5 rd:5 \ &rrri rn=%reg_movprfx imm=%imm8_16_10 +# SVE2 extract vector (constructive) +EXT_sve2 00000101 011 ..... 000 ... rn:5 rd:5 \ + &rri imm=%imm8_16_10 + ### SVE Permute - Unpredicated Group # SVE broadcast general register @@ -588,9 +592,12 @@ REVH 00000101 .. 1001 01 100 ... ..... ..... @rd_pg_rn REVW 00000101 .. 1001 10 100 ... ..... ..... @rd_pg_rn RBIT 00000101 .. 1001 11 100 ... ..... ..... @rd_pg_rn -# SVE vector splice (predicated) +# SVE vector splice (predicated, destructive) SPLICE 00000101 .. 101 100 100 ... ..... ..... @rdn_pg_rm +# SVE2 vector splice (predicated, constructive) +SPLICE_sve2 00000101 .. 101 101 100 ... ..... ..... @rd_pg_rn + ### SVE Select Vectors Group # SVE select vector elements (predicated) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0afae9646f..e9feb05da3 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -2266,18 +2266,18 @@ static bool trans_CPY_z_i(DisasContext *s, arg_CPY_z_i *a) *** SVE Permute Extract Group */ -static bool trans_EXT(DisasContext *s, arg_EXT *a) +static bool do_EXT(DisasContext *s, int rd, int rn, int rm, int imm) { if (!sve_access_check(s)) { return true; } unsigned vsz = vec_full_reg_size(s); - unsigned n_ofs = a->imm >= vsz ? 0 : a->imm; + unsigned n_ofs = imm >= vsz ? 0 : imm; unsigned n_siz = vsz - n_ofs; - unsigned d = vec_full_reg_offset(s, a->rd); - unsigned n = vec_full_reg_offset(s, a->rn); - unsigned m = vec_full_reg_offset(s, a->rm); + unsigned d = vec_full_reg_offset(s, rd); + unsigned n = vec_full_reg_offset(s, rn); + unsigned m = vec_full_reg_offset(s, rm); /* Use host vector move insns if we have appropriate sizes * and no unfortunate overlap. @@ -2296,6 +2296,19 @@ static bool trans_EXT(DisasContext *s, arg_EXT *a) return true; } +static bool trans_EXT(DisasContext *s, arg_EXT *a) +{ + return do_EXT(s, a->rd, a->rn, a->rm, a->imm); +} + +static bool trans_EXT_sve2(DisasContext *s, arg_rri *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_EXT(s, a->rd, a->rn, (a->rn + 1) % 32, a->imm); +} + /* *** SVE Permute - Unpredicated Group */ @@ -3013,6 +3026,18 @@ static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a) return true; } +static bool trans_SPLICE_sve2(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzp(s, gen_helper_sve_splice, + a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz); + } + return true; +} + /* *** SVE Integer Compare - Vectors Group */ From patchwork Fri Apr 16 21:02:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422636 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp739353jaf; Fri, 16 Apr 2021 14:41:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzF0GgWhsFfRWw1HtuVVkQeGQ8+q6DRRaUtt2pIZduPUlgjVW+351Sy3ZsoLZ7Qizh/Q83K X-Received: by 2002:a25:888b:: with SMTP id d11mr1744576ybl.385.1618609314835; Fri, 16 Apr 2021 14:41:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609314; cv=none; d=google.com; s=arc-20160816; b=hnN2PIUiQFht2UztsGNiUatWiIj2n2TXTjx43pPj1A18jvFMJlMVY0kCk4m4IbRepJ 4abKEeT7++8m2pNYg2kizJ8ZxTS0scYE7KJRi9ortWEU+vn6ifVsd+swhY0O0t8ORUkx 2D2CoOb2UmOrtUT4Vhxejk0+baKAl9H0SrSY/8Ix4buYevcbBFaw+HFGFrCAg1h3u/8a 4a0XQYlQJIRwWxhEiHA6Af5S7hY5zMTHkNcWC+JUN/fXx/ICTsDaB47s+vFRleF3iFJF kX757PZNwRKqsQ9f2Tp2xwD8f/jGCRGgVRWEs60yhyF8f2QIrBgtmC0jfghQ0Ijemqnr 0Xhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=8zgKYnfxZBw0Kr430/mX7qtdsnSyYLs5XMO3Y50nXxM=; b=J2VRcodSW3cCuPIWrHIVS5qLZWuLITp6OoilejuTZeektVOt1ZF0qub2lH7OGxcFoC VnOHbhgmll8PB6iSdbBUXsypWgLqgpIGE9oOI3N2jvdlGsXhGzP3AQ/QeoGXEJ9zbAEV Ec2keDW8nDrHDFOUjB7JxpaGRkaZ/k3Tin3kfbWPXZhL0vyFueQk8yxbYh/wcN9UGbes XzPr7PTm8W3JWHAdrWwXobKaujNVQPfrcuZG2b1M/5+7O4U9dzt00GU7fPNhMAPYQP4v qVo0V8MFjtbrLWxaAQHndvWg/3QuqiWP2eAGCDLpc9BhCVE/Ibpk1jcdCFITnl9R+cWt zohQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=qDgzQCLF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n125si6801522ybn.14.2021.04.16.14.41.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:41:54 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=qDgzQCLF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45386 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWDq-0003Tn-2v for patch@linaro.org; Fri, 16 Apr 2021 17:41:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44964) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0005k0-PO for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:21 -0400 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]:35686) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdB-0001gC-Ed for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:21 -0400 Received: by mail-pf1-x431.google.com with SMTP id h15so1450604pfv.2 for ; Fri, 16 Apr 2021 14:04:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8zgKYnfxZBw0Kr430/mX7qtdsnSyYLs5XMO3Y50nXxM=; b=qDgzQCLFjeB4D60hHJyBZd31B4bV7w/gn23am5OC3YFUb1rdniisC1gwyWlT05fcRu 9HezJenVAmXhXc9R23BJOXbJDxD0beJMwARUQvsiot7hT86B0dSHz35eLdRnydt+XfTs O/fhWZKbIXTyTEMEw5zfdCas7juduhYF3WKUXE8c9Ud1tK5mUI5reYhQEOgIzbrNBjfK ic6o2GUUqLcqk3QdpMbHyjuxf3N6EbMZXZJOXxMK1WtEJcHsTf7ax8LQ7xzZaYtaYsx2 Y2BRsS5r0FK3ni/xJwNkT+AgLZtlsD+pGbmR6rZoqpYNZxi0DEaPq3qGqc0MjajHsu5Z Xjfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8zgKYnfxZBw0Kr430/mX7qtdsnSyYLs5XMO3Y50nXxM=; b=miVDpl4Km3WpC8iVU0ly79eJv/Z9L3UDOKsXZWAnyAMLAekJk6Fm299cHbc5G1ixhW 2BuXSLuRNb9KtNMXhfbo9EUD+3hebC/f+dsasAELUO67qbNkz7YhaDHKUZ+/o9I4XQeV aI/zGfUD9R6SpVOvIlYIK57wCM0z01BoSgeqhjCFK/M7hr7C5VEm5e/aCUf2EMAaOhFk Y/9+aR5lf2Jz7qNZcN50SpHyj1sz/6lMIHGZ1z4y7CFFa0ypkgH7T+cuyLALoU62eeBU u8VBDNWjR7C4eCuuuChukZVWSbU61FzpscX9KrHYy05Ca1w0jZlnp163rQHuWq757sU8 57YQ== X-Gm-Message-State: AOAM533ImsAAo+JhaK8CB8bwTqwcHXCUWgM1SNn0Kgb9WbBk0lBx96jO oE/8iQE9b63Rqe4+e2eEPB/2gPIxM27QWg== X-Received: by 2002:a65:5c42:: with SMTP id v2mr839163pgr.339.1618607040041; Fri, 16 Apr 2021 14:04:00 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.03.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:03:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 48/81] target/arm: Pass separate addend to {U, S}DOT helpers Date: Fri, 16 Apr 2021 14:02:07 -0700 Message-Id: <20210416210240.1591291-49-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For SVE, we potentially have a 4th argument coming from the movprfx instruction. Currently we do not optimize movprfx, so the problem is not visible. Signed-off-by: Richard Henderson --- v4: Fix double addition (zhiwei). --- target/arm/helper.h | 20 +++--- target/arm/sve.decode | 7 +- target/arm/translate-a64.c | 15 +++- target/arm/translate-sve.c | 13 ++-- target/arm/vec_helper.c | 120 ++++++++++++++++++-------------- target/arm/translate-neon.c.inc | 10 +-- 6 files changed, 109 insertions(+), 76 deletions(-) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 23a7ec5638..f4b092ee1c 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -608,15 +608,19 @@ DEF_HELPER_FLAGS_5(sve2_sqrdmlah_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcaddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 67b6466a1e..04ef38f148 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -756,12 +756,13 @@ UMIN_zzi 00100101 .. 101 011 110 ........ ..... @rdn_i8u MUL_zzi 00100101 .. 110 000 110 ........ ..... @rdn_i8s # SVE integer dot product (unpredicated) -DOT_zzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 ra=%reg_movprfx +DOT_zzzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 \ + ra=%reg_movprfx # SVE integer dot product (indexed) -DOT_zzx 01000100 101 index:2 rm:3 00000 u:1 rn:5 rd:5 \ +DOT_zzxw 01000100 101 index:2 rm:3 00000 u:1 rn:5 rd:5 \ sz=0 ra=%reg_movprfx -DOT_zzx 01000100 111 index:1 rm:4 00000 u:1 rn:5 rd:5 \ +DOT_zzxw 01000100 111 index:1 rm:4 00000 u:1 rn:5 rd:5 \ sz=1 ra=%reg_movprfx # SVE floating-point complex add (predicated) diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index cd8408e84c..004ff8c019 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -698,6 +698,17 @@ static void gen_gvec_op3_qc(DisasContext *s, bool is_q, int rd, int rn, tcg_temp_free_ptr(qc_ptr); } +/* Expand a 4-operand operation using an out-of-line helper. */ +static void gen_gvec_op4_ool(DisasContext *s, bool is_q, int rd, int rn, + int rm, int ra, int data, gen_helper_gvec_4 *fn) +{ + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), + is_q ? 16 : 8, vec_full_reg_size(s), data, fn); +} + /* Set ZF and NF based on a 64 bit result. This is alas fiddlier * than the 32 bit equivalent. */ @@ -12198,7 +12209,7 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) return; case 0x2: /* SDOT / UDOT */ - gen_gvec_op3_ool(s, is_q, rd, rn, rm, 0, + gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, 0, u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b); return; @@ -13457,7 +13468,7 @@ static void disas_simd_indexed(DisasContext *s, uint32_t insn) switch (16 * u + opcode) { case 0x0e: /* SDOT */ case 0x1e: /* UDOT */ - gen_gvec_op3_ool(s, is_q, rd, rn, rm, index, + gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, index, u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b); return; diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index e9feb05da3..0317d5386a 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3800,28 +3800,29 @@ DO_ZZI(UMIN, umin) #undef DO_ZZI -static bool trans_DOT_zzz(DisasContext *s, arg_DOT_zzz *a) +static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) { - static gen_helper_gvec_3 * const fns[2][2] = { + static gen_helper_gvec_4 * const fns[2][2] = { { gen_helper_gvec_sdot_b, gen_helper_gvec_sdot_h }, { gen_helper_gvec_udot_b, gen_helper_gvec_udot_h } }; if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, 0); + gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, a->ra, 0); } return true; } -static bool trans_DOT_zzx(DisasContext *s, arg_DOT_zzx *a) +static bool trans_DOT_zzxw(DisasContext *s, arg_DOT_zzxw *a) { - static gen_helper_gvec_3 * const fns[2][2] = { + static gen_helper_gvec_4 * const fns[2][2] = { { gen_helper_gvec_sdot_idx_b, gen_helper_gvec_sdot_idx_h }, { gen_helper_gvec_udot_idx_b, gen_helper_gvec_udot_idx_h } }; if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, a->index); + gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, + a->ra, a->index); } return true; } diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index a3d80ecad0..f88e572132 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -375,71 +375,76 @@ void HELPER(sve2_sqrdmlsh_d)(void *vd, void *vn, void *vm, * All elements are treated equally, no matter where they are. */ -void HELPER(gvec_sdot_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_b)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint32_t *d = vd; + int32_t *d = vd, *a = va; int8_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 4; ++i) { - d[i] += n[i * 4 + 0] * m[i * 4 + 0] - + n[i * 4 + 1] * m[i * 4 + 1] - + n[i * 4 + 2] * m[i * 4 + 2] - + n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + n[i * 4 + 0] * m[i * 4 + 0] + + n[i * 4 + 1] * m[i * 4 + 1] + + n[i * 4 + 2] * m[i * 4 + 2] + + n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_b)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint32_t *d = vd; + uint32_t *d = vd, *a = va; uint8_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 4; ++i) { - d[i] += n[i * 4 + 0] * m[i * 4 + 0] - + n[i * 4 + 1] * m[i * 4 + 1] - + n[i * 4 + 2] * m[i * 4 + 2] - + n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + n[i * 4 + 0] * m[i * 4 + 0] + + n[i * 4 + 1] * m[i * 4 + 1] + + n[i * 4 + 2] * m[i * 4 + 2] + + n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_sdot_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint64_t *d = vd; + int64_t *d = vd, *a = va; int16_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - d[i] += (int64_t)n[i * 4 + 0] * m[i * 4 + 0] - + (int64_t)n[i * 4 + 1] * m[i * 4 + 1] - + (int64_t)n[i * 4 + 2] * m[i * 4 + 2] - + (int64_t)n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + (int64_t)n[i * 4 + 0] * m[i * 4 + 0] + + (int64_t)n[i * 4 + 1] * m[i * 4 + 1] + + (int64_t)n[i * 4 + 2] * m[i * 4 + 2] + + (int64_t)n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint64_t *d = vd; + uint64_t *d = vd, *a = va; uint16_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - d[i] += (uint64_t)n[i * 4 + 0] * m[i * 4 + 0] - + (uint64_t)n[i * 4 + 1] * m[i * 4 + 1] - + (uint64_t)n[i * 4 + 2] * m[i * 4 + 2] - + (uint64_t)n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + (uint64_t)n[i * 4 + 0] * m[i * 4 + 0] + + (uint64_t)n[i * 4 + 1] * m[i * 4 + 1] + + (uint64_t)n[i * 4 + 2] * m[i * 4 + 2] + + (uint64_t)n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; intptr_t index = simd_data(desc); - uint32_t *d = vd; + int32_t *d = vd, *a = va; int8_t *n = vn; int8_t *m_indexed = (int8_t *)vm + H4(index) * 4; @@ -455,10 +460,11 @@ void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) int8_t m3 = m_indexed[i * 4 + 3]; do { - d[i] += n[i * 4 + 0] * m0 - + n[i * 4 + 1] * m1 - + n[i * 4 + 2] * m2 - + n[i * 4 + 3] * m3; + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); } while (++i < segend); segend = i + 4; } while (i < opr_sz_4); @@ -466,11 +472,12 @@ void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; intptr_t index = simd_data(desc); - uint32_t *d = vd; + uint32_t *d = vd, *a = va; uint8_t *n = vn; uint8_t *m_indexed = (uint8_t *)vm + H4(index) * 4; @@ -486,10 +493,11 @@ void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) uint8_t m3 = m_indexed[i * 4 + 3]; do { - d[i] += n[i * 4 + 0] * m0 - + n[i * 4 + 1] * m1 - + n[i * 4 + 2] * m2 - + n[i * 4 + 3] * m3; + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); } while (++i < segend); segend = i + 4; } while (i < opr_sz_4); @@ -497,11 +505,12 @@ void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc), opr_sz_8 = opr_sz / 8; intptr_t index = simd_data(desc); - uint64_t *d = vd; + int64_t *d = vd, *a = va; int16_t *n = vn; int16_t *m_indexed = (int16_t *)vm + index * 4; @@ -509,30 +518,33 @@ void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) * Process the entire segment all at once, writing back the results * only after we've consumed all of the inputs. */ - for (i = 0; i < opr_sz_8 ; i += 2) { - uint64_t d0, d1; + for (i = 0; i < opr_sz_8; i += 2) { + int64_t d0, d1; - d0 = n[i * 4 + 0] * (int64_t)m_indexed[i * 4 + 0]; + d0 = a[i + 0]; + d0 += n[i * 4 + 0] * (int64_t)m_indexed[i * 4 + 0]; d0 += n[i * 4 + 1] * (int64_t)m_indexed[i * 4 + 1]; d0 += n[i * 4 + 2] * (int64_t)m_indexed[i * 4 + 2]; d0 += n[i * 4 + 3] * (int64_t)m_indexed[i * 4 + 3]; - d1 = n[i * 4 + 4] * (int64_t)m_indexed[i * 4 + 0]; + + d1 = a[i + 1]; + d1 += n[i * 4 + 4] * (int64_t)m_indexed[i * 4 + 0]; d1 += n[i * 4 + 5] * (int64_t)m_indexed[i * 4 + 1]; d1 += n[i * 4 + 6] * (int64_t)m_indexed[i * 4 + 2]; d1 += n[i * 4 + 7] * (int64_t)m_indexed[i * 4 + 3]; - d[i + 0] += d0; - d[i + 1] += d1; + d[i + 0] = d0; + d[i + 1] = d1; } - clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_idx_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc), opr_sz_8 = opr_sz / 8; intptr_t index = simd_data(desc); - uint64_t *d = vd; + uint64_t *d = vd, *a = va; uint16_t *n = vn; uint16_t *m_indexed = (uint16_t *)vm + index * 4; @@ -540,22 +552,24 @@ void HELPER(gvec_udot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) * Process the entire segment all at once, writing back the results * only after we've consumed all of the inputs. */ - for (i = 0; i < opr_sz_8 ; i += 2) { + for (i = 0; i < opr_sz_8; i += 2) { uint64_t d0, d1; - d0 = n[i * 4 + 0] * (uint64_t)m_indexed[i * 4 + 0]; + d0 = a[i + 0]; + d0 += n[i * 4 + 0] * (uint64_t)m_indexed[i * 4 + 0]; d0 += n[i * 4 + 1] * (uint64_t)m_indexed[i * 4 + 1]; d0 += n[i * 4 + 2] * (uint64_t)m_indexed[i * 4 + 2]; d0 += n[i * 4 + 3] * (uint64_t)m_indexed[i * 4 + 3]; - d1 = n[i * 4 + 4] * (uint64_t)m_indexed[i * 4 + 0]; + + d1 = a[i + 1]; + d1 += n[i * 4 + 4] * (uint64_t)m_indexed[i * 4 + 0]; d1 += n[i * 4 + 5] * (uint64_t)m_indexed[i * 4 + 1]; d1 += n[i * 4 + 6] * (uint64_t)m_indexed[i * 4 + 2]; d1 += n[i * 4 + 7] * (uint64_t)m_indexed[i * 4 + 3]; - d[i + 0] += d0; - d[i + 1] += d1; + d[i + 0] = d0; + d[i + 1] = d1; } - clear_tail(d, opr_sz, simd_maxsz(desc)); } diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index a02b8369a1..8ecfef1567 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -221,7 +221,7 @@ static bool trans_VCADD(DisasContext *s, arg_VCADD *a) static bool trans_VDOT(DisasContext *s, arg_VDOT *a) { int opr_sz; - gen_helper_gvec_3 *fn_gvec; + gen_helper_gvec_4 *fn_gvec; if (!dc_isar_feature(aa32_dp, s)) { return false; @@ -243,9 +243,10 @@ static bool trans_VDOT(DisasContext *s, arg_VDOT *a) opr_sz = (1 + a->q) * 8; fn_gvec = a->u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b; - tcg_gen_gvec_3_ool(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->vm), + vfp_reg_offset(1, a->vd), opr_sz, opr_sz, 0, fn_gvec); return true; } @@ -323,7 +324,7 @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) { - gen_helper_gvec_3 *fn_gvec; + gen_helper_gvec_4 *fn_gvec; int opr_sz; TCGv_ptr fpst; @@ -348,9 +349,10 @@ static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) fn_gvec = a->u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b; opr_sz = (1 + a->q) * 8; fpst = fpstatus_ptr(FPST_STD); - tcg_gen_gvec_3_ool(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->rm), + vfp_reg_offset(1, a->vd), opr_sz, opr_sz, a->index, fn_gvec); tcg_temp_free_ptr(fpst); return true; From patchwork Fri Apr 16 21:02:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422661 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp749974jaf; Fri, 16 Apr 2021 15:02:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwR5iQssHruMyppMowfP7ARFQMUwKrylL1Gv10W2cllMqQyWKcHFbwXR40Qm4yH5TLl0nql X-Received: by 2002:ab0:57ca:: with SMTP id u10mr1269121uac.71.1618610579651; Fri, 16 Apr 2021 15:02:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610579; cv=none; d=google.com; s=arc-20160816; b=EyC8EiivzjzIzh6Uw6+S2u8DQPjYFcZyLWVnQ51iaGbkiqBf0f8bTlhuOVZbAvC0Ou WZgGCQ2WMD4L+yGLUWWXWKdwKj0pqoalTBFK2ILc++PIt8kXDh52/fcSLSWx4bdv+oew 6pXpv82wlQ+1KrBmKBSY3mLrS0ggkdfPSoUmpWJvCYTePJg0WszucNJWKbNCF6SyRvO+ YX++bWi24IWB6wFa+rIeQppJn0Q7PlPa4hxZ3TCcdYacLPWfM2vcDueVAboR9wz2adhG qw1ehBLhj2wQTRvk8kSujQolSCDJa1w5sMoUkfoo8a+2SiUJQU7yZWPKDadVxYwXN9jo ks9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=tTGUPYLRanHesNN2/5sC3VdhU409Nhotte30zMeHJNQ=; b=csyEIbNJf2sdguerEFoiZhyq273v0pqoC8KljE3mIMGA/8Ca0saPYdyxmMUpgJd4lB vQP1DhRCpXz+WEmS/g6fLDt4XeIwhxOZGMcQ5rdIJlaa1ekemLSWj143MQ9bwMADCM1h /WjWOYvCXBPSj/EAO7LeLqqQnpSwATdtssxFzTCA+rFhalIsF4LYjhFOTONq9segn8qJ UC9FvKv9hGXuRSfelF68B0B37xtw/cgResOKHFGT/87IodK7eEIkp5qZ5gkb96g8u5hy d03v2C6UMkzFzsD95REj6MEfFiahM2MEJRzQQKNzfnyyZF0ZJdMryWRNlPXfm/RdgMEQ l+eA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VvfzCxDH; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 43si4025268uav.44.2021.04.16.15.02.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:02:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VvfzCxDH; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53596 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWYE-0004Wo-VT for patch@linaro.org; Fri, 16 Apr 2021 18:02:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45106) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVda-0005uY-IQ for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:26 -0400 Received: from mail-pg1-x532.google.com ([2607:f8b0:4864:20::532]:37616) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdF-0001gs-97 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:26 -0400 Received: by mail-pg1-x532.google.com with SMTP id p2so4545670pgh.4 for ; Fri, 16 Apr 2021 14:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tTGUPYLRanHesNN2/5sC3VdhU409Nhotte30zMeHJNQ=; b=VvfzCxDHgpuyrRpWx5frnhwPwDwGxtlX9BWY2U9WXQq7SDg9hMYr7wpZOoTK5WjwPm 9EaIfX7CGDxv6afqo7wsmH6sbXMCprZXCWNZVPRug9xDjeZzN0XHdxtXKx2tYl1KwV0w W+juRNrYYm3ZloeJrvdrHBjvngW3CUzxfvOTO7tX5C1jNErinlnGj6TlxHNUNEzliu1u 3GKN8lCMg96I1qMxgFG6yBTtBuIMgKp0bKzXvK8mOaIXHFo0oC9/BeAU6G1O9HjEVRgG wSonUF8RpcKAv818udkob+7LzXv0Ty+z7kEuFgznz3Y9XsjjSsdhk0qfCimqQBZUAE1m oooQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tTGUPYLRanHesNN2/5sC3VdhU409Nhotte30zMeHJNQ=; b=OynowGZFPbnqptEoPG/8dbeeAyWt+2KD3XkNQOpYtXh779klbEoxFuIbrl0wn6nJmO tT7JHDNgA6ZV0pF8ptUY8dgGEcFhujdCRKg8r+USSMS1vOMpePro/RDMwzFSxYEmGcB9 WbUYAMmOnGVYdOVbwa2vJAKPY5O1OvOZWAz7Mg3V0kO0lMasIG/1IJLEYoyEHd56Cneo 25GCr6tFxRDZ/ecUGLrKnpl010ad0EquJS8/Wz8RQvEPMem51kIFiHYUKg/b5Bf6mfoe vXrJij1UUiz4W9Pts23GbeVjdtwZ1py7UVM3nlIUGTBqARDJBE2byOvjNt6njgdJ1g60 vxPw== X-Gm-Message-State: AOAM5338Au1M6veyWVKuSEQCBfIgzcP7rF7Kj34LsI21FchvPdLFCHGN Jw/+XIN7H5ktI5jLFl64sXd63k7Tcvgu1Q== X-Received: by 2002:aa7:904e:0:b029:25a:4469:222a with SMTP id n14-20020aa7904e0000b029025a4469222amr3746101pfo.72.1618607041111; Fri, 16 Apr 2021 14:04:01 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 49/81] target/arm: Pass separate addend to FCMLA helpers Date: Fri, 16 Apr 2021 14:02:08 -0700 Message-Id: <20210416210240.1591291-50-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::532; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x532.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For SVE, we potentially have a 4th argument coming from the movprfx instruction. Currently we do not optimize movprfx, so the problem is not visible. Signed-off-by: Richard Henderson --- target/arm/helper.h | 20 ++++++------- target/arm/translate-a64.c | 28 ++++++++++++++---- target/arm/translate-sve.c | 5 ++-- target/arm/vec_helper.c | 50 +++++++++++++-------------------- target/arm/translate-neon.c.inc | 10 ++++--- 5 files changed, 62 insertions(+), 51 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.h b/target/arm/helper.h index f4b092ee1c..72c5bf6aca 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -629,16 +629,16 @@ DEF_HELPER_FLAGS_5(gvec_fcadds, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_fcaddd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlah, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlah_idx, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlas, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlas_idx, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlad, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlah, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlah_idx, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlas, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlas_idx, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlad, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(neon_paddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(neon_pmaxh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 004ff8c019..f45b81e56d 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -709,6 +709,23 @@ static void gen_gvec_op4_ool(DisasContext *s, bool is_q, int rd, int rn, is_q ? 16 : 8, vec_full_reg_size(s), data, fn); } +/* + * Expand a 4-operand + fpstatus pointer + simd data value operation using + * an out-of-line helper. + */ +static void gen_gvec_op4_fpst(DisasContext *s, bool is_q, int rd, int rn, + int rm, int ra, bool is_fp16, int data, + gen_helper_gvec_4_ptr *fn) +{ + TCGv_ptr fpst = fpstatus_ptr(is_fp16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), fpst, + is_q ? 16 : 8, vec_full_reg_size(s), data, fn); + tcg_temp_free_ptr(fpst); +} + /* Set ZF and NF based on a 64 bit result. This is alas fiddlier * than the 32 bit equivalent. */ @@ -12220,15 +12237,15 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) rot = extract32(opcode, 0, 2); switch (size) { case 1: - gen_gvec_op3_fpst(s, is_q, rd, rn, rm, true, rot, + gen_gvec_op4_fpst(s, is_q, rd, rn, rm, rd, true, rot, gen_helper_gvec_fcmlah); break; case 2: - gen_gvec_op3_fpst(s, is_q, rd, rn, rm, false, rot, + gen_gvec_op4_fpst(s, is_q, rd, rn, rm, rd, false, rot, gen_helper_gvec_fcmlas); break; case 3: - gen_gvec_op3_fpst(s, is_q, rd, rn, rm, false, rot, + gen_gvec_op4_fpst(s, is_q, rd, rn, rm, rd, false, rot, gen_helper_gvec_fcmlad); break; default: @@ -13479,9 +13496,10 @@ static void disas_simd_indexed(DisasContext *s, uint32_t insn) { int rot = extract32(insn, 13, 2); int data = (index << 2) | rot; - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd), + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, rd), vec_full_reg_offset(s, rn), - vec_full_reg_offset(s, rm), fpst, + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, rd), fpst, is_q ? 16 : 8, vec_full_reg_size(s), data, size == MO_64 ? gen_helper_gvec_fcmlas_idx diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0317d5386a..ffae6884d2 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4383,7 +4383,7 @@ static bool trans_FCMLA_zpzzz(DisasContext *s, arg_FCMLA_zpzzz *a) static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a) { - static gen_helper_gvec_3_ptr * const fns[2] = { + static gen_helper_gvec_4_ptr * const fns[2] = { gen_helper_gvec_fcmlah_idx, gen_helper_gvec_fcmlas_idx, }; @@ -4393,9 +4393,10 @@ static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a) if (sve_access_check(s)) { unsigned vsz = vec_full_reg_size(s); TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), vec_full_reg_offset(s, a->rn), vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), status, vsz, vsz, a->index * 4 + a->rot, fns[a->esz - 1]); diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index f88e572132..b19877e0d3 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -657,13 +657,11 @@ void HELPER(gvec_fcaddd)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float16 *d = vd; - float16 *n = vn; - float16 *m = vm; + float16 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -680,19 +678,17 @@ void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, float16 e4 = e2; float16 e3 = m[H2(i + 1 - flip)] ^ neg_imag; - d[H2(i)] = float16_muladd(e2, e1, d[H2(i)], 0, fpst); - d[H2(i + 1)] = float16_muladd(e4, e3, d[H2(i + 1)], 0, fpst); + d[H2(i)] = float16_muladd(e2, e1, a[H2(i)], 0, fpst); + d[H2(i + 1)] = float16_muladd(e4, e3, a[H2(i + 1)], 0, fpst); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float16 *d = vd; - float16 *n = vn; - float16 *m = vm; + float16 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -716,20 +712,18 @@ void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, float16 e2 = n[H2(j + flip)]; float16 e4 = e2; - d[H2(j)] = float16_muladd(e2, e1, d[H2(j)], 0, fpst); - d[H2(j + 1)] = float16_muladd(e4, e3, d[H2(j + 1)], 0, fpst); + d[H2(j)] = float16_muladd(e2, e1, a[H2(j)], 0, fpst); + d[H2(j + 1)] = float16_muladd(e4, e3, a[H2(j + 1)], 0, fpst); } } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float32 *d = vd; - float32 *n = vn; - float32 *m = vm; + float32 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -746,19 +740,17 @@ void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, float32 e4 = e2; float32 e3 = m[H4(i + 1 - flip)] ^ neg_imag; - d[H4(i)] = float32_muladd(e2, e1, d[H4(i)], 0, fpst); - d[H4(i + 1)] = float32_muladd(e4, e3, d[H4(i + 1)], 0, fpst); + d[H4(i)] = float32_muladd(e2, e1, a[H4(i)], 0, fpst); + d[H4(i + 1)] = float32_muladd(e4, e3, a[H4(i + 1)], 0, fpst); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float32 *d = vd; - float32 *n = vn; - float32 *m = vm; + float32 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -782,20 +774,18 @@ void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, float32 e2 = n[H4(j + flip)]; float32 e4 = e2; - d[H4(j)] = float32_muladd(e2, e1, d[H4(j)], 0, fpst); - d[H4(j + 1)] = float32_muladd(e4, e3, d[H4(j + 1)], 0, fpst); + d[H4(j)] = float32_muladd(e2, e1, a[H4(j)], 0, fpst); + d[H4(j + 1)] = float32_muladd(e4, e3, a[H4(j + 1)], 0, fpst); } } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float64 *d = vd; - float64 *n = vn; - float64 *m = vm; + float64 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint64_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -812,8 +802,8 @@ void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, float64 e4 = e2; float64 e3 = m[i + 1 - flip] ^ neg_imag; - d[i] = float64_muladd(e2, e1, d[i], 0, fpst); - d[i + 1] = float64_muladd(e4, e3, d[i + 1], 0, fpst); + d[i] = float64_muladd(e2, e1, a[i], 0, fpst); + d[i + 1] = float64_muladd(e4, e3, a[i + 1], 0, fpst); } clear_tail(d, opr_sz, simd_maxsz(desc)); } diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index 8ecfef1567..6f418bd8de 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -146,7 +146,7 @@ static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) { int opr_sz; TCGv_ptr fpst; - gen_helper_gvec_3_ptr *fn_gvec_ptr; + gen_helper_gvec_4_ptr *fn_gvec_ptr; if (!dc_isar_feature(aa32_vcma, s) || (a->size == MO_16 && !dc_isar_feature(aa32_fp16_arith, s))) { @@ -171,9 +171,10 @@ static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) fpst = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD); fn_gvec_ptr = (a->size == MO_16) ? gen_helper_gvec_fcmlah : gen_helper_gvec_fcmlas; - tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ptr(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->vm), + vfp_reg_offset(1, a->vd), fpst, opr_sz, opr_sz, a->rot, fn_gvec_ptr); tcg_temp_free_ptr(fpst); @@ -284,7 +285,7 @@ static bool trans_VFML(DisasContext *s, arg_VFML *a) static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) { - gen_helper_gvec_3_ptr *fn_gvec_ptr; + gen_helper_gvec_4_ptr *fn_gvec_ptr; int opr_sz; TCGv_ptr fpst; @@ -313,9 +314,10 @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) gen_helper_gvec_fcmlah_idx : gen_helper_gvec_fcmlas_idx; opr_sz = (1 + a->q) * 8; fpst = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD); - tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ptr(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->vm), + vfp_reg_offset(1, a->vd), fpst, opr_sz, opr_sz, (a->index << 2) | a->rot, fn_gvec_ptr); tcg_temp_free_ptr(fpst); From patchwork Fri Apr 16 21:02:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422634 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp738027jaf; Fri, 16 Apr 2021 14:39:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyfygqokNRKeJTFtnEZiG5IZl8rnUwhTwfVIumR1JUJKBAnAXhKK2R/Uglouno5vFUe99NC X-Received: by 2002:a92:2e13:: with SMTP id v19mr8544181ile.198.1618609152140; Fri, 16 Apr 2021 14:39:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609152; cv=none; d=google.com; s=arc-20160816; b=QsGMQOEzdOsz8eLr+9c8VJpndFgdWJTjHZXt1R5GlRuasOCQKzl9S9RyPzoXqPUCth u3nrOLtDa6XBImyI1yv0BJH9sttgbRd5srOLyOM0zvVr7gyg/MQcfF36VLf3HVk80wa8 BJbDGqztnvMDfq2WGaU0hp49BzSwYgKf3j+yuXh5w60jD4xcH3I0xRCY6Yv562gdz3Og bu+y3H0gNTD8hWJFqNCpZZcBCpA/DoHB3xv6JkUbiqvlTPk1IonbD1x02nWD9EQxJfu1 IondDh5Cb20H2F5QADX02GSZ2uw73+BUJcRfMjkqoTMJdTEuN5hgXyQOJvWJ3DpGd37Y FNyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=RfpadBU7KlY8uj8mgdV4cd4GjWhl0iserVzzK3wcZFQ=; b=mZo/j/IMgqaZk1eFo0LLE03qNZZ4NHnz637ePMlymzUHAV3ZXz/jlCuBXu/QmX52mx Hsj4hFlkIqoKvCNv1fYjzl1nOQWE9NLvwBTXxliArRi/alCzbYJFqCcctiddMeGOmIQO +EEXj8Sqo8Npp25uPL73cx+YhIEIWKaLdXNqkNmVRpsKu1f+KJ9dpTQR42Y4bzVJ/fiv XpOSWAEDv6Gga90FuOw3r7t9y1Q1Pq/PgqG24G9J7ULXM/cjATLen31HJT/rwZszrHtF EajabBba4k8hpSYepZhUS56rCidaziwOrG5FoIcxg+qCrtOkPUWdj3muevjSBY4NWz62 FSfA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kg7MgVoC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r5si3142814ill.85.2021.04.16.14.39.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:39:12 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kg7MgVoC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36688 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWBD-0008A4-Gn for patch@linaro.org; Fri, 16 Apr 2021 17:39:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45010) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdX-0005mn-Bz for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:23 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]:39679) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdF-0001ha-86 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:23 -0400 Received: by mail-pf1-x42a.google.com with SMTP id c17so19147539pfn.6 for ; Fri, 16 Apr 2021 14:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RfpadBU7KlY8uj8mgdV4cd4GjWhl0iserVzzK3wcZFQ=; b=kg7MgVoCdIUtkq831P9sBFKCsyxYC8F8ifW7uLgIrYew/4gXVD6iMtSessLGWZr+PQ L8N30RdkLoszL2Ei82Yb9I/SwkE3EUlMwMjH+yDRvDKsvyAk2lcXeKRGm02VQ0FbUZoG vI4iqDHOdP9Pa3UVwHe01SMzvLtWjTeaAHYng6fbJ4MXJ0mGCQTIRYKlXhtNBO648fH5 /IL6M5BL65/AFrFYrAul0rKlM9q5xwiY48I+EejbShLbfgv8+wkSUWWwyZOE4NhD0xpG GE2D8NGm5NdpF3bFrytlDDzVw52JavPMnAS8cTM/nvkkZCF1/bmdrspWucj5tDl9sUuP 5/Gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RfpadBU7KlY8uj8mgdV4cd4GjWhl0iserVzzK3wcZFQ=; b=S74TANyeq90pPGfD2aqvfpTySLyp17PLqtY2oifL9LKoIXA07y6QUAvzTAcJDaNVHx SvI6pd6LccAEKsg/zmQfdo00PkqcKgF+PI6RTAgCJXRl10sRNG8EiwOROAvxqJrF3N+9 4Xqt8+AbSI4SyfZFoUiu9PqddHEG+rGNQMUlhXfQa+zQ2ATyaEZbkZdfLsPQHYqAGPRV 7a5scyX2azHy2Ve4u5sd9IGyjxpEBElXsI4IbJoc3Qe5BszsxKixsIOZbdG+wORlRTIK VB7l7K1c/ib3UTPzinY1+8Q0eZNtDiaeL92d61DpYyyJnenfhZzuzTV40EN/e/tB9iNZ qsnw== X-Gm-Message-State: AOAM530Jn+GF3dPW3MLRL8jlX7R8AuAmIZ7spcTfDcfoh07XrAyerawb tXxcykPu6CY1tkxsIvsMPF88PmoyekruWA== X-Received: by 2002:aa7:8d84:0:b029:1f8:3449:1bc6 with SMTP id i4-20020aa78d840000b02901f834491bc6mr9275949pfr.76.1618607042027; Fri, 16 Apr 2021 14:04:02 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 50/81] target/arm: Split out formats for 2 vectors + 1 index Date: Fri, 16 Apr 2021 14:02:09 -0700 Message-Id: <20210416210240.1591291-51-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Currently only used by FMUL, but will shortly be used more. Signed-off-by: Richard Henderson --- target/arm/sve.decode | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 04ef38f148..a504b55dad 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -67,6 +67,7 @@ &rri_esz rd rn imm esz &rrri_esz rd rn rm imm esz &rrr_esz rd rn rm esz +&rrx_esz rd rn rm index esz &rpr_esz rd pg rn esz &rpr_s rd pg rn s &rprr_s rd pg rn rm s @@ -245,6 +246,12 @@ @rpri_scatter_store ....... msz:2 .. imm:5 ... pg:3 rn:5 rd:5 \ &rpri_scatter_store +# Two registers and a scalar by N-bit index +@rrx_3 ........ .. . .. rm:3 ...... rn:5 rd:5 \ + &rrx_esz index=%index3_22_19 +@rrx_2 ........ .. . index:2 rm:3 ...... rn:5 rd:5 &rrx_esz +@rrx_1 ........ .. . index:1 rm:4 ...... rn:5 rd:5 &rrx_esz + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -792,10 +799,9 @@ FMLA_zzxz 01100100 111 index:1 rm:4 00000 sub:1 rn:5 rd:5 \ ### SVE FP Multiply Indexed Group # SVE floating-point multiply (indexed) -FMUL_zzx 01100100 0.1 .. rm:3 001000 rn:5 rd:5 \ - index=%index3_22_19 esz=1 -FMUL_zzx 01100100 101 index:2 rm:3 001000 rn:5 rd:5 esz=2 -FMUL_zzx 01100100 111 index:1 rm:4 001000 rn:5 rd:5 esz=3 +FMUL_zzx 01100100 0. 1 ..... 001000 ..... ..... @rrx_3 esz=1 +FMUL_zzx 01100100 10 1 ..... 001000 ..... ..... @rrx_2 esz=2 +FMUL_zzx 01100100 11 1 ..... 001000 ..... ..... @rrx_1 esz=3 ### SVE FP Fast Reduction Group From patchwork Fri Apr 16 21:02:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422658 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp747516jaf; Fri, 16 Apr 2021 14:59:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzYiyTm48Z/Cl5I8ZXIVBMjJ3fayY/QbE4SXBTBeCabeyfc5sf3Ms/jPf0Q8e3BtUp49sHw X-Received: by 2002:a25:7107:: with SMTP id m7mr1868527ybc.55.1618610354575; Fri, 16 Apr 2021 14:59:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610354; cv=none; d=google.com; s=arc-20160816; b=Hs1JqXCeseDhdkJXl4IPv0Uc4UXBwS7Yt+wrszr5NLs/CySZHQB8U2r9xmA6qWyBSx YtsIQdf13Xs9KiNN28eZcxJ/8I9aAmQVgVaqbnOv4e9MZtol2HO1v16rEwDi54rMZA8J 09lQ2ivWnswJ0RJQ85ZegZ44IGjAWHjtf8sZbx8j7sZ/elhV2hSqscnPA22xzxRxcMWr j3RCIbPQtvGa6BPVlmoEPsNfvguXv4NHAgRSWwpt8qmPye5ypoSg1w6So+KRXArej4Ti zC/mufgwG9P/wryjpOV3cD3NY3DlJzrucR0/wvJHEbfoV5CejqjgXGdqu/7L/b68p59P qq3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=zK5X/zS8IlLeFPtG0rq3DCnm5yvOzCFnx5s48eBO5HI=; b=sPo8IiCFlYGISon6TKTqbzNdsMTD0Mmw/dDyEKTcYI7ozmiXpMjK37kEx/0qjDc4yz GHUAyDrrYg9oEwyvLZFkmGkGF3aBlyzao88MOaBzhWkz8M31wxrdA2vrYmEKK+iYdAo7 aXaXf9RW7gF0Zm6iBUiz6Kxea2MBQrmlaUA73uYw0XTsYKpH1BmSyIlVHel5/30Xw+hM UN7vMKV+UW+ydUwfolXmL3peauWxCYhlC3GGmPM7SO/hFiQIcS9u2Sj94FexsdUhuaVK dGeECbxahJ3pfWwdD1mDc4QJVrrQLnTTZQcIZphMb3r5TytDP6GwEZCIynL8f4gVD8Yx MYHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mJXeLb7U; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g206si6228583ybg.281.2021.04.16.14.59.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:59:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mJXeLb7U; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42522 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWUb-0008QB-VU for patch@linaro.org; Fri, 16 Apr 2021 17:59:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45028) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdY-0005p4-7Q for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:24 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]:37479) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdF-0001hm-8R for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:23 -0400 Received: by mail-pf1-x42c.google.com with SMTP id o123so19143535pfb.4 for ; Fri, 16 Apr 2021 14:04:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zK5X/zS8IlLeFPtG0rq3DCnm5yvOzCFnx5s48eBO5HI=; b=mJXeLb7UUJXY8c+Pd7I54W4wnXKiMtEn1h6r82o5xJIX1Y0A1/GiF/iVpOAWG5bnQ6 lUtSVc2rPlszZNzbgsm1CiM22NSgtOMUAe+Vlzk8WybDhMUtL3N/4l6i/3rhRnMiN3Gx RQfbELqS/OtkUA9qcL6TDHLozPWrZUNa+JACERWZhWavWCiYq8guRChT/XGTAz5iE5iX jO9W6wRkmFWBXamjx1Jfi92eGBWRHHtEzGG30Tz5yBzjvpuqOjDjufp4m/nGMQceTEkd VggXO75ypNxg48xApu8axpfaOgQ+1U9Xs1cUWXjKjhSiRv8oQHdweig1fX95pVc3bVcS IZ9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zK5X/zS8IlLeFPtG0rq3DCnm5yvOzCFnx5s48eBO5HI=; b=gcwrb4ZhY9CdA7lFQvYt8haEZm9cmSDjUgnk8oPvi7k1Rvuz+zg1f/FKHmW4rhsZKy JKR5DQFaE2KSjcZVNm/UNV4CBH6RwoJDKg4mb+mPJfuoBfk831QezxWKJ4y/e3HeoPF5 Jr1CVsnvHXmTceV5Ttlsq9eYJA/THENu6MgD1BbZfn0WoMSqqpIOjgEqUHqE/AVIOOf9 5bjwjuOYcQ57UN/u+4CAYSGVu7jykKbuea+WnVWoaMQmKxYm2/tnwm6dRG/NWO/3XhRz beL3tRyYymKn9BhAeR445IZU+6VQc7TjJlzAlJhxH2HfrnLRoXLbEVcB+1A7CP53WOmz P8PA== X-Gm-Message-State: AOAM5338Fdbb/4LGgm/zHoAL/PaWce6BPllhstZuBCEU+5q4f3udA/D9 0k0L85p5iukDs0q6IiHL7/grE249gyrdlA== X-Received: by 2002:a63:575b:: with SMTP id h27mr883629pgm.180.1618607042874; Fri, 16 Apr 2021 14:04:02 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:02 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 51/81] target/arm: Split out formats for 3 vectors + 1 index Date: Fri, 16 Apr 2021 14:02:10 -0700 Message-Id: <20210416210240.1591291-52-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Used by FMLA and DOT, but will shortly be used more. Split FMLA from FMLS to avoid an extra sub field; similarly for SDOT from UDOT. Signed-off-by: Richard Henderson --- target/arm/sve.decode | 29 +++++++++++++++++++---------- target/arm/translate-sve.c | 38 ++++++++++++++++++++++++++++---------- 2 files changed, 47 insertions(+), 20 deletions(-) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index a504b55dad..74ac72bdbd 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -73,6 +73,7 @@ &rprr_s rd pg rn rm s &rprr_esz rd pg rn rm esz &rrrr_esz rd ra rn rm esz +&rrxr_esz rd rn rm ra index esz &rprrr_esz rd pg rn rm ra esz &rpri_esz rd pg rn imm esz &ptrue rd esz pat s @@ -252,6 +253,14 @@ @rrx_2 ........ .. . index:2 rm:3 ...... rn:5 rd:5 &rrx_esz @rrx_1 ........ .. . index:1 rm:4 ...... rn:5 rd:5 &rrx_esz +# Three registers and a scalar by N-bit index +@rrxr_3 ........ .. . .. rm:3 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx index=%index3_22_19 +@rrxr_2 ........ .. . index:2 rm:3 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx +@rrxr_1 ........ .. . index:1 rm:4 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -767,10 +776,10 @@ DOT_zzzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 \ ra=%reg_movprfx # SVE integer dot product (indexed) -DOT_zzxw 01000100 101 index:2 rm:3 00000 u:1 rn:5 rd:5 \ - sz=0 ra=%reg_movprfx -DOT_zzxw 01000100 111 index:1 rm:4 00000 u:1 rn:5 rd:5 \ - sz=1 ra=%reg_movprfx +SDOT_zzxw_s 01000100 10 1 ..... 000000 ..... ..... @rrxr_2 esz=2 +SDOT_zzxw_d 01000100 11 1 ..... 000000 ..... ..... @rrxr_1 esz=3 +UDOT_zzxw_s 01000100 10 1 ..... 000001 ..... ..... @rrxr_2 esz=2 +UDOT_zzxw_d 01000100 11 1 ..... 000001 ..... ..... @rrxr_1 esz=3 # SVE floating-point complex add (predicated) FCADD 01100100 esz:2 00000 rot:1 100 pg:3 rm:5 rd:5 \ @@ -789,12 +798,12 @@ FCMLA_zzxz 01100100 11 1 index:1 rm:4 0001 rot:2 rn:5 rd:5 \ ### SVE FP Multiply-Add Indexed Group # SVE floating-point multiply-add (indexed) -FMLA_zzxz 01100100 0.1 .. rm:3 00000 sub:1 rn:5 rd:5 \ - ra=%reg_movprfx index=%index3_22_19 esz=1 -FMLA_zzxz 01100100 101 index:2 rm:3 00000 sub:1 rn:5 rd:5 \ - ra=%reg_movprfx esz=2 -FMLA_zzxz 01100100 111 index:1 rm:4 00000 sub:1 rn:5 rd:5 \ - ra=%reg_movprfx esz=3 +FMLA_zzxz 01100100 0. 1 ..... 000000 ..... ..... @rrxr_3 esz=1 +FMLA_zzxz 01100100 10 1 ..... 000000 ..... ..... @rrxr_2 esz=2 +FMLA_zzxz 01100100 11 1 ..... 000000 ..... ..... @rrxr_1 esz=3 +FMLS_zzxz 01100100 0. 1 ..... 000001 ..... ..... @rrxr_3 esz=1 +FMLS_zzxz 01100100 10 1 ..... 000001 ..... ..... @rrxr_2 esz=2 +FMLS_zzxz 01100100 11 1 ..... 000001 ..... ..... @rrxr_1 esz=3 ### SVE FP Multiply Indexed Group diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ffae6884d2..2eb21b28e1 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3813,26 +3813,34 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) return true; } -static bool trans_DOT_zzxw(DisasContext *s, arg_DOT_zzxw *a) +static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn) { - static gen_helper_gvec_4 * const fns[2][2] = { - { gen_helper_gvec_sdot_idx_b, gen_helper_gvec_sdot_idx_h }, - { gen_helper_gvec_udot_idx_b, gen_helper_gvec_udot_idx_h } - }; - + if (fn == NULL) { + return false; + } if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, - a->ra, a->index); + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); } return true; } +#define DO_RRXR(NAME, FUNC) \ + static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ + { return do_zzxz_ool(s, a, FUNC); } + +DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b) +DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) +DO_RRXR(trans_UDOT_zzxw_s, gen_helper_gvec_udot_idx_b) +DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) + +#undef DO_RRXR /* *** SVE Floating Point Multiply-Add Indexed Group */ -static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) +static bool do_FMLA_zzxz(DisasContext *s, arg_rrxr_esz *a, bool sub) { static gen_helper_gvec_4_ptr * const fns[3] = { gen_helper_gvec_fmla_idx_h, @@ -3847,13 +3855,23 @@ static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) vec_full_reg_offset(s, a->rn), vec_full_reg_offset(s, a->rm), vec_full_reg_offset(s, a->ra), - status, vsz, vsz, (a->index << 1) | a->sub, + status, vsz, vsz, (a->index << 1) | sub, fns[a->esz - 1]); tcg_temp_free_ptr(status); } return true; } +static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) +{ + return do_FMLA_zzxz(s, a, false); +} + +static bool trans_FMLS_zzxz(DisasContext *s, arg_FMLA_zzxz *a) +{ + return do_FMLA_zzxz(s, a, true); +} + /* *** SVE Floating Point Multiply Indexed Group */ From patchwork Fri Apr 16 21:02:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422631 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp737086jaf; Fri, 16 Apr 2021 14:37:24 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwTFM/mEqANl5zQN+aF6uoV6yyjrfA5Nh27yV1jB4FJbNrMb983ejPoDer3G3JCF4KEJDOl X-Received: by 2002:a92:de4e:: with SMTP id e14mr2774160ilr.129.1618609044349; Fri, 16 Apr 2021 14:37:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609044; cv=none; d=google.com; s=arc-20160816; b=i53LJv22zwepDUj/JbTf9oGF7ajaZjuiSjkuZgg5tOHUv1UuwwNpyhYK9qybyTKDCz VlqialAO0yXlxTlLejMbH1gMJ1f5HHVsV3iJn24R/itspsZ+2+jYxO2CYNd08rGtnRS0 9NOQfX0hghzpQ80voKRYaHZ4S8FX/z4tOoCiNhVKdD6knWHaaj6CrEAK7r4psbqArkOZ 3fNguSyyWMgj/+P30qAmKMdjwZ12v4F/j35+8JbAQz9pm5D9Y3/hcL6ik9EusI6gGkqy ox4sbLYiHSn17gTs5A2Oa9j5JBDYUzTZbYyD+pD29xQ5lO2UY1YWfvA5k1iBloNj2/23 Bnmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=mJ6X3+A6T4zH0IAmYnnf1DVj8KtUnubpSuPtkLtMp7I=; b=IGrVUtKsKYl2AkWqRcaFQcZuF6jJgS8s9Ev/lzK9c+hSQIbsPq/J3ueBwU5yyKCrIX 8O+KNyaoWkgXeO60mtm3f4O9+sYTrwlI1+1bUZI+1V3S3QQJB3R4jxHIw0vzw2shGIOV y0TofX6LKibjKIZY2tu+RjGZhQk45Y9qA7hAi6UvlFReA7MWDeVysOaMtW8hw9k9f11E +IbeaKvmQeWXEM0fSUGBjpU3zhkyOYCr8E6cCQQKuMzPWjyluxNLfppMMXkj8oUYWCWo wxEekK+DD5yrJWSO8zVfHOmQRpr2kT/2kiLkrL+LOaQfqO9i9raNcMfxugyVvvf4oOf+ XzQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tVyo3NjO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h15si4652642ili.57.2021.04.16.14.37.24 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:37:24 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tVyo3NjO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57262 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXW9T-0004eA-RG for patch@linaro.org; Fri, 16 Apr 2021 17:37:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45084) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdZ-0005so-RA for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:25 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]:33327) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdF-0001hv-A4 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:25 -0400 Received: by mail-pf1-x42f.google.com with SMTP id h11so1491825pfn.0 for ; Fri, 16 Apr 2021 14:04:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mJ6X3+A6T4zH0IAmYnnf1DVj8KtUnubpSuPtkLtMp7I=; b=tVyo3NjOiKE8IS9oP/C9Cq66AmOGvskS6gtIic9iLnLxwb9gObsZ08fzDs4dh15h6m Lk2jL8UTK4n1cTlQ7dSoLdMLWaeIugKis91nqWSSDcBFwzXjMbZ9w/uePioP+JnNMuoz cRbczMs41Aq0RV4GXvph0R22Fn7EzrxMQC1nJWLJLv1zpXRHEiX94g+Ab3Nn6+SDC4G5 7fDs+AqQrErNpqS2xPS9hQSKIqRnPPXPOlms8W5+jrSjM5UJ0DPaI/TMonkEuAnjEP6c 9lABorKCkWFQCkdJGc7Tw4MrUHDfZcvqQv4KkODp5j4gyIXIlumrLHL7sPKg8rK3JSdg Hl0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mJ6X3+A6T4zH0IAmYnnf1DVj8KtUnubpSuPtkLtMp7I=; b=eq26Z0mawdJXgbLrv/9d0pYSVHsq3u2P+MMZLhEvt+J740vjPJi3P2XA4p7FgBParg 8xiorMknctFV7IBZ3s9Fd9K4pR+W9FPAPvKEPqHnhCXTZC2LcS641E2Hs2xG8LdK/2hB /10qCbTgfiYyL6VPOR4KNNrPdmqPObiZhAm/LkJTraOwbWkCoysAt8tsF8s3Qw3vkIMc EiAoIRJQNY1sbgzaMWt8mtWmUsFRU8pIXgL/xQnso3bQtcTPctjOrBT4RGuEasfytnfW KeCWVqTzLxqtJTArSsDSTYM3oEbD6oIIFaxW4T7BHaIp7xhJC7sm6xUalUKtECHSZ3LS x06A== X-Gm-Message-State: AOAM530k6Pj1jdPutHFGSPoByEt6HiQSdKUekjh1HtRzbgQNdhDo3hzN RTCFNn0dsdLyYwHMdCMXhus0eeXk3GFoeQ== X-Received: by 2002:a05:6a00:15c6:b029:24e:9dd1:8813 with SMTP id o6-20020a056a0015c6b029024e9dd18813mr9342198pfu.80.1618607043668; Fri, 16 Apr 2021 14:04:03 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:03 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 52/81] target/arm: Implement SVE2 integer multiply (indexed) Date: Fri, 16 Apr 2021 14:02:11 -0700 Message-Id: <20210416210240.1591291-53-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 7 +++++++ target/arm/translate-sve.c | 30 ++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 74ac72bdbd..65cb0a2206 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -775,12 +775,19 @@ MUL_zzi 00100101 .. 110 000 110 ........ ..... @rdn_i8s DOT_zzzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 \ ra=%reg_movprfx +#### SVE Multiply - Indexed + # SVE integer dot product (indexed) SDOT_zzxw_s 01000100 10 1 ..... 000000 ..... ..... @rrxr_2 esz=2 SDOT_zzxw_d 01000100 11 1 ..... 000000 ..... ..... @rrxr_1 esz=3 UDOT_zzxw_s 01000100 10 1 ..... 000001 ..... ..... @rrxr_2 esz=2 UDOT_zzxw_d 01000100 11 1 ..... 000001 ..... ..... @rrxr_1 esz=3 +# SVE2 integer multiply (indexed) +MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 +MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 +MUL_zzx_d 01000100 11 1 ..... 111110 ..... ..... @rrx_1 esz=3 + # SVE floating-point complex add (predicated) FCADD 01100100 esz:2 00000 rot:1 100 pg:3 rm:5 rd:5 \ rn=%reg_movprfx diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 2eb21b28e1..0de8445fb4 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3813,6 +3813,10 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) return true; } +/* + * SVE Multiply - Indexed + */ + static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, gen_helper_gvec_4 *fn) { @@ -3836,6 +3840,32 @@ DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) #undef DO_RRXR +static bool do_sve2_zzx_ool(DisasContext *s, arg_rrx_esz *a, + gen_helper_gvec_3 *fn) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, a->index, fn); + } + return true; +} + +#define DO_SVE2_RRX(NAME, FUNC) \ + static bool NAME(DisasContext *s, arg_rrx_esz *a) \ + { return do_sve2_zzx_ool(s, a, FUNC); } + +DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h) +DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s) +DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) + +#undef DO_SVE2_RRX + /* *** SVE Floating Point Multiply-Add Indexed Group */ From patchwork Fri Apr 16 21:02:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422640 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp740920jaf; Fri, 16 Apr 2021 14:44:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyY2xm2Ne2xx7kV69pZpOmSty6i88B1O8yJzxgd3LZeToq6B70z+VfsNx3dCvo8Q0kfl6gf X-Received: by 2002:a25:ea47:: with SMTP id o7mr1616168ybe.415.1618609499836; Fri, 16 Apr 2021 14:44:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609499; cv=none; d=google.com; s=arc-20160816; b=XSz/c3WqQFxP6dkthsmw20IeVOJBKoPeRuYcnQWVaJO6JrQz+edbmvmDAyYPBD8Xg8 JuMGUZXwPD/x+56aTBKL1OoS4EOinQ8TAhrdd1TEzE6PGIZaJsXt+goS6DyyPCg8n18r pU+j2OrccD9+XZc/TvRZ2bP+u2VnTp554eO1sHPBJEyHodcorbl0tnsa+mY/zryXi9io eO5hioZDM2nrQNEAZtMhzqpIv8s+3O8Onki72sBPt/5MIKXZSKwJYI/W/Flo/iHBxUeY f6eIuXEyAT8PSCZXhlOStqpheUJQandZOwnA5uMFtLQa0ToEYF8fS/AkJZpqzZvOs/0U wmIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=wwTGY29rJKdn3GggDiDCMT7mmPQzpwnC/ENztXM6VVY=; b=lN0uMaV7izT9uzT+HaFZRE8cyWjpLgciSGUZCd87mN76SRRHFThvpFsTOPeBLUeF5e 69SdBRcQqrPYtSIQL9EoZgkHKdCCisQQVMaEkXMRw4TeRZpzmqs9XyCcrj7yH5RKRfQr ERnxJBsAZ8qN9LFiBX3r3jIaDSCltK02z4e4GkzC4HwOHjyTFgRrwPnJUXtzOn6/B2Nf VWWPbMtgzYy9MTkKVf3trQ3SYEIA8qW1q/cmEacahThodWZEZP+KfmLVcl9wQQuAlIsk MLQZ7xxqaYIjlFfqUmzITjPRr7W5W8dEuqZ+Jh5UZUVAkvXDVxj78LuYSm4seVXHLsZF DiCg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TZ924Yxc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g11si6902951ybj.80.2021.04.16.14.44.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:44:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TZ924Yxc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55360 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWGp-0007Vn-86 for patch@linaro.org; Fri, 16 Apr 2021 17:44:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45188) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVde-00063x-25 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:30 -0400 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]:33734) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdK-0001i8-UQ for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:29 -0400 Received: by mail-pj1-x102e.google.com with SMTP id kb13-20020a17090ae7cdb02901503d67f0beso3640508pjb.0 for ; Fri, 16 Apr 2021 14:04:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wwTGY29rJKdn3GggDiDCMT7mmPQzpwnC/ENztXM6VVY=; b=TZ924Yxc6dBCBVE/IrscPqnkeZuEYP7Q0zKoelhyezGh88pGfG65rVsuphPakjKkuu 5QREu5N/1rXCynafPB0lLpxf2+06usO9DgzbWablmoOWd5PnhMba5cP6re9d9ne/L9/y +TBa/WSnF4FGEVy68zJDoASGcZqmb/wMPVQpkjpgqkHNaOSzeQUoTtuiYKEkdj/PJreE q/zTxhEeWCapmurz+p8csvnpUdfqH97G4WUAFQIX7gX84j6HxBr8yxa4xYlrLPt4fILb gfd6Ut6q2YEMAWWaVKR1oCJOy09fZHN8VIxWGb/GtD38O8r6fnzhjXaNO6QuVp/Y96fa myGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wwTGY29rJKdn3GggDiDCMT7mmPQzpwnC/ENztXM6VVY=; b=uOy/zaCJJ4Ie3W3hov39hVBZQRtrIp3j3MBRAs4CtRqTIF0L3MiRKLFix/ptFSfyrA oI5XZEXTKPC8p1XUYCgCCngFIV3JLMENcNIHzIx387n2tdc1Tri2PPIUdfIvnCvDic0u ubVtqrohphNyCJjahMl6170EBPWLvzo7+qPMzD21T8sns1YN1zgyqmCcB4wU89Ctz9AC Z8meM4YnqUxdEmH8Fp11GZz97nYttA7wP17fXF+Z4H2hfIG2MVU2CR7SZqGu9u+QlXdj G0pOz97ynIGU8o+gIu3ohk11wFWn2IRIWjuj1X4xHCWnr6rTPdua6w32WDU4bJsj5cPa 9IWw== X-Gm-Message-State: AOAM530thSMWIPSm/wRTgXQm4VYJHD6KqwsWVqN/zffhItISVYtaq6uA fBZEYjbBtwRZEUe7lHUl0nSW/RkI6hO8yw== X-Received: by 2002:a17:90a:b10a:: with SMTP id z10mr11554023pjq.161.1618607044519; Fri, 16 Apr 2021 14:04:04 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 53/81] target/arm: Implement SVE2 integer multiply-add (indexed) Date: Fri, 16 Apr 2021 14:02:12 -0700 Message-Id: <20210416210240.1591291-54-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102e; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 8 ++++++++ target/arm/translate-sve.c | 23 +++++++++++++++++++++++ 2 files changed, 31 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 65cb0a2206..9bfaf737b7 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -783,6 +783,14 @@ SDOT_zzxw_d 01000100 11 1 ..... 000000 ..... ..... @rrxr_1 esz=3 UDOT_zzxw_s 01000100 10 1 ..... 000001 ..... ..... @rrxr_2 esz=2 UDOT_zzxw_d 01000100 11 1 ..... 000001 ..... ..... @rrxr_1 esz=3 +# SVE2 integer multiply-add (indexed) +MLA_zzxz_h 01000100 0. 1 ..... 000010 ..... ..... @rrxr_3 esz=1 +MLA_zzxz_s 01000100 10 1 ..... 000010 ..... ..... @rrxr_2 esz=2 +MLA_zzxz_d 01000100 11 1 ..... 000010 ..... ..... @rrxr_1 esz=3 +MLS_zzxz_h 01000100 0. 1 ..... 000011 ..... ..... @rrxr_3 esz=1 +MLS_zzxz_s 01000100 10 1 ..... 000011 ..... ..... @rrxr_2 esz=2 +MLS_zzxz_d 01000100 11 1 ..... 000011 ..... ..... @rrxr_1 esz=3 + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0de8445fb4..25dadabe28 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3866,6 +3866,29 @@ DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) #undef DO_SVE2_RRX +static bool do_sve2_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zzxz_ool(s, a, fn); +} + +#define DO_SVE2_RRXR(NAME, FUNC) \ + static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ + { return do_sve2_zzxz_ool(s, a, FUNC); } + +DO_SVE2_RRXR(trans_MLA_zzxz_h, gen_helper_gvec_mla_idx_h) +DO_SVE2_RRXR(trans_MLA_zzxz_s, gen_helper_gvec_mla_idx_s) +DO_SVE2_RRXR(trans_MLA_zzxz_d, gen_helper_gvec_mla_idx_d) + +DO_SVE2_RRXR(trans_MLS_zzxz_h, gen_helper_gvec_mls_idx_h) +DO_SVE2_RRXR(trans_MLS_zzxz_s, gen_helper_gvec_mls_idx_s) +DO_SVE2_RRXR(trans_MLS_zzxz_d, gen_helper_gvec_mls_idx_d) + +#undef DO_SVE2_RRXR + /* *** SVE Floating Point Multiply-Add Indexed Group */ From patchwork Fri Apr 16 21:02:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422644 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp743424jaf; Fri, 16 Apr 2021 14:50:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyfrL9t+QOgy3e+34ZAFbgAEhFbFLPrLycX9e7mk1gl9JWhr2jMPT8vCd3EIzGsDYCVt7D2 X-Received: by 2002:a02:6654:: with SMTP id l20mr6011681jaf.55.1618609813235; Fri, 16 Apr 2021 14:50:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609813; cv=none; d=google.com; s=arc-20160816; b=KcFfd8lFXGSFu1KOW7RWRdtQY989ynLuWzHGw66dX22EZIL4Me4bfkqKZ7C9smEV3O hLgc2N4reQ3/6Bj1dhybaNZANpksnhgj6BBqIL8DW4mPw7h5MHaNYAUWr0GUERlYqk8m nhkkU8gNdAtDcKolF2AiGDQo3eYTzf4JPhVcsnecUE1K+pv2vV4rZ8OpCCqCGhel23xd 5FPlZC3JbNukOAwWgbpPBYDINi0gGgqFzOaKIlX9botianIlUeHusIK1cJYvaaEKi3M0 UGb4Uomkmgmzc3nSet99dJPYAlGHnCMIzOBqARAttmWDVR5jcr6rLx6L1bsbj6rRsDq+ ycrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=YSDZDgwTisSXoeVIgFgCvP0NdTqkSYat+07hmk+/QA8=; b=roL0ShuZbtN270/oxlYTH8k0/D4j0kJbr3/MMMtZgXO6xhS2/yuTpdzO0BWotJcmW6 DsKc1FRi8pyyAW4nge1FRfy27t07Q+9GURyNmcUZizjNBLkfdqGaqJLQgD6F5IqPEM4B Fhsi5LE9/JPutmXviyvm3gVyRI5ScbThzblX2ITmbeEtyX7+/2OBnAxWl8h7n1aG30gA yA26B+M1KnKTLNyPKZaApU8lyCWhkJHXzKbYC41AoYq9GX2iSS/9pUj9omEuTHD1drqW DnKdQfU7T8K7REN7Mr7i8OM4QL8CBf+hvEGyw2mZm9/evE0S45jSxWwoPIVoJGFYmF/I lWrQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jiQnotNJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w13si6889406ilq.89.2021.04.16.14.50.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:50:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jiQnotNJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36890 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWLs-000376-Ma for patch@linaro.org; Fri, 16 Apr 2021 17:50:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45212) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVde-00065C-IP for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:30 -0400 Received: from mail-pg1-x52b.google.com ([2607:f8b0:4864:20::52b]:41621) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdO-0001iH-Og for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:30 -0400 Received: by mail-pg1-x52b.google.com with SMTP id f29so20003081pgm.8 for ; Fri, 16 Apr 2021 14:04:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YSDZDgwTisSXoeVIgFgCvP0NdTqkSYat+07hmk+/QA8=; b=jiQnotNJe9Ot9+i+JZAtEZhnqvsjm7J8JHZm5VjcPBzuUzmjsS3fjghrHawXwv5gfy lgPIBMTqISjVdYHaCaNYRXxO+vqM2tEcigpPWILkBMoKI583r1PvM25D08K7zhBFI4W6 EKPEASrmK+HczxWWMn2zwS9HBxgD0eHtJIPMjC9Zm8GWDWPALQh+9bjQpHgAUKUF666b oLVOXOqxJslwVeJY2QDufBJrIOZIwZo8SuxqVbW8AIbVA0NJdDuVdMMXDLF25sPqYcBW o6HIa/hroBrDC0jtBE+qCJ4nmE0EP6/x7mkp7bxBICIES4eZB2P2FkKy6MNnVpkira4X qafw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YSDZDgwTisSXoeVIgFgCvP0NdTqkSYat+07hmk+/QA8=; b=j1bN3yRPHuMUTq9T+m5KTpB7SHYM1wgQ9Pe01cJofjCYq05tUgirLpiHokSMd1zNv6 9JqchqqKPszUvvPrzazxg4RSdJ/Hkywgr8QJItd/l2SGcQAXFOkjlJ4TWzDAaSSt1lmj STrFJsxXAjtTAS+D2sd+0iJMgJpwdgbLaiE23ceFyxkJl+LTYfIs3xsqRnAxo3sa9mWz oO9yIA0tKvd5POEXz5yU3Kij7SqUNAQQ3qBlwMPIQNliBYeFWV21D+2wGRjABGNlOX3n m8QoCuwC0VRK/q/pihEytNd5vHWJDBw60+XqdIMBbS6cz0A1NEQjkTR8K4/9A85wkkfB 7pDQ== X-Gm-Message-State: AOAM533GrhybALqQ9FPG5cOm1/rUPZYtnatGmsyAyxiiix88dQLQTeG7 sBaVoxvLVvMQ/SDx1RYG9tAatlE2GZ/oqA== X-Received: by 2002:a63:5b5c:: with SMTP id l28mr796647pgm.363.1618607045307; Fri, 16 Apr 2021 14:04:05 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 54/81] target/arm: Implement SVE2 saturating multiply-add high (indexed) Date: Fri, 16 Apr 2021 14:02:13 -0700 Message-Id: <20210416210240.1591291-55-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52b; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 14 +++++++++++++ target/arm/sve.decode | 8 ++++++++ target/arm/sve_helper.c | 40 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 8 ++++++++ 4 files changed, 70 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 7e99dcd119..fe67574741 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2665,3 +2665,17 @@ DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_6(fmmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(fmmla_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 9bfaf737b7..1956d96ad5 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -791,6 +791,14 @@ MLS_zzxz_h 01000100 0. 1 ..... 000011 ..... ..... @rrxr_3 esz=1 MLS_zzxz_s 01000100 10 1 ..... 000011 ..... ..... @rrxr_2 esz=2 MLS_zzxz_d 01000100 11 1 ..... 000011 ..... ..... @rrxr_1 esz=3 +# SVE2 saturating multiply-add high (indexed) +SQRDMLAH_zzxz_h 01000100 0. 1 ..... 000100 ..... ..... @rrxr_3 esz=1 +SQRDMLAH_zzxz_s 01000100 10 1 ..... 000100 ..... ..... @rrxr_2 esz=2 +SQRDMLAH_zzxz_d 01000100 11 1 ..... 000100 ..... ..... @rrxr_1 esz=3 +SQRDMLSH_zzxz_h 01000100 0. 1 ..... 000101 ..... ..... @rrxr_3 esz=1 +SQRDMLSH_zzxz_s 01000100 10 1 ..... 000101 ..... ..... @rrxr_2 esz=2 +SQRDMLSH_zzxz_d 01000100 11 1 ..... 000101 ..... ..... @rrxr_1 esz=3 + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index f285c90b70..fc4a943029 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1487,9 +1487,49 @@ DO_CMLA(sve2_sqrdcmlah_zzzz_h, int16_t, H2, DO_SQRDMLAH_H) DO_CMLA(sve2_sqrdcmlah_zzzz_s, int32_t, H4, DO_SQRDMLAH_S) DO_CMLA(sve2_sqrdcmlah_zzzz_d, int64_t, , DO_SQRDMLAH_D) +#undef DO_SQRDMLAH_B +#undef DO_SQRDMLAH_H +#undef DO_SQRDMLAH_S +#undef DO_SQRDMLAH_D #undef do_cmla #undef DO_CMLA +#define DO_ZZXZ(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, idx = simd_data(desc); \ + TYPE *d = vd, *a = va, *n = vn, *m = (TYPE *)vm + H(idx); \ + for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ + TYPE mm = m[i]; \ + for (j = 0; j < segment; j++) { \ + d[i + j] = OP(n[i + j], mm, a[i + j]); \ + } \ + } \ +} + +#define DO_SQRDMLAH_H(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_h(N, M, A, false, true, &discard); }) +#define DO_SQRDMLAH_S(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_s(N, M, A, false, true, &discard); }) +#define DO_SQRDMLAH_D(N, M, A) do_sqrdmlah_d(N, M, A, false, true) + +DO_ZZXZ(sve2_sqrdmlah_idx_h, int16_t, H2, DO_SQRDMLAH_H) +DO_ZZXZ(sve2_sqrdmlah_idx_s, int32_t, H4, DO_SQRDMLAH_S) +DO_ZZXZ(sve2_sqrdmlah_idx_d, int64_t, , DO_SQRDMLAH_D) + +#define DO_SQRDMLSH_H(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_h(N, M, A, true, true, &discard); }) +#define DO_SQRDMLSH_S(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_s(N, M, A, true, true, &discard); }) +#define DO_SQRDMLSH_D(N, M, A) do_sqrdmlah_d(N, M, A, true, true) + +DO_ZZXZ(sve2_sqrdmlsh_idx_h, int16_t, H2, DO_SQRDMLSH_H) +DO_ZZXZ(sve2_sqrdmlsh_idx_s, int32_t, H4, DO_SQRDMLSH_S) +DO_ZZXZ(sve2_sqrdmlsh_idx_d, int64_t, , DO_SQRDMLSH_D) + +#undef DO_ZZXZ + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 25dadabe28..42d8582707 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3887,6 +3887,14 @@ DO_SVE2_RRXR(trans_MLS_zzxz_h, gen_helper_gvec_mls_idx_h) DO_SVE2_RRXR(trans_MLS_zzxz_s, gen_helper_gvec_mls_idx_s) DO_SVE2_RRXR(trans_MLS_zzxz_d, gen_helper_gvec_mls_idx_d) +DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_h, gen_helper_sve2_sqrdmlah_idx_h) +DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_s, gen_helper_sve2_sqrdmlah_idx_s) +DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_d, gen_helper_sve2_sqrdmlah_idx_d) + +DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_h, gen_helper_sve2_sqrdmlsh_idx_h) +DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_s, gen_helper_sve2_sqrdmlsh_idx_s) +DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d) + #undef DO_SVE2_RRXR /* From patchwork Fri Apr 16 21:02:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422663 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp752052jaf; Fri, 16 Apr 2021 15:06:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy4i7sUTdkiQgzapXK8luLTTh98VIHvDQg2b9i++xJr9YChx5qRZBQJ+6EXUTz0ApcMLvKf X-Received: by 2002:a1f:1e01:: with SMTP id e1mr9299909vke.8.1618610784950; Fri, 16 Apr 2021 15:06:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610784; cv=none; d=google.com; s=arc-20160816; b=MkVBKVb6Vk/C4NVBuRUlNMovJTq6e77Mt/lNpgoiDEGQSAfmasb2/ZLJPVTBqc1WmM PC+qYHOsKnDhvxwRWqlqxqYVX+gZH5+g70sU29D/joS9MaXfVcUsQOV2+b0kVj0Vs9oq /0zl6/jWADBt+D06Nc8Zk0Lbrlnsu6/qhp4gRhtKhOtFSADIxCM1z5GcdE2ySGQsuMmb zk8zIRm1FWuiMTcLh49TcnAZVUx6lbsoqE+gaKfjWguXKnqJemXZIVqT0rPUV+ou3nVT zFXQYljzLIjDaDelStxUx2BOuMtCYBiZJgj4Z7h3ED27j8E7dfqdSCLTkGZaqboSjH9G Pt8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=jTQupde6YzsHjhMQLFeEGZDoiQ5cfoyX6mtthKyHZqc=; b=RKkXbkeTnlSlUiKzdyK+MswehqPg8vlukvgPRYEEq+yhgmucFZszxKZcFWhnZOXio+ Y56VMnaG94TP2OvvaYCvt1zSoFLjXm7gNZdtZAkSxsCPllGbVzmRMqNS3N3OSK3T1cTI HSdaq02aR2AE2okoORqOu1nFbH9T3aa1uqD3glwVNapaT/yJgNVCPudoW478hbZHpa+f 3rSqPDvvBvEW3n73zpyrRfD5AKcRRUPVTztCCmfALQtNIpEOQ1/J5/9vWHFj7eHmVrjp kl9M2xiguwLoTtqMEE2DgveOWlllOAsamRjDbXyCe3J3IioJGHFu/JA16N5nutha3h8Z JjZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ayW5VLiK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k20si136487ual.183.2021.04.16.15.06.24 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:06:24 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ayW5VLiK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34108 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWbY-0008Lt-AJ for patch@linaro.org; Fri, 16 Apr 2021 18:06:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45170) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdd-00061T-5a for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:29 -0400 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]:45801) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdL-0001iS-VK for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:28 -0400 Received: by mail-pj1-x102f.google.com with SMTP id nm3-20020a17090b19c3b029014e1bbf6c60so11072990pjb.4 for ; Fri, 16 Apr 2021 14:04:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jTQupde6YzsHjhMQLFeEGZDoiQ5cfoyX6mtthKyHZqc=; b=ayW5VLiKWrNIogZTem2WRZsLT9MIYvwUb+rb5jjYyrGJTEPjoLKNTBU0QQ8oXT9Dts P4L5XDB9S4rk7/SeIEDXStVHXUGYsBvedqwkBPb9MkbTp4aJoJe4mxJK8ihVIzY+ZtfJ HK+CLyWbUY5Cts59XPbRgzNfRpIk1If3qn69BcMw0RbVjbpI2/24QLK/PFWM6YTW+Jl0 Ir78BfdaD9tNp1/ecUj7HoOIiH6hYCf8VwPapRst4wJeoooMhK5dCXgyrK4O9B2GuLdp HZ3M14+h4B+jIrgRuSlfKVTGZyL5c9LNPwoPUM9BHrnjkzJ1QhDKY7DIx7OatEvYFwY6 F8GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jTQupde6YzsHjhMQLFeEGZDoiQ5cfoyX6mtthKyHZqc=; b=ZAe5Q6fx6npzD2yhDkL8qRxCbHZP4a5xTUAkl/PsQfdDTSIUN9wGAV3k5NvihdqWfJ KiQap+qeFjHuCO0ZEMKWFPaAigX1uc1YyGSjfkV9xkDch5Q3giwF1HW3F67CitXMHcHW CZpw15VJDwd14oPsq2hVcdcYM3b930S14duZwfQkB3KXTVe9i8vKaOs6UQYnKdmYW+5E WPuMVMjzHwAzVTDj8bZ/E0h9VvuQGvV/EPL3pvMxSc/g9OtMIQa7m4j2eG1ailBHCEGj YY3M5S5hUcpJeP5ap1bp9LqN+pmRmOphO0WKrSmOnDl2beEfONdcoIIhoeVzTNEd+HIb 6ZxA== X-Gm-Message-State: AOAM532NWe4u99UEgWZ+4Yy+pi9QNcqckkIAr4hctyZ44kfdh8qhowhS n+UWsrgumnNwY4+MLjkDZH9r/mx9qzQOLQ== X-Received: by 2002:a17:90a:fc92:: with SMTP id ci18mr11246698pjb.130.1618607046186; Fri, 16 Apr 2021 14:04:06 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:05 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 55/81] target/arm: Implement SVE2 saturating multiply-add (indexed) Date: Fri, 16 Apr 2021 14:02:14 -0700 Message-Id: <20210416210240.1591291-56-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 9 +++++++++ target/arm/sve.decode | 18 ++++++++++++++++++ target/arm/sve_helper.c | 30 ++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 32 ++++++++++++++++++++++++-------- 4 files changed, 81 insertions(+), 8 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index fe67574741..08398800bd 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2679,3 +2679,12 @@ DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqdmlal_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlal_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 1956d96ad5..8d2709d3cc 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -30,6 +30,8 @@ %size_23 23:2 %dtype_23_13 23:2 13:2 %index3_22_19 22:1 19:2 +%index3_19_11 19:2 11:1 +%index2_20_11 20:1 11:1 # A combination of tsz:imm3 -- extract esize. %tszimm_esz 22:2 5:5 !function=tszimm_esz @@ -261,6 +263,12 @@ @rrxr_1 ........ .. . index:1 rm:4 ...... rn:5 rd:5 \ &rrxr_esz ra=%reg_movprfx +# Three registers and a scalar by N-bit index, alternate +@rrxr_3a ........ .. ... rm:3 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx index=%index3_19_11 +@rrxr_2a ........ .. .. rm:4 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx index=%index2_20_11 + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -799,6 +807,16 @@ SQRDMLSH_zzxz_h 01000100 0. 1 ..... 000101 ..... ..... @rrxr_3 esz=1 SQRDMLSH_zzxz_s 01000100 10 1 ..... 000101 ..... ..... @rrxr_2 esz=2 SQRDMLSH_zzxz_d 01000100 11 1 ..... 000101 ..... ..... @rrxr_1 esz=3 +# SVE2 saturating multiply-add (indexed) +SQDMLALB_zzxw_s 01000100 10 1 ..... 0010.0 ..... ..... @rrxr_3a esz=2 +SQDMLALB_zzxw_d 01000100 11 1 ..... 0010.0 ..... ..... @rrxr_2a esz=3 +SQDMLALT_zzxw_s 01000100 10 1 ..... 0010.1 ..... ..... @rrxr_3a esz=2 +SQDMLALT_zzxw_d 01000100 11 1 ..... 0010.1 ..... ..... @rrxr_2a esz=3 +SQDMLSLB_zzxw_s 01000100 10 1 ..... 0011.0 ..... ..... @rrxr_3a esz=2 +SQDMLSLB_zzxw_d 01000100 11 1 ..... 0011.0 ..... ..... @rrxr_2a esz=3 +SQDMLSLT_zzxw_s 01000100 10 1 ..... 0011.1 ..... ..... @rrxr_3a esz=2 +SQDMLSLT_zzxw_d 01000100 11 1 ..... 0011.1 ..... ..... @rrxr_2a esz=3 + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index fc4a943029..c43c38044b 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1530,6 +1530,36 @@ DO_ZZXZ(sve2_sqrdmlsh_idx_d, int64_t, , DO_SQRDMLSH_D) #undef DO_ZZXZ +#define DO_ZZXW(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + intptr_t idx = extract32(desc, SIMD_DATA_SHIFT + 1, 3) * sizeof(TYPEN); \ + for (i = 0; i < oprsz; i += 16) { \ + TYPEW mm = *(TYPEN *)(vm + i + idx); \ + for (j = 0; j < 16; j += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + j + sel)); \ + TYPEW aa = *(TYPEW *)(va + HW(i + j)); \ + *(TYPEW *)(vd + HW(i + j)) = OP(nn, mm, aa); \ + } \ + } \ +} + +#define DO_SQDMLAL_S(N, M, A) DO_SQADD_S(A, do_sqdmull_s(N, M)) +#define DO_SQDMLAL_D(N, M, A) do_sqadd_d(A, do_sqdmull_d(N, M)) + +DO_ZZXW(sve2_sqdmlal_idx_s, int32_t, int16_t, H1_4, H1_2, DO_SQDMLAL_S) +DO_ZZXW(sve2_sqdmlal_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLAL_D) + +#define DO_SQDMLSL_S(N, M, A) DO_SQSUB_S(A, do_sqdmull_s(N, M)) +#define DO_SQDMLSL_D(N, M, A) do_sqsub_d(A, do_sqdmull_d(N, M)) + +DO_ZZXW(sve2_sqdmlsl_idx_s, int32_t, int16_t, H1_4, H1_2, DO_SQDMLSL_S) +DO_ZZXW(sve2_sqdmlsl_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLSL_D) + +#undef DO_ZZXW + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 42d8582707..4c304c0124 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3817,21 +3817,21 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) * SVE Multiply - Indexed */ -static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, - gen_helper_gvec_4 *fn) +static bool do_zzxz_data(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn, int data) { if (fn == NULL) { return false; } if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); } return true; } #define DO_RRXR(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_zzxz_ool(s, a, FUNC); } + { return do_zzxz_data(s, a, FUNC, a->index); } DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b) DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) @@ -3866,18 +3866,18 @@ DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) #undef DO_SVE2_RRX -static bool do_sve2_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, - gen_helper_gvec_4 *fn) +static bool do_sve2_zzxz_data(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn, int data) { if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zzxz_ool(s, a, fn); + return do_zzxz_data(s, a, fn, data); } #define DO_SVE2_RRXR(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_sve2_zzxz_ool(s, a, FUNC); } + { return do_sve2_zzxz_data(s, a, FUNC, a->index); } DO_SVE2_RRXR(trans_MLA_zzxz_h, gen_helper_gvec_mla_idx_h) DO_SVE2_RRXR(trans_MLA_zzxz_s, gen_helper_gvec_mla_idx_s) @@ -3897,6 +3897,22 @@ DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d) #undef DO_SVE2_RRXR +#define DO_SVE2_RRXR_TB(NAME, FUNC, TOP) \ + static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ + { return do_sve2_zzxz_data(s, a, FUNC, (a->index << 1) | TOP); } + +DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, false) +DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, false) +DO_SVE2_RRXR_TB(trans_SQDMLALT_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, true) +DO_SVE2_RRXR_TB(trans_SQDMLALT_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, true) + +DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, false) +DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, false) +DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, true) +DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, true) + +#undef DO_SVE2_RRXR_TB + /* *** SVE Floating Point Multiply-Add Indexed Group */ From patchwork Fri Apr 16 21:02:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422653 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp745639jaf; Fri, 16 Apr 2021 14:55:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwDg1+R6bM1UY7OvCPWXI0aRda/G2LSNmqm6IsIur2MNoCEMsk86z4NLK0k17PCdge58/3Y X-Received: by 2002:a5b:585:: with SMTP id l5mr1622264ybp.213.1618610120358; Fri, 16 Apr 2021 14:55:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610120; cv=none; d=google.com; s=arc-20160816; b=asYP6Vg6mshUzNcKAPSgejyEnnxWEqOMr2zNf+9yKjmQFyO5F8fx7oJtvr5n8Yh8kx qmlZqf98zwae2viYOKoakijc32mxPN+UjetnzAd9KvPrNXHWpH9E2KqX3YIo6cSpEQ8Q GIMRPUlNxdSAWUxnaqx+Jlz/E3H8AfeA/v/0DRIA1l4qD1c9u+NNUqmWjWha5SLDojvL Ty/DKl+29oJybl1wvDszyH8sN32CDX9Yhufk5hm40anaFmZUaWzafwSxNEwxDrNQdKZj 5snCKETKDTlk+OW/KaFB6m3bcnacsjVCgrr7X2jrQSnmUj8GJ4cfmLOPYZ7bxtirKMee IbKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=63xBkYPF4qfo/eA4hVG2Q333ePWRFmMPgR5uNHsN55s=; b=i+xtpzd74KwKiu/0oytv5xEN14TGq7Yf9+3NqxD8ROniuwsP56Ow3wNVOKDAzvaiZS TDDYGspsnXlkHzPe9jWK9WELejZso+amtLNJUHo6YkVNfEl4FI9to9MWZ71t2eKZ6hZO UM5Nabg8xD0Tv7ycV2G4vD65k+NLCQu37dEHJvGogipmn/4fm4vexwnX4Iqj6GP9wSua mt6FXlYHMhCdMOYdpq/VqDiQTo48LXjBIeB+w6teLw39seM96JZBW2i2IDUjeLn8v41e XtZJlHovosDorErV03QmS3rIGmK3/DXrzwSLjhyXPkxzAwjB9KAHJpWP+/oqZ3oi8SDZ vx4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xv3Ng13i; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j9si7033293ybp.439.2021.04.16.14.55.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:55:20 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xv3Ng13i; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55220 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWQp-000286-Qw for patch@linaro.org; Fri, 16 Apr 2021 17:55:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45302) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdh-0006E1-QB for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:33 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]:38564) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdO-0001ia-On for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:33 -0400 Received: by mail-pf1-x42c.google.com with SMTP id g16so5844577pfq.5 for ; Fri, 16 Apr 2021 14:04:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=63xBkYPF4qfo/eA4hVG2Q333ePWRFmMPgR5uNHsN55s=; b=xv3Ng13iYvVKaq/7gglXrjiTiPlexFuxhF1wzBfkHzRbBZrwHumkDNqIh88rvXLWys YL0gAbwMViFupzhZ47Vct+xYDOy1rcWJOlmEH7D/JSUiSCxya32ll/X1D0b/LzshUMeA G4gxhtGP8VUCJcl/fwGqyDKAjfwEjMaVsAiTJBWTjVIyo5qMvCC0fyrXKLVn/FPgbUrV seWxKMnM/c2j544OndL7lLMBYJgmN2FGKXDzgdvrnvhSYRgiJSYRz8Ce87IPSgwf8oxP KKebr3zwsMkK5aCYMJGCBn5omEYV+fYl54yl/QvxhMBdb+dhDdJ09N0GkcwpOopIjziS mKZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=63xBkYPF4qfo/eA4hVG2Q333ePWRFmMPgR5uNHsN55s=; b=CvhmIRRBBdeOvP5Xq1iSSAl8Ic9YVb/LvnunIpl2FfZrEa6tEzeUziu53xUVyF7lcI DavHDh+0Nc5d7YiN5zfl6SUZ5SxBc9Fz8kpARYvPmWlggTXBdfh6k7BC4RA0e6xr7pIL cxyupnk5PjOP7VM+iqPqn8pLmj9JUy606MV4sde2LvBYgQ++b6sgDzM6cWf96zcF1Ph0 5VkyoIqpuRPBElNKleEZ/LKN8u6RTEYDSf55DKZ5sZJKw2uu9jnGdku1E9SHRcld/KxJ NSl2wiaT083K8yCktVvSfVxBCxvGK7gd009R0D8/8+UvaiO/iKbimUzkjcKfzu6TBzpN 7I/Q== X-Gm-Message-State: AOAM533ueaiE0x2KlRFRm8yEY1kuFWzv5Hw/hOl1V5ui1eAnlIwv+3r8 mwPKPuHruetiOBrEV+2EOuZph87DqD/Oqg== X-Received: by 2002:aa7:95ae:0:b029:247:b279:9f48 with SMTP id a14-20020aa795ae0000b0290247b2799f48mr9672164pfk.29.1618607047160; Fri, 16 Apr 2021 14:04:07 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:06 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 56/81] target/arm: Implement SVE2 saturating multiply (indexed) Date: Fri, 16 Apr 2021 14:02:15 -0700 Message-Id: <20210416210240.1591291-57-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 12 ++++++++++++ target/arm/sve_helper.c | 20 ++++++++++++++++++++ target/arm/translate-sve.c | 19 +++++++++++++++---- 4 files changed, 52 insertions(+), 4 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 08398800bd..0be0d90bee 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2688,3 +2688,8 @@ DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 8d2709d3cc..a3b9fb95f9 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -255,6 +255,12 @@ @rrx_2 ........ .. . index:2 rm:3 ...... rn:5 rd:5 &rrx_esz @rrx_1 ........ .. . index:1 rm:4 ...... rn:5 rd:5 &rrx_esz +# Two registers and a scalar by N-bit index, alternate +@rrx_3a ........ .. . .. rm:3 ...... rn:5 rd:5 \ + &rrx_esz index=%index3_19_11 +@rrx_2a ........ .. . . rm:4 ...... rn:5 rd:5 \ + &rrx_esz index=%index2_20_11 + # Three registers and a scalar by N-bit index @rrxr_3 ........ .. . .. rm:3 ...... rn:5 rd:5 \ &rrxr_esz ra=%reg_movprfx index=%index3_22_19 @@ -817,6 +823,12 @@ SQDMLSLB_zzxw_d 01000100 11 1 ..... 0011.0 ..... ..... @rrxr_2a esz=3 SQDMLSLT_zzxw_s 01000100 10 1 ..... 0011.1 ..... ..... @rrxr_3a esz=2 SQDMLSLT_zzxw_d 01000100 11 1 ..... 0011.1 ..... ..... @rrxr_2a esz=3 +# SVE2 saturating multiply (indexed) +SQDMULLB_zzx_s 01000100 10 1 ..... 1110.0 ..... ..... @rrx_3a esz=2 +SQDMULLB_zzx_d 01000100 11 1 ..... 1110.0 ..... ..... @rrx_2a esz=3 +SQDMULLT_zzx_s 01000100 10 1 ..... 1110.1 ..... ..... @rrx_3a esz=2 +SQDMULLT_zzx_d 01000100 11 1 ..... 1110.1 ..... ..... @rrx_2a esz=3 + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index c43c38044b..e8a8425522 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1560,6 +1560,26 @@ DO_ZZXW(sve2_sqdmlsl_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLSL_D) #undef DO_ZZXW +#define DO_ZZX(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + intptr_t idx = extract32(desc, SIMD_DATA_SHIFT + 1, 3) * sizeof(TYPEN); \ + for (i = 0; i < oprsz; i += 16) { \ + TYPEW mm = *(TYPEN *)(vm + i + idx); \ + for (j = 0; j < 16; j += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + j + sel)); \ + *(TYPEW *)(vd + HW(i + j)) = OP(nn, mm); \ + } \ + } \ +} + +DO_ZZX(sve2_sqdmull_idx_s, int32_t, int16_t, H1_4, H1_2, do_sqdmull_s) +DO_ZZX(sve2_sqdmull_idx_d, int64_t, int32_t, , H1_4, do_sqdmull_d) + +#undef DO_ZZX + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 4c304c0124..d3fcf2e4c1 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3840,8 +3840,8 @@ DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) #undef DO_RRXR -static bool do_sve2_zzx_ool(DisasContext *s, arg_rrx_esz *a, - gen_helper_gvec_3 *fn) +static bool do_sve2_zzx_data(DisasContext *s, arg_rrx_esz *a, + gen_helper_gvec_3 *fn, int data) { if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { return false; @@ -3851,14 +3851,14 @@ static bool do_sve2_zzx_ool(DisasContext *s, arg_rrx_esz *a, tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), vec_full_reg_offset(s, a->rn), vec_full_reg_offset(s, a->rm), - vsz, vsz, a->index, fn); + vsz, vsz, data, fn); } return true; } #define DO_SVE2_RRX(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrx_esz *a) \ - { return do_sve2_zzx_ool(s, a, FUNC); } + { return do_sve2_zzx_data(s, a, FUNC, a->index); } DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h) DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s) @@ -3866,6 +3866,17 @@ DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) #undef DO_SVE2_RRX +#define DO_SVE2_RRX_TB(NAME, FUNC, TOP) \ + static bool NAME(DisasContext *s, arg_rrx_esz *a) \ + { return do_sve2_zzx_data(s, a, FUNC, (a->index << 1) | TOP); } + +DO_SVE2_RRX_TB(trans_SQDMULLB_zzx_s, gen_helper_sve2_sqdmull_idx_s, false) +DO_SVE2_RRX_TB(trans_SQDMULLB_zzx_d, gen_helper_sve2_sqdmull_idx_d, false) +DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_s, gen_helper_sve2_sqdmull_idx_s, true) +DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_d, gen_helper_sve2_sqdmull_idx_d, true) + +#undef DO_SVE2_RRX_TB + static bool do_sve2_zzxz_data(DisasContext *s, arg_rrxr_esz *a, gen_helper_gvec_4 *fn, int data) { From patchwork Fri Apr 16 21:02:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422639 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp740123jaf; Fri, 16 Apr 2021 14:43:28 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxKml9e1/WxrT24FMJUXInf+0AJ7TQNsiZn3+kHkjn4ahsBv4TIuxUeS5x7aMuuk5QrR392 X-Received: by 2002:a25:f40f:: with SMTP id q15mr1662824ybd.73.1618609408394; Fri, 16 Apr 2021 14:43:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609408; cv=none; d=google.com; s=arc-20160816; b=OEV+jqibAvl47qlg8lWmjz03JX9+fkLGZj9AKLOcp/yfeFD+1VLzu70OiAqsYqMUZU EpFQ6RWV4Ky2+4jpynD6vo4ZexaT9WghkbTMRFMldhzO/Wg8mb/zMtnn/d9GObm/mFpN lKPHkq9CmmrxWNdM3gynulw563PfLJ0B55Cv8TgdTNE+vE1JYUsX4rcvC5Y+XfrKVlR7 8o+fgRPDlK7cbxlbBFr7zkelXjOCrAltg9P0qp5HVnVmU/ETGmrdK/vSDbIZK98Z5GBL hXtqE8MKwXxKNM+ex52r1NDBbXYrnSUlLDAqwRvhT1jMmFXWtJgayRtX5Pf0xAILyq62 U/sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=gXohVRGuyQOJc/h/jdj7wHe2XbYitvQhklfiQ4g4gH8=; b=SX1pbgjkrg9urxhQIj0x8xVmDQAcSRnblVjQCtxKyUIVT+OIfU+yJKi44mcK1oPQHG dpIpHX00RKyrIeKW0tuBlk+5TDTVhpKD5iUoc7XL5fm+W0gPw0iALAjCo5TNQBDC2GSm TIFcW275JZehrBS04wj/Vqgan5QzxCLdt0ETn4Kg0u5nIgtAIE5nIGWlhF+dMGB98Q6g vLk85JvctLvQPLZhptXrdmmQ7XuA2q+j8BG0fQOsybRJZtmzmeWmZzoWvdfn2tFnHtDx rtNiSAd7zIasvrFqc8PtnIWMJkEkDvS5ZOqW1rCilDREQ68leOuqQaD8KQObueHrhy2L YPig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mF8Psq2v; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i9si7108450ybo.197.2021.04.16.14.43.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:43:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mF8Psq2v; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50406 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWFL-0005W1-Q4 for patch@linaro.org; Fri, 16 Apr 2021 17:43:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45392) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdl-0006Nl-8M for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:37 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]:44974) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001im-IA for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:36 -0400 Received: by mail-pf1-x42f.google.com with SMTP id m11so19141701pfc.11 for ; Fri, 16 Apr 2021 14:04:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gXohVRGuyQOJc/h/jdj7wHe2XbYitvQhklfiQ4g4gH8=; b=mF8Psq2vYNRu2HerlvWqmATI/6I1O9SCfsrGnptgRSlZunri6FL95ufh11woI+jK7M g544rkKAn0Dx/AicCIaWjwdnfUU56oJyqKweVImSUIAsTv5KMypAoFC4q5ds5mJwnNuH L+LOjVEQuvPOzgIx0f+tZrcj6KX3lMj+UGeSKwDoVS413UJK7avdNKXTS6m4YBclAQQN A3TVjATMvdSO8DiYdE62t7utk74dZuYoPNs2ukFOAJgkk+Ya0PNVJI5pzUqnwEtuA/KH 3yyBsN+AwdHUZcggq+OVjAHoxGFmgn+A4uER0kvZzWV6hGujzZn3bf5YyYSDpiPZupXi WwfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gXohVRGuyQOJc/h/jdj7wHe2XbYitvQhklfiQ4g4gH8=; b=AOOWn3VtOyztDbZ1tCYVljwewreCa8Ob5/vrtsVeJ4NnWAYl+NO533VaYtZQrsH2rX 5OdpQYdL30Pa6o+PHESiOLgJh05pR7i20LD7xW8/CTylnhNetoaHP9XKLXa1M13yyojZ Hlvcj4nct0e50YAC2y+fEcU8+0nJrvYTa5dUE8aDG2DpxlVdz4jzTT2HvbqnfCzMYaqm 35vvOq0okOx+d2m3uJKnQynr+p2hw91iEa3/tagPEO7mzZk9BxkXcVwYxfP44V1GwxHV Ch2NlblcB5opboec4GUgDGeaBhbRs+mHiNfbfOz/SlOkmhwSEywa5Nz6ywyY15l2OcYG zFeQ== X-Gm-Message-State: AOAM530biQtMNCGI7BfRDl66myMrqNawPYUnefo93xJgPwZzFsPDKOqi MG0IgsAYiOSoNNUnWS2E1zgjv54J/Fl3oA== X-Received: by 2002:a63:d714:: with SMTP id d20mr807021pgg.285.1618607048212; Fri, 16 Apr 2021 14:04:08 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:07 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 57/81] target/arm: Implement SVE2 signed saturating doubling multiply high Date: Fri, 16 Apr 2021 14:02:16 -0700 Message-Id: <20210416210240.1591291-58-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.h | 10 +++++ target/arm/sve.decode | 4 ++ target/arm/translate-sve.c | 18 ++++++++ target/arm/vec_helper.c | 84 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 116 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 72c5bf6aca..eb94b6b1e6 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -957,6 +957,16 @@ DEF_HELPER_FLAGS_5(neon_sqrdmulh_h, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(neon_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) #ifdef TARGET_AARCH64 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index a3b9fb95f9..407d3019d1 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1202,6 +1202,10 @@ SMULH_zzz 00000100 .. 1 ..... 0110 10 ..... ..... @rd_rn_rm UMULH_zzz 00000100 .. 1 ..... 0110 11 ..... ..... @rd_rn_rm PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 +# SVE2 signed saturating doubling multiply high (unpredicated) +SQDMULH_zzz 00000100 .. 1 ..... 0111 00 ..... ..... @rd_rn_rm +SQRDMULH_zzz 00000100 .. 1 ..... 0111 01 ..... ..... @rd_rn_rm + ### SVE2 Integer - Predicated SADALP_zpzz 01000100 .. 000 100 101 ... ..... ..... @rdm_pg_rn diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index d3fcf2e4c1..dd4de9e57f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6419,6 +6419,24 @@ static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a) return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b); } +static bool trans_SQDMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqdmulh_b, gen_helper_sve2_sqdmulh_h, + gen_helper_sve2_sqdmulh_s, gen_helper_sve2_sqdmulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + +static bool trans_SQRDMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqrdmulh_b, gen_helper_sve2_sqrdmulh_h, + gen_helper_sve2_sqrdmulh_s, gen_helper_sve2_sqrdmulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + /* * SVE2 Integer - Predicated */ diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index b19877e0d3..25061c15e1 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -81,6 +81,26 @@ void HELPER(sve2_sqrdmlsh_b)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], 0, false, false); + } +} + +void HELPER(sve2_sqrdmulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], 0, false, true); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */ int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, bool neg, bool round, uint32_t *sat) @@ -198,6 +218,28 @@ void HELPER(sve2_sqrdmlsh_h)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], 0, false, false, &discard); + } +} + +void HELPER(sve2_sqrdmulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], 0, false, true, &discard); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, bool neg, bool round, uint32_t *sat) @@ -309,6 +351,28 @@ void HELPER(sve2_sqrdmlsh_s)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], 0, false, false, &discard); + } +} + +void HELPER(sve2_sqrdmulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], 0, false, true, &discard); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 64-bit */ static int64_t do_sat128_d(Int128 r) { @@ -368,6 +432,26 @@ void HELPER(sve2_sqrdmlsh_d)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], 0, false, false); + } +} + +void HELPER(sve2_sqrdmulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], 0, false, true); + } +} + /* Integer 8 and 16-bit dot-product. * * Note that for the loops herein, host endianness does not matter From patchwork Fri Apr 16 21:02:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422637 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp739463jaf; Fri, 16 Apr 2021 14:42:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyIMnrEJIsIFRjgw9aXPqJv+gvRdeZK1E1UY0l3LT08JSA9dQ3lVyWQ+QMB9qyiRCnQVSO2 X-Received: by 2002:a25:ce09:: with SMTP id x9mr1589337ybe.81.1618609328182; Fri, 16 Apr 2021 14:42:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609328; cv=none; d=google.com; s=arc-20160816; b=ONwDim2NketJMraqKjMAY7VV7WJWD0n3+2GS8zyANC9rae7FsSY2w76+vFqYklZJJG U0xEwxRunOqankdJpAGo4nGtCXUnlYAER2VSjjSgaXcu2RILUjkBQ8YEwidIB+5SVe0a WKUQO43NLr9RHTIwUCK2m01sphEkpx9HeVdt6pFlt3kkeUDdVpVhe6j4kegvNYMSJTjV h2aUvzoSCjVA7D4aIcvw3OWbXP2SFR6cv1FWMHTut2d8OK25zUFVgG7nhUJqHvVJADpb OpeFiMgZ4LqDYFjl548voUAeRL8Fz1tF+8lSqiORUs7+MeLpkDRoVTc2ZN2/L3MT9bzJ DAbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=AmPoZIIhtmHK3pFU80wMlIIFBV5aGkRIo467G1t+i0Y=; b=dDrbijAkbSyd4NbYAzLHfiJ8/0W1VTxcFStYSGaRA7Ffa29QKLxbKeTEKTJTZ6m7M+ 0VUhmqWm7ho2Y9DnT8r3nGtK5YCuMyNNa/OspzRsNBaPv3NotL8STAJIMk4WVCNDKQ/n ZFHIP16obyZvFY221No7xsDI7DGsdA83SBHeBBUW4J2K3cnF03GyTeNa78Ya4GTvQxFt wp+Y2r5r7JmgzzjZjmw2GtzRV0xfzStRDWvhk2CpyiIwX0IRbfG/3f347kVA5B6UNfpR +e49JHnDrQbACUrs2nhM3hqdFGDR1dxTSzb06s28OukMw5e/rpe+F+0u9KDv7h7JZ6rS KmOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DkJkO2Bk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f17si6394631ybr.50.2021.04.16.14.42.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:42:08 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DkJkO2Bk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46776 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWE3-00042u-JT for patch@linaro.org; Fri, 16 Apr 2021 17:42:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45156) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdc-00060M-PT for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:28 -0400 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]:46877) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdK-0001j2-US for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:28 -0400 Received: by mail-pg1-x531.google.com with SMTP id 31so4544531pgn.13 for ; Fri, 16 Apr 2021 14:04:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AmPoZIIhtmHK3pFU80wMlIIFBV5aGkRIo467G1t+i0Y=; b=DkJkO2BktbcYypDGMm9yDDSYLUGgLUzLwqkOaGzg4ryWsSDrODChZU24fT475iav9I i+2SW+MhgoMIFki5mXeZI5B5clnA2kyBFhj7AjZfbF9/BYrft4IuEduVY0bZXY7udVMs XEnArFF0hMQql3PTN78lmjjRhKss8L38m3nsvC9+eRreijlGhvK0ymrooivBaB2H368C DzhUacPQv4gX2x9Mp8/IKrdQCnSbFQRA0W71SD0JZE60KenPQFgNXYOFlvc7Q82ABg5K W63souu57o/Zv5d9Y995gPlO0vHl+9+ktQxceayPAiTlTkv9rJm3CQn+P7q1aRbCOFNw uvGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AmPoZIIhtmHK3pFU80wMlIIFBV5aGkRIo467G1t+i0Y=; b=FPiwE5R3ow8nOvvBuSA2/ZPtdurqEuorvyivYkeObFU+FTnW9/6Vhel4nIIlxgVCVD Wbx7rcm/Jxf3RH0XYDc4cJ/K6+aUBva4fELSKMWIw1EQ1OYKTajK48AvEmNarRDPm9m0 l/6gD5FFP8tVzGQXXwasvCsF3Hu2TF0pXO6kaIMu2PotgJPKSzf/2EjFONQ5WhdUF6Gz mZ4kyQXf/sq7X8COjCBKWOEzIzyqF9OEhAI8PH2Q8lCKgNKDy5SyvXF1jJ/EKKawRxRG 8Ex3O1kD3mQGl2y3uqhlfsiI41AghTu+0j1FoLrFqpvAdd2sUWcFBd/jtj3D9p3dsV80 FpGw== X-Gm-Message-State: AOAM531YdeoF4dR0BWxwdSTMDnrKxuqw+pxiq64wbnpiSKsyAKghSy0f K5HgtIQJe2TrL96dXeUFMpQ9oO60BRBgGA== X-Received: by 2002:a63:5361:: with SMTP id t33mr817409pgl.439.1618607049221; Fri, 16 Apr 2021 14:04:09 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:08 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 58/81] target/arm: Implement SVE2 saturating multiply high (indexed) Date: Fri, 16 Apr 2021 14:02:17 -0700 Message-Id: <20210416210240.1591291-59-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::531; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x531.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.h | 14 ++++++ target/arm/sve.decode | 8 ++++ target/arm/translate-sve.c | 8 ++++ target/arm/vec_helper.c | 88 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 118 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index eb94b6b1e6..e7c463fff5 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -967,6 +967,20 @@ DEF_HELPER_FLAGS_4(sve2_sqrdmulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqrdmulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) #ifdef TARGET_AARCH64 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 407d3019d1..35010d755f 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -829,6 +829,14 @@ SQDMULLB_zzx_d 01000100 11 1 ..... 1110.0 ..... ..... @rrx_2a esz=3 SQDMULLT_zzx_s 01000100 10 1 ..... 1110.1 ..... ..... @rrx_3a esz=2 SQDMULLT_zzx_d 01000100 11 1 ..... 1110.1 ..... ..... @rrx_2a esz=3 +# SVE2 saturating multiply high (indexed) +SQDMULH_zzx_h 01000100 0. 1 ..... 111100 ..... ..... @rrx_3 esz=1 +SQDMULH_zzx_s 01000100 10 1 ..... 111100 ..... ..... @rrx_2 esz=2 +SQDMULH_zzx_d 01000100 11 1 ..... 111100 ..... ..... @rrx_1 esz=3 +SQRDMULH_zzx_h 01000100 0. 1 ..... 111101 ..... ..... @rrx_3 esz=1 +SQRDMULH_zzx_s 01000100 10 1 ..... 111101 ..... ..... @rrx_2 esz=2 +SQRDMULH_zzx_d 01000100 11 1 ..... 111101 ..... ..... @rrx_1 esz=3 + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index dd4de9e57f..b43bf939f5 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3864,6 +3864,14 @@ DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h) DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s) DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) +DO_SVE2_RRX(trans_SQDMULH_zzx_h, gen_helper_sve2_sqdmulh_idx_h) +DO_SVE2_RRX(trans_SQDMULH_zzx_s, gen_helper_sve2_sqdmulh_idx_s) +DO_SVE2_RRX(trans_SQDMULH_zzx_d, gen_helper_sve2_sqdmulh_idx_d) + +DO_SVE2_RRX(trans_SQRDMULH_zzx_h, gen_helper_sve2_sqrdmulh_idx_h) +DO_SVE2_RRX(trans_SQRDMULH_zzx_s, gen_helper_sve2_sqrdmulh_idx_s) +DO_SVE2_RRX(trans_SQRDMULH_zzx_d, gen_helper_sve2_sqrdmulh_idx_d) + #undef DO_SVE2_RRX #define DO_SVE2_RRX_TB(NAME, FUNC, TOP) \ diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 25061c15e1..8b7269d8e1 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -240,6 +240,36 @@ void HELPER(sve2_sqrdmulh_h)(void *vd, void *vn, void *vm, uint32_t desc) } } +void HELPER(sve2_sqdmulh_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int16_t *d = vd, *n = vn, *m = (int16_t *)vm + H2(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 2; i += 16 / 2) { + int16_t mm = m[i]; + for (j = 0; j < 16 / 2; ++j) { + d[i + j] = do_sqrdmlah_h(n[i + j], mm, 0, false, false, &discard); + } + } +} + +void HELPER(sve2_sqrdmulh_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int16_t *d = vd, *n = vn, *m = (int16_t *)vm + H2(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 2; i += 16 / 2) { + int16_t mm = m[i]; + for (j = 0; j < 16 / 2; ++j) { + d[i + j] = do_sqrdmlah_h(n[i + j], mm, 0, false, true, &discard); + } + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, bool neg, bool round, uint32_t *sat) @@ -373,6 +403,36 @@ void HELPER(sve2_sqrdmulh_s)(void *vd, void *vn, void *vm, uint32_t desc) } } +void HELPER(sve2_sqdmulh_idx_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int32_t *d = vd, *n = vn, *m = (int32_t *)vm + H4(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 4; i += 16 / 4) { + int32_t mm = m[i]; + for (j = 0; j < 16 / 4; ++j) { + d[i + j] = do_sqrdmlah_s(n[i + j], mm, 0, false, false, &discard); + } + } +} + +void HELPER(sve2_sqrdmulh_idx_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int32_t *d = vd, *n = vn, *m = (int32_t *)vm + H4(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 4; i += 16 / 4) { + int32_t mm = m[i]; + for (j = 0; j < 16 / 4; ++j) { + d[i + j] = do_sqrdmlah_s(n[i + j], mm, 0, false, true, &discard); + } + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 64-bit */ static int64_t do_sat128_d(Int128 r) { @@ -452,6 +512,34 @@ void HELPER(sve2_sqrdmulh_d)(void *vd, void *vn, void *vm, uint32_t desc) } } +void HELPER(sve2_sqdmulh_idx_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int64_t *d = vd, *n = vn, *m = (int64_t *)vm + idx; + + for (i = 0; i < opr_sz / 8; i += 16 / 8) { + int64_t mm = m[i]; + for (j = 0; j < 16 / 8; ++j) { + d[i + j] = do_sqrdmlah_d(n[i + j], mm, 0, false, false); + } + } +} + +void HELPER(sve2_sqrdmulh_idx_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int64_t *d = vd, *n = vn, *m = (int64_t *)vm + idx; + + for (i = 0; i < opr_sz / 8; i += 16 / 8) { + int64_t mm = m[i]; + for (j = 0; j < 16 / 8; ++j) { + d[i + j] = do_sqrdmlah_d(n[i + j], mm, 0, false, true); + } + } +} + /* Integer 8 and 16-bit dot-product. * * Note that for the loops herein, host endianness does not matter From patchwork Fri Apr 16 21:02:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422667 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp755589jaf; Fri, 16 Apr 2021 15:12:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw+UODDySsslOAMlfqPtQ1uoar2dVC05YqFkMiOzngxfEiZGLJMDufI/Dj4vUcp3f1mbWGb X-Received: by 2002:a1f:4343:: with SMTP id q64mr8465463vka.4.1618611129284; Fri, 16 Apr 2021 15:12:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618611129; cv=none; d=google.com; s=arc-20160816; b=rsPIi02rjHwlfeNnFfzCM0r/XppbRHh19Y3MtTH49H9nQUV1Yuem8mlMjxlW9EkzVE rEDcXnMzgVZk8mAl4pOuvUi4xiHeki1IwHVRy+UI0qqCtnHDr7SuHoIvO++veR+hDVDn 9SVFZHvm+Ia0IPNixj2l3UmBuBVksczIto5B9mNgBkpRdlCLSBB7TS0SmQQ6B6A+8VrQ NFBP5gC73jRsHX7j4G1kdz06ZnU5iWRA94LUYHJ17SXZtKgG6xs9a8EFpAFPGAng5GvM HjUKEQTylyEXTDD51C+eeIJl5MrkY+fqZQnhQHg6aLDREjaZpLgWiN4eBMYF2m4t+4UO OGjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=hE9TgsBmlGieqIDzowyo5vXw3quweqaP+9DzBYJUIRE=; b=0h9OVwvHOGwtzXWglErO401QXIcXJMIK7qNyeWsJKs/1JPEP49jgAgULew1EW70Q2l 7Od2WB0eMiuNPxD6Im/jb4g/x1FUA7K6SwrMqgtp229fRnyhD3iPuk3e/qU/x4siKA6r 4dY8XFEgx7K5Mz0o4rjDWw/8GFy9naTLdu9CMch2VXZNtX7AMqg+QmFNITmE21FXxQn5 XjsDNyoRap5LbCHKxssk1nZjRBpfgaivD5umSD6n/0Vy39C/O2zLUcy/DpnNyic2fD6B Ve1p6+y3JmJEhyZkvgm5/TmVkXNTO6mFcrb8ItNRZ3o1s0KgZohJ2m5tqFH1pS3optfF BWSg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CQlPYEMo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c13si3041485uak.249.2021.04.16.15.12.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:12:09 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CQlPYEMo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52752 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWh6-00089m-84 for patch@linaro.org; Fri, 16 Apr 2021 18:12:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45382) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdl-0006N0-06 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:37 -0400 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]:34734) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001jA-Hl for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:36 -0400 Received: by mail-pl1-x62a.google.com with SMTP id t22so14310770ply.1 for ; Fri, 16 Apr 2021 14:04:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hE9TgsBmlGieqIDzowyo5vXw3quweqaP+9DzBYJUIRE=; b=CQlPYEMouLjD90rPiX1Xn6jPNltluWYIB2th80uMKcJN8ekwStHJGhcJDETeO5sBQd 1APjONTHRFct9eHSh2JlHghvULl9+69r3rUFGUvr/xQHV478hKqxf4maoZQ8gQKrOaLa NgFcTJaoiGs+f61nrwQiBKoXSYCpYuVCFc438u+eS6seWsXC+SuRlxdjSuUSeb4JwsMY o+Z5iOdf96nAPCg/3GjRJsMx/BWyJbF/lJFQwkTG8MowQySWnwNxqGaM/aoDLqLUmqAA 4ocNBeLKvrfrpwB+oUCuHohnuZZqzO28OU0F6FetHxlB20aue5wVAxIlzl3GUc0oNroP 0GLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hE9TgsBmlGieqIDzowyo5vXw3quweqaP+9DzBYJUIRE=; b=ZmuBQD5jQeVLgIp/AAmeRijxooGgFgeg7IdXXFg2mcMSWlil6PVeJpYP6PRgbIdYIM DhjWmMe893ff4Vhf4O2ZvFxFKqUAZVLf5gMHeF1ASrc62NCeUo3y6tqNSB6kUZjdpU82 SsBEDxhEV4ifgK+/KntlVzOxiTlxCGJ4rnidEYxWRyrJTuQadpDnbURlvTxf0Kycijkz 6l2V+TuPFneEKZd/4RgW2eLyxGUdtTH0Vlh4QWX+ATJTfRfWufOJRbP1bMy7s/1qowQB XToIKOjukx/aDckIOE0o+efeS2Ik9eGKOu7EUXB1Hqo+DKKz6dxYv3RAY7wAiVxopHEP QjGQ== X-Gm-Message-State: AOAM530WeaGo1UmKl1erOD3P006bJ6wboaTWApJ/Rn2dMj4Ffaus35bh h/p2AiqsLL9aqfibG9zfnQFsDh6CkQQe5g== X-Received: by 2002:a17:90a:e512:: with SMTP id t18mr9429819pjy.34.1618607050109; Fri, 16 Apr 2021 14:04:10 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:09 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 59/81] target/arm: Implement SVE mixed sign dot product (indexed) Date: Fri, 16 Apr 2021 14:02:18 -0700 Message-Id: <20210416210240.1591291-60-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++ target/arm/helper.h | 4 +++ target/arm/sve.decode | 4 +++ target/arm/translate-sve.c | 16 +++++++++ target/arm/vec_helper.c | 68 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 97 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index e44bb8973a..132ac5d8ec 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4246,6 +4246,11 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve_i8mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, I8MM) != 0; +} + static inline bool isar_feature_aa64_sve2_f32mm(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, F32MM) != 0; diff --git a/target/arm/helper.h b/target/arm/helper.h index e7c463fff5..e4c6458f98 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -621,6 +621,10 @@ DEF_HELPER_FLAGS_5(gvec_sdot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_udot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sudot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_usdot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcaddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 35010d755f..05360e2608 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -813,6 +813,10 @@ SQRDMLSH_zzxz_h 01000100 0. 1 ..... 000101 ..... ..... @rrxr_3 esz=1 SQRDMLSH_zzxz_s 01000100 10 1 ..... 000101 ..... ..... @rrxr_2 esz=2 SQRDMLSH_zzxz_d 01000100 11 1 ..... 000101 ..... ..... @rrxr_1 esz=3 +# SVE mixed sign dot product (indexed) +USDOT_zzxw_s 01000100 10 1 ..... 000110 ..... ..... @rrxr_2 esz=2 +SUDOT_zzxw_s 01000100 10 1 ..... 000111 ..... ..... @rrxr_2 esz=2 + # SVE2 saturating multiply-add (indexed) SQDMLALB_zzxw_s 01000100 10 1 ..... 0010.0 ..... ..... @rrxr_3a esz=2 SQDMLALB_zzxw_d 01000100 11 1 ..... 0010.0 ..... ..... @rrxr_2a esz=3 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index b43bf939f5..1f07131cff 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3838,6 +3838,22 @@ DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) DO_RRXR(trans_UDOT_zzxw_s, gen_helper_gvec_udot_idx_b) DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) +static bool trans_SUDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) +{ + if (!dc_isar_feature(aa64_sve_i8mm, s)) { + return false; + } + return do_zzxz_data(s, a, gen_helper_gvec_sudot_idx_b, a->index); +} + +static bool trans_USDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) +{ + if (!dc_isar_feature(aa64_sve_i8mm, s)) { + return false; + } + return do_zzxz_data(s, a, gen_helper_gvec_usdot_idx_b, a->index); +} + #undef DO_RRXR static bool do_sve2_zzx_data(DisasContext *s, arg_rrx_esz *a, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 8b7269d8e1..98b707f4f5 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -677,6 +677,74 @@ void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(gvec_sudot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; + intptr_t index = simd_data(desc); + int32_t *d = vd, *a = va; + int8_t *n = vn; + uint8_t *m_indexed = (uint8_t *)vm + index * 4; + + /* + * Notice the special case of opr_sz == 8, from aa64/aa32 advsimd. + * Otherwise opr_sz is a multiple of 16. + */ + segend = MIN(4, opr_sz_4); + i = 0; + do { + uint8_t m0 = m_indexed[i * 4 + 0]; + uint8_t m1 = m_indexed[i * 4 + 1]; + uint8_t m2 = m_indexed[i * 4 + 2]; + uint8_t m3 = m_indexed[i * 4 + 3]; + + do { + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); + } while (++i < segend); + segend = i + 4; + } while (i < opr_sz_4); + + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_usdot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; + intptr_t index = simd_data(desc); + uint32_t *d = vd, *a = va; + uint8_t *n = vn; + int8_t *m_indexed = (int8_t *)vm + index * 4; + + /* + * Notice the special case of opr_sz == 8, from aa64/aa32 advsimd. + * Otherwise opr_sz is a multiple of 16. + */ + segend = MIN(4, opr_sz_4); + i = 0; + do { + int8_t m0 = m_indexed[i * 4 + 0]; + int8_t m1 = m_indexed[i * 4 + 1]; + int8_t m2 = m_indexed[i * 4 + 2]; + int8_t m3 = m_indexed[i * 4 + 3]; + + do { + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); + } while (++i < segend); + segend = i + 4; + } while (i < opr_sz_4); + + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { From patchwork Fri Apr 16 21:02:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422643 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp742386jaf; Fri, 16 Apr 2021 14:48:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyZkrSo+DKiYO1TpYmnWccZj+S+YeEdwcpOosXdyWcYAaeK8Egx9x3DL6Y+BNNRIzxeWDZq X-Received: by 2002:a02:a807:: with SMTP id f7mr6028956jaj.54.1618609688757; Fri, 16 Apr 2021 14:48:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609688; cv=none; d=google.com; s=arc-20160816; b=lomWbuMM++ETo7xqVhBIPTJwKB+p2BqlLl+u62nvnalN5a+6LoarJwn8Ha5LukMWtt +2J9Ztkq6Qx3wpqtmy8URYTzp8JFeDEjanqRx0ft4Tz00ziPPK5Z7gBE/dVq6BF409VE j4D/avhxBeidXREywUttPJ6b7PoWObBjTuQhi0lI72Qq4V67QD07wUmy0EEJgL4LnsDV TCF71u6DKeUbDI8otKUzQWlyxj+9aC2LWFqt9fftyGwXYJ8uG+56+gO7b54QXvTlubVc ZZ+wGgZxX2JbCfAEStgHDBVB/azRX+aGOZlC/qT8W++DD1fbe4P6x8L7uLyo8FX3X8U4 12aQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=6+bWqDpe9bFb8HpI6D4DHTRWk4CTefRX0cnIpSJEYXs=; b=km+iWFAmdVEObvl2qbXl5F063p+qqmi2BUCTOp3E51vI60liZCpYXAo70scjZBNSLi sz0kYqaT89qGBLKmKmIy+eDSkefB4rgJv1eAYUyyjhIASd9WK0FK0ZuGRIcVeQrTciC0 Anxmmo7sdwZWkiG7w4JCLYt1rUYE+rNuIqHpJqvxfQhtlWUwP3XzHUdLvzGwnPkt4E7Q gFR6n/PsMWGh4fkkCvtu4ugn68t6Thr4GotDu4/s4SoAFxwZtqSAqIM/rMqBjfHHGMQI s/zQVwjySpRTG5w39PnJSVDZJWq38+oMWt2IonJ5RbeIB/dTHiqNuFipI9LGzApvb/jE ob/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xLr+k5PZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l9si7326681ils.131.2021.04.16.14.48.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:48:08 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xLr+k5PZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60586 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWJs-0001CQ-61 for patch@linaro.org; Fri, 16 Apr 2021 17:48:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45468) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdo-0006Vs-Dm for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:40 -0400 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]:54221) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001jU-KJ for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:40 -0400 Received: by mail-pj1-x1035.google.com with SMTP id nk8so1511949pjb.3 for ; Fri, 16 Apr 2021 14:04:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6+bWqDpe9bFb8HpI6D4DHTRWk4CTefRX0cnIpSJEYXs=; b=xLr+k5PZgpuOZBW90nsmRSiaL1uAfsb3ytPt0y4sZoB0K2hxDMbkRwgOgW/OTD5UkG Cp0TV77e8kxJda06niIshs6BY9r1gcSSIU0bzXVS9Y4h5k3sUznGZynNTTkq12f2P2zZ q5WJu6EhVW+D0j+t53TdK4Jt5CNG97BPAu2uUQOOnsDGBQByeHsewNbHz9u4Hy/hnPER v3cGvusFZuwrE9FVeohCQmBMinjf2qsbcdg12+hARsLh0zI3tGKo34KRUqIAa7w9GJIZ p5+zBqte8iGaXgkuEAyQKHoEEryjDjw8JaXh+buKj1EiW+osA6+OcrhXPZrFfK5Ga0pQ Gg6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6+bWqDpe9bFb8HpI6D4DHTRWk4CTefRX0cnIpSJEYXs=; b=ARaWm+M/sxoXwQPyxC7JL1lGHxnk6TGZurGOhDdW5ffq7ij6OzS6w6mnijlj0LRs6+ m90h/UDV+d+MJPWoklbQSIGXxKUY4WLGFoYnFELdp5wXxvWEjElLusUiOkPEjwP/tIR5 m9DOz3oHaBI1zs8CCY35rTNzbmy+VyUiE7hrgh3H8WdoTSvuR+BlJrdfkBPDz/bqfWkr agamq4CTgtuscVXVB4yuZZc24O4vRw4bBcS1N0GDHPMgXEAljoTB1tLbwmobN9XicjP3 d4mxPmgdNXNEg3rJde9eklUrl+rWQqAehyEi5751otumIY741y1ap0tXolRUSlpUdlc3 9eIQ== X-Gm-Message-State: AOAM531pH37mi1EeSB72sJOaO/HpfC1ZDvzOPaz6qGZTpI5ihf16tOWJ GGg+q5NVomkd8Dror6/PMNdp1X/NsnyXLQ== X-Received: by 2002:a17:90b:388a:: with SMTP id mu10mr11202454pjb.203.1618607051049; Fri, 16 Apr 2021 14:04:11 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:10 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 60/81] target/arm: Implement SVE mixed sign dot product Date: Fri, 16 Apr 2021 14:02:19 -0700 Message-Id: <20210416210240.1591291-61-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1035.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.h | 2 ++ target/arm/sve.decode | 4 ++++ target/arm/translate-sve.c | 16 ++++++++++++++++ target/arm/vec_helper.c | 18 ++++++++++++++++++ 4 files changed, 40 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index e4c6458f98..86f938c938 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -612,6 +612,8 @@ DEF_HELPER_FLAGS_5(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_udot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_usdot_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_sdot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 05360e2608..73f1348313 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1481,6 +1481,10 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx +## SVE mixed sign dot product + +USDOT_zzzz 01000100 .. 0 ..... 011 110 ..... ..... @rda_rn_rm + ### SVE2 floating point matrix multiply accumulate FMMLA 01100100 .. 1 ..... 111001 ..... ..... @rda_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 1f07131cff..0da4a48199 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8056,3 +8056,19 @@ static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a) } return true; } + +static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) +{ + if (a->esz != 2 || !dc_isar_feature(aa64_sve_i8mm, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + vsz, vsz, 0, gen_helper_gvec_usdot_b); + } + return true; +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 98b707f4f5..9b2a4d5b7e 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -579,6 +579,24 @@ void HELPER(gvec_udot_b)(void *vd, void *vn, void *vm, void *va, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(gvec_usdot_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *a = va; + uint8_t *n = vn; + int8_t *m = vm; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = (a[i] + + n[i * 4 + 0] * m[i * 4 + 0] + + n[i * 4 + 1] * m[i * 4 + 1] + + n[i * 4 + 2] * m[i * 4 + 2] + + n[i * 4 + 3] * m[i * 4 + 3]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + void HELPER(gvec_sdot_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); From patchwork Fri Apr 16 21:02:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422635 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp738995jaf; Fri, 16 Apr 2021 14:41:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzvOau3MfJXYgqA7q7c3Nd2801eZw7Qk8Mdz9kOUknpSTMGbbECeKU9SW/MwroKJPgNtQUN X-Received: by 2002:a5d:97cb:: with SMTP id k11mr5244062ios.204.1618609273570; Fri, 16 Apr 2021 14:41:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609273; cv=none; d=google.com; s=arc-20160816; b=yZnx+W+gkP2mBslRxeojZ+q4aRZp4NFekGerlx3saaW/bgKCwg6O/YprE7KW6Fn59w yMTvKsyUmcYVWECuwPmoqjATqCYrL/LH3p7iKKtyjqGu5yXujuak0/x+5M3zTv/8nVK/ W1hJrhQL4iA3GCPpffweJw15/Lkm6HAfnb2tL9DTCFqiMLw5eopvtEsanUZ39U24GOFR wQB3HeaDQ4vP7e4XbQgiGSkak8tWQY4veMpq9/pC5uEsGLYFWpnIkNSuk95cH7BHk0oN iY6A6aLj9F5RiZ40dPpxTR0vcgv5H8LlTcfHgvaYWTkjx8dqaXYj+jiqZieeDrASwSnq U+dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=MCDHvXiyFXS1mzEiFOdu1UrmPEyO05QZiiThz71om18=; b=Si+AtiDntIbC4/ztMqCTpTbc9n3WuwDIXZA/AgioYT1INMF36Yef0ZpP/+adSwOnTY XaZKHGXeyGRBDpLFGIR4rI5XZnJGX1M+YwStl7sVbE3eAuea3DRSZDZk805r26NZI9QF QsxzPsg66QZk9C9nyXcuJeDR+RN1kgZMGq1LPdehdLdp5otgSw34sIVCE7/MimQPvehq +V3DxpanleTVDuP1OoU0OmKczNKLvJ3MTUaedZQrW3ytvmSWewhPVn++7Q1nmFJvAt4H HfzRZ7IMLHV+HVmasJadOL2YqKk7mhx/4UbSjrQ5KIv7mJyVhx+JfpkUTJj+IsmNfkVz iXXg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KVAj4DKa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g13si3343198iow.1.2021.04.16.14.41.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:41:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KVAj4DKa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40320 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWDA-0001ND-WE for patch@linaro.org; Fri, 16 Apr 2021 17:41:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45254) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdf-00068Q-Od for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:31 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]:43789) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdQ-0001je-KL for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:31 -0400 Received: by mail-pf1-x42d.google.com with SMTP id p67so14195073pfp.10 for ; Fri, 16 Apr 2021 14:04:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MCDHvXiyFXS1mzEiFOdu1UrmPEyO05QZiiThz71om18=; b=KVAj4DKavMVJw5wI5GH7QMxT6FW2E6YDuOuPBeY4z4tDTmfUCans+feT5uli14Pttv ff+JQ2O3W7n+DbVczxme2y7lXpYycyMmtqzaQllI3D1mSxFLuzx9toZB6z+9WeVIEjH7 Wpz1uo58AhTmdKy4la8tdXliFNS8dJ2GfjY5/5D8Jhz5/xVsXsqelzpGWCLYWx2TNuXc RwFPNqa+/yEqXPvihi9PmE/bvw8qyBcyFylq/yZAJ3OqK4m6CeukoDP4tJlePPv52aIJ s96xgUJTpUeq9Sl684FICuPrdO4d8Bd0zq2xJMu4h8bMHMHyQorFI34ziImSa1U/YJbe yfyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MCDHvXiyFXS1mzEiFOdu1UrmPEyO05QZiiThz71om18=; b=KKsvZ1QESjHh5uSoGPlD2uOcE0QMiYi68qwfZOUTHOGu+4tdt93tF+F0jioAvQ4t0R SPFYPpeFtbbgox1nl5x8qfdp4IfqjbCsvP+jLYqxAKzvUytk2oMLG2pSYmVZ4R4lfMvT SWmMCBNMrhG0UAEGa7oGA9UjbyJq9URBksCI6iAGHFBuZz/vqtw3gDgqfq1RoNwK8W2Y nEdivjbbSO39Yjff/th2ixNLRySWKm5wos5Mrz0BzRfPXU1e545OGryTFG8t0w1yUkRR DdESYD5pqlmIohD3TXINxeWOAaCVo/3sDZUSHTGShPqlxQNEeh8CsAkUpYTPnhfv0tZR 7Ipg== X-Gm-Message-State: AOAM531yjEUun2WIsj/jPic1ijjjrdmaoWTgg2oLdeVUTCAoJBvTartH N02DC/5CqOgGdzkNZudhmb8SfBdHaNvsVA== X-Received: by 2002:a63:5223:: with SMTP id g35mr864332pgb.164.1618607051735; Fri, 16 Apr 2021 14:04:11 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:11 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 61/81] target/arm: Implement SVE2 crypto unary operations Date: Fri, 16 Apr 2021 14:02:20 -0700 Message-Id: <20210416210240.1591291-62-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 6 ++++++ target/arm/translate-sve.c | 11 +++++++++++ 2 files changed, 17 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 73f1348313..6ab13b2f78 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1509,3 +1509,9 @@ STNT1_zprz 1110010 .. 00 ..... 001 ... ..... ..... \ # SVE2 32-bit scatter non-temporal store (vector plus scalar) STNT1_zprz 1110010 .. 10 ..... 001 ... ..... ..... \ @rprr_scatter_store xs=0 esz=2 scale=0 + +### SVE2 Crypto Extensions + +# SVE2 crypto unary operations +# AESMC and AESIMC +AESMC 01000101 00 10000011100 decrypt:1 00000 rd:5 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0da4a48199..4213411caa 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8072,3 +8072,14 @@ static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) } return true; } + +static bool trans_AESMC(DisasContext *s, arg_AESMC *a) +{ + if (!dc_isar_feature(aa64_sve2_aes, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zz(s, gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt); + } + return true; +} From patchwork Fri Apr 16 21:02:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422648 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp744633jaf; Fri, 16 Apr 2021 14:53:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzzb0n2CBo6fRabElyf2Q9aBJGR9UPe/o7hRW4aE+sGNdeoY4wnaGPBQjdMBpfSPj/y4paB X-Received: by 2002:a05:6602:1da:: with SMTP id w26mr5268854iot.170.1618609987057; Fri, 16 Apr 2021 14:53:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609987; cv=none; d=google.com; s=arc-20160816; b=KKnVUjYNi3VPuGfzB5Ii8V1VhfbKxJBsHXWB9QL57NdpD2pZ9EDLxkBImYsO6FDe15 rgOgDa2YvdaJ7fq82GkhHa7cbhrZB7ubIQQLkO4r8AI49fF4bLxU6ua9hflC0khLW0oa HeMjv4WO+93Li3Efao2BeqM8z/53GHU7yhABV6o60oMlUZfsiO7A/JD6VLwdsYVKmtuD 45DbtI72KlkmiZW+H9iiJX24K46WIJ+0gOqREcfnXEOG/ROGj0d9xAbtYZf0mESZ95fB ps9Y+bxpLkXpw4CQ+pvuw3qWNpbaPKzW9HHoBsCJJRX3e6BPcgG8QfWtO8JqvHzzVKir DP0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=qO7nX/keqWMY4kFJdwer6bjckmqvkxXLwxerKkDKNvA=; b=m8COotDpyiOO7UJjrzmvogZCo6tffPXodujAodi7iUgP5rgEz3JhKSGRw4kDIRlOw4 u2a7l1rhnLMwzLjEFEIFTf1X53bGK9M3Vun2OmY3f2KzzQrvjeMvAmwwE46uIOMVk/us wTXspY/5bWXPTGW+SrUlpnVxxMD9mnj/4cWn2v2S+YTTyu6HkU9LlNgIg+MALqxj2Nnt jer1EV8Y+YYR+/uvic2mlGDCgae8ZlgLnMSuKF+hl48MaQdY/5P7n5WT3htF3L0A9+EC i4uC/FiLg6gCWYTcglOLq6RS8tYDYP0J5MFoi31dEdT5OpzJWKgv8d+b1jT0DKtyJpxF peLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NbSJgHjw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x7si624861ilv.97.2021.04.16.14.53.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:53:07 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NbSJgHjw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45460 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWOg-0006gU-8i for patch@linaro.org; Fri, 16 Apr 2021 17:53:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45288) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdh-0006CX-7t for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:33 -0400 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]:43943) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001k5-EA for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:32 -0400 Received: by mail-pj1-x1035.google.com with SMTP id x21-20020a17090a5315b029012c4a622e4aso15261616pjh.2 for ; Fri, 16 Apr 2021 14:04:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qO7nX/keqWMY4kFJdwer6bjckmqvkxXLwxerKkDKNvA=; b=NbSJgHjwQselsNtZ3gvP6mjDDx8hz9ihxjcRidOFkO4fVfARuAWzVG+hYzIMoXc1ts NYWDl8+CHob4iYLQQxr58EhxpJKqJd//BoXPki0xysXwIdGoCQ3hqXuIMM/g1pjs7ihR ky137UiGqeZUvqnNaqNIolx+IGQtLcvw1gLilNKOXyPzB/BOUEG8UhknR3wcE+xO8f6q fMR5cS6hEPCyj8bn3e30wDmQcugxn8t2HXuQtw2gAORzDRzLrIGjsiZLfXT6uAntJp49 KPOyJKK5nWl7/Em3KTC2yy5GFS4rsF6d0Z4MRAEwJ1UryVsZS8qkx5mv7o6R60LwJMf0 WPUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qO7nX/keqWMY4kFJdwer6bjckmqvkxXLwxerKkDKNvA=; b=Qk0JsCccsxAj94pSVJotHTzboYoDDH12Xkn7EeVtGSsusfPBI06VImJOPCTFxhDF+q eUECLQJe1IST2s/OATgyWsMwMFFHruNAnjDvzUNQvAKKAJvVQQgGnWnHRHugRQpI/pOt +gjsxKjPRo9zsT3S+MI7VTsGI1S4h92x5xRJBwZQX0SR/+9x3Sov2JacUlGcsX3nxcsA FrKAVYvPZmJ0sSRBFzWt1KPvRv1HRdKmq/1RpgxfafMFBzACcKCeeaGNc8FgNmKFyJFe R1tmg3J3U+1YXaz/zgIaR/Wy13cDspLjJ0KVDIg/txOTbY9m5QlQWMOvKzyT86tda0oh 9WnA== X-Gm-Message-State: AOAM5339MYW5WSqK6bM/vV8Z96X665S1WtePGcQ57Pi1mFrigq1bP1y7 Z1/79HRYGQOmzGB+iZxx5noJD66m+/VdIQ== X-Received: by 2002:a17:90a:1990:: with SMTP id 16mr4970193pji.172.1618607052898; Fri, 16 Apr 2021 14:04:12 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:12 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 62/81] target/arm: Implement SVE2 crypto destructive binary operations Date: Fri, 16 Apr 2021 14:02:21 -0700 Message-Id: <20210416210240.1591291-63-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1035.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++++ target/arm/sve.decode | 7 +++++++ target/arm/translate-sve.c | 38 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 132ac5d8ec..904f5da290 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4246,6 +4246,11 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve2_sm4(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SM4) != 0; +} + static inline bool isar_feature_aa64_sve_i8mm(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, I8MM) != 0; diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 6ab13b2f78..fb4d32691e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -118,6 +118,8 @@ @pd_pn_pm ........ esz:2 .. rm:4 ....... rn:4 . rd:4 &rrr_esz @rdn_rm ........ esz:2 ...... ...... rm:5 rd:5 \ &rrr_esz rn=%reg_movprfx +@rdn_rm_e0 ........ .. ...... ...... rm:5 rd:5 \ + &rrr_esz rn=%reg_movprfx esz=0 @rdn_sh_i8u ........ esz:2 ...... ...... ..... rd:5 \ &rri_esz rn=%reg_movprfx imm=%sh8_i8u @rdn_i8u ........ esz:2 ...... ... imm:8 rd:5 \ @@ -1515,3 +1517,8 @@ STNT1_zprz 1110010 .. 10 ..... 001 ... ..... ..... \ # SVE2 crypto unary operations # AESMC and AESIMC AESMC 01000101 00 10000011100 decrypt:1 00000 rd:5 + +# SVE2 crypto destructive binary operations +AESE 01000101 00 10001 0 11100 0 ..... ..... @rdn_rm_e0 +AESD 01000101 00 10001 0 11100 1 ..... ..... @rdn_rm_e0 +SM4E 01000101 00 10001 1 11100 0 ..... ..... @rdn_rm_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 4213411caa..681bbc6174 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8083,3 +8083,41 @@ static bool trans_AESMC(DisasContext *s, arg_AESMC *a) } return true; } + +static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt) +{ + if (!dc_isar_feature(aa64_sve2_aes, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, gen_helper_crypto_aese, + a->rd, a->rn, a->rm, decrypt); + } + return true; +} + +static bool trans_AESE(DisasContext *s, arg_rrr_esz *a) +{ + return do_aese(s, a, false); +} + +static bool trans_AESD(DisasContext *s, arg_rrr_esz *a) +{ + return do_aese(s, a, true); +} + +static bool do_sm4(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn) +{ + if (!dc_isar_feature(aa64_sve2_sm4, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); + } + return true; +} + +static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a) +{ + return do_sm4(s, a, gen_helper_crypto_sm4e); +} From patchwork Fri Apr 16 21:02:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422660 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp749647jaf; Fri, 16 Apr 2021 15:02:28 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyzz1iQVR9MkkervedKMECWQd6yQwEDMhYmv6NrBhNdKUoDdF8xrBEcu7f6tAPvffV81j9i X-Received: by 2002:a17:906:4cc5:: with SMTP id q5mr10373280ejt.302.1618610548241; Fri, 16 Apr 2021 15:02:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610548; cv=none; d=google.com; s=arc-20160816; b=ft+XH5MSz84OxnFNpZaGhgVVywrpHpM0biTN6GJBxmeINYA+/jdw5pRmrB9vdH3LmL EulkKMpU2ltdcu4lDcAntWch2YhOyMMvSY5IDeQ89DL6h6hp6IF1BtY6C60v/ILeRZG+ BzoV+piEd+ewvdlsp2tgvgTxDhT5WL1obixMWp2h1zieJX6cqfAtQKBvOr9RpHjuLJ7E G/QDjtbx5YIP97WCF/xykdvD41iBPRzgJBokpOG38kG4wGU877WUn9khim52jj4MjT9a 8k9C2VVtmSNai1cx3Medx+2XrQZBGzK7UKQpOlay0ctGcucvY+HY3r5OzXSgiQtQ/WFo 645w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=08G/gmmis9kTq4Wos8KgqE9HaSoRMxfx4IEhQdOfahw=; b=IPnu6mKTg8I7sn+aW8lAlhrU6VNMF951hxjmPjhAcBebVWiANpMix0g+M+njw1VtME Lv42HAiHOL85Phi30XrxyYkYKiBNGqLcfTLX9SEYQcsI9yAG8/7l+gpRA3bWaEGzTbCI o5j9hJZnugJjPrl9SirEVf0e3An59tmHyDgMv4p/aygpEjtJtPxAI6beAxdAQkYHQZvb /n5mBqZnQ/trDOj8j1uOgxwzE65F9orsJSZHkPbhDj5smBhtDZ6HnZMKhtpOv14XJklp 1X7AN8jo/iYGOIXYzBBRLsF846vjxd+UmOjdEY1aGdy1IEcT5CC2MOylcY4wl6rHrL9b AK9w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bX6wkcAp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b13si5929183ede.383.2021.04.16.15.02.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:02:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bX6wkcAp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:48882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWXj-0002cq-9V for patch@linaro.org; Fri, 16 Apr 2021 18:02:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45460) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdo-0006VA-5l for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:40 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]:33426) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001kV-JK for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:39 -0400 Received: by mail-pg1-x530.google.com with SMTP id t22so20043965pgu.0 for ; Fri, 16 Apr 2021 14:04:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=08G/gmmis9kTq4Wos8KgqE9HaSoRMxfx4IEhQdOfahw=; b=bX6wkcApBUdRtW3onP015fnjn/e0q9c4D54oStV9cNg8cbKTyWq1BNDjzE2VJS1ffn 8NJCjlUImC25+ZfGfV6tYp6aEi+QDZSg8p0frW+FVOP9wU566Fy52pwtxTJf99kR8YPK u8kBzMqzWWroaPQHQqohk2KbCZX5PfjOkHiMVJKsWtfnmMeX7T6Z5HRzwabY5FRXLTu3 IW2w1sWuHVBJ7VnMEXYlCeDTUAyQO/zIneAvrhw57FbcEEn4RVaOK0fD034jkU9ukf1I Y2iXK03pQ8kKwEPkWNgBnQnZOKZWUgIwjYAoYNHItopEIVW/7Yj0y6HZEO6Paf5ajF6R u5sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=08G/gmmis9kTq4Wos8KgqE9HaSoRMxfx4IEhQdOfahw=; b=p+Sz3UNiWZSSI8xWY95lVa98AQGK5ZOAe4QfyLpAIoE6/J9JEIIA8WmigeIgP8JAQh jaCSYSGyLJiXKhhGp/hP5IVzDgoIC9J0GYRYYha3PK6GL06hqgWM08fHU8F0YsNWiPyR q/2cRKwF7Pf/kq4qEI71YX3TsDsvyvAvithgWGyt6BUTe9UoUtD1gBuUIafQMy4sltCs IT1gRQxbzuG6BU7aAyvf2UCE4q5GKP6AwIHXfdiYzK4n3Nhv3Jvulz8tcZ/OHT3Q7m5w CZhmdpP4mlcYHHl5I/0/t3BxHKs7aBGkFXyk2GACRSG4PzN2UwS0lxj9gVDZDokE5B8d afdw== X-Gm-Message-State: AOAM531ylIYXMIWDr2cOQXowjDIwr5N5/77R1b/Q4d/S9MYGu6hwQsJl 1IMy3vq50oaVsBU7RrSlfYLXOMmieExwdw== X-Received: by 2002:a63:5807:: with SMTP id m7mr816391pgb.73.1618607053934; Fri, 16 Apr 2021 14:04:13 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:13 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 63/81] target/arm: Implement SVE2 crypto constructive binary operations Date: Fri, 16 Apr 2021 14:02:22 -0700 Message-Id: <20210416210240.1591291-64-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++++ target/arm/sve.decode | 4 ++++ target/arm/translate-sve.c | 16 ++++++++++++++++ 3 files changed, 25 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 904f5da290..b43fd066ba 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4246,6 +4246,11 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve2_sha3(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SHA3) != 0; +} + static inline bool isar_feature_aa64_sve2_sm4(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SM4) != 0; diff --git a/target/arm/sve.decode b/target/arm/sve.decode index fb4d32691e..7a2770cb0c 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1522,3 +1522,7 @@ AESMC 01000101 00 10000011100 decrypt:1 00000 rd:5 AESE 01000101 00 10001 0 11100 0 ..... ..... @rdn_rm_e0 AESD 01000101 00 10001 0 11100 1 ..... ..... @rdn_rm_e0 SM4E 01000101 00 10001 1 11100 0 ..... ..... @rdn_rm_e0 + +# SVE2 crypto constructive binary operations +SM4EKEY 01000101 00 1 ..... 11110 0 ..... ..... @rd_rn_rm_e0 +RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 681bbc6174..de8a6b2a15 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8121,3 +8121,19 @@ static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a) { return do_sm4(s, a, gen_helper_crypto_sm4e); } + +static bool trans_SM4EKEY(DisasContext *s, arg_rrr_esz *a) +{ + return do_sm4(s, a, gen_helper_crypto_sm4ekey); +} + +static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_sha3, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzz(s, gen_gvec_rax1, MO_64, a->rd, a->rn, a->rm); + } + return true; +} From patchwork Fri Apr 16 21:02:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422642 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp741975jaf; Fri, 16 Apr 2021 14:47:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyCsWYdp0hNi0mj3q2NZgvMzBD2LIIoZIQsq3Rp4WbfBmLZKVu696ibnlH6KBieVAwYXvPS X-Received: by 2002:a25:cf08:: with SMTP id f8mr1857423ybg.132.1618609632482; Fri, 16 Apr 2021 14:47:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609632; cv=none; d=google.com; s=arc-20160816; b=CwUCVf1KdxTV/fcxhEbxNqSbyJL/x4vNtYX9Cu9mmE9UPeMPRFG2bEMVyAsRJKEl4X TYO5YZIPbWRUCeRQS6ZP2JzQnsYNd+QVtXRF9oT0yTWKFZs583U0eHyIDA56FVbl5cKJ 5pK0E2LwquxfpQVmiz50HEn/M3+G8h7jFYKQoK1gbOsFo+DNnKhtS3/4K437o3sjZD/b m/5hlTYjZoM4vrnNOjg3DdSLAB202M/XW6VkWv0WuM9BshU7NJyxo9UdYQjnOvSjLxu4 zVcXa/3kJnsoVDnFEGUOA4RKzp4b++M/IEjTFbw2kUUg0UcWSJP/mEe/f9cXlyOvxFc/ ojhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ol1lmCzaRhxk8Z5yEk7BOIP02pWavr9/vkWdoeRx6P8=; b=C0VtY+rbpBwl7uLInND0FWk0yA5N3NnA2DXDR78Y9yZhc3UAnM3u4SqTKuQh2n/447 r+ZXzPfyk5Ilu250cbPQAPgp5U1tq0s9e7Fg2HW+0hseonGHLbKFNHv1WN/uV+YBNpXL Po9+8Fmr5CGIZp3AEDa3OGOBiTv3uMCwUHxWKrFmBGq9vHYH05qH6/kRb8hTU/Ds2og3 BGDjfKBxCwazfYJmmjhctZC3t8gjx/9rKKvqRyN+0bNLqJiYWJb/UwD38R/Zhvprkloh xjd6q0yQfx8YNh4oFkNSaK6/S83fH3ZDh7nc2gyAcdLcPalEK3kUtX64THyq6d2wU4Vs MFWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QfJ1Rlhg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r6si7136187ybb.325.2021.04.16.14.47.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:47:12 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QfJ1Rlhg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59082 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWIx-0000bF-T6 for patch@linaro.org; Fri, 16 Apr 2021 17:47:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45344) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdj-0006H6-0V for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:35 -0400 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]:37782) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001kk-EQ for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:34 -0400 Received: by mail-pl1-x635.google.com with SMTP id h20so14641675plr.4 for ; Fri, 16 Apr 2021 14:04:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ol1lmCzaRhxk8Z5yEk7BOIP02pWavr9/vkWdoeRx6P8=; b=QfJ1Rlhg8DdXvGK8kPm/vjPWe2dZlfB93fQWyL9R4EYBGiWTW4w+9ZxK2juA6InDOH UEwWZ2AT8NE2pw2sxY8le35bvCjqYFR46eSlT4nvPZOl2e7cYTUAw2qenYhJiIj0rXdE LhnRDRakQ2WMt3+v9l7sVa/iMTO7+LwyUtjRcl30Oq+0armsTUYSrm//+E6O546+ysBe Kk5rcLLTwrWikIi1lGbYtrB1cwSgmuBnJec6rkUsb0HeJSnb/Z6JqbaLqgocRzUZ4bo2 n32hWkrXu43igNQnKJDvOl9GrCH+cwUFFqCa4Nhl0hSftBwLAH11X4CQjZW25BXkMqp8 BDPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ol1lmCzaRhxk8Z5yEk7BOIP02pWavr9/vkWdoeRx6P8=; b=BGHqK/Acg8lA7Xh3a5RQlEwZVbOKbc7oUpBCPS01bb2ATvkMAgvinoYfujeAPvho1R uPeaIqvjCKU6mYwD4DrTrxfK2HL7JJJdHkFyz3i84d1jnzYA3ucELZNcgwkSTwsurRTA agf8EDHPG0ZP0ah4J/Cr3KeHlUNM/uomAmuAkHX2LanJSgsedDL5HB9qv/y9Qel7pUPy 4H8kXlyIOR/Fmp0PkNOYSeLQLV77MnLCrp8j/GDv01TwxfjZGazD8GFGs8UtP7kMIq3X 2VmrTGxO9siGj6KsnNbC23+CkCBKcFxtsaKU71PiTxNNMOuTCZRE8HlJRfOKHOLf2rmu b6AQ== X-Gm-Message-State: AOAM533ZX2ikAVLcswGm0TUw66milfDh0Lnohwn5ylN7hH+IyIR7nshP V4bIjcq6EmWB/0d7yf5JoZZIWK0eME6yWg== X-Received: by 2002:a17:90b:3008:: with SMTP id hg8mr6920646pjb.115.1618607054976; Fri, 16 Apr 2021 14:04:14 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:14 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 64/81] target/arm: Implement SVE2 TBL, TBX Date: Fri, 16 Apr 2021 14:02:23 -0700 Message-Id: <20210416210240.1591291-65-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::635; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428144352.9275-1-steplong@quicinc.com> [rth: rearrange the macros a little and rebase] Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 10 +++++ target/arm/sve.decode | 5 +++ target/arm/sve_helper.c | 90 ++++++++++++++++++++++++++++++-------- target/arm/translate-sve.c | 33 ++++++++++++++ 4 files changed, 119 insertions(+), 19 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 0be0d90bee..41c08a963b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -661,6 +661,16 @@ DEF_HELPER_FLAGS_4(sve_tbl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_tbl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_tbl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_tbx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_tbx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_tbx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_tbx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(sve_sunpk_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_sunpk_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_sunpk_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7a2770cb0c..38aaf1b37e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -558,6 +558,11 @@ TBL 00000101 .. 1 ..... 001100 ..... ..... @rd_rn_rm # SVE unpack vector elements UNPK 00000101 esz:2 1100 u:1 h:1 001110 rn:5 rd:5 +# SVE2 Table Lookup (three sources) + +TBL_sve2 00000101 .. 1 ..... 001010 ..... ..... @rd_rn_rm +TBX 00000101 .. 1 ..... 001011 ..... ..... @rd_rn_rm + ### SVE Permute - Predicates Group # SVE permute predicate elements diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index e8a8425522..8dc04441aa 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2915,28 +2915,80 @@ void HELPER(sve_rev_d)(void *vd, void *vn, uint32_t desc) } } -#define DO_TBL(NAME, TYPE, H) \ -void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ -{ \ - intptr_t i, opr_sz = simd_oprsz(desc); \ - uintptr_t elem = opr_sz / sizeof(TYPE); \ - TYPE *d = vd, *n = vn, *m = vm; \ - ARMVectorReg tmp; \ - if (unlikely(vd == vn)) { \ - n = memcpy(&tmp, vn, opr_sz); \ - } \ - for (i = 0; i < elem; i++) { \ - TYPE j = m[H(i)]; \ - d[H(i)] = j < elem ? n[H(j)] : 0; \ - } \ +typedef void tb_impl_fn(void *, void *, void *, void *, uintptr_t, bool); + +static inline void do_tbl1(void *vd, void *vn, void *vm, uint32_t desc, + bool is_tbx, tb_impl_fn *fn) +{ + ARMVectorReg scratch; + uintptr_t oprsz = simd_oprsz(desc); + + if (unlikely(vd == vn)) { + vn = memcpy(&scratch, vn, oprsz); + } + + fn(vd, vn, NULL, vm, oprsz, is_tbx); } -DO_TBL(sve_tbl_b, uint8_t, H1) -DO_TBL(sve_tbl_h, uint16_t, H2) -DO_TBL(sve_tbl_s, uint32_t, H4) -DO_TBL(sve_tbl_d, uint64_t, ) +static inline void do_tbl2(void *vd, void *vn0, void *vn1, void *vm, + uint32_t desc, bool is_tbx, tb_impl_fn *fn) +{ + ARMVectorReg scratch; + uintptr_t oprsz = simd_oprsz(desc); -#undef TBL + if (unlikely(vd == vn0)) { + vn0 = memcpy(&scratch, vn0, oprsz); + if (vd == vn1) { + vn1 = vn0; + } + } else if (unlikely(vd == vn1)) { + vn1 = memcpy(&scratch, vn1, oprsz); + } + + fn(vd, vn0, vn1, vm, oprsz, is_tbx); +} + +#define DO_TB(SUFF, TYPE, H) \ +static inline void do_tb_##SUFF(void *vd, void *vt0, void *vt1, \ + void *vm, uintptr_t oprsz, bool is_tbx) \ +{ \ + TYPE *d = vd, *tbl0 = vt0, *tbl1 = vt1, *indexes = vm; \ + uintptr_t i, nelem = oprsz / sizeof(TYPE); \ + for (i = 0; i < nelem; ++i) { \ + TYPE index = indexes[H1(i)], val = 0; \ + if (index < nelem) { \ + val = tbl0[H(index)]; \ + } else { \ + index -= nelem; \ + if (tbl1 && index < nelem) { \ + val = tbl1[H(index)]; \ + } else if (is_tbx) { \ + continue; \ + } \ + } \ + d[H(i)] = val; \ + } \ +} \ +void HELPER(sve_tbl_##SUFF)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + do_tbl1(vd, vn, vm, desc, false, do_tb_##SUFF); \ +} \ +void HELPER(sve2_tbl_##SUFF)(void *vd, void *vn0, void *vn1, \ + void *vm, uint32_t desc) \ +{ \ + do_tbl2(vd, vn0, vn1, vm, desc, false, do_tb_##SUFF); \ +} \ +void HELPER(sve2_tbx_##SUFF)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + do_tbl1(vd, vn, vm, desc, true, do_tb_##SUFF); \ +} + +DO_TB(b, uint8_t, H1) +DO_TB(h, uint16_t, H2) +DO_TB(s, uint32_t, H4) +DO_TB(d, uint64_t, ) + +#undef DO_TB #define DO_UNPK(NAME, TYPED, TYPES, HD, HS) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index de8a6b2a15..63e79fafe5 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -2417,6 +2417,39 @@ static bool trans_TBL(DisasContext *s, arg_rrr_esz *a) return true; } +static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_4 * const fns[4] = { + gen_helper_sve2_tbl_b, gen_helper_sve2_tbl_h, + gen_helper_sve2_tbl_s, gen_helper_sve2_tbl_d + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, + (a->rn + 1) % 32, a->rm, 0); + } + return true; +} + +static bool trans_TBX(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_tbx_b, gen_helper_sve2_tbx_h, + gen_helper_sve2_tbx_s, gen_helper_sve2_tbx_d + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); + } + return true; +} + static bool trans_UNPK(DisasContext *s, arg_UNPK *a) { static gen_helper_gvec_2 * const fns[4][2] = { From patchwork Fri Apr 16 21:02:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422646 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp743754jaf; Fri, 16 Apr 2021 14:50:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz4PrJawaUnggrzvfyeYjsj+gmZJm7MtNeG7dO5ez9HeuNnvFKZF7gK6VwECKb5P1MwoH92 X-Received: by 2002:a25:c08b:: with SMTP id c133mr1778480ybf.67.1618609859234; Fri, 16 Apr 2021 14:50:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609859; cv=none; d=google.com; s=arc-20160816; b=cldP+siCNitriXymz1eHY+CrRM7GuIQotkpuqKcFtPuvQ5nsi7vwcjAA4rkneROq09 viLEXL6SqAjAfhgKlI1RDhETkGlp/l4pmPbBVutVaR38PHJBmChtJDQ/AHj95UItA4Va FBBRl2xnEwYC98vdaw0l0gxDugZUVlL3wvwX7TnuEckwh9k/glb7hQ5BVUK1s2iaqRhH 60eOU1dDHzrQ5od6Pqt8bzlQtO8HnupIJrX00ibBDwBlTF5oTR9rCxy/bLl6WRqd1TOP oyVazuguQEG1NicCakOhNxuod6O9IDTMSb8mGjMRqU1fgAg9DaNzR1fOZ+7CNtXq3Qbw w2DQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Qi7ydIxfQkB8AeeB+foEIc9co/N1EA+zUALSSBLM1Ow=; b=NjabaxcTGrqivnaoaynokLwB5NpmHRmmA1JakC8C4cn8XUpYgKx4xwl0m47YqxoBkb z0lQS3e6JGvm2mxDWuxcYevNdSPv1sqfI7f/Qr+sLwcZbxTUbK2Itm7BOPQgNvGPK+W4 oZit9v7JmZ+26zL3pgH6scn7T6vSp4LsKXWzpgv8/ulIRmDnXbsNFeb785228bbtd/p6 Vv2Qmhi/s1dLbQdRwiMqZ6RMDwES1TvvcQCFCWhECXh5VlLa6otxEskPfPy54tarfkkz y0rBWXz+c+iGch/1t0DTU9oFsHyMktVEaFUCtw3dy0IdOJ9tJI9eedmB2M1Gy8ewuawB Fp7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Bw1Eqvjv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t5si6939126ybo.273.2021.04.16.14.50.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:50:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Bw1Eqvjv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40688 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWMc-0004fO-Kr for patch@linaro.org; Fri, 16 Apr 2021 17:50:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45502) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdp-0006ZE-PW for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:41 -0400 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]:46909) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001kv-L2 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:41 -0400 Received: by mail-pl1-x630.google.com with SMTP id m18so12423922plc.13 for ; Fri, 16 Apr 2021 14:04:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Qi7ydIxfQkB8AeeB+foEIc9co/N1EA+zUALSSBLM1Ow=; b=Bw1EqvjvPJ0YtYg/G0keZJoxkkzLy+sSCmtZ+tv+NHKVw5af/oCY5L3AmiL+ELi2Vc TUREsVlMlifP+K8WxmhPN4cntub79ulGPXULNmsV06RdV31kGICU+UcIxc7S+7ZxP6xv YQNe0CS3DNdrHgUkC0AIgBHCO7A4PSUrTy2BeWuBe78I+kjNY3GuP9p8I5YTKEWD8Kcy FFiQ2J3iARWH435FHCuy0q+9ATGBOdtYItdcsqApQiD8gWr/J047ZB26QdGNSlueLyYi EGseMT3w69boMgulLmWtsCKmY+ru7sw1U7JH6CEHXJGez/OcSx8TqBjAq6hUxUd7u7m1 hwyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Qi7ydIxfQkB8AeeB+foEIc9co/N1EA+zUALSSBLM1Ow=; b=lGIms2+8a5RI6R4niYZblLzfIDhKS8QfdLPfRU4zs8GsbiVgMNTQf8qHBKVJwoCM/R PzhKgncx5t5DQOVjRhQHtzwcE6y7Ra/2rGSijX+sKwWPTcQEnuug9w5Rbr8ifARXpqJE AGNJde3YYIcuBynBzpnAxgzCBuw+zY/vj07TzkF5dG1Wa/gpTZ6Cq8beEftojjqtYGBd e5xD5NUQivo+7ZzI1Wv+iLCmCSnHVxByyQaGbWu18MkS0zT9nxw1btvCfALvnSRyaOmu oWrfS6oqgpY583fOzOk3FlfGbNZaM3vZ63ewoB1ni4XrzACv8xd7ZvounPY70Q2J79EL 3ukw== X-Gm-Message-State: AOAM533ESotFW5GVorbnGUycEb3uG5JyB/aUcvqSlINZrcD2j0sn4kP1 PzxdsN0yhfb0inxKgwkLViD6Cj8wz0z5Ng== X-Received: by 2002:a17:90b:4a52:: with SMTP id lb18mr11556556pjb.1.1618607055985; Fri, 16 Apr 2021 14:04:15 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:15 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 65/81] target/arm: Implement SVE2 FCVTNT Date: Fri, 16 Apr 2021 14:02:24 -0700 Message-Id: <20210416210240.1591291-66-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428174332.17162-2-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 4 ++++ target/arm/sve_helper.c | 20 ++++++++++++++++++++ target/arm/translate-sve.c | 16 ++++++++++++++++ 4 files changed, 45 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 41c08a963b..d6b064bdc9 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2703,3 +2703,8 @@ DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_fcvtnt_sh, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_fcvtnt_ds, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 38aaf1b37e..afc53639ac 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1531,3 +1531,7 @@ SM4E 01000101 00 10001 1 11100 0 ..... ..... @rdn_rm_e0 # SVE2 crypto constructive binary operations SM4EKEY 01000101 00 1 ..... 11110 0 ..... ..... @rd_rn_rm_e0 RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 + +### SVE2 floating-point convert precision odd elements +FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 +FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 8dc04441aa..6164ae17cc 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7448,3 +7448,23 @@ void HELPER(fmmla_d)(void *vd, void *vn, void *vm, void *va, d[3] = float64_add(a[3], float64_add(p0, p1, status), status); } } + +#define DO_FCVTNT(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ +{ \ + intptr_t i = simd_oprsz(desc); \ + uint64_t *g = vg; \ + do { \ + uint64_t pg = g[(i - 1) >> 6]; \ + do { \ + i -= sizeof(TYPEW); \ + if (likely((pg >> (i & 63)) & 1)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + *(TYPEN *)(vd + HN(i + sizeof(TYPEN))) = OP(nn, status); \ + } \ + } while (i & 63); \ + } while (i != 0); \ +} + +DO_FCVTNT(sve2_fcvtnt_sh, uint32_t, uint16_t, H1_4, H1_2, sve_f32_to_f16) +DO_FCVTNT(sve2_fcvtnt_ds, uint64_t, uint32_t, H1_4, H1_2, float64_to_float32) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 63e79fafe5..df52736e3b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8170,3 +8170,19 @@ static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a) } return true; } + +static bool trans_FCVTNT_sh(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_sh); +} + +static bool trans_FCVTNT_ds(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_ds); +} From patchwork Fri Apr 16 21:02:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422665 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp754190jaf; Fri, 16 Apr 2021 15:09:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwCyjSHpHqtWHhadqOQ+KBjKFI3MblB3giD3SLpZ+2I0SonrhQaj59iMaHiTH+TJXTYwle6 X-Received: by 2002:a67:790c:: with SMTP id u12mr9173857vsc.19.1618610988151; Fri, 16 Apr 2021 15:09:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610988; cv=none; d=google.com; s=arc-20160816; b=TJfkBjZwq0m+dAxRkR3CS0dE14crOqbTmlwdbX4zy6nPlRlVfB0/sV6z4jMo7VHMDs /d1bS/E5FTYHh5Hvj2E3w0TOGU6GMZrZde89i/h18gKyLg3jphxokSp3h0leB4abWodU yXtSKwXYgNzeGPgV6PSc1eY31CJp0HRfLJzN2mdwv1L3aFEV+yGOcATsQPrinO6NWxr/ f15V0h7upJIYPJKOOyMX4kPNhtScUMcv1rXjDf3545RKSq+sZLHiv+uILgq4webJEqya o0YT8wgj8dOp1Q5zS99fGhRrzwIyNnxD7kAtPmWTWJpKBCTQSTy5uyi0ktXDuF9HJEv2 11Gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=TefYznISTQm0pzCtVV51Stsil3QGM04wAsqxqTtOHyw=; b=quya9WCB1G/IEuu7hISNLH86XJ9DYcBzRSBMfMZK7ejrvWMZz2c/XhajV7NIZyvhVJ 5Qb876w8w1R2m+W0TyY4kNgfsi2yTSet77wXJen8QriblE0tefwJP6orhC89lrkRdiPt OoDS0ouqHrB6CegoxCNijRjWp5C0/CA4kylYAfwkPRDiIe0HFHOX3SVo0TwiIR3puKWn /fnW2OlEQ8k0T1nyigoEaO4Z+3YNPGd4RzJ38qx9gbAQP1h/fX0lCHYwlANO+nsOUoMh 0dy8VX+Vvu1X25R5H1Z9jUV2lMaLsYbt5X5Gqn9OrcpjRPETbDOMyjkAOfvb6nk2bGAt GZzg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=eG4pzepR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k22si648037vkp.15.2021.04.16.15.09.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:09:48 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=eG4pzepR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43898 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWep-0004Tk-Gt for patch@linaro.org; Fri, 16 Apr 2021 18:09:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45324) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdi-0006Fv-Ha for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:34 -0400 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]:43543) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001lj-FN for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:34 -0400 Received: by mail-pl1-x62f.google.com with SMTP id u15so6084764plf.10 for ; Fri, 16 Apr 2021 14:04:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TefYznISTQm0pzCtVV51Stsil3QGM04wAsqxqTtOHyw=; b=eG4pzepRDHVzSW5X9CaoqwAj58daoondH8PQJAwJaGYMskbG6QR7x3fL3LCSC5BXBJ f8kGzQJQWA6i+mauQLcvVrt16AKfIFl/+SZCymaWuJ4S8RL6ZQp9oOv1p1dlgHQOUZ5G 3KNEHFej7yUg8unc6YO4EMhGHVDzfLo+oN7htcSRenzudH6xP6kZEVAPRafCnSlMQlal VbBV36nbggRoJ7FdERd9ZV73tX+hcCnyaJzB5zl5wj+lfjtPw+Hf68WlCKiG+rxE1kdp fgmq674OI85+C1Q72ntN+BW8o9JHh4k0sWzanVoT7ucCy23FPUBaHKHJDeWAS256AO8s yJ3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TefYznISTQm0pzCtVV51Stsil3QGM04wAsqxqTtOHyw=; b=nTdLbXAik5MDOBHg8XFbBiUszaX5dg3/8T5/v4L337vtnb4sqR/yC+y78LUREMBUFG 62BW1PpbOyqjDePr7T/A6pnJD+2vq6gsrjsl1pwd4GSU2RolBxt73KEw+AzBZ6IvgB6X 6ngxr/uZn1PrqSoPm/EatfMPI9l9yZm2LfyblMlU/vgGw4CORRejydn3L3nRF9WZuNnF oa45IAP03kZS309C23cPeKBWB4CM8lJyv5R3wmotYh2nUnv1trY2mur/zaJN+l3Oq17z U99zUtzQjLTOIsljc0Y2DZoLRhbQniyFkdQ039MUWGMKyG8UStgW/0BuLDgjK8y/u/6t CtZQ== X-Gm-Message-State: AOAM5319QQIAKgvrT0SzqW5s9xCJL8XLSAPH5DlkoPW/nHcFgI3SNc98 zPuTyNZypZ0hMoKyLrtTV2d7XzqlyHcBdw== X-Received: by 2002:a17:90a:288:: with SMTP id w8mr11793103pja.163.1618607057008; Fri, 16 Apr 2021 14:04:17 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:16 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 66/81] target/arm: Implement SVE2 FCVTLT Date: Fri, 16 Apr 2021 14:02:25 -0700 Message-Id: <20210416210240.1591291-67-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62f; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428174332.17162-3-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 23 +++++++++++++++++++++++ target/arm/translate-sve.c | 16 ++++++++++++++++ 4 files changed, 46 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index d6b064bdc9..30b6dc49c8 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2708,3 +2708,8 @@ DEF_HELPER_FLAGS_5(sve2_fcvtnt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_fcvtnt_ds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_fcvtlt_hs, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_fcvtlt_sd, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index afc53639ac..fb998f5f34 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1534,4 +1534,6 @@ RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 ### SVE2 floating-point convert precision odd elements FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 +FCVTLT_hs 01100100 10 0010 01 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 +FCVTLT_sd 01100100 11 0010 11 101 ... ..... ..... @rd_pg_rn_e0 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 6164ae17cc..2684f40a62 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7468,3 +7468,26 @@ void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ DO_FCVTNT(sve2_fcvtnt_sh, uint32_t, uint16_t, H1_4, H1_2, sve_f32_to_f16) DO_FCVTNT(sve2_fcvtnt_ds, uint64_t, uint32_t, H1_4, H1_2, float64_to_float32) + +#define DO_FCVTLT(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ +{ \ + intptr_t i = simd_oprsz(desc); \ + uint64_t *g = vg; \ + do { \ + uint64_t pg = g[(i - 1) >> 6]; \ + do { \ + i -= sizeof(TYPEW); \ + if (likely((pg >> (i & 63)) & 1)) { \ + TYPEN nn = *(TYPEN *)(vn + HN(i + sizeof(TYPEN))); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, status); \ + } \ + } while (i & 63); \ + } while (i != 0); \ +} + +DO_FCVTLT(sve2_fcvtlt_hs, uint32_t, uint16_t, H1_4, H1_2, sve_f16_to_f32) +DO_FCVTLT(sve2_fcvtlt_sd, uint64_t, uint32_t, H1_4, H1_2, float32_to_float64) + +#undef DO_FCVTLT +#undef DO_FCVTNT diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index df52736e3b..9cad93cb98 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8186,3 +8186,19 @@ static bool trans_FCVTNT_ds(DisasContext *s, arg_rpr_esz *a) } return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_ds); } + +static bool trans_FCVTLT_hs(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_hs); +} + +static bool trans_FCVTLT_sd(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_sd); +} From patchwork Fri Apr 16 21:02:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422669 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp758351jaf; Fri, 16 Apr 2021 15:16:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzSvYDuH/tUEPVz6NDyXlDdadbt5l/sdzdR2hGhHl96HQkgjZG+HVbQek3xY97KHTFr69sS X-Received: by 2002:a05:6638:258e:: with SMTP id s14mr6145487jat.28.1618611394619; Fri, 16 Apr 2021 15:16:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618611394; cv=none; d=google.com; s=arc-20160816; b=TIsr/Jif52j2HINntcQUf+squ7n1KhCbS8PL1/kwUbTHFhmk1Rozl9Px2lTNtmBtKq xPOqv7faDGM8T9n3JqwL3/5Nr4HThtQLbH/HEgAVkRQ/dZv5k/8tQi5fGoE7iFoME5df dffsrpHwh9f/U+Q33rWPUEmehbbBTIUrXPikPDv8lObFJmOC44qyupvCPs84F5CD0LUG Uajknz2VwTdlXC5D+D19G9NTGhU4q0o9JKcJgdTBPQbT2oN+A63kMxA1nNzofgNa8hOa 3tJ1LvX9Pud/eoKjoo7KXDNy7rRhevhNFfveatfKdI2NhuCzvuDnJ1gnXn/f5hT0ig/D C4bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=T7nNbpS6ns3g2zobopRt+7U4FUmG0OZq/G9VxwXU/Zw=; b=J6jspjdUvcscon1iYEW2E6X/9U9dSUf0MOlrM0zbjvmorYKvedHVSKdUPy17k+1BMe yQV7krXpUTgwmLXpFiN/iGMVhbJhxndFdh8qHRNuMCJbr8JzEk3YaCQdKkPepn1DPTi8 GJcdVkSc+rl0sb4du6/NCC2PTiUVWUsx0eYi5zJlZ/VZbIK0GHy8Xv9xZrkrvc6AoUe6 FIaI9QTAkZWu2WRY6pWzuy/E+dS39N+ztb6xUPkrRyv13xfyc9x54u3gy8Iy7yY6XxOj bCjOsDvazxoYNB1lcRHYz8OcM6YwYRrRSoRTji/LNQo6KYFn8e+7GPJy9/CGZ2tuUYgS Et8A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rfNu+dMV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t19si3722764jam.106.2021.04.16.15.16.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:16:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rfNu+dMV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37168 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWlO-00052x-4B for patch@linaro.org; Fri, 16 Apr 2021 18:16:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45432) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdm-0006Rj-U5 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:38 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]:35442) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001lt-GO for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:38 -0400 Received: by mail-pl1-x62c.google.com with SMTP id q11so1533196plx.2 for ; Fri, 16 Apr 2021 14:04:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=T7nNbpS6ns3g2zobopRt+7U4FUmG0OZq/G9VxwXU/Zw=; b=rfNu+dMV1j0nPvjskRRCyQ8nmXth8IiyW2qUy9Ar4e/AoGB2YpQpDV8UCPF8o8qtxU j2YqsCwPrIp9XiBEopnpratnY1Xs4qCIiIN2MHsr7gpSweimOfBStJr1dkBmMctAgY3W AiqPyx6XCG7WHaNroeBQ3ltLvKeB/ukycFpUzJzKSuPC8XJ0H/Z7W2IXoSppTxAIH5kd YYvvQKytuMJs4xBNRZ4EhqbQ5Q4r9GJdkRiTm4fQFt1Ml1tzkHV8kSNlrxNG8cnbbwFh c5TdOanjReKKBJOSDX0EBnmtAzSuC59gUoNigRTrJzh15g8DTV/HFfA1mwvVdinmytx7 Bw9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=T7nNbpS6ns3g2zobopRt+7U4FUmG0OZq/G9VxwXU/Zw=; b=Pj8YjZdSGnOFrxd8QOCYKlronjOHwCFG1I2ZD5XMS2NfmklKtPiW4YVqXuAKX6Ph6q +rIIyY1ANJPN0M53LPdd6Etm6mPZMSXrz6Qc34d6HcaoXB354c9uU2Hr+Axw4uxCiiBx 2Wh9FFGLvvoJfx0PG4KMjk7mMXxQQS045pK5lmRApcJJ28ZKp1v3JdDCkqCrLLEiUo01 rsQDWIxeoQDIf2xQAQLMlysr556etHdXxxVzMDXgxNuiaKgsi5rjlzr90yxWF9V8M5K2 RSFwYO6YwAKfWAdDEOxNZdK35NkI84gGrPwXUGTV94qzUl3wAU8Ck5zO9Hhq7ZmUim4R a1Ag== X-Gm-Message-State: AOAM530XyZuFbALAWbz45vZol2xdmefXos05ZSOysmZXi0lvaVYBYlTA kt6VHcZ+vmHrxs43rQTcimt1x+++Gqa8Dw== X-Received: by 2002:a17:90b:390f:: with SMTP id ob15mr11485296pjb.100.1618607058029; Fri, 16 Apr 2021 14:04:18 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:17 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 67/81] target/arm: Implement SVE2 FCVTXNT, FCVTX Date: Fri, 16 Apr 2021 14:02:26 -0700 Message-Id: <20210416210240.1591291-68-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428174332.17162-4-steplong@quicinc.com> [rth: Use do_frint_mode, which avoids a specific runtime helper.] Signed-off-by: Richard Henderson --- target/arm/sve.decode | 2 ++ target/arm/translate-sve.c | 49 ++++++++++++++++++++++++++++++-------- 2 files changed, 41 insertions(+), 10 deletions(-) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index fb998f5f34..46153d6a84 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1533,6 +1533,8 @@ SM4EKEY 01000101 00 1 ..... 11110 0 ..... ..... @rd_rn_rm_e0 RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 ### SVE2 floating-point convert precision odd elements +FCVTXNT_ds 01100100 00 0010 10 101 ... ..... ..... @rd_pg_rn_e0 +FCVTX_ds 01100101 00 0010 10 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 FCVTLT_hs 01100100 10 0010 01 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 9cad93cb98..5b78298777 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4715,11 +4715,9 @@ static bool trans_FRINTX(DisasContext *s, arg_rpr_esz *a) return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, fns[a->esz - 1]); } -static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode) +static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, + int mode, gen_helper_gvec_3_ptr *fn) { - if (a->esz == 0) { - return false; - } if (sve_access_check(s)) { unsigned vsz = vec_full_reg_size(s); TCGv_i32 tmode = tcg_const_i32(mode); @@ -4730,7 +4728,7 @@ static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode) tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), vec_full_reg_offset(s, a->rn), pred_full_reg_offset(s, a->pg), - status, vsz, vsz, 0, frint_fns[a->esz - 1]); + status, vsz, vsz, 0, fn); gen_helper_set_rmode(tmode, tmode, status); tcg_temp_free_i32(tmode); @@ -4741,27 +4739,42 @@ static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode) static bool trans_FRINTN(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_nearest_even); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz - 1]); } static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_up); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_up, frint_fns[a->esz - 1]); } static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_down); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_down, frint_fns[a->esz - 1]); } static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_to_zero); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz - 1]); } static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_ties_away); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz - 1]); } static bool trans_FRECPX(DisasContext *s, arg_rpr_esz *a) @@ -8202,3 +8215,19 @@ static bool trans_FCVTLT_sd(DisasContext *s, arg_rpr_esz *a) } return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_sd); } + +static bool trans_FCVTX_ds(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve_fcvt_ds); +} + +static bool trans_FCVTXNT_ds(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve2_fcvtnt_ds); +} From patchwork Fri Apr 16 21:02:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422668 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp757399jaf; Fri, 16 Apr 2021 15:14:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyXWQTT7gxBCQYHpvqlxfZqdwlym56FU4Ys49//IVNn/aMKMgasxWSycpap8oh3s8J9Y4Fq X-Received: by 2002:a05:6638:371e:: with SMTP id k30mr6101628jav.4.1618611298589; Fri, 16 Apr 2021 15:14:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618611298; cv=none; d=google.com; s=arc-20160816; b=Hrg1eWShbuyjNj2tc70B+s+khi268MCSQ4FonfDt1G4xcr43QHWkhReUdGm3w7tAVi AiiKgWColXLkDC0imbYCPCs8vU+NRU50VHOA7nUljG4AfuBuh05Gf6KNOPz3OtZ/kJpZ matllo4mpDoUyqzewcMc9ISDLmAt5dofNPNps1wTh+6cdYPCAbTT6G1ZxJEgxrEBTDid KOBvUcBYhV5aYomjd1eiKXTQymykk0mcJzDOcEt0nZPADAveA9ZH7FzUnaF90FDibrij ay/g3kOP8jdpTNRwTYc137A9oFzqInVA2QY7MXWemqn7rRLcrfbDxnUK+oZzG7O7BO0E Y88Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=qR0OQ59R49nnoUzy7pA6/SiglCGIjRHKKV8eiOZlkXE=; b=d/N4qpUL1yEiJaIWtJ7XPWu/pQD8X1qsoYurumd7a1lms4/BMmP+Wet3h2t+a3RQ7/ 2LuQZwsjV/MiMDs17Uoy3FsqGM6Lx9W2jfdGO795XA0RcOelc96l+n1B3FxPf9OGjLj9 Ex66urlZsWvMfNBMAx2UJEfW0JDkz+AvB0npzwKj1pdXiEWZQ0clEvtYhKt8qbl+sprn Vrf8R9aCmUBVb1/pkP2enWFB3hb/AZyydZG5BfFNZQtu4V29s8xazy1WeMAspFXhE0aJ HuGydYCk6vg7ZlTenvb5JeDeT7JkXAcSpxMiJpqpnKxmvqFeQCQnfURSWXxdReyA6dht HLCQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cIYXI12v; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e2si8711702ioc.23.2021.04.16.15.14.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:14:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cIYXI12v; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60624 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWjq-0002xq-2H for patch@linaro.org; Fri, 16 Apr 2021 18:14:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45400) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdl-0006Oj-Na for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:37 -0400 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]:43933) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001m3-I9 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:37 -0400 Received: by mail-pj1-x102a.google.com with SMTP id x21-20020a17090a5315b029012c4a622e4aso15261742pjh.2 for ; Fri, 16 Apr 2021 14:04:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qR0OQ59R49nnoUzy7pA6/SiglCGIjRHKKV8eiOZlkXE=; b=cIYXI12vVi39ByHfWisobA8+8MDkNomudNmJBnCK0iWJvNR1TjkMFyliUYHY75yBYF C47FayMc0REbR44t1oXEUPWAKOudcZoXRU9qkFXDm2pKlrmgloPqk0q7rtjBXzGopdjf So7y7i3kzzOhp2kFo5v+ncLrDX2XdzAho9AWwNRSIwZk57HCSriOCiNaepyjuIjhYx5C KOKzJPm3/zWPKO0PxI4TQdlfnRMheiU1v1W6uMax0CKa+W83EoDv+LwDWPGFB9BgAwyW KnAYWnFhlP9dzxBWteC1vBuVXq8cOJglZ1XeQBgI57MFo2er/A4BG7bvqanBxqC1zHrY hzBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qR0OQ59R49nnoUzy7pA6/SiglCGIjRHKKV8eiOZlkXE=; b=aTVfKrxvBnwsaoZDEUkjdeRwv0riXs4gb3Nzourb9mC1V5vDk350xnhs1hYjMBk5nX Ge1o5WG7c9RGP96LEkC87IzW/hzwLQFdyZpA2zrZYNw7O1tcqAgk/2HhoX6W/1AWE+24 mM9prhdKjpDbC6U9ZQCBKvvboqaz4oYPhAbfq7jTM91oOx8fElBnPL0qGtHlttRDH3rx QkS9hPWuBoGphMfLPYwATZL6Z4q73m/pJB3m81tSVSTZHB5Jh1aCVamNCS6hfg/RqPFq nT6cBP0JJM2l3DKl1NZlmxAxjtppA18lJQMx/Q/woRLMNXzVYmygPqirsV+K1FNeQGU3 x/Ig== X-Gm-Message-State: AOAM531AZoTvQ6gSNPFokv/BL1myZbOL/GNjKqYY65k9NpcfhnTnhcOB IK630Im1o95Vq4UA2+sH8Jp99buzFJB5pA== X-Received: by 2002:a17:90b:4c47:: with SMTP id np7mr11281761pjb.26.1618607058907; Fri, 16 Apr 2021 14:04:18 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:18 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 68/81] target/arm: Implement SVE2 FLOGB Date: Fri, 16 Apr 2021 14:02:27 -0700 Message-Id: <20210416210240.1591291-69-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200430191405.21641-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fixed esz index and c++ comments v3: Fixed denormal arithmetic and raise invalid. --- target/arm/helper-sve.h | 4 +++ target/arm/sve.decode | 3 +++ target/arm/sve_helper.c | 52 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 24 ++++++++++++++++++ 4 files changed, 83 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 30b6dc49c8..96bd200e73 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2713,3 +2713,7 @@ DEF_HELPER_FLAGS_5(sve2_fcvtlt_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_fcvtlt_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(flogb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(flogb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(flogb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 46153d6a84..17adb393ff 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1539,3 +1539,6 @@ FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 FCVTLT_hs 01100100 10 0010 01 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 FCVTLT_sd 01100100 11 0010 11 101 ... ..... ..... @rd_pg_rn_e0 + +### SVE2 floating-point convert to integer +FLOGB 01100101 00 011 esz:2 0101 pg:3 rn:5 rd:5 &rpr_esz diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 2684f40a62..754301a3a6 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -4575,6 +4575,58 @@ DO_ZPZ_FP(sve_ucvt_dh, uint64_t, , uint64_to_float16) DO_ZPZ_FP(sve_ucvt_ds, uint64_t, , uint64_to_float32) DO_ZPZ_FP(sve_ucvt_dd, uint64_t, , uint64_to_float64) +static int16_t do_float16_logb_as_int(float16 a, float_status *s) +{ + if (float16_is_normal(a)) { + return extract16(a, 10, 5) - 15; + } else if (float16_is_infinity(a)) { + return INT16_MAX; + } else if (float16_is_any_nan(a) || float16_is_zero(a)) { + float_raise(float_flag_invalid, s); + return INT16_MIN; + } else { + /* + * denormal: bias - fractional_zeros + * = bias + masked_zeros - uint32_zeros + */ + return -15 + 22 - clz32(extract16(a, 0, 10)); + } +} + +static int32_t do_float32_logb_as_int(float32 a, float_status *s) +{ + if (float32_is_normal(a)) { + return extract32(a, 23, 8) - 127; + } else if (float32_is_infinity(a)) { + return INT32_MAX; + } else if (float32_is_any_nan(a) || float32_is_zero(a)) { + float_raise(float_flag_invalid, s); + return INT32_MIN; + } else { + /* denormal (see above) */ + return -127 + 9 - clz32(extract32(a, 0, 23)); + } +} + +static int64_t do_float64_logb_as_int(float64 a, float_status *s) +{ + if (float64_is_normal(a)) { + return extract64(a, 52, 11) - 1023; + } else if (float64_is_infinity(a)) { + return INT64_MAX; + } else if (float64_is_any_nan(a) || float64_is_zero(a)) { + float_raise(float_flag_invalid, s); + return INT64_MIN; + } else { + /* denormal (see above) */ + return -1023 + 12 - clz64(extract64(a, 0, 52)); + } +} + +DO_ZPZ_FP(flogb_h, float16, H1_2, do_float16_logb_as_int) +DO_ZPZ_FP(flogb_s, float32, H1_4, do_float32_logb_as_int) +DO_ZPZ_FP(flogb_d, float64, , do_float64_logb_as_int) + #undef DO_ZPZ_FP static void do_fmla_zpzzz_h(void *vd, void *vn, void *vm, void *va, void *vg, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 5b78298777..fe8f87d55e 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8231,3 +8231,27 @@ static bool trans_FCVTXNT_ds(DisasContext *s, arg_rpr_esz *a) } return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve2_fcvtnt_ds); } + +static bool trans_FLOGB(DisasContext *s, arg_rpr_esz *a) +{ + static gen_helper_gvec_3_ptr * const fns[] = { + NULL, gen_helper_flogb_h, + gen_helper_flogb_s, gen_helper_flogb_d + }; + + if (!dc_isar_feature(aa64_sve2, s) || fns[a->esz] == NULL) { + return false; + } + if (sve_access_check(s)) { + TCGv_ptr status = + fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + unsigned vsz = vec_full_reg_size(s); + + tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + pred_full_reg_offset(s, a->pg), + status, vsz, vsz, 0, fns[a->esz]); + tcg_temp_free_ptr(status); + } + return true; +} From patchwork Fri Apr 16 21:02:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422657 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp747344jaf; Fri, 16 Apr 2021 14:58:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxnQq+nGiyhRY6jWqsQ2WhTMSp+djarPKRNAApurz7lAmzy0et6mZKfioRpTUMwH5XL2k49 X-Received: by 2002:a25:ae0f:: with SMTP id a15mr1657232ybj.294.1618610330900; Fri, 16 Apr 2021 14:58:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610330; cv=none; d=google.com; s=arc-20160816; b=Bvp3jVRuT5RXaf6sFrekXFAu+2DbeCnoNlAWuXh/XvN6Z+dfEjeQf25qKTwdF+ogDv wLhcojGTt2XW1c4jbBD3TWWB7uiufXzSZW5KJG+lYk9o+7eSkQbFeN/u7PMaQSuuAAQX D2kYMWVJ0jshMQvum18nX7jvi2KqSndv3WD3tlrAmsy0iP+5AFCiXj4v6hiHVMP3cHz1 xATIkzCFCjnqzcXLoAh/yOBRQcuTq7BSJZaklQm6+QNy1OQY8nx0uoeFoCYH9FsMZYuF M0YpjHb5voDymDRtrReTczqr5dlIlApa9xg9YbDTaB55bGOK1uqXVrX8d6Qw8OFIX26X ri4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Nzy1xDOFf+uPUAwqPevG83MmScnqopiQ3mxZt4fgtoc=; b=US4m93JexjY6fCXR560v1+Yh6nzFLy1q6SrijiQC0/627dD17N4l7Fv5yhdVDd93jQ oRKbNJc1jmG/hKqp8vrGhnAZs2/+4ju5HwrmXXyohauldPy0+w8XNo46k0BPaEsggmvY n//Jj1pZ7c2Ho8lXuORmCul9ir/uZfObZ+B19U/p7JECkOyjdxzKp38SiDmAkvbtc98a AcA2TqfWu2XX+xN54cFSS9fP1TcjCUwO2OwIWYYDoecVCwIkFLQCBagSlqe6XxfnGf5p Alc4pcjTsv/72YcdY2aYJ+L8C8aH6fN/HOdZBoopoQmCnzlieJhNQFCp17VZKAQK6kmx xbig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Uop53OCD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 207si6803506ybn.268.2021.04.16.14.58.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:58:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Uop53OCD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40030 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWUE-0007QX-B0 for patch@linaro.org; Fri, 16 Apr 2021 17:58:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45450) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdn-0006Tc-JU for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:39 -0400 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]:34721) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001mC-Ik for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:39 -0400 Received: by mail-pj1-x102a.google.com with SMTP id em21-20020a17090b0155b029014e204a81e6so6951510pjb.1 for ; Fri, 16 Apr 2021 14:04:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Nzy1xDOFf+uPUAwqPevG83MmScnqopiQ3mxZt4fgtoc=; b=Uop53OCD+0b1isatRsOdXELnfVCwQ6EqPY3INyfFGOlml6oe04blH2JtsR3etJJrUt x/+DyynsvjYQXrU02/7k2ecMtG28XbMO1O+4+0ExCxFYN/6HHHxzE+od3XzN0FX5q1lk V3+0cZ4sktQd+qtKhI7WBAH+lhQPmc80aLT0NZECq14II7ia3V8OAhyL4CQfvAOVznvX dpC3ewimsKn9blQVw8KOtDcLmE/Ud3StjpbVZaNZxEVyaDnReMZAJzOubgAqqBKRSi/W pVKaZrdOZCkdLyyJIhNMI2puUVNRtW9IKGRkLfQoAtO62tvy/iQnkdmkFkHGaQjmLf4P Wbhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Nzy1xDOFf+uPUAwqPevG83MmScnqopiQ3mxZt4fgtoc=; b=OLm7mW1o4dHNEj+/hhvg1lIKxWcjDbCKKyLII1IkthmqqGphwuclry+XtpBPlKSHhT ZmR6aGSTSA/ik28AtJYXjRPCfXyXaY+sjYiHb6PDHKK958tlwD6W2afS0O0uGuCs1dhl PVAAv0cIa6Z/nVWWJYAjvi0GONrRGBExwDze8G28mgYUdohAy7/291O0ijC6VS74VaNq BgOD0W3NeudSuCKqPAT8dYvIeHXyJGRYAx4XgVkoW9NPtngNXf/YWSn4ed9gF+EKAJhx Y1mLLks2Qi7SNLQZS5vFAHSdyY1Or2fKR5KROdHzGK+3rM89AzkJjfb+YGoOLKqBIJQC 4Zgg== X-Gm-Message-State: AOAM533WCi0GPdQPa8Tku5qAsq5xOc5wKumQr9/AzyoLPdCxDtOoWokH cP07yJwIp9ibv0MtzcHCpVNE0gsIThRVig== X-Received: by 2002:a17:902:7788:b029:e9:11:5334 with SMTP id o8-20020a1709027788b02900e900115334mr11377757pll.70.1618607059628; Fri, 16 Apr 2021 14:04:19 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:19 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 69/81] target/arm: Share table of sve load functions Date: Fri, 16 Apr 2021 14:02:28 -0700 Message-Id: <20210416210240.1591291-70-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The table used by do_ldrq is a subset of the table used by do_ld_zpa; we can share them by passing dtype instead of msz to do_ldrq. Signed-off-by: Richard Henderson --- target/arm/translate-sve.c | 254 ++++++++++++++++++------------------- 1 file changed, 126 insertions(+), 128 deletions(-) -- 2.25.1 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index fe8f87d55e..04efa037f2 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5153,128 +5153,130 @@ static void do_mem_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, tcg_temp_free_i32(t_desc); } +/* Indexed by [mte][be][dtype][nreg] */ +static gen_helper_gvec_mem * const ldr_fns[2][2][16][4] = { + { /* mte inactive, little-endian */ + { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, + gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, + { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_le_r, gen_helper_sve_ld2hh_le_r, + gen_helper_sve_ld3hh_le_r, gen_helper_sve_ld4hh_le_r }, + { gen_helper_sve_ld1hsu_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_le_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld2ss_le_r, + gen_helper_sve_ld3ss_le_r, gen_helper_sve_ld4ss_le_r }, + { gen_helper_sve_ld1sdu_le_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_le_r, gen_helper_sve_ld2dd_le_r, + gen_helper_sve_ld3dd_le_r, gen_helper_sve_ld4dd_le_r } }, + + /* mte inactive, big-endian */ + { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, + gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, + { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_be_r, gen_helper_sve_ld2hh_be_r, + gen_helper_sve_ld3hh_be_r, gen_helper_sve_ld4hh_be_r }, + { gen_helper_sve_ld1hsu_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_be_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld2ss_be_r, + gen_helper_sve_ld3ss_be_r, gen_helper_sve_ld4ss_be_r }, + { gen_helper_sve_ld1sdu_be_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_be_r, gen_helper_sve_ld2dd_be_r, + gen_helper_sve_ld3dd_be_r, gen_helper_sve_ld4dd_be_r } } }, + + { /* mte active, little-endian */ + { { gen_helper_sve_ld1bb_r_mte, + gen_helper_sve_ld2bb_r_mte, + gen_helper_sve_ld3bb_r_mte, + gen_helper_sve_ld4bb_r_mte }, + { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_le_r_mte, + gen_helper_sve_ld2hh_le_r_mte, + gen_helper_sve_ld3hh_le_r_mte, + gen_helper_sve_ld4hh_le_r_mte }, + { gen_helper_sve_ld1hsu_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_le_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_le_r_mte, + gen_helper_sve_ld2ss_le_r_mte, + gen_helper_sve_ld3ss_le_r_mte, + gen_helper_sve_ld4ss_le_r_mte }, + { gen_helper_sve_ld1sdu_le_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_le_r_mte, + gen_helper_sve_ld2dd_le_r_mte, + gen_helper_sve_ld3dd_le_r_mte, + gen_helper_sve_ld4dd_le_r_mte } }, + + /* mte active, big-endian */ + { { gen_helper_sve_ld1bb_r_mte, + gen_helper_sve_ld2bb_r_mte, + gen_helper_sve_ld3bb_r_mte, + gen_helper_sve_ld4bb_r_mte }, + { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_be_r_mte, + gen_helper_sve_ld2hh_be_r_mte, + gen_helper_sve_ld3hh_be_r_mte, + gen_helper_sve_ld4hh_be_r_mte }, + { gen_helper_sve_ld1hsu_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_be_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_be_r_mte, + gen_helper_sve_ld2ss_be_r_mte, + gen_helper_sve_ld3ss_be_r_mte, + gen_helper_sve_ld4ss_be_r_mte }, + { gen_helper_sve_ld1sdu_be_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_be_r_mte, + gen_helper_sve_ld2dd_be_r_mte, + gen_helper_sve_ld3dd_be_r_mte, + gen_helper_sve_ld4dd_be_r_mte } } }, +}; + static void do_ld_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype, int nreg) { - static gen_helper_gvec_mem * const fns[2][2][16][4] = { - { /* mte inactive, little-endian */ - { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, - gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, - { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_le_r, gen_helper_sve_ld2hh_le_r, - gen_helper_sve_ld3hh_le_r, gen_helper_sve_ld4hh_le_r }, - { gen_helper_sve_ld1hsu_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_le_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld2ss_le_r, - gen_helper_sve_ld3ss_le_r, gen_helper_sve_ld4ss_le_r }, - { gen_helper_sve_ld1sdu_le_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_le_r, gen_helper_sve_ld2dd_le_r, - gen_helper_sve_ld3dd_le_r, gen_helper_sve_ld4dd_le_r } }, - - /* mte inactive, big-endian */ - { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, - gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, - { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_be_r, gen_helper_sve_ld2hh_be_r, - gen_helper_sve_ld3hh_be_r, gen_helper_sve_ld4hh_be_r }, - { gen_helper_sve_ld1hsu_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_be_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld2ss_be_r, - gen_helper_sve_ld3ss_be_r, gen_helper_sve_ld4ss_be_r }, - { gen_helper_sve_ld1sdu_be_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_be_r, gen_helper_sve_ld2dd_be_r, - gen_helper_sve_ld3dd_be_r, gen_helper_sve_ld4dd_be_r } } }, - - { /* mte active, little-endian */ - { { gen_helper_sve_ld1bb_r_mte, - gen_helper_sve_ld2bb_r_mte, - gen_helper_sve_ld3bb_r_mte, - gen_helper_sve_ld4bb_r_mte }, - { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_le_r_mte, - gen_helper_sve_ld2hh_le_r_mte, - gen_helper_sve_ld3hh_le_r_mte, - gen_helper_sve_ld4hh_le_r_mte }, - { gen_helper_sve_ld1hsu_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_le_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_le_r_mte, - gen_helper_sve_ld2ss_le_r_mte, - gen_helper_sve_ld3ss_le_r_mte, - gen_helper_sve_ld4ss_le_r_mte }, - { gen_helper_sve_ld1sdu_le_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_le_r_mte, - gen_helper_sve_ld2dd_le_r_mte, - gen_helper_sve_ld3dd_le_r_mte, - gen_helper_sve_ld4dd_le_r_mte } }, - - /* mte active, big-endian */ - { { gen_helper_sve_ld1bb_r_mte, - gen_helper_sve_ld2bb_r_mte, - gen_helper_sve_ld3bb_r_mte, - gen_helper_sve_ld4bb_r_mte }, - { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_be_r_mte, - gen_helper_sve_ld2hh_be_r_mte, - gen_helper_sve_ld3hh_be_r_mte, - gen_helper_sve_ld4hh_be_r_mte }, - { gen_helper_sve_ld1hsu_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_be_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_be_r_mte, - gen_helper_sve_ld2ss_be_r_mte, - gen_helper_sve_ld3ss_be_r_mte, - gen_helper_sve_ld4ss_be_r_mte }, - { gen_helper_sve_ld1sdu_be_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_be_r_mte, - gen_helper_sve_ld2dd_be_r_mte, - gen_helper_sve_ld3dd_be_r_mte, - gen_helper_sve_ld4dd_be_r_mte } } }, - }; gen_helper_gvec_mem *fn - = fns[s->mte_active[0]][s->be_data == MO_BE][dtype][nreg]; + = ldr_fns[s->mte_active[0]][s->be_data == MO_BE][dtype][nreg]; /* * While there are holes in the table, they are not @@ -5512,14 +5514,8 @@ static bool trans_LDNF1_zpri(DisasContext *s, arg_rpri_load *a) return true; } -static void do_ldrq(DisasContext *s, int zt, int pg, TCGv_i64 addr, int msz) +static void do_ldrq(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype) { - static gen_helper_gvec_mem * const fns[2][4] = { - { gen_helper_sve_ld1bb_r, gen_helper_sve_ld1hh_le_r, - gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld1dd_le_r }, - { gen_helper_sve_ld1bb_r, gen_helper_sve_ld1hh_be_r, - gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld1dd_be_r }, - }; unsigned vsz = vec_full_reg_size(s); TCGv_ptr t_pg; TCGv_i32 t_desc; @@ -5551,7 +5547,9 @@ static void do_ldrq(DisasContext *s, int zt, int pg, TCGv_i64 addr, int msz) t_pg = tcg_temp_new_ptr(); tcg_gen_addi_ptr(t_pg, cpu_env, poff); - fns[s->be_data == MO_BE][msz](cpu_env, t_pg, addr, t_desc); + gen_helper_gvec_mem *fn + = ldr_fns[s->mte_active[0]][s->be_data == MO_BE][dtype][0]; + fn(cpu_env, t_pg, addr, t_desc); tcg_temp_free_ptr(t_pg); tcg_temp_free_i32(t_desc); @@ -5573,7 +5571,7 @@ static bool trans_LD1RQ_zprr(DisasContext *s, arg_rprr_load *a) TCGv_i64 addr = new_tmp_a64(s); tcg_gen_shli_i64(addr, cpu_reg(s, a->rm), msz); tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, a->rn)); - do_ldrq(s, a->rd, a->pg, addr, msz); + do_ldrq(s, a->rd, a->pg, addr, a->dtype); } return true; } @@ -5583,7 +5581,7 @@ static bool trans_LD1RQ_zpri(DisasContext *s, arg_rpri_load *a) if (sve_access_check(s)) { TCGv_i64 addr = new_tmp_a64(s); tcg_gen_addi_i64(addr, cpu_reg_sp(s, a->rn), a->imm * 16); - do_ldrq(s, a->rd, a->pg, addr, dtype_msz(a->dtype)); + do_ldrq(s, a->rd, a->pg, addr, a->dtype); } return true; } From patchwork Fri Apr 16 21:02:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422662 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp750984jaf; Fri, 16 Apr 2021 15:04:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz0CgsXtINxFRpAxZxWF5pjkVjcwlPye5swtrfCicSX+mC5n1+K7WyD0lnd0YCM2w0lWymE X-Received: by 2002:ab0:20a9:: with SMTP id y9mr1301920ual.57.1618610675832; Fri, 16 Apr 2021 15:04:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610675; cv=none; d=google.com; s=arc-20160816; b=o9rWUxP4AJszLY4Ypr+bQM6Fe9V6d36yhLw7y7JQ45NEmwKqwerc3OXyxBP4lr8F6e WE+fDH04tIfNWAhS+UN7D+UIsM9Gtpwf/pMfHHeGLEmJsSIJCRM5yAl7Q5s2ro7zarax 4W1qz/Ncatd6XnjqoFE6Zw/8uVKyQEgAuYJKtGWzIVw+UjoCJiRU04ZQELiz7WY9uO5I yyifvs3nd/lSLwgb431HmlMQT7dlMrqfgFYiOYv94AfQ85OplRZCNtc7YzEVZNMMK2t+ lWH5p5XYLM64o1ya92xKyZ4EoqEWX2tFESXoLuLEy+Ph+zqp76SiiFH+565S92bVPib0 l7ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=EnSyPCZmyd4z3y6dSeIzgcP6+JdzBQ6BgXyeRXzNj2k=; b=yczVdfr7PCl3KpZEOSlxx8df/aCjDsPOCpPKYIRT5a2W7ObLEgpcvCEF39VKDA2Lt/ 8KyThfm4eN+ZOH9BhjWGuIMUS7ykb0uvnxpL3pNdgVGB2BUx7OsL0Z3tskrGEg8nK5+I KhX72K2amujXLT/W7tPIBOC8CRtqgatUa4CYB/zEsFkJ5pJT7K5PEYaZ2cdMbBoi4oA3 k19bi9wSYUTpZ6vP1eT8CB+nPMRkbigtX5EX4z1hDvi+7xNqDLhHmBamQgUwV2L9X3ID ZJaOxKz9bjaVyX1vMAHd/eHkLPOmqUeHw/hBAQmb/8IuzpPspIqGfG8/q/Iv0bdkXN+J mGtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XDNKqLCX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h5si3189494vsf.63.2021.04.16.15.04.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:04:35 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XDNKqLCX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57854 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWZn-0006Ex-02 for patch@linaro.org; Fri, 16 Apr 2021 18:04:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45506) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdq-0006aG-6h for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:42 -0400 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]:39692) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdV-0001mL-Ua for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:41 -0400 Received: by mail-pf1-x436.google.com with SMTP id c17so19148060pfn.6 for ; Fri, 16 Apr 2021 14:04:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EnSyPCZmyd4z3y6dSeIzgcP6+JdzBQ6BgXyeRXzNj2k=; b=XDNKqLCXgaWN9hVZRw4eOItipgPOL+l5D2RKjGcPFUTgyDk7z6jcQ5ktHKnIhLIe1u sWnIYWMhADRyMwox5iU22xVDz78rrHrny/9HrEII8nWUvFBIFxU89O9PU0m5wqkNTI/a V6kP9apnS6cwPjmJ7Ff3c418p6LdVLW3wSpNLB5fel4V5CEhjLmj/36+0I5bPHIb6EgQ tQKDDGJwGMvsZ5TjAms9XNKGXJ8W064ex5Yq4v7CzcJHSuHAIvEptNw16fbstOeJiKdG Im5Ylfrdru08Qxp9FvkPwARVXagByHpTs+lzOHjynN0lHEQFwc9Q1XPRCbNCfkCh6v38 jkKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EnSyPCZmyd4z3y6dSeIzgcP6+JdzBQ6BgXyeRXzNj2k=; b=cTixCkWW3fcuZKEjMSeVC62d9AOQXiGXK4+HzdU875rrygDMwD8ecJe3Jf4yF4Eop5 bnvPKLSivYGzWhjdDRdBNpmvD7itFd1qN3rhgqfRDKvbe9c8nLmoN+YSK2Y+ilNrQ54x +eqIgVU3ymtMMTA+qjule9GkOxrF8e+ORe9+3ikaF86JuILqYgECuXDaCNmvl9hC8Rnn vXEiMvWfIE9DcjcwvkRSVTtEh5+KlPHROj+5oA8gFnxCPaTQ2Q3XH66RR/CwgMah5mjL vny/fO1VGzqzkQNtZ1s2Zz6Un4abYlEQpZitMCJsRZXK6DPPK/KxNSZL5pFQgD/gmPEL bH9Q== X-Gm-Message-State: AOAM5336X3HoQT/Dnn8KFx41x9/gFlo3fUTZYaYpNHnXA+Rtovm8Go7t sDKGYUt4yPj9frG+VccHERGClmgDQ/1XOg== X-Received: by 2002:a63:d755:: with SMTP id w21mr867382pgi.400.1618607060694; Fri, 16 Apr 2021 14:04:20 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:20 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 70/81] target/arm: Implement SVE2 LD1RO Date: Fri, 16 Apr 2021 14:02:29 -0700 Message-Id: <20210416210240.1591291-71-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 4 ++ target/arm/translate-sve.c | 97 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 101 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 17adb393ff..df870ce23b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1077,11 +1077,15 @@ LD_zpri 1010010 .. nreg:2 0.... 111 ... ..... ..... @rpri_load_msz # SVE load and broadcast quadword (scalar plus scalar) LD1RQ_zprr 1010010 .. 00 ..... 000 ... ..... ..... \ @rprr_load_msz nreg=0 +LD1RO_zprr 1010010 .. 01 ..... 000 ... ..... ..... \ + @rprr_load_msz nreg=0 # SVE load and broadcast quadword (scalar plus immediate) # LD1RQB, LD1RQH, LD1RQS, LD1RQD LD1RQ_zpri 1010010 .. 00 0.... 001 ... ..... ..... \ @rpri_load_msz nreg=0 +LD1RO_zpri 1010010 .. 01 0.... 001 ... ..... ..... \ + @rpri_load_msz nreg=0 # SVE 32-bit gather prefetch (scalar plus 32-bit scaled offsets) PRF 1000010 00 -1 ----- 0-- --- ----- 0 ---- diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 04efa037f2..1cc98a1447 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5586,6 +5586,103 @@ static bool trans_LD1RQ_zpri(DisasContext *s, arg_rpri_load *a) return true; } +static void do_ldro(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype) +{ + unsigned vsz = vec_full_reg_size(s); + unsigned vsz_r32; + TCGv_ptr t_pg; + TCGv_i32 t_desc; + int desc, poff, doff; + + if (vsz < 32) { + /* + * Note that this UNDEFINED check comes after CheckSVEEnabled() + * in the ARM pseudocode, which is the sve_access_check() done + * in our caller. We should not now return false from the caller. + */ + unallocated_encoding(s); + return; + } + + /* Load the first octaword using the normal predicated load helpers. */ + + poff = pred_full_reg_offset(s, pg); + if (vsz > 32) { + /* + * Zero-extend the first 32 bits of the predicate into a temporary. + * This avoids triggering an assert making sure we don't have bits + * set within a predicate beyond VQ, but we have lowered VQ to 2 + * for this load operation. + */ + TCGv_i64 tmp = tcg_temp_new_i64(); +#ifdef HOST_WORDS_BIGENDIAN + poff += 4; +#endif + tcg_gen_ld32u_i64(tmp, cpu_env, poff); + + poff = offsetof(CPUARMState, vfp.preg_tmp); + tcg_gen_st_i64(tmp, cpu_env, poff); + tcg_temp_free_i64(tmp); + } + + t_pg = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(t_pg, cpu_env, poff); + + desc = simd_desc(32, 32, zt); + t_desc = tcg_const_i32(desc); + + gen_helper_gvec_mem *fn + = ldr_fns[s->mte_active[0]][s->be_data == MO_BE][dtype][0]; + fn(cpu_env, t_pg, addr, t_desc); + + tcg_temp_free_ptr(t_pg); + tcg_temp_free_i32(t_desc); + + /* + * Replicate that first octaword. + * The replication happens in units of 32; if the full vector size + * is not a multiple of 32, the final bits are zeroed. + */ + doff = vec_full_reg_offset(s, zt); + vsz_r32 = QEMU_ALIGN_DOWN(vsz, 32); + if (vsz >= 64) { + tcg_gen_gvec_dup_mem(5, doff + 32, doff, vsz_r32 - 32, vsz - 32); + } else if (vsz > vsz_r32) { + /* Nop move, with side effect of clearing the tail. */ + tcg_gen_gvec_mov(MO_64, doff, doff, vsz_r32, vsz); + } +} + +static bool trans_LD1RO_zprr(DisasContext *s, arg_rprr_load *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + if (a->rm == 31) { + return false; + } + if (sve_access_check(s)) { + TCGv_i64 addr = new_tmp_a64(s); + tcg_gen_shli_i64(addr, cpu_reg(s, a->rm), dtype_msz(a->dtype)); + tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, a->rn)); + do_ldro(s, a->rd, a->pg, addr, a->dtype); + } + return true; +} + +static bool trans_LD1RO_zpri(DisasContext *s, arg_rpri_load *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + if (sve_access_check(s)) { + TCGv_i64 addr = new_tmp_a64(s); + tcg_gen_addi_i64(addr, cpu_reg_sp(s, a->rn), a->imm * 32); + do_ldro(s, a->rd, a->pg, addr, a->dtype); + } + return true; +} + /* Load and broadcast element. */ static bool trans_LD1R_zpri(DisasContext *s, arg_rpri_load *a) { From patchwork Fri Apr 16 21:02:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422649 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp744712jaf; Fri, 16 Apr 2021 14:53:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz92dRCVhUStuUICdaiinX+TPLQKY93b/+BjtJmI9Q45xHLzzCQCwHTrg4BbiMKl+RCgNnr X-Received: by 2002:a6b:8fd7:: with SMTP id r206mr5264060iod.71.1618610000660; Fri, 16 Apr 2021 14:53:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610000; cv=none; d=google.com; s=arc-20160816; b=MQaDlOkB7ARf8nLq6uMpTW1ZPHjowMJ35GI8PR1Xlwljeo3M9Y9LsMt/ehHRO0xcAD QkRyUjtYCKDpB+bVqZUDFsT0eHtHpsMDzPwf9JMTQMrvrS9Tza8scaLcjjt7NriF1JY+ 0tlgfH6PJNh4h/RYQTOm9dvbV1pk40TxG4NNPt51f3kT0HW6fG559d8IOR2rY2CvOiVc Elw8d/nGyN2KEAW7SmCLKiu2dFfo4xkpl1jugbpmwj1hEa5XZr0FqPEjbX/CnfVaJqaQ o/aPjXYiIjnnmZQUj/if84VnJRWL9ehTIgNznmcde9x1M8v5yQz2YxtG+wHz+asDwQIN 6n/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=8WE8+/U1Z3Zm6Ai/dPG+UHhpoeyvgwdlIhVlAeBkl70=; b=UEwV0q3K2iq5BX9ivlZ+psPcxFJx3qH0LJWDfYjXVeqnAWEDqrJ6pFUa61ehmvhWuc gU5xSyX0ZxmCbHOuO3XnEvkzr2ra0svpYoqixOMk73OeQ8ILezD6B9t7wjl0NFko3qiu p77LvGL2SLHjVoq19Ry+5/bFsLbsnP4dtZx2GNO2rLpMmqXNvxl0rDW7VF0z/oqwMWzE uBbMptKA62yr8ZAuKeC6zkLv15oIMS7sMNlr4nOKxOEP9NwmrEEbuA4r6wW4wgx4XUHY eiqTZlHVrtyGT6kzWma1MgAd9dgqtQOlDmQBd2E5H/N2goLaqlQ7q+pYPSYwprS4i1Ab 9fNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Il1f/GZW"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m5si1706613ilq.49.2021.04.16.14.53.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:53:20 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Il1f/GZW"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:49296 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWOu-0008Cz-2B for patch@linaro.org; Fri, 16 Apr 2021 17:53:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45568) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVds-0006gc-My for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:44 -0400 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]:39634) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdW-0001mh-W9 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:44 -0400 Received: by mail-pj1-x102b.google.com with SMTP id ot17-20020a17090b3b51b0290109c9ac3c34so17057684pjb.4 for ; Fri, 16 Apr 2021 14:04:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8WE8+/U1Z3Zm6Ai/dPG+UHhpoeyvgwdlIhVlAeBkl70=; b=Il1f/GZWewBFRsLqE12bGqAWPw2IpRwBasRGB6V3cDv9Nz//s5utX/q1MLyfQ11Um+ UE+t5fAeWK5jo2HYoHGvMAtbj15dLdV1XW43Q9tAC4plVEt6IFX5S5ufjwYzCS1zimBk XhhIBvMdwwWdxZsmAykZE+Vu9VvxnFblanJ5o5a1cQeXUmlCIAoFRaH+iK8lOZG3p5ti XFw35VBPXGu35kwuGNZSN0Xnp8o4u21/b4PLN/dYrs8uj6vtEzYsJCDK3SdvsFxICnOU gkYJmS531P3gruMm2Lh25IQuBHt1xeAHZamjsJkDAiss9XGBTC7T/s/X+kI/EbyBFU0D Rk5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8WE8+/U1Z3Zm6Ai/dPG+UHhpoeyvgwdlIhVlAeBkl70=; b=eP8i0YxSi/jVbw3f6FCtIVnpKsPdbqdsyWvtGLDRSpJe2iYBTFHG3ozPooIA+63yWc oEF+QXmA5DxkpbQM4dLD53FjoHzh2e3SPNmDpobi5hIREFCPFDXkqB9a9WKICxX8VxiA c9Slgswtw0N4bckpFiNQ731+iNe/TwcGJfs6HmGk6vSvt9I29HLYxgj5B6kaTw0vuZPh VFydKQ4xffuiw5e5lKC0WHbPII3jOjwBIp1cQPTHXsW6Jbiy2O4mvzCbWOzl08Ytnyff r8TjG1Ex/rMPyO+jOduiiDn5lG/K59QnssuBQzc/O4QR/TWmWVdVAyROr5HTUlbrsn5w prig== X-Gm-Message-State: AOAM530j9Vz0iO6orxDp0ryfFE0bL7+AwS8f+hXG3DU24svirAs4koF1 sKcM9EuhursG/T9KUCHO+IbAQWbDUmUKcw== X-Received: by 2002:a17:902:c407:b029:e7:2272:d164 with SMTP id k7-20020a170902c407b02900e72272d164mr11454227plk.84.1618607061722; Fri, 16 Apr 2021 14:04:21 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:21 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 71/81] target/arm: Implement 128-bit ZIP, UZP, TRN Date: Fri, 16 Apr 2021 14:02:30 -0700 Message-Id: <20210416210240.1591291-72-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102b; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 3 ++ target/arm/sve.decode | 8 ++++++ target/arm/sve_helper.c | 29 +++++++++++++------ target/arm/translate-sve.c | 58 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 90 insertions(+), 8 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 96bd200e73..6e9479800d 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -689,16 +689,19 @@ DEF_HELPER_FLAGS_4(sve_zip_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_zip_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_zip_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_zip_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_zip_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uzp_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_trn_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_compact_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_compact_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index df870ce23b..32e11301a5 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -590,6 +590,14 @@ UZP2_z 00000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm TRN1_z 00000101 .. 1 ..... 011 100 ..... ..... @rd_rn_rm TRN2_z 00000101 .. 1 ..... 011 101 ..... ..... @rd_rn_rm +# SVE2 permute vector segments +ZIP1_q 00000101 10 1 ..... 000 000 ..... ..... @rd_rn_rm_e0 +ZIP2_q 00000101 10 1 ..... 000 001 ..... ..... @rd_rn_rm_e0 +UZP1_q 00000101 10 1 ..... 000 010 ..... ..... @rd_rn_rm_e0 +UZP2_q 00000101 10 1 ..... 000 011 ..... ..... @rd_rn_rm_e0 +TRN1_q 00000101 10 1 ..... 000 110 ..... ..... @rd_rn_rm_e0 +TRN2_q 00000101 10 1 ..... 000 111 ..... ..... @rd_rn_rm_e0 + ### SVE Permute - Predicated Group # SVE compress active elements diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 754301a3a6..d5701cb4e8 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3338,36 +3338,45 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ *(TYPE *)(vd + H(2 * i + 0)) = *(TYPE *)(vn + H(i)); \ *(TYPE *)(vd + H(2 * i + sizeof(TYPE))) = *(TYPE *)(vm + H(i)); \ } \ + if (sizeof(TYPE) == 16 && unlikely(oprsz & 16)) { \ + memset(vd + oprsz - 16, 0, 16); \ + } \ } DO_ZIP(sve_zip_b, uint8_t, H1) DO_ZIP(sve_zip_h, uint16_t, H1_2) DO_ZIP(sve_zip_s, uint32_t, H1_4) DO_ZIP(sve_zip_d, uint64_t, ) +DO_ZIP(sve2_zip_q, Int128, ) #define DO_UZP(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ intptr_t oprsz = simd_oprsz(desc); \ - intptr_t oprsz_2 = oprsz / 2; \ intptr_t odd_ofs = simd_data(desc); \ - intptr_t i; \ + intptr_t i, p; \ ARMVectorReg tmp_m; \ if (unlikely((vm - vd) < (uintptr_t)oprsz)) { \ vm = memcpy(&tmp_m, vm, oprsz); \ } \ - for (i = 0; i < oprsz_2; i += sizeof(TYPE)) { \ - *(TYPE *)(vd + H(i)) = *(TYPE *)(vn + H(2 * i + odd_ofs)); \ - } \ - for (i = 0; i < oprsz_2; i += sizeof(TYPE)) { \ - *(TYPE *)(vd + H(oprsz_2 + i)) = *(TYPE *)(vm + H(2 * i + odd_ofs)); \ - } \ + i = 0, p = odd_ofs; \ + do { \ + *(TYPE *)(vd + H(i)) = *(TYPE *)(vn + H(p)); \ + i += sizeof(TYPE), p += 2 * sizeof(TYPE); \ + } while (p < oprsz); \ + p -= oprsz; \ + do { \ + *(TYPE *)(vd + H(i)) = *(TYPE *)(vm + H(p)); \ + i += sizeof(TYPE), p += 2 * sizeof(TYPE); \ + } while (p < oprsz); \ + tcg_debug_assert(i == oprsz); \ } DO_UZP(sve_uzp_b, uint8_t, H1) DO_UZP(sve_uzp_h, uint16_t, H1_2) DO_UZP(sve_uzp_s, uint32_t, H1_4) DO_UZP(sve_uzp_d, uint64_t, ) +DO_UZP(sve2_uzp_q, Int128, ) #define DO_TRN(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ @@ -3381,12 +3390,16 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ *(TYPE *)(vd + H(i + 0)) = ae; \ *(TYPE *)(vd + H(i + sizeof(TYPE))) = be; \ } \ + if (sizeof(TYPE) == 16 && unlikely(oprsz & 16)) { \ + memset(vd + oprsz - 16, 0, 16); \ + } \ } DO_TRN(sve_trn_b, uint8_t, H1) DO_TRN(sve_trn_h, uint16_t, H1_2) DO_TRN(sve_trn_s, uint32_t, H1_4) DO_TRN(sve_trn_d, uint64_t, ) +DO_TRN(sve2_trn_q, Int128, ) #undef DO_ZIP #undef DO_UZP diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 1cc98a1447..0cfc90b483 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -2624,6 +2624,32 @@ static bool trans_ZIP2_z(DisasContext *s, arg_rrr_esz *a) return do_zip(s, a, true); } +static bool do_zip_q(DisasContext *s, arg_rrr_esz *a, bool high) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0; + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn) + high_ofs, + vec_full_reg_offset(s, a->rm) + high_ofs, + vsz, vsz, 0, gen_helper_sve2_zip_q); + } + return true; +} + +static bool trans_ZIP1_q(DisasContext *s, arg_rrr_esz *a) +{ + return do_zip_q(s, a, false); +} + +static bool trans_ZIP2_q(DisasContext *s, arg_rrr_esz *a) +{ + return do_zip_q(s, a, true); +} + static gen_helper_gvec_3 * const uzp_fns[4] = { gen_helper_sve_uzp_b, gen_helper_sve_uzp_h, gen_helper_sve_uzp_s, gen_helper_sve_uzp_d, @@ -2639,6 +2665,22 @@ static bool trans_UZP2_z(DisasContext *s, arg_rrr_esz *a) return do_zzz_data_ool(s, a, 1 << a->esz, uzp_fns[a->esz]); } +static bool trans_UZP1_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 0, gen_helper_sve2_uzp_q); +} + +static bool trans_UZP2_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 16, gen_helper_sve2_uzp_q); +} + static gen_helper_gvec_3 * const trn_fns[4] = { gen_helper_sve_trn_b, gen_helper_sve_trn_h, gen_helper_sve_trn_s, gen_helper_sve_trn_d, @@ -2654,6 +2696,22 @@ static bool trans_TRN2_z(DisasContext *s, arg_rrr_esz *a) return do_zzz_data_ool(s, a, 1 << a->esz, trn_fns[a->esz]); } +static bool trans_TRN1_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 0, gen_helper_sve2_trn_q); +} + +static bool trans_TRN2_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 16, gen_helper_sve2_trn_q); +} + /* *** SVE Permute Vector - Predicated Group */ From patchwork Fri Apr 16 21:02:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422670 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp759925jaf; Fri, 16 Apr 2021 15:19:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxaPpc7ZnekHIErgtWWDbMApqEskEs+iMlpHBktGsb5ou4q8oGBKUDdaRhIGOj6O8US5PSx X-Received: by 2002:a0c:fb4e:: with SMTP id b14mr10897967qvq.28.1618611576442; Fri, 16 Apr 2021 15:19:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618611576; cv=none; d=google.com; s=arc-20160816; b=rNWIXnOgDWZpDMDxLsw/qhX+s7MxpDbROHjIJr56h5y1OiJRt1lRLAnFGpdvMM9OUA fV0ohRp3FYp235JUZ4XT37hqew0aS26oWamIZVFkb7l9vlAVvnKujFqrGDNVAIuh+/4T rzrlaM+sZUQhOuH1+e6vqzpIyq4qZlfRLblHzyGVUhsF/xEmG8qRJz66WAnPFdZh7VgY rg3qiaji5z5zC56Ap4Vn/Hv9gAPVWDpfMvLNPUhIVQTusYSgQHauL+d3mhWk0tWIfWmq WkHO3wbBr89LsWM26JdEX9RnH15c3EQ0DX/l9c1RC2VUIoOUXaFllwkYtHnGj4PA4E21 KEjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=SfmJvMQuIYR8WFaI/bEqnt8au8APyoTLP4XYJ+DE47U=; b=X/0gFaqMIPz27V6h6yg8PkQJLAZf4aPfL7+YTZIAoTT4/aDzITyyIwT211Zo7xlg4Z /8i0JQILwhg/6lrhHY8xgZKTdBouYd6f8c7N223LQNOdVNDEhkYTtld6P3kLz9zP1/y7 4gC2EVGbQd4dGk/K4mtpe2zA5CD9yLK9fWmaUURh6SBO/VxUzwMIK52i59DqHEfiBILn fLP2oEP1Omcf3bfmaNF8rtBOEkJtOcaJFZX1EWOk4LdHdTYPM/h/qhcf/Baqtfos4Ymj bvjg4S96rOTEasAw+20zcg6iUw1fUSl87VrpszBotPEHBwzL1kTQ6FfDS47m8b3AcJR6 TNvg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ayjpxGgE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t9si2921634qta.269.2021.04.16.15.19.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:19:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ayjpxGgE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43198 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWoJ-0007ZO-PR for patch@linaro.org; Fri, 16 Apr 2021 18:19:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45618) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdu-0006mG-TZ for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:46 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]:52010) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdY-0001nl-3O for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:46 -0400 Received: by mail-pj1-x1033.google.com with SMTP id lt13so5818831pjb.1 for ; Fri, 16 Apr 2021 14:04:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SfmJvMQuIYR8WFaI/bEqnt8au8APyoTLP4XYJ+DE47U=; b=ayjpxGgEcVrs9WBZnUJ/7ifQMHCzMA2jIKJgJMzAf96fgGY2F/Cjtrhdb6o+RND4sY S2mizd7f3BM1wO4OVj7BnR6j+zqRErtc4AJo2li2xEvVeyrVp8Bqrfz1IdyjhENssDAm FzZgEqV2rT6Jp3UG0c2WZSYVcBVUyy3hwDjL9sJzWQoHYwVsuS3oytgOB7rli/2CFKe3 gi+toHiSaxLUwyg6Mw7VI1EDGK9x3Yqd0YRnODjBkYP0ojfKNmJYyfNqCB2uBV+P6anu 4vdZ1mdh0qyiDq4zdfjiW8UBKuGBk9+9NM62329asvsOdngvuRjCOVNh9kyfuoIXt0yq lIng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SfmJvMQuIYR8WFaI/bEqnt8au8APyoTLP4XYJ+DE47U=; b=BgqDvffmcvV8WX3bBTw4H71AdsyGJN+l00MzdReEsS8hzLQnHPsncwSc499iLf1x5F K95BSf3XE6p4Iv1rNpIOC2gL/ciacwtz8toZS53ub9st5ijnaUhKyRhCf3QgU3+gmifa HYwfvGIm3dsB7TRLdAEADMQQuTaZ+zDGEOg7TEaEUvnTKk77fPn+ZrQofIaKGi6DKmDL tYmKHn/hza3YwoSB8nX8Bkdv3NFl8FxsiDNhglu7nBjUlOhwRrDr8Vgg2dy9pvVbrBBQ 53lKQKinhzC8+YnrgH+ILf/xc1NKirkOYn/YpqwUi84QKUppRRm/i2jI3f0IpTkiNoP2 GENA== X-Gm-Message-State: AOAM5328useMFGI2duNAumcFLLgr/F6jcghVczL57QX1LaziMPf18DH5 DKoSgcpfb5Y/10H34/9nep5Leo2hqrCRbg== X-Received: by 2002:a17:902:ce8b:b029:eb:5fd4:51c5 with SMTP id f11-20020a170902ce8bb02900eb5fd451c5mr11785968plg.31.1618607062759; Fri, 16 Apr 2021 14:04:22 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:22 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 72/81] target/arm: Implement SVE2 bitwise shift immediate Date: Fri, 16 Apr 2021 14:02:31 -0700 Message-Id: <20210416210240.1591291-73-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1033.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Implements SQSHL/UQSHL, SRSHR/URSHR, and SQSHLU Signed-off-by: Stephen Long Message-Id: <20200430194159.24064-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 33 +++++++++++++++++++++ target/arm/sve.decode | 5 ++++ target/arm/sve_helper.c | 35 ++++++++++++++++++++++ target/arm/translate-sve.c | 60 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 133 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 6e9479800d..fa7418e706 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2707,6 +2707,39 @@ DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_srshr_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_srshr_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_srshr_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_srshr_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_urshr_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_urshr_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_urshr_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_urshr_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqshlu_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshlu_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshlu_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshlu_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_fcvtnt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_fcvtnt_ds, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 32e11301a5..cfdee8955b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -340,6 +340,11 @@ ASR_zpzi 00000100 .. 000 000 100 ... .. ... ..... @rdn_pg_tszimm_shr LSR_zpzi 00000100 .. 000 001 100 ... .. ... ..... @rdn_pg_tszimm_shr LSL_zpzi 00000100 .. 000 011 100 ... .. ... ..... @rdn_pg_tszimm_shl ASRD 00000100 .. 000 100 100 ... .. ... ..... @rdn_pg_tszimm_shr +SQSHL_zpzi 00000100 .. 000 110 100 ... .. ... ..... @rdn_pg_tszimm_shl +UQSHL_zpzi 00000100 .. 000 111 100 ... .. ... ..... @rdn_pg_tszimm_shl +SRSHR 00000100 .. 001 100 100 ... .. ... ..... @rdn_pg_tszimm_shr +URSHR 00000100 .. 001 101 100 ... .. ... ..... @rdn_pg_tszimm_shr +SQSHLU 00000100 .. 001 111 100 ... .. ... ..... @rdn_pg_tszimm_shl # SVE bitwise shift by vector (predicated) ASR_zpzz 00000100 .. 010 000 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index d5701cb4e8..c5c3017745 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2084,6 +2084,41 @@ DO_ZPZI(sve_asrd_h, int16_t, H1_2, DO_ASRD) DO_ZPZI(sve_asrd_s, int32_t, H1_4, DO_ASRD) DO_ZPZI_D(sve_asrd_d, int64_t, DO_ASRD) +/* SVE2 bitwise shift by immediate */ +DO_ZPZI(sve2_sqshl_zpzi_b, int8_t, H1, do_sqshl_b) +DO_ZPZI(sve2_sqshl_zpzi_h, int16_t, H1_2, do_sqshl_h) +DO_ZPZI(sve2_sqshl_zpzi_s, int32_t, H1_4, do_sqshl_s) +DO_ZPZI_D(sve2_sqshl_zpzi_d, int64_t, do_sqshl_d) + +DO_ZPZI(sve2_uqshl_zpzi_b, uint8_t, H1, do_uqshl_b) +DO_ZPZI(sve2_uqshl_zpzi_h, uint16_t, H1_2, do_uqshl_h) +DO_ZPZI(sve2_uqshl_zpzi_s, uint32_t, H1_4, do_uqshl_s) +DO_ZPZI_D(sve2_uqshl_zpzi_d, uint64_t, do_uqshl_d) + +DO_ZPZI(sve2_srshr_b, int8_t, H1, do_srshr) +DO_ZPZI(sve2_srshr_h, int16_t, H1_2, do_srshr) +DO_ZPZI(sve2_srshr_s, int32_t, H1_4, do_srshr) +DO_ZPZI_D(sve2_srshr_d, int64_t, do_srshr) + +DO_ZPZI(sve2_urshr_b, uint8_t, H1, do_urshr) +DO_ZPZI(sve2_urshr_h, uint16_t, H1_2, do_urshr) +DO_ZPZI(sve2_urshr_s, uint32_t, H1_4, do_urshr) +DO_ZPZI_D(sve2_urshr_d, uint64_t, do_urshr) + +#define do_suqrshl_b(n, m) \ + ({ uint32_t discard; do_suqrshl_bhs(n, (int8_t)m, 8, false, &discard); }) +#define do_suqrshl_h(n, m) \ + ({ uint32_t discard; do_suqrshl_bhs(n, (int16_t)m, 16, false, &discard); }) +#define do_suqrshl_s(n, m) \ + ({ uint32_t discard; do_suqrshl_bhs(n, m, 32, false, &discard); }) +#define do_suqrshl_d(n, m) \ + ({ uint32_t discard; do_suqrshl_d(n, m, false, &discard); }) + +DO_ZPZI(sve2_sqshlu_b, int8_t, H1, do_suqrshl_b) +DO_ZPZI(sve2_sqshlu_h, int16_t, H1_2, do_suqrshl_h) +DO_ZPZI(sve2_sqshlu_s, int32_t, H1_4, do_suqrshl_s) +DO_ZPZI_D(sve2_sqshlu_d, int64_t, do_suqrshl_d) + #undef DO_ASRD #undef DO_ZPZI #undef DO_ZPZI_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0cfc90b483..26377b1227 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -1044,6 +1044,66 @@ static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a) } } +static bool trans_SQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqshl_zpzi_b, gen_helper_sve2_sqshl_zpzi_h, + gen_helper_sve2_sqshl_zpzi_s, gen_helper_sve2_sqshl_zpzi_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_UQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_uqshl_zpzi_b, gen_helper_sve2_uqshl_zpzi_h, + gen_helper_sve2_uqshl_zpzi_s, gen_helper_sve2_uqshl_zpzi_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_SRSHR(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_srshr_b, gen_helper_sve2_srshr_h, + gen_helper_sve2_srshr_s, gen_helper_sve2_srshr_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_URSHR(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_urshr_b, gen_helper_sve2_urshr_h, + gen_helper_sve2_urshr_s, gen_helper_sve2_urshr_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_SQSHLU(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqshlu_b, gen_helper_sve2_sqshlu_h, + gen_helper_sve2_sqshlu_s, gen_helper_sve2_sqshlu_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + /* *** SVE Bitwise Shift - Predicated Group */ From patchwork Fri Apr 16 21:02:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422655 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp746377jaf; Fri, 16 Apr 2021 14:56:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwEWY2C0go5eSPsDZyD/zaiGON1O6ReUDvCEwlmVIt091ewMGs9qqlNwRJGkSQUtG7nQTEx X-Received: by 2002:a5d:81c8:: with SMTP id t8mr5203024iol.19.1618610213406; Fri, 16 Apr 2021 14:56:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610213; cv=none; d=google.com; s=arc-20160816; b=m6+BCLgIuHmf0wM7zDYkXpyXij7rbuQnRefFBAvqh0KtsWTSpd/z3ifWnlCGVpgw/V 5njp4ivSuPYu8a6y/9QiDq1iCD8dpeAIFmi15t6KoQvvdY5Iq8FoiIcavxfyFqwlt/GB 7EjcxFXQrsqfN3d4wyO6hWgEtNtOwufRM3fo+nexIaTFGyF1zgAL/NpnBPJqwtgnKMSE LgmTsm+KfC90OL5RqrkgeYRstbWezma/z/TS7ABFfLEPxWmo2xcB82WkNV47ANQz/aeM d9ROrBrVqG/X+GyhpO42IxUM5DNjg9kPLSBH4Rb1UlLpPRNT+XKmWrvK+4rGOTn0aisr eoWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=iExyyyc5vS9YN2Us+0ohNYVQqJJ+6cn11cKRgRG0gvI=; b=YgQ9W94Y8UmkU3edl/mniEl2AxehKobNZuqNhf17S2fDGHtGES8tg6SJ0Kd/yBL5G8 TUsv6zVKKU6NScmry2k3fmcq7nOjIO9YRYR8/msBtInOdLbptpyCs6hiHSFrsLyYs2hJ kM0D+RR4TwD2UYazLjwxyBANxfQiu4T8QNZi4XWkDivcz0eMl34aWGLpKGnZOeeugaAS jG/YAMy5Wr3Vt/yuzGVKPsM2yf7WkZso0/W4/q7BJUG7hfWYnq0FcTAxyTcoLimtsByo eWg3+3+4A57RJX+jAYtc3CZ6IkgVprFdNmu4lrY4OPDgAASqizcMek8KLrQVKrIV//HI Uq/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rtoHt451; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l127si7147870ioa.2.2021.04.16.14.56.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:56:53 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rtoHt451; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59172 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWSK-0003iy-Pg for patch@linaro.org; Fri, 16 Apr 2021 17:56:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45614) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdu-0006l7-Fm for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:46 -0400 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]:42535) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdZ-0001oQ-5S for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:46 -0400 Received: by mail-pl1-x632.google.com with SMTP id v13so1042947ple.9 for ; Fri, 16 Apr 2021 14:04:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iExyyyc5vS9YN2Us+0ohNYVQqJJ+6cn11cKRgRG0gvI=; b=rtoHt451I847eBMZvh+094Fc/MYa26KuD0kUdbz/e9vxi+gvfdQww13wutjvJ7ePCH PWiIelM0mCVuNm/pZeIcXhNNZuDB7vyOerwtmZCKDlz9YiChOeZHNhAS6N59l7JjqeFL hw6QtWXuPhSOtwxFVT+TxCpnfL6iAVvGOVi/oWJRd6QluxcD5XAaQSCSfAI1D18j2qGx Vgnj8m/b9ODGufyAgqPSPxqMKdCZOWpfBbB1LgqmvuEIVOxtg14UCNFrOK5aTCKtWJyh i/ORVmU/1Qd0Ooyp9L8icoo/XWOBq43VKfop4Klk1pMqx+jYfO8Bpfjd5EdJ10hyfQfQ JrHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iExyyyc5vS9YN2Us+0ohNYVQqJJ+6cn11cKRgRG0gvI=; b=UJh6AnNvnN7iWhaMjVZgrVjtmfg4wlZ4WTL8Wm/bLHnrFGuFk4v7h470x2xqi7TPIl DInKTCBkeqlfF1D5ipLJq0ABBSOtx36VtwwxQ3Sc5qhfPCJdbsLqr6qmcpJ+t7R+mbjD BEgqkQJYWjjMxMuTnrAujk6hI+to6sdsUtmHQnkllHSoeO9MVzPoClrISGRT17NaZnEy 5ItawOkOM73mQ5WouTeGtBo1bCBLgiIom5wK5wRxrfOoRT3B52+ZuEM5exQCcGY1zFAO wYp4f+YsUABeGXMX7rdV+QZOPnVNkYNLtuiWOT0mCRdy6C2oHhlxsn9uHOereN3+2OBA M5pg== X-Gm-Message-State: AOAM532GNhXDssDcjyBYVLkMsNtzAqDe5DsLDoM+UZ/7gMOATVYWA2M2 tJLvISG/bhTRM5M7h1SYbkwAfhIFY8UQMA== X-Received: by 2002:a17:902:d4d0:b029:eb:59ea:2f32 with SMTP id o16-20020a170902d4d0b02900eb59ea2f32mr11416058plg.58.1618607063749; Fri, 16 Apr 2021 14:04:23 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:23 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 73/81] target/arm: Implement SVE2 fp multiply-add long Date: Fri, 16 Apr 2021 14:02:32 -0700 Message-Id: <20210416210240.1591291-74-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::632; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x632.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Implements both vectored and indexed FMLALB, FMLALT, FMLSLB, FMLSLT Signed-off-by: Stephen Long Message-Id: <20200504171240.11220-1-steplong@quicinc.com> [rth: Rearrange to use float16_to_float32_by_bits.] Signed-off-by: Richard Henderson --- target/arm/helper.h | 5 +++ target/arm/sve.decode | 14 +++++++ target/arm/translate-sve.c | 75 ++++++++++++++++++++++++++++++++++++++ target/arm/vec_helper.c | 51 ++++++++++++++++++++++++++ 4 files changed, 145 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 86f938c938..e8b16a401f 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -987,6 +987,11 @@ DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmlal_zzxw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) #ifdef TARGET_AARCH64 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index cfdee8955b..63870b7539 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -132,6 +132,8 @@ &rrrr_esz ra=%reg_movprfx # Four operand with unused vector element size +@rda_rn_rm_e0 ........ ... rm:5 ... ... rn:5 rd:5 \ + &rrrr_esz esz=0 ra=%reg_movprfx @rdn_ra_rm_e0 ........ ... rm:5 ... ... ra:5 rd:5 \ &rrrr_esz esz=0 rn=%reg_movprfx @@ -1559,3 +1561,15 @@ FCVTLT_sd 01100100 11 0010 11 101 ... ..... ..... @rd_pg_rn_e0 ### SVE2 floating-point convert to integer FLOGB 01100101 00 011 esz:2 0101 pg:3 rn:5 rd:5 &rpr_esz + +### SVE2 floating-point multiply-add long (vectors) +FMLALB_zzzw 01100100 10 1 ..... 10 0 00 0 ..... ..... @rda_rn_rm_e0 +FMLALT_zzzw 01100100 10 1 ..... 10 0 00 1 ..... ..... @rda_rn_rm_e0 +FMLSLB_zzzw 01100100 10 1 ..... 10 1 00 0 ..... ..... @rda_rn_rm_e0 +FMLSLT_zzzw 01100100 10 1 ..... 10 1 00 1 ..... ..... @rda_rn_rm_e0 + +### SVE2 floating-point multiply-add long (indexed) +FMLALB_zzxw 01100100 10 1 ..... 0100.0 ..... ..... @rrxr_3a esz=2 +FMLALT_zzxw 01100100 10 1 ..... 0100.1 ..... ..... @rrxr_3a esz=2 +FMLSLB_zzxw 01100100 10 1 ..... 0110.0 ..... ..... @rrxr_3a esz=2 +FMLSLT_zzxw 01100100 10 1 ..... 0110.1 ..... ..... @rrxr_3a esz=2 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 26377b1227..ae628968da 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8468,3 +8468,78 @@ static bool trans_FLOGB(DisasContext *s, arg_rpr_esz *a) } return true; } + +static bool do_FMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sub, bool sel) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + cpu_env, vsz, vsz, (sel << 1) | sub, + gen_helper_sve2_fmlal_zzzw_s); + } + return true; +} + +static bool trans_FMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, false, false); +} + +static bool trans_FMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, false, true); +} + +static bool trans_FMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, true, false); +} + +static bool trans_FMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, true, true); +} + +static bool do_FMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sub, bool sel) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + cpu_env, vsz, vsz, + (a->index << 2) | (sel << 1) | sub, + gen_helper_sve2_fmlal_zzxw_s); + } + return true; +} + +static bool trans_FMLALB_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, false, false); +} + +static bool trans_FMLALT_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, false, true); +} + +static bool trans_FMLSLB_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, true, false); +} + +static bool trans_FMLSLT_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, true, true); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 9b2a4d5b7e..ea08b15c55 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -29,10 +29,14 @@ so addressing units smaller than that needs a host-endian fixup. */ #ifdef HOST_WORDS_BIGENDIAN #define H1(x) ((x) ^ 7) +#define H1_2(x) ((x) ^ 6) +#define H1_4(x) ((x) ^ 4) #define H2(x) ((x) ^ 3) #define H4(x) ((x) ^ 1) #else #define H1(x) (x) +#define H1_2(x) (x) +#define H1_4(x) (x) #define H2(x) (x) #define H4(x) (x) #endif @@ -1907,6 +1911,27 @@ void HELPER(gvec_fmlal_a64)(void *vd, void *vn, void *vm, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } +void HELPER(sve2_fmlal_zzzw_s)(void *vd, void *vn, void *vm, void *va, + void *venv, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + uint16_t negn = extract32(desc, SIMD_DATA_SHIFT, 1) << 15; + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(float16); + CPUARMState *env = venv; + float_status *status = &env->vfp.fp_status; + bool fz16 = get_flush_inputs_to_zero(&env->vfp.fp_status_f16); + + for (i = 0; i < oprsz; i += sizeof(float32)) { + float16 nn_16 = *(float16 *)(vn + H1_2(i + sel)) ^ negn; + float16 mm_16 = *(float16 *)(vm + H1_2(i + sel)); + float32 nn = float16_to_float32_by_bits(nn_16, fz16); + float32 mm = float16_to_float32_by_bits(mm_16, fz16); + float32 aa = *(float32 *)(va + H1_4(i)); + + *(float32 *)(vd + H1_4(i)) = float32_muladd(nn, mm, aa, 0, status); + } +} + static void do_fmlal_idx(float32 *d, void *vn, void *vm, float_status *fpst, uint32_t desc, bool fz16) { @@ -1951,6 +1976,32 @@ void HELPER(gvec_fmlal_idx_a64)(void *vd, void *vn, void *vm, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } +void HELPER(sve2_fmlal_zzxw_s)(void *vd, void *vn, void *vm, void *va, + void *venv, uint32_t desc) +{ + intptr_t i, j, oprsz = simd_oprsz(desc); + uint16_t negn = extract32(desc, SIMD_DATA_SHIFT, 1) << 15; + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(float16); + intptr_t idx = extract32(desc, SIMD_DATA_SHIFT + 2, 3) * sizeof(float16); + CPUARMState *env = venv; + float_status *status = &env->vfp.fp_status; + bool fz16 = get_flush_inputs_to_zero(&env->vfp.fp_status_f16); + + for (i = 0; i < oprsz; i += 16) { + float16 mm_16 = *(float16 *)(vm + i + idx); + float32 mm = float16_to_float32_by_bits(mm_16, fz16); + + for (j = 0; j < 16; j += sizeof(float32)) { + float16 nn_16 = *(float16 *)(vn + H1_2(i + j + sel)) ^ negn; + float32 nn = float16_to_float32_by_bits(nn_16, fz16); + float32 aa = *(float32 *)(va + H1_4(i + j)); + + *(float32 *)(vd + H1_4(i + j)) = + float32_muladd(nn, mm, aa, 0, status); + } + } +} + void HELPER(gvec_sshl_b)(void *vd, void *vn, void *vm, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); From patchwork Fri Apr 16 21:02:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422647 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp744336jaf; Fri, 16 Apr 2021 14:52:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzkG0xr8HpeJX1s+KZGoi7dwhRkiyuEPdE0zYdX+ImroTYjZv4vERe7Qgc4SWUm2dutcJ30 X-Received: by 2002:a05:6902:4ab:: with SMTP id r11mr1696373ybs.158.1618609939784; Fri, 16 Apr 2021 14:52:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618609939; cv=none; d=google.com; s=arc-20160816; b=OMfSFPLUX1rLGfAMKScYEQRAgWgdZa3qP7fxZ8OC9mUPgi/XaIbNb+iBPhLqAJCCPU igP4MsDrD1Z3gx4BRMGpo3papItYGWvZzC6j6CHmWmrmz/zNGabF0BQbW3rqrZMrwZgO v7GkwLrEZgZ4DbBAIqMmd7XTz4FTUyDdM1plh71MD2ztBZmkslVqUD9A0A++cDlTtn66 s55ItUTkcGRjM8HJXfVP42M3oIUh1TSSiTJDvpjP5zBXoy85OK71fmAUv1oOnrnAY5BG uuKEPLdxq9+FqQF/BEPXxpA438uRUeEKTmN5I6FQxxzYBinPLYafqHqHM9xKbmDvMJdD 58TA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=BxrpeVsOh7NSGL6r65vW+3WtU9d9sINqZuLtaN2MbiQ=; b=AZkd4Qncc4rBajm14K8FGC4Nsag6FZlPeySVx33C/e9/llaAPUOzPnRktZDiBrTK8J igspWfbgWZEWdNSPt6JAtQ8tDUEgYROLpoBK73ZJlVkSqtSXId2YYBtvdgyIQ/YFa8hX oTKhhxcTmvgSOEAHl7fIS50VN78rXxkayO9WQf5RKsgZEYie536r1F43c0+fRRED4PDM 57lTnsR5ifLnjhQOlqp/hs5fOQY+W3XYluD9dzLnGOem03xeS4T7UnPdNQIJaOW73z8y wHLJPcuh3cBduwGsI0LsCS2T7lIEblTHBrBEF1d44lEawuvrHYbvuHPhRVyWAuND8nMC vh/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dQtGuTpc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r13si6586783ybp.209.2021.04.16.14.52.19 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:52:19 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dQtGuTpc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43174 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWNv-0005mJ-6Z for patch@linaro.org; Fri, 16 Apr 2021 17:52:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45630) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdv-0006ms-5J for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:47 -0400 Received: from mail-pg1-x52f.google.com ([2607:f8b0:4864:20::52f]:43905) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVda-0001pD-0M for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:46 -0400 Received: by mail-pg1-x52f.google.com with SMTP id p12so20003761pgj.10 for ; Fri, 16 Apr 2021 14:04:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BxrpeVsOh7NSGL6r65vW+3WtU9d9sINqZuLtaN2MbiQ=; b=dQtGuTpc+NnybSBeYfxxsdU2HzlD1xhbiW2S68xYO+cmm+HEjsLGTZhqlBko8Ftxyw zUFjXLBfHwFsm3LKUeMGWMT27T61FHPSAOyXNxTQpwa6fVeXOTM+C0ZTc1yLAIVOT0wf vydxPPOuI5p9ZaOx//njoBwC0VMW4w7oPKWAw7fITsrFLmYBNb4vOrILT4qky/XxRuzV R9FkVt70x7aPJTDnjoFPTToD9pulhax+xuEcqS/O3cZ9P82hZk4DUuY4ckSBWGrt9Vmp Lyn7C226SBWP68Ly9HPRRWWDhNpWynYnUe4HFI2YzgODlL/EHI1diZ2U54TQp6yPEXfn zUbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BxrpeVsOh7NSGL6r65vW+3WtU9d9sINqZuLtaN2MbiQ=; b=cNh4mqZ7UqjOCiaZgpeVF0orjNKfpRCfQf5Xf7tyQFFa3Wm0GfyBTVdWTcRdt/uT4p RuSRbJvroVULna0DJpcZegU0Saf0pVMkPV/eEvN5f21uiaQDgdZYv7hLGS4zezJyiuLI Wk1/BSTDdhINuTMRd+DRJHOGW3YZ1SIFCGMW29JAh2YWdG/SJnZaV3iTGlPFkyObZGwF nU6CJVnu9u8wgo0bt7dhUryQtIo3aS3MK41IwpCkPdKiCJg7+rhz4Ey2d4P1K4YckHu+ RP1q5lV7VP2H4cgBWoozcuTWXMISZaYWHYja/NhVXSL1EDvxcnq+rgKzspfJV/eXF0dT MSNw== X-Gm-Message-State: AOAM530/2gFamqy346tOX3Orl+uSGPjOS6zcTHUZdume+bkOKGZVGvd8 GohTQciEVq2fMFcSAgToT0B4GEMH78qVnA== X-Received: by 2002:a65:6184:: with SMTP id c4mr857679pgv.200.1618607064762; Fri, 16 Apr 2021 14:04:24 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:24 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 74/81] target/arm: Implement aarch64 SUDOT, USDOT Date: Fri, 16 Apr 2021 14:02:33 -0700 Message-Id: <20210416210240.1591291-75-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52f; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++++ target/arm/translate-a64.c | 25 +++++++++++++++++++++++++ 2 files changed, 30 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index b43fd066ba..a0865e224c 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4206,6 +4206,11 @@ static inline bool isar_feature_aa64_rcpc_8_4(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64isar1, ID_AA64ISAR1, LRCPC) >= 2; } +static inline bool isar_feature_aa64_i8mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64isar1, ID_AA64ISAR1, I8MM) != 0; +} + static inline bool isar_feature_aa64_ccidx(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, CCIDX) != 0; diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index f45b81e56d..0d45a44f51 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -12190,6 +12190,13 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) } feature = dc_isar_feature(aa64_dp, s); break; + case 0x03: /* USDOT */ + if (size != MO_32) { + unallocated_encoding(s); + return; + } + feature = dc_isar_feature(aa64_i8mm, s); + break; case 0x18: /* FCMLA, #0 */ case 0x19: /* FCMLA, #90 */ case 0x1a: /* FCMLA, #180 */ @@ -12230,6 +12237,10 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b); return; + case 0x3: /* USDOT */ + gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, 0, gen_helper_gvec_usdot_b); + return; + case 0x8: /* FCMLA, #0 */ case 0x9: /* FCMLA, #90 */ case 0xa: /* FCMLA, #180 */ @@ -13375,6 +13386,13 @@ static void disas_simd_indexed(DisasContext *s, uint32_t insn) return; } break; + case 0x0f: /* SUDOT, USDOT */ + if (is_scalar || (size & 1) || !dc_isar_feature(aa64_i8mm, s)) { + unallocated_encoding(s); + return; + } + size = MO_32; + break; case 0x11: /* FCMLA #0 */ case 0x13: /* FCMLA #90 */ case 0x15: /* FCMLA #180 */ @@ -13489,6 +13507,13 @@ static void disas_simd_indexed(DisasContext *s, uint32_t insn) u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b); return; + case 0x0f: /* SUDOT, USDOT */ + gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, index, + extract32(insn, 23, 1) + ? gen_helper_gvec_usdot_idx_b + : gen_helper_gvec_sudot_idx_b); + return; + case 0x11: /* FCMLA #0 */ case 0x13: /* FCMLA #90 */ case 0x15: /* FCMLA #180 */ From patchwork Fri Apr 16 21:02:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422650 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp744977jaf; Fri, 16 Apr 2021 14:53:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzmITtk3wq7U4lhf/l6T8466WFHIq8leru5eLm2LSley3sDEZ6NbVinMoLBoeqekengyhjc X-Received: by 2002:a25:be09:: with SMTP id h9mr1796449ybk.239.1618610037482; Fri, 16 Apr 2021 14:53:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610037; cv=none; d=google.com; s=arc-20160816; b=f0A9pwl3Yf+pu66sHLHI30zoKH5crX7BEGq1PF5zd6T8GUHpA6fytmDTP8uq6rMNQd xr9HmnS3CKSQuAlzI/hBqOUvZyz2O43LpvY64eRp68g/gnmcia9hbmF5iiEr6F2DnA9U +R+ZSlfpjlOXhBce5YHpsT7dygsLO+V6Y2vwwlmnVlbLAOi5uoQz470uW0BClzTzGaVX aIdGSaK/We2l6x2N7PyNpAA2XyVdRrjA29u+PUoQ4SrQB3u3/1gXmSt+v0LzRgGx1a7y sWkZb0/JpHd+EAsJQV1m6jq437XNCeao7bSlUvL5l6sjMYVamD9jTX26tgBBBzuRgQ8O UEPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=SejLQptV3Xbdii/z6eoM9zkt++LAZtRH0Va0O76ztI4=; b=bbWaf6PtE7WUKbiFEP6GCNQxRFhMPsKI4/wSiKaYwGeCUJmV2L33e2+K8UOujYwWM4 qQZwIQMs2Cx3FOl6rDu9JDLPo1ZUcpmvvJZvLdCyAlxiWKXVz6qJHIVbHW51JC0VpeKC kd42mbfiu3SIwAKyJSqIhtsiBzwnKJBB9m4NPeGVx2y29S6gWvnZknLLO6OSB32NyU0z LOPsiGbLtx/I7j4Jzpv06yLW34Va9dy9pkgfrcbAIC5D03/hcXuiLh3nnHyfIAHlXNNo Nk5jvf5eOXI2LV3SgLN5vQjNOOAh3hN/L63rWoMWYYA4GHwySSQdNdr+bcvAhZii0WLI 68Lg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ar7Apl2L; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c8si6714238ybj.340.2021.04.16.14.53.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:53:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ar7Apl2L; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52886 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWPU-0001Dd-TK for patch@linaro.org; Fri, 16 Apr 2021 17:53:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45692) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdx-0006rg-8l for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:49 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]:41718) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVda-0001pU-Tb for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:48 -0400 Received: by mail-pl1-x62c.google.com with SMTP id e2so10328438plh.8 for ; Fri, 16 Apr 2021 14:04:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SejLQptV3Xbdii/z6eoM9zkt++LAZtRH0Va0O76ztI4=; b=ar7Apl2LZBFiJ6a6aVAVFvIrCpAyQW3lgFmo5RVKpvZTJ5stWUKNrCBcJT84eoUbVg ZdrFso93Xt2xZSoU2JW2Atrb395ssj+UmD/ZRv4POMJMBuUrmuSCIcLhdgd2SXuKEX6g r9dSgxflrHo3853Y7AP842VGVD3QwUeaV80VdLr/49h1ckQthh7ysgpI7+Gnht8JXIdk kDwlrTkXPG0SRPxlubEOBcktwrNk8ZzRzYEYmD5xn9K4VH789kJnVNFD+Du3l3cWTTOb 8pp76AEQsZqYErJP+3NRbwS0/E6Slv6kQWUotJk50Q4Tvnaycs/bH2W/C/8+KK+s1Zhr ihQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SejLQptV3Xbdii/z6eoM9zkt++LAZtRH0Va0O76ztI4=; b=Q1dy/EeWWyWXNEv4HTfQOArnJQSvnS5oGu7AXVsG10uxaLbgO1VS+tTN1vynea+S0X 35deU7SoMuM6ys7suTs379Ep7/MSLfeZbkXQcb2HQ4+UbANiKSBGHdtRU1F2JiCusaEE xP6VBIZhjJh8VlPWJSvbxmQwVz3g1nw60Vq414USLRv3rwjGPmv3cOIGGOdb0d8XD4m5 McGls6orQA3txH61bvVtLa3w95/waIgba9LVkGlwGK/Clq5zQGiUNTSIB8v2HAzdykeK uoowE9PEz3Xozg+BwOBirWsms6geZUKQ9NLkebfSOxyvo94o/IJGiVE/X793t6mQa00H pBJg== X-Gm-Message-State: AOAM530+piFYklSrt4uShyD2WO4QqCSD94Q9iS5jY4n7RcAkh+aVlg+7 08hWXneQnJPpRIzqh9vsVktUQux/lThG4w== X-Received: by 2002:a17:90a:dc07:: with SMTP id i7mr11577441pjv.16.1618607065598; Fri, 16 Apr 2021 14:04:25 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:25 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 75/81] target/arm: Split out do_neon_ddda_fpst Date: Fri, 16 Apr 2021 14:02:34 -0700 Message-Id: <20210416210240.1591291-76-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Split out a helper that can handle the 4-register format for helpers shared with SVE. Signed-off-by: Richard Henderson --- target/arm/translate-neon.c.inc | 98 +++++++++++++++------------------ 1 file changed, 43 insertions(+), 55 deletions(-) -- 2.25.1 diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index 6f418bd8de..6385d13a7e 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -142,24 +142,21 @@ static void neon_store_element64(int reg, int ele, MemOp size, TCGv_i64 var) } } -static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) +static bool do_neon_ddda_fpst(DisasContext *s, int q, int vd, int vn, int vm, + int data, ARMFPStatusFlavour fp_flavor, + gen_helper_gvec_4_ptr *fn_gvec_ptr) { - int opr_sz; - TCGv_ptr fpst; - gen_helper_gvec_4_ptr *fn_gvec_ptr; - - if (!dc_isar_feature(aa32_vcma, s) - || (a->size == MO_16 && !dc_isar_feature(aa32_fp16_arith, s))) { - return false; - } - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { + if (((vd | vn | vm) & 0x10) && !dc_isar_feature(aa32_simd_r32, s)) { return false; } - if ((a->vn | a->vm | a->vd) & a->q) { + /* + * UNDEF accesses to odd registers for each bit of Q. + * Q will be 0b111 for all Q-reg instructions, otherwise + * when we have mixed Q- and D-reg inputs. + */ + if (((vd & 1) * 4 | (vn & 1) * 2 | (vm & 1)) & q) { return false; } @@ -167,20 +164,34 @@ static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) return true; } - opr_sz = (1 + a->q) * 8; - fpst = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD); - fn_gvec_ptr = (a->size == MO_16) ? - gen_helper_gvec_fcmlah : gen_helper_gvec_fcmlas; - tcg_gen_gvec_4_ptr(vfp_reg_offset(1, a->vd), - vfp_reg_offset(1, a->vn), - vfp_reg_offset(1, a->vm), - vfp_reg_offset(1, a->vd), - fpst, opr_sz, opr_sz, a->rot, - fn_gvec_ptr); + int opr_sz = q ? 16 : 8; + TCGv_ptr fpst = fpstatus_ptr(fp_flavor); + + tcg_gen_gvec_4_ptr(vfp_reg_offset(1, vd), + vfp_reg_offset(1, vn), + vfp_reg_offset(1, vm), + vfp_reg_offset(1, vd), + fpst, opr_sz, opr_sz, data, fn_gvec_ptr); tcg_temp_free_ptr(fpst); return true; } +static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) +{ + if (!dc_isar_feature(aa32_vcma, s)) { + return false; + } + if (a->size == MO_16) { + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + return do_neon_ddda_fpst(s, a->q * 7, a->vd, a->vn, a->vm, a->rot, + FPST_STD_F16, gen_helper_gvec_fcmlah); + } + return do_neon_ddda_fpst(s, a->q * 7, a->vd, a->vn, a->vm, a->rot, + FPST_STD, gen_helper_gvec_fcmlas); +} + static bool trans_VCADD(DisasContext *s, arg_VCADD *a) { int opr_sz; @@ -285,43 +296,20 @@ static bool trans_VFML(DisasContext *s, arg_VFML *a) static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) { - gen_helper_gvec_4_ptr *fn_gvec_ptr; - int opr_sz; - TCGv_ptr fpst; + int data = (a->index << 2) | a->rot; if (!dc_isar_feature(aa32_vcma, s)) { return false; } - if (a->size == MO_16 && !dc_isar_feature(aa32_fp16_arith, s)) { - return false; + if (a->size == MO_16) { + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + return do_neon_ddda_fpst(s, a->q * 6, a->vd, a->vn, a->vm, data, + FPST_STD_F16, gen_helper_gvec_fcmlah_idx); } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - - if ((a->vd | a->vn) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - fn_gvec_ptr = (a->size == MO_16) ? - gen_helper_gvec_fcmlah_idx : gen_helper_gvec_fcmlas_idx; - opr_sz = (1 + a->q) * 8; - fpst = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD); - tcg_gen_gvec_4_ptr(vfp_reg_offset(1, a->vd), - vfp_reg_offset(1, a->vn), - vfp_reg_offset(1, a->vm), - vfp_reg_offset(1, a->vd), - fpst, opr_sz, opr_sz, - (a->index << 2) | a->rot, fn_gvec_ptr); - tcg_temp_free_ptr(fpst); - return true; + return do_neon_ddda_fpst(s, a->q * 6, a->vd, a->vn, a->vm, data, + FPST_STD, gen_helper_gvec_fcmlas_idx); } static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) From patchwork Fri Apr 16 21:02:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422671 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp760812jaf; Fri, 16 Apr 2021 15:21:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyswdEWi24VTP+DN4krA9lzq6vfOb9o1uGEQ8cWR05hbVOgTmF5FB+LmDdIRNjmcHe4uPgM X-Received: by 2002:a25:c082:: with SMTP id c124mr1754974ybf.396.1618611676006; Fri, 16 Apr 2021 15:21:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618611676; cv=none; d=google.com; s=arc-20160816; b=0pwQYoMCh9XKuOykImcBAR5ddCNxYntJ1D+upo2TQGRJR4URMhhx7yeLGg1F/Q3Q1h 1vRSNvqNn+Mzxv/qP2OTzg8szPhZ/B2hDS1hMr1s5n3rNUjvgiGgwifK7MkEILzT3ZhU Dubjn45MMWmKRCyN9EUi+HHrFwnnZI3f3AjPfj+rvlEwfSJ+tgxWD6iHOZOOn5xeadWE v2zf45QcV5gUqZn2UQ79RXfopuPnjoz48QqhX7sHWyn3C2Vs/DXEryMVKki1PefhkNDM SlhENs2Hp4BuNlGHPnTtbVP1MCXDi3VyLnmVQXbqLPwBEJF68WBhxp/lzbdaHWiPKaeh CmAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=8Vthia0uCLhbBTsJ1gqUumg4ccjBueHeraGUzoP/hUo=; b=0PyBGFnm6GJjrPmsE/JCN/nxiKWUvYfuXN2ZoBPqQ+0SB8zvAPC/W0pQpmJFu150tf JzT9MzKOVVZTrtGzWN/COJ/z4OBk1Zf3e+1K+kWYjBcmbYzlBoOxg2eGWx2UYZTNfVM7 EaafhJB3Xp4Cob4F5sDI5ZIF6ZgyvbI2TaO4/QXD/VWeo6wZdNEnLiy3PPr4x7g+qi4o KDpAnbtlABwNJMzAL4MSG+yC9GLDMLWZ2zMOKhQ/aw/uNyGburQB+aV1h01kOg13+KmA r1Pcq4hNYYbPGGmVDj7My40MsXxIyUKUysb4wNuK3TxXwGtue6sC90xppTbMRsKJf/zB 0zAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mPN6yWUb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r8si7262327ybb.369.2021.04.16.15.21.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:21:15 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mPN6yWUb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWpv-0000qx-HS for patch@linaro.org; Fri, 16 Apr 2021 18:21:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45658) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdw-0006q4-Jp for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:48 -0400 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]:38888) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdb-0001pk-Nt for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:48 -0400 Received: by mail-pl1-x631.google.com with SMTP id o16so1028419plg.5 for ; Fri, 16 Apr 2021 14:04:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8Vthia0uCLhbBTsJ1gqUumg4ccjBueHeraGUzoP/hUo=; b=mPN6yWUbNavADuJXATWt8nvOEpoCFjFSe56nvdRqQMCUZs8HtfXsi2B9j9qjL5qBVo 770Y/9yYZwTH+4mP8dAUbg2aWF63UxA1F7K+nzJfX8tF6ELw5kMnHe5KHTfOuRrRUlHd jWxCCyrBZ0EoI3zYk+ksBykYVQsrWmJF72Ur5JiJ1lHrFradv0AQGineOWlr7RE6KgeK z1OJodYCfloC3mmaQ95YU8AkwJrwNIgdLPI2ZCCikSgeRWUwqDrxnugGvwvECpiooNO9 o9a3RVyldTq83H1uHYPJ9DfehsuKDDrSX5wfAuc2ClEs834NsUkvA8bUfeGreLE2tnDR 2Mng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8Vthia0uCLhbBTsJ1gqUumg4ccjBueHeraGUzoP/hUo=; b=Zz4T1CIYarML8HzfU1pRxqPFQ9x7iTvHPzhrVNqmsioQsct1LRtFOKl55936OpZLvs W84Gh4aAFBBtcdbUN0SY+f84S8LexSNNdPnHVdOFvyiyfKoF6lr66I0iE20ONaWC2vWM TkUTWkSR9bCVDfd8jJ/WyQ+1Ek4o806sWa2oFFeE/VaNYPqnoEPiASTW/IfRho2QlnFK 1RTgXbUt21O5ubFHzBUS6PIoyABWfHObfbRr/Dqp/KMLL+Pl/+R/8rGy7yOPxIsv2Way aG960ufSl5uUb8A0EmOB5HOKn9UU14sTVrlsNwmEvyKmDMAu8moW+T4fgxvN603ZTAs4 3ztw== X-Gm-Message-State: AOAM5313V3VxoDgoXyT5Ltn9FZg8VKy2nhlfVytdxOXwe+Ai5/LuhAdS G31zwvr/+uSMej1aZP7AhkLRHT0ce1GBfg== X-Received: by 2002:a17:90a:fb89:: with SMTP id cp9mr11363346pjb.47.1618607066445; Fri, 16 Apr 2021 14:04:26 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:26 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 76/81] target/arm: Remove unused fpst from VDOT_scalar Date: Fri, 16 Apr 2021 14:02:35 -0700 Message-Id: <20210416210240.1591291-77-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::631; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x631.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Cut and paste error from another pattern. Signed-off-by: Richard Henderson --- target/arm/translate-neon.c.inc | 3 --- 1 file changed, 3 deletions(-) -- 2.25.1 diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index 6385d13a7e..c1fbe21ae6 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -316,7 +316,6 @@ static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) { gen_helper_gvec_4 *fn_gvec; int opr_sz; - TCGv_ptr fpst; if (!dc_isar_feature(aa32_dp, s)) { return false; @@ -338,13 +337,11 @@ static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) fn_gvec = a->u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b; opr_sz = (1 + a->q) * 8; - fpst = fpstatus_ptr(FPST_STD); tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->rm), vfp_reg_offset(1, a->vd), opr_sz, opr_sz, a->index, fn_gvec); - tcg_temp_free_ptr(fpst); return true; } From patchwork Fri Apr 16 21:02:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422656 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp746822jaf; Fri, 16 Apr 2021 14:57:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzQmeQ86qmruqfYbQeUipIY6F388FtSemH7v7VkWzwU+zTnFsz3C8KY/bXq1BOa8N1ST6j4 X-Received: by 2002:a02:6d66:: with SMTP id e38mr5982027jaf.69.1618610260527; Fri, 16 Apr 2021 14:57:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610260; cv=none; d=google.com; s=arc-20160816; b=AkDWpI/8gncZPBLD1EiSr9rYm0gW9TMJ1Knfx2SpRJsaBq++564t/myBPwyXPoZIuV tS+JJ8zCuYP/X1AKuMQYPfD12G9KlwDxALYporYWMPhE6O2KAYdoD2RAXv8GQv+QS3J8 IiRwNYIVxxUljx88yrp5Cv40q2G6qtqbrmpqWGKOOl3BxhyPs7I83Gc4hsTgfhQNxblX N31WP0dBj4IkYs9bcT+Hz9EM2g4tuU7Z52rOu+O/9ie+F+0491l6q8WVclfoYutuQUMY 17I6cK9DB/JqBIZpYW5OTux1Q4iw30FjMNxu6FXzfWLXPUhwYTQ76LGO3geoKeXzjssr BIRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9Ga4fLXN7AbcOvfFg3KvJnaRVScnkhscfervqpvFkDI=; b=G2YhkQSYcgnwAoVoXn9OCWZFwx0+u+haU+xePXMwZ2ICAZeVj/3vdY8VTyz9fbGLZC Ecft1hLqOLSySYKtb3OR6lSrxTSIHURsTlkTpFXKXFn/S3aeww6Q7MwSm4kIOX7R6DN+ rXq422tmqKVZSKatlilodXQgqGTR3z2GsX5idyCvLGFMFlDqV48j2qnSr4MLOJP58vnB lQ1cVqHvjODNiKNPvoSHMjZVgQXcFoZrQ26jW5N4y8G+UUpGk9jWHrmq1jp3hrXlo57W w9ifRKmTSrkUzBVNmOYRzCLsIfGd6w6tGSN8DjhXjMcuoxaXpDjLWBA+GQ5lqJ3PIHYM Y/CA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QJAVayBN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l22si7982764jad.0.2021.04.16.14.57.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:57:40 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QJAVayBN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33292 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWT5-0004kE-VC for patch@linaro.org; Fri, 16 Apr 2021 17:57:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdx-0006tR-TK for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:49 -0400 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]:39637) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVdc-0001qF-Kj for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:49 -0400 Received: by mail-pj1-x102e.google.com with SMTP id ot17-20020a17090b3b51b0290109c9ac3c34so17057809pjb.4 for ; Fri, 16 Apr 2021 14:04:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9Ga4fLXN7AbcOvfFg3KvJnaRVScnkhscfervqpvFkDI=; b=QJAVayBNxp5z8raq5Fn66ONMBSlexsWAzBdmcqP7GPdEK7Dab0h61XzV9cM7LHvTEO k3ewNx12X0SNTH7y2eesb0xZeN7rxWLvPzw/uJfBoXG2PLSgalPclHxzCBqmjZzFJCdd RIDe2EqS9zuKez82ROxMJBu+iUp+OO+tvcOXhByyBjPo3c2RwGvimhjU1J7wr3NkUOPA vaV0J7MzJeoOm4Vg9IJ+rRoQfeLtmYki4f/9fVZ+ZtU7POMAqzc+ISERp9AueZDRcNTy GcQ5/yfIcKYrSd8swiEzHeElcrtYFk3my+I/vTKosDC1H96V+zMUf/GVrd3y5pE8P/w1 OSFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9Ga4fLXN7AbcOvfFg3KvJnaRVScnkhscfervqpvFkDI=; b=IfVMcn6Qna3eFtMtoCgEVYWVPkRkMc4HZvx1nS6KtYxyg+k4U1M95V6x8/uT1LwI6D gVHMudbX2ZQ3XaQ1Pc+Udl32bhXdf8JK1J9zFu2qdUgdPimEdcCAkSoWrkOkoGOkP+zk Ph+NsoB3BHLff35WMaUsPyJoApm8rKZWWMNpE1ZEOoLq0+emCeCqO/VA6d0bzSguNLDg Z6rQ/AqQQVa1pMi9dbuUHUksQw6QMw98G0i3BOYzrWFmPKmE5uKs4vIMpMbHCHVI+Bzn jqxIsecT1TkTRf8HLfYynjWk2Z1XOwxO1c/BADk8uHKCvDmv0NkUW5CxnTdj4E5mW3ao JZhA== X-Gm-Message-State: AOAM532uYtZpa4RsqydNscmDim3KK0THFM+sB2oFrefIV1UKzyhXkOZg DwV3QQvqgMGhv0PddRUmCSPyfxMHLsMzbw== X-Received: by 2002:a17:90a:9511:: with SMTP id t17mr11847273pjo.235.1618607067265; Fri, 16 Apr 2021 14:04:27 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:26 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 77/81] target/arm: Fix decode for VDOT (indexed) Date: Fri, 16 Apr 2021 14:02:36 -0700 Message-Id: <20210416210240.1591291-78-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102e; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We were extracting the M register twice, once incorrectly as M:vm and once correctly as rm. Remove the incorrect name and remove the incorrect decode. Signed-off-by: Richard Henderson --- target/arm/neon-shared.decode | 4 +- target/arm/translate-neon.c.inc | 90 ++++++++++++++------------------- 2 files changed, 40 insertions(+), 54 deletions(-) -- 2.25.1 diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index ca0c699072..facb621450 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -61,8 +61,8 @@ VCMLA_scalar 1111 1110 0 . rot:2 .... .... 1000 . q:1 index:1 0 vm:4 \ VCMLA_scalar 1111 1110 1 . rot:2 .... .... 1000 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp size=2 index=0 -VDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 u:1 rm:4 \ - vm=%vm_dp vn=%vn_dp vd=%vd_dp +VDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 u:1 vm:4 \ + vn=%vn_dp vd=%vd_dp %vfml_scalar_q0_rm 0:3 5:1 %vfml_scalar_q1_index 5:1 3:1 diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index c1fbe21ae6..d9901c0153 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -142,6 +142,36 @@ static void neon_store_element64(int reg, int ele, MemOp size, TCGv_i64 var) } } +static bool do_neon_ddda(DisasContext *s, int q, int vd, int vn, int vm, + int data, gen_helper_gvec_4 *fn_gvec) +{ + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (((vd | vn | vm) & 0x10) && !dc_isar_feature(aa32_simd_r32, s)) { + return false; + } + + /* + * UNDEF accesses to odd registers for each bit of Q. + * Q will be 0b111 for all Q-reg instructions, otherwise + * when we have mixed Q- and D-reg inputs. + */ + if (((vd & 1) * 4 | (vn & 1) * 2 | (vm & 1)) & q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + int opr_sz = q ? 16 : 8; + tcg_gen_gvec_4_ool(vfp_reg_offset(1, vd), + vfp_reg_offset(1, vn), + vfp_reg_offset(1, vm), + vfp_reg_offset(1, vd), + opr_sz, opr_sz, data, fn_gvec); + return true; +} + static bool do_neon_ddda_fpst(DisasContext *s, int q, int vd, int vn, int vm, int data, ARMFPStatusFlavour fp_flavor, gen_helper_gvec_4_ptr *fn_gvec_ptr) @@ -232,35 +262,13 @@ static bool trans_VCADD(DisasContext *s, arg_VCADD *a) static bool trans_VDOT(DisasContext *s, arg_VDOT *a) { - int opr_sz; - gen_helper_gvec_4 *fn_gvec; - if (!dc_isar_feature(aa32_dp, s)) { return false; } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - - if ((a->vn | a->vm | a->vd) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - opr_sz = (1 + a->q) * 8; - fn_gvec = a->u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b; - tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), - vfp_reg_offset(1, a->vn), - vfp_reg_offset(1, a->vm), - vfp_reg_offset(1, a->vd), - opr_sz, opr_sz, 0, fn_gvec); - return true; + return do_neon_ddda(s, a->q * 7, a->vd, a->vn, a->vm, 0, + a->u + ? gen_helper_gvec_udot_b + : gen_helper_gvec_sdot_b); } static bool trans_VFML(DisasContext *s, arg_VFML *a) @@ -314,35 +322,13 @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) { - gen_helper_gvec_4 *fn_gvec; - int opr_sz; - if (!dc_isar_feature(aa32_dp, s)) { return false; } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn) & 0x10)) { - return false; - } - - if ((a->vd | a->vn) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - fn_gvec = a->u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b; - opr_sz = (1 + a->q) * 8; - tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), - vfp_reg_offset(1, a->vn), - vfp_reg_offset(1, a->rm), - vfp_reg_offset(1, a->vd), - opr_sz, opr_sz, a->index, fn_gvec); - return true; + return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, + a->u + ? gen_helper_gvec_udot_idx_b + : gen_helper_gvec_sdot_idx_b); } static bool trans_VFML_scalar(DisasContext *s, arg_VFML_scalar *a) From patchwork Fri Apr 16 21:02:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422666 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp754535jaf; Fri, 16 Apr 2021 15:10:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx3hgaIbfo8P9ijXHYcFtfhjIOy+gJfWiNJy2ROa9X98wqN1e2HgA4KYXaIwoet5LL7yFYy X-Received: by 2002:a5d:9659:: with SMTP id d25mr5391595ios.146.1618611022063; Fri, 16 Apr 2021 15:10:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618611022; cv=none; d=google.com; s=arc-20160816; b=d9TM9xFTNF+OQjd3/ghVuP0xhJhKoHIBORei3tyFACLoRqpSGZI3hD8TiKYWbOHq7Q C7NMfP+Ey/F0PPYlMqOPfN+vFJ0CVi8g8vFpehmoV8rIh46NsQ3jcBFJJiefHRyxaWgM t2K+vmyHv7ld6xh3m17+h1sFhC5NzNkwiehjCe+miX1/3VGKVcfFXLCFr7qdxAposCVk 3H5sAnypL1Ngw88XEpSvmya5eyEcdejYi+OxB/HPN0o5Mbl9iL48tUEn0nRmW0X2qzau VJYGQciPMmpbin5rYWl9s31HHrwnH0Lw/W/OMEqrfuHRxS0hjVAfBhPu/WikTBhqn9qa t1CA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=pwtVsWMVmbv9sqp1N/B2+DpRKMYOpBDEtVpM5RexTu0=; b=Kz3cc3jRw2ajunktrTRuRkcS0KH6AvdM43NxMMSc/8v6IluSxBCx/lXVsFfYpMTHFq MdoStJoD9XNEJW5gu+4kAINUmHxla0+VDUj6QFXQPEvLpqzloM9hfUSDj9r9vtMl2Vlk h5QrNeeaPSeSllztep/fa8Iq+T+IDGZkPX1yurOZ78NoA4mYEWwcbVwNuEOJLIngXZA3 GZ1bYDynJrFs6L+8d9APIU4tvvfWyGcX9E/11RTps+1Ko3LkR6pPySNvCm0oz3h2Pl7t a7eTmx/3sS0nC/tniOTeN5g6n35fqhzFxyHUtP8BcLsTUh1t7NLIk/UfEhQ3+pVZoEe/ STqg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=c1n0aLkt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z6si5046854ilb.46.2021.04.16.15.10.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:10:22 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=c1n0aLkt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44124 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWfN-0004ZZ-Fl for patch@linaro.org; Fri, 16 Apr 2021 18:10:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45730) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdy-0006vb-S9 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:50 -0400 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]:39931) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVde-0001rw-0a for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:50 -0400 Received: by mail-pl1-x631.google.com with SMTP id u7so12823361plr.6 for ; Fri, 16 Apr 2021 14:04:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pwtVsWMVmbv9sqp1N/B2+DpRKMYOpBDEtVpM5RexTu0=; b=c1n0aLktOe+6zT06/QQuLNvggpDg+M/p98tmU6Db0PSssv29DVciG0AUjSEVAr4wwe eh+IsBBJO/00ZIY3c+S+2V8h+dtGKn8V1MkSStJy3Ck0jK7S/BC1WMC868So/CASfbGZ Es3t+G1P1VxobcXzlHFvNr94AUnhEzqYB44BSUjHRjqLVR7EQnR2pT/wHiDYEGvsxV99 6aHm6SY6CY9InooK9Up6o435tXBIge7PnIZf8ihZPb5YQQuwfNfrokGp+Aq9OibazJ8V d04i2Q5LlNVJOYlyxLs6kNOLy0AZ48OExF0buzknAWrBANcvN8EoZ4pbKEC/d+mT0JNl e+fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pwtVsWMVmbv9sqp1N/B2+DpRKMYOpBDEtVpM5RexTu0=; b=Lz3Dv9yH4B+2c8lFF1l1R9d6WLfHlnNYoIwR0l7dzGXvGu/9XIGDRCM+uGCTz8sIVI fqwVDyCZpnO3FznIthYErLPUEjbMwgdu/NWO9hNQemxey3AXrnqJB0YpczFT8oN+oQsr 6GTU/zWRyHjLu2wLWVDpEX0p6/lM7e4hCS2blDBej2b0WsKpgtbm4DGE+qHo9USgqopk 9tD4lRrSGwfW29zI2BH/EYEa78cA/kWL/Tpf/jiG7EST/RsOcig1tdhsTQGwLOfsal3C /ynC6QISVT53JqAPWgIhfcXUnWoCL78uJUUshyXq+KffL2DoaKZdjlR5l3p85pufAm2L hjjg== X-Gm-Message-State: AOAM531xXry6nz5Kn1lfBX/DGWT5nxzfBxAsn+sm1TGp/azqbim3ohhU awkMAdwZxACZ4vmon+kzGcI+/hnxKfu8/g== X-Received: by 2002:a17:90a:670a:: with SMTP id n10mr11561574pjj.176.1618607068373; Fri, 16 Apr 2021 14:04:28 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:28 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 78/81] target/arm: Split decode of VSDOT and VUDOT Date: Fri, 16 Apr 2021 14:02:37 -0700 Message-Id: <20210416210240.1591291-79-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::631; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x631.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Now that we have a common helper, sharing decode does not save much. Also, this will solve an upcoming naming problem. Signed-off-by: Richard Henderson --- target/arm/neon-shared.decode | 9 ++++++--- target/arm/translate-neon.c.inc | 30 ++++++++++++++++++++++-------- 2 files changed, 28 insertions(+), 11 deletions(-) -- 2.25.1 diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index facb621450..2d94369750 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -46,8 +46,9 @@ VCMLA 1111 110 rot:2 . 1 . .... .... 1000 . q:1 . 0 .... \ VCADD 1111 110 rot:1 1 . 0 . .... .... 1000 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp size=%vcadd_size -# VUDOT and VSDOT -VDOT 1111 110 00 . 10 .... .... 1101 . q:1 . u:1 .... \ +VSDOT 1111 110 00 . 10 .... .... 1101 . q:1 . 0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp +VUDOT 1111 110 00 . 10 .... .... 1101 . q:1 . 1 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp # VFM[AS]L @@ -61,7 +62,9 @@ VCMLA_scalar 1111 1110 0 . rot:2 .... .... 1000 . q:1 index:1 0 vm:4 \ VCMLA_scalar 1111 1110 1 . rot:2 .... .... 1000 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp size=2 index=0 -VDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 u:1 vm:4 \ +VSDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 0 vm:4 \ + vn=%vn_dp vd=%vd_dp +VUDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 1 vm:4 \ vn=%vn_dp vd=%vd_dp %vfml_scalar_q0_rm 0:3 5:1 diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index d9901c0153..2fd6478d3c 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -260,15 +260,22 @@ static bool trans_VCADD(DisasContext *s, arg_VCADD *a) return true; } -static bool trans_VDOT(DisasContext *s, arg_VDOT *a) +static bool trans_VSDOT(DisasContext *s, arg_VSDOT *a) { if (!dc_isar_feature(aa32_dp, s)) { return false; } return do_neon_ddda(s, a->q * 7, a->vd, a->vn, a->vm, 0, - a->u - ? gen_helper_gvec_udot_b - : gen_helper_gvec_sdot_b); + gen_helper_gvec_sdot_b); +} + +static bool trans_VUDOT(DisasContext *s, arg_VUDOT *a) +{ + if (!dc_isar_feature(aa32_dp, s)) { + return false; + } + return do_neon_ddda(s, a->q * 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_udot_b); } static bool trans_VFML(DisasContext *s, arg_VFML *a) @@ -320,15 +327,22 @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) FPST_STD, gen_helper_gvec_fcmlas_idx); } -static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) +static bool trans_VSDOT_scalar(DisasContext *s, arg_VSDOT_scalar *a) { if (!dc_isar_feature(aa32_dp, s)) { return false; } return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, - a->u - ? gen_helper_gvec_udot_idx_b - : gen_helper_gvec_sdot_idx_b); + gen_helper_gvec_sdot_idx_b); +} + +static bool trans_VUDOT_scalar(DisasContext *s, arg_VUDOT_scalar *a) +{ + if (!dc_isar_feature(aa32_dp, s)) { + return false; + } + return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, + gen_helper_gvec_udot_idx_b); } static bool trans_VFML_scalar(DisasContext *s, arg_VFML_scalar *a) From patchwork Fri Apr 16 21:02:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422659 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp747580jaf; Fri, 16 Apr 2021 14:59:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyytZsoIQVutMlWPd6PNSaQhvQFyMicq7nb7UYksZkp9wD+5E6DcMdYxrQSG4/l1D0IoCVY X-Received: by 2002:a25:7909:: with SMTP id u9mr1713617ybc.22.1618610365151; Fri, 16 Apr 2021 14:59:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610365; cv=none; d=google.com; s=arc-20160816; b=EvEhxPB1sJvsBdXloeSTD4ufymPPEz5rR61gf6K6XYH5ZFjm5TD3p0AnpU4GWmb07/ gzaysieZOBtoQ4/LmmnC6OZuElFggAmbdlhJX+fBOs5b5utMy6obzf30hzoy78PBzMc3 leuzkNlDKlLvxCGSuWXBvCgFkhwg1L2n1j3EX6NKbj5TJjDhHQmp/Gha9FlVUafBimFQ qkq7opADTw9xT1bD6O6FMcxUa+I46YanA+hIZwMdi4VD09D32YOF+BbdQYppQDA4QWQe Z22mevg+F9b1AlqU+H+U6u0R1B6G3qf4jIMaJ9Vd9d6Qwjvsza73v3GGy+N2fRcN5i9n SyIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=JwuGS1PiLu9HVQcfTsdjpr9pKvjxyw9I/bAPxOgmgZ0=; b=NjRWeb5zSB5+d96vmdj6vLqKjaXKzmE7ovVdp6WSsxVRCOLVdOLsvgi2UzlmwF3ncM Eai+U9TupQ9J4Ogemut9ZZwHmkeHQET9KMKpgxdJwasgoIdw5RMFVks8i0Bv2e3N3eDc Vnpyhnqg0gN4f7LzGfaje2MzAIjD/dyTfBYA+9XzLMTOpu0SySOxSHtmTVCXLeqbAYte 74WxMZAPjMpq2RWEwE1ZRsAqGAjW7ZMsGXX8YiO9R5ijI39AWwGmvi91Ri78nYf6yvsV r0eaLDxot71to8glgLCPOkrIWtsGsEYoZ0Uw5PFQ2rKeh3d54dASnCwwyveaQMys9Qrz ++Dw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YnIT1Hkm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y1si7620355ybq.124.2021.04.16.14.59.25 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 14:59:25 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YnIT1Hkm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43040 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWUm-0000CH-LD for patch@linaro.org; Fri, 16 Apr 2021 17:59:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45732) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVdy-0006vd-TZ for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:50 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]:41719) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVde-0001sK-PP for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:04:50 -0400 Received: by mail-pl1-x62d.google.com with SMTP id e2so10328511plh.8 for ; Fri, 16 Apr 2021 14:04:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JwuGS1PiLu9HVQcfTsdjpr9pKvjxyw9I/bAPxOgmgZ0=; b=YnIT1Hkm3P7fhbmLtKoUyLfAkcKd6hBN9pcjOmdOKU+Tu6i9UmPALnFj3ksaJSMQ25 f4/LLVVhiaY1qTZSg/yw9bISLC/1eFXr55MkAdvUgxtMibZblBYnFK+JOq6N+fc6+YIp iViCgNgmvY8YfKEEiz3ANZE7fENGDcEDhCzcsvyg1NGgxI0CA4iCnEojbvh4VFOEMJC6 aGjUCKg/p7fQpxoWRLp4085F4ARSszqSaB24yjjnAkl7tRUE0v3+rMVRqOugIX843vMI TQhhHcw1BdF+czyFdxEbYIFhzJY6ngOEYh0xBJPyyU2Q+r7Wh6em7Ej840NSdSitiLBr MFlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JwuGS1PiLu9HVQcfTsdjpr9pKvjxyw9I/bAPxOgmgZ0=; b=N3CUZsOlb/RSLNXrFGiEEaml7NsheeHisdrKYC0Oc+UBlB3zlN9vfGAdQSaYwLA845 id97UobYPJxrb7MxsFsBifiO1SHYJBxcalndw2y9d9uuetw2ObqvsaSitctNaUgk4YBs vlzhnm6oWTkMVxf2GJ+k2EBdAlcDA+J9JLx/zAG43QE6LreIOl715c8n3cwxT3BjNnZZ l5fEIUjmeSAu7oMGkGWhZhpPLko8O7VF05rSqz165/8V8WedGilnQ3fagpb56ig7G2Gb /5ETe7QN1UfXt0J9/MqMXNibLrJ71QyFnZMQS0JfjPqzu1vl1VGR5UEJr9sZDoZLsuN3 n/Fw== X-Gm-Message-State: AOAM531nIk+tHYK5Jinn9iWy3Xlw15KS67pOqk+0jb1M2M/0c59r13HZ FaGTY1AGUIs1pol5VleMoBGHY8+HjTTZRQ== X-Received: by 2002:a17:90a:df17:: with SMTP id gp23mr11206111pjb.7.1618607069495; Fri, 16 Apr 2021 14:04:29 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id m9sm5766734pgt.65.2021.04.16.14.04.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:04:29 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 79/81] target/arm: Implement aarch32 VSUDOT, VUSDOT Date: Fri, 16 Apr 2021 14:02:38 -0700 Message-Id: <20210416210240.1591291-80-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++++ target/arm/neon-shared.decode | 6 ++++++ target/arm/translate-neon.c.inc | 27 +++++++++++++++++++++++++++ 3 files changed, 38 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index a0865e224c..134dc65e34 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -3783,6 +3783,11 @@ static inline bool isar_feature_aa32_predinv(const ARMISARegisters *id) return FIELD_EX32(id->id_isar6, ID_ISAR6, SPECRES) != 0; } +static inline bool isar_feature_aa32_i8mm(const ARMISARegisters *id) +{ + return FIELD_EX32(id->id_isar6, ID_ISAR6, I8MM) != 0; +} + static inline bool isar_feature_aa32_ras(const ARMISARegisters *id) { return FIELD_EX32(id->id_pfr0, ID_PFR0, RAS) != 0; diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index 2d94369750..5befaec87b 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -50,6 +50,8 @@ VSDOT 1111 110 00 . 10 .... .... 1101 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp VUDOT 1111 110 00 . 10 .... .... 1101 . q:1 . 1 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp +VUSDOT 1111 110 01 . 10 .... .... 1101 . q:1 . 0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp # VFM[AS]L VFML 1111 110 0 s:1 . 10 .... .... 1000 . 0 . 1 .... \ @@ -66,6 +68,10 @@ VSDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 0 vm:4 \ vn=%vn_dp vd=%vd_dp VUDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 1 vm:4 \ vn=%vn_dp vd=%vd_dp +VUSDOT_scalar 1111 1110 1 . 00 .... .... 1101 . q:1 index:1 0 vm:4 \ + vn=%vn_dp vd=%vd_dp +VSUDOT_scalar 1111 1110 1 . 00 .... .... 1101 . q:1 index:1 1 vm:4 \ + vn=%vn_dp vd=%vd_dp %vfml_scalar_q0_rm 0:3 5:1 %vfml_scalar_q1_index 5:1 3:1 diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index 2fd6478d3c..c322615915 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -278,6 +278,15 @@ static bool trans_VUDOT(DisasContext *s, arg_VUDOT *a) gen_helper_gvec_udot_b); } +static bool trans_VUSDOT(DisasContext *s, arg_VUSDOT *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, a->q * 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_usdot_b); +} + static bool trans_VFML(DisasContext *s, arg_VFML *a) { int opr_sz; @@ -345,6 +354,24 @@ static bool trans_VUDOT_scalar(DisasContext *s, arg_VUDOT_scalar *a) gen_helper_gvec_udot_idx_b); } +static bool trans_VUSDOT_scalar(DisasContext *s, arg_VUSDOT_scalar *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, + gen_helper_gvec_usdot_idx_b); +} + +static bool trans_VSUDOT_scalar(DisasContext *s, arg_VSUDOT_scalar *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, + gen_helper_gvec_sudot_idx_b); +} + static bool trans_VFML_scalar(DisasContext *s, arg_VFML_scalar *a) { int opr_sz; From patchwork Fri Apr 16 21:18:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422664 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp752534jaf; Fri, 16 Apr 2021 15:07:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw/9IGl1+dHPtr+8GZK3Z1O2CUlzau/peqrJ/jrm6WzOEbjJtrthRUIw/SbVM2RkMu0qpgG X-Received: by 2002:a05:6102:358:: with SMTP id e24mr9053374vsa.53.1618610831551; Fri, 16 Apr 2021 15:07:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618610831; cv=none; d=google.com; s=arc-20160816; b=qkbwOVVcUrk2pWu9azjeCOAheNl8ytJJiBFi254hICvsehfqbSbcZE/KmxPuedRrDU GujsC0GqO1C5dbqebz1Gm+e9HvUAOJ5VMmqNEU93bhan/5aMmdfK4V0w0feTAvWAw80z u4Bji8FOg2TCYXZHnAkSyOy3VHu6ba/OsBhQGI+a6xk/mcn73owIIqd627oABcj4SQfI 0Hd+boCMnLQ898jKxm7LVXYvPhVeDJqPBrOihn4ayDe490IvAh72ubDYoDaEcEqwpwlR zLmNz0tynLmQZCsu6XIb5kmRYguP2nCbfMOUQFM580fr3kWVyRslIb/PZ11A2knnUdXW 1dZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=l2DdQUpZAAYz/+QgM/RAZaDAb6uzfP9zN/oGnx1vuJ0=; b=yDqZjnDilMmYK92BrTXp2zSPo1q7OoLmyy07Jf+2imJBd3XbIw4jtW3RvjL/85bYXe ZHxffqC2n3tBDPT/hSACMDxgW5URdgmYb6dv8vngdPuiJ7CGg6Ll9FWupTw9TLZlKzpv m7NaJXgxx6Ih9S0pz9DkPtn+I/qxJzyj39U81lplPABCy60nolBwl7yYB1jaXvV1UMQh qlJmnAytnxB+KxyLbhg7lVPbYjfhIb9RDKU3OsVQza4anMu5LLQa4eXSzBscwKE4c9vU PbEA8AmmOQ/L6yaVEavFBhW8Wu7xFOg3eFLExU8rqPdTFjW2xI0bERkGpCeTphaWxCos +DPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="d6S/D6ma"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w1si3415898uan.170.2021.04.16.15.07.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:07:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="d6S/D6ma"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33934 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWcI-0008Hi-TK for patch@linaro.org; Fri, 16 Apr 2021 18:07:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47882) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVrf-0000Dm-7I for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:18:59 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]:46927) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVrd-0001OA-0S for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:18:58 -0400 Received: by mail-pf1-x42d.google.com with SMTP id d124so19127929pfa.13 for ; Fri, 16 Apr 2021 14:18:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l2DdQUpZAAYz/+QgM/RAZaDAb6uzfP9zN/oGnx1vuJ0=; b=d6S/D6mahlOitw85J2YwwWMKTyowdfSdr+eBMmK9uYpxajtdSzqExbKEzCcGaEAx7x Y0/Dj1PqLe/eSQofsorTFC599rtbvWXQVlMKaEHo4vPldg9wqdi5O6zb0C40HAvWWHST 9VV1Z3cb4RQutvrJnIhIg8L0GudKMJyE2CUr3aq+B8UnCAAu9R8cHSRk2E3RacBYoLgV +kEspkZqX7pSgRBLEuhoWYQsbC2Ea1+loJiIla7adtj2b04JnajiV7vOAwQuP001OMrF hv8dYp4fA/At130fCNayF6RV95eWrx/3YCC4CwuCj4JEts4HIf+okKhGIlu7vwK/YHHM wKmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l2DdQUpZAAYz/+QgM/RAZaDAb6uzfP9zN/oGnx1vuJ0=; b=f5qyKB9Z1PI0SVZsZ5rmb4rGtSs0vPz9YCll8DNhSUEq7dHjxUQGvT7AvE8lwawSyT MH9lrGH90ZLy+3cZJFBVMmyUOLc7xj4E8m9yHvydcPZ76EANs7q1BYw1RpO1AElfQ55H OdIs1tZ9L8h14yHSCSi+vyMiyYGioULP6tWKMUgfPf4bSQhR/1HpNCnQt/+kCRxlf1qV /nzk4Cm3xYwpvPfXw3L4lRhPG13mYHeuCftVC55I2y8eguKC98uMwMFBW1y+fDYKREBa 6WamBpUib4SQAzZFMd3geCUGXUA2XRzRVmLI7IokZwRX28Eows3uzifqdsIocLW4H8U8 gX+A== X-Gm-Message-State: AOAM53128CFhbbOmee+keZ29xuGn7/+JygyWx5UarCaxeO/TGOk2RkUp zQTSnm4K3U43QYvn5bojwx8L9uUk/uPI7A== X-Received: by 2002:a63:c042:: with SMTP id z2mr909471pgi.136.1618607935267; Fri, 16 Apr 2021 14:18:55 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id r127sm5586266pfr.163.2021.04.16.14.18.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:18:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 80/81] target/arm: Implement integer matrix multiply accumulate Date: Fri, 16 Apr 2021 14:18:53 -0700 Message-Id: <20210416211853.1591708-1-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This is {S,U,US}MMLA for both AArch64 AdvSIMD and SVE, and V{S,U,US}MMLA.S8 for AArch32 NEON. Signed-off-by: Richard Henderson --- target/arm/helper.h | 7 +++ target/arm/neon-shared.decode | 7 +++ target/arm/sve.decode | 6 +++ target/arm/translate-a64.c | 18 ++++++++ target/arm/translate-sve.c | 27 ++++++++++++ target/arm/vec_helper.c | 77 +++++++++++++++++++++++++++++++++ target/arm/translate-neon.c.inc | 27 ++++++++++++ 7 files changed, 169 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index e8b16a401f..33df62f44d 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -994,6 +994,13 @@ DEF_HELPER_FLAGS_6(sve2_fmlal_zzxw_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_smmla_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_ummla_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_usmmla_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + #ifdef TARGET_AARCH64 #include "helper-a64.h" #include "helper-sve.h" diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index 5befaec87b..cc9f4cdd85 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -59,6 +59,13 @@ VFML 1111 110 0 s:1 . 10 .... .... 1000 . 0 . 1 .... \ VFML 1111 110 0 s:1 . 10 .... .... 1000 . 1 . 1 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp q=1 +VSMMLA 1111 1100 0.10 .... .... 1100 .1.0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp +VUMMLA 1111 1100 0.10 .... .... 1100 .1.1 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp +VUSMMLA 1111 1100 1.10 .... .... 1100 .1.0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp + VCMLA_scalar 1111 1110 0 . rot:2 .... .... 1000 . q:1 index:1 0 vm:4 \ vn=%vn_dp vd=%vd_dp size=1 VCMLA_scalar 1111 1110 1 . rot:2 .... .... 1000 . q:1 . 0 .... \ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 63870b7539..3d7c4fa6e5 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1364,6 +1364,12 @@ USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl EORBT 01000101 .. 0 ..... 10010 0 ..... ..... @rd_rn_rm EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm +## SVE integer matrix multiply accumulate + +SMMLA 01000101 00 0 ..... 10011 0 ..... ..... @rda_rn_rm_e0 +USMMLA 01000101 10 0 ..... 10011 0 ..... ..... @rda_rn_rm_e0 +UMMLA 01000101 11 0 ..... 10011 0 ..... ..... @rda_rn_rm_e0 + ## SVE2 bitwise permute BEXT 01000101 .. 0 ..... 1011 00 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 0d45a44f51..668edf3a00 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -12197,6 +12197,15 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) } feature = dc_isar_feature(aa64_i8mm, s); break; + case 0x04: /* SMMLA */ + case 0x14: /* UMMLA */ + case 0x05: /* USMMLA */ + if (!is_q || size != MO_32) { + unallocated_encoding(s); + return; + } + feature = dc_isar_feature(aa64_i8mm, s); + break; case 0x18: /* FCMLA, #0 */ case 0x19: /* FCMLA, #90 */ case 0x1a: /* FCMLA, #180 */ @@ -12241,6 +12250,15 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, 0, gen_helper_gvec_usdot_b); return; + case 0x04: /* SMMLA, UMMLA */ + gen_gvec_op4_ool(s, 1, rd, rn, rm, rd, 0, + u ? gen_helper_gvec_ummla_b + : gen_helper_gvec_smmla_b); + return; + case 0x05: /* USMMLA */ + gen_gvec_op4_ool(s, 1, rd, rn, rm, rd, 0, gen_helper_gvec_usmmla_b); + return; + case 0x8: /* FCMLA, #0 */ case 0x9: /* FCMLA, #90 */ case 0xa: /* FCMLA, #180 */ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ae628968da..cb0e7a1f68 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8543,3 +8543,30 @@ static bool trans_FMLSLT_zzxw(DisasContext *s, arg_rrxr_esz *a) { return do_FMLAL_zzxw(s, a, true, true); } + +static bool do_i8mm_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, + gen_helper_gvec_4 *fn, int data) +{ + if (!dc_isar_feature(aa64_sve_i8mm, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); + } + return true; +} + +static bool trans_SMMLA(DisasContext *s, arg_rrrr_esz *a) +{ + return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_smmla_b, 0); +} + +static bool trans_USMMLA(DisasContext *s, arg_rrrr_esz *a) +{ + return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_usmmla_b, 0); +} + +static bool trans_UMMLA(DisasContext *s, arg_rrrr_esz *a) +{ + return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_ummla_b, 0); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index ea08b15c55..6c9f1e5146 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -2578,3 +2578,80 @@ void HELPER(gvec_xar_d)(void *vd, void *vn, void *vm, uint32_t desc) } clear_tail(d, opr_sz * 8, simd_maxsz(desc)); } + +/* + * Integer matrix-multiply accumulate + */ + +static uint32_t do_smmla_b(uint32_t sum, void *vn, void *vm) +{ + int8_t *n = vn, *m = vm; + + for (intptr_t k = 0; k < 8; ++k) { + sum += n[H1(k)] * m[H1(k)]; + } + return sum; +} + +static uint32_t do_ummla_b(uint32_t sum, void *vn, void *vm) +{ + uint8_t *n = vn, *m = vm; + + for (intptr_t k = 0; k < 8; ++k) { + sum += n[H1(k)] * m[H1(k)]; + } + return sum; +} + +static uint32_t do_usmmla_b(uint32_t sum, void *vn, void *vm) +{ + uint8_t *n = vn; + int8_t *m = vm; + + for (intptr_t k = 0; k < 8; ++k) { + sum += n[H1(k)] * m[H1(k)]; + } + return sum; +} + +static void do_mmla_b(void *vd, void *vn, void *vm, void *va, uint32_t desc, + uint32_t (*inner_loop)(uint32_t, void *, void *)) +{ + intptr_t seg, opr_sz = simd_oprsz(desc); + + for (seg = 0; seg < opr_sz; seg += 16) { + uint32_t *d = vd + seg; + uint32_t *a = va + seg; + uint32_t sum0, sum1, sum2, sum3; + + /* + * Process the entire segment at once, writing back the + * results only after we've consumed all of the inputs. + * + * Key to indicies by column: + * i j i j + */ + sum0 = a[H4(0 + 0)]; + sum0 = inner_loop(sum0, vn + seg + 0, vm + seg + 0); + sum1 = a[H4(0 + 1)]; + sum1 = inner_loop(sum1, vn + seg + 0, vm + seg + 8); + sum2 = a[H4(2 + 0)]; + sum2 = inner_loop(sum2, vn + seg + 8, vm + seg + 0); + sum3 = a[H4(2 + 1)]; + sum3 = inner_loop(sum3, vn + seg + 8, vm + seg + 8); + + d[H4(0)] = sum0; + d[H4(1)] = sum1; + d[H4(2)] = sum2; + d[H4(3)] = sum3; + } + clear_tail(vd, opr_sz, simd_maxsz(desc)); +} + +#define DO_MMLA_B(NAME, INNER) \ + void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ + { do_mmla_b(vd, vn, vm, va, desc, INNER); } + +DO_MMLA_B(gvec_smmla_b, do_smmla_b) +DO_MMLA_B(gvec_ummla_b, do_ummla_b) +DO_MMLA_B(gvec_usmmla_b, do_usmmla_b) diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index c322615915..f1893b1dc8 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -4027,3 +4027,30 @@ static bool trans_VTRN(DisasContext *s, arg_2misc *a) tcg_temp_free_i32(tmp2); return true; } + +static bool trans_VSMMLA(DisasContext *s, arg_VSMMLA *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_smmla_b); +} + +static bool trans_VUMMLA(DisasContext *s, arg_VUMMLA *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_ummla_b); +} + +static bool trans_VUSMMLA(DisasContext *s, arg_VUSMMLA *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_usmmla_b); +} From patchwork Fri Apr 16 21:19:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 422672 Delivered-To: patch@linaro.org Received: by 2002:a02:6a6f:0:0:0:0:0 with SMTP id m47csp762693jaf; Fri, 16 Apr 2021 15:24:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxKAaQ4o8dG7J6bOWGm0WsY+zdnNfOpnHGsdBY3ZiU/C3wDJXqvYzAp4OUz+7OO/t9q6eph X-Received: by 2002:a5b:144:: with SMTP id c4mr1882608ybp.136.1618611898340; Fri, 16 Apr 2021 15:24:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1618611898; cv=none; d=google.com; s=arc-20160816; b=naBBHUNycm5mzc7xbLgBlspCX8S4RvJTp5R07HMRKJ4N9XoGN5xpJLsoNel3YqeuTu hAP1zDSVA3IsvM7Uo8AAOrdTZ/hG9kY/wb2mmoWNDKA5EaMg5YJMUDOcOmI0Ln8Rdmg3 M7OuV9MKh0hFEvFJtwEg82ATWX1QR94iZTro5HxCEPgRoLP49JedrtZFHyjzqMXkxbbF IBtKqA/76FUQttpbHucaj/rZTa42EhNs9141CXQ+GRFU+j0j3RByTgYG2MfAiacTecM8 XGK7XQ5fpMycGgUUNamAnbM1Rmg1UBpF9Ape2NvQs39FR8zilxQkvr/P2XZ+CIVdnFR9 z3rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=0MbRdnRJn0x9uVTH+qro7/oyKU7eLBQcZeSxN6NVvH4=; b=1GiWPECVsK88rxBH/0TInJhtef8njO2goYFRsdYJtggGZdFxsAlF4QUea7RPbcm9It mRfckYblAGX36RBofkvkxCi/IZpv2oR+Sfg7dNcWM05KaB8J67nacFpjOeLs+Seq/oyg mIy7Xtz0qIo+pNCgJ1z65N6Jb2Dld7Um/dIX0j2KXGxmzJK9e+u5Sf6DDpjDy3Xr6nmw Vu5kxkhvZrs3NuvvBCfk+9pinvjg10wTxodiOoyZf3f9gruyUfWlijSpVxVPBI2u88My ij8gLqRhnw5/Ua2nmpZnJTjor9Gvhu/GiNz9B+CJbAlcwYTsI5ZNrqcs52oE+I0U5bgD Rrvg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qaXmsrGp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s76si7068720ybc.416.2021.04.16.15.24.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 16 Apr 2021 15:24:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qaXmsrGp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53286 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lXWtV-0003Qi-Rb for patch@linaro.org; Fri, 16 Apr 2021 18:24:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47978) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lXVsJ-0001IK-15 for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:19:39 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]:37557) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lXVsH-0001kO-AY for qemu-devel@nongnu.org; Fri, 16 Apr 2021 17:19:38 -0400 Received: by mail-pf1-x432.google.com with SMTP id o123so19166532pfb.4 for ; Fri, 16 Apr 2021 14:19:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0MbRdnRJn0x9uVTH+qro7/oyKU7eLBQcZeSxN6NVvH4=; b=qaXmsrGpI9i5K8Edy6l7VAnAbkhP/t+MCdGi+211l8uLRVZYWu38GBc8aq5kqgtCTR 8qPnoZlRs2qP36+W9LhuS/8sUhpsvNuGRUAyKQXYHKTO8340E5+QFkZ+7G4sMTakAaqI 2VUWK1ZlLN7N8pu1EvXmuFzAYDjILxnUVaN9btg6rB1kp7Z0OyfRVJELzMsIK91+SqT8 0iRrllnrFHVAVF4lfMZFWGm/lXYnw/5MEmqbt7gQTTuUOKT4pPJ7fTDPGbzKlPgMxwYj SwieOtpt0EV+HkqPPdi4DB0EAK7oBQqA5FVd4tNMOv86uSBQmng5alA4XA1Cms2vKoLo iJxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0MbRdnRJn0x9uVTH+qro7/oyKU7eLBQcZeSxN6NVvH4=; b=Ulc9OZdBCX0QUf4UGi0+fhAHIPytKe2z5SKsASoam40zvWZAZVav8tD5I+Yph8eq9x MIc3psj7nuXsBRkbcc3X4UP7Dxveq8Y7WivA+L5anb+ZySOC5VNN2s/FFp74vmpBoIT8 mS2pvFJjSH9rVHji6qGgBz0ntzkaArBdPVC5ygDGYHkRW+hxO2CSskCA7mOAXQOwtDcj ZMZZVQ6L5c1orduhBilpGXBfemAwFG6x+r2aRYtmfd0CluGjZQ+LnqrEHcCi4VCIeoTu iS9y7V4muw32sZkePC1D6kijJLdptO5g9fnlgz88SFI+CQBTcG58prRlU69Fd56CjwVY 1Eng== X-Gm-Message-State: AOAM533O8Fqj2R1s5VaiNhPffDQFI2boKunj2KQ338v9e5DAvUhKHA4c LdIqBBa/LkgZ5GLa86yqjw8ZDyQU2vq+0w== X-Received: by 2002:a62:4e96:0:b029:248:effc:9a4d with SMTP id c144-20020a624e960000b0290248effc9a4dmr9428224pfb.71.1618607976014; Fri, 16 Apr 2021 14:19:36 -0700 (PDT) Received: from localhost.localdomain ([71.212.131.83]) by smtp.gmail.com with ESMTPSA id i28sm5648479pgi.42.2021.04.16.14.19.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Apr 2021 14:19:35 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v5 81/81] target/arm: Enable SVE2 and some extensions Date: Fri, 16 Apr 2021 14:19:34 -0700 Message-Id: <20210416211934.1591764-1-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416210240.1591291-1-richard.henderson@linaro.org> References: <20210416210240.1591291-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.c | 1 + target/arm/cpu64.c | 13 +++++++++++++ 2 files changed, 14 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.c b/target/arm/cpu.c index 0dd623e590..30fd5d5ff7 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -1464,6 +1464,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp) u = cpu->isar.id_isar6; u = FIELD_DP32(u, ID_ISAR6, JSCVT, 0); + u = FIELD_DP32(u, ID_ISAR6, I8MM, 0); cpu->isar.id_isar6 = u; u = cpu->isar.mvfr0; diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index f0a9e968c9..379f90fab8 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -662,6 +662,7 @@ static void aarch64_max_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64ISAR1, SPECRES, 1); t = FIELD_DP64(t, ID_AA64ISAR1, FRINTTS, 1); t = FIELD_DP64(t, ID_AA64ISAR1, LRCPC, 2); /* ARMv8.4-RCPC */ + t = FIELD_DP64(t, ID_AA64ISAR1, I8MM, 1); cpu->isar.id_aa64isar1 = t; t = cpu->isar.id_aa64pfr0; @@ -702,6 +703,17 @@ static void aarch64_max_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64MMFR2, ST, 1); /* TTST */ cpu->isar.id_aa64mmfr2 = t; + t = cpu->isar.id_aa64zfr0; + t = FIELD_DP64(t, ID_AA64ZFR0, SVEVER, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, AES, 2); /* PMULL */ + t = FIELD_DP64(t, ID_AA64ZFR0, BITPERM, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, SHA3, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, SM4, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, I8MM, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, F32MM, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, F64MM, 1); + cpu->isar.id_aa64zfr0 = t; + /* Replicate the same data to the 32-bit id registers. */ u = cpu->isar.id_isar5; u = FIELD_DP32(u, ID_ISAR5, AES, 2); /* AES + PMULL */ @@ -718,6 +730,7 @@ static void aarch64_max_initfn(Object *obj) u = FIELD_DP32(u, ID_ISAR6, FHM, 1); u = FIELD_DP32(u, ID_ISAR6, SB, 1); u = FIELD_DP32(u, ID_ISAR6, SPECRES, 1); + u = FIELD_DP32(u, ID_ISAR6, I8MM, 1); cpu->isar.id_isar6 = u; u = cpu->isar.id_pfr0;