From patchwork Tue Mar 9 16:19:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396016 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2766084jai; Tue, 9 Mar 2021 09:19:12 -0800 (PST) X-Google-Smtp-Source: ABdhPJzR11MQtFgZ7S8UoVohCl4POCBY2q8UCFHdhoPnMfufyoqHCQfXVTfuUHlN9sTOcwvUNYOo X-Received: by 2002:a02:3ec7:: with SMTP id s190mr29489867jas.11.1615310352716; Tue, 09 Mar 2021 09:19:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615310352; cv=none; d=google.com; s=arc-20160816; b=M0LMiIqLYLzBILXvVL5VP2mhIPd9+UC5uSkrZz+ttuK0hqrXwV3CWPb6C53o290O8B qnZmsR/AtHiLGGTMtI2lOWOTDHp2VIgfygYkmt2bT1yqu4eZ6Z2Qf1lwfCi3WoWB+i48 ACdb3f4gSzMwCq3PSI6tO/0ZFL2mykOsLws+LRgpM1gfNXwSfCMysE7eTA/8Ssz2i6g+ l5kZZGxS75iqHsgJwmWNrAVJY/8gQ/to8gKi1uAiId3P7aUH6ebqIkDmeJBrNU54Z+yL Quf1FPssintqc8QzrmvgnvCQQ2vLwgDJTT/oL/k4GtorpIRGSPdIms7g+qaxk92n2IzY vBaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=hYJXwe4FbJfgTdVPMN7My2daIn3Bg5z6ffUixEKobJI=; b=tDoWbwVkVzN0C/sXZMt/Jm/3SbjW+2S+iUKECplu2dd9+4N5bHgeQkIwlbKb9G68JU Qd3vcQCkA3h/n+En9+0P9yXyXwlLiahEMXsQkmTJt0B3cOtZkbcUjuvCqr7bDNPF6cPD XdcMtl/BLalt4mYhw0SbwQkNjUk93mTu2LmqeX/NGGDiG9177HEfUJUZHdcGvEN+kQSc XJjQME2GGMIANRV4NtS5ktuTb7cHMLb35udUrJJlXX74GfcgXUH2UUDA+ou0rMu8BQ7s 0LclSAM6p9Z7TFsQF+K4eJWNHeHCjnjSxlNe9YHzlM6D2fbnu1oP10bGBKXjVxb9Gv37 nfKA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=H3usu8MF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v7si13275281ilu.80.2021.03.09.09.19.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:19:12 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=H3usu8MF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57288 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJg0m-0001Yb-1r for patch@linaro.org; Tue, 09 Mar 2021 12:19:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45114) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6L-0006w5-3t for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:54 -0500 Received: from mail-oi1-x22e.google.com ([2607:f8b0:4864:20::22e]:35192) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6E-0002Nh-F4 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:52 -0500 Received: by mail-oi1-x22e.google.com with SMTP id u6so8187521oic.2 for ; Tue, 09 Mar 2021 08:20:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hYJXwe4FbJfgTdVPMN7My2daIn3Bg5z6ffUixEKobJI=; b=H3usu8MFzXk95EuTUYgzbCBYkuTSnaT2kJRw72Y3R+tuSgAz56SxHuGNH7gdhC2A5X 20V3EyL2kLhB+8d6dHZjLP5ErtnVhQNws2df7nyTdpBn35sIuq6oHzv4ZgngRnSfWWYT 0vQiYV/mLk+MuGF/kc92nIWO3gV9/mu/5T5xmqp8iHJ62HIhIRukH+rkzfp2C3Ls3vbC H8y3gDXIMA+zt9iKmT1fl60kydLpCVeXYyuXzb8t4xHfQNcgLheC4UhdbcgI/UTgjnwA /EQqoipyJ0FoXdYRbdNczgGpZ+wE63xSijN99T4cNMZdeKgQOIU+tIIEABYiw1pbYWFA ilzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hYJXwe4FbJfgTdVPMN7My2daIn3Bg5z6ffUixEKobJI=; b=Vk0eCW7MqyOehGNW2ShCcWYICnet52AHZEfz83fFdxRsRHstYIY/w3qhuAxxDYuShS bJb4T0mUhB9454sSRMq621YtjDOml2icw/eYDIWuJdBhixPA7wpzeRb6UT4B8Z43dv4N Up8obOLLryOc4+rm/SbApQflaR52tej6zaOSF4kLFECZqKBme9wTDMCovvXi4DyPQ98P I5qYUm+CTBTdJXdcacBBJw39WmBMBbUYmmxM7F+W8lAJoeaaY4m43oj9tArzlSKqvoEK Y8BO6lFtySZNzDPuq1A0/33QlreHNKAqcvJPz+VGOBLqrsiMluImz+z827zmddl+Fyki C1jQ== X-Gm-Message-State: AOAM531D9DAHR21kMfi1YKY7k60li+2SUqB2VKNHpQNTD/2zDn7c2cFX untohp5X7J/7YwbhuUpbkpN94SLjKISB7clZ X-Received: by 2002:aca:de82:: with SMTP id v124mr3481050oig.125.1615306844573; Tue, 09 Mar 2021 08:20:44 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:44 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 01/78] target/arm: Add ID_AA64ZFR0 fields and isar_feature_aa64_sve2 Date: Tue, 9 Mar 2021 08:19:24 -0800 Message-Id: <20210309162041.23124-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22e; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Will be used for SVE2 isa subset enablement. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- v2: Do not read zfr0 from kvm unless sve is available. --- target/arm/cpu.h | 16 ++++++++++++++++ target/arm/helper.c | 3 +-- target/arm/kvm64.c | 11 +++++++++++ 3 files changed, 28 insertions(+), 2 deletions(-) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 193a49ec7f..ee92ea876e 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -942,6 +942,7 @@ struct ARMCPU { uint64_t id_aa64mmfr2; uint64_t id_aa64dfr0; uint64_t id_aa64dfr1; + uint64_t id_aa64zfr0; } isar; uint64_t midr; uint32_t revidr; @@ -2029,6 +2030,16 @@ FIELD(ID_AA64DFR0, DOUBLELOCK, 36, 4) FIELD(ID_AA64DFR0, TRACEFILT, 40, 4) FIELD(ID_AA64DFR0, MTPMU, 48, 4) +FIELD(ID_AA64ZFR0, SVEVER, 0, 4) +FIELD(ID_AA64ZFR0, AES, 4, 4) +FIELD(ID_AA64ZFR0, BITPERM, 16, 4) +FIELD(ID_AA64ZFR0, BFLOAT16, 20, 4) +FIELD(ID_AA64ZFR0, SHA3, 32, 4) +FIELD(ID_AA64ZFR0, SM4, 40, 4) +FIELD(ID_AA64ZFR0, I8MM, 44, 4) +FIELD(ID_AA64ZFR0, F32MM, 52, 4) +FIELD(ID_AA64ZFR0, F64MM, 56, 4) + FIELD(ID_DFR0, COPDBG, 0, 4) FIELD(ID_DFR0, COPSDBG, 4, 4) FIELD(ID_DFR0, MMAPDBG, 8, 4) @@ -4182,6 +4193,11 @@ static inline bool isar_feature_aa64_ssbs(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, SSBS) != 0; } +static inline bool isar_feature_aa64_sve2(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SVEVER) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper.c b/target/arm/helper.c index 904b0927cd..48c00de6ca 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -7561,8 +7561,7 @@ void register_cp_regs_for_features(ARMCPU *cpu) .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 4, .access = PL1_R, .type = ARM_CP_CONST, .accessfn = access_aa64_tid3, - /* At present, only SVEver == 0 is defined anyway. */ - .resetvalue = 0 }, + .resetvalue = cpu->isar.id_aa64zfr0 }, { .name = "ID_AA64PFR5_EL1_RESERVED", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 5, .access = PL1_R, .type = ARM_CP_CONST, diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index dff85f6db9..e8729b53fd 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -567,6 +567,17 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf) err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64mmfr2, ARM64_SYS_REG(3, 0, 0, 7, 2)); + /* + * Before v5.1, KVM did not support SVE and did not expose + * ID_AA64ZFR0_EL1 even as RAZ. After v5.1, KVM still does + * not expose the register to "user" requests like this + * unless the host supports SVE. + */ + if (isar_feature_aa64_sve(&ahcf->isar)) { + err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64zfr0, + ARM64_SYS_REG(3, 0, 0, 4, 4)); + } + /* * Note that if AArch32 support is not present in the host, * the AArch32 sysregs are present to be read, but will From patchwork Tue Mar 9 16:19:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396017 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2769193jai; Tue, 9 Mar 2021 09:23:03 -0800 (PST) X-Google-Smtp-Source: ABdhPJzxelzydespGi5hTNjhhyKoqjlzM0DlCk5H11GjmmCa81JpBf8HDmOxM+5ZXsYqdKHXJcqB X-Received: by 2002:a67:ec94:: with SMTP id h20mr3038366vsp.51.1615310583287; Tue, 09 Mar 2021 09:23:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615310583; cv=none; d=google.com; s=arc-20160816; b=MBCPIvX4gJeP6yIdK1LGS1hd0ntzNYLoX6bhKd8aPAEic/ilsGMZnMP9uEbJbUwjUe 8I35I8LNEoQA+tGqUD7FVDdhWlWhWlCWm+drjJ0T93KRDuRXJfaT0F4L9jUjWWjBhW10 sP2qTiaVIm6TQ3eNKwxTCSddTOi7ETBSQqITg/74S/YOxD8/90y7CuG8PYKIqidYTLLE pSEwA9ECWH3ufT1ANZBSNB90xCxppMK5sK/EqOsZ6VefMlwPv4AgE5Dywl0NXuCtIZb3 Sz//hT7o1JxRdcNQI+/pnBSI2paTwYGlGmjCQhaQvq6CcFLydwyKdwiYOCokAPqVzGYG mLhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=HvSrOl22actPyuRTqNBVMMjUXh2g8f+fmwTAA2vQd8U=; b=dwNGXVVWHMeBFugQN636PY6NqspA4eMsZNlXw2t+o2Ib+4GFux9XDeY/9i3UgbQ+It lQpySF9P8RO+MhYpapr6cEnlS/hXg0gSPxY/QrB8WinGqCZ2ZL0C3F49+ijlKCCidEYO E3tPMVpR7ZJm/03r2oFrjqGQW0P3f3PUCTPa93HHsWi1/N5l5IPGJu0qcckhAl8KkIrR LNumDeP6soNAWLTGQEoEfRLZfnGiyjcDrtUNo5E2CxPwjP0pXd4b1NR9Dg5c4GDxJVQS DvEay7SnqmzTPxuUDsRTXHy1Fn9b0V0T79p/EPjWT2cNvkaQMFk6ZaV+uKzmtCAef7q/ rDww== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RifNVV1X; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n14si2271029ual.82.2021.03.09.09.23.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:23:03 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RifNVV1X; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:32836 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJg4U-0003Ho-JJ for patch@linaro.org; Tue, 09 Mar 2021 12:23:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45118) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6L-0006w6-7x for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:54 -0500 Received: from mail-oo1-xc29.google.com ([2607:f8b0:4864:20::c29]:35187) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6E-0002Np-SE for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:53 -0500 Received: by mail-oo1-xc29.google.com with SMTP id p6so3179182oot.2 for ; Tue, 09 Mar 2021 08:20:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HvSrOl22actPyuRTqNBVMMjUXh2g8f+fmwTAA2vQd8U=; b=RifNVV1XV9YWWu7edWc6wDDTJVbiIIxQ0RYQW+UjFnzzgAfrPbqmaM69sHcF98E2C8 uyImPK/VDNSa6h4KUM+khu3/MAiIiVb7dt71MrFfmWATntgW7/KvevOqZGiBfixukDy3 JROLtRSiQ3ElLc94t6qrvbcITF44LuyEYAridz0utSl43X4+SYlfspsKJb3uytpijxD6 qXBQl8OTDxwBhwX5D0kcMaWVDC3mm0wtSXVNCUmNtcj/xz4Tcs3h1Alm2B3jBxoLxgkM 07TNhKr96ga4iNgSqnog+rSN3EihnMlDPAxnsJwAG8p5s3W/adLQEbiW9fN4NVPVyKmd LfJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HvSrOl22actPyuRTqNBVMMjUXh2g8f+fmwTAA2vQd8U=; b=KNkYsPYq85QVmF5/RmVlvvKk5CBESM7Yh5tqjaNlVzT8MkAnAuuDKU2iPAE3NjxOmb 25a+Rhcuz8KmbZJOP/lZZI6vR2KVb/+yUe0nnUHFl52gleRUHZgXw4aVWjyOvmcHDjxE +rhuJ578VQXoWfpWmCQt+rRQy6dmK/uq9jAxcreSoHV8v8eQheJB1THVicDZbE6q3lA1 Lh1K2c3wdB3db8b1UyfB9wjzPCR4xiQU3/90mpY0dXHpFp0d+lXi6/yRWPPs3CQTllsQ hp8d/5yyd1RpSHVhRpTPVB8eLoHDzO0iiJsDzhjYg3VYdjN/MwnkJntblq6m24F7ROIa pamQ== X-Gm-Message-State: AOAM531OM1L/StRIeuho6bDRFXPDDpee6Y81nQRQe13CGI5rekuTM08p f/50kyYnrl5Ri6U7sMS2yN9mzfehHXZCLNi4 X-Received: by 2002:a4a:b787:: with SMTP id a7mr23199487oop.18.1615306845733; Tue, 09 Mar 2021 08:20:45 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:45 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 02/78] target/arm: Implement SVE2 Integer Multiply - Unpredicated Date: Tue, 9 Mar 2021 08:19:25 -0800 Message-Id: <20210309162041.23124-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c29; envelope-from=richard.henderson@linaro.org; helo=mail-oo1-xc29.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For MUL, we can rely on generic support. For SMULH and UMULH, create some trivial helpers. For PMUL, back in a21bb78e5817, we organized helper_gvec_pmul_b in preparation for this use. Signed-off-by: Richard Henderson --- target/arm/helper.h | 10 ++++ target/arm/sve.decode | 10 ++++ target/arm/translate-sve.c | 50 ++++++++++++++++++++ target/arm/vec_helper.c | 96 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 166 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index ff8148ddc6..2c412ffd3b 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -828,6 +828,16 @@ DEF_HELPER_FLAGS_3(gvec_cgt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(gvec_cge0_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(gvec_cge0_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_umulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_sshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_sshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_ushl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 5c90603358..557706cacb 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1090,3 +1090,13 @@ ST1_zprz 1110010 .. 00 ..... 100 ... ..... ..... \ @rprr_scatter_store xs=0 esz=3 scale=0 ST1_zprz 1110010 .. 00 ..... 110 ... ..... ..... \ @rprr_scatter_store xs=1 esz=3 scale=0 + +#### SVE2 Support + +### SVE2 Integer Multiply - Unpredicated + +# SVE2 integer multiply vectors (unpredicated) +MUL_zzz 00000100 .. 1 ..... 0110 00 ..... ..... @rd_rn_rm +SMULH_zzz 00000100 .. 1 ..... 0110 10 ..... ..... @rd_rn_rm +UMULH_zzz 00000100 .. 1 ..... 0110 11 ..... ..... @rd_rn_rm +PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0eefb61214..06b906bce9 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5796,3 +5796,53 @@ static bool trans_MOVPRFX_z(DisasContext *s, arg_rpr_esz *a) { return do_movz_zpz(s, a->rd, a->rn, a->pg, a->esz, false); } + +/* + * SVE2 Integer Multiply - Unpredicated + */ + +static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzz(s, tcg_gen_gvec_mul, a->esz, a->rd, a->rn, a->rm); + } + return true; +} + +static bool do_sve2_zzz_ool(DisasContext *s, arg_rrr_esz *a, + gen_helper_gvec_3 *fn) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); + } + return true; +} + +static bool trans_SMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_gvec_smulh_b, gen_helper_gvec_smulh_h, + gen_helper_gvec_smulh_s, gen_helper_gvec_smulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + +static bool trans_UMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_gvec_umulh_b, gen_helper_gvec_umulh_h, + gen_helper_gvec_umulh_s, gen_helper_gvec_umulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + +static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a) +{ + return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 3fbeae87cb..40b92100bf 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -1985,3 +1985,99 @@ void HELPER(simd_tblx)(void *vd, void *vm, void *venv, uint32_t desc) clear_tail(vd, oprsz, simd_maxsz(desc)); } #endif + +/* + * NxN -> N highpart multiply + * + * TODO: expose this as a generic vector operation. + */ + +void HELPER(gvec_smulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ((int32_t)n[i] * m[i]) >> 8; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_smulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = ((int32_t)n[i] * m[i]) >> 16; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_smulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = ((int64_t)n[i] * m[i]) >> 32; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_smulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn, *m = vm; + uint64_t discard; + + for (i = 0; i < opr_sz / 8; ++i) { + muls64(&discard, &d[i], n[i], m[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ((uint32_t)n[i] * m[i]) >> 8; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint16_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = ((uint32_t)n[i] * m[i]) >> 16; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint32_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = ((uint64_t)n[i] * m[i]) >> 32; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn, *m = vm; + uint64_t discard; + + for (i = 0; i < opr_sz / 8; ++i) { + mulu64(&discard, &d[i], n[i], m[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} From patchwork Tue Mar 9 16:19:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396007 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2737782jai; Tue, 9 Mar 2021 08:43:05 -0800 (PST) X-Google-Smtp-Source: ABdhPJwdNbExxcZI9zb8i4oep5uQ90+ZYZV/jXXwgRvFqsl3xmvvoyZfK+ItXT4o2WGvTURLNlnP X-Received: by 2002:a92:d201:: with SMTP id y1mr26500160ily.129.1615308185888; Tue, 09 Mar 2021 08:43:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615308185; cv=none; d=google.com; s=arc-20160816; b=AwkmaOvPn5Cv2RCCsA9MH4MD/ViiKPu4Nj/HMYfoO2DpjfzALfsgbLD6ODYpLjziLZ VfDo6k7XQMNTT8XGPSKROjnMfFWFEanCPGdfovEMpX92nOgixBv7uPMILaGxktsA+K8e QfxdlJdxv/W6b37BDdZ4N9lwMRe/VSMdyBdhNpqIt+yi6lcNsnLhwDRQcJjWVPto9PYr hUuslJ54Y5kKPzWPA14tgO+XZ7S7zhKxyv5A9IbX5ZqbcwQuumYuYWA5y5s2L5uCoVWN uKyhJ/qLQ4oxiAVN2P79Ox6dIoQxVn+p8mPUdAawQXjnajDDo/yt3mB7Arzw2/UL/Etd dOTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=hxGq0ZhXNKQdyORVC2X6O56UWpurZDIjXhlbnnV/c4c=; b=TrzJPMJybr+/fIwBP8s4nQjtENVyHr5qdJNOBxCLBpMX7MqGwXuiffzTDgvy1eXY2n 9o533wduOJ6EoX1Qt0yUaGffjnv7+nbp3uN1ioCQ69yuwT9wlAB1NtHVxOt+DQuEGNd2 vjqpkRm5lBtYU/jrV8cbhRorxmFZhGRPrBW6KnSAcCCI5bZW0ZPmL/Y4/PHCi+T6nHHu +1nkqobPCxbSa31/fL5UZ2Vz2Tz9LrbLQXTnjs1dbINPNanMsv6s8RA64VVbbHFcinaB W/qtwf+mGWVa+1cIakee2XaiJ4FsXPI+HLaf+YCRd4Ju7Q2iO/w2gRjXFmroFJ7Qi4dL ipCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vEHxcwsl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c6si14334260jag.27.2021.03.09.08.43.05 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 08:43:05 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vEHxcwsl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46156 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJfRp-0002LX-3E for patch@linaro.org; Tue, 09 Mar 2021 11:43:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45158) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6M-0006w8-CY for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:54 -0500 Received: from mail-oi1-x232.google.com ([2607:f8b0:4864:20::232]:39498) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6F-0002O5-Tl for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:54 -0500 Received: by mail-oi1-x232.google.com with SMTP id z126so15523020oiz.6 for ; Tue, 09 Mar 2021 08:20:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hxGq0ZhXNKQdyORVC2X6O56UWpurZDIjXhlbnnV/c4c=; b=vEHxcwsll2OL7B4wPd2UzdoCTPKBWqQo/x+vErN644MUv7skSxqY7aJZPou+AchHhj aYP6/qI7RrQT6wz/qAJFO6Mmauvg+IAUeFWokO4paRD5dZb/Zy7ewl2+k/3KA90Sqz2m lZsRoBqghk8KLKFG3i2FuewlLQCNMqtdRz8J1e/QVSUxrMVMAfQvOgV5V+PDrV+jTyr1 BeLD+Z/CZTZYzWQXT148bmupu2rPgwcRLfWZULQHQH8LEuBSY/nHUsv8mHzd98s8xDym xvJcc0byCZ0V+U9b8tEoJBppsUVlfmkT3+36+EVWAYggnVff7cDjzu+uKE8fZ59SeYsp K1uQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hxGq0ZhXNKQdyORVC2X6O56UWpurZDIjXhlbnnV/c4c=; b=q5Q4Y9ZrIDD8dMYrsQG13p0mFKfbgqaj/jx4S16HT/jipFHLN+IbBTcs+XtGqZZQ4J gBpQPwbw7BKWpkKaYzVoa7WxmCDl3MMCG7xsYAS+26snu32X2cgsXWIRJ8dp+5r2a3aE lJfa32HuhHCN/hsPclKJBv3SHu1hfW4CLHZkaAIbQmLqeltDK+1dMC3Ob32y1Tuufn+A 7jJuc0ic51Ifc4YcKOMCX9FLv8YBLoSR1IzRIpT/riV0mxI+wrpGuHzQj8buOJ3QvMti 9IKSekorkrj78o8K11QF6hSkmuDcv/e092U6xcJl+YKu0GY4dgQ86T7irbxEEJlGwsYZ QJyg== X-Gm-Message-State: AOAM532zp1asQp3IbZ6mhqGS+8GilDkZCzyqCLcewEhD49NUTlqyKmPb oubBqHzbZdwKUg1dO4LPHiZ+hJwl7tevlBg3 X-Received: by 2002:aca:482:: with SMTP id 124mr1517622oie.21.1615306846873; Tue, 09 Mar 2021 08:20:46 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:46 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 03/78] target/arm: Implement SVE2 integer pairwise add and accumulate long Date: Tue, 9 Mar 2021 08:19:26 -0800 Message-Id: <20210309162041.23124-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::232; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x232.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 14 ++++++++++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 44 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 39 +++++++++++++++++++++++++++++++++ 4 files changed, 102 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index e4cadd2a65..b2a274b40b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -158,6 +158,20 @@ DEF_HELPER_FLAGS_5(sve_umulh_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve_umulh_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sadalp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sadalp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sadalp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_d, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 557706cacb..0524c01fcf 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1100,3 +1100,8 @@ MUL_zzz 00000100 .. 1 ..... 0110 00 ..... ..... @rd_rn_rm SMULH_zzz 00000100 .. 1 ..... 0110 10 ..... ..... @rd_rn_rm UMULH_zzz 00000100 .. 1 ..... 0110 11 ..... ..... @rd_rn_rm PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 + +### SVE2 Integer - Predicated + +SADALP_zpzz 01000100 .. 000 100 101 ... ..... ..... @rdm_pg_rn +UADALP_zpzz 01000100 .. 000 101 101 ... ..... ..... @rdm_pg_rn diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index fd6c58f96a..00bb82da91 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -517,6 +517,50 @@ DO_ZPZZ_D(sve_asr_zpzz_d, int64_t, DO_ASR) DO_ZPZZ_D(sve_lsr_zpzz_d, uint64_t, DO_LSR) DO_ZPZZ_D(sve_lsl_zpzz_d, uint64_t, DO_LSL) +static inline uint16_t do_sadalp_h(uint16_t n, uint16_t m) +{ + int8_t n1 = n, n2 = n >> 8; + return m + n1 + n2; +} + +static inline uint32_t do_sadalp_s(uint32_t n, uint32_t m) +{ + int16_t n1 = n, n2 = n >> 16; + return m + n1 + n2; +} + +static inline uint64_t do_sadalp_d(uint64_t n, uint64_t m) +{ + int32_t n1 = n, n2 = n >> 32; + return m + n1 + n2; +} + +DO_ZPZZ(sve2_sadalp_zpzz_h, int16_t, H1_2, do_sadalp_h) +DO_ZPZZ(sve2_sadalp_zpzz_s, int32_t, H1_4, do_sadalp_s) +DO_ZPZZ_D(sve2_sadalp_zpzz_d, uint64_t, do_sadalp_d) + +static inline uint16_t do_uadalp_h(uint16_t n, uint16_t m) +{ + uint8_t n1 = n, n2 = n >> 8; + return m + n1 + n2; +} + +static inline uint32_t do_uadalp_s(uint32_t n, uint32_t m) +{ + uint16_t n1 = n, n2 = n >> 16; + return m + n1 + n2; +} + +static inline uint64_t do_uadalp_d(uint64_t n, uint64_t m) +{ + uint32_t n1 = n, n2 = n >> 32; + return m + n1 + n2; +} + +DO_ZPZZ(sve2_uadalp_zpzz_h, int16_t, H1_2, do_uadalp_h) +DO_ZPZZ(sve2_uadalp_zpzz_s, int32_t, H1_4, do_uadalp_s) +DO_ZPZZ_D(sve2_uadalp_zpzz_d, uint64_t, do_uadalp_d) + #undef DO_ZPZZ #undef DO_ZPZZ_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 06b906bce9..603e673696 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5846,3 +5846,42 @@ static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a) { return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b); } + +/* + * SVE2 Integer - Predicated + */ + +static bool do_sve2_zpzz_ool(DisasContext *s, arg_rprr_esz *a, + gen_helper_gvec_4 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzz_ool(s, a, fn); +} + +static bool trans_SADALP_zpzz(DisasContext *s, arg_rprr_esz *a) +{ + static gen_helper_gvec_4 * const fns[3] = { + gen_helper_sve2_sadalp_zpzz_h, + gen_helper_sve2_sadalp_zpzz_s, + gen_helper_sve2_sadalp_zpzz_d, + }; + if (a->esz == 0) { + return false; + } + return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); +} + +static bool trans_UADALP_zpzz(DisasContext *s, arg_rprr_esz *a) +{ + static gen_helper_gvec_4 * const fns[3] = { + gen_helper_sve2_uadalp_zpzz_h, + gen_helper_sve2_uadalp_zpzz_s, + gen_helper_sve2_uadalp_zpzz_d, + }; + if (a->esz == 0) { + return false; + } + return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); +} From patchwork Tue Mar 9 16:19:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396009 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2742076jai; Tue, 9 Mar 2021 08:49:33 -0800 (PST) X-Google-Smtp-Source: ABdhPJzaUe8vQZWbSQ/umbNsRnwvVdjXMyzZqArRpyKNg9fYW2dNko4l0JZFzTuHDBpVTXYk3qV9 X-Received: by 2002:a05:622a:102:: with SMTP id u2mr25801612qtw.37.1615308573356; Tue, 09 Mar 2021 08:49:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615308573; cv=none; d=google.com; s=arc-20160816; b=RCcCUXahcTbFkMBfqOL3gBmPNd3pRnjfMWFn8ew5mvMX+ao4N6BanDhg/f9C9iVVwC Hr7sRZFADweC6QMFRdxNlJ7fShQ/q2Rp4Azc2zb4BrdOcvuHslOD526baz9I2URkpwbY 87UmQ/UMUSLF8i8FbS2MUIeBVuHKSTx0TPxYuusYsZa46sTDbbcN21IwzR1WqHwZLjcp D3u0Rbh5MpcCQ3jY+1Fq/rUqfNeFBB7cydWJKKzBEDD3Q0yu5BCALOMjVRL8wzZu1vfo wgp8JYVpkpm5UHsalPabMdKIO30UW7lM0LAAduy9K+b6N6jIzHv0tHmpayBPkKXn5Mj9 YWuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=4JrM/gCMwPo4dSZSFmIGMUubRJmYQic6TLHlnfTln/s=; b=osH6eb/2QT/AzxNQfEhoHVwy7Qst3to/KsTSEVXUVHcpa6OPZhQvVLWy8MGgkx39ir NGefhQC1JVe3yW5ypGxu8hFJbZPeND+hGEwJmvNfSuyI7GewKRzzcyRhsPkLlpzi2N0F uHz0pzgeyz4RlKl8N8IzYaPcp1Uki2Lq4mC5EhvOy1Xv3p1R3zq5zBAhzxYS96uo/2SO lX0N8o9Ss1vcj0ckAZ9Z9+IJDCnIxzqhQFzgKDjOY4up66iaEjnucoEdwXL6d+Wxhxul RbLCtQOLFGXc/QTrc6OYdhIhFlwRc/0d8X9lacMcmgy/tOdHReRvcapzCBQ6rKDGXqJq +kew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vmIIWa5Q; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d30si9070697qtm.5.2021.03.09.08.49.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 08:49:33 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vmIIWa5Q; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54936 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJfY4-0006Q5-Ep for patch@linaro.org; Tue, 09 Mar 2021 11:49:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45202) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6N-0006xU-NK for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:56 -0500 Received: from mail-ot1-x333.google.com ([2607:f8b0:4864:20::333]:46160) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6K-0002PV-Ak for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:54 -0500 Received: by mail-ot1-x333.google.com with SMTP id r24so6268975otq.13 for ; Tue, 09 Mar 2021 08:20:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4JrM/gCMwPo4dSZSFmIGMUubRJmYQic6TLHlnfTln/s=; b=vmIIWa5QRdpnNdriVDsaP7WiGyINJeBalh+QCf55Js2ovzW/43RBjwC1mTxlcXLgk8 2cflmMI6XxWciZMDd/MtGkJLy9nHytYp2d0L/oivywWNsyeTnKy70inOtoNmw9AJAQgG fUrzGSOZika+1mY1TU2XNSW+eht9FJstGTKZKWM2dy7TiP3XFq+RCGHgr9ENh4VuJkQ2 eYhaxh0XfBak1e+xO4Qg8GsJmCJ9WDKjZSwpe6RQx7ZJquv+dlCa/G4IQRedYVQT6WzK YcrWFq2IwuVsrRiYcVUwOneYlubwaq+lVBLYQn7YW/qzUSQLB3xcgKgHGJMlIj+sOq4q ej6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4JrM/gCMwPo4dSZSFmIGMUubRJmYQic6TLHlnfTln/s=; b=lqTzdiBuCbbPokZZ0wCudhG9EU0pv+zzysFCCWYaoKL/Dk844dbrbb7cWkk0SdOhXi PPyXMn8MhS2eN/oICOAV8XWL6qADmfH3U2fizwvHfcL+vNFMR30ztHP+xJk05kiTGJML NkkSF2l6pR+WMNfd0wSm5CISBkRHe1VILvfHdIfs4dHUYFHu1UOi4b++uPHVlYaZOIH5 B0dCghUg4mUJzWWHVo7JZJuGgeyZjp1LwbcLg8Wm233s+KsnEaSSNBBKaM4EPeBX15hd rPYTbuX1XootPOLdMwtCah+DEbnUJUyyRx9/KOIBtwQCnm8wWs+bsof48wGdlgTvqMBs mcvw== X-Gm-Message-State: AOAM533PnmN4ALoOcaH4LiGOp33qVxWtxAVyA3j5Fsx4tLFYfzT3VVdm id18OkzRv67JB2AYow1yQ/W144LUOSzFgWrM X-Received: by 2002:a9d:4e14:: with SMTP id p20mr16850572otf.94.1615306847959; Tue, 09 Mar 2021 08:20:47 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:47 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 04/78] target/arm: Implement SVE2 integer unary operations (predicated) Date: Tue, 9 Mar 2021 08:19:27 -0800 Message-Id: <20210309162041.23124-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::333; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x333.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix sqabs, sqneg (laurent desnogues) --- target/arm/helper-sve.h | 13 +++++++++++ target/arm/sve.decode | 7 ++++++ target/arm/sve_helper.c | 29 +++++++++++++++++++---- target/arm/translate-sve.c | 47 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 92 insertions(+), 4 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index b2a274b40b..9992e93e2b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -502,6 +502,19 @@ DEF_HELPER_FLAGS_4(sve_rbit_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_rbit_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_rbit_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqneg_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqneg_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqneg_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqneg_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_urecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ursqrte_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_splice, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_cmpeq_ppzz_b, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 0524c01fcf..5ba542969b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1105,3 +1105,10 @@ PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 SADALP_zpzz 01000100 .. 000 100 101 ... ..... ..... @rdm_pg_rn UADALP_zpzz 01000100 .. 000 101 101 ... ..... ..... @rdm_pg_rn + +### SVE2 integer unary operations (predicated) + +URECPE 01000100 .. 000 000 101 ... ..... ..... @rd_pg_rn +URSQRTE 01000100 .. 000 001 101 ... ..... ..... @rd_pg_rn +SQABS 01000100 .. 001 000 101 ... ..... ..... @rd_pg_rn +SQNEG 01000100 .. 001 001 101 ... ..... ..... @rd_pg_rn diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 00bb82da91..5dcdd26db0 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -535,8 +535,8 @@ static inline uint64_t do_sadalp_d(uint64_t n, uint64_t m) return m + n1 + n2; } -DO_ZPZZ(sve2_sadalp_zpzz_h, int16_t, H1_2, do_sadalp_h) -DO_ZPZZ(sve2_sadalp_zpzz_s, int32_t, H1_4, do_sadalp_s) +DO_ZPZZ(sve2_sadalp_zpzz_h, uint16_t, H1_2, do_sadalp_h) +DO_ZPZZ(sve2_sadalp_zpzz_s, uint32_t, H1_4, do_sadalp_s) DO_ZPZZ_D(sve2_sadalp_zpzz_d, uint64_t, do_sadalp_d) static inline uint16_t do_uadalp_h(uint16_t n, uint16_t m) @@ -557,8 +557,8 @@ static inline uint64_t do_uadalp_d(uint64_t n, uint64_t m) return m + n1 + n2; } -DO_ZPZZ(sve2_uadalp_zpzz_h, int16_t, H1_2, do_uadalp_h) -DO_ZPZZ(sve2_uadalp_zpzz_s, int32_t, H1_4, do_uadalp_s) +DO_ZPZZ(sve2_uadalp_zpzz_h, uint16_t, H1_2, do_uadalp_h) +DO_ZPZZ(sve2_uadalp_zpzz_s, uint32_t, H1_4, do_uadalp_s) DO_ZPZZ_D(sve2_uadalp_zpzz_d, uint64_t, do_uadalp_d) #undef DO_ZPZZ @@ -728,6 +728,27 @@ DO_ZPZ(sve_rbit_h, uint16_t, H1_2, revbit16) DO_ZPZ(sve_rbit_s, uint32_t, H1_4, revbit32) DO_ZPZ_D(sve_rbit_d, uint64_t, revbit64) +#define DO_SQABS(X) \ + ({ __typeof(X) x_ = (X), min_ = 1ull << (sizeof(X) * 8 - 1); \ + x_ >= 0 ? x_ : x_ == min_ ? -min_ - 1 : -x_; }) + +DO_ZPZ(sve2_sqabs_b, int8_t, H1, DO_SQABS) +DO_ZPZ(sve2_sqabs_h, int16_t, H1_2, DO_SQABS) +DO_ZPZ(sve2_sqabs_s, int32_t, H1_4, DO_SQABS) +DO_ZPZ_D(sve2_sqabs_d, int64_t, DO_SQABS) + +#define DO_SQNEG(X) \ + ({ __typeof(X) x_ = (X), min_ = 1ull << (sizeof(X) * 8 - 1); \ + x_ == min_ ? -min_ - 1 : -x_; }) + +DO_ZPZ(sve2_sqneg_b, uint8_t, H1, DO_SQNEG) +DO_ZPZ(sve2_sqneg_h, uint16_t, H1_2, DO_SQNEG) +DO_ZPZ(sve2_sqneg_s, uint32_t, H1_4, DO_SQNEG) +DO_ZPZ_D(sve2_sqneg_d, uint64_t, DO_SQNEG) + +DO_ZPZ(sve2_urecpe_s, uint32_t, H1_4, helper_recpe_u32) +DO_ZPZ(sve2_ursqrte_s, uint32_t, H1_4, helper_rsqrte_u32) + /* Three-operand expander, unpredicated, in which the third operand is "wide". */ #define DO_ZZW(NAME, TYPE, TYPEW, H, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 603e673696..2d8b9d2e70 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5885,3 +5885,50 @@ static bool trans_UADALP_zpzz(DisasContext *s, arg_rprr_esz *a) } return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); } + +/* + * SVE2 integer unary operations (predicated) + */ + +static bool do_sve2_zpz_ool(DisasContext *s, arg_rpr_esz *a, + gen_helper_gvec_3 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ool(s, a, fn); +} + +static bool trans_URECPE(DisasContext *s, arg_rpr_esz *a) +{ + if (a->esz != 2) { + return false; + } + return do_sve2_zpz_ool(s, a, gen_helper_sve2_urecpe_s); +} + +static bool trans_URSQRTE(DisasContext *s, arg_rpr_esz *a) +{ + if (a->esz != 2) { + return false; + } + return do_sve2_zpz_ool(s, a, gen_helper_sve2_ursqrte_s); +} + +static bool trans_SQABS(DisasContext *s, arg_rpr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqabs_b, gen_helper_sve2_sqabs_h, + gen_helper_sve2_sqabs_s, gen_helper_sve2_sqabs_d, + }; + return do_sve2_zpz_ool(s, a, fns[a->esz]); +} + +static bool trans_SQNEG(DisasContext *s, arg_rpr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqneg_b, gen_helper_sve2_sqneg_h, + gen_helper_sve2_sqneg_s, gen_helper_sve2_sqneg_d, + }; + return do_sve2_zpz_ool(s, a, fns[a->esz]); +} From patchwork Tue Mar 9 16:19:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396025 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2779693jai; Tue, 9 Mar 2021 09:35:32 -0800 (PST) X-Google-Smtp-Source: ABdhPJyXkQZJaaXU/C35FYqfM5T4R45YKH9PMyzI+QiZBO6HSCxOe87cLnmlydYayuQDInhVxN9H X-Received: by 2002:a67:e319:: with SMTP id j25mr18381834vsf.9.1615311332748; Tue, 09 Mar 2021 09:35:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311332; cv=none; d=google.com; s=arc-20160816; b=tcBcqGQXswFs5qD/UUO+HfcNyiXFA3p6yUGU2JRwY7YsA3gQbWA4JhajiGRLZ0MpGs 6lVvcYzlUJ60z4zQ6upEUR4gAPsylcEikHtfZMcnriDX4vTWU7j+hhR+sbGvkOIa9S9l aGo8/AjYg2F6ruwXs3FkXfY/w1JgiKi0/PzWQ8+j74TIRGjaIvjkDZ3Sn5m5qniAjjgV f7U7b1BSSmegj11MUTIjvVVwS2TvfNmIHJi6TNZ4EZtnzTeiZF+eDZH6wZbeFLf88mmJ WGgsDI1cc7BxFIt72VbEJGqwaZVRaL1kWpq4LpMqq6gLIVtuLFfme4K+hp83DJ+szwWw DcFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=DS4qOR8a0wrDRRaz52YdBuOsYQMMqkUfZ7NchqJV82A=; b=u506o/28Wrl7kEIcEcokBwrsbPXkmMpXgRwtwxdS7FkHOcIffo9Pip7fI5D0gxF/bs TY124hqakBFUFDBdWYREr3c5V/HDKXVJQP5OrSAOqO0nKjJgo6hZMqObO5sBaVc7G5CT UQyxxxUlQaFngV2p1pfIsh2kAIjiyA0nLHJ5C6+9HMqUlHOZ6SrACOaszhhm0NaiJySq VwJz9IPCTSqPcLrMkZxnYjF6yhLwhl2wp1uD7c/HoZz7I+OjoynuLHzOTnBodTRzeSXA w5FHXHj17fYA3QNWXRepOxz65aBkvm4DGh4rvcdPgmnsaZEmqvWfXcBSKHLWwJO+Xc1z 2FTg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qpvJ7L2C; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y6si2683049uap.21.2021.03.09.09.35.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:35:32 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qpvJ7L2C; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55988 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgGa-0004zw-1j for patch@linaro.org; Tue, 09 Mar 2021 12:35:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45262) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6P-0006z7-Vm for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:58 -0500 Received: from mail-oi1-x22e.google.com ([2607:f8b0:4864:20::22e]:32770) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6K-0002Po-AT for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:57 -0500 Received: by mail-oi1-x22e.google.com with SMTP id d16so6238377oic.0 for ; Tue, 09 Mar 2021 08:20:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DS4qOR8a0wrDRRaz52YdBuOsYQMMqkUfZ7NchqJV82A=; b=qpvJ7L2CYh/J4MVNmHabjOct6+9KGiARNPheeGvKEYqTQWD7crKg8RYLDgy8YhDeWK 4XvuDj3F3UMdl4RcKSuCGqHo+2DEHrjiNqyWMrWYiV2AGsH+nz8DmbvazoQ2zPFWXOno l9novqbE4lUtZtuaV6Lmm8jaGEc/FxNQscrP9ELeSsKNA9ddDckuLD9SuykGvsAHlNIn 6TH8s5p3aXDHFLIwj3YbRmkJgGRNppGZ/2tVMkIpAgNzZSLqKmYj/526rhqQik6AuMWb 1MyVRL1NrSQqL/WkMw1wNf5amv1P3lAcODitsoXRStdtE88sXvpxSI+odgCfCqdL0dcI GkAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DS4qOR8a0wrDRRaz52YdBuOsYQMMqkUfZ7NchqJV82A=; b=OfqvajsfbORwOjcc/Il7szOrbCPwnGZpQBJp7FxsQAmwFyfWwIuMBzRCF55hJO1/q+ /dazTi9KuN2lomkXYJWNUMB+j8uXa7fc2TnItzScl/azqWIyqJjuR/q/jB10SbSRBX7D GebkRsXaBwmrXg9oDy3EfZmQms41IqgQTEv1QxZsEXloa/YVNinZnCEMHNpgnOb9V442 ZqJnDwCuyj6LvfPMTbnVxLuCudsbRIJOZUDkw342zC6ODKWdqdvzYk08Gyog31Petyic SQdziCNs5FpegMdTeRr1JIc0Sxl7y/t8miNThsnqdENROHjdCsukvC2WlyEtL+n92YFt uPoA== X-Gm-Message-State: AOAM530K6/kAImIKLuVKEGpGZP/AqZTD2pdGyy6QALOBpysPsSozSc3n qLGCp5ONDyS6SzWgKUMnc/NLkEfbah2P2fjA X-Received: by 2002:aca:4712:: with SMTP id u18mr3614067oia.61.1615306849048; Tue, 09 Mar 2021 08:20:49 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:48 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 05/78] target/arm: Split out saturating/rounding shifts from neon Date: Tue, 9 Mar 2021 08:19:28 -0800 Message-Id: <20210309162041.23124-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22e; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Split these operations out into a header that can be shared between neon and sve. The "sat" pointer acts both as a boolean for control of saturating behavior and controls the difference in behavior between neon and sve -- QC bit or no QC bit. Widen the shift operand in the new helpers, as the SVE2 insns treat the whole input element as significant. For the neon uses, truncate the shift to int8_t while passing the parameter. Implement right-shift rounding as tmp = src >> (shift - 1); dst = (tmp >> 1) + (tmp & 1); This is the same number of instructions as the current tmp = 1 << (shift - 1); dst = (src + tmp) >> shift; without any possibility of intermediate overflow. Signed-off-by: Richard Henderson --- v2: Widen the shift operand (laurent desnouges) --- target/arm/vec_internal.h | 138 +++++++++++ target/arm/neon_helper.c | 507 +++++++------------------------------- 2 files changed, 221 insertions(+), 424 deletions(-) -- 2.25.1 diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h index e3eb3e7a6b..0102547a10 100644 --- a/target/arm/vec_internal.h +++ b/target/arm/vec_internal.h @@ -30,4 +30,142 @@ static inline void clear_tail(void *vd, uintptr_t opr_sz, uintptr_t max_sz) } } +static inline int32_t do_sqrshl_bhs(int32_t src, int32_t shift, int bits, + bool round, uint32_t *sat) +{ + if (shift <= -bits) { + /* Rounding the sign bit always produces 0. */ + if (round) { + return 0; + } + return src >> 31; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < bits) { + int32_t val = src << shift; + if (bits == 32) { + if (!sat || val >> shift == src) { + return val; + } + } else { + int32_t extval = sextract32(val, 0, bits); + if (!sat || val == extval) { + return extval; + } + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return (1u << (bits - 1)) - (src >= 0); +} + +static inline uint32_t do_uqrshl_bhs(uint32_t src, int32_t shift, int bits, + bool round, uint32_t *sat) +{ + if (shift <= -(bits + round)) { + return 0; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < bits) { + uint32_t val = src << shift; + if (bits == 32) { + if (!sat || val >> shift == src) { + return val; + } + } else { + uint32_t extval = extract32(val, 0, bits); + if (!sat || val == extval) { + return extval; + } + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return MAKE_64BIT_MASK(0, bits); +} + +static inline int32_t do_suqrshl_bhs(int32_t src, int32_t shift, int bits, + bool round, uint32_t *sat) +{ + if (src < 0) { + *sat = 1; + return 0; + } + return do_uqrshl_bhs(src, shift, bits, round, sat); +} + +static inline int64_t do_sqrshl_d(int64_t src, int64_t shift, + bool round, uint32_t *sat) +{ + if (shift <= -64) { + /* Rounding the sign bit always produces 0. */ + if (round) { + return 0; + } + return src >> 63; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < 64) { + int64_t val = src << shift; + if (!sat || val >> shift == src) { + return val; + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return src < 0 ? INT64_MIN : INT64_MAX; +} + +static inline uint64_t do_uqrshl_d(uint64_t src, int64_t shift, + bool round, uint32_t *sat) +{ + if (shift <= -(64 + round)) { + return 0; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < 64) { + uint64_t val = src << shift; + if (!sat || val >> shift == src) { + return val; + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return UINT64_MAX; +} + +static inline int64_t do_suqrshl_d(int64_t src, int64_t shift, + bool round, uint32_t *sat) +{ + if (src < 0) { + *sat = 1; + return 0; + } + return do_uqrshl_d(src, shift, round, sat); +} + #endif /* TARGET_ARM_VEC_INTERNALS_H */ diff --git a/target/arm/neon_helper.c b/target/arm/neon_helper.c index b637265691..338b9189d5 100644 --- a/target/arm/neon_helper.c +++ b/target/arm/neon_helper.c @@ -11,6 +11,7 @@ #include "cpu.h" #include "exec/helper-proto.h" #include "fpu/softfloat.h" +#include "vec_internal.h" #define SIGNBIT (uint32_t)0x80000000 #define SIGNBIT64 ((uint64_t)1 << 63) @@ -576,496 +577,154 @@ NEON_POP(pmax_s16, neon_s16, 2) NEON_POP(pmax_u16, neon_u16, 2) #undef NEON_FN -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8 || \ - tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, false, NULL)) NEON_VOP(shl_u16, neon_u16, 2) #undef NEON_FN -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> (sizeof(src1) * 8 - 1); \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, false, NULL)) NEON_VOP(shl_s16, neon_s16, 2) #undef NEON_FN -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if ((tmp >= (ssize_t)sizeof(src1) * 8) \ - || (tmp <= -(ssize_t)sizeof(src1) * 8)) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 8, true, NULL)) NEON_VOP(rshl_s8, neon_s8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, true, NULL)) NEON_VOP(rshl_s16, neon_s16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_rshl_s32)(uint32_t valop, uint32_t shiftop) +uint32_t HELPER(neon_rshl_s32)(uint32_t val, uint32_t shift) { - int32_t dest; - int32_t val = (int32_t)valop; - int8_t shift = (int8_t)shiftop; - if ((shift >= 32) || (shift <= -32)) { - dest = 0; - } else if (shift < 0) { - int64_t big_dest = ((int64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - } - return dest; + return do_sqrshl_bhs(val, (int8_t)shift, 32, true, NULL); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_rshl_s64)(uint64_t valop, uint64_t shiftop) +uint64_t HELPER(neon_rshl_s64)(uint64_t val, uint64_t shift) { - int8_t shift = (int8_t)shiftop; - int64_t val = valop; - if ((shift >= 64) || (shift <= -64)) { - val = 0; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == INT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x4000000000000000LL; - } else { - val++; - val >>= 1; - } - } else { - val <<= shift; - } - return val; + return do_sqrshl_d(val, (int8_t)shift, true, NULL); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8 || \ - tmp < -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp == -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> (-tmp - 1); \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 8, true, NULL)) NEON_VOP(rshl_u8, neon_u8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, true, NULL)) NEON_VOP(rshl_u16, neon_u16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_rshl_u32)(uint32_t val, uint32_t shiftop) +uint32_t HELPER(neon_rshl_u32)(uint32_t val, uint32_t shift) { - uint32_t dest; - int8_t shift = (int8_t)shiftop; - if (shift >= 32 || shift < -32) { - dest = 0; - } else if (shift == -32) { - dest = val >> 31; - } else if (shift < 0) { - uint64_t big_dest = ((uint64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - } - return dest; + return do_uqrshl_bhs(val, (int8_t)shift, 32, true, NULL); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_rshl_u64)(uint64_t val, uint64_t shiftop) +uint64_t HELPER(neon_rshl_u64)(uint64_t val, uint64_t shift) { - int8_t shift = (uint8_t)shiftop; - if (shift >= 64 || shift < -64) { - val = 0; - } else if (shift == -64) { - /* Rounding a 1-bit result just preserves that bit. */ - val >>= 63; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == UINT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x8000000000000000ULL; - } else { - val++; - val >>= 1; - } - } else { - val <<= shift; - } - return val; + return do_uqrshl_d(val, (int8_t)shift, true, NULL); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = ~0; \ - } else { \ - dest = 0; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = ~0; \ - } \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 8, false, env->vfp.qc)) NEON_VOP_ENV(qshl_u8, neon_u8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, false, env->vfp.qc)) NEON_VOP_ENV(qshl_u16, neon_u16, 2) -NEON_VOP_ENV(qshl_u32, neon_u32, 1) #undef NEON_FN -uint64_t HELPER(neon_qshl_u64)(CPUARMState *env, uint64_t val, uint64_t shiftop) +uint32_t HELPER(neon_qshl_u32)(CPUARMState *env, uint32_t val, uint32_t shift) { - int8_t shift = (int8_t)shiftop; - if (shift >= 64) { - if (val) { - val = ~(uint64_t)0; - SET_QC(); - } - } else if (shift <= -64) { - val = 0; - } else if (shift < 0) { - val >>= -shift; - } else { - uint64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = ~(uint64_t)0; - } - } - return val; + return do_uqrshl_bhs(val, (int8_t)shift, 32, false, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = (uint32_t)(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } else { \ - dest = src1; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> 31; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = (uint32_t)(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } \ - }} while (0) +uint64_t HELPER(neon_qshl_u64)(CPUARMState *env, uint64_t val, uint64_t shift) +{ + return do_uqrshl_d(val, (int8_t)shift, false, env->vfp.qc); +} + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 8, false, env->vfp.qc)) NEON_VOP_ENV(qshl_s8, neon_s8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, false, env->vfp.qc)) NEON_VOP_ENV(qshl_s16, neon_s16, 2) -NEON_VOP_ENV(qshl_s32, neon_s32, 1) #undef NEON_FN -uint64_t HELPER(neon_qshl_s64)(CPUARMState *env, uint64_t valop, uint64_t shiftop) +uint32_t HELPER(neon_qshl_s32)(CPUARMState *env, uint32_t val, uint32_t shift) { - int8_t shift = (uint8_t)shiftop; - int64_t val = valop; - if (shift >= 64) { - if (val) { - SET_QC(); - val = (val >> 63) ^ ~SIGNBIT64; - } - } else if (shift <= -64) { - val >>= 63; - } else if (shift < 0) { - val >>= -shift; - } else { - int64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = (tmp >> 63) ^ ~SIGNBIT64; - } - } - return val; + return do_sqrshl_bhs(val, (int8_t)shift, 32, false, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - if (src1 & (1 << (sizeof(src1) * 8 - 1))) { \ - SET_QC(); \ - dest = 0; \ - } else { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = ~0; \ - } else { \ - dest = 0; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = ~0; \ - } \ - } \ - }} while (0) -NEON_VOP_ENV(qshlu_s8, neon_u8, 4) -NEON_VOP_ENV(qshlu_s16, neon_u16, 2) +uint64_t HELPER(neon_qshl_s64)(CPUARMState *env, uint64_t val, uint64_t shift) +{ + return do_sqrshl_d(val, (int8_t)shift, false, env->vfp.qc); +} + +#define NEON_FN(dest, src1, src2) \ + (dest = do_suqrshl_bhs(src1, (int8_t)src2, 8, false, env->vfp.qc)) +NEON_VOP_ENV(qshlu_s8, neon_s8, 4) #undef NEON_FN -uint32_t HELPER(neon_qshlu_s32)(CPUARMState *env, uint32_t valop, uint32_t shiftop) +#define NEON_FN(dest, src1, src2) \ + (dest = do_suqrshl_bhs(src1, (int8_t)src2, 16, false, env->vfp.qc)) +NEON_VOP_ENV(qshlu_s16, neon_s16, 2) +#undef NEON_FN + +uint32_t HELPER(neon_qshlu_s32)(CPUARMState *env, uint32_t val, uint32_t shift) { - if ((int32_t)valop < 0) { - SET_QC(); - return 0; - } - return helper_neon_qshl_u32(env, valop, shiftop); + return do_suqrshl_bhs(val, (int8_t)shift, 32, false, env->vfp.qc); } -uint64_t HELPER(neon_qshlu_s64)(CPUARMState *env, uint64_t valop, uint64_t shiftop) +uint64_t HELPER(neon_qshlu_s64)(CPUARMState *env, uint64_t val, uint64_t shift) { - if ((int64_t)valop < 0) { - SET_QC(); - return 0; - } - return helper_neon_qshl_u64(env, valop, shiftop); + return do_suqrshl_d(val, (int8_t)shift, false, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = ~0; \ - } else { \ - dest = 0; \ - } \ - } else if (tmp < -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp == -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> (sizeof(src1) * 8 - 1); \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = ~0; \ - } \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 8, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_u8, neon_u8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_u16, neon_u16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_qrshl_u32)(CPUARMState *env, uint32_t val, uint32_t shiftop) +uint32_t HELPER(neon_qrshl_u32)(CPUARMState *env, uint32_t val, uint32_t shift) { - uint32_t dest; - int8_t shift = (int8_t)shiftop; - if (shift >= 32) { - if (val) { - SET_QC(); - dest = ~0; - } else { - dest = 0; - } - } else if (shift < -32) { - dest = 0; - } else if (shift == -32) { - dest = val >> 31; - } else if (shift < 0) { - uint64_t big_dest = ((uint64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - if ((dest >> shift) != val) { - SET_QC(); - dest = ~0; - } - } - return dest; + return do_uqrshl_bhs(val, (int8_t)shift, 32, true, env->vfp.qc); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_qrshl_u64)(CPUARMState *env, uint64_t val, uint64_t shiftop) +uint64_t HELPER(neon_qrshl_u64)(CPUARMState *env, uint64_t val, uint64_t shift) { - int8_t shift = (int8_t)shiftop; - if (shift >= 64) { - if (val) { - SET_QC(); - val = ~0; - } - } else if (shift < -64) { - val = 0; - } else if (shift == -64) { - val >>= 63; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == UINT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x8000000000000000ULL; - } else { - val++; - val >>= 1; - } - } else { \ - uint64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = ~0; - } - } - return val; + return do_uqrshl_d(val, (int8_t)shift, true, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = (typeof(dest))(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } else { \ - dest = 0; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = (uint32_t)(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 8, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_s8, neon_s8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_s16, neon_s16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_qrshl_s32)(CPUARMState *env, uint32_t valop, uint32_t shiftop) +uint32_t HELPER(neon_qrshl_s32)(CPUARMState *env, uint32_t val, uint32_t shift) { - int32_t dest; - int32_t val = (int32_t)valop; - int8_t shift = (int8_t)shiftop; - if (shift >= 32) { - if (val) { - SET_QC(); - dest = (val >> 31) ^ ~SIGNBIT; - } else { - dest = 0; - } - } else if (shift <= -32) { - dest = 0; - } else if (shift < 0) { - int64_t big_dest = ((int64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - if ((dest >> shift) != val) { - SET_QC(); - dest = (val >> 31) ^ ~SIGNBIT; - } - } - return dest; + return do_sqrshl_bhs(val, (int8_t)shift, 32, true, env->vfp.qc); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_qrshl_s64)(CPUARMState *env, uint64_t valop, uint64_t shiftop) +uint64_t HELPER(neon_qrshl_s64)(CPUARMState *env, uint64_t val, uint64_t shift) { - int8_t shift = (uint8_t)shiftop; - int64_t val = valop; - - if (shift >= 64) { - if (val) { - SET_QC(); - val = (val >> 63) ^ ~SIGNBIT64; - } - } else if (shift <= -64) { - val = 0; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == INT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x4000000000000000ULL; - } else { - val++; - val >>= 1; - } - } else { - int64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = (tmp >> 63) ^ ~SIGNBIT64; - } - } - return val; + return do_sqrshl_d(val, (int8_t)shift, true, env->vfp.qc); } uint32_t HELPER(neon_add_u8)(uint32_t a, uint32_t b) From patchwork Tue Mar 9 16:19:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396010 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2746506jai; Tue, 9 Mar 2021 08:55:55 -0800 (PST) X-Google-Smtp-Source: ABdhPJy0PcCWbntqyc2YkD7t9BPvXTWdBBLwlCaZA41MhwOzxJKEL0S0fgK2ryGiF00HzhYIkQU1 X-Received: by 2002:ad4:50d0:: with SMTP id e16mr2816992qvq.37.1615308955210; Tue, 09 Mar 2021 08:55:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615308955; cv=none; d=google.com; s=arc-20160816; b=eMFqJqFWKXEDbqcIiOKbzOXQhFTdL5mXmVoFF97Ref2yaz471d/ZcNGOm1AHGLXIN9 f8OsbatA6PelPYWkkksLTJmdY9Rl8W6ERAqTiXakYgB4J7wChRgdZoNGa+XqAWwesbXf YyUbOC5gZhHmTC+GPXFGwtR9JBKam3qu+tL2mw4WRBJAQ68zkVu8EowYFIspkiYLA1Eu W096CwhCng/NOzr4iPLU8vZ9dUOyh3ubK3h2ihOHfjbbzWfkF+KWmcqth3sgyUPY/FNz c24e3w7Sh0TrNqeOEeT+/lW0j2qlKNL43xSkG8xa/5F2A5mtm9MxVSgzdY1BjBTWoc2H 3QNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=tvbQbRXJincws9vxpJQp1b+ub8fQnU0GehgSc3qlp/c=; b=hfvIrs+cSDUpNZyW8abj6FXGtjx19bBv8joznhsyKnxgZr3mZYEXAr2k3y/bhxSHim amr/xAq1SCxsTLVwnM8JzNx7DC6TWxLIdBD/EvX/tAmlWo3EXaebykxA8zA0wohUAd+H VsX+qjlAHkrcdHimqzjMsIc5yZ4H7p3w8+2o34yRUrXlef2mO099V68io3+cPfgXwO4h 6E+QRAtSkxIYeEdFQo8V25LiPm1kenkgRsIzVTNNX7OlLlEToWYlarCU4iVFNgISj/CO lIXrUeiypKsCftqTilQsfoXM6A8QiM1BZ4Bg7L0xqIsACVQnjp0NTeg55JE4Wvm7RaPu TPWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i++NZKyT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 203si7869898qkh.175.2021.03.09.08.55.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 08:55:55 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i++NZKyT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37014 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJfeE-000316-Fv for patch@linaro.org; Tue, 09 Mar 2021 11:55:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45230) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6O-0006xz-Rj for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:57 -0500 Received: from mail-ot1-x32e.google.com ([2607:f8b0:4864:20::32e]:42013) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6K-0002Pv-BA for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:56 -0500 Received: by mail-ot1-x32e.google.com with SMTP id e45so13331787ote.9 for ; Tue, 09 Mar 2021 08:20:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tvbQbRXJincws9vxpJQp1b+ub8fQnU0GehgSc3qlp/c=; b=i++NZKyTPMraH8mGWhiUQdr2rI7JdzM/08WahnLiXRJi2d7bslqygx/XmB2dsNIu5N KPXHrPPyMUILWdpwLnTdVVLQko/rbuGYPEnD2m/DMZB8U9l6gvdeQf/2UGt/5+4dv12z fmzAgp3KuXUVS2qRdtA2CUNxCPMJiKcmbLLJK748jiC0HLpBWiwblz+UIMclK3UFSQbK BbkdQw3lVP6p5J6JLy+NZ22HlPY66muhLH3jSWN5gXmUJNlhrOu0VIGbv+p6b1e97QrA 5cnDcPLahEHjvO8Us8elmfuHzHkvRsQVo0T/JBfwmiNReSm/RMUUFOjblH2eQYbyBgre FXpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tvbQbRXJincws9vxpJQp1b+ub8fQnU0GehgSc3qlp/c=; b=PYBlsSUClLtA1oL++Q1mj6ph8VA/vSE11tki66ap4ozzcG+HWCKfWULxjvAb2ipzL+ QGF4t3+KHUtKmGN5fO3VFXouXq/bBkeQg2o2rXt3BLyoEfwRz7eQlN443mXqyDvFrHjN FVY61J5/d5ZzqcXCsw3yaLFENDBJ7RZYbDG+unqZBS3xgUG+lROc+xHeYlKH8RYkgz5z oXuIFMoYO6WcgLljTOBWqX/u+Yv6o2N3nvIKb7TyGiFHB3Zii2JCvDFrCgCcmLs8NWaM 5DOKWV+HsUw1xZ6K3xSTGYyAEvhnlsphZ/BpHfqE5fvRD3sSZFXDd76JhmB7INPbFyri STPw== X-Gm-Message-State: AOAM530nE4wQfpbBZHUBqIzyo8bo1JSvKDssYP7hBbHqn846lh+7YR7F unHED8Kkheq0c1+JMujWoL8yDQeax1cXpvPt X-Received: by 2002:a05:6830:1f55:: with SMTP id u21mr25823401oth.103.1615306850169; Tue, 09 Mar 2021 08:20:50 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:49 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 06/78] target/arm: Implement SVE2 saturating/rounding bitwise shift left (predicated) Date: Tue, 9 Mar 2021 08:19:29 -0800 Message-Id: <20210309162041.23124-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32e; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Shift values are always signed (laurent desnogues). --- target/arm/helper-sve.h | 54 ++++++++++++++++++++++++++ target/arm/sve.decode | 17 +++++++++ target/arm/sve_helper.c | 78 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 18 +++++++++ 4 files changed, 167 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 9992e93e2b..62106c74be 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -172,6 +172,60 @@ DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_d, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 5ba542969b..93f2479693 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1112,3 +1112,20 @@ URECPE 01000100 .. 000 000 101 ... ..... ..... @rd_pg_rn URSQRTE 01000100 .. 000 001 101 ... ..... ..... @rd_pg_rn SQABS 01000100 .. 001 000 101 ... ..... ..... @rd_pg_rn SQNEG 01000100 .. 001 001 101 ... ..... ..... @rd_pg_rn + +### SVE2 saturating/rounding bitwise shift left (predicated) + +SRSHL 01000100 .. 000 010 100 ... ..... ..... @rdn_pg_rm +URSHL 01000100 .. 000 011 100 ... ..... ..... @rdn_pg_rm +SRSHL 01000100 .. 000 110 100 ... ..... ..... @rdm_pg_rn # SRSHLR +URSHL 01000100 .. 000 111 100 ... ..... ..... @rdm_pg_rn # URSHLR + +SQSHL 01000100 .. 001 000 100 ... ..... ..... @rdn_pg_rm +UQSHL 01000100 .. 001 001 100 ... ..... ..... @rdn_pg_rm +SQSHL 01000100 .. 001 100 100 ... ..... ..... @rdm_pg_rn # SQSHLR +UQSHL 01000100 .. 001 101 100 ... ..... ..... @rdm_pg_rn # UQSHLR + +SQRSHL 01000100 .. 001 010 100 ... ..... ..... @rdn_pg_rm +UQRSHL 01000100 .. 001 011 100 ... ..... ..... @rdn_pg_rm +SQRSHL 01000100 .. 001 110 100 ... ..... ..... @rdm_pg_rn # SQRSHLR +UQRSHL 01000100 .. 001 111 100 ... ..... ..... @rdm_pg_rn # UQRSHLR diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 5dcdd26db0..5c911963ec 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -26,6 +26,7 @@ #include "tcg/tcg-gvec-desc.h" #include "fpu/softfloat.h" #include "tcg/tcg.h" +#include "vec_internal.h" /* Note that vector data is stored in host-endian 64-bit chunks, @@ -561,6 +562,83 @@ DO_ZPZZ(sve2_uadalp_zpzz_h, uint16_t, H1_2, do_uadalp_h) DO_ZPZZ(sve2_uadalp_zpzz_s, uint32_t, H1_4, do_uadalp_s) DO_ZPZZ_D(sve2_uadalp_zpzz_d, uint64_t, do_uadalp_d) +#define do_srshl_b(n, m) do_sqrshl_bhs(n, m, 8, true, NULL) +#define do_srshl_h(n, m) do_sqrshl_bhs(n, m, 16, true, NULL) +#define do_srshl_s(n, m) do_sqrshl_bhs(n, m, 32, true, NULL) +#define do_srshl_d(n, m) do_sqrshl_d(n, m, true, NULL) + +DO_ZPZZ(sve2_srshl_zpzz_b, int8_t, H1_2, do_srshl_b) +DO_ZPZZ(sve2_srshl_zpzz_h, int16_t, H1_2, do_srshl_h) +DO_ZPZZ(sve2_srshl_zpzz_s, int32_t, H1_4, do_srshl_s) +DO_ZPZZ_D(sve2_srshl_zpzz_d, int64_t, do_srshl_d) + +#define do_urshl_b(n, m) do_uqrshl_bhs(n, (int8_t)m, 8, true, NULL) +#define do_urshl_h(n, m) do_uqrshl_bhs(n, (int16_t)m, 16, true, NULL) +#define do_urshl_s(n, m) do_uqrshl_bhs(n, m, 32, true, NULL) +#define do_urshl_d(n, m) do_uqrshl_d(n, m, true, NULL) + +DO_ZPZZ(sve2_urshl_zpzz_b, uint8_t, H1_2, do_urshl_b) +DO_ZPZZ(sve2_urshl_zpzz_h, uint16_t, H1_2, do_urshl_h) +DO_ZPZZ(sve2_urshl_zpzz_s, uint32_t, H1_4, do_urshl_s) +DO_ZPZZ_D(sve2_urshl_zpzz_d, uint64_t, do_urshl_d) + +/* Unlike the NEON and AdvSIMD versions, there is no QC bit to set. */ +#define do_sqshl_b(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 8, false, &discard); }) +#define do_sqshl_h(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 16, false, &discard); }) +#define do_sqshl_s(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 32, false, &discard); }) +#define do_sqshl_d(n, m) \ + ({ uint32_t discard; do_sqrshl_d(n, m, false, &discard); }) + +DO_ZPZZ(sve2_sqshl_zpzz_b, int8_t, H1_2, do_sqshl_b) +DO_ZPZZ(sve2_sqshl_zpzz_h, int16_t, H1_2, do_sqshl_h) +DO_ZPZZ(sve2_sqshl_zpzz_s, int32_t, H1_4, do_sqshl_s) +DO_ZPZZ_D(sve2_sqshl_zpzz_d, int64_t, do_sqshl_d) + +#define do_uqshl_b(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int8_t)m, 8, false, &discard); }) +#define do_uqshl_h(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int16_t)m, 16, false, &discard); }) +#define do_uqshl_s(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, m, 32, false, &discard); }) +#define do_uqshl_d(n, m) \ + ({ uint32_t discard; do_uqrshl_d(n, m, false, &discard); }) + +DO_ZPZZ(sve2_uqshl_zpzz_b, uint8_t, H1_2, do_uqshl_b) +DO_ZPZZ(sve2_uqshl_zpzz_h, uint16_t, H1_2, do_uqshl_h) +DO_ZPZZ(sve2_uqshl_zpzz_s, uint32_t, H1_4, do_uqshl_s) +DO_ZPZZ_D(sve2_uqshl_zpzz_d, uint64_t, do_uqshl_d) + +#define do_sqrshl_b(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 8, true, &discard); }) +#define do_sqrshl_h(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 16, true, &discard); }) +#define do_sqrshl_s(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 32, true, &discard); }) +#define do_sqrshl_d(n, m) \ + ({ uint32_t discard; do_sqrshl_d(n, m, true, &discard); }) + +DO_ZPZZ(sve2_sqrshl_zpzz_b, int8_t, H1_2, do_sqrshl_b) +DO_ZPZZ(sve2_sqrshl_zpzz_h, int16_t, H1_2, do_sqrshl_h) +DO_ZPZZ(sve2_sqrshl_zpzz_s, int32_t, H1_4, do_sqrshl_s) +DO_ZPZZ_D(sve2_sqrshl_zpzz_d, int64_t, do_sqrshl_d) + +#define do_uqrshl_b(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int8_t)m, 8, true, &discard); }) +#define do_uqrshl_h(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int16_t)m, 16, true, &discard); }) +#define do_uqrshl_s(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, m, 32, true, &discard); }) +#define do_uqrshl_d(n, m) \ + ({ uint32_t discard; do_uqrshl_d(n, m, true, &discard); }) + +DO_ZPZZ(sve2_uqrshl_zpzz_b, uint8_t, H1_2, do_uqrshl_b) +DO_ZPZZ(sve2_uqrshl_zpzz_h, uint16_t, H1_2, do_uqrshl_h) +DO_ZPZZ(sve2_uqrshl_zpzz_s, uint32_t, H1_4, do_uqrshl_s) +DO_ZPZZ_D(sve2_uqrshl_zpzz_d, uint64_t, do_uqrshl_d) + #undef DO_ZPZZ #undef DO_ZPZZ_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 2d8b9d2e70..0950c32c09 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5932,3 +5932,21 @@ static bool trans_SQNEG(DisasContext *s, arg_rpr_esz *a) }; return do_sve2_zpz_ool(s, a, fns[a->esz]); } + +#define DO_SVE2_ZPZZ(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ +{ \ + static gen_helper_gvec_4 * const fns[4] = { \ + gen_helper_sve2_##name##_zpzz_b, gen_helper_sve2_##name##_zpzz_h, \ + gen_helper_sve2_##name##_zpzz_s, gen_helper_sve2_##name##_zpzz_d, \ + }; \ + return do_sve2_zpzz_ool(s, a, fns[a->esz]); \ +} + +DO_SVE2_ZPZZ(SQSHL, sqshl) +DO_SVE2_ZPZZ(SQRSHL, sqrshl) +DO_SVE2_ZPZZ(SRSHL, srshl) + +DO_SVE2_ZPZZ(UQSHL, uqshl) +DO_SVE2_ZPZZ(UQRSHL, uqrshl) +DO_SVE2_ZPZZ(URSHL, urshl) From patchwork Tue Mar 9 16:19:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396011 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2748090jai; Tue, 9 Mar 2021 08:58:39 -0800 (PST) X-Google-Smtp-Source: ABdhPJwCHEPCwbMSu/TVEqUW+yrE2tWQ0PmmXMV5PBs+3OkKLyeVlVHfneq/SUBr1zX4r0Mqrzik X-Received: by 2002:a6b:3bc7:: with SMTP id i190mr24046897ioa.163.1615309119262; Tue, 09 Mar 2021 08:58:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615309119; cv=none; d=google.com; s=arc-20160816; b=ngwJooYvMF7QMtJsxlApq+LD9AuxCBWBmBATBQNf8a7nqOH9TpvrN4966NZQnbPkl7 mrKt8LJHPr4JlLhfzU+CD7dFrPsNEpFwxkPQgWtIx3aDsEW/OjhDIW3VNnHcP/JfxLkb WjwzMf+xn13tjwpoqTPBEDoCkpm7uGkzmRt2aCe05H61uw1AAgx5DI9Y2ZKigL76bZ3v roS4rw0xRQDG8qYnAQYJ6aHuPJFQ/A8jAT8dP+SNOUyvJs9Y3SqJEtQkLkS9X1GD9SCC qDtigyvWEN8rNAJ8NKzuT4JiTTYfnG7FzU6Nd3fwJLPJonUHlFOKiBvDUWlII6FrBEcV U87A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=boAjEAjRJcOocs6tCIDq/S2Fa1ZUyTzEni2HeQU7C9w=; b=En8XzuGZ6WFjnlDWLSWY832YF+Z8Wpa1/it5JbCQyqDHaaGHPKiPbycJwnmF/vHKNE mV6mTai7ZSy5Aq/2A43TEObILadXQdTwZL//gTO6fzSvTe2XmGSKZY8hgaIjfA14r1ny 9Z4d/Hl47TFsKAcRnZnXELceLjNjzB/TOLaRJoyo0C4iovfbR6xDF4C8WW134JfFPi3n uwsv2uHhgh8mX2u9aY/xFsFfW6/PXRtx+z9ZKxwK8rJMlaeTi4lVPQWNStLOgS2eFVbi gFek4FxnFJ+CnFgrJOFRts5aU0i6tnrHhCCzh6SRrXzAJlKri/m1pVV5Rb973ezYLuNq Lp7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TJDpIg+b; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b13si12303677jat.120.2021.03.09.08.58.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 08:58:39 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TJDpIg+b; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45730 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJfgs-0007Fe-KR for patch@linaro.org; Tue, 09 Mar 2021 11:58:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45238) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6P-0006y2-4y for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:57 -0500 Received: from mail-oi1-x232.google.com ([2607:f8b0:4864:20::232]:47070) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6K-0002Q6-E6 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:56 -0500 Received: by mail-oi1-x232.google.com with SMTP id f3so15491943oiw.13 for ; Tue, 09 Mar 2021 08:20:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=boAjEAjRJcOocs6tCIDq/S2Fa1ZUyTzEni2HeQU7C9w=; b=TJDpIg+bytAmngBdyi6xgjrXxFTzXxroidOugO3DEpTlrhe3y+yo6IFkXokmXHuHoH +pUUsoMLadytQOCAjmgyGgsexRxD+dbNFJMQp7EOKn+noQfp83XE9BNhDEwj/fNmURAV kCBIY2gUKPfJClE2dnR+bpVIFKmEs709NWgB1UO8yfi8zltt6x7HcJCuUmZpU4xh4JYk senlB2nVGSoWQtUj8I8vK1p4nPS4klFANjXXyEuQLtQui4izh9VSFvRZmIgkToaheFkR 7S5/RUP/GNfmddaECoe3irrWr3NbcfCOwJqn7qdn+25TAf0XGJDUeM8pyFytQ2HwpN33 lhSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=boAjEAjRJcOocs6tCIDq/S2Fa1ZUyTzEni2HeQU7C9w=; b=JT+A599K/2Ug53M/Zp6muC1SvEmJiiOrKtUGUcVmd0riZJeAzdYat4QvBZ/BtCsnnE 5fq2OOl9EpdE4eo3nh7oNYU/yQhMphESO9wj9qkqEYBhkF/9nhSB2wYTeWbZ+QYiCgKn k1Vid7yPo1Rb/w/YJ0w9Gg67VWTcYxiplaL/RKeL5jzEsSOxL0K4lQZQQmVKN6XeSv5H OPKaO1Wqco6dkkCPnLddFE7Xkt33tPnto0IWNxq+GqoMVpKk15FDK5Z2487wGlZHANx3 /Gvkeg5ZiKbmOWtEfECpNiV0E/T3zZjnoz/JKqbjBG4Jb9myAs0PAA723m5/d9dt+uoY qylg== X-Gm-Message-State: AOAM5315flGbuzHydBCn8PwCXvd5CbBbIGkxRCjovMPJ5h4iiwaPuJmJ HV71U99aGZL9Yu+P12n1/5e6V8dPhjYdRSGf X-Received: by 2002:aca:b555:: with SMTP id e82mr3426409oif.65.1615306851437; Tue, 09 Mar 2021 08:20:51 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:50 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 07/78] target/arm: Implement SVE2 integer halving add/subtract (predicated) Date: Tue, 9 Mar 2021 08:19:30 -0800 Message-Id: <20210309162041.23124-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::232; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x232.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 54 ++++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 11 ++++++++ target/arm/sve_helper.c | 39 +++++++++++++++++++++++++++ target/arm/translate-sve.c | 8 ++++++ 4 files changed, 112 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 62106c74be..5fdc0d223a 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -226,6 +226,60 @@ DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_d, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 93f2479693..58c3f7ede4 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1129,3 +1129,14 @@ SQRSHL 01000100 .. 001 010 100 ... ..... ..... @rdn_pg_rm UQRSHL 01000100 .. 001 011 100 ... ..... ..... @rdn_pg_rm SQRSHL 01000100 .. 001 110 100 ... ..... ..... @rdm_pg_rn # SQRSHLR UQRSHL 01000100 .. 001 111 100 ... ..... ..... @rdm_pg_rn # UQRSHLR + +### SVE2 integer halving add/subtract (predicated) + +SHADD 01000100 .. 010 000 100 ... ..... ..... @rdn_pg_rm +UHADD 01000100 .. 010 001 100 ... ..... ..... @rdn_pg_rm +SHSUB 01000100 .. 010 010 100 ... ..... ..... @rdn_pg_rm +UHSUB 01000100 .. 010 011 100 ... ..... ..... @rdn_pg_rm +SRHADD 01000100 .. 010 100 100 ... ..... ..... @rdn_pg_rm +URHADD 01000100 .. 010 101 100 ... ..... ..... @rdn_pg_rm +SHSUB 01000100 .. 010 110 100 ... ..... ..... @rdm_pg_rn # SHSUBR +UHSUB 01000100 .. 010 111 100 ... ..... ..... @rdm_pg_rn # UHSUBR diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 5c911963ec..a7a934f3a4 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -639,6 +639,45 @@ DO_ZPZZ(sve2_uqrshl_zpzz_h, uint16_t, H1_2, do_uqrshl_h) DO_ZPZZ(sve2_uqrshl_zpzz_s, uint32_t, H1_4, do_uqrshl_s) DO_ZPZZ_D(sve2_uqrshl_zpzz_d, uint64_t, do_uqrshl_d) +#define DO_HADD_BHS(n, m) (((int64_t)n + m) >> 1) +#define DO_HADD_D(n, m) ((n >> 1) + (m >> 1) + (n & m & 1)) + +DO_ZPZZ(sve2_shadd_zpzz_b, int8_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_shadd_zpzz_h, int16_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_shadd_zpzz_s, int32_t, H1_4, DO_HADD_BHS) +DO_ZPZZ_D(sve2_shadd_zpzz_d, int64_t, DO_HADD_D) + +DO_ZPZZ(sve2_uhadd_zpzz_b, uint8_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_uhadd_zpzz_h, uint16_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_uhadd_zpzz_s, uint32_t, H1_4, DO_HADD_BHS) +DO_ZPZZ_D(sve2_uhadd_zpzz_d, uint64_t, DO_HADD_D) + +#define DO_RHADD_BHS(n, m) (((int64_t)n + m + 1) >> 1) +#define DO_RHADD_D(n, m) ((n >> 1) + (m >> 1) + ((n | m) & 1)) + +DO_ZPZZ(sve2_srhadd_zpzz_b, int8_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_srhadd_zpzz_h, int16_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_srhadd_zpzz_s, int32_t, H1_4, DO_RHADD_BHS) +DO_ZPZZ_D(sve2_srhadd_zpzz_d, int64_t, DO_RHADD_D) + +DO_ZPZZ(sve2_urhadd_zpzz_b, uint8_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_urhadd_zpzz_h, uint16_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_urhadd_zpzz_s, uint32_t, H1_4, DO_RHADD_BHS) +DO_ZPZZ_D(sve2_urhadd_zpzz_d, uint64_t, DO_RHADD_D) + +#define DO_HSUB_BHS(n, m) (((int64_t)n - m) >> 1) +#define DO_HSUB_D(n, m) ((n >> 1) - (m >> 1) - (~n & m & 1)) + +DO_ZPZZ(sve2_shsub_zpzz_b, int8_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_shsub_zpzz_h, int16_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_shsub_zpzz_s, int32_t, H1_4, DO_HSUB_BHS) +DO_ZPZZ_D(sve2_shsub_zpzz_d, int64_t, DO_HSUB_D) + +DO_ZPZZ(sve2_uhsub_zpzz_b, uint8_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_uhsub_zpzz_h, uint16_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_uhsub_zpzz_s, uint32_t, H1_4, DO_HSUB_BHS) +DO_ZPZZ_D(sve2_uhsub_zpzz_d, uint64_t, DO_HSUB_D) + #undef DO_ZPZZ #undef DO_ZPZZ_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0950c32c09..07e1315cca 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5950,3 +5950,11 @@ DO_SVE2_ZPZZ(SRSHL, srshl) DO_SVE2_ZPZZ(UQSHL, uqshl) DO_SVE2_ZPZZ(UQRSHL, uqrshl) DO_SVE2_ZPZZ(URSHL, urshl) + +DO_SVE2_ZPZZ(SHADD, shadd) +DO_SVE2_ZPZZ(SRHADD, srhadd) +DO_SVE2_ZPZZ(SHSUB, shsub) + +DO_SVE2_ZPZZ(UHADD, uhadd) +DO_SVE2_ZPZZ(URHADD, urhadd) +DO_SVE2_ZPZZ(UHSUB, uhsub) From patchwork Tue Mar 9 16:19:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396012 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2754374jai; Tue, 9 Mar 2021 09:05:32 -0800 (PST) X-Google-Smtp-Source: ABdhPJxHSSYypSy+HoVxM4INEEkY1eKNmEzpicJHI4+B8kh/CLX19HvuCWnDZMLDAvTjNTWxUvw6 X-Received: by 2002:ac5:c296:: with SMTP id h22mr17797615vkk.9.1615309532151; Tue, 09 Mar 2021 09:05:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615309532; cv=none; d=google.com; s=arc-20160816; b=D8baFzwPy2ncjHac7PMDvAdb6MRrmYzfcdkhWJ0lutEBf4Chj4BdyImWoE+IDl5WtG Nc7/hMDVXCUkuVs8liEhN0awfm57x/SZ+vxtFSPcxmn7BOX9nMPNdOa1T6vN2ZyuAbXJ xvCpJQ5NbXwtjyuEhbCESK69n79NYXgTuyDrB9kIbRy8cVRWdNLgZO7Lq6UKNjMQTEE7 cO9klvhLMjHcJceCtpGUvfxctzwNQ6nW4uJkLNUE9fOyTc0DDjTOxD44uhCDmePE0UdR A5yjxanOTTHwltRMKlZZUVLxWphoSmozNGrihgZaJ/jUUgoFWe6Ly+vzr+yPExWPtCNS /abA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=+s4Ex6WJ2oxr+RR4s+3+c7rbv+0uvqpEzs763JIOfME=; b=kSYLYCis36ScIBvKXXIexrLzOR5G5jTh2dkgK5bjAk4RLGYXl8rnFzrdpuxE1bwfwY dOcCotJJNZqasFQx1yZ0sVvKN8BKrrK6/B1z5ImRAEk0oj6xaB6DWvjSadm9nHpquw3p 2dIvDAsRej5QXbyv+XTPENcOZBc3CJmvEW7vRWy8OFWJgzHHgyn4FFOsBTQ2IG0cPQTs XoWNLqnuLiN9GGyJZCUc6kqZIuulzNoroeFS4pnP90PaD7unGz/au9TZWuir6RMszz+o VRLAvWq6XLyjceQBQpjVGq8z9Jxsec905hh64v+GDuSvLi1ShS0jQzsTBk2FCmkfD9kD jGVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=y4Y7ubhR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p16si185089vsf.92.2021.03.09.09.05.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:05:32 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=y4Y7ubhR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54322 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJfnX-0002gn-FK for patch@linaro.org; Tue, 09 Mar 2021 12:05:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45280) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6Q-00070U-Kv for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:58 -0500 Received: from mail-ot1-x333.google.com ([2607:f8b0:4864:20::333]:42018) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6L-0002QP-Mb for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:20:58 -0500 Received: by mail-ot1-x333.google.com with SMTP id e45so13331919ote.9 for ; Tue, 09 Mar 2021 08:20:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+s4Ex6WJ2oxr+RR4s+3+c7rbv+0uvqpEzs763JIOfME=; b=y4Y7ubhRzzGM1uGSydXZo2o0+lJ90YagFX3foXSRID0URQ0J74P8ndPTmRtiILXyhu oGQUUFULMICx8mmPle8AOg8u5XXY8kux2Ba+5jiToj3FKSD2SO68TqTwT1jFhLNRTAta 7CbXdMFN5GmHMJVCOr7JqMS5nN61sa25BttreG7d1EjnA3rHa9FMPuakAcq7tnNce44t Lp1g26I+z0KyOJB+jIXTwXiJsmRspOp2hTYgwlgAlR8J8fOQDYiHPDR5EDjQUhk66BpP Kaq1ybRY36zTwookMlKr7UEti333D+c9maEuFmC9z2X+HgQqIPhuFuveoP2BOr2nj/OY ee0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+s4Ex6WJ2oxr+RR4s+3+c7rbv+0uvqpEzs763JIOfME=; b=t1FVvgfcKPbwnVTVHPfMevSB7jBXgXIUu+DsUy+iLXnO2inw1HnFrZI1RgetNM6MXl tAm9UBDLxLeOZE08K8boaIhzbGyVLdM2djp4p2N5nOK7Ev/vG9+2Xp/HwSicjAJ2nbW+ KXECmHtt556itmY8gkAlhsssJiWFH6n+RAmJ2p7/+t1YPsNlNsK1alYgzXCEmiInME07 tCYs0RMDzbRjpcUusYyWoNZawNcJNfxFNHqoAIrILVMS8FZKaBfKYQor2n34oq9KFu9O pT6Sqrc+4BXfrRTNmzHUnhbznHqUSLX4Is0ndRleakcc/EmZfZGyhetsrkFcQk87+Xe8 8smA== X-Gm-Message-State: AOAM532XmciR/GWXTDHpvBB01rUOlrtSypGCPL3ArmM1AjKuQrtJ4dgs aI1GyggzuNR62Beckugj5bmLbPxZdWu9pr0y X-Received: by 2002:a05:6830:558:: with SMTP id l24mr24494637otb.209.1615306852671; Tue, 09 Mar 2021 08:20:52 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:52 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 08/78] target/arm: Implement SVE2 integer pairwise arithmetic Date: Tue, 9 Mar 2021 08:19:31 -0800 Message-Id: <20210309162041.23124-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::333; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x333.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Load all inputs before writing any output (laurent desnogues) --- target/arm/helper-sve.h | 45 ++++++++++++++++++++++ target/arm/sve.decode | 8 ++++ target/arm/sve_helper.c | 76 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 6 +++ 4 files changed, 135 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 5fdc0d223a..09bc067dd4 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -326,6 +326,51 @@ DEF_HELPER_FLAGS_5(sve_sel_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve_sel_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_asr_zpzw_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_asr_zpzw_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 58c3f7ede4..61a3321325 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1140,3 +1140,11 @@ SRHADD 01000100 .. 010 100 100 ... ..... ..... @rdn_pg_rm URHADD 01000100 .. 010 101 100 ... ..... ..... @rdn_pg_rm SHSUB 01000100 .. 010 110 100 ... ..... ..... @rdm_pg_rn # SHSUBR UHSUB 01000100 .. 010 111 100 ... ..... ..... @rdm_pg_rn # UHSUBR + +### SVE2 integer pairwise arithmetic + +ADDP 01000100 .. 010 001 101 ... ..... ..... @rdn_pg_rm +SMAXP 01000100 .. 010 100 101 ... ..... ..... @rdn_pg_rm +UMAXP 01000100 .. 010 101 101 ... ..... ..... @rdn_pg_rm +SMINP 01000100 .. 010 110 101 ... ..... ..... @rdn_pg_rm +UMINP 01000100 .. 010 111 101 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index a7a934f3a4..19ae7722f0 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -681,6 +681,82 @@ DO_ZPZZ_D(sve2_uhsub_zpzz_d, uint64_t, DO_HSUB_D) #undef DO_ZPZZ #undef DO_ZPZZ_D +/* + * Three operand expander, operating on element pairs. + * If the slot I is even, the elements from from VN {I, I+1}. + * If the slot I is odd, the elements from from VM {I-1, I}. + * Load all of the input elements in each pair before overwriting output. + */ +#define DO_ZPZZ_PAIR(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + TYPE n0 = *(TYPE *)(vn + H(i)); \ + TYPE m0 = *(TYPE *)(vm + H(i)); \ + TYPE n1 = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE m1 = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(n0, n1); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(m0, m1); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ +} + +/* Similarly, specialized for 64-bit operands. */ +#define DO_ZPZZ_PAIR_D(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; \ + TYPE *d = vd, *n = vn, *m = vm; \ + uint8_t *pg = vg; \ + for (i = 0; i < opr_sz; i += 2) { \ + TYPE n0 = n[i], n1 = n[i + 1]; \ + TYPE m0 = m[i], m1 = m[i + 1]; \ + if (pg[H1(i)] & 1) { \ + d[i] = OP(n0, n1); \ + } \ + if (pg[H1(i + 1)] & 1) { \ + d[i + 1] = OP(m0, m1); \ + } \ + } \ +} + +DO_ZPZZ_PAIR(sve2_addp_zpzz_b, uint8_t, H1_2, DO_ADD) +DO_ZPZZ_PAIR(sve2_addp_zpzz_h, uint16_t, H1_2, DO_ADD) +DO_ZPZZ_PAIR(sve2_addp_zpzz_s, uint32_t, H1_4, DO_ADD) +DO_ZPZZ_PAIR_D(sve2_addp_zpzz_d, uint64_t, DO_ADD) + +DO_ZPZZ_PAIR(sve2_umaxp_zpzz_b, uint8_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_umaxp_zpzz_h, uint16_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_umaxp_zpzz_s, uint32_t, H1_4, DO_MAX) +DO_ZPZZ_PAIR_D(sve2_umaxp_zpzz_d, uint64_t, DO_MAX) + +DO_ZPZZ_PAIR(sve2_uminp_zpzz_b, uint8_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_uminp_zpzz_h, uint16_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_uminp_zpzz_s, uint32_t, H1_4, DO_MIN) +DO_ZPZZ_PAIR_D(sve2_uminp_zpzz_d, uint64_t, DO_MIN) + +DO_ZPZZ_PAIR(sve2_smaxp_zpzz_b, int8_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_smaxp_zpzz_h, int16_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_smaxp_zpzz_s, int32_t, H1_4, DO_MAX) +DO_ZPZZ_PAIR_D(sve2_smaxp_zpzz_d, int64_t, DO_MAX) + +DO_ZPZZ_PAIR(sve2_sminp_zpzz_b, int8_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_sminp_zpzz_h, int16_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_sminp_zpzz_s, int32_t, H1_4, DO_MIN) +DO_ZPZZ_PAIR_D(sve2_sminp_zpzz_d, int64_t, DO_MIN) + +#undef DO_ZPZZ_PAIR +#undef DO_ZPZZ_PAIR_D + /* Three-operand expander, controlled by a predicate, in which the * third operand is "wide". That is, for D = N op M, the same 64-bit * value of M is used with all of the narrower values of N. diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 07e1315cca..0675abf1d8 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5958,3 +5958,9 @@ DO_SVE2_ZPZZ(SHSUB, shsub) DO_SVE2_ZPZZ(UHADD, uhadd) DO_SVE2_ZPZZ(URHADD, urhadd) DO_SVE2_ZPZZ(UHSUB, uhsub) + +DO_SVE2_ZPZZ(ADDP, addp) +DO_SVE2_ZPZZ(SMAXP, smaxp) +DO_SVE2_ZPZZ(UMAXP, umaxp) +DO_SVE2_ZPZZ(SMINP, sminp) +DO_SVE2_ZPZZ(UMINP, uminp) From patchwork Tue Mar 9 16:19:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396013 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2757346jai; Tue, 9 Mar 2021 09:08:49 -0800 (PST) X-Google-Smtp-Source: ABdhPJxIK81rposKOYVPsIycCjw5gpeUmT2KSw7r0sv8HYfG4mPg5Rnpnpw2m06sqA1BunRKQYOO X-Received: by 2002:ab0:498c:: with SMTP id e12mr16472367uad.10.1615309729379; Tue, 09 Mar 2021 09:08:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615309729; cv=none; d=google.com; s=arc-20160816; b=ZNJEyf+1kMJ7a7SXP+KkDAYU58KKln6SAsr8+zmALQJEyXuBCdh1eyNvQk+hJQ243V uzexRXawtvLsPRfRadWS0/72CxrCLkA0f5qJaWoPLPni5MB0IBPzwVOBLPlOGyu1/kXh DJrq1E3XdhHmeEgT4yBNb1/9Fmn08crZwHQNI178+QhXDx3652j4/V1+yGRlMcxi0jTW suLNwW86Ayn9CgmR+hclkaq20NQWddRmGyhS9VBnXUF83AzRiBAGb/Vhrt+txQ+rY3QS dhmP0mXrJay13M+4kkELdW/0n45b3/SdzpHsY8sj4gPomszg/y1plE1JFH04mD94tySd uoyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=gB6zuzZoOjocfFDpiWWgFoXH2tOq/6UVbrgzv+51ugs=; b=pQfOoph+zNzGNacbSdb2hDqoMKSZzKwofo6F8j3FUjI/ssfXvqlalxwZZPfpTDctxT a/7fEKKtu1Bto5IzqJitdM2IYYNYSTwB58EUaZjFv/9SzLc+F5AJlsxXCmDUmy5c9RuT l6bOOz92xoZdLBQz+DHb7GlXRSUqeEr3dHjMH7RUkSm4pmeFoUO6ZDSlRDO7d3iLW28v Zt6PDeOijCYpV0pnYbUZG/qXGA+4T/WiXpkVw6pUw5gMGxerco43BeIpHnt70Pfpy9c7 ulPV9ldws8RMWuVd0Ko13/sTVYBF9rxq3IayWDYokDY50S8b/L5w6TM3yMMC0wLsMlEf r0rg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=y9ZfojB7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c20si2585233uar.68.2021.03.09.09.08.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:08:49 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=y9ZfojB7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJfqi-00086J-Mr for patch@linaro.org; Tue, 09 Mar 2021 12:08:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45406) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6Y-00077R-Av for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:07 -0500 Received: from mail-oi1-x232.google.com ([2607:f8b0:4864:20::232]:37259) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6M-0002Qj-OH for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:06 -0500 Received: by mail-oi1-x232.google.com with SMTP id u198so10576577oia.4 for ; Tue, 09 Mar 2021 08:20:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gB6zuzZoOjocfFDpiWWgFoXH2tOq/6UVbrgzv+51ugs=; b=y9ZfojB76qCbWjQKLtK+nTRbWBPSBOPiVteUu+D2yCyGSpBOSTQ/XPkIM+TnepbAwA K5eyrCt74FCBLnOrVnhxfH416/moytrP0PVJZ31ZqerEWi7ZxPBTKL0bac1nFu0eBDSd ZJvicWgLMAUCIk1ZOmPa5rtLWZ7gumPj+DQAKLpGZBlzSj49jf8ZAyk8B2I3AQjhDMfM yyRiT1QzvAb9ZCpo4x3his80XP8KocqVle34g1BcfLZ+fS5BfL8hMpX7ssMsEm6tC1+b pbVZs3/Y7/Jz17XvGC3oSZWw/bRmHRrrYQGfHB4SxRglkbmQ35QWiBLPzbLhSiMYVL6e 6Dww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gB6zuzZoOjocfFDpiWWgFoXH2tOq/6UVbrgzv+51ugs=; b=RFHu75/5QJWBEYWdV0OX/xop6g6HB0/72aAJ55vX02J8wQq9bdJYQekuB7IkTmkdfh 1lrx+ZGfM+kSykEYDXZExo0VEa0Kf3yN1gdcmK+Tss24sngR01p/sIlY4Vjk6OkLwe7A b66/dDBzv80DRtTEXvQMRRjMweGAvcV0Q5BLssegTcaBzJiuKvdQDmWquKZtPRkY+Au5 8RDE+cKwd+fV8XPfPsPlNNaJBjfO1C+7oB43jvcN5K4di1DPMPwjHfzt/6ucxZGh+T6O sI9rRuMw735NTnTFCPB9t70Omoeq3T4MesfXe6U6jXsPTpDiTZxzzMsUawnSHWo8K7hn z9TA== X-Gm-Message-State: AOAM530EnNmF5zxOQg0lD6S0HACVivSu4sKOLRI3rLaQTYKBiRBYfxeS Sg7ZoU3gs1ti82+UXICmgNo9IrLsjN/Sq+X6 X-Received: by 2002:aca:d6d3:: with SMTP id n202mr3547619oig.175.1615306853699; Tue, 09 Mar 2021 08:20:53 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:53 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 09/78] target/arm: Implement SVE2 saturating add/subtract (predicated) Date: Tue, 9 Mar 2021 08:19:32 -0800 Message-Id: <20210309162041.23124-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::232; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x232.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 54 +++++++++++ target/arm/sve.decode | 11 +++ target/arm/sve_helper.c | 194 ++++++++++++++++++++++++++----------- target/arm/translate-sve.c | 7 ++ 4 files changed, 210 insertions(+), 56 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 09bc067dd4..37461c9927 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -371,6 +371,60 @@ DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_asr_zpzw_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_asr_zpzw_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 61a3321325..cd4f73265f 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1148,3 +1148,14 @@ SMAXP 01000100 .. 010 100 101 ... ..... ..... @rdn_pg_rm UMAXP 01000100 .. 010 101 101 ... ..... ..... @rdn_pg_rm SMINP 01000100 .. 010 110 101 ... ..... ..... @rdn_pg_rm UMINP 01000100 .. 010 111 101 ... ..... ..... @rdn_pg_rm + +### SVE2 saturating add/subtract (predicated) + +SQADD_zpzz 01000100 .. 011 000 100 ... ..... ..... @rdn_pg_rm +UQADD_zpzz 01000100 .. 011 001 100 ... ..... ..... @rdn_pg_rm +SQSUB_zpzz 01000100 .. 011 010 100 ... ..... ..... @rdn_pg_rm +UQSUB_zpzz 01000100 .. 011 011 100 ... ..... ..... @rdn_pg_rm +SUQADD 01000100 .. 011 100 100 ... ..... ..... @rdn_pg_rm +USQADD 01000100 .. 011 101 100 ... ..... ..... @rdn_pg_rm +SQSUB_zpzz 01000100 .. 011 110 100 ... ..... ..... @rdm_pg_rn # SQSUBR +UQSUB_zpzz 01000100 .. 011 111 100 ... ..... ..... @rdm_pg_rn # UQSUBR diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 19ae7722f0..009d49e81d 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -678,6 +678,135 @@ DO_ZPZZ(sve2_uhsub_zpzz_h, uint16_t, H1_2, DO_HSUB_BHS) DO_ZPZZ(sve2_uhsub_zpzz_s, uint32_t, H1_4, DO_HSUB_BHS) DO_ZPZZ_D(sve2_uhsub_zpzz_d, uint64_t, DO_HSUB_D) +static inline int32_t do_sat_bhs(int64_t val, int64_t min, int64_t max) +{ + return val >= max ? max : val <= min ? min : val; +} + +#define DO_SQADD_B(n, m) do_sat_bhs((int64_t)n + m, INT8_MIN, INT8_MAX) +#define DO_SQADD_H(n, m) do_sat_bhs((int64_t)n + m, INT16_MIN, INT16_MAX) +#define DO_SQADD_S(n, m) do_sat_bhs((int64_t)n + m, INT32_MIN, INT32_MAX) + +static inline int64_t do_sqadd_d(int64_t n, int64_t m) +{ + int64_t r = n + m; + if (((r ^ n) & ~(n ^ m)) < 0) { + /* Signed overflow. */ + return r < 0 ? INT64_MAX : INT64_MIN; + } + return r; +} + +DO_ZPZZ(sve2_sqadd_zpzz_b, int8_t, H1_2, DO_SQADD_B) +DO_ZPZZ(sve2_sqadd_zpzz_h, int16_t, H1_2, DO_SQADD_H) +DO_ZPZZ(sve2_sqadd_zpzz_s, int32_t, H1_4, DO_SQADD_S) +DO_ZPZZ_D(sve2_sqadd_zpzz_d, int64_t, do_sqadd_d) + +#define DO_UQADD_B(n, m) do_sat_bhs((int64_t)n + m, 0, UINT8_MAX) +#define DO_UQADD_H(n, m) do_sat_bhs((int64_t)n + m, 0, UINT16_MAX) +#define DO_UQADD_S(n, m) do_sat_bhs((int64_t)n + m, 0, UINT32_MAX) + +static inline uint64_t do_uqadd_d(uint64_t n, uint64_t m) +{ + uint64_t r = n + m; + return r < n ? UINT64_MAX : r; +} + +DO_ZPZZ(sve2_uqadd_zpzz_b, uint8_t, H1_2, DO_UQADD_B) +DO_ZPZZ(sve2_uqadd_zpzz_h, uint16_t, H1_2, DO_UQADD_H) +DO_ZPZZ(sve2_uqadd_zpzz_s, uint32_t, H1_4, DO_UQADD_S) +DO_ZPZZ_D(sve2_uqadd_zpzz_d, uint64_t, do_uqadd_d) + +#define DO_SQSUB_B(n, m) do_sat_bhs((int64_t)n - m, INT8_MIN, INT8_MAX) +#define DO_SQSUB_H(n, m) do_sat_bhs((int64_t)n - m, INT16_MIN, INT16_MAX) +#define DO_SQSUB_S(n, m) do_sat_bhs((int64_t)n - m, INT32_MIN, INT32_MAX) + +static inline int64_t do_sqsub_d(int64_t n, int64_t m) +{ + int64_t r = n - m; + if (((r ^ n) & (n ^ m)) < 0) { + /* Signed overflow. */ + return r < 0 ? INT64_MAX : INT64_MIN; + } + return r; +} + +DO_ZPZZ(sve2_sqsub_zpzz_b, int8_t, H1_2, DO_SQSUB_B) +DO_ZPZZ(sve2_sqsub_zpzz_h, int16_t, H1_2, DO_SQSUB_H) +DO_ZPZZ(sve2_sqsub_zpzz_s, int32_t, H1_4, DO_SQSUB_S) +DO_ZPZZ_D(sve2_sqsub_zpzz_d, int64_t, do_sqsub_d) + +#define DO_UQSUB_B(n, m) do_sat_bhs((int64_t)n - m, 0, UINT8_MAX) +#define DO_UQSUB_H(n, m) do_sat_bhs((int64_t)n - m, 0, UINT16_MAX) +#define DO_UQSUB_S(n, m) do_sat_bhs((int64_t)n - m, 0, UINT32_MAX) + +static inline uint64_t do_uqsub_d(uint64_t n, uint64_t m) +{ + return n > m ? n - m : 0; +} + +DO_ZPZZ(sve2_uqsub_zpzz_b, uint8_t, H1_2, DO_UQSUB_B) +DO_ZPZZ(sve2_uqsub_zpzz_h, uint16_t, H1_2, DO_UQSUB_H) +DO_ZPZZ(sve2_uqsub_zpzz_s, uint32_t, H1_4, DO_UQSUB_S) +DO_ZPZZ_D(sve2_uqsub_zpzz_d, uint64_t, do_uqsub_d) + +#define DO_SUQADD_B(n, m) \ + do_sat_bhs((int64_t)(int8_t)n + m, INT8_MIN, INT8_MAX) +#define DO_SUQADD_H(n, m) \ + do_sat_bhs((int64_t)(int16_t)n + m, INT16_MIN, INT16_MAX) +#define DO_SUQADD_S(n, m) \ + do_sat_bhs((int64_t)(int32_t)n + m, INT32_MIN, INT32_MAX) + +static inline int64_t do_suqadd_d(int64_t n, uint64_t m) +{ + uint64_t r = n + m; + + if (n < 0) { + /* Note that m - abs(n) cannot underflow. */ + if (r > INT64_MAX) { + /* Result is either very large positive or negative. */ + if (m > -n) { + /* m > abs(n), so r is a very large positive. */ + return INT64_MAX; + } + /* Result is negative. */ + } + } else { + /* Both inputs are positive: check for overflow. */ + if (r < m || r > INT64_MAX) { + return INT64_MAX; + } + } + return r; +} + +DO_ZPZZ(sve2_suqadd_zpzz_b, uint8_t, H1_2, DO_SUQADD_B) +DO_ZPZZ(sve2_suqadd_zpzz_h, uint16_t, H1_2, DO_SUQADD_H) +DO_ZPZZ(sve2_suqadd_zpzz_s, uint32_t, H1_4, DO_SUQADD_S) +DO_ZPZZ_D(sve2_suqadd_zpzz_d, uint64_t, do_suqadd_d) + +#define DO_USQADD_B(n, m) \ + do_sat_bhs((int64_t)n + (int8_t)m, 0, UINT8_MAX) +#define DO_USQADD_H(n, m) \ + do_sat_bhs((int64_t)n + (int16_t)m, 0, UINT16_MAX) +#define DO_USQADD_S(n, m) \ + do_sat_bhs((int64_t)n + (int32_t)m, 0, UINT32_MAX) + +static inline uint64_t do_usqadd_d(uint64_t n, int64_t m) +{ + uint64_t r = n + m; + + if (m < 0) { + return n < -m ? 0 : r; + } + return r < n ? UINT64_MAX : r; +} + +DO_ZPZZ(sve2_usqadd_zpzz_b, uint8_t, H1_2, DO_USQADD_B) +DO_ZPZZ(sve2_usqadd_zpzz_h, uint16_t, H1_2, DO_USQADD_H) +DO_ZPZZ(sve2_usqadd_zpzz_s, uint32_t, H1_4, DO_USQADD_S) +DO_ZPZZ_D(sve2_usqadd_zpzz_d, uint64_t, do_usqadd_d) + #undef DO_ZPZZ #undef DO_ZPZZ_D @@ -1614,13 +1743,7 @@ void HELPER(sve_sqaddi_b)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int8_t)) { - int r = *(int8_t *)(a + i) + b; - if (r > INT8_MAX) { - r = INT8_MAX; - } else if (r < INT8_MIN) { - r = INT8_MIN; - } - *(int8_t *)(d + i) = r; + *(int8_t *)(d + i) = DO_SQADD_B(b, *(int8_t *)(a + i)); } } @@ -1629,13 +1752,7 @@ void HELPER(sve_sqaddi_h)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int16_t)) { - int r = *(int16_t *)(a + i) + b; - if (r > INT16_MAX) { - r = INT16_MAX; - } else if (r < INT16_MIN) { - r = INT16_MIN; - } - *(int16_t *)(d + i) = r; + *(int16_t *)(d + i) = DO_SQADD_H(b, *(int16_t *)(a + i)); } } @@ -1644,13 +1761,7 @@ void HELPER(sve_sqaddi_s)(void *d, void *a, int64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int32_t)) { - int64_t r = *(int32_t *)(a + i) + b; - if (r > INT32_MAX) { - r = INT32_MAX; - } else if (r < INT32_MIN) { - r = INT32_MIN; - } - *(int32_t *)(d + i) = r; + *(int32_t *)(d + i) = DO_SQADD_S(b, *(int32_t *)(a + i)); } } @@ -1659,13 +1770,7 @@ void HELPER(sve_sqaddi_d)(void *d, void *a, int64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int64_t)) { - int64_t ai = *(int64_t *)(a + i); - int64_t r = ai + b; - if (((r ^ ai) & ~(ai ^ b)) < 0) { - /* Signed overflow. */ - r = (r < 0 ? INT64_MAX : INT64_MIN); - } - *(int64_t *)(d + i) = r; + *(int64_t *)(d + i) = do_sqadd_d(b, *(int64_t *)(a + i)); } } @@ -1678,13 +1783,7 @@ void HELPER(sve_uqaddi_b)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint8_t)) { - int r = *(uint8_t *)(a + i) + b; - if (r > UINT8_MAX) { - r = UINT8_MAX; - } else if (r < 0) { - r = 0; - } - *(uint8_t *)(d + i) = r; + *(uint8_t *)(d + i) = DO_UQADD_B(b, *(uint8_t *)(a + i)); } } @@ -1693,13 +1792,7 @@ void HELPER(sve_uqaddi_h)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint16_t)) { - int r = *(uint16_t *)(a + i) + b; - if (r > UINT16_MAX) { - r = UINT16_MAX; - } else if (r < 0) { - r = 0; - } - *(uint16_t *)(d + i) = r; + *(uint16_t *)(d + i) = DO_UQADD_H(b, *(uint16_t *)(a + i)); } } @@ -1708,13 +1801,7 @@ void HELPER(sve_uqaddi_s)(void *d, void *a, int64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint32_t)) { - int64_t r = *(uint32_t *)(a + i) + b; - if (r > UINT32_MAX) { - r = UINT32_MAX; - } else if (r < 0) { - r = 0; - } - *(uint32_t *)(d + i) = r; + *(uint32_t *)(d + i) = DO_UQADD_S(b, *(uint32_t *)(a + i)); } } @@ -1723,11 +1810,7 @@ void HELPER(sve_uqaddi_d)(void *d, void *a, uint64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint64_t)) { - uint64_t r = *(uint64_t *)(a + i) + b; - if (r < b) { - r = UINT64_MAX; - } - *(uint64_t *)(d + i) = r; + *(uint64_t *)(d + i) = do_uqadd_d(b, *(uint64_t *)(a + i)); } } @@ -1736,8 +1819,7 @@ void HELPER(sve_uqsubi_d)(void *d, void *a, uint64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint64_t)) { - uint64_t ai = *(uint64_t *)(a + i); - *(uint64_t *)(d + i) = (ai < b ? 0 : ai - b); + *(uint64_t *)(d + i) = do_uqsub_d(*(uint64_t *)(a + i), b); } } diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0675abf1d8..fc8e2a84f2 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5964,3 +5964,10 @@ DO_SVE2_ZPZZ(SMAXP, smaxp) DO_SVE2_ZPZZ(UMAXP, umaxp) DO_SVE2_ZPZZ(SMINP, sminp) DO_SVE2_ZPZZ(UMINP, uminp) + +DO_SVE2_ZPZZ(SQADD_zpzz, sqadd) +DO_SVE2_ZPZZ(UQADD_zpzz, uqadd) +DO_SVE2_ZPZZ(SQSUB_zpzz, sqsub) +DO_SVE2_ZPZZ(UQSUB_zpzz, uqsub) +DO_SVE2_ZPZZ(SUQADD, suqadd) +DO_SVE2_ZPZZ(USQADD, usqadd) From patchwork Tue Mar 9 16:19:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396022 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2775726jai; Tue, 9 Mar 2021 09:30:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJwXAWfRXbySVPdYigwKLvOhMsyDGqeXWp3m2NcVsedjZ7dlgSM0dqWYgQhxia5J3aaV56A8 X-Received: by 2002:a1f:3492:: with SMTP id b140mr6201156vka.17.1615311057310; Tue, 09 Mar 2021 09:30:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311057; cv=none; d=google.com; s=arc-20160816; b=p9RXAH16aAyL3XX3bNXFWLGp5xZjz5KRwA6dLw3+il0pGuH5A5t50UgALiulW9Kdjx NdDTpEixdsSI2I1dn6y37TJUFhasXTa4YZJbdKECSlZkaxsQtLVLMhcoO7RjbxGXgvCK 7MMam4Cpp2MOTAGnH6g/x5fQJloVQKilp6fg2ovwEC/fU7I8mUd7KMN3CjDW8Jpp/k35 UPur+lEbDKqrEkAWLyd69JCDMEgtYMRWliZxJVVOZSPfUafQD1CM/62oEkLruztFyzmH 6YlpzGd8q0AlUNsTOn+8wXqTLn9xzSbJGA5ERo4TnFgZLP2jH6r9CgAqh5qW5gNHXdMp I8VA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ALuD3u4vRQ1W3ilZFnOTMRI40+P7noMwmrau2Rl0oOs=; b=zSOU0nPMxwYCOrP9bGemNCx3yTHd/A3KKdb0C9iDJDmUV5g64Z7W8eqkaj4IauG1Ff pE7+mZmofrmRVfC8bns2mgb9i/8+4AkYyUrIk+Frw+QQ3F9tH6RVG7cBAGzi1a1IaCAd XQG4G3QJYAAKh+3uxblRBzu5YFMXFqH1zFVngA6s+j6k4lMcrMuO0d+E10KJntXdw1KH /UV7Dlc7nanYTWNBOKahxoNxga9hzbMoVgXAnh8bE4II/5zWpoypZ3lnKl9CVpFs3XAQ s1V7WoENi6QdoJo7P4up8JvpxZPdVN0nw2kVrK1dDG6WUIObvE80eIJOsCgqybJeKd7A +2LA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VA+lmLkL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t193si2556389vst.73.2021.03.09.09.30.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:30:57 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VA+lmLkL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46820 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgC8-000129-IX for patch@linaro.org; Tue, 09 Mar 2021 12:30:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45490) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6d-00079O-4s for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:11 -0500 Received: from mail-oi1-x233.google.com ([2607:f8b0:4864:20::233]:45925) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6O-0002R3-8n for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:10 -0500 Received: by mail-oi1-x233.google.com with SMTP id t83so5550481oih.12 for ; Tue, 09 Mar 2021 08:20:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ALuD3u4vRQ1W3ilZFnOTMRI40+P7noMwmrau2Rl0oOs=; b=VA+lmLkLyJ7OOnHoHD0nTuRs+/4t6k53adCvz0cl5yr9GwOqyjhIlNptMmMU6E+Eqp cEgdabQ221LwL9oGZZcMaX1Rt7Vu4MrRotqDCRv7//oJUCs217c2IEWe+wLkT+fAifoy dgR8vPkuLsPAgwpdYfUIwPTuEtY9iHkVeKbObn/64wjuEkVZVWVkPXz3DDMe+jmHQiBC b21K21S87FLCdagT7hqAG050j/D98oYJ7q8GMas3YrKa6cjO+96b3dWS+WPw5FWwir2I ed2IimXcsl6aK7w3n72snfOBGGwykg2OopgdSIibdv2bow1KQ4HMNqaryN3/v93voLgz 3rZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ALuD3u4vRQ1W3ilZFnOTMRI40+P7noMwmrau2Rl0oOs=; b=LKx4ZnuCRWYnWPyttUNVGyXKcj9scoKuu7KeyQA0nXsqCSOVfczcytq8BGqrR4jcvd bRFdOAucNkZwUrnrhTeIZjWIIkTes0fgEEEeqEi1JGg5W5UeAguhhVjBz/9VxhNwpOph WBUoWquxGIUbsr/9JBRf6dxMi5CNkcAYZRtsopCq3bCrqaPpXGYHjPhXrKmTZLFA8wIQ iYGLsKSBZITuhiHNqOW4dYx9llNGilELsknkO27d4Fwu4JI8Sm8waLkDJM7fUr4TZajE Y+Ig+vLmk157HvQvBqDfyFWZt7GUZl7MV5pwKLpYSm/qEHgtwutoShHDrFasTMn6H4S3 4PvA== X-Gm-Message-State: AOAM532aNEurpBf62fO0EpdXohVPsrqmmZe2sUiHmL/pAqvt0uTV7Llx GOc2rbrZs1x9T198VHr/LtV+BZenPCK0Zq0H X-Received: by 2002:a54:4494:: with SMTP id v20mr3492740oiv.147.1615306854731; Tue, 09 Mar 2021 08:20:54 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:54 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 10/78] target/arm: Implement SVE2 integer add/subtract long Date: Tue, 9 Mar 2021 08:19:33 -0800 Message-Id: <20210309162041.23124-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::233; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x233.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix select offsets (laurent desnogues). --- target/arm/helper-sve.h | 24 ++++++++++++++++++++ target/arm/sve.decode | 19 ++++++++++++++++ target/arm/sve_helper.c | 43 +++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 46 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 132 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 37461c9927..a81297b387 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1367,6 +1367,30 @@ DEF_HELPER_FLAGS_5(sve_ftmad_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_ftmad_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_ftmad_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_ssubl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sabdl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sabdl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sabdl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uaddl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_usubl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uabdl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uabdl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uabdl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(sve_ld1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld2bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld3bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index cd4f73265f..fbfd57b23a 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1159,3 +1159,22 @@ SUQADD 01000100 .. 011 100 100 ... ..... ..... @rdn_pg_rm USQADD 01000100 .. 011 101 100 ... ..... ..... @rdn_pg_rm SQSUB_zpzz 01000100 .. 011 110 100 ... ..... ..... @rdm_pg_rn # SQSUBR UQSUB_zpzz 01000100 .. 011 111 100 ... ..... ..... @rdm_pg_rn # UQSUBR + +#### SVE2 Widening Integer Arithmetic + +## SVE2 integer add/subtract long + +SADDLB 01000101 .. 0 ..... 00 0000 ..... ..... @rd_rn_rm +SADDLT 01000101 .. 0 ..... 00 0001 ..... ..... @rd_rn_rm +UADDLB 01000101 .. 0 ..... 00 0010 ..... ..... @rd_rn_rm +UADDLT 01000101 .. 0 ..... 00 0011 ..... ..... @rd_rn_rm + +SSUBLB 01000101 .. 0 ..... 00 0100 ..... ..... @rd_rn_rm +SSUBLT 01000101 .. 0 ..... 00 0101 ..... ..... @rd_rn_rm +USUBLB 01000101 .. 0 ..... 00 0110 ..... ..... @rd_rn_rm +USUBLT 01000101 .. 0 ..... 00 0111 ..... ..... @rd_rn_rm + +SABDLB 01000101 .. 0 ..... 00 1100 ..... ..... @rd_rn_rm +SABDLT 01000101 .. 0 ..... 00 1101 ..... ..... @rd_rn_rm +UABDLB 01000101 .. 0 ..... 00 1110 ..... ..... @rd_rn_rm +UABDLT 01000101 .. 0 ..... 00 1111 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 009d49e81d..8f22e025e6 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1113,6 +1113,49 @@ DO_ZZW(sve_lsl_zzw_s, uint32_t, uint64_t, H1_4, DO_LSL) #undef DO_ZPZ #undef DO_ZPZ_D +/* + * Three-operand expander, unpredicated, in which the two inputs are + * selected from the top or bottom half of the wide column. + */ +#define DO_ZZZ_TB(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sel1 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + int sel2 = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel1)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel2)); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, mm); \ + } \ +} + +DO_ZZZ_TB(sve2_saddl_h, int16_t, int8_t, H1_2, H1, DO_ADD) +DO_ZZZ_TB(sve2_saddl_s, int32_t, int16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_TB(sve2_saddl_d, int64_t, int32_t, , H1_4, DO_ADD) + +DO_ZZZ_TB(sve2_ssubl_h, int16_t, int8_t, H1_2, H1, DO_SUB) +DO_ZZZ_TB(sve2_ssubl_s, int32_t, int16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_TB(sve2_ssubl_d, int64_t, int32_t, , H1_4, DO_SUB) + +DO_ZZZ_TB(sve2_sabdl_h, int16_t, int8_t, H1_2, H1, DO_ABD) +DO_ZZZ_TB(sve2_sabdl_s, int32_t, int16_t, H1_4, H1_2, DO_ABD) +DO_ZZZ_TB(sve2_sabdl_d, int64_t, int32_t, , H1_4, DO_ABD) + +DO_ZZZ_TB(sve2_uaddl_h, uint16_t, uint8_t, H1_2, H1, DO_ADD) +DO_ZZZ_TB(sve2_uaddl_s, uint32_t, uint16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_TB(sve2_uaddl_d, uint64_t, uint32_t, , H1_4, DO_ADD) + +DO_ZZZ_TB(sve2_usubl_h, uint16_t, uint8_t, H1_2, H1, DO_SUB) +DO_ZZZ_TB(sve2_usubl_s, uint32_t, uint16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_TB(sve2_usubl_d, uint64_t, uint32_t, , H1_4, DO_SUB) + +DO_ZZZ_TB(sve2_uabdl_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) +DO_ZZZ_TB(sve2_uabdl_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) +DO_ZZZ_TB(sve2_uabdl_d, uint64_t, uint32_t, , H1_4, DO_ABD) + +#undef DO_ZZZ_TB + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index fc8e2a84f2..b94561b92b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5971,3 +5971,49 @@ DO_SVE2_ZPZZ(SQSUB_zpzz, sqsub) DO_SVE2_ZPZZ(UQSUB_zpzz, uqsub) DO_SVE2_ZPZZ(SUQADD, suqadd) DO_SVE2_ZPZZ(USQADD, usqadd) + +/* + * SVE2 Widening Integer Arithmetic + */ + +static bool do_sve2_zzw_ool(DisasContext *s, arg_rrr_esz *a, + gen_helper_gvec_3 *fn, int data) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, data, fn); + } + return true; +} + +#define DO_SVE2_ZZZ_TB(NAME, name, SEL1, SEL2) \ +static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_h, \ + gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ + }; \ + return do_sve2_zzw_ool(s, a, fns[a->esz], (SEL2 << 1) | SEL1); \ +} + +DO_SVE2_ZZZ_TB(SADDLB, saddl, false, false) +DO_SVE2_ZZZ_TB(SSUBLB, ssubl, false, false) +DO_SVE2_ZZZ_TB(SABDLB, sabdl, false, false) + +DO_SVE2_ZZZ_TB(UADDLB, uaddl, false, false) +DO_SVE2_ZZZ_TB(USUBLB, usubl, false, false) +DO_SVE2_ZZZ_TB(UABDLB, uabdl, false, false) + +DO_SVE2_ZZZ_TB(SADDLT, saddl, true, true) +DO_SVE2_ZZZ_TB(SSUBLT, ssubl, true, true) +DO_SVE2_ZZZ_TB(SABDLT, sabdl, true, true) + +DO_SVE2_ZZZ_TB(UADDLT, uaddl, true, true) +DO_SVE2_ZZZ_TB(USUBLT, usubl, true, true) +DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true) From patchwork Tue Mar 9 16:19:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396019 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2772069jai; Tue, 9 Mar 2021 09:26:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJxQFAihb3yTt15ZB3uFwx+68fs/LwKoSSe6GFIdGxSVnij7SIDVQP36/yZrzTZZorKlByxP X-Received: by 2002:a05:6102:b05:: with SMTP id b5mr3242065vst.29.1615310788529; Tue, 09 Mar 2021 09:26:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615310788; cv=none; d=google.com; s=arc-20160816; b=Fwyf+c8hU4LsaPQypgPbgkB95XDuikClYTUoOFZPr2A0RVTFgzME6fN4h2+hrMIfuh 9h5cwkjSCHL3X6/umN/AZQwQK+GLdnBH9iB+Hn5w7XbgMFxeEzZXCzLbjCVS5XnVNkJq Qm2s/0UE/rENJLIWxAGbRjNSFhW5kuafZ5FnjN2FFSLIxyQ0vaPm3tNG0r9AGSDuG3ks 89X9NA5KbYk6VXbE2Nv8XyvraSggemKvriSEkxEdmHxgwnC0Tn05HzQv0pmaMkcKjkGm W+M3iAFteiVQwAGdAQeUwpkXFdvE4Fl4N+kJjNs/0qM071VwVd42EU6389IjZFoDvlAW OKRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=MBA7VlgaL7sj30xRXq/QblNTMSN0MmjXoz+f7WQsSFg=; b=QLEVxCgrEJqVJL0K/pTMukayQ+RQMQh3802PeCqZ0wkVNzwzpb+5sQPC80KcHvPOAl 4WrC7v0dPB3KmYVxRXid1M1v4jseivLKrYvQW0v9lPKLle5lKB92JhNKQeiGPlvc9zro 2ZHMFNORRZWxDr/LP2BdPibNkgHCKnJiygOMqnRfUxUJd3qWIpD+6cqx2kF9F1xLDKMa NtDiambK9K0Ob7SGGobJdxMiTyx867QzhhZy55QioDcTl88EL3dbUqns6vxN6h2BuvtV 8ZrTcaOfhLBlU0bSHWDYv0JeAehxp7dlA7oB5qorOYh6nMNtvdwgeXbabsVzPjAOrF4j pN4g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=j7w3w+yK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w201si2579053vkw.40.2021.03.09.09.26.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:26:28 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=j7w3w+yK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40074 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJg7n-0006SD-S7 for patch@linaro.org; Tue, 09 Mar 2021 12:26:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45568) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6f-0007DG-Ag for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:13 -0500 Received: from mail-ot1-x335.google.com ([2607:f8b0:4864:20::335]:46163) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6R-0002S4-TT for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:13 -0500 Received: by mail-ot1-x335.google.com with SMTP id r24so6269567otq.13 for ; Tue, 09 Mar 2021 08:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MBA7VlgaL7sj30xRXq/QblNTMSN0MmjXoz+f7WQsSFg=; b=j7w3w+yKwBoa+bodyFP8uP5Pk8rlksQZUAZGaZcI/erPHCXHpQBQ2hsyjSphBqW1/o 1t5n4LecC96y55LAsoLB7SvIrSmYfakWlqZDAluT+cZPT4V9xJgKnKrwpSSNhWKCezGE cWVEtsg013kc8cLSwawfITEG0frpBh04D+rQ+uQvo7SgHn0P5psIxnY/VuAN0hLyqGx2 pwVpSzHN0vAuUlLMxIZUxVIWBBFSUfK0zsVqs3w5T4Rn+ZbvaJYVNuA0ywiM/Vk+I1U3 u8cWgw4uySBVttH75BA1XrTJ5g3HFBRsZ/T21U3NgTr+6pUmARa+fFh+G25IeHvAvS2x 0IoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MBA7VlgaL7sj30xRXq/QblNTMSN0MmjXoz+f7WQsSFg=; b=rU9edvQbmSvtpbLZ3rdJO3tWLFv6ChJ7Wg15F8apOwQj2bB2zit9aiFqKONzPXCALq 4szhHivy/wRNBgv1xDcpxW1iXzyDueYeXq//wOLo7LOyMCRutXhJW4EexCb/jgnNz7bU VDVDvX7t8QYqcKcX4JUVDns/tgpUGgIyePNCt0+JnahE/do3m7an6vzJnpeoiHleJhOE 5djMdyBmwsjUZk6roraJMUsN6+EcHFzrjb2ISI5w8UY5jrJ6y6L2y4OdVp2Z30nNL53W hie/of0Sj7kOQsP5/2ClVeNRgFcs/pSPNXgVfBs/gxiXVI4xMnXU+7YulExOvMskzAdG s/jw== X-Gm-Message-State: AOAM531SceuC4SzdWD8LqJUownbKz7UPBkzGWypQh0K+M5kpfqNaiOyG EtoS6FRWceJErmwouTqIkJy4XsJ7rCjgwJlC X-Received: by 2002:a9d:823:: with SMTP id 32mr23535685oty.306.1615306855811; Tue, 09 Mar 2021 08:20:55 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:55 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 11/78] target/arm: Implement SVE2 integer add/subtract interleaved long Date: Tue, 9 Mar 2021 08:19:34 -0800 Message-Id: <20210309162041.23124-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::335; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x335.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 6 ++++++ target/arm/translate-sve.c | 4 ++++ 2 files changed, 10 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index fbfd57b23a..12be0584a8 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1178,3 +1178,9 @@ SABDLB 01000101 .. 0 ..... 00 1100 ..... ..... @rd_rn_rm SABDLT 01000101 .. 0 ..... 00 1101 ..... ..... @rd_rn_rm UABDLB 01000101 .. 0 ..... 00 1110 ..... ..... @rd_rn_rm UABDLT 01000101 .. 0 ..... 00 1111 ..... ..... @rd_rn_rm + +## SVE2 integer add/subtract interleaved long + +SADDLBT 01000101 .. 0 ..... 1000 00 ..... ..... @rd_rn_rm +SSUBLBT 01000101 .. 0 ..... 1000 10 ..... ..... @rd_rn_rm +SSUBLTB 01000101 .. 0 ..... 1000 11 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index b94561b92b..105c70dc8c 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6017,3 +6017,7 @@ DO_SVE2_ZZZ_TB(SABDLT, sabdl, true, true) DO_SVE2_ZZZ_TB(UADDLT, uaddl, true, true) DO_SVE2_ZZZ_TB(USUBLT, usubl, true, true) DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true) + +DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true) +DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true) +DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false) From patchwork Tue Mar 9 16:19:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396014 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2762250jai; Tue, 9 Mar 2021 09:14:30 -0800 (PST) X-Google-Smtp-Source: ABdhPJz//NneN6eciG7tIAtE+R242ohYg2qo/Iq+QrD+xh65Dz89/zym0Fz2YCU7+p3g3aebiM5A X-Received: by 2002:a9f:238b:: with SMTP id 11mr7639994uao.45.1615310070017; Tue, 09 Mar 2021 09:14:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615310070; cv=none; d=google.com; s=arc-20160816; b=RS8GoMyKe5SUEX1Zq57Zg+eg+RvLwl2J3sOOjxvZ04vY/QkgFyJAdhr0ajTewkEfna wneiNeKgDozFPRvtGQ6rYICf6Gc9M8jr/2qQ0QakueJsnCAOqNeJq50BDELRznG+Rp9i PELTi+C+c9D/wxv48EIDP1JLrQ+nSEYB3RZoULrHHPJOE7GZO9dJMxbTdBoDYCrtTIo1 NRtuCB11gpTwn3hoRVC4+Em95+Zyxqoz4Xk+UQc+W98voC7qY5nUJspcoAQj6FHrSOQT J7oMF0p9aIrX6DPG2kJd0YjpP6jen23NV/7tkqj8y31x0xHhrWsvsbN3blsAQl1VDihy d2rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=mipsCScn6r1g4sWz6wUddE3d54Z8gTngCu5kJItzM9c=; b=zTqdTtWN8w0igWAdQaF4YAwzttOKxcRDxhE4H8id13G8sO0398hWmWARcKolBp2xQ9 /edVq1St/rqmoPEgy3oDCGiYfXXD6gsAALqe35SNnQbe8tOP6FD6u7UUx3lSnQuZd2Xj RJGUeCOydEw8nbQQDnfZqS/dA45wBP1ciSKZiJhlLXN3nnZj7pqo90ZpoWIOeha3H5eL +LrwaA0QdMbrV72Y0xGnlErFmT5Hl5ADs+ZD8mcJBfVlLxfv/jex4O0yX4M6r/cgnLhZ nv4OV+NmbB+MEGt0mFwcWiDSyTl2ZhNZayiscqBFX2ipq0l7t1GvaG10u1/W5QzTQLHe zYTg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jtAOYkEW; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y4si324148vsg.334.2021.03.09.09.14.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:14:30 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jtAOYkEW; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46666 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJfwD-0004Qk-6u for patch@linaro.org; Tue, 09 Mar 2021 12:14:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45512) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6e-0007A5-2D for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:12 -0500 Received: from mail-oi1-x232.google.com ([2607:f8b0:4864:20::232]:37260) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6P-0002RW-Qw for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:11 -0500 Received: by mail-oi1-x232.google.com with SMTP id u198so10576768oia.4 for ; Tue, 09 Mar 2021 08:20:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mipsCScn6r1g4sWz6wUddE3d54Z8gTngCu5kJItzM9c=; b=jtAOYkEWezcMR+4TsfAsGTYDhc4LRXxHX70yqEFjEFmBo4PXALLUQSKk1cEHQcoBrQ XSTq4zLpN/rAV6dLHHKRMGLWthhw/FfT2+OOPPN+ltoiwgNcorJBJJVQsgxjVXdGiaxt niWHPBMPh2Uy6zMpqGwhxAu/yIWL1hewkJPcb6smJMOvV2T2760SJqsI2ZrM5/nR0WAu gxueUol4juZhm3vYwSoLT0yty5mxzTrGCNfgLUQSZ2fFl865R3+GLbpJXvpOKfsVmrSS K8+60lsLN1JTEu85gNkWo9fGSZnD8jLPthCm24SgcJaCC5Npxuaij1MXG3/GVtl8Df3A CDKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mipsCScn6r1g4sWz6wUddE3d54Z8gTngCu5kJItzM9c=; b=cAFxvTbfuBUL2F5E75GwdMGz9mF0epYL6oVaPU4LWD92mawArKg90nB/RqGaZk6RVD JN+zRbfkg9ZdBZJV/xBKWAZ/eGihm3/3WD1r03rv2iZMEE63b9ok0p954PeCPoLzRfmP i6vc+/E63fzZ1mXP78xySr34w0Iv2k0Qumuikibmf1+5i5fYyeCtunl9HCzC27KO3urw /hElnyjNaUSXp2G3rul0HdfkKcG7vYO2hralaXy0LskWma8bhTJ0hpbVOjaeZ9OAaYyG hUkI52qtp4ejZQUOdQyyJJzjia6pKW8ANh3q9DzyPiYOiez/BrGGypeDp8BEUJprJpGx uIFQ== X-Gm-Message-State: AOAM533wH+3+CUFjAbaOKZZOd638iU+Mj4wo6cTpwswKgWaM+XUDzgEb 3ghpTTq7Z5eYfdTt837DHavK6AFq8Hh2VAZd X-Received: by 2002:aca:b787:: with SMTP id h129mr2391453oif.58.1615306856885; Tue, 09 Mar 2021 08:20:56 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:56 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 12/78] target/arm: Implement SVE2 integer add/subtract wide Date: Tue, 9 Mar 2021 08:19:35 -0800 Message-Id: <20210309162041.23124-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::232; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x232.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix select offsets (laurent desnogues). --- target/arm/helper-sve.h | 16 ++++++++++++++++ target/arm/sve.decode | 12 ++++++++++++ target/arm/sve_helper.c | 30 ++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 20 ++++++++++++++++++++ 4 files changed, 78 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index a81297b387..3286a9c205 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1391,6 +1391,22 @@ DEF_HELPER_FLAGS_4(sve2_uabdl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_uabdl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_uabdl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_ssubw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uaddw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_usubw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(sve_ld1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld2bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld3bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 12be0584a8..f6f21426ef 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1184,3 +1184,15 @@ UABDLT 01000101 .. 0 ..... 00 1111 ..... ..... @rd_rn_rm SADDLBT 01000101 .. 0 ..... 1000 00 ..... ..... @rd_rn_rm SSUBLBT 01000101 .. 0 ..... 1000 10 ..... ..... @rd_rn_rm SSUBLTB 01000101 .. 0 ..... 1000 11 ..... ..... @rd_rn_rm + +## SVE2 integer add/subtract wide + +SADDWB 01000101 .. 0 ..... 010 000 ..... ..... @rd_rn_rm +SADDWT 01000101 .. 0 ..... 010 001 ..... ..... @rd_rn_rm +UADDWB 01000101 .. 0 ..... 010 010 ..... ..... @rd_rn_rm +UADDWT 01000101 .. 0 ..... 010 011 ..... ..... @rd_rn_rm + +SSUBWB 01000101 .. 0 ..... 010 100 ..... ..... @rd_rn_rm +SSUBWT 01000101 .. 0 ..... 010 101 ..... ..... @rd_rn_rm +USUBWB 01000101 .. 0 ..... 010 110 ..... ..... @rd_rn_rm +USUBWT 01000101 .. 0 ..... 010 111 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 8f22e025e6..5b7b8d27b8 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1156,6 +1156,36 @@ DO_ZZZ_TB(sve2_uabdl_d, uint64_t, uint32_t, , H1_4, DO_ABD) #undef DO_ZZZ_TB +#define DO_ZZZ_WTB(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sel2 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel2)); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, mm); \ + } \ +} + +DO_ZZZ_WTB(sve2_saddw_h, int16_t, int8_t, H1_2, H1, DO_ADD) +DO_ZZZ_WTB(sve2_saddw_s, int32_t, int16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_WTB(sve2_saddw_d, int64_t, int32_t, , H1_4, DO_ADD) + +DO_ZZZ_WTB(sve2_ssubw_h, int16_t, int8_t, H1_2, H1, DO_SUB) +DO_ZZZ_WTB(sve2_ssubw_s, int32_t, int16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_WTB(sve2_ssubw_d, int64_t, int32_t, , H1_4, DO_SUB) + +DO_ZZZ_WTB(sve2_uaddw_h, uint16_t, uint8_t, H1_2, H1, DO_ADD) +DO_ZZZ_WTB(sve2_uaddw_s, uint32_t, uint16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_WTB(sve2_uaddw_d, uint64_t, uint32_t, , H1_4, DO_ADD) + +DO_ZZZ_WTB(sve2_usubw_h, uint16_t, uint8_t, H1_2, H1, DO_SUB) +DO_ZZZ_WTB(sve2_usubw_s, uint32_t, uint16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_WTB(sve2_usubw_d, uint64_t, uint32_t, , H1_4, DO_SUB) + +#undef DO_ZZZ_WTB + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 105c70dc8c..e88787318c 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6021,3 +6021,23 @@ DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true) DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true) DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true) DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false) + +#define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \ +static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_h, \ + gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ + }; \ + return do_sve2_zzw_ool(s, a, fns[a->esz], SEL2); \ +} + +DO_SVE2_ZZZ_WTB(SADDWB, saddw, false) +DO_SVE2_ZZZ_WTB(SADDWT, saddw, true) +DO_SVE2_ZZZ_WTB(SSUBWB, ssubw, false) +DO_SVE2_ZZZ_WTB(SSUBWT, ssubw, true) + +DO_SVE2_ZZZ_WTB(UADDWB, uaddw, false) +DO_SVE2_ZZZ_WTB(UADDWT, uaddw, true) +DO_SVE2_ZZZ_WTB(USUBWB, usubw, false) +DO_SVE2_ZZZ_WTB(USUBWT, usubw, true) From patchwork Tue Mar 9 16:19:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396024 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2779153jai; Tue, 9 Mar 2021 09:34:55 -0800 (PST) X-Google-Smtp-Source: ABdhPJzzHuB5q7h0yQLVbGu2I184GoXAoUX0dp9lxAhxFaOg0Hp/Ib42w9e6iNqhuTFml9wdJeIc X-Received: by 2002:a02:77d0:: with SMTP id g199mr28973858jac.118.1615311295278; Tue, 09 Mar 2021 09:34:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311295; cv=none; d=google.com; s=arc-20160816; b=Bi01xUDT3lBhW86+dZv9en9vTJ1SsdgLjxpP3ddxme2XWWshcYeNIa5EjpOpWJDFA3 27a1Ryq/8BKm7g9feKRnLFqHIlPRJiedFvUSs3B62LUg/aHsghro212AjoFNCHs1n/e+ wyNY1gtFr+o0BWp50SkbzsoUZ2wWF5x9DeJaKcDr9quccMbX1gq6vAcYFSJGj7bMbv4o 3qmYPS+QPL9I6ILP7txRukuXdbgj9+Q7NzEUWfV6NIaPm1rx7hlcjLWjTRekli97co23 ezImiF6aHAQF10bVbbjNmE/pWZmHq6L8T2oVE1M9olhN/Yazx7RgAPJsvub+RCbweXvJ XRPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=/w1/PeuZSTxpxUDHY11OvShK/rLcq0Hrz8JHi0g1Cbs=; b=zC+GQuG3Fhi4gptOOaJlytTOVTEbl1CQ5h5t4xgTpcljuTm2Tl5QwVOGHmNYQ6TCy9 Y2y4VCPiYvgedM4Cu/gCuni/6eCR5JNz8ge4k6FK3O8TRo6UzJFkjKUKB2klMS/gW6j+ bVqjlv4g+GkvpF7g72tAIGGUJOEFk5j42qpCXRl+PDiUWOoDs2kPlVE92ISV9f0kkHl5 7uOmMQeX21qOvvUpyXplweU9bmNPrISMfjHZiNYJ5GJP/aPA2zb3qxzGZOvFMUjd+Qn2 QI9kOdohUnNVyJQI39L65aY8RnMohrRxy1q48JZdh9HfuplW3kzBMUQCcdhOlilrnvJO 33WQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=m4j39iVo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l9si13002781ilv.110.2021.03.09.09.34.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:34:55 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=m4j39iVo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57650 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgFy-0005f3-Hl for patch@linaro.org; Tue, 09 Mar 2021 12:34:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45530) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6e-0007BB-Iu for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:12 -0500 Received: from mail-oi1-x229.google.com ([2607:f8b0:4864:20::229]:47063) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6Q-0002Rm-V1 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:12 -0500 Received: by mail-oi1-x229.google.com with SMTP id f3so15492345oiw.13 for ; Tue, 09 Mar 2021 08:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/w1/PeuZSTxpxUDHY11OvShK/rLcq0Hrz8JHi0g1Cbs=; b=m4j39iVowofGQn4C6GTtRO3E174pNhL4/3av0Ogvp29dTweGdQ2k/7KFd044+bFKal /SUda8AhNT3cFVqmsdXi/6E6d/4kH4CZvLuvto/N4ZFiLi7tDcpp8BXRKk5YHbEA6Xw0 7viJFsNwuQFmWboFm2y4ijiTsi5EadvWs7yjjLRQkSXOQnJDDdWDl7kPeBF/eiPUD+mX 9gQ5a5YGn6EbrNP/wxZjNRnXIbwQ+kMPoYKoN2vhOht7zWLhmsOEJkwsaLFcYbxlztHb D+Lv5Uy6sSFeXWb9sXimCuLxldx5dyZ7vQeHyoabC4JC6En1R45FSsVV86/7BCL+1ZzU LPrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/w1/PeuZSTxpxUDHY11OvShK/rLcq0Hrz8JHi0g1Cbs=; b=bWZhSeJ8uFfiqsLa1E4oxvZAsfFcisUOVms6stNe4qiPI/3upjXwjni/Z6r31ZM7UK BbBhkjoQ60i+ah1m3Zsby7BsYIwmqhwyvFZG2DWRVep57HLLpsa6BfUaRndV71ojoWtP TgGgddYj8ewpN93CgNbf7+bUOrKEqU1F7HjIpIvP8HDGSObrVuSBdFjtEghaeO/sA66z SlqH4rx5HM1TNkRThQPTorDKE4CWnW3vH0qX0I9BEnLYS1+rV4esq5hRvOeYO7C/lC5U Ds9HAU0KQIxPlxWpodEARI270AWS7RNzh7Zcs3N6I7htYHajN94yGESzed1yJD/MV2sV 2sUA== X-Gm-Message-State: AOAM530KaZRyW1MaGDqmLLujXPFSTP3cotgVpoFX181DEaBROHG8Rbdl +z87rloNajecqJSGhJZITPiUSD2wHaO9Us8A X-Received: by 2002:aca:b108:: with SMTP id a8mr3496355oif.49.1615306857898; Tue, 09 Mar 2021 08:20:57 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:57 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 13/78] target/arm: Implement SVE2 integer multiply long Date: Tue, 9 Mar 2021 08:19:36 -0800 Message-Id: <20210309162041.23124-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::229; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x229.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Exclude PMULL from this category for the moment. Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 15 +++++++++++++++ target/arm/sve.decode | 9 +++++++++ target/arm/sve_helper.c | 31 +++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 9 +++++++++ 4 files changed, 64 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 3286a9c205..ad8121eec6 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2347,4 +2347,19 @@ DEF_HELPER_FLAGS_6(sve_stdd_le_zd_mte, TCG_CALL_NO_WG, DEF_HELPER_FLAGS_6(sve_stdd_be_zd_mte, TCG_CALL_NO_WG, void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_zzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_zzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_zzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_smull_zzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_smull_zzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_smull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_umull_zzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_umull_zzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_umull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(sve2_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index f6f21426ef..d9a72b7661 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1196,3 +1196,12 @@ SSUBWB 01000101 .. 0 ..... 010 100 ..... ..... @rd_rn_rm SSUBWT 01000101 .. 0 ..... 010 101 ..... ..... @rd_rn_rm USUBWB 01000101 .. 0 ..... 010 110 ..... ..... @rd_rn_rm USUBWT 01000101 .. 0 ..... 010 111 ..... ..... @rd_rn_rm + +## SVE2 integer multiply long + +SQDMULLB_zzz 01000101 .. 0 ..... 011 000 ..... ..... @rd_rn_rm +SQDMULLT_zzz 01000101 .. 0 ..... 011 001 ..... ..... @rd_rn_rm +SMULLB_zzz 01000101 .. 0 ..... 011 100 ..... ..... @rd_rn_rm +SMULLT_zzz 01000101 .. 0 ..... 011 101 ..... ..... @rd_rn_rm +UMULLB_zzz 01000101 .. 0 ..... 011 110 ..... ..... @rd_rn_rm +UMULLT_zzz 01000101 .. 0 ..... 011 111 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 5b7b8d27b8..7877a5bdf5 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1154,6 +1154,37 @@ DO_ZZZ_TB(sve2_uabdl_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) DO_ZZZ_TB(sve2_uabdl_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) DO_ZZZ_TB(sve2_uabdl_d, uint64_t, uint32_t, , H1_4, DO_ABD) +DO_ZZZ_TB(sve2_smull_zzz_h, int16_t, int8_t, H1_2, H1, DO_MUL) +DO_ZZZ_TB(sve2_smull_zzz_s, int32_t, int16_t, H1_4, H1_2, DO_MUL) +DO_ZZZ_TB(sve2_smull_zzz_d, int64_t, int32_t, , H1_4, DO_MUL) + +DO_ZZZ_TB(sve2_umull_zzz_h, uint16_t, uint8_t, H1_2, H1, DO_MUL) +DO_ZZZ_TB(sve2_umull_zzz_s, uint32_t, uint16_t, H1_4, H1_2, DO_MUL) +DO_ZZZ_TB(sve2_umull_zzz_d, uint64_t, uint32_t, , H1_4, DO_MUL) + +/* Note that the multiply cannot overflow, but the doubling can. */ +static inline int16_t do_sqdmull_h(int16_t n, int16_t m) +{ + int16_t val = n * m; + return DO_SQADD_H(val, val); +} + +static inline int32_t do_sqdmull_s(int32_t n, int32_t m) +{ + int32_t val = n * m; + return DO_SQADD_S(val, val); +} + +static inline int64_t do_sqdmull_d(int64_t n, int64_t m) +{ + int64_t val = n * m; + return do_sqadd_d(val, val); +} + +DO_ZZZ_TB(sve2_sqdmull_zzz_h, int16_t, int8_t, H1_2, H1, do_sqdmull_h) +DO_ZZZ_TB(sve2_sqdmull_zzz_s, int32_t, int16_t, H1_4, H1_2, do_sqdmull_s) +DO_ZZZ_TB(sve2_sqdmull_zzz_d, int64_t, int32_t, , H1_4, do_sqdmull_d) + #undef DO_ZZZ_TB #define DO_ZZZ_WTB(NAME, TYPEW, TYPEN, HW, HN, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index e88787318c..8da87a6ce0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6022,6 +6022,15 @@ DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true) DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true) DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false) +DO_SVE2_ZZZ_TB(SQDMULLB_zzz, sqdmull_zzz, false, false) +DO_SVE2_ZZZ_TB(SQDMULLT_zzz, sqdmull_zzz, true, true) + +DO_SVE2_ZZZ_TB(SMULLB_zzz, smull_zzz, false, false) +DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true) + +DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false) +DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true) + #define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ { \ From patchwork Tue Mar 9 16:19:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396015 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2765387jai; Tue, 9 Mar 2021 09:18:21 -0800 (PST) X-Google-Smtp-Source: ABdhPJykUe88+llzc/vnhfhGvzdv3++5Yk8C/8E/xoheVeHfjYAENT8j1nRXdM5AUbfDSjGiVF8Q X-Received: by 2002:a05:6602:722:: with SMTP id g2mr23521268iox.1.1615310301000; Tue, 09 Mar 2021 09:18:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615310300; cv=none; d=google.com; s=arc-20160816; b=0r9WMU/wMXrIyC6DboWhaicQB+pZ0SZAW9/hIHUGtEVruuP6SkQ5JkXGspE8l3mxB+ o0P+J9ITXy9JdZNZwp8aA+QzPxZEly4P7frzDuJIYHyQqVB2r+KVve6ze9NiCHbGL/9l 12fD8CNYOelJq42x3bU1Q89+TE3VpKbmboxt6Rt4jwviou2Mk1nAayFNsbsBAFl0o2Tr LKRsuQG2zY/Xo1sIZWSPsJpme9JuuxMrmiz7lgWtKRUQHqfxfdXh2HKw/xhe5w56ayTE GRUgkq35aw5X0CYnJgzyiPTaPzb2QsnanJqJe8lhH8UjsOP/J6DNdwcSXuavWqGTWgdj cJYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=FdA8tFtpyXbNCIxq+Lmcx9QVlUew+tHqOvL8JhiAxSE=; b=P54Pu5uMIxVSI1E0L2vdXBM/oLcNpZ1oq2+4LmHq3SC4+j6NsKr2CruuXF09sY4IU/ O3OMWK9Ib1NdysxiiVu4LZL2HEwpzcVtucoY/PQizsuPHa80QyYrvFNLS6vrgIKOEIt8 XszKaYJzjDFQHjeuzbrnvwWj3Y4ubCJ5Tc5mpwIygDteZ+thOFiUbrGg9npTQ7ggNQgS /p20GyHEgAqO+eiMA/BUKxPFdts9ScR+ZZ+ZG/Mi1rnA6Wp54qJ/QPOGdbEenu0+ZY8G CzP+1zuEkjs2e4m+PkMT8OKsSzfagEhG7wss66sY7Vd99zT7ELNle3lhpJU/7YU+Sf/j nXrw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KWZ5hzYb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l1si3453676jaj.50.2021.03.09.09.18.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:18:20 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KWZ5hzYb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55322 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJfzw-0000iD-94 for patch@linaro.org; Tue, 09 Mar 2021 12:18:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45578) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6f-0007Dm-Jy for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:13 -0500 Received: from mail-ot1-x331.google.com ([2607:f8b0:4864:20::331]:44752) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6S-0002S5-0m for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:13 -0500 Received: by mail-ot1-x331.google.com with SMTP id f33so13306104otf.11 for ; Tue, 09 Mar 2021 08:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FdA8tFtpyXbNCIxq+Lmcx9QVlUew+tHqOvL8JhiAxSE=; b=KWZ5hzYb/Cy/mtS/FrvezYm9PeHrluo2e5vf1QQ1uDT3eUN/kMPJzgUnq03y4plxrm 4jKCJvQKSTwKcp2aZJt1H6NN0hWqWUIq9ISlySWOF88W43UDzAQnuVzkjfwihWNOfyYT g0BNurTNHETVJ3BS+t8G8KwdZVndkIoZquTYIiyVVSGEcFjWNTOArV2LRLDcuje9yJLu UKz6ofpHG8PDpUu9rrKz+iUjwVc0Kq2fcOwdM8QUCyq1FlNShLLTQa19RPCeODjLHsuo 9GrP9dSPQJMJmXbjdfllvSe8X376MOC8vQBszPbxgq8LIRW1awW4jDJ8f6iBr0yUp9a1 0qJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FdA8tFtpyXbNCIxq+Lmcx9QVlUew+tHqOvL8JhiAxSE=; b=IZ7m5c0YrkthJuJnTcS43KpXr38FfkRU0iZd3NiYqMSFpSeqGpJPbr9cTRzU14FkBx nEEAYbYoEADHBCipBmKZwOgQb68o/go1rnB5pOEhqShPVQ/BOZdQG/R5RlmKPgbm4BCN 6qNAherpEe4wkHm+wjW4BY2mOIW7DxA80u79mBWfVW0TmuvZtc0a8Z6Vvyjq+8ZnojyB WU2P9kIQ0TmIljYkPvlbH92a0sGGaFNYl6SiQ8W5B6LoM2OA3WE86UuaJGkdmTAgxCVg Sz+VbxFMEdLhpjh3nFIP5m7SpjOmQKZ6O1p6oo1r3WEWz08Lp72tsZyd1StgXntAfGVh Ankw== X-Gm-Message-State: AOAM5300G+9LSKkSjQYxNX4mbrWUsTTRwCnKo/jy5IIcBKiFBJgPujN6 U5VSCf1d/ytb5gLnlD8SSMsrhnJOyFz/J+RO X-Received: by 2002:a05:6830:158c:: with SMTP id i12mr23783504otr.7.1615306858887; Tue, 09 Mar 2021 08:20:58 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:58 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 14/78] target/arm: Implement PMULLB and PMULLT Date: Tue, 9 Mar 2021 08:19:37 -0800 Message-Id: <20210309162041.23124-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::331; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x331.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 10 ++++++++++ target/arm/helper-sve.h | 1 + target/arm/sve.decode | 2 ++ target/arm/translate-sve.c | 22 ++++++++++++++++++++++ target/arm/vec_helper.c | 24 ++++++++++++++++++++++++ 5 files changed, 59 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index ee92ea876e..824365209b 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4198,6 +4198,16 @@ static inline bool isar_feature_aa64_sve2(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SVEVER) != 0; } +static inline bool isar_feature_aa64_sve2_aes(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, AES) != 0; +} + +static inline bool isar_feature_aa64_sve2_pmull128(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, AES) >= 2; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index ad8121eec6..bf3e533eb4 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2363,3 +2363,4 @@ DEF_HELPER_FLAGS_4(sve2_umull_zzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_umull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_pmull_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index d9a72b7661..016c15ebb6 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1201,6 +1201,8 @@ USUBWT 01000101 .. 0 ..... 010 111 ..... ..... @rd_rn_rm SQDMULLB_zzz 01000101 .. 0 ..... 011 000 ..... ..... @rd_rn_rm SQDMULLT_zzz 01000101 .. 0 ..... 011 001 ..... ..... @rd_rn_rm +PMULLB 01000101 .. 0 ..... 011 010 ..... ..... @rd_rn_rm +PMULLT 01000101 .. 0 ..... 011 011 ..... ..... @rd_rn_rm SMULLB_zzz 01000101 .. 0 ..... 011 100 ..... ..... @rd_rn_rm SMULLT_zzz 01000101 .. 0 ..... 011 101 ..... ..... @rd_rn_rm UMULLB_zzz 01000101 .. 0 ..... 011 110 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 8da87a6ce0..260337d111 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6031,6 +6031,28 @@ DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true) DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false) DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true) +static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_gvec_pmull_q, gen_helper_sve2_pmull_h, + NULL, gen_helper_sve2_pmull_d, + }; + if (a->esz == 0 && !dc_isar_feature(aa64_sve2_pmull128, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], sel); +} + +static bool trans_PMULLB(DisasContext *s, arg_rrr_esz *a) +{ + return do_trans_pmull(s, a, false); +} + +static bool trans_PMULLT(DisasContext *s, arg_rrr_esz *a) +{ + return do_trans_pmull(s, a, true); +} + #define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ { \ diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 40b92100bf..b0ce597060 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -1750,6 +1750,30 @@ void HELPER(sve2_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) d[i] = pmull_h(nn, mm); } } + +static uint64_t pmull_d(uint64_t op1, uint64_t op2) +{ + uint64_t result = 0; + int i; + + for (i = 0; i < 32; ++i) { + uint64_t mask = -((op1 >> i) & 1); + result ^= (op2 << i) & mask; + } + return result; +} + +void HELPER(sve2_pmull_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t sel = H4(simd_data(desc)); + intptr_t i, opr_sz = simd_oprsz(desc); + uint32_t *n = vn, *m = vm; + uint64_t *d = vd; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = pmull_d(n[2 * i + sel], m[2 * i + sel]); + } +} #endif #define DO_CMP0(NAME, TYPE, OP) \ From patchwork Tue Mar 9 16:19:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396033 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2786951jai; Tue, 9 Mar 2021 09:45:19 -0800 (PST) X-Google-Smtp-Source: ABdhPJwavlldKjcHMXrk3lGLPVjDmJCu4et7n432AbugOXh3IHXDcg0HkKHJvgivuO073OiFnitq X-Received: by 2002:a5d:8490:: with SMTP id t16mr23277040iom.91.1615311919864; Tue, 09 Mar 2021 09:45:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311919; cv=none; d=google.com; s=arc-20160816; b=0GfMZy2+wRuCTe5pvWYXB0VLNMAETuqqo9oe1+m4bkoB3eAP8hv2ONVnYPlNJHGLVQ mCEjTxxnGXS3m3xzuwc3RonTlUDHOPilEPsZzwq8/4pMWbq9+2bAsNbCaR77iN4On8Oi J6Hx0RWu3ES2F2FpuGecmqfaNLh0gYekOOZdxxMJWxi2fZz6Btim5LsK8nXoIobS8kmo oX7QttNrcXv0XJdixVnp1PwzXjKspC7WeyISD3r2YYdDRgaiBcBxBSYueQEg2BzMcLZE VT4dugpjV1Bz01quJqa9YBf2YuVMal2xXZCj79RNyZ9Nz1orRAL8IGyLX5cJ9M1OVzkE 3cWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=tgusoMtmDQWI+mFp+GeBtWOtPA1PyCYaH+NXyjthEkA=; b=kz7EuAQEf4P1evdqWIym1Q6HNYvTEwksRqA7+cUU62r4zWAOkwoGn0W6UUEZvlFpOI GmlDQyVmXzpK0FLggP8deAHQzXWki9lzzS3V/4LaSW4OozhV+v4GylZRMvn4pJYXvyJV Nj2OfYG64/N+lQn09wJgeEnYK83gL/qFAgiX7seEZY7t+Rvpkx9SiLmnolXmT7IRWo1u 4rOUK8WrA9DglB+Ibeky8RCyu5g03wj8EYki8OAMoLA7P23scNBmiHpchD/9RDLFEXQE ZBch74VbbxXX0vYhr7O4MOz+Qe+09fVJyjWyoAXioN0F81xKheWrrEnCRcf7NYgxXNsL 0frw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pqzv2niG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b9si13225794ior.88.2021.03.09.09.45.19 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:45:19 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pqzv2niG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:49134 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgQ3-00067e-63 for patch@linaro.org; Tue, 09 Mar 2021 12:45:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45704) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6j-0007Lx-5c for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:17 -0500 Received: from mail-oi1-x236.google.com ([2607:f8b0:4864:20::236]:35875) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6X-0002SU-Dc for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:16 -0500 Received: by mail-oi1-x236.google.com with SMTP id o22so5780587oic.3 for ; Tue, 09 Mar 2021 08:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tgusoMtmDQWI+mFp+GeBtWOtPA1PyCYaH+NXyjthEkA=; b=pqzv2niGVMP0OGtuk0ONtdcvfGynuzAROnLj1lNdN7kDrKSvVaSe1ZImDJFp4T3+6K NHSHg2lRnLjxB6s3ZUkif8qvI6P+89Y+dl3w80O2h7Gv8674gXDBzgz52uWcvUHgELv4 9b8GgjsDMw7chlm+zC3CHFuRBWyAZnQ1YVAW83x2lg9R49m8btowfp8+AHfxa9wa8LMu aug5ba/ISn9P7nAZwjqD/wTrPpOGos+rVvjd350b56roBILCUayzarWgAXFKXYgpCbVd TNtfA0N4fdgB+wOb+IWo1lKG5tdnGC+JehBBgrNnumxQ4vZo7Zh66cmHbo7jMPQz+2dk 5vSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tgusoMtmDQWI+mFp+GeBtWOtPA1PyCYaH+NXyjthEkA=; b=mCmV2wxlcpkB1W7cpnew9MKdfncY62ou1M5axUArXzcv4X5KGgdqJlpBvyC3dKAHFK gfFYEnLfxNhJlndRxOZpcSUMpZ4mecGMcNxeRv0RB0b0SGuYZlOcm2dsKK6Gn4Vv8GuW hax77xsBknNtWs3s2ZdZKiai39un5j8G5B/kbCd6buGzt/aJ3Jb/wx8tQQLw1B/2pun5 cdBCvozPVUPW9g/7tqYeGqlmI/WuQeJw/J5WLjTYrtp6QYLmG62e7BR0yoX/JMm/2HiJ vAnBqib3oY97hUU+VaFMpuuqei8m4a1tETwXEU0mMbxyddvkKA/VJXu/DLnjG+fuvpns x1hg== X-Gm-Message-State: AOAM532JYqLlfi+Qi3rMDz+cXxwQrO7KgOQ6GMlVil2lUiUe8VVa5KQO JSJ0y1DN7rL/i7MmfSKvKjwVzQLDiB+rfP/p X-Received: by 2002:a05:6808:1290:: with SMTP id a16mr3483280oiw.161.1615306859972; Tue, 09 Mar 2021 08:20:59 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.20.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:20:59 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 15/78] target/arm: Implement SVE2 bitwise shift left long Date: Tue, 9 Mar 2021 08:19:38 -0800 Message-Id: <20210309162041.23124-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::236; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x236.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 8 ++ target/arm/sve.decode | 8 ++ target/arm/sve_helper.c | 26 ++++++ target/arm/translate-sve.c | 159 +++++++++++++++++++++++++++++++++++++ 4 files changed, 201 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index bf3e533eb4..740939e7a8 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2364,3 +2364,11 @@ DEF_HELPER_FLAGS_4(sve2_umull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_pmull_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sshll_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sshll_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sshll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_ushll_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_ushll_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_ushll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 016c15ebb6..a3191eba7b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1207,3 +1207,11 @@ SMULLB_zzz 01000101 .. 0 ..... 011 100 ..... ..... @rd_rn_rm SMULLT_zzz 01000101 .. 0 ..... 011 101 ..... ..... @rd_rn_rm UMULLB_zzz 01000101 .. 0 ..... 011 110 ..... ..... @rd_rn_rm UMULLT_zzz 01000101 .. 0 ..... 011 111 ..... ..... @rd_rn_rm + +## SVE2 bitwise shift left long + +# Note bit23 == 0 is handled by esz > 0 in do_sve2_shll_tb. +SSHLLB 01000101 .. 0 ..... 1010 00 ..... ..... @rd_rn_tszimm_shl +SSHLLT 01000101 .. 0 ..... 1010 01 ..... ..... @rd_rn_tszimm_shl +USHLLB 01000101 .. 0 ..... 1010 10 ..... ..... @rd_rn_tszimm_shl +USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 7877a5bdf5..3e80b612ea 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -625,6 +625,8 @@ DO_ZPZZ(sve2_sqrshl_zpzz_h, int16_t, H1_2, do_sqrshl_h) DO_ZPZZ(sve2_sqrshl_zpzz_s, int32_t, H1_4, do_sqrshl_s) DO_ZPZZ_D(sve2_sqrshl_zpzz_d, int64_t, do_sqrshl_d) +#undef do_sqrshl_d + #define do_uqrshl_b(n, m) \ ({ uint32_t discard; do_uqrshl_bhs(n, (int8_t)m, 8, true, &discard); }) #define do_uqrshl_h(n, m) \ @@ -639,6 +641,8 @@ DO_ZPZZ(sve2_uqrshl_zpzz_h, uint16_t, H1_2, do_uqrshl_h) DO_ZPZZ(sve2_uqrshl_zpzz_s, uint32_t, H1_4, do_uqrshl_s) DO_ZPZZ_D(sve2_uqrshl_zpzz_d, uint64_t, do_uqrshl_d) +#undef do_uqrshl_d + #define DO_HADD_BHS(n, m) (((int64_t)n + m) >> 1) #define DO_HADD_D(n, m) ((n >> 1) + (m >> 1) + (n & m & 1)) @@ -1217,6 +1221,28 @@ DO_ZZZ_WTB(sve2_usubw_d, uint64_t, uint32_t, , H1_4, DO_SUB) #undef DO_ZZZ_WTB +#define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + intptr_t sel = (simd_data(desc) & 1) * sizeof(TYPEN); \ + int shift = simd_data(desc) >> 1; \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel)); \ + *(TYPEW *)(vd + HW(i)) = nn << shift; \ + } \ +} + +DO_ZZI_SHLL(sve2_sshll_h, int16_t, int8_t, H1_2, H1) +DO_ZZI_SHLL(sve2_sshll_s, int32_t, int16_t, H1_4, H1_2) +DO_ZZI_SHLL(sve2_sshll_d, int64_t, int32_t, , H1_4) + +DO_ZZI_SHLL(sve2_ushll_h, uint16_t, uint8_t, H1_2, H1) +DO_ZZI_SHLL(sve2_ushll_s, uint32_t, uint16_t, H1_4, H1_2) +DO_ZZI_SHLL(sve2_ushll_d, uint64_t, uint32_t, , H1_4) + +#undef DO_ZZI_SHLL + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 260337d111..0f8686d47e 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6072,3 +6072,162 @@ DO_SVE2_ZZZ_WTB(UADDWB, uaddw, false) DO_SVE2_ZZZ_WTB(UADDWT, uaddw, true) DO_SVE2_ZZZ_WTB(USUBWB, usubw, false) DO_SVE2_ZZZ_WTB(USUBWT, usubw, true) + +static void gen_sshll_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t imm) +{ + int top = imm & 1; + int shl = imm >> 1; + int halfbits = 4 << vece; + + if (top) { + if (shl == halfbits) { + TCGv_vec t = tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(halfbits, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); + } else { + tcg_gen_sari_vec(vece, d, n, halfbits); + tcg_gen_shli_vec(vece, d, d, shl); + } + } else { + tcg_gen_shli_vec(vece, d, n, halfbits); + tcg_gen_sari_vec(vece, d, d, halfbits - shl); + } +} + +static void gen_ushll_i64(unsigned vece, TCGv_i64 d, TCGv_i64 n, int imm) +{ + int halfbits = 4 << vece; + int top = imm & 1; + int shl = (imm >> 1); + int shift; + uint64_t mask; + + mask = MAKE_64BIT_MASK(0, halfbits); + mask <<= shl; + mask = dup_const(vece, mask); + + shift = shl - top * halfbits; + if (shift < 0) { + tcg_gen_shri_i64(d, n, -shift); + } else { + tcg_gen_shri_i64(d, n, shift); + } + tcg_gen_andi_i64(d, d, mask); +} + +static void gen_ushll16_i64(TCGv_i64 d, TCGv_i64 n, int64_t imm) +{ + gen_ushll_i64(MO_16, d, n, imm); +} + +static void gen_ushll32_i64(TCGv_i64 d, TCGv_i64 n, int64_t imm) +{ + gen_ushll_i64(MO_32, d, n, imm); +} + +static void gen_ushll64_i64(TCGv_i64 d, TCGv_i64 n, int64_t imm) +{ + gen_ushll_i64(MO_64, d, n, imm); +} + +static void gen_ushll_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t imm) +{ + int halfbits = 4 << vece; + int top = imm & 1; + int shl = imm >> 1; + + if (top) { + if (shl == halfbits) { + TCGv_vec t = tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(halfbits, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); + } else { + tcg_gen_shri_vec(vece, d, n, halfbits); + tcg_gen_shli_vec(vece, d, d, shl); + } + } else { + if (shl == 0) { + TCGv_vec t = tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); + } else { + tcg_gen_shli_vec(vece, d, n, halfbits); + tcg_gen_shri_vec(vece, d, d, halfbits - shl); + } + } +} + +static bool do_sve2_shll_tb(DisasContext *s, arg_rri_esz *a, + bool sel, bool uns) +{ + static const TCGOpcode sshll_list[] = { + INDEX_op_shli_vec, INDEX_op_sari_vec, 0 + }; + static const TCGOpcode ushll_list[] = { + INDEX_op_shli_vec, INDEX_op_shri_vec, 0 + }; + static const GVecGen2i ops[2][3] = { + { { .fniv = gen_sshll_vec, + .opt_opc = sshll_list, + .fno = gen_helper_sve2_sshll_h, + .vece = MO_16 }, + { .fniv = gen_sshll_vec, + .opt_opc = sshll_list, + .fno = gen_helper_sve2_sshll_s, + .vece = MO_32 }, + { .fniv = gen_sshll_vec, + .opt_opc = sshll_list, + .fno = gen_helper_sve2_sshll_d, + .vece = MO_64 } }, + { { .fni8 = gen_ushll16_i64, + .fniv = gen_ushll_vec, + .opt_opc = ushll_list, + .fno = gen_helper_sve2_ushll_h, + .vece = MO_16 }, + { .fni8 = gen_ushll32_i64, + .fniv = gen_ushll_vec, + .opt_opc = ushll_list, + .fno = gen_helper_sve2_ushll_s, + .vece = MO_32 }, + { .fni8 = gen_ushll64_i64, + .fniv = gen_ushll_vec, + .opt_opc = ushll_list, + .fno = gen_helper_sve2_ushll_d, + .vece = MO_64 } }, + }; + + if (a->esz < 0 || a->esz > 2 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2i(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vsz, vsz, (a->imm << 1) | sel, + &ops[uns][a->esz]); + } + return true; +} + +static bool trans_SSHLLB(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, false, false); +} + +static bool trans_SSHLLT(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, true, false); +} + +static bool trans_USHLLB(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, false, true); +} + +static bool trans_USHLLT(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, true, true); +} From patchwork Tue Mar 9 16:19:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396031 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2784222jai; Tue, 9 Mar 2021 09:41:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJwuFZvw2/p52zWj7/8YY8AxH108YKzGSaIcglugbCKYn2YXw1twnq9vApLbwe67EBFVxEq5 X-Received: by 2002:a02:c894:: with SMTP id m20mr30272290jao.80.1615311688869; Tue, 09 Mar 2021 09:41:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311688; cv=none; d=google.com; s=arc-20160816; b=py0pxE2WRQ2jXfhr6P+0SykYjGBres7pIiv5Pbhc3sfOJxUxD8AabsAnd42XSc4FNv KcFfBVT4xyQUrNj7TIgmQl5YXyBK0c3M0OxwrEt/jTUjt3d89heQc1Z5duXBymxvZ+KT SXTVzVyrrnjCQMy41iLKKtOy6D6OPPg35ZgsEZXR7UsOFIKJySkcHjzncvxHJwnhbcho MlI4TMFUsEP/r15PTZ5bhvTj81KjwvlrOhoFrYNGb93RLAivQlMAkXqaeXUqhIXh1AlW HRE1FYIqu3Gwur7FSLvYpi5GMqMQOCZYwRSBFLKPv2VAfH0mdn3qstHKBM1w9XtZYgWh bDlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=1ZQl5tAHE23Q9wb6v8En0geqPTjfeiUtEihoaGG8gow=; b=J2fQNfwtXwpGIj9ix0P/UR3VVvOMuwXnKNd5tdTa9S87l+XkqNFRkepxpKZ8SOINGa VDvA8jxOwXExRqUoF/HP3CK9BbTivO8AmBujaaoriY5bscc4R3elagrA64vVdo0Fz6ub q2pZQXrdjiW+T4wscKIenaTKqOI51pfc7tNJnBPILBH2SxGE09E+GLYuOXYi1CpJhF/C gxU4PJzy9IpvOvEZO5KCL4HyuaVpOVidjCwP1VQmXPB9Tz8tPSkQ8dvgwHtUcwfMOTBR OWTQEKkD983yuj9JA3uCS19UsZsPZdsVL2fLSUAt+mf5dErCHuprd5P0TLU2H7hrj0Ct um3A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=IkrZQtGS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j12si13182305ila.38.2021.03.09.09.41.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:41:28 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=IkrZQtGS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40488 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgMK-0002HW-7w for patch@linaro.org; Tue, 09 Mar 2021 12:41:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45650) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6h-0007JH-Ne for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:15 -0500 Received: from mail-oi1-x22a.google.com ([2607:f8b0:4864:20::22a]:44661) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6X-0002Sf-DF for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:15 -0500 Received: by mail-oi1-x22a.google.com with SMTP id w195so8891023oif.11 for ; Tue, 09 Mar 2021 08:21:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1ZQl5tAHE23Q9wb6v8En0geqPTjfeiUtEihoaGG8gow=; b=IkrZQtGSZl0n9+EqICXW9HCsxrQ7gUaoQVgPHWvhphUYvWbXf6vyA+z3v1m+TnmfJT 06+v1Aphui73fWutYl4yrRyddhs+06mtKYtU62ZAcCaqL1+29lGmgub+Mux6f3PKEEGQ Enb0cmOmO6Fuc54YFdKiS8597c/yI/50fSpPt7ocqz/epOPnT5WM7S0uMHM9nIQEqnMd xxHg389+aO9MV3Gt74BIfZBIHHU8VbMStz8HgY8gwSObubkSdVmRcPEonmskYlMl5HED NuHJ4dGFhBGcs+Qq5mKJj1PartIR2/I1KJLCvQsj3AGiGBy1JgJrA+pS3r5cPuaw9pia oiSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1ZQl5tAHE23Q9wb6v8En0geqPTjfeiUtEihoaGG8gow=; b=W+7fXJw/SoUKINZJlm2zR0Aj8PQZFdMBfIYAFD387Xboem2dbItYg1PGL1xCzjSa3V g+cFSJZA5wGkmMkDRwts9sTspBSUF38xFt1vPy6ad1DU89xS/InEhRQCpwX1SOPjkSrQ OUiEEL8MC+/Usw5ESw8xyA1q0861f7e1DKnr1dg9qLocduORMnX/0pj9Q9esVdBhRlgT Y6GZ3gSbTlV4wicRy1VpL6sR7VxzJArCx4UDus1XCYP40Fbm6NADjdHDvuuMY9+WcFbl WalqRGcyp+Giz60xsUchSSbKRN/D5iyJwmYpydykJTcpMe/EHWH+FfxvU3JHHHk57ygX Zx0A== X-Gm-Message-State: AOAM533YRCXWHLIxtB8jyDj3JPl1jcdea5s2LeMDjigLdPSsnf2Xu/PR SrPvIMocQR/gXi936pQd66+zXnN0YOJ+oNrz X-Received: by 2002:a54:488b:: with SMTP id r11mr3635864oic.166.1615306861043; Tue, 09 Mar 2021 08:21:01 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:00 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 16/78] target/arm: Implement SVE2 bitwise exclusive-or interleaved Date: Tue, 9 Mar 2021 08:19:39 -0800 Message-Id: <20210309162041.23124-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22a; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 20 ++++++++++++++++++++ target/arm/translate-sve.c | 19 +++++++++++++++++++ 4 files changed, 49 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 740939e7a8..f65818da05 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2372,3 +2372,8 @@ DEF_HELPER_FLAGS_3(sve2_sshll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_ushll_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_ushll_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_ushll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_eoril_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_eoril_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_eoril_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_eoril_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index a3191eba7b..0922a44829 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1215,3 +1215,8 @@ SSHLLB 01000101 .. 0 ..... 1010 00 ..... ..... @rd_rn_tszimm_shl SSHLLT 01000101 .. 0 ..... 1010 01 ..... ..... @rd_rn_tszimm_shl USHLLB 01000101 .. 0 ..... 1010 10 ..... ..... @rd_rn_tszimm_shl USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl + +## SVE2 bitwise exclusive-or interleaved + +EORBT 01000101 .. 0 ..... 10010 0 ..... ..... @rd_rn_rm +EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 3e80b612ea..9f5d143348 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1221,6 +1221,26 @@ DO_ZZZ_WTB(sve2_usubw_d, uint64_t, uint32_t, , H1_4, DO_SUB) #undef DO_ZZZ_WTB +#define DO_ZZZ_NTB(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + intptr_t sel1 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPE); \ + intptr_t sel2 = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(TYPE); \ + for (i = 0; i < opr_sz; i += 2 * sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + H(i + sel1)); \ + TYPE mm = *(TYPE *)(vm + H(i + sel2)); \ + *(TYPE *)(vd + H(i + sel1)) = OP(nn, mm); \ + } \ +} + +DO_ZZZ_NTB(sve2_eoril_b, uint8_t, H1, DO_EOR) +DO_ZZZ_NTB(sve2_eoril_h, uint16_t, H1_2, DO_EOR) +DO_ZZZ_NTB(sve2_eoril_s, uint32_t, H1_4, DO_EOR) +DO_ZZZ_NTB(sve2_eoril_d, uint64_t, , DO_EOR) + +#undef DO_ZZZ_NTB + #define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0f8686d47e..ea03bf6609 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6031,6 +6031,25 @@ DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true) DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false) DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true) +static bool do_eor_tb(DisasContext *s, arg_rrr_esz *a, bool sel1) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_eoril_b, gen_helper_sve2_eoril_h, + gen_helper_sve2_eoril_s, gen_helper_sve2_eoril_d, + }; + return do_sve2_zzw_ool(s, a, fns[a->esz], (!sel1 << 1) | sel1); +} + +static bool trans_EORBT(DisasContext *s, arg_rrr_esz *a) +{ + return do_eor_tb(s, a, false); +} + +static bool trans_EORTB(DisasContext *s, arg_rrr_esz *a) +{ + return do_eor_tb(s, a, true); +} + static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel) { static gen_helper_gvec_3 * const fns[4] = { From patchwork Tue Mar 9 16:19:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396028 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2782509jai; Tue, 9 Mar 2021 09:39:13 -0800 (PST) X-Google-Smtp-Source: ABdhPJyluU8LyjEe9sZ68RbzDTyDRZXKNIGe+dZTvt+3huJCyvidTYDel8evY/+uGP/lZcKcufxq X-Received: by 2002:a05:6602:2048:: with SMTP id z8mr23983868iod.143.1615311553304; Tue, 09 Mar 2021 09:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311553; cv=none; d=google.com; s=arc-20160816; b=yAki4g2vY5Da0F/30vfbhjmpiqg0m8goHAvhwHDttfHKoRGOgxDrXZSRGj0534A1eH gnYyNygb9L8NRekEXXDcaaBgIZHQY9sLrq5+xUS/wkf9dH7wpBY/3i43rci9MSfxt2XZ 8LMvALYpsXfkndNH73Hd+Z5OQqPSvRJZTDkBbC4deO8+gN02G8Hn/IueQEn1azjYTPxB hhA3mWHKQtK2kIJ8pPlBUdS81Z8SsjwGEAxKrJAwPalwHXU1sn6W1jawdLuQgsscDZSm 4FNPVFSkmFdfulHNcMxY52N0BBfWWZ9kiEjsERnnFzzRhXfQjt6+8C92q4Ick/RdqyX1 345w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=JFABDHCDp3HL/YAORggSqEppCmdhvx5EoY62umVHCko=; b=rfUxBcce+lL7otmKTIt4UhSl5YzwVHgCD03iOHsC+9++zqElN6aeMa4NCRbP9Q6xj6 QkZlHOY+RFF74GSdy24eMDOVX0o6+NNFvHT/o/yC6Qq5x/0gA9Z+7L95oiCvHDTDto4i 9E2k+WnPWx3jQxtfXxRhi6c8Ryr954IgqZA9cmHMxNStFtgR4E3x4uWL/MCT9psa53Z1 c6MrpxWGl3ZhpnIxA2BRkS8v6Q0+H+jt1o3N4a6AFp/sZC8INdlSMouaU0IHXgG5EI73 /zLL051tsDWjk0inPo0cOeyK+Mpu4QSuDby3dbzlBa9o2b+/39tujgCXptX6cVLL9CK8 8QOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hsTdpc9z; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k11si14048750ilo.42.2021.03.09.09.39.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hsTdpc9z; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38116 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgK8-0001EQ-MK for patch@linaro.org; Tue, 09 Mar 2021 12:39:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45656) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6i-0007Jv-1l for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:16 -0500 Received: from mail-oi1-x236.google.com ([2607:f8b0:4864:20::236]:35203) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6X-0002Sp-Db for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:15 -0500 Received: by mail-oi1-x236.google.com with SMTP id u6so8188870oic.2 for ; Tue, 09 Mar 2021 08:21:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JFABDHCDp3HL/YAORggSqEppCmdhvx5EoY62umVHCko=; b=hsTdpc9zKjY5nOzrRPvd4u/3wtkuaVepm9JCeWX8DeuSQ25dbel+OM4L0xr9mr4wkT Ekuo6t5NG3ntvMuKBLw3fVUg/QDuCnzqEVEBjn0jv+vlj9lI4b+9K2kYNYh3IceX6QBA M4ChE+4NrjRiF0ywtsDeR+wfWUzZEW9b5RJPYfLE9B5dsSJ0vMNrom6PrtZQsNwRZv/8 Q4Gk7VgenGfjfQYmOJWx2DNDI5GrXt6xBMr9QHtzc+RG+mQxNdXSoGCTBYllSkw9W52q EB78NbpJgkHgys+9zA+4zinXOX5wM+zNObiAkNSpEbsvYlVgWiojiHQKfWWPx9IwFEk0 z3Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JFABDHCDp3HL/YAORggSqEppCmdhvx5EoY62umVHCko=; b=OUMIlGX+aCyXMIWMdmL76gdgWV/RgtUmhCmg4YnW4tRd2CbJvrCjLEL+SBZQhhJpl1 w7v+lwZ6dDmQr4gnrmz1UENWBrESEHIGHHzsri0VW5Kgv9NEBliZEo5h6Amuhh251bpW z0c9v29bxrGi2ZOVpZCCiI527e7c9sidtoajAYXH4CC7QqRROVm40bBzUdNwt3RoJidP IVbCZnELUOpku9fPFliKkvLj+olapziYMs1Oi47uz0EOiuETAptwZ9AQZUZrrT4P9XpI 5h/OSAfq0aSvzoWJ/TQi7aNO5S8NIF2sdzRE0ZC0V7eNaVX5GmUCSIbtDGMAsoffoZn1 vo2A== X-Gm-Message-State: AOAM533vsGZR5dmTPDoMroG6z3t1rMGKYTsbOmUW/xL9enuBkwuRQdQi lT2lcLlrbqXu2w4iPcpndswyuGfmxOcUtham X-Received: by 2002:aca:fc11:: with SMTP id a17mr1346633oii.68.1615306862128; Tue, 09 Mar 2021 08:21:02 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:01 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 17/78] target/arm: Implement SVE2 bitwise permute Date: Tue, 9 Mar 2021 08:19:40 -0800 Message-Id: <20210309162041.23124-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::236; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x236.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++ target/arm/helper-sve.h | 15 ++++++++ target/arm/sve.decode | 6 ++++ target/arm/sve_helper.c | 73 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 36 +++++++++++++++++++ 5 files changed, 135 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 824365209b..57d03addb4 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4208,6 +4208,11 @@ static inline bool isar_feature_aa64_sve2_pmull128(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, AES) >= 2; } +static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index f65818da05..4861481fe0 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2377,3 +2377,18 @@ DEF_HELPER_FLAGS_4(sve2_eoril_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_eoril_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_eoril_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_eoril_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_bext_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bext_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bext_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bext_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_bdep_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bdep_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bdep_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bdep_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_bgrp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bgrp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bgrp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bgrp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 0922a44829..7cb89a0d47 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1220,3 +1220,9 @@ USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl EORBT 01000101 .. 0 ..... 10010 0 ..... ..... @rd_rn_rm EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm + +## SVE2 bitwise permute + +BEXT 01000101 .. 0 ..... 1011 00 ..... ..... @rd_rn_rm +BDEP 01000101 .. 0 ..... 1011 01 ..... ..... @rd_rn_rm +BGRP 01000101 .. 0 ..... 1011 10 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 9f5d143348..ef9d40591a 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1241,6 +1241,79 @@ DO_ZZZ_NTB(sve2_eoril_d, uint64_t, , DO_EOR) #undef DO_ZZZ_NTB +#define DO_BITPERM(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + i); \ + TYPE mm = *(TYPE *)(vm + i); \ + *(TYPE *)(vd + i) = OP(nn, mm, sizeof(TYPE) * 8); \ + } \ +} + +static uint64_t bitextract(uint64_t data, uint64_t mask, int n) +{ + uint64_t res = 0; + int db, rb = 0; + + for (db = 0; db < n; ++db) { + if ((mask >> db) & 1) { + res |= ((data >> db) & 1) << rb; + ++rb; + } + } + return res; +} + +DO_BITPERM(sve2_bext_b, uint8_t, bitextract) +DO_BITPERM(sve2_bext_h, uint16_t, bitextract) +DO_BITPERM(sve2_bext_s, uint32_t, bitextract) +DO_BITPERM(sve2_bext_d, uint64_t, bitextract) + +static uint64_t bitdeposit(uint64_t data, uint64_t mask, int n) +{ + uint64_t res = 0; + int rb, db = 0; + + for (rb = 0; rb < n; ++rb) { + if ((mask >> rb) & 1) { + res |= ((data >> db) & 1) << rb; + ++db; + } + } + return res; +} + +DO_BITPERM(sve2_bdep_b, uint8_t, bitdeposit) +DO_BITPERM(sve2_bdep_h, uint16_t, bitdeposit) +DO_BITPERM(sve2_bdep_s, uint32_t, bitdeposit) +DO_BITPERM(sve2_bdep_d, uint64_t, bitdeposit) + +static uint64_t bitgroup(uint64_t data, uint64_t mask, int n) +{ + uint64_t resm = 0, resu = 0; + int db, rbm = 0, rbu = 0; + + for (db = 0; db < n; ++db) { + uint64_t val = (data >> db) & 1; + if ((mask >> db) & 1) { + resm |= val << rbm++; + } else { + resu |= val << rbu++; + } + } + + return resm | (resu << rbm); +} + +DO_BITPERM(sve2_bgrp_b, uint8_t, bitgroup) +DO_BITPERM(sve2_bgrp_h, uint16_t, bitgroup) +DO_BITPERM(sve2_bgrp_s, uint32_t, bitgroup) +DO_BITPERM(sve2_bgrp_d, uint64_t, bitgroup) + +#undef DO_BITPERM + #define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ea03bf6609..06812405f2 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6250,3 +6250,39 @@ static bool trans_USHLLT(DisasContext *s, arg_rri_esz *a) { return do_sve2_shll_tb(s, a, true, true); } + +static bool trans_BEXT(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_bext_b, gen_helper_sve2_bext_h, + gen_helper_sve2_bext_s, gen_helper_sve2_bext_d, + }; + if (!dc_isar_feature(aa64_sve2_bitperm, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], 0); +} + +static bool trans_BDEP(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_bdep_b, gen_helper_sve2_bdep_h, + gen_helper_sve2_bdep_s, gen_helper_sve2_bdep_d, + }; + if (!dc_isar_feature(aa64_sve2_bitperm, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], 0); +} + +static bool trans_BGRP(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_bgrp_b, gen_helper_sve2_bgrp_h, + gen_helper_sve2_bgrp_s, gen_helper_sve2_bgrp_d, + }; + if (!dc_isar_feature(aa64_sve2_bitperm, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], 0); +} From patchwork Tue Mar 9 16:19:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396032 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2785310jai; Tue, 9 Mar 2021 09:43:02 -0800 (PST) X-Google-Smtp-Source: ABdhPJx3FL9iMIyql32w+8tZUGJN8HiyZK2JFaNb9DHiFoTnfAH04zUgWNz9hWNJpZLOHhKWAgGP X-Received: by 2002:a02:661c:: with SMTP id k28mr29956762jac.78.1615311782817; Tue, 09 Mar 2021 09:43:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311782; cv=none; d=google.com; s=arc-20160816; b=X/08kjwtBrUyHjLY80uA1zAsUk1wczJPQxQRWxTSrIr1x8R2dg8YUojCkslR5PUE0X oRjzQ8LUvVLpvU4EvfE6w+/PkKcYQYAn3fiKZ4Gwo5hNEQf7UZhkrL/43rY4+Rekx43c I+ubAKuZebC5mI+3utWWI/amncn4WYHzAcz7rNgVyT5smStdqV++Vi5K2KhwmnY49rIK V0aVGBCq1i3QgTjlN5lCo3H/HQBcOqXHIFtAo+v8o0zHHpUAN72YzoSlF2nGHwW5zOnm It1uRX0+a8OKyogiA2lhY+EEYFfSPSnSTfJacRf6FkW7cdRgATiSdiVFnIMHT8B2banb JzFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=rDEY1GrPbC6zgYAZ0Uc0BKJUrlnah2xC2PMQNWF2rHg=; b=umvCEY4eUXO9bPlseZU0apdGeqro9fzdRv+bpNMbWqncwsUkF5bdgTCyv6gpX+62/V pztZzmNlJxo/broGGFAh8kp4DVCvCc/HlChKL9HgyXP4tpTSI7tqJBneGhDLo0vjYfV2 hse8QT24KF9cAJRtSPR5bWw8snqxTpSH075tzrOfS/OTSWP8jC8f/4M4qJIXTkWRA5ob 2plpgteBbVew8mj5bhNk3q4YXW22IuFNO9wIU565GDWRVU3buaZGqbwBHpNKFU0QqaXB 8RaXlGflq1nSXG0B91f0eSiZwUHSeP7QlDG4RPLbGOYVl9qReHTRqVH9IYf4M7+/ZRfo Za4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fsQlb2ve; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q126si14612099iod.60.2021.03.09.09.43.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:43:02 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fsQlb2ve; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47106 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgNq-0005GW-4D for patch@linaro.org; Tue, 09 Mar 2021 12:43:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45710) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6j-0007MO-Em for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:21 -0500 Received: from mail-oo1-xc30.google.com ([2607:f8b0:4864:20::c30]:39830) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6X-0002Sy-E0 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:17 -0500 Received: by mail-oo1-xc30.google.com with SMTP id i11so3175307ood.6 for ; Tue, 09 Mar 2021 08:21:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rDEY1GrPbC6zgYAZ0Uc0BKJUrlnah2xC2PMQNWF2rHg=; b=fsQlb2ve2NwjOuuXAEO7JyEDSrXnwhH7Wck4bjfEaaSKfyO9LKgFWyhljSkyCbSyR0 Vlr5xvo7D3p54uctpQksLlcQQe1EAJJfQtc9/DBhzU47quesGnJhmyERrZKDZocmjrCO NfKcVdtaOd2KDXiUw5GKAiOAzQC9+v9Da5xh5mGXL2cDbEvis09ITW30mX/RMzPToETn ozV6S/cXY2Txd99Q1pNj+0CwfyzNaTShSgZtBVF/x3mxdnlMFOZnlbWd47TdpNHCoMp3 vuEMP6qwq3oODARRvge9zLarU/MWwHXhUWE4Z/fCYYdAvb5dlnrEzhDQscNWFxT3P89B Hv8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rDEY1GrPbC6zgYAZ0Uc0BKJUrlnah2xC2PMQNWF2rHg=; b=r4Y7USkYmgVWgi3RXtwSuqIiAj2kiBfXJEupev6uXWhoJ+L1uG0Phq1GVbO3G1s5CA Ow5h2hHHnHWArH/xhErMedfxiulx+xSkILYNTlBaeqjCp+tGlpvtWlJpJAr2lGYGSHOq dJm6HkBGjAb1+QAJeravxHRDMD374/skIJzg3qZTUnJ+yR6CNzsPntyQ8os+oCFSATQa nuIoWPmVJg9mYu4BPtUHYG8K1vIIDhaeeAKfj17i2YLvgJ7ylptmUzmzxkzBRqPOmTrg VyULbct1GOBsR0psPj4S5zp5jgPfIfzDkllUzDYK1cwbkRmuSqBwmG9VMGUnZQvuOQlG ZQ/A== X-Gm-Message-State: AOAM5339BbPlEYYKRPKW1hHAmuNnuQ81Ub4qX8ugIrx8qw1SyDBagMKA lmudbQTligBIbixmkx2tzqGza05ToWhc7dgM X-Received: by 2002:a4a:e2c6:: with SMTP id l6mr22918563oot.31.1615306863209; Tue, 09 Mar 2021 08:21:03 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:02 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 18/78] target/arm: Implement SVE2 complex integer add Date: Tue, 9 Mar 2021 08:19:41 -0800 Message-Id: <20210309162041.23124-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c30; envelope-from=richard.henderson@linaro.org; helo=mail-oo1-xc30.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix subtraction ordering (laurent desnogues). --- target/arm/helper-sve.h | 10 +++++++++ target/arm/sve.decode | 9 ++++++++ target/arm/sve_helper.c | 42 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 31 ++++++++++++++++++++++++++++ 4 files changed, 92 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 4861481fe0..c2155cc544 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2392,3 +2392,13 @@ DEF_HELPER_FLAGS_4(sve2_bgrp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_bgrp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_bgrp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_bgrp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_cadd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_cadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_cadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_cadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqcadd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqcadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqcadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqcadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7cb89a0d47..7508b901d0 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1226,3 +1226,12 @@ EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm BEXT 01000101 .. 0 ..... 1011 00 ..... ..... @rd_rn_rm BDEP 01000101 .. 0 ..... 1011 01 ..... ..... @rd_rn_rm BGRP 01000101 .. 0 ..... 1011 10 ..... ..... @rd_rn_rm + +#### SVE2 Accumulate + +## SVE2 complex integer add + +CADD_rot90 01000101 .. 00000 0 11011 0 ..... ..... @rdn_rm +CADD_rot270 01000101 .. 00000 0 11011 1 ..... ..... @rdn_rm +SQCADD_rot90 01000101 .. 00000 1 11011 0 ..... ..... @rdn_rm +SQCADD_rot270 01000101 .. 00000 1 11011 1 ..... ..... @rdn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index ef9d40591a..6d70d10701 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1314,6 +1314,48 @@ DO_BITPERM(sve2_bgrp_d, uint64_t, bitgroup) #undef DO_BITPERM +#define DO_CADD(NAME, TYPE, H, ADD_OP, SUB_OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sub_r = simd_data(desc); \ + if (sub_r) { \ + for (i = 0; i < opr_sz; i += 2 * sizeof(TYPE)) { \ + TYPE acc_r = *(TYPE *)(vn + H(i)); \ + TYPE acc_i = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE el2_r = *(TYPE *)(vm + H(i)); \ + TYPE el2_i = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + acc_r = ADD_OP(acc_r, el2_i); \ + acc_i = SUB_OP(acc_i, el2_r); \ + *(TYPE *)(vd + H(i)) = acc_r; \ + *(TYPE *)(vd + H(i + sizeof(TYPE))) = acc_i; \ + } \ + } else { \ + for (i = 0; i < opr_sz; i += 2 * sizeof(TYPE)) { \ + TYPE acc_r = *(TYPE *)(vn + H(i)); \ + TYPE acc_i = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE el2_r = *(TYPE *)(vm + H(i)); \ + TYPE el2_i = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + acc_r = SUB_OP(acc_r, el2_i); \ + acc_i = ADD_OP(acc_i, el2_r); \ + *(TYPE *)(vd + H(i)) = acc_r; \ + *(TYPE *)(vd + H(i + sizeof(TYPE))) = acc_i; \ + } \ + } \ +} + +DO_CADD(sve2_cadd_b, int8_t, H1, DO_ADD, DO_SUB) +DO_CADD(sve2_cadd_h, int16_t, H1_2, DO_ADD, DO_SUB) +DO_CADD(sve2_cadd_s, int32_t, H1_4, DO_ADD, DO_SUB) +DO_CADD(sve2_cadd_d, int64_t, , DO_ADD, DO_SUB) + +DO_CADD(sve2_sqcadd_b, int8_t, H1, DO_SQADD_B, DO_SQSUB_B) +DO_CADD(sve2_sqcadd_h, int16_t, H1_2, DO_SQADD_H, DO_SQSUB_H) +DO_CADD(sve2_sqcadd_s, int32_t, H1_4, DO_SQADD_S, DO_SQSUB_S) +DO_CADD(sve2_sqcadd_d, int64_t, , do_sqadd_d, do_sqsub_d) + +#undef DO_CADD + #define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 06812405f2..e4eca9f16d 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6286,3 +6286,34 @@ static bool trans_BGRP(DisasContext *s, arg_rrr_esz *a) } return do_sve2_zzw_ool(s, a, fns[a->esz], 0); } + +static bool do_cadd(DisasContext *s, arg_rrr_esz *a, bool sq, bool rot) +{ + static gen_helper_gvec_3 * const fns[2][4] = { + { gen_helper_sve2_cadd_b, gen_helper_sve2_cadd_h, + gen_helper_sve2_cadd_s, gen_helper_sve2_cadd_d }, + { gen_helper_sve2_sqcadd_b, gen_helper_sve2_sqcadd_h, + gen_helper_sve2_sqcadd_s, gen_helper_sve2_sqcadd_d }, + }; + return do_sve2_zzw_ool(s, a, fns[sq][a->esz], rot); +} + +static bool trans_CADD_rot90(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, false, false); +} + +static bool trans_CADD_rot270(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, false, true); +} + +static bool trans_SQCADD_rot90(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, true, false); +} + +static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, true, true); +} From patchwork Tue Mar 9 16:19:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396023 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2776819jai; Tue, 9 Mar 2021 09:32:10 -0800 (PST) X-Google-Smtp-Source: ABdhPJxtz0rh/DwaE2nSx+tSOibTO1VAEwq54ZljoL9HUfy+9mkw9iGfSenieEToC8nmawtm6A3R X-Received: by 2002:a92:8752:: with SMTP id d18mr24055206ilm.283.1615311130557; Tue, 09 Mar 2021 09:32:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311130; cv=none; d=google.com; s=arc-20160816; b=Hik3pai3uvM0aliNEjg2vhhHSeh4a7QnICaNvlG4AMsrz3LpKW+FrdhINO4TGKbaKh nICUGoMXGZtiyDl7eHTtTZ3N2INurDjHavpYznSY/n/h/2aG6Wt/2xCO3XNb3lpJO4YP fway0j9EqGE0Ac8Nl7ibPAmE1ZnSDD0qkqwU18KBOVqbcN5fDSr2lIcLKIuOhiUz/GTd Hw62XLanApKo6M/I6Yz2/klYT17QqTo8ZE9Pa61fsMN+u2zwz7QW4DEYQJhoOKFD353+ r+jq0CZOc9S6JZYiQIbr6tEHurJLKghRKlW8lhEpz8g5lENArtkE0cvTYZLgYgw6FeRQ QETw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=s6m7gD9TSwnsmEAd7UsDrChSJzeJfhUW4ksDD+ynVvo=; b=XbVlMNyHZMrNvOWgJlkHRCLBgFBi6l5YB5BCy78klXn7Eqi7l5FHHhJm+Ghqk09uIp snlLu/fDGKjsu3I8+EVn1eLOpFp0w87SCctATLlezdD3d8soMxfxo3FdqMup+5I50M3j nwxpSQ91beVg4+LqvOYbqAwsnOz0hvS/ZbXWSrbPAquT3jYXt7v1dpXxDTKKftVSck7U Y9E+dV/jHw8JJZAS8EpAtREO9KDxEbA0mzFn0cALk1U0dfbfYgkVB5aMD6vtNTIj1T0b QesIToxMil2/dSwUGNqi5jGFKE4ZdtyHpAIqonGit3T0Xeoy3faat/xAwOVJioKQWLBG Ak7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VDbkviV7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x3si12475452ilj.150.2021.03.09.09.32.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:32:10 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VDbkviV7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:48582 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgDJ-0001lI-P6 for patch@linaro.org; Tue, 09 Mar 2021 12:32:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45672) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6i-0007KN-Fj for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:16 -0500 Received: from mail-oi1-x22d.google.com ([2607:f8b0:4864:20::22d]:40392) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6X-0002T6-DW for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:16 -0500 Received: by mail-oi1-x22d.google.com with SMTP id w65so15514493oie.7 for ; Tue, 09 Mar 2021 08:21:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=s6m7gD9TSwnsmEAd7UsDrChSJzeJfhUW4ksDD+ynVvo=; b=VDbkviV7z80Ab7PYlUFre1xirKLNZvbB1E4V44YCRX19nKcDzctngMaQcDMujqbbij FQdEZPwz6ltLVwBdTmt17ZvzJOhE/V6WPc1KwSyDSbTpAjKqq76FwPFfZ8IT8IlIugl9 9wJRJ5a0DnYVM8gnjksN1NswG5lslP0UsxNIOZmwRN4ZNx+bNAPFTAOreKo2V0qjw5kW tB2UzCGzYsaT5Euaz2MS8kmHR7DioZGB5nhlJWwSJl7D8viOU4bKxGsfAeI6bzW0DvE5 LkaxEs8RU1WK4MsJpwp/620YE9117MgXsUWhXh8AVvxJmUneII/qFHB8AabFF4BmIslW To3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s6m7gD9TSwnsmEAd7UsDrChSJzeJfhUW4ksDD+ynVvo=; b=c5QbJhuMFajAI8YFYA/9yL/49b67oUDN/X/B4UGGAnuBRRmrZL3++iXdhExK8cM/T8 JpMmtD9mymqR89wzjizqs9z8oYWH4HHAqiRVRY1G6kixsB86vHDQ1rLD5SlymflGTQBS D46ETP+g4a8RaIgktC7WSfYbWkJQCOyjUbJnirAMNPLe4cPu8igssaPwUEZQHm4VjFQn FocD1/AEw37NcLPDGoXnASheplaXJXQtyfmhynLjzHcWvdhvg2lpp3ZQ4Up9iLh8TjjT yYpYLIRDc2cKzWx3K8OOmpqbhZKjTGcLUiV+/rQD8OElKty4kS1IJCq4Gw65fYJIrhhw pEcQ== X-Gm-Message-State: AOAM5307Pwi/sDSNYgNj/8eXtlf0J4hxWVgx2S2PwB6x0kBU0R3A+rKw 12MHtsZD2BhP6LzBskBaHCuu1KsETi//ivwc X-Received: by 2002:aca:3509:: with SMTP id c9mr3454792oia.51.1615306864442; Tue, 09 Mar 2021 08:21:04 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:04 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 19/78] target/arm: Implement SVE2 integer absolute difference and accumulate long Date: Tue, 9 Mar 2021 08:19:42 -0800 Message-Id: <20210309162041.23124-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22d; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix select offsetting and argument order (laurent desnogues). --- target/arm/helper-sve.h | 14 ++++++++++ target/arm/sve.decode | 12 +++++++++ target/arm/sve_helper.c | 23 ++++++++++++++++ target/arm/translate-sve.c | 55 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 104 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index c2155cc544..229fb396b2 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2402,3 +2402,17 @@ DEF_HELPER_FLAGS_4(sve2_sqcadd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqcadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqcadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqcadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sabal_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sabal_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sabal_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uabal_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uabal_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uabal_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7508b901d0..56b7353bfa 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -70,6 +70,7 @@ &rpr_s rd pg rn s &rprr_s rd pg rn rm s &rprr_esz rd pg rn rm esz +&rrrr_esz rd ra rn rm esz &rprrr_esz rd pg rn rm ra esz &rpri_esz rd pg rn imm esz &ptrue rd esz pat s @@ -119,6 +120,10 @@ @rdn_i8s ........ esz:2 ...... ... imm:s8 rd:5 \ &rri_esz rn=%reg_movprfx +# Four operand, vector element size +@rda_rn_rm ........ esz:2 . rm:5 ... ... rn:5 rd:5 \ + &rrrr_esz ra=%reg_movprfx + # Three operand with "memory" size, aka immediate left shift @rd_rn_msz_rm ........ ... rm:5 .... imm:2 rn:5 rd:5 &rrri @@ -1235,3 +1240,10 @@ CADD_rot90 01000101 .. 00000 0 11011 0 ..... ..... @rdn_rm CADD_rot270 01000101 .. 00000 0 11011 1 ..... ..... @rdn_rm SQCADD_rot90 01000101 .. 00000 1 11011 0 ..... ..... @rdn_rm SQCADD_rot270 01000101 .. 00000 1 11011 1 ..... ..... @rdn_rm + +## SVE2 integer absolute difference and accumulate long + +SABALB 01000101 .. 0 ..... 1100 00 ..... ..... @rda_rn_rm +SABALT 01000101 .. 0 ..... 1100 01 ..... ..... @rda_rn_rm +UABALB 01000101 .. 0 ..... 1100 10 ..... ..... @rda_rn_rm +UABALT 01000101 .. 0 ..... 1100 11 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 6d70d10701..dd2c6a7404 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1241,6 +1241,29 @@ DO_ZZZ_NTB(sve2_eoril_d, uint64_t, , DO_EOR) #undef DO_ZZZ_NTB +#define DO_ZZZW_ACC(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + intptr_t sel1 = simd_data(desc) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel1)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel1)); \ + TYPEW aa = *(TYPEW *)(va + HW(i)); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, mm) + aa; \ + } \ +} + +DO_ZZZW_ACC(sve2_sabal_h, int16_t, int8_t, H1_2, H1, DO_ABD) +DO_ZZZW_ACC(sve2_sabal_s, int32_t, int16_t, H1_4, H1_2, DO_ABD) +DO_ZZZW_ACC(sve2_sabal_d, int64_t, int32_t, , H1_4, DO_ABD) + +DO_ZZZW_ACC(sve2_uabal_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) +DO_ZZZW_ACC(sve2_uabal_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) +DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) + +#undef DO_ZZZW_ACC + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index e4eca9f16d..c179e1f123 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -163,6 +163,18 @@ static void gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn, vsz, vsz, data, fn); } +/* Invoke an out-of-line helper on 4 Zregs. */ +static void gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn, + int rd, int rn, int rm, int ra, int data) +{ + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), + vsz, vsz, data, fn); +} + /* Invoke an out-of-line helper on 2 Zregs and a predicate. */ static void gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn, int rd, int rn, int pg, int data) @@ -6317,3 +6329,46 @@ static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a) { return do_cadd(s, a, true, true); } + +static bool do_sve2_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, + gen_helper_gvec_4 *fn, int data) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); + } + return true; +} + +static bool do_abal(DisasContext *s, arg_rrrr_esz *a, bool uns, bool sel) +{ + static gen_helper_gvec_4 * const fns[2][4] = { + { NULL, gen_helper_sve2_sabal_h, + gen_helper_sve2_sabal_s, gen_helper_sve2_sabal_d }, + { NULL, gen_helper_sve2_uabal_h, + gen_helper_sve2_uabal_s, gen_helper_sve2_uabal_d }, + }; + return do_sve2_zzzz_ool(s, a, fns[uns][a->esz], sel); +} + +static bool trans_SABALB(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, false, false); +} + +static bool trans_SABALT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, false, true); +} + +static bool trans_UABALB(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, true, false); +} + +static bool trans_UABALT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, true, true); +} From patchwork Tue Mar 9 16:19:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396018 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2769790jai; Tue, 9 Mar 2021 09:23:47 -0800 (PST) X-Google-Smtp-Source: ABdhPJxYMhTpJ1pIaYzto8bdAEfqSKQHzW5fH/1LdZRjmyqSrHZdyxQmPCOMmO2nGbu28JSFRIz0 X-Received: by 2002:a1f:a692:: with SMTP id p140mr17491252vke.0.1615310627589; Tue, 09 Mar 2021 09:23:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615310627; cv=none; d=google.com; s=arc-20160816; b=KyN/T6pML4pcmS8zwCP5qu6gu4LJa31sG38naM4719ddBr0AAlypbiQ90rqKBIJgTz Ml+FJVGwqpqXSu5uH1AuBTI8ZSGHLanAeXeQe/be/00waGn7XnNlzVgjbgTXaAZoBvaq Yu4o+y4hP/kuGgW9+MrAh8dLz1fKJxVlp34AbcvoMmbaJrcMHflXf0maNFnSLT1Ie6ia POzNyUNdUaxaoT1VO29FeCFJOgQ0u3j1QtdtP6qNXdYIvXPgNghgkqIAU3IxU5f46ETT /o68ljtkynTl9FFDMykQ7opY+KIRR/zfnfHeSuQ+5ywkJAahXi1Vp+qzy29j/+VNcHt+ rgNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=OB55iI1IUo2lwcmr8zS27hEcvopOYX8FmeGTqkG2if0=; b=KDNFGV024Q95BHMqGnrHqjK1r54s1lWU24nD3tfctEbIyWv+YOrZN2pAqq/M+1w6FZ qm/fY2hzhf0Qh7dSnsnAlmO8xJoHqrnfBeoTk4nwipobel64rclrjHulIY2S0b7KRmEG Ukb5XN/h4RY/nTx5gPVYfR2KRLBMDmI30FXo4rwLfO77HBmyLK1eer1lHPm7FcDdpDSH 1m5yYPmiSNbv1Tc4hd/G8NCbB1hBkkG9NxvCVh3iEt6s2EmQz9mje52aiWoJoHlZrxrJ ppdY911rb78XmyDVBKNhZXrA4ThA4V+y8nof8+egG30REMZkT67KJ0d0w7lPiknT5ydS Ohzg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FTpfUFX6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v6si2601909vsl.189.2021.03.09.09.23.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:23:47 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FTpfUFX6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36890 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJg5B-00051C-TD for patch@linaro.org; Tue, 09 Mar 2021 12:23:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45830) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6r-0007Nl-I8 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:27 -0500 Received: from mail-oi1-x22e.google.com ([2607:f8b0:4864:20::22e]:40393) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6Z-0002TH-SH for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:25 -0500 Received: by mail-oi1-x22e.google.com with SMTP id w65so15514542oie.7 for ; Tue, 09 Mar 2021 08:21:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OB55iI1IUo2lwcmr8zS27hEcvopOYX8FmeGTqkG2if0=; b=FTpfUFX6nSlrAv862BSzH6bjrB1HSO5Uj33nOwgkVsqVbH6StlKBUxxqXK+MwbfC2r sR8Ptu3bcnPSbjQG8D5IcPLETGsHJsIDkRaNY8040OUr4GurLi/QXuyIgIOG0hjlGgBo r0RdOLfW7R6Ab/XRgzam5JxQs2s/IQLvvXZSwNA1OoXgQjdEGIzXdHoxLmr2ebgtVsTm FEioWmFuzOz5tt1VYqcKntEht44gg27NvhvNT35pMQkdJUmgUT+T4fcRYVB6SqUX7M90 whkC4jCIvg6heG8p6epKQt0elS04JD1roevKe+x2r/YYKQXAEQE1HgtumOUPQbjxea2R TEAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OB55iI1IUo2lwcmr8zS27hEcvopOYX8FmeGTqkG2if0=; b=ZwxzLaR2LfL4B+XaGwNBmZJtOaBe65bY7GOZcSkwSPHG9ZvM9gHeAMRC54RUwF3JZ4 EJAn4Y+Iw7Xt9/dhJgaMyMrG3imqZjeyk4VKco0IQFxfpjKevF4xftwdQOmmp3HT3Ptd MzvQp15dzdpAMP+MbsGUmldXp7jfxCOPksx2B8IG7ywF8quzI8Wr710bjj6Pr/a6uzyA XzJNM7IFoVNR+kN+oufIvSjyEXZSC87yBie2Pxa5jP24hxf9QZIgnzqyW0dIrTYDvO8w IF44+iAOFdDdgAGETKbq6p6AaOWki5rR/I0jcyNdmGtV+sXrheMAg85nYSSpNyDrHZPs SIhA== X-Gm-Message-State: AOAM530EsWAmr1wiAQuvP0dDsivfeFvyufLvTgAshDT/IcT49++N4qQ1 Z8KMVPYsHkzjINatkiImPW5SyN6EXSgKyTij X-Received: by 2002:aca:f3d6:: with SMTP id r205mr3643357oih.48.1615306865567; Tue, 09 Mar 2021 08:21:05 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:05 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 20/78] target/arm: Implement SVE2 integer add/subtract long with carry Date: Tue, 9 Mar 2021 08:19:43 -0800 Message-Id: <20210309162041.23124-21-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22e; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix sel indexing and argument order (laurent desnogues). --- target/arm/helper-sve.h | 3 +++ target/arm/sve.decode | 6 ++++++ target/arm/sve_helper.c | 34 ++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 23 +++++++++++++++++++++++ 4 files changed, 66 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 229fb396b2..4a62012850 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2416,3 +2416,6 @@ DEF_HELPER_FLAGS_5(sve2_uabal_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_uabal_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_adcl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_adcl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 56b7353bfa..79046d81e3 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1247,3 +1247,9 @@ SABALB 01000101 .. 0 ..... 1100 00 ..... ..... @rda_rn_rm SABALT 01000101 .. 0 ..... 1100 01 ..... ..... @rda_rn_rm UABALB 01000101 .. 0 ..... 1100 10 ..... ..... @rda_rn_rm UABALT 01000101 .. 0 ..... 1100 11 ..... ..... @rda_rn_rm + +## SVE2 integer add/subtract long with carry + +# ADC and SBC decoded via size in helper dispatch. +ADCLB 01000101 .. 0 ..... 11010 0 ..... ..... @rda_rn_rm +ADCLT 01000101 .. 0 ..... 11010 1 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index dd2c6a7404..aea04a60fd 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1264,6 +1264,40 @@ DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) #undef DO_ZZZW_ACC +void HELPER(sve2_adcl_s)(void *vd, void *vn, void *vm, void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int sel = H4(extract32(desc, SIMD_DATA_SHIFT, 1)); + uint32_t inv = -extract32(desc, SIMD_DATA_SHIFT + 1, 1); + uint32_t *a = va, *n = vn; + uint64_t *d = vd, *m = vm; + + for (i = 0; i < opr_sz / 8; ++i) { + uint32_t e1 = a[2 * i + H4(0)]; + uint32_t e2 = n[2 * i + sel] ^ inv; + uint64_t c = extract64(m[i], 32, 1); + /* Compute and store the entire 33-bit result at once. */ + d[i] = c + e1 + e2; + } +} + +void HELPER(sve2_adcl_d)(void *vd, void *vn, void *vm, void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int sel = extract32(desc, SIMD_DATA_SHIFT, 1); + uint64_t inv = -(uint64_t)extract32(desc, SIMD_DATA_SHIFT + 1, 1); + uint64_t *d = vd, *a = va, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 8; i += 2) { + Int128 e1 = int128_make64(a[i]); + Int128 e2 = int128_make64(n[i + sel] ^ inv); + Int128 c = int128_make64(m[i + 1] & 1); + Int128 r = int128_add(int128_add(e1, e2), c); + d[i + 0] = int128_getlo(r); + d[i + 1] = int128_gethi(r); + } +} + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c179e1f123..92420d3ec7 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6372,3 +6372,26 @@ static bool trans_UABALT(DisasContext *s, arg_rrrr_esz *a) { return do_abal(s, a, true, true); } + +static bool do_adcl(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[2] = { + gen_helper_sve2_adcl_s, + gen_helper_sve2_adcl_d, + }; + /* + * Note that in this case the ESZ field encodes both size and sign. + * Split out 'subtract' into bit 1 of the data field for the helper. + */ + return do_sve2_zzzz_ool(s, a, fns[a->esz & 1], (a->esz & 2) | sel); +} + +static bool trans_ADCLB(DisasContext *s, arg_rrrr_esz *a) +{ + return do_adcl(s, a, false); +} + +static bool trans_ADCLT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_adcl(s, a, true); +} From patchwork Tue Mar 9 16:19:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396026 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2780745jai; Tue, 9 Mar 2021 09:36:55 -0800 (PST) X-Google-Smtp-Source: ABdhPJyyMhL+ForPkHXpvsAS6Qzv+mPyTCTgA1yuj131YC9o2l6UL+Um04BgvGko/tfDHpbQ6xH3 X-Received: by 2002:a1f:abcf:: with SMTP id u198mr3228021vke.19.1615311415240; Tue, 09 Mar 2021 09:36:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311415; cv=none; d=google.com; s=arc-20160816; b=0Pi4onntjD8CMlM7vEiH7uaS7F7Z4+VIpkDm2uCCktFi3LKkYoGQJlrE1xk6A6qi+t 3OR+qNSbnWsHvnt2mbgioE7b2xokqCyD/QmdG+Aa/eZZCF5ZIAQsE/iUtigDqupovAJj 52dK72NsDEYpdqeywlhTS/flhUc9GNN3fiG1b8pmWSMx4jf/b1aB5S7KaaHlSoC3q0ik qwCl/+nciBWqLYcmeUqkZ5CSj8JGhQF7sySJe3aRrptSZo+u7xGTYMDmegyCuEyLrGfx CwU86Wkjng2cUz9wrqTuY1lZVc3qKMePgrAQA5SKFX3jlczxuhAyZFxj+DXTyic0OafA 4/tg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=aFif56JV+fTnJ9XuXaPYxfPPOlJI5WKkI90jCx4L/Os=; b=XyiR3K6Oa14GtPwsWsxkhBQQLFpqpfgu3Ap8RddyJzk8r1+P1OE2qa8WceD/xRtiMQ +4tkYAx0AV/7MZvYYPHzzKEQE8OSgA0WJIaQWzbM16CjRg9Vslvf9LPwfFyLtpnLsZaH Yot5vPfeCY3E8SThPakUcKxIhRIyJGrHTRCxYzz80rW7aWS8wacAtUFl4PPceUw/yHER u5dEyDzH/1VvTU+JV86OJIEy1NDbWzdz85LUd4xqtDES0tAoA3VmmYG3NetEMtyINY2e 4RB1ICDFZoLYPuFCREMH1aUBErJZ7r57kh8PuC6Fgz7eqSYxUaUP/pgxtvRps8nGfbff D5HQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=npnOlkSC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b135si1190850vsd.224.2021.03.09.09.36.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:36:55 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=npnOlkSC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33380 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgHu-0007Xz-Ik for patch@linaro.org; Tue, 09 Mar 2021 12:36:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45906) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6x-0007Ok-5D for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:32 -0500 Received: from mail-oi1-x233.google.com ([2607:f8b0:4864:20::233]:44670) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6Z-0002TS-SQ for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:29 -0500 Received: by mail-oi1-x233.google.com with SMTP id w195so8891355oif.11 for ; Tue, 09 Mar 2021 08:21:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aFif56JV+fTnJ9XuXaPYxfPPOlJI5WKkI90jCx4L/Os=; b=npnOlkSC9RW5z868zDrdc4wPYr69gdikOhINi9glBWFpsBhMhihFDvJ7uwhiPI0coK y89uzWd+LobqKxlzdQuEpMIVV/L0Vtkf/CvhiGAGq84E2jmtXRNRJnxhxkLRrwmQX+3y zgCdoQJNSDEHq+zUBEyYlc2dk1oYmLf0npmYBXh+idCl5AO9ZyA+7au76HCkdCjxRDBQ bfg7W5NmWfCmlfAOpF06e0Mu9XXIch+Fgq2L3wdvvalrf4A8bGsRc9xLa4RoGhK7U0pp 3CxM7pWQ06vesdrS8o6KWd8xnOZxyiQwzu5fe3OTSdW0ztKe8KsjiJn/+v7SmSulBIM/ RXTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aFif56JV+fTnJ9XuXaPYxfPPOlJI5WKkI90jCx4L/Os=; b=fR+tBNUElxO+S1V3LHLMR20hyuccCEy9jYUu8P8hfqlN4ToojYmmXVEL7ZN38ZQ91y VgDgcFRdRjqRJHdBTKwQc5M+GzIMqFrvnZCaDaoVlsKUFIek8dpQ6GEyVwU8XfN5srFH xd6iWdy6UwtU9XooV4iVBWuzya2QSKcm7W2/ulc/iM+4dUcSNWoj4ta+ppOKRybcJoNv G0fAHIX8XnPyW6UIoWbDv2VGfoqz/YrEIZ4AjAteCjbaEfrWXacvjVLb9A82Oy3/dOJa iiKtpDe3MJHDmUFu5b/2TpRW4cNkCOkBwfqdsO+vRct1wvfs7hc3gYsg1cUjmB3u1yUZ o8FA== X-Gm-Message-State: AOAM532uNhu5RG66H7+qkYo8MO6M00+huS22xFyqNpP4stqFc/8p9vXs Q1FkmybdMuPl3AcvaBPVgPSWYHM5S0Wp4P+g X-Received: by 2002:aca:acc2:: with SMTP id v185mr3383689oie.28.1615306866788; Tue, 09 Mar 2021 08:21:06 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:06 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 21/78] target/arm: Implement SVE2 bitwise shift right and accumulate Date: Tue, 9 Mar 2021 08:19:44 -0800 Message-Id: <20210309162041.23124-22-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::233; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x233.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 8 ++++++++ target/arm/translate-sve.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 79046d81e3..d3c4ec6dd1 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1253,3 +1253,11 @@ UABALT 01000101 .. 0 ..... 1100 11 ..... ..... @rda_rn_rm # ADC and SBC decoded via size in helper dispatch. ADCLB 01000101 .. 0 ..... 11010 0 ..... ..... @rda_rn_rm ADCLT 01000101 .. 0 ..... 11010 1 ..... ..... @rda_rn_rm + +## SVE2 bitwise shift right and accumulate + +# TODO: Use @rda and %reg_movprfx here. +SSRA 01000101 .. 0 ..... 1110 00 ..... ..... @rd_rn_tszimm_shr +USRA 01000101 .. 0 ..... 1110 01 ..... ..... @rd_rn_tszimm_shr +SRSRA 01000101 .. 0 ..... 1110 10 ..... ..... @rd_rn_tszimm_shr +URSRA 01000101 .. 0 ..... 1110 11 ..... ..... @rd_rn_tszimm_shr diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 92420d3ec7..bd4d6f1a69 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6395,3 +6395,37 @@ static bool trans_ADCLT(DisasContext *s, arg_rrrr_esz *a) { return do_adcl(s, a, true); } + +static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn) +{ + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + unsigned rd_ofs = vec_full_reg_offset(s, a->rd); + unsigned rn_ofs = vec_full_reg_offset(s, a->rn); + fn(a->esz, rd_ofs, rn_ofs, a->imm, vsz, vsz); + } + return true; +} + +static bool trans_SSRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_ssra); +} + +static bool trans_USRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_usra); +} + +static bool trans_SRSRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_srsra); +} + +static bool trans_URSRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_ursra); +} From patchwork Tue Mar 9 16:19:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396020 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2774957jai; Tue, 9 Mar 2021 09:30:04 -0800 (PST) X-Google-Smtp-Source: ABdhPJwI0roEVD49YEgEBs3CkroRBxml7Xz5HLtln1K7qRlqrKd/GFcMhyNLlszjmKEyWhUmJ5m0 X-Received: by 2002:a92:2c04:: with SMTP id t4mr25867047ile.99.1615311004430; Tue, 09 Mar 2021 09:30:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311004; cv=none; d=google.com; s=arc-20160816; b=frEMz8KE8oe3zInuev/mMuCjWR3q5+rVe9s7xbvfPgoWwMsyFZgAlSCfhGaKw4CndH 5VA5vNA2x8kyJ1FQFnTWMLUCzElQMjscGUXVfjqdpHF9eaR0/3lsUS43xiy0UFSgjgF+ B7dMtvPK+nAJFn65szRJ0lyQjp7JV/5m98MI4GCoYoJu79CNa8YgDjC89k/qSiIsajki 0fwInnI4rLM1Nvzv+4xc9TEl/7k6savHhLeDe7inGqgOQdSkd/hn0TDrmbfUBMuFMtvq V1BlS299Re2bcnqJF/izotqP1tz8Dl0ydkqsnO5QiHPK951CpUbKJSA8eNeVts6ESOgU gvJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=K7rOIpXmUiahZQykYscgkQbVm3sAJxgER+idBTOFlEo=; b=zwbromdIEScyF/5bvuq99QxsZxlTgH+fPo2pGHJU/Cy7TWUI/IZlKvTtBWgWf3ziLA SWT7uA/9aQTpQLc2041PRrIPh44LcSVUdOdxp+l1Ponr37wJdcGrTuGB2OWVKrOMJrXP 8SdQso8lc/mwOFIvNC/FV1+199+zhHSF/ddFe93E8U6E16ApkqM2Tc7ShNhag0HOobSa qKLNLkRwaQeofwMYqs3YN3kGn/pjv/qYt8WGsStIO3eHX+HRZ1cwwlzyz6WLyozvYW70 I9osi1O2qHkJYzIbiQ1VTkYHVw/h6zj+10ZMmHWa/QTAjKdXTX0vU5Ge9L1XolA41aG8 cOwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kG0Ezhpv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b16si14646085ilq.33.2021.03.09.09.30.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:30:04 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kG0Ezhpv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45442 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgBH-0000T0-QM for patch@linaro.org; Tue, 09 Mar 2021 12:30:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45910) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6x-0007Op-73 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:32 -0500 Received: from mail-oi1-x229.google.com ([2607:f8b0:4864:20::229]:38108) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6b-0002Te-H3 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:30 -0500 Received: by mail-oi1-x229.google.com with SMTP id v192so7957014oia.5 for ; Tue, 09 Mar 2021 08:21:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K7rOIpXmUiahZQykYscgkQbVm3sAJxgER+idBTOFlEo=; b=kG0EzhpvRl6o9MWf329J/+5Xvf9lP3t/hnpnNbQA3goQxSgTY1KWTyxbGLwqeLFB5f f0lCBW+v+3lF/N+8lF6uJv3+IwO5dZ/BaBF1Sifpfl6xLxz4DWdKkmhzYYEQkwBEsPEN VLwCwl1ocvbrKgo8TzL3bz2IWi79VBOrKnVJLpgSAMCgUmnARQk+1VXOReR/frKoWZZP dRD3SKJiU5iUfHHdoU0nnvI7240XyXuBzylsHAiTLNu9ZZ4YCy0G48lhq0KYlX2rpBXt A1Bu3N+lvQG/L20dd4zPdjYomWRpDtiy5+B0YlB86JJwVRJ5OPsmO/aUGFkJOCajWFup rfkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K7rOIpXmUiahZQykYscgkQbVm3sAJxgER+idBTOFlEo=; b=Ke70VYZZmMtAzzC+ShihPb83Ge/XRHEiX07/dzLeSHOaCFGcOxue4wslgXgVocA5bn ol8clxNGX00g6x3HZCwC9liopQ+j9H4Qa9zC9zbrJ6qukhbdJ+YIeOCF0Q8nS3zQDlOO zRLErtRaDhlbB//teYlmkPZRrFggl9e6znS6dUjnpKB0rwLPsPhXwtDBTYwYFpyokelD 8su9CfL65PgtXDsqqEVxJF9IRAIH7ekkR3KqV44i0H1rQY9dxdK8kcWKe731mj7whDJs 4LRPwwPgvJv0LcBuQwcn0VAanllcH4rvE+/ZbyKPqB4Y1nZ9p8LMw00vAqohhsbP3wzq D2/Q== X-Gm-Message-State: AOAM530mhCrdU0scuwOc5SovXiVR2rFlE5pAV8H80gybhPQQHIm27kxN rkOgTg2kT3A6yIom7vLuWusG9lbAaIKIK5jv X-Received: by 2002:aca:5bd7:: with SMTP id p206mr3460021oib.66.1615306867820; Tue, 09 Mar 2021 08:21:07 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:07 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 22/78] target/arm: Implement SVE2 bitwise shift and insert Date: Tue, 9 Mar 2021 08:19:45 -0800 Message-Id: <20210309162041.23124-23-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::229; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x229.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 5 +++++ target/arm/translate-sve.c | 10 ++++++++++ 2 files changed, 15 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index d3c4ec6dd1..695a16551e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1261,3 +1261,8 @@ SSRA 01000101 .. 0 ..... 1110 00 ..... ..... @rd_rn_tszimm_shr USRA 01000101 .. 0 ..... 1110 01 ..... ..... @rd_rn_tszimm_shr SRSRA 01000101 .. 0 ..... 1110 10 ..... ..... @rd_rn_tszimm_shr URSRA 01000101 .. 0 ..... 1110 11 ..... ..... @rd_rn_tszimm_shr + +## SVE2 bitwise shift and insert + +SRI 01000101 .. 0 ..... 11110 0 ..... ..... @rd_rn_tszimm_shr +SLI 01000101 .. 0 ..... 11110 1 ..... ..... @rd_rn_tszimm_shl diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index bd4d6f1a69..d41fbd9697 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6429,3 +6429,13 @@ static bool trans_URSRA(DisasContext *s, arg_rri_esz *a) { return do_sve2_fn2i(s, a, gen_gvec_ursra); } + +static bool trans_SRI(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_sri); +} + +static bool trans_SLI(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_sli); +} From patchwork Tue Mar 9 16:19:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396036 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2789516jai; Tue, 9 Mar 2021 09:48:47 -0800 (PST) X-Google-Smtp-Source: ABdhPJzEsrFmgLSLPqFKWmDnAc8mgWUGpzknzo3DtWBWHl7X3kEELcIjfSsMywq3SWv04AyDmhz0 X-Received: by 2002:a02:4087:: with SMTP id n129mr30117523jaa.143.1615312126912; Tue, 09 Mar 2021 09:48:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312126; cv=none; d=google.com; s=arc-20160816; b=AnBXYt0BPASgJxJOz5St+y/io5X0Oz/xrtdunC870aEhSOrej9TbAg7/QVIVzePUra hK/rjXkmklIwOrE6eSyTnaDXl0R15GlLl0hcytw1jfrh17u2UF1nWGekjQV3pARqUQA+ i5qWlW8M/O4aTwgPiOJDyNgyARuXRJune7CI6vtF16IBX3M4YGePRMe8a9bnX5/zwpcI j16JGEzaZ/islBW8Os8vjVAWocskmc2FiDoyYNG8r2zCAHYs4Unc9O4roemfTlend6hD QdmPiK4vghh/LJqlgIEP/McTAdhkkQNc+5eSx3aZKKXL9GXOeLXCoYBDknt0mOs6mEOg pgMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=PNu/k9q7y7xSUi1QwShDN3GcUxd7gykmxrek3+gYYGs=; b=IOPZKoBwoeuMDm2gUilRvkQmoFkjHaUeA64LjD/wazNDBXLukmg7VQaTRj6dIHMsdc XNIAe9duT7V15mvXGrMXLYlUD/uY8E9gDE3Gej22g2thybLUmPRjIViO3Dsq4Gm86OLm 3jaGCqbgl8gCEeiVYAHnhKyXLMV+OT2XXFlj8v4AMeSr3P6oMf/mWX9tszXcbNJtMJbZ qABgmbs/VVlT/AtYSMUTFDx/RHCM4/L+GnpioohV8yTssU34NMsGG60rUR+OzrzCt/Wl OUYv2H362I0HPyHKQckdnN76oJAseg9sVowXInOMfnQQBH9WpjUYs4c3I0U5dZrAVdYa lA3Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=M12jkkYq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g3si14490763ild.84.2021.03.09.09.48.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:48:46 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=M12jkkYq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60076 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgTO-0002Zb-8z for patch@linaro.org; Tue, 09 Mar 2021 12:48:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45934) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf6z-0007Pm-7v for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:34 -0500 Received: from mail-oo1-xc35.google.com ([2607:f8b0:4864:20::c35]:33801) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6b-0002To-SG for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:32 -0500 Received: by mail-oo1-xc35.google.com with SMTP id x23so3183290oop.1 for ; Tue, 09 Mar 2021 08:21:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PNu/k9q7y7xSUi1QwShDN3GcUxd7gykmxrek3+gYYGs=; b=M12jkkYqw9CF3LWYBlAFIk3eJ/3n8aSy1g7SZFhkOCwDMXyDgf6f/yt87+guBlqB4g fhI0xc+hLV7hAw7jN0W+m1etKTSI1EGrh6d8o7NE5GcxHGJ0Bg9hp0zFxfR+D/TaDRZY mclH4ioIVhORR+xQ9CVxh0oQJ/6nVNfpENH2dmnwFlHJL1ScsqSagdPXoq1gn5alYuWl wDfJNeYH+0q21s2GZyw+GfJ3zvO8UXtxjeaFqyyv7ugcVOl4sFnOmnh+Daj1SHgMD4K1 hj2SPaL8ZLI9b9joJWGEep6c34SZZKsAvUDe1Xtlb7+gMMbToDuWbqdJQ+hQvVnmySGS WrkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PNu/k9q7y7xSUi1QwShDN3GcUxd7gykmxrek3+gYYGs=; b=Tr7MIHkdPKIwlyXuAau2YkgdBTVLLyXNTw6ZLbSLS8n7LOR0zhwtmONTFBnMj9WNGb yA2BYZILuGvJWFpOLDPAkI5TTO8idSBQl0A7rqvRdAgFfQyvTLNsktMvkEA/oGxb4vY+ RQjHWo6dJZTcBpkFzi9Mbs1U5srpEm+5uVJeutc0npa+kXQM5ddCl3rpB1PXvPodqD4v e/OTiUZbJ11gojzwiIynH3H13ooEawBIvZNGCiLbcxqwG3EaNdgwS3HzwpAwLXf44JX7 Ey20ePwXykHgXqCyM5/kSOAxDTty2EdYZtoQiPR+gRA5txuSspB+uQfHm0aZwp6c1k2U LcuA== X-Gm-Message-State: AOAM531jKSsVaRW7EUVQFq+RkaTlBhSbXXKQKhKMOE0IJJdaUPdeBXy0 1cj8L7TqHQij6psKSlGROVCBxwhpN/AQoAX+ X-Received: by 2002:a4a:96b3:: with SMTP id s48mr22923006ooi.11.1615306868833; Tue, 09 Mar 2021 08:21:08 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:08 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 23/78] target/arm: Implement SVE2 integer absolute difference and accumulate Date: Tue, 9 Mar 2021 08:19:46 -0800 Message-Id: <20210309162041.23124-24-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c35; envelope-from=richard.henderson@linaro.org; helo=mail-oo1-xc35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 6 ++++++ target/arm/translate-sve.c | 21 +++++++++++++++++++++ 2 files changed, 27 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 695a16551e..32b15e4192 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1266,3 +1266,9 @@ URSRA 01000101 .. 0 ..... 1110 11 ..... ..... @rd_rn_tszimm_shr SRI 01000101 .. 0 ..... 11110 0 ..... ..... @rd_rn_tszimm_shr SLI 01000101 .. 0 ..... 11110 1 ..... ..... @rd_rn_tszimm_shl + +## SVE2 integer absolute difference and accumulate + +# TODO: Use @rda and %reg_movprfx here. +SABA 01000101 .. 0 ..... 11111 0 ..... ..... @rd_rn_rm +UABA 01000101 .. 0 ..... 11111 1 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index d41fbd9697..d27fab2b80 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6439,3 +6439,24 @@ static bool trans_SLI(DisasContext *s, arg_rri_esz *a) { return do_sve2_fn2i(s, a, gen_gvec_sli); } + +static bool do_sve2_fn_zzz(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm); + } + return true; +} + +static bool trans_SABA(DisasContext *s, arg_rrr_esz *a) +{ + return do_sve2_fn_zzz(s, a, gen_gvec_saba); +} + +static bool trans_UABA(DisasContext *s, arg_rrr_esz *a) +{ + return do_sve2_fn_zzz(s, a, gen_gvec_uaba); +} From patchwork Tue Mar 9 16:19:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396044 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2795033jai; Tue, 9 Mar 2021 09:56:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJxXbpgcoEFuRknxWYNmSH4AKaSKG+oYOKid6/Yka4eAXmtr/++WLkDRGBkaMiYJVlNPitPO X-Received: by 2002:a05:6602:80d:: with SMTP id z13mr22994860iow.17.1615312613346; Tue, 09 Mar 2021 09:56:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312613; cv=none; d=google.com; s=arc-20160816; b=Rtu+/5VhRt615IZG0s2JRN+7UOf6sdzloDlVIBgPbu4Z0SJ7COiIH8IL+HA8FTaO2k SpqB9gtEB4968DfcGZUZqtRGsCYgDxlUupb+04dvfOgk3QQ6zuIyAYxGdvGYVX+bvMgd MuFg2/f0T3DS+ygbR7aRrvDu6/Pc2Gl3Iv0BOgNcMkSW57gdiGTeTPhc2uDA0nBJurRf rA6DX6iBeqTEimA8/QluYNn9iOqM6JVoxSzNdE80sTnh30d6OMw9bOa2akG75EjeKRWo bXhh35HR5vSMiMInbp828rWm5ziilyixyMOSzuUTEyapNMR3GLDuK470QF7M3urqLFPS NKUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=O3EZKnXFTtqPZ2MBMJ6BzW7ax9HuO5PgBXq+C5BPyHY=; b=ylupvGowQGu6gZhkFxvIhf6mhlQWbePQK5BpUT43c7TwJgmxulnpmxcLOWazcIviXd TQzLQ0sEDLU0hXc1eQ0nYRGpQ6vxKJGRGUNpdswythrElnfBAbF1/md9DZ2+mS6k6AJu KaFLjHtVN5Nn0B0qyrWKHmn4+2/oaaGpfFj+dCCUeH6JyH/S6KjkOJCD3as1PrGGWq92 Gn9gEjK9JxBaivsXWw6EyzAcIEf2EL+OS43qxYPP6K/jh9HrP5LBt5IdYcQaZc1JU3qB cdR0PzPr1LsHZc48/9NXRu84Eogg/38AbP3mjdz7U5OHSksuOpRX90Tiemp0NPZgd8aU XR7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=s9siPygL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i9si13447810ilu.162.2021.03.09.09.56.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:56:53 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=s9siPygL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50276 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgbE-0001vX-FR for patch@linaro.org; Tue, 09 Mar 2021 12:56:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46022) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf75-0007VQ-UR for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:41 -0500 Received: from mail-oo1-xc2c.google.com ([2607:f8b0:4864:20::c2c]:43012) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6c-0002U0-Uv for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:39 -0500 Received: by mail-oo1-xc2c.google.com with SMTP id x19so3163800ooj.10 for ; Tue, 09 Mar 2021 08:21:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=O3EZKnXFTtqPZ2MBMJ6BzW7ax9HuO5PgBXq+C5BPyHY=; b=s9siPygLtngVa8o5WuiyDY5+UCaRpZiiyWbBPk5E8dFZmyBZxbeaSuY9lif7SHmNyX /vVxltk0gJZmYu0F9HY2FqoVl0Gqkegn1+yRp3TjFDbtg242hJVzabT2peb8iKk5RcuE Xzvu7PnciR/DDGjb1cu6BrtxXqRfK3XfLjrVymZMursrsuDaHQt+2q/7mI7zYd+PNy8X yRzLS82G90nUnzLGKNbdVzohD9kgtl3l7/QCt3vZC6E79M8ExndAUebeTbyb4m5U15tG s807/CSx4dgx2318W22dU4bInBa9ZlGKEVDAYLSLKsUvpVs3hYcU3VKDmZgUdm3MXz/5 TYtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=O3EZKnXFTtqPZ2MBMJ6BzW7ax9HuO5PgBXq+C5BPyHY=; b=lZQKpyZvFLVsfhE1TDcFvmdjz0ZHn92il/kE3ElgCUp6Vra4lk8pe7C8Di+VJ42rLU 21d2DFooFnlfhADDEAN1AozhNazOIFYJP5QGlqJob0u0DUIR27WBy+1rYK9/Ix4ALn/g Bezlwro1DQmOXUcXYz2oPBc7gpvXfkvQekkJvaLEn+Ir9EPjhHMG6VTm6cJiGgfzrwn7 VAcNu9CxECm4LKeZSIMd9BzhI5lAidltk9M4tmn6WNkFB33rd/kTFwy29Z4kcJYGzvpR Et01jKo1xSojQ0YJYLrtAXGwPTQ1bVwmcctuH/Ld6qGRAZv0YEvKy8PaB5tAteEkQ5qo AOfA== X-Gm-Message-State: AOAM531jzO5lgerzcWFRZgoMrUIUvAp6OG6VhNN3gowivW3iOCRd1UqV Qn2Ney9mqX6JeUfWfN8GK+fi7AeFwhjBW9bx X-Received: by 2002:a4a:6b04:: with SMTP id g4mr6710220ooc.78.1615306869834; Tue, 09 Mar 2021 08:21:09 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:09 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 24/78] target/arm: Implement SVE2 saturating extract narrow Date: Tue, 9 Mar 2021 08:19:47 -0800 Message-Id: <20210309162041.23124-25-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c2c; envelope-from=richard.henderson@linaro.org; helo=mail-oo1-xc2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 24 ++++ target/arm/sve.decode | 12 ++ target/arm/sve_helper.c | 56 +++++++++ target/arm/translate-sve.c | 238 +++++++++++++++++++++++++++++++++++++ 4 files changed, 330 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 4a62012850..b302203ce8 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2419,3 +2419,27 @@ DEF_HELPER_FLAGS_5(sve2_uabal_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_adcl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_adcl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqxtnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtunb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqxtnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 32b15e4192..19866ec4c6 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1272,3 +1272,15 @@ SLI 01000101 .. 0 ..... 11110 1 ..... ..... @rd_rn_tszimm_shl # TODO: Use @rda and %reg_movprfx here. SABA 01000101 .. 0 ..... 11111 0 ..... ..... @rd_rn_rm UABA 01000101 .. 0 ..... 11111 1 ..... ..... @rd_rn_rm + +#### SVE2 Narrowing + +## SVE2 saturating extract narrow + +# Bits 23, 18-16 are zero, limited in the translator via esz < 3 & imm == 0. +SQXTNB 01000101 .. 1 ..... 010 000 ..... ..... @rd_rn_tszimm_shl +SQXTNT 01000101 .. 1 ..... 010 001 ..... ..... @rd_rn_tszimm_shl +UQXTNB 01000101 .. 1 ..... 010 010 ..... ..... @rd_rn_tszimm_shl +UQXTNT 01000101 .. 1 ..... 010 011 ..... ..... @rd_rn_tszimm_shl +SQXTUNB 01000101 .. 1 ..... 010 100 ..... ..... @rd_rn_tszimm_shl +SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index aea04a60fd..115daa50a5 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1264,6 +1264,62 @@ DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) #undef DO_ZZZW_ACC +#define DO_XTNB(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + i); \ + nn = OP(nn) & MAKE_64BIT_MASK(0, sizeof(TYPE) * 4); \ + *(TYPE *)(vd + i) = nn; \ + } \ +} + +#define DO_XTNT(NAME, TYPE, TYPEN, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc), odd = H(sizeof(TYPEN)); \ + for (i = 0; i < opr_sz; i += sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + i); \ + *(TYPEN *)(vd + i + odd) = OP(nn); \ + } \ +} + +#define DO_SQXTN_H(n) do_sat_bhs(n, INT8_MIN, INT8_MAX) +#define DO_SQXTN_S(n) do_sat_bhs(n, INT16_MIN, INT16_MAX) +#define DO_SQXTN_D(n) do_sat_bhs(n, INT32_MIN, INT32_MAX) + +DO_XTNB(sve2_sqxtnb_h, int16_t, DO_SQXTN_H) +DO_XTNB(sve2_sqxtnb_s, int32_t, DO_SQXTN_S) +DO_XTNB(sve2_sqxtnb_d, int64_t, DO_SQXTN_D) + +DO_XTNT(sve2_sqxtnt_h, int16_t, int8_t, H1, DO_SQXTN_H) +DO_XTNT(sve2_sqxtnt_s, int32_t, int16_t, H1_2, DO_SQXTN_S) +DO_XTNT(sve2_sqxtnt_d, int64_t, int32_t, H1_4, DO_SQXTN_D) + +#define DO_UQXTN_H(n) do_sat_bhs(n, 0, UINT8_MAX) +#define DO_UQXTN_S(n) do_sat_bhs(n, 0, UINT16_MAX) +#define DO_UQXTN_D(n) do_sat_bhs(n, 0, UINT32_MAX) + +DO_XTNB(sve2_uqxtnb_h, uint16_t, DO_UQXTN_H) +DO_XTNB(sve2_uqxtnb_s, uint32_t, DO_UQXTN_S) +DO_XTNB(sve2_uqxtnb_d, uint64_t, DO_UQXTN_D) + +DO_XTNT(sve2_uqxtnt_h, uint16_t, uint8_t, H1, DO_UQXTN_H) +DO_XTNT(sve2_uqxtnt_s, uint32_t, uint16_t, H1_2, DO_UQXTN_S) +DO_XTNT(sve2_uqxtnt_d, uint64_t, uint32_t, H1_4, DO_UQXTN_D) + +DO_XTNB(sve2_sqxtunb_h, int16_t, DO_UQXTN_H) +DO_XTNB(sve2_sqxtunb_s, int32_t, DO_UQXTN_S) +DO_XTNB(sve2_sqxtunb_d, int64_t, DO_UQXTN_D) + +DO_XTNT(sve2_sqxtunt_h, int16_t, int8_t, H1, DO_UQXTN_H) +DO_XTNT(sve2_sqxtunt_s, int32_t, int16_t, H1_2, DO_UQXTN_S) +DO_XTNT(sve2_sqxtunt_d, int64_t, int32_t, H1_4, DO_UQXTN_D) + +#undef DO_XTNB +#undef DO_XTNT + void HELPER(sve2_adcl_s)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index d27fab2b80..65f1b11848 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6460,3 +6460,241 @@ static bool trans_UABA(DisasContext *s, arg_rrr_esz *a) { return do_sve2_fn_zzz(s, a, gen_gvec_uaba); } + +static bool do_sve2_narrow_extract(DisasContext *s, arg_rri_esz *a, + const GVecGen2 ops[3]) +{ + if (a->esz < 0 || a->esz > MO_32 || a->imm != 0 || + !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vsz, vsz, &ops[a->esz]); + } + return true; +} + +static const TCGOpcode sqxtn_list[] = { + INDEX_op_shli_vec, INDEX_op_smin_vec, INDEX_op_smax_vec, 0 +}; + +static void gen_sqxtnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t mask = (1ull << halfbits) - 1; + int64_t min = -1ull << (halfbits - 1); + int64_t max = -min - 1; + + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, d, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, d, d, t); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_and_vec(vece, d, d, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtnb_vec, + .opt_opc = sqxtn_list, + .fno = gen_helper_sve2_sqxtnb_h, + .vece = MO_16 }, + { .fniv = gen_sqxtnb_vec, + .opt_opc = sqxtn_list, + .fno = gen_helper_sve2_sqxtnb_s, + .vece = MO_32 }, + { .fniv = gen_sqxtnb_vec, + .opt_opc = sqxtn_list, + .fno = gen_helper_sve2_sqxtnb_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static void gen_sqxtnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t mask = (1ull << halfbits) - 1; + int64_t min = -1ull << (halfbits - 1); + int64_t max = -min - 1; + + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtnt_vec, + .opt_opc = sqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtnt_h, + .vece = MO_16 }, + { .fniv = gen_sqxtnt_vec, + .opt_opc = sqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtnt_s, + .vece = MO_32 }, + { .fniv = gen_sqxtnt_vec, + .opt_opc = sqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtnt_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static const TCGOpcode uqxtn_list[] = { + INDEX_op_shli_vec, INDEX_op_umin_vec, 0 +}; + +static void gen_uqxtnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_UQXTNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_uqxtnb_vec, + .opt_opc = uqxtn_list, + .fno = gen_helper_sve2_uqxtnb_h, + .vece = MO_16 }, + { .fniv = gen_uqxtnb_vec, + .opt_opc = uqxtn_list, + .fno = gen_helper_sve2_uqxtnb_s, + .vece = MO_32 }, + { .fniv = gen_uqxtnb_vec, + .opt_opc = uqxtn_list, + .fno = gen_helper_sve2_uqxtnb_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static void gen_uqxtnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_UQXTNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_uqxtnt_vec, + .opt_opc = uqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_uqxtnt_h, + .vece = MO_16 }, + { .fniv = gen_uqxtnt_vec, + .opt_opc = uqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_uqxtnt_s, + .vece = MO_32 }, + { .fniv = gen_uqxtnt_vec, + .opt_opc = uqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_uqxtnt_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static const TCGOpcode sqxtun_list[] = { + INDEX_op_shli_vec, INDEX_op_umin_vec, INDEX_op_smax_vec, 0 +}; + +static void gen_sqxtunb_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, d, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, d, d, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTUNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtunb_vec, + .opt_opc = sqxtun_list, + .fno = gen_helper_sve2_sqxtunb_h, + .vece = MO_16 }, + { .fniv = gen_sqxtunb_vec, + .opt_opc = sqxtun_list, + .fno = gen_helper_sve2_sqxtunb_s, + .vece = MO_32 }, + { .fniv = gen_sqxtunb_vec, + .opt_opc = sqxtun_list, + .fno = gen_helper_sve2_sqxtunb_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static void gen_sqxtunt_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTUNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtunt_vec, + .opt_opc = sqxtun_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtunt_h, + .vece = MO_16 }, + { .fniv = gen_sqxtunt_vec, + .opt_opc = sqxtun_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtunt_s, + .vece = MO_32 }, + { .fniv = gen_sqxtunt_vec, + .opt_opc = sqxtun_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtunt_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} From patchwork Tue Mar 9 16:19:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396030 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2784220jai; Tue, 9 Mar 2021 09:41:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJz2wkVzwDt1sDVLTXU4b0T1gMQkSXwUU2IjimSset4KwsEuoOuta8W6ILiz1vX1R+LN66N6 X-Received: by 2002:a02:cb48:: with SMTP id k8mr29566109jap.52.1615311688647; Tue, 09 Mar 2021 09:41:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311688; cv=none; d=google.com; s=arc-20160816; b=sQ+l+Uv/hywBHKMQxpZ5IqySlWC4G8KSs2EdqMIjUdgu3Wn7oW2+tHkNDC6T9MmPcQ gEhQtpmJC+NxqBidhi4jGAeGS9cI1Dqt82DoJTzoV8JApob0QHLqKyHEsU6+DknPtGfj vDSzFchvs69zxL3Z7sbnxtfyCsR6KjJRoTVZH59YzGZqUHm3Xw3xvzp97Hv6GmkZCOtj ZGrtLcfJQkubNeLMwkWX9yn9luGSTPuuRlDOFD/kOdA5drdsYtv9oxHlZ+fEvXPu65vs djfwhbiL6CancTeaVjJXKTgzgnNVWQolat4PnUOkM+g8UbSbWNnWScy3GwiNvvL0ToQL svFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=hzN4bO1Q9kWzjd+yXsPtlaqCNoI3movNX0neMlD42A0=; b=aDSWwYz3ANV1TbYfIiB4oSEUw7bXLXvkJG/or+zDnENF4Mo+33kU1xxo1H9tPNkiXA 85m3Unc0AM6emVo+LtAx1ZpKKgsLe8/y+NXTClqVLBgPp5a9ji2ehulUZIkKnbPc0No/ 8VPo5tFsMxG4ShwYCuUXA5yIhz/FDjo3thdE+RMxu30jbkJ8k4TuvsXHnCQOKd1J7BWf j1wketrUdmI/OJVng8QShHR7uePqdZORJRA24J/Keunk23FWow4guJVNse9uHPu+twWy ZHIkh5Sr2s2Rb55EdS2E5vu3RiFfIo8C3FQ2Vd672wD3yHXvGqRhH1SmtofAKnlcBhLP rhXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i5KJQKN9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r7si12103484jad.72.2021.03.09.09.41.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:41:28 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i5KJQKN9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42968 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgMK-0003QN-0i for patch@linaro.org; Tue, 09 Mar 2021 12:41:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45952) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf71-0007RH-32 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:35 -0500 Received: from mail-oi1-x233.google.com ([2607:f8b0:4864:20::233]:41366) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6d-0002UI-VT for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:34 -0500 Received: by mail-oi1-x233.google.com with SMTP id y131so12654434oia.8 for ; Tue, 09 Mar 2021 08:21:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hzN4bO1Q9kWzjd+yXsPtlaqCNoI3movNX0neMlD42A0=; b=i5KJQKN9bkqEcjBqGiOCWwGMjhmab9bcMrL4qpn5XJJJhlqqhHKg1eMmBqWvWg5MMb RsRFl0gFyiMZCRzLZ09L8A5BpaEgUVDEM1E9vQJ1LzJ1WMLIHbex/L0VkPDD15vo/nq9 Sct5eqCgZwzfPOErepBVkVs6+8VwW7oJe1/KV76tl4n5kfYomUEBFDkNT32ChGSYjQdN hq3maIX4uV2QVTU3Us9pjYgYaqBrMt0UVZ2LrvcuQ+rNWseqs5Mz+s8wAiI8o8/DTGbp taQpJ00aLC/ayqbtivTa1JAvUDzcMSW+I3R8adao6YEgyPHdSBUXH/Ezdv7ZISc0HVY7 rwZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hzN4bO1Q9kWzjd+yXsPtlaqCNoI3movNX0neMlD42A0=; b=Nbe9jVlm4UHN1d13uqQF7A+CouQDqiua0Fk+UY27XauOXi4uGtzVDuToXMP+cah3kN r1xOPgKv4saJV8yuESf3fygvTaMO8A5MWZMQY9qdwrnr7T0fIsq2OSgG7t1LIwnuFHUZ npTasGQvVERDmqA/uBxcTS6tZex6KHDjwkA3CyYTXos1HzAr0P3zKN16BP5FzJkQbgCC f3s9hbpUJ8HZQeOQdK6MBg6Jk79ac2+sTB07QTllo6Phz3+dGUQR0YNYN5+gmIdQVLIZ aGIKiOp7sEQeXjPQtK9RVTnCC8xyAKh3/S+4ff7TRS7ga3hNZnVgGDKFYRJ6nf9Ivcbf ahoQ== X-Gm-Message-State: AOAM530FPDCs9QnH4AZjNNzNjnnczLEt7RdcSQkQuQ6qlReDreJxXl1m YcYjRwj4ouUKBWo3SKAzxcf2jcjkBPbVvfgg X-Received: by 2002:aca:d6d3:: with SMTP id n202mr3548714oig.175.1615306871020; Tue, 09 Mar 2021 08:21:11 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:10 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 25/78] target/arm: Implement SVE2 floating-point pairwise Date: Tue, 9 Mar 2021 08:19:48 -0800 Message-Id: <20210309162041.23124-26-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::233; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x233.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Reviewed-by: Richard Henderson Signed-off-by: Richard Henderson --- v2: Load all inputs before writing any output (laurent desnogues) --- target/arm/helper-sve.h | 35 +++++++++++++++++++++++++++++ target/arm/sve.decode | 8 +++++++ target/arm/sve_helper.c | 46 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 25 +++++++++++++++++++++ 4 files changed, 114 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index b302203ce8..a033b5f6b2 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2443,3 +2443,38 @@ DEF_HELPER_FLAGS_3(sve2_uqxtnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 19866ec4c6..9c75ac94c0 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1284,3 +1284,11 @@ UQXTNB 01000101 .. 1 ..... 010 010 ..... ..... @rd_rn_tszimm_shl UQXTNT 01000101 .. 1 ..... 010 011 ..... ..... @rd_rn_tszimm_shl SQXTUNB 01000101 .. 1 ..... 010 100 ..... ..... @rd_rn_tszimm_shl SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl + +## SVE2 floating-point pairwise operations + +FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm +FMAXNMP 01100100 .. 010 10 0 100 ... ..... ..... @rdn_pg_rm +FMINNMP 01100100 .. 010 10 1 100 ... ..... ..... @rdn_pg_rm +FMAXP 01100100 .. 010 11 0 100 ... ..... ..... @rdn_pg_rm +FMINP 01100100 .. 010 11 1 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 115daa50a5..6da734e7b4 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -890,6 +890,52 @@ DO_ZPZZ_PAIR_D(sve2_sminp_zpzz_d, int64_t, DO_MIN) #undef DO_ZPZZ_PAIR #undef DO_ZPZZ_PAIR_D +#define DO_ZPZZ_PAIR_FP(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, \ + void *status, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + TYPE n0 = *(TYPE *)(vn + H(i)); \ + TYPE m0 = *(TYPE *)(vm + H(i)); \ + TYPE n1 = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE m1 = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(n0, n1, status); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(m0, m1, status); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ +} + +DO_ZPZZ_PAIR_FP(sve2_faddp_zpzz_h, float16, H1_2, float16_add) +DO_ZPZZ_PAIR_FP(sve2_faddp_zpzz_s, float32, H1_4, float32_add) +DO_ZPZZ_PAIR_FP(sve2_faddp_zpzz_d, float64, , float64_add) + +DO_ZPZZ_PAIR_FP(sve2_fmaxnmp_zpzz_h, float16, H1_2, float16_maxnum) +DO_ZPZZ_PAIR_FP(sve2_fmaxnmp_zpzz_s, float32, H1_4, float32_maxnum) +DO_ZPZZ_PAIR_FP(sve2_fmaxnmp_zpzz_d, float64, , float64_maxnum) + +DO_ZPZZ_PAIR_FP(sve2_fminnmp_zpzz_h, float16, H1_2, float16_minnum) +DO_ZPZZ_PAIR_FP(sve2_fminnmp_zpzz_s, float32, H1_4, float32_minnum) +DO_ZPZZ_PAIR_FP(sve2_fminnmp_zpzz_d, float64, , float64_minnum) + +DO_ZPZZ_PAIR_FP(sve2_fmaxp_zpzz_h, float16, H1_2, float16_max) +DO_ZPZZ_PAIR_FP(sve2_fmaxp_zpzz_s, float32, H1_4, float32_max) +DO_ZPZZ_PAIR_FP(sve2_fmaxp_zpzz_d, float64, , float64_max) + +DO_ZPZZ_PAIR_FP(sve2_fminp_zpzz_h, float16, H1_2, float16_min) +DO_ZPZZ_PAIR_FP(sve2_fminp_zpzz_s, float32, H1_4, float32_min) +DO_ZPZZ_PAIR_FP(sve2_fminp_zpzz_d, float64, , float64_min) + +#undef DO_ZPZZ_PAIR_FP + /* Three-operand expander, controlled by a predicate, in which the * third operand is "wide". That is, for D = N op M, the same 64-bit * value of M is used with all of the narrower values of N. diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 65f1b11848..ae2ac20302 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6698,3 +6698,28 @@ static bool trans_SQXTUNT(DisasContext *s, arg_rri_esz *a) }; return do_sve2_narrow_extract(s, a, ops); } + +static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, + gen_helper_gvec_4_ptr *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzz_fp(s, a, fn); +} + +#define DO_SVE2_ZPZZ_FP(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ +{ \ + static gen_helper_gvec_4_ptr * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_zpzz_h, \ + gen_helper_sve2_##name##_zpzz_s, gen_helper_sve2_##name##_zpzz_d \ + }; \ + return do_sve2_zpzz_fp(s, a, fns[a->esz]); \ +} + +DO_SVE2_ZPZZ_FP(FADDP, faddp) +DO_SVE2_ZPZZ_FP(FMAXNMP, fmaxnmp) +DO_SVE2_ZPZZ_FP(FMINNMP, fminnmp) +DO_SVE2_ZPZZ_FP(FMAXP, fmaxp) +DO_SVE2_ZPZZ_FP(FMINP, fminp) From patchwork Tue Mar 9 16:19:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396034 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2787973jai; Tue, 9 Mar 2021 09:46:44 -0800 (PST) X-Google-Smtp-Source: ABdhPJzzc3IHPX0LOyjexhsbDw3NbAiZnDaGnavWc8FhVOJbXLLw6rrFlQfI1dFdwFu8xCSlC+vL X-Received: by 2002:a92:c644:: with SMTP id 4mr25535248ill.237.1615312004837; Tue, 09 Mar 2021 09:46:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312004; cv=none; d=google.com; s=arc-20160816; b=FnvYpQ7mfwwfu6K/RR17bIOWr9jpo3Ypds+gOswPV03yzp4vBmU8XJVCPwKdquiq/w a/3vJ0PqjR7ztSLPxSTxTsq+jBgW1zp25wsNhvoMFMakjJK2M6K/j5amhDezwKzP4x2q Zb3IaVbSNYditey0oTr29QxojZIQeYINy7M9eRJ70+NWlPlfK7CDXl3tdptw0LlGm3Rc nrmLsy9VLETYMds2/wJ/FL41QXRjKA9gScZyG7raI86Xp++pzkDvmgEFSQSn5UjLdNpJ qH2n38LBAiOlFe1NngU52N1gIpvg+tSgLWaC5LpT0mQh/6pNYoFnes2v44z0lDEOh+zD Lu1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=qtOtzTyTf51XMjw29ZLW9+FKKnq6ZOqwHnw1rjmdIyU=; b=VlRygoqzOC2f6E21s2WyFVdku070S5KPok5NxxuhA1gzfwK/yup1ada15tDeKQ3Cbl CWjCe6yV76dR/F58hhbilE4BDZ71CKYizyzKwfV3k0NW8ViCTYqhGGbF34/TNYabsBUa jrFzc+3JccZbWkHkEDsKYn2tOLUUtfr97tXjl0SklquUfNoc+bov/8fl8lLZaD1tWzzg gWqDGD13TmTMkazPoP60ZqLdNCJx8B6OtHBgm6SpsEXwe4Yfn7Z0URwwM90c1RQFo7cp mMAEqmvY68FHp9SE9LCn/21g+o0KJvxbCN10yTsKjQvCeLO7xdmutqw9uSzYzfaznFpE WV/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="rHTSSqe/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q4si13922467jat.34.2021.03.09.09.46.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:46:44 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="rHTSSqe/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51904 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgRP-0007Ih-PS for patch@linaro.org; Tue, 09 Mar 2021 12:46:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46016) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf75-0007V6-Et for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:39 -0500 Received: from mail-oo1-xc2f.google.com ([2607:f8b0:4864:20::c2f]:43015) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6f-0002Ug-7L for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:39 -0500 Received: by mail-oo1-xc2f.google.com with SMTP id x19so3163831ooj.10 for ; Tue, 09 Mar 2021 08:21:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qtOtzTyTf51XMjw29ZLW9+FKKnq6ZOqwHnw1rjmdIyU=; b=rHTSSqe/FSzun7Yhs9P4cuWS02equ6rRhkADexFgLOFoQI4BTb3aD9qPBXDsU23CF8 IK+12PMc8ZU8ksRzpOaR0OdxjMr9WKcayFQ5xLYR0tK86niQs6C1z4UasjcmE3tcOZPm Jx4yyFc612LqkjzYrp7RLngN/aNPBnKJc6wMjW8yWyNV4m0q3Pv9mT3B3PdTV8ivKNig roNyo9yCnHSbsdrTnQq1/OzRMYC1l4xImN1vtCc4OWk85Omdz/Z7edGXmDG2fW9NGrvJ VccGDRTGe6bsUsgNrSyxsbd3vGplxEHP/oZDbJF/6zojvH58bAHmI5TZvNg/G7X+5eQl 0Z4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qtOtzTyTf51XMjw29ZLW9+FKKnq6ZOqwHnw1rjmdIyU=; b=n8UzVmtPyUNe9xzHkKv1fGiOcV+eIScQuyh5DuXVEvpnkc+XewfyIw6d1iVdVoJKHG nxQGY8IpOO2VolLa+YCxn+zsYS+ZvebwiBGhpX/hGxcKEvpLqxdGqsBP48Z5psyuAf+S k6Gws6RHvXK2UH3caZM13hbtasNO3y+K373fwXjDWL21uIGiCEVKy5GuK9YC6HmBeDno 4KVdzJcmOBeHLZSaEWRitmHk53mWUssToWuCtPtbd5HoKGWw5tDYb2yxgxwH7eivQyWj 69rySUT+7q8e9XSPAytZhmfoePKdn5yHarLRmN4S8FGgMCxwRpCH4K4OuQ/gHgCnpbPa ytLg== X-Gm-Message-State: AOAM530JNGGIkchUeKdPW6FURsNxwsJtyIkXL4FhXtYI3UHfkykwfL79 UVIeSYKWN9O1u2eb1CyWP5TfQUVHjJM95ojM X-Received: by 2002:a4a:9671:: with SMTP id r46mr7714445ooi.69.1615306872106; Tue, 09 Mar 2021 08:21:12 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:11 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 26/78] target/arm: Implement SVE2 SHRN, RSHRN Date: Tue, 9 Mar 2021 08:19:49 -0800 Message-Id: <20210309162041.23124-27-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c2f; envelope-from=richard.henderson@linaro.org; helo=mail-oo1-xc2f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix typo in gen_shrnb_vec (laurent desnogues) v3: Replace DO_RSHR with an inline function --- target/arm/helper-sve.h | 16 ++++ target/arm/sve.decode | 8 ++ target/arm/sve_helper.c | 54 ++++++++++++- target/arm/translate-sve.c | 160 +++++++++++++++++++++++++++++++++++++ 4 files changed, 236 insertions(+), 2 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index a033b5f6b2..2b2ebea631 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2444,6 +2444,22 @@ DEF_HELPER_FLAGS_3(sve2_sqxtunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_shrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_rshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_rshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 9c75ac94c0..169486ecb2 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1285,6 +1285,14 @@ UQXTNT 01000101 .. 1 ..... 010 011 ..... ..... @rd_rn_tszimm_shl SQXTUNB 01000101 .. 1 ..... 010 100 ..... ..... @rd_rn_tszimm_shl SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl +## SVE2 bitwise shift right narrow + +# Bit 23 == 0 is handled by esz > 0 in the translator. +SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr +SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr +RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr +RSHRNT 01000101 .. 1 ..... 00 0111 ..... ..... @rd_rn_tszimm_shr + ## SVE2 floating-point pairwise operations FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 6da734e7b4..b8e4152f03 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1863,6 +1863,17 @@ void HELPER(NAME)(void *vd, void *vn, void *vg, uint32_t desc) \ when N is negative, add 2**M-1. */ #define DO_ASRD(N, M) ((N + (N < 0 ? ((__typeof(N))1 << M) - 1 : 0)) >> M) +static inline uint64_t do_urshr(uint64_t x, unsigned sh) +{ + if (likely(sh < 64)) { + return (x >> sh) + ((x >> (sh - 1)) & 1); + } else if (sh == 64) { + return x >> 63; + } else { + return 0; + } +} + DO_ZPZI(sve_asr_zpzi_b, int8_t, H1, DO_SHR) DO_ZPZI(sve_asr_zpzi_h, int16_t, H1_2, DO_SHR) DO_ZPZI(sve_asr_zpzi_s, int32_t, H1_4, DO_SHR) @@ -1883,12 +1894,51 @@ DO_ZPZI(sve_asrd_h, int16_t, H1_2, DO_ASRD) DO_ZPZI(sve_asrd_s, int32_t, H1_4, DO_ASRD) DO_ZPZI_D(sve_asrd_d, int64_t, DO_ASRD) -#undef DO_SHR -#undef DO_SHL #undef DO_ASRD #undef DO_ZPZI #undef DO_ZPZI_D +#define DO_SHRNB(NAME, TYPEW, TYPEN, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int shift = simd_data(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + i); \ + *(TYPEW *)(vd + i) = (TYPEN)OP(nn, shift); \ + } \ +} + +#define DO_SHRNT(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int shift = simd_data(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + *(TYPEN *)(vd + HN(i + sizeof(TYPEN))) = OP(nn, shift); \ + } \ +} + +DO_SHRNB(sve2_shrnb_h, uint16_t, uint8_t, DO_SHR) +DO_SHRNB(sve2_shrnb_s, uint32_t, uint16_t, DO_SHR) +DO_SHRNB(sve2_shrnb_d, uint64_t, uint32_t, DO_SHR) + +DO_SHRNT(sve2_shrnt_h, uint16_t, uint8_t, H1_2, H1, DO_SHR) +DO_SHRNT(sve2_shrnt_s, uint32_t, uint16_t, H1_4, H1_2, DO_SHR) +DO_SHRNT(sve2_shrnt_d, uint64_t, uint32_t, , H1_4, DO_SHR) + +DO_SHRNB(sve2_rshrnb_h, uint16_t, uint8_t, do_urshr) +DO_SHRNB(sve2_rshrnb_s, uint32_t, uint16_t, do_urshr) +DO_SHRNB(sve2_rshrnb_d, uint64_t, uint32_t, do_urshr) + +DO_SHRNT(sve2_rshrnt_h, uint16_t, uint8_t, H1_2, H1, do_urshr) +DO_SHRNT(sve2_rshrnt_s, uint32_t, uint16_t, H1_4, H1_2, do_urshr) +DO_SHRNT(sve2_rshrnt_d, uint64_t, uint32_t, , H1_4, do_urshr) + +#undef DO_SHRNB +#undef DO_SHRNT + /* Fully general four-operand expander, controlled by a predicate. */ #define DO_ZPZZZ(NAME, TYPE, H, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ae2ac20302..91e1c4186c 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6699,6 +6699,166 @@ static bool trans_SQXTUNT(DisasContext *s, arg_rri_esz *a) return do_sve2_narrow_extract(s, a, ops); } +static bool do_sve2_shr_narrow(DisasContext *s, arg_rri_esz *a, + const GVecGen2i ops[3]) +{ + if (a->esz < 0 || a->esz > MO_32 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + assert(a->imm > 0 && a->imm <= (8 << a->esz)); + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2i(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vsz, vsz, a->imm, &ops[a->esz]); + } + return true; +} + +static void gen_shrnb_i64(unsigned vece, TCGv_i64 d, TCGv_i64 n, int shr) +{ + int halfbits = 4 << vece; + uint64_t mask = dup_const(vece, MAKE_64BIT_MASK(0, halfbits)); + + tcg_gen_shri_i64(d, n, shr); + tcg_gen_andi_i64(d, d, mask); +} + +static void gen_shrnb16_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnb_i64(MO_16, d, n, shr); +} + +static void gen_shrnb32_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnb_i64(MO_32, d, n, shr); +} + +static void gen_shrnb64_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnb_i64(MO_64, d, n, shr); +} + +static void gen_shrnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + uint64_t mask = MAKE_64BIT_MASK(0, halfbits); + + tcg_gen_shri_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_SHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { INDEX_op_shri_vec, 0 }; + static const GVecGen2i ops[3] = { + { .fni8 = gen_shrnb16_i64, + .fniv = gen_shrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_shrnb_h, + .vece = MO_16 }, + { .fni8 = gen_shrnb32_i64, + .fniv = gen_shrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_shrnb_s, + .vece = MO_32 }, + { .fni8 = gen_shrnb64_i64, + .fniv = gen_shrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_shrnb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_shrnt_i64(unsigned vece, TCGv_i64 d, TCGv_i64 n, int shr) +{ + int halfbits = 4 << vece; + uint64_t mask = dup_const(vece, MAKE_64BIT_MASK(0, halfbits)); + + tcg_gen_shli_i64(n, n, halfbits - shr); + tcg_gen_andi_i64(n, n, ~mask); + tcg_gen_andi_i64(d, d, mask); + tcg_gen_or_i64(d, d, n); +} + +static void gen_shrnt16_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnt_i64(MO_16, d, n, shr); +} + +static void gen_shrnt32_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnt_i64(MO_32, d, n, shr); +} + +static void gen_shrnt64_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + tcg_gen_shri_i64(n, n, shr); + tcg_gen_deposit_i64(d, d, n, 32, 32); +} + +static void gen_shrnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + uint64_t mask = MAKE_64BIT_MASK(0, halfbits); + + tcg_gen_shli_vec(vece, n, n, halfbits - shr); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { INDEX_op_shli_vec, 0 }; + static const GVecGen2i ops[3] = { + { .fni8 = gen_shrnt16_i64, + .fniv = gen_shrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_shrnt_h, + .vece = MO_16 }, + { .fni8 = gen_shrnt32_i64, + .fniv = gen_shrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_shrnt_s, + .vece = MO_32 }, + { .fni8 = gen_shrnt64_i64, + .fniv = gen_shrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_shrnt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_RSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_rshrnb_h }, + { .fno = gen_helper_sve2_rshrnb_s }, + { .fno = gen_helper_sve2_rshrnb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_RSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_rshrnt_h }, + { .fno = gen_helper_sve2_rshrnt_s }, + { .fno = gen_helper_sve2_rshrnt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Tue Mar 9 16:19:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396040 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2792487jai; Tue, 9 Mar 2021 09:53:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJwuDsMvGA+mwlZ0ls8GZAbUdCXcoyep3ok+xe8xVkYrRqvefQ6aQjCz+OjC6nhNjf1LklOZ X-Received: by 2002:a25:8006:: with SMTP id m6mr41715251ybk.336.1615312387117; Tue, 09 Mar 2021 09:53:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312387; cv=none; d=google.com; s=arc-20160816; b=k/IcPufFr8A5+v0TXMoEtvuooXcyIOwyqFbZGztKlfYsjooMFj+wnYJg4KAhQHLsIH BPkdGjRBf9cicJScYTxUnGC7pZZ5+bakJIuAIMqWEBj9cyNv/BxtCsy9J/oABp6qhKN1 7pIApiB/nVbXVBq6gMu/n+pTVaHqkwl8b+h4uVkm7NC+vfP63QX9tBZJ3jByi9lAtebh HjvzJMmJHdWx3oanzqg0rjZ7biPAfIRmxUj0S3A81vVY6hlZTaCpW4ISrXQul1mOaGYQ sfwTIK6dtW89OCwQuzNCRtInt4Z50WGl2fGJhwJ2vrdWeoYjwV73trsUoKawS8329IWX oU5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=b3LCEzIegBZA/KKLIZt6ZumkwDD1iAx7cVD9vUey8jQ=; b=1IPLK/gi4LSwTHWC8o6f/7WGxn+96S82zWpZO2/4VHDAlLFo1vkP5v7EUOYb1KKJnV K5eQcQdyPm0OAzYSxxfr7VphrSDHatAQtnDrd4wa7nRAs+YZHDZgl72219EZbC5SrCPa 6q/Uxb2xqHJ/ynL2qVftgCx65FfCQ7wX13w/uYJfazIXTTn134lfWUxcHEbPcT1+5Avy 9X8THNHXYPMW1n+DDxBflzfCCCEBZLCrOg9QrujfQ8BIBqMSQjrU5zpCrqHKFyGEzbvH +7WDwWO2EdVorGvRYmEdMVsHGlH6euZpNoI/N4NneWBKYtqZInSUPr0H9iRaHL+G/jRr Clrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=y2VmB+Y1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x7si15961254ybo.123.2021.03.09.09.53.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:53:07 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=y2VmB+Y1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41524 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgXa-0006dP-G1 for patch@linaro.org; Tue, 09 Mar 2021 12:53:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46004) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf73-0007Ui-RF for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:39 -0500 Received: from mail-oi1-x230.google.com ([2607:f8b0:4864:20::230]:38115) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6g-0002V2-9F for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:37 -0500 Received: by mail-oi1-x230.google.com with SMTP id v192so7957395oia.5 for ; Tue, 09 Mar 2021 08:21:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=b3LCEzIegBZA/KKLIZt6ZumkwDD1iAx7cVD9vUey8jQ=; b=y2VmB+Y1CR44IU10Gs6jdsfkzEtbpKSMPI4eaoKnNltVUTlLFVDBIJD1pH3nzO/PaM 3XjwGNhwT0l+L3rdrtKPBHB0cWo429vA+3qdqkGOxj5VN057cgfzAtcEdCcWBP2v2chv v3d2KdUMcATSnGi6ApQVF9DuPC80Hek/V0CtJyN1qKMY14XTaOnDPHe7TUyei01SJJPN isuk/tdsfBdZVyDUkt+QVzY543Er7lH7175NBz7hQGGIPgXGrxClJnloJ4vXdXTyosL3 cySRkTQJG077HRvQGZzqUe03XTAEDrwAuAPqk2q7XoD9KfVhwUzdW8BB7zJznQ31ku8F 3tvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b3LCEzIegBZA/KKLIZt6ZumkwDD1iAx7cVD9vUey8jQ=; b=pmkjSyySdo5+N4Lzo1+FlLNXpHWIQFbnBJezqrPjnJhRil6hqqrcPxO8ZK5+8kPZvo J5GMP9Gf8lOKMmF/9LizuNe7HfppZ3+iPXo3IpD8VK9rAvtEELJPsL9lgHFg2wI7W73j mhmt9l42QmsofC/QOHAdj7d9S1u8wgQc/ob0ApPFA/ZtNyyC0yU0NPnxkeeqfHS7Rqq9 lQOMV5+UhRc0JmWtq9fIrfWkD7p1cZkkUEycGmihAFkj8SW+XXITTvWrAHZ7yFcxQsGp Xu3TnsgbYGRjfMZA3OUNg9zmbloeER2nTgIzZp+tcm3BTgvs0xf+DK2MwnjcK3G6HtaQ wiOA== X-Gm-Message-State: AOAM530hdudG1BVlV0ph9UZw2OT8aGbaL72X7whkQz9CazwQ6uA8V0Hg 0iXmXoX6/O4xCl0kd2Tx3fZnGxBeqMyXcV4c X-Received: by 2002:aca:b787:: with SMTP id h129mr2392434oif.58.1615306873237; Tue, 09 Mar 2021 08:21:13 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:12 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 27/78] target/arm: Implement SVE2 SQSHRUN, SQRSHRUN Date: Tue, 9 Mar 2021 08:19:50 -0800 Message-Id: <20210309162041.23124-28-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::230; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x230.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 16 +++++++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 35 ++++++++++++++ target/arm/translate-sve.c | 98 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 153 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 2b2ebea631..2e80d9d27b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2460,6 +2460,22 @@ DEF_HELPER_FLAGS_3(sve2_rshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_rshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_rshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrunb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 169486ecb2..18faa900ca 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1288,6 +1288,10 @@ SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl ## SVE2 bitwise shift right narrow # Bit 23 == 0 is handled by esz > 0 in the translator. +SQSHRUNB 01000101 .. 1 ..... 00 0000 ..... ..... @rd_rn_tszimm_shr +SQSHRUNT 01000101 .. 1 ..... 00 0001 ..... ..... @rd_rn_tszimm_shr +SQRSHRUNB 01000101 .. 1 ..... 00 0010 ..... ..... @rd_rn_tszimm_shr +SQRSHRUNT 01000101 .. 1 ..... 00 0011 ..... ..... @rd_rn_tszimm_shr SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index b8e4152f03..96a4c76e94 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1874,6 +1874,16 @@ static inline uint64_t do_urshr(uint64_t x, unsigned sh) } } +static inline int64_t do_srshr(int64_t x, unsigned sh) +{ + if (likely(sh < 64)) { + return (x >> sh) + ((x >> (sh - 1)) & 1); + } else { + /* Rounding the sign bit always produces 0. */ + return 0; + } +} + DO_ZPZI(sve_asr_zpzi_b, int8_t, H1, DO_SHR) DO_ZPZI(sve_asr_zpzi_h, int16_t, H1_2, DO_SHR) DO_ZPZI(sve_asr_zpzi_s, int32_t, H1_4, DO_SHR) @@ -1936,6 +1946,31 @@ DO_SHRNT(sve2_rshrnt_h, uint16_t, uint8_t, H1_2, H1, do_urshr) DO_SHRNT(sve2_rshrnt_s, uint32_t, uint16_t, H1_4, H1_2, do_urshr) DO_SHRNT(sve2_rshrnt_d, uint64_t, uint32_t, , H1_4, do_urshr) +#define DO_SQSHRUN_H(x, sh) do_sat_bhs((int64_t)(x) >> sh, 0, UINT8_MAX) +#define DO_SQSHRUN_S(x, sh) do_sat_bhs((int64_t)(x) >> sh, 0, UINT16_MAX) +#define DO_SQSHRUN_D(x, sh) \ + do_sat_bhs((int64_t)(x) >> (sh < 64 ? sh : 63), 0, UINT32_MAX) + +DO_SHRNB(sve2_sqshrunb_h, int16_t, uint8_t, DO_SQSHRUN_H) +DO_SHRNB(sve2_sqshrunb_s, int32_t, uint16_t, DO_SQSHRUN_S) +DO_SHRNB(sve2_sqshrunb_d, int64_t, uint32_t, DO_SQSHRUN_D) + +DO_SHRNT(sve2_sqshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQSHRUN_H) +DO_SHRNT(sve2_sqshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQSHRUN_S) +DO_SHRNT(sve2_sqshrunt_d, int64_t, uint32_t, , H1_4, DO_SQSHRUN_D) + +#define DO_SQRSHRUN_H(x, sh) do_sat_bhs(do_srshr(x, sh), 0, UINT8_MAX) +#define DO_SQRSHRUN_S(x, sh) do_sat_bhs(do_srshr(x, sh), 0, UINT16_MAX) +#define DO_SQRSHRUN_D(x, sh) do_sat_bhs(do_srshr(x, sh), 0, UINT32_MAX) + +DO_SHRNB(sve2_sqrshrunb_h, int16_t, uint8_t, DO_SQRSHRUN_H) +DO_SHRNB(sve2_sqrshrunb_s, int32_t, uint16_t, DO_SQRSHRUN_S) +DO_SHRNB(sve2_sqrshrunb_d, int64_t, uint32_t, DO_SQRSHRUN_D) + +DO_SHRNT(sve2_sqrshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRUN_H) +DO_SHRNT(sve2_sqrshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRUN_S) +DO_SHRNT(sve2_sqrshrunt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRUN_D) + #undef DO_SHRNB #undef DO_SHRNT diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 91e1c4186c..fe247d758c 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6859,6 +6859,104 @@ static bool trans_RSHRNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static void gen_sqshrunb_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRUNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_sari_vec, INDEX_op_smax_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrunb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrunb_h, + .vece = MO_16 }, + { .fniv = gen_sqshrunb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrunb_s, + .vece = MO_32 }, + { .fniv = gen_sqshrunb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrunb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_sqshrunt_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRUNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shli_vec, INDEX_op_sari_vec, + INDEX_op_smax_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrunt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrunt_h, + .vece = MO_16 }, + { .fniv = gen_sqshrunt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrunt_s, + .vece = MO_32 }, + { .fniv = gen_sqshrunt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrunt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRUNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrunb_h }, + { .fno = gen_helper_sve2_sqrshrunb_s }, + { .fno = gen_helper_sve2_sqrshrunb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRUNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrunt_h }, + { .fno = gen_helper_sve2_sqrshrunt_s }, + { .fno = gen_helper_sve2_sqrshrunt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Tue Mar 9 16:19:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396037 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2789868jai; Tue, 9 Mar 2021 09:49:16 -0800 (PST) X-Google-Smtp-Source: ABdhPJwTBjctmv6Y9RIdCrPlGFO72WNHkYJcPld68ObaMRLw9LQJQvSJY9yhgV/QL/Eiz/l8omaC X-Received: by 2002:aed:3104:: with SMTP id 4mr26354474qtg.341.1615312156178; Tue, 09 Mar 2021 09:49:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312156; cv=none; d=google.com; s=arc-20160816; b=YmTJFIKMkW2o0GCztJSqi5fiI+sTVGW/oAGeTLMnZnwa5SaLNDEoooUN7fkPP5HKiV 0I3b2nHe4bk+DKaXZHqoA05nF57b50zTHaX5CTKRug6OPTd748YFlFBw7wA9WbY1DjaM bf62OZwSIQjWgRj6WynFeu5krPde8uIrurq+OvQaI3UPnldh92GQqveX5AxWkpgO+2d6 AhfVxi8p7Xb9XJaSy/owsFSd6jAv2BgtRGzCK8YtamyiIK2GxsqQKx6cEPMtEP6CSZvJ 6beqCDs09ipa3lwQtSWso5w/LMQvSA+Y94Lx+vREKYhitaptuM2ofumvbbrDUtpVtOqO RZlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=hoz+RRUt4NVeZFe30oZNFywcAnPrk+J9cCbokJUYKMc=; b=LFREBwz/w0jEP1i3fblekbpgSsEZNHFweQpKD1S+3TuvuxsZWmHCeJwm9J8GHsgc/+ DyAIbFyWjhh542GqrTjG7Bn2238X/lverLcYiFkOyE9kqGAbWqjdmflxKwoc5ASDfzWQ 2WYkNVoc5l4A0uG+eBCD6D8YOteTPaKg/djE3fswQG3Dl9o/sUsqNpeWf2p/UsijSdWI iwiQT8blo3mji8RfgOmCWB0IWe0i+eaGYK9RPdhu1vO/pXPBEpNavJDJ5uCPzpQVT/m8 y2zir8FbNf7W2gkE08aj0QTXqiHnGfIgJ5zRBC37k7ZXCe3uBliS7+WI0M9GnGf/7wN/ XlbQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qP1GpV2X; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z128si5462274qkc.181.2021.03.09.09.49.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:49:16 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qP1GpV2X; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58264 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgTr-0001px-Fz for patch@linaro.org; Tue, 09 Mar 2021 12:49:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46064) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf77-0007XB-WF for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:42 -0500 Received: from mail-oi1-x232.google.com ([2607:f8b0:4864:20::232]:43943) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6h-0002VI-Dy for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:41 -0500 Received: by mail-oi1-x232.google.com with SMTP id d20so15508362oiw.10 for ; Tue, 09 Mar 2021 08:21:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hoz+RRUt4NVeZFe30oZNFywcAnPrk+J9cCbokJUYKMc=; b=qP1GpV2XDwLC0d8IoBEa4QpmQ/N3MQT33eWqhbRmFgA3um8l2es++97TTpfN8FLS8B a54fgtSE8SNfp4rM+Kmd/6MgFC4kjIQOVeLLbgoubkbkUqn5I9PA5ZnH5jVoNOP1nhq9 2EExRAcI935AxYyx8UDQ+1yFfnVPJnwuzCaBqiwkzQlDVXMNtzYrdJnqnNkPfxVHCObD y9vcugyg3QBlqRdgSju6lBYta1Hp3WJqvdJciuOC0ww80tgEBiB/AGhEYa39iAdKDoQq eyb2s39DGJwPqavwjrU+3H61upAeVbrBKZDTWK2whO5mZQYjU/5FoZ0DK45eS49sIJRY VnGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hoz+RRUt4NVeZFe30oZNFywcAnPrk+J9cCbokJUYKMc=; b=EKJmwJ9+grjItvudWprr2vXY606zZBEnGazOJt57KMwdYuczy+oK8G9Uvp5rb3lnbV bebkWdTgnDQcAads0tWKrgOYu7ZD/agbpSJx6YSmGp87ALwAeVqJiDNGNx6rv56zqaaC 2TSauY7E6eMH8lYyj2eZQx83ARID4haOeVkBLYbWlasHPq12Hy9BXZ1NsB29LwN0ZPGZ DC1DImBsvfSN6AGaLi+sQGG/eBI4jO1CWEkl4/Xg00P9nkmZvO5p1BC4ff95InNCSdgc S18bAJZuTRmdjadmg84U3jhMEYgIHbcyTzNoB3EsV644GhjGnpwnKAszAW5K6LZAHhWQ hwaw== X-Gm-Message-State: AOAM532+ahmQMuz0ZcRh5FvkZsakeBjnR9jmYDk5kdZeS1gXlBKQsWmZ 9hiXPg9YdlXKyEwUJMjwHm1qgWVOIo/ZYN6k X-Received: by 2002:aca:ed10:: with SMTP id l16mr3513793oih.132.1615306874400; Tue, 09 Mar 2021 08:21:14 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:14 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 28/78] target/arm: Implement SVE2 UQSHRN, UQRSHRN Date: Tue, 9 Mar 2021 08:19:51 -0800 Message-Id: <20210309162041.23124-29-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::232; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x232.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 16 +++++++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 24 ++++++++++ target/arm/translate-sve.c | 93 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 137 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 2e80d9d27b..ba6a24fc8b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2476,6 +2476,22 @@ DEF_HELPER_FLAGS_3(sve2_sqrshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqrshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 18faa900ca..13b5da0856 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1296,6 +1296,10 @@ SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr RSHRNT 01000101 .. 1 ..... 00 0111 ..... ..... @rd_rn_tszimm_shr +UQSHRNB 01000101 .. 1 ..... 00 1100 ..... ..... @rd_rn_tszimm_shr +UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr +UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr +UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr ## SVE2 floating-point pairwise operations diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 96a4c76e94..83d3547f67 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1971,6 +1971,30 @@ DO_SHRNT(sve2_sqrshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRUN_H) DO_SHRNT(sve2_sqrshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRUN_S) DO_SHRNT(sve2_sqrshrunt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRUN_D) +#define DO_UQSHRN_H(x, sh) MIN(x >> sh, UINT8_MAX) +#define DO_UQSHRN_S(x, sh) MIN(x >> sh, UINT16_MAX) +#define DO_UQSHRN_D(x, sh) MIN(x >> sh, UINT32_MAX) + +DO_SHRNB(sve2_uqshrnb_h, uint16_t, uint8_t, DO_UQSHRN_H) +DO_SHRNB(sve2_uqshrnb_s, uint32_t, uint16_t, DO_UQSHRN_S) +DO_SHRNB(sve2_uqshrnb_d, uint64_t, uint32_t, DO_UQSHRN_D) + +DO_SHRNT(sve2_uqshrnt_h, uint16_t, uint8_t, H1_2, H1, DO_UQSHRN_H) +DO_SHRNT(sve2_uqshrnt_s, uint32_t, uint16_t, H1_4, H1_2, DO_UQSHRN_S) +DO_SHRNT(sve2_uqshrnt_d, uint64_t, uint32_t, , H1_4, DO_UQSHRN_D) + +#define DO_UQRSHRN_H(x, sh) MIN(do_urshr(x, sh), UINT8_MAX) +#define DO_UQRSHRN_S(x, sh) MIN(do_urshr(x, sh), UINT16_MAX) +#define DO_UQRSHRN_D(x, sh) MIN(do_urshr(x, sh), UINT32_MAX) + +DO_SHRNB(sve2_uqrshrnb_h, uint16_t, uint8_t, DO_UQRSHRN_H) +DO_SHRNB(sve2_uqrshrnb_s, uint32_t, uint16_t, DO_UQRSHRN_S) +DO_SHRNB(sve2_uqrshrnb_d, uint64_t, uint32_t, DO_UQRSHRN_D) + +DO_SHRNT(sve2_uqrshrnt_h, uint16_t, uint8_t, H1_2, H1, DO_UQRSHRN_H) +DO_SHRNT(sve2_uqrshrnt_s, uint32_t, uint16_t, H1_4, H1_2, DO_UQRSHRN_S) +DO_SHRNT(sve2_uqrshrnt_d, uint64_t, uint32_t, , H1_4, DO_UQRSHRN_D) + #undef DO_SHRNB #undef DO_SHRNT diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index fe247d758c..6a3c34a4fc 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6957,6 +6957,99 @@ static bool trans_SQRSHRUNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static void gen_uqshrnb_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_shri_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_UQSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shri_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_uqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_uqshrnb_h, + .vece = MO_16 }, + { .fniv = gen_uqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_uqshrnb_s, + .vece = MO_32 }, + { .fniv = gen_uqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_uqshrnb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_uqshrnt_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_shri_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_UQSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shli_vec, INDEX_op_shri_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_uqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_uqshrnt_h, + .vece = MO_16 }, + { .fniv = gen_uqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_uqshrnt_s, + .vece = MO_32 }, + { .fniv = gen_uqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_uqshrnt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_UQRSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_uqrshrnb_h }, + { .fno = gen_helper_sve2_uqrshrnb_s }, + { .fno = gen_helper_sve2_uqrshrnb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_uqrshrnt_h }, + { .fno = gen_helper_sve2_uqrshrnt_s }, + { .fno = gen_helper_sve2_uqrshrnt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Tue Mar 9 16:19:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396049 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2800585jai; Tue, 9 Mar 2021 10:03:43 -0800 (PST) X-Google-Smtp-Source: ABdhPJzM442/mwW6JVROBcsbayQGzo/q2KLs/w1HdDveWkEgByRO46j725EPgUKCmAVLoIw5wI+K X-Received: by 2002:a67:61c7:: with SMTP id v190mr16467846vsb.47.1615313022959; Tue, 09 Mar 2021 10:03:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313022; cv=none; d=google.com; s=arc-20160816; b=LzbUMyot7EBp9qFn/dr12zxhbWW/UimK2MeoQgkiGOGsWmi3Ai+GTLJDje6axRaq7t ewDLH+iDjOXe7soxdfSuLnbv3lYZqbg5dFNh6vEW83HSormDWH1HtAfZFIcj6MTL6OOn bPsINoi+oveMVcKR18f28Rh6rGklFulyJQGtAM69ICWiFIwuVnZ2CXB7eMNGdAnUV1hq MATMPoRlQOFHzLxE2cL72NOC5JmwaFOABnn5ILKk+2kzhns6fBHAG9aA/zp3tBrO38kU YxiDwZX888ecdJnHPeLVCwQ5+9Wg/mJnzgQnVxn9nvMGPV/E4Xb1kBOWXahmQLA5MnLa 2gGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=tqdKhCk5c5yScfkTSz3TJ7AtRc7gWT1n4eCkVBF5BsQ=; b=SRsGKNJQhkyajuPIp1JWDkiAUjeCVy/7ov6HbOS2Kqas2Bz1JKZCq26FUSyimhgypE CMHS+hIgFPg5m+zr245firYKv6llL9S4lETc+3fLevxV+gojAVJkjleKTULNpN4jdfib UsxPwrZtzKqHYK179Rpkxavr8JNSEQSyGBiky7VM6d8K0z/mS9J+p0yvQaGLyeI9Pk/3 y6bfG7Mc4KDPfbz9/ICacOW5ZJ0VMADpec2Kbdg9Ag/Ty6vvrTJynDYlHnmapmjZ3cvU OkX1KC2kaHuU8xmgEQbYYjhuNQn7d0lCePAm2iOwakfA/jUxG+7S+vh+27bCfHiPNhNo y9CQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HR2bZZ3t; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q23si2583850vsk.52.2021.03.09.10.03.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:03:42 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HR2bZZ3t; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60128 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJghp-0006Gn-W6 for patch@linaro.org; Tue, 09 Mar 2021 13:03:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46128) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7A-0007dh-E8 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:44 -0500 Received: from mail-oi1-x234.google.com ([2607:f8b0:4864:20::234]:39505) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6i-0002Vh-W5 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:44 -0500 Received: by mail-oi1-x234.google.com with SMTP id z126so15524737oiz.6 for ; Tue, 09 Mar 2021 08:21:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tqdKhCk5c5yScfkTSz3TJ7AtRc7gWT1n4eCkVBF5BsQ=; b=HR2bZZ3tdv/TymFNmL57UxfwRoqMdldC6FyJmrE008FG9TLdWBl80a3c/r6aULJhsz JxKnxtXt9bn8Lnwfe/EoQ0vOYpQYvvaCHAhVVz6ZPm+kUiW2Eg8TujW8dlube7r9lDqX vP3zmSZGj091NHLz6ySbAvsKz7a9EbQj4c2uFBiGaCwLD3oB+ZzBkuzBK7Hv0HWl1hEN LedhbVtBOd8cX1b6UzBmDpon2WjTZ9LaOLhNW12cLzXWnbZyBURVL+JZ+k727nyd8jYy KpWNj0ah94PnXotm8JauvDbMAf9CS+ld5rYbzWfhjOPFqANm+leGpr/RfL3D0Jlu77G/ gceQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tqdKhCk5c5yScfkTSz3TJ7AtRc7gWT1n4eCkVBF5BsQ=; b=tFsDYlsKwMdNZlTG4eW9MWEr/67uVhOKrDWIA4i1WO9Q/wrKzLMK0EEXWFfuQve5Ln Blenor9R//TL9qlIlLiqPR+4L6rb0GopzxTmEWq+Jv5ojdLDpsRQeeePis69fFFw3LqP BNsGZX0fiJdO5sjeCYLQ9AIOa2ztax15dmX6jOueLAZIq6HlaJWJeMPW7+GwmMK4E2+o WIZBI23PlJVoVJxLhVx+y0+wWlNL6P+hP2cM0E4hhPQ1Dx+P2winl+ExTFuSYKfSkajn ScZTJhLc2yLrhd11lLBuJlT8dx6wIXwuVCkbXCD02LGo2lfwWByxE228ExuQnoEZAKvF jL2w== X-Gm-Message-State: AOAM533NNTV0cNxyRivU44LvkeuROeacCjPVNJBO3B4/klP+0xwrUANS jeC1lQPkAEp6+e4XFoNrILL5uWH+63F+xWt1 X-Received: by 2002:aca:5cd6:: with SMTP id q205mr3487126oib.94.1615306875831; Tue, 09 Mar 2021 08:21:15 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:15 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 29/78] target/arm: Implement SVE2 SQSHRN, SQRSHRN Date: Tue, 9 Mar 2021 08:19:52 -0800 Message-Id: <20210309162041.23124-30-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::234; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x234.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This completes the section "SVE2 bitwise shift right narrow". Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 16 ++++++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 24 +++++++++ target/arm/translate-sve.c | 105 +++++++++++++++++++++++++++++++++++++ 4 files changed, 149 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index ba6a24fc8b..1c7fe8e417 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2476,6 +2476,22 @@ DEF_HELPER_FLAGS_3(sve2_sqrshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(sve2_uqshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 13b5da0856..0674464695 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1296,6 +1296,10 @@ SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr RSHRNT 01000101 .. 1 ..... 00 0111 ..... ..... @rd_rn_tszimm_shr +SQSHRNB 01000101 .. 1 ..... 00 1000 ..... ..... @rd_rn_tszimm_shr +SQSHRNT 01000101 .. 1 ..... 00 1001 ..... ..... @rd_rn_tszimm_shr +SQRSHRNB 01000101 .. 1 ..... 00 1010 ..... ..... @rd_rn_tszimm_shr +SQRSHRNT 01000101 .. 1 ..... 00 1011 ..... ..... @rd_rn_tszimm_shr UQSHRNB 01000101 .. 1 ..... 00 1100 ..... ..... @rd_rn_tszimm_shr UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 83d3547f67..4b487d6f5f 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1971,6 +1971,30 @@ DO_SHRNT(sve2_sqrshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRUN_H) DO_SHRNT(sve2_sqrshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRUN_S) DO_SHRNT(sve2_sqrshrunt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRUN_D) +#define DO_SQSHRN_H(x, sh) do_sat_bhs(x >> sh, INT8_MIN, INT8_MAX) +#define DO_SQSHRN_S(x, sh) do_sat_bhs(x >> sh, INT16_MIN, INT16_MAX) +#define DO_SQSHRN_D(x, sh) do_sat_bhs(x >> sh, INT32_MIN, INT32_MAX) + +DO_SHRNB(sve2_sqshrnb_h, int16_t, uint8_t, DO_SQSHRN_H) +DO_SHRNB(sve2_sqshrnb_s, int32_t, uint16_t, DO_SQSHRN_S) +DO_SHRNB(sve2_sqshrnb_d, int64_t, uint32_t, DO_SQSHRN_D) + +DO_SHRNT(sve2_sqshrnt_h, int16_t, uint8_t, H1_2, H1, DO_SQSHRN_H) +DO_SHRNT(sve2_sqshrnt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQSHRN_S) +DO_SHRNT(sve2_sqshrnt_d, int64_t, uint32_t, , H1_4, DO_SQSHRN_D) + +#define DO_SQRSHRN_H(x, sh) do_sat_bhs(do_srshr(x, sh), INT8_MIN, INT8_MAX) +#define DO_SQRSHRN_S(x, sh) do_sat_bhs(do_srshr(x, sh), INT16_MIN, INT16_MAX) +#define DO_SQRSHRN_D(x, sh) do_sat_bhs(do_srshr(x, sh), INT32_MIN, INT32_MAX) + +DO_SHRNB(sve2_sqrshrnb_h, int16_t, uint8_t, DO_SQRSHRN_H) +DO_SHRNB(sve2_sqrshrnb_s, int32_t, uint16_t, DO_SQRSHRN_S) +DO_SHRNB(sve2_sqrshrnb_d, int64_t, uint32_t, DO_SQRSHRN_D) + +DO_SHRNT(sve2_sqrshrnt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRN_H) +DO_SHRNT(sve2_sqrshrnt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRN_S) +DO_SHRNT(sve2_sqrshrnt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRN_D) + #define DO_UQSHRN_H(x, sh) MIN(x >> sh, UINT8_MAX) #define DO_UQSHRN_S(x, sh) MIN(x >> sh, UINT16_MAX) #define DO_UQSHRN_D(x, sh) MIN(x >> sh, UINT32_MAX) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6a3c34a4fc..5380ed26c1 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6957,6 +6957,111 @@ static bool trans_SQRSHRUNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static void gen_sqshrnb_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = MAKE_64BIT_MASK(0, halfbits - 1); + int64_t min = -max - 1; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_sari_vec, INDEX_op_smax_vec, INDEX_op_smin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrnb_h, + .vece = MO_16 }, + { .fniv = gen_sqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrnb_s, + .vece = MO_32 }, + { .fniv = gen_sqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrnb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_sqshrnt_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = MAKE_64BIT_MASK(0, halfbits - 1); + int64_t min = -max - 1; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shli_vec, INDEX_op_sari_vec, + INDEX_op_smax_vec, INDEX_op_smin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrnt_h, + .vece = MO_16 }, + { .fniv = gen_sqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrnt_s, + .vece = MO_32 }, + { .fniv = gen_sqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrnt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrnb_h }, + { .fno = gen_helper_sve2_sqrshrnb_s }, + { .fno = gen_helper_sve2_sqrshrnb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrnt_h }, + { .fno = gen_helper_sve2_sqrshrnt_s }, + { .fno = gen_helper_sve2_sqrshrnt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static void gen_uqshrnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr) { From patchwork Tue Mar 9 16:19:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396041 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2792684jai; Tue, 9 Mar 2021 09:53:23 -0800 (PST) X-Google-Smtp-Source: ABdhPJxdDBX26RK5HXmFRBH9Or2ymyAvNF1p20EBD0rC+iaWQdoHVKfkGJWNJ7yBhNZwzIeQPjhQ X-Received: by 2002:ac8:5e4d:: with SMTP id i13mr25846595qtx.68.1615312403400; Tue, 09 Mar 2021 09:53:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312403; cv=none; d=google.com; s=arc-20160816; b=gYPaGJaAtE4e6qKlbwcrpQRav5K6OQOvPSF1Seb0RQ9u71UFuw4+O416z9eoIzN9OY AXhpFQsrcwQkmac/DRW7dCrVCKzkDo4ETFuWg5HU9mssn2SuqPAUbY7+fLEIK6dbInak Vu6aYbozptJHt4EcDfP9N6edIKkXbJ/uF2eWXWZLV7JQ9mqe31SWwdIRqyHULTCi1vx+ MSti/xVIc8eLY6IhNA81FoS12xCqct1IWB8upFti5iwZp3F/eL6ikCJwJp9I3FjBzPbJ JfBaVdjGlMRrheNYx8icp6vroQ6EAwgJejN2jkOmaObtKkw5RVv8wq8eEi+dAcd7sYdo vepQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=X5qNtlKyR3HiK96NPS7iXZ0SUIEgp+QlNj8LO8hQyAM=; b=jb2EJFUisLiQmkrzjxh8N/CR+AGUEBFjI4a+aSh18V4kpGmX+Xhnribyujy4YWl3+c edHEQhjZJ4FBXG3hOGxl/pgt8opsZrnJ6uC5sA2jCJE2fTSlQEW0yQ3tT/UN7zXgZthu Fi4jeEyK43fWK0xxksjw9EBHWDWji3LJAZzsDmczwhmx/v8/VmrdybdPwFM/lLCkipaU P81emPiHWjxCxJT1ae86/jFNaeukdk+dzSQZUD1mAhmVe2CFzLyYQwfVSD3xSvniDbWI S+vLQFHkwZRKVjEFYzFVX1hSgUCM9BnbXiDARP5Wea43Gu0TR9BqeZs6fGa/BSylDC7D 98Vw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=N+YLCFKu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l16si8956879qkj.55.2021.03.09.09.53.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:53:23 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=N+YLCFKu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:39852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgXq-0005yS-OC for patch@linaro.org; Tue, 09 Mar 2021 12:53:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46126) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7A-0007db-Eg for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:44 -0500 Received: from mail-ot1-x32a.google.com ([2607:f8b0:4864:20::32a]:46156) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6l-0002W5-Fq for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:44 -0500 Received: by mail-ot1-x32a.google.com with SMTP id r24so6270719otq.13 for ; Tue, 09 Mar 2021 08:21:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=X5qNtlKyR3HiK96NPS7iXZ0SUIEgp+QlNj8LO8hQyAM=; b=N+YLCFKuHpHa80yDhQmA/0pb8QkwdYp1OMdY5f/Tiia43E513T517vthrv9htKFlrn GLVqAIDJ7O/WyIDcTb9eesnxX8Ou0wjjrBJ8LxxVn7wCnOtI+I6E543JnLDmBowl7HYk q0iXmgT6fYryCkgxKZ2ojymvRJX40HBKc5r0pExTr3ZyZug2ef2zUM9p2aPzuHWM6EWC AgVKkpYX7ueP0NFeuqCzrEZLXm4dRVscHvzkTJaygtOg0FRId+d1F8Yp/9FBcatk5L6B s4F0tVgPswCIkq0++LB5nMosSmLhJvdeH4LiJcc6ksdAo36sNIV1P0xRXw6272V3ixt6 wWMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=X5qNtlKyR3HiK96NPS7iXZ0SUIEgp+QlNj8LO8hQyAM=; b=auUu1Vej1n9guqIQ+SCkpA76eoWpWcDwE66c0rlwjWwSY0ya8lacv3nlkBlKIFTIVF kAudfBmBOR/2FF5xJA/Chs7BJaa3hQVAkaMyB8eFpgLyobNJszMsHml4DjhmQ0N3lb06 GU2L7h/0OChaBbPWwjR7dzDVy+x3BpP3RUh6yymOG850YNdUFNhoou0cWVCJj04vd2Zd UmbryTXZkFj/RMonYIrTfc0C7p/V49zjHGbDzS4f4rzKI6biXaJQzxSPSVZnOJI2O3y3 Xx5v2ENwL/9kssrEm4aSVRzP6AA0la5fYimklKdLt5/4OVvIijcmXreuBvz61AUKf6+G R7KA== X-Gm-Message-State: AOAM533oLPFOKUwbIemVE6V2FfdgQLBw5935vEMBzePYyvHwklMF4SeS 8hRMmz1Rp465sjRGjuOobjwVmZULLufW7Ym0 X-Received: by 2002:a05:6830:8f:: with SMTP id a15mr2285756oto.299.1615306877007; Tue, 09 Mar 2021 08:21:17 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:16 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 30/78] target/arm: Implement SVE2 WHILEGT, WHILEGE, WHILEHI, WHILEHS Date: Tue, 9 Mar 2021 08:19:53 -0800 Message-Id: <20210309162041.23124-31-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32a; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Rename the existing sve_while (less-than) helper to sve_whilel to make room for a new sve_whileg helper for greater-than. Signed-off-by: Richard Henderson --- v2: Use a new helper function to implement this. v4: Update for PREDDESC. --- target/arm/helper-sve.h | 3 +- target/arm/sve.decode | 2 +- target/arm/sve_helper.c | 38 +++++++++++++++++++++++++- target/arm/translate-sve.c | 56 ++++++++++++++++++++++++++++---------- 4 files changed, 82 insertions(+), 17 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 1c7fe8e417..5bf9fdc7a3 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -913,7 +913,8 @@ DEF_HELPER_FLAGS_4(sve_brkns, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_cntp, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) -DEF_HELPER_FLAGS_3(sve_while, TCG_CALL_NO_RWG, i32, ptr, i32, i32) +DEF_HELPER_FLAGS_3(sve_whilel, TCG_CALL_NO_RWG, i32, ptr, i32, i32) +DEF_HELPER_FLAGS_3(sve_whileg, TCG_CALL_NO_RWG, i32, ptr, i32, i32) DEF_HELPER_FLAGS_4(sve_subri_b, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(sve_subri_h, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 0674464695..ae853d21f2 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -700,7 +700,7 @@ SINCDECP_z 00100101 .. 1010 d:1 u:1 10000 00 .... ..... @incdec2_pred CTERM 00100101 1 sf:1 1 rm:5 001000 rn:5 ne:1 0000 # SVE integer compare scalar count and limit -WHILE 00100101 esz:2 1 rm:5 000 sf:1 u:1 1 rn:5 eq:1 rd:4 +WHILE 00100101 esz:2 1 rm:5 000 sf:1 u:1 lt:1 rn:5 eq:1 rd:4 ### SVE Integer Wide Immediate - Unpredicated Group diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 4b487d6f5f..700112e6cb 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3745,7 +3745,7 @@ uint64_t HELPER(sve_cntp)(void *vn, void *vg, uint32_t pred_desc) return sum; } -uint32_t HELPER(sve_while)(void *vd, uint32_t count, uint32_t pred_desc) +uint32_t HELPER(sve_whilel)(void *vd, uint32_t count, uint32_t pred_desc) { intptr_t oprsz = FIELD_EX32(pred_desc, PREDDESC, OPRSZ); intptr_t esz = FIELD_EX32(pred_desc, PREDDESC, ESZ); @@ -3771,6 +3771,42 @@ uint32_t HELPER(sve_while)(void *vd, uint32_t count, uint32_t pred_desc) return predtest_ones(d, oprsz, esz_mask); } +uint32_t HELPER(sve_whileg)(void *vd, uint32_t count, uint32_t pred_desc) +{ + intptr_t oprsz = FIELD_EX32(pred_desc, PREDDESC, OPRSZ); + intptr_t esz = FIELD_EX32(pred_desc, PREDDESC, ESZ); + uint64_t esz_mask = pred_esz_masks[esz]; + ARMPredicateReg *d = vd; + intptr_t i, invcount, oprbits; + uint64_t bits; + + if (count == 0) { + return do_zero(d, oprsz); + } + + oprbits = oprsz * 8; + tcg_debug_assert(count <= oprbits); + + bits = esz_mask; + if (oprbits & 63) { + bits &= MAKE_64BIT_MASK(0, oprbits & 63); + } + + invcount = oprbits - count; + for (i = (oprsz - 1) / 8; i > invcount / 64; --i) { + d->p[i] = bits; + bits = esz_mask; + } + + d->p[i] = bits & MAKE_64BIT_MASK(invcount & 63, 64); + + while (--i >= 0) { + d->p[i] = 0; + } + + return predtest_ones(d, oprsz, esz_mask); +} + /* Recursive reduction on a function; * C.f. the ARM ARM function ReducePredicated. * diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 5380ed26c1..f833bd5e33 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3112,7 +3112,14 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) unsigned vsz = vec_full_reg_size(s); unsigned desc = 0; TCGCond cond; + uint64_t maxval; + /* Note that GE/HS has a->eq == 0 and GT/HI has a->eq == 1. */ + bool eq = a->eq == a->lt; + /* The greater-than conditions are all SVE2. */ + if (!a->lt && !dc_isar_feature(aa64_sve2, s)) { + return false; + } if (!sve_access_check(s)) { return true; } @@ -3135,22 +3142,42 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) */ t0 = tcg_temp_new_i64(); t1 = tcg_temp_new_i64(); - tcg_gen_sub_i64(t0, op1, op0); + + if (a->lt) { + tcg_gen_sub_i64(t0, op1, op0); + if (a->u) { + maxval = a->sf ? UINT64_MAX : UINT32_MAX; + cond = eq ? TCG_COND_LEU : TCG_COND_LTU; + } else { + maxval = a->sf ? INT64_MAX : INT32_MAX; + cond = eq ? TCG_COND_LE : TCG_COND_LT; + } + } else { + tcg_gen_sub_i64(t0, op0, op1); + if (a->u) { + maxval = 0; + cond = eq ? TCG_COND_GEU : TCG_COND_GTU; + } else { + maxval = a->sf ? INT64_MIN : INT32_MIN; + cond = eq ? TCG_COND_GE : TCG_COND_GT; + } + } tmax = tcg_const_i64(vsz >> a->esz); - if (a->eq) { + if (eq) { /* Equality means one more iteration. */ tcg_gen_addi_i64(t0, t0, 1); - /* If op1 is max (un)signed integer (and the only time the addition - * above could overflow), then we produce an all-true predicate by - * setting the count to the vector length. This is because the - * pseudocode is described as an increment + compare loop, and the - * max integer would always compare true. + /* + * For the less-than while, if op1 is maxval (and the only time + * the addition above could overflow), then we produce an all-true + * predicate by setting the count to the vector length. This is + * because the pseudocode is described as an increment + compare + * loop, and the maximum integer would always compare true. + * Similarly, the greater-than while has the same issue with the + * minimum integer due to the decrement + compare loop. */ - tcg_gen_movi_i64(t1, (a->sf - ? (a->u ? UINT64_MAX : INT64_MAX) - : (a->u ? UINT32_MAX : INT32_MAX))); + tcg_gen_movi_i64(t1, maxval); tcg_gen_movcond_i64(TCG_COND_EQ, t0, op1, t1, tmax, t0); } @@ -3159,9 +3186,6 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) tcg_temp_free_i64(tmax); /* Set the count to zero if the condition is false. */ - cond = (a->u - ? (a->eq ? TCG_COND_LEU : TCG_COND_LTU) - : (a->eq ? TCG_COND_LE : TCG_COND_LT)); tcg_gen_movi_i64(t1, 0); tcg_gen_movcond_i64(cond, t0, op0, op1, t0, t1); tcg_temp_free_i64(t1); @@ -3181,7 +3205,11 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) ptr = tcg_temp_new_ptr(); tcg_gen_addi_ptr(ptr, cpu_env, pred_full_reg_offset(s, a->rd)); - gen_helper_sve_while(t2, ptr, t2, t3); + if (a->lt) { + gen_helper_sve_whilel(t2, ptr, t2, t3); + } else { + gen_helper_sve_whileg(t2, ptr, t2, t3); + } do_pred_flags(t2); tcg_temp_free_ptr(ptr); From patchwork Tue Mar 9 16:19:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396051 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2803927jai; Tue, 9 Mar 2021 10:07:26 -0800 (PST) X-Google-Smtp-Source: ABdhPJxRfLcDuFQrpdIK8c9bZUfdykR/90xier/eIfSokLhCeb9g66PdNdA0MgQoUJk5Kfy8hfKW X-Received: by 2002:a05:6e02:1c83:: with SMTP id w3mr26526795ill.216.1615313246752; Tue, 09 Mar 2021 10:07:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313246; cv=none; d=google.com; s=arc-20160816; b=NPRXgtXi5RjwmVoATxIBm99PDSyxcIuBYQkaZgi5eOC8kdIAQucPOGmMQic9osLM7P F9XQe1dnmE25gefyGDYxoCYfqlr4UuxmFlCzlULFbotBIMh9tpMjerrukMZCp8sofWWi 3GoegT9FmvjaaWDB/Qa3RQRc7ZW6rgmcuW4NJSFZ0URv8iF+e55pSIyij22AtqF98u1Q C/3b8rhDC4pIqUK2CqANp8IAMu2cohhcawcQojKGz4KUGk95++JO4aTqS8xE9iVv6xzE c3HvEhQtILrlVk9l73N6Lqsar7JZlNKWj8wa1IL4jYwCcdC6RZg0N3whTCO4RnzdLrb2 mZtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=OMON+uvzPOXYOLaXkReV00C37gevE1AfAAGsidJktiM=; b=xHebg7IlgbMpYEsWsiWlS0G5ZmG12hpgynpVitKYzkACawwJHglFMcQRxPb2EcYsbC F63HdZQwcyl4EZVtDougjGNvre1KpniiOPkiNG4yvZhvjO4bJnjDWQowc+fY8c8KWqjU K0nPWi/m659mmlvmxCbU8KdnO6wcaXsrcEP9T6EsGDbtFcq1UEXIsYaSWd0pSO5ZhgM+ eH6/w96XQ0LyMqX2TjrPqU/DaSTnhKi8zsDHaJwKME79SF6XOfvgo28zj/WAAi5FD0CU YcVRsF9dcRsubjTjfDWKRdwI3HsxHlbPvFUt0+98jQgXVE11PD8bAgwpYeP3AjJhScky KgCQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=APWEuaf+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b5si14595921jat.99.2021.03.09.10.07.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:07:26 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=APWEuaf+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40540 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJglS-0001fs-17 for patch@linaro.org; Tue, 09 Mar 2021 13:07:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46138) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7A-0007eR-Mk for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:44 -0500 Received: from mail-ot1-x32a.google.com ([2607:f8b0:4864:20::32a]:34744) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6l-0002WJ-U5 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:44 -0500 Received: by mail-ot1-x32a.google.com with SMTP id n23so7728988otq.1 for ; Tue, 09 Mar 2021 08:21:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OMON+uvzPOXYOLaXkReV00C37gevE1AfAAGsidJktiM=; b=APWEuaf++rtiRyvVmr4jLO5CzwnWiKJxuezQBul9sXUTx1yjAcqT6o0CcKbL7+5e7O noDWRUy4F0MAW2rI8oAyYqcKctodTzCnPHef9ktQZ1hmpUw7QsW7QBeB/Bpfsia2fJZj DDUA2U0eCuuvsjhNS+1Glknksdk+GMuvBsDCa7amx20yY/a3b1/GDU61uiCvuyicIDyw AgdI3ULAkCJ9DYb/vN8uUw8d5VO9sGGVEMbsxH5YB8o4+8oq3ctiRpc5o1gTc8RhdnaZ G1Cmug+umZWbN4Gj8qFBNVp+rYAztJ7ztEXmeBhxN0WQSe00MWpStzctjt3l4si/ah50 H5Gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OMON+uvzPOXYOLaXkReV00C37gevE1AfAAGsidJktiM=; b=Pt5hu0m/CJMiszplb0BcguPKRNfAQovGKo9PXnsMwQ2RkVDVtyRsuZs/rjlYgSk2Nu KZYmWWYvqV/Oagy8KIobdEf9rETdEVr3UMh59wsXGoa5fzeTW37TyBNUvhW0GVYMftOs veNC8uS1owJArNTjTIIbWkYEHF5zniXiPYaoSi+Ezb4Fr1v1D7ZSICH/m+YpCga/Fx07 mEBtZGTryOHzQd4HosaJ3rMAHdyWmCtW3Vq8wAEAUjEIo2wk2i0fxaY/j2jypHx1DBxX Ci4H+eCGXt4IZu15TlUf/mSl5BKrKzpw51oF7V3Ri6Xgor0xlealhVE5d8tm1mT86FlN OXbQ== X-Gm-Message-State: AOAM531ActL/xwwxZDYHuwwuCohGumLhoQ2Po1UD5mHLdt3sd7Ej538y WEpayrr+WY8rFDZrdvjl1ffshBOa1E+RwBau X-Received: by 2002:a05:6830:4109:: with SMTP id w9mr24886152ott.176.1615306878286; Tue, 09 Mar 2021 08:21:18 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:17 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 31/78] target/arm: Implement SVE2 WHILERW, WHILEWR Date: Tue, 9 Mar 2021 08:19:54 -0800 Message-Id: <20210309162041.23124-32-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32a; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix decodetree typo v3: Fix iteration counts (zhiwei). v4: Update for PREDDESC. --- target/arm/sve.decode | 3 ++ target/arm/translate-sve.c | 67 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index ae853d21f2..f365907518 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -702,6 +702,9 @@ CTERM 00100101 1 sf:1 1 rm:5 001000 rn:5 ne:1 0000 # SVE integer compare scalar count and limit WHILE 00100101 esz:2 1 rm:5 000 sf:1 u:1 lt:1 rn:5 eq:1 rd:4 +# SVE2 pointer conflict compare +WHILE_ptr 00100101 esz:2 1 rm:5 001 100 rn:5 rw:1 rd:4 + ### SVE Integer Wide Immediate - Unpredicated Group # SVE broadcast floating-point immediate (unpredicated) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index f833bd5e33..9e93223e62 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3218,6 +3218,73 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) return true; } +static bool trans_WHILE_ptr(DisasContext *s, arg_WHILE_ptr *a) +{ + TCGv_i64 op0, op1, diff, t1, tmax; + TCGv_i32 t2, t3; + TCGv_ptr ptr; + unsigned vsz = vec_full_reg_size(s); + unsigned desc = 0; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (!sve_access_check(s)) { + return true; + } + + op0 = read_cpu_reg(s, a->rn, 1); + op1 = read_cpu_reg(s, a->rm, 1); + + tmax = tcg_const_i64(vsz); + diff = tcg_temp_new_i64(); + + if (a->rw) { + /* WHILERW */ + /* diff = abs(op1 - op0), noting that op0/1 are unsigned. */ + t1 = tcg_temp_new_i64(); + tcg_gen_sub_i64(diff, op0, op1); + tcg_gen_sub_i64(t1, op1, op0); + tcg_gen_movcond_i64(TCG_COND_GEU, diff, op0, op1, diff, t1); + tcg_temp_free_i64(t1); + /* Round down to a multiple of ESIZE. */ + tcg_gen_andi_i64(diff, diff, -1 << a->esz); + /* If op1 == op0, diff == 0, and the condition is always true. */ + tcg_gen_movcond_i64(TCG_COND_EQ, diff, op0, op1, tmax, diff); + } else { + /* WHILEWR */ + tcg_gen_sub_i64(diff, op1, op0); + /* Round down to a multiple of ESIZE. */ + tcg_gen_andi_i64(diff, diff, -1 << a->esz); + /* If op0 >= op1, diff <= 0, the condition is always true. */ + tcg_gen_movcond_i64(TCG_COND_GEU, diff, op0, op1, tmax, diff); + } + + /* Bound to the maximum. */ + tcg_gen_umin_i64(diff, diff, tmax); + tcg_temp_free_i64(tmax); + + /* Since we're bounded, pass as a 32-bit type. */ + t2 = tcg_temp_new_i32(); + tcg_gen_extrl_i64_i32(t2, diff); + tcg_temp_free_i64(diff); + + desc = FIELD_DP32(desc, PREDDESC, OPRSZ, vsz / 8); + desc = FIELD_DP32(desc, PREDDESC, ESZ, a->esz); + t3 = tcg_const_i32(desc); + + ptr = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(ptr, cpu_env, pred_full_reg_offset(s, a->rd)); + + gen_helper_sve_whilel(t2, ptr, t2, t3); + do_pred_flags(t2); + + tcg_temp_free_ptr(ptr); + tcg_temp_free_i32(t2); + tcg_temp_free_i32(t3); + return true; +} + /* *** SVE Integer Wide Immediate - Unpredicated Group */ From patchwork Tue Mar 9 16:19:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396027 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2781526jai; Tue, 9 Mar 2021 09:37:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJzsKxqH3XZ9QPy8CY4efTEsP5q2/haPXTEJOSe7uHewSZAtEw8rPwatw6yVFoJV8jcZifan X-Received: by 2002:a5e:8e41:: with SMTP id r1mr24014826ioo.5.1615311473261; Tue, 09 Mar 2021 09:37:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311473; cv=none; d=google.com; s=arc-20160816; b=YQ7ytJRjWSC3rVWQoQEi0Qpp0seYjE2KEQTexrTe+iOr10Jp7+qZ5IT3PFHovmugt1 CADAkd6TzNGHcZHd2/h72RFRfkdWz9CCxyyonxeWwlME0cAkZBN62sGiY5ZTkE+3+ZiU G+snbqN8JAJgCGdJQR074mssk2aTuamtjtK5HqHH5EEl8gCmfz0PsURvi2twsmUCUHW2 7VHgensjNvzk+280jAPkvJDLnh7h8rq1aL0WrENVomZpRFUl/iIUf/LUO1s7/lnj9nJh 5Q9RpLnRPqfYt0815T3lZhDUw1cXDYGeT3ojgCTutx/AXJx3i3751jdvi5XU8OW/Hu2e zRtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=XGx0WYoYKWuyK1VYGtlBKMofDhawLxHmw/HrFOnsEOQ=; b=sUA+t4BnP+SvrVFNVzFoyKFwvl4xpskEX8D1RWeeCZKHmYgAe3DCR1kiNb9rvKaxHI n0BRZzP0aHWweFEy51KWcAJTTKmcDNd8WEajUmPZRQUWmx6+pTSzQqT4G3wrWzyaQ5ar Zj6qrY07UrtqWhV134diFrEXIUMs1yvrmmef8+Jnxbyov0CY4IlKZGnbeFzsZ84DMFo2 VGs2QLq8W3dpGHjAW6tzd0WEjmHcKFKjJIbd42kbeh85vaXwjLISGx3LNjFVqBMl4mUD fU7dJtr2alWnZroXME/keHNgvD2h8J4Z9wcHy1AqySTaS6zqaVfB25PBV+2Xz6ATENez JstQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EDRbFUxN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i22si11892938jaj.113.2021.03.09.09.37.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:37:53 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EDRbFUxN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:32830 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgIq-0007Gk-H2 for patch@linaro.org; Tue, 09 Mar 2021 12:37:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46164) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7B-0007gb-MJ for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:45 -0500 Received: from mail-ot1-x32d.google.com ([2607:f8b0:4864:20::32d]:38383) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6n-0002WT-B7 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:45 -0500 Received: by mail-ot1-x32d.google.com with SMTP id a17so13370678oto.5 for ; Tue, 09 Mar 2021 08:21:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XGx0WYoYKWuyK1VYGtlBKMofDhawLxHmw/HrFOnsEOQ=; b=EDRbFUxNVxt+nJZVEL1uROB+WhpuQw679tcuJJJ44vjSonEmYSPC/+GGTYqaXBJ8WY AA1j7XBoe0lXZWeLA7JB0YqxeGAjCLX9jMRSlLAsx15OrC+5wwYfqFQQCsW90m87jYja ojtZ8r9FlbAUR/dBeNEQ2PrMCSof647ngu8sOMn3XU5tObuRi2xHUIi6NytKnlE6zUsx 7s7XlSxFXalYIc98ZqtPi3+j3xARlDLNTuGxUf6iHEF7/TccyD4Vu4kXWoGPAbaa31rz h2I8F5fvyvYtqHRZEcgnfKk9VmnnSwHcF0GqakrMpGHzgvOQ5UqIpXpPGcCSrkkhtkky 9U+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XGx0WYoYKWuyK1VYGtlBKMofDhawLxHmw/HrFOnsEOQ=; b=bsDSzLbB6NgFfheLZu/Qi7WhB2od2zma19q+NMbeGZrCHQYWs9UInaatY3rZDC/7wm IsB9p3x2M6rVCWvh/Dqe7OH3Mt1jTxhBk89dWF72FZYLznQA9mwh8SSm2Ub9vElkOjOE 9PGExk/BNy4LOTKVlrnqWXqQgyBYAdYnG5KiORU2/2wiLLjhDrH0pu/xe1dqcQGOflSP 9pPY59uCSVk6YsQD+fi7YjjghjoWi1fjCmO/HMRiXD7KN7ObUXyU0BECTLuZ/5pZEx/g JcgAT97oEGb5jUvrNxyXpbt8wyxw+7cVlTKY1I5WwfcVo0kObPDYGLFsmgvzn4jYy4yv d0iA== X-Gm-Message-State: AOAM533KI80R6G1qlq/g/qPne4tfTfUzap2Tp3U1QTm79IdOJKvR872+ veEHyRIxxw3mIHPt0asOa5VeY/A0aXNIkCZH X-Received: by 2002:a05:6830:1113:: with SMTP id w19mr12335946otq.55.1615306879458; Tue, 09 Mar 2021 08:21:19 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:19 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 32/78] target/arm: Implement SVE2 bitwise ternary operations Date: Tue, 9 Mar 2021 08:19:55 -0800 Message-Id: <20210309162041.23124-33-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32d; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 6 ++ target/arm/sve.decode | 12 +++ target/arm/sve_helper.c | 50 +++++++++ target/arm/translate-sve.c | 213 +++++++++++++++++++++++++++++++++++++ 4 files changed, 281 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 5bf9fdc7a3..df617e3351 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2543,3 +2543,9 @@ DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_eor3, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_bcax, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_bsl1n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_bsl2n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_nbsl, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index f365907518..bf673e2f16 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -124,6 +124,10 @@ @rda_rn_rm ........ esz:2 . rm:5 ... ... rn:5 rd:5 \ &rrrr_esz ra=%reg_movprfx +# Four operand with unused vector element size +@rdn_ra_rm_e0 ........ ... rm:5 ... ... ra:5 rd:5 \ + &rrrr_esz esz=0 rn=%reg_movprfx + # Three operand with "memory" size, aka immediate left shift @rd_rn_msz_rm ........ ... rm:5 .... imm:2 rn:5 rd:5 &rrri @@ -379,6 +383,14 @@ ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +# SVE2 bitwise ternary operations +EOR3 00000100 00 1 ..... 001 110 ..... ..... @rdn_ra_rm_e0 +BSL 00000100 00 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 +BCAX 00000100 01 1 ..... 001 110 ..... ..... @rdn_ra_rm_e0 +BSL1N 00000100 01 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 +BSL2N 00000100 10 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 +NBSL 00000100 11 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 + ### SVE Index Generation Group # SVE index generation (immediate start, immediate increment) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 700112e6cb..4af8661cf5 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -6824,3 +6824,53 @@ DO_ST1_ZPZ_D(dd_be, zd, MO_64) #undef DO_ST1_ZPZ_S #undef DO_ST1_ZPZ_D + +void HELPER(sve2_eor3)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = n[i] ^ m[i] ^ k[i]; + } +} + +void HELPER(sve2_bcax)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = n[i] ^ (m[i] & ~k[i]); + } +} + +void HELPER(sve2_bsl1n)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = (~n[i] & k[i]) | (m[i] & ~k[i]); + } +} + +void HELPER(sve2_bsl2n)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = (n[i] & k[i]) | (~m[i] & ~k[i]); + } +} + +void HELPER(sve2_nbsl)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ~((n[i] & k[i]) | (m[i] & ~k[i])); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 9e93223e62..a4e1230ef1 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -217,6 +217,17 @@ static void gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn, vec_full_reg_offset(s, rm), vsz, vsz); } +/* Invoke a vector expander on four Zregs. */ +static void gen_gvec_fn_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn, + int esz, int rd, int rn, int rm, int ra) +{ + unsigned vsz = vec_full_reg_size(s); + gvec_fn(esz, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), vsz, vsz); +} + /* Invoke a vector move on two Zregs. */ static bool do_mov_z(DisasContext *s, int rd, int rn) { @@ -329,6 +340,208 @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a) return do_zzz_fn(s, a, tcg_gen_gvec_andc); } +static bool do_sve2_zzzz_fn(DisasContext *s, arg_rrrr_esz *a, GVecGen4Fn *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzzz(s, fn, a->esz, a->rd, a->rn, a->rm, a->ra); + } + return true; +} + +static void gen_eor3_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_xor_i64(d, n, m); + tcg_gen_xor_i64(d, d, k); +} + +static void gen_eor3_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + tcg_gen_xor_vec(vece, d, n, m); + tcg_gen_xor_vec(vece, d, d, k); +} + +static void gen_eor3(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_eor3_i64, + .fniv = gen_eor3_vec, + .fno = gen_helper_sve2_eor3, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_EOR3(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_eor3); +} + +static void gen_bcax_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_andc_i64(d, m, k); + tcg_gen_xor_i64(d, d, n); +} + +static void gen_bcax_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + tcg_gen_andc_vec(vece, d, m, k); + tcg_gen_xor_vec(vece, d, d, n); +} + +static void gen_bcax(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_bcax_i64, + .fniv = gen_bcax_vec, + .fno = gen_helper_sve2_bcax, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_BCAX(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bcax); +} + +static void gen_bsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + /* BSL differs from the generic bitsel in argument ordering. */ + tcg_gen_gvec_bitsel(vece, d, a, n, m, oprsz, maxsz); +} + +static bool trans_BSL(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bsl); +} + +static void gen_bsl1n_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_andc_i64(n, k, n); + tcg_gen_andc_i64(m, m, k); + tcg_gen_or_i64(d, n, m); +} + +static void gen_bsl1n_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + if (TCG_TARGET_HAS_bitsel_vec) { + tcg_gen_not_vec(vece, n, n); + tcg_gen_bitsel_vec(vece, d, k, n, m); + } else { + tcg_gen_andc_vec(vece, n, k, n); + tcg_gen_andc_vec(vece, m, m, k); + tcg_gen_or_vec(vece, d, n, m); + } +} + +static void gen_bsl1n(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_bsl1n_i64, + .fniv = gen_bsl1n_vec, + .fno = gen_helper_sve2_bsl1n, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_BSL1N(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bsl1n); +} + +static void gen_bsl2n_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + /* + * Z[dn] = (n & k) | (~m & ~k) + * = | ~(m | k) + */ + tcg_gen_and_i64(n, n, k); + if (TCG_TARGET_HAS_orc_i64) { + tcg_gen_or_i64(m, m, k); + tcg_gen_orc_i64(d, n, m); + } else { + tcg_gen_nor_i64(m, m, k); + tcg_gen_or_i64(d, n, m); + } +} + +static void gen_bsl2n_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + if (TCG_TARGET_HAS_bitsel_vec) { + tcg_gen_not_vec(vece, m, m); + tcg_gen_bitsel_vec(vece, d, k, n, m); + } else { + tcg_gen_and_vec(vece, n, n, k); + tcg_gen_or_vec(vece, m, m, k); + tcg_gen_orc_vec(vece, d, n, m); + } +} + +static void gen_bsl2n(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_bsl2n_i64, + .fniv = gen_bsl2n_vec, + .fno = gen_helper_sve2_bsl2n, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_BSL2N(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bsl2n); +} + +static void gen_nbsl_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_and_i64(n, n, k); + tcg_gen_andc_i64(m, m, k); + tcg_gen_nor_i64(d, n, m); +} + +static void gen_nbsl_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + tcg_gen_bitsel_vec(vece, d, k, n, m); + tcg_gen_not_vec(vece, d, d); +} + +static void gen_nbsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_nbsl_i64, + .fniv = gen_nbsl_vec, + .fno = gen_helper_sve2_nbsl, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_NBSL(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_nbsl); +} + /* *** SVE Integer Arithmetic - Unpredicated Group */ From patchwork Tue Mar 9 16:19:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396029 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2784135jai; Tue, 9 Mar 2021 09:41:22 -0800 (PST) X-Google-Smtp-Source: ABdhPJyNkzV00LTRhgrCLEb1etI2DnmD2UTqIqNVaWePIg7OEDfNZp73/Pm7S/osWCMPiwNJQSQr X-Received: by 2002:a92:cccb:: with SMTP id u11mr24985533ilq.44.1615311682153; Tue, 09 Mar 2021 09:41:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615311682; cv=none; d=google.com; s=arc-20160816; b=m2FBsfD6+Sv87FQR8wzP5A6YXueatc/Qy09n6za985Dun5wZ6ONaOeF4ypTNlUPgZ3 YSPqL5TvRGx7SBgeHT2t8wXrHksAbsowLAFO5CDS2pVAPp18M8mgMeJOOTC70NlsXM03 zeaH0j3AFpH8hn2P976x+GFdhOxnP0xR7kFvg4a74v9aEBcewnOQ791sYGQjKjWEzJe5 ujkLPXXwPJm9LVgN7zzgSBH1yiN5UY6+0HxQmNJgmmRhhtmc4Qe79AyudRIwVg1eBAGO usmeQNJcWTV7vAO8HT3tP5q8djQ16BWHwP4EkX8v/4+844vH1sGUrU/f5TAQD+WK5SqA 9XZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=zjgMuM7VtBxm6ii284K+FR//b3PnTa2tgQtaP6TgvsM=; b=eArZT/Erio2im0RBOpKhEVxFVmPYkyjsUU8G97K8/RoBrWlcIY+UXKeID535EbZsPy fMrwzfj4c4imihQLIMbvSHm/G1+DzMpWHgv/dgKVeTo7ffp+s/h0RhXt3NoLhtAIda0C ttjaZg+oQENulK0lcMgU6mAu4g+k4+mhrz4JNayePNDIWhJunZK7XDIB8qgfIF1oQJNx WDUuoIZRtYErBNGY7/Hw014nHt0qxrwEHluoMH5+cgK3m02RD6sEJ5ytFLkMQX2RQNX1 x/Mbxrihi7/eOkZMAzwwdMY8GPG2h1LZkaG2I+BnpsYfP6/hxAbMgXn5Y37zg/t8rftU Rk7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y4+AntUX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i13si15808847ilm.11.2021.03.09.09.41.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:41:22 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y4+AntUX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42564 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgMD-0003ES-Du for patch@linaro.org; Tue, 09 Mar 2021 12:41:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46198) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7D-0007kT-0z for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:47 -0500 Received: from mail-ot1-x336.google.com ([2607:f8b0:4864:20::336]:44760) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6n-0002Wi-Jb for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:46 -0500 Received: by mail-ot1-x336.google.com with SMTP id f33so13307300otf.11 for ; Tue, 09 Mar 2021 08:21:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zjgMuM7VtBxm6ii284K+FR//b3PnTa2tgQtaP6TgvsM=; b=Y4+AntUXf25f+LReilg5EWsFxGvmwRlRU6Uc79YLaK7dR559TKErExGwZQbkJYjuFn GJu+q5Na0fy1X0947SvpLkBIySnJzVpz/PvgSU682pf6zpztsWwfuAOGqxk1yRdw+puC ea2CezTQvToHp5lQxV5mQaObJNdhEqUZFeQyj6x20ulmOhHeX6XvNsLRs/6kJGx/jS+q F72puiGovIorkq1UYOhDeA1cA2bkCHzbaj7RLhEhvM4RbKWodS9hFiieyHOXwHN+CcDN jTxHm8eSt37l2A28IJ3XmZMpWGjmnvIkwusHmtScIYSEW4RFFuXk2KtWModpty6QfrNw GymQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zjgMuM7VtBxm6ii284K+FR//b3PnTa2tgQtaP6TgvsM=; b=b96dvlTeCjcz/5IgFHEhtQ2MaP21kbadv2k67uKFGNCC6yX/4Z7r51Qg41Ho3AFJjh xoq2+H3QYmw2iCGrQw+RPHZoA9qNMfUTwyehCFDfgKBD2CbcT5bnqr/vIaT6WKFmxoZK q3bDXKXjNZhaigE+qY8dzpm+OuMTDJB2jjn9YB+5CiKWkkc0ZvinmY8MkgOe57XE9yiW d6zf84PDidiMkMoQXTzovyZ1JIDQNiGk4HLSXGzNY/HUY/+Y1v68zOQfLCpe3wkZfDt5 TBgJoHpF2CUwlIp+KeeofbqmDEFPjpd0CNABeTmK8lSuulTe9upHitCjUmln3yb40al3 X8vQ== X-Gm-Message-State: AOAM532Z9wiP6/CRBAZX+sK7JT+AzVXZqr9LsUpra/FW7ItInUL78qz1 mQ36PUJsZ5Zx+WJrawSlaeKXmpH9tT4jZ/aJ X-Received: by 2002:a05:6830:139a:: with SMTP id d26mr11081513otq.256.1615306880636; Tue, 09 Mar 2021 08:21:20 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:20 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 33/78] target/arm: Implement SVE2 MATCH, NMATCH Date: Tue, 9 Mar 2021 08:19:56 -0800 Message-Id: <20210309162041.23124-34-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::336; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x336.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Reviewed-by: Richard Henderson Signed-off-by: Stephen Long Message-Id: <20200415145915.2859-1-steplong@quicinc.com> [rth: Expanded comment for do_match2] Signed-off-by: Richard Henderson --- v2: Apply esz_mask to input pg to fix output flags. --- target/arm/helper-sve.h | 10 ++++++ target/arm/sve.decode | 5 +++ target/arm/sve_helper.c | 64 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 22 +++++++++++++ 4 files changed, 101 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index df617e3351..11dc6870de 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2509,6 +2509,16 @@ DEF_HELPER_FLAGS_3(sve2_uqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_b, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_h, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index bf673e2f16..47fca5e12d 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1320,6 +1320,11 @@ UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr +### SVE2 Character Match + +MATCH 01000101 .. 1 ..... 100 ... ..... 0 .... @pd_pg_rn_rm +NMATCH 01000101 .. 1 ..... 100 ... ..... 1 .... @pd_pg_rn_rm + ## SVE2 floating-point pairwise operations FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 4af8661cf5..7a2af036c1 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -6874,3 +6874,67 @@ void HELPER(sve2_nbsl)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) d[i] = ~((n[i] & k[i]) | (m[i] & ~k[i])); } } + +/* + * Returns true if m0 or m1 contains the low uint8_t/uint16_t in n. + * See hasless(v,1) from + * https://graphics.stanford.edu/~seander/bithacks.html#ZeroInWord + */ +static inline bool do_match2(uint64_t n, uint64_t m0, uint64_t m1, int esz) +{ + int bits = 8 << esz; + uint64_t ones = dup_const(esz, 1); + uint64_t signs = ones << (bits - 1); + uint64_t cmp0, cmp1; + + cmp1 = dup_const(esz, n); + cmp0 = cmp1 ^ m0; + cmp1 = cmp1 ^ m1; + cmp0 = (cmp0 - ones) & ~cmp0; + cmp1 = (cmp1 - ones) & ~cmp1; + return (cmp0 | cmp1) & signs; +} + +static inline uint32_t do_match(void *vd, void *vn, void *vm, void *vg, + uint32_t desc, int esz, bool nmatch) +{ + uint16_t esz_mask = pred_esz_masks[esz]; + intptr_t opr_sz = simd_oprsz(desc); + uint32_t flags = PREDTEST_INIT; + intptr_t i, j, k; + + for (i = 0; i < opr_sz; i += 16) { + uint64_t m0 = *(uint64_t *)(vm + i); + uint64_t m1 = *(uint64_t *)(vm + i + 8); + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)) & esz_mask; + uint16_t out = 0; + + for (j = 0; j < 16; j += 8) { + uint64_t n = *(uint64_t *)(vn + i + j); + + for (k = 0; k < 8; k += 1 << esz) { + if (pg & (1 << (j + k))) { + bool o = do_match2(n >> (k * 8), m0, m1, esz); + out |= (o ^ nmatch) << (j + k); + } + } + } + *(uint16_t *)(vd + H1_2(i >> 3)) = out; + flags = iter_predtest_fwd(out, pg, flags); + } + return flags; +} + +#define DO_PPZZ_MATCH(NAME, ESZ, INV) \ +uint32_t HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + return do_match(vd, vn, vm, vg, desc, ESZ, INV); \ +} + +DO_PPZZ_MATCH(sve2_match_ppzz_b, MO_8, false) +DO_PPZZ_MATCH(sve2_match_ppzz_h, MO_16, false) + +DO_PPZZ_MATCH(sve2_nmatch_ppzz_b, MO_8, true) +DO_PPZZ_MATCH(sve2_nmatch_ppzz_h, MO_16, true) + +#undef DO_PPZZ_MATCH diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index a4e1230ef1..c339702cd3 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7463,6 +7463,28 @@ static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, + gen_helper_gvec_flags_4 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_ppzz_flags(s, a, fn); +} + +#define DO_SVE2_PPZZ_MATCH(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ +{ \ + static gen_helper_gvec_flags_4 * const fns[4] = { \ + gen_helper_sve2_##name##_ppzz_b, gen_helper_sve2_##name##_ppzz_h, \ + NULL, NULL \ + }; \ + return do_sve2_ppzz_flags(s, a, fns[a->esz]); \ +} + +DO_SVE2_PPZZ_MATCH(MATCH, match) +DO_SVE2_PPZZ_MATCH(NMATCH, nmatch) + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Tue Mar 9 16:19:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396043 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2794953jai; Tue, 9 Mar 2021 09:56:46 -0800 (PST) X-Google-Smtp-Source: ABdhPJy2ynBhjNR6agTjtAqSspTUdPb5QNb4QJUhABFLFqpa4Bo6p9iuOGirRez8K+uWusHub8oh X-Received: by 2002:a05:6602:cc:: with SMTP id z12mr22569158ioe.190.1615312606105; Tue, 09 Mar 2021 09:56:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312606; cv=none; d=google.com; s=arc-20160816; b=vbleDGJIYHeUHpURxLUxnLVHvy8zwA5m9l3hawR9bJn6XualLG9eN+CFhBl6PhIXqn 0Yn4ytEyLYD35OQpVFoUS2l9S8LnkvX5nuynhNvVaO6v6ti8AkFGVRTJyzRGtCwYhm3O 7/OioOdRvOuFZLBgpwWLDpOKgLSvCMAfE68scPQZ1AGpYWmRUV3v9JFu+MqAE4UtpgZM eSN4qGYwU4xo+qqANvOIcCpFCp8YSmZXIsiePd3/hTA94tu5ttCyMw815iUiD2CjENA3 U8NEdtaoo2OidTWN2JDXMhZV6nrdeTZd3caydE8D515mN4dpOPlLyxjqw7EjHln8lbdY J6Pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=J/syEFKHrWPbx1KGImwloEHiyCqnRrD0lkTeI6w8l6k=; b=aBd2t9ZLSf8eKTwc6tcmkww2NhiHkq1jLtHQrU2NxHWyqHLPImpTVpDsKyGBTREuXJ U4qkW3bfJtbW3TJD/bU3i4le0fTC+eaBd1v8y8QQ851JCObDovrefmQExWuNV+nLCTVf FT5Ts6OvhlEzwZ+6GmY+3kyhUUXY4hfm+FhwQlvtYAq2XlxMbChv4XrxC+EwNqnJ5pDw 0984HeZIBszS01c0NKqYw1L1uMotdUn0hXgG4NKSnVwnjMpnjpwIWr1QbUUWQ5tWYdyr iBSDjRb7hMZ2H1MYTGtYBgWsXUBhJmbOOxIxc7EqVj2ZPHySz7hWFwtaDNLzkJWLl+SE rnIA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OxBzAATv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z8si15345011jan.13.2021.03.09.09.56.45 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:56:46 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OxBzAATv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:49532 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgb7-0001aH-CG for patch@linaro.org; Tue, 09 Mar 2021 12:56:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46196) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7D-0007kR-05 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:47 -0500 Received: from mail-ot1-x335.google.com ([2607:f8b0:4864:20::335]:43895) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6p-0002X6-3x for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:46 -0500 Received: by mail-ot1-x335.google.com with SMTP id v12so13329327ott.10 for ; Tue, 09 Mar 2021 08:21:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J/syEFKHrWPbx1KGImwloEHiyCqnRrD0lkTeI6w8l6k=; b=OxBzAATvDSzvsxUcA6SAlvFmP0uYHrNkz5RIpFgjRlyaVm+FPPHap/ffj6CMWn4e7y ZGN5v15cxS4u+dfMMHDiZYEWwlOqlsfLgVwH9wDW/tw3sB3m39Zl7dJzsrCcs/0d1gQ+ b+plBkbxjYbLkrSnMAzWXqYlH0iylQx5uL97H1zuRJ7SlfuqLf2T2EkMC4iDc+kFqAoz 5408Rza6N1lKCkHpZYTM+w49GiDE6ZHgQj3STY3ynLD2YZhVPxMs0qCupvlWTeZJrH/l RAQwNw4aY88qQ5cOnZxxaFyx76S8yG7v2UyiBnFr6oWJkNei2mbXRhExLNMeWIEt6F4f PCnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J/syEFKHrWPbx1KGImwloEHiyCqnRrD0lkTeI6w8l6k=; b=hVHQYVUia0F+HsaMJlpd7GbfLkLQnUWGkfpgDExaN9PIYFy4wDMNLIY/6XptEj65qV Dw1pS6i/CnuChApT1l6k/VALRCwsX5NH6rq+xA3XmsulmQLcbFNmGocyM8ysYhWyMrBB 8uCrO1b83ExV3h7PB6dHTip0r4GkzIF4/g5ZnznaBG3TLFyNbQd1bw6UmSDXAxND+B46 3tfciZAyeFconM9nOJvo6Tv58Yw1QGI4H/LXw9C/IYM32IaRZEMXwDPnFn2Ro7R+WRJ/ hXjuKFH0ZwCrHDRUiXhbakE3tT4qKhApc0u7+fpQ+3FhuPU55bJXubnOiuL9Javw/HsE rG+g== X-Gm-Message-State: AOAM531OWH+LDwzFzXMnFD+VhLbbtD3+Jtr3oeAeknFyLU+rRhZ0p3zw r/2Tc5A6irhjpHmnEccR2bYOPtBsskS4cg34 X-Received: by 2002:a9d:2e4:: with SMTP id 91mr24998809otl.60.1615306881986; Tue, 09 Mar 2021 08:21:21 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:21 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 34/78] target/arm: Implement SVE2 saturating multiply-add long Date: Tue, 9 Mar 2021 08:19:57 -0800 Message-Id: <20210309162041.23124-35-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::335; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x335.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 14 ++++++++++ target/arm/sve.decode | 14 ++++++++++ target/arm/sve_helper.c | 30 +++++++++++++++++++++ target/arm/translate-sve.c | 54 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 112 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 11dc6870de..d8f390617c 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2559,3 +2559,17 @@ DEF_HELPER_FLAGS_5(sve2_bcax, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_bsl1n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_bsl2n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_nbsl, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqdmlal_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlal_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 47fca5e12d..52f615b39e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1332,3 +1332,17 @@ FMAXNMP 01100100 .. 010 10 0 100 ... ..... ..... @rdn_pg_rm FMINNMP 01100100 .. 010 10 1 100 ... ..... ..... @rdn_pg_rm FMAXP 01100100 .. 010 11 0 100 ... ..... ..... @rdn_pg_rm FMINP 01100100 .. 010 11 1 100 ... ..... ..... @rdn_pg_rm + +#### SVE Integer Multiply-Add (unpredicated) + +## SVE2 saturating multiply-add long + +SQDMLALB_zzzw 01000100 .. 0 ..... 0110 00 ..... ..... @rda_rn_rm +SQDMLALT_zzzw 01000100 .. 0 ..... 0110 01 ..... ..... @rda_rn_rm +SQDMLSLB_zzzw 01000100 .. 0 ..... 0110 10 ..... ..... @rda_rn_rm +SQDMLSLT_zzzw 01000100 .. 0 ..... 0110 11 ..... ..... @rda_rn_rm + +## SVE2 saturating multiply-add interleaved long + +SQDMLALBT 01000100 .. 0 ..... 00001 0 ..... ..... @rda_rn_rm +SQDMLSLBT 01000100 .. 0 ..... 00001 1 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 7a2af036c1..bd7332be43 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1400,6 +1400,36 @@ void HELPER(sve2_adcl_d)(void *vd, void *vn, void *vm, void *va, uint32_t desc) } } +#define DO_SQDMLAL(NAME, TYPEW, TYPEN, HW, HN, DMUL_OP, SUM_OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sel1 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + int sel2 = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel1)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel2)); \ + TYPEW aa = *(TYPEW *)(va + HW(i)); \ + *(TYPEW *)(vd + HW(i)) = SUM_OP(aa, DMUL_OP(nn, mm)); \ + } \ +} + +DO_SQDMLAL(sve2_sqdmlal_zzzw_h, int16_t, int8_t, H1_2, H1, + do_sqdmull_h, DO_SQADD_H) +DO_SQDMLAL(sve2_sqdmlal_zzzw_s, int32_t, int16_t, H1_4, H1_2, + do_sqdmull_s, DO_SQADD_S) +DO_SQDMLAL(sve2_sqdmlal_zzzw_d, int64_t, int32_t, , H1_4, + do_sqdmull_d, do_sqadd_d) + +DO_SQDMLAL(sve2_sqdmlsl_zzzw_h, int16_t, int8_t, H1_2, H1, + do_sqdmull_h, DO_SQSUB_H) +DO_SQDMLAL(sve2_sqdmlsl_zzzw_s, int32_t, int16_t, H1_4, H1_2, + do_sqdmull_s, DO_SQSUB_S) +DO_SQDMLAL(sve2_sqdmlsl_zzzw_d, int64_t, int32_t, , H1_4, + do_sqdmull_d, do_sqsub_d) + +#undef DO_SQDMLAL + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c339702cd3..15cbca0674 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7509,3 +7509,57 @@ DO_SVE2_ZPZZ_FP(FMAXNMP, fmaxnmp) DO_SVE2_ZPZZ_FP(FMINNMP, fminnmp) DO_SVE2_ZPZZ_FP(FMAXP, fmaxp) DO_SVE2_ZPZZ_FP(FMINP, fminp) + +/* + * SVE Integer Multiply-Add (unpredicated) + */ + +static bool do_sqdmlal_zzzw(DisasContext *s, arg_rrrr_esz *a, + bool sel1, bool sel2) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_sqdmlal_zzzw_h, + gen_helper_sve2_sqdmlal_zzzw_s, gen_helper_sve2_sqdmlal_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], (sel2 << 1) | sel1); +} + +static bool do_sqdmlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, + bool sel1, bool sel2) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_sqdmlsl_zzzw_h, + gen_helper_sve2_sqdmlsl_zzzw_s, gen_helper_sve2_sqdmlsl_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], (sel2 << 1) | sel1); +} + +static bool trans_SQDMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlal_zzzw(s, a, false, false); +} + +static bool trans_SQDMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlal_zzzw(s, a, true, true); +} + +static bool trans_SQDMLALBT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlal_zzzw(s, a, false, true); +} + +static bool trans_SQDMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlsl_zzzw(s, a, false, false); +} + +static bool trans_SQDMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlsl_zzzw(s, a, true, true); +} + +static bool trans_SQDMLSLBT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlsl_zzzw(s, a, false, true); +} From patchwork Tue Mar 9 16:19:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396038 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2791121jai; Tue, 9 Mar 2021 09:51:08 -0800 (PST) X-Google-Smtp-Source: ABdhPJxH23AnghbLljeFbKHPiOc5n6bYRAbI6LL4/dqjyRPWYi1CB58ua6zPCz3OzIfzPJprQGUh X-Received: by 2002:aed:31e2:: with SMTP id 89mr10326603qth.19.1615312268022; Tue, 09 Mar 2021 09:51:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312268; cv=none; d=google.com; s=arc-20160816; b=uuvim5aD68zEIIBx2be7EJAB9SexjYhh7choz/gc9LWHpJutHguciWrMAaSuSY2PP/ lGRSr6jf+YWUWhbXK31OXON2Ujl6G1AHbLHvbqN7YJas3i5VCJCm3YohiyPDVDjHVeqX edTqX9Knm3wuaPoXHtlHyeRZSTcr6LJdzCDNYBAmARnL/RANzESsOhzFhPxuRh1POP+X cFq+uBoX/MmQhmlgB3MvrbnfaaINolFmYoXh7BI2/kPlREAjvsLCOuhB7CVjJkMZ18wa K8nzmWRidvvypW8aU/2xqZJ2SnTHIk7yDOOUrs1gZR2PwMtx6qnlgIdssyjH57mPAufy +TUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=yIctWEimOpT3ofhcc26IOMfX6Wto+emY+iSYhkedMtU=; b=aGlOLjM5aMdL0PRdspSICRRnPcq7FzVmGW/D+55vXN87GOhVvXUB0ASkRGN06kAsVD 1ozE35t6G+QTwEvFfJiWgmbP411w3XOlNct/eh+JcaGRjf+sL+JcP5nMwr/tXkRRvMRv HoBm05KGHjNIKb3sV/hMbXM4OkMzqEChlYhHql843mx7ilhDc0eAII3iCgM1PaNu0C7u zTpEWeIGvtCx+8w6pRG+SnghpRHwZZ7rdbV4t8JUYRdQ+ABgDv1m0VQcrj7PgEjroEAH YVAChKaG3xFzDDjGnLIXAqqn0ibj8M8udwNlvHFjHIhVAPdXh8av4a952BoDGFZpudRR kEMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=l0qq8OhI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c6si8082021qko.346.2021.03.09.09.51.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:51:08 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=l0qq8OhI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34186 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgVf-0003ZD-Br for patch@linaro.org; Tue, 09 Mar 2021 12:51:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46242) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7E-0007o7-DM for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:48 -0500 Received: from mail-ot1-x32d.google.com ([2607:f8b0:4864:20::32d]:42018) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6r-0002Xo-6F for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:48 -0500 Received: by mail-ot1-x32d.google.com with SMTP id e45so13333823ote.9 for ; Tue, 09 Mar 2021 08:21:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yIctWEimOpT3ofhcc26IOMfX6Wto+emY+iSYhkedMtU=; b=l0qq8OhI5nS+ZFa5BGDM0y+aNWfficIuAJDPwB1pjoTq+kREq/1u30pbf6KCWnMQNI G8SeVAmvtwS/m1uVuI2GNPe6Xq+26CDVLP5TjTINYAgBolXXdg+p0Yh5//RFOreEixtx WPHbhJf6K1bazNIYV9vVhYGmALpjfJNS/9z7FpILqbnqKJxQH6Ni5ic6lj0ySlPrAz1q a//4anocXDlR94qvbTCV1NJZl3BgNGtxUq2ObeA2dtaQqhOvW3MNorWcDuhrt3sR8Mxx IreGEr81WoBiaqN3E4922CRpUcyBIkc27k79rA4njtl7cuGA5AH3mnvxe4uQx/7vCBkn 81WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yIctWEimOpT3ofhcc26IOMfX6Wto+emY+iSYhkedMtU=; b=ijDGGp1CQpgbaRhF9zrTJMsNSfChKl37UJFlrR2lFk9G/8j7Vz09Nxokkdv2hANCGt kFjvK+g1CsF/UkvusalPbja2vFMD3kYLq3n2lQU4rYyuILImVwP6IMWwdntBC3e8RL5O JX3MpHoS5w2K8tzfqikXy+2F8qatT9RCm63v2d+a9OMcUjnvfZUiouGlfHEx1HnYE6Qg Z+cP6jB3bj2qvmO+UnwUm1J9KFwV4MGns6WpmDUM5w3FVrHmkB/hTMgREoYBpO/N9p/X bQL7sjRTB6TYT1vo9BNbisbuGinzoZM8T9nMVYc+MRGRmcnegrzu0JPsrRlyaiSn4Vjt 8sWA== X-Gm-Message-State: AOAM531vJvvmfEWUJLjdKOngKKahQPMBObfk8BioGCTVM2lQbZ9Bxl6k Pcujs/1G7gaTTNqIEvai12YryLcYIcczTdGU X-Received: by 2002:a9d:3f37:: with SMTP id m52mr24075414otc.92.1615306883289; Tue, 09 Mar 2021 08:21:23 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:22 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 35/78] target/arm: Implement SVE2 saturating multiply-add high Date: Tue, 9 Mar 2021 08:19:58 -0800 Message-Id: <20210309162041.23124-36-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32d; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" SVE2 has two additional sizes of the operation and unlike NEON, there is no saturation flag. Create new entry points for SVE2 that do not set QC. Signed-off-by: Richard Henderson --- target/arm/helper.h | 17 ++++ target/arm/sve.decode | 5 ++ target/arm/translate-sve.c | 18 +++++ target/arm/vec_helper.c | 161 +++++++++++++++++++++++++++++++++++-- 4 files changed, 195 insertions(+), 6 deletions(-) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 2c412ffd3b..6bb0b0ddc0 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -591,6 +591,23 @@ DEF_HELPER_FLAGS_5(gvec_qrdmlah_s32, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_qrdmlsh_s32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 52f615b39e..8308c9238a 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1346,3 +1346,8 @@ SQDMLSLT_zzzw 01000100 .. 0 ..... 0110 11 ..... ..... @rda_rn_rm SQDMLALBT 01000100 .. 0 ..... 00001 0 ..... ..... @rda_rn_rm SQDMLSLBT 01000100 .. 0 ..... 00001 1 ..... ..... @rda_rn_rm + +## SVE2 saturating multiply-add high + +SQRDMLAH_zzzz 01000100 .. 0 ..... 01110 0 ..... ..... @rda_rn_rm +SQRDMLSH_zzzz 01000100 .. 0 ..... 01110 1 ..... ..... @rda_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 15cbca0674..f6eea9554b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7563,3 +7563,21 @@ static bool trans_SQDMLSLBT(DisasContext *s, arg_rrrr_esz *a) { return do_sqdmlsl_zzzw(s, a, false, true); } + +static bool trans_SQRDMLAH_zzzz(DisasContext *s, arg_rrrr_esz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_sqrdmlah_b, gen_helper_sve2_sqrdmlah_h, + gen_helper_sve2_sqrdmlah_s, gen_helper_sve2_sqrdmlah_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); +} + +static bool trans_SQRDMLSH_zzzz(DisasContext *s, arg_rrrr_esz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_sqrdmlsh_b, gen_helper_sve2_sqrdmlsh_h, + gen_helper_sve2_sqrdmlsh_s, gen_helper_sve2_sqrdmlsh_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index b0ce597060..c56337e724 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -22,6 +22,7 @@ #include "exec/helper-proto.h" #include "tcg/tcg-gvec-desc.h" #include "fpu/softfloat.h" +#include "qemu/int128.h" #include "vec_internal.h" /* Note that vector data is stored in host-endian 64-bit chunks, @@ -36,15 +37,55 @@ #define H4(x) (x) #endif +/* Signed saturating rounding doubling multiply-accumulate high half, 8-bit */ +static int8_t do_sqrdmlah_b(int8_t src1, int8_t src2, int8_t src3, + bool neg, bool round) +{ + /* + * Simplify: + * = ((a3 << 8) + ((e1 * e2) << 1) + (round << 7)) >> 8 + * = ((a3 << 7) + (e1 * e2) + (round << 6)) >> 7 + */ + int32_t ret = (int32_t)src1 * src2; + if (neg) { + ret = -ret; + } + ret += ((int32_t)src3 << 7) + (round << 6); + ret >>= 7; + + if (ret != (int8_t)ret) { + ret = (ret < 0 ? INT8_MIN : INT8_MAX); + } + return ret; +} + +void HELPER(sve2_sqrdmlah_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], a[i], false, true); + } +} + +void HELPER(sve2_sqrdmlsh_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], a[i], true, true); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */ static int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, bool neg, bool round, uint32_t *sat) { - /* - * Simplify: - * = ((a3 << 16) + ((e1 * e2) << 1) + (1 << 15)) >> 16 - * = ((a3 << 15) + (e1 * e2) + (1 << 14)) >> 15 - */ + /* Simplify similarly to do_sqrdmlah_b above. */ int32_t ret = (int32_t)src1 * src2; if (neg) { ret = -ret; @@ -133,11 +174,35 @@ void HELPER(neon_sqrdmulh_h)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(sve2_sqrdmlah_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], a[i], false, true, &discard); + } +} + +void HELPER(sve2_sqrdmlsh_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], a[i], true, true, &discard); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ static int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, bool neg, bool round, uint32_t *sat) { - /* Simplify similarly to int_qrdmlah_s16 above. */ + /* Simplify similarly to do_sqrdmlah_b above. */ int64_t ret = (int64_t)src1 * src2; if (neg) { ret = -ret; @@ -220,6 +285,90 @@ void HELPER(neon_sqrdmulh_s)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(sve2_sqrdmlah_s)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], a[i], false, true, &discard); + } +} + +void HELPER(sve2_sqrdmlsh_s)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], a[i], true, true, &discard); + } +} + +/* Signed saturating rounding doubling multiply-accumulate high half, 64-bit */ +static int64_t do_sat128_d(Int128 r) +{ + int64_t ls = int128_getlo(r); + int64_t hs = int128_gethi(r); + + if (unlikely(hs != (ls >> 63))) { + return hs < 0 ? INT64_MIN : INT64_MAX; + } + return ls; +} + +static int64_t do_sqrdmlah_d(int64_t n, int64_t m, int64_t a, + bool neg, bool round) +{ + uint64_t l, h; + Int128 r, t; + + /* As in do_sqrdmlah_b, but with 128-bit arithmetic. */ + muls64(&l, &h, m, n); + r = int128_make128(l, h); + if (neg) { + r = int128_neg(r); + } + if (a) { + t = int128_exts64(a); + t = int128_lshift(t, 63); + r = int128_add(r, t); + } + if (round) { + t = int128_exts64(1ll << 62); + r = int128_add(r, t); + } + r = int128_rshift(r, 63); + + return do_sat128_d(r); +} + +void HELPER(sve2_sqrdmlah_d)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], a[i], false, true); + } +} + +void HELPER(sve2_sqrdmlsh_d)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], a[i], true, true); + } +} + /* Integer 8 and 16-bit dot-product. * * Note that for the loops herein, host endianness does not matter From patchwork Tue Mar 9 16:19:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396061 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2812486jai; Tue, 9 Mar 2021 10:18:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJw/xq0/iPyrq2MvnhY5T1/HXrV7aNqS9Pqv1FSRrMkAMsm1oLn2BLiELHxYMGFBFqgIcDc7 X-Received: by 2002:a05:6638:144e:: with SMTP id l14mr29409358jad.76.1615313908110; Tue, 09 Mar 2021 10:18:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313908; cv=none; d=google.com; s=arc-20160816; b=0vQIFhOS40pKUSIJN+xpGGGFd+ETjkvwRn8mShWUqJBTv0ep5DAcs28O/AvXTwhT3V Rp5aH0VdxyfnSDbBAOO2VTCbpvBrQWxYwinQdx109K9T3pZNOY1Hh+NqDQetoFeUjUjb A3K8wifLWpiO/El2LqoXKLSiOcLo0myMFByWddcoFlj7yX0xG3d4fp5nABqB9rLtx5cw kpN9kqGWXBvR+h5aBXERXP/N77YyoEIbX97wDQY4u8b9X6nt8vLjDVp57JnfBlreAUbL 9MsMagyZJ2HNZLL28Nb3M9SqyffFVUU3odUN8eiWXikU6g5g2pKXtCbN4a1f24BsAsu2 eTbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=MSMseqklTx5XGInPbltkyfVEXNwiNbXjqnPSDdzuxDo=; b=odhFQisaV9dJpmwey9nvHB7VlIC52Pm8Fo1GVEEiBAF5nDWqoq/1h9eTgmezs9/SzN a6t23W+RsBYKIs1NWwQ4t0Ft4rdEnCJbT5nppz5DseGG3tM4x9DVy8Wdh/CnA7TGFCqu 7t6rZhtllpLdPTVDXnnMPBTcRTndnR921g99afuWXfF716xmbfqMo4kwxLzvt5hiVWGo tnhoiMHh1IItbgeL555ZN7BEVMkfOvbOzPP9wqky78cs76h7eAqY+Ml23CkeNZk416Z6 qQvaKQls0t6Nee6Ah/y1O9Zv3zxAxgCXTtSbg6QBicX4CLY6kHZKotdLeUw/ZVjsKhEc Ednw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TgkozMef; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b28si13763689jav.105.2021.03.09.10.18.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:18:28 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TgkozMef; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34696 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgw7-00055X-3o for patch@linaro.org; Tue, 09 Mar 2021 13:18:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46302) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7G-0007tJ-I5 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:50 -0500 Received: from mail-oi1-x22a.google.com ([2607:f8b0:4864:20::22a]:47070) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6w-0002Y3-Uo for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:50 -0500 Received: by mail-oi1-x22a.google.com with SMTP id f3so15494070oiw.13 for ; Tue, 09 Mar 2021 08:21:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MSMseqklTx5XGInPbltkyfVEXNwiNbXjqnPSDdzuxDo=; b=TgkozMefSqZ+GhCKjKxmVoP9sOUMytnoBHBk/KDSlyJkdrpzRzuOsFikQ7Mr/W2L5U i7IbZqO8JdRo0wjfRW+KyVzEjfncqLuYrZ68wobXnsieO/A2NjI6iFztm2sAvDVhc8sJ FSx19wYUT5S5sCRywRWDdTxxzto2ANfYokd/WbNRE4139+pew7sO/f1IwCtdwppFXgxS ZmSTgUVDaJy1p0LpxazzvSR4LLl1mHO9gcNrYb6EC+R6WTw1qv1tq+H6XiA/+jT5UrDa OR9NUvemvTRTBzHwrX9RuHxcRO1VKLLfOz1ookGatjPUQNiEs1QzUCpNgvLnGD3eSMs3 vncg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MSMseqklTx5XGInPbltkyfVEXNwiNbXjqnPSDdzuxDo=; b=EJ6kr4rjWpkJiWhRvPKx0Dhvm0ceN26XYbC2TNx+zg2zx3pH/qHW/PguogDaEkg+Vu nBDjrXZ4kmaGMu/4VqwQiHOQlf8dHN9gBB70uWPmyGQKkRL09HEtXDzvREuWx0eeg/Rp Bx2iDP+8mMXW55piei9yymk5uuqomJ7B6dn4fNt+dV6+P3BNkt9nf/iz5QgznjQA720V SQHClp043+OAzIebCaFrdFE7KBhsM1nMKyxJnSubpxuy0dG8TZhiAgrHZhjMh9ld7Vhk lYgSHaYlL0B6vqXhorxWCwSiA1kHtPSd9JmcStNnwoCOs6EPu3mYWECs5sjYSDI22DsA bZfQ== X-Gm-Message-State: AOAM532YXVpa0sAovQfJVYVV/3OprvszpN5P48PqPNpj6qNuSkvsAXK2 7P9vIFiG/3+LEX6R2gVPeBfOcTIPwEHxZNPe X-Received: by 2002:a54:488b:: with SMTP id r11mr3637344oic.166.1615306884402; Tue, 09 Mar 2021 08:21:24 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:24 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 36/78] target/arm: Implement SVE2 integer multiply-add long Date: Tue, 9 Mar 2021 08:19:59 -0800 Message-Id: <20210309162041.23124-37-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22a; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 28 ++++++++++++++ target/arm/sve.decode | 11 ++++++ target/arm/sve_helper.c | 18 +++++++++ target/arm/translate-sve.c | 76 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 133 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index d8f390617c..457a421455 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2573,3 +2573,31 @@ DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_smlal_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlal_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_umlal_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlal_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_smlsl_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlsl_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlsl_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 8308c9238a..b28b50e05c 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1351,3 +1351,14 @@ SQDMLSLBT 01000100 .. 0 ..... 00001 1 ..... ..... @rda_rn_rm SQRDMLAH_zzzz 01000100 .. 0 ..... 01110 0 ..... ..... @rda_rn_rm SQRDMLSH_zzzz 01000100 .. 0 ..... 01110 1 ..... ..... @rda_rn_rm + +## SVE2 integer multiply-add long + +SMLALB_zzzw 01000100 .. 0 ..... 010 000 ..... ..... @rda_rn_rm +SMLALT_zzzw 01000100 .. 0 ..... 010 001 ..... ..... @rda_rn_rm +UMLALB_zzzw 01000100 .. 0 ..... 010 010 ..... ..... @rda_rn_rm +UMLALT_zzzw 01000100 .. 0 ..... 010 011 ..... ..... @rda_rn_rm +SMLSLB_zzzw 01000100 .. 0 ..... 010 100 ..... ..... @rda_rn_rm +SMLSLT_zzzw 01000100 .. 0 ..... 010 101 ..... ..... @rda_rn_rm +UMLSLB_zzzw 01000100 .. 0 ..... 010 110 ..... ..... @rda_rn_rm +UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index bd7332be43..61903cded6 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1308,6 +1308,24 @@ DO_ZZZW_ACC(sve2_uabal_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) DO_ZZZW_ACC(sve2_uabal_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) +DO_ZZZW_ACC(sve2_smlal_zzzw_h, int16_t, int8_t, H1_2, H1, DO_MUL) +DO_ZZZW_ACC(sve2_smlal_zzzw_s, int32_t, int16_t, H1_4, H1_2, DO_MUL) +DO_ZZZW_ACC(sve2_smlal_zzzw_d, int64_t, int32_t, , H1_4, DO_MUL) + +DO_ZZZW_ACC(sve2_umlal_zzzw_h, uint16_t, uint8_t, H1_2, H1, DO_MUL) +DO_ZZZW_ACC(sve2_umlal_zzzw_s, uint32_t, uint16_t, H1_4, H1_2, DO_MUL) +DO_ZZZW_ACC(sve2_umlal_zzzw_d, uint64_t, uint32_t, , H1_4, DO_MUL) + +#define DO_NMUL(N, M) -(N * M) + +DO_ZZZW_ACC(sve2_smlsl_zzzw_h, int16_t, int8_t, H1_2, H1, DO_NMUL) +DO_ZZZW_ACC(sve2_smlsl_zzzw_s, int32_t, int16_t, H1_4, H1_2, DO_NMUL) +DO_ZZZW_ACC(sve2_smlsl_zzzw_d, int64_t, int32_t, , H1_4, DO_NMUL) + +DO_ZZZW_ACC(sve2_umlsl_zzzw_h, uint16_t, uint8_t, H1_2, H1, DO_NMUL) +DO_ZZZW_ACC(sve2_umlsl_zzzw_s, uint32_t, uint16_t, H1_4, H1_2, DO_NMUL) +DO_ZZZW_ACC(sve2_umlsl_zzzw_d, uint64_t, uint32_t, , H1_4, DO_NMUL) + #undef DO_ZZZW_ACC #define DO_XTNB(NAME, TYPE, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index f6eea9554b..c112548db4 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7581,3 +7581,79 @@ static bool trans_SQRDMLSH_zzzz(DisasContext *s, arg_rrrr_esz *a) }; return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); } + +static bool do_smlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_smlal_zzzw_h, + gen_helper_sve2_smlal_zzzw_s, gen_helper_sve2_smlal_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_SMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlal_zzzw(s, a, false); +} + +static bool trans_SMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlal_zzzw(s, a, true); +} + +static bool do_umlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_umlal_zzzw_h, + gen_helper_sve2_umlal_zzzw_s, gen_helper_sve2_umlal_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_UMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlal_zzzw(s, a, false); +} + +static bool trans_UMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlal_zzzw(s, a, true); +} + +static bool do_smlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_smlsl_zzzw_h, + gen_helper_sve2_smlsl_zzzw_s, gen_helper_sve2_smlsl_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_SMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlsl_zzzw(s, a, false); +} + +static bool trans_SMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlsl_zzzw(s, a, true); +} + +static bool do_umlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_umlsl_zzzw_h, + gen_helper_sve2_umlsl_zzzw_s, gen_helper_sve2_umlsl_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_UMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlsl_zzzw(s, a, false); +} + +static bool trans_UMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlsl_zzzw(s, a, true); +} From patchwork Tue Mar 9 16:20:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396057 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2809919jai; Tue, 9 Mar 2021 10:14:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJzH1LYGTPZIhFxSya0F7TElRTqBL6MqOqXWAgqWeVfR+rDAwYanKOMQg8bFfguDh0QtJcsW X-Received: by 2002:a92:d0c3:: with SMTP id y3mr24669310ila.303.1615313697416; Tue, 09 Mar 2021 10:14:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313697; cv=none; d=google.com; s=arc-20160816; b=CM5BuWo0/IQbhXgwcwM+FxCqlw7/RWmCpZkKn/8olK9FgpvwZqYUUEsZjrh5P3U2lO ZdJv1OiwflC9Jksn5NIrb8FbBpGx38jKLiJPzkcoNd9EO+QKHMNZO69UbcZisYsouABQ KSULdRHBEdKGWzzlAUETq2ddNwBbCaiB4IhfTeUx3LNuO+NgHgv6PBf5KjgoXFhj41S/ uRMssnrtiZwm43dHmbUQfr1glQsH8DyI0dD3I3haPupVdgHFjvBmeJC0rh2tu2ds1nxv OVeRhDLJIZn+vTA9ZpcUafGT9qYXm2ay77Mt4snCVNRObIVxaajUuseR+yhZWt93X8b2 V8Rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=UzDsHngEWpbjz3McPd9SBlyGKK1l5sx+C92oFmNJwrM=; b=F4LJNtlSHvRKRY9J8aw8CXFSvpN7yuX7qUzDzFzVtkbpNV1LIpN4N4Eg+dN5D95DEN wxDsCLa/Rtfv1xgpw3r8km0Pkjn+TTqF+BzXTcorWIMzc+e7sCtirn2hNX9q32UaCwqz tAujnGrnd02YTYJaFDTKiE3SsWmpsxoAh4CBKoTHoa3vnE17W1Lw+aYvmWZmPoHrUW+C UvDB137A9Jkvv0p9tPluCnOd2OQpJ84V56RWRgtNeBaX3nDS8Fi6Rmt4JrYumBJ/jg9F ZN1MX4zICHchx3bQ79FjurTahEq/rRP130DY6uxUUFgETqYasoS4mK0zT2X/AViARfQT 9N1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qPtbruXa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i16si14053106ioi.40.2021.03.09.10.14.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:14:57 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qPtbruXa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52228 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgsi-00083c-MZ for patch@linaro.org; Tue, 09 Mar 2021 13:14:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46252) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7F-0007pq-0c for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:49 -0500 Received: from mail-oi1-x22f.google.com ([2607:f8b0:4864:20::22f]:35201) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6t-0002Yc-AU for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:48 -0500 Received: by mail-oi1-x22f.google.com with SMTP id u6so8190384oic.2 for ; Tue, 09 Mar 2021 08:21:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UzDsHngEWpbjz3McPd9SBlyGKK1l5sx+C92oFmNJwrM=; b=qPtbruXaIDR7lLBJEqaV/LWzw3uECdp285rUPfN1o2gZ9KwJxMpNbKHIGSeFU5r4Mz ePIwa0osuWXH+N0I+x77qplRTjwqBQlmAVlCxQ9BC1rWAhaI7pIzUhMiw8MhhaAPjOLo 7upPCUmaHUT8iB9Of43wS1Z20/rJFKOaJZRFdUJnNntQcHErA5ZeCGKFHl1m9v8Hxbrp cZurqBle+w+QZWKbxjflwDYoxIFd928Nt0PmRunspQiT4GuEe2IXDNo8LRSGIxy8OOcP 3N0zoAtZltR2MeuS46ZW1uGb98HmOSb0JEVQaHqQMvXvTWeD8Txo3MDQQQaXq5y4ZXgD ZuTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UzDsHngEWpbjz3McPd9SBlyGKK1l5sx+C92oFmNJwrM=; b=ems4NAc8HQ3Utla1wfuv4msIJ94IlMv9Vn4YAiY0hhRJ24blH7h5aDd6cJHahB18fT Kc3pVz7D4WEiZApIAh2WcliyRKEIbROMxS1m+9tuu8JPUYeQQ8wddwhRXBfjWFWY4Ih5 pHW6XHXQsYpz6fqd18zkYmcv31nx2x8ISTA7HmGHQDa3D98LKk6CgJ3/fNhbUL2vDNtS lPn+5GsL2WBju0tQ1tjt8RvKcbKVaqrao/3V7jdOGwpBWxf0prh/AYxfNAlOAejfP/38 MFPS7zb5ahMOJX4ffhnXEzh0Iteiyau16AhWJL+Gx2CpEqxoX009WkpxX1ftz1JGuFHR 3IKQ== X-Gm-Message-State: AOAM532GfxPxgLtF718LPheDveQzMK7Qv4cV9uG+J2g8xrqWtr6cCw60 wb0IFJ3SaMFFhyzBSPxR+TrY8v6aUPUcpVCs X-Received: by 2002:aca:fc11:: with SMTP id a17mr1348034oii.68.1615306885533; Tue, 09 Mar 2021 08:21:25 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:25 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 37/78] target/arm: Implement SVE2 complex integer multiply-add Date: Tue, 9 Mar 2021 08:20:00 -0800 Message-Id: <20210309162041.23124-38-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22f; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix do_sqrdmlah_d (laurent desnogues) --- target/arm/helper-sve.h | 18 ++++++++++++++++ target/arm/vec_internal.h | 5 +++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 42 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 32 +++++++++++++++++++++++++++++ target/arm/vec_helper.c | 15 +++++++------- 6 files changed, 109 insertions(+), 8 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 457a421455..d154218452 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2601,3 +2601,21 @@ DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h index 0102547a10..200fb55909 100644 --- a/target/arm/vec_internal.h +++ b/target/arm/vec_internal.h @@ -168,4 +168,9 @@ static inline int64_t do_suqrshl_d(int64_t src, int64_t shift, return do_uqrshl_d(src, shift, round, sat); } +int8_t do_sqrdmlah_b(int8_t, int8_t, int8_t, bool, bool); +int16_t do_sqrdmlah_h(int16_t, int16_t, int16_t, bool, bool, uint32_t *); +int32_t do_sqrdmlah_s(int32_t, int32_t, int32_t, bool, bool, uint32_t *); +int64_t do_sqrdmlah_d(int64_t, int64_t, int64_t, bool, bool); + #endif /* TARGET_ARM_VEC_INTERNALS_H */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index b28b50e05c..936977eacb 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1362,3 +1362,8 @@ SMLSLB_zzzw 01000100 .. 0 ..... 010 100 ..... ..... @rda_rn_rm SMLSLT_zzzw 01000100 .. 0 ..... 010 101 ..... ..... @rda_rn_rm UMLSLB_zzzw 01000100 .. 0 ..... 010 110 ..... ..... @rda_rn_rm UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm + +## SVE2 complex integer multiply-add + +CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx +SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 61903cded6..3bf81d96ae 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1448,6 +1448,48 @@ DO_SQDMLAL(sve2_sqdmlsl_zzzw_d, int64_t, int32_t, , H1_4, #undef DO_SQDMLAL +#define DO_CMLA(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / sizeof(TYPE); \ + int rot = simd_data(desc); \ + int sel_a = rot & 1, sel_b = sel_a ^ 1; \ + bool sub_r = rot == 1 || rot == 2; \ + bool sub_i = rot >= 2; \ + TYPE *d = vd, *n = vn, *m = vm, *a = va; \ + for (i = 0; i < opr_sz; i += 2) { \ + TYPE elt1_a = n[H(i + sel_a)]; \ + TYPE elt2_a = m[H(i + sel_a)]; \ + TYPE elt2_b = m[H(i + sel_b)]; \ + d[H(i)] = OP(elt1_a, elt2_a, a[H(i)], sub_r); \ + d[H(i + 1)] = OP(elt1_a, elt2_b, a[H(i + 1)], sub_i); \ + } \ +} + +#define do_cmla(N, M, A, S) (A + (N * M) * (S ? -1 : 1)) + +DO_CMLA(sve2_cmla_zzzz_b, uint8_t, H1, do_cmla) +DO_CMLA(sve2_cmla_zzzz_h, uint16_t, H2, do_cmla) +DO_CMLA(sve2_cmla_zzzz_s, uint32_t, H4, do_cmla) +DO_CMLA(sve2_cmla_zzzz_d, uint64_t, , do_cmla) + +#define DO_SQRDMLAH_B(N, M, A, S) \ + do_sqrdmlah_b(N, M, A, S, true) +#define DO_SQRDMLAH_H(N, M, A, S) \ + ({ uint32_t discard; do_sqrdmlah_h(N, M, A, S, true, &discard); }) +#define DO_SQRDMLAH_S(N, M, A, S) \ + ({ uint32_t discard; do_sqrdmlah_s(N, M, A, S, true, &discard); }) +#define DO_SQRDMLAH_D(N, M, A, S) \ + do_sqrdmlah_d(N, M, A, S, true) + +DO_CMLA(sve2_sqrdcmlah_zzzz_b, int8_t, H1, DO_SQRDMLAH_B) +DO_CMLA(sve2_sqrdcmlah_zzzz_h, int16_t, H2, DO_SQRDMLAH_H) +DO_CMLA(sve2_sqrdcmlah_zzzz_s, int32_t, H4, DO_SQRDMLAH_S) +DO_CMLA(sve2_sqrdcmlah_zzzz_d, int64_t, , DO_SQRDMLAH_D) + +#undef do_cmla +#undef DO_CMLA + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c112548db4..29d33389c2 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7657,3 +7657,35 @@ static bool trans_UMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) { return do_umlsl_zzzw(s, a, true); } + +static bool trans_CMLA_zzzz(DisasContext *s, arg_CMLA_zzzz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_cmla_zzzz_b, gen_helper_sve2_cmla_zzzz_h, + gen_helper_sve2_cmla_zzzz_s, gen_helper_sve2_cmla_zzzz_d, + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot); + } + return true; +} + +static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_sqrdcmlah_zzzz_b, gen_helper_sve2_sqrdcmlah_zzzz_h, + gen_helper_sve2_sqrdcmlah_zzzz_s, gen_helper_sve2_sqrdcmlah_zzzz_d, + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot); + } + return true; +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index c56337e724..19006f50f7 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -38,8 +38,8 @@ #endif /* Signed saturating rounding doubling multiply-accumulate high half, 8-bit */ -static int8_t do_sqrdmlah_b(int8_t src1, int8_t src2, int8_t src3, - bool neg, bool round) +int8_t do_sqrdmlah_b(int8_t src1, int8_t src2, int8_t src3, + bool neg, bool round) { /* * Simplify: @@ -82,8 +82,8 @@ void HELPER(sve2_sqrdmlsh_b)(void *vd, void *vn, void *vm, } /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */ -static int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, - bool neg, bool round, uint32_t *sat) +int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, + bool neg, bool round, uint32_t *sat) { /* Simplify similarly to do_sqrdmlah_b above. */ int32_t ret = (int32_t)src1 * src2; @@ -199,8 +199,8 @@ void HELPER(sve2_sqrdmlsh_h)(void *vd, void *vn, void *vm, } /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ -static int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, - bool neg, bool round, uint32_t *sat) +int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, + bool neg, bool round, uint32_t *sat) { /* Simplify similarly to do_sqrdmlah_b above. */ int64_t ret = (int64_t)src1 * src2; @@ -321,8 +321,7 @@ static int64_t do_sat128_d(Int128 r) return ls; } -static int64_t do_sqrdmlah_d(int64_t n, int64_t m, int64_t a, - bool neg, bool round) +int64_t do_sqrdmlah_d(int64_t n, int64_t m, int64_t a, bool neg, bool round) { uint64_t l, h; Int128 r, t; From patchwork Tue Mar 9 16:20:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396065 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2815054jai; Tue, 9 Mar 2021 10:22:13 -0800 (PST) X-Google-Smtp-Source: ABdhPJwWfD1epZvl57fqS/pihx6p9IkJh5ClgQSSLZG6S37KknRvGcW7bnk0q0/GbAweTvMaEU+U X-Received: by 2002:a05:6e02:1c0b:: with SMTP id l11mr24194587ilh.187.1615314133148; Tue, 09 Mar 2021 10:22:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314133; cv=none; d=google.com; s=arc-20160816; b=hd63hWiRbbNpDh297YOcE7UZmVReP4SFwLUqvgmoB3eRMlt8TyCxCAoGeatHwS4tBZ OOqb3Cel/nGTv79wM0zdLh0pi/3fVy2P3cmHATY9pRovYZyHxAUdbHPwgGId2YIrbTOJ H7iJfgId3b1Ei4ju65JeynFkukZkgf1pb5XM+3oGfi0nKFNPRtyuaioUv3vM5c4W/Usw eWR42dGOn3P1xPfNrDaYlXdWzPcd2kfWpJ7ZiNLJlh8+bgi0AD9xNoF13JOttC/Q/mn9 T/DZTILmivk1ZwMsFMxu/vDF//8MHWapfO4CZ7UX8Fn4Ry30rEhOlE0G3vMlWSs1lO11 KsYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=IBLMO5eVc4EnK/tNZqjKH+YJHmSPUKVvxUQrcUP5MI8=; b=QVxWfry1mRMM165ul4wqgZ9eEoaJRvKeEfsVEWG6jFGzjGYkqRpbUmk3cVYGj/9ajW BKwbl3DF/ezaDxj5gjhCEfwdD8EQDQ8zi9zdql+oiPbwhHdgiWCqU7unrgUbmTKu3Buh R+CaPLf2JXaC3WMZTtnBWDehWpkWrK+a0Mtjb3J0dkcu117OMZQe6ttsLoqHvCOsvhE6 EsTGaaGVlCwsQN8rAp+Sr66o8AXf3ezsiNA9BcKkmoEvba8HZ0ZwS3u1AtTCxpQ01wDN RkApg05UuWHyLcD0DapczcURJQ7XYchZEln66Rq6QRnK8wclfy0OwYJ9Z1/lfHJnDPtj 04Mg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RXSIVygG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v7si13403143ilu.80.2021.03.09.10.22.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:22:13 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RXSIVygG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44174 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgzk-0000lk-Er for patch@linaro.org; Tue, 09 Mar 2021 13:22:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46316) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7H-0007ux-6v for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:51 -0500 Received: from mail-ot1-x331.google.com ([2607:f8b0:4864:20::331]:41542) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6w-0002Yn-VY for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:50 -0500 Received: by mail-ot1-x331.google.com with SMTP id f8so8308635otp.8 for ; Tue, 09 Mar 2021 08:21:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IBLMO5eVc4EnK/tNZqjKH+YJHmSPUKVvxUQrcUP5MI8=; b=RXSIVygGg4iGOzB5iKHMhMSKEdQFqKnl996mj4LBAbmVqdy+WIh1V3fCPmX62H/6mF zvrfDdp/kjKDII5hy5sZz5MizJKWkxW/2reGoCOqrdzh/+uaOVHDdmYHWO4NYy/xiLjz cdMR/q8KuSBNSdPr7a48l7BAn9MZvilFxgyyIqEsRHaLmuX06jggOo6g18nsvDpAO7av SW0QEYKKxWLXQ8h58OutEYSFTC8dRN4AyH9k5g6bCAjLdck8k5ssCQFS6qxL7oRMgCc0 epdeQPigV2WKgItf221UyhDL4/a40AbpLMPECG3BrT/u3GrA6SD0PunAuhacfAU6N065 TF4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IBLMO5eVc4EnK/tNZqjKH+YJHmSPUKVvxUQrcUP5MI8=; b=mB1G2DB4pIdiGQaOm77cqvjg9KeQ8lIjEEauEsdTbim+Nmbypw8KdmC1gmOP2Htn/e rKo5Rq8oXzAnimnGdeaO7BO0NIKq9F3GU+sKAbFpfIA5y6QW8njH26VUtNqJ6jyEqYdQ H6z0xeXsbNez4jOiWexWyMuMphJ1K9A0EdFkMEH3tkVHvspP1GEXg9+uoGaJZnuMKGn2 W+yX1aYCyAXvb1Uj4HAruwtLj2vEoV2tpWg7k3uxSPA8jpAaiwTcwCXjA2Am/ErjZm53 cfUdrVLxljwpzxd/YCkQgH8tKibatuyWnLx7MlVTXz7YUN5WfGVFLnWDWQDWpdPqQk/e pgyQ== X-Gm-Message-State: AOAM532UBSwDFlTMJjSYAG+JsSm5p4xMLUI6wsc7N/o2pb0YOi5ttIfQ iEU6HDVGR/QgJbh79WRHWYl/IHf3lw2e09LS X-Received: by 2002:a9d:4e14:: with SMTP id p20mr16852972otf.94.1615306886661; Tue, 09 Mar 2021 08:21:26 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:26 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 38/78] target/arm: Implement SVE2 ADDHNB, ADDHNT Date: Tue, 9 Mar 2021 08:20:01 -0800 Message-Id: <20210309162041.23124-39-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::331; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x331.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-2-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 36 ++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 13 +++++++++++++ 4 files changed, 62 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index d154218452..a369fd2391 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2509,6 +2509,14 @@ DEF_HELPER_FLAGS_3(sve2_uqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_addhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 936977eacb..72dd36a5c8 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1320,6 +1320,11 @@ UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr +## SVE2 integer add/subtract narrow high part + +ADDHNB 01000101 .. 1 ..... 011 000 ..... ..... @rd_rn_rm +ADDHNT 01000101 .. 1 ..... 011 001 ..... ..... @rd_rn_rm + ### SVE2 Character Match MATCH 01000101 .. 1 ..... 100 ... ..... 0 .... @pd_pg_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 3bf81d96ae..2e2ad8fb09 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2112,6 +2112,42 @@ DO_SHRNT(sve2_uqrshrnt_d, uint64_t, uint32_t, , H1_4, DO_UQRSHRN_D) #undef DO_SHRNB #undef DO_SHRNT +#define DO_BINOPNB(NAME, TYPEW, TYPEN, SHIFT, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + i); \ + TYPEW mm = *(TYPEW *)(vm + i); \ + *(TYPEW *)(vd + i) = (TYPEN)OP(nn, mm, SHIFT); \ + } \ +} + +#define DO_BINOPNT(NAME, TYPEW, TYPEN, SHIFT, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + TYPEW mm = *(TYPEW *)(vm + HW(i)); \ + *(TYPEN *)(vd + HN(i + sizeof(TYPEN))) = OP(nn, mm, SHIFT); \ + } \ +} + +#define DO_ADDHN(N, M, SH) ((N + M) >> SH) + +DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) +DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) +DO_BINOPNB(sve2_addhnb_d, uint64_t, uint32_t, 32, DO_ADDHN) + +DO_BINOPNT(sve2_addhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_ADDHN) +DO_BINOPNT(sve2_addhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_ADDHN) +DO_BINOPNT(sve2_addhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_ADDHN) + +#undef DO_ADDHN + +#undef DO_BINOPNB + /* Fully general four-operand expander, controlled by a predicate. */ #define DO_ZPZZZ(NAME, TYPE, H, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 29d33389c2..931f1177a5 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7463,6 +7463,19 @@ static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +#define DO_SVE2_ZZZ_NARROW(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_h, \ + gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ + }; \ + return do_sve2_zzz_ool(s, a, fns[a->esz]); \ +} + +DO_SVE2_ZZZ_NARROW(ADDHNB, addhnb) +DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt) + static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) { From patchwork Tue Mar 9 16:20:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396048 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2800001jai; Tue, 9 Mar 2021 10:03:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJzmBhvrWHdN0dmAZ0z+MNpzeuju4mgsOD7Hzj4kqb3jRNiQ1DdY2wrNfpKJ5I8QolhtMrAB X-Received: by 2002:a67:401:: with SMTP id 1mr17246975vse.6.1615312987208; Tue, 09 Mar 2021 10:03:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312987; cv=none; d=google.com; s=arc-20160816; b=xhHI7S5r7d8CWNgs/Csr0iiZ8gta6MXoxCY1j/RH+ZNzU/ow/SC4/bxxYZL0ks+lhA rF9BBEwAGmbsNXYoOsX0joF/Ui4y4QDrTza1OxqJNM7bSdL+Bxga4mqMNSWEueGqQaAu gHJVh/QurWLehlNurdu87Xu+YBawbuT+qXJKgdVTatNCOnT/efzuzuBT+pQd8wvIwfoc ccRsSLNNWQXPThXpXfx1/ii7Y04vIwbavzOenP3bb4XWblhi3o0JY2K/87tITmptFiEY nU8pHInYJGqj1YwVdq+i9SllAxye7jnNRVkQFGK4JunEMARYCNfonaCEpVBm1Ibdn6+D b0Dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Z/S0kD58xUXjkywGS7Cu/nFSr7dKf86ON5tXYzK+tQE=; b=n+JJZm3IxY2n5UuhPu4+YP8/DMTOc1rWjUO5j7eHuxxu4eE+nkIpFNg6HTCF0KJB13 JojFb2jpdbF2r0hNqxxxpsh/mqV/rvgVwkWiOlPnFZeiLjjecsvyuEqVRBgms1TdOHoS Ckmiq7XPEt7olBkQAGunYykuGS04cJNtBhz5H1JBvIC5vliCCoSBLYYZqU5wUBidjyxl dsq5NMrZQLr5Sn5+A3sf38oMmYV+9wdfEK84bIsgoTB+s5VHP+sK+ojbCMDwvdoj3wqu r84ZmkLs+93vhZenpEbblGTJqCBZsK6loBM/JLkEF6asnYEDKolO5UwqfOvdNcMp4BOG t4mg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pEXGMtxa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z21si2600531uar.157.2021.03.09.10.03.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:03:07 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pEXGMtxa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59188 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJghG-0005kG-GQ for patch@linaro.org; Tue, 09 Mar 2021 13:03:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46324) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7H-0007vQ-Dp for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:51 -0500 Received: from mail-oi1-x230.google.com ([2607:f8b0:4864:20::230]:39505) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf6w-0002ZE-Vl for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:21:51 -0500 Received: by mail-oi1-x230.google.com with SMTP id z126so15525466oiz.6 for ; Tue, 09 Mar 2021 08:21:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z/S0kD58xUXjkywGS7Cu/nFSr7dKf86ON5tXYzK+tQE=; b=pEXGMtxaFB0INmat8EhJqD9dBV1gQg5n7Ji8nQryvwZHAzIZBw4TZqc8vBDhpJhKMA uneUtcvgQhv3sh3b9ZMYv3OS7CvthkHpKHJCCkDZrkiknrQk6ZpX1JKV6MOPXGLGw85C H4OEnJY6H96NG7MXdqfzEZaB+8s0GKDPcG6lXPnjSFDzdqdu0SilOIqYXEjPZKwQLTgU YGcoPeuvGdG+QddHxOB6upGuwuTCpskBuZwCaOK68UoGTrGCHpjNGaSBeZdd5IWiEAlx gKjbXGCUXHX9UHYJXfYi/cCjelITu7ySvy9QGztkPoBUb+O7HxjX3yRC2Y620HI1QMde /qZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z/S0kD58xUXjkywGS7Cu/nFSr7dKf86ON5tXYzK+tQE=; b=icGdAL8jhUBVgC4M15kbYhxT1svIChjG8GLr5y7wO81GILUw2nlFxkR0+GVg/FWveP X83OPxOjH9LgolCxL9xdugYqFfxuM4CpWwLwUD1GpIizlNYQW8k10/XK15Jep+z9kPiY Tj4nW4AdmtoQ/n+K7Hq0Wn7/4MsAtbO6tUiIo9eE3H87qeQyBp8M9bSMKwqNAx7C5o+/ 5C+HSONxJu5gc6FWbkFerRbXIzPd0U0BYcQ/I4JvR45VpOCUDdVhcBlvYFykAxgCZu7t tB4xe4rDcIDMlI8g6/53ZSq9afK08glTLxgBkFGITwGdisypsvpl4JRZssajbumQmgTU Tu7w== X-Gm-Message-State: AOAM532LjC4vBT7HOUtfjgPoyeL/lus8oBdRCGfP9vHN12lIy539r7fV uFMQwsQE8W2wX80cZ3bZIMJ3JLo9uw7FcIv3 X-Received: by 2002:aca:de82:: with SMTP id v124mr3483571oig.125.1615306887676; Tue, 09 Mar 2021 08:21:27 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id v23sm2516474ots.63.2021.03.09.08.21.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:27 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 39/78] target/arm: Implement SVE2 RADDHNB, RADDHNT Date: Tue, 9 Mar 2021 08:20:02 -0800 Message-Id: <20210309162041.23124-40-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::230; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x230.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-3-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fix round bit type (laurent desnogues) --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 10 ++++++++++ target/arm/translate-sve.c | 2 ++ 4 files changed, 22 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index a369fd2391..8d95c87694 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2517,6 +2517,14 @@ DEF_HELPER_FLAGS_4(sve2_addhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_addhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_addhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_raddhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 72dd36a5c8..dfcfab4bc0 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1324,6 +1324,8 @@ UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr ADDHNB 01000101 .. 1 ..... 011 000 ..... ..... @rd_rn_rm ADDHNT 01000101 .. 1 ..... 011 001 ..... ..... @rd_rn_rm +RADDHNB 01000101 .. 1 ..... 011 010 ..... ..... @rd_rn_rm +RADDHNT 01000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm ### SVE2 Character Match diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 2e2ad8fb09..8be0d28a8a 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2135,6 +2135,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ } #define DO_ADDHN(N, M, SH) ((N + M) >> SH) +#define DO_RADDHN(N, M, SH) ((N + M + ((__typeof(N))1 << (SH - 1))) >> SH) DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) @@ -2144,6 +2145,15 @@ DO_BINOPNT(sve2_addhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_ADDHN) DO_BINOPNT(sve2_addhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_ADDHN) DO_BINOPNT(sve2_addhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_ADDHN) +DO_BINOPNB(sve2_raddhnb_h, uint16_t, uint8_t, 8, DO_RADDHN) +DO_BINOPNB(sve2_raddhnb_s, uint32_t, uint16_t, 16, DO_RADDHN) +DO_BINOPNB(sve2_raddhnb_d, uint64_t, uint32_t, 32, DO_RADDHN) + +DO_BINOPNT(sve2_raddhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_RADDHN) +DO_BINOPNT(sve2_raddhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_RADDHN) +DO_BINOPNT(sve2_raddhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_RADDHN) + +#undef DO_RADDHN #undef DO_ADDHN #undef DO_BINOPNB diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 931f1177a5..375f75e086 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7475,6 +7475,8 @@ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ DO_SVE2_ZZZ_NARROW(ADDHNB, addhnb) DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt) +DO_SVE2_ZZZ_NARROW(RADDHNB, raddhnb) +DO_SVE2_ZZZ_NARROW(RADDHNT, raddhnt) static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) From patchwork Tue Mar 9 16:20:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396069 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2817580jai; Tue, 9 Mar 2021 10:25:39 -0800 (PST) X-Google-Smtp-Source: ABdhPJwkOOm2jv729MMbh2ThxPV84ACV+s8k5JeA8S/5CaMqsPmSaMH9Vqoh2JWq4iQSNsYia3NA X-Received: by 2002:a02:77d0:: with SMTP id g199mr29171265jac.118.1615314339009; Tue, 09 Mar 2021 10:25:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314339; cv=none; d=google.com; s=arc-20160816; b=DKxaj1CAG7xe3JsXv/XXmMg3SxUdswhhm+Kem3z4lOwSvcAF1N0IEa/w4/LMyWbyxn 3Eb5shBvZPKCpIkRW4PovqzsODGOhFjw4FAN7yKwO+5y7p5EG2yi8aJZg4lyidUB2+MK i0hCmJ9zq5avIqVMi66F2ZDhqLcLHDgfOBi2Ev1xkGJ6Gcg0w5s8dJUvZwjYWBnC1dUZ l+8vlUt0ZKO/kZQZ9ODXuBTFiyrTBDHE4FjlES87uUg66lss5vYTJVvG9zAbGMg66LHe q4sL73vC6sBimhQtijd+hQ1xAdiTnRyfxKM5sKqnodBXavDASrA2xnhuiZnrZxzjCzFf t7hQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=dmOaMq0cyoTud0OTLwGo1bmU+yPSQ++q9U9cjzeKa4M=; b=F/0/O9Od8pGKxr2zV78lUgr407v1rXU+f2W169SHahi4JkJvTIABs08o+tDrmXyhQM 4DdDvo7rh7ki4CGgORXYDrHstokq0DGFhxNYDwPk9zydASdgHtzXXZLnm5rYPtLmdkWD E3zEFifuFT8M3ShyzWWyZ/F9iJkTx44yZ0dvyecr4To3LsFbXERpwGXgNze447CqynKi 6/IGwwBqtkezz+xCwxNLjW9nWFiFMsFmmrXdNGGLfTs8QxgUAo1Xc1CWauZAOYjmIeiN 7m5skn2j07jQWOIAut61lMyGagU5w0jynczt1HugY8MdTc4jyB5E8ywe6WFhGzSSWit5 trVQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=u0x80t9C; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t14si17333418iov.85.2021.03.09.10.25.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:25:39 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=u0x80t9C; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52460 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJh34-0004Q1-Ay for patch@linaro.org; Tue, 09 Mar 2021 13:25:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46454) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7S-0008P1-EC for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:02 -0500 Received: from mail-ot1-x32d.google.com ([2607:f8b0:4864:20::32d]:38389) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7Q-0002jd-MQ for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:02 -0500 Received: by mail-ot1-x32d.google.com with SMTP id a17so13373028oto.5 for ; Tue, 09 Mar 2021 08:22:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dmOaMq0cyoTud0OTLwGo1bmU+yPSQ++q9U9cjzeKa4M=; b=u0x80t9C7FpPKMWMeYbn9liCw4tnVADINA+wbbvfWzhNdoP1sjDVexwTtt+SniWWl0 FKwM3cJ23RbR8/p0Qh1Qojuk0e6UOtDXGaamO67omqeXV++A1LurS2w5DjRwMsiKaJiH D725oJbMT9nBw7US8I3nyat+e95bKQdGqKUP8fOSCOMvN5buwxH0dMXEdy9nMelWORp0 5UJRT0VlHYP34mGHGbnTRh4OmF655jiGcv+MqD9JlSjqTEljzOFnrGwE8GXzQwAZVUsn zB8wjE4v2DMXrw4zWPxLOoeDklztLKcDzhzjQGR6chAh7jB4Pxd4691GofbTrwHk4+Eb iBjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dmOaMq0cyoTud0OTLwGo1bmU+yPSQ++q9U9cjzeKa4M=; b=J3Y3ZVis083RJ0cerH6WguMr5GHIYXGq9GGm1rTABrC9bq5a09Ba3MdqM7rLVoWxRC Xdq+8H+53/f5jBEoglHsnWHnEdiMZ5pikzmsbR7s/ybVkZwKpjulbwORs4Fs8jqj2r5i Qi+IZF03pqjajhKs7a8G3hMVv9UfZkPnm8Q5YGnTHaIxbReJR+bUnTqPOsSsZJ9A4PL8 NWlTpEoCX3Qq+aRwOPmQ3MjhLXp1MZeRbCSnNPEJfFQd/HGnvHrFArTDyEFdMexXSnTo F3vkLOOgRDWBtLW8d8MgiwPi7UmdyAOLeMQyzkzkKvYeRL6flf8bQcYN/tQbuCNlRt/u lHow== X-Gm-Message-State: AOAM531MlaorEprprbgEtDgmowujxPmd+DS0WYY1XysO5UN1K1ET2GW/ NRyPlMlkiGJO2RO8TcUETFhFi/vBpeflW5n5 X-Received: by 2002:a9d:750c:: with SMTP id r12mr9669622otk.322.1615306919692; Tue, 09 Mar 2021 08:21:59 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.21.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:21:59 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 40/78] target/arm: Implement SVE2 SUBHNB, SUBHNT Date: Tue, 9 Mar 2021 08:20:03 -0800 Message-Id: <20210309162041.23124-41-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32d; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-4-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 10 ++++++++++ target/arm/translate-sve.c | 3 +++ 4 files changed, 23 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 8d95c87694..3642e7c820 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2525,6 +2525,14 @@ DEF_HELPER_FLAGS_4(sve2_raddhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_raddhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_raddhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_subhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index dfcfab4bc0..c68bfcf6ed 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1326,6 +1326,8 @@ ADDHNB 01000101 .. 1 ..... 011 000 ..... ..... @rd_rn_rm ADDHNT 01000101 .. 1 ..... 011 001 ..... ..... @rd_rn_rm RADDHNB 01000101 .. 1 ..... 011 010 ..... ..... @rd_rn_rm RADDHNT 01000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm +SUBHNB 01000101 .. 1 ..... 011 100 ..... ..... @rd_rn_rm +SUBHNT 01000101 .. 1 ..... 011 101 ..... ..... @rd_rn_rm ### SVE2 Character Match diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 8be0d28a8a..229dde282e 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2136,6 +2136,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ #define DO_ADDHN(N, M, SH) ((N + M) >> SH) #define DO_RADDHN(N, M, SH) ((N + M + ((__typeof(N))1 << (SH - 1))) >> SH) +#define DO_SUBHN(N, M, SH) ((N - M) >> SH) DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) @@ -2153,6 +2154,15 @@ DO_BINOPNT(sve2_raddhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_RADDHN) DO_BINOPNT(sve2_raddhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_RADDHN) DO_BINOPNT(sve2_raddhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_RADDHN) +DO_BINOPNB(sve2_subhnb_h, uint16_t, uint8_t, 8, DO_SUBHN) +DO_BINOPNB(sve2_subhnb_s, uint32_t, uint16_t, 16, DO_SUBHN) +DO_BINOPNB(sve2_subhnb_d, uint64_t, uint32_t, 32, DO_SUBHN) + +DO_BINOPNT(sve2_subhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_SUBHN) +DO_BINOPNT(sve2_subhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_SUBHN) +DO_BINOPNT(sve2_subhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_SUBHN) + +#undef DO_SUBHN #undef DO_RADDHN #undef DO_ADDHN diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 375f75e086..792ec38cb0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7478,6 +7478,9 @@ DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt) DO_SVE2_ZZZ_NARROW(RADDHNB, raddhnb) DO_SVE2_ZZZ_NARROW(RADDHNT, raddhnt) +DO_SVE2_ZZZ_NARROW(SUBHNB, subhnb) +DO_SVE2_ZZZ_NARROW(SUBHNT, subhnt) + static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) { From patchwork Tue Mar 9 16:20:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396045 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2795325jai; Tue, 9 Mar 2021 09:57:22 -0800 (PST) X-Google-Smtp-Source: ABdhPJxyXZKV0R+gdsJn0MJPD3KShSWRy3ZEfemcTFGK+XksORyjKgFUscNmtUtn16vVXGU+nVFo X-Received: by 2002:a25:e08b:: with SMTP id x133mr41358215ybg.138.1615312642447; Tue, 09 Mar 2021 09:57:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312642; cv=none; d=google.com; s=arc-20160816; b=tMtVCHsRWfHsybBvAWOPEKlRtzKt22ChEUixhvC35lTkQCUxZuzBIKaAShAK8eom7A J9NARrcrUYb6TAQ/xsyOocT1eSOTsXJxoYZK2ppM+06Mfrk+zfGjWPjSYjg11MyexwNN fZ/rVLnH0+cwxCdaW7trQBQXEECwAyploTcU41JvLw06gYTe4TY3+LEF1KXgmCQUiajR WdQfhY+cBd5CL3cCyqhIOOR1Ouv+vB38r3s+zRRdx7Q9czjQPesvZjDcgto4XQK+CXUR HOPJgeU+cfDXvU8t72IzPCWx5+IbIxJtVTtiZJ0o0lfoGFWIet4Nb+cuqwFGsdTX+wAm fX1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Hd9WE+PagjBcQKfB5basSfMLdVZ3AAupsplglzUvPSg=; b=jrT+8sBvwRKaWQ4hoKSOw+h30Uc6J2acuHYs+gBtyi2XMwN5o3NrX+wc0HGFEF/7Lc c3xR1dUkoA18aV1//Pyfr0hujavpPZ9yfPEh8KNkKOqpvTnr3NCF0hf7ooAQALGyh3Fm 6RsqhF6QpiXFj9KRg5lz8gZBUU/OQUWr58YgA+v+QDVtd7aCtrV5zUGdAcIr1W79OFfn M0c1lHw+YjGDj1kCy71Jp8drDo5NCFXPf82haBUFEHSIRsEtv0TikA19bfLRmeSd/QfZ lXssMcOm40uDnT7mN96ySWTFTXZYRSaXKEd83NcJHLwnoPto4va9awgQgTuJYjkuMFV1 t0RA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wkzuxtEK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f130si16848147ybg.79.2021.03.09.09.57.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:57:22 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wkzuxtEK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:48912 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgbh-0001Id-Or for patch@linaro.org; Tue, 09 Mar 2021 12:57:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46582) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7W-000088-Vl for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:07 -0500 Received: from mail-oi1-x231.google.com ([2607:f8b0:4864:20::231]:41373) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7R-0002jl-T8 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:06 -0500 Received: by mail-oi1-x231.google.com with SMTP id y131so12657590oia.8 for ; Tue, 09 Mar 2021 08:22:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Hd9WE+PagjBcQKfB5basSfMLdVZ3AAupsplglzUvPSg=; b=wkzuxtEK7luGbXzArxop8L2YAqXPwJj30XO+XdjUasAVq9Zq8zR2CA4DrYaD30uRSP /6BhRwl68BcM+mA+zgQL89lAZG8BZcCbhKN+ZXCrxrioWFf9aXZTxqiJw5EXSO7WZCi1 BABFSH+NxZVpaUW+vOPXSF5O9T6EaSdYjK51VyApJFGenlPK6iTnFkRYl4n6s/FDyxRf a426mkfwpVG91J8VLa+VxyYTVhRHP1nSfkY9EaXt9suUBOBf9iqImbuT3Lzc+Z3zFAVR H+sMeYAOXlBBzAP7FF62YFG8Bq2Ker/ip1zqKTjNX+adHOfeYmX0PZ4Pva3gLcktA8qC Y7Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Hd9WE+PagjBcQKfB5basSfMLdVZ3AAupsplglzUvPSg=; b=B8Fh8tL+NV2hox8OGKqvWvaja0zGUoOEyN+/JkSdz779vQVfIqUdZJvJ9eWdao0K4I og/r6U0cZPhZMlzLKY5j8nFJS4ExePeyRZkaKG6GeZtKggUFISlmzU4rcsCJ/Tfi5uP0 1GaWwflYBdsZV5qn4RyibQZtR1PPVsk+m+LrPKkLiVaUOtQRFy3/BScbmMq1MErEt+/X DiwMojj1qVSmgLfzRCoUy8WXA1Z9kxdw7A4HtCpQ+kuPSfsX7IJiVQUaeVef3bPHJ5UX lY17mKGtrsmKE+Um3YUnsA/VcAX9+8BoCdH71miB9Fu9SI6WTKoP2zQQtcwOHXQ6kHOQ gFrQ== X-Gm-Message-State: AOAM531EO7qSVHTdCGfXZly7z5dUvdqFyWSjyHo0t1TJJIBPd/n1B9Jh uaqAkfDmMCtHwfHbY3VQ3QsDIitb3IaZZnjT X-Received: by 2002:a05:6808:148a:: with SMTP id e10mr3145299oiw.138.1615306920747; Tue, 09 Mar 2021 08:22:00 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.21.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:00 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 41/78] target/arm: Implement SVE2 RSUBHNB, RSUBHNT Date: Tue, 9 Mar 2021 08:20:04 -0800 Message-Id: <20210309162041.23124-42-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::231; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x231.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long This completes the section 'SVE2 integer add/subtract narrow high part' Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-5-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fix round bit type (laurent desnogues) --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 10 ++++++++++ target/arm/translate-sve.c | 2 ++ 4 files changed, 22 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 3642e7c820..98e6b57e38 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2533,6 +2533,14 @@ DEF_HELPER_FLAGS_4(sve2_subhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_subhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_subhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_rsubhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index c68bfcf6ed..388bf92acf 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1328,6 +1328,8 @@ RADDHNB 01000101 .. 1 ..... 011 010 ..... ..... @rd_rn_rm RADDHNT 01000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm SUBHNB 01000101 .. 1 ..... 011 100 ..... ..... @rd_rn_rm SUBHNT 01000101 .. 1 ..... 011 101 ..... ..... @rd_rn_rm +RSUBHNB 01000101 .. 1 ..... 011 110 ..... ..... @rd_rn_rm +RSUBHNT 01000101 .. 1 ..... 011 111 ..... ..... @rd_rn_rm ### SVE2 Character Match diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 229dde282e..860ab564cd 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2137,6 +2137,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ #define DO_ADDHN(N, M, SH) ((N + M) >> SH) #define DO_RADDHN(N, M, SH) ((N + M + ((__typeof(N))1 << (SH - 1))) >> SH) #define DO_SUBHN(N, M, SH) ((N - M) >> SH) +#define DO_RSUBHN(N, M, SH) ((N - M + ((__typeof(N))1 << (SH - 1))) >> SH) DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) @@ -2162,6 +2163,15 @@ DO_BINOPNT(sve2_subhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_SUBHN) DO_BINOPNT(sve2_subhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_SUBHN) DO_BINOPNT(sve2_subhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_SUBHN) +DO_BINOPNB(sve2_rsubhnb_h, uint16_t, uint8_t, 8, DO_RSUBHN) +DO_BINOPNB(sve2_rsubhnb_s, uint32_t, uint16_t, 16, DO_RSUBHN) +DO_BINOPNB(sve2_rsubhnb_d, uint64_t, uint32_t, 32, DO_RSUBHN) + +DO_BINOPNT(sve2_rsubhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_RSUBHN) +DO_BINOPNT(sve2_rsubhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_RSUBHN) +DO_BINOPNT(sve2_rsubhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_RSUBHN) + +#undef DO_RSUBHN #undef DO_SUBHN #undef DO_RADDHN #undef DO_ADDHN diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 792ec38cb0..08622432bd 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7480,6 +7480,8 @@ DO_SVE2_ZZZ_NARROW(RADDHNT, raddhnt) DO_SVE2_ZZZ_NARROW(SUBHNB, subhnb) DO_SVE2_ZZZ_NARROW(SUBHNT, subhnt) +DO_SVE2_ZZZ_NARROW(RSUBHNB, rsubhnb) +DO_SVE2_ZZZ_NARROW(RSUBHNT, rsubhnt) static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) From patchwork Tue Mar 9 16:20:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396035 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2788146jai; Tue, 9 Mar 2021 09:47:00 -0800 (PST) X-Google-Smtp-Source: ABdhPJyINybUmFp8PVwLFkHxSy+WjqP1XFiNYzAblMIUmZdsSdRyfSq210Oha5BjiWWIe3px5o/n X-Received: by 2002:a92:cd8a:: with SMTP id r10mr24800110ilb.110.1615312020456; Tue, 09 Mar 2021 09:47:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312020; cv=none; d=google.com; s=arc-20160816; b=eCpn4LCzpZocRGpNsfQbB5rznwcygApo7WZa0Frhql3upvszXzx77YimJAR8Gy3PfH QS33YTHfylSxXNlbjXHIdYEuxDgk1FuR4rs+lXphvHtnAr7n6nAimDhjnFxF6JbvdVXJ TcLcHuDdBlTguurFoNqyiuRcaFGjlaVEZwmPpr0YhxuVhAIFKV5zUgMsLi25szIad7Yt 54zz225tSNowoJirAHDkorrvkLAqND+rB02DlCXU9nMz+pajd5iPCyCm9/3rwYEVNYPd OFtu+OYbPfZAIUXZ75KzI5hSuVJwi+vpR3iPa9fdzk/zTaquqtTmw2n7XDBv8LAZHxfi PxrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=3y/+qgHE/0Io69b/bOTYSEDVvU6QnkuY6CtVyDU1GPc=; b=M/nTU+VZiND/ZMXGrlXSxzEaLDGBq7g5jqIYPqLZX39h5o0a2hQNNCGt5w2kFa1vtf nEQSAfv5wnyfvV3iNRle1YfsOS6fXhKweTb3dbc1fPm95JapjJk4aotgK+RlIovSmzdN NASyd5/SZDVbMUH0D3ez8i68wdR/jdPg9fteP5fXU65DyfuEUJ9Z8ZGfYwE8E4le2JrE qid6YQUh65OPLQd6ZC4ou0tZ4YwwQvPbuaDrH+rsUZj1oD6ZFnnVE1RrlpRBTPbzWupp arx2EjZrI/2Cczs09zB1rcaInAAjOhKb1M6ilOi2Z7MIYKtfNpAxjxGM5FrkmHVaNf5t S0Ow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wDbTdXvO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i7si13617682ilu.26.2021.03.09.09.47.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:47:00 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wDbTdXvO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53102 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgRf-0007tI-RF for patch@linaro.org; Tue, 09 Mar 2021 12:46:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46546) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7V-0008Vr-GL for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:05 -0500 Received: from mail-oi1-x233.google.com ([2607:f8b0:4864:20::233]:39513) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7T-0002k1-3h for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:05 -0500 Received: by mail-oi1-x233.google.com with SMTP id z126so15527711oiz.6 for ; Tue, 09 Mar 2021 08:22:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3y/+qgHE/0Io69b/bOTYSEDVvU6QnkuY6CtVyDU1GPc=; b=wDbTdXvOOhZeYcquoZj0/bE/nMn7dPcsxVVPwkMxZ02AHEn/qNW4j/OU070swwW1it YxkHw5Wd+ym8MREhgjOe8APFg8MTS+NLp/WfrC6KwjJtvIy6dTpiLt7LAvI5geS3f6gV 6bWuDU18OkzGshroaxSH3pcqs02HGoVruEiaQHyWaPjo82w8UhOwVCBIRUTQa06noCy6 cwHI8zHd9JEpxg9VDumnk+2j8bgPCUDZ98AeX1IbMy/u9lvrFbcSAI4SXDTDfZA6FCPH ypA9hYYhhu0hckH7UHOSIY4wbC0SLJYfCmL3hJv0OUYiBinqqZe6t9hNaWqi0AVhcF+6 O4dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3y/+qgHE/0Io69b/bOTYSEDVvU6QnkuY6CtVyDU1GPc=; b=EyB+gmBK8+qy1bt+eQbfO1ITHOC8t1y/mRRWwovIiPkhpSf6Ij/CwYwbC070iG2WC3 Q87g87XvIH95mYVkZlGfvrzjfHT0cN0A1JoFIBOAYDq0L4FPE7XuoYh84r6Z9N3Tzere 7WInqXK0d9exNi3bYl5QelIstisGjDVPzLbIwvzodOQMwPRLtcXp99H6jqLlu7zUqIup Pjd8GFkFaEFIvGCfa4hzjzcRSDs4+MvC0uAiI/iWG7MLBTZvgNnZ8VN4uJXIHCgumtEF oSRgxFu5hhnZ0Ik7CANga+4KpwyMDDuNDU9p7liTWAbm8tImPsFVbQOspMsiyW4JsXKv cVXw== X-Gm-Message-State: AOAM531vF+u0ZzU2vYvCCj7xcJXKMtJUIYyxqRxlGeSpWRmwcDsR79Hp lDsp132BYiQDArPY0oyX5PLkhfjC0OofazEd X-Received: by 2002:a05:6808:114b:: with SMTP id u11mr3386371oiu.52.1615306922117; Tue, 09 Mar 2021 08:22:02 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:01 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 42/78] target/arm: Implement SVE2 HISTCNT, HISTSEG Date: Tue, 9 Mar 2021 08:20:05 -0800 Message-Id: <20210309162041.23124-43-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::233; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x233.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200416173109.8856-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fix overlap between output and input vectors. v4: Fix histseg counting (zhiwei). --- target/arm/helper-sve.h | 7 ++ target/arm/sve.decode | 6 ++ target/arm/sve_helper.c | 131 +++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 19 ++++++ 4 files changed, 163 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 98e6b57e38..507a2fea8e 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2551,6 +2551,13 @@ DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_b, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_h, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_histcnt_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_histcnt_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_histseg, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 388bf92acf..8f501a083c 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -146,6 +146,7 @@ &rprrr_esz rn=%reg_movprfx @rdn_pg_rm_ra ........ esz:2 . ra:5 ... pg:3 rm:5 rd:5 \ &rprrr_esz rn=%reg_movprfx +@rd_pg_rn_rm ........ esz:2 . rm:5 ... pg:3 rn:5 rd:5 &rprr_esz # One register operand, with governing predicate, vector element size @rd_pg_rn ........ esz:2 ... ... ... pg:3 rn:5 rd:5 &rpr_esz @@ -1336,6 +1337,11 @@ RSUBHNT 01000101 .. 1 ..... 011 111 ..... ..... @rd_rn_rm MATCH 01000101 .. 1 ..... 100 ... ..... 0 .... @pd_pg_rn_rm NMATCH 01000101 .. 1 ..... 100 ... ..... 1 .... @pd_pg_rn_rm +### SVE2 Histogram Computation + +HISTCNT 01000101 .. 1 ..... 110 ... ..... ..... @rd_pg_rn_rm +HISTSEG 01000101 .. 1 ..... 101 000 ..... ..... @rd_rn_rm + ## SVE2 floating-point pairwise operations FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 860ab564cd..88a5e64277 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7094,3 +7094,134 @@ DO_PPZZ_MATCH(sve2_nmatch_ppzz_b, MO_8, true) DO_PPZZ_MATCH(sve2_nmatch_ppzz_h, MO_16, true) #undef DO_PPZZ_MATCH + +void HELPER(sve2_histcnt_s)(void *vd, void *vn, void *vm, void *vg, + uint32_t desc) +{ + ARMVectorReg scratch; + intptr_t i, j; + intptr_t opr_sz = simd_oprsz(desc); + uint32_t *d = vd, *n = vn, *m = vm; + uint8_t *pg = vg; + + if (d == n) { + n = memcpy(&scratch, n, opr_sz); + if (d == m) { + m = n; + } + } else if (d == m) { + m = memcpy(&scratch, m, opr_sz); + } + + for (i = 0; i < opr_sz; i += 4) { + uint64_t count = 0; + uint8_t pred; + + pred = pg[H1(i >> 3)] >> (i & 7); + if (pred & 1) { + uint32_t nn = n[H4(i >> 2)]; + + for (j = 0; j <= i; j += 4) { + pred = pg[H1(j >> 3)] >> (j & 7); + if ((pred & 1) && nn == m[H4(j >> 2)]) { + ++count; + } + } + } + d[H4(i >> 2)] = count; + } +} + +void HELPER(sve2_histcnt_d)(void *vd, void *vn, void *vm, void *vg, + uint32_t desc) +{ + ARMVectorReg scratch; + intptr_t i, j; + intptr_t opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn, *m = vm; + uint8_t *pg = vg; + + if (d == n) { + n = memcpy(&scratch, n, opr_sz); + if (d == m) { + m = n; + } + } else if (d == m) { + m = memcpy(&scratch, m, opr_sz); + } + + for (i = 0; i < opr_sz / 8; ++i) { + uint64_t count = 0; + if (pg[H1(i)] & 1) { + uint64_t nn = n[i]; + for (j = 0; j <= i; ++j) { + if ((pg[H1(j)] & 1) && nn == m[j]) { + ++count; + } + } + } + d[i] = count; + } +} + +/* + * Returns the number of bytes in m0 and m1 that match n. + * Unlike do_match2 we don't just need true/false, we need an exact count. + * This requires two extra logical operations. + */ +static inline uint64_t do_histseg_cnt(uint8_t n, uint64_t m0, uint64_t m1) +{ + const uint64_t mask = dup_const(MO_8, 0x7f); + uint64_t cmp0, cmp1; + + cmp1 = dup_const(MO_8, n); + cmp0 = cmp1 ^ m0; + cmp1 = cmp1 ^ m1; + + /* + * 1: clear msb of each byte to avoid carry to next byte (& mask) + * 2: carry in to msb if byte != 0 (+ mask) + * 3: set msb if cmp has msb set (| cmp) + * 4: set ~msb to ignore them (| mask) + * We now have 0xff for byte != 0 or 0x7f for byte == 0. + * 5: invert, resulting in 0x80 if and only if byte == 0. + */ + cmp0 = ~(((cmp0 & mask) + mask) | cmp0 | mask); + cmp1 = ~(((cmp1 & mask) + mask) | cmp1 | mask); + + /* + * Combine the two compares in a way that the bits do + * not overlap, and so preserves the count of set bits. + * If the host has an efficient instruction for ctpop, + * then ctpop(x) + ctpop(y) has the same number of + * operations as ctpop(x | (y >> 1)). If the host does + * not have an efficient ctpop, then we only want to + * use it once. + */ + return ctpop64(cmp0 | (cmp1 >> 1)); +} + +void HELPER(sve2_histseg)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j; + intptr_t opr_sz = simd_oprsz(desc); + + for (i = 0; i < opr_sz; i += 16) { + uint64_t n0 = *(uint64_t *)(vn + i); + uint64_t m0 = *(uint64_t *)(vm + i); + uint64_t n1 = *(uint64_t *)(vn + i + 8); + uint64_t m1 = *(uint64_t *)(vm + i + 8); + uint64_t out0 = 0; + uint64_t out1 = 0; + + for (j = 0; j < 64; j += 8) { + uint64_t cnt0 = do_histseg_cnt(n0 >> j, m0, m1); + uint64_t cnt1 = do_histseg_cnt(n1 >> j, m0, m1); + out0 |= cnt0 << j; + out1 |= cnt1 << j; + } + + *(uint64_t *)(vd + i) = out0; + *(uint64_t *)(vd + i + 8) = out1; + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 08622432bd..acda88cb2c 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7505,6 +7505,25 @@ static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ DO_SVE2_PPZZ_MATCH(MATCH, match) DO_SVE2_PPZZ_MATCH(NMATCH, nmatch) +static bool trans_HISTCNT(DisasContext *s, arg_rprr_esz *a) +{ + static gen_helper_gvec_4 * const fns[2] = { + gen_helper_sve2_histcnt_s, gen_helper_sve2_histcnt_d + }; + if (a->esz < 2) { + return false; + } + return do_sve2_zpzz_ool(s, a, fns[a->esz - 2]); +} + +static bool trans_HISTSEG(DisasContext *s, arg_rrr_esz *a) +{ + if (a->esz != 0) { + return false; + } + return do_sve2_zzz_ool(s, a, gen_helper_sve2_histseg); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Tue Mar 9 16:20:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396047 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2796925jai; Tue, 9 Mar 2021 09:59:45 -0800 (PST) X-Google-Smtp-Source: ABdhPJzWCtiNxZ0IIwi4PeRvBQQpiSPfzPRMr5BaBjzDqGnNDE/QHKkAg1S14En9awurDYcY+ej6 X-Received: by 2002:a37:bc8:: with SMTP id 191mr26378503qkl.76.1615312784890; Tue, 09 Mar 2021 09:59:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312784; cv=none; d=google.com; s=arc-20160816; b=ztHW6feEpI+4kGTWhvhjFxRJ+vH1tkLdGouZ34+Tn9iQ7b87r75Rt6W/VWJxU7yW99 T3qGQ7G6UG4mFRd6VV7FmUNzsTGbf4ngafJBK+HfBlPSUuCI3k7JAI3h2bFn4pY0rEly yzrW3QO07f3FciZXr6fwjlpPki+qeG4D0sUJ/TYHYNbWMuIjuXYfxs3iAjNTru6E92Sk P6dU3RLrlzKGqsgD2NGlXyilWHqwAIe/NZq6V0SkcXenH0FD+qRrHHz+Tzpetxqti2X+ kFQ8QgSAkvOx7nLHp7eNR90fbYgCjHaOmApsCtVK1XnHJjHcuTNaPva9Q9So1tUblnlU AVlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=adanJyUxqOWc9Dhq8EIk3bW7dNNKqybJfKCeN5b9hiE=; b=WeJjKf5bQkwbMzAi170G03jsXcI316/8X+4tKy/8zhO9PhNfEqCT6+eTjUph4z/zOT 13I5CtRqmNw8G6oDnx5hLJEmmJHSICqz5hbDDcLzuGikU7ouxFgW+jcPjtKpmrowosog xnaxq84PALR2z8QfiJKDoJ9ser0Qf4pOJSnCmMyBOoll8/tYOKaEdlclftLWqRQtpW+t E5AAvF3GIEMVvRJElib5tbC5XNXBCFujf3QOH0y55J0sEkBOAud/49L5IMxU5YTrgkSI j3kw4EG9hym2GWGlwN2fPe8k0Mx64IzrsVLKBaZy2bhHQlHBjrbURw8mvv+Qy7ciH3EL L7zw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rRzMroVD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f14si8564576qvd.200.2021.03.09.09.59.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:59:44 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=rRzMroVD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57378 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJge0-0004uA-7x for patch@linaro.org; Tue, 09 Mar 2021 12:59:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46620) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7Y-0000Ay-1R for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:08 -0500 Received: from mail-ot1-x32e.google.com ([2607:f8b0:4864:20::32e]:34753) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7U-0002kF-6e for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:07 -0500 Received: by mail-ot1-x32e.google.com with SMTP id n23so7731666otq.1 for ; Tue, 09 Mar 2021 08:22:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=adanJyUxqOWc9Dhq8EIk3bW7dNNKqybJfKCeN5b9hiE=; b=rRzMroVDH+kXWy/hu6PDgmO5Gc6ZYGbJznpGLle8WV9D9FkNySPcyOIFR+vM1BT8MI vEgiPlqczHEXRdn0UnRCDiGNVALrC+py3kMg989Qfx6geFW4oOZAvT5wfshuJe7z2VRi kMQ0OTg6DqCvhCh+hfXn5NcFRcIyfDji+ZAuRd8XrVM8NEHtWcifSQPkdcjuFzbWMTcf 4h8gjpE/lSL8I6tUtW1J16YpStq1HsR/0qudmc4+TGPzdu+YCpMCOiPWlnmcUkl9cIct b0hxellvspd1qrHMTEMW7Qi17tu7faxpKSuAnNxMiTPvldDAQRP3bDwz32blqnIQTXFc 5eeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=adanJyUxqOWc9Dhq8EIk3bW7dNNKqybJfKCeN5b9hiE=; b=gaCtN9aZwXAgv6ajo7qrfRlPSj0HupkgludFdBPUsoUNxx0M36swTckOmjacD0I5iy d+tBNEMcrHwJptLvNnelDkbUJpMSNBwkpbuO8aDAnUH9y89FvRK89Dv6qSlTjtNtQUSi jfLacJ4fOLqvEu4UyQfOfn9D9UYkUf98woI24oxyPCvHthTLWgeSIuncvta9bSa00rDx /ilDwToeqpJGIUEFvXZYPNBd4N79QgxDQNiqePgVyaDs8i780iX6kpxXuZGDekmlVN4g MWUo6IrMugbq1VI0rgQaKSIZ3jG8j0tOPcGvHzkqZGaSuAaeug+mx1niwEvPRhOv8wfo lEww== X-Gm-Message-State: AOAM530COxIKLUfDYnmWKnMj0Ov3LsR5DKEj5XhrumC7aSOLt+zza8WP svyFISWh/1t9pB3cg3JOLve7+xC5g5cTh4y/ X-Received: by 2002:a05:6830:1304:: with SMTP id p4mr24459350otq.185.1615306923128; Tue, 09 Mar 2021 08:22:03 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:02 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 43/78] target/arm: Implement SVE2 XAR Date: Tue, 9 Mar 2021 08:20:06 -0800 Message-Id: <20210309162041.23124-44-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32e; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" In addition, use the same vector generator interface for AdvSIMD. This fixes a bug in which the AdvSIMD insn failed to clear the high bits of the SVE register. Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 4 ++ target/arm/helper.h | 2 + target/arm/translate-a64.h | 3 ++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 39 ++++++++++++++ target/arm/translate-a64.c | 25 ++------- target/arm/translate-sve.c | 104 +++++++++++++++++++++++++++++++++++++ target/arm/vec_helper.c | 12 +++++ 8 files changed, 172 insertions(+), 21 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 507a2fea8e..28b8f00201 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2558,6 +2558,10 @@ DEF_HELPER_FLAGS_5(sve2_histcnt_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_4(sve2_histseg, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_xar_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_xar_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_xar_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/helper.h b/target/arm/helper.h index 6bb0b0ddc0..23a7ec5638 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -953,6 +953,8 @@ DEF_HELPER_FLAGS_5(neon_sqrdmulh_h, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(neon_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + #ifdef TARGET_AARCH64 #include "helper-a64.h" #include "helper-sve.h" diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h index 3668b671dd..cdf2b2d1be 100644 --- a/target/arm/translate-a64.h +++ b/target/arm/translate-a64.h @@ -122,5 +122,8 @@ bool disas_sve(DisasContext *, uint32_t); void gen_gvec_rax1(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_xar(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t rm_ofs, int64_t shift, + uint32_t opr_sz, uint32_t max_sz); #endif /* TARGET_ARM_TRANSLATE_A64_H */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 8f501a083c..7645587469 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -65,6 +65,7 @@ &rr_dbm rd rn dbm &rrri rd rn rm imm &rri_esz rd rn imm esz +&rrri_esz rd rn rm imm esz &rrr_esz rd rn rm esz &rpr_esz rd pg rn esz &rpr_s rd pg rn s @@ -384,6 +385,9 @@ ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +XAR 00000100 .. 1 ..... 001 101 rm:5 rd:5 &rrri_esz \ + rn=%reg_movprfx esz=%tszimm16_esz imm=%tszimm16_shr + # SVE2 bitwise ternary operations EOR3 00000100 00 1 ..... 001 110 ..... ..... @rdn_ra_rm_e0 BSL 00000100 00 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 88a5e64277..0a02deef70 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7225,3 +7225,42 @@ void HELPER(sve2_histseg)(void *vd, void *vn, void *vm, uint32_t desc) *(uint64_t *)(vd + i + 8) = out1; } } + +void HELPER(sve2_xar_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + int shr = simd_data(desc); + int shl = 8 - shr; + uint64_t mask = dup_const(MO_8, 0xff >> shr); + uint64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + uint64_t t = n[i] ^ m[i]; + d[i] = ((t >> shr) & mask) | ((t << shl) & ~mask); + } +} + +void HELPER(sve2_xar_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + int shr = simd_data(desc); + int shl = 16 - shr; + uint64_t mask = dup_const(MO_16, 0xffff >> shr); + uint64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + uint64_t t = n[i] ^ m[i]; + d[i] = ((t >> shr) & mask) | ((t << shl) & ~mask); + } +} + +void HELPER(sve2_xar_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 4; + int shr = simd_data(desc); + uint32_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ror32(n[i] ^ m[i], shr); + } +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index b591f096df..491519fa6c 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -14355,8 +14355,6 @@ static void disas_crypto_xar(DisasContext *s, uint32_t insn) int imm6 = extract32(insn, 10, 6); int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); - TCGv_i64 tcg_op1, tcg_op2, tcg_res[2]; - int pass; if (!dc_isar_feature(aa64_sha3, s)) { unallocated_encoding(s); @@ -14367,25 +14365,10 @@ static void disas_crypto_xar(DisasContext *s, uint32_t insn) return; } - tcg_op1 = tcg_temp_new_i64(); - tcg_op2 = tcg_temp_new_i64(); - tcg_res[0] = tcg_temp_new_i64(); - tcg_res[1] = tcg_temp_new_i64(); - - for (pass = 0; pass < 2; pass++) { - read_vec_element(s, tcg_op1, rn, pass, MO_64); - read_vec_element(s, tcg_op2, rm, pass, MO_64); - - tcg_gen_xor_i64(tcg_res[pass], tcg_op1, tcg_op2); - tcg_gen_rotri_i64(tcg_res[pass], tcg_res[pass], imm6); - } - write_vec_element(s, tcg_res[0], rd, 0, MO_64); - write_vec_element(s, tcg_res[1], rd, 1, MO_64); - - tcg_temp_free_i64(tcg_op1); - tcg_temp_free_i64(tcg_op2); - tcg_temp_free_i64(tcg_res[0]); - tcg_temp_free_i64(tcg_res[1]); + gen_gvec_xar(MO_64, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), imm6, 16, + vec_full_reg_size(s)); } /* Crypto three-reg imm2 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index acda88cb2c..cad61f4577 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -340,6 +340,110 @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a) return do_zzz_fn(s, a, tcg_gen_gvec_andc); } +static void gen_xar8_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) +{ + TCGv_i64 t = tcg_temp_new_i64(); + uint64_t mask = dup_const(MO_8, 0xff >> sh); + + tcg_gen_xor_i64(t, n, m); + tcg_gen_shri_i64(d, t, sh); + tcg_gen_shli_i64(t, t, 8 - sh); + tcg_gen_andi_i64(d, d, mask); + tcg_gen_andi_i64(t, t, ~mask); + tcg_gen_or_i64(d, d, t); + tcg_temp_free_i64(t); +} + +static void gen_xar16_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) +{ + TCGv_i64 t = tcg_temp_new_i64(); + uint64_t mask = dup_const(MO_16, 0xffff >> sh); + + tcg_gen_xor_i64(t, n, m); + tcg_gen_shri_i64(d, t, sh); + tcg_gen_shli_i64(t, t, 16 - sh); + tcg_gen_andi_i64(d, d, mask); + tcg_gen_andi_i64(t, t, ~mask); + tcg_gen_or_i64(d, d, t); + tcg_temp_free_i64(t); +} + +static void gen_xar_i32(TCGv_i32 d, TCGv_i32 n, TCGv_i32 m, int32_t sh) +{ + tcg_gen_xor_i32(d, n, m); + tcg_gen_rotri_i32(d, d, sh); +} + +static void gen_xar_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) +{ + tcg_gen_xor_i64(d, n, m); + tcg_gen_rotri_i64(d, d, sh); +} + +static void gen_xar_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, int64_t sh) +{ + tcg_gen_xor_vec(vece, d, n, m); + tcg_gen_rotri_vec(vece, d, d, sh); +} + +void gen_gvec_xar(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t rm_ofs, int64_t shift, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop[] = { INDEX_op_rotli_vec, 0 }; + static const GVecGen3i ops[4] = { + { .fni8 = gen_xar8_i64, + .fniv = gen_xar_vec, + .fno = gen_helper_sve2_xar_b, + .opt_opc = vecop, + .vece = MO_8 }, + { .fni8 = gen_xar16_i64, + .fniv = gen_xar_vec, + .fno = gen_helper_sve2_xar_h, + .opt_opc = vecop, + .vece = MO_16 }, + { .fni4 = gen_xar_i32, + .fniv = gen_xar_vec, + .fno = gen_helper_sve2_xar_s, + .opt_opc = vecop, + .vece = MO_32 }, + { .fni8 = gen_xar_i64, + .fniv = gen_xar_vec, + .fno = gen_helper_gvec_xar_d, + .opt_opc = vecop, + .vece = MO_64 } + }; + int esize = 8 << vece; + + /* The SVE2 range is 1 .. esize; the AdvSIMD range is 0 .. esize-1. */ + tcg_debug_assert(shift >= 0); + tcg_debug_assert(shift <= esize); + shift &= esize - 1; + + if (shift == 0) { + /* xar with no rotate devolves to xor. */ + tcg_gen_gvec_xor(vece, rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz); + } else { + tcg_gen_gvec_3i(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, + shift, &ops[vece]); + } +} + +static bool trans_XAR(DisasContext *s, arg_rrri_esz *a) +{ + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + gen_gvec_xar(a->esz, vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), a->imm, vsz, vsz); + } + return true; +} + static bool do_sve2_zzzz_fn(DisasContext *s, arg_rrrr_esz *a, GVecGen4Fn *fn) { if (!dc_isar_feature(aa64_sve2, s)) { diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 19006f50f7..a3d80ecad0 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -2253,3 +2253,15 @@ void HELPER(gvec_umulh_d)(void *vd, void *vn, void *vm, uint32_t desc) } clear_tail(d, opr_sz, simd_maxsz(desc)); } + +void HELPER(gvec_xar_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + int shr = simd_data(desc); + uint64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ror64(n[i] ^ m[i], shr); + } + clear_tail(d, opr_sz * 8, simd_maxsz(desc)); +} From patchwork Tue Mar 9 16:20:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396052 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2805516jai; Tue, 9 Mar 2021 10:09:24 -0800 (PST) X-Google-Smtp-Source: ABdhPJzKlgBk9knNBvLo8cKrs4nQZtJt7LdgXzKS8VgQ01MHSR48UenpQ7ok6KGzEOFTuSUgO+fG X-Received: by 2002:a6b:c881:: with SMTP id y123mr23056137iof.152.1615313364010; Tue, 09 Mar 2021 10:09:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313364; cv=none; d=google.com; s=arc-20160816; b=wFAKdEoSaYPW8KI5JrBYaY3wUw56B2bEdzHnuEONkLaQQ4crDIJFdgmdlFOhGKeeyu dz2Xqz+S75GZvCtC1YR4Tax1wocOt3IZg50wXOxFm1VV1frO18aeJxgluuqwITWxlCKU 7aR6Q/8BYXN3p4igJnqD3tamXwPTy3hHMb50Ju+tqph6EOH2HlCb3OxpWYXYzhP9WJAV Cw+46AqcCoRxYp6Zovx+H6PDjIvMqIUJqNh5WQZUGCYPv8QkC1HzeoldrBwgGWpX80vc HugtWg/ce/L3VtfrITwgWvL5C9NVieX4w91lctrQwh8ovfRzIIdy1XNWGpAo/P6oZ/aK DjEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=34lemQd5zIzYiuHieHMv0eB4fxYwFhBfNJ3IIztazZk=; b=y4URQR/Zrzbq05ZNd5irc9xa7KFORHVURL8+ptBstTNVVvQJhzJYIm98dg7Ijq1ebl 7DbMTC5xxpHt24m2OIJtIzN8V4gtKBPqbkf/7YzLwA70GDcFQE9qYYR475gcWdQPNk3c 6n7wuh30jyfrNOV3/KQYEBQIxxg404Tnzkc0OmiQ1aofNx914+LpeWpK8RzC8YQhSTtS ZvCI8s8agxgvg7N6PfKO87ZhODaAnXPWwwI3PdurO5WTJM1iBeN3enuAyT7Xe8S8hvFQ EqaZagmWHyzEptSM/inP8/4P+RfKv1X1ErwBq9Fu/o3omNRnfwr5IDFF/YaWKkbVhYpa ywKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=q+G3nnup; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y5si14064796iof.34.2021.03.09.10.09.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:09:23 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=q+G3nnup; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43344 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgnL-00033o-8e for patch@linaro.org; Tue, 09 Mar 2021 13:09:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46626) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7Y-0000Bj-AX for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:08 -0500 Received: from mail-oi1-x22d.google.com ([2607:f8b0:4864:20::22d]:35879) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7V-0002kY-94 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:08 -0500 Received: by mail-oi1-x22d.google.com with SMTP id o22so5784705oic.3 for ; Tue, 09 Mar 2021 08:22:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=34lemQd5zIzYiuHieHMv0eB4fxYwFhBfNJ3IIztazZk=; b=q+G3nnupCHprgfnvNQwKm5tEFHHItpSSzY4MT7WkgFPmusKZ9UQj8raPo3bMwhqiyK 2X53mDkTV038gOIKvBz03NUs6g/iNNhG3UJOYSkGn5zJsNPJU5gFkzx8wx95wGGjVD39 QhrtB3aKx+G8CZvF6a4hEEMsC1I/yLIOvBDAR+WV+GnMunO/VYfladoWVka2SBOkAgRx J0yUv0XcU2HrDCTxvzULUF/kuqFjWSxtoMeMl8Lo5q+/orh4WzY7TW/Pt7d++kmyCb26 rypY8you0fsEINPcN7joDVb5zMviCXo1HDW+bMu7bN3kNKqQByFKkrTapyqvySY2fKsi 93DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=34lemQd5zIzYiuHieHMv0eB4fxYwFhBfNJ3IIztazZk=; b=N0NykQ57GpG1RzUE64p7Y9YIvF0CmHjmMbwIcUA7PCFoGHBZLZerh5ikBQYCGyBUl+ 5uWua9HfPpkxvlZL0qKBcwa1zeCoRPXpUBoVNlVNj+hJiuMcGXKijIsyqtlSOvOJghfh 3WDys3KTH+77NrIxGw6DYzIORRwET63HiTcyBFd4jdvGNCt+0xlJjf7nUbk2xqzTXMQa JWtFRBsTVnXAzatbBPBK0TUtPk6BljONZW7A9sIYNapvIFQDhi7kptCVVtrjyR8moDMf HMEj5v9TbJCR9MI0BsrvoLVBwVjDyHGl+en9jFqh/wAAKn0vPmWAmhhxLjX+/YvCibys kkzQ== X-Gm-Message-State: AOAM530Y/QpRzp03J2/5hSpm7eYFJcgs7m11/C7WRE0O0x4wcyIUAFpb CWbi3BuAfPcCVNj8jQ8NBRa4N57ewbP0EA6Q X-Received: by 2002:aca:de82:: with SMTP id v124mr3485851oig.125.1615306924296; Tue, 09 Mar 2021 08:22:04 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:03 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 44/78] target/arm: Implement SVE2 scatter store insns Date: Tue, 9 Mar 2021 08:20:07 -0800 Message-Id: <20210309162041.23124-45-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22d; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Add decoding logic for SVE2 64-bit/32-bit scatter non-temporal store insns. 64-bit * STNT1B (vector plus scalar) * STNT1H (vector plus scalar) * STNT1W (vector plus scalar) * STNT1D (vector plus scalar) 32-bit * STNT1B (vector plus scalar) * STNT1H (vector plus scalar) * STNT1W (vector plus scalar) Signed-off-by: Stephen Long Message-Id: <20200422141553.8037-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/sve.decode | 10 ++++++++++ target/arm/translate-sve.c | 8 ++++++++ 2 files changed, 18 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7645587469..5cfe6df0d2 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1388,3 +1388,13 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx + +### SVE2 Memory Store Group + +# SVE2 64-bit scatter non-temporal store (vector plus scalar) +STNT1_zprz 1110010 .. 00 ..... 001 ... ..... ..... \ + @rprr_scatter_store xs=2 esz=3 scale=0 + +# SVE2 32-bit scatter non-temporal store (vector plus scalar) +STNT1_zprz 1110010 .. 10 ..... 001 ... ..... ..... \ + @rprr_scatter_store xs=0 esz=2 scale=0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index cad61f4577..895522e3cf 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6168,6 +6168,14 @@ static bool trans_ST1_zpiz(DisasContext *s, arg_ST1_zpiz *a) return true; } +static bool trans_STNT1_zprz(DisasContext *s, arg_ST1_zprz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return trans_ST1_zprz(s, a); +} + /* * Prefetches */ From patchwork Tue Mar 9 16:20:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396055 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2809839jai; Tue, 9 Mar 2021 10:14:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJxFBq8vhH+cFFpenMdxo+QzYxX59A/YMsyyT6cKjCPNEIKn1Hd+GyYwygdswlA5kI+3fG25 X-Received: by 2002:a6b:f00a:: with SMTP id w10mr24540680ioc.2.1615313691586; Tue, 09 Mar 2021 10:14:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313691; cv=none; d=google.com; s=arc-20160816; b=pLYJFkTOZKv+VEydG0xn2B9cSB13tBbh1c/d/hqgG3CuG5+d/HS0axPHmBEdGfIWYG iELYvKRC5vIms2e65YCkf7kMmaktpNyb8WPhIYRR6H9TD7Sg/nZdsDroMbNvY3zsazZ7 /967VryfNlFtJiSQ7HFu3pJD9jUgygV7k9MN9DtgNJrTU6/0r8loRPEDt5vJCooY9Yub 9TS9Np7D6WGvXzCmBwnhpZWpwAktG17/o/QzGA0TJbzAKiGHWKMCW3Zppu4yZStQ8FuV HWRLvUO7xznw+0TToBN8oLkCryLQxk84VX5df/JmUqEZU9fOgc/Nw0Q/79nVbhIqKlxG Kc9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=bFsXvn8twBtNJsU+A9bHI+28KQAx75XSf3FzGr0LfTA=; b=dkaKAOxpKK96Hk+8B3WrMrvEC2sqZROcg3oMcgCo0USkAhsRp0/T7lWJR5x9O7y5bx Y0B5SpgtFdun8EeE1SM2oUPaloO1LGlKIqvm3nh07tNoBXxZ5RqaN6FsQz/lA0Bxz5DL M52/7mJdK1NNUk14lhzinHpX9cWEV2coWsdvdssCh8STi/dna7k2h90H4Y5gp5g/3biN 4+ngDGKorhhKabRjBDsY0Fupr8EtbEoFTE5tFOJL6KVTy/ma1Jp50YWF8T9rHvpwPGvO 1r04k+NqnlCupa1OC6nsiZEjQNysmDEfWK6KMdtnUCEwF1oRt8O7jQjJ9amYRM6dcF92 pUhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=q0fu5sSo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b5si13502610ilj.34.2021.03.09.10.14.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:14:51 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=q0fu5sSo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51770 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgsc-0007s6-Ve for patch@linaro.org; Tue, 09 Mar 2021 13:14:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46646) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7Z-0000DZ-02 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:09 -0500 Received: from mail-oo1-xc2a.google.com ([2607:f8b0:4864:20::c2a]:45387) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7W-0002km-He for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:08 -0500 Received: by mail-oo1-xc2a.google.com with SMTP id s23so3177378oot.12 for ; Tue, 09 Mar 2021 08:22:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bFsXvn8twBtNJsU+A9bHI+28KQAx75XSf3FzGr0LfTA=; b=q0fu5sSoZ7yFFwRNiK535wkOmqDR8Mm3uSK5rzUFfY3DTblLR3s1rdAEvKm7FMmCz5 Q3qgB+VjI1PLSCRsjpFuXcldMrKfo7Arvn086/zoNRQZb8066wF45pbk39AUS1NdudOg F8wND/7en4V0R62H43M3qafTpxkpYX/AvGWTPJLpGVbApaNdv8eIQKK4OgLknLuUjI3g J61P2wMlYOnA0aj8PQtJyBIP0lwjaNjrSxQkn1mtyXzR0kb0UAjYqC3fKP0oDmyLipx5 QSq/2y7LzaybBmTLPusjytWgpVWt4AwGrfIQpJLI6p8B34UFVsDJ7CQJPZuzDcGJZ8xj Ayew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bFsXvn8twBtNJsU+A9bHI+28KQAx75XSf3FzGr0LfTA=; b=XEvJTHt2JebZAVSKY5/l+ZvZlBaRYpTiehQRRKFX9LS0VnVw4ymzRRlYXFJ+n/fAdy c9nancEY+lZoP4XjN8GfrXv9dPinHF+zMDeUCS4/eK3pB56suH/ecl2owrGuq+eyZY+m jBmCF61L+4VeYtPeB164x1MDRXnrpG19OwjG76FPqRfhqFIfblBR5p9G6GplWzhkTutE bHZf1dUct3VbsggubTfX3yr89Pd2kAttF/uXSICqG/7AMCpgahBkHnrWoXlxIltMoXxm 5Lm4HWZ3KeOPei77HJUm6aCS+Co98B1nlPnysKFC8a1UVteGiAY6FLUgVke0wXTC0HaY nwRw== X-Gm-Message-State: AOAM531dd3cWx5IbocKjIyNLV3ePJUDfCp96A1UTptJmJ8KvW2BdkyCt hdnx1NMHePgNdmxFz5Faa0KggOUJq59aD0Z9 X-Received: by 2002:a4a:d1da:: with SMTP id a26mr22970496oos.58.1615306925485; Tue, 09 Mar 2021 08:22:05 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:05 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 45/78] target/arm: Implement SVE2 gather load insns Date: Tue, 9 Mar 2021 08:20:08 -0800 Message-Id: <20210309162041.23124-46-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c2a; envelope-from=richard.henderson@linaro.org; helo=mail-oo1-xc2a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Add decoding logic for SVE2 64-bit/32-bit gather non-temporal load insns. 64-bit * LDNT1SB * LDNT1B (vector plus scalar) * LDNT1SH * LDNT1H (vector plus scalar) * LDNT1SW * LDNT1W (vector plus scalar) * LDNT1D (vector plus scalar) 32-bit * LDNT1SB * LDNT1B (vector plus scalar) * LDNT1SH * LDNT1H (vector plus scalar) * LDNT1W (vector plus scalar) Signed-off-by: Stephen Long Message-Id: <20200422152343.12493-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/sve.decode | 11 +++++++++++ target/arm/translate-sve.c | 8 ++++++++ 2 files changed, 19 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 5cfe6df0d2..c3958bed6a 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1389,6 +1389,17 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx +### SVE2 Memory Gather Load Group + +# SVE2 64-bit gather non-temporal load +# (scalar plus unpacked 32-bit unscaled offsets) +LDNT1_zprz 1100010 msz:2 00 rm:5 1 u:1 0 pg:3 rn:5 rd:5 \ + &rprr_gather_load xs=0 esz=3 scale=0 ff=0 + +# SVE2 32-bit gather non-temporal load (scalar plus 32-bit unscaled offsets) +LDNT1_zprz 1000010 msz:2 00 rm:5 10 u:1 pg:3 rn:5 rd:5 \ + &rprr_gather_load xs=0 esz=2 scale=0 ff=0 + ### SVE2 Memory Store Group # SVE2 64-bit scatter non-temporal store (vector plus scalar) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 895522e3cf..21ecb535ca 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6016,6 +6016,14 @@ static bool trans_LD1_zpiz(DisasContext *s, arg_LD1_zpiz *a) return true; } +static bool trans_LDNT1_zprz(DisasContext *s, arg_LD1_zprz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return trans_LD1_zprz(s, a); +} + /* Indexed by [mte][be][xs][msz]. */ static gen_helper_gvec_mem_scatter * const scatter_store_fn32[2][2][2][3] = { { /* MTE Inactive */ From patchwork Tue Mar 9 16:20:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396039 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2791289jai; Tue, 9 Mar 2021 09:51:21 -0800 (PST) X-Google-Smtp-Source: ABdhPJxx11oHYvA4UqgvAQVSiAhIxC0q7KUQdwPUZUs4whppdv4nZrC1heUUTzuFJBevLOFkpRD8 X-Received: by 2002:a05:620a:12da:: with SMTP id e26mr7955702qkl.92.1615312281267; Tue, 09 Mar 2021 09:51:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312281; cv=none; d=google.com; s=arc-20160816; b=d2okSNXXwl45y9HrHT0GLMnKkU2Agqb1VpHnCAXjPYMnyOVOwoLK1/u7O9uE1rBp6X lSXBIjCKv4PIaMRfRHumomuQaUq0AlcQRNYVCb1GepUcCKEAl7l02BIttUqiKhff5pJT ileDJW1mkwGcFeMb7yrSH6xE6Fzcg9rAZZPrz7+9AD9a9YsC/6LTKwoAp8SD9MlogSbY rFHPPFhB6PEzsA3uzGyDdjtSf2qEfS/v8Pib8uP/juralrS04Ki1jxfIwf7b9Lv7J6K3 HaowMINnC717Ic2p1HObJTIUXc1HOwlkNsICK5M9vmrNF7GXzXm4Fh5Oa2UJY6EIvh0g dQQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=r8qWPPJzVA86rxlts7U6E1EFTR6n39CVbZo89lBzbCI=; b=i2zWjVh/243q+0KTOVcn71yjyWuj+Er/gEeiTRt/egLD0838HAiisdN2rmjI91rkjH tsNHjPMkW00WcV1qciflm2XzcGpIoHZlaL3vX7lR4OIraYaFpPwj1hSXCCJrRx4tKfut AAwsxLoUWuazYi+HaRKF8+xlydT0Akra4rMPr5/wfOlUq1LhIjpmeMj3PwSaNBucB40g pFOE48kH/azXR1LBm6URM1RODnpA+7Ycyyfncbdxsi7C1o2AKbb1nKIbdo+iitYunGdZ SD6f3f4OppPDN536XfZueqSaZcCMeWNtGmoVSelEMUPakZnjBAnq/uB0jBHTfKfCDQXC g4fQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YczDUld1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h5si8120344qvm.59.2021.03.09.09.51.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:51:21 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YczDUld1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37234 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgVs-0004np-Jb for patch@linaro.org; Tue, 09 Mar 2021 12:51:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46684) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7Z-0000GJ-TW for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:09 -0500 Received: from mail-oo1-xc31.google.com ([2607:f8b0:4864:20::c31]:33228) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7X-0002l6-Qo for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:09 -0500 Received: by mail-oo1-xc31.google.com with SMTP id z22so3178561oop.0 for ; Tue, 09 Mar 2021 08:22:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=r8qWPPJzVA86rxlts7U6E1EFTR6n39CVbZo89lBzbCI=; b=YczDUld1qtyPKcICrQV1pUYue/CPwvzw0XK91yJMR1F1MltJMHLoY4eekTASUymXne 77XzisG7vPMk0kB3nkackK91l1jldzZuW8tJBd5eGo21L05ROt4cOwPlFGF36AaeuaCB Eyq82kEz2saWhlIXdw7frQoxM3DuGm1WiBCGLxn29IJ63P0Z6M43YRWR1n1aMq1LGbuu 2wIN499cDVKBkhfvPiWKMVcDwkpRhmnOIapTFijF2I/57Nx9+6tGn/8cbl7Kwlx7/+Wz 9LpI54H5WD1cpWJDtRGgfcQ5GUbFJBv8bGrCwOX7ZVP6AsVaG8FTdYade6lF3PFfDlt7 tx4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r8qWPPJzVA86rxlts7U6E1EFTR6n39CVbZo89lBzbCI=; b=ikxUijiDmAzZIsEsS/UrhgeFOTDyz3N8tUSgf2gsEuiioh/VLCK7WAs3cpG+vQtlwB yr8esKggxwFNPTgasykrdcHLT6B/4+JfOhAhBKplG7Oo/fMjDJDQ0IS44T96rpsxYrkL GLxxtt62jWgB7vhFv8I8tQ1OP9v07xksB0nkcwY6uqSXDohxuNuoQnTEma1Qnmg6ZNGf uhnh624yZZeXQBP6oA+hLSNxxyxLSjjBFzTcr8yKlpJEbYhQBibGtNK/4juX7miS1jlB K8pFSEL085Ju0l0rSYtbEOmCFMnp10Cp4w/+VEU6+UAneGqyhk6CgzgkslYgCogrfYQv FJxw== X-Gm-Message-State: AOAM530iS0rLuHnj1Wg5KOB3AUpiP1k7zmAw0RvfAZ114CN2f4cCJbbw OLGXWybaza2O48+QsJ37t4JvORmy2uSyUJdS X-Received: by 2002:a4a:9101:: with SMTP id k1mr8329364oog.7.1615306926693; Tue, 09 Mar 2021 08:22:06 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:06 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 46/78] target/arm: Implement SVE2 FMMLA Date: Tue, 9 Mar 2021 08:20:09 -0800 Message-Id: <20210309162041.23124-47-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c31; envelope-from=richard.henderson@linaro.org; helo=mail-oo1-xc31.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200422165503.13511-1-steplong@quicinc.com> [rth: Fix indexing in helpers, expand macro to straight functions.] Signed-off-by: Richard Henderson --- target/arm/cpu.h | 10 ++++++ target/arm/helper-sve.h | 3 ++ target/arm/sve.decode | 4 +++ target/arm/sve_helper.c | 74 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 34 ++++++++++++++++++ 5 files changed, 125 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 57d03addb4..3f19c4cbed 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4213,6 +4213,16 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve2_f32mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, F32MM) != 0; +} + +static inline bool isar_feature_aa64_sve2_f64mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, F64MM) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 28b8f00201..7e99dcd119 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2662,3 +2662,6 @@ DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(fmmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(fmmla_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index c3958bed6a..cb2ee86228 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1389,6 +1389,10 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx +### SVE2 floating point matrix multiply accumulate + +FMMLA 01100100 .. 1 ..... 111001 ..... ..... @rda_rn_rm + ### SVE2 Memory Gather Load Group # SVE2 64-bit gather non-temporal load diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 0a02deef70..fcfb7d1f41 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7264,3 +7264,77 @@ void HELPER(sve2_xar_s)(void *vd, void *vn, void *vm, uint32_t desc) d[i] = ror32(n[i] ^ m[i], shr); } } + +void HELPER(fmmla_s)(void *vd, void *vn, void *vm, void *va, + void *status, uint32_t desc) +{ + intptr_t s, opr_sz = simd_oprsz(desc) / (sizeof(float32) * 4); + + for (s = 0; s < opr_sz; ++s) { + float32 *n = vn + s * sizeof(float32) * 4; + float32 *m = vm + s * sizeof(float32) * 4; + float32 *a = va + s * sizeof(float32) * 4; + float32 *d = vd + s * sizeof(float32) * 4; + float32 n00 = n[H4(0)], n01 = n[H4(1)]; + float32 n10 = n[H4(2)], n11 = n[H4(3)]; + float32 m00 = m[H4(0)], m01 = m[H4(1)]; + float32 m10 = m[H4(2)], m11 = m[H4(3)]; + float32 p0, p1; + + /* i = 0, j = 0 */ + p0 = float32_mul(n00, m00, status); + p1 = float32_mul(n01, m01, status); + d[H4(0)] = float32_add(a[H4(0)], float32_add(p0, p1, status), status); + + /* i = 0, j = 1 */ + p0 = float32_mul(n00, m10, status); + p1 = float32_mul(n01, m11, status); + d[H4(1)] = float32_add(a[H4(1)], float32_add(p0, p1, status), status); + + /* i = 1, j = 0 */ + p0 = float32_mul(n10, m00, status); + p1 = float32_mul(n11, m01, status); + d[H4(2)] = float32_add(a[H4(2)], float32_add(p0, p1, status), status); + + /* i = 1, j = 1 */ + p0 = float32_mul(n10, m10, status); + p1 = float32_mul(n11, m11, status); + d[H4(3)] = float32_add(a[H4(3)], float32_add(p0, p1, status), status); + } +} + +void HELPER(fmmla_d)(void *vd, void *vn, void *vm, void *va, + void *status, uint32_t desc) +{ + intptr_t s, opr_sz = simd_oprsz(desc) / (sizeof(float64) * 4); + + for (s = 0; s < opr_sz; ++s) { + float64 *n = vn + s * sizeof(float64) * 4; + float64 *m = vm + s * sizeof(float64) * 4; + float64 *a = va + s * sizeof(float64) * 4; + float64 *d = vd + s * sizeof(float64) * 4; + float64 n00 = n[0], n01 = n[1], n10 = n[2], n11 = n[3]; + float64 m00 = m[0], m01 = m[1], m10 = m[2], m11 = m[3]; + float64 p0, p1; + + /* i = 0, j = 0 */ + p0 = float64_mul(n00, m00, status); + p1 = float64_mul(n01, m01, status); + d[0] = float64_add(a[0], float64_add(p0, p1, status), status); + + /* i = 0, j = 1 */ + p0 = float64_mul(n00, m10, status); + p1 = float64_mul(n01, m11, status); + d[1] = float64_add(a[1], float64_add(p0, p1, status), status); + + /* i = 1, j = 0 */ + p0 = float64_mul(n10, m00, status); + p1 = float64_mul(n11, m01, status); + d[2] = float64_add(a[2], float64_add(p0, p1, status), status); + + /* i = 1, j = 1 */ + p0 = float64_mul(n10, m10, status); + p1 = float64_mul(n11, m11, status); + d[3] = float64_add(a[3], float64_add(p0, p1, status), status); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 21ecb535ca..8c93d87897 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7673,6 +7673,40 @@ DO_SVE2_ZPZZ_FP(FMINP, fminp) * SVE Integer Multiply-Add (unpredicated) */ +static bool trans_FMMLA(DisasContext *s, arg_rrrr_esz *a) +{ + gen_helper_gvec_4_ptr *fn; + + switch (a->esz) { + case MO_32: + if (!dc_isar_feature(aa64_sve2_f32mm, s)) { + return false; + } + fn = gen_helper_fmmla_s; + break; + case MO_64: + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + fn = gen_helper_fmmla_d; + break; + default: + return false; + } + + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + status, vsz, vsz, 0, fn); + tcg_temp_free_ptr(status); + } + return true; +} + static bool do_sqdmlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel1, bool sel2) { From patchwork Tue Mar 9 16:20:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396059 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2812406jai; Tue, 9 Mar 2021 10:18:22 -0800 (PST) X-Google-Smtp-Source: ABdhPJwNHt6B1B6P48OlFYRodulXpzDXnPSU8zOi+27kI4eErIf/VAR/x7VcJlL9tJscVIHFGU8o X-Received: by 2002:a6b:e312:: with SMTP id u18mr23726499ioc.58.1615313902198; Tue, 09 Mar 2021 10:18:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313902; cv=none; d=google.com; s=arc-20160816; b=FjM/gRYY0WhUtXqtFnSgun+nh3LeiFwqtLQnQpXSycHJ0mwPtXI7d/dPtNR15ozgNy Deresrw68drZ2PQxogeVRnz6YjRy9jEIEMc9UUOZ0L01utZtTpJhvbt3YyV5eRtRPev/ qsfvFG3JKj7FpdqTFzJ0YUkVnyvfoJ/ekQXsI7GEdrivUaLPkL5qgmtU2G0+6WmDlm00 H9u6UREPtlDI2M9JDJrnzwu9slVKgio7VupxicWscfiHYz9cW8sGFhKANBIFYfSRC9/F 0qFD1WVeOJJ9DzjmfAGfQ1hA0rXS7ggj+YRMxP1Rntdt3ZDmZbKsi/9rnVgLSU1x9jk5 C9CA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=HEwyxw34JNS3xvvmtmlho/yxHK1LKHErEB/8mITxEEA=; b=BWjyAmV7k+EIM+IXaQwWNNpQzqqf0vIfAJF24/OmwBcaAJ6idSuETSEaLENNIqicUj QpDwbAjhgmZrE9x+x6WE3I9ci8B9nLGcF6Bmrl2oWzDUqANdNB4pbbPUyvymKe2nZWEA VBwD9ul5/8fNglmZLjaxdMIpz09c7vpJAZYVZzrcdQ0926Ss/asEvVeBM9gkHGz8Dq6o da9p6O4kcnjcMWekvp8qMll45u3PZeb56tNiqxIUSCRZnv+exd3WL0OiUDUwP5Z3LPz8 qYUTIKyx0NVXcSIDVphn5FoWU1NRUsB7W5eq7gg9MTjgcPdGiepCjsBnvpqLK+HCL06u s+TA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xKL1Veqg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o16si15019733ilk.56.2021.03.09.10.18.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:18:22 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xKL1Veqg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60110 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgw1-0003uS-FT for patch@linaro.org; Tue, 09 Mar 2021 13:18:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46696) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7a-0000Hg-RT for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:10 -0500 Received: from mail-oo1-xc34.google.com ([2607:f8b0:4864:20::c34]:46336) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7Y-0002lU-Rt for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:10 -0500 Received: by mail-oo1-xc34.google.com with SMTP id l11so3166995oov.13 for ; Tue, 09 Mar 2021 08:22:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HEwyxw34JNS3xvvmtmlho/yxHK1LKHErEB/8mITxEEA=; b=xKL1VeqgOiuR1tF7zh9YSdfceKKUpt3AVxq74NOV8Syh0+01kMfHyFun0NBhwVtz71 AC9Nd0ZHLJ7o+nPH1pf8EXedClHxdefahUu0GhmNWy6gLI2OS+ngEz6u/t+IVVL4zDGi /fPxQCPawnXRh73QcTlBr2fNpCePw3oxWdYant7MZVrCG8w4OXIMPhhaCgUlXaOBwnQl Qt5ncFSg4F79R3fhAtkLgqgRSepXz2QTjL1pARXFE19DbpKjCx8cnYAMAoXCbEH9pmWD EvjMHJj78zBpuYu1eMLKv4ut+YWxAq//pqqg/AXn9b4zq/3PZQjiXJbUaQfaI6DlaFIo Ovwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HEwyxw34JNS3xvvmtmlho/yxHK1LKHErEB/8mITxEEA=; b=nyDJHT0aYVrBjK8LwwW4GXJuc4flVbHhL4Jt3xMSWEp51QN7FO1wBVnbXNdRnhhp/V TIoGjzsgSSdy+5oCUjTWvq8I22fniRX9ZDVlndoGIGNtZ1Hz8A9U9FpkgQFrOx0SX6R/ s4J30Nnz+FHGaBNRjihZHrPzkp5EojVnHm32ZSLUsh9vt2fU6aX1DnDYOSUaIRBaTPPZ yf/yGVCAISuJrWn4fWpQqDsQuSNYxJpUQPez70yg5S9icBS9hGgf74bPKUgy/dnlFQbU MzAhZajpsdnCDICAmzBo4g0t5FybopNj9tmRbnzdABZ9v6JqKkDFbC9XW9F8fs/MtnO4 vvrw== X-Gm-Message-State: AOAM530kOHyHf6yAX3/36w/5oKIHGvCFHv1wi19YwFGe3WaDmB514HCA gTXDAyXuqmPiD/92UrefoiZ4yTiIL+TgRoZ7 X-Received: by 2002:a4a:d994:: with SMTP id k20mr22846236oou.70.1615306927863; Tue, 09 Mar 2021 08:22:07 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:07 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 47/78] target/arm: Implement SVE2 SPLICE, EXT Date: Tue, 9 Mar 2021 08:20:10 -0800 Message-Id: <20210309162041.23124-48-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c34; envelope-from=richard.henderson@linaro.org; helo=mail-oo1-xc34.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200423180347.9403-1-steplong@quicinc.com> [rth: Rename the trans_* functions to *_sve2.] Signed-off-by: Richard Henderson --- target/arm/sve.decode | 11 +++++++++-- target/arm/translate-sve.c | 35 ++++++++++++++++++++++++++++++----- 2 files changed, 39 insertions(+), 7 deletions(-) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index cb2ee86228..67b6466a1e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -494,10 +494,14 @@ CPY_z_i 00000101 .. 01 .... 00 . ........ ..... @rdn_pg4 imm=%sh8_i8s ### SVE Permute - Extract Group -# SVE extract vector (immediate offset) +# SVE extract vector (destructive) EXT 00000101 001 ..... 000 ... rm:5 rd:5 \ &rrri rn=%reg_movprfx imm=%imm8_16_10 +# SVE2 extract vector (constructive) +EXT_sve2 00000101 011 ..... 000 ... rn:5 rd:5 \ + &rri imm=%imm8_16_10 + ### SVE Permute - Unpredicated Group # SVE broadcast general register @@ -588,9 +592,12 @@ REVH 00000101 .. 1001 01 100 ... ..... ..... @rd_pg_rn REVW 00000101 .. 1001 10 100 ... ..... ..... @rd_pg_rn RBIT 00000101 .. 1001 11 100 ... ..... ..... @rd_pg_rn -# SVE vector splice (predicated) +# SVE vector splice (predicated, destructive) SPLICE 00000101 .. 101 100 100 ... ..... ..... @rdn_pg_rm +# SVE2 vector splice (predicated, constructive) +SPLICE_sve2 00000101 .. 101 101 100 ... ..... ..... @rd_pg_rn + ### SVE Select Vectors Group # SVE select vector elements (predicated) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 8c93d87897..cfcf4893ac 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -2266,18 +2266,18 @@ static bool trans_CPY_z_i(DisasContext *s, arg_CPY_z_i *a) *** SVE Permute Extract Group */ -static bool trans_EXT(DisasContext *s, arg_EXT *a) +static bool do_EXT(DisasContext *s, int rd, int rn, int rm, int imm) { if (!sve_access_check(s)) { return true; } unsigned vsz = vec_full_reg_size(s); - unsigned n_ofs = a->imm >= vsz ? 0 : a->imm; + unsigned n_ofs = imm >= vsz ? 0 : imm; unsigned n_siz = vsz - n_ofs; - unsigned d = vec_full_reg_offset(s, a->rd); - unsigned n = vec_full_reg_offset(s, a->rn); - unsigned m = vec_full_reg_offset(s, a->rm); + unsigned d = vec_full_reg_offset(s, rd); + unsigned n = vec_full_reg_offset(s, rn); + unsigned m = vec_full_reg_offset(s, rm); /* Use host vector move insns if we have appropriate sizes * and no unfortunate overlap. @@ -2296,6 +2296,19 @@ static bool trans_EXT(DisasContext *s, arg_EXT *a) return true; } +static bool trans_EXT(DisasContext *s, arg_EXT *a) +{ + return do_EXT(s, a->rd, a->rn, a->rm, a->imm); +} + +static bool trans_EXT_sve2(DisasContext *s, arg_rri *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_EXT(s, a->rd, a->rn, (a->rn + 1) % 32, a->imm); +} + /* *** SVE Permute - Unpredicated Group */ @@ -3013,6 +3026,18 @@ static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a) return true; } +static bool trans_SPLICE_sve2(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzp(s, gen_helper_sve_splice, + a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz); + } + return true; +} + /* *** SVE Integer Compare - Vectors Group */ From patchwork Tue Mar 9 16:20:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396042 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2794663jai; Tue, 9 Mar 2021 09:56:20 -0800 (PST) X-Google-Smtp-Source: ABdhPJyqGYJfgqtvyFTxwVznq/qav1S/IyA4BhObrKpaJxIHdArpRW2+OFz2ac+7gAXkAt4wn5MD X-Received: by 2002:a92:940b:: with SMTP id c11mr23585084ili.132.1615312580852; Tue, 09 Mar 2021 09:56:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312580; cv=none; d=google.com; s=arc-20160816; b=J4UXXZ8fVsdKpNfVoy1F6stTAlY5rjdfTocbWx8gk9eHxh7bRXO3/LvROCZzkvqR3n yUh9/BMUfCiiadlEnjXJ1jKbAQP3rIUly3HZkJCaF/lhSSPj8UrzPzsCqlzjhfw4XylG sZH5kjO/SWyibCZcJKEn0/BdJJBZ4+LXaaPCGTS/OaqB58k8ZsveIDg5RG3mbkwv8lbi Ik98R/85/eRx2CEWrR5hhYZKj3qRDOTtdVERvl38NPt7hSZTUQIIcpk7UugWE2NlsPUp 9dENQbqZg9nmGcq0Y7C2bW6Ck6WGsGOlQ8lLZ23NYybUVA6KV5V9Bn9pLSz/U/oblF0J 3HeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=RTxFZBryI2G+vq0yXPAmaQzM2kjegfds9m7qc3Y2i3k=; b=PC5mJbdp/5GzYaf8oBkh5mtFL2NZ5uYNtA1Cin4NRDWL910blM/3eqVSuu9h7gkkyE 4UgciB36P8dYtJuvQ2kjnl5uRGBAf9IUactRSNKkr4XvwOqR6YM8VE998ShDJhws2Iym EGni61YO0U1iiJuO/QuRcXn92/LRNejblsPUuV5qcfYeMpwP5RPegGz3NcnAf4UAP/8D N8ePcYyhQKbuWhypXjly5iKT+769RvCCih809j9h0QzhnhJqifsyyFnKGobOYuTsLZF2 +c6HyAR47Z2xMWzd0atk3GNu5as0QtVL/kCDfVSN19vcjlLzwlMUOMOCtNusdsIMFLoN GJtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=kewpJO7f; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 22si14126184ioz.62.2021.03.09.09.56.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:56:20 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=kewpJO7f; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45886 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgai-0008RJ-2z for patch@linaro.org; Tue, 09 Mar 2021 12:56:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46788) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7g-0000Mo-7w for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:17 -0500 Received: from mail-oo1-xc2f.google.com ([2607:f8b0:4864:20::c2f]:39832) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7a-0002m4-5B for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:14 -0500 Received: by mail-oo1-xc2f.google.com with SMTP id i11so3176233ood.6 for ; Tue, 09 Mar 2021 08:22:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RTxFZBryI2G+vq0yXPAmaQzM2kjegfds9m7qc3Y2i3k=; b=kewpJO7fZPI3B35WnjNsa4VDoQRKZS7K5JiDFXffkPgoEuAqXaKkDFYcijD1rVFR0F b0FVPw40HvjinNnS2/oDfGQ5S7VFX2gUFl/Tt1MVGxJFp/KQn06v4MHFZ5BBtUAqHTmG xMONX1pJX0qgRXqoo8KozTJV78kbCADspHfW4D0AjaKjmE9NlLPNiH1OkeXHcNh7ZcIt uc2bJrbsrP4W7YsU7G0Z5UAVEDFBPIRWWBX35WnwxANlE9SULIIvUIe79NtPEBcSBbbP b/lr8jK1kzQxUUuLlXX7PeiffsYhoYG+2gicKlu8X1JuAemHeYt/hAZaFl7nIDNm9733 Perw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RTxFZBryI2G+vq0yXPAmaQzM2kjegfds9m7qc3Y2i3k=; b=qYMNiZdFtu3WB7jh+sMDoescxSDvvlnqxOr34U6k9xoCWWRzTMdq2IETCxwokXUWc5 CY2WHyBXgU1x1ISQ6q9pWlCHdoGUc78U0UPmeqxVBJlMtBHYgv3Wuw15Roc4nmCvY1ek D8K6NYvraBP2eRaeio4x5axg7hlVql9mVkUKTWSrUmWRoyV1Hv3KwCaxLoHokDent5oB EgTYULk3VP7LB5cLrRpOZQkU+4+VjBf5Rx7l8+qGqtN3FDjPIGVEjvDIYdeRPtcRc+b9 7/CSe81L0RMYwDQ+wOFjUENjNhBwiWWZNYGY7AKaB41TD9gU0CA3dMEnU/AQmbS7y9dG KENw== X-Gm-Message-State: AOAM530qaxfeEcaZqGeV8YGZWzJ/+1JYqiwTERfzJHFBupufFZzK/U+s 06sAgOdvElfS0rWXOcjLn5O5WXVutYZIdpyQ X-Received: by 2002:a4a:9019:: with SMTP id i25mr22793349oog.8.1615306928974; Tue, 09 Mar 2021 08:22:08 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:08 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 48/78] target/arm: Pass separate addend to {U, S}DOT helpers Date: Tue, 9 Mar 2021 08:20:11 -0800 Message-Id: <20210309162041.23124-49-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c2f; envelope-from=richard.henderson@linaro.org; helo=mail-oo1-xc2f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For SVE, we potentially have a 4th argument coming from the movprfx instruction. Currently we do not optimize movprfx, so the problem is not visible. Signed-off-by: Richard Henderson --- v4: Fix double addition (zhiwei). --- target/arm/helper.h | 20 +++--- target/arm/sve.decode | 7 +- target/arm/translate-a64.c | 15 +++- target/arm/translate-sve.c | 13 ++-- target/arm/vec_helper.c | 120 ++++++++++++++++++-------------- target/arm/translate-neon.c.inc | 10 +-- 6 files changed, 109 insertions(+), 76 deletions(-) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 23a7ec5638..f4b092ee1c 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -608,15 +608,19 @@ DEF_HELPER_FLAGS_5(sve2_sqrdmlah_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcaddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 67b6466a1e..04ef38f148 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -756,12 +756,13 @@ UMIN_zzi 00100101 .. 101 011 110 ........ ..... @rdn_i8u MUL_zzi 00100101 .. 110 000 110 ........ ..... @rdn_i8s # SVE integer dot product (unpredicated) -DOT_zzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 ra=%reg_movprfx +DOT_zzzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 \ + ra=%reg_movprfx # SVE integer dot product (indexed) -DOT_zzx 01000100 101 index:2 rm:3 00000 u:1 rn:5 rd:5 \ +DOT_zzxw 01000100 101 index:2 rm:3 00000 u:1 rn:5 rd:5 \ sz=0 ra=%reg_movprfx -DOT_zzx 01000100 111 index:1 rm:4 00000 u:1 rn:5 rd:5 \ +DOT_zzxw 01000100 111 index:1 rm:4 00000 u:1 rn:5 rd:5 \ sz=1 ra=%reg_movprfx # SVE floating-point complex add (predicated) diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 491519fa6c..c221f1b4ea 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -699,6 +699,17 @@ static void gen_gvec_op3_qc(DisasContext *s, bool is_q, int rd, int rn, tcg_temp_free_ptr(qc_ptr); } +/* Expand a 4-operand operation using an out-of-line helper. */ +static void gen_gvec_op4_ool(DisasContext *s, bool is_q, int rd, int rn, + int rm, int ra, int data, gen_helper_gvec_4 *fn) +{ + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), + is_q ? 16 : 8, vec_full_reg_size(s), data, fn); +} + /* Set ZF and NF based on a 64 bit result. This is alas fiddlier * than the 32 bit equivalent. */ @@ -12189,7 +12200,7 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) return; case 0x2: /* SDOT / UDOT */ - gen_gvec_op3_ool(s, is_q, rd, rn, rm, 0, + gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, 0, u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b); return; @@ -13448,7 +13459,7 @@ static void disas_simd_indexed(DisasContext *s, uint32_t insn) switch (16 * u + opcode) { case 0x0e: /* SDOT */ case 0x1e: /* UDOT */ - gen_gvec_op3_ool(s, is_q, rd, rn, rm, index, + gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, index, u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b); return; diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index cfcf4893ac..37f14af35a 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3800,28 +3800,29 @@ DO_ZZI(UMIN, umin) #undef DO_ZZI -static bool trans_DOT_zzz(DisasContext *s, arg_DOT_zzz *a) +static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) { - static gen_helper_gvec_3 * const fns[2][2] = { + static gen_helper_gvec_4 * const fns[2][2] = { { gen_helper_gvec_sdot_b, gen_helper_gvec_sdot_h }, { gen_helper_gvec_udot_b, gen_helper_gvec_udot_h } }; if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, 0); + gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, a->ra, 0); } return true; } -static bool trans_DOT_zzx(DisasContext *s, arg_DOT_zzx *a) +static bool trans_DOT_zzxw(DisasContext *s, arg_DOT_zzxw *a) { - static gen_helper_gvec_3 * const fns[2][2] = { + static gen_helper_gvec_4 * const fns[2][2] = { { gen_helper_gvec_sdot_idx_b, gen_helper_gvec_sdot_idx_h }, { gen_helper_gvec_udot_idx_b, gen_helper_gvec_udot_idx_h } }; if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, a->index); + gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, + a->ra, a->index); } return true; } diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index a3d80ecad0..f88e572132 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -375,71 +375,76 @@ void HELPER(sve2_sqrdmlsh_d)(void *vd, void *vn, void *vm, * All elements are treated equally, no matter where they are. */ -void HELPER(gvec_sdot_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_b)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint32_t *d = vd; + int32_t *d = vd, *a = va; int8_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 4; ++i) { - d[i] += n[i * 4 + 0] * m[i * 4 + 0] - + n[i * 4 + 1] * m[i * 4 + 1] - + n[i * 4 + 2] * m[i * 4 + 2] - + n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + n[i * 4 + 0] * m[i * 4 + 0] + + n[i * 4 + 1] * m[i * 4 + 1] + + n[i * 4 + 2] * m[i * 4 + 2] + + n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_b)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint32_t *d = vd; + uint32_t *d = vd, *a = va; uint8_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 4; ++i) { - d[i] += n[i * 4 + 0] * m[i * 4 + 0] - + n[i * 4 + 1] * m[i * 4 + 1] - + n[i * 4 + 2] * m[i * 4 + 2] - + n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + n[i * 4 + 0] * m[i * 4 + 0] + + n[i * 4 + 1] * m[i * 4 + 1] + + n[i * 4 + 2] * m[i * 4 + 2] + + n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_sdot_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint64_t *d = vd; + int64_t *d = vd, *a = va; int16_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - d[i] += (int64_t)n[i * 4 + 0] * m[i * 4 + 0] - + (int64_t)n[i * 4 + 1] * m[i * 4 + 1] - + (int64_t)n[i * 4 + 2] * m[i * 4 + 2] - + (int64_t)n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + (int64_t)n[i * 4 + 0] * m[i * 4 + 0] + + (int64_t)n[i * 4 + 1] * m[i * 4 + 1] + + (int64_t)n[i * 4 + 2] * m[i * 4 + 2] + + (int64_t)n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint64_t *d = vd; + uint64_t *d = vd, *a = va; uint16_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - d[i] += (uint64_t)n[i * 4 + 0] * m[i * 4 + 0] - + (uint64_t)n[i * 4 + 1] * m[i * 4 + 1] - + (uint64_t)n[i * 4 + 2] * m[i * 4 + 2] - + (uint64_t)n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + (uint64_t)n[i * 4 + 0] * m[i * 4 + 0] + + (uint64_t)n[i * 4 + 1] * m[i * 4 + 1] + + (uint64_t)n[i * 4 + 2] * m[i * 4 + 2] + + (uint64_t)n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; intptr_t index = simd_data(desc); - uint32_t *d = vd; + int32_t *d = vd, *a = va; int8_t *n = vn; int8_t *m_indexed = (int8_t *)vm + H4(index) * 4; @@ -455,10 +460,11 @@ void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) int8_t m3 = m_indexed[i * 4 + 3]; do { - d[i] += n[i * 4 + 0] * m0 - + n[i * 4 + 1] * m1 - + n[i * 4 + 2] * m2 - + n[i * 4 + 3] * m3; + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); } while (++i < segend); segend = i + 4; } while (i < opr_sz_4); @@ -466,11 +472,12 @@ void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; intptr_t index = simd_data(desc); - uint32_t *d = vd; + uint32_t *d = vd, *a = va; uint8_t *n = vn; uint8_t *m_indexed = (uint8_t *)vm + H4(index) * 4; @@ -486,10 +493,11 @@ void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) uint8_t m3 = m_indexed[i * 4 + 3]; do { - d[i] += n[i * 4 + 0] * m0 - + n[i * 4 + 1] * m1 - + n[i * 4 + 2] * m2 - + n[i * 4 + 3] * m3; + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); } while (++i < segend); segend = i + 4; } while (i < opr_sz_4); @@ -497,11 +505,12 @@ void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc), opr_sz_8 = opr_sz / 8; intptr_t index = simd_data(desc); - uint64_t *d = vd; + int64_t *d = vd, *a = va; int16_t *n = vn; int16_t *m_indexed = (int16_t *)vm + index * 4; @@ -509,30 +518,33 @@ void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) * Process the entire segment all at once, writing back the results * only after we've consumed all of the inputs. */ - for (i = 0; i < opr_sz_8 ; i += 2) { - uint64_t d0, d1; + for (i = 0; i < opr_sz_8; i += 2) { + int64_t d0, d1; - d0 = n[i * 4 + 0] * (int64_t)m_indexed[i * 4 + 0]; + d0 = a[i + 0]; + d0 += n[i * 4 + 0] * (int64_t)m_indexed[i * 4 + 0]; d0 += n[i * 4 + 1] * (int64_t)m_indexed[i * 4 + 1]; d0 += n[i * 4 + 2] * (int64_t)m_indexed[i * 4 + 2]; d0 += n[i * 4 + 3] * (int64_t)m_indexed[i * 4 + 3]; - d1 = n[i * 4 + 4] * (int64_t)m_indexed[i * 4 + 0]; + + d1 = a[i + 1]; + d1 += n[i * 4 + 4] * (int64_t)m_indexed[i * 4 + 0]; d1 += n[i * 4 + 5] * (int64_t)m_indexed[i * 4 + 1]; d1 += n[i * 4 + 6] * (int64_t)m_indexed[i * 4 + 2]; d1 += n[i * 4 + 7] * (int64_t)m_indexed[i * 4 + 3]; - d[i + 0] += d0; - d[i + 1] += d1; + d[i + 0] = d0; + d[i + 1] = d1; } - clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_idx_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc), opr_sz_8 = opr_sz / 8; intptr_t index = simd_data(desc); - uint64_t *d = vd; + uint64_t *d = vd, *a = va; uint16_t *n = vn; uint16_t *m_indexed = (uint16_t *)vm + index * 4; @@ -540,22 +552,24 @@ void HELPER(gvec_udot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) * Process the entire segment all at once, writing back the results * only after we've consumed all of the inputs. */ - for (i = 0; i < opr_sz_8 ; i += 2) { + for (i = 0; i < opr_sz_8; i += 2) { uint64_t d0, d1; - d0 = n[i * 4 + 0] * (uint64_t)m_indexed[i * 4 + 0]; + d0 = a[i + 0]; + d0 += n[i * 4 + 0] * (uint64_t)m_indexed[i * 4 + 0]; d0 += n[i * 4 + 1] * (uint64_t)m_indexed[i * 4 + 1]; d0 += n[i * 4 + 2] * (uint64_t)m_indexed[i * 4 + 2]; d0 += n[i * 4 + 3] * (uint64_t)m_indexed[i * 4 + 3]; - d1 = n[i * 4 + 4] * (uint64_t)m_indexed[i * 4 + 0]; + + d1 = a[i + 1]; + d1 += n[i * 4 + 4] * (uint64_t)m_indexed[i * 4 + 0]; d1 += n[i * 4 + 5] * (uint64_t)m_indexed[i * 4 + 1]; d1 += n[i * 4 + 6] * (uint64_t)m_indexed[i * 4 + 2]; d1 += n[i * 4 + 7] * (uint64_t)m_indexed[i * 4 + 3]; - d[i + 0] += d0; - d[i + 1] += d1; + d[i + 0] = d0; + d[i + 1] = d1; } - clear_tail(d, opr_sz, simd_maxsz(desc)); } diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index f6c68e30ab..4f8903a6a4 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -221,7 +221,7 @@ static bool trans_VCADD(DisasContext *s, arg_VCADD *a) static bool trans_VDOT(DisasContext *s, arg_VDOT *a) { int opr_sz; - gen_helper_gvec_3 *fn_gvec; + gen_helper_gvec_4 *fn_gvec; if (!dc_isar_feature(aa32_dp, s)) { return false; @@ -243,9 +243,10 @@ static bool trans_VDOT(DisasContext *s, arg_VDOT *a) opr_sz = (1 + a->q) * 8; fn_gvec = a->u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b; - tcg_gen_gvec_3_ool(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->vm), + vfp_reg_offset(1, a->vd), opr_sz, opr_sz, 0, fn_gvec); return true; } @@ -323,7 +324,7 @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) { - gen_helper_gvec_3 *fn_gvec; + gen_helper_gvec_4 *fn_gvec; int opr_sz; TCGv_ptr fpst; @@ -348,9 +349,10 @@ static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) fn_gvec = a->u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b; opr_sz = (1 + a->q) * 8; fpst = fpstatus_ptr(FPST_STD); - tcg_gen_gvec_3_ool(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->rm), + vfp_reg_offset(1, a->vd), opr_sz, opr_sz, a->index, fn_gvec); tcg_temp_free_ptr(fpst); return true; From patchwork Tue Mar 9 16:20:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396046 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2796009jai; Tue, 9 Mar 2021 09:58:22 -0800 (PST) X-Google-Smtp-Source: ABdhPJwEpu8BJ/Wl7qpjQAgtng/ScaVQaBZJjNFtsKbr4tC1rRCbdTEAWboU/a4CAmldAtB8UASn X-Received: by 2002:a05:6638:191c:: with SMTP id p28mr28327357jal.39.1615312702040; Tue, 09 Mar 2021 09:58:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615312702; cv=none; d=google.com; s=arc-20160816; b=gpxaGYV5OzaFTiCtb16HRKo8LmEhTWI5XIrZ5fYgVtAeVka/mCYM6cC//UWEmt9EBO /wMZ8hGjUKlCXdJA8gWERixc6/3LVopQnfi+X4EF+sJmHsluMnEGOuphme7nUhTIyHGO X6JRqypjuZFGujRzTQMC8aNnmAGsnx/is3b0PLrLdPCqxrskpAnwoRUtmjqJ6t8c4MbL D8KEouMydbmxo2wSXt2L444NIzzJRnWNTmMLyDQbBqhaC/XHk3tLr1sxVWJudcZn36Hg /CD42GZpHmJHwqPfoxsHB43ul7UP504jaZS4seS8Q23KnSqAhNFo8DpLGwEU0cLZusRp ImIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=wFHrsLIpIaTDXv1HcswMi0L2moQ2c4srYWvmZF6fAUY=; b=l6T8PRrNmOiW6rNjVnHmkzGqcLm75p7SpcWqccPDijv066wbwvO0L22vbFUaF+0D30 pTl5gKW6mcyITJ3lqRaLz5Ub3gjo0h4lwXBSSpzq50PDjo7CwSH4KgaB45jzXK54aRCI XP4rGYh07SG6yBRPIkSQ1cTYzpwQfc0m9EKhEMYS/Er29tWykpwyOgIs2HTsZWQJlpkw idkzzW1vUYQbIfNHJxKZlwMP1D73kPcSbhHlBWdAgJMjqx67M5AdtEdAs/wAeeR82af4 ygP/hmgJXcxqkZ7oiv0StjFm9EUU8NbnK/BFBKaXcaL0nNwF8qUjIANmtPhvg3j+Xy8U 31hA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HRwp7HCo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s9si14328509ilu.130.2021.03.09.09.58.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 09:58:22 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HRwp7HCo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54564 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgcf-0003iy-EA for patch@linaro.org; Tue, 09 Mar 2021 12:58:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46840) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7i-0000Np-8C for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:20 -0500 Received: from mail-ot1-x331.google.com ([2607:f8b0:4864:20::331]:46169) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7b-0002md-5g for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:16 -0500 Received: by mail-ot1-x331.google.com with SMTP id r24so6273560otq.13 for ; Tue, 09 Mar 2021 08:22:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wFHrsLIpIaTDXv1HcswMi0L2moQ2c4srYWvmZF6fAUY=; b=HRwp7HCoxrh1HXwrlGEaBndez/K540x6g13hSOKQ5tceDAgDxO1M27GnfSuG91cv3S PbuZ0WFK6G0xU3+8YLDSBoAPRJHrGS6G39XLyUjFXYZsk7TXdpWuMSs/RO5j/NQV+8jB wae2Cw/p1hvRmYsBHgaFVeRinQ0BKKSA82nmlXShpZ6MFWm7cZCm1nMeWPVJliju/k5r Lwm1sf3ickqAiuDTCSVKE1skwOaibMDh+N2/KEZh0cBkOIEkQ/9vm6zgIdJh8kUgDo4x IMcI80K75CRYq5g2uPpGgu9LYeo+U/2GpHgw4odyoHVKmBs+wMvDO/AYUwJ9OhKLW4hh ta3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wFHrsLIpIaTDXv1HcswMi0L2moQ2c4srYWvmZF6fAUY=; b=I1K38XlWygjoxEEbJXgge6TJ5uaR7uqxP0KEsUGAZthQnuSOud5AZMM21u/IfWkVSx nAgVhPds2To8MKFnQHTuy6ueWOIPKgPcW7Xipsli66gGPHwACeAsYB9JcA9H8icUfexx ei94yIQvfhgt9315Qcj5Fbr5RiGvLBxTUA2jMW9RNsfyf058zeOWT6+3wGjs5do0JcRR XurL6GxSDd+OdCMklfp6/U+6wrvAZA0ywID+L/uL9goCCUHIjeJSSc4BJ9E5Z5b9r08e V5qxGi5oVR0lKjbwxc9RaguBT0+fvEzaRumx3KDs/iM5NseQQMtzG40RxJBMHOJui2p2 bjlw== X-Gm-Message-State: AOAM53151e+IYqwrwF+l/z1oz+gRs29P33KhLUk2q7xc1d0TGar1pFpD kMnj5+ffSsQpzLj2W0kOEBUUU2Q9uAK03Tuf X-Received: by 2002:a05:6830:619:: with SMTP id w25mr2151537oti.371.1615306930085; Tue, 09 Mar 2021 08:22:10 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:09 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 49/78] target/arm: Pass separate addend to FCMLA helpers Date: Tue, 9 Mar 2021 08:20:12 -0800 Message-Id: <20210309162041.23124-50-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::331; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x331.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For SVE, we potentially have a 4th argument coming from the movprfx instruction. Currently we do not optimize movprfx, so the problem is not visible. Signed-off-by: Richard Henderson --- target/arm/helper.h | 20 ++++++------- target/arm/translate-a64.c | 28 ++++++++++++++---- target/arm/translate-sve.c | 5 ++-- target/arm/vec_helper.c | 50 +++++++++++++-------------------- target/arm/translate-neon.c.inc | 10 ++++--- 5 files changed, 62 insertions(+), 51 deletions(-) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index f4b092ee1c..72c5bf6aca 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -629,16 +629,16 @@ DEF_HELPER_FLAGS_5(gvec_fcadds, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_fcaddd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlah, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlah_idx, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlas, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlas_idx, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlad, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlah, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlah_idx, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlas, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlas_idx, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlad, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(neon_paddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(neon_pmaxh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index c221f1b4ea..d3a1368090 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -710,6 +710,23 @@ static void gen_gvec_op4_ool(DisasContext *s, bool is_q, int rd, int rn, is_q ? 16 : 8, vec_full_reg_size(s), data, fn); } +/* + * Expand a 4-operand + fpstatus pointer + simd data value operation using + * an out-of-line helper. + */ +static void gen_gvec_op4_fpst(DisasContext *s, bool is_q, int rd, int rn, + int rm, int ra, bool is_fp16, int data, + gen_helper_gvec_4_ptr *fn) +{ + TCGv_ptr fpst = fpstatus_ptr(is_fp16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), fpst, + is_q ? 16 : 8, vec_full_reg_size(s), data, fn); + tcg_temp_free_ptr(fpst); +} + /* Set ZF and NF based on a 64 bit result. This is alas fiddlier * than the 32 bit equivalent. */ @@ -12211,15 +12228,15 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) rot = extract32(opcode, 0, 2); switch (size) { case 1: - gen_gvec_op3_fpst(s, is_q, rd, rn, rm, true, rot, + gen_gvec_op4_fpst(s, is_q, rd, rn, rm, rd, true, rot, gen_helper_gvec_fcmlah); break; case 2: - gen_gvec_op3_fpst(s, is_q, rd, rn, rm, false, rot, + gen_gvec_op4_fpst(s, is_q, rd, rn, rm, rd, false, rot, gen_helper_gvec_fcmlas); break; case 3: - gen_gvec_op3_fpst(s, is_q, rd, rn, rm, false, rot, + gen_gvec_op4_fpst(s, is_q, rd, rn, rm, rd, false, rot, gen_helper_gvec_fcmlad); break; default: @@ -13470,9 +13487,10 @@ static void disas_simd_indexed(DisasContext *s, uint32_t insn) { int rot = extract32(insn, 13, 2); int data = (index << 2) | rot; - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd), + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, rd), vec_full_reg_offset(s, rn), - vec_full_reg_offset(s, rm), fpst, + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, rd), fpst, is_q ? 16 : 8, vec_full_reg_size(s), data, size == MO_64 ? gen_helper_gvec_fcmlas_idx diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 37f14af35a..7439dd928b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4383,7 +4383,7 @@ static bool trans_FCMLA_zpzzz(DisasContext *s, arg_FCMLA_zpzzz *a) static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a) { - static gen_helper_gvec_3_ptr * const fns[2] = { + static gen_helper_gvec_4_ptr * const fns[2] = { gen_helper_gvec_fcmlah_idx, gen_helper_gvec_fcmlas_idx, }; @@ -4393,9 +4393,10 @@ static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a) if (sve_access_check(s)) { unsigned vsz = vec_full_reg_size(s); TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), vec_full_reg_offset(s, a->rn), vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), status, vsz, vsz, a->index * 4 + a->rot, fns[a->esz - 1]); diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index f88e572132..b19877e0d3 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -657,13 +657,11 @@ void HELPER(gvec_fcaddd)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float16 *d = vd; - float16 *n = vn; - float16 *m = vm; + float16 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -680,19 +678,17 @@ void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, float16 e4 = e2; float16 e3 = m[H2(i + 1 - flip)] ^ neg_imag; - d[H2(i)] = float16_muladd(e2, e1, d[H2(i)], 0, fpst); - d[H2(i + 1)] = float16_muladd(e4, e3, d[H2(i + 1)], 0, fpst); + d[H2(i)] = float16_muladd(e2, e1, a[H2(i)], 0, fpst); + d[H2(i + 1)] = float16_muladd(e4, e3, a[H2(i + 1)], 0, fpst); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float16 *d = vd; - float16 *n = vn; - float16 *m = vm; + float16 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -716,20 +712,18 @@ void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, float16 e2 = n[H2(j + flip)]; float16 e4 = e2; - d[H2(j)] = float16_muladd(e2, e1, d[H2(j)], 0, fpst); - d[H2(j + 1)] = float16_muladd(e4, e3, d[H2(j + 1)], 0, fpst); + d[H2(j)] = float16_muladd(e2, e1, a[H2(j)], 0, fpst); + d[H2(j + 1)] = float16_muladd(e4, e3, a[H2(j + 1)], 0, fpst); } } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float32 *d = vd; - float32 *n = vn; - float32 *m = vm; + float32 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -746,19 +740,17 @@ void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, float32 e4 = e2; float32 e3 = m[H4(i + 1 - flip)] ^ neg_imag; - d[H4(i)] = float32_muladd(e2, e1, d[H4(i)], 0, fpst); - d[H4(i + 1)] = float32_muladd(e4, e3, d[H4(i + 1)], 0, fpst); + d[H4(i)] = float32_muladd(e2, e1, a[H4(i)], 0, fpst); + d[H4(i + 1)] = float32_muladd(e4, e3, a[H4(i + 1)], 0, fpst); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float32 *d = vd; - float32 *n = vn; - float32 *m = vm; + float32 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -782,20 +774,18 @@ void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, float32 e2 = n[H4(j + flip)]; float32 e4 = e2; - d[H4(j)] = float32_muladd(e2, e1, d[H4(j)], 0, fpst); - d[H4(j + 1)] = float32_muladd(e4, e3, d[H4(j + 1)], 0, fpst); + d[H4(j)] = float32_muladd(e2, e1, a[H4(j)], 0, fpst); + d[H4(j + 1)] = float32_muladd(e4, e3, a[H4(j + 1)], 0, fpst); } } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float64 *d = vd; - float64 *n = vn; - float64 *m = vm; + float64 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint64_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -812,8 +802,8 @@ void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, float64 e4 = e2; float64 e3 = m[i + 1 - flip] ^ neg_imag; - d[i] = float64_muladd(e2, e1, d[i], 0, fpst); - d[i + 1] = float64_muladd(e4, e3, d[i + 1], 0, fpst); + d[i] = float64_muladd(e2, e1, a[i], 0, fpst); + d[i + 1] = float64_muladd(e4, e3, a[i + 1], 0, fpst); } clear_tail(d, opr_sz, simd_maxsz(desc)); } diff --git a/target/arm/translate-neon.c.inc b/target/arm/translate-neon.c.inc index 4f8903a6a4..17657c2310 100644 --- a/target/arm/translate-neon.c.inc +++ b/target/arm/translate-neon.c.inc @@ -146,7 +146,7 @@ static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) { int opr_sz; TCGv_ptr fpst; - gen_helper_gvec_3_ptr *fn_gvec_ptr; + gen_helper_gvec_4_ptr *fn_gvec_ptr; if (!dc_isar_feature(aa32_vcma, s) || (a->size == MO_16 && !dc_isar_feature(aa32_fp16_arith, s))) { @@ -171,9 +171,10 @@ static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) fpst = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD); fn_gvec_ptr = (a->size == MO_16) ? gen_helper_gvec_fcmlah : gen_helper_gvec_fcmlas; - tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ptr(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->vm), + vfp_reg_offset(1, a->vd), fpst, opr_sz, opr_sz, a->rot, fn_gvec_ptr); tcg_temp_free_ptr(fpst); @@ -284,7 +285,7 @@ static bool trans_VFML(DisasContext *s, arg_VFML *a) static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) { - gen_helper_gvec_3_ptr *fn_gvec_ptr; + gen_helper_gvec_4_ptr *fn_gvec_ptr; int opr_sz; TCGv_ptr fpst; @@ -313,9 +314,10 @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) gen_helper_gvec_fcmlah_idx : gen_helper_gvec_fcmlas_idx; opr_sz = (1 + a->q) * 8; fpst = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD); - tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ptr(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->vm), + vfp_reg_offset(1, a->vd), fpst, opr_sz, opr_sz, (a->index << 2) | a->rot, fn_gvec_ptr); tcg_temp_free_ptr(fpst); From patchwork Tue Mar 9 16:20:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396074 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2819863jai; Tue, 9 Mar 2021 10:29:08 -0800 (PST) X-Google-Smtp-Source: ABdhPJzgTMyNMrHb9L5TjURqwS7VVmgtJ4GeBQFVCY4xG4T05UAS1wA5BFtMl/7fItl0XLPJ9lfF X-Received: by 2002:ad4:4dc6:: with SMTP id cw6mr5125916qvb.31.1615314548520; Tue, 09 Mar 2021 10:29:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314548; cv=none; d=google.com; s=arc-20160816; b=uW8Wz8crTq6aTMBEJYwkFQihYO9rp/DRadV3wM46slABSr6LiurU2CfQLN0x/Mouht CGWnfuuAxLeR53x8e03Z6JN7s7yTlQINbu8sqCr9dydtbLFzxbzfMlUoNC/tg8YQDV+e QXI9g1NU0OzojqiI8l0/ThJgiqB280lHNOmtOBGa96LrtaM71gV8ZJzaPukBHJUadBZS kHS7wrRvUkbf0buwgaBRtQtLZoZ0ZofY3FIkmoZl/k5Nzk0k2jewmJjfSQmXunRkvxMG 70tEInZWKWoNxvIbNvvYVnvPy4lfkcDxS/fJC2GVBWZdWTGs8pF8yp81hBHCK+BHA5lY igpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=gqJnI67ZsQPzrPFFIP/tf6r9PpztXTYxjNrV84rekq4=; b=leUL1YmN38i/omjhkxLvY0G6gV82M2hoVfonwY9hYFoex7RoMtKrSbbsIPSBHUCGqM 5XJlvS2o2Z8B1Fq4H9/iHpUsqYIBIqVmndzGVEgBA2NXaXbG4UqacoXo2YMeCdWE1Te1 wjkZ5+1ZiK5jPmdjIU0kHhQjw6SZhOdbCxU0I9O27600MFBe8OGtjT04GCJW5v0B42IU rJ3JAb0xGNIqfOwhpVoOTDOKlqw+LAZblco8cAyW6/Vcw+Eeaqw8LrEWU9bCyH+im31s YnapZ4Ue91gfhvxGk7xWxhJT17C47KwX9FJ4a/k/WhehLihWxKfVsM/vvjJvurW+z3Nb GxLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uumtE7Ra; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n185si2700010qkc.170.2021.03.09.10.29.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:29:08 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uumtE7Ra; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34076 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJh6N-0000DM-Vh for patch@linaro.org; Tue, 09 Mar 2021 13:29:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46844) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7i-0000Nz-9P for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:20 -0500 Received: from mail-ot1-x331.google.com ([2607:f8b0:4864:20::331]:34906) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7c-0002n8-5Q for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:17 -0500 Received: by mail-ot1-x331.google.com with SMTP id j22so7518307otp.2 for ; Tue, 09 Mar 2021 08:22:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gqJnI67ZsQPzrPFFIP/tf6r9PpztXTYxjNrV84rekq4=; b=uumtE7RaWimHkpHQG3IuIYrCoycoH9N7U5i/2c1Ue/7SOqhIwiIss/2e/fDXGVRcsG rgrklQPMmdB750IVU0hnibtYaaMN5mNHPQenrObw9Y6TK/2esuV7xu9BWynwtDuDes7z GsCIGhf5JvuErBWURNSft+lmyHY8ZeN9PrsdDSeLTGVilcdFYj3DR3awNO4ix6URMvIi A2EPbnSUTZPLK5p3IMNngi/tFDXIiEifcPqBhsB411AtDcD9WK+z2q3NErlaKo9H06hb 6wsWzzDuGQHPjDnaPZYABtXRk0rhPcEsf0/e+F4nipzYigbiXh85LqzMSFJV8Dzx4MHX nh9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gqJnI67ZsQPzrPFFIP/tf6r9PpztXTYxjNrV84rekq4=; b=jTbMwNI5IBzZoHU9nGIryjU8SNBqTzgNcnHf/2cJz9FE4Cr8JhWj1/ysLYaOq15WVE YDhDiPSL8AZo8PJMU7LXJ4VBcAHIL73G/tWKbJOGOGt8O2Kl7NUJB/i2sjGyaF754FNO +UTld7gvdla7YtBKt/sEZJuT1Zs/yqoReUH5hOLzyCidHWhVEDs8oIYEQgMNGZTiVgWq C/rAIUGEFJgWKZqPawBEmqE7n/uHlp10ONe5hX5ktJ2b/mBiSPB/KncPTPrFcX7NFbaI 1wFm0N6gK6rH8FhCTYi2qWy+PsMExMdY9w46FLX7jSgHuRx5ZeB1e0nNB5Csa9TIynp/ LLmg== X-Gm-Message-State: AOAM531jxPKJYZtINHWoOuWj19dFVScEU+z6p5FMJ/w8pWI9Qm8aMV3H x8ljHUI2zwvbfG7Y7fbBvK6BggcVzufwq1cG X-Received: by 2002:a05:6830:802:: with SMTP id r2mr21007229ots.110.1615306931216; Tue, 09 Mar 2021 08:22:11 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:10 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 50/78] target/arm: Split out formats for 2 vectors + 1 index Date: Tue, 9 Mar 2021 08:20:13 -0800 Message-Id: <20210309162041.23124-51-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::331; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x331.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Currently only used by FMUL, but will shortly be used more. Signed-off-by: Richard Henderson --- target/arm/sve.decode | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 04ef38f148..eb5d4b052e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -67,6 +67,7 @@ &rri_esz rd rn imm esz &rrri_esz rd rn rm imm esz &rrr_esz rd rn rm esz +&rrx_esz rd rn rm index esz &rpr_esz rd pg rn esz &rpr_s rd pg rn s &rprr_s rd pg rn rm s @@ -245,6 +246,14 @@ @rpri_scatter_store ....... msz:2 .. imm:5 ... pg:3 rn:5 rd:5 \ &rpri_scatter_store +# Two registers and a scalar by index +@rrx_h ........ 0. . .. rm:3 ...... rn:5 rd:5 \ + &rrx_esz index=%index3_22_19 esz=1 +@rrx_s ........ 10 . index:2 rm:3 ...... rn:5 rd:5 \ + &rrx_esz esz=2 +@rrx_d ........ 11 . index:1 rm:4 ...... rn:5 rd:5 \ + &rrx_esz esz=3 + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -792,10 +801,9 @@ FMLA_zzxz 01100100 111 index:1 rm:4 00000 sub:1 rn:5 rd:5 \ ### SVE FP Multiply Indexed Group # SVE floating-point multiply (indexed) -FMUL_zzx 01100100 0.1 .. rm:3 001000 rn:5 rd:5 \ - index=%index3_22_19 esz=1 -FMUL_zzx 01100100 101 index:2 rm:3 001000 rn:5 rd:5 esz=2 -FMUL_zzx 01100100 111 index:1 rm:4 001000 rn:5 rd:5 esz=3 +FMUL_zzx 01100100 .. 1 ..... 001000 ..... ..... @rrx_h +FMUL_zzx 01100100 .. 1 ..... 001000 ..... ..... @rrx_s +FMUL_zzx 01100100 .. 1 ..... 001000 ..... ..... @rrx_d ### SVE FP Fast Reduction Group From patchwork Tue Mar 9 16:20:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396063 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2814306jai; Tue, 9 Mar 2021 10:21:03 -0800 (PST) X-Google-Smtp-Source: ABdhPJwn7XnvRqMdHl/jJzOdgxHOWDjcvHVW+LIhhVMnVRkA/O5gaJUfRi6vFi9OipEVJd10u5L4 X-Received: by 2002:a5d:8d94:: with SMTP id b20mr24004826ioj.200.1615314063063; Tue, 09 Mar 2021 10:21:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314063; cv=none; d=google.com; s=arc-20160816; b=qGDhSPBD+voor5z4C1RWv1vQrqS4TBDLLWkLHvydqpG1OXEh+9cwlvO+bRKQhELZ2Z QFjTJgsfGl9i+kVFj+EB4PiIQrdOOQ9UovP0RD4r2fPDVPTIiJ4GAzv3+dbsSj0yo412 PuB/UNaV1xLJgow8gGrcrNxzXQQAsSdPzvGVHE2voSYVz/0x63TvomKUsevXufp5kYxs 7lqCLiSbmddAfzK9bPVmAhxBPxTLQxYKEI3kwQJ3ZU4xPXaC8ocdWQko1WsiJhco347f lMIsgVuBHbYGJlQeU+WC/HsVH3YyGVMwIcMuMvHJHPLUvD0abBOftptcIWVvPFM86+TO S00g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=s9kzQTeQJ4h6WpYSnNsPxeHmHj0fKNTfgZmJnvNGHZM=; b=t2tB21YBB8swRyzKtKy8VfmIOEuit1qNb5S8mjmrameniIDacK03mgTq2FhUjQ2uG/ LJOATpwL2w0BqBUbYbavhBwwfxsZBBMGhDI7Vyu+nLyfZlo+IISq11mogSFXlMJ92Drt qHKMDo76037wRqkYjri9tAO8mHSCgy69202+hhSurXVOLdYNZp9xzEqlEB60LIe7eKVT DtnXcO4cM6vlYZtkYrf9lSVXNnc/M1uXN9io6LngD6SjfDGZ5z3+w2GUm7TeFjevsC7e PkTctYGNwjzjf8o6SA/RaUnTzu52h9mTaC6Mqg5WKRGg7E2m002ARsneFoCKGCBTNPjS kFlQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xoHV7LlK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p12si13833217ilo.118.2021.03.09.10.21.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:21:03 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xoHV7LlK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40452 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgyc-0007cY-D9 for patch@linaro.org; Tue, 09 Mar 2021 13:21:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46868) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7k-0000OJ-Dd for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:21 -0500 Received: from mail-ot1-x330.google.com ([2607:f8b0:4864:20::330]:36524) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7e-0002nM-5H for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:18 -0500 Received: by mail-ot1-x330.google.com with SMTP id t16so13349691ott.3 for ; Tue, 09 Mar 2021 08:22:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=s9kzQTeQJ4h6WpYSnNsPxeHmHj0fKNTfgZmJnvNGHZM=; b=xoHV7LlKDUSGz3Zt98i8ioNxeu69NIr4vq9Hnfh8QtMbaKIddpsBuAMQDsm2VlcgAG a/zmJaLAaF7vkxENoNX/2L92Z+SCqLoLU9opR3k5YQlT3SxuSyyiOEWHVPKkJf2k9Q6O EKI/VML1ODfF0qqTWMJtkS7vtBo9JO+vzC30X2VAY56abUuQHUN7Xc0Ku+CLmvhp4Ite 6s95XHbwNYlgx63WCaBsZg4iLckTF4a/ztAza1W5TcU4AtqNYQxI+C4e2V2YiKvbTiyF l5kAeFE5yI6TGDkE5rsUwLDwSH7ccZ9j+6Jowc+0gvqock8HFgZrTfKmZGQ03zATWYx3 whdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s9kzQTeQJ4h6WpYSnNsPxeHmHj0fKNTfgZmJnvNGHZM=; b=dHlnnKJwyaBRyk6Ouw7vZ9Y3EnmB7DUFU6ucO05UuseJb0jEuHoF9vDMgjl45WddjV UGT/K3tnlHiNqclRK2BTsFT4pDJIyFxZpiD33/13sJn2dTSr9BeM1c6EUDzcX/P3dNR3 ck/R1tEVgoYyWvGpGf2Zl4y79TnZsu1hXlTAmOzFSYxor7kOlMeJqae0IGhMj+pIPsZ8 A2EYtTMr3f0dIgY4XhrGOdbRcp3AFfMFpX0s478JaQ7gmvatvD/Rwv0h8bvgOc3C1kPS FugEO90aPYXSRdaUgUMkRhbvjsW7tMbiwkk+MQNpvJ5jjSx80XgqKF2hmWgqItOZMZI2 gZ9A== X-Gm-Message-State: AOAM5304l8Fj1OODgih+a0ulvoI9SkjnpPF78AQJnoCFJ/cMLJmqIP7c 0crbaxyjfUQshIRU0giNXRWxxKKmxekK1mOp X-Received: by 2002:a05:6830:1e51:: with SMTP id e17mr24399527otj.292.1615306932220; Tue, 09 Mar 2021 08:22:12 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:11 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 51/78] target/arm: Split out formats for 3 vectors + 1 index Date: Tue, 9 Mar 2021 08:20:14 -0800 Message-Id: <20210309162041.23124-52-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::330; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x330.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Used by FMLA and DOT, but will shortly be used more. Split FMLA from FMLS to avoid an extra sub field; similarly for SDOT from UDOT. Signed-off-by: Richard Henderson --- target/arm/sve.decode | 29 +++++++++++++++++++---------- target/arm/translate-sve.c | 38 ++++++++++++++++++++++++++++---------- 2 files changed, 47 insertions(+), 20 deletions(-) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index eb5d4b052e..87ee5b8629 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -73,6 +73,7 @@ &rprr_s rd pg rn rm s &rprr_esz rd pg rn rm esz &rrrr_esz rd ra rn rm esz +&rrxr_esz rd rn rm ra index esz &rprrr_esz rd pg rn rm ra esz &rpri_esz rd pg rn imm esz &ptrue rd esz pat s @@ -254,6 +255,14 @@ @rrx_d ........ 11 . index:1 rm:4 ...... rn:5 rd:5 \ &rrx_esz esz=3 +# Three registers and a scalar by index +@rrxr_h ........ 0. . .. rm:3 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx index=%index3_22_19 esz=1 +@rrxr_s ........ 10 . index:2 rm:3 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx esz=2 +@rrxr_d ........ 11 . index:1 rm:4 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx esz=3 + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -769,10 +778,10 @@ DOT_zzzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 \ ra=%reg_movprfx # SVE integer dot product (indexed) -DOT_zzxw 01000100 101 index:2 rm:3 00000 u:1 rn:5 rd:5 \ - sz=0 ra=%reg_movprfx -DOT_zzxw 01000100 111 index:1 rm:4 00000 u:1 rn:5 rd:5 \ - sz=1 ra=%reg_movprfx +SDOT_zzxw_s 01000100 .. 1 ..... 000000 ..... ..... @rrxr_s +SDOT_zzxw_d 01000100 .. 1 ..... 000000 ..... ..... @rrxr_d +UDOT_zzxw_s 01000100 .. 1 ..... 000001 ..... ..... @rrxr_s +UDOT_zzxw_d 01000100 .. 1 ..... 000001 ..... ..... @rrxr_d # SVE floating-point complex add (predicated) FCADD 01100100 esz:2 00000 rot:1 100 pg:3 rm:5 rd:5 \ @@ -791,12 +800,12 @@ FCMLA_zzxz 01100100 11 1 index:1 rm:4 0001 rot:2 rn:5 rd:5 \ ### SVE FP Multiply-Add Indexed Group # SVE floating-point multiply-add (indexed) -FMLA_zzxz 01100100 0.1 .. rm:3 00000 sub:1 rn:5 rd:5 \ - ra=%reg_movprfx index=%index3_22_19 esz=1 -FMLA_zzxz 01100100 101 index:2 rm:3 00000 sub:1 rn:5 rd:5 \ - ra=%reg_movprfx esz=2 -FMLA_zzxz 01100100 111 index:1 rm:4 00000 sub:1 rn:5 rd:5 \ - ra=%reg_movprfx esz=3 +FMLA_zzxz 01100100 .. 1 ..... 000000 ..... ..... @rrxr_h +FMLA_zzxz 01100100 .. 1 ..... 000000 ..... ..... @rrxr_s +FMLA_zzxz 01100100 .. 1 ..... 000000 ..... ..... @rrxr_d +FMLS_zzxz 01100100 .. 1 ..... 000001 ..... ..... @rrxr_h +FMLS_zzxz 01100100 .. 1 ..... 000001 ..... ..... @rrxr_s +FMLS_zzxz 01100100 .. 1 ..... 000001 ..... ..... @rrxr_d ### SVE FP Multiply Indexed Group diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 7439dd928b..35d02df493 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3813,26 +3813,34 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) return true; } -static bool trans_DOT_zzxw(DisasContext *s, arg_DOT_zzxw *a) +static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn) { - static gen_helper_gvec_4 * const fns[2][2] = { - { gen_helper_gvec_sdot_idx_b, gen_helper_gvec_sdot_idx_h }, - { gen_helper_gvec_udot_idx_b, gen_helper_gvec_udot_idx_h } - }; - + if (fn == NULL) { + return false; + } if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, - a->ra, a->index); + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); } return true; } +#define DO_RRXR(NAME, FUNC) \ + static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ + { return do_zzxz_ool(s, a, FUNC); } + +DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b) +DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) +DO_RRXR(trans_UDOT_zzxw_s, gen_helper_gvec_udot_idx_b) +DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) + +#undef DO_RRXR /* *** SVE Floating Point Multiply-Add Indexed Group */ -static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) +static bool do_FMLA_zzxz(DisasContext *s, arg_rrxr_esz *a, bool sub) { static gen_helper_gvec_4_ptr * const fns[3] = { gen_helper_gvec_fmla_idx_h, @@ -3847,13 +3855,23 @@ static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) vec_full_reg_offset(s, a->rn), vec_full_reg_offset(s, a->rm), vec_full_reg_offset(s, a->ra), - status, vsz, vsz, (a->index << 1) | a->sub, + status, vsz, vsz, (a->index << 1) | sub, fns[a->esz - 1]); tcg_temp_free_ptr(status); } return true; } +static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) +{ + return do_FMLA_zzxz(s, a, false); +} + +static bool trans_FMLS_zzxz(DisasContext *s, arg_FMLA_zzxz *a) +{ + return do_FMLA_zzxz(s, a, true); +} + /* *** SVE Floating Point Multiply Indexed Group */ From patchwork Tue Mar 9 16:20:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396076 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2822698jai; Tue, 9 Mar 2021 10:33:11 -0800 (PST) X-Google-Smtp-Source: ABdhPJyxtNkZBRlxfuPy0ldX5I7uq6e9suTgvId+PNISfGd9q4ALEW6nr2fpcx4UEZSNCB1hWfQ7 X-Received: by 2002:a05:6638:1653:: with SMTP id a19mr29601336jat.113.1615314791577; Tue, 09 Mar 2021 10:33:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314791; cv=none; d=google.com; s=arc-20160816; b=AvOShV+2GtOQN6s1HbTLHO/OEpRu9t6itLX9exHK+9WpAoaKeT3GfpHq1TyW8g2NAx DRXKA8qLqI0MsWhoB06Zn+Rtq+fip/H3xj2RhVx73S/bpnmXCoYb356K5wIkHQ1Qd3V8 E9/zF0W41lH8xYxngTUivtdD8ecDPfU63VaZCaQ+hJGrPGkfZKPywbDEO9gZnkG8dlNM ITxR2TUUFk/23n6VRSh1izK2Y1Qj31LVHAFPeh0gzacgxXo80YrZwVEz2fRqQk65WYcC it3HfmeZrJWTunNqHV7Kr1Bg4hNqAJbqh9ggxYqYGFDLF0oW2JjJdqkzm6uBVP9wTGy3 nR8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=XJukSUnh/ZapdqYElhaMrdNjfVgk9LJj1DVZruN+l2s=; b=Vt/MMG4ZxZMDKt0Elhb1FajLSH+ljBq1J7Xh7/HAfjEuZ+nVhsG8PAGyoKi0Vpz82R fYfDH5Qn5Zx1KZ9NjLF7MYEri3KjuKyYwrJL7rCDxbSKnM/GqXVBfJcKFF02BgJisM1j 22Jpf1cVfIQAq91jQKSODdagyBv5c8LREia8tr/yCxfop8I10OVFRowu4ju+0SeF7Zbh XePZAFneUHo+3vaql2g5lx9SLuM/nKTvHBqHLWHF0czF5Ypc0ZodzyWqbIwtxFMJ2Hsh dOAhPbnDgEVUnkH233WleroglOuhmHX6q+IuB/OUL7LAlR5inPRWm6I59iD6/Qg7X9oz oHgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NEvaxp3J; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d6si5921952ioy.98.2021.03.09.10.33.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:33:11 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NEvaxp3J; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42592 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhAM-0003r2-VI for patch@linaro.org; Tue, 09 Mar 2021 13:33:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46926) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7p-0000PZ-Qf for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:29 -0500 Received: from mail-ot1-x332.google.com ([2607:f8b0:4864:20::332]:37042) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7e-0002nb-C1 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:24 -0500 Received: by mail-ot1-x332.google.com with SMTP id 75so10096268otn.4 for ; Tue, 09 Mar 2021 08:22:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XJukSUnh/ZapdqYElhaMrdNjfVgk9LJj1DVZruN+l2s=; b=NEvaxp3JgyvtJgZ8a914jHnXXBs9ScOzLYAnhMiVjNu0LrXN6LUUZkxy8EZXnVdeCB YPXBzHuGmdDoRe0v0B1Apqv76LYm2chAz8sg/Nvsnu7RCY+IKaHVaY4j+XGjLdIOuBhC c/BTrJaeUyLcwhbtw1nr2g1M7RiU8owhZEvNrjAoZbxOk5UpzpOLRvMJiAXMp1nHNuGM s2BzSj7E9y5TB7285FVOMQXRnxcMmPRg0PiNsFREfs8YTl2/qs6iN7VOohrPT0SyMSGw X0qoTARqMb0Fhwa4sTkF5DIHXliCjQeup7pLOoCZt41D4hrZ4aTSvecBu1NlmQwFv0mB D9/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XJukSUnh/ZapdqYElhaMrdNjfVgk9LJj1DVZruN+l2s=; b=BFn51HDSbv+HGVZ3y6iC+9Q04et+KqCpY97T70riegBS2/c1lj1zhl2hTVVD1PIVem 3XmCiVWYvFfvZSSn82gxr3Xres7dm28bpD0Xk1Y9cg3rWu6pVc60riUs/94oraBoW3rT 4DX1NXA0/DIesXymOuQXirwCNNP8CbKZhsgPfsI7eryATYserBRnwzedubYJXoTYsh0B 6u40V9UF5VJbDQuUz58OYzDMicCfBzCYLkxGtgjyyPRxUa9alJwEmAQx8yFsKIU+6vFH vQ2iuxGkOTKcLtoJTNnGmpLlHbIfxeQQ56WIN6lUfROdH+iDdtZ2IBqVU9Wk/7xVZOVW UodA== X-Gm-Message-State: AOAM530vaPoXDBKvNNbQANqwRevo6hgictWVWh/o8H5G2nYhDNvPFfpr 70B3NwW2D9gWTcrI6JCrI1if3nvgeEfY4EE6 X-Received: by 2002:a05:6830:1404:: with SMTP id v4mr15878259otp.2.1615306933276; Tue, 09 Mar 2021 08:22:13 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:12 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 52/78] target/arm: Implement SVE2 integer multiply (indexed) Date: Tue, 9 Mar 2021 08:20:15 -0800 Message-Id: <20210309162041.23124-53-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::332; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x332.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 7 +++++++ target/arm/translate-sve.c | 30 ++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 87ee5b8629..5d20392887 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -777,12 +777,19 @@ MUL_zzi 00100101 .. 110 000 110 ........ ..... @rdn_i8s DOT_zzzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 \ ra=%reg_movprfx +#### SVE Multiply - Indexed + # SVE integer dot product (indexed) SDOT_zzxw_s 01000100 .. 1 ..... 000000 ..... ..... @rrxr_s SDOT_zzxw_d 01000100 .. 1 ..... 000000 ..... ..... @rrxr_d UDOT_zzxw_s 01000100 .. 1 ..... 000001 ..... ..... @rrxr_s UDOT_zzxw_d 01000100 .. 1 ..... 000001 ..... ..... @rrxr_d +# SVE2 integer multiply (indexed) +MUL_zzx_h 01000100 .. 1 ..... 111110 ..... ..... @rrx_h +MUL_zzx_s 01000100 .. 1 ..... 111110 ..... ..... @rrx_s +MUL_zzx_d 01000100 .. 1 ..... 111110 ..... ..... @rrx_d + # SVE floating-point complex add (predicated) FCADD 01100100 esz:2 00000 rot:1 100 pg:3 rm:5 rd:5 \ rn=%reg_movprfx diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 35d02df493..ee2ef91355 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3813,6 +3813,10 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) return true; } +/* + * SVE Multiply - Indexed + */ + static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, gen_helper_gvec_4 *fn) { @@ -3836,6 +3840,32 @@ DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) #undef DO_RRXR +static bool do_sve2_zzx_ool(DisasContext *s, arg_rrx_esz *a, + gen_helper_gvec_3 *fn) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, a->index, fn); + } + return true; +} + +#define DO_SVE2_RRX(NAME, FUNC) \ + static bool NAME(DisasContext *s, arg_rrx_esz *a) \ + { return do_sve2_zzx_ool(s, a, FUNC); } + +DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h) +DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s) +DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) + +#undef DO_SVE2_RRX + /* *** SVE Floating Point Multiply-Add Indexed Group */ From patchwork Tue Mar 9 16:20:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396068 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2816376jai; Tue, 9 Mar 2021 10:23:56 -0800 (PST) X-Google-Smtp-Source: ABdhPJxNm36vFZVIg2abyy9+G1d8uwbH2KAezJLSmiuS50etLc8xUAKu11DIhUvnu6R6XmHjGD+z X-Received: by 2002:a92:c644:: with SMTP id 4mr25657679ill.237.1615314236074; Tue, 09 Mar 2021 10:23:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314236; cv=none; d=google.com; s=arc-20160816; b=rS+FERGUsEdXuhr4SkFSOYe5VfsOsSzUExuvy4j1ysfbdvHL7UvWfcrX8X666DEEmK RYLW1BTs9MWDbHV58OjRff46kFZj/FGmIQjI7lZ/Ea682qbTzIuCvD0y1Rx4MDmEanfm gORR7oKW38981bbqNx/uKDJW4jhdsg+hBWSxMcAGZymh0B8wPbIzR8byixtLIagZ8cJa L3WK96pNk/ck745ucCBdRWYkHB3so9Zw8R1pgro4CiidaZI9INZ5DB6Bs5ZkwFoWf7fb IaB/2seQaOJ86ZqaoDJX1iO8D9CmlZfOD1H2/kybzOc3XYV/Nj3SlXEDOBXhqptvwM3k AblA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=0olgZkdMsWEvekxSRg+Mk6d/nWiC65iXDYtyv4A1kpY=; b=hD/WNIgCApUgW0XG5CyLwU50aJ9QG+FxIpZ7i8lWF+kh17rjViHPcX7dB7s9DPFf8O 5IXoM3PQ7B5iyUAxBU20yxOy2WmS8k6O7Z4T/jiIKAmemZ0cAhuW5rjQudK2tTSblWgG EDSyEeKnuEVacw1ueZiTeGmyukX9BS4O8NPHrE1FMGg+61WTFjqUhm1cbS3pgUlbULXc zNzAmiuQwEXZtzLSO8WR3T4QkvXv8a4Vyxy9hskNUhGK4thfmBndy1I/fxZ1QscNj2hE AxFAoRGw4Y5raYka1UmVHEckjF1zV58iiMSZoG0NZuuc/Yb/1eCKU3GGq2eiJYIjKTFv 4MbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mKNJl1GU; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w17si13676961jai.126.2021.03.09.10.23.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:23:56 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mKNJl1GU; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:48700 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJh1P-0002kp-GO for patch@linaro.org; Tue, 09 Mar 2021 13:23:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46980) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7t-0000Pu-HX for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:31 -0500 Received: from mail-ot1-x32b.google.com ([2607:f8b0:4864:20::32b]:46164) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7g-0002nm-1a for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:25 -0500 Received: by mail-ot1-x32b.google.com with SMTP id r24so6273810otq.13 for ; Tue, 09 Mar 2021 08:22:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0olgZkdMsWEvekxSRg+Mk6d/nWiC65iXDYtyv4A1kpY=; b=mKNJl1GUCbXJOkcwk5bvuCPDXnf4TqSJf/thUR3zitcEI2Sttr80kgloiqe79QR6Jg ajhUAZpvcAM57d/qdVQHbjNARH+1u9j89T59tdYQEKwzZSq9+LRriC8Gblzp/9RFAcp+ m6k+2kU/cr+HV2XISkwWRIau2keuqVV5F5vWhwplASKARLp9xhIRs0DmiZP7bljBOHIF Ivk/OZ6ILbsI5cbi04Ya+o/DD5hf+Q4ql4RHSlsGddA4keqgXlAwg63fQMw9f+6yxHPW nwL/zUmyVpRSQpHXlfh8fsvEmUgDF0cgonEC639MKWhJ8exCaF+PoB0j45CEmlQeBq8P esyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0olgZkdMsWEvekxSRg+Mk6d/nWiC65iXDYtyv4A1kpY=; b=NiOs+UTGDkw5QpZKcBv4t0j3v0guSvm4VSwAJG8ucNE1RPBCRCAtlp4/JxeOIJcIWh t+6muKD6N3TTRcz5wTAYtKYTp57qjMbZETXQRNR4jQPPnCeXJMYoIlRQWkCM8QJWP5ZE 8nPUPvV2l3EPl+3ONdpE6CxNpkpR9hiEg7nld80TeElOCs+o9l0v1fUrya2tr54kUKvx DeMDrfb1Fu6ORswuK05GGXQuSF7j0O0mbL1544oNx9bRjppO77n4BkvghHXenLKQLyn2 vMRz7j9mCSKJ2eYuehsNl2NDV3XNjcxIwrC9nHv5a0wAV3hmH/WVw4OOQxuXns24g5Jz pNig== X-Gm-Message-State: AOAM531sIBcJO3YtaQlzB1Vq/Sh32Q0HG+QWiv+MkP2q9eDMrJyYP5cP NYesk5wFT2dAspAhtGao8HqmnvObSPLV9vie X-Received: by 2002:a9d:4b83:: with SMTP id k3mr23919051otf.21.1615306934421; Tue, 09 Mar 2021 08:22:14 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:14 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 53/78] target/arm: Implement SVE2 integer multiply-add (indexed) Date: Tue, 9 Mar 2021 08:20:16 -0800 Message-Id: <20210309162041.23124-54-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32b; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 8 ++++++++ target/arm/translate-sve.c | 23 +++++++++++++++++++++++ 2 files changed, 31 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 5d20392887..8d761edb74 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -785,6 +785,14 @@ SDOT_zzxw_d 01000100 .. 1 ..... 000000 ..... ..... @rrxr_d UDOT_zzxw_s 01000100 .. 1 ..... 000001 ..... ..... @rrxr_s UDOT_zzxw_d 01000100 .. 1 ..... 000001 ..... ..... @rrxr_d +# SVE2 integer multiply-add (indexed) +MLA_zzxz_h 01000100 .. 1 ..... 000010 ..... ..... @rrxr_h +MLA_zzxz_s 01000100 .. 1 ..... 000010 ..... ..... @rrxr_s +MLA_zzxz_d 01000100 .. 1 ..... 000010 ..... ..... @rrxr_d +MLS_zzxz_h 01000100 .. 1 ..... 000011 ..... ..... @rrxr_h +MLS_zzxz_s 01000100 .. 1 ..... 000011 ..... ..... @rrxr_s +MLS_zzxz_d 01000100 .. 1 ..... 000011 ..... ..... @rrxr_d + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 .. 1 ..... 111110 ..... ..... @rrx_h MUL_zzx_s 01000100 .. 1 ..... 111110 ..... ..... @rrx_s diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ee2ef91355..bee3e5a235 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3866,6 +3866,29 @@ DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) #undef DO_SVE2_RRX +static bool do_sve2_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zzxz_ool(s, a, fn); +} + +#define DO_SVE2_RRXR(NAME, FUNC) \ + static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ + { return do_sve2_zzxz_ool(s, a, FUNC); } + +DO_SVE2_RRXR(trans_MLA_zzxz_h, gen_helper_gvec_mla_idx_h) +DO_SVE2_RRXR(trans_MLA_zzxz_s, gen_helper_gvec_mla_idx_s) +DO_SVE2_RRXR(trans_MLA_zzxz_d, gen_helper_gvec_mla_idx_d) + +DO_SVE2_RRXR(trans_MLS_zzxz_h, gen_helper_gvec_mls_idx_h) +DO_SVE2_RRXR(trans_MLS_zzxz_s, gen_helper_gvec_mls_idx_s) +DO_SVE2_RRXR(trans_MLS_zzxz_d, gen_helper_gvec_mls_idx_d) + +#undef DO_SVE2_RRXR + /* *** SVE Floating Point Multiply-Add Indexed Group */ From patchwork Tue Mar 9 16:20:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396053 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2805855jai; Tue, 9 Mar 2021 10:09:47 -0800 (PST) X-Google-Smtp-Source: ABdhPJxYf4BW7ODwzUV9plCtcKpsKJhr86BqR5uoNnjWTOfx7Zxs3Ly9wyNN88JcgxJVmjNk3oif X-Received: by 2002:a92:90f:: with SMTP id y15mr6634558ilg.143.1615313387538; Tue, 09 Mar 2021 10:09:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313387; cv=none; d=google.com; s=arc-20160816; b=Nym2fqFJNBunM/6Xt+M++D3GwD9DV+q/cLbmzlene1LR+/weL9Jfxu8/oyYXDIepaU ZpgMTl+KcfBIIcZYtLdv77Bsj8wqXn4IoHmZNB2SBR9ccz1boLtioyL9ydppTC0RBG7M PqulcGSCETWqpBjmn/imYvitXbSKc/ZpNirdNCWC4h9n/rGBF2D5QIO9cn4yXrMxnvjt +5KBLJJKLNHdO5vGixLhCqMru9iP3qTtEhpA02feml+4YpqkeDSOicIjagSn8EC1D3uj L52OpysEjIaRYhe78HpmT6KPgA3XrQeYWbCHMf/jHYY7EJb07Pq/YNpxFY9Z00mcK6y6 Szvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=es5j0v4Vs43+K3nW8rNBcTSafAsKaEV9dJ00NX0aUAc=; b=jqcjxeNW6A1uFJXogpAB0QzVuWvkhH7Pn3XU46a3L8rjZRE+yN2HMuI8tOTxAcLCPD y8tfDP9OQcOW3wDsG/0Yv8I/i1m+aG9jWxy71P1aV3JAMFnv7KTqMJR/dn/sz0xpLnrQ 3UHIvUep79y2tz+2amCRbaIbAlcssLMG0vTpKJ5b9Lh2RffXbWIcQ7PeUJTK7s/GOudi YAj5SzaZMLrQwE17Wfz2tttrsIecE84X3qK4lAxeFTBspAWjcPwSsDdw+DwzxlgTi2Wf 4QFmktcnT3CUu8P3YS/jIH+dCgvcJs9u2Im5DsMNver0dC68s3iHDWUzvs3pIIZnV5P0 dvmA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TgiY6IBZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b7si14389501ilv.124.2021.03.09.10.09.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:09:47 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TgiY6IBZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43880 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgni-0003Qd-UQ for patch@linaro.org; Tue, 09 Mar 2021 13:09:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47170) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf85-0000Rc-4k for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:41 -0500 Received: from mail-oi1-x22c.google.com ([2607:f8b0:4864:20::22c]:32785) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7h-0002oA-Vm for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:37 -0500 Received: by mail-oi1-x22c.google.com with SMTP id d16so6243874oic.0 for ; Tue, 09 Mar 2021 08:22:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=es5j0v4Vs43+K3nW8rNBcTSafAsKaEV9dJ00NX0aUAc=; b=TgiY6IBZXp4XronzTC4hXDfPcS00gDmsdiJUIlwxce9HfCxQgqJzaOvBXS/6K6mDhl /jjgDRt6LOCd7PkRtCBUrzohD2KBQN8SNpkgoWs0HJOuKsrVEb7zw2bt559HLs2/GmXl j/iad949Q0qKWPO78K7OnqoHa4IivHYtqy42kfLZaZOZNU/qQ+nDISfL3Kebq8SLhwtk ju3v//xru2ny9f0CavzPtLmeVHg8mQePQJA0nG94D1gLS+6dTkvv2e1Xi9mnx82QpLkO BcbeNCBMHGwKp/S0MTcr6C71cxlm905JeoGqNA5NeTtwvIUWJPD8m0mzAJAgcXiJbGn8 KNVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=es5j0v4Vs43+K3nW8rNBcTSafAsKaEV9dJ00NX0aUAc=; b=lxJLZnZVs4EXJbqfBIj8gUlQQ97/7VoPO+g5J33HNKIi3jO0cis7rB6aMU2U7lUMqP 0V3+bLj43DqNnTHlWr85KV9PpSpMh9XrD8p86/53Ij+gp9HtsI23hxB9pe6WIFI9gTXG 14QG8tBpNebwzm+O7lfg7ctfTYle2ZPvRtJWzEqaxdo0faqurgDSm2nlUKjMV7kkaVQx TlP1vF1wZ34+YYMKwXE5h3xpjDn3MB+g/rMbh0KNCEMG4TfbvBeTn69Kas05aiwytpOj R5IQ5mybi1uMs44j5HpERcAH1mTSSTabsyRm0SgBPTuprgt+emdh0gsUsdiy2mkcc5cu nAsg== X-Gm-Message-State: AOAM533HO81EchtDVbYxlgQxAGytPpK7AVa3JNkgkfePy/ysCLC6dFEv hRjd6pG99AHNMcUGOEEc6YziQNIsBaKvOm9i X-Received: by 2002:aca:bc89:: with SMTP id m131mr3354028oif.62.1615306935506; Tue, 09 Mar 2021 08:22:15 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:15 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 54/78] target/arm: Implement SVE2 saturating multiply-add high (indexed) Date: Tue, 9 Mar 2021 08:20:17 -0800 Message-Id: <20210309162041.23124-55-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22c; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 14 +++++++++++++ target/arm/sve.decode | 8 ++++++++ target/arm/sve_helper.c | 40 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 8 ++++++++ 4 files changed, 70 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 7e99dcd119..fe67574741 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2665,3 +2665,17 @@ DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_6(fmmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(fmmla_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 8d761edb74..786f6837c5 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -793,6 +793,14 @@ MLS_zzxz_h 01000100 .. 1 ..... 000011 ..... ..... @rrxr_h MLS_zzxz_s 01000100 .. 1 ..... 000011 ..... ..... @rrxr_s MLS_zzxz_d 01000100 .. 1 ..... 000011 ..... ..... @rrxr_d +# SVE2 saturating multiply-add high (indexed) +SQRDMLAH_zzxz_h 01000100 .. 1 ..... 000100 ..... ..... @rrxr_h +SQRDMLAH_zzxz_s 01000100 .. 1 ..... 000100 ..... ..... @rrxr_s +SQRDMLAH_zzxz_d 01000100 .. 1 ..... 000100 ..... ..... @rrxr_d +SQRDMLSH_zzxz_h 01000100 .. 1 ..... 000101 ..... ..... @rrxr_h +SQRDMLSH_zzxz_s 01000100 .. 1 ..... 000101 ..... ..... @rrxr_s +SQRDMLSH_zzxz_d 01000100 .. 1 ..... 000101 ..... ..... @rrxr_d + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 .. 1 ..... 111110 ..... ..... @rrx_h MUL_zzx_s 01000100 .. 1 ..... 111110 ..... ..... @rrx_s diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index fcfb7d1f41..b4726bb6c5 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1487,9 +1487,49 @@ DO_CMLA(sve2_sqrdcmlah_zzzz_h, int16_t, H2, DO_SQRDMLAH_H) DO_CMLA(sve2_sqrdcmlah_zzzz_s, int32_t, H4, DO_SQRDMLAH_S) DO_CMLA(sve2_sqrdcmlah_zzzz_d, int64_t, , DO_SQRDMLAH_D) +#undef DO_SQRDMLAH_B +#undef DO_SQRDMLAH_H +#undef DO_SQRDMLAH_S +#undef DO_SQRDMLAH_D #undef do_cmla #undef DO_CMLA +#define DO_ZZXZ(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, idx = simd_data(desc); \ + TYPE *d = vd, *a = va, *n = vn, *m = (TYPE *)vm + H(idx); \ + for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ + TYPE mm = m[i]; \ + for (j = 0; j < segment; j++) { \ + d[i + j] = OP(n[i + j], mm, a[i + j]); \ + } \ + } \ +} + +#define DO_SQRDMLAH_H(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_h(N, M, A, false, true, &discard); }) +#define DO_SQRDMLAH_S(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_s(N, M, A, false, true, &discard); }) +#define DO_SQRDMLAH_D(N, M, A) do_sqrdmlah_d(N, M, A, false, true) + +DO_ZZXZ(sve2_sqrdmlah_idx_h, int16_t, H2, DO_SQRDMLAH_H) +DO_ZZXZ(sve2_sqrdmlah_idx_s, int32_t, H4, DO_SQRDMLAH_S) +DO_ZZXZ(sve2_sqrdmlah_idx_d, int64_t, , DO_SQRDMLAH_D) + +#define DO_SQRDMLSH_H(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_h(N, M, A, true, true, &discard); }) +#define DO_SQRDMLSH_S(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_s(N, M, A, true, true, &discard); }) +#define DO_SQRDMLSH_D(N, M, A) do_sqrdmlah_d(N, M, A, true, true) + +DO_ZZXZ(sve2_sqrdmlsh_idx_h, int16_t, H2, DO_SQRDMLSH_H) +DO_ZZXZ(sve2_sqrdmlsh_idx_s, int32_t, H4, DO_SQRDMLSH_S) +DO_ZZXZ(sve2_sqrdmlsh_idx_d, int64_t, , DO_SQRDMLSH_D) + +#undef DO_ZZXZ + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index bee3e5a235..8e3c6df7a2 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3887,6 +3887,14 @@ DO_SVE2_RRXR(trans_MLS_zzxz_h, gen_helper_gvec_mls_idx_h) DO_SVE2_RRXR(trans_MLS_zzxz_s, gen_helper_gvec_mls_idx_s) DO_SVE2_RRXR(trans_MLS_zzxz_d, gen_helper_gvec_mls_idx_d) +DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_h, gen_helper_sve2_sqrdmlah_idx_h) +DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_s, gen_helper_sve2_sqrdmlah_idx_s) +DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_d, gen_helper_sve2_sqrdmlah_idx_d) + +DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_h, gen_helper_sve2_sqrdmlsh_idx_h) +DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_s, gen_helper_sve2_sqrdmlsh_idx_s) +DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d) + #undef DO_SVE2_RRXR /* From patchwork Tue Mar 9 16:20:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396082 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2828616jai; Tue, 9 Mar 2021 10:41:56 -0800 (PST) X-Google-Smtp-Source: ABdhPJz0ejsxfMVQw5rn5Jd4FkNO7z4EoHqM9yawm7TcK5S4WgCNeCWerFz96AXmctV7LeYnI/Sw X-Received: by 2002:a92:cd0c:: with SMTP id z12mr26120707iln.109.1615315316239; Tue, 09 Mar 2021 10:41:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615315316; cv=none; d=google.com; s=arc-20160816; b=fvI5v4aQzapGyUugN/mp6FEqiaULuPHrq0mj1Z2eFSBkFbX1RuzP9+mZbGrQyFFh6S gseuit2lL8Fxs4Y5fk0RJgC9yxyjr4w3xgi7D0HW/Z5taATBvhKR/xAXw4asmDCNja/c J2+0XK1U8xb2NaHR71mNxptJDiXr9QWFEfcIa0mTobL3WJ4BYxMO8oQcMC2//bHiqjFT SKRWfGcn8sLzhuwFZ8321QqdJA0wRdM3WAMRiNGr3d5QCTvdafE7x1BMdirWSUjjGnjk kYAhCz+T9G6xrRe0KDC4NeIO7Mf5eCOla4EfFTfTWL9k6CSoKYVBMzLxeWTc8Vlw3PLx tL5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=z4dwnR3n8lYNVy9WPNW7GU/Li5hUkOXVq2BJazu+LS8=; b=H2xZJdGz2L+KIQigbCqZ5D9OAoTOJp0IC6lY03py6+BEumg86PXdhlGVNCIVpa7LeN ifm0KXLOi5GzgUANu+kQt6xEgWcy6JLQpXxUznSVmYNrKXPHnMkvg71ve6YMxWkMsCSU zoUdwCfDD3NSz1O85fOd+zL2FFiI700zBa/7BCUtKif5LOTd/iGwLSqIdaiphdxdfB9s f5lncxzyuTPP7cqQtjkVAIkUXFwJkqpK1CARwxw14pImT2rvyvOpilE/eVxU2ux6trhf 6Ttnc2ecapZuOo+lUaTSopbeV1J5yr0eDlJYqYfHj/zeFQQa8my9s0S7WiIs1jTNuYQt 2OuQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="rHA17z/V"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x8si13995185jao.12.2021.03.09.10.41.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:41:56 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="rHA17z/V"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60906 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhIp-00036i-Jl for patch@linaro.org; Tue, 09 Mar 2021 13:41:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47086) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7y-0000QO-0q for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:39 -0500 Received: from mail-ot1-x334.google.com ([2607:f8b0:4864:20::334]:42031) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7h-0002oN-WB for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:31 -0500 Received: by mail-ot1-x334.google.com with SMTP id e45so13337022ote.9 for ; Tue, 09 Mar 2021 08:22:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z4dwnR3n8lYNVy9WPNW7GU/Li5hUkOXVq2BJazu+LS8=; b=rHA17z/V4gLqSLYcOpZRMFW42TZMRnbOaVByGXDQw9Tytp+hObWiOaW0sDl0XX9Cym bJaohtAjby4i0ZW6xFHz4VDU4I8KiAXxSyU98QNkXdtU9BrXGFKIHUZNqBxZv8rb0Wn5 lYxw2FQcGhQiWd2glvD+Q1MOqRWFCZ6qkXXWmpXBnhWrPtfjUFFZzj3r1icRWhrz6W2H g8h5utu3bDc9WpLzdgNH2vBdCiSJsMbGSKBlzGdduxZufNV6rDEcz/NBPLH2ttuvFT7r QzWI/Po+9Xyaek58bhbnEUHDbn3A9+dfN4+9VQAwwtsJh2bCW1ulRgqYd7wdKIpNr7o+ M/rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z4dwnR3n8lYNVy9WPNW7GU/Li5hUkOXVq2BJazu+LS8=; b=t/nAOEoevs0n0Ixhxm06G6Wf3Xv9BTyJiEOVQFJ9sX2VLUBFTRjPGZ1A+ZdSltmKya dDtEBxDI7POuC/AX6SaIKCZSIRqe7L13Q6p7oGt3Mgo2lDWeV7RH+e2K2z34Qrdaqqxp N+drW+eCuCdoGZ+Kf1ZlHZ1QQjRO0e89siSIKIC4KYaoQyj3DwebiMzwqtVS2P23QEBi EJXieqMOdD5fUhlyCbFJlDrMpDBviwjibArPvvYx8AichCuAOSJWx8XZfpQ3Q836Eeb/ PJhOJYl5e5ouPdNiKt7ti8ykjfTfuiJJZaokkZjRmNgYaMCDQIAhT5AfOxj3CDTurOkZ sI/A== X-Gm-Message-State: AOAM5313cD2xnWVPMdv+7bnmxrAV8orfiU9EY6lEn7t7Xv0dyKdd9o4D yV98t7EohuT1ZZsWhCi6ikad8ZKM6fhQm0Wd X-Received: by 2002:a05:6830:144e:: with SMTP id w14mr19933304otp.0.1615306936622; Tue, 09 Mar 2021 08:22:16 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:16 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 55/78] target/arm: Implement SVE2 saturating multiply-add (indexed) Date: Tue, 9 Mar 2021 08:20:18 -0800 Message-Id: <20210309162041.23124-56-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::334; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x334.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 9 +++++++++ target/arm/sve.decode | 18 ++++++++++++++++++ target/arm/sve_helper.c | 30 ++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 32 ++++++++++++++++++++++++-------- 4 files changed, 81 insertions(+), 8 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index fe67574741..08398800bd 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2679,3 +2679,12 @@ DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqdmlal_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlal_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 786f6837c5..6900e79492 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -30,6 +30,8 @@ %size_23 23:2 %dtype_23_13 23:2 13:2 %index3_22_19 22:1 19:2 +%index3_19_11 19:2 11:1 +%index2_20_11 20:1 11:1 # A combination of tsz:imm3 -- extract esize. %tszimm_esz 22:2 5:5 !function=tszimm_esz @@ -263,6 +265,12 @@ @rrxr_d ........ 11 . index:1 rm:4 ...... rn:5 rd:5 \ &rrxr_esz ra=%reg_movprfx esz=3 +# Three registers and a scalar by index, wide +@rrxw_s ........ 10 ... rm:3 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx index=%index3_19_11 esz=2 +@rrxw_d ........ 11 .. rm:4 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx index=%index2_20_11 esz=3 + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -801,6 +809,16 @@ SQRDMLSH_zzxz_h 01000100 .. 1 ..... 000101 ..... ..... @rrxr_h SQRDMLSH_zzxz_s 01000100 .. 1 ..... 000101 ..... ..... @rrxr_s SQRDMLSH_zzxz_d 01000100 .. 1 ..... 000101 ..... ..... @rrxr_d +# SVE2 saturating multiply-add (indexed) +SQDMLALB_zzxw_s 01000100 .. 1 ..... 0010.0 ..... ..... @rrxw_s +SQDMLALB_zzxw_d 01000100 .. 1 ..... 0010.0 ..... ..... @rrxw_d +SQDMLALT_zzxw_s 01000100 .. 1 ..... 0010.1 ..... ..... @rrxw_s +SQDMLALT_zzxw_d 01000100 .. 1 ..... 0010.1 ..... ..... @rrxw_d +SQDMLSLB_zzxw_s 01000100 .. 1 ..... 0011.0 ..... ..... @rrxw_s +SQDMLSLB_zzxw_d 01000100 .. 1 ..... 0011.0 ..... ..... @rrxw_d +SQDMLSLT_zzxw_s 01000100 .. 1 ..... 0011.1 ..... ..... @rrxw_s +SQDMLSLT_zzxw_d 01000100 .. 1 ..... 0011.1 ..... ..... @rrxw_d + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 .. 1 ..... 111110 ..... ..... @rrx_h MUL_zzx_s 01000100 .. 1 ..... 111110 ..... ..... @rrx_s diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index b4726bb6c5..6ad1158455 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1530,6 +1530,36 @@ DO_ZZXZ(sve2_sqrdmlsh_idx_d, int64_t, , DO_SQRDMLSH_D) #undef DO_ZZXZ +#define DO_ZZXW(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + intptr_t idx = extract32(desc, SIMD_DATA_SHIFT + 1, 3) * sizeof(TYPEN); \ + for (i = 0; i < oprsz; i += 16) { \ + TYPEW mm = *(TYPEN *)(vm + i + idx); \ + for (j = 0; j < 16; j += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + j + sel)); \ + TYPEW aa = *(TYPEW *)(va + HW(i + j)); \ + *(TYPEW *)(vd + HW(i + j)) = OP(nn, mm, aa); \ + } \ + } \ +} + +#define DO_SQDMLAL_S(N, M, A) DO_SQADD_S(A, do_sqdmull_s(N, M)) +#define DO_SQDMLAL_D(N, M, A) do_sqadd_d(A, do_sqdmull_d(N, M)) + +DO_ZZXW(sve2_sqdmlal_idx_s, int32_t, int16_t, H1_4, H1_2, DO_SQDMLAL_S) +DO_ZZXW(sve2_sqdmlal_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLAL_D) + +#define DO_SQDMLSL_S(N, M, A) DO_SQSUB_S(A, do_sqdmull_s(N, M)) +#define DO_SQDMLSL_D(N, M, A) do_sqsub_d(A, do_sqdmull_d(N, M)) + +DO_ZZXW(sve2_sqdmlsl_idx_s, int32_t, int16_t, H1_4, H1_2, DO_SQDMLSL_S) +DO_ZZXW(sve2_sqdmlsl_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLSL_D) + +#undef DO_ZZXW + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 8e3c6df7a2..a3261561c0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3817,21 +3817,21 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) * SVE Multiply - Indexed */ -static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, - gen_helper_gvec_4 *fn) +static bool do_zzxz_data(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn, int data) { if (fn == NULL) { return false; } if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); } return true; } #define DO_RRXR(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_zzxz_ool(s, a, FUNC); } + { return do_zzxz_data(s, a, FUNC, a->index); } DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b) DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) @@ -3866,18 +3866,18 @@ DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) #undef DO_SVE2_RRX -static bool do_sve2_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, - gen_helper_gvec_4 *fn) +static bool do_sve2_zzxz_data(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn, int data) { if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zzxz_ool(s, a, fn); + return do_zzxz_data(s, a, fn, data); } #define DO_SVE2_RRXR(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_sve2_zzxz_ool(s, a, FUNC); } + { return do_sve2_zzxz_data(s, a, FUNC, a->index); } DO_SVE2_RRXR(trans_MLA_zzxz_h, gen_helper_gvec_mla_idx_h) DO_SVE2_RRXR(trans_MLA_zzxz_s, gen_helper_gvec_mla_idx_s) @@ -3897,6 +3897,22 @@ DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d) #undef DO_SVE2_RRXR +#define DO_SVE2_RRXR_TB(NAME, FUNC, TOP) \ + static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ + { return do_sve2_zzxz_data(s, a, FUNC, (a->index << 1) | TOP); } + +DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, false) +DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, false) +DO_SVE2_RRXR_TB(trans_SQDMLALT_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, true) +DO_SVE2_RRXR_TB(trans_SQDMLALT_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, true) + +DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, false) +DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, false) +DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, true) +DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, true) + +#undef DO_SVE2_RRXR_TB + /* *** SVE Floating Point Multiply-Add Indexed Group */ From patchwork Tue Mar 9 16:20:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396079 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2824571jai; Tue, 9 Mar 2021 10:36:00 -0800 (PST) X-Google-Smtp-Source: ABdhPJwpTlKKoXKO2AzKD5ZjYKRkiH8JdLVdjS7JyA6eS5wlRJnX9bkMoiwS92t0+2OblOFyeA/I X-Received: by 2002:a92:5211:: with SMTP id g17mr26807409ilb.116.1615314960815; Tue, 09 Mar 2021 10:36:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314960; cv=none; d=google.com; s=arc-20160816; b=LZtSvRTKSzhSNH0yavF3upUwTl5sUl9BPlhUMg4PWhJrLhGQ3p3IV8ANAZ0NP5KOTk nqmDTbmNJCavG4wSHB85A7LQEaualLKPwD6LZVIZ2XJMdDNuHOepzMeSBQzEhCTMQuXi 3MmOY3fFyb2WT9xv84AicrG4gFawuOwjYmrVmsYdKXUaeYOjVelnZA6xpuDDSckuvpmp NmS3Xm8UQDYbUYKE4cAsTeZuN39UcpLmqwSw8Z9BUef590OBNNj1D20UWOR6XtklPFQU z/L6KtgEEGBQJ6/Au3iaOfdVmnTQeIPpdyzDf+S5KUd/oxepTXIjL3lMFCAMRMoxhsTl RgZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=gAITA8fVumfgJ7BWNimYNqBynCeOl0cQ9txvmPBZezA=; b=VJhxlgeVFXxDfmVwgcGx5G+cRVgF38J2oGx0zMpZxVISieDY4uvs/NG9njAtlDk3IU qEwveOyZRkn9svMRxA7k5dAGVJTYm+OhB+rTpVr4GkHAXUm9tnA8kul02J6WStoXQVBc 5WR3fvLAIMUiVj+wuJlgJbUf4uts9n5APCT5JHoDl98KS8I2w75Tttr7f3wmc2u1XAuO 0MdtfrgmxlgPbEH0icHHGDVfHzbUGlWlBd9NhvoG19fCLbIQIxxADZ2ufBJgwmZR3I1D nw4vCOAWzxL3pZfJ4JA8MWfbjwUGqbyzWFmnXEbXssuyuJUGZMJcCh+jTcu0j8IS4Pz/ ZUYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y9D4dGUj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a5si14993331ilp.121.2021.03.09.10.36.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:36:00 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y9D4dGUj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52304 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhD6-0007rq-5H for patch@linaro.org; Tue, 09 Mar 2021 13:36:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47090) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf7y-0000Qb-1F for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:39 -0500 Received: from mail-oi1-x22e.google.com ([2607:f8b0:4864:20::22e]:41373) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7j-0002oY-IB for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:33 -0500 Received: by mail-oi1-x22e.google.com with SMTP id y131so12658601oia.8 for ; Tue, 09 Mar 2021 08:22:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gAITA8fVumfgJ7BWNimYNqBynCeOl0cQ9txvmPBZezA=; b=Y9D4dGUjAu1wb98gLhfsbm1/jkqbWR2PIVsHfHeOs2hREjuZqpjqrATb+PS8aDu7oI 53uOgm9jETIlm578QCwp8oW8l1LjhR5J8axvykjMpWjfKawFR+wS8mpr9tRiiTBBf83Z FYAENDYDd3jAYVxc9v6vS9C+5FxzZe//qkB/nLriMkMyeoWT0P8FUfkkAa4MS7MwKDc8 9l7XnXslOPr31+WRGVauqGSs8gFQ/eWuHk6q5wrhJGac/dCSnvsY6+si1Xwi4Bxw97qU 22W5bsk0D8yXFesxHruum9qqA9SoEWhdvLO+BfD6gfjOAj1oKbTr8r5o6Q+ZQwp2r1q1 WSfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gAITA8fVumfgJ7BWNimYNqBynCeOl0cQ9txvmPBZezA=; b=q0KE+gTLmADFSw8sUdSyuyTIyHW38BMUOdB19xpKkmm2KaTRQbL4zvJjGIQGehPjQw BWxTZIKG6LI1wp0x68/rotkH1kRXoM7IHkOXXLjV41Ht4EOaWdQd+L/Xrr7iu6TXLH2J k+1TzD74354Bd8AIAg2SPDgaKj65ncjfF+YeTYa8U20l30MHsIIJh1nFqa0JcxP0NAde X+XP4njEzsIBv2Rp95zRu0/AUjTKl6sy/+4apbkt06yZ11ZMRvbiXvnmBpPoNz8mMTD2 KG+czMKSBPwcG+iNGfNnLBBXyDoKrCE8TthRNBc3SpIxTRhGjNrBheW0ivz8u0vZYvjC R8Cg== X-Gm-Message-State: AOAM533GjvplWiTbaL8D8ygm7uNlNES4zT50Rgk2wcUauK/mC+88EcDG h+WmzyHAT+MeiHnON5rns0YcVql+2iiKhD1O X-Received: by 2002:aca:c2d7:: with SMTP id s206mr3317786oif.95.1615306937661; Tue, 09 Mar 2021 08:22:17 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:17 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 56/78] target/arm: Implement SVE2 integer multiply long (indexed) Date: Tue, 9 Mar 2021 08:20:19 -0800 Message-Id: <20210309162041.23124-57-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22e; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 16 ++++++++++++++++ target/arm/sve_helper.c | 23 +++++++++++++++++++++++ target/arm/translate-sve.c | 24 ++++++++++++++++++++---- 4 files changed, 64 insertions(+), 4 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 08398800bd..a3690082af 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2688,3 +2688,8 @@ DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_smull_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_smull_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_umull_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_umull_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 6900e79492..ebb892748b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -257,6 +257,12 @@ @rrx_d ........ 11 . index:1 rm:4 ...... rn:5 rd:5 \ &rrx_esz esz=3 +# Two registers and a scalar by index, wide +@rrxl_s ........ 10 ... rm:3 ...... rn:5 rd:5 \ + &rrx_esz index=%index3_19_11 esz=2 +@rrxl_d ........ 11 .. rm:4 ...... rn:5 rd:5 \ + &rrx_esz index=%index2_20_11 esz=3 + # Three registers and a scalar by index @rrxr_h ........ 0. . .. rm:3 ...... rn:5 rd:5 \ &rrxr_esz ra=%reg_movprfx index=%index3_22_19 esz=1 @@ -819,6 +825,16 @@ SQDMLSLB_zzxw_d 01000100 .. 1 ..... 0011.0 ..... ..... @rrxw_d SQDMLSLT_zzxw_s 01000100 .. 1 ..... 0011.1 ..... ..... @rrxw_s SQDMLSLT_zzxw_d 01000100 .. 1 ..... 0011.1 ..... ..... @rrxw_d +# SVE2 integer multiply long (indexed) +SMULLB_zzx_s 01000100 .. 1 ..... 1100.0 ..... ..... @rrxl_s +SMULLB_zzx_d 01000100 .. 1 ..... 1100.0 ..... ..... @rrxl_d +SMULLT_zzx_s 01000100 .. 1 ..... 1100.1 ..... ..... @rrxl_s +SMULLT_zzx_d 01000100 .. 1 ..... 1100.1 ..... ..... @rrxl_d +UMULLB_zzx_s 01000100 .. 1 ..... 1101.0 ..... ..... @rrxl_s +UMULLB_zzx_d 01000100 .. 1 ..... 1101.0 ..... ..... @rrxl_d +UMULLT_zzx_s 01000100 .. 1 ..... 1101.1 ..... ..... @rrxl_s +UMULLT_zzx_d 01000100 .. 1 ..... 1101.1 ..... ..... @rrxl_d + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 .. 1 ..... 111110 ..... ..... @rrx_h MUL_zzx_s 01000100 .. 1 ..... 111110 ..... ..... @rrx_s diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 6ad1158455..b64e1f243d 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1560,6 +1560,29 @@ DO_ZZXW(sve2_sqdmlsl_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLSL_D) #undef DO_ZZXW +#define DO_ZZX(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + intptr_t idx = extract32(desc, SIMD_DATA_SHIFT + 1, 3) * sizeof(TYPEN); \ + for (i = 0; i < oprsz; i += 16) { \ + TYPEW mm = *(TYPEN *)(vm + i + idx); \ + for (j = 0; j < 16; j += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + j + sel)); \ + *(TYPEW *)(vd + HW(i + j)) = OP(nn, mm); \ + } \ + } \ +} + +DO_ZZX(sve2_smull_idx_s, int32_t, int16_t, H1_4, H1_2, DO_MUL) +DO_ZZX(sve2_smull_idx_d, int64_t, int32_t, , H1_4, DO_MUL) + +DO_ZZX(sve2_umull_idx_s, uint32_t, uint16_t, H1_4, H1_2, DO_MUL) +DO_ZZX(sve2_umull_idx_d, uint64_t, uint32_t, , H1_4, DO_MUL) + +#undef DO_ZZX + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index a3261561c0..6376a21b98 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3840,8 +3840,8 @@ DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) #undef DO_RRXR -static bool do_sve2_zzx_ool(DisasContext *s, arg_rrx_esz *a, - gen_helper_gvec_3 *fn) +static bool do_sve2_zzx_data(DisasContext *s, arg_rrx_esz *a, + gen_helper_gvec_3 *fn, int data) { if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { return false; @@ -3851,14 +3851,14 @@ static bool do_sve2_zzx_ool(DisasContext *s, arg_rrx_esz *a, tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), vec_full_reg_offset(s, a->rn), vec_full_reg_offset(s, a->rm), - vsz, vsz, a->index, fn); + vsz, vsz, data, fn); } return true; } #define DO_SVE2_RRX(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrx_esz *a) \ - { return do_sve2_zzx_ool(s, a, FUNC); } + { return do_sve2_zzx_data(s, a, FUNC, a->index); } DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h) DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s) @@ -3866,6 +3866,22 @@ DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) #undef DO_SVE2_RRX +#define DO_SVE2_RRX_TB(NAME, FUNC, TOP) \ + static bool NAME(DisasContext *s, arg_rrx_esz *a) \ + { return do_sve2_zzx_data(s, a, FUNC, (a->index << 1) | TOP); } + +DO_SVE2_RRX_TB(trans_SMULLB_zzx_s, gen_helper_sve2_smull_idx_s, false) +DO_SVE2_RRX_TB(trans_SMULLB_zzx_d, gen_helper_sve2_smull_idx_d, false) +DO_SVE2_RRX_TB(trans_SMULLT_zzx_s, gen_helper_sve2_smull_idx_s, true) +DO_SVE2_RRX_TB(trans_SMULLT_zzx_d, gen_helper_sve2_smull_idx_d, true) + +DO_SVE2_RRX_TB(trans_UMULLB_zzx_s, gen_helper_sve2_umull_idx_s, false) +DO_SVE2_RRX_TB(trans_UMULLB_zzx_d, gen_helper_sve2_umull_idx_d, false) +DO_SVE2_RRX_TB(trans_UMULLT_zzx_s, gen_helper_sve2_umull_idx_s, true) +DO_SVE2_RRX_TB(trans_UMULLT_zzx_d, gen_helper_sve2_umull_idx_d, true) + +#undef DO_SVE2_RRX_TB + static bool do_sve2_zzxz_data(DisasContext *s, arg_rrxr_esz *a, gen_helper_gvec_4 *fn, int data) { From patchwork Tue Mar 9 16:20:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396083 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2831857jai; Tue, 9 Mar 2021 10:46:50 -0800 (PST) X-Google-Smtp-Source: ABdhPJyHVNBeUUzoN3/OGj6W2doYrPHSTmM5W0s9cMN3Ym5PSjJwYGUsljLmIwwC4DaIKCaPlSOw X-Received: by 2002:a6b:b2cd:: with SMTP id b196mr24380995iof.67.1615315610481; Tue, 09 Mar 2021 10:46:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615315610; cv=none; d=google.com; s=arc-20160816; b=jbYf0rK0CiMQ3bFDGWynvNKku0qCnlZ34N/D7A9otXCRAXBfwI/mzkQWaOQORAWn/P kwO7wG8SxCwyaa5uYSqpWDL3MjE7wrO/vDvyTEme2cAFWkBzu+9PLKAxPCOtO0GdNtKO 1ZxmybJtEiEhdU+Ju0zrMdUFYAtFyCphFfE2rdwdpblmk1OGnIWMuRXkXm+8YYHURbDg 0atFxgcW3bya9EIiOGJOS5gGxuLxDB++hfYU7+R4YxuLl6Oodb+qONpMGfExlZldi1Li v21uj0rSDStBXIbKDXsYt4LVuZt/ErG3biiLkW8wht47UEhmSiKht3KXEbikuErPdTyN Rbiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=n5+mVnW6j6a5LEyFP98f4FS2kW0fwkO2c1u4z0El7gk=; b=viIuhyvGWFOC6prmnH4WukOQ/Kh+0uKb08XqKJhfnP0e0N7GuZALbFdI4lopv+YqVG c15zQ8z2XFTiP+w4XPKn8CNo65jyPMxGAwfrhYgT52jGERHNvThWCxj+x1mjrZws7qbP Wjn+ucX10Tse9wQ8GpK1r6oC9QYevoIsQ60ZBQxhKksHg0R3rWXO7KtvcW3nJ797Yqbl GMWsaC3Oip4ejLnyXAeruqZ9iKnuNSnPA8lDQm3iNIb78m4NOPtuxSWRdk3hbPuuw3H5 Mm2JXiK+dsqaeEqP/YVA8Ent9EUmiyXVTuZk7KECniXhY9w9349aN4zohJb7kgnIgdVo b9vQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hsiAzyPX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k3si15239887iol.70.2021.03.09.10.46.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:46:50 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hsiAzyPX; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41214 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhNZ-0007QM-NT for patch@linaro.org; Tue, 09 Mar 2021 13:46:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47130) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf80-0000RS-4j for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:41 -0500 Received: from mail-ot1-x331.google.com ([2607:f8b0:4864:20::331]:46170) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7k-0002p9-1f for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:35 -0500 Received: by mail-ot1-x331.google.com with SMTP id r24so6274043otq.13 for ; Tue, 09 Mar 2021 08:22:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=n5+mVnW6j6a5LEyFP98f4FS2kW0fwkO2c1u4z0El7gk=; b=hsiAzyPXdjdgysBMTktOGPo6fAPQzYOum+zleUo4r3I/1eJM6vezTRuBIbhEHnvPSg 9v8MMKzXgnSxz816g5Zz7JCOlN0JlKlhJP3+MXgvrDnOvkknL0tXi17fjMAW/o76UTRo LEtZm5hJOlLagHWikzMzqR5N2QpECWsasQTkLRTiaTInKLan6H7rv7V8hhNAZzhZ9fFi w1PQLe0582JoQCm2nOjCIaTUecjVa0rZ7rBbJPjc0xyTnHMP5ZSJ1YLdS97W8Q7whDR4 oaorxmEM2/DJjv4ei1QX2Tgbb49bglBniDckXj/zeUCfLt3JjmZSwAUdEZzMMMaOa2xr 2yDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n5+mVnW6j6a5LEyFP98f4FS2kW0fwkO2c1u4z0El7gk=; b=OLc+aRDtMghBPIpuwih1G9A7aflpNcyRqsNIRVZ93xSNbBQQ7W8z9AkPJI0Dr7crBL 3k3ay/yl4X7ViVvDIm3j/uI0pgfU6DMij40QyF57RRhh6PiD1O+KUGC1HOhN7BBr50Yj gvENIz8GyjhsxTupZYxXC4kHLRtTvoKX+I420JKwUUiqx/NJyrFJacflmtSEq4OvV4su fqMdniWVAyrM2l6pu421b3lGGLj9EM7OC7zb/dYEzFfreq9B9L7Nknp+TRg/rFxcGH9/ SsIazqjbvO5ocSH4SAyoFo8cqZ3n8XhWNR/B0Pq3agXHIvIVZaQ++r9NJlCzahRmvzX/ GUkQ== X-Gm-Message-State: AOAM533bJtOxX7U5P88w1LIDmzTffAo8c3cyUYY4ad/dYBhD2SO2KcXF SWuwu+puSGpu0P/cdZ4LztdV5VI4E9fKWGTG X-Received: by 2002:a9d:4e1a:: with SMTP id p26mr903207otf.202.1615306938661; Tue, 09 Mar 2021 08:22:18 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:18 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 57/78] target/arm: Implement SVE2 saturating multiply (indexed) Date: Tue, 9 Mar 2021 08:20:20 -0800 Message-Id: <20210309162041.23124-58-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::331; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x331.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 6 ++++++ target/arm/sve_helper.c | 3 +++ target/arm/translate-sve.c | 5 +++++ 4 files changed, 19 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index a3690082af..548dedf24c 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2693,3 +2693,8 @@ DEF_HELPER_FLAGS_4(sve2_smull_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_smull_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_umull_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_umull_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index ebb892748b..a0c930ed66 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -835,6 +835,12 @@ UMULLB_zzx_d 01000100 .. 1 ..... 1101.0 ..... ..... @rrxl_d UMULLT_zzx_s 01000100 .. 1 ..... 1101.1 ..... ..... @rrxl_s UMULLT_zzx_d 01000100 .. 1 ..... 1101.1 ..... ..... @rrxl_d +# SVE2 saturating multiply (indexed) +SQDMULLB_zzx_s 01000100 .. 1 ..... 1110.0 ..... ..... @rrxl_s +SQDMULLB_zzx_d 01000100 .. 1 ..... 1110.0 ..... ..... @rrxl_d +SQDMULLT_zzx_s 01000100 .. 1 ..... 1110.1 ..... ..... @rrxl_s +SQDMULLT_zzx_d 01000100 .. 1 ..... 1110.1 ..... ..... @rrxl_d + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 .. 1 ..... 111110 ..... ..... @rrx_h MUL_zzx_s 01000100 .. 1 ..... 111110 ..... ..... @rrx_s diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index b64e1f243d..95bda0ae38 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1581,6 +1581,9 @@ DO_ZZX(sve2_smull_idx_d, int64_t, int32_t, , H1_4, DO_MUL) DO_ZZX(sve2_umull_idx_s, uint32_t, uint16_t, H1_4, H1_2, DO_MUL) DO_ZZX(sve2_umull_idx_d, uint64_t, uint32_t, , H1_4, DO_MUL) +DO_ZZX(sve2_sqdmull_idx_s, int32_t, int16_t, H1_4, H1_2, do_sqdmull_s) +DO_ZZX(sve2_sqdmull_idx_d, int64_t, int32_t, , H1_4, do_sqdmull_d) + #undef DO_ZZX #define DO_BITPERM(NAME, TYPE, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6376a21b98..9b27f32b54 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3880,6 +3880,11 @@ DO_SVE2_RRX_TB(trans_UMULLB_zzx_d, gen_helper_sve2_umull_idx_d, false) DO_SVE2_RRX_TB(trans_UMULLT_zzx_s, gen_helper_sve2_umull_idx_s, true) DO_SVE2_RRX_TB(trans_UMULLT_zzx_d, gen_helper_sve2_umull_idx_d, true) +DO_SVE2_RRX_TB(trans_SQDMULLB_zzx_s, gen_helper_sve2_sqdmull_idx_s, false) +DO_SVE2_RRX_TB(trans_SQDMULLB_zzx_d, gen_helper_sve2_sqdmull_idx_d, false) +DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_s, gen_helper_sve2_sqdmull_idx_s, true) +DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_d, gen_helper_sve2_sqdmull_idx_d, true) + #undef DO_SVE2_RRX_TB static bool do_sve2_zzxz_data(DisasContext *s, arg_rrxr_esz *a, From patchwork Tue Mar 9 16:20:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396050 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2803865jai; Tue, 9 Mar 2021 10:07:22 -0800 (PST) X-Google-Smtp-Source: ABdhPJwPqM9/Fd+cOQXmp7i84WRFS25wD1yrY2AlsitQRB9Pm/VGGp8wASz6Ip1dt92pS1xPzO+y X-Received: by 2002:a05:6e02:4a4:: with SMTP id e4mr23995646ils.114.1615313242110; Tue, 09 Mar 2021 10:07:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313242; cv=none; d=google.com; s=arc-20160816; b=Yu94zHJ91so6fRvdcWb6Fgo9RBKynif8J0HcPxIhXgIWR9hzhwl7J/UEJtvW+5CSBf k966q+kM9piYGL1xOIPWnZAHMFbvicnbx+lrUZKa15G6WfXPuRT+Q/9cndk9r++MaM/u 8ffvX9skGfbjxFtNsuUD3mj9Rhcjve4+DRqBdKkkKgJkBgoIWbXJnDq7SU6Y/pV6QCRG iBinX5s7lAPIIxcCqmMbGGkq2cdpzgH74FnOoKBiclas8NIod0Vm4fj1H12sLJ6IF8Sy SXOpOYOd9A6tRgZYU3qUDdtcP0PTO0ASBhgCaB7jTKDFtV8pqt8ODEuXJOa8Gzz1CFMY cVhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=M6NWbJEGQ5xCtqVe/UozKEUpF+Si9GTRje/9jcqBvVA=; b=m5X/5/b6DOLRUwpPVAuQ0JlarY6wePZz7KTmK1X43EaedzxpzQi8TzI7X1xwplAVyN X/VqPdby/gL0Pjaii9QZiwlNajf1WEQNlVBcbzOLDL0racaL204CAb4Mn3SWit/6CpNw hMFjbrcJQx6wPlkOmQRekRYYPzq2X2I+5uyNTBEjq12T2ZAX0Kjqx67fqDndOC6JvsG1 WvisRvtKyqze3Ep+hqkiL9uAiJDRaF8uySxWJf0NB9ugpKgDBzWjZxkdWYBf461zAHGG lmw4E8kxFtCgWVQzOztUcj3IAmjeLvUoq+ufQNK3Zdc2ZDZUm1WxaIVNCxfCNw70Pfve GXdw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=de53WvI+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k31si13060098jav.116.2021.03.09.10.07.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:07:22 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=de53WvI+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37562 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJglN-0000MB-EV for patch@linaro.org; Tue, 09 Mar 2021 13:07:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47148) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf81-0000RW-NT for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:41 -0500 Received: from mail-oi1-x22d.google.com ([2607:f8b0:4864:20::22d]:40404) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7n-0002pK-RL for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:37 -0500 Received: by mail-oi1-x22d.google.com with SMTP id w65so15519232oie.7 for ; Tue, 09 Mar 2021 08:22:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=M6NWbJEGQ5xCtqVe/UozKEUpF+Si9GTRje/9jcqBvVA=; b=de53WvI+icZQgxisgwr5fRpDajM6rZZRXxUhpLhWyH27fLvn+SM5F1PJLuvP8CLFVz Ajl/hwk21I1Vl5iOzewwCrM+/G8Kmg3uaB4RO1jRlF9q9AugtZ4CIgow4247dnjTTSNX XWlbwlBktKaBDZTYL1XLNV6blimeW9eAVdhfxsZbRy7PRlbcsTW7JGzwkbU8KHxA8W/c bcMJv+GWKyaiA7iEE8q5t408JxhGGrhYkGuA9ekmYvJTiB3qc9HKTB5HLszZimDJNV0W uErCdvrjVx82ZW3skwl4w8XXErbFy1Z2HQ77juhJnAFaBvH4hBSZWt2T1jcUmmnqwTpK st7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=M6NWbJEGQ5xCtqVe/UozKEUpF+Si9GTRje/9jcqBvVA=; b=k+T578a15/OkmtrryZ4FEHyBbOGi6jV+mPXZCUGtVAV1UqOclXi64zZqdzmYtvCbD3 IAR4zXKDmlHZHFYDMTngshu1CCs9cktGnWTVReNgkyxnr9E8PlbTvdznuZMKH8SPbo1H RgbyjSLU2L8WAhpuMnI1k/Flh1OOLokBi2dD5aWGvZZ8UJwnV5YbMVR90cEH4hkxYVbl pJ6gk0Xqut2RfzQR4qezDuCswMm/SbIeu39iyLuacIgWsNJImufKqrcTQcrpcIBJCn0J 2+rgFSFrCHC6nwp2GjKlRO3fjkoqZpq+y6rfth0qbxDNIZPTikigA/zjB6UDGu2tOU/H GFbA== X-Gm-Message-State: AOAM533pu8VXLp7TFax7iVUbA8kzBqg6p2BJ1YagA0+zpUYf69zktM75 PNiQ2uxVCYwJDUInfT+XbBoxIBgYBtyGJIKP X-Received: by 2002:aca:d6d3:: with SMTP id n202mr3553013oig.175.1615306939693; Tue, 09 Mar 2021 08:22:19 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:19 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 58/78] target/arm: Implement SVE2 signed saturating doubling multiply high Date: Tue, 9 Mar 2021 08:20:21 -0800 Message-Id: <20210309162041.23124-59-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22d; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.h | 10 +++++ target/arm/sve.decode | 4 ++ target/arm/translate-sve.c | 18 ++++++++ target/arm/vec_helper.c | 84 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 116 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 72c5bf6aca..eb94b6b1e6 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -957,6 +957,16 @@ DEF_HELPER_FLAGS_5(neon_sqrdmulh_h, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(neon_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) #ifdef TARGET_AARCH64 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index a0c930ed66..8c063e77ae 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1214,6 +1214,10 @@ SMULH_zzz 00000100 .. 1 ..... 0110 10 ..... ..... @rd_rn_rm UMULH_zzz 00000100 .. 1 ..... 0110 11 ..... ..... @rd_rn_rm PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 +# SVE2 signed saturating doubling multiply high (unpredicated) +SQDMULH_zzz 00000100 .. 1 ..... 0111 00 ..... ..... @rd_rn_rm +SQRDMULH_zzz 00000100 .. 1 ..... 0111 01 ..... ..... @rd_rn_rm + ### SVE2 Integer - Predicated SADALP_zpzz 01000100 .. 000 100 101 ... ..... ..... @rdm_pg_rn diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 9b27f32b54..27a10a9759 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6430,6 +6430,24 @@ static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a) return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b); } +static bool trans_SQDMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqdmulh_b, gen_helper_sve2_sqdmulh_h, + gen_helper_sve2_sqdmulh_s, gen_helper_sve2_sqdmulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + +static bool trans_SQRDMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqrdmulh_b, gen_helper_sve2_sqrdmulh_h, + gen_helper_sve2_sqrdmulh_s, gen_helper_sve2_sqrdmulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + /* * SVE2 Integer - Predicated */ diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index b19877e0d3..25061c15e1 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -81,6 +81,26 @@ void HELPER(sve2_sqrdmlsh_b)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], 0, false, false); + } +} + +void HELPER(sve2_sqrdmulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], 0, false, true); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */ int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, bool neg, bool round, uint32_t *sat) @@ -198,6 +218,28 @@ void HELPER(sve2_sqrdmlsh_h)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], 0, false, false, &discard); + } +} + +void HELPER(sve2_sqrdmulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], 0, false, true, &discard); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, bool neg, bool round, uint32_t *sat) @@ -309,6 +351,28 @@ void HELPER(sve2_sqrdmlsh_s)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], 0, false, false, &discard); + } +} + +void HELPER(sve2_sqrdmulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], 0, false, true, &discard); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 64-bit */ static int64_t do_sat128_d(Int128 r) { @@ -368,6 +432,26 @@ void HELPER(sve2_sqrdmlsh_d)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], 0, false, false); + } +} + +void HELPER(sve2_sqrdmulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], 0, false, true); + } +} + /* Integer 8 and 16-bit dot-product. * * Note that for the loops herein, host endianness does not matter From patchwork Tue Mar 9 16:20:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396060 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2812491jai; Tue, 9 Mar 2021 10:18:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJzAl+aXOmuUC/GeDf3v3iE0fuVVSJVKLO0g7e8gRJyZn2CFVUWSPxGe+fBI7klK/7VZgcwq X-Received: by 2002:a92:2c04:: with SMTP id t4mr26028318ile.99.1615313908402; Tue, 09 Mar 2021 10:18:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313908; cv=none; d=google.com; s=arc-20160816; b=zXVT+CiWPlEo+yLRDJ7OB/+WxTmEM87DxBNx3t7uSPo/nzUyqkAk4jDAXx5OHBhBn9 8SsC9Iw2/AtY5832jcnslFeAlDAImYTbDf5Kk9m0L0WPLDJaf7dUo/jm+lRV5zAJ2oJ+ TBkvQQqniR9d+K7Mc96RB+YvT6bWGnMZ3APqj2ehsF4u/OBSN3qU7/9J4nUDcjbbdxCS /HNicw4zrZu4oergufoojYjVvt8R1EgP1BbKrw9fGpclYF910ZdFST9BTN3NP/EHsGIu oJ/F7rgvfBbVEeouICruPHiEkfU3aOHnK2EuQsMH602itXhhyoz0G4JJGxkdo8D0oQiH pLpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=mg0Np1qlEsA3IXrJan/GCfIAaTldychqLU2815ExZjw=; b=y92zCDz525HSCoCsmks0ukhL77EcSgPAVkc9Q1Z5o7gFS0iiAOMJSSgOmhgJJVEhTH JvIS4jsGV4n/O3mLLlmB+hpZhZKKoAQIPLrcs2Po3XHjqRMM6P6rVSlQPduYDCV0PfBC 7Zm0L6zvqjZeS0I2RD8qB/Ok22RWoCCCx7DeRk2ohCAp1CdAG6WlrRSs0Pu20RauJ3Ud dDGJ/se1o5U5+ml5gOehyV96WNIOXlz0mIIAjg0UVwVEpRpRtpKpllwGt77eSY4LxpnI iNkWaUvHViedpLaWFBY44R6IwcpjNgpENcfoS5jHk9bxLr6rkXSyBDVuZNYKU8z8cASN /LkA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hkzNxkvC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id u10si14189838ilv.64.2021.03.09.10.18.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:18:28 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hkzNxkvC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60550 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgw7-00045y-ON for patch@linaro.org; Tue, 09 Mar 2021 13:18:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47218) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf87-0000T7-L5 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:43 -0500 Received: from mail-ot1-x32f.google.com ([2607:f8b0:4864:20::32f]:38394) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7s-0002pZ-1e for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:43 -0500 Received: by mail-ot1-x32f.google.com with SMTP id a17so13374394oto.5 for ; Tue, 09 Mar 2021 08:22:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mg0Np1qlEsA3IXrJan/GCfIAaTldychqLU2815ExZjw=; b=hkzNxkvCErn6801t1KbUNwsvkHtu3nuSEn+z0MPc8JZPpZLUZy0Xb4B2SgippGGw1W ZlMnb4fEbvMCnw/hDwlGtitbRl8qKUKgCjwcms1GoQP1pTw3QAHyLWKGKXcOHlrYqt7t DC3G3utAmO3lRIHrBs/9gVvAYWfMs+vuO/elBjDR0SKgsEA6xHi0xQ/QsZMLeGyZ6cNu SmFqFAJULkCrSNgtJReG+y4PZUXu/x7f9IxLf+k6MrF7ftc/3UNz/d/2LR9XU41mjP8R lDc/ir7q/hKRYmh1IndDYOOG7v4KwIvQDMnm++2eHaLuMyhijqi+DEesHh2PFhrGiHml ZVnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mg0Np1qlEsA3IXrJan/GCfIAaTldychqLU2815ExZjw=; b=NdTkFwHB+OC9Tv/MczUGeJmE47csKdczKW44ZV3PDOIqzGyrD7xB59cg360RJk4GUf RqoZejR7Z6tj/8lXivkd9Und4w0JHv5E0mUwDNbcSwc1UIeo2rfY069FlSJoQf5rYUJV bPDSdGMoZuxJ8zpp6UN9zAXQ2S8dfPUOxqh8AqCUVcLJJXgBeOcuM4V58NAMRHWq/3ps nrtzK6nLq1LeBRg/4PjD6xwfSeZat2GZJMCEwBf8V6LjLqcLcOR5bfHEsO4IAsc1IeBu LcqlOtvqE8bW4xZYm05EVHSW9eJctfdIZuqnImrO+RU81hA8wfOxFb+1q9wdrwwBfBoG 4QXA== X-Gm-Message-State: AOAM530FlJGBrrqeJTv5qQcpB87ZDgb0oCTc8tzKFaqjkExeToaASog8 AUx5IfXqUq8Y8fhjrbRTFm1v2pLHDTw3elVH X-Received: by 2002:a05:6830:1258:: with SMTP id s24mr24451109otp.341.1615306940723; Tue, 09 Mar 2021 08:22:20 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:20 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 59/78] target/arm: Implement SVE2 saturating multiply high (indexed) Date: Tue, 9 Mar 2021 08:20:22 -0800 Message-Id: <20210309162041.23124-60-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32f; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.h | 14 ++++++ target/arm/sve.decode | 8 ++++ target/arm/translate-sve.c | 8 ++++ target/arm/vec_helper.c | 88 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 118 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index eb94b6b1e6..e7c463fff5 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -967,6 +967,20 @@ DEF_HELPER_FLAGS_4(sve2_sqrdmulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqrdmulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) #ifdef TARGET_AARCH64 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 8c063e77ae..31a3d909f3 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -841,6 +841,14 @@ SQDMULLB_zzx_d 01000100 .. 1 ..... 1110.0 ..... ..... @rrxl_d SQDMULLT_zzx_s 01000100 .. 1 ..... 1110.1 ..... ..... @rrxl_s SQDMULLT_zzx_d 01000100 .. 1 ..... 1110.1 ..... ..... @rrxl_d +# SVE2 saturating multiply high (indexed) +SQDMULH_zzx_h 01000100 .. 1 ..... 111100 ..... ..... @rrx_h +SQDMULH_zzx_s 01000100 .. 1 ..... 111100 ..... ..... @rrx_s +SQDMULH_zzx_d 01000100 .. 1 ..... 111100 ..... ..... @rrx_d +SQRDMULH_zzx_h 01000100 .. 1 ..... 111101 ..... ..... @rrx_h +SQRDMULH_zzx_s 01000100 .. 1 ..... 111101 ..... ..... @rrx_s +SQRDMULH_zzx_d 01000100 .. 1 ..... 111101 ..... ..... @rrx_d + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 .. 1 ..... 111110 ..... ..... @rrx_h MUL_zzx_s 01000100 .. 1 ..... 111110 ..... ..... @rrx_s diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 27a10a9759..a3e7d4f38f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3864,6 +3864,14 @@ DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h) DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s) DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) +DO_SVE2_RRX(trans_SQDMULH_zzx_h, gen_helper_sve2_sqdmulh_idx_h) +DO_SVE2_RRX(trans_SQDMULH_zzx_s, gen_helper_sve2_sqdmulh_idx_s) +DO_SVE2_RRX(trans_SQDMULH_zzx_d, gen_helper_sve2_sqdmulh_idx_d) + +DO_SVE2_RRX(trans_SQRDMULH_zzx_h, gen_helper_sve2_sqrdmulh_idx_h) +DO_SVE2_RRX(trans_SQRDMULH_zzx_s, gen_helper_sve2_sqrdmulh_idx_s) +DO_SVE2_RRX(trans_SQRDMULH_zzx_d, gen_helper_sve2_sqrdmulh_idx_d) + #undef DO_SVE2_RRX #define DO_SVE2_RRX_TB(NAME, FUNC, TOP) \ diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 25061c15e1..8b7269d8e1 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -240,6 +240,36 @@ void HELPER(sve2_sqrdmulh_h)(void *vd, void *vn, void *vm, uint32_t desc) } } +void HELPER(sve2_sqdmulh_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int16_t *d = vd, *n = vn, *m = (int16_t *)vm + H2(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 2; i += 16 / 2) { + int16_t mm = m[i]; + for (j = 0; j < 16 / 2; ++j) { + d[i + j] = do_sqrdmlah_h(n[i + j], mm, 0, false, false, &discard); + } + } +} + +void HELPER(sve2_sqrdmulh_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int16_t *d = vd, *n = vn, *m = (int16_t *)vm + H2(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 2; i += 16 / 2) { + int16_t mm = m[i]; + for (j = 0; j < 16 / 2; ++j) { + d[i + j] = do_sqrdmlah_h(n[i + j], mm, 0, false, true, &discard); + } + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, bool neg, bool round, uint32_t *sat) @@ -373,6 +403,36 @@ void HELPER(sve2_sqrdmulh_s)(void *vd, void *vn, void *vm, uint32_t desc) } } +void HELPER(sve2_sqdmulh_idx_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int32_t *d = vd, *n = vn, *m = (int32_t *)vm + H4(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 4; i += 16 / 4) { + int32_t mm = m[i]; + for (j = 0; j < 16 / 4; ++j) { + d[i + j] = do_sqrdmlah_s(n[i + j], mm, 0, false, false, &discard); + } + } +} + +void HELPER(sve2_sqrdmulh_idx_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int32_t *d = vd, *n = vn, *m = (int32_t *)vm + H4(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 4; i += 16 / 4) { + int32_t mm = m[i]; + for (j = 0; j < 16 / 4; ++j) { + d[i + j] = do_sqrdmlah_s(n[i + j], mm, 0, false, true, &discard); + } + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 64-bit */ static int64_t do_sat128_d(Int128 r) { @@ -452,6 +512,34 @@ void HELPER(sve2_sqrdmulh_d)(void *vd, void *vn, void *vm, uint32_t desc) } } +void HELPER(sve2_sqdmulh_idx_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int64_t *d = vd, *n = vn, *m = (int64_t *)vm + idx; + + for (i = 0; i < opr_sz / 8; i += 16 / 8) { + int64_t mm = m[i]; + for (j = 0; j < 16 / 8; ++j) { + d[i + j] = do_sqrdmlah_d(n[i + j], mm, 0, false, false); + } + } +} + +void HELPER(sve2_sqrdmulh_idx_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int64_t *d = vd, *n = vn, *m = (int64_t *)vm + idx; + + for (i = 0; i < opr_sz / 8; i += 16 / 8) { + int64_t mm = m[i]; + for (j = 0; j < 16 / 8; ++j) { + d[i + j] = do_sqrdmlah_d(n[i + j], mm, 0, false, true); + } + } +} + /* Integer 8 and 16-bit dot-product. * * Note that for the loops herein, host endianness does not matter From patchwork Tue Mar 9 16:20:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396084 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2834378jai; Tue, 9 Mar 2021 10:50:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJyNYdCst7RKrUMOAUByXv1zuFPeWvdy/WJ/KbpSHzr/E+BtdlVpkvjJF+CbX9TemQWoRSHv X-Received: by 2002:a5e:8d05:: with SMTP id m5mr3254654ioj.114.1615315857610; Tue, 09 Mar 2021 10:50:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615315857; cv=none; d=google.com; s=arc-20160816; b=kvLRUy1zUdnpuq6Bd1aPjcsp284lvLezFVqeXwGDQE5/dd1CtF2I8Z1NqaOBHuT4o1 /S6QYXlURDBYH/PSRL29nSOrgPM+CAcHdt3nF3Zqx0Ro6Hynk4/KLZKT1feesSgiAMlU EzKuTMyWKwehtmoH59HcBfAx/Er+okbWjOBQ8e/4eqsjDlC+LaZefuDgo5hdkwIcRJTY w8eW8O4TrSAKTSJR8KIJaJn72G2BUo8t5t4v+LCyfxavDncBzqIjub8pvbvwYzMw/zvo /mZA7rxOtR83pb9QmA0BfItIo5pKPcY4879oSXrdQ+5+U2jgCeDNmRIU8jEKECAAz/Du gqQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=6ScXgfjow8dkwDZFECjUC9yhYbtEGzBMTrAi5glL4G8=; b=nT/n9V+tsuKnzsR5tReD1lgmNuynE14EtyJDCEokboXiP+Sg4bwhiOmonlPQrNcHgu vOR6qQWXfpvGJxoleFNWaYzy7fma6hfbuPazQDpVhxLIsu/t2sFh5ICpRbGVHxyrSBJM avKoDuYv4tlleC0IKXERfadkmRxk1vcTw7MiK06+lYca6LkIEr0dArkqhpPoXoSo/zQ9 1KtPSc9GPLOIP2tF97e3EDtfKXkf7P0CqlaSFfIw8PLwUjio2CX6FeoeP3ZJPmzCNMiv oZsLg2PmyIMoR7mvS2HrM0zBVFb+BVdTb8Wlu10wQ4tC1JY2x5dKLtxZkfxNg3RF90kM ZLAg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=F8Mkuk7H; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t14si17403085iov.85.2021.03.09.10.50.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:50:57 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=F8Mkuk7H; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52282 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhRY-0004Rc-W3 for patch@linaro.org; Tue, 09 Mar 2021 13:50:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47254) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf88-0000W6-NE for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:44 -0500 Received: from mail-ot1-x336.google.com ([2607:f8b0:4864:20::336]:40426) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7r-0002pi-9i for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:44 -0500 Received: by mail-ot1-x336.google.com with SMTP id b8so13349384oti.7 for ; Tue, 09 Mar 2021 08:22:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6ScXgfjow8dkwDZFECjUC9yhYbtEGzBMTrAi5glL4G8=; b=F8Mkuk7H34IIvSWZDvKeivpUmA0wlmpMkB0iqM1VtDP53QBS4z37R4uFHSueCKIYRL lXg1YNE30t5yFDvEACjXdYgxj6O0eJQ2PwZukHi+ngu1zWXrgsXHDm9zjqs47XWlSC6h 7iISZQuwtxSj10zDfkH919Dozu7vk1Y2AYjvLE7xZ3zMYVE5AiM9dQJKHrG6kdB7aCOE Fpr1sch1WbCXjogH4Jq+DYb956faPY1VEH6On7wnLiQPbdg6a15PDdr7H98cPI8jrIU8 q39MpWZRSxnDfkUtJpG/s0lIaEkqXv0H7EorI8XC+Z7NXd1pipW1e6dCJiA2PrAvJNaV J1Fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6ScXgfjow8dkwDZFECjUC9yhYbtEGzBMTrAi5glL4G8=; b=Z74SJRGLt2F88uFeXfZoaoUsAwnArTGKb9v5VTKbhCwAddlaQ8hrv0sBtNq0GK5VJ3 ce+zxH24AVSwPRH5z/H/OMm9AOSnKxRJNMYjVMNkDxG/Xp+uvZZeUkQnTBwgVwI6JVh2 KCKB4MD30SZ5qI7EDU6ipTSRRtEbGiY3vm5U4G//e5/V8cp5Tzh8TXDCu7mHvKnqb/x+ iiQFo+bgfOfjbsud0tpOikFcj68W2kiQuAXVIn+SGhuKX1ut/hbLAjPHaQIPqXpamlKg esqg0knWGfDyVH/Y2aDwtwOQfc/OafBKCvEVvDd2C4NPwW7HLooG/aiJzp8kdFYTHF9V h/Fw== X-Gm-Message-State: AOAM532ZiacsjsTXHJkVQZEaUWhlcjhvpMcMquXv7aPbGcAkiWQfEndI qKgQy6uEVIlN8HmvluEdkLeEeCdgFrHf2Kax X-Received: by 2002:a05:6830:24a1:: with SMTP id v1mr3877097ots.119.1615306941726; Tue, 09 Mar 2021 08:22:21 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:21 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 60/78] target/arm: Implement SVE2 multiply-add long (indexed) Date: Tue, 9 Mar 2021 08:20:23 -0800 Message-Id: <20210309162041.23124-61-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::336; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x336.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 17 +++++++++++++++++ target/arm/sve.decode | 18 ++++++++++++++++++ target/arm/sve_helper.c | 16 ++++++++++++++++ target/arm/translate-sve.c | 20 ++++++++++++++++++++ 4 files changed, 71 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 548dedf24c..5e3a2922c8 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2698,3 +2698,20 @@ DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_smlal_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlal_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlsl_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlsl_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlal_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlal_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlsl_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlsl_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 31a3d909f3..c77adf8ca6 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -825,6 +825,24 @@ SQDMLSLB_zzxw_d 01000100 .. 1 ..... 0011.0 ..... ..... @rrxw_d SQDMLSLT_zzxw_s 01000100 .. 1 ..... 0011.1 ..... ..... @rrxw_s SQDMLSLT_zzxw_d 01000100 .. 1 ..... 0011.1 ..... ..... @rrxw_d +# SVE2 multiply-add long (indexed) +SMLALB_zzxw_s 01000100 .. 1 ..... 1000.0 ..... ..... @rrxw_s +SMLALB_zzxw_d 01000100 .. 1 ..... 1000.0 ..... ..... @rrxw_d +SMLALT_zzxw_s 01000100 .. 1 ..... 1000.1 ..... ..... @rrxw_s +SMLALT_zzxw_d 01000100 .. 1 ..... 1000.1 ..... ..... @rrxw_d +UMLALB_zzxw_s 01000100 .. 1 ..... 1001.0 ..... ..... @rrxw_s +UMLALB_zzxw_d 01000100 .. 1 ..... 1001.0 ..... ..... @rrxw_d +UMLALT_zzxw_s 01000100 .. 1 ..... 1001.1 ..... ..... @rrxw_s +UMLALT_zzxw_d 01000100 .. 1 ..... 1001.1 ..... ..... @rrxw_d +SMLSLB_zzxw_s 01000100 .. 1 ..... 1010.0 ..... ..... @rrxw_s +SMLSLB_zzxw_d 01000100 .. 1 ..... 1010.0 ..... ..... @rrxw_d +SMLSLT_zzxw_s 01000100 .. 1 ..... 1010.1 ..... ..... @rrxw_s +SMLSLT_zzxw_d 01000100 .. 1 ..... 1010.1 ..... ..... @rrxw_d +UMLSLB_zzxw_s 01000100 .. 1 ..... 1011.0 ..... ..... @rrxw_s +UMLSLB_zzxw_d 01000100 .. 1 ..... 1011.0 ..... ..... @rrxw_d +UMLSLT_zzxw_s 01000100 .. 1 ..... 1011.1 ..... ..... @rrxw_s +UMLSLT_zzxw_d 01000100 .. 1 ..... 1011.1 ..... ..... @rrxw_d + # SVE2 integer multiply long (indexed) SMULLB_zzx_s 01000100 .. 1 ..... 1100.0 ..... ..... @rrxl_s SMULLB_zzx_d 01000100 .. 1 ..... 1100.0 ..... ..... @rrxl_d diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 95bda0ae38..c64321368f 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1546,6 +1546,20 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ } \ } +#define DO_MLA(N, M, A) (A + N * M) + +DO_ZZXW(sve2_smlal_idx_s, int32_t, int16_t, H1_4, H1_2, DO_MLA) +DO_ZZXW(sve2_smlal_idx_d, int64_t, int32_t, , H1_4, DO_MLA) +DO_ZZXW(sve2_umlal_idx_s, uint32_t, uint16_t, H1_4, H1_2, DO_MLA) +DO_ZZXW(sve2_umlal_idx_d, uint64_t, uint32_t, , H1_4, DO_MLA) + +#define DO_MLS(N, M, A) (A - N * M) + +DO_ZZXW(sve2_smlsl_idx_s, int32_t, int16_t, H1_4, H1_2, DO_MLS) +DO_ZZXW(sve2_smlsl_idx_d, int64_t, int32_t, , H1_4, DO_MLS) +DO_ZZXW(sve2_umlsl_idx_s, uint32_t, uint16_t, H1_4, H1_2, DO_MLS) +DO_ZZXW(sve2_umlsl_idx_d, uint64_t, uint32_t, , H1_4, DO_MLS) + #define DO_SQDMLAL_S(N, M, A) DO_SQADD_S(A, do_sqdmull_s(N, M)) #define DO_SQDMLAL_D(N, M, A) do_sqadd_d(A, do_sqdmull_d(N, M)) @@ -1558,6 +1572,8 @@ DO_ZZXW(sve2_sqdmlal_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLAL_D) DO_ZZXW(sve2_sqdmlsl_idx_s, int32_t, int16_t, H1_4, H1_2, DO_SQDMLSL_S) DO_ZZXW(sve2_sqdmlsl_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLSL_D) +#undef DO_MLA +#undef DO_MLS #undef DO_ZZXW #define DO_ZZX(NAME, TYPEW, TYPEN, HW, HN, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index a3e7d4f38f..6a6698bfe9 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3940,6 +3940,26 @@ DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, false) DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, true) DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, true) +DO_SVE2_RRXR_TB(trans_SMLALB_zzxw_s, gen_helper_sve2_smlal_idx_s, false) +DO_SVE2_RRXR_TB(trans_SMLALB_zzxw_d, gen_helper_sve2_smlal_idx_d, false) +DO_SVE2_RRXR_TB(trans_SMLALT_zzxw_s, gen_helper_sve2_smlal_idx_s, true) +DO_SVE2_RRXR_TB(trans_SMLALT_zzxw_d, gen_helper_sve2_smlal_idx_d, true) + +DO_SVE2_RRXR_TB(trans_UMLALB_zzxw_s, gen_helper_sve2_umlal_idx_s, false) +DO_SVE2_RRXR_TB(trans_UMLALB_zzxw_d, gen_helper_sve2_umlal_idx_d, false) +DO_SVE2_RRXR_TB(trans_UMLALT_zzxw_s, gen_helper_sve2_umlal_idx_s, true) +DO_SVE2_RRXR_TB(trans_UMLALT_zzxw_d, gen_helper_sve2_umlal_idx_d, true) + +DO_SVE2_RRXR_TB(trans_SMLSLB_zzxw_s, gen_helper_sve2_smlsl_idx_s, false) +DO_SVE2_RRXR_TB(trans_SMLSLB_zzxw_d, gen_helper_sve2_smlsl_idx_d, false) +DO_SVE2_RRXR_TB(trans_SMLSLT_zzxw_s, gen_helper_sve2_smlsl_idx_s, true) +DO_SVE2_RRXR_TB(trans_SMLSLT_zzxw_d, gen_helper_sve2_smlsl_idx_d, true) + +DO_SVE2_RRXR_TB(trans_UMLSLB_zzxw_s, gen_helper_sve2_umlsl_idx_s, false) +DO_SVE2_RRXR_TB(trans_UMLSLB_zzxw_d, gen_helper_sve2_umlsl_idx_d, false) +DO_SVE2_RRXR_TB(trans_UMLSLT_zzxw_s, gen_helper_sve2_umlsl_idx_s, true) +DO_SVE2_RRXR_TB(trans_UMLSLT_zzxw_d, gen_helper_sve2_umlsl_idx_d, true) + #undef DO_SVE2_RRXR_TB /* From patchwork Tue Mar 9 16:20:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396086 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2841422jai; Tue, 9 Mar 2021 11:01:20 -0800 (PST) X-Google-Smtp-Source: ABdhPJyBMwb9T7Bbw+HxhXO37HeFT2WrI2dqpK7vIihau/RCOIcxGwMqvYNjO3cGcbKZr2oN1hdg X-Received: by 2002:a5d:840e:: with SMTP id i14mr23882971ion.131.1615316480646; Tue, 09 Mar 2021 11:01:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615316480; cv=none; d=google.com; s=arc-20160816; b=E4TJni0mpz5ZlHMIwK6KSbELYDD7JtQRYqcu6Ugxv+5fNNOoibzLYCy4oJrEF7u6Z9 Sp4g6ZjjFx5UGl4HNCyqZp+ymUN+ryVGJyZ42Kv0lQn13CWmlzczLHv86HlBZqQUW5MC 2JnEVluQWoPdyoIMUDPZk+I9rGzFg7hqGpFz9HcdFQggSbBd00p/GRi3YLypzBsJK5fb g3GVkEnqp7RwInz+o2IBrh5wotX7/txxt8MELtKdC/qmC9hfyFYKArLnhdMKw037wzK6 PznnGUDIAM1V2mjVbCTQ5MK2lhfX3EcbN9PCnr5a9Sfolaf8+wwwZQOM2sRHtWDlfBC/ w/FA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=LG30+AJqxoFo0xo1KAMI/CixIM8aZhWQwPJGMcWeV98=; b=i9jYdQJZzvn8/Is0Mi9Xp1lGUaqSfXsLBkcfGkaF9/wAnFJ7kut5ooQtYFBuT8VyqI np/NUHqDW5s7v+FlEKD0k/WrUTHuTs4yGczVJek/LW7s95PuE8nLWG5kjrIIEtnP6X+1 Gs6jepg4jwfXXVnVxm5MS2RzgQsd2fB5lcPFZW/9vKV4+2p2NR7qtgvPQ/24BaJQDQNG i877Y23HVNcBDZgfIJDSGo2XHndYiQnCq7MCJ7FBzO7xQw4mBCglELWbuJ5D+qDD1vuL uxyafXIEK8kEHV28ay5bOpSEMBt/0KSce+a8Qu1H68UIln9X2BsmylUjPu4Tlb1d0XvJ dHMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=arjvTj6t; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j3si14685790ilo.36.2021.03.09.11.01.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 11:01:20 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=arjvTj6t; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:49868 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhbb-0006uH-SC for patch@linaro.org; Tue, 09 Mar 2021 14:01:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47398) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8D-0000gP-1W for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:49 -0500 Received: from mail-ot1-x32a.google.com ([2607:f8b0:4864:20::32a]:33145) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7x-0002pp-La for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:48 -0500 Received: by mail-ot1-x32a.google.com with SMTP id j8so13395400otc.0 for ; Tue, 09 Mar 2021 08:22:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LG30+AJqxoFo0xo1KAMI/CixIM8aZhWQwPJGMcWeV98=; b=arjvTj6tHpYeYB1gVsgbRtL9+kH0IVBQtAnBJift33UPd0Eh/Onxhs2yK1c643utgg B8eO3PHNv+wNrLRuDKaXP6/bIkNbDh2AmQ7CzdabCJqHbMf3v8LzjRYNFrPMHYk+KIIj ODspbC3FPQ0ixNOLN0k5PSpwb44dPPWVGrF1zQ6C7CC96hPfbiVK2dvVTkI/tQ1Xt1eh 3XmcnYY7x0fEO0a2+d3o2AdIegE8p+Hy4rPuCcfM2KACnb5pwzuCHje+UrJNxzCndkr2 wLZoNA7L+HCz9ZNosMS+g8+Uwu8WfpB8DZk/U52ywS5qkxK64EFbqIB3sSE+uaYI5llo fq1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LG30+AJqxoFo0xo1KAMI/CixIM8aZhWQwPJGMcWeV98=; b=U0kBSj6YKb9l0Py6zfwUm6bdQUVsTcfAbevhL7k94XKYoivbj01RAgUOAxF6Z2z7GH tHC/K714o7khpixMIlyZ63Xh0xeAHxViUalWWyZ5PAJ5skx5qncMBgSSJyBcTSFWZkCy nIKkbZqhQCgXD8KVM6SlExF+vrwLa5hs/Cy/1MAZjR7gtmDxW9n/NJs+740dCum8+SN4 XvTCGHl4qqnBid1fhidvsdyP1uO/79v+JVMt5sMBvxzTzHYBinM5TK4Zi84ktmULndZT vcv8bG9PRgkIA5lWdhIFTmiOMA0QukNVUk1Uq86JYdDnnBDYz9dz97tql239UgrNE+cX w0BA== X-Gm-Message-State: AOAM533A0CaoFW5ut5NIo7PtF8ly89CBk8zKDRIiufysuE2gXFsNRw+b XIUiNde4YvU44L/bptC2OVjYci2UiHdgd1rM X-Received: by 2002:a9d:3c1:: with SMTP id f59mr13381622otf.330.1615306942778; Tue, 09 Mar 2021 08:22:22 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:22 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 61/78] target/arm: Implement SVE2 complex integer multiply-add (indexed) Date: Tue, 9 Mar 2021 08:20:24 -0800 Message-Id: <20210309162041.23124-62-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32a; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 9 +++ target/arm/sve.decode | 12 ++++ target/arm/sve_helper.c | 142 +++++++++++++++++++++++++++++++------ target/arm/translate-sve.c | 38 +++++++--- 4 files changed, 169 insertions(+), 32 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 5e3a2922c8..0243ac32ec 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2715,3 +2715,12 @@ DEF_HELPER_FLAGS_5(sve2_umlsl_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_umlsl_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_cmla_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cmla_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index c77adf8ca6..9117196352 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -825,6 +825,18 @@ SQDMLSLB_zzxw_d 01000100 .. 1 ..... 0011.0 ..... ..... @rrxw_d SQDMLSLT_zzxw_s 01000100 .. 1 ..... 0011.1 ..... ..... @rrxw_s SQDMLSLT_zzxw_d 01000100 .. 1 ..... 0011.1 ..... ..... @rrxw_d +# SVE2 complex integer multiply-add (indexed) +CMLA_zzxz_h 01000100 10 1 index:2 rm:3 0110 rot:2 rn:5 rd:5 \ + ra=%reg_movprfx +CMLA_zzxz_s 01000100 11 1 index:1 rm:4 0110 rot:2 rn:5 rd:5 \ + ra=%reg_movprfx + +# SVE2 complex saturating integer multiply-add (indexed) +SQRDCMLAH_zzxz_h 01000100 10 1 index:2 rm:3 0111 rot:2 rn:5 rd:5 \ + ra=%reg_movprfx +SQRDCMLAH_zzxz_s 01000100 11 1 index:1 rm:4 0111 rot:2 rn:5 rd:5 \ + ra=%reg_movprfx + # SVE2 multiply-add long (indexed) SMLALB_zzxw_s 01000100 .. 1 ..... 1000.0 ..... ..... @rrxw_s SMLALB_zzxw_d 01000100 .. 1 ..... 1000.0 ..... ..... @rrxw_d diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index c64321368f..13f40acd88 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1466,34 +1466,132 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ } \ } -#define do_cmla(N, M, A, S) (A + (N * M) * (S ? -1 : 1)) +static int8_t do_cmla_b(int8_t n, int8_t m, int8_t a, bool sub) +{ + return n * m * (sub ? -1 : 1) + a; +} -DO_CMLA(sve2_cmla_zzzz_b, uint8_t, H1, do_cmla) -DO_CMLA(sve2_cmla_zzzz_h, uint16_t, H2, do_cmla) -DO_CMLA(sve2_cmla_zzzz_s, uint32_t, H4, do_cmla) -DO_CMLA(sve2_cmla_zzzz_d, uint64_t, , do_cmla) +static int16_t do_cmla_h(int16_t n, int16_t m, int16_t a, bool sub) +{ + return n * m * (sub ? -1 : 1) + a; +} -#define DO_SQRDMLAH_B(N, M, A, S) \ - do_sqrdmlah_b(N, M, A, S, true) -#define DO_SQRDMLAH_H(N, M, A, S) \ - ({ uint32_t discard; do_sqrdmlah_h(N, M, A, S, true, &discard); }) -#define DO_SQRDMLAH_S(N, M, A, S) \ - ({ uint32_t discard; do_sqrdmlah_s(N, M, A, S, true, &discard); }) -#define DO_SQRDMLAH_D(N, M, A, S) \ - do_sqrdmlah_d(N, M, A, S, true) +static int32_t do_cmla_s(int32_t n, int32_t m, int32_t a, bool sub) +{ + return n * m * (sub ? -1 : 1) + a; +} -DO_CMLA(sve2_sqrdcmlah_zzzz_b, int8_t, H1, DO_SQRDMLAH_B) -DO_CMLA(sve2_sqrdcmlah_zzzz_h, int16_t, H2, DO_SQRDMLAH_H) -DO_CMLA(sve2_sqrdcmlah_zzzz_s, int32_t, H4, DO_SQRDMLAH_S) -DO_CMLA(sve2_sqrdcmlah_zzzz_d, int64_t, , DO_SQRDMLAH_D) +static int64_t do_cmla_d(int64_t n, int64_t m, int64_t a, bool sub) +{ + return n * m * (sub ? -1 : 1) + a; +} + +DO_CMLA(sve2_cmla_zzzz_b, uint8_t, H1, do_cmla_b) +DO_CMLA(sve2_cmla_zzzz_h, uint16_t, H2, do_cmla_h) +DO_CMLA(sve2_cmla_zzzz_s, uint32_t, H4, do_cmla_s) +DO_CMLA(sve2_cmla_zzzz_d, uint64_t, , do_cmla_d) + +static int8_t do_sqrdcmlah_b(int8_t n, int8_t m, int8_t a, bool sub) +{ + return do_sqrdmlah_b(n, m, a, sub, true); +} + +static int16_t do_sqrdcmlah_h(int16_t n, int16_t m, int16_t a, bool sub) +{ + uint32_t discard; + return do_sqrdmlah_h(n, m, a, sub, true, &discard); +} + +static int32_t do_sqrdcmlah_s(int32_t n, int32_t m, int32_t a, bool sub) +{ + uint32_t discard; + return do_sqrdmlah_s(n, m, a, sub, true, &discard); +} + +static int64_t do_sqrdcmlah_d(int64_t n, int64_t m, int64_t a, bool sub) +{ + return do_sqrdmlah_d(n, m, a, sub, true); +} + +DO_CMLA(sve2_sqrdcmlah_zzzz_b, int8_t, H1, do_sqrdcmlah_b) +DO_CMLA(sve2_sqrdcmlah_zzzz_h, int16_t, H2, do_sqrdcmlah_h) +DO_CMLA(sve2_sqrdcmlah_zzzz_s, int32_t, H4, do_sqrdcmlah_s) +DO_CMLA(sve2_sqrdcmlah_zzzz_d, int64_t, , do_sqrdcmlah_d) -#undef DO_SQRDMLAH_B -#undef DO_SQRDMLAH_H -#undef DO_SQRDMLAH_S -#undef DO_SQRDMLAH_D -#undef do_cmla #undef DO_CMLA +static void do_cmla_idx_h(int16_t *d, int16_t *n, int16_t *m, + int16_t *a, uint32_t desc, + int16_t (*fn)(int16_t, int16_t, int16_t, bool)) +{ + intptr_t i, j, oprsz = simd_oprsz(desc); + int rot = extract32(desc, SIMD_DATA_SHIFT, 2); + int idx = extract32(desc, SIMD_DATA_SHIFT + 2, 2) * 2; + int sel_a = rot & 1, sel_b = sel_a ^ 1; + bool sub_r = rot == 1 || rot == 2; + bool sub_i = rot >= 2; + + for (i = 0; i < oprsz / 2; i += 16 / 2) { + int16_t elt2_a = m[H2(i + idx + sel_a)]; + int16_t elt2_b = m[H2(i + idx + sel_b)]; + + for (j = 0; j < 16 / 2; j += 2) { + int16_t elt1_a = n[H2(i + j + sel_a)]; + + d[H2(i + j)] = fn(elt1_a, elt2_a, a[H2(i + j)], sub_r); + d[H2(i + j + 1)] = fn(elt1_a, elt2_b, a[H2(i + j + 1)], sub_i); + } + } +} + +void HELPER(sve2_cmla_idx_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + do_cmla_idx_h(vd, vn, vm, va, desc, do_cmla_h); +} + +void HELPER(sve2_sqrdcmlah_idx_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + do_cmla_idx_h(vd, vn, vm, va, desc, do_sqrdcmlah_h); +} + +static void do_cmla_idx_s(int32_t *d, int32_t *n, int32_t *m, + int32_t *a, uint32_t desc, + int32_t (*fn)(int32_t, int32_t, int32_t, bool)) +{ + intptr_t i, j, oprsz = simd_oprsz(desc); + int rot = extract32(desc, SIMD_DATA_SHIFT, 2); + int idx = extract32(desc, SIMD_DATA_SHIFT + 2, 2) * 2; + int sel_a = rot & 1, sel_b = sel_a ^ 1; + bool sub_r = rot == 1 || rot == 2; + bool sub_i = rot >= 2; + + for (i = 0; i < oprsz / 4; i += 16 / 4) { + int32_t elt2_a = m[H4(i + idx + sel_a)]; + int32_t elt2_b = m[H4(i + idx + sel_b)]; + + for (j = 0; j < 16 / 4; j += 2) { + int32_t elt1_a = n[H4(i + j + sel_a)]; + + d[H4(i + j)] = fn(elt1_a, elt2_a, a[H4(i + j)], sub_r); + d[H4(i + j + 1)] = fn(elt1_a, elt2_b, a[H4(i + j + 1)], sub_i); + } + } +} + +void HELPER(sve2_cmla_idx_s)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + do_cmla_idx_s(vd, vn, vm, va, desc, do_cmla_s); +} + +void HELPER(sve2_sqrdcmlah_idx_s)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + do_cmla_idx_s(vd, vn, vm, va, desc, do_sqrdcmlah_s); +} + #define DO_ZZXZ(NAME, TYPE, H, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6a6698bfe9..371e8cf05a 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3817,21 +3817,21 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) * SVE Multiply - Indexed */ -static bool do_zzxz_data(DisasContext *s, arg_rrxr_esz *a, +static bool do_zzxz_data(DisasContext *s, int rd, int rn, int rm, int ra, gen_helper_gvec_4 *fn, int data) { if (fn == NULL) { return false; } if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); + gen_gvec_ool_zzzz(s, fn, rd, rn, rm, ra, data); } return true; } #define DO_RRXR(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_zzxz_data(s, a, FUNC, a->index); } + { return do_zzxz_data(s, a->rd, a->rn, a->rm, a->ra, FUNC, a->index); } DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b) DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) @@ -3895,18 +3895,18 @@ DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_d, gen_helper_sve2_sqdmull_idx_d, true) #undef DO_SVE2_RRX_TB -static bool do_sve2_zzxz_data(DisasContext *s, arg_rrxr_esz *a, - gen_helper_gvec_4 *fn, int data) +static bool do_sve2_zzxz_data(DisasContext *s, int rd, int rn, int rm, + int ra, gen_helper_gvec_4 *fn, int data) { if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zzxz_data(s, a, fn, data); + return do_zzxz_data(s, rd, rn, rm, ra, fn, data); } #define DO_SVE2_RRXR(NAME, FUNC) \ - static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_sve2_zzxz_data(s, a, FUNC, a->index); } +static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ +{ return do_sve2_zzxz_data(s, a->rd, a->rn, a->rm, a->ra, FUNC, a->index); } DO_SVE2_RRXR(trans_MLA_zzxz_h, gen_helper_gvec_mla_idx_h) DO_SVE2_RRXR(trans_MLA_zzxz_s, gen_helper_gvec_mla_idx_s) @@ -3927,8 +3927,11 @@ DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d) #undef DO_SVE2_RRXR #define DO_SVE2_RRXR_TB(NAME, FUNC, TOP) \ - static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_sve2_zzxz_data(s, a, FUNC, (a->index << 1) | TOP); } +static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ +{ \ + return do_sve2_zzxz_data(s, a->rd, a->rn, a->rm, a->ra, \ + FUNC, (a->index << 1) | TOP); \ +} DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, false) DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, false) @@ -3962,6 +3965,21 @@ DO_SVE2_RRXR_TB(trans_UMLSLT_zzxw_d, gen_helper_sve2_umlsl_idx_d, true) #undef DO_SVE2_RRXR_TB +#define DO_SVE2_RRXR_ROT(NAME, FUNC) \ +static bool trans_##NAME(DisasContext *s, arg_##NAME *a) \ +{ \ + return do_sve2_zzxz_data(s, a->rd, a->rn, a->rm, a->ra, \ + FUNC, (a->index << 2) | a->rot); \ +} + +DO_SVE2_RRXR_ROT(CMLA_zzxz_h, gen_helper_sve2_cmla_idx_h) +DO_SVE2_RRXR_ROT(CMLA_zzxz_s, gen_helper_sve2_cmla_idx_s) + +DO_SVE2_RRXR_ROT(SQRDCMLAH_zzxz_h, gen_helper_sve2_sqrdcmlah_idx_h) +DO_SVE2_RRXR_ROT(SQRDCMLAH_zzxz_s, gen_helper_sve2_sqrdcmlah_idx_s) + +#undef DO_SVE2_RRXR_ROT + /* *** SVE Floating Point Multiply-Add Indexed Group */ From patchwork Tue Mar 9 16:20:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396056 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2809895jai; Tue, 9 Mar 2021 10:14:56 -0800 (PST) X-Google-Smtp-Source: ABdhPJySZ2wtDHpwvUu6sIWzw8uVyPo/X/rmdqbC4XSFAOfjf1F0TJWIUiTpRPseBBiMZKBQJ08H X-Received: by 2002:a05:6e02:e87:: with SMTP id t7mr25086230ilj.211.1615313696012; Tue, 09 Mar 2021 10:14:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313696; cv=none; d=google.com; s=arc-20160816; b=BPxTb+Gaz523iYnbT5yeGeXRxKZ0caFpy5UcR16ipyRStV9Mfo8DrwQRxLuv3xS/8n i3Zgt/e8ti5Fdy5F/nyNcIj4tcVfn8LWwWT85ssRmTrJvVn7PalDOsG+Ju0YNb8+Y4vi LFNUdSdd8Xp21NtXpCYGsurPhpfq90wfa8QBaah883WySUu2Q9F9W6jOlkiiqh5MdWzB HLNsgexqU4O0mcUJ60TQws5fhP5czZVzOGHSga2qF5t6T8GzfEOvyFasGVl1fWP4D+ml YqLRKtfgjcKYU0tKDaHWD/wDQq7hkGfzlbo/Xy0RoNXTJy2zIUetjvLUpEhYxxBMQIJH qoOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=HqfAKqJRxXesCfGA44gwR9XW/bgbySRjf69IpODQuuc=; b=NygCJ2CjOFDGPJ9P2h3D+bo6ofUcCeBYOFlsedOtCR3ZVKwVALC7pkPO3uLHd1GC1V n+GSOqgpv/N7ADj2BhgGn5Jpp3IcX/7A7GPGRcvJQTeIACNEidVDwdrIrc7R/QKPjs1A e073828qdUSzMRjnDLt2uDpQBg94EV8ZWIJjVnFwGvRCKFX6l1NXScohR1Xle5Aqvo9O Jk96boKWdEhAn1v/oLg8nbBWasz4wG0EAHAmejIsZQfojQIT45q6e+U9gyV3L60J5jOx UA139qYg/UbVJiFezU2v7sLPXdlm3sb2v3/pjA3z4/C+pWrbFH415dEldVIzTCcAczK6 0idw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BYGlgVBa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k14si14177276ion.44.2021.03.09.10.14.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:14:56 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BYGlgVBa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52190 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgsh-00082l-Bs for patch@linaro.org; Tue, 09 Mar 2021 13:14:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47208) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf87-0000SD-1m for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:43 -0500 Received: from mail-ot1-x331.google.com ([2607:f8b0:4864:20::331]:40422) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7p-0002py-J0 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:42 -0500 Received: by mail-ot1-x331.google.com with SMTP id b8so13349515oti.7 for ; Tue, 09 Mar 2021 08:22:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HqfAKqJRxXesCfGA44gwR9XW/bgbySRjf69IpODQuuc=; b=BYGlgVBaECI92X/MxzkpS6XxpHpgEKgZ+OUA4/tJZ/u69SJdyJBaQxkHrfZ3um9q6h tO/ZyVXjfQ1hFAiy/BJ8s0JdMn2mmxI1pcH6V8cfY1XDx/hyi2aJsNMN6ZhKceI25QOj wXkYarbJ4Vy2S6LI/JgMQr8NmUq9YK1IrYBVMJ/8yM3fgOADGPwdkN0VQuhOz1rEHMvg V9BDZ/amZPuhstfRE7A6hfHcw9II/EGw2IzfZeArZNNalGUSWCAbAn88MOnelims4xMd WhvsHFnVytAddXwTTntqgDaCssPYgfirk2BY7zOAEG36dnuE8eZUhChuvn/WWmon9lZm mrmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HqfAKqJRxXesCfGA44gwR9XW/bgbySRjf69IpODQuuc=; b=BG+NHfaDqeVhVmoZ/vV3l+GRoV8bbId1jZA9PIgsO4eTeYaFjKh2CtXPdFoleLwFYK A7u5lYxxauGCjkPCxW/Ss4Yf9dnNfU/4P2oKzysGlrCvAqgVP2fiuHy+Co0MJ6DpjGKN ISVqE2k02QSks124BQHQlC1t7hto0+mHWkiervCtu2zSZBY+9/czgnPvn/X9g/ROwzeS UR88Rd/6nvfcglDz4WLYaE4aqFdgJsJVsVCR3Fwn8l4YFFmcbmnuK9/jZ+KaMuBu5cFS TZYjqS+l2TkVLq4DI6hTxxRefUGlMHMKBtmcM9HTx3VIgAZ5zHT/ftE/2zLrmYZEs9P3 Wl8Q== X-Gm-Message-State: AOAM532jSzrBjjm01M96UZK2hkLuaRPhuiTziLxpIhmoEOxitNvmYnyl 7l6OVmDfmBGavJo6qefE0+heHSxewpZXP0O2 X-Received: by 2002:a9d:70c9:: with SMTP id w9mr11116233otj.62.1615306943932; Tue, 09 Mar 2021 08:22:23 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:23 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 62/78] target/arm: Implement SVE mixed sign dot product (indexed) Date: Tue, 9 Mar 2021 08:20:25 -0800 Message-Id: <20210309162041.23124-63-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::331; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x331.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++ target/arm/helper.h | 4 +++ target/arm/sve.decode | 4 +++ target/arm/translate-sve.c | 18 ++++++++++ target/arm/vec_helper.c | 68 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 99 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 3f19c4cbed..8c84bfb62b 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4213,6 +4213,11 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve2_i8mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, I8MM) != 0; +} + static inline bool isar_feature_aa64_sve2_f32mm(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, F32MM) != 0; diff --git a/target/arm/helper.h b/target/arm/helper.h index e7c463fff5..e4c6458f98 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -621,6 +621,10 @@ DEF_HELPER_FLAGS_5(gvec_sdot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_udot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sudot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_usdot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcaddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 9117196352..499d989087 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -815,6 +815,10 @@ SQRDMLSH_zzxz_h 01000100 .. 1 ..... 000101 ..... ..... @rrxr_h SQRDMLSH_zzxz_s 01000100 .. 1 ..... 000101 ..... ..... @rrxr_s SQRDMLSH_zzxz_d 01000100 .. 1 ..... 000101 ..... ..... @rrxr_d +# SVE mixed sign dot product (indexed) +USDOT_zzxw_s 01000100 .. 1 ..... 000110 ..... ..... @rrxr_s +SUDOT_zzxw_s 01000100 .. 1 ..... 000111 ..... ..... @rrxr_s + # SVE2 saturating multiply-add (indexed) SQDMLALB_zzxw_s 01000100 .. 1 ..... 0010.0 ..... ..... @rrxw_s SQDMLALB_zzxw_d 01000100 .. 1 ..... 0010.0 ..... ..... @rrxw_d diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 371e8cf05a..6f80c944d3 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3838,6 +3838,24 @@ DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) DO_RRXR(trans_UDOT_zzxw_s, gen_helper_gvec_udot_idx_b) DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) +static bool trans_SUDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_i8mm, s)) { + return false; + } + return do_zzxz_data(s, a->rd, a->rn, a->rm, a->ra, + gen_helper_gvec_sudot_idx_b, a->index); +} + +static bool trans_USDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_i8mm, s)) { + return false; + } + return do_zzxz_data(s, a->rd, a->rn, a->rm, a->ra, + gen_helper_gvec_usdot_idx_b, a->index); +} + #undef DO_RRXR static bool do_sve2_zzx_data(DisasContext *s, arg_rrx_esz *a, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 8b7269d8e1..98b707f4f5 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -677,6 +677,74 @@ void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(gvec_sudot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; + intptr_t index = simd_data(desc); + int32_t *d = vd, *a = va; + int8_t *n = vn; + uint8_t *m_indexed = (uint8_t *)vm + index * 4; + + /* + * Notice the special case of opr_sz == 8, from aa64/aa32 advsimd. + * Otherwise opr_sz is a multiple of 16. + */ + segend = MIN(4, opr_sz_4); + i = 0; + do { + uint8_t m0 = m_indexed[i * 4 + 0]; + uint8_t m1 = m_indexed[i * 4 + 1]; + uint8_t m2 = m_indexed[i * 4 + 2]; + uint8_t m3 = m_indexed[i * 4 + 3]; + + do { + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); + } while (++i < segend); + segend = i + 4; + } while (i < opr_sz_4); + + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_usdot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; + intptr_t index = simd_data(desc); + uint32_t *d = vd, *a = va; + uint8_t *n = vn; + int8_t *m_indexed = (int8_t *)vm + index * 4; + + /* + * Notice the special case of opr_sz == 8, from aa64/aa32 advsimd. + * Otherwise opr_sz is a multiple of 16. + */ + segend = MIN(4, opr_sz_4); + i = 0; + do { + int8_t m0 = m_indexed[i * 4 + 0]; + int8_t m1 = m_indexed[i * 4 + 1]; + int8_t m2 = m_indexed[i * 4 + 2]; + int8_t m3 = m_indexed[i * 4 + 3]; + + do { + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); + } while (++i < segend); + segend = i + 4; + } while (i < opr_sz_4); + + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { From patchwork Tue Mar 9 16:20:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396080 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2825836jai; Tue, 9 Mar 2021 10:38:03 -0800 (PST) X-Google-Smtp-Source: ABdhPJzYzBAW4Pwf1Onz8dQwYAoR/9ZFTy9W/s2dksKxfeNCXPf4UmqUgsHGNdOTH9PfY1OsseOU X-Received: by 2002:a5d:9418:: with SMTP id v24mr23818032ion.61.1615315083185; Tue, 09 Mar 2021 10:38:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615315083; cv=none; d=google.com; s=arc-20160816; b=jR6jCVC1Ii/yaNo2mtCRZdSpV3J7eHetuVgg6BkRi3lHbD2HdzuwJqhWqIqB+QEek6 j17AiDnIgWWngD1DTjW0J8SA5G5wiVw+HwNsOJKgeeIxYna4bCAaycYAwASlXGqz41mQ aQ83VSm9LrGIfqK4CkAGShMD/qLNA7qZW9uzzPCQYvUm2Klr0/7kevLXLJEcpYADBvZB 1WXKQrZTWVUz7Ad0/J63+fypwEfCXbldmiynwJhHZBdmdYg89P5Vo0MsJYPUOZu1jxyx fqtEOPzF1dJX2+RC/7DSGtSbZqlFsAnffe6HVDbGpz5r+DADw+6A7TOjoWuFTQEW4S6g Dnjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=SdeTKTulbQDqH8eRRD1J9G4EQUVyNG3m/VHui2jrnMM=; b=S0S7ovy3VNqd03bT3CqHIEWW+8xgGYrV51orRGj7M+bfVmC11MhcnMsi+uGSbiiLem 2VVH44Kd39hmPevkBN1y1FUkTrsXDt0xLL+dO3F0HGZGU662+syGac4bikE10QgQA8Sz +jTSyNO03njg5LZwUY8GSsrYVl7YTNDNswnOgWbTO2zyjhVJ9w8mNL8aSlRlq9K//VRV FSWGAXtkBCd0bPiEOTjRWeBMSs556xiTdqfuEPD2A/s/yq0TmVFXBeyRFMr4vLH7Y8OJ +XvwNdLDieN20h1Mry1NtumsVwRrf2JrVuTZFn8/KbrBlxhLxY2vfuRYJNgbduzyaATe UOsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="ss2LAf/2"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j3si13644554jam.97.2021.03.09.10.38.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:38:03 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="ss2LAf/2"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54252 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhF4-0000FI-J4 for patch@linaro.org; Tue, 09 Mar 2021 13:38:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47438) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8E-0000i6-A5 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:55 -0500 Received: from mail-oi1-x236.google.com ([2607:f8b0:4864:20::236]:33980) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7x-0002qG-N6 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:50 -0500 Received: by mail-oi1-x236.google.com with SMTP id x78so15565386oix.1 for ; Tue, 09 Mar 2021 08:22:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SdeTKTulbQDqH8eRRD1J9G4EQUVyNG3m/VHui2jrnMM=; b=ss2LAf/2SGarxL80xk5r+s4Miq0am4KEwjBvd8Z9Xvpvz+EDk1V2pcfRn7rRkC4s0P dTOwxKU276Byn5TKIgeRanQkeXD5efa5a8IYGDuIfjtSQ228khHcYT5ZbmY3NwFWbFXP LbjXSTLnTN+MTGN7Y/v7DY1GdvJvEX+hfFVqkVPGoXSOCYrxZUOvnwvmaF4lNeE3lvpZ 7xOXdpPIiBfUJNO1hXUBD+E2bLA0daAQy9Ky4fT3EVdNNht2/bI6GnvopNxJ69dQT09L ASUih0K34IKpTiuiLKv3pOT8Pqw+0jk5PHfDLG2O7Yq445oraiQJDX7L195M6MvYkSFK ftrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SdeTKTulbQDqH8eRRD1J9G4EQUVyNG3m/VHui2jrnMM=; b=IKWh+0YIdHxj0B0apfvKrkOSQBksTvR8UPbwzuwKG8FXgRoBt8ll9/3N32Cp+Ht3UQ vWl6pRRvd989y41vljPmukjaImqdal3ZAP8b7lu5nGmu9j+lMU+jvrOd3jxP6gt66o3P mSyXXGbZAbRZ3OGynDED1tbnzqeVzIdyDg6V7KVcl640nZSNy5OwW3hIVEaex4wjtp8H tBtKjvQARFDRupbdm9yiy/SRKpZPBDvQsQ6Dzm5TFvYdJpCETb8Hl1J9ugnzgPvEcFZO JUWcapEZ69foObDERG5rEzXFpvVpebikRlLtC2Pd2HOI3a7c7r1cnU9IgvAuwi6NIHpK zPMA== X-Gm-Message-State: AOAM532OmZjKakVmhbTpNgSkNj+/SWRZ3JpngyVWWvdSLBuUpCTKK8av iEo0t0HPi5ZNwSlnnP3z3tQgQFiNVDytwTvy X-Received: by 2002:aca:4e13:: with SMTP id c19mr3571969oib.40.1615306945414; Tue, 09 Mar 2021 08:22:25 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:24 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 63/78] target/arm: Implement SVE mixed sign dot product Date: Tue, 9 Mar 2021 08:20:26 -0800 Message-Id: <20210309162041.23124-64-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::236; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x236.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.h | 2 ++ target/arm/sve.decode | 4 ++++ target/arm/translate-sve.c | 16 ++++++++++++++++ target/arm/vec_helper.c | 18 ++++++++++++++++++ 4 files changed, 40 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index e4c6458f98..86f938c938 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -612,6 +612,8 @@ DEF_HELPER_FLAGS_5(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_udot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_usdot_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_sdot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 499d989087..6c4002de60 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1523,6 +1523,10 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx +## SVE mixed sign dot product + +USDOT_zzzz 01000100 .. 0 ..... 011 110 ..... ..... @rda_rn_rm + ### SVE2 floating point matrix multiply accumulate FMMLA 01100100 .. 1 ..... 111001 ..... ..... @rda_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6f80c944d3..23ffad33b0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8107,3 +8107,19 @@ static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a) } return true; } + +static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) +{ + if (a->esz != 2 || !dc_isar_feature(aa64_sve2_i8mm, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + vsz, vsz, 0, gen_helper_gvec_usdot_b); + } + return true; +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 98b707f4f5..9b2a4d5b7e 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -579,6 +579,24 @@ void HELPER(gvec_udot_b)(void *vd, void *vn, void *vm, void *va, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(gvec_usdot_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *a = va; + uint8_t *n = vn; + int8_t *m = vm; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = (a[i] + + n[i * 4 + 0] * m[i * 4 + 0] + + n[i * 4 + 1] * m[i * 4 + 1] + + n[i * 4 + 2] * m[i * 4 + 2] + + n[i * 4 + 3] * m[i * 4 + 3]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + void HELPER(gvec_sdot_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); From patchwork Tue Mar 9 16:20:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396058 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2811756jai; Tue, 9 Mar 2021 10:17:29 -0800 (PST) X-Google-Smtp-Source: ABdhPJzRfDfRL3tWwNUDm6gun4VTK9+O5XJ8jVSwUS2rp4o0zMjPF9TVPTHorhBtKgJJxc2b7RcW X-Received: by 2002:a05:6638:2047:: with SMTP id t7mr29648546jaj.134.1615313849056; Tue, 09 Mar 2021 10:17:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313849; cv=none; d=google.com; s=arc-20160816; b=LJJFSW1PRntKIIm2D6N3F6s8nVb0JPhe4187R/fo3yi95b1M6OKfo5p4bERDDLvRpt FSsbX9dROdEqJ53TF4LtnxmnAuDv5rpfaMkok4T/t2Dfg1WrqDc46r4SECAmDCUiPO+K 2iuL7nxXEp4o52SAkdlVo6YUWuGHsny7n18OoU345/qZ7mDjZbmh/cOIptCRjFNkxAJV arRpPMPSAPAgvRFUdF7pGJusOGKYjTHZeK9Avlk74nzVE4KQTQ384OhmzGe8YSes8e58 DZWTtDfVvoEo5ety+fifT0Zm0yt5d7SfWdVTfUb2M80wtgPiHvD4qVbGO/CtH9fAy/rG CIFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ebb1J4U0PlPdNLq+xM9aspyyU60NgXV87zexXZC7NNM=; b=GWz0MlcJd/7mbKy6HkSt4PKuFRfUEY1jh/yo3doFojn/rJsyIYeL7JyVWd2nwH4oAi Psd7n3TfhmKEB3XsDg1w8Pdk1Eogyks7FKecAPrcehn/bkM9SiVEDTLFiNfwIuBx3TC8 epz9OVVG80CpACMiNY7w/zuhm1JXc4Fg6jktPdVUx/8WhYEfZ4ibbJTfG+KDTrXC2wZb qVgkaOMKKnTJgO9Ldfir3plMp37Rls05wVRWJwoJGjYFCMICmw/qv0/Hp2c33Etf/d01 s2zJyCPmgg2ld0c4A+/yOQQx/+w8x0TIIAQSJlbRT+8mFfBOpVjMlwzz6Cs/WuRi+Ze4 w8dg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RQBvFspD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j12si13253263ila.38.2021.03.09.10.17.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:17:29 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RQBvFspD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56704 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgvA-0002Hp-D0 for patch@linaro.org; Tue, 09 Mar 2021 13:17:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47420) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8D-0000hJ-EW for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:49 -0500 Received: from mail-oo1-xc2d.google.com ([2607:f8b0:4864:20::c2d]:43016) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7x-0002qS-M9 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:49 -0500 Received: by mail-oo1-xc2d.google.com with SMTP id x19so3164881ooj.10 for ; Tue, 09 Mar 2021 08:22:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ebb1J4U0PlPdNLq+xM9aspyyU60NgXV87zexXZC7NNM=; b=RQBvFspD+gvwA1XfZNvOAMkIjd3qJEb0rUBQkZ8Fx8mnSkTFHgnz7wEHSruq1VHoC8 +0kVGGf5sVhB0iwNqUI7M2gNv1hee8JhPTUVUnOWj8RQbx/Jg25X4vMDAEL1J5TP6v2R 93hiROeVudEj7IEF/hcYR53KPwYQeZlmnu7kIb/sOzdL9J/8u6+FdpWmGhHkqbTKjN65 48MYQzQ0i3OMAeUrrHC8py6Iv0rvvHaPV3RxcItqB/ks4MxWWTe+k///bVeJV7VBVTUQ iHOpvxE/NDYgyEW7X+U3tRUn/ZLSqv+5QLzJRdmD69t8WdjeAihw/Cyu+Bu4krQ3ZTHF hGxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ebb1J4U0PlPdNLq+xM9aspyyU60NgXV87zexXZC7NNM=; b=JFAnEJFrWwm4E8sdclP4TpJ1+5UMStyPMuMU/pAxh6NFvpiDYDSSf27tpmiYmbvN9o FhR8M7mGQL1U09GkTvC2Qa6Im5vZbcMJr9J3lPCVg+KQzj2uAvboWVlC+Njunh1uGs2t ASVkiEn+A6tIVOfkrRe1UrObfBruVWIj/BWllYrqH/ZmuptcBnyMUOxRc2tnCjdYDcMe TKXEh7+Kd6zFEugOlZUtQECTgiwpXvm1mwRCDH0ols0rJKTUiYNCXWob/Vsml6LUIZOt CrfDO9SJCzLvWOAIj+OJL9RStocRW3cA9vEkI5LoPsNTmAHBhxAKfXZR2UKFR3UaoztP Z/FA== X-Gm-Message-State: AOAM532qWvxQhMXgqbjSo1xdBJ0BAen1LH94XoYZzGsZWGD21a3wc2px YiRKN+Zre6+mK39IcAhtDluDtu6PqtgosDqR X-Received: by 2002:a05:6820:3c8:: with SMTP id s8mr22820856ooj.49.1615306946483; Tue, 09 Mar 2021 08:22:26 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:26 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 64/78] target/arm: Implement SVE2 crypto unary operations Date: Tue, 9 Mar 2021 08:20:27 -0800 Message-Id: <20210309162041.23124-65-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c2d; envelope-from=richard.henderson@linaro.org; helo=mail-oo1-xc2d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 6 ++++++ target/arm/translate-sve.c | 11 +++++++++++ 2 files changed, 17 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 6c4002de60..b161334e93 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1551,3 +1551,9 @@ STNT1_zprz 1110010 .. 00 ..... 001 ... ..... ..... \ # SVE2 32-bit scatter non-temporal store (vector plus scalar) STNT1_zprz 1110010 .. 10 ..... 001 ... ..... ..... \ @rprr_scatter_store xs=0 esz=2 scale=0 + +### SVE2 Crypto Extensions + +# SVE2 crypto unary operations +# AESMC and AESIMC +AESMC 01000101 00 10000011100 decrypt:1 00000 rd:5 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 23ffad33b0..f61e6ff6ea 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8123,3 +8123,14 @@ static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) } return true; } + +static bool trans_AESMC(DisasContext *s, arg_AESMC *a) +{ + if (!dc_isar_feature(aa64_sve2_aes, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zz(s, gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt); + } + return true; +} From patchwork Tue Mar 9 16:20:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396054 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2807012jai; Tue, 9 Mar 2021 10:11:11 -0800 (PST) X-Google-Smtp-Source: ABdhPJwKH2fFV5SYTdySGI73E+eLaCmmQbkv8/w36JjDqSxIS09bXKod3CJ1luGhRqicyBYYExFt X-Received: by 2002:a02:cad9:: with SMTP id f25mr29310198jap.26.1615313471159; Tue, 09 Mar 2021 10:11:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615313471; cv=none; d=google.com; s=arc-20160816; b=AKj9YJLCrjGz9g9l4FF3u7NzKctEA2d2n/HZ0BC9kn48M8EqxRVtuPexZB1u93tTth eWugIeyQz63gSGLMQA774IZ5y0lDzquPkkI2gxizwpaNhHu5ja4GPCMNyUElcQo4CBtV /im6A6ExdXfISpgAgm/jxreJUS2LKpUd8S3mKIrhIiCmEUdyT9mTU60ocNqVvWfp3GLX kKf3oAapRPeFli6PF5H674LNETgOBFZhfvUz9KplJ2lEQN+yHjsKqY11jjnzQLHivJGh FIrhbvE6F4Dp6MYeGQnW+GnPweETBUGxrwrhASl8466CWDRV3lQORnSIwM2jUEY/hX3l 0Jeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=k7HUMIRolyiVf+4bR2Asr5i8Un4DIvsL9C7fcAzpPAA=; b=xgbPFp3UUkzeyg6yHE9pAVZhp6lGfZvmdsQjpBeDr2QD7HoVHTD6TNgAloK9HpP4KT iXY84jEyvITMV72csi++TlzmXzvqcZ/R8b/h/lm5SVHDkh0qg+HAJ25URZLfNgd6ahja hFkb7Pu6Qn7Z95cF0kNqh4bAG5NvgNOGBpMQbraUVQXpPXwcblvtqQ2NvQHjmDnVfo+h TcwNIwPPlKyk5Bkv5+QwKLbObr49YJc16qnjCToVJ6a1L0+r2aIxGBl8kO4jLpI0MRbR eBT3F1Vsb9mLAaGSFdzl4lqT3iyCVzDabguQ+gwNtdlaxVqCV9GaULd2GiM/cFSHlwfB 0IGw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yLcMPzIj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h8si14015666iow.31.2021.03.09.10.11.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:11:11 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yLcMPzIj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46320 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgp4-0004tP-I8 for patch@linaro.org; Tue, 09 Mar 2021 13:11:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47250) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf88-0000VZ-H6 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:44 -0500 Received: from mail-oi1-x22b.google.com ([2607:f8b0:4864:20::22b]:35881) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7t-0002qg-D1 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:44 -0500 Received: by mail-oi1-x22b.google.com with SMTP id o22so5786296oic.3 for ; Tue, 09 Mar 2021 08:22:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=k7HUMIRolyiVf+4bR2Asr5i8Un4DIvsL9C7fcAzpPAA=; b=yLcMPzIjEcBtO4Qlig/bEO7aFwlHLnVgs1hFqivL9FZq++JyfV7SGfT0KvcGFKA1Tl b03X+idNPu/1Mqt9acHuXvQ5Uums+rawTWNcklg5DwK9eqQ/duwOpQYPgP1Du2YmKY2e VfxPOW5ByPad3OtBcF/BcuR6j2RBpP2MkCOm2RfOEex3n3YPcLBJ2iyH14eimTkKJMK5 lXL3lSFyAQnR9hFHKDJ3iAZ7sQmf5Cup49QDmnGoglA8JM9SR8yL5ySnxlyYzfgFJgxn 8Auei4AjoA4C47hdHr8lFUIxccTRhGxm9QtnB4fRXlgq3YjYekQyGbdTh0sos39qz7T7 WZ3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k7HUMIRolyiVf+4bR2Asr5i8Un4DIvsL9C7fcAzpPAA=; b=sThjcBtADWRAADqSdDXlVlQaV3IMg6nDHCX/kmRUMcS2Y4iilSDEt1C9mU7wtYQt6H Bark9EEDMvVmUE7ed245Q964tTK55AUZdWxBtKEE8EYEEECrWwTTwThWrJwADzfvF/r9 gb8z6nfvo7qrmdzKUpM+YMEn/nyPCcEJukpjBru0ACgP14zrT9rQqRC691JgucUKDwD6 pOqVIlxtKqzVKhjLW0Gs1auZGwqpcCWzAjSYEtsV8zgLHl4ZsUPiSA01kStTr+mtmy6f 1bbX7wVGPx92tsBltCstw7Gp4Qd9Efzn6u7ngEzprtATm/E5flSI2x6YSe4H4hH63uJN 9khQ== X-Gm-Message-State: AOAM530PcCfWt04q4kykcGSuRQSe305t7Pkzecq8B8QmJTxu3OzCjGjK UiuUt8OwzdgEza1kCaxbncKFDGY4oSehoq47 X-Received: by 2002:aca:a9c7:: with SMTP id s190mr3511345oie.19.1615306947449; Tue, 09 Mar 2021 08:22:27 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:27 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 65/78] target/arm: Implement SVE2 crypto destructive binary operations Date: Tue, 9 Mar 2021 08:20:28 -0800 Message-Id: <20210309162041.23124-66-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22b; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++++ target/arm/sve.decode | 7 +++++++ target/arm/translate-sve.c | 38 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 8c84bfb62b..8c482f229b 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4213,6 +4213,11 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve2_sm4(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SM4) != 0; +} + static inline bool isar_feature_aa64_sve2_i8mm(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, I8MM) != 0; diff --git a/target/arm/sve.decode b/target/arm/sve.decode index b161334e93..bbb1dbd990 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -118,6 +118,8 @@ @pd_pn_pm ........ esz:2 .. rm:4 ....... rn:4 . rd:4 &rrr_esz @rdn_rm ........ esz:2 ...... ...... rm:5 rd:5 \ &rrr_esz rn=%reg_movprfx +@rdn_rm_e0 ........ .. ...... ...... rm:5 rd:5 \ + &rrr_esz rn=%reg_movprfx esz=0 @rdn_sh_i8u ........ esz:2 ...... ...... ..... rd:5 \ &rri_esz rn=%reg_movprfx imm=%sh8_i8u @rdn_i8u ........ esz:2 ...... ... imm:8 rd:5 \ @@ -1557,3 +1559,8 @@ STNT1_zprz 1110010 .. 10 ..... 001 ... ..... ..... \ # SVE2 crypto unary operations # AESMC and AESIMC AESMC 01000101 00 10000011100 decrypt:1 00000 rd:5 + +# SVE2 crypto destructive binary operations +AESE 01000101 00 10001 0 11100 0 ..... ..... @rdn_rm_e0 +AESD 01000101 00 10001 0 11100 1 ..... ..... @rdn_rm_e0 +SM4E 01000101 00 10001 1 11100 0 ..... ..... @rdn_rm_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index f61e6ff6ea..14360b2b08 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8134,3 +8134,41 @@ static bool trans_AESMC(DisasContext *s, arg_AESMC *a) } return true; } + +static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt) +{ + if (!dc_isar_feature(aa64_sve2_aes, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, gen_helper_crypto_aese, + a->rd, a->rn, a->rm, decrypt); + } + return true; +} + +static bool trans_AESE(DisasContext *s, arg_rrr_esz *a) +{ + return do_aese(s, a, false); +} + +static bool trans_AESD(DisasContext *s, arg_rrr_esz *a) +{ + return do_aese(s, a, true); +} + +static bool do_sm4(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn) +{ + if (!dc_isar_feature(aa64_sve2_sm4, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); + } + return true; +} + +static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a) +{ + return do_sm4(s, a, gen_helper_crypto_sm4e); +} From patchwork Tue Mar 9 16:20:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396085 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2838500jai; Tue, 9 Mar 2021 10:57:11 -0800 (PST) X-Google-Smtp-Source: ABdhPJxMrNs2Z04+eNqvU7SB4PoNXo0ttPgLfslQbgDigNuN4O42CCHRbiWCzm899qUR7EdjvMuu X-Received: by 2002:a25:b09e:: with SMTP id f30mr43136327ybj.199.1615316231744; Tue, 09 Mar 2021 10:57:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615316231; cv=none; d=google.com; s=arc-20160816; b=lE/KL7qhtbXKyTnOjkCnWcoRLMwzO34L3g7DufXOXltR7vN/PbNMGRaRxavxzBi4Hq IK0MG6u0Jc4t/gRxx4GuaHZO1XctNYvaFAszjkVna7AHBU6IgrdHJBvxpJKcn0aJkQO5 YAI1bH+lDAPhZYXSs0hlIQQBUKYtgadOt45McK4F4dE5rOHJ8JfBjZa6jNDfu/T5s+Wi pyjWaTjduXS/kT+GTI5fSQ6zirFryRX52Xb9J/dfoDXY6RYpuzA3BZQ8zSNthxi0p7wQ w8XXdzen5rKKQDTnPTFz3/AaS9oh1WLG9Mh2tGI32qT5mq80oU53kBdLrg9W+gQxH+kQ +VtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=CYN66BzpOIeY3pk3bLTjjeoAsrmmNKLt+cAT2PgztsE=; b=ikBMU+GIV4whh0xyPwqpvqK7VkMflZZfmN3tsU6fMNC+mXBbn5EFrOYHuDaZM1B4fm YOB64RnNaljhvrWEMJYP3jCcSwlmKyJxJQuDVlpxht7fwE7SS+g49a0O3xYf1gyNYLFB tTGYxwDl2Zq8BLlmvpOvbogbG64uY7CO/g5BdKWNkjeMylSvKFD7yp5Z+gcMV7s+m/GM Jfn1k86U4ZxRbDq6KjrBPodnWmwp+lTed6oglrrEwS4+SRFcatnJfEVasNhbZZtxiAHD Q+aoD5yS+aN1mXMY/M/e3ptJ9EAaG+UnMSRY4Bel+LCS0wcPrz7f4s4zLMb2lvhhs99S qeOw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=j2NXNAt1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t16si15310478ybq.375.2021.03.09.10.57.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:57:11 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=j2NXNAt1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40124 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhXb-0002uc-2e for patch@linaro.org; Tue, 09 Mar 2021 13:57:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47342) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8B-0000c8-BL for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:47 -0500 Received: from mail-oi1-x22e.google.com ([2607:f8b0:4864:20::22e]:38125) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7v-0002qu-3m for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:47 -0500 Received: by mail-oi1-x22e.google.com with SMTP id v192so7962210oia.5 for ; Tue, 09 Mar 2021 08:22:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CYN66BzpOIeY3pk3bLTjjeoAsrmmNKLt+cAT2PgztsE=; b=j2NXNAt1EvgD4xFtn5T0eQlxs1pwduriDBQd+RE9FSs5qp8SHRcIi4/r+MR2Q3fxkK Vm7Ymtvdc42bb2R/soyNvUFcKtFT68EueMHNNFJbao1vQucQ+NTVqhqz6I12ujYgycan IvNoPKkZPG5Q+cxSnugtbqNlBhR085JSgYH5B0gR9+HoVh+ThGTJUZt+jvFwnEWy20MI iWhuqqD5SputC++32VmApa3Lp7o/qGOk1ZShUq8WxbCfqNbvrcA0tsVVe63oAzc8ZerD c74cDXRwlRHYS8IVXrw3yHNoaRc2x/YlWd3Y0hthkJsnoV3MfB/AusuHgG8iiRvdS+KJ YT9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CYN66BzpOIeY3pk3bLTjjeoAsrmmNKLt+cAT2PgztsE=; b=LnOp/9PfamUGHKmupT592m/GsvkUdx1aBEX0wt8UIznUo3TQA+kAOk0N9i5XZmBFEY K3lmuTrQ7I8edd3JvxKnktFbP8Y98PT8RbNH/juYc7WhZUyTR++eRJnkKOErpCrPd8U+ NQDBoUNlIdHx9jPQznwBLe2/rK4OdPR1OK6vGn3Trek9fB45kJ51vLhnTBBSB7vpqiqK 4V05jmpREZnW6W/ejqrWPsD49hx1+QPjCKwAYXsB+U0T3xRHnDpNML95/KeRPq0ctJ6A aar+vWtfFRAlfUjfOPgC4E1LRPsaz5KKSPj2HCDHmqptip+KaJuaxRdhEif4OOyK32Km aR7w== X-Gm-Message-State: AOAM530xcThoz1+Bfv108R1WU5n+0Nsoav+Ikbdb+FjaZ/Ij6Nkz11TG pjQGQ4EDI2i1SdXClWDJ3DqE78t49sqvd+UG X-Received: by 2002:aca:5882:: with SMTP id m124mr3508467oib.137.1615306948459; Tue, 09 Mar 2021 08:22:28 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:28 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 66/78] target/arm: Implement SVE2 crypto constructive binary operations Date: Tue, 9 Mar 2021 08:20:29 -0800 Message-Id: <20210309162041.23124-67-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22e; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++++ target/arm/sve.decode | 4 ++++ target/arm/translate-sve.c | 16 ++++++++++++++++ 3 files changed, 25 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 8c482f229b..1c5ad44bad 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4213,6 +4213,11 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve2_sha3(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SHA3) != 0; +} + static inline bool isar_feature_aa64_sve2_sm4(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SM4) != 0; diff --git a/target/arm/sve.decode b/target/arm/sve.decode index bbb1dbd990..26385a9c2f 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1564,3 +1564,7 @@ AESMC 01000101 00 10000011100 decrypt:1 00000 rd:5 AESE 01000101 00 10001 0 11100 0 ..... ..... @rdn_rm_e0 AESD 01000101 00 10001 0 11100 1 ..... ..... @rdn_rm_e0 SM4E 01000101 00 10001 1 11100 0 ..... ..... @rdn_rm_e0 + +# SVE2 crypto constructive binary operations +SM4EKEY 01000101 00 1 ..... 11110 0 ..... ..... @rd_rn_rm_e0 +RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 14360b2b08..6ee4d10ece 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8172,3 +8172,19 @@ static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a) { return do_sm4(s, a, gen_helper_crypto_sm4e); } + +static bool trans_SM4EKEY(DisasContext *s, arg_rrr_esz *a) +{ + return do_sm4(s, a, gen_helper_crypto_sm4ekey); +} + +static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_sha3, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzz(s, gen_gvec_rax1, MO_64, a->rd, a->rn, a->rm); + } + return true; +} From patchwork Tue Mar 9 16:20:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396064 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2814417jai; Tue, 9 Mar 2021 10:21:14 -0800 (PST) X-Google-Smtp-Source: ABdhPJwVVSoNBumxyIGKlauev4NpvZ7fkWWiN/luHcS+7/fNMoF+twLBgYsCCNeAYKY4MhcO+RX8 X-Received: by 2002:a92:bd06:: with SMTP id c6mr26892013ile.158.1615314074360; Tue, 09 Mar 2021 10:21:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314074; cv=none; d=google.com; s=arc-20160816; b=TlrRIt9dOZnfcOA65qkK5WhnrmucNtXcQvnUKiDAJCM2Sc5OwpAR4DYy6rkEvqv8Df OjshhwPkc3zgj44J7vVH3c0UKxobTfFUmC+wBNO/RvHzfu1aQap8+6nYRFwIK0iqWy5C I5H5YUsrijNajramoReI8PsZQDpmbELEtwH4z5e5WfcB0iVxXGI9qFjmsxQLb7m8E5tD 6742aKds56vMydtWhbiQl6Zov4Zu1vbjhUNoSWKGWjZP8bGOJKgbQMnHM6vE3fxeZ76o qsR6ZUBSbwfiWC0yq14xYT4rSX8RzC7I8MHrbgASMvogcq0iDGysh0YxaGvSkvdxbKLl 7zOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=2qq8MdiFwPp8/qR7ptjD04Ks+V7JZ0l3fieJu8sZgO0=; b=p/FxEA+roViGpXvF30GQZbhCa9RXvlenbFGWYk2KgpltmuLzRJ6Lo7R2GediZIXcIq Xb6g51TwP92hdeQqCw1AG0gH4gMb9am87ziKsAaVCgQB8sStZhRaARpW5UsMwnhiuGWY 2bDTXo9T00qX6AVHdGRUAqu4FB3yCedhVoMDLVNNrAU+DhIuWX53IFUG0w9/GZfzN/Fo MaEsaIKRBikJIbDMgEud0bglatLWZvfRqIrsMBvUweEAXuGgRRcqL81OrYHldG8K1Id1 FoZ6hU7jkH+Hfo5qegy2kBu3fC0NrJfN/5jCdwkx1Ydirs0t6UWSY85msUr33Gjy6+3Q 3fUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tnn0q21u; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a4si12918536ios.8.2021.03.09.10.21.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:21:14 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tnn0q21u; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41436 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgyn-000832-N0 for patch@linaro.org; Tue, 09 Mar 2021 13:21:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47396) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8D-0000gO-0d for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:49 -0500 Received: from mail-oi1-x236.google.com ([2607:f8b0:4864:20::236]:44688) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7v-0002rD-3u for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:48 -0500 Received: by mail-oi1-x236.google.com with SMTP id w195so8896565oif.11 for ; Tue, 09 Mar 2021 08:22:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2qq8MdiFwPp8/qR7ptjD04Ks+V7JZ0l3fieJu8sZgO0=; b=tnn0q21u5KeKZpgSUWHOBM3PVQi3O1jRDERzSM4h1MX3iop1mNKcVfY/p3Tmq1FUvI PBw7dE/AwqrD1VBWBNP+GOO07ljU54+3a1aKx0msE2a5rrNdylaPc8ylawa9Lp1hDqF7 93W3mTDtDI/Bk7iOCVUtQQR4FAZIqV995zu1mG2U27xFsnh3tCyojZCrSbP+mhyou1YC b6kOHKHms7r+3kReShDYX93E+JYKA/To15SrM0flBEWn9qPEIKm+UZgJcBKjzrpDn5mX NdtmxL7k3fl8Q5tG+uXE2SZyVhgvt/3AII89JfYTaS7Cj9jk8bwsMyL6xfEf5TiX83ak M4SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2qq8MdiFwPp8/qR7ptjD04Ks+V7JZ0l3fieJu8sZgO0=; b=PooK+mVs1DjuEqifOc2dWAfRiOBUA5YqQ7w6NZSMdoFJdBi3aOtKJ/JO7we/KMtQ3M JAJGX9yIR85bicIEBcdd+w+yLB1bnVX2s+Ln3hvVxU41pIVKauehDRADZpx4MVQQptDj TAocQ7MhMGlvpgWlcdgdsyiQVI6sL6mkwvk8H1gQ5cGNb7s5ytZy3elxmiGoD5Tt2cuf eaduDKMxnEKs3z64+GcJQ4wWbPuGeguPjZyxJtJdxSp/ivYbnfPvEegz5QWUWuf55RhL rCrqQCCzXTl8xJNNWGsVe4E+/ZmAniGDhMVxCnCf5pTiKMM0U+Nv5uunApoo9SoZ8QYA Ihxg== X-Gm-Message-State: AOAM532Os4JeIuUguibRh9rqEuecfHfVSjKsLrwgaOKF8LWxNFR4OCxf gWkcIH9jyc0iPSdr7dUfdrYTpmJutxzTmjfl X-Received: by 2002:aca:afc6:: with SMTP id y189mr3455225oie.109.1615306949638; Tue, 09 Mar 2021 08:22:29 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:29 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 67/78] target/arm: Implement SVE2 TBL, TBX Date: Tue, 9 Mar 2021 08:20:30 -0800 Message-Id: <20210309162041.23124-68-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::236; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x236.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428144352.9275-1-steplong@quicinc.com> [rth: rearrange the macros a little and rebase] Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 10 +++++ target/arm/sve.decode | 5 +++ target/arm/sve_helper.c | 90 ++++++++++++++++++++++++++++++-------- target/arm/translate-sve.c | 33 ++++++++++++++ 4 files changed, 119 insertions(+), 19 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 0243ac32ec..c85cd85a0b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -661,6 +661,16 @@ DEF_HELPER_FLAGS_4(sve_tbl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_tbl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_tbl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_tbx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_tbx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_tbx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_tbx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(sve_sunpk_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_sunpk_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_sunpk_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 26385a9c2f..9bf37e322f 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -560,6 +560,11 @@ TBL 00000101 .. 1 ..... 001100 ..... ..... @rd_rn_rm # SVE unpack vector elements UNPK 00000101 esz:2 1100 u:1 h:1 001110 rn:5 rd:5 +# SVE2 Table Lookup (three sources) + +TBL_sve2 00000101 .. 1 ..... 001010 ..... ..... @rd_rn_rm +TBX 00000101 .. 1 ..... 001011 ..... ..... @rd_rn_rm + ### SVE Permute - Predicates Group # SVE permute predicate elements diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 13f40acd88..c406ae289c 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3035,28 +3035,80 @@ void HELPER(sve_rev_d)(void *vd, void *vn, uint32_t desc) } } -#define DO_TBL(NAME, TYPE, H) \ -void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ -{ \ - intptr_t i, opr_sz = simd_oprsz(desc); \ - uintptr_t elem = opr_sz / sizeof(TYPE); \ - TYPE *d = vd, *n = vn, *m = vm; \ - ARMVectorReg tmp; \ - if (unlikely(vd == vn)) { \ - n = memcpy(&tmp, vn, opr_sz); \ - } \ - for (i = 0; i < elem; i++) { \ - TYPE j = m[H(i)]; \ - d[H(i)] = j < elem ? n[H(j)] : 0; \ - } \ +typedef void tb_impl_fn(void *, void *, void *, void *, uintptr_t, bool); + +static inline void do_tbl1(void *vd, void *vn, void *vm, uint32_t desc, + bool is_tbx, tb_impl_fn *fn) +{ + ARMVectorReg scratch; + uintptr_t oprsz = simd_oprsz(desc); + + if (unlikely(vd == vn)) { + vn = memcpy(&scratch, vn, oprsz); + } + + fn(vd, vn, NULL, vm, oprsz, is_tbx); } -DO_TBL(sve_tbl_b, uint8_t, H1) -DO_TBL(sve_tbl_h, uint16_t, H2) -DO_TBL(sve_tbl_s, uint32_t, H4) -DO_TBL(sve_tbl_d, uint64_t, ) +static inline void do_tbl2(void *vd, void *vn0, void *vn1, void *vm, + uint32_t desc, bool is_tbx, tb_impl_fn *fn) +{ + ARMVectorReg scratch; + uintptr_t oprsz = simd_oprsz(desc); -#undef TBL + if (unlikely(vd == vn0)) { + vn0 = memcpy(&scratch, vn0, oprsz); + if (vd == vn1) { + vn1 = vn0; + } + } else if (unlikely(vd == vn1)) { + vn1 = memcpy(&scratch, vn1, oprsz); + } + + fn(vd, vn0, vn1, vm, oprsz, is_tbx); +} + +#define DO_TB(SUFF, TYPE, H) \ +static inline void do_tb_##SUFF(void *vd, void *vt0, void *vt1, \ + void *vm, uintptr_t oprsz, bool is_tbx) \ +{ \ + TYPE *d = vd, *tbl0 = vt0, *tbl1 = vt1, *indexes = vm; \ + uintptr_t i, nelem = oprsz / sizeof(TYPE); \ + for (i = 0; i < nelem; ++i) { \ + TYPE index = indexes[H1(i)], val = 0; \ + if (index < nelem) { \ + val = tbl0[H(index)]; \ + } else { \ + index -= nelem; \ + if (tbl1 && index < nelem) { \ + val = tbl1[H(index)]; \ + } else if (is_tbx) { \ + continue; \ + } \ + } \ + d[H(i)] = val; \ + } \ +} \ +void HELPER(sve_tbl_##SUFF)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + do_tbl1(vd, vn, vm, desc, false, do_tb_##SUFF); \ +} \ +void HELPER(sve2_tbl_##SUFF)(void *vd, void *vn0, void *vn1, \ + void *vm, uint32_t desc) \ +{ \ + do_tbl2(vd, vn0, vn1, vm, desc, false, do_tb_##SUFF); \ +} \ +void HELPER(sve2_tbx_##SUFF)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + do_tbl1(vd, vn, vm, desc, true, do_tb_##SUFF); \ +} + +DO_TB(b, uint8_t, H1) +DO_TB(h, uint16_t, H2) +DO_TB(s, uint32_t, H4) +DO_TB(d, uint64_t, ) + +#undef DO_TB #define DO_UNPK(NAME, TYPED, TYPES, HD, HS) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6ee4d10ece..6959cc9d9c 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -2417,6 +2417,39 @@ static bool trans_TBL(DisasContext *s, arg_rrr_esz *a) return true; } +static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_4 * const fns[4] = { + gen_helper_sve2_tbl_b, gen_helper_sve2_tbl_h, + gen_helper_sve2_tbl_s, gen_helper_sve2_tbl_d + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, + (a->rn + 1) % 32, a->rm, 0); + } + return true; +} + +static bool trans_TBX(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_tbx_b, gen_helper_sve2_tbx_h, + gen_helper_sve2_tbx_s, gen_helper_sve2_tbx_d + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); + } + return true; +} + static bool trans_UNPK(DisasContext *s, arg_UNPK *a) { static gen_helper_gvec_2 * const fns[4][2] = { From patchwork Tue Mar 9 16:20:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396073 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2819592jai; Tue, 9 Mar 2021 10:28:43 -0800 (PST) X-Google-Smtp-Source: ABdhPJyynGn518/DaA70VIi9KL3RhOsudzt6YG3bhDQtmPoattS+arhiXNztElC0dc3UXwWwKNeP X-Received: by 2002:a05:6e02:138f:: with SMTP id d15mr25648942ilo.217.1615314523114; Tue, 09 Mar 2021 10:28:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314523; cv=none; d=google.com; s=arc-20160816; b=sO/8cXM+YAC7iNJtjl7K0MlZQ1K0NhHTubalioL5+0JLcsV+MrNC3U6ZLumUID1MAP NPYxSwENqXLcY7zUv7scj6QovkRsO6eIjcF1F5OMOs1XpPuEsb1GXy27K9k6yHaC0u5G FgZ/ZFWxxMIH67jfhw7GHOGR68Cx2Vb1t/ecdwZcFxjIqA6SEYuxSWtgCV3G+9YH/lrT Vt7MijFe3/B4hLOpQv3FD1aQsiyyaV4PqiLB1SDK3T6HgnBMq4pdTCGCXvJHviJ2TMI1 F/EV19Tkg7QB4/9lFZOQqKAlWEcvm1Ime1s8YRDpMln0oD+vlzIzCpO2vXxXEbKwEYn3 CVhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=2LRthBYH3plyRGRdELd90NdrMXuX6CtYeCD9OlWaRNY=; b=GajaLlpV/HU+yMUqU5p0DyfKKqQVGk/j1swK/92XUJXIhRq4Z3HFfpW1WWaIwXg8yu W91pHBbgCDQb9dl0jPv+CMJfReGoRtN/GhjzYw37HzSLTfiqvDMPL5UZIgIQjtdT94ez IDdKwnMLhYWh/5Lc0nJmeq/Pijrr7XZB+W2Nz66niTZAZhNCarEjlOTupEf1kxxM0LN/ AmiPFo01dvuKXidjKUeBx2ATLwQHlCZo0i0ogoGKcKPG1Ti/+TrghOmPOnxGRD8VP1d5 OkkM+doKuccSS0URWuYciQV2+Yq9xs3UDtxowwuskKXD2BYp7OZbBcGy5i/gYCIaGPaJ SWAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WhqUF+Ff; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i11si3223041ioh.75.2021.03.09.10.28.43 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:28:43 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WhqUF+Ff; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36016 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJh62-00013e-Fs for patch@linaro.org; Tue, 09 Mar 2021 13:28:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47374) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8C-0000eu-Ak for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:48 -0500 Received: from mail-ot1-x336.google.com ([2607:f8b0:4864:20::336]:40428) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7x-0002rO-LU for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:48 -0500 Received: by mail-ot1-x336.google.com with SMTP id b8so13349901oti.7 for ; Tue, 09 Mar 2021 08:22:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2LRthBYH3plyRGRdELd90NdrMXuX6CtYeCD9OlWaRNY=; b=WhqUF+FfAtW2uKJ3oe2r6CJaDgeRBKSjGdDyy1KNVwEV/SsogVVNxVWrfbIQJ3xlAQ O4UiNc23+s9x1F6aPVHzBhOY70eWlTWBRvp0GFtyCpLao+0fwgojFdrnanIlA1/i37qB m8f+IAvhWhBLu1eYD/BLf9DXtZ7opHPW/IeMDzpToLMoh3mLGlC7awb3qnpZDqIo+JoQ JkjKYFSJHndtS3eALo5Y71HoPlkhN3mnRchZH11daCsi1RRQrrJY+8nXKqMY5aMLHKhi y2N2/FZ/5hdjU2YIrFb1OGvx9BXJJSZwrns455Vdik0+DRd05VmKxdVjVilibGkZhZxW eL4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2LRthBYH3plyRGRdELd90NdrMXuX6CtYeCD9OlWaRNY=; b=p23IT+sRk7l5+cmMZJSAILuDtDaLpuYcfJ2Dpb3Ffa23D4iteMNmyqQShgpsCqz1Ne Z5Xobe+D8GbYqMDWD8GV3+0/z9kmvu3NamBHuBGTHPXgOnAkmfsNRvbDxcbx8NpAfjzx rIgsP4FtfmzI3pIQgAi4/cIGfglUvX1fz7VBVHFLeg42IQPpmW3QWbuLkF/pfXdOWbVf JXLDGlx86i0uNNfSmJCl7bh8tPLndvikRCcqTEQvQF3LJEYbbNatbE4IeyO8Os86mF7g N3sKvDW/hSu0dAff4PQeJxXZVhzqjbKWiZskmJ8PXepO3CyW7Ecw2aT4HWLbgG6mc01c JUQA== X-Gm-Message-State: AOAM532Wi2ZnnxKS+jPk5DlxYDFG3lUgL6XygYYooT0DQ7BFk81OVkRr 157oz49FFUOENJl9o0y5+kuioyZ+eGjdp3/g X-Received: by 2002:a9d:a0a:: with SMTP id 10mr25057090otg.181.1615306950877; Tue, 09 Mar 2021 08:22:30 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:30 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 68/78] target/arm: Implement SVE2 FCVTNT Date: Tue, 9 Mar 2021 08:20:31 -0800 Message-Id: <20210309162041.23124-69-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::336; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x336.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428174332.17162-2-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 4 ++++ target/arm/sve_helper.c | 20 ++++++++++++++++++++ target/arm/translate-sve.c | 16 ++++++++++++++++ 4 files changed, 45 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index c85cd85a0b..eedb70eb50 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2734,3 +2734,8 @@ DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_fcvtnt_sh, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_fcvtnt_ds, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 9bf37e322f..2c38fe4691 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1573,3 +1573,7 @@ SM4E 01000101 00 10001 1 11100 0 ..... ..... @rdn_rm_e0 # SVE2 crypto constructive binary operations SM4EKEY 01000101 00 1 ..... 11110 0 ..... ..... @rd_rn_rm_e0 RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 + +### SVE2 floating-point convert precision odd elements +FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 +FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index c406ae289c..b719a3aacf 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7600,3 +7600,23 @@ void HELPER(fmmla_d)(void *vd, void *vn, void *vm, void *va, d[3] = float64_add(a[3], float64_add(p0, p1, status), status); } } + +#define DO_FCVTNT(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ +{ \ + intptr_t i = simd_oprsz(desc); \ + uint64_t *g = vg; \ + do { \ + uint64_t pg = g[(i - 1) >> 6]; \ + do { \ + i -= sizeof(TYPEW); \ + if (likely((pg >> (i & 63)) & 1)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + *(TYPEN *)(vd + HN(i + sizeof(TYPEN))) = OP(nn, status); \ + } \ + } while (i & 63); \ + } while (i != 0); \ +} + +DO_FCVTNT(sve2_fcvtnt_sh, uint32_t, uint16_t, H1_4, H1_2, sve_f32_to_f16) +DO_FCVTNT(sve2_fcvtnt_ds, uint64_t, uint32_t, H1_4, H1_2, float64_to_float32) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6959cc9d9c..83fab9a6fc 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8221,3 +8221,19 @@ static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a) } return true; } + +static bool trans_FCVTNT_sh(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_sh); +} + +static bool trans_FCVTNT_ds(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_ds); +} From patchwork Tue Mar 9 16:20:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396062 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2814096jai; Tue, 9 Mar 2021 10:20:43 -0800 (PST) X-Google-Smtp-Source: ABdhPJw30dLEvNuSnmcD0DRCpX40QlTchRLb6qaoHXvnkNXobnIwJ/HN5InFOMZUlgpDQNRAiw2i X-Received: by 2002:a05:6e02:dce:: with SMTP id l14mr25124671ilj.102.1615314043306; Tue, 09 Mar 2021 10:20:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314043; cv=none; d=google.com; s=arc-20160816; b=NBJp8l0J78xMW2ATYllo4BUSW9y013qqYRXOgNUpDnszxDzAPH6QMgr4hC3q0rJGF9 IgOa0AcR6MEbyuLkiI2j5sYuOiTs4RGDArKTxFeyRAX4+CH0fCisHZ2BG0+9lC0cPiFV XOvXL/yZX4AhGI84ECv/hKUg/JVeoJnCrJel2B3I0D5GR29WplDc+IeMyyxYoRyw5zTu LscZbur7IA0ixzkVSkwgFycZJkqORCrva83peJCxhSxV9h16DbCyDJnMbomBCPfF/ie3 +JDWm/SB0CMDQLN5vQImIOPJXGQ3f+cuyNqNIzL4v4EMy/1egEnmcG4JmE2TtXvoP50E KTWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=bptJGYx73w0iLgXPC0ahsEqU5HiDk0G0B7MTmQ2AFnA=; b=jaBPa1CKa/jfz6q/NsE+63bNGXa8HT9Abwj/zAav3FNRYdPiKIXOOUea8gxuhsNiwW 7HDzljZVbnzDJGSV1PIHtrU2dFoDBOtZnZKc0DptG/lkMXIk8X/+670dqNXjjAhRkotn d5UcmhcoyLQHOX+K9rUd4L6DGlRaR6IEYB6WlYigagUbbLBh/IK6oPTYGrsmTSf6SfJN ooH6lf3HRXBCTtFb8MP4EKPl4aiD4khnjynRRPz6DeMxZGULlzq+yUYspjQIGsBopi55 OG46wV6dSpN0HQ9Xhi/WpxG7w3tmWSyywGbG8T8eyKY0/5iuEfVCpn/BtVDY2vLruCBy dfKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yCkrIyVp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s190si14691542ios.22.2021.03.09.10.20.43 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:20:43 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yCkrIyVp; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36742 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJgyI-0005ve-En for patch@linaro.org; Tue, 09 Mar 2021 13:20:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47432) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8D-0000hf-Si for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:49 -0500 Received: from mail-ot1-x333.google.com ([2607:f8b0:4864:20::333]:36530) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7x-0002rb-Lz for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:49 -0500 Received: by mail-ot1-x333.google.com with SMTP id t16so13350850ott.3 for ; Tue, 09 Mar 2021 08:22:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bptJGYx73w0iLgXPC0ahsEqU5HiDk0G0B7MTmQ2AFnA=; b=yCkrIyVpXMuyWp5Xowa6qQlOP7GAwLSF0pBCvOIrcEChHqsauXQ1Fko5mdQ/HWkbCy aWwuPk+3tKdzCoKp7oodAayj/RzcZkzznjksMRSiLoWKvIK9r/33kx7XaBvt5ERq1aSB gpWPaAB4jphBMsBQTKSITpPWdHg5B+R049fKa1qAn+GLSXgCKgxXnf3xOoYSoEtN7J45 E+r4uTpfKIk2dsqvSNwuHbM1mID9QP3VucEUiB2mRXn+J21w6Kc1d2RQJKP1zLK2BW+x vvCxWX3uoIZX0c2FRzajmUqB4OCikmhkPXN5bDCKSAAKsT/a8k4NDfRBEqtpK80HuCkp yP5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bptJGYx73w0iLgXPC0ahsEqU5HiDk0G0B7MTmQ2AFnA=; b=mz7dbn7qkYwnWVPOPVrZJhwgcXyL+zvnn/x3Xsy1dvfEqpJu2cjiMrxG5xMcLvKN7V J8tCjE/x9Nu8OGWxZG5ipJMIMLxhQFaWbQneZfCGJyPTSx/1xWOLd13UR454yqxmIsi2 OTUV0gynCufS/inhKRc8m2Tzq+lu6ZF+5+BWnRBBsFQ8RHr1+kMZbEQR9R8Qcu2BxOHY da2Ne974pZFqSs9n8AWM75QnIFIcd6sicDF/JfTnbWEVTwGMeWLokyYtQJ7NWarZB1wS +6dorLZzN2ZL863lstDuwx6kGbISfFj2atXT5Tc9zmzB9/n6o3Fsg3z5eC09TSwWV+us iAmA== X-Gm-Message-State: AOAM530xS0ZopduuIWOV0sThh5A+hqqI9y78EwZehgIESBeiFDteyKlO ogXF481N9cS+vaBAWU7f/7sy7SSNNilwynV1 X-Received: by 2002:a05:6830:3495:: with SMTP id c21mr24550583otu.97.1615306952107; Tue, 09 Mar 2021 08:22:32 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:31 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 69/78] target/arm: Implement SVE2 FCVTLT Date: Tue, 9 Mar 2021 08:20:32 -0800 Message-Id: <20210309162041.23124-70-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::333; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x333.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428174332.17162-3-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 23 +++++++++++++++++++++++ target/arm/translate-sve.c | 16 ++++++++++++++++ 4 files changed, 46 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index eedb70eb50..8569d384b9 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2739,3 +2739,8 @@ DEF_HELPER_FLAGS_5(sve2_fcvtnt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_fcvtnt_ds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_fcvtlt_hs, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_fcvtlt_sd, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 2c38fe4691..d152193e89 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1576,4 +1576,6 @@ RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 ### SVE2 floating-point convert precision odd elements FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 +FCVTLT_hs 01100100 10 0010 01 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 +FCVTLT_sd 01100100 11 0010 11 101 ... ..... ..... @rd_pg_rn_e0 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index b719a3aacf..3b3f1eca5d 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7620,3 +7620,26 @@ void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ DO_FCVTNT(sve2_fcvtnt_sh, uint32_t, uint16_t, H1_4, H1_2, sve_f32_to_f16) DO_FCVTNT(sve2_fcvtnt_ds, uint64_t, uint32_t, H1_4, H1_2, float64_to_float32) + +#define DO_FCVTLT(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ +{ \ + intptr_t i = simd_oprsz(desc); \ + uint64_t *g = vg; \ + do { \ + uint64_t pg = g[(i - 1) >> 6]; \ + do { \ + i -= sizeof(TYPEW); \ + if (likely((pg >> (i & 63)) & 1)) { \ + TYPEN nn = *(TYPEN *)(vn + HN(i + sizeof(TYPEN))); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, status); \ + } \ + } while (i & 63); \ + } while (i != 0); \ +} + +DO_FCVTLT(sve2_fcvtlt_hs, uint32_t, uint16_t, H1_4, H1_2, sve_f16_to_f32) +DO_FCVTLT(sve2_fcvtlt_sd, uint64_t, uint32_t, H1_4, H1_2, float32_to_float64) + +#undef DO_FCVTLT +#undef DO_FCVTNT diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 83fab9a6fc..c29984d244 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8237,3 +8237,19 @@ static bool trans_FCVTNT_ds(DisasContext *s, arg_rpr_esz *a) } return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_ds); } + +static bool trans_FCVTLT_hs(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_hs); +} + +static bool trans_FCVTLT_sd(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_sd); +} From patchwork Tue Mar 9 16:20:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396067 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2815928jai; Tue, 9 Mar 2021 10:23:20 -0800 (PST) X-Google-Smtp-Source: ABdhPJy9KuN2BdirfKQTZtW0MrhK3zVuo3v7A8Sn921u64m9M/KQ3Xo2JB2uJRewVS7Xl/FHkv2Q X-Received: by 2002:a92:c641:: with SMTP id 1mr26995700ill.94.1615314200652; Tue, 09 Mar 2021 10:23:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314200; cv=none; d=google.com; s=arc-20160816; b=cNcA7m9seQdKWoA9n9gAXLbJb5nV+ORVPawEkm5O2IQsNmZ3PUhGaXyjVJUfeQCyJS KcHjuTAQHYcpybowomJsjrZLWUVSuQlE1JQiTdtYAm+C7b9OsgNw04CHTtFi0gcGnAF2 8K3DSuttUGN+Mh3FAMz5gWl7pAHfA0k3Y3bm/QzHkdjEZFc3Xy1zCJjg9JcrH+bx9xh6 waPyZ3xHdCO8F7+zdmIrx3BeJW6T6dieDfdTbbYTObqyMajooBmXwe4RvVjtcFxxga6V AqNoYwaA46r5QKDrMDHMBq+DmrSQwQH/qyhAEhnGFvs/nBggH9H2W0th0ccqupNt5LzH sO6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=eeIkjfuK4lIZhLLIPgTtWdu0JToAdaQwwLqvqAcG254=; b=S017bqA+aCLn5EYb1I249/nKYc/sZHhY1/vK/hvEKqhcIbQoxSUzXU6p5KPujny895 FWKul0O0+0v6RoBSh2wnczqZdzb8GeLc7Pe+PK4eJ5Lqg7AZSdvSNI/OdyMwj1gDov/C 3TiBzULSfZX7rEpb4BHN1hwK+LFZX7fgNF6K0bPQP6FjL99h9mr8UgGr3Dr/dwSHE6iG zexnEAN0B/G/qCXfwZyYcRtCHaqQ1p9Mhfa5mZIgJ8UsBUrzqgCG7e6V/RpDitA7+QVX j23Vbpm1xPqIJHwLLfdVKS+da3mOMeyQOvFcOEWEDLHIb/nu+ZZk9DZAXyFrG+r/q3Eb yd7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aJYF0Ek5; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k3si15190875iol.70.2021.03.09.10.23.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:23:20 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aJYF0Ek5; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:49744 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJh0p-0003Cx-RT for patch@linaro.org; Tue, 09 Mar 2021 13:23:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47472) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8H-0000iE-7B for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:56 -0500 Received: from mail-oi1-x229.google.com ([2607:f8b0:4864:20::229]:42966) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7z-0002rk-Mn for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:52 -0500 Received: by mail-oi1-x229.google.com with SMTP id x135so11047526oia.9 for ; Tue, 09 Mar 2021 08:22:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eeIkjfuK4lIZhLLIPgTtWdu0JToAdaQwwLqvqAcG254=; b=aJYF0Ek554kl/uUuyu83mgE6ixaCJIlo2YxwjmIm8dwjYKdOPL6IxvavdRv90amKhN nKUIfnttPNxv4PqYf5ov5gBFwMAi6+0Xm2sRjb62far8uqBl3qi4lY9+E/a/6fdvoKRG Xx1eHM233bpqqbDbEfZKk2vFg7OuhZ3UYmoBLMnVDGotqI7UDWsR84Gu1Bb7Sqj0OQUU HmQ3CMozI2c/r1HYqCO5qaDUTS3BXlWM1uxqQPgpZNVd112DkTN+G8QuVWRdZCUj57A5 r4Ls/ZbJJtxu0SN7nlwd1HAQDO43xjjOyswbDAEqmQ0z5Jig81xNQgnksLO0A0lUbABp iCiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eeIkjfuK4lIZhLLIPgTtWdu0JToAdaQwwLqvqAcG254=; b=F1Ns0t56CuhJzemlNEgQ2IMeMhDemeNtMMxg7WAMNhwX4kmv0SLRfTRgE+wiLXGzQ/ /i0M5rLZLIdKPfBPYkQ3md/VgjoTIqzv2oZcRzRlfEP1VAkKxrLAULMQn3OpBaf3zMNJ pFyj6unptNCKNnCcjh8KTQayydRasQfMNJlTybJ1NJKnPuKppEwIMpxTBZHt691hSYkj U8ogLL98k9mMJyxEnddA9jsqyejow99JJDjAuPPgS2a9IKWR6bH99J+mEy+9iHkaFjom 9Herd27MkjKjIW/SvJIbwu4e4HZqlpm5Pk575b2alnc/+a04z5Y9v7S+1GmQOxgBfiTt jlYw== X-Gm-Message-State: AOAM531x3GPQIaEEyZQjGJQO/jbBuQyVLqD7fcLFLXb3fnD4TFnVibTS ZpRrSDR2nqfDdpOyYYiiay49JMLQGPQi6JcD X-Received: by 2002:aca:2104:: with SMTP id 4mr3523575oiz.124.1615306953244; Tue, 09 Mar 2021 08:22:33 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:32 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 70/78] target/arm: Implement SVE2 FCVTXNT, FCVTX Date: Tue, 9 Mar 2021 08:20:33 -0800 Message-Id: <20210309162041.23124-71-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::229; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x229.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428174332.17162-4-steplong@quicinc.com> [rth: Use do_frint_mode, which avoids a specific runtime helper.] Signed-off-by: Richard Henderson --- target/arm/sve.decode | 2 ++ target/arm/translate-sve.c | 49 ++++++++++++++++++++++++++++++-------- 2 files changed, 41 insertions(+), 10 deletions(-) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index d152193e89..4fcd1b43f7 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1575,6 +1575,8 @@ SM4EKEY 01000101 00 1 ..... 11110 0 ..... ..... @rd_rn_rm_e0 RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 ### SVE2 floating-point convert precision odd elements +FCVTXNT_ds 01100100 00 0010 10 101 ... ..... ..... @rd_pg_rn_e0 +FCVTX_ds 01100101 00 0010 10 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 FCVTLT_hs 01100100 10 0010 01 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c29984d244..14a6a53320 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4765,11 +4765,9 @@ static bool trans_FRINTX(DisasContext *s, arg_rpr_esz *a) return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, fns[a->esz - 1]); } -static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode) +static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, + int mode, gen_helper_gvec_3_ptr *fn) { - if (a->esz == 0) { - return false; - } if (sve_access_check(s)) { unsigned vsz = vec_full_reg_size(s); TCGv_i32 tmode = tcg_const_i32(mode); @@ -4780,7 +4778,7 @@ static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode) tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), vec_full_reg_offset(s, a->rn), pred_full_reg_offset(s, a->pg), - status, vsz, vsz, 0, frint_fns[a->esz - 1]); + status, vsz, vsz, 0, fn); gen_helper_set_rmode(tmode, tmode, status); tcg_temp_free_i32(tmode); @@ -4791,27 +4789,42 @@ static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode) static bool trans_FRINTN(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_nearest_even); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz - 1]); } static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_up); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_up, frint_fns[a->esz - 1]); } static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_down); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_down, frint_fns[a->esz - 1]); } static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_to_zero); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz - 1]); } static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_ties_away); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz - 1]); } static bool trans_FRECPX(DisasContext *s, arg_rpr_esz *a) @@ -8253,3 +8266,19 @@ static bool trans_FCVTLT_sd(DisasContext *s, arg_rpr_esz *a) } return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_sd); } + +static bool trans_FCVTX_ds(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve_fcvt_ds); +} + +static bool trans_FCVTXNT_ds(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve2_fcvtnt_ds); +} From patchwork Tue Mar 9 16:20:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396077 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2822761jai; Tue, 9 Mar 2021 10:33:17 -0800 (PST) X-Google-Smtp-Source: ABdhPJw5nITJHfgEdo0tH+j0UyHw7qZfadQGFC6UnemuxXJl4QwhIyeUTIgFdmZcssR1UvlEebzZ X-Received: by 2002:a05:6638:43:: with SMTP id a3mr28619405jap.102.1615314797376; Tue, 09 Mar 2021 10:33:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314797; cv=none; d=google.com; s=arc-20160816; b=Z19IOzZYIQhxTuGN5vmE0/M8Pz2+/eygVhTSH+9Q0NC4JnOc0TNWtY/ioYkrKRzyvv G7EJ/BToGZs7SBv1vYG8rBkInr6FcNBhxUuHR79r4aMCiU5l+FEbl7nMJ0wclsitpaTW Xt35VOFTnCtsrrLdkR6hd8D3yLa/jyRpyqTkCqvic+fxkhW7IJm3H6pJ5Z5nva3gSXQF shnnNoLoRUKHcxurCU/tzEGa9JG7Wmx2ASMS90NrVSTHRHKtN8+HDys24ijnX/luQ61T N8TmHbyzuPz3JaFPqepOLwTreS9E6weXEXVobKroZpggJS3RDGJoAnx3wvAxo6fIOGpa HP1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=XLsd+chH4Trrrh1V30MCg7dAKKeY9MhnPocQKywEdeE=; b=zGmR1C2dkWofRjqn5CXmLHc4h7yA4SnxdKicyoAZSOroxN9btjqBvpu/qnfcsDFXP/ mUJNN3rVO1O9+yVQw915CQizkJs1uPaCjIYEzB7bWu4uR/zxIhiamvdlz6HtmidpsI5g tcGVQnbnnQQhY40DmfUdns+yzu/pWsghnwWVkdUxXooQOlLrVl4ldOlqHu75CS+/V1VW 4FnkQAjGzS5sI3ev2QzYzJqU5gnP+7sb7YHvmzfm8jJ6LCa72bmra6oI6xK00ZUVlEkV ouSdEKa4mVtWPeZqCywr6bU6feR7McoR0bAVrFVzZsrDbflYF/3mSXvwdzbZx7cq0dvX A8Qw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uZlSvWF0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i4si16111468iov.52.2021.03.09.10.33.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:33:17 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uZlSvWF0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhAS-0005AF-O2 for patch@linaro.org; Tue, 09 Mar 2021 13:33:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47474) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8H-0000iF-Ph for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:56 -0500 Received: from mail-ot1-x334.google.com ([2607:f8b0:4864:20::334]:40427) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf7z-0002s3-NE for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:53 -0500 Received: by mail-ot1-x334.google.com with SMTP id b8so13350074oti.7 for ; Tue, 09 Mar 2021 08:22:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XLsd+chH4Trrrh1V30MCg7dAKKeY9MhnPocQKywEdeE=; b=uZlSvWF0/XoC/Dme45TzdV84Y4WFpvy7diAvjduWO089HvAFvKb0vzWACNqcDnHh5r Z8+6Q/a6fHksE5zoV2GWjfwLPJMc1YuPDyNAJMu3aAHAto5We2jaKdF3kveDR2rqI/4F fllbb7iRl/2y+h1RpQ3JAS/yO5INUHXcmc6hRl84ioktbTkVbQwswba1NeQsDwr3MSNC VxX0oZJcPifiY6j46B1fvsdlKSLV6DORC6LdcZedMzs1W0Uu4uOAQtIJB6ZQCyS1blPI G8NsPnogl9TKZhQsXS/PPUUQuPPCQp64GwAeM4UXN051W8PS0/umyTHAHsrgADCQ0aDY n6eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XLsd+chH4Trrrh1V30MCg7dAKKeY9MhnPocQKywEdeE=; b=WT6X1VfchJFA5ulTcuQApxiAp4LpKOSVm11lMT5DREU3BPNKKnnedbcHc45s+CdKxH 7uphj4tMbLNdw2WDgLdnHRmWWquOMt92CaOSR732DabZBeJbUwhDWO4j1xF585ZgX0s8 pFLfNgUjpShBIn31BEe/CAvb9IqY0UMwnILZN40jXIB5ylmHsEDMrbOwNH0pDQWxYuyp 81xOJJI+ZrGHXWmkH3apwX63lgJ+w22ic7I0q/HGntop2nfR/PWa/5Wstco5z6MzhFan k9c4oxK/YLqlCzRztiev7FeG7UdAvDQN3Rg+fblcxLtfwPt9j9zwL5sSMinqlbi8dhtO NOzA== X-Gm-Message-State: AOAM532GL3FPxA49AAcz2FFMZxUsPxYROqqREKN2Rw6T/I1QIksXK9l6 IPPkYp6X2Knv6Bkxl6udYmQ5BD6JrmHD6VD2 X-Received: by 2002:a9d:6390:: with SMTP id w16mr19810705otk.178.1615306954275; Tue, 09 Mar 2021 08:22:34 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:33 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 71/78] target/arm: Implement SVE2 FLOGB Date: Tue, 9 Mar 2021 08:20:34 -0800 Message-Id: <20210309162041.23124-72-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::334; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x334.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200430191405.21641-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fixed esz index and c++ comments v3: Fixed denormal arithmetic and raise invalid. --- target/arm/helper-sve.h | 4 +++ target/arm/sve.decode | 3 +++ target/arm/sve_helper.c | 52 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 24 ++++++++++++++++++ 4 files changed, 83 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 8569d384b9..0a0ee8587b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2744,3 +2744,7 @@ DEF_HELPER_FLAGS_5(sve2_fcvtlt_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_fcvtlt_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(flogb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(flogb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(flogb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 4fcd1b43f7..9ad017d4d1 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1581,3 +1581,6 @@ FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 FCVTLT_hs 01100100 10 0010 01 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 FCVTLT_sd 01100100 11 0010 11 101 ... ..... ..... @rd_pg_rn_e0 + +### SVE2 floating-point convert to integer +FLOGB 01100101 00 011 esz:2 0101 pg:3 rn:5 rd:5 &rpr_esz diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 3b3f1eca5d..1e7c9c4f66 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -4695,6 +4695,58 @@ DO_ZPZ_FP(sve_ucvt_dh, uint64_t, , uint64_to_float16) DO_ZPZ_FP(sve_ucvt_ds, uint64_t, , uint64_to_float32) DO_ZPZ_FP(sve_ucvt_dd, uint64_t, , uint64_to_float64) +static int16_t do_float16_logb_as_int(float16 a, float_status *s) +{ + if (float16_is_normal(a)) { + return extract16(a, 10, 5) - 15; + } else if (float16_is_infinity(a)) { + return INT16_MAX; + } else if (float16_is_any_nan(a) || float16_is_zero(a)) { + float_raise(float_flag_invalid, s); + return INT16_MIN; + } else { + /* + * denormal: bias - fractional_zeros + * = bias + masked_zeros - uint32_zeros + */ + return -15 + 22 - clz32(extract16(a, 0, 10)); + } +} + +static int32_t do_float32_logb_as_int(float32 a, float_status *s) +{ + if (float32_is_normal(a)) { + return extract32(a, 23, 8) - 127; + } else if (float32_is_infinity(a)) { + return INT32_MAX; + } else if (float32_is_any_nan(a) || float32_is_zero(a)) { + float_raise(float_flag_invalid, s); + return INT32_MIN; + } else { + /* denormal (see above) */ + return -127 + 9 - clz32(extract32(a, 0, 23)); + } +} + +static int64_t do_float64_logb_as_int(float64 a, float_status *s) +{ + if (float64_is_normal(a)) { + return extract64(a, 52, 11) - 1023; + } else if (float64_is_infinity(a)) { + return INT64_MAX; + } else if (float64_is_any_nan(a) || float64_is_zero(a)) { + float_raise(float_flag_invalid, s); + return INT64_MIN; + } else { + /* denormal (see above) */ + return -1023 + 12 - clz64(extract64(a, 0, 52)); + } +} + +DO_ZPZ_FP(flogb_h, float16, H1_2, do_float16_logb_as_int) +DO_ZPZ_FP(flogb_s, float32, H1_4, do_float32_logb_as_int) +DO_ZPZ_FP(flogb_d, float64, , do_float64_logb_as_int) + #undef DO_ZPZ_FP static void do_fmla_zpzzz_h(void *vd, void *vn, void *vm, void *va, void *vg, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 14a6a53320..d7c3ad047f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8282,3 +8282,27 @@ static bool trans_FCVTXNT_ds(DisasContext *s, arg_rpr_esz *a) } return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve2_fcvtnt_ds); } + +static bool trans_FLOGB(DisasContext *s, arg_rpr_esz *a) +{ + static gen_helper_gvec_3_ptr * const fns[] = { + NULL, gen_helper_flogb_h, + gen_helper_flogb_s, gen_helper_flogb_d + }; + + if (!dc_isar_feature(aa64_sve2, s) || fns[a->esz] == NULL) { + return false; + } + if (sve_access_check(s)) { + TCGv_ptr status = + fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + unsigned vsz = vec_full_reg_size(s); + + tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + pred_full_reg_offset(s, a->pg), + status, vsz, vsz, 0, fns[a->esz]); + tcg_temp_free_ptr(status); + } + return true; +} From patchwork Tue Mar 9 16:20:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396066 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2815623jai; Tue, 9 Mar 2021 10:22:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJy5DxEhSSXH02xRpv7KyTkrmuCxOGbjtnNpMh+xGXl6JfAMXvnfWkGKTlby4QV3kV3fFiSt X-Received: by 2002:a6b:5112:: with SMTP id f18mr23996310iob.196.1615314172911; Tue, 09 Mar 2021 10:22:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314172; cv=none; d=google.com; s=arc-20160816; b=meNfH0BYBpNZ1Vi3uVsNGOC3G0Pde+ExpaJbqQqcLF3Sn+IUZDx+CiskdHa8wscw1y XMwld+3NioE9Tk5T4vCyHj8gZ18Rps87GLlU8ArVom1K8cZ7k4hutA0OH1MeQX9ZtW58 QeO/4zXua1sQ+vx8iM2KeqSyDiuO+X1+UJ4m/jjpIOqn+BK3YFYv9g88skZyyukMDKSe 1G0srFQ6T5lNyrrS58RG2ze/lrNHm9HUaDps/4T0TgA4nIgWFW/8RA92h79yom78xt0J IosIygDmAChaqyvvi7fwHGCOkmfHaZyJSmvneTI7i/xpzba0SZ68APMFJjxe9hAJeb0w RFsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=WetfNKkPM8tDPE/v/V+6AUBIG8HNbSZnOY/2KKUmIqc=; b=X/pDP89vmyCxeTRDgpYsTHbaqxmbBOuOU5oJ3YQIl/7vJ0J2F4rNRi1vhKmSrxBJDQ 0Elz15fZaJ/d/aqa8xuyxvumICQEJwUB7DiKDmXQl4Cbx+EaNBRiLYCkchBRupQOOcfV sRWaIuuuDg7DsQNS1XwxE4+zF0r71lDEGTVx+n3IO8iQA1l0uVcVp/KsytC1h1wHOoOp whs7dkGlObqK8h0c7c5ok6Q4LVAg3Y2NabhVZTx7a3LC3JzRqw2C/1GWZPVTQ95lI3AQ k+cyxYivIPFDly/SaP1LRb4V5z/Xv4OtjAj2PQS5GltygkPiPDR8oIEItctDhUayQ3RI JhRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=r+bvgRzP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a5si1257674ilv.16.2021.03.09.10.22.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:22:52 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=r+bvgRzP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45044 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJh0O-0001A6-5s for patch@linaro.org; Tue, 09 Mar 2021 13:22:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47518) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8K-0000iL-0n for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:58 -0500 Received: from mail-oi1-x22c.google.com ([2607:f8b0:4864:20::22c]:47084) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf80-0002sG-AY for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:55 -0500 Received: by mail-oi1-x22c.google.com with SMTP id f3so15498550oiw.13 for ; Tue, 09 Mar 2021 08:22:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WetfNKkPM8tDPE/v/V+6AUBIG8HNbSZnOY/2KKUmIqc=; b=r+bvgRzPSnmJDuzanddzlCEXYYGsvou9+tHRdSSOxsa6kqaZGpsOk8i6L8iulFZrks n9MOjvwqVaJ85pHiHSDNVVbeV6znA7viSPRdGI4sduRFgjlMVuRs2ZxNUP9WFtel2uuP k1LHR+rmVIAJK4z+0BWDd4pYWqvAvmRBH8OMjctI9mvlULNbh5+BKCjFAgKiQ6A4BD4E Nx1lZZC7sUQ92gH/fsFe9mCCJqViknla8JZ1RVuOT5TIbyr7oQFKRZQCvsz4TO2LnFVC Yzq0Pue4qoP1/PgiTNKMp+oFVuo9yWEgMZShmwnNuxo6pGNnEgy3isRM8fm3VDAcoQjH sCPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WetfNKkPM8tDPE/v/V+6AUBIG8HNbSZnOY/2KKUmIqc=; b=AgGg9YukoZUUEj7r7fM1BsgCJOIst3ygYQFfGKGPKsaa2uSe9bP9d7HfQKoDQc+hjT shNYl3Ky363tYSAj6xqChgz6EZ3eGv0xjkZwNCuUW9dPc/nIb5Zc3k3/8q/a9Eq6psw0 o4vcXJs3zfTuEZOU9L+BPpkFtbRmwSS4hZMKQ882y0F+RcgqiIiXglhLCXhxJL3H9fJB 7QXidVIJSVTW13M/naj75OwBpZbhiPMTlBwqX9Rex6mkc7JOQWdzg94l/twbeqPU6L6l txl13UfVgCuQCvEQNBN3LUgaN3euzKRhRoDVz4fxzLjiu8cYpAm4KT8Ar5ZmKkkoniEB Yhhw== X-Gm-Message-State: AOAM530c8HCVv/4Z27lCs+sVSvVC3LXZXnGUw0rL+ixueYgMcMpTNtoe k7LfP/lB2gcj//a3B5c3CQt2fv+J9fW8wGiQ X-Received: by 2002:aca:ea83:: with SMTP id i125mr3553595oih.18.1615306955398; Tue, 09 Mar 2021 08:22:35 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:35 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 72/78] target/arm: Share table of sve load functions Date: Tue, 9 Mar 2021 08:20:35 -0800 Message-Id: <20210309162041.23124-73-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22c; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The table used by do_ldrq is a subset of the table used by do_ld_zpa; we can share them by passing dtype instead of msz to do_ldrq. Signed-off-by: Richard Henderson --- target/arm/translate-sve.c | 254 ++++++++++++++++++------------------- 1 file changed, 126 insertions(+), 128 deletions(-) -- 2.25.1 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index d7c3ad047f..130abea7a5 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5204,128 +5204,130 @@ static void do_mem_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, tcg_temp_free_i32(t_desc); } +/* Indexed by [mte][be][dtype][nreg] */ +static gen_helper_gvec_mem * const ldr_fns[2][2][16][4] = { + { /* mte inactive, little-endian */ + { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, + gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, + { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_le_r, gen_helper_sve_ld2hh_le_r, + gen_helper_sve_ld3hh_le_r, gen_helper_sve_ld4hh_le_r }, + { gen_helper_sve_ld1hsu_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_le_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld2ss_le_r, + gen_helper_sve_ld3ss_le_r, gen_helper_sve_ld4ss_le_r }, + { gen_helper_sve_ld1sdu_le_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_le_r, gen_helper_sve_ld2dd_le_r, + gen_helper_sve_ld3dd_le_r, gen_helper_sve_ld4dd_le_r } }, + + /* mte inactive, big-endian */ + { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, + gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, + { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_be_r, gen_helper_sve_ld2hh_be_r, + gen_helper_sve_ld3hh_be_r, gen_helper_sve_ld4hh_be_r }, + { gen_helper_sve_ld1hsu_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_be_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld2ss_be_r, + gen_helper_sve_ld3ss_be_r, gen_helper_sve_ld4ss_be_r }, + { gen_helper_sve_ld1sdu_be_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_be_r, gen_helper_sve_ld2dd_be_r, + gen_helper_sve_ld3dd_be_r, gen_helper_sve_ld4dd_be_r } } }, + + { /* mte active, little-endian */ + { { gen_helper_sve_ld1bb_r_mte, + gen_helper_sve_ld2bb_r_mte, + gen_helper_sve_ld3bb_r_mte, + gen_helper_sve_ld4bb_r_mte }, + { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_le_r_mte, + gen_helper_sve_ld2hh_le_r_mte, + gen_helper_sve_ld3hh_le_r_mte, + gen_helper_sve_ld4hh_le_r_mte }, + { gen_helper_sve_ld1hsu_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_le_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_le_r_mte, + gen_helper_sve_ld2ss_le_r_mte, + gen_helper_sve_ld3ss_le_r_mte, + gen_helper_sve_ld4ss_le_r_mte }, + { gen_helper_sve_ld1sdu_le_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_le_r_mte, + gen_helper_sve_ld2dd_le_r_mte, + gen_helper_sve_ld3dd_le_r_mte, + gen_helper_sve_ld4dd_le_r_mte } }, + + /* mte active, big-endian */ + { { gen_helper_sve_ld1bb_r_mte, + gen_helper_sve_ld2bb_r_mte, + gen_helper_sve_ld3bb_r_mte, + gen_helper_sve_ld4bb_r_mte }, + { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_be_r_mte, + gen_helper_sve_ld2hh_be_r_mte, + gen_helper_sve_ld3hh_be_r_mte, + gen_helper_sve_ld4hh_be_r_mte }, + { gen_helper_sve_ld1hsu_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_be_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_be_r_mte, + gen_helper_sve_ld2ss_be_r_mte, + gen_helper_sve_ld3ss_be_r_mte, + gen_helper_sve_ld4ss_be_r_mte }, + { gen_helper_sve_ld1sdu_be_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_be_r_mte, + gen_helper_sve_ld2dd_be_r_mte, + gen_helper_sve_ld3dd_be_r_mte, + gen_helper_sve_ld4dd_be_r_mte } } }, +}; + static void do_ld_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype, int nreg) { - static gen_helper_gvec_mem * const fns[2][2][16][4] = { - { /* mte inactive, little-endian */ - { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, - gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, - { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_le_r, gen_helper_sve_ld2hh_le_r, - gen_helper_sve_ld3hh_le_r, gen_helper_sve_ld4hh_le_r }, - { gen_helper_sve_ld1hsu_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_le_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld2ss_le_r, - gen_helper_sve_ld3ss_le_r, gen_helper_sve_ld4ss_le_r }, - { gen_helper_sve_ld1sdu_le_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_le_r, gen_helper_sve_ld2dd_le_r, - gen_helper_sve_ld3dd_le_r, gen_helper_sve_ld4dd_le_r } }, - - /* mte inactive, big-endian */ - { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, - gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, - { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_be_r, gen_helper_sve_ld2hh_be_r, - gen_helper_sve_ld3hh_be_r, gen_helper_sve_ld4hh_be_r }, - { gen_helper_sve_ld1hsu_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_be_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld2ss_be_r, - gen_helper_sve_ld3ss_be_r, gen_helper_sve_ld4ss_be_r }, - { gen_helper_sve_ld1sdu_be_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_be_r, gen_helper_sve_ld2dd_be_r, - gen_helper_sve_ld3dd_be_r, gen_helper_sve_ld4dd_be_r } } }, - - { /* mte active, little-endian */ - { { gen_helper_sve_ld1bb_r_mte, - gen_helper_sve_ld2bb_r_mte, - gen_helper_sve_ld3bb_r_mte, - gen_helper_sve_ld4bb_r_mte }, - { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_le_r_mte, - gen_helper_sve_ld2hh_le_r_mte, - gen_helper_sve_ld3hh_le_r_mte, - gen_helper_sve_ld4hh_le_r_mte }, - { gen_helper_sve_ld1hsu_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_le_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_le_r_mte, - gen_helper_sve_ld2ss_le_r_mte, - gen_helper_sve_ld3ss_le_r_mte, - gen_helper_sve_ld4ss_le_r_mte }, - { gen_helper_sve_ld1sdu_le_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_le_r_mte, - gen_helper_sve_ld2dd_le_r_mte, - gen_helper_sve_ld3dd_le_r_mte, - gen_helper_sve_ld4dd_le_r_mte } }, - - /* mte active, big-endian */ - { { gen_helper_sve_ld1bb_r_mte, - gen_helper_sve_ld2bb_r_mte, - gen_helper_sve_ld3bb_r_mte, - gen_helper_sve_ld4bb_r_mte }, - { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_be_r_mte, - gen_helper_sve_ld2hh_be_r_mte, - gen_helper_sve_ld3hh_be_r_mte, - gen_helper_sve_ld4hh_be_r_mte }, - { gen_helper_sve_ld1hsu_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_be_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_be_r_mte, - gen_helper_sve_ld2ss_be_r_mte, - gen_helper_sve_ld3ss_be_r_mte, - gen_helper_sve_ld4ss_be_r_mte }, - { gen_helper_sve_ld1sdu_be_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_be_r_mte, - gen_helper_sve_ld2dd_be_r_mte, - gen_helper_sve_ld3dd_be_r_mte, - gen_helper_sve_ld4dd_be_r_mte } } }, - }; gen_helper_gvec_mem *fn - = fns[s->mte_active[0]][s->be_data == MO_BE][dtype][nreg]; + = ldr_fns[s->mte_active[0]][s->be_data == MO_BE][dtype][nreg]; /* * While there are holes in the table, they are not @@ -5563,14 +5565,8 @@ static bool trans_LDNF1_zpri(DisasContext *s, arg_rpri_load *a) return true; } -static void do_ldrq(DisasContext *s, int zt, int pg, TCGv_i64 addr, int msz) +static void do_ldrq(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype) { - static gen_helper_gvec_mem * const fns[2][4] = { - { gen_helper_sve_ld1bb_r, gen_helper_sve_ld1hh_le_r, - gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld1dd_le_r }, - { gen_helper_sve_ld1bb_r, gen_helper_sve_ld1hh_be_r, - gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld1dd_be_r }, - }; unsigned vsz = vec_full_reg_size(s); TCGv_ptr t_pg; TCGv_i32 t_desc; @@ -5602,7 +5598,9 @@ static void do_ldrq(DisasContext *s, int zt, int pg, TCGv_i64 addr, int msz) t_pg = tcg_temp_new_ptr(); tcg_gen_addi_ptr(t_pg, cpu_env, poff); - fns[s->be_data == MO_BE][msz](cpu_env, t_pg, addr, t_desc); + gen_helper_gvec_mem *fn + = ldr_fns[s->mte_active[0]][s->be_data == MO_BE][dtype][0]; + fn(cpu_env, t_pg, addr, t_desc); tcg_temp_free_ptr(t_pg); tcg_temp_free_i32(t_desc); @@ -5624,7 +5622,7 @@ static bool trans_LD1RQ_zprr(DisasContext *s, arg_rprr_load *a) TCGv_i64 addr = new_tmp_a64(s); tcg_gen_shli_i64(addr, cpu_reg(s, a->rm), msz); tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, a->rn)); - do_ldrq(s, a->rd, a->pg, addr, msz); + do_ldrq(s, a->rd, a->pg, addr, a->dtype); } return true; } @@ -5634,7 +5632,7 @@ static bool trans_LD1RQ_zpri(DisasContext *s, arg_rpri_load *a) if (sve_access_check(s)) { TCGv_i64 addr = new_tmp_a64(s); tcg_gen_addi_i64(addr, cpu_reg_sp(s, a->rn), a->imm * 16); - do_ldrq(s, a->rd, a->pg, addr, dtype_msz(a->dtype)); + do_ldrq(s, a->rd, a->pg, addr, a->dtype); } return true; } From patchwork Tue Mar 9 16:20:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396070 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2818032jai; Tue, 9 Mar 2021 10:26:19 -0800 (PST) X-Google-Smtp-Source: ABdhPJzldkELDMQXK9wsB9AOm4TsRIA6fOCkJdKZm/9M36esTgs4outye1gcqfDTubBSNbEobyCT X-Received: by 2002:a25:e7d7:: with SMTP id e206mr41685529ybh.110.1615314379216; Tue, 09 Mar 2021 10:26:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314379; cv=none; d=google.com; s=arc-20160816; b=SBDJnHs0tmuEAJ2xujiEajV37YK0x4lq0gnxWxJL/uPVc9MYOfJ8KRFdCMDtIgwjku px37ZlWkyAnaJsjJ49yfY1fLnR07qA0xDEwUiB6mcMEnj27Y3xBywZdZg5I3GIY6QUtG ia78zmoXId/UjBCmfolgMOyExEDfkLrm9Sskubzgk8IBKt/Eb/vffsmhw4e1mWMRsxJ3 l01tXAhj10l4un/+pJiNwUX6L0MLst/ClKGqvgs9D8SpeoxKD7kFUscsGan15y9cvNoD 593BWSMHbJWTd3RAF0w+rcJSebpQbZT+cwCPy4jH23KisJHt1hkwkVyrveShsc6um38g wYcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Gp/V5Vy9Sz7v8gKnc23SVNqKOB8MLAcvhTvbwZmR4cw=; b=JQ2ZlwFoheO9cstaj0/Tv72tNl5Db8XzYbl1sUxFvCI5oStuvN+sEwMCgWcqP0aqtK GUJe/EHoYMPDPmFcK9wwvoVdSOJ02fFqvll3+V0P+JCIfDo535VPs8pk0HuRYi17N9mB zKZXSyugto9OabZfbVgXaPT0k+S+5lguVvBBcJqfHkZfF2d6pzmLnOYT/g3CKu7k4uhf w/+qk/2GdkzdUTIuRURuatocE46wr1v7oDSi1A64UrFxvqPkhcqVBUdXzduSZZH1vMer BqluEq42mOvuTP++oxHl5gby8OtQ5d44RW199iCTy69TYBm0yHlNB2Rj652kQP0FDlZW qCZw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QjtuEqSl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o10si5486314ybj.1.2021.03.09.10.26.19 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:26:19 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QjtuEqSl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54630 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJh3i-0005MC-G5 for patch@linaro.org; Tue, 09 Mar 2021 13:26:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47536) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8L-0000iW-Ah for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:58 -0500 Received: from mail-ot1-x32b.google.com ([2607:f8b0:4864:20::32b]:45317) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf81-0002sT-Kz for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:57 -0500 Received: by mail-ot1-x32b.google.com with SMTP id r24so5242110otp.12 for ; Tue, 09 Mar 2021 08:22:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Gp/V5Vy9Sz7v8gKnc23SVNqKOB8MLAcvhTvbwZmR4cw=; b=QjtuEqSlAxWuqa9Eru5+j3X3/uCBcKVXR8N0W5nUmfC77AX0WYnFs0rjxIu9Lpsxol nZ3QGzm9BsMQWnoILqgEUkWR7iSm2ZAFrTskYgCzbWT1UUoM7tYpnU/Gc89vp0DNJjNx mE11AQaEIznKY06yeMEKPjrLT9AD++1GF5zDwCaS3opkpnetyDOM1LmV6q8yAlzMWCZF WSH4Goe0QRiYewswc03bF0S+JORZbuuTi4G8V85gvnDsIQNwofUcmMdrDSVLPdV8eaek eop+rowTwtHimdqmATTzUsExyqiVIdV7fyAZON72mKBm4zD1zAEKkO2L/E8I6qjTXbXH 9gRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Gp/V5Vy9Sz7v8gKnc23SVNqKOB8MLAcvhTvbwZmR4cw=; b=QSNlBE+K3rIGT4ui6CQEv12g/EXAC9Z1GgwANYthg87ZzGOqslGC8wpHdXnoanCgMa oDe5mgVRZ2BrdkPFSUfn7boUQu2ldWgE7vNhvVMzuVEZoz6cbjqV9b2uR9llNc1LH/M6 6TNQC4+0WHUQQvYcdDdlOECX5NWxBsJx+ad1TXp3tYqp+GxEhgX6RR2PxCMvpSQSHu2L nBxUuYch9ZkDthery6CbNQlz5mU20hTm0DoPxikXvsF9q5I9ME2reS3rYT0mbBRtAf1Q lMY/Yn4gldXY2vdgGnYYh3Da5F9ocGJrE+45lJcEaLVWboxX/cSyc1UfNP+o749XmLgA KD3A== X-Gm-Message-State: AOAM530RCvteMTWHiWmLHyNtw7HBGNAc/EDr8PuA7g6Iv4VDXVjyWQ/S fHjxtnvbO/suwpbqr8zIGky3SebDTyAG3gHc X-Received: by 2002:a9d:12a4:: with SMTP id g33mr24446868otg.308.1615306956664; Tue, 09 Mar 2021 08:22:36 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:36 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 73/78] target/arm: Implement SVE2 LD1RO Date: Tue, 9 Mar 2021 08:20:36 -0800 Message-Id: <20210309162041.23124-74-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32b; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 4 ++ target/arm/translate-sve.c | 97 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 101 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 9ad017d4d1..f3b405b741 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1119,11 +1119,15 @@ LD_zpri 1010010 .. nreg:2 0.... 111 ... ..... ..... @rpri_load_msz # SVE load and broadcast quadword (scalar plus scalar) LD1RQ_zprr 1010010 .. 00 ..... 000 ... ..... ..... \ @rprr_load_msz nreg=0 +LD1RO_zprr 1010010 .. 01 ..... 000 ... ..... ..... \ + @rprr_load_msz nreg=0 # SVE load and broadcast quadword (scalar plus immediate) # LD1RQB, LD1RQH, LD1RQS, LD1RQD LD1RQ_zpri 1010010 .. 00 0.... 001 ... ..... ..... \ @rpri_load_msz nreg=0 +LD1RO_zpri 1010010 .. 01 0.... 001 ... ..... ..... \ + @rpri_load_msz nreg=0 # SVE 32-bit gather prefetch (scalar plus 32-bit scaled offsets) PRF 1000010 00 -1 ----- 0-- --- ----- 0 ---- diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 130abea7a5..58bebbfedd 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5637,6 +5637,103 @@ static bool trans_LD1RQ_zpri(DisasContext *s, arg_rpri_load *a) return true; } +static void do_ldro(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype) +{ + unsigned vsz = vec_full_reg_size(s); + unsigned vsz_r32; + TCGv_ptr t_pg; + TCGv_i32 t_desc; + int desc, poff, doff; + + if (vsz < 32) { + /* + * Note that this UNDEFINED check comes after CheckSVEEnabled() + * in the ARM pseudocode, which is the sve_access_check() done + * in our caller. We should not now return false from the caller. + */ + unallocated_encoding(s); + return; + } + + /* Load the first octaword using the normal predicated load helpers. */ + + poff = pred_full_reg_offset(s, pg); + if (vsz > 32) { + /* + * Zero-extend the first 32 bits of the predicate into a temporary. + * This avoids triggering an assert making sure we don't have bits + * set within a predicate beyond VQ, but we have lowered VQ to 2 + * for this load operation. + */ + TCGv_i64 tmp = tcg_temp_new_i64(); +#ifdef HOST_WORDS_BIGENDIAN + poff += 4; +#endif + tcg_gen_ld32u_i64(tmp, cpu_env, poff); + + poff = offsetof(CPUARMState, vfp.preg_tmp); + tcg_gen_st_i64(tmp, cpu_env, poff); + tcg_temp_free_i64(tmp); + } + + t_pg = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(t_pg, cpu_env, poff); + + desc = simd_desc(32, 32, zt); + t_desc = tcg_const_i32(desc); + + gen_helper_gvec_mem *fn + = ldr_fns[s->mte_active[0]][s->be_data == MO_BE][dtype][0]; + fn(cpu_env, t_pg, addr, t_desc); + + tcg_temp_free_ptr(t_pg); + tcg_temp_free_i32(t_desc); + + /* + * Replicate that first octaword. + * The replication happens in units of 32; if the full vector size + * is not a multiple of 32, the final bits are zeroed. + */ + doff = vec_full_reg_offset(s, zt); + vsz_r32 = QEMU_ALIGN_DOWN(vsz, 32); + if (vsz >= 64) { + tcg_gen_gvec_dup_mem(5, doff + 32, doff, vsz_r32 - 32, vsz - 32); + } else if (vsz > vsz_r32) { + /* Nop move, with side effect of clearing the tail. */ + tcg_gen_gvec_mov(MO_64, doff, doff, vsz_r32, vsz); + } +} + +static bool trans_LD1RO_zprr(DisasContext *s, arg_rprr_load *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + if (a->rm == 31) { + return false; + } + if (sve_access_check(s)) { + TCGv_i64 addr = new_tmp_a64(s); + tcg_gen_shli_i64(addr, cpu_reg(s, a->rm), dtype_msz(a->dtype)); + tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, a->rn)); + do_ldro(s, a->rd, a->pg, addr, a->dtype); + } + return true; +} + +static bool trans_LD1RO_zpri(DisasContext *s, arg_rpri_load *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + if (sve_access_check(s)) { + TCGv_i64 addr = new_tmp_a64(s); + tcg_gen_addi_i64(addr, cpu_reg_sp(s, a->rn), a->imm * 32); + do_ldro(s, a->rd, a->pg, addr, a->dtype); + } + return true; +} + /* Load and broadcast element. */ static bool trans_LD1R_zpri(DisasContext *s, arg_rpri_load *a) { From patchwork Tue Mar 9 16:20:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396071 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2818527jai; Tue, 9 Mar 2021 10:27:04 -0800 (PST) X-Google-Smtp-Source: ABdhPJwfygfC0ycbJN7ESpigLHkMyirJHY+BcogJNNDBgcFSYufbgRqoCGKvufqgQ3cyFJKJ8zv3 X-Received: by 2002:a6b:6e06:: with SMTP id d6mr23607043ioh.116.1615314424660; Tue, 09 Mar 2021 10:27:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314424; cv=none; d=google.com; s=arc-20160816; b=PDf6RUY6pNkBHdZqcvwNZnH+XfFMdpKgVUuJjos1k9n8RMlFe/YfgsdkTsXN893oyK wXaGWHzl9YxHo7TAHtYypBlngTMZyEo4OnmKAMfUnw7KlOFOLPKfU4x6zc1xwCU5qeqY yJXLaTAr+11eHmErt3RTWK/QXeyEr0SZAnrpzZRp76tA3j+VsK7DYFSpjTK8p7bwDL0D VV/EbZqTYrzYH0SadmhdRTumLgJQIGgIi+qDKZ5EmS1QZxtom+vnKLbdqVX8UiYtfln3 alZGN3CRW/1ddzxeB8W4iBUX67byOdV67+xc8rMheJiuqimt63rLyJ/rm1/goXr0Y6KB T5EA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=p0cZaouKijzDmN1CVclrNUq7YJSRe7kJ4eRsMD3tCro=; b=SZxKW2/GlBukgWYxGx/Qew8rJg1DaLMU3Ph6FG9Ajo4r1iU7P3OgdNCX0VeX2+HHAI GZp1wJNfej58q7ahXrGJEZF7MqG6gwiYhRRPAlN9YKRe6A8tF0LJP4yHiysG0UXn6IJS E0H/DgXLI6+BE4XDl3NugLy5qe0MKeqkOrR2mPGBlpotUFuycrkaq1xIMLVEDYVXODbv nsyBjWEaRR6SqpTW+uays+4ETP64Of1P6DVQ3IrDfTHMhPtjdRmut8F3Dl/W6X7KqaxE ttZueG3AxEKXsAzXaRZ3tYQ9y+EpWqyDHuB3tLUufmjLPtAK+tDXh6cXxaKTLeBigzaZ T/Rg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GYF+IQOi; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m2si14917034ilg.69.2021.03.09.10.27.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:27:04 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GYF+IQOi; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60568 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJh4R-0007xR-QE for patch@linaro.org; Tue, 09 Mar 2021 13:27:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47552) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8M-0000jb-Il for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:59 -0500 Received: from mail-oi1-x22f.google.com ([2607:f8b0:4864:20::22f]:44683) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf84-0002sj-Ug for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:58 -0500 Received: by mail-oi1-x22f.google.com with SMTP id w195so8897118oif.11 for ; Tue, 09 Mar 2021 08:22:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=p0cZaouKijzDmN1CVclrNUq7YJSRe7kJ4eRsMD3tCro=; b=GYF+IQOiNcgJzYqdAkVK2FsyoXyuF2yXsYS2vqL9n1TLWdSMKUW6idYsbkc1O9fBcd M8fTGF9wR1NhHISpz46SOOs429Quv9W4ETP+MQZAZt2efz4JwmRUvxbe3tFhVikl+vHu dqjxC6s36SfOpx56ZgvOyh5vWzCHykO/xUE92OIpuGtzOd3fX7VLGikIFHZhaWRS40o5 8RleSfgkUW8BHQR3szVQmczcxQ7wiuflmGtQPSsPycI7KgxUFHuiBOthVUDqlQNVytuE HHv48C8QMLOEos05Twoea7l9cwBiUk6mhSGPO5O0qNmlnqw/vswT5LcJcay63tkJ2bFW +Raw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p0cZaouKijzDmN1CVclrNUq7YJSRe7kJ4eRsMD3tCro=; b=cBjCsUZXGMdoTD5GEyArNNWOLEF6LwL/w8oDuGF8E5dK3VjcE6aWZK2mh2+7Oys2Cc MBJ8pMOOr0tPrAYZKd4yCVWevtJdfw9aUh6ZcX+FwJqD3HSWOijnojWo7cMjVaxkuFIy 8pmgJDZLRd2OMfeH6NgQTjkB2f27KR8PijRa8eLN5eza1FgfhHMFUhnv+Lxhn81hu+uY jH+Nm7cyltgM6uLo4sS2IkcDsszbGiUpddMBI1KKPuqHcsk0zCsfD9pidxrOntHmHnxr lMt8QKmcDDDMUyiuE5XXzVnLLhEskCkCbxYQe2C0YaucTiMUBcmcmW+8Hlnorrrqg5TQ FIXg== X-Gm-Message-State: AOAM532nvc6MZaFyDOQer13z5osuAbOMoZho2HuPrFCjI5c88Mrr0h6Z nJb/QKnMTIeIK5xx+LxKEYMZ8VvUnsY33Okh X-Received: by 2002:aca:62c1:: with SMTP id w184mr3376965oib.177.1615306957797; Tue, 09 Mar 2021 08:22:37 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:37 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 74/78] target/arm: Implement 128-bit ZIP, UZP, TRN Date: Tue, 9 Mar 2021 08:20:37 -0800 Message-Id: <20210309162041.23124-75-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22f; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 3 ++ target/arm/sve.decode | 8 ++++++ target/arm/sve_helper.c | 29 +++++++++++++------ target/arm/translate-sve.c | 58 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 90 insertions(+), 8 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 0a0ee8587b..ddda7f35ed 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -689,16 +689,19 @@ DEF_HELPER_FLAGS_4(sve_zip_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_zip_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_zip_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_zip_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_zip_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uzp_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_trn_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_compact_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_compact_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index f3b405b741..78452d416d 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -592,6 +592,14 @@ UZP2_z 00000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm TRN1_z 00000101 .. 1 ..... 011 100 ..... ..... @rd_rn_rm TRN2_z 00000101 .. 1 ..... 011 101 ..... ..... @rd_rn_rm +# SVE2 permute vector segments +ZIP1_q 00000101 10 1 ..... 000 000 ..... ..... @rd_rn_rm_e0 +ZIP2_q 00000101 10 1 ..... 000 001 ..... ..... @rd_rn_rm_e0 +UZP1_q 00000101 10 1 ..... 000 010 ..... ..... @rd_rn_rm_e0 +UZP2_q 00000101 10 1 ..... 000 011 ..... ..... @rd_rn_rm_e0 +TRN1_q 00000101 10 1 ..... 000 110 ..... ..... @rd_rn_rm_e0 +TRN2_q 00000101 10 1 ..... 000 111 ..... ..... @rd_rn_rm_e0 + ### SVE Permute - Predicated Group # SVE compress active elements diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 1e7c9c4f66..d879ffce9e 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3458,36 +3458,45 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ *(TYPE *)(vd + H(2 * i + 0)) = *(TYPE *)(vn + H(i)); \ *(TYPE *)(vd + H(2 * i + sizeof(TYPE))) = *(TYPE *)(vm + H(i)); \ } \ + if (sizeof(TYPE) == 16 && unlikely(oprsz & 16)) { \ + memset(vd + oprsz - 16, 0, 16); \ + } \ } DO_ZIP(sve_zip_b, uint8_t, H1) DO_ZIP(sve_zip_h, uint16_t, H1_2) DO_ZIP(sve_zip_s, uint32_t, H1_4) DO_ZIP(sve_zip_d, uint64_t, ) +DO_ZIP(sve2_zip_q, Int128, ) #define DO_UZP(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ intptr_t oprsz = simd_oprsz(desc); \ - intptr_t oprsz_2 = oprsz / 2; \ intptr_t odd_ofs = simd_data(desc); \ - intptr_t i; \ + intptr_t i, p; \ ARMVectorReg tmp_m; \ if (unlikely((vm - vd) < (uintptr_t)oprsz)) { \ vm = memcpy(&tmp_m, vm, oprsz); \ } \ - for (i = 0; i < oprsz_2; i += sizeof(TYPE)) { \ - *(TYPE *)(vd + H(i)) = *(TYPE *)(vn + H(2 * i + odd_ofs)); \ - } \ - for (i = 0; i < oprsz_2; i += sizeof(TYPE)) { \ - *(TYPE *)(vd + H(oprsz_2 + i)) = *(TYPE *)(vm + H(2 * i + odd_ofs)); \ - } \ + i = 0, p = odd_ofs; \ + do { \ + *(TYPE *)(vd + H(i)) = *(TYPE *)(vn + H(p)); \ + i += sizeof(TYPE), p += 2 * sizeof(TYPE); \ + } while (p < oprsz); \ + p -= oprsz; \ + do { \ + *(TYPE *)(vd + H(i)) = *(TYPE *)(vm + H(p)); \ + i += sizeof(TYPE), p += 2 * sizeof(TYPE); \ + } while (p < oprsz); \ + tcg_debug_assert(i == oprsz); \ } DO_UZP(sve_uzp_b, uint8_t, H1) DO_UZP(sve_uzp_h, uint16_t, H1_2) DO_UZP(sve_uzp_s, uint32_t, H1_4) DO_UZP(sve_uzp_d, uint64_t, ) +DO_UZP(sve2_uzp_q, Int128, ) #define DO_TRN(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ @@ -3501,12 +3510,16 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ *(TYPE *)(vd + H(i + 0)) = ae; \ *(TYPE *)(vd + H(i + sizeof(TYPE))) = be; \ } \ + if (sizeof(TYPE) == 16 && unlikely(oprsz & 16)) { \ + memset(vd + oprsz - 16, 0, 16); \ + } \ } DO_TRN(sve_trn_b, uint8_t, H1) DO_TRN(sve_trn_h, uint16_t, H1_2) DO_TRN(sve_trn_s, uint32_t, H1_4) DO_TRN(sve_trn_d, uint64_t, ) +DO_TRN(sve2_trn_q, Int128, ) #undef DO_ZIP #undef DO_UZP diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 58bebbfedd..ef2ca6a0b6 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -2624,6 +2624,32 @@ static bool trans_ZIP2_z(DisasContext *s, arg_rrr_esz *a) return do_zip(s, a, true); } +static bool do_zip_q(DisasContext *s, arg_rrr_esz *a, bool high) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0; + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn) + high_ofs, + vec_full_reg_offset(s, a->rm) + high_ofs, + vsz, vsz, 0, gen_helper_sve2_zip_q); + } + return true; +} + +static bool trans_ZIP1_q(DisasContext *s, arg_rrr_esz *a) +{ + return do_zip_q(s, a, false); +} + +static bool trans_ZIP2_q(DisasContext *s, arg_rrr_esz *a) +{ + return do_zip_q(s, a, true); +} + static gen_helper_gvec_3 * const uzp_fns[4] = { gen_helper_sve_uzp_b, gen_helper_sve_uzp_h, gen_helper_sve_uzp_s, gen_helper_sve_uzp_d, @@ -2639,6 +2665,22 @@ static bool trans_UZP2_z(DisasContext *s, arg_rrr_esz *a) return do_zzz_data_ool(s, a, 1 << a->esz, uzp_fns[a->esz]); } +static bool trans_UZP1_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 0, gen_helper_sve2_uzp_q); +} + +static bool trans_UZP2_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 16, gen_helper_sve2_uzp_q); +} + static gen_helper_gvec_3 * const trn_fns[4] = { gen_helper_sve_trn_b, gen_helper_sve_trn_h, gen_helper_sve_trn_s, gen_helper_sve_trn_d, @@ -2654,6 +2696,22 @@ static bool trans_TRN2_z(DisasContext *s, arg_rrr_esz *a) return do_zzz_data_ool(s, a, 1 << a->esz, trn_fns[a->esz]); } +static bool trans_TRN1_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 0, gen_helper_sve2_trn_q); +} + +static bool trans_TRN2_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 16, gen_helper_sve2_trn_q); +} + /* *** SVE Permute Vector - Predicated Group */ From patchwork Tue Mar 9 16:20:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396072 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2819529jai; Tue, 9 Mar 2021 10:28:35 -0800 (PST) X-Google-Smtp-Source: ABdhPJwqS9m3jPFAdYKoalhnbbiru8gVcsVPR122SNtaT/HdZ8u6yiNerLfUlYwOTpoo3hCr0qdP X-Received: by 2002:a05:6602:2f0c:: with SMTP id q12mr23340813iow.82.1615314515243; Tue, 09 Mar 2021 10:28:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314515; cv=none; d=google.com; s=arc-20160816; b=e1Luf8/TZ70D5h6w9y4SO/F83jHA4hpad6J7Pp+pPKaT0tr50AcuTHY922YIQzN8/1 vwwIVkiUE7i0uESUYC90OcF9v3Aznfbx66jdbsA6R4ldJqFcTVGUds3IYOTIWpvk9bTH hGrYwiftMLvszpuKGV8mfliKL6N20BIHJDYFldF6APqovszubkumbfD9s4FY4mA4Sj2q OoOt/LUOnFebxmOfwpUIRZ0/AqHaE4yAmheXDQ20MU5A5JfiYOKfkt1wmVcZsjdZFw/G uT4Kfs43rjTjeLCtUWQj7NVRiFSjFiyLd3KuI6cIpzJGhTzrv1dNkwGM0+W9y2moTne9 G8HA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=LvTrM/eXKhblOjuZfTaVJavnaVQDrBaCdov+m5Ya39I=; b=l1+BeCQ39MWJzBz/ZXDp9V0I0L3bwfIyOZBI2GrvNM2xJoK7SCsH26gqVa3I1sUweU Q0EMwwcoSAqMN9FZzOPIJMlLKvgklzyhdZIPlLV6Wx8nuslcirOROlF6ezqt5xAyl8QN QE5XOS3XxhOx5yrHptuJbWaEKHk1vK4TIR47bNhvduz7WvDCrLCtVd/NRa4K67fIsSnb 1L7rDZ6571aVpkJn8NP+K6VGqfzuL8Ic0jK7Wpu11cx0rElzSdBx2Rf4I5n110Anx5IJ 4H/6HlFPsgCWT3aKGKrivtHEgCjC5NMCT1HDGTsfeZJLqzJgMcKE7HwrwFX3JhUiyMFi X+yQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wTqbY1oC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e11si13374252ilu.32.2021.03.09.10.28.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:28:35 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wTqbY1oC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35178 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJh5u-0000gw-Kr for patch@linaro.org; Tue, 09 Mar 2021 13:28:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47564) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8N-0000jc-JN for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:23:00 -0500 Received: from mail-ot1-x331.google.com ([2607:f8b0:4864:20::331]:44766) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf84-0002t0-V5 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:22:59 -0500 Received: by mail-ot1-x331.google.com with SMTP id f33so13311868otf.11 for ; Tue, 09 Mar 2021 08:22:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LvTrM/eXKhblOjuZfTaVJavnaVQDrBaCdov+m5Ya39I=; b=wTqbY1oCNnPeKD7rceZHto++6npcutrPKfKgEq9223TW5NSPOt2VXCn/532Kf8zVSR h9LmKE5Z6ACm8DD4ya/QY/eTXuaVKiAmTI4sOuI2gr651tmHDmKftZQLEEHJK/b702g/ rFz/mzJLl2mJ5mXbLdVvQ4SbmbsjcKhQ92ebvr0cIomffHcnHbt2O5qa7tKUlYxS88lv tfv84S66ZlNzCFIUJFz6v6Y1ZXX+8IGwwyFVrtAZsXybB0h8eMYnYAI49MH4hhpYvfOn /Q96xxDQuO8xD8ZIcmxBdqzXkzHyWF8QCbBArX3XVk+glKLnShzkMgdewdaclN6HZT7e xEkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LvTrM/eXKhblOjuZfTaVJavnaVQDrBaCdov+m5Ya39I=; b=l9TCaTT8WyYcn/lFG28vkE2PwfjAEucRLnXLR0EuxMqb9Xd+c7BxnU0qPnYD95P3aK fJWFMaauf3yS2Yf+QStEj4MUCjfco/vDL/vvrBCWbS2nZ/Y/zE3E4rsqU6i1FCcW67x/ uv06yF0y0c7aL6QlDH4+Jt7dyXLSKKIRROXc7H4D8PqRE8UJgvt+1iVJRlwkXlXkZ3mu yAGYwZylK32cI66+OzVMnuhdIDcVhO7PUb+X1oHw77rU7SBdK81BfD9DJXIsmALsp4Sy WgQth/cIFao1NYLNtIUVCxMObDVaP/Ps4UodAd5Hs4HcAATaeLmoyhF90IlEz2P2WcVn MQ4A== X-Gm-Message-State: AOAM533MPPKl7GGTDgc8heC6A96Rn8DJbC+cQsz4y+TDNBAtBPKB62Rs ClRup+bwmThaLXa1yPMTb8NdE0Mwd3vN7I3e X-Received: by 2002:a05:6830:1da1:: with SMTP id z1mr9808514oti.64.1615306959262; Tue, 09 Mar 2021 08:22:39 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:38 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 75/78] target/arm: Implement SVE2 bitwise shift immediate Date: Tue, 9 Mar 2021 08:20:38 -0800 Message-Id: <20210309162041.23124-76-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::331; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x331.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Implements SQSHL/UQSHL, SRSHR/URSHR, and SQSHLU Signed-off-by: Stephen Long Message-Id: <20200430194159.24064-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 33 +++++++++++++++++++++ target/arm/sve.decode | 5 ++++ target/arm/sve_helper.c | 35 ++++++++++++++++++++++ target/arm/translate-sve.c | 60 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 133 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index ddda7f35ed..43ee4238de 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2738,6 +2738,39 @@ DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_idx_h, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_srshr_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_srshr_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_srshr_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_srshr_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_urshr_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_urshr_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_urshr_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_urshr_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqshlu_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshlu_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshlu_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshlu_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_fcvtnt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_fcvtnt_ds, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 78452d416d..f2a66a159a 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -342,6 +342,11 @@ ASR_zpzi 00000100 .. 000 000 100 ... .. ... ..... @rdn_pg_tszimm_shr LSR_zpzi 00000100 .. 000 001 100 ... .. ... ..... @rdn_pg_tszimm_shr LSL_zpzi 00000100 .. 000 011 100 ... .. ... ..... @rdn_pg_tszimm_shl ASRD 00000100 .. 000 100 100 ... .. ... ..... @rdn_pg_tszimm_shr +SQSHL_zpzi 00000100 .. 000 110 100 ... .. ... ..... @rdn_pg_tszimm_shl +UQSHL_zpzi 00000100 .. 000 111 100 ... .. ... ..... @rdn_pg_tszimm_shl +SRSHR 00000100 .. 001 100 100 ... .. ... ..... @rdn_pg_tszimm_shr +URSHR 00000100 .. 001 101 100 ... .. ... ..... @rdn_pg_tszimm_shr +SQSHLU 00000100 .. 001 111 100 ... .. ... ..... @rdn_pg_tszimm_shl # SVE bitwise shift by vector (predicated) ASR_zpzz 00000100 .. 010 000 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index d879ffce9e..d08da9fdc1 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2204,6 +2204,41 @@ DO_ZPZI(sve_asrd_h, int16_t, H1_2, DO_ASRD) DO_ZPZI(sve_asrd_s, int32_t, H1_4, DO_ASRD) DO_ZPZI_D(sve_asrd_d, int64_t, DO_ASRD) +/* SVE2 bitwise shift by immediate */ +DO_ZPZI(sve2_sqshl_zpzi_b, int8_t, H1, do_sqshl_b) +DO_ZPZI(sve2_sqshl_zpzi_h, int16_t, H1_2, do_sqshl_h) +DO_ZPZI(sve2_sqshl_zpzi_s, int32_t, H1_4, do_sqshl_s) +DO_ZPZI_D(sve2_sqshl_zpzi_d, int64_t, do_sqshl_d) + +DO_ZPZI(sve2_uqshl_zpzi_b, uint8_t, H1, do_uqshl_b) +DO_ZPZI(sve2_uqshl_zpzi_h, uint16_t, H1_2, do_uqshl_h) +DO_ZPZI(sve2_uqshl_zpzi_s, uint32_t, H1_4, do_uqshl_s) +DO_ZPZI_D(sve2_uqshl_zpzi_d, uint64_t, do_uqshl_d) + +DO_ZPZI(sve2_srshr_b, int8_t, H1, do_srshr) +DO_ZPZI(sve2_srshr_h, int16_t, H1_2, do_srshr) +DO_ZPZI(sve2_srshr_s, int32_t, H1_4, do_srshr) +DO_ZPZI_D(sve2_srshr_d, int64_t, do_srshr) + +DO_ZPZI(sve2_urshr_b, uint8_t, H1, do_urshr) +DO_ZPZI(sve2_urshr_h, uint16_t, H1_2, do_urshr) +DO_ZPZI(sve2_urshr_s, uint32_t, H1_4, do_urshr) +DO_ZPZI_D(sve2_urshr_d, uint64_t, do_urshr) + +#define do_suqrshl_b(n, m) \ + ({ uint32_t discard; do_suqrshl_bhs(n, (int8_t)m, 8, false, &discard); }) +#define do_suqrshl_h(n, m) \ + ({ uint32_t discard; do_suqrshl_bhs(n, (int16_t)m, 16, false, &discard); }) +#define do_suqrshl_s(n, m) \ + ({ uint32_t discard; do_suqrshl_bhs(n, m, 32, false, &discard); }) +#define do_suqrshl_d(n, m) \ + ({ uint32_t discard; do_suqrshl_d(n, m, false, &discard); }) + +DO_ZPZI(sve2_sqshlu_b, int8_t, H1, do_suqrshl_b) +DO_ZPZI(sve2_sqshlu_h, int16_t, H1_2, do_suqrshl_h) +DO_ZPZI(sve2_sqshlu_s, int32_t, H1_4, do_suqrshl_s) +DO_ZPZI_D(sve2_sqshlu_d, int64_t, do_suqrshl_d) + #undef DO_ASRD #undef DO_ZPZI #undef DO_ZPZI_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ef2ca6a0b6..4002bc5ed4 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -1044,6 +1044,66 @@ static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a) } } +static bool trans_SQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqshl_zpzi_b, gen_helper_sve2_sqshl_zpzi_h, + gen_helper_sve2_sqshl_zpzi_s, gen_helper_sve2_sqshl_zpzi_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_UQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_uqshl_zpzi_b, gen_helper_sve2_uqshl_zpzi_h, + gen_helper_sve2_uqshl_zpzi_s, gen_helper_sve2_uqshl_zpzi_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_SRSHR(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_srshr_b, gen_helper_sve2_srshr_h, + gen_helper_sve2_srshr_s, gen_helper_sve2_srshr_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_URSHR(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_urshr_b, gen_helper_sve2_urshr_h, + gen_helper_sve2_urshr_s, gen_helper_sve2_urshr_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_SQSHLU(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqshlu_b, gen_helper_sve2_sqshlu_h, + gen_helper_sve2_sqshlu_s, gen_helper_sve2_sqshlu_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + /* *** SVE Bitwise Shift - Predicated Group */ From patchwork Tue Mar 9 16:20:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396075 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2821717jai; Tue, 9 Mar 2021 10:31:44 -0800 (PST) X-Google-Smtp-Source: ABdhPJyliPm1AA2vJFr7gAx8OyCFT7FnkY8Ev5YWfaN9T+yCmQThrw/vU08TqzTwapzMLMDGzYS1 X-Received: by 2002:a05:6638:406:: with SMTP id q6mr30741577jap.126.1615314704403; Tue, 09 Mar 2021 10:31:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314704; cv=none; d=google.com; s=arc-20160816; b=zVhqaCswnCZYnbj+eI+3dpEF0zdegu30UKFmElrE7IUx55UjmeXYQTbzZBsldwWh8y TNoMIILLLXVIczDVSa5f+4heE3oxPRjv780xDOymGySy5Nob1Cv17HzbiNX33mT6GeDy DZm3KhzyRwHfATq4IS0OOplviMxKInWnCZZQh7AYrJLLmbQXJqerul5sPm2UZpgjwJw2 ZPBK7MDH3rEYgMdlOgxA4wsP/3IPIaWNvLsFuZdHHENiiJVm8nrrlUE68E1funSKaa2i gHHofzNzi9cDhI6WVB50FYjPOuG12gsgycc3m+C3s/AZchEeF4uFPbU3D0bQmbAArHas ncdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=uNObzhk2VNVhoLK83yWNuS3fwlbTMuGpnWo81q3J43E=; b=nFDGFT0RZ8A+Vq9f5+Ob45KvQ2yKZVQ4ZpZrjS03wuWVIRpyx51dGkRkK1G/vw5YH9 8zv+6npomVlSjvAMtZAja7+1oa9/zNElDB5N5SxnKBdP9H2FRhgl2QLjzxNci3HS2qUh OJ8KH51MNmxAg0nMwnPr3ur125LxCK1jKneUrCWP1lE6wznxoRgrzGFFvq4Br9li9FCc +n4Qqz30c7bEpBCiRH7hP5TTrB0qeEFLpMBSs/5XYzysGlo9zyF3j6n1tzHfp9tkpVFl 2gMe3dta5gatDzBXK+ARASqkajYSCTCpHA//29ksa7phoYI3ghPLgAdGLMw7Q0k+q4DM BsjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hjX1lJBR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h23si14209026ila.44.2021.03.09.10.31.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:31:44 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hjX1lJBR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42150 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJh8x-0003gH-P0 for patch@linaro.org; Tue, 09 Mar 2021 13:31:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47578) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8O-0000kB-D8 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:23:01 -0500 Received: from mail-oi1-x232.google.com ([2607:f8b0:4864:20::232]:41380) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf86-0002tI-Ng for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:23:00 -0500 Received: by mail-oi1-x232.google.com with SMTP id y131so12660185oia.8 for ; Tue, 09 Mar 2021 08:22:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uNObzhk2VNVhoLK83yWNuS3fwlbTMuGpnWo81q3J43E=; b=hjX1lJBRj85ajk/9oVOo6QL3G+U1zwzG7Ouz5SnjU7Wxfx6kKqM/GXaZPYVIumXrdq bqdffZbkrHDCdNf6GCCoR8lfvIYRt6G/oRj8KLNPKKe/rymjKV1wbizc9pYVMzGthrzb njKrrOdvSPjcp/FKJKTATZX7qltZeyYUz0m6hyAC3OdBNnusV2Yg2aTWSFQhSx4fdhmh a/POpkqppQ8fuLg00sLHItFwyIhBV/ju3LyXTjkqfkJIvgFAl0+DIP6txzN/NBLv+kXm Pt4gp3fxB73VdZTQtxe9D7qkm+KE5yonvYKV8mOrSPFiDEEOZYnJ6DzA7cSeGYSeGJ0E qBpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uNObzhk2VNVhoLK83yWNuS3fwlbTMuGpnWo81q3J43E=; b=HVSpzI8AoXUhgSHxTwX8u6AmwE+GwbwtelLze4X8ZjaHH2RXiIeyfMeOAbUUFbFKAw 1vJFp+dS2xtHwmWWtkeGGIDkcEmc53up2My9n8aw9MGGE6//RXKxH8XsfKUwjFyMR9Ci E74hjIFBifFM9LWwst2cyIxF5VXdgflj5KoP5W04m/w31OoDj5+4pHFJM1mDDAgEJMep +yL0gVEiOqyUrUS50mhoFYrfFsixXjDoGCU9Tdfge4UN2KMm3WYi3Lyk93ieSccQfy1l wVCFIpiU2zWT6hhBdWyB/nsKQHr8qCN772AGT5Ahauz/fPfaYGBCwLuVjrs3Lziak7ib dM8A== X-Gm-Message-State: AOAM533lAOVULVFEGpJq8Fb/ZI0QsTAY7Z7Ra7ETtENn2cFxTWvcdc9o n5z7a56/aVTfSQ95SKLmJYhh4CnQcd9/2rXl X-Received: by 2002:aca:cfd3:: with SMTP id f202mr3470893oig.155.1615306960747; Tue, 09 Mar 2021 08:22:40 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:40 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 76/78] target/arm: Implement SVE2 fp multiply-add long Date: Tue, 9 Mar 2021 08:20:39 -0800 Message-Id: <20210309162041.23124-77-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::232; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x232.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Implements both vectored and indexed FMLALB, FMLALT, FMLSLB, FMLSLT Signed-off-by: Stephen Long Message-Id: <20200504171240.11220-1-steplong@quicinc.com> [rth: Rearrange to use float16_to_float32_by_bits.] Signed-off-by: Richard Henderson --- target/arm/helper.h | 5 +++ target/arm/sve.decode | 12 ++++++ target/arm/translate-sve.c | 75 ++++++++++++++++++++++++++++++++++++++ target/arm/vec_helper.c | 51 ++++++++++++++++++++++++++ 4 files changed, 143 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 86f938c938..e8b16a401f 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -987,6 +987,11 @@ DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmlal_zzxw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) #ifdef TARGET_AARCH64 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index f2a66a159a..b5c53b4c40 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1601,3 +1601,15 @@ FCVTLT_sd 01100100 11 0010 11 101 ... ..... ..... @rd_pg_rn_e0 ### SVE2 floating-point convert to integer FLOGB 01100101 00 011 esz:2 0101 pg:3 rn:5 rd:5 &rpr_esz + +### SVE2 floating-point multiply-add long (vectors) +FMLALB_zzzw 01100100 .. 1 ..... 10 0 00 0 ..... ..... @rda_rn_rm +FMLALT_zzzw 01100100 .. 1 ..... 10 0 00 1 ..... ..... @rda_rn_rm +FMLSLB_zzzw 01100100 .. 1 ..... 10 1 00 0 ..... ..... @rda_rn_rm +FMLSLT_zzzw 01100100 .. 1 ..... 10 1 00 1 ..... ..... @rda_rn_rm + +### SVE2 floating-point multiply-add long (indexed) +FMLALB_zzxw 01100100 .. 1 ..... 0100.0 ..... ..... @rrxw_s +FMLALT_zzxw 01100100 .. 1 ..... 0100.1 ..... ..... @rrxw_s +FMLSLB_zzxw 01100100 .. 1 ..... 0110.0 ..... ..... @rrxw_s +FMLSLT_zzxw 01100100 .. 1 ..... 0110.1 ..... ..... @rrxw_s diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 4002bc5ed4..af2c6fbb9e 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8519,3 +8519,78 @@ static bool trans_FLOGB(DisasContext *s, arg_rpr_esz *a) } return true; } + +static bool do_FMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sub, bool sel) +{ + if (a->esz != MO_32 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + cpu_env, vsz, vsz, (sel << 1) | sub, + gen_helper_sve2_fmlal_zzzw_s); + } + return true; +} + +static bool trans_FMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, false, false); +} + +static bool trans_FMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, false, true); +} + +static bool trans_FMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, true, false); +} + +static bool trans_FMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, true, true); +} + +static bool do_FMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sub, bool sel) +{ + if (a->esz != MO_32 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + cpu_env, vsz, vsz, + (a->index << 2) | (sel << 1) | sub, + gen_helper_sve2_fmlal_zzxw_s); + } + return true; +} + +static bool trans_FMLALB_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, false, false); +} + +static bool trans_FMLALT_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, false, true); +} + +static bool trans_FMLSLB_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, true, false); +} + +static bool trans_FMLSLT_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, true, true); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 9b2a4d5b7e..ea08b15c55 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -29,10 +29,14 @@ so addressing units smaller than that needs a host-endian fixup. */ #ifdef HOST_WORDS_BIGENDIAN #define H1(x) ((x) ^ 7) +#define H1_2(x) ((x) ^ 6) +#define H1_4(x) ((x) ^ 4) #define H2(x) ((x) ^ 3) #define H4(x) ((x) ^ 1) #else #define H1(x) (x) +#define H1_2(x) (x) +#define H1_4(x) (x) #define H2(x) (x) #define H4(x) (x) #endif @@ -1907,6 +1911,27 @@ void HELPER(gvec_fmlal_a64)(void *vd, void *vn, void *vm, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } +void HELPER(sve2_fmlal_zzzw_s)(void *vd, void *vn, void *vm, void *va, + void *venv, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + uint16_t negn = extract32(desc, SIMD_DATA_SHIFT, 1) << 15; + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(float16); + CPUARMState *env = venv; + float_status *status = &env->vfp.fp_status; + bool fz16 = get_flush_inputs_to_zero(&env->vfp.fp_status_f16); + + for (i = 0; i < oprsz; i += sizeof(float32)) { + float16 nn_16 = *(float16 *)(vn + H1_2(i + sel)) ^ negn; + float16 mm_16 = *(float16 *)(vm + H1_2(i + sel)); + float32 nn = float16_to_float32_by_bits(nn_16, fz16); + float32 mm = float16_to_float32_by_bits(mm_16, fz16); + float32 aa = *(float32 *)(va + H1_4(i)); + + *(float32 *)(vd + H1_4(i)) = float32_muladd(nn, mm, aa, 0, status); + } +} + static void do_fmlal_idx(float32 *d, void *vn, void *vm, float_status *fpst, uint32_t desc, bool fz16) { @@ -1951,6 +1976,32 @@ void HELPER(gvec_fmlal_idx_a64)(void *vd, void *vn, void *vm, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } +void HELPER(sve2_fmlal_zzxw_s)(void *vd, void *vn, void *vm, void *va, + void *venv, uint32_t desc) +{ + intptr_t i, j, oprsz = simd_oprsz(desc); + uint16_t negn = extract32(desc, SIMD_DATA_SHIFT, 1) << 15; + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(float16); + intptr_t idx = extract32(desc, SIMD_DATA_SHIFT + 2, 3) * sizeof(float16); + CPUARMState *env = venv; + float_status *status = &env->vfp.fp_status; + bool fz16 = get_flush_inputs_to_zero(&env->vfp.fp_status_f16); + + for (i = 0; i < oprsz; i += 16) { + float16 mm_16 = *(float16 *)(vm + i + idx); + float32 mm = float16_to_float32_by_bits(mm_16, fz16); + + for (j = 0; j < 16; j += sizeof(float32)) { + float16 nn_16 = *(float16 *)(vn + H1_2(i + j + sel)) ^ negn; + float32 nn = float16_to_float32_by_bits(nn_16, fz16); + float32 aa = *(float32 *)(va + H1_4(i + j)); + + *(float32 *)(vd + H1_4(i + j)) = + float32_muladd(nn, mm, aa, 0, status); + } + } +} + void HELPER(gvec_sshl_b)(void *vd, void *vn, void *vm, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); From patchwork Tue Mar 9 16:20:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396078 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2823487jai; Tue, 9 Mar 2021 10:34:31 -0800 (PST) X-Google-Smtp-Source: ABdhPJzC45gZnD9CY4hZIfx695bEXKv0JhPznecDT4hQe9qTvXF44gWgWNPaI2hegV0xRZ1mjobe X-Received: by 2002:a05:6e02:11a3:: with SMTP id 3mr24809572ilj.126.1615314871809; Tue, 09 Mar 2021 10:34:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615314871; cv=none; d=google.com; s=arc-20160816; b=NnpNceT88qICzlodY79VrdYYppQT5Mqwn72p4Pk4VmQvaKfvGzRNsGmj+gzgaAEmL+ 1ymxDdscAFYATkNBOvzrbSs/+oIW8NTKBN2dK8yTt4uq9AP8WO8RP8Bqj2s95qCCbwSj ukdrF76X/v7DOncqvQJ52qY1VGLEcY23qkrsZBx2R80iLHq3JWf2J7I9d2iyt7CpyEPy GlvV2TFI55sowhXjWHN9JYFSNEY9qXU8/HaGKCYbYborWr5cK56yyrN60uWdafCJo3xW Fn/EW6iB8oh77OFQ64560h5QewOCu9u3t+d+4fjhab6arq+enhSmOTdZXms80GiFt6S9 tImA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=WagiVrDfUgMksBOhokHdQsJkyD0LF4+ZrDTc70WJHXM=; b=ZoKZizZvtN6A4MHweSkmTT7lU0x+LbVr71ZlxValDHIs8sdpBTcOY8HM6RNph8qKry BzRgIBwnO7hqBynUHd/fH+6NNd2OG3x7ajKMPtigYZGJyXE2RENqkVRWB4NH83aQcBRw KJ+66qoOtkcEZvIpPAU8cjTxtRqRyOwQcK4i0kmjRnEZFWECDUXE4zs36p2cfOO5v5MA u/Z+OAhAl3mNr9Y/9kCWS5xPbvcuCNo7oC4C+7VlGHonDPBJVpj/65B6iAb3+zoDFssK /TmQEd3aC1941sNsj7kI1xZq11nTzbOZgd6iX0vSw7aC9cOtPpknjctJWD3YMomghfBd 0Xhw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PNVqX2QF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s17si607339jat.66.2021.03.09.10.34.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 10:34:31 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PNVqX2QF; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46292 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhBf-0005Pt-16 for patch@linaro.org; Tue, 09 Mar 2021 13:34:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47666) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8S-0000mq-IP for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:23:04 -0500 Received: from mail-oi1-x230.google.com ([2607:f8b0:4864:20::230]:33977) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf86-0002tU-R2 for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:23:04 -0500 Received: by mail-oi1-x230.google.com with SMTP id x78so15566466oix.1 for ; Tue, 09 Mar 2021 08:22:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WagiVrDfUgMksBOhokHdQsJkyD0LF4+ZrDTc70WJHXM=; b=PNVqX2QFLjutgz74hzJHUwhuoveZ0LoTQOQi/EHylwnRhgRKw9cqqSNHsyYfJRCKuz yiOAr4D1dPSucfUayf3xY9LTP3Oa1fRtsDgYemmkL0DdVUkb6/AtPTEyi1UAIxhHWKrk dJfP5q1tA/WpZKJMVVJ5HVNhWFD3l6xT3wxAdevi9m0GdpRZ6L18tcLlFjVGu//1yx/c KPh0UWvUc+WMjFiL+MqUN2jA/1P1zsfHcdSmKe8vQ3DIR68Av7OTwudKtKbAoBHk/kcx YIrpuDg38vSGJiSLvRWXKJJgcd5t314ufHw0+DYsb1MStTtsyym6u71i7UGlOo/UlnBX Aikw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WagiVrDfUgMksBOhokHdQsJkyD0LF4+ZrDTc70WJHXM=; b=dqvBeHtxgRUMSOyo9XyK4ocuMptGO8rd/BDelnIhXYiKnXxfUJJrIVmllGaW0xOayz Gp5ZOD46kAhYGFjQIzNsSZhJfe/cgiCsipolJVnNubbSVTQyTTOcgJt+vHZG1+OiufoK jr408CRpf7RaIRbfO/NT++rzKjcUFEg5DSkU/Fae6dufRFktB8pyXg9adS3J0k73y7J6 ++4Dz0ZMdZuXd92UU70aQeKMIWh1VeCF9c/8eqc/YrtXwwgxk2xJhfeW5AUyV4QPan02 95UZ7VwLVd9qHPUROkmA8XW7m3HkXFDZVUSilYeQdW7mHhrzspaPW19AbDWdLlBuYTsi lr8A== X-Gm-Message-State: AOAM530crL5lmtv0dh3kfjNTn0D2amBnRSeahDWn6SR2fA9dDFNo8adM JtptVK76kLZoPGuApSGW2cebTdtV4+iOdmXX X-Received: by 2002:aca:4e13:: with SMTP id c19mr3572899oib.40.1615306961818; Tue, 09 Mar 2021 08:22:41 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:41 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 77/78] target/arm: Implement SVE2 complex integer dot product Date: Tue, 9 Mar 2021 08:20:40 -0800 Message-Id: <20210309162041.23124-78-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::230; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x230.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 10 ++++ target/arm/sve.decode | 9 ++++ target/arm/sve_helper.c | 99 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 17 +++++++ 4 files changed, 135 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 43ee4238de..955d14b7ac 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2784,3 +2784,13 @@ DEF_HELPER_FLAGS_5(sve2_fcvtlt_sd, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(flogb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(flogb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(flogb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_cdot_zzzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cdot_zzzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_cdot_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cdot_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index b5c53b4c40..289a3576c3 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -811,6 +811,9 @@ MUL_zzi 00100101 .. 110 000 110 ........ ..... @rdn_i8s DOT_zzzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 \ ra=%reg_movprfx +# SVE2 complex dot product (vectors) +CDOT_zzzz 01000100 esz:2 0 rm:5 0001 rot:2 rn:5 rd:5 ra=%reg_movprfx + #### SVE Multiply - Indexed # SVE integer dot product (indexed) @@ -849,6 +852,12 @@ SQDMLSLB_zzxw_d 01000100 .. 1 ..... 0011.0 ..... ..... @rrxw_d SQDMLSLT_zzxw_s 01000100 .. 1 ..... 0011.1 ..... ..... @rrxw_s SQDMLSLT_zzxw_d 01000100 .. 1 ..... 0011.1 ..... ..... @rrxw_d +# SVE2 complex integer dot product (indexed) +CDOT_zzxw_s 01000100 10 1 index:2 rm:3 0100 rot:2 rn:5 rd:5 \ + ra=%reg_movprfx +CDOT_zzxw_d 01000100 11 1 index:1 rm:4 0100 rot:2 rn:5 rd:5 \ + ra=%reg_movprfx + # SVE2 complex integer multiply-add (indexed) CMLA_zzxz_h 01000100 10 1 index:2 rm:3 0110 rot:2 rn:5 rd:5 \ ra=%reg_movprfx diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index d08da9fdc1..0a583b9898 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1592,6 +1592,105 @@ void HELPER(sve2_sqrdcmlah_idx_s)(void *vd, void *vn, void *vm, do_cmla_idx_s(vd, vn, vm, va, desc, do_sqrdcmlah_s); } +/* Note N and M are 4 elements bundled into one unit. */ +static int32_t do_cdot_s(uint32_t n, uint32_t m, int32_t a, + int sel_a, int sel_b, int sub_i) +{ + for (int i = 0; i <= 1; i++) { + int32_t elt1_r = (int8_t)(n >> (16 * i)); + int32_t elt1_i = (int8_t)(n >> (16 * i + 8)); + int32_t elt2_a = (int8_t)(m >> (16 * i + 8 * sel_a)); + int32_t elt2_b = (int8_t)(m >> (16 * i + 8 * sel_b)); + + a += elt1_r * elt2_a + elt1_i * elt2_b * sub_i; + } + return a; +} + +static int64_t do_cdot_d(uint64_t n, uint64_t m, int64_t a, + int sel_a, int sel_b, int sub_i) +{ + for (int i = 0; i <= 1; i++) { + int64_t elt1_r = (int16_t)(n >> (32 * i + 0)); + int64_t elt1_i = (int16_t)(n >> (32 * i + 16)); + int64_t elt2_a = (int16_t)(m >> (32 * i + 16 * sel_a)); + int64_t elt2_b = (int16_t)(m >> (32 * i + 16 * sel_b)); + + a += elt1_r * elt2_a + elt1_i * elt2_b * sub_i; + } + return a; +} + +void HELPER(sve2_cdot_zzzz_s)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + int opr_sz = simd_oprsz(desc); + int rot = simd_data(desc); + int sel_a = rot & 1; + int sel_b = sel_a ^ 1; + int sub_i = (rot == 0 || rot == 3 ? -1 : 1); + uint32_t *d = vd, *n = vn, *m = vm, *a = va; + + for (int e = 0; e < opr_sz / 4; e++) { + d[e] = do_cdot_s(n[e], m[e], a[e], sel_a, sel_b, sub_i); + } +} + +void HELPER(sve2_cdot_zzzz_d)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + int opr_sz = simd_oprsz(desc); + int rot = simd_data(desc); + int sel_a = rot & 1; + int sel_b = sel_a ^ 1; + int sub_i = (rot == 0 || rot == 3 ? -1 : 1); + uint64_t *d = vd, *n = vn, *m = vm, *a = va; + + for (int e = 0; e < opr_sz / 8; e++) { + d[e] = do_cdot_d(n[e], m[e], a[e], sel_a, sel_b, sub_i); + } +} + +void HELPER(sve2_cdot_idx_s)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + int opr_sz = simd_oprsz(desc); + int rot = extract32(desc, SIMD_DATA_SHIFT, 2); + int idx = H4(extract32(desc, SIMD_DATA_SHIFT + 2, 2)); + int sel_a = rot & 1; + int sel_b = sel_a ^ 1; + int sub_i = (rot == 0 || rot == 3 ? -1 : 1); + uint32_t *d = vd, *n = vn, *m = vm, *a = va; + + for (int seg = 0; seg < opr_sz / 4; seg += 4) { + uint32_t seg_m = m[seg + idx]; + for (int e = 0; e < 4; e++) { + d[seg + e] = do_cdot_s(n[seg + e], seg_m, a[seg + e], + sel_a, sel_b, sub_i); + } + } +} + +void HELPER(sve2_cdot_idx_d)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + int seg, opr_sz = simd_oprsz(desc); + int rot = extract32(desc, SIMD_DATA_SHIFT, 2); + int idx = extract32(desc, SIMD_DATA_SHIFT + 2, 2); + int sel_a = rot & 1; + int sel_b = sel_a ^ 1; + int sub_i = (rot == 0 || rot == 3 ? -1 : 1); + uint64_t *d = vd, *n = vn, *m = vm, *a = va; + + for (seg = 0; seg < opr_sz / 8; seg += 2) { + uint64_t seg_m = m[seg + idx]; + for (int e = 0; e < 2; e++) { + d[seg + e] = do_cdot_d(n[seg + e], seg_m, a[seg + e], + sel_a, sel_b, sub_i); + } + } +} + #define DO_ZZXZ(NAME, TYPE, H, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index af2c6fbb9e..3c44a0d182 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4147,6 +4147,9 @@ DO_SVE2_RRXR_ROT(CMLA_zzxz_s, gen_helper_sve2_cmla_idx_s) DO_SVE2_RRXR_ROT(SQRDCMLAH_zzxz_h, gen_helper_sve2_sqrdcmlah_idx_h) DO_SVE2_RRXR_ROT(SQRDCMLAH_zzxz_s, gen_helper_sve2_sqrdcmlah_idx_s) +DO_SVE2_RRXR_ROT(CDOT_zzxw_s, gen_helper_sve2_cdot_idx_s) +DO_SVE2_RRXR_ROT(CDOT_zzxw_d, gen_helper_sve2_cdot_idx_d) + #undef DO_SVE2_RRXR_ROT /* @@ -8351,6 +8354,20 @@ static bool trans_CMLA_zzzz(DisasContext *s, arg_CMLA_zzzz *a) return true; } +static bool trans_CDOT_zzzz(DisasContext *s, arg_CMLA_zzzz *a) +{ + if (!dc_isar_feature(aa64_sve2, s) || a->esz < MO_32) { + return false; + } + if (sve_access_check(s)) { + gen_helper_gvec_4 *fn = (a->esz == MO_32 + ? gen_helper_sve2_cdot_zzzz_s + : gen_helper_sve2_cdot_zzzz_d); + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->rot); + } + return true; +} + static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a) { static gen_helper_gvec_4 * const fns[] = { From patchwork Tue Mar 9 16:20:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 396088 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp2845352jai; Tue, 9 Mar 2021 11:05:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJwAvMu2K+KruEcMoDEHWjYRxn1N5OhRrGA+KEJkrTEqYdHORK0AciII8+U0hHnfy6jfzMdX X-Received: by 2002:a5e:8c16:: with SMTP id n22mr24107840ioj.156.1615316757689; Tue, 09 Mar 2021 11:05:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615316757; cv=none; d=google.com; s=arc-20160816; b=HFkxVPR8F8WpIKPK716GGx2Ie+RTrgGbSpLbOXovIkW9KHFqlvzoRxI4YzGuW/e/g/ SyeqHM8GgUbjEyOzoe9A5R5dj9t1/h26nOh+YBv5ejhFyqVsbqqyYJFPzZEh3G329Of9 rlwlGHCBEDSY8SMAUTNpZs/LJA2AUq+pLFTX59P91kWfCnLy9nPgiAgSb9cLlT59qCkC +ISlpWVRppubPqlWyNPst4fH+Ore+4Jp5FDqjiAj/FPwo+BxQXANT4bHPZA2v7nk3BzQ GxQe/YvJhQbUREhvlQoxpKKRNGr62FhT6kfRjZZn8UuLG6j7vFKrmnk0GFmiaGvTo9s7 gUNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=KA40eVLdhdRfN2JR4MJ3Si1YCqVpEUGM3POPrQ47iFo=; b=qMfMTUlZNJQ6v4U7SNoEltKvzd/behKAYOzflyMp0WEG296Ijat36DF17mC8H4md5R DlWg+vRCdhrcIg/DUhXqARxfJWHCPAinsPrDmwmvSywW9wOaoo2eUcLBDfmFvyGq5z8E nMrKJHXcAOFtTzJeSc+qNpgcxYTvIJdQG4XVlpDz+HbJzBlWYzfNTvPv01Z3kgVTox5z 14z5tEyIBvDThd9kv70cV4Etl/mdIrffWwRvydL5YjZKZoLeBRoWGUwq27ng+XNMpb4i ruCJKBtUofvHLJlgZi5BUiySntJ8/Iba2EefdQ/rs5vHdDAPMwlgrqfLavS/c2TeQ/7N C+LQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qD0MOHKO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y6si12821989ill.138.2021.03.09.11.05.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Mar 2021 11:05:57 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qD0MOHKO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58958 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lJhg3-0002S4-I5 for patch@linaro.org; Tue, 09 Mar 2021 14:05:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:47630) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lJf8Q-0000kj-6s for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:23:02 -0500 Received: from mail-ot1-x32f.google.com ([2607:f8b0:4864:20::32f]:46172) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lJf88-0002ti-4Y for qemu-devel@nongnu.org; Tue, 09 Mar 2021 11:23:01 -0500 Received: by mail-ot1-x32f.google.com with SMTP id r24so6275477otq.13 for ; Tue, 09 Mar 2021 08:22:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KA40eVLdhdRfN2JR4MJ3Si1YCqVpEUGM3POPrQ47iFo=; b=qD0MOHKOp6bP/YsVj8koY8qGBD0BmVEgkJhNjcO5ZVl5Y0EQ6Ec5Fx1l1/F3IeTNYr X+7gf+PIU1/HXjQQF8lFgVqK6sTr16j6sSMiuDV++Y2hpGULonPr51y19Rs0UIlXMNuB mT23KnHS291zDdY/HRY/P/mJvxO9964oFu54YSUEUdZY2lCgKDjRBiAGTUDiIkl1YhqV LDl7OwAI4WglzV8CxDML4EfMjzvx8r51VqY7+xHM7Suc9t8aONa/ApxVK/Mn3bmcJ5rI 0ia9MA22gHEijhojbrq3PtVohgmurbPWpCRHmMnQLn7FpxmO8sL3d+S5/xaWvM4GcGE2 y/IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KA40eVLdhdRfN2JR4MJ3Si1YCqVpEUGM3POPrQ47iFo=; b=VRvxIK8G0z3EtRhqpO21HjA3DT8BQJPWc0v3XKNvLvi5lbJwYj2ZEHAbapVO3ohjIg CrFbNX6hUOafX0UP301RQ/xwL9RVxwTK4Dy5buHH8sfXzsSAQMwqgw9TKNAqDVBHax9T BOmX7GNEOb1Dh5/4BZuJGDG/nmMYWS5hsMh90CR1OZxodXTHmXSlDz5rewc1VTvRIXYY wwfeVu7SpIJEGr6/OlCaMElvw9a4yqw+3qU20tfTnTUACqN1r+EDuDGa3bDH0LIqxYZH N7Y+EaQXZM7YhvMHJpZlEGz/iOqkpSQzl+ZvR8mHSRH81PGcgW1UiBPlWU3ijdccIe6N bpag== X-Gm-Message-State: AOAM531ejOg2HHfx7Z9xnN4OjjnvJFfXcMWmNVv8oedqwZ9cS7iG2G7K 084ZeZWY/+hUMO7kdtosVgyKFyGreqaCm+2v X-Received: by 2002:a9d:7513:: with SMTP id r19mr9976102otk.85.1615306962934; Tue, 09 Mar 2021 08:22:42 -0800 (PST) Received: from localhost.localdomain (fixed-187-189-51-144.totalplay.net. [187.189.51.144]) by smtp.gmail.com with ESMTPSA id u19sm3470898ote.15.2021.03.09.08.22.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Mar 2021 08:22:42 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v4 78/78] target/arm: Enable SVE2 and some extensions Date: Tue, 9 Mar 2021 08:20:41 -0800 Message-Id: <20210309162041.23124-79-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309162041.23124-1-richard.henderson@linaro.org> References: <20210309162041.23124-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32f; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu64.c | 11 +++++++++++ 1 file changed, 11 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index f0a9e968c9..b717b096c3 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -702,6 +702,17 @@ static void aarch64_max_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64MMFR2, ST, 1); /* TTST */ cpu->isar.id_aa64mmfr2 = t; + t = cpu->isar.id_aa64zfr0; + t = FIELD_DP64(t, ID_AA64ZFR0, SVEVER, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, AES, 2); /* PMULL */ + t = FIELD_DP64(t, ID_AA64ZFR0, BITPERM, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, SHA3, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, SM4, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, I8MM, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, F32MM, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, F64MM, 1); + cpu->isar.id_aa64zfr0 = t; + /* Replicate the same data to the 32-bit id registers. */ u = cpu->isar.id_isar5; u = FIELD_DP32(u, ID_ISAR5, AES, 2); /* AES + PMULL */