From patchwork Wed Sep 18 01:58:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173932 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp1870822ill; Tue, 17 Sep 2019 18:58:53 -0700 (PDT) X-Google-Smtp-Source: APXvYqxJXsy1RwszPtxcTzLjOcs2KZqSQGT+8V69c/jY354q97ST30NGCtNOVVKojucIxuaqiTna X-Received: by 2002:a17:907:2124:: with SMTP id qo4mr7262914ejb.311.1568771933013; Tue, 17 Sep 2019 18:58:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568771933; cv=none; d=google.com; s=arc-20160816; b=WbejKEHbofybaBzYd+jIyBgKCVyJt3TnTgWrzcWuvKBU8Zbc45IuA5rY4b+a3g+qFX VSjgGs+HB2xy2yBrEu2ENpYAXymWU7W3HjjLpFJqV8wP1Vc8Ijic8s9MVGBvZYpQ+g9F 2qlMbbjJ/TLaS7brsAYL2jExdus48YsQXtMmtkfuFt9n2JcqrgqhhndQgJvjosbqTh0q 0uJLBu6pkgeZGo/IE5YBNUD+KhAEUgXy2/bYfnuqXbEWZUGkHnz8CPkKjKd20nkIdFCt iYMYskend34ZTGQvGltMpPV3TgVOq/giZkVLfxnxjJVKY9D/nxIWE5JPkkZ8cVz+OctP UKjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature; bh=d3yhuSAWObdwPGMzVydKVQhvY4RLHDLQyz9FmjRVkY0=; b=xc05riWBJAzN8772cUZn3mv2iwG9Yy0OTNXcvbwfaDDqH6g3ylIaJdivjfS0XssDk1 ev1lgRIu4N/jB8sywpE5sQSSiCh9YQ349aenvhP2FB9WgHNN5En3oL4YDjHMME8wNd2f 5/RIBmkO89OeC1ikmDBtF+dw/X/iXpX1PUp6fqiVfpfxleksgo1FwT9erl4OfoN6UttW AKkVNp+m5IwChMeT/0UWDrd1ICD1xkoCiw1opjRG3ynBWqd0BuZ9HuCF6RQgOB29Pizu 4RdqibnibRY6TRPNXFfWDlE9pNWO6F/r9OrhP50RX+fNJRfsnQ9d754FKJ+I+HSB+dXq WmJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=uYsObqQ1; dkim=pass header.i=@linaro.org header.s=google header.b=DDFQ0Qp6; spf=pass (google.com: domain of gcc-patches-return-509144-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509144-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id si9si665770ejb.97.2019.09.17.18.58.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Sep 2019 18:58:53 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-509144-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=uYsObqQ1; dkim=pass header.i=@linaro.org header.s=google header.b=DDFQ0Qp6; spf=pass (google.com: domain of gcc-patches-return-509144-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509144-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; q=dns; s= default; b=PDuqx2svmzwHM6Vvh16OjY+GN7sdxTlkM2oLVgVCROTl0VZb3W+/e P2J9g+GFJssnRWva3E4WSopIhsIHQC4u43UboOj3dCb29QdDFLc4y0hCKV+2VJuT QaHp3z3WRlcqJ73+5YcOJ7r+yDJw/9iT2RpOiFSWozVnLWqqm8NC6o= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; s= default; bh=tVRqMsS1+XO9Ln3uc0U7Wx5Oi2I=; b=uYsObqQ1/MQlpMrAZzIu emw8IRIL2R3Tc1lsoF0xtK7eHEMhXttOQNuuZKsTCP06tO9sZyL/MY0dRzmm81S8 ALROBD0p+xix7DkUNZGZDIRCgqEAvF4GdDMrXt8p892bmkUGVKFlZRSn/pJtrnNI mEsYJJW44y96KMxLdmN2Fy8= Received: (qmail 120487 invoked by alias); 18 Sep 2019 01:58:26 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 120474 invoked by uid 89); 18 Sep 2019 01:58:25 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-19.4 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.1 spammy= X-HELO: mail-pl1-f170.google.com Received: from mail-pl1-f170.google.com (HELO mail-pl1-f170.google.com) (209.85.214.170) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 18 Sep 2019 01:58:24 +0000 Received: by mail-pl1-f170.google.com with SMTP id e5so2350899pls.9 for ; Tue, 17 Sep 2019 18:58:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=d3yhuSAWObdwPGMzVydKVQhvY4RLHDLQyz9FmjRVkY0=; b=DDFQ0Qp6ATET1UR1JZpwwXvRNDjubyWFSMXzofK/JBrdEx+mOf/4JKZPC5XY+OnLbH l6B4FKK2DB0KaFfagv1l3uxh9eRHuLLIIxIQ0Z6aDM/6lx90gfd/58hghJ/SdGMrG1c8 OHL+n9nyLEopG7Af0w+PlQydJCXzUsOd9ful7PbcRk1W5rtnYf+wxR4ZO/Vdk5yD9ghf oa4PrJTbwhKtf8Le9XZs8NcOz0UODfaNkxK2NyxV37yRBrrIe3MGwLX/hj3rnMClgTPy 7TaDCN8PUQFLw9ZakG/FCveo3FWvuAKT0vOt16LSi2phg9HRt2akAi4BAVuLKFaJHmMl Mm8Q== Return-Path: Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id d10sm6614306pfn.50.2019.09.17.18.58.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 18:58:20 -0700 (PDT) From: Richard Henderson To: gcc-patches@gcc.gnu.org Cc: Wilco.Dijkstra@arm.com, kyrylo.tkachov@foss.arm.com, Marcus.Shawcroft@arm.com, James.Greenhalgh@arm.com Subject: [PATCH, AArch64 v4 1/6] aarch64: Extend %R for integer registers Date: Tue, 17 Sep 2019 18:58:12 -0700 Message-Id: <20190918015817.24408-2-richard.henderson@linaro.org> In-Reply-To: <20190918015817.24408-1-richard.henderson@linaro.org> References: <20190918015817.24408-1-richard.henderson@linaro.org> * config/aarch64/aarch64.c (aarch64_print_operand): Allow integer registers with %R. --- gcc/config/aarch64/aarch64.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) -- 2.17.1 diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index 232317d4a5a..99d51e2aef9 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -8420,7 +8420,7 @@ sizetochar (int size) 'S/T/U/V': Print a FP/SIMD register name for a register list. The register printed is the FP/SIMD register name of X + 0/1/2/3 for S/T/U/V. - 'R': Print a scalar FP/SIMD register name + 1. + 'R': Print a scalar Integer/FP/SIMD register name + 1. 'X': Print bottom 16 bits of integer constant in hex. 'w/x': Print a general register name or the zero register (32-bit or 64-bit). @@ -8623,12 +8623,13 @@ aarch64_print_operand (FILE *f, rtx x, int code) break; case 'R': - if (!REG_P (x) || !FP_REGNUM_P (REGNO (x))) - { - output_operand_lossage ("incompatible floating point / vector register operand for '%%%c'", code); - return; - } - asm_fprintf (f, "q%d", REGNO (x) - V0_REGNUM + 1); + if (REG_P (x) && FP_REGNUM_P (REGNO (x))) + asm_fprintf (f, "q%d", REGNO (x) - V0_REGNUM + 1); + else if (REG_P (x) && GP_REGNUM_P (REGNO (x))) + asm_fprintf (f, "x%d", REGNO (x) - R0_REGNUM + 1); + else + output_operand_lossage ("incompatible register operand for '%%%c'", + code); break; case 'X': From patchwork Wed Sep 18 01:58:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173933 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp1870963ill; Tue, 17 Sep 2019 18:59:05 -0700 (PDT) X-Google-Smtp-Source: APXvYqxBanVzF39t1WaHx7AEH84v9MPwFrqBYj2IWbgXrYUZbNGXaUdVQypl2gBr+gjk7D9j5BFU X-Received: by 2002:a17:906:1e0e:: with SMTP id g14mr7287936ejj.247.1568771945702; Tue, 17 Sep 2019 18:59:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568771945; cv=none; d=google.com; s=arc-20160816; b=eDaIey19dijvStI8ZwwWhQU5HTuxSChElLsuSxlR77bh5qqIA5pmqJ+zkxL3BZYd16 4pPaOFs7DZFNnSvYTSuUwNh8lbmFQ4JJb/hXNCOB+31l4ZfUYJMLUPTO5zzjVS5Dk1KK bGNu/L5HXuZlSSti7u/G9ojBEmtDc4MQeli1LKfZ8Hi57M6fV0rc9zdL4uAhdY9Apqif SBYrz1vmfhzsCcuSpGt0vngdo/l2IhpOIKzpWfLicZk4evk+DcDLdkHbNJPaI0Cojyih FnjbjaRSjVL3uJzKT60mA7IrPUePJ9aIYsV1lHxIf9bN64/W+np0lnGCxeNg6BTHJZJz +fug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature; bh=LCor1pkgTu7UhqGMy/ZO4Ubcr0JnZ56kgSWlTCKyc6Q=; b=HCgxpsp+dBmwGgclVJZX9oisxq0wuUwlTWLa3zWtTyKYU4Qy4GQkp6RXEEobBlefPc +glDLwiBAX/myXd7p21n/JcsHzMLg4RlOp0BTwmTtAkjPszgYrp7JiqalJnYm753eRkR yEG0aFb2fud2ofI/IvyT+R5BgVYqxaE05o1UwQNo1JWAE4SqboJPMYDIu/7QFytkBD3u pnbOZpvV1q+SYf3DObIoCmk7Sa0NwKjLFqEHYePWMogEwDwmJAW86eXXhQxmPqw+kwFW xCcKUJUfiQaCxUO735xdCJ4jjx7yMpsAyPYulc4fICv2RSQQuLQIlk/Nv/fP8O6+J2F0 VD6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=IOhonaOg; dkim=pass header.i=@linaro.org header.s=google header.b=pcobBO1G; spf=pass (google.com: domain of gcc-patches-return-509145-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509145-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id b9si2163465eju.269.2019.09.17.18.59.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Sep 2019 18:59:05 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-509145-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=IOhonaOg; dkim=pass header.i=@linaro.org header.s=google header.b=pcobBO1G; spf=pass (google.com: domain of gcc-patches-return-509145-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509145-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; q=dns; s= default; b=U/QVxpESaZi+7L9W7GpQe4tEM4VwGKHvHNf5UhZxy3a+ElI+1HHyb YkdPM8+Va5EFPNL8FKX2rANSTsVMh0A0BYVAQdUfKs7g1aRM94xdRFwJfgo91ufi GC3mCdt0ac9k3PxLhwNqyJ5bJg4tYRTm0bliFIMkzG5sHYhiu6vxCQ= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; s= default; bh=rNeulZTJHdDjvRZ6EJWt7fPx+r0=; b=IOhonaOg12Gyc9CfOmmA 6FcSXJfndFO/YsQjHxyqTtsNG9NSJlURw74LVFS9atF7rg92b1sFTQydNntO4jVm hohMTn1gLoNqJwr8VP4QHCKfPSUnH9MBBD9IhAvb7g4bJEGiJWs9emngB9rmwtSN 9xiWu12rdgkravyhaObfio4= Received: (qmail 120691 invoked by alias); 18 Sep 2019 01:58:27 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 120623 invoked by uid 89); 18 Sep 2019 01:58:27 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-23.2 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.1 spammy=HX-Received:90a, emit_insn X-HELO: mail-pf1-f172.google.com Received: from mail-pf1-f172.google.com (HELO mail-pf1-f172.google.com) (209.85.210.172) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 18 Sep 2019 01:58:25 +0000 Received: by mail-pf1-f172.google.com with SMTP id q21so3285002pfn.11 for ; Tue, 17 Sep 2019 18:58:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LCor1pkgTu7UhqGMy/ZO4Ubcr0JnZ56kgSWlTCKyc6Q=; b=pcobBO1GrS2eJYklbV1NAQzPyDzN2g8h7LH3LkXmv5Qm4ZmZCGU5E2wrZUZ1SbL9JH +lOMmQe9E6j3zenikDx4II+naFTIupEmIkGbXz+LxcZ6pfPopMaO2AsFfaizXA5lZr05 cB0HGAbhEBS0OHkrUsGS8S0TfyrRyAhpeuhj9OYOaFkKnj8RjTaJr3aZuTioOoafYz6Q 5OR+7zrVWGl7RQkQT3RaxBVK/5Qe63XAwObp+a0v6IPZ38/EEVDv0ERNSXzsuRrhyLpJ 5yPHwYKNXItYg7uBSDkSJK0YKu75fq8kJpMba3iOCvw5hujy62492bWdOHT3kq20kUCC VTOw== Return-Path: Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id d10sm6614306pfn.50.2019.09.17.18.58.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 18:58:22 -0700 (PDT) From: Richard Henderson To: gcc-patches@gcc.gnu.org Cc: Wilco.Dijkstra@arm.com, kyrylo.tkachov@foss.arm.com, Marcus.Shawcroft@arm.com, James.Greenhalgh@arm.com Subject: [PATCH, AArch64 v4 2/6] aarch64: Implement TImode compare-and-swap Date: Tue, 17 Sep 2019 18:58:13 -0700 Message-Id: <20190918015817.24408-3-richard.henderson@linaro.org> In-Reply-To: <20190918015817.24408-1-richard.henderson@linaro.org> References: <20190918015817.24408-1-richard.henderson@linaro.org> This pattern will only be used with the __sync functions, because we do not yet have a bare TImode atomic load. * config/aarch64/aarch64.c (aarch64_gen_compare_reg): Add support for NE comparison of TImode values. (aarch64_emit_load_exclusive): Add support for TImode. (aarch64_emit_store_exclusive): Likewise. (aarch64_split_compare_and_swap): Disable strong_zero_p for TImode. * config/aarch64/atomics.md (@atomic_compare_and_swap): Change iterator from ALLI to ALLI_TI. (@atomic_compare_and_swap): New. (@atomic_compare_and_swap_lse): New. (aarch64_load_exclusive_pair): New. (aarch64_store_exclusive_pair): New. * config/aarch64/iterators.md (JUST_TI): New. --- gcc/config/aarch64/aarch64.c | 48 ++++++++++++++--- gcc/config/aarch64/atomics.md | 93 +++++++++++++++++++++++++++++++-- gcc/config/aarch64/iterators.md | 3 ++ 3 files changed, 131 insertions(+), 13 deletions(-) -- 2.17.1 diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index 99d51e2aef9..a5c4f55627d 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -2039,10 +2039,33 @@ emit_set_insn (rtx x, rtx y) rtx aarch64_gen_compare_reg (RTX_CODE code, rtx x, rtx y) { - machine_mode mode = SELECT_CC_MODE (code, x, y); - rtx cc_reg = gen_rtx_REG (mode, CC_REGNUM); + machine_mode cmp_mode = GET_MODE (x); + machine_mode cc_mode; + rtx cc_reg; - emit_set_insn (cc_reg, gen_rtx_COMPARE (mode, x, y)); + if (cmp_mode == TImode) + { + gcc_assert (code == NE); + + cc_mode = CCmode; + cc_reg = gen_rtx_REG (cc_mode, CC_REGNUM); + + rtx x_lo = operand_subword (x, 0, 0, TImode); + rtx y_lo = operand_subword (y, 0, 0, TImode); + emit_set_insn (cc_reg, gen_rtx_COMPARE (cc_mode, x_lo, y_lo)); + + rtx x_hi = operand_subword (x, 1, 0, TImode); + rtx y_hi = operand_subword (y, 1, 0, TImode); + emit_insn (gen_ccmpdi (cc_reg, cc_reg, x_hi, y_hi, + gen_rtx_EQ (cc_mode, cc_reg, const0_rtx), + GEN_INT (AARCH64_EQ))); + } + else + { + cc_mode = SELECT_CC_MODE (code, x, y); + cc_reg = gen_rtx_REG (cc_mode, CC_REGNUM); + emit_set_insn (cc_reg, gen_rtx_COMPARE (cc_mode, x, y)); + } return cc_reg; } @@ -2593,7 +2616,6 @@ aarch64_zero_extend_const_eq (machine_mode xmode, rtx x, gcc_assert (r != NULL); return rtx_equal_p (x, r); } - /* Return TARGET if it is nonnull and a register of mode MODE. Otherwise, return a fresh register of mode MODE if we can, @@ -16814,16 +16836,26 @@ static void aarch64_emit_load_exclusive (machine_mode mode, rtx rval, rtx mem, rtx model_rtx) { - emit_insn (gen_aarch64_load_exclusive (mode, rval, mem, model_rtx)); + if (mode == TImode) + emit_insn (gen_aarch64_load_exclusive_pair (gen_lowpart (DImode, rval), + gen_highpart (DImode, rval), + mem, model_rtx)); + else + emit_insn (gen_aarch64_load_exclusive (mode, rval, mem, model_rtx)); } /* Emit store exclusive. */ static void aarch64_emit_store_exclusive (machine_mode mode, rtx bval, - rtx rval, rtx mem, rtx model_rtx) + rtx mem, rtx rval, rtx model_rtx) { - emit_insn (gen_aarch64_store_exclusive (mode, bval, rval, mem, model_rtx)); + if (mode == TImode) + emit_insn (gen_aarch64_store_exclusive_pair + (bval, mem, operand_subword (rval, 0, 0, TImode), + operand_subword (rval, 1, 0, TImode), model_rtx)); + else + emit_insn (gen_aarch64_store_exclusive (mode, bval, mem, rval, model_rtx)); } /* Mark the previous jump instruction as unlikely. */ @@ -16950,7 +16982,7 @@ aarch64_split_compare_and_swap (rtx operands[]) CBNZ scratch, .label1 .label2: CMP rval, 0. */ - bool strong_zero_p = !is_weak && oldval == const0_rtx; + bool strong_zero_p = !is_weak && oldval == const0_rtx && mode != TImode; label1 = NULL; if (!is_weak) diff --git a/gcc/config/aarch64/atomics.md b/gcc/config/aarch64/atomics.md index a679270cd38..f8bdd048b37 100644 --- a/gcc/config/aarch64/atomics.md +++ b/gcc/config/aarch64/atomics.md @@ -21,11 +21,11 @@ ;; Instruction patterns. (define_expand "@atomic_compare_and_swap" - [(match_operand:SI 0 "register_operand") ;; bool out - (match_operand:ALLI 1 "register_operand") ;; val out - (match_operand:ALLI 2 "aarch64_sync_memory_operand") ;; memory - (match_operand:ALLI 3 "nonmemory_operand") ;; expected - (match_operand:ALLI 4 "aarch64_reg_or_zero") ;; desired + [(match_operand:SI 0 "register_operand" "") ;; bool out + (match_operand:ALLI_TI 1 "register_operand" "") ;; val out + (match_operand:ALLI_TI 2 "aarch64_sync_memory_operand" "") ;; memory + (match_operand:ALLI_TI 3 "nonmemory_operand" "") ;; expected + (match_operand:ALLI_TI 4 "aarch64_reg_or_zero" "") ;; desired (match_operand:SI 5 "const_int_operand") ;; is_weak (match_operand:SI 6 "const_int_operand") ;; mod_s (match_operand:SI 7 "const_int_operand")] ;; mod_f @@ -88,6 +88,30 @@ } ) +(define_insn_and_split "@aarch64_compare_and_swap" + [(set (reg:CC CC_REGNUM) ;; bool out + (unspec_volatile:CC [(const_int 0)] UNSPECV_ATOMIC_CMPSW)) + (set (match_operand:JUST_TI 0 "register_operand" "=&r") ;; val out + (match_operand:JUST_TI 1 "aarch64_sync_memory_operand" "+Q")) ;; memory + (set (match_dup 1) + (unspec_volatile:JUST_TI + [(match_operand:JUST_TI 2 "aarch64_reg_or_zero" "rZ") ;; expect + (match_operand:JUST_TI 3 "aarch64_reg_or_zero" "rZ") ;; desired + (match_operand:SI 4 "const_int_operand") ;; is_weak + (match_operand:SI 5 "const_int_operand") ;; mod_s + (match_operand:SI 6 "const_int_operand")] ;; mod_f + UNSPECV_ATOMIC_CMPSW)) + (clobber (match_scratch:SI 7 "=&r"))] + "" + "#" + "&& reload_completed" + [(const_int 0)] + { + aarch64_split_compare_and_swap (operands); + DONE; + } +) + (define_insn "@aarch64_compare_and_swap_lse" [(set (match_operand:SI 0 "register_operand" "+r") ;; val out (zero_extend:SI @@ -133,6 +157,28 @@ return "casal\t%0, %2, %1"; }) +(define_insn "@aarch64_compare_and_swap_lse" + [(set (match_operand:JUST_TI 0 "register_operand" "+r") ;; val out + (match_operand:JUST_TI 1 "aarch64_sync_memory_operand" "+Q")) ;; memory + (set (match_dup 1) + (unspec_volatile:JUST_TI + [(match_dup 0) ;; expect + (match_operand:JUST_TI 2 "register_operand" "r") ;; desired + (match_operand:SI 3 "const_int_operand")] ;; mod_s + UNSPECV_ATOMIC_CMPSW))] + "TARGET_LSE" +{ + enum memmodel model = memmodel_from_int (INTVAL (operands[3])); + if (is_mm_relaxed (model)) + return "casp\t%0, %R0, %2, %R2, %1"; + else if (is_mm_acquire (model) || is_mm_consume (model)) + return "caspa\t%0, %R0, %2, %R2, %1"; + else if (is_mm_release (model)) + return "caspl\t%0, %R0, %2, %R2, %1"; + else + return "caspal\t%0, %R0, %2, %R2, %1"; +}) + (define_expand "atomic_exchange" [(match_operand:ALLI 0 "register_operand") (match_operand:ALLI 1 "aarch64_sync_memory_operand") @@ -581,6 +627,24 @@ } ) +(define_insn "aarch64_load_exclusive_pair" + [(set (match_operand:DI 0 "register_operand" "=r") + (unspec_volatile:DI + [(match_operand:TI 2 "aarch64_sync_memory_operand" "Q") + (match_operand:SI 3 "const_int_operand")] + UNSPECV_LX)) + (set (match_operand:DI 1 "register_operand" "=r") + (unspec_volatile:DI [(match_dup 2) (match_dup 3)] UNSPECV_LX))] + "" + { + enum memmodel model = memmodel_from_int (INTVAL (operands[3])); + if (is_mm_relaxed (model) || is_mm_consume (model) || is_mm_release (model)) + return "ldxp\t%0, %1, %2"; + else + return "ldaxp\t%0, %1, %2"; + } +) + (define_insn "@aarch64_store_exclusive" [(set (match_operand:SI 0 "register_operand" "=&r") (unspec_volatile:SI [(const_int 0)] UNSPECV_SX)) @@ -599,6 +663,25 @@ } ) +(define_insn "aarch64_store_exclusive_pair" + [(set (match_operand:SI 0 "register_operand" "=&r") + (unspec_volatile:SI [(const_int 0)] UNSPECV_SX)) + (set (match_operand:TI 1 "aarch64_sync_memory_operand" "=Q") + (unspec_volatile:TI + [(match_operand:DI 2 "aarch64_reg_or_zero" "rZ") + (match_operand:DI 3 "aarch64_reg_or_zero" "rZ") + (match_operand:SI 4 "const_int_operand")] + UNSPECV_SX))] + "" + { + enum memmodel model = memmodel_from_int (INTVAL (operands[3])); + if (is_mm_relaxed (model) || is_mm_consume (model) || is_mm_acquire (model)) + return "stxp\t%w0, %x2, %x3, %1"; + else + return "stlxp\t%w0, %x2, %x3, %1"; + } +) + (define_expand "mem_thread_fence" [(match_operand:SI 0 "const_int_operand")] "" diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md index d23f0fcbc2f..03b3ce36302 100644 --- a/gcc/config/aarch64/iterators.md +++ b/gcc/config/aarch64/iterators.md @@ -29,6 +29,9 @@ ;; Iterator for HI, SI, DI, some instructions can only work on these modes. (define_mode_iterator GPI_I16 [(HI "AARCH64_ISA_F16") SI DI]) +;; "Iterator" for just TI -- features like @pattern only work with iterators. +(define_mode_iterator JUST_TI [TI]) + ;; Iterator for QI and HI modes (define_mode_iterator SHORT [QI HI]) From patchwork Wed Sep 18 01:58:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173934 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp1871102ill; Tue, 17 Sep 2019 18:59:17 -0700 (PDT) X-Google-Smtp-Source: APXvYqzEglCFuXrTeAK2M/h5tNd9i1nytZk6c9nR2VNiYZksgJ6Ivqa7GcZEJKXtk/vNZ7vIdk9i X-Received: by 2002:a05:6402:782:: with SMTP id d2mr6515785edy.296.1568771957317; Tue, 17 Sep 2019 18:59:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568771957; cv=none; d=google.com; s=arc-20160816; b=tQY0sXX3+xMw7s8nFlqhPh0pcLPb0dwE34e+eFsnLiY3L0qhRJgx23h10wPRwu9bZE WxDhkRZLY9IemRhWSK1ZjJT2vTEEmFUlQgN0kSLYDKRF/MGJYXQ4YTwgELSuPFp1vOmN 9gokcnAIJtWyHD7LoSfys8BOD9/VcpCP4CQ7JcGxbQd0rzm4xOQIgo9E05dYrhTS3EQi daf/9ZmZoYsc/GhvzM4O4qrlHyL9OKcYNKQI3tInzj2X0iPvca6flQ80F+uv+Z0cC2TN Yc7DoEbhqBcYnC2LaOp+If9CfRSIh0pHo8gefbc0q9sJogP9O2LKJWAQ1TErFMzQUUM9 aKng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature; bh=LYy7CYDV00a+Fq0naW7EFGUs/iRqLDxrtNHdtv2Pguk=; b=W75OfYWfSqxQf/TnPWH8YQCjJtgw5XX7K7L10RDBvRuLAgICC8aNdywkZIX8VHn5rt SICjMm2Dnt7mq2Ps77Z93R6PsM28y58kudXHvLZasi1lKcSsEvz/VSfcLdjSRZsL3MIo GiqV5yMl0iN1ETl+1ahNReQsT2PRhKEz4TZ2biIh7skF3n1mgy/blAATOBxDvto1zjFD iLquDY/otaLVXvSqTQ7OJJjvHOewcZql6TKFC4qrVbUfmb/qBiZ0iNUxU7z0vdDrlPod SoQiLN9P9qygokGWBeIYfpjUwHIabWa0XGhPTFWfAGAa1NgALTQ+46iT22L/oygq0J1C ms7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=bzSgFTWX; dkim=pass header.i=@linaro.org header.s=google header.b="Oic/3aqq"; spf=pass (google.com: domain of gcc-patches-return-509146-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509146-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id b1si2570147edm.271.2019.09.17.18.59.16 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Sep 2019 18:59:17 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-509146-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=bzSgFTWX; dkim=pass header.i=@linaro.org header.s=google header.b="Oic/3aqq"; spf=pass (google.com: domain of gcc-patches-return-509146-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509146-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; q=dns; s= default; b=wvb1zAXTXTi5+OWre2HicJG2I58sx2uGOWY15PbNC6EbQb5K/3itY 0Bn2uVSR4dbmVR796YVGe09fkBhCfYGqrSDVTtH9oWic2rB68v6rWewM4mADnnq5 Ulix1B+YpFy0RPU9iSNxZkvn3lD7S7hypEhiwjbLoPLyszmbqt05jQ= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; s= default; bh=82SfhKTlDHEdIaQdIjraQM4yllg=; b=bzSgFTWXI3kNvO1YgK70 FhUZqzL0bVOJShsZ+SPBzQJhsflGW0ntWUjPcrCnO0f3qae8YKZYLD4Qlg939VaS 7qLFlZalNyPdk7nW7qwsSAXyifpq7jqoad5+ROz2LTIJ++RCnELBcKEznjbV8j+i 1R/SY+dnWcgyClaW+9q1noc= Received: (qmail 120779 invoked by alias); 18 Sep 2019 01:58:28 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 120696 invoked by uid 89); 18 Sep 2019 01:58:27 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-24.4 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.1 spammy=emit_insn, codes X-HELO: mail-pl1-f193.google.com Received: from mail-pl1-f193.google.com (HELO mail-pl1-f193.google.com) (209.85.214.193) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 18 Sep 2019 01:58:26 +0000 Received: by mail-pl1-f193.google.com with SMTP id x6so874981plv.6 for ; Tue, 17 Sep 2019 18:58:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LYy7CYDV00a+Fq0naW7EFGUs/iRqLDxrtNHdtv2Pguk=; b=Oic/3aqq11ymCWuS69PlQUDtUridVYBzRHIdsJwf3iRcoRwNruLPwhyThp35qTCdCR IST2JOjfQFCPvF5f6+O2qKJOr1AGWL9mprE+3gOKQ1kBohl5bnyoLQsQphYm1D/Hn73s f+hkQ/xSddaB9iGMPUYRfsmCqsFNs0vhE2Co2bSVlZM8O2X23qi3ACUXf2UfWvzXDMHn 2gouXML5HHVoEnjw7PUennxYfq/f1JaNudI43HNPOI5rsVFODhOLSddk7OEyIwWgKTyL 3/GiRV0kMNslNPL9aPG3fy6iCEEleaEleefXkGRP6p1XpTTOzMNyG4jxkGlapxwtk04V ZiDA== Return-Path: Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id d10sm6614306pfn.50.2019.09.17.18.58.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 18:58:23 -0700 (PDT) From: Richard Henderson To: gcc-patches@gcc.gnu.org Cc: Wilco.Dijkstra@arm.com, kyrylo.tkachov@foss.arm.com, Marcus.Shawcroft@arm.com, James.Greenhalgh@arm.com Subject: [PATCH, AArch64 v4 3/6] aarch64: Tidy aarch64_split_compare_and_swap Date: Tue, 17 Sep 2019 18:58:14 -0700 Message-Id: <20190918015817.24408-4-richard.henderson@linaro.org> In-Reply-To: <20190918015817.24408-1-richard.henderson@linaro.org> References: <20190918015817.24408-1-richard.henderson@linaro.org> With aarch64_track_speculation, we had extra code to do exactly what the !strong_zero_p path already did. The rest is reducing code duplication. * config/aarch64/aarch64 (aarch64_split_compare_and_swap): Disable strong_zero_p for aarch64_track_speculation; unify some code paths; use aarch64_gen_compare_reg instead of open-coding. --- gcc/config/aarch64/aarch64.c | 50 ++++++++++-------------------------- 1 file changed, 14 insertions(+), 36 deletions(-) -- 2.17.1 diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index a5c4f55627d..b937514e6f8 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -16955,13 +16955,11 @@ aarch64_emit_post_barrier (enum memmodel model) void aarch64_split_compare_and_swap (rtx operands[]) { - rtx rval, mem, oldval, newval, scratch; + rtx rval, mem, oldval, newval, scratch, x, model_rtx; machine_mode mode; bool is_weak; rtx_code_label *label1, *label2; - rtx x, cond; enum memmodel model; - rtx model_rtx; rval = operands[0]; mem = operands[1]; @@ -16982,7 +16980,8 @@ aarch64_split_compare_and_swap (rtx operands[]) CBNZ scratch, .label1 .label2: CMP rval, 0. */ - bool strong_zero_p = !is_weak && oldval == const0_rtx && mode != TImode; + bool strong_zero_p = (!is_weak && !aarch64_track_speculation && + oldval == const0_rtx && mode != TImode); label1 = NULL; if (!is_weak) @@ -16995,35 +16994,20 @@ aarch64_split_compare_and_swap (rtx operands[]) /* The initial load can be relaxed for a __sync operation since a final barrier will be emitted to stop code hoisting. */ if (is_mm_sync (model)) - aarch64_emit_load_exclusive (mode, rval, mem, - GEN_INT (MEMMODEL_RELAXED)); + aarch64_emit_load_exclusive (mode, rval, mem, GEN_INT (MEMMODEL_RELAXED)); else aarch64_emit_load_exclusive (mode, rval, mem, model_rtx); if (strong_zero_p) - { - if (aarch64_track_speculation) - { - /* Emit an explicit compare instruction, so that we can correctly - track the condition codes. */ - rtx cc_reg = aarch64_gen_compare_reg (NE, rval, const0_rtx); - x = gen_rtx_NE (GET_MODE (cc_reg), cc_reg, const0_rtx); - } - else - x = gen_rtx_NE (VOIDmode, rval, const0_rtx); - - x = gen_rtx_IF_THEN_ELSE (VOIDmode, x, - gen_rtx_LABEL_REF (Pmode, label2), pc_rtx); - aarch64_emit_unlikely_jump (gen_rtx_SET (pc_rtx, x)); - } + x = gen_rtx_NE (VOIDmode, rval, const0_rtx); else { - cond = aarch64_gen_compare_reg_maybe_ze (NE, rval, oldval, mode); - x = gen_rtx_NE (VOIDmode, cond, const0_rtx); - x = gen_rtx_IF_THEN_ELSE (VOIDmode, x, - gen_rtx_LABEL_REF (Pmode, label2), pc_rtx); - aarch64_emit_unlikely_jump (gen_rtx_SET (pc_rtx, x)); + rtx cc_reg = aarch64_gen_compare_reg_maybe_ze (NE, rval, oldval, mode); + x = gen_rtx_NE (VOIDmode, cc_reg, const0_rtx); } + x = gen_rtx_IF_THEN_ELSE (VOIDmode, x, + gen_rtx_LABEL_REF (Pmode, label2), pc_rtx); + aarch64_emit_unlikely_jump (gen_rtx_SET (pc_rtx, x)); aarch64_emit_store_exclusive (mode, scratch, mem, newval, model_rtx); @@ -17044,22 +17028,16 @@ aarch64_split_compare_and_swap (rtx operands[]) aarch64_emit_unlikely_jump (gen_rtx_SET (pc_rtx, x)); } else - { - cond = gen_rtx_REG (CCmode, CC_REGNUM); - x = gen_rtx_COMPARE (CCmode, scratch, const0_rtx); - emit_insn (gen_rtx_SET (cond, x)); - } + aarch64_gen_compare_reg (NE, scratch, const0_rtx); emit_label (label2); + /* If we used a CBNZ in the exchange loop emit an explicit compare with RVAL to set the condition flags. If this is not used it will be removed by later passes. */ if (strong_zero_p) - { - cond = gen_rtx_REG (CCmode, CC_REGNUM); - x = gen_rtx_COMPARE (CCmode, rval, const0_rtx); - emit_insn (gen_rtx_SET (cond, x)); - } + aarch64_gen_compare_reg (NE, rval, const0_rtx); + /* Emit any final barrier needed for a __sync operation. */ if (is_mm_sync (model)) aarch64_emit_post_barrier (model); From patchwork Wed Sep 18 01:58:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173936 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp1871370ill; Tue, 17 Sep 2019 18:59:42 -0700 (PDT) X-Google-Smtp-Source: APXvYqzgplqa18ztgCoWKJ0Qk3313DPn+ael5m/gqYBfiFGx3VD3qISbcKnpG83uUmTPlstvPty+ X-Received: by 2002:a05:6402:160d:: with SMTP id f13mr7905646edv.227.1568771982154; Tue, 17 Sep 2019 18:59:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568771982; cv=none; d=google.com; s=arc-20160816; b=SNgcG55ADicyKliwOhP/C1029k8DdIqTdakYuaVDS0NIFLgNNJxshwsoMa6lTZeD7w wNQkVGY4PiEEd8cjSJ2TRgN5or6pRyjQgkhwSSTGOMY/wzLlJ9RQEkSTbfhYuj4+d3le 2q2dcDkavglRzhyZr5jJrpHbJ5vr+fcwQRlzsrE0YwnlKcW7IRXAcImSTTRWJGkETcOt P0B+AFy/AtBHOeTOzVQXd7z+OZ/UfBwzlCpLwrlM1KOKwC3mQUpQboHxf8PqG1wW7bDN AeJorR5QzQnPIKlFSzVil05Wj9+D6cf4nodzrIGbHyP/qN2ueyjAVFPCiR3bDTWceZUJ 4+dA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature; bh=kphIlyNV9CkVhM0qbANn1TIsctdbowMVWnA6Xyn0vxQ=; b=QznzQtNqWMyMH2mYSXlCsL+XacVvuGDELZu3cF/nGjoFNe+yp1vE3IhgLxNTB6SpJ+ gFTRQdcwMstmMPX8iq9Idwwp8yArRNWJwr3cMwHuSF3xZWCUkgF86GjrqXKO5NWSSUkF p2WhTYQ5SY0nXdpWqvHne2Kvkncx/TIeUoTkFtTTwVpL74yKkSgACHrUFSoE2lRL0+c6 yWJn2rnL+aW1ERDz0ivigTVjlB8k7i0QAciiE7IVJP9zJvnhYFhSe8oPO3HG3M2UtSFL R5l/GmtRzqbaI2cB3yrz46Ukx9lEtJQriiDm+f990JYXULTQeUgJV/RlSZirTc5cqQ2a 6KNQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=IXs2iDv0; dkim=pass header.i=@linaro.org header.s=google header.b=ydYT+PT+; spf=pass (google.com: domain of gcc-patches-return-509148-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509148-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id x8si2136748ejb.358.2019.09.17.18.59.41 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Sep 2019 18:59:42 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-509148-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=IXs2iDv0; dkim=pass header.i=@linaro.org header.s=google header.b=ydYT+PT+; spf=pass (google.com: domain of gcc-patches-return-509148-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509148-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; q=dns; s= default; b=eh6a8GXbj4jJCWlAGA98jUOJP5Nfaju2E/VT4mksplQ/VRkECvJn/ NnE1OIsT83ip6fCw66wne+wChULgSuC215+Ftcx+QIFHrIEHoOT4bW9mwInBM3ye DxkT4nVDpYxOU+iUMA5Ol7SX03S5wIPwrtVx++8/lL861+18Hi4YW4= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; s= default; bh=AG3MsOZSi7QwttiTrFcclFX3Wo8=; b=IXs2iDv0bKO+pxmRKes5 idEtCoxao6cde+CbX4Zi+CTP/sS1VAylVnGRrdDAW3VTJiAEOGUp1xUDnnSBFeOF ILOmKPO7qxsaqS7PHGwf466WkLSm6Zpwqsb5HRZOt9wDFx+gRSj9cLIGT6gmJ0fW EI1t01AibPzuScj/cYdj2FE= Received: (qmail 121321 invoked by alias); 18 Sep 2019 01:58:32 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 121190 invoked by uid 89); 18 Sep 2019 01:58:31 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-25.0 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.1 spammy=examine, 3.1, linaro, concatenate X-HELO: mail-pl1-f180.google.com Received: from mail-pl1-f180.google.com (HELO mail-pl1-f180.google.com) (209.85.214.180) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 18 Sep 2019 01:58:27 +0000 Received: by mail-pl1-f180.google.com with SMTP id x6so875017plv.6 for ; Tue, 17 Sep 2019 18:58:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kphIlyNV9CkVhM0qbANn1TIsctdbowMVWnA6Xyn0vxQ=; b=ydYT+PT+vp9l5S6naOrHCwL7gzRR3gNaJO0kSluUZVxV2Hm2aLiqfijbfQ3Sr9myNC VCTrfeXzoc4nPJZnDNK+Sdbe79ZEyQlwDz3b6HI4E39QjR2tHSYWKZOPId3UrcdX8Mcd JX4j7Z0ltaWRa6KA07VcYYDF/DHTKVhrrDdiVD2iWxqRq4ipM5jPkCoekKqmwGYSqQFa tOLch/VSVjYczoKg6T4aM7Tb4vkHM+7J/YDmBVgRtkbP5eoLM026YgzSvYhqR7Gq6DqZ oV7C0zaTjqCvR3RiI4GM1RcsfQHmlg6k1NY5gl/a/WwDkYXqSIioqqmtueDNyi7kHipv 771g== Return-Path: Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id d10sm6614306pfn.50.2019.09.17.18.58.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 18:58:24 -0700 (PDT) From: Richard Henderson To: gcc-patches@gcc.gnu.org Cc: Wilco.Dijkstra@arm.com, kyrylo.tkachov@foss.arm.com, Marcus.Shawcroft@arm.com, James.Greenhalgh@arm.com Subject: [PATCH, AArch64 v4 4/6] aarch64: Add out-of-line functions for LSE atomics Date: Tue, 17 Sep 2019 18:58:15 -0700 Message-Id: <20190918015817.24408-5-richard.henderson@linaro.org> In-Reply-To: <20190918015817.24408-1-richard.henderson@linaro.org> References: <20190918015817.24408-1-richard.henderson@linaro.org> This is the libgcc part of the interface -- providing the functions. Rationale is provided at the top of libgcc/config/aarch64/lse.S. * config/aarch64/lse-init.c: New file. * config/aarch64/lse.S: New file. * config/aarch64/t-lse: New file. * config.host: Add t-lse to all aarch64 tuples. --- libgcc/config/aarch64/lse-init.c | 45 ++++++ libgcc/config.host | 4 + libgcc/config/aarch64/lse.S | 235 +++++++++++++++++++++++++++++++ libgcc/config/aarch64/t-lse | 44 ++++++ 4 files changed, 328 insertions(+) create mode 100644 libgcc/config/aarch64/lse-init.c create mode 100644 libgcc/config/aarch64/lse.S create mode 100644 libgcc/config/aarch64/t-lse -- 2.17.1 diff --git a/libgcc/config/aarch64/lse-init.c b/libgcc/config/aarch64/lse-init.c new file mode 100644 index 00000000000..51fb21d45c9 --- /dev/null +++ b/libgcc/config/aarch64/lse-init.c @@ -0,0 +1,45 @@ +/* Out-of-line LSE atomics for AArch64 architecture, Init. + Copyright (C) 2018 Free Software Foundation, Inc. + Contributed by Linaro Ltd. + +This file is part of GCC. + +GCC is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free +Software Foundation; either version 3, or (at your option) any later +version. + +GCC is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +for more details. + +Under Section 7 of GPL version 3, you are granted additional +permissions described in the GCC Runtime Library Exception, version +3.1, as published by the Free Software Foundation. + +You should have received a copy of the GNU General Public License and +a copy of the GCC Runtime Library Exception along with this program; +see the files COPYING3 and COPYING.RUNTIME respectively. If not, see +. */ + +/* Define the symbol gating the LSE implementations. */ +_Bool __aarch64_have_lse_atomics + __attribute__((visibility("hidden"), nocommon)); + +/* Disable initialization of __aarch64_have_lse_atomics during bootstrap. */ +#ifndef inhibit_libc +# include + +/* Disable initialization if the system headers are too old. */ +# if defined(AT_HWCAP) && defined(HWCAP_ATOMICS) + +static void __attribute__((constructor)) +init_have_lse_atomics (void) +{ + unsigned long hwcap = getauxval (AT_HWCAP); + __aarch64_have_lse_atomics = (hwcap & HWCAP_ATOMICS) != 0; +} + +# endif /* HWCAP */ +#endif /* inhibit_libc */ diff --git a/libgcc/config.host b/libgcc/config.host index 728e543ea39..122113fc519 100644 --- a/libgcc/config.host +++ b/libgcc/config.host @@ -350,12 +350,14 @@ aarch64*-*-elf | aarch64*-*-rtems*) extra_parts="$extra_parts crtbegin.o crtend.o crti.o crtn.o" extra_parts="$extra_parts crtfastmath.o" tmake_file="${tmake_file} ${cpu_type}/t-aarch64" + tmake_file="${tmake_file} ${cpu_type}/t-lse t-slibgcc-libgcc" tmake_file="${tmake_file} ${cpu_type}/t-softfp t-softfp t-crtfm" md_unwind_header=aarch64/aarch64-unwind.h ;; aarch64*-*-freebsd*) extra_parts="$extra_parts crtfastmath.o" tmake_file="${tmake_file} ${cpu_type}/t-aarch64" + tmake_file="${tmake_file} ${cpu_type}/t-lse t-slibgcc-libgcc" tmake_file="${tmake_file} ${cpu_type}/t-softfp t-softfp t-crtfm" md_unwind_header=aarch64/freebsd-unwind.h ;; @@ -367,12 +369,14 @@ aarch64*-*-netbsd*) ;; aarch64*-*-fuchsia*) tmake_file="${tmake_file} ${cpu_type}/t-aarch64" + tmake_file="${tmake_file} ${cpu_type}/t-lse t-slibgcc-libgcc" tmake_file="${tmake_file} ${cpu_type}/t-softfp t-softfp" ;; aarch64*-*-linux*) extra_parts="$extra_parts crtfastmath.o" md_unwind_header=aarch64/linux-unwind.h tmake_file="${tmake_file} ${cpu_type}/t-aarch64" + tmake_file="${tmake_file} ${cpu_type}/t-lse t-slibgcc-libgcc" tmake_file="${tmake_file} ${cpu_type}/t-softfp t-softfp t-crtfm" ;; alpha*-*-linux*) diff --git a/libgcc/config/aarch64/lse.S b/libgcc/config/aarch64/lse.S new file mode 100644 index 00000000000..c24a39242ca --- /dev/null +++ b/libgcc/config/aarch64/lse.S @@ -0,0 +1,235 @@ +/* Out-of-line LSE atomics for AArch64 architecture. + Copyright (C) 2018 Free Software Foundation, Inc. + Contributed by Linaro Ltd. + +This file is part of GCC. + +GCC is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free +Software Foundation; either version 3, or (at your option) any later +version. + +GCC is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +for more details. + +Under Section 7 of GPL version 3, you are granted additional +permissions described in the GCC Runtime Library Exception, version +3.1, as published by the Free Software Foundation. + +You should have received a copy of the GNU General Public License and +a copy of the GCC Runtime Library Exception along with this program; +see the files COPYING3 and COPYING.RUNTIME respectively. If not, see +. */ + +/* + * The problem that we are trying to solve is operating system deployment + * of ARMv8.1-Atomics, also known as Large System Exensions (LSE). + * + * There are a number of potential solutions for this problem which have + * been proposed and rejected for various reasons. To recap: + * + * (1) Multiple builds. The dynamic linker will examine /lib64/atomics/ + * if HWCAP_ATOMICS is set, allowing entire libraries to be overwritten. + * However, not all Linux distributions are happy with multiple builds, + * and anyway it has no effect on main applications. + * + * (2) IFUNC. We could put these functions into libgcc_s.so, and have + * a single copy of each function for all DSOs. However, ARM is concerned + * that the branch-to-indirect-branch that is implied by using a PLT, + * as required by IFUNC, is too much overhead for smaller cpus. + * + * (3) Statically predicted direct branches. This is the approach that + * is taken here. These functions are linked into every DSO that uses them. + * All of the symbols are hidden, so that the functions are called via a + * direct branch. The choice of LSE vs non-LSE is done via one byte load + * followed by a well-predicted direct branch. The functions are compiled + * separately to minimize code size. + */ + +/* Tell the assembler to accept LSE instructions. */ + .arch armv8-a+lse + +/* Declare the symbol gating the LSE implementations. */ + .hidden __aarch64_have_lse_atomics + +/* Turn size and memory model defines into mnemonic fragments. */ +#if SIZE == 1 +# define S b +# define UXT uxtb +#elif SIZE == 2 +# define S h +# define UXT uxth +#elif SIZE == 4 || SIZE == 8 || SIZE == 16 +# define S +# define UXT mov +#else +# error +#endif + +#if MODEL == 1 +# define SUFF _relax +# define A +# define L +#elif MODEL == 2 +# define SUFF _acq +# define A a +# define L +#elif MODEL == 3 +# define SUFF _rel +# define A +# define L l +#elif MODEL == 4 +# define SUFF _acq_rel +# define A a +# define L l +#else +# error +#endif + +/* Concatenate symbols. */ +#define glue2_(A, B) A ## B +#define glue2(A, B) glue2_(A, B) +#define glue3_(A, B, C) A ## B ## C +#define glue3(A, B, C) glue3_(A, B, C) +#define glue4_(A, B, C, D) A ## B ## C ## D +#define glue4(A, B, C, D) glue4_(A, B, C, D) + +/* Select the size of a register, given a regno. */ +#define x(N) glue2(x, N) +#define w(N) glue2(w, N) +#if SIZE < 8 +# define s(N) w(N) +#else +# define s(N) x(N) +#endif + +#define NAME(BASE) glue4(__aarch64_, BASE, SIZE, SUFF) +#define LDXR glue4(ld, A, xr, S) +#define STXR glue4(st, L, xr, S) + +/* Temporary registers used. Other than these, only the return value + register (x0) and the flags are modified. */ +#define tmp0 16 +#define tmp1 17 +#define tmp2 15 + +/* Start and end a function. */ +.macro STARTFN name + .text + .balign 16 + .globl \name + .hidden \name + .type \name, %function + .cfi_startproc +\name: +.endm + +.macro ENDFN name + .cfi_endproc + .size \name, . - \name +.endm + +/* Branch to LABEL if LSE is disabled. */ +.macro JUMP_IF_NOT_LSE label + adrp x(tmp0), __aarch64_have_lse_atomics + ldrb w(tmp0), [x(tmp0), :lo12:__aarch64_have_lse_atomics] + cbz w(tmp0), \label +.endm + +#ifdef L_cas + +STARTFN NAME(cas) + JUMP_IF_NOT_LSE 8f + +#if SIZE < 16 +#define CAS glue4(cas, A, L, S) + + CAS s(0), s(1), [x2] + ret + +8: UXT s(tmp0), s(0) +0: LDXR s(0), [x2] + cmp s(0), s(tmp0) + bne 1f + STXR w(tmp1), s(1), [x2] + cbnz w(tmp1), 0b +1: ret + +#else +#define LDXP glue3(ld, A, xp) +#define STXP glue3(st, L, xp) +#define CASP glue3(casp, A, L) + + CASP x0, x1, x2, x3, [x4] + ret + +8: mov x(tmp0), x0 + mov x(tmp1), x1 +0: LDXP x0, x1, [x4] + cmp x0, x(tmp0) + ccmp x1, x(tmp1), #0, eq + bne 1f + STXP w(tmp2), x(tmp0), x(tmp1), [x4] + cbnz w(tmp2), 0b +1: ret + +#endif + +ENDFN NAME(cas) +#endif + +#ifdef L_swp +#define SWP glue4(swp, A, L, S) + +STARTFN NAME(swp) + JUMP_IF_NOT_LSE 8f + + SWP s(0), s(0), [x1] + ret + +8: mov s(tmp0), s(0) +0: LDXR s(0), [x1] + STXR w(tmp1), s(tmp0), [x1] + cbnz w(tmp1), 0b + ret + +ENDFN NAME(swp) +#endif + +#if defined(L_ldadd) || defined(L_ldclr) \ + || defined(L_ldeor) || defined(L_ldset) + +#ifdef L_ldadd +#define LDNM ldadd +#define OP add +#elif defined(L_ldclr) +#define LDNM ldclr +#define OP bic +#elif defined(L_ldeor) +#define LDNM ldeor +#define OP eor +#elif defined(L_ldset) +#define LDNM ldset +#define OP orr +#else +#error +#endif +#define LDOP glue4(LDNM, A, L, S) + +STARTFN NAME(LDNM) + JUMP_IF_NOT_LSE 8f + + LDOP s(0), s(0), [x1] + ret + +8: mov s(tmp0), s(0) +0: LDXR s(0), [x1] + OP s(tmp1), s(0), s(tmp0) + STXR w(tmp1), s(tmp1), [x1] + cbnz w(tmp1), 0b + ret + +ENDFN NAME(LDNM) +#endif diff --git a/libgcc/config/aarch64/t-lse b/libgcc/config/aarch64/t-lse new file mode 100644 index 00000000000..c7f4223cd45 --- /dev/null +++ b/libgcc/config/aarch64/t-lse @@ -0,0 +1,44 @@ +# Out-of-line LSE atomics for AArch64 architecture. +# Copyright (C) 2018 Free Software Foundation, Inc. +# Contributed by Linaro Ltd. +# +# This file is part of GCC. +# +# GCC is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3, or (at your option) +# any later version. +# +# GCC is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GCC; see the file COPYING3. If not see +# . + +# Compare-and-swap has 5 sizes and 4 memory models. +S0 := $(foreach s, 1 2 4 8 16, $(addsuffix _$(s), cas)) +O0 := $(foreach m, 1 2 3 4, $(addsuffix _$(m)$(objext), $(S0))) + +# Swap, Load-and-operate have 4 sizes and 4 memory models +S1 := $(foreach s, 1 2 4 8, $(addsuffix _$(s), swp ldadd ldclr ldeor ldset)) +O1 := $(foreach m, 1 2 3 4, $(addsuffix _$(m)$(objext), $(S1))) + +LSE_OBJS := $(O0) $(O1) + +libgcc-objects += $(LSE_OBJS) lse-init$(objext) + +empty = +space = $(empty) $(empty) +PAT_SPLIT = $(subst _,$(space),$(*F)) +PAT_BASE = $(word 1,$(PAT_SPLIT)) +PAT_N = $(word 2,$(PAT_SPLIT)) +PAT_M = $(word 3,$(PAT_SPLIT)) + +lse-init$(objext): $(srcdir)/config/aarch64/lse-init.c + $(gcc_compile) -c $< + +$(LSE_OBJS): $(srcdir)/config/aarch64/lse.S + $(gcc_compile) -DL_$(PAT_BASE) -DSIZE=$(PAT_N) -DMODEL=$(PAT_M) -c $< From patchwork Wed Sep 18 01:58:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173937 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp1871540ill; Tue, 17 Sep 2019 18:59:56 -0700 (PDT) X-Google-Smtp-Source: APXvYqy5Es274uuKtZY7Hiaj+4CYEfBM8MXJ9eVZ1QL5E9kHAZMrQMtnSepoDzE9S2JAgA5YJ71J X-Received: by 2002:a05:6402:1681:: with SMTP id a1mr7876912edv.218.1568771995941; Tue, 17 Sep 2019 18:59:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568771995; cv=none; d=google.com; s=arc-20160816; b=xDGzMpCsSJ75gOlH6uCzBEXGdYYbDobpsLSKCK9EnqI3xPheJ+dhVsoDQ2/m8FUUdh Zb/EJ+st8yzI2rExyJpVneUMoYEeaZnO7zleoQLVr1Yv1z1aXU5hGM2Ta/SRKKQHJ3lI KDdsdfRS4jGAkZUJdX1oY7jADn08ZHcvmHJCmYjmuAn0ljGy1KS1W/fLRv06V5iEtTPK do7/CizLcSN/UlPl60nGK75Wg4+4RGqjQBUZsZTSVRbm4cdw4hK/Ux1oNJQgigI80P2A Y4TjazFKDfi1ybFYDazVfOuoR2KyYr5uNJZmy3V7hwHNvoL+tlX+iuWYOLl2CskzQ+YX IbOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature; bh=18/BRjEy0HxWg9UwIH1eHtf1Qyx7PzRFl2k+DcLz8zw=; b=kxj3iI1yVyNtNUtEy+YoLUuxxdAM2DxPYL31lDJufpJE0IDJFW94dsHO3jr34Zbm5v e/da4v1wDNgDkISSv8eOv5VSGPEpxJzK05Efu2AMpijzb1vOidgCjHeperdddRxpkl67 GGSO/Ej9j8pQAyGzdFnWf/EbivPBAhJditUT6GFrhiUtkecILpnKYZhDlNZygAzMOXY6 A9Gu2R9PDkgYCLxEz0Y/M0FbHLF85t5L20P1F53Sh8/vpC9C6RkEow6B9JC0wvMYJteV 6xewB0YbC4bD9plIP206DVNzQ8jMtd5CjAwvbgnNZ01ZzJNimSgrBBey1IN8yzBh9CGd AVYw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=ckQq+NWh; dkim=pass header.i=@linaro.org header.s=google header.b=mF3PY8vi; spf=pass (google.com: domain of gcc-patches-return-509149-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509149-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id x14si2338417edq.49.2019.09.17.18.59.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Sep 2019 18:59:55 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-509149-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=ckQq+NWh; dkim=pass header.i=@linaro.org header.s=google header.b=mF3PY8vi; spf=pass (google.com: domain of gcc-patches-return-509149-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509149-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; q=dns; s= default; b=l5v85ONojz6PELs129aqiyaQ6YtzZmiEdagRGU9as8t8jBYS/u7j9 7lNehflre6AILRbvelUA7CHgD/BJBu/go4+c6QtFLrzmjPhaINRJdEFo3ZjUUY2H 387P4Y9VYtY84AcCzZNehXJcWHodO0sS1UaXW+yWMcrSB+gU0tM354= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; s= default; bh=HHcpxF0w2YYqrqiklBMDDNfhGzc=; b=ckQq+NWhqCpgndfq/UNY ORiDHq8bSyVrL03evf2tIrAtzkxTqtHYNCnqHJBqxEAiJ0tA2B6TPMPqIZ3Zz3Ak TzBPWDdrOn2Xezubso1+93YBVUrHt+GxEHA/z7HmoFUG87pbSiEEiAitzpX/RH86 gRwPIscxCN7kZ7bKjZn0koU= Received: (qmail 121640 invoked by alias); 18 Sep 2019 01:58:34 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 121555 invoked by uid 89); 18 Sep 2019 01:58:33 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-25.7 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.1 spammy=NAMES, PLUS, gol, force_reg X-HELO: mail-pg1-f176.google.com Received: from mail-pg1-f176.google.com (HELO mail-pg1-f176.google.com) (209.85.215.176) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 18 Sep 2019 01:58:29 +0000 Received: by mail-pg1-f176.google.com with SMTP id c17so3046302pgg.4 for ; Tue, 17 Sep 2019 18:58:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=18/BRjEy0HxWg9UwIH1eHtf1Qyx7PzRFl2k+DcLz8zw=; b=mF3PY8vitPP1d1iFxrNJebZej4FtTSjJFtNLaxyEbOHe9wgzhg9kwoEY5TjxhqMGgu WR5N08XXw4SOaaXDJhPNVoDf/EsFeNl0pbwarQuNyKkNFmbP0vMZqOtRLQqJcUWnO5I5 AiHsOBY5d501xa0pK90ZhpmxrnfqPjAEaOWe1G6ui+Sy+y1nmKPA1g8fA/VTOtbUvBNC pM3rOOGkfqZD+7b8GXR9KSlqU43maX9LSjbg2en7Cp6Ynz8MgzQT2QeKUCoeDz2qodri mr1s1IVu71wcAWC9hwsPym3Yeg0QgyHMiIdq4jLpOn+n+/aRSucdK6/3EnqzkVwo990r 5Biw== Return-Path: Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id d10sm6614306pfn.50.2019.09.17.18.58.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 18:58:26 -0700 (PDT) From: Richard Henderson To: gcc-patches@gcc.gnu.org Cc: Wilco.Dijkstra@arm.com, kyrylo.tkachov@foss.arm.com, Marcus.Shawcroft@arm.com, James.Greenhalgh@arm.com Subject: [PATCH, AArch64 v4 5/6] aarch64: Implement -moutline-atomics Date: Tue, 17 Sep 2019 18:58:16 -0700 Message-Id: <20190918015817.24408-6-richard.henderson@linaro.org> In-Reply-To: <20190918015817.24408-1-richard.henderson@linaro.org> References: <20190918015817.24408-1-richard.henderson@linaro.org> * config/aarch64/aarch64.opt (-moutline-atomics): New. * config/aarch64/aarch64.c (aarch64_atomic_ool_func): New. (aarch64_ool_cas_names, aarch64_ool_swp_names): New. (aarch64_ool_ldadd_names, aarch64_ool_ldset_names): New. (aarch64_ool_ldclr_names, aarch64_ool_ldeor_names): New. (aarch64_expand_compare_and_swap): Honor TARGET_OUTLINE_ATOMICS. * config/aarch64/atomics.md (atomic_exchange): Likewise. (atomic_): Likewise. (atomic_fetch_): Likewise. (atomic__fetch): Likewise. testsuite/ * gcc.target/aarch64/atomic-op-acq_rel.c: Use -mno-outline-atomics. * gcc.target/aarch64/atomic-comp-swap-release-acquire.c: Likewise. * gcc.target/aarch64/atomic-op-acquire.c: Likewise. * gcc.target/aarch64/atomic-op-char.c: Likewise. * gcc.target/aarch64/atomic-op-consume.c: Likewise. * gcc.target/aarch64/atomic-op-imm.c: Likewise. * gcc.target/aarch64/atomic-op-int.c: Likewise. * gcc.target/aarch64/atomic-op-long.c: Likewise. * gcc.target/aarch64/atomic-op-relaxed.c: Likewise. * gcc.target/aarch64/atomic-op-release.c: Likewise. * gcc.target/aarch64/atomic-op-seq_cst.c: Likewise. * gcc.target/aarch64/atomic-op-short.c: Likewise. * gcc.target/aarch64/atomic_cmp_exchange_zero_reg_1.c: Likewise. * gcc.target/aarch64/atomic_cmp_exchange_zero_strong_1.c: Likewise. * gcc.target/aarch64/sync-comp-swap.c: Likewise. * gcc.target/aarch64/sync-op-acquire.c: Likewise. * gcc.target/aarch64/sync-op-full.c: Likewise. --- gcc/config/aarch64/aarch64-protos.h | 13 +++ gcc/config/aarch64/aarch64.c | 87 +++++++++++++++++ .../atomic-comp-swap-release-acquire.c | 2 +- .../gcc.target/aarch64/atomic-op-acq_rel.c | 2 +- .../gcc.target/aarch64/atomic-op-acquire.c | 2 +- .../gcc.target/aarch64/atomic-op-char.c | 2 +- .../gcc.target/aarch64/atomic-op-consume.c | 2 +- .../gcc.target/aarch64/atomic-op-imm.c | 2 +- .../gcc.target/aarch64/atomic-op-int.c | 2 +- .../gcc.target/aarch64/atomic-op-long.c | 2 +- .../gcc.target/aarch64/atomic-op-relaxed.c | 2 +- .../gcc.target/aarch64/atomic-op-release.c | 2 +- .../gcc.target/aarch64/atomic-op-seq_cst.c | 2 +- .../gcc.target/aarch64/atomic-op-short.c | 2 +- .../aarch64/atomic_cmp_exchange_zero_reg_1.c | 2 +- .../atomic_cmp_exchange_zero_strong_1.c | 2 +- .../gcc.target/aarch64/sync-comp-swap.c | 2 +- .../gcc.target/aarch64/sync-op-acquire.c | 2 +- .../gcc.target/aarch64/sync-op-full.c | 2 +- gcc/config/aarch64/aarch64.opt | 3 + gcc/config/aarch64/atomics.md | 94 +++++++++++++++++-- gcc/doc/invoke.texi | 16 +++- 22 files changed, 221 insertions(+), 26 deletions(-) -- 2.17.1 diff --git a/gcc/config/aarch64/aarch64-protos.h b/gcc/config/aarch64/aarch64-protos.h index c4b73d26df6..1c1aac7201a 100644 --- a/gcc/config/aarch64/aarch64-protos.h +++ b/gcc/config/aarch64/aarch64-protos.h @@ -696,4 +696,17 @@ poly_uint64 aarch64_regmode_natural_size (machine_mode); bool aarch64_high_bits_all_ones_p (HOST_WIDE_INT); +struct atomic_ool_names +{ + const char *str[5][4]; +}; + +rtx aarch64_atomic_ool_func(machine_mode mode, rtx model_rtx, + const atomic_ool_names *names); +extern const atomic_ool_names aarch64_ool_swp_names; +extern const atomic_ool_names aarch64_ool_ldadd_names; +extern const atomic_ool_names aarch64_ool_ldset_names; +extern const atomic_ool_names aarch64_ool_ldclr_names; +extern const atomic_ool_names aarch64_ool_ldeor_names; + #endif /* GCC_AARCH64_PROTOS_H */ diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index b937514e6f8..56a4a47db73 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -16867,6 +16867,82 @@ aarch64_emit_unlikely_jump (rtx insn) add_reg_br_prob_note (jump, profile_probability::very_unlikely ()); } +/* We store the names of the various atomic helpers in a 5x4 array. + Return the libcall function given MODE, MODEL and NAMES. */ + +rtx +aarch64_atomic_ool_func(machine_mode mode, rtx model_rtx, + const atomic_ool_names *names) +{ + memmodel model = memmodel_base (INTVAL (model_rtx)); + int mode_idx, model_idx; + + switch (mode) + { + case E_QImode: + mode_idx = 0; + break; + case E_HImode: + mode_idx = 1; + break; + case E_SImode: + mode_idx = 2; + break; + case E_DImode: + mode_idx = 3; + break; + case E_TImode: + mode_idx = 4; + break; + default: + gcc_unreachable (); + } + + switch (model) + { + case MEMMODEL_RELAXED: + model_idx = 0; + break; + case MEMMODEL_CONSUME: + case MEMMODEL_ACQUIRE: + model_idx = 1; + break; + case MEMMODEL_RELEASE: + model_idx = 2; + break; + case MEMMODEL_ACQ_REL: + case MEMMODEL_SEQ_CST: + model_idx = 3; + break; + default: + gcc_unreachable (); + } + + return init_one_libfunc_visibility (names->str[mode_idx][model_idx], + VISIBILITY_HIDDEN); +} + +#define DEF0(B, N) \ + { "__aarch64_" #B #N "_relax", \ + "__aarch64_" #B #N "_acq", \ + "__aarch64_" #B #N "_rel", \ + "__aarch64_" #B #N "_acq_rel" } + +#define DEF4(B) DEF0(B, 1), DEF0(B, 2), DEF0(B, 4), DEF0(B, 8), \ + { NULL, NULL, NULL, NULL } +#define DEF5(B) DEF0(B, 1), DEF0(B, 2), DEF0(B, 4), DEF0(B, 8), DEF0(B, 16) + +static const atomic_ool_names aarch64_ool_cas_names = { { DEF5(cas) } }; +const atomic_ool_names aarch64_ool_swp_names = { { DEF4(swp) } }; +const atomic_ool_names aarch64_ool_ldadd_names = { { DEF4(ldadd) } }; +const atomic_ool_names aarch64_ool_ldset_names = { { DEF4(ldset) } }; +const atomic_ool_names aarch64_ool_ldclr_names = { { DEF4(ldclr) } }; +const atomic_ool_names aarch64_ool_ldeor_names = { { DEF4(ldeor) } }; + +#undef DEF0 +#undef DEF4 +#undef DEF5 + /* Expand a compare and swap pattern. */ void @@ -16913,6 +16989,17 @@ aarch64_expand_compare_and_swap (rtx operands[]) newval, mod_s)); cc_reg = aarch64_gen_compare_reg_maybe_ze (NE, rval, oldval, mode); } + else if (TARGET_OUTLINE_ATOMICS) + { + /* Oldval must satisfy compare afterward. */ + if (!aarch64_plus_operand (oldval, mode)) + oldval = force_reg (mode, oldval); + rtx func = aarch64_atomic_ool_func (mode, mod_s, &aarch64_ool_cas_names); + rval = emit_library_call_value (func, NULL_RTX, LCT_NORMAL, r_mode, + oldval, mode, newval, mode, + XEXP (mem, 0), Pmode); + cc_reg = aarch64_gen_compare_reg_maybe_ze (NE, rval, oldval, mode); + } else { /* The oldval predicate varies by mode. Test it and force to reg. */ diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-comp-swap-release-acquire.c b/gcc/testsuite/gcc.target/aarch64/atomic-comp-swap-release-acquire.c index 49ca5d0d09c..a828a72aa75 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-comp-swap-release-acquire.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-comp-swap-release-acquire.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2 -fno-ipa-icf" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -fno-ipa-icf -mno-outline-atomics" } */ #include "atomic-comp-swap-release-acquire.x" diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-acq_rel.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-acq_rel.c index 74f26348e42..6823ce381b2 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-acq_rel.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-acq_rel.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "atomic-op-acq_rel.x" diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-acquire.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-acquire.c index 66c1b1efe20..87937de378a 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-acquire.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-acquire.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "atomic-op-acquire.x" diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-char.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-char.c index c09d0434ecf..60955e57da3 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-char.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-char.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "atomic-op-char.x" diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-consume.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-consume.c index 5783ab84f5c..16cb11aeeaf 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-consume.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-consume.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "atomic-op-consume.x" diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-imm.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-imm.c index 18b8f0b04e9..bcab4e481e3 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-imm.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-imm.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ int v = 0; diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-int.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-int.c index 8520f0839ba..040e4a8d168 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-int.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-int.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "atomic-op-int.x" diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-long.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-long.c index d011f8c5ce2..fc88b92cd3e 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-long.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-long.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ long v = 0; diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-relaxed.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-relaxed.c index ed96bfdb978..503d62b0280 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-relaxed.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-relaxed.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "atomic-op-relaxed.x" diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-release.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-release.c index fc4be17de89..efe14aea7e4 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-release.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-release.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "atomic-op-release.x" diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-seq_cst.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-seq_cst.c index 613000fe490..09973bf82ba 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-seq_cst.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-seq_cst.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "atomic-op-seq_cst.x" diff --git a/gcc/testsuite/gcc.target/aarch64/atomic-op-short.c b/gcc/testsuite/gcc.target/aarch64/atomic-op-short.c index e82c8118ece..e1dcebb0f89 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic-op-short.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic-op-short.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "atomic-op-short.x" diff --git a/gcc/testsuite/gcc.target/aarch64/atomic_cmp_exchange_zero_reg_1.c b/gcc/testsuite/gcc.target/aarch64/atomic_cmp_exchange_zero_reg_1.c index f2a21ddf2e1..29246979bfb 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic_cmp_exchange_zero_reg_1.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic_cmp_exchange_zero_reg_1.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-O2 -march=armv8-a+nolse" } */ +/* { dg-options "-O2 -march=armv8-a+nolse -mno-outline-atomics" } */ /* { dg-skip-if "" { *-*-* } { "-mcpu=*" } { "" } } */ int diff --git a/gcc/testsuite/gcc.target/aarch64/atomic_cmp_exchange_zero_strong_1.c b/gcc/testsuite/gcc.target/aarch64/atomic_cmp_exchange_zero_strong_1.c index 8d2ae67dfbe..6daf9b08f5a 100644 --- a/gcc/testsuite/gcc.target/aarch64/atomic_cmp_exchange_zero_strong_1.c +++ b/gcc/testsuite/gcc.target/aarch64/atomic_cmp_exchange_zero_strong_1.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-O2 -march=armv8-a+nolse" } */ +/* { dg-options "-O2 -march=armv8-a+nolse -mno-outline-atomics" } */ /* { dg-skip-if "" { *-*-* } { "-mcpu=*" } { "" } } */ int diff --git a/gcc/testsuite/gcc.target/aarch64/sync-comp-swap.c b/gcc/testsuite/gcc.target/aarch64/sync-comp-swap.c index e571b2f13b3..f56415f3354 100644 --- a/gcc/testsuite/gcc.target/aarch64/sync-comp-swap.c +++ b/gcc/testsuite/gcc.target/aarch64/sync-comp-swap.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2 -fno-ipa-icf" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -fno-ipa-icf -mno-outline-atomics" } */ #include "sync-comp-swap.x" diff --git a/gcc/testsuite/gcc.target/aarch64/sync-op-acquire.c b/gcc/testsuite/gcc.target/aarch64/sync-op-acquire.c index 357bf1be3b2..39b3144aa36 100644 --- a/gcc/testsuite/gcc.target/aarch64/sync-op-acquire.c +++ b/gcc/testsuite/gcc.target/aarch64/sync-op-acquire.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "sync-op-acquire.x" diff --git a/gcc/testsuite/gcc.target/aarch64/sync-op-full.c b/gcc/testsuite/gcc.target/aarch64/sync-op-full.c index c6ba1629965..6b8b2043f40 100644 --- a/gcc/testsuite/gcc.target/aarch64/sync-op-full.c +++ b/gcc/testsuite/gcc.target/aarch64/sync-op-full.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-march=armv8-a+nolse -O2" } */ +/* { dg-options "-march=armv8-a+nolse -O2 -mno-outline-atomics" } */ #include "sync-op-full.x" diff --git a/gcc/config/aarch64/aarch64.opt b/gcc/config/aarch64/aarch64.opt index 55d466068b8..865b6a6d8ca 100644 --- a/gcc/config/aarch64/aarch64.opt +++ b/gcc/config/aarch64/aarch64.opt @@ -255,3 +255,6 @@ user-land code. TargetVariable long aarch64_stack_protector_guard_offset = 0 +moutline-atomics +Target Report Mask(OUTLINE_ATOMICS) Save +Generate local calls to out-of-line atomic operations. diff --git a/gcc/config/aarch64/atomics.md b/gcc/config/aarch64/atomics.md index f8bdd048b37..2e59b868420 100644 --- a/gcc/config/aarch64/atomics.md +++ b/gcc/config/aarch64/atomics.md @@ -186,16 +186,27 @@ (match_operand:SI 3 "const_int_operand")] "" { - rtx (*gen) (rtx, rtx, rtx, rtx); - /* Use an atomic SWP when available. */ if (TARGET_LSE) - gen = gen_aarch64_atomic_exchange_lse; + { + emit_insn (gen_aarch64_atomic_exchange_lse + (operands[0], operands[1], operands[2], operands[3])); + } + else if (TARGET_OUTLINE_ATOMICS) + { + machine_mode mode = mode; + rtx func = aarch64_atomic_ool_func (mode, operands[3], + &aarch64_ool_swp_names); + rtx rval = emit_library_call_value (func, operands[0], LCT_NORMAL, + mode, operands[2], mode, + XEXP (operands[1], 0), Pmode); + emit_move_insn (operands[0], rval); + } else - gen = gen_aarch64_atomic_exchange; - - emit_insn (gen (operands[0], operands[1], operands[2], operands[3])); - + { + emit_insn (gen_aarch64_atomic_exchange + (operands[0], operands[1], operands[2], operands[3])); + } DONE; } ) @@ -280,6 +291,39 @@ } operands[1] = force_reg (mode, operands[1]); } + else if (TARGET_OUTLINE_ATOMICS) + { + const atomic_ool_names *names; + switch () + { + case MINUS: + operands[1] = expand_simple_unop (mode, NEG, operands[1], + NULL, 1); + /* fallthru */ + case PLUS: + names = &aarch64_ool_ldadd_names; + break; + case IOR: + names = &aarch64_ool_ldset_names; + break; + case XOR: + names = &aarch64_ool_ldeor_names; + break; + case AND: + operands[1] = expand_simple_unop (mode, NOT, operands[1], + NULL, 1); + names = &aarch64_ool_ldclr_names; + break; + default: + gcc_unreachable (); + } + machine_mode mode = mode; + rtx func = aarch64_atomic_ool_func (mode, operands[2], names); + emit_library_call_value (func, NULL_RTX, LCT_NORMAL, mode, + operands[1], mode, + XEXP (operands[0], 0), Pmode); + DONE; + } else gen = gen_aarch64_atomic_; @@ -405,6 +449,40 @@ } operands[2] = force_reg (mode, operands[2]); } + else if (TARGET_OUTLINE_ATOMICS) + { + const atomic_ool_names *names; + switch () + { + case MINUS: + operands[2] = expand_simple_unop (mode, NEG, operands[2], + NULL, 1); + /* fallthru */ + case PLUS: + names = &aarch64_ool_ldadd_names; + break; + case IOR: + names = &aarch64_ool_ldset_names; + break; + case XOR: + names = &aarch64_ool_ldeor_names; + break; + case AND: + operands[2] = expand_simple_unop (mode, NOT, operands[2], + NULL, 1); + names = &aarch64_ool_ldclr_names; + break; + default: + gcc_unreachable (); + } + machine_mode mode = mode; + rtx func = aarch64_atomic_ool_func (mode, operands[3], names); + rtx rval = emit_library_call_value (func, operands[0], LCT_NORMAL, mode, + operands[2], mode, + XEXP (operands[1], 0), Pmode); + emit_move_insn (operands[0], rval); + DONE; + } else gen = gen_aarch64_atomic_fetch_; @@ -494,7 +572,7 @@ { /* Use an atomic load-operate instruction when possible. In this case we will re-compute the result from the original mem value. */ - if (TARGET_LSE) + if (TARGET_LSE || TARGET_OUTLINE_ATOMICS) { rtx tmp = gen_reg_rtx (mode); operands[2] = force_reg (mode, operands[2]); diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index 0e3693598e7..900fda1efb2 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -643,7 +643,8 @@ Objective-C and Objective-C++ Dialects}. -march=@var{name} -mcpu=@var{name} -mtune=@var{name} @gol -moverride=@var{string} -mverbose-cost-dump @gol -mstack-protector-guard=@var{guard} -mstack-protector-guard-reg=@var{sysreg} @gol --mstack-protector-guard-offset=@var{offset} -mtrack-speculation } +-mstack-protector-guard-offset=@var{offset} -mtrack-speculation @gol +-moutline-atomics } @emph{Adapteva Epiphany Options} @gccoptlist{-mhalf-reg-file -mprefer-short-insn-regs @gol @@ -15874,6 +15875,19 @@ be used by the compiler when expanding calls to @code{__builtin_speculation_safe_copy} to permit a more efficient code sequence to be generated. +@item -moutline-atomics +@itemx -mno-outline-atomics +Enable or disable calls to out-of-line helpers to implement atomic operations. +These helpers will, at runtime, determine if ARMv8.1-Atomics instructions +should be used; if not, they will use the load/store-exclusive instructions +that are present in the base ARMv8.0 ISA. + +This option is only applicable when compiling for the base ARMv8.0 +instruction set. If using a later revision, e.g. @option{-march=armv8.1-a} +or @option{-march=armv8-a+lse}, the ARMv8.1-Atomics instructions will be +used directly. The same applies when using @option{-mcpu=} when the +selected cpu supports the @samp{lse} feature. + @item -march=@var{name} @opindex march Specify the name of the target architecture and, optionally, one or From patchwork Wed Sep 18 01:58:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173935 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp1871243ill; Tue, 17 Sep 2019 18:59:29 -0700 (PDT) X-Google-Smtp-Source: APXvYqyd4dTHTj3t2BditBq0h6eawo5cfNJrqQTQiHPUBNJ4cohE7sSRzqeTAfZlLcFt/tb4ykxk X-Received: by 2002:a50:ee92:: with SMTP id f18mr7833649edr.253.1568771969228; Tue, 17 Sep 2019 18:59:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568771969; cv=none; d=google.com; s=arc-20160816; b=AY/s+x37CeKoeFS8EncPRHdV5lb4G2l1Wcpss9cotDfc29g17Pj3TbEqYrFyG5qrP3 ajzIw04bZ82wSpXs/hjdAD+UAP3IFHQG8j/r7K2ixtlHoEGidOK2ZePLVwlWXWXGwLJf XmsweErbHclWbTreiMYGFW02Zsb6lroBHKFI3UGMGk4Ll4bbeoScisxUvskHd82qv9Cc n8NpozztkYPrfqMym+ZU9/JNVnlZYSEuMvGo4GHl4BzviJRcBp1h/vEBnotxNInlauCt lIORhYtfT1ain971dmI6R0QYSyRfI42005IptjiFY7hjgVR+QrGt4LfNdKH5wEWDNasf NckA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature; bh=duu9sBVEKHzrPUuhKCsMXazwNS7oUAgJ9EfTCO8Ioao=; b=Q3Q0uDlWM9+dBTenvAjyVvHuyJVTUxl1mA0AFA9ospK2f0MYrXKmkBbCJ2l81gbUY/ MI6Znf3OEG44j+7kIMR2iAz8Hg9Ap5Wpni2c2KgcbXOdCN5X4y7yJ6TvZGWA7t3qoDIP A1c1uimnV7zV7HwsY1s+Gwi04d3mIsnkpOm3NdIsqk3RVfg+5ElzcIlxRFHAiBlP5XzW DkqrjCwSBEL5vWt0TZ6hKw+M3V9T/8Xkx1MjxnJhXPHrrPaZ4WUDeg/nBQvCP2OLd165 b6lo2hTvDIHPQZGTqn0H8w15dQgHWWIMK/sF4ww0UDI08NGGsi5uT0lJvAuELxyEx/dD 4ppA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=T6w3yzhw; dkim=pass header.i=@linaro.org header.s=google header.b="nuY/cPjy"; spf=pass (google.com: domain of gcc-patches-return-509147-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509147-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id g21si1645081edq.289.2019.09.17.18.59.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Sep 2019 18:59:29 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-509147-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=T6w3yzhw; dkim=pass header.i=@linaro.org header.s=google header.b="nuY/cPjy"; spf=pass (google.com: domain of gcc-patches-return-509147-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-509147-patch=linaro.org@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; q=dns; s= default; b=CzUtoBo3t4oxIaPOwDbDn9B4Lor5V6oRwFfTf9QOmPoOwQf37DojM 4bDbDiLY9OOq5ipth7W9f40fd6GSfhdMmpkNhncw0bmFcFteA9XdntSX4BgJat46 wRJvbGsq5jtuasw+OxPPE5UN16DsjWGxU3p7mq35PRcPa+mCBjyEcY= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references; s= default; bh=Aelddb1bnTGtevk9tCSsITp6NIA=; b=T6w3yzhwkIA1hvIAwDTi b8ShaCA8tYcLyaNboxX0Er+8dajld0y3koRLRUdXxod9jzptFstBRJZ1atg/cl2s NnptOeCMutAgLhRQKrbN2WfoTWG2QFO8QNGWn7svrGgTxUPliCvIdNMs+qjOFtjW Joqdh19yr8BFpAQPMvUqPug= Received: (qmail 121295 invoked by alias); 18 Sep 2019 01:58:32 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 121182 invoked by uid 89); 18 Sep 2019 01:58:31 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-25.4 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.1 spammy= X-HELO: mail-pf1-f182.google.com Received: from mail-pf1-f182.google.com (HELO mail-pf1-f182.google.com) (209.85.210.182) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 18 Sep 2019 01:58:30 +0000 Received: by mail-pf1-f182.google.com with SMTP id i1so3301231pfa.6 for ; Tue, 17 Sep 2019 18:58:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=duu9sBVEKHzrPUuhKCsMXazwNS7oUAgJ9EfTCO8Ioao=; b=nuY/cPjy+FpEv2QELSeDQ+TNVl/lFn9GQIjYQchpbLvc+j71Ti4u6Q00QnIcAQlGt+ yG93yvHu9HeO3jCBj7wKWYTFrx4yzf7OsXHK5y1FpnwnYMoOTbeVDpz3WlUwUViLlxaG wi+m4J2rDQ6wtyn8D3BQHH4aVdEA5FrUM2qxojSJCPLNi2ZCNSFoNgCSW1uf4tzzTbb8 ai/ckOz8XeiDqInpuKMBaNe9maXycMvu65bTbxdIuSYH48u6IW1PwZoekAOpqbGFXq6b ka/QNFgK9agBMRhjbXGs5TiAW6YWy18/f4IpDSkgOO9qqOk6gFNn18chd3rseQ3u2i2F JL7Q== Return-Path: Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id d10sm6614306pfn.50.2019.09.17.18.58.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 18:58:27 -0700 (PDT) From: Richard Henderson To: gcc-patches@gcc.gnu.org Cc: Wilco.Dijkstra@arm.com, kyrylo.tkachov@foss.arm.com, Marcus.Shawcroft@arm.com, James.Greenhalgh@arm.com Subject: [PATCH, AArch64 v4 6/6] TESTING: Enable -moutline-atomics by default Date: Tue, 17 Sep 2019 18:58:17 -0700 Message-Id: <20190918015817.24408-7-richard.henderson@linaro.org> In-Reply-To: <20190918015817.24408-1-richard.henderson@linaro.org> References: <20190918015817.24408-1-richard.henderson@linaro.org> --- gcc/common/config/aarch64/aarch64-common.c | 6 ++++-- gcc/config/aarch64/aarch64.c | 6 ++++-- 2 files changed, 8 insertions(+), 4 deletions(-) -- 2.17.1 diff --git a/gcc/common/config/aarch64/aarch64-common.c b/gcc/common/config/aarch64/aarch64-common.c index 07c03253951..2bbf454eea9 100644 --- a/gcc/common/config/aarch64/aarch64-common.c +++ b/gcc/common/config/aarch64/aarch64-common.c @@ -32,9 +32,11 @@ #include "diagnostic.h" #include "params.h" -#ifdef TARGET_BIG_ENDIAN_DEFAULT #undef TARGET_DEFAULT_TARGET_FLAGS -#define TARGET_DEFAULT_TARGET_FLAGS (MASK_BIG_END) +#ifdef TARGET_BIG_ENDIAN_DEFAULT +#define TARGET_DEFAULT_TARGET_FLAGS (MASK_BIG_END | MASK_OUTLINE_ATOMICS) +#else +#define TARGET_DEFAULT_TARGET_FLAGS (MASK_OUTLINE_ATOMICS) #endif #undef TARGET_HANDLE_OPTION diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index 56a4a47db73..ca4363e7831 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -20535,9 +20535,11 @@ aarch64_run_selftests (void) #undef TARGET_C_MODE_FOR_SUFFIX #define TARGET_C_MODE_FOR_SUFFIX aarch64_c_mode_for_suffix -#ifdef TARGET_BIG_ENDIAN_DEFAULT #undef TARGET_DEFAULT_TARGET_FLAGS -#define TARGET_DEFAULT_TARGET_FLAGS (MASK_BIG_END) +#ifdef TARGET_BIG_ENDIAN_DEFAULT +#define TARGET_DEFAULT_TARGET_FLAGS (MASK_BIG_END | MASK_OUTLINE_ATOMICS) +#else +#define TARGET_DEFAULT_TARGET_FLAGS (MASK_OUTLINE_ATOMICS) #endif #undef TARGET_CLASS_MAX_NREGS