From patchwork Fri Oct 3 15:11:26 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Newton X-Patchwork-Id: 38331 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ee0-f70.google.com (mail-ee0-f70.google.com [74.125.83.70]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 801702057C for ; Fri, 3 Oct 2014 15:12:18 +0000 (UTC) Received: by mail-ee0-f70.google.com with SMTP id c13sf949100eek.5 for ; Fri, 03 Oct 2014 08:12:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:mailing-list :precedence:list-id:list-unsubscribe:list-subscribe:list-archive :list-post:list-help:sender:delivered-to:from:to:subject:date :message-id:in-reply-to:references:x-original-sender :x-original-authentication-results; bh=OP5peEof8C2L01Dx4Tvnc+12gpNmW/lVkltL3CgvkmQ=; b=Y9FylMSh//QGuqihApTBQfYvnYlrcFLMlMlIxPRRoyv97rz/WiuM1+nNWwb+e4jcz4 s3/Qohytx23lISB6Kvo+ts+AJAGcbyWcjd9SNQexESFZvHFTejVTMQWg9vTH2pkOaxjb h+ibF4Eoo0VFQErZpu2qmnmgn6gGPmxhZaa2/+JixmyrqVZe1T9GhzWxR5UitVMGI7yM wW/Wf96OT6AbMo8kAAzo1TMqjZ09go2B7pySantsZgvekyCy9yYg355deQWQCpg4bYLF XceWEoiVxGQEYzuovFsDZmr5z/K+nnEsT0JpKJ/qjljLbpnnRmvdLriAECDAHcISRTcN pV1w== X-Gm-Message-State: ALoCoQnhd1VZ0YxoCbtvYj2za/ojYR5hjuxN2nljCXyEV2a4L+sCBXy2EfAGvvkczCqBeVsMpqfq X-Received: by 10.152.4.4 with SMTP id g4mr1101257lag.2.1412349137128; Fri, 03 Oct 2014 08:12:17 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.19.42 with SMTP id b10ls337362lae.28.gmail; Fri, 03 Oct 2014 08:12:17 -0700 (PDT) X-Received: by 10.112.151.99 with SMTP id up3mr6272335lbb.45.1412349136986; Fri, 03 Oct 2014 08:12:16 -0700 (PDT) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [2a00:1450:4010:c03::231]) by mx.google.com with ESMTPS id to5si11602227lbb.93.2014.10.03.08.12.16 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 03 Oct 2014 08:12:16 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 2a00:1450:4010:c03::231 as permitted sender) client-ip=2a00:1450:4010:c03::231; Received: by mail-la0-f49.google.com with SMTP id q1so1226246lam.36 for ; Fri, 03 Oct 2014 08:12:16 -0700 (PDT) X-Received: by 10.152.22.137 with SMTP id d9mr6573852laf.29.1412349136579; Fri, 03 Oct 2014 08:12:16 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.130.169 with SMTP id of9csp245996lbb; Fri, 3 Oct 2014 08:12:15 -0700 (PDT) X-Received: by 10.70.1.111 with SMTP id 15mr1581567pdl.28.1412349135032; Fri, 03 Oct 2014 08:12:15 -0700 (PDT) Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id rl12si7325844pab.232.2014.10.03.08.12.14 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Oct 2014 08:12:15 -0700 (PDT) Received-SPF: pass (google.com: domain of libc-alpha-return-53276-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Received: (qmail 3523 invoked by alias); 3 Oct 2014 15:11:42 -0000 Mailing-List: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org Precedence: list List-Id: List-Unsubscribe: , List-Subscribe: List-Archive: List-Post: , List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 3435 invoked by uid 89); 3 Oct 2014 15:11:41 -0000 X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.1 required=5.0 tests=AWL, BAYES_00, RCVD_IN_DNSWL_LOW, SPF_PASS autolearn=ham version=3.3.2 X-HELO: mail-wi0-f171.google.com X-Received: by 10.194.120.101 with SMTP id lb5mr8025447wjb.84.1412349097136; Fri, 03 Oct 2014 08:11:37 -0700 (PDT) From: Will Newton To: libc-alpha@sourceware.org Subject: [PATCH 3/3] sysdeps/arm/bits/atomic.h: Use relaxed atomics for catomic_* Date: Fri, 3 Oct 2014 16:11:26 +0100 Message-Id: <1412349086-11473-4-git-send-email-will.newton@linaro.org> In-Reply-To: <1412349086-11473-1-git-send-email-will.newton@linaro.org> References: <1412349086-11473-1-git-send-email-will.newton@linaro.org> X-Original-Sender: will.newton@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 2a00:1450:4010:c03::231 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org; dkim=pass header.i=@sourceware.org X-Google-Group-Id: 836684582541 Using the relaxed memory model for atomics when single-threaded allows a reduction in the number of barriers (dmb) executed and an improvement in single thread performance on the malloc benchtest: Before: 259.073 After: 246.749 ChangeLog: 2014-10-03 Will Newton * sysdeps/arm/bits/atomic.h [__GNUC_PREREQ (4, 7) && __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4] (__atomic_is_single_thread): New define. (atomic_exchange_and_add_relaxed): Likewise. (catomic_exchange_and_add): Use relaxed memory model if single threaded. (atomic_and_relaxed): New define. (catomic_and): Use relaxed memory model if single threaded. (atomic_or_relaxed): New define. (catomic_or): Use relaxed memory model if single threaded. --- sysdeps/arm/bits/atomic.h | 55 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 52 insertions(+), 3 deletions(-) diff --git a/sysdeps/arm/bits/atomic.h b/sysdeps/arm/bits/atomic.h index be314e4..0fbd82b 100644 --- a/sysdeps/arm/bits/atomic.h +++ b/sysdeps/arm/bits/atomic.h @@ -52,6 +52,19 @@ void __arm_link_error (void); a pattern to do this efficiently. */ #if __GNUC_PREREQ (4, 7) && defined __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4 +# if defined IS_IN_libpthread || !defined NOT_IN_libc +# ifdef IS_IN_libpthread +extern int __pthread_multiple_threads attribute_hidden; +# define __atomic_is_single_thread (__pthread_multiple_threads == 0) +# else +extern int __libc_multiple_threads attribute_hidden; +# define __atomic_is_single_thread (__libc_multiple_threads == 0) +# endif +# else +# define __atomic_is_single_thread 0 +# endif + + /* Compare and exchange. For all "bool" routines, we return FALSE if exchange successful. */ @@ -180,7 +193,19 @@ void __arm_link_error (void); __atomic_val_bysize (__arch_exchange_and_add, int, mem, value, \ __ATOMIC_RELEASE) -# define catomic_exchange_and_add atomic_exchange_and_add +# define atomic_exchange_and_add_relaxed(mem, value) \ + __atomic_val_bysize (__arch_exchange_and_add, int, mem, value, \ + __ATOMIC_RELAXED) + +# define catomic_exchange_and_add(mem, value) \ + ({ \ + __typeof (*(mem)) __res; \ + if (__atomic_is_single_thread) \ + __res = atomic_exchange_and_add_relaxed (mem, value); \ + else \ + __res = atomic_exchange_and_add_acq (mem, value); \ + __res; \ + }) /* Atomically bitwise and value and return the previous value. */ @@ -200,9 +225,21 @@ void __arm_link_error (void); __atomic_val_bysize (__arch_exchange_and_and, int, mem, value, \ __ATOMIC_ACQUIRE) +# define atomic_and_relaxed(mem, value) \ + __atomic_val_bysize (__arch_exchange_and_and, int, mem, value, \ + __ATOMIC_RELAXED) + # define atomic_and_val atomic_and -# define catomic_and atomic_and +# define catomic_and(mem, value) \ + ({ \ + __typeof (*(mem)) __res; \ + if (__atomic_is_single_thread) \ + __res = atomic_and_relaxed (mem, value); \ + else \ + __res = atomic_and (mem, value); \ + __res; \ + }) /* Atomically bitwise or value and return the previous value. */ @@ -222,9 +259,21 @@ void __arm_link_error (void); __atomic_val_bysize (__arch_exchange_and_or, int, mem, value, \ __ATOMIC_ACQUIRE) +# define atomic_or_relaxed(mem, value) \ + __atomic_val_bysize (__arch_exchange_and_or, int, mem, value, \ + __ATOMIC_RELAXED) + # define atomic_or_val atomic_or -# define catomic_or atomic_or +# define catomic_or(mem, value) \ + ({ \ + __typeof (*(mem)) __res; \ + if (__atomic_is_single_thread) \ + __res = atomic_or_relaxed (mem, value); \ + else \ + __res = atomic_or (mem, value); \ + __res; \ + }) #elif defined __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4