From patchwork Wed Oct 7 10:59:28 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 54577 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wi0-f200.google.com (mail-wi0-f200.google.com [209.85.212.200]) by patches.linaro.org (Postfix) with ESMTPS id 94D9C22FF8 for ; Wed, 7 Oct 2015 11:00:05 +0000 (UTC) Received: by wicmn1 with SMTP id mn1sf6458269wic.1 for ; Wed, 07 Oct 2015 04:00:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:sender:precedence:list-id:x-original-sender :x-original-authentication-results:mailing-list:list-post:list-help :list-archive:list-unsubscribe; bh=ccQbs2BTHf1KEeq3crrW+sB982sVCG997VnhjZsf0+U=; b=eN/C+QyGn2YdYQeWmeHxVGndI59hSi2KT0oSU9YGv+y0ewXJpwO88SBjeZiP4/iiOj QLpXghU/FuLODQuAuevPUwmn+WIIBfYPFIJhUjL6X2cBuVoUVfkG/mSMrjVo46GLMXkr ytkw0DdTdGTLPPkpLs1TLaoKQCzzJlYzFnU5CasdMtdXjkoKmAxfSZwZk11UG7NV12k2 lmA92UTXDhzq3q6haK1b8SC8eqtWPgskjYv8GzEkysmQyXS2R6fypUzBp7vwvH6O6jKR YIkdSc0PNbA/Dhq2B0GhlgunyO+U2eU0IMo0G5U4wR5dxijmWBN+aVx+f0yc4O8BkZzI 9TyQ== X-Gm-Message-State: ALoCoQnjWy6rjaaBoNjcerep253zuXbbBNioGZ91NFtsMeKp5AwU8N7qUGECKYC2NDKsYYZ8vOD7 X-Received: by 10.112.147.7 with SMTP id tg7mr91840lbb.16.1444215604826; Wed, 07 Oct 2015 04:00:04 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.25.83.82 with SMTP id h79ls44544lfb.76.gmail; Wed, 07 Oct 2015 04:00:04 -0700 (PDT) X-Received: by 10.25.24.224 with SMTP id 93mr145103lfy.84.1444215604624; Wed, 07 Oct 2015 04:00:04 -0700 (PDT) Received: from mail-la0-f49.google.com (mail-la0-f49.google.com. [209.85.215.49]) by mx.google.com with ESMTPS id al9si24669106lbc.51.2015.10.07.04.00.04 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 07 Oct 2015 04:00:04 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.49 as permitted sender) client-ip=209.85.215.49; Received: by laddr2 with SMTP id dr2so7825317lad.3 for ; Wed, 07 Oct 2015 04:00:04 -0700 (PDT) X-Received: by 10.112.156.167 with SMTP id wf7mr224604lbb.88.1444215604051; Wed, 07 Oct 2015 04:00:04 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.59.35 with SMTP id w3csp2372789lbq; Wed, 7 Oct 2015 04:00:01 -0700 (PDT) X-Received: by 10.66.219.162 with SMTP id pp2mr426945pac.59.1444215601316; Wed, 07 Oct 2015 04:00:01 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id vy5si56921218pac.2.2015.10.07.04.00.00; Wed, 07 Oct 2015 04:00:01 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753406AbbJGK77 (ORCPT + 30 others); Wed, 7 Oct 2015 06:59:59 -0400 Received: from cam-admin0.cambridge.arm.com ([217.140.96.50]:42219 "EHLO cam-admin0.cambridge.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751039AbbJGK75 (ORCPT ); Wed, 7 Oct 2015 06:59:57 -0400 Received: from edgewater-inn.cambridge.arm.com (edgewater-inn.cambridge.arm.com [10.1.203.122]) by cam-admin0.cambridge.arm.com (8.12.6/8.12.6) with ESMTP id t97AxVWr023087; Wed, 7 Oct 2015 11:59:31 +0100 (BST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 289461AE2FFC; Wed, 7 Oct 2015 11:59:32 +0100 (BST) From: Will Deacon To: linux-arch@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Will Deacon , Boqun Feng , "Paul E. McKenney" , Peter Zijlstra Subject: [PATCH v2] barriers: introduce smp_mb__release_acquire and update documentation Date: Wed, 7 Oct 2015 11:59:28 +0100 Message-Id: <1444215568-24732-1-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: will.deacon@arm.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.49 as permitted sender) smtp.mailfrom=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , As much as we'd like to live in a world where RELEASE -> ACQUIRE is always cheaply ordered and can be used to construct UNLOCK -> LOCK definitions with similar guarantees, the grim reality is that this isn't even possible on x86 (thanks to Paul for bringing us crashing down to Earth). This patch handles the issue by introducing a new barrier macro, smp_mb__release_acquire, that can be placed between a RELEASE and a subsequent ACQUIRE operation in order to upgrade them to a full memory barrier. At the moment, it doesn't have any users, so its existence serves mainly as a documentation aid. Documentation/memory-barriers.txt is updated to describe more clearly the ACQUIRE and RELEASE ordering in this area and to show an example of the new barrier in action. Cc: Boqun Feng Cc: Paul E. McKenney Cc: Peter Zijlstra Signed-off-by: Will Deacon --- v1 -> v2: - Clarified that this barrier affects only access performed by the executing CPU - Definitions for all TSO architectures Documentation/memory-barriers.txt | 26 +++++++++++++++++++++++++- arch/ia64/include/asm/barrier.h | 1 + arch/powerpc/include/asm/barrier.h | 1 + arch/s390/include/asm/barrier.h | 2 ++ arch/sparc/include/asm/barrier_64.h | 5 +++-- arch/x86/include/asm/barrier.h | 2 ++ include/asm-generic/barrier.h | 4 ++++ 7 files changed, 38 insertions(+), 3 deletions(-) diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index 2ba8461b0631..0f69f1dc74fc 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -459,11 +459,18 @@ And a couple of implicit varieties: RELEASE on that same variable are guaranteed to be visible. In other words, within a given variable's critical section, all accesses of all previous critical sections for that variable are guaranteed to have - completed. + completed. If the RELEASE and ACQUIRE operations act on independent + variables, an smp_mb__release_acquire() barrier can be placed between + them to upgrade the sequence to a full barrier. This means that ACQUIRE acts as a minimal "acquire" operation and RELEASE acts as a minimal "release" operation. +A subset of the atomic operations described in atomic_ops.txt have ACQUIRE +and RELEASE variants in addition to fully-ordered and relaxed (no barrier +semantics) definitions. For compound atomics performing both a load and +a store, ACQUIRE semantics apply only to the load and RELEASE semantics +only to the store portion of the operation. Memory barriers are only required where there's a possibility of interaction between two CPUs or between a CPU and a device. If it can be guaranteed that @@ -1895,6 +1902,23 @@ the RELEASE would simply complete, thereby avoiding the deadlock. a sleep-unlock race, but the locking primitive needs to resolve such races properly in any case. +Where the RELEASE and ACQUIRE operations are performed by the same CPU, +ordering can be enforced by use of an smp_mb__release_acquire() barrier: + + *A = a; + RELEASE M + smp_mb__release_acquire(); + ACQUIRE N + *B = b; + +in which case, the only permitted sequences are: + + STORE *A, RELEASE M, ACQUIRE N, STORE *B + STORE *A, ACQUIRE N, RELEASE M, STORE *B + +Note that smp_mb__release_acquire() has no effect on ACQUIRE or RELEASE +operations performed by other CPUs. + Locks and semaphores may not provide any guarantee of ordering on UP compiled systems, and so cannot be counted on in such a situation to actually achieve anything at all - especially with respect to I/O accesses - unless combined diff --git a/arch/ia64/include/asm/barrier.h b/arch/ia64/include/asm/barrier.h index df896a1c41d3..9dceee6c2f20 100644 --- a/arch/ia64/include/asm/barrier.h +++ b/arch/ia64/include/asm/barrier.h @@ -77,6 +77,7 @@ do { \ ___p1; \ }) +#define smp_mb__release_acquire() smp_mb() #define smp_store_mb(var, value) do { WRITE_ONCE(var, value); mb(); } while (0) /* diff --git a/arch/powerpc/include/asm/barrier.h b/arch/powerpc/include/asm/barrier.h index 0eca6efc0631..919624634d0a 100644 --- a/arch/powerpc/include/asm/barrier.h +++ b/arch/powerpc/include/asm/barrier.h @@ -87,6 +87,7 @@ do { \ ___p1; \ }) +#define smp_mb__release_acquire() smp_mb() #define smp_mb__before_atomic() smp_mb() #define smp_mb__after_atomic() smp_mb() #define smp_mb__before_spinlock() smp_mb() diff --git a/arch/s390/include/asm/barrier.h b/arch/s390/include/asm/barrier.h index d48fe0162331..0c150b5fdd1c 100644 --- a/arch/s390/include/asm/barrier.h +++ b/arch/s390/include/asm/barrier.h @@ -53,4 +53,6 @@ do { \ ___p1; \ }) +#define smp_mb__release_acquire() smp_mb() + #endif /* __ASM_BARRIER_H */ diff --git a/arch/sparc/include/asm/barrier_64.h b/arch/sparc/include/asm/barrier_64.h index 14a928601657..4ae875cd9e78 100644 --- a/arch/sparc/include/asm/barrier_64.h +++ b/arch/sparc/include/asm/barrier_64.h @@ -71,7 +71,8 @@ do { \ ___p1; \ }) -#define smp_mb__before_atomic() barrier() -#define smp_mb__after_atomic() barrier() +#define smp_mb__release_acquire() smp_mb() +#define smp_mb__before_atomic() barrier() +#define smp_mb__after_atomic() barrier() #endif /* !(__SPARC64_BARRIER_H) */ diff --git a/arch/x86/include/asm/barrier.h b/arch/x86/include/asm/barrier.h index 0681d2532527..1c61ad251e0e 100644 --- a/arch/x86/include/asm/barrier.h +++ b/arch/x86/include/asm/barrier.h @@ -85,6 +85,8 @@ do { \ ___p1; \ }) +#define smp_mb__release_acquire() smp_mb() + #endif /* Atomic operations are already serializing on x86 */ diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h index b42afada1280..61ae95199397 100644 --- a/include/asm-generic/barrier.h +++ b/include/asm-generic/barrier.h @@ -119,5 +119,9 @@ do { \ ___p1; \ }) +#ifndef smp_mb__release_acquire +#define smp_mb__release_acquire() do { } while (0) +#endif + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_GENERIC_BARRIER_H */