From patchwork Fri Apr 5 13:59:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 161846 Delivered-To: patch@linaro.org Received: by 2002:a02:c6d8:0:0:0:0:0 with SMTP id r24csp471685jan; Fri, 5 Apr 2019 06:59:59 -0700 (PDT) X-Google-Smtp-Source: APXvYqyqIgIgxiOzGERLqVqI7f5acx8AY2UjnpK7F+PphJxaXzVzcsOMa9x4e+bGSzU8lugzEED0 X-Received: by 2002:a63:54b:: with SMTP id 72mr11783540pgf.323.1554472799505; Fri, 05 Apr 2019 06:59:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554472799; cv=none; d=google.com; s=arc-20160816; b=QyZTLtO/P/3Fi3lCLNLJeNpGb4dbVBJagUqrZMWRoq7DA58Nndsb3giIszFSPOg/h9 gvwLG5D3Q7v9WUteuVnAsiqWVCASOTX2a2ES/4t+8fz7vqF+IhkK/YoR6R+3iM7KfiKY e4OhfVeDnZySS7geGYakyCTRct2dwQyyqJAifdREPfFxS78dkNbPDiBQcHY2JJS3oXeY h7mH43JOXFMRSh9w0cduGFMvD4MSybVNzgQB5JGLiAeRZrVyeGI17z6wiGUrPYg7yHA0 Aq4BnL8AG7O2XZSUPeEgrAwHYzB2ZkIw3ErMSEhNpUoLczqd0jXTPqg0kVyx2RWSm4b1 SHEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=vI8xheGFhIt/sRxFxrL2NmKPCQ71Nq9kyke9lX3lj9s=; b=i1tqpC5ECsldBUxNM5maWAL7qhIRX02rg/QbA/74XZsmSdfobK1epOwWwA00iYyi9J NPFaFIJaFdRt1h60FueknkUdnB/hnX5whPV7FBt995co9J0EESsPdr7vlbMO6p9/Sgi2 rvumWaO702DOnOYmD5td8uzXS6ob3SLqNbam2pnqIBlgLWSIPJupEEZ4Aakpl97bMkBG kPPzHsPEUYi2LWMahbUWWyMxwbcgINMk8vZKp4Hz54x8AyNRkia7XDYKOmG7xboIxgQf pwGU9iKTp2gDI3DuvewTG+bEfAsLgCUPqKAKSxv9fHBCtYH2wgF6Gt8KvcIX3//mubrq tfmw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j9si18547946pgp.17.2019.04.05.06.59.59; Fri, 05 Apr 2019 06:59:59 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731239AbfDEN75 (ORCPT + 31 others); Fri, 5 Apr 2019 09:59:57 -0400 Received: from foss.arm.com ([217.140.101.70]:49048 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726694AbfDEN7x (ORCPT ); Fri, 5 Apr 2019 09:59:53 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A707119BF; Fri, 5 Apr 2019 06:59:52 -0700 (PDT) Received: from fuggles.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D13C53F68F; Fri, 5 Apr 2019 06:59:48 -0700 (PDT) From: Will Deacon To: linux-arch@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Will Deacon , "Paul E. McKenney" , Benjamin Herrenschmidt , Michael Ellerman , Arnd Bergmann , Peter Zijlstra , Andrea Parri , Palmer Dabbelt , Daniel Lustig , David Howells , Alan Stern , Linus Torvalds , "Maciej W. Rozycki" , Paul Burton , Ingo Molnar , Yoshinori Sato , Rich Felker , Tony Luck , Mikulas Patocka , Akira Yokosawa , Luis Chamberlain , Nicholas Piggin Subject: [PATCH v2 02/21] asm-generic/mmiowb: Add generic implementation of mmiowb() tracking Date: Fri, 5 Apr 2019 14:59:17 +0100 Message-Id: <20190405135936.7266-3-will.deacon@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190405135936.7266-1-will.deacon@arm.com> References: <20190405135936.7266-1-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation for removing all explicit mmiowb() calls from driver code, implement a tracking system in asm-generic based loosely on the PowerPC implementation. This allows architectures with a non-empty mmiowb() definition to have the barrier automatically inserted in spin_unlock() following a critical section containing an I/O write. Signed-off-by: Will Deacon --- include/asm-generic/mmiowb.h | 63 ++++++++++++++++++++++++++++++++++++++ include/asm-generic/mmiowb_types.h | 12 ++++++++ kernel/Kconfig.locks | 7 +++++ kernel/locking/spinlock.c | 7 +++++ 4 files changed, 89 insertions(+) create mode 100644 include/asm-generic/mmiowb.h create mode 100644 include/asm-generic/mmiowb_types.h -- 2.11.0 diff --git a/include/asm-generic/mmiowb.h b/include/asm-generic/mmiowb.h new file mode 100644 index 000000000000..9439ff037b2d --- /dev/null +++ b/include/asm-generic/mmiowb.h @@ -0,0 +1,63 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_GENERIC_MMIOWB_H +#define __ASM_GENERIC_MMIOWB_H + +/* + * Generic implementation of mmiowb() tracking for spinlocks. + * + * If your architecture doesn't ensure that writes to an I/O peripheral + * within two spinlocked sections on two different CPUs are seen by the + * peripheral in the order corresponding to the lock handover, then you + * need to follow these FIVE easy steps: + * + * 1. Implement mmiowb() (and arch_mmiowb_state() if you're fancy) + * in asm/mmiowb.h, then #include this file + * 2. Ensure your I/O write accessors call mmiowb_set_pending() + * 3. Select ARCH_HAS_MMIOWB + * 4. Untangle the resulting mess of header files + * 5. Complain to your architects + */ +#ifdef CONFIG_MMIOWB + +#include +#include + +#ifndef arch_mmiowb_state +#include +#include + +DECLARE_PER_CPU(struct mmiowb_state, __mmiowb_state); +#define __mmiowb_state() this_cpu_ptr(&__mmiowb_state) +#else +#define __mmiowb_state() arch_mmiowb_state() +#endif /* arch_mmiowb_state */ + +static inline void mmiowb_set_pending(void) +{ + struct mmiowb_state *ms = __mmiowb_state(); + ms->mmiowb_pending = ms->nesting_count; +} + +static inline void mmiowb_spin_lock(void) +{ + struct mmiowb_state *ms = __mmiowb_state(); + ms->nesting_count++; +} + +static inline void mmiowb_spin_unlock(void) +{ + struct mmiowb_state *ms = __mmiowb_state(); + + if (unlikely(ms->mmiowb_pending)) { + ms->mmiowb_pending = 0; + mmiowb(); + } + + ms->nesting_count--; +} +#else +#define mmiowb_set_pending() do { } while (0) +#define mmiowb_spin_lock() do { } while (0) +#define mmiowb_spin_unlock() do { } while (0) +#endif /* CONFIG_MMIOWB */ +#endif /* __ASM_GENERIC_MMIOWB_H */ diff --git a/include/asm-generic/mmiowb_types.h b/include/asm-generic/mmiowb_types.h new file mode 100644 index 000000000000..8eb0095655e7 --- /dev/null +++ b/include/asm-generic/mmiowb_types.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_GENERIC_MMIOWB_TYPES_H +#define __ASM_GENERIC_MMIOWB_TYPES_H + +#include + +struct mmiowb_state { + u16 nesting_count; + u16 mmiowb_pending; +}; + +#endif /* __ASM_GENERIC_MMIOWB_TYPES_H */ diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks index fbba478ae522..6ba2570eddad 100644 --- a/kernel/Kconfig.locks +++ b/kernel/Kconfig.locks @@ -251,3 +251,10 @@ config ARCH_USE_QUEUED_RWLOCKS config QUEUED_RWLOCKS def_bool y if ARCH_USE_QUEUED_RWLOCKS depends on SMP + +config ARCH_HAS_MMIOWB + bool + +config MMIOWB + def_bool y if ARCH_HAS_MMIOWB + depends on SMP diff --git a/kernel/locking/spinlock.c b/kernel/locking/spinlock.c index 936f3d14dd6b..0ff08380f531 100644 --- a/kernel/locking/spinlock.c +++ b/kernel/locking/spinlock.c @@ -22,6 +22,13 @@ #include #include +#ifdef CONFIG_MMIOWB +#ifndef arch_mmiowb_state +DEFINE_PER_CPU(struct mmiowb_state, __mmiowb_state); +EXPORT_PER_CPU_SYMBOL(__mmiowb_state); +#endif +#endif + /* * If lockdep is enabled then we use the non-preemption spin-ops * even on CONFIG_PREEMPT, because lockdep assumes that interrupts are