From patchwork Fri Feb 22 18:50:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 159062 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp2070429jaa; Fri, 22 Feb 2019 10:51:20 -0800 (PST) X-Google-Smtp-Source: AHgI3Ia9iPkUzP7fowOIxvC2m8TH78RkbD0/S9n9wN5nUBMQXC++JakGi7iWpjuNz5JczP85Waj2 X-Received: by 2002:a62:560f:: with SMTP id k15mr5528859pfb.231.1550861480423; Fri, 22 Feb 2019 10:51:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550861480; cv=none; d=google.com; s=arc-20160816; b=Z9OZc6FeHonUPGUOCeRc6aS9YFfUadHAvx1hKO90BeWiXaAOsy+3qYp03PsBsPaZOU 5/NlleSkndCJBH2AvmjZLkPGm4z7SrrRog7R7u3pa/3hRXN09eAWhQwouIB/zzsUkdrs lH35coUeeFaWEKLoi15unwJ0/8J2ERoAg8zuC2Ki4M/CBME/BPXA+Qrk9xorqFnWaWnd 96MHf08YSalIIgTRnq6FmC6dfZPt8RznIOBAFL6bqM+rt35UXu+qz8ltYqbuOj8qi5XH wYv8+yPlALnFf2mmKYRh9BliqTByEFuExPJxErQ0dAllLhOykHukMObPjRSjkXxbTPy8 DUvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=uDll+RtJwTRAnHGjfnj1wtODh0Kv0U7d8mn3e14bOHg=; b=fdCxb67AKaih/r+ssS8J9n4MsQYMMKkGrFTWBrY2okBeZ/yAuyqlkR0kzrJFipvjT2 9CmxtOP2tYw54ygdUeQRxhueU22EK5v1wgzqwO9KcwCPfR/RytlSLUyYqAQ5+PKBkhTG SvWcDfHUEX2jBsd50n8PiGKkyNBAdHH80EwFhhv4wtn+XuvZJUg9DLCkwGLyoLASjB33 3XhMse9RZvNmj1kbb6bTwoVrO+gqQndHlyn8HpMKKUYsdkB1GiTBqbv3WosTQPA9Dxec Sl51pQHoZgPNBpWodegvOv2cUCtXO1YUEc+eL7lTq01Djs0KlxtMGBgcdoPuyQQw7UZz jO9Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 1si177506ply.232.2019.02.22.10.51.20; Fri, 22 Feb 2019 10:51:20 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727608AbfBVSvS (ORCPT + 32 others); Fri, 22 Feb 2019 13:51:18 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39488 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726368AbfBVSvR (ORCPT ); Fri, 22 Feb 2019 13:51:17 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 281531682; Fri, 22 Feb 2019 10:51:17 -0800 (PST) Received: from fuggles.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E1B193F5C1; Fri, 22 Feb 2019 10:51:13 -0800 (PST) From: Will Deacon To: linux-arch@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Will Deacon , "Paul E. McKenney" , Benjamin Herrenschmidt , Michael Ellerman , Arnd Bergmann , Peter Zijlstra , Andrea Parri , Palmer Dabbelt , Daniel Lustig , David Howells , Alan Stern , Linus Torvalds , "Maciej W. Rozycki" , Paul Burton , Ingo Molnar , Yoshinori Sato , Rich Felker , Tony Luck Subject: [RFC PATCH 12/20] powerpc: mmiowb: Hook up mmwiob() implementation to asm-generic code Date: Fri, 22 Feb 2019 18:50:18 +0000 Message-Id: <20190222185026.10973-13-will.deacon@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190222185026.10973-1-will.deacon@arm.com> References: <20190222185026.10973-1-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In a bid to kill off explicit mmiowb() usage in driver code, hook up the asm-generic mmiowb() tracking code but override all of the functions so that the existing (flawed) implementation is preserved on Power. Future work should either use the asm-generic implementation entirely, or implement a similar counting mechanism to avoid erroneously skipping barrier invocations when they are needed. Signed-off-by: Will Deacon --- arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/Kbuild | 1 - arch/powerpc/include/asm/io.h | 33 +++-------------------------- arch/powerpc/include/asm/mmiowb.h | 41 +++++++++++++++++++++++++++++++++++++ arch/powerpc/include/asm/spinlock.h | 17 --------------- 5 files changed, 45 insertions(+), 48 deletions(-) create mode 100644 arch/powerpc/include/asm/mmiowb.h -- 2.11.0 diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 2890d36eb531..242ee248b922 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -134,6 +134,7 @@ config PPC select ARCH_HAS_ELF_RANDOMIZE select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_GCOV_PROFILE_ALL + select ARCH_HAS_MMIOWB if PPC64 && SMP select ARCH_HAS_PHYS_TO_DMA select ARCH_HAS_PMEM_API if PPC64 select ARCH_HAS_PTE_SPECIAL diff --git a/arch/powerpc/include/asm/Kbuild b/arch/powerpc/include/asm/Kbuild index 57bd1f6660f4..77ff7fb24823 100644 --- a/arch/powerpc/include/asm/Kbuild +++ b/arch/powerpc/include/asm/Kbuild @@ -8,7 +8,6 @@ generic-y += irq_regs.h generic-y += irq_work.h generic-y += local64.h generic-y += mcs_spinlock.h -generic-y += mmiowb.h generic-y += preempt.h generic-y += rwsem.h generic-y += vtime.h diff --git a/arch/powerpc/include/asm/io.h b/arch/powerpc/include/asm/io.h index 7f19fbd3ba55..828100476ba6 100644 --- a/arch/powerpc/include/asm/io.h +++ b/arch/powerpc/include/asm/io.h @@ -34,14 +34,11 @@ extern struct pci_dev *isa_bridge_pcidev; #include #include #include +#include #include #include #include -#ifdef CONFIG_PPC64 -#include -#endif - #define SIO_CONFIG_RA 0x398 #define SIO_CONFIG_RD 0x399 @@ -107,12 +104,6 @@ extern bool isa_io_special; * */ -#ifdef CONFIG_PPC64 -#define IO_SET_SYNC_FLAG() do { local_paca->io_sync = 1; } while(0) -#else -#define IO_SET_SYNC_FLAG() -#endif - #define DEF_MMIO_IN_X(name, size, insn) \ static inline u##size name(const volatile u##size __iomem *addr) \ { \ @@ -127,7 +118,7 @@ static inline void name(volatile u##size __iomem *addr, u##size val) \ { \ __asm__ __volatile__("sync;"#insn" %1,%y0" \ : "=Z" (*addr) : "r" (val) : "memory"); \ - IO_SET_SYNC_FLAG(); \ + mmiowb_set_pending(); \ } #define DEF_MMIO_IN_D(name, size, insn) \ @@ -144,7 +135,7 @@ static inline void name(volatile u##size __iomem *addr, u##size val) \ { \ __asm__ __volatile__("sync;"#insn"%U0%X0 %1,%0" \ : "=m" (*addr) : "r" (val) : "memory"); \ - IO_SET_SYNC_FLAG(); \ + mmiowb_set_pending(); \ } DEF_MMIO_IN_D(in_8, 8, lbz); @@ -652,24 +643,6 @@ static inline void name at \ #include -#ifdef CONFIG_PPC32 -#define mmiowb() -#else -/* - * Enforce synchronisation of stores vs. spin_unlock - * (this does it explicitly, though our implementation of spin_unlock - * does it implicitely too) - */ -static inline void mmiowb(void) -{ - unsigned long tmp; - - __asm__ __volatile__("sync; li %0,0; stb %0,%1(13)" - : "=&r" (tmp) : "i" (offsetof(struct paca_struct, io_sync)) - : "memory"); -} -#endif /* !CONFIG_PPC32 */ - static inline void iosync(void) { __asm__ __volatile__ ("sync" : : : "memory"); diff --git a/arch/powerpc/include/asm/mmiowb.h b/arch/powerpc/include/asm/mmiowb.h new file mode 100644 index 000000000000..7647ffcced1c --- /dev/null +++ b/arch/powerpc/include/asm/mmiowb.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_POWERPC_MMIOWB_H +#define _ASM_POWERPC_MMIOWB_H + +#ifdef CONFIG_ARCH_HAS_MMIOWB + +#include +#include +#include + +#define mmiowb() mb() + +/* + * Note: this is broken, as it doesn't correctly handle sequences such as: + * + * spin_lock(&foo); // io_sync = 0 + * outb(42, port); // io_sync = 1 + * spin_lock(&bar); // io_sync = 0 + * ... + * spin_unlock(&bar); + * spin_unlock(&foo); + * + * The implementation in asm-generic/mmiowb.h deals with this by counting + * the critical sections. + */ +#define mmiowb_set_pending() do { local_paca->io_sync = 1; } while (0) +#define mmiowb_spin_lock() do { get_paca()->io_sync = 0; } while (0) + +#define mmiowb_spin_unlock() do { \ + if (unlikely(get_paca()->io_sync)) { \ + mmiowb(); \ + get_paca()->io_sync = 0; \ + } \ +} while (0) +#else +#define mmiowb() do { } while (0) +#endif /* CONFIG_ARCH_HAS_MMIOWB */ + +#include + +#endif /* _ASM_POWERPC_MMIOWB_H */ diff --git a/arch/powerpc/include/asm/spinlock.h b/arch/powerpc/include/asm/spinlock.h index 685c72310f5d..15b39c407c4e 100644 --- a/arch/powerpc/include/asm/spinlock.h +++ b/arch/powerpc/include/asm/spinlock.h @@ -39,19 +39,6 @@ #define LOCK_TOKEN 1 #endif -#if defined(CONFIG_PPC64) && defined(CONFIG_SMP) -#define CLEAR_IO_SYNC (get_paca()->io_sync = 0) -#define SYNC_IO do { \ - if (unlikely(get_paca()->io_sync)) { \ - mb(); \ - get_paca()->io_sync = 0; \ - } \ - } while (0) -#else -#define CLEAR_IO_SYNC -#define SYNC_IO -#endif - #ifdef CONFIG_PPC_PSERIES #define vcpu_is_preempted vcpu_is_preempted static inline bool vcpu_is_preempted(int cpu) @@ -99,7 +86,6 @@ static inline unsigned long __arch_spin_trylock(arch_spinlock_t *lock) static inline int arch_spin_trylock(arch_spinlock_t *lock) { - CLEAR_IO_SYNC; return __arch_spin_trylock(lock) == 0; } @@ -130,7 +116,6 @@ extern void __rw_yield(arch_rwlock_t *lock); static inline void arch_spin_lock(arch_spinlock_t *lock) { - CLEAR_IO_SYNC; while (1) { if (likely(__arch_spin_trylock(lock) == 0)) break; @@ -148,7 +133,6 @@ void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long flags) { unsigned long flags_dis; - CLEAR_IO_SYNC; while (1) { if (likely(__arch_spin_trylock(lock) == 0)) break; @@ -167,7 +151,6 @@ void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long flags) static inline void arch_spin_unlock(arch_spinlock_t *lock) { - SYNC_IO; __asm__ __volatile__("# arch_spin_unlock\n\t" PPC_RELEASE_BARRIER: : :"memory"); lock->slock = 0;