From patchwork Thu Feb 15 15:29:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 128498 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp2021724ljc; Thu, 15 Feb 2018 10:02:01 -0800 (PST) X-Google-Smtp-Source: AH8x225C7t0oqD9HY+TIcDZeH7X5IYZk7eTTDEOxjyC8Q9vJj5X385pY/wnqRtlGO7rpTWNetp3H X-Received: by 10.99.36.70 with SMTP id k67mr2928300pgk.48.1518717721786; Thu, 15 Feb 2018 10:02:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518717721; cv=none; d=google.com; s=arc-20160816; b=hFTgt4ou+MBZK3ZOIV7dwuDu04Z46Wct5/pYzvSAN+I/kjc7a3kXQ/1MTQ9OPHZdzX yRatsJvMoWezlL3A6ubUSz1iiWwq5ArCEimOpNL+pgfyTwm3yyB7D4Ul8ZgYGJQUa+gV sr/5ycaB6jsmS5QOuQkIusQgFrn6Hd29vLweWc0LpSru4UBpBXY5U1i87ilo+QqjR8ol gSn9lzB/biOlXCbBl8/7e4X1nWLMDkYZ+mDoOiggyl6cmfaUUd7FC4NQA4Q0iVfSfDEK 2ai9teOtmavodqM+zpbYbHFSIvNrOlmx9WKLJ80QPcDt220L5vuncY9Kd0CC7nRCAq36 tfjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=vRafagUfTI2Cp462QPpxphE9QbS0qrGfDszWZgl8ppQ=; b=gVQ0gYJBONY1/NtpLoeVWI3tvjlw4zj4yy0W1gS48m1RdDWSysmrp4W4bOe6LCAz/j qLvj2ktsbDSiPm9QbHJWs1fdu8QXgtTw73Sgn7JxNBJOJJs+nqJBgPEbzkH2pxbBjmdV CENe3w6OJPq2zO/37qoFMYu5Ncj9Cj0e64yhuCX9aNOArk2cEmF2DsaFOHzNmjgcfPu0 EsRMjAv0+I98E0vv2K5UZv7jhmxpaOMAFkU21QLsi1IV0LdoSzv+XyuBJVQpp0G342oI mtTGnmeMwhOjiMId9mxJ9B6nxN1o2Zkd/QH1/WJFDUig9HGuCzWWtbxM/VG+V8uur4c2 NzlA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c41-v6si224758plj.682.2018.02.15.10.01.57; Thu, 15 Feb 2018 10:02:01 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1946166AbeBOR4E (ORCPT + 28 others); Thu, 15 Feb 2018 12:56:04 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:55968 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1164178AbeBOPaO (ORCPT ); Thu, 15 Feb 2018 10:30:14 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 48A0315AD; Thu, 15 Feb 2018 07:30:14 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 1AC283F592; Thu, 15 Feb 2018 07:30:14 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 7C7441AE5466; Thu, 15 Feb 2018 15:30:21 +0000 (GMT) From: Will Deacon To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, peterz@infradead.org, mingo@kernel.org, Will Deacon Subject: [RFC PATCH 1/5] arm64: fpsimd: include in fpsimd.h Date: Thu, 15 Feb 2018 15:29:31 +0000 Message-Id: <1518708575-12284-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1518708575-12284-1-git-send-email-will.deacon@arm.com> References: <1518708575-12284-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org fpsimd.h uses the __init annotation, so pull in linux/init.h Signed-off-by: Will Deacon --- arch/arm64/include/asm/fpsimd.h | 1 + 1 file changed, 1 insertion(+) -- 2.1.4 diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 8857a0f0d0f7..fc3527b985ca 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -22,6 +22,7 @@ #ifndef __ASSEMBLY__ #include +#include #include /* From patchwork Thu Feb 15 15:29:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 128497 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp2021634ljc; Thu, 15 Feb 2018 10:01:57 -0800 (PST) X-Google-Smtp-Source: AH8x227204iThAjH6fw/hUwsauIggSVlpa6ZDA3V/aGfK3NJq0iZwiKqw+uk0qFSYkQos5YJPECy X-Received: by 10.101.100.87 with SMTP id s23mr2873495pgv.413.1518717717575; Thu, 15 Feb 2018 10:01:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518717717; cv=none; d=google.com; s=arc-20160816; b=qn4Uxvst4LsdntK34TuQw/pHInYSDBob2zwQJoePHT8qUxwQylnIeKyfOR7D4rgZUz 0aCogyxXetBJLcsnCJaa22cZSZcmrFSTO8zJh+pVMIaV3psQu2F+IoKN52t0NatLBCgy zgqkYSpeSgm6SzZVyiomEntWkTrtBGPBwggRJkdvqa9IoYr02p0GnneZQCGhsJNGoAnW 4pbU40KRim63r/NpWIu9MaoH2hZiMEZXISMKzWJ0CWNmD+y4kYu/6xNZbi4x5MCIhvuv Zw5MjJe/HZoEAvD/3r2bmFNo8BfEKQpv0wcroCp7pmoZ/FPBp2IZMFFXVsjXAy4tJBda rErw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=Ff00QHp/6aMOsqR2NVb1BLj3eeLDi/3C3oRkWuAgAEw=; b=rXjNuKxXCbMCJBmunyMHShv1gzWCvKdaYYVEzkEyKihO1B7W6z0rFrptaa85Rr7q/Z gtNR4sdtI4rbGUgvW+WVzJM6hpNB07TajGdJ62AuaAq/Gw6629G+zyBgG8lmwg083C1l m6aJ6dtFkXfrI1pBvtK5ZkRClpLiOYbpwzl+vyyRpNmZ0iou8VcE6N2G4LMYji4jYi6Y JLNgGu5Q2omzFUUJcTwEBbCoVval+JLJdb99sT0+1MSW1q1yai5j5UxjIvIibzRdoTTn Wv9pAEHv+XAU+T6XyyDU1pf+yhDmRrkmz5VWRykSDtxUAvb+c0YNkW+vZal4OGXv5dw9 jOLw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c41-v6si224758plj.682.2018.02.15.10.01.52; Thu, 15 Feb 2018 10:01:57 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1165873AbeBOR4F (ORCPT + 28 others); Thu, 15 Feb 2018 12:56:05 -0500 Received: from foss.arm.com ([217.140.101.70]:55978 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1164181AbeBOPaO (ORCPT ); Thu, 15 Feb 2018 10:30:14 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5B5B315BF; Thu, 15 Feb 2018 07:30:14 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2D6D43F5C0; Thu, 15 Feb 2018 07:30:14 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 8CB801AE5468; Thu, 15 Feb 2018 15:30:21 +0000 (GMT) From: Will Deacon To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, peterz@infradead.org, mingo@kernel.org, Will Deacon Subject: [RFC PATCH 2/5] asm-generic: Avoid including linux/kernel.h in asm-generic/bug.h Date: Thu, 15 Feb 2018 15:29:32 +0000 Message-Id: <1518708575-12284-3-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1518708575-12284-1-git-send-email-will.deacon@arm.com> References: <1518708575-12284-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org asm-generic/bug.h unnecessarily includes linux/kernel.h whereas it can get away with linux/types.h instead. lib/errseq.c relies on this transitive include, so update it to include linux/kernel.h instead. Signed-off-by: Will Deacon --- include/asm-generic/bug.h | 2 +- lib/errseq.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) -- 2.1.4 diff --git a/include/asm-generic/bug.h b/include/asm-generic/bug.h index 963b755d19b0..ffda1247f639 100644 --- a/include/asm-generic/bug.h +++ b/include/asm-generic/bug.h @@ -15,7 +15,7 @@ #endif #ifndef __ASSEMBLY__ -#include +#include #ifdef CONFIG_BUG diff --git a/lib/errseq.c b/lib/errseq.c index df782418b333..ef3b10516eab 100644 --- a/lib/errseq.c +++ b/lib/errseq.c @@ -3,6 +3,7 @@ #include #include #include +#include /* * An errseq_t is a way of recording errors in one place, and allowing any From patchwork Thu Feb 15 15:29:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 128499 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp2021797ljc; Thu, 15 Feb 2018 10:02:05 -0800 (PST) X-Google-Smtp-Source: AH8x22697Dg42y5GY1YB9yAF+pM+jqoZ0Wdb97eEdqUb08MwRn+qjmVfhMDbHEaMgH0xyELDhzdd X-Received: by 10.101.100.87 with SMTP id s23mr2873969pgv.413.1518717725829; Thu, 15 Feb 2018 10:02:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518717725; cv=none; d=google.com; s=arc-20160816; b=NajZoWYpjbC1meAV+0mGhD1tTSaCEyNzA+ECZy72YXHtQ/rjAjfQVBfAfKKkvBKZvy tbLpFrsDBRwxx3re842TLgzWyEkj6bv5NXDZEy0XMU83yjj1WxyvXveAAzwJLWsQgiU8 bhytOB9JQxQ65SdWbMTLMQNfAzIR3mbAhKmHDzpJHxWnQunRPOGxMrhO4Au9Tx7S2SVm C6b1TRBYbOXpwoBOBAtm4NDp9LX4FovoiHx7GSYL+QP+EE3WgM48AD7Ag04qd0eJ2Id/ J98Xp+gpSbV7/nTcG4HUx2vuirA3WtO64ZOefuyUCbiESNqLJYWuDreUIA5WVV7mAqd1 X3nA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=c/os1MuMosp5zJD5cVc5NhXRINYbfOISuBB38ksbCE4=; b=ZHZgLfw4JTIfH2l2z11IvkSXrOCVTyDTx5wO3zjkfgWFFHJBwhexzG01q/mg0ZrIc6 ZM0dkd/BcIE2qURpGwAorN52SJ46UlDJxaWzEEo3CzmvsR9fsKsp5iDHgoYr3+zexwiX y80eo++cJHZarVdvkn8Z0JgJnHCUCRGsnkVQ8eRiN15er6e8WlB3cENOb8Gi5AUX+Q9U zbMk3j9s+81gjOVV/1bwyS4R5Bm0gGHb+/3oX/+zyuMA/hmoL/7YQcbVmRJRNtS9ygds iutw7WQwEXWfsGbr6Ja6G/mDDnpgh95KFZrAmMNdJENQYYu1oIBdYsmKKcV6AM03V+JT NF7A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c41-v6si224758plj.682.2018.02.15.10.02.01; Thu, 15 Feb 2018 10:02:05 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S968039AbeBOR4C (ORCPT + 28 others); Thu, 15 Feb 2018 12:56:02 -0500 Received: from foss.arm.com ([217.140.101.70]:55986 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1164183AbeBOPaO (ORCPT ); Thu, 15 Feb 2018 10:30:14 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7559D1610; Thu, 15 Feb 2018 07:30:14 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 474D53F24D; Thu, 15 Feb 2018 07:30:14 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 9F0D51AE5469; Thu, 15 Feb 2018 15:30:21 +0000 (GMT) From: Will Deacon To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, peterz@infradead.org, mingo@kernel.org, Will Deacon Subject: [RFC PATCH 3/5] asm-generic/bitops/atomic.h: Rewrite using atomic_fetch_* Date: Thu, 15 Feb 2018 15:29:33 +0000 Message-Id: <1518708575-12284-4-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1518708575-12284-1-git-send-email-will.deacon@arm.com> References: <1518708575-12284-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The atomic bitops can actually be implemented pretty efficiently using the atomic_fetch_* ops, rather than explicit use of spinlocks. Cc: Peter Zijlstra Cc: Ingo Molnar Signed-off-by: Will Deacon --- include/asm-generic/bitops/atomic.h | 219 ++++++++++++------------------------ 1 file changed, 71 insertions(+), 148 deletions(-) -- 2.1.4 diff --git a/include/asm-generic/bitops/atomic.h b/include/asm-generic/bitops/atomic.h index 04deffaf5f7d..69b7915e291d 100644 --- a/include/asm-generic/bitops/atomic.h +++ b/include/asm-generic/bitops/atomic.h @@ -2,189 +2,112 @@ #ifndef _ASM_GENERIC_BITOPS_ATOMIC_H_ #define _ASM_GENERIC_BITOPS_ATOMIC_H_ -#include -#include +#include +#include +#include -#ifdef CONFIG_SMP -#include -#include /* we use L1_CACHE_BYTES */ - -/* Use an array of spinlocks for our atomic_ts. - * Hash function to index into a different SPINLOCK. - * Since "a" is usually an address, use one spinlock per cacheline. +/* + * Implementation of atomic bitops using atomic-fetch ops. + * See Documentation/atomic_bitops.txt for details. */ -# define ATOMIC_HASH_SIZE 4 -# define ATOMIC_HASH(a) (&(__atomic_hash[ (((unsigned long) a)/L1_CACHE_BYTES) & (ATOMIC_HASH_SIZE-1) ])) - -extern arch_spinlock_t __atomic_hash[ATOMIC_HASH_SIZE] __lock_aligned; - -/* Can't use raw_spin_lock_irq because of #include problems, so - * this is the substitute */ -#define _atomic_spin_lock_irqsave(l,f) do { \ - arch_spinlock_t *s = ATOMIC_HASH(l); \ - local_irq_save(f); \ - arch_spin_lock(s); \ -} while(0) - -#define _atomic_spin_unlock_irqrestore(l,f) do { \ - arch_spinlock_t *s = ATOMIC_HASH(l); \ - arch_spin_unlock(s); \ - local_irq_restore(f); \ -} while(0) +static inline void set_bit(unsigned int nr, volatile unsigned long *p) +{ + p += BIT_WORD(nr); + atomic_long_fetch_or_relaxed(BIT_MASK(nr), (atomic_long_t *)p); +} -#else -# define _atomic_spin_lock_irqsave(l,f) do { local_irq_save(f); } while (0) -# define _atomic_spin_unlock_irqrestore(l,f) do { local_irq_restore(f); } while (0) -#endif +static inline void clear_bit(unsigned int nr, volatile unsigned long *p) +{ + p += BIT_WORD(nr); + atomic_long_fetch_andnot_relaxed(BIT_MASK(nr), (atomic_long_t *)p); +} -/* - * NMI events can occur at any time, including when interrupts have been - * disabled by *_irqsave(). So you can get NMI events occurring while a - * *_bit function is holding a spin lock. If the NMI handler also wants - * to do bit manipulation (and they do) then you can get a deadlock - * between the original caller of *_bit() and the NMI handler. - * - * by Keith Owens - */ +static inline void change_bit(unsigned int nr, volatile unsigned long *p) +{ + p += BIT_WORD(nr); + atomic_long_fetch_xor_relaxed(BIT_MASK(nr), (atomic_long_t *)p); +} -/** - * set_bit - Atomically set a bit in memory - * @nr: the bit to set - * @addr: the address to start counting from - * - * This function is atomic and may not be reordered. See __set_bit() - * if you do not require the atomic guarantees. - * - * Note: there are no guarantees that this function will not be reordered - * on non x86 architectures, so if you are writing portable code, - * make sure not to rely on its reordering guarantees. - * - * Note that @nr may be almost arbitrarily large; this function is not - * restricted to acting on a single-word quantity. - */ -static inline void set_bit(int nr, volatile unsigned long *addr) +static inline int test_and_set_bit(unsigned int nr, volatile unsigned long *p) { + long old; unsigned long mask = BIT_MASK(nr); - unsigned long *p = ((unsigned long *)addr) + BIT_WORD(nr); - unsigned long flags; - _atomic_spin_lock_irqsave(p, flags); - *p |= mask; - _atomic_spin_unlock_irqrestore(p, flags); + p += BIT_WORD(nr); + if (READ_ONCE(*p) & mask) + return 1; + + old = atomic_long_fetch_or(mask, (atomic_long_t *)p); + return !!(old & mask); } -/** - * clear_bit - Clears a bit in memory - * @nr: Bit to clear - * @addr: Address to start counting from - * - * clear_bit() is atomic and may not be reordered. However, it does - * not contain a memory barrier, so if it is used for locking purposes, - * you should call smp_mb__before_atomic() and/or smp_mb__after_atomic() - * in order to ensure changes are visible on other processors. - */ -static inline void clear_bit(int nr, volatile unsigned long *addr) +static inline int test_and_clear_bit(unsigned int nr, volatile unsigned long *p) { + long old; unsigned long mask = BIT_MASK(nr); - unsigned long *p = ((unsigned long *)addr) + BIT_WORD(nr); - unsigned long flags; - _atomic_spin_lock_irqsave(p, flags); - *p &= ~mask; - _atomic_spin_unlock_irqrestore(p, flags); + p += BIT_WORD(nr); + if (!(READ_ONCE(*p) & mask)) + return 0; + + old = atomic_long_fetch_andnot(mask, (atomic_long_t *)p); + return !!(old & mask); } -/** - * change_bit - Toggle a bit in memory - * @nr: Bit to change - * @addr: Address to start counting from - * - * change_bit() is atomic and may not be reordered. It may be - * reordered on other architectures than x86. - * Note that @nr may be almost arbitrarily large; this function is not - * restricted to acting on a single-word quantity. - */ -static inline void change_bit(int nr, volatile unsigned long *addr) +static inline int test_and_change_bit(unsigned int nr, volatile unsigned long *p) { + long old; unsigned long mask = BIT_MASK(nr); - unsigned long *p = ((unsigned long *)addr) + BIT_WORD(nr); - unsigned long flags; - _atomic_spin_lock_irqsave(p, flags); - *p ^= mask; - _atomic_spin_unlock_irqrestore(p, flags); + p += BIT_WORD(nr); + old = atomic_long_fetch_xor(mask, (atomic_long_t *)p); + return !!(old & mask); } -/** - * test_and_set_bit - Set a bit and return its old value - * @nr: Bit to set - * @addr: Address to count from - * - * This operation is atomic and cannot be reordered. - * It may be reordered on other architectures than x86. - * It also implies a memory barrier. - */ -static inline int test_and_set_bit(int nr, volatile unsigned long *addr) +static inline int test_and_set_bit_lock(unsigned int nr, + volatile unsigned long *p) { + long old; unsigned long mask = BIT_MASK(nr); - unsigned long *p = ((unsigned long *)addr) + BIT_WORD(nr); - unsigned long old; - unsigned long flags; - _atomic_spin_lock_irqsave(p, flags); - old = *p; - *p = old | mask; - _atomic_spin_unlock_irqrestore(p, flags); + p += BIT_WORD(nr); + if (READ_ONCE(*p) & mask) + return 1; - return (old & mask) != 0; + old = atomic_long_fetch_or_acquire(mask, (atomic_long_t *)p); + return !!(old & mask); } -/** - * test_and_clear_bit - Clear a bit and return its old value - * @nr: Bit to clear - * @addr: Address to count from - * - * This operation is atomic and cannot be reordered. - * It can be reorderdered on other architectures other than x86. - * It also implies a memory barrier. - */ -static inline int test_and_clear_bit(int nr, volatile unsigned long *addr) +static inline void clear_bit_unlock(unsigned int nr, volatile unsigned long *p) { - unsigned long mask = BIT_MASK(nr); - unsigned long *p = ((unsigned long *)addr) + BIT_WORD(nr); - unsigned long old; - unsigned long flags; + p += BIT_WORD(nr); + atomic_long_fetch_andnot_release(BIT_MASK(nr), (atomic_long_t *)p); +} - _atomic_spin_lock_irqsave(p, flags); - old = *p; - *p = old & ~mask; - _atomic_spin_unlock_irqrestore(p, flags); +static inline void __clear_bit_unlock(unsigned int nr, + volatile unsigned long *p) +{ + unsigned long old; - return (old & mask) != 0; + p += BIT_WORD(nr); + old = READ_ONCE(*p); + old &= ~BIT_MASK(nr); + smp_store_release(p, old); } -/** - * test_and_change_bit - Change a bit and return its old value - * @nr: Bit to change - * @addr: Address to count from - * - * This operation is atomic and cannot be reordered. - * It also implies a memory barrier. - */ -static inline int test_and_change_bit(int nr, volatile unsigned long *addr) +#ifndef clear_bit_unlock_is_negative_byte +static inline bool clear_bit_unlock_is_negative_byte(unsigned int nr, + volatile unsigned long *p) { + long old; unsigned long mask = BIT_MASK(nr); - unsigned long *p = ((unsigned long *)addr) + BIT_WORD(nr); - unsigned long old; - unsigned long flags; - - _atomic_spin_lock_irqsave(p, flags); - old = *p; - *p = old ^ mask; - _atomic_spin_unlock_irqrestore(p, flags); - return (old & mask) != 0; + p += BIT_WORD(nr); + old = atomic_long_fetch_andnot_release(mask, (atomic_long_t *)p); + return !!(old & BIT(7)); } +#define clear_bit_unlock_is_negative_byte clear_bit_unlock_is_negative_byte +#endif #endif /* _ASM_GENERIC_BITOPS_ATOMIC_H */ From patchwork Thu Feb 15 15:29:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 128500 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp2022441ljc; Thu, 15 Feb 2018 10:02:38 -0800 (PST) X-Google-Smtp-Source: AH8x225gReLEPcum/Iq5H9qEuCN5UOSMQDlb8zIJ41FCRxoRwaWcokcmHfoBUBRdJx/HMDMT/jM/ X-Received: by 10.98.198.1 with SMTP id m1mr3345440pfg.90.1518717747942; Thu, 15 Feb 2018 10:02:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518717747; cv=none; d=google.com; s=arc-20160816; b=EFDMjdmMDnlZrY+9WL4oNaD6iydG04sopz+daQvzYT4YGY5LVkxz7l8GGIzelK+Tkz 08zi2mFZxFzWOC09h12goKuGlKZ+ioI8iO/YnURmPsG36l/F9QUOwu04GA+q3AMZvEXC 2RoM9dBG3HoiHhYLZrbpz+/ObRxcBY2YbKP1uaQIPKmnTrcuQnk7x7I6d8cHBGGBIfmy HsEcxqCUL1ky+HwTDjy/EAQkjCfShx2gJbtHvSV01h8L1bfJIlc1z41aMnVQPH9LagQj RDxBUE02qA0ZzGORVF+yFU+jBnr+k0zLoBZYmC+RvdrqmDwwxcg7rzlaXypfOYb/G+2A tnfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=/+VEGj06cId9d/VU3wf6YG12ZvlZpRQpvBF/x8dHNtE=; b=jGe3+Hx/TxMpFzARp19niQQh/XudKUqN+XLyAh2t9xKpHilo5fK6L1xRybl3rVJnxR 3KaA5AVIQnG+hUBcnOKOobjUUY1QZDck4W9+PBdTtlTs5L0RSmolGCITW/x33p5v787S OGjM36M11f3qAjIwfQum4sL3+Mo3GVPNxQhPmZiTjfIQXo294Vwr99BY/MR0J/AehE83 GBbZVHo8bJUpF2UzptXXiEBpB7BvYSRluO5bcObOaJKrFSwttiFs7XlRpyeIzkH41Day HrKkwHtgQA4nF2PqkeLHjac6a63qkrJf1C9o0ZHA7yVk1Ai0CqGTb38mxK7EiTCVwAbP vF4Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u12-v6si1976952plm.481.2018.02.15.10.02.27; Thu, 15 Feb 2018 10:02:27 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S967990AbeBOR4B (ORCPT + 28 others); Thu, 15 Feb 2018 12:56:01 -0500 Received: from foss.arm.com ([217.140.101.70]:55998 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1164184AbeBOPaO (ORCPT ); Thu, 15 Feb 2018 10:30:14 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 87559164F; Thu, 15 Feb 2018 07:30:14 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 597763F592; Thu, 15 Feb 2018 07:30:14 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id B98461AE546A; Thu, 15 Feb 2018 15:30:21 +0000 (GMT) From: Will Deacon To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, peterz@infradead.org, mingo@kernel.org, Will Deacon Subject: [RFC PATCH 4/5] arm64: Replace our atomic bitops implementation with asm-generic Date: Thu, 15 Feb 2018 15:29:34 +0000 Message-Id: <1518708575-12284-5-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1518708575-12284-1-git-send-email-will.deacon@arm.com> References: <1518708575-12284-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The asm-generic/bitops/atomic.h implementation is built around the atomic-fetch ops, which we implement efficiently for both LSE and LL/SC systems. Use that instead of our hand-rolled, out-of-line bitops.S. Signed-off-by: Will Deacon --- arch/arm64/include/asm/bitops.h | 13 +------ arch/arm64/lib/Makefile | 2 +- arch/arm64/lib/bitops.S | 76 ----------------------------------------- 3 files changed, 2 insertions(+), 89 deletions(-) delete mode 100644 arch/arm64/lib/bitops.S -- 2.1.4 diff --git a/arch/arm64/include/asm/bitops.h b/arch/arm64/include/asm/bitops.h index 9c19594ce7cb..817233fbf249 100644 --- a/arch/arm64/include/asm/bitops.h +++ b/arch/arm64/include/asm/bitops.h @@ -17,22 +17,11 @@ #define __ASM_BITOPS_H #include -#include #ifndef _LINUX_BITOPS_H #error only can be included directly #endif -/* - * Little endian assembly atomic bitops. - */ -extern void set_bit(int nr, volatile unsigned long *p); -extern void clear_bit(int nr, volatile unsigned long *p); -extern void change_bit(int nr, volatile unsigned long *p); -extern int test_and_set_bit(int nr, volatile unsigned long *p); -extern int test_and_clear_bit(int nr, volatile unsigned long *p); -extern int test_and_change_bit(int nr, volatile unsigned long *p); - #include #include #include @@ -44,8 +33,8 @@ extern int test_and_change_bit(int nr, volatile unsigned long *p); #include #include -#include +#include #include #include diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile index 4e696f96451f..73095a04c0ad 100644 --- a/arch/arm64/lib/Makefile +++ b/arch/arm64/lib/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -lib-y := bitops.o clear_user.o delay.o copy_from_user.o \ +lib-y := clear_user.o delay.o copy_from_user.o \ copy_to_user.o copy_in_user.o copy_page.o \ clear_page.o memchr.o memcpy.o memmove.o memset.o \ memcmp.o strcmp.o strncmp.o strlen.o strnlen.o \ diff --git a/arch/arm64/lib/bitops.S b/arch/arm64/lib/bitops.S deleted file mode 100644 index 43ac736baa5b..000000000000 --- a/arch/arm64/lib/bitops.S +++ /dev/null @@ -1,76 +0,0 @@ -/* - * Based on arch/arm/lib/bitops.h - * - * Copyright (C) 2013 ARM Ltd. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -#include -#include -#include - -/* - * x0: bits 5:0 bit offset - * bits 31:6 word offset - * x1: address - */ - .macro bitop, name, llsc, lse -ENTRY( \name ) - and w3, w0, #63 // Get bit offset - eor w0, w0, w3 // Clear low bits - mov x2, #1 - add x1, x1, x0, lsr #3 // Get word offset -alt_lse " prfm pstl1strm, [x1]", "nop" - lsl x3, x2, x3 // Create mask - -alt_lse "1: ldxr x2, [x1]", "\lse x3, [x1]" -alt_lse " \llsc x2, x2, x3", "nop" -alt_lse " stxr w0, x2, [x1]", "nop" -alt_lse " cbnz w0, 1b", "nop" - - ret -ENDPROC(\name ) - .endm - - .macro testop, name, llsc, lse -ENTRY( \name ) - and w3, w0, #63 // Get bit offset - eor w0, w0, w3 // Clear low bits - mov x2, #1 - add x1, x1, x0, lsr #3 // Get word offset -alt_lse " prfm pstl1strm, [x1]", "nop" - lsl x4, x2, x3 // Create mask - -alt_lse "1: ldxr x2, [x1]", "\lse x4, x2, [x1]" - lsr x0, x2, x3 -alt_lse " \llsc x2, x2, x4", "nop" -alt_lse " stlxr w5, x2, [x1]", "nop" -alt_lse " cbnz w5, 1b", "nop" -alt_lse " dmb ish", "nop" - - and x0, x0, #1 - ret -ENDPROC(\name ) - .endm - -/* - * Atomic bit operations. - */ - bitop change_bit, eor, steor - bitop clear_bit, bic, stclr - bitop set_bit, orr, stset - - testop test_and_change_bit, eor, ldeoral - testop test_and_clear_bit, bic, ldclral - testop test_and_set_bit, orr, ldsetal From patchwork Thu Feb 15 15:29:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 128491 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp2015212ljc; Thu, 15 Feb 2018 09:55:22 -0800 (PST) X-Google-Smtp-Source: AH8x227/CWlJ8PRKAh9lIAEQ1AkYscGP5ryaxdLoIu4K/GaCRKA9RWkztiz4eRfmkwq2VbFt85No X-Received: by 10.99.188.2 with SMTP id q2mr2898730pge.101.1518717322645; Thu, 15 Feb 2018 09:55:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518717322; cv=none; d=google.com; s=arc-20160816; b=n6h2xsJH+bME3rfksJubrCyW9C0rMqIz2S3K++a9BerjhvmaFO1iZkYCuh+n+WKn4N ncEMFgVthFVYebMclWYXCpK6+7wgooZD6qumTNp9NbkAPnDY1836hstWjtDR2hWbnBkM PT1ASVWB46csZjbvjIg7WrbNIRx/Yak5ZUqNt7wT/qKFlLWvHzWctGSgPeaGBfqlVA4p REKS1A7ZK5+I0yww9FtQIZqldf+Ojw195KNYbQAimRHrU/DbA9s56gCiSutXjh7R0gYC RR+zZAieI+nvLo/11/YJllyKKwRw6HfbOiZhA6TiaEa1v7imkv+PSUD5VLrJlhl7XZqQ I8yA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=SMO6lswS/OwEUvMNDTWIoFbQ4cdk26/MVVVLIZrPKek=; b=xfCerjUcROv/3Ic1fF7YqsYVIBr9j0Oh1vLXzz8XuC3Z6wsEBMM4ji0C2Z46uY8D/Z g/I9J+/VVex4Y0KAwU0jEym9d+bEepRyE3plkvrlgbIUY/4sklNoRvMOO0sFFM6LruNU MM6evXV4Jd6lCwMs3EH+ko8EP542EmzfgVH8bKo+ARVZaInPnxSJoNLKmsjyYWjsQ2OC H8VWaCRCx9TsoxBM1NfIXyu6zu1uLmCt7D4smvHrZ5vM8FBXzoXAtCwVakjT9hMg1vC5 8rafjEFm0OjvCi1zLMz4USGoovFSUzovikMSW5uYup73RauU6ocpsXoJ3xPpgO6y0CY4 guUA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q17si1923562pfg.301.2018.02.15.09.55.21; Thu, 15 Feb 2018 09:55:22 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1946151AbeBORzS (ORCPT + 28 others); Thu, 15 Feb 2018 12:55:18 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:56002 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1164190AbeBOPaP (ORCPT ); Thu, 15 Feb 2018 10:30:15 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 29A8C165C; Thu, 15 Feb 2018 07:30:15 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id F009D3F24D; Thu, 15 Feb 2018 07:30:14 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id CBCF81AE546B; Thu, 15 Feb 2018 15:30:21 +0000 (GMT) From: Will Deacon To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, peterz@infradead.org, mingo@kernel.org, Will Deacon Subject: [RFC PATCH 5/5] arm64: bitops: Include Date: Thu, 15 Feb 2018 15:29:35 +0000 Message-Id: <1518708575-12284-6-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1518708575-12284-1-git-send-email-will.deacon@arm.com> References: <1518708575-12284-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org asm-generic/bitops/ext2-atomic-setbit.h provides the ext2 atomic bitop definitions, so we don't need to define our own. Signed-off-by: Will Deacon --- arch/arm64/include/asm/bitops.h | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/bitops.h b/arch/arm64/include/asm/bitops.h index 817233fbf249..564ffe562b7a 100644 --- a/arch/arm64/include/asm/bitops.h +++ b/arch/arm64/include/asm/bitops.h @@ -37,11 +37,6 @@ #include #include #include - -/* - * Ext2 is defined to use little-endian byte ordering. - */ -#define ext2_set_bit_atomic(lock, nr, p) test_and_set_bit_le(nr, p) -#define ext2_clear_bit_atomic(lock, nr, p) test_and_clear_bit_le(nr, p) +#include #endif /* __ASM_BITOPS_H */