From patchwork Mon Jul 16 11:30:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 141999 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp2305708ljj; Mon, 16 Jul 2018 04:30:48 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdk8nSI1/roFfGNXbrNep4LqRLCGlL5FwM/jZumv/602g0Y+6otXRouWnSSVMtGRgbZpggJ X-Received: by 2002:a62:d842:: with SMTP id e63-v6mr17847042pfg.88.1531740648158; Mon, 16 Jul 2018 04:30:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531740648; cv=none; d=google.com; s=arc-20160816; b=u4Qgr2InyW8Ul/zLhZYtExWA7ZpG5sfVlwvIv2BF6jWtP64Gb/lp/Y+yOv/lOsady/ LSbTx5FMJFmjPvY2DBb9Y9SvwfqdxftFVH91XCjsYVJlTiYr1Uu5aIFCZcI+JgEH/F+L oJnlkaWIYHpH7BjtPugcyoAczTxSw7S9KaMR4y8ya+9DUGcPpyJcRm5HUt306FBd1elC 7bEa+usjKa75NxyKCXgIYnE9hgNxODVX/Jc/U8aMTZQWodWRKbkF5QS6a46u0qCvXKma UY4vHMvj6XisDmVxx7xkaIg6HfQDNkIY2+d11z3mCxjoMXPVxE8vaKO1kKUxliWFLE6w JK5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=Wt1HOaYEr4lxKGpOtxPuSCzw2erphYIZ59Hijnoy+VU=; b=gAJef1jdCgJEvPQjldr3VCntKkmiAXRb9/yhXY2yTlbw2QXF/ilsmhPK8fY5PC2SX1 PtbmZiWzWkV5h95ZjV4d6kcf9mZF/Elqi8DdNM0QH58bOokc8RROe58ubGD/7x7LRraA 4JaD2vZZ4TdJBltRWqI6BmSgTTHrvtHulm1K6ZAy46jjnYsi+Kbi+y+QyPoK7/Df0cSF /3xP/cX6zGkGCSDOUOfZCSHVI/oRx2jXAJ92GT2zwN4vEY48nxzNagEeMzaad0+yC6dE y7roh9F8VHnJxurTjcxYfluneHMI5Ydo9VM8kGeBLrzmi/+N0FIyJVC3KSbTbobn+o/s 8R8g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c2-v6si28204718pge.124.2018.07.16.04.30.47; Mon, 16 Jul 2018 04:30:48 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731203AbeGPL5p (ORCPT + 31 others); Mon, 16 Jul 2018 07:57:45 -0400 Received: from foss.arm.com ([217.140.101.70]:57630 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728643AbeGPL5o (ORCPT ); Mon, 16 Jul 2018 07:57:44 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A85FB1682; Mon, 16 Jul 2018 04:30:45 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B8CC93F589; Mon, 16 Jul 2018 04:30:42 -0700 (PDT) From: Mark Rutland To: mingo@kernel.org, mingo@redhat.com Cc: andy.shevchenko@gmail.com, arnd@arndb.de, aryabinin@virtuozzo.com, boqun.feng@gmail.com, catalin.marinas@arm.com, dvyukov@google.com, glider@google.com, hpa@zytor.com, linux-kernel@vger.kernel.org, mark.rutland@arm.com, parri.andrea@gmail.com, peter@hurleysoftware.com, peterz@infradead.org, tglx@linutronix.de, will.deacon@arm.com, linux-arm-kernel@lists.infradead.org Subject: [PATCHv4 03/12] atomics: simplify cmpxchg() instrumentation Date: Mon, 16 Jul 2018 12:30:08 +0100 Message-Id: <20180716113017.3909-4-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180716113017.3909-1-mark.rutland@arm.com> References: <20180716113017.3909-1-mark.rutland@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Currently we define some fairly verbose wrappers for the cmpxchg() family so that we can pass a pointer and size into kasan_check_write(). The wrappers duplicate the size-switching logic necessary in arch code, and only work for scalar types. On some architectures, (cmp)xchg are used on non-scalar types, and thus the instrumented wrappers need to be able to handle this. We could take the type-punning logic from {READ,WRITE}_ONCE(), but this makes the wrappers even more verbose, and requires several local variables in the macros. Instead, let's simplify the wrappers into simple macros which: * snapshot the pointer into a single local variable, called __ai_ptr to avoid conflicts with variables in the scope of the caller. * call kasan_check_write() on __ai_ptr. * invoke the relevant arch_*() function, passing the original arguments, bar __ai_ptr being substituted for ptr. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Acked-by: Peter Zijlstra (Intel) Acked-by: Will Deacon Cc: Boqun Feng Cc: Dmitry Vyukov Cc: Ingo Molnar --- include/asm-generic/atomic-instrumented.h | 100 +++++------------------------- 1 file changed, 15 insertions(+), 85 deletions(-) -- 2.11.0 diff --git a/include/asm-generic/atomic-instrumented.h b/include/asm-generic/atomic-instrumented.h index 3c64e95d5ed0..c7c3e4cdd942 100644 --- a/include/asm-generic/atomic-instrumented.h +++ b/include/asm-generic/atomic-instrumented.h @@ -408,109 +408,39 @@ static __always_inline bool atomic64_add_negative(s64 i, atomic64_t *v) } #endif -static __always_inline unsigned long -cmpxchg_size(volatile void *ptr, unsigned long old, unsigned long new, int size) -{ - kasan_check_write(ptr, size); - switch (size) { - case 1: - return arch_cmpxchg((u8 *)ptr, (u8)old, (u8)new); - case 2: - return arch_cmpxchg((u16 *)ptr, (u16)old, (u16)new); - case 4: - return arch_cmpxchg((u32 *)ptr, (u32)old, (u32)new); - case 8: - BUILD_BUG_ON(sizeof(unsigned long) != 8); - return arch_cmpxchg((u64 *)ptr, (u64)old, (u64)new); - } - BUILD_BUG(); - return 0; -} - #define cmpxchg(ptr, old, new) \ ({ \ - ((__typeof__(*(ptr)))cmpxchg_size((ptr), (unsigned long)(old), \ - (unsigned long)(new), sizeof(*(ptr)))); \ + typeof(ptr) __ai_ptr = (ptr); \ + kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \ + arch_cmpxchg(__ai_ptr, (old), (new)); \ }) -static __always_inline unsigned long -sync_cmpxchg_size(volatile void *ptr, unsigned long old, unsigned long new, - int size) -{ - kasan_check_write(ptr, size); - switch (size) { - case 1: - return arch_sync_cmpxchg((u8 *)ptr, (u8)old, (u8)new); - case 2: - return arch_sync_cmpxchg((u16 *)ptr, (u16)old, (u16)new); - case 4: - return arch_sync_cmpxchg((u32 *)ptr, (u32)old, (u32)new); - case 8: - BUILD_BUG_ON(sizeof(unsigned long) != 8); - return arch_sync_cmpxchg((u64 *)ptr, (u64)old, (u64)new); - } - BUILD_BUG(); - return 0; -} - #define sync_cmpxchg(ptr, old, new) \ ({ \ - ((__typeof__(*(ptr)))sync_cmpxchg_size((ptr), \ - (unsigned long)(old), (unsigned long)(new), \ - sizeof(*(ptr)))); \ + typeof(ptr) __ai_ptr = (ptr); \ + kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \ + arch_sync_cmpxchg(__ai_ptr, (old), (new)); \ }) -static __always_inline unsigned long -cmpxchg_local_size(volatile void *ptr, unsigned long old, unsigned long new, - int size) -{ - kasan_check_write(ptr, size); - switch (size) { - case 1: - return arch_cmpxchg_local((u8 *)ptr, (u8)old, (u8)new); - case 2: - return arch_cmpxchg_local((u16 *)ptr, (u16)old, (u16)new); - case 4: - return arch_cmpxchg_local((u32 *)ptr, (u32)old, (u32)new); - case 8: - BUILD_BUG_ON(sizeof(unsigned long) != 8); - return arch_cmpxchg_local((u64 *)ptr, (u64)old, (u64)new); - } - BUILD_BUG(); - return 0; -} - #define cmpxchg_local(ptr, old, new) \ ({ \ - ((__typeof__(*(ptr)))cmpxchg_local_size((ptr), \ - (unsigned long)(old), (unsigned long)(new), \ - sizeof(*(ptr)))); \ + typeof(ptr) __ai_ptr = (ptr); \ + kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \ + arch_cmpxchg_local(__ai_ptr, (old), (new)); \ }) -static __always_inline u64 -cmpxchg64_size(volatile u64 *ptr, u64 old, u64 new) -{ - kasan_check_write(ptr, sizeof(*ptr)); - return arch_cmpxchg64(ptr, old, new); -} - #define cmpxchg64(ptr, old, new) \ ({ \ - ((__typeof__(*(ptr)))cmpxchg64_size((ptr), (u64)(old), \ - (u64)(new))); \ + typeof(ptr) __ai_ptr = (ptr); \ + kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \ + arch_cmpxchg64(__ai_ptr, (old), (new)); \ }) -static __always_inline u64 -cmpxchg64_local_size(volatile u64 *ptr, u64 old, u64 new) -{ - kasan_check_write(ptr, sizeof(*ptr)); - return arch_cmpxchg64_local(ptr, old, new); -} - #define cmpxchg64_local(ptr, old, new) \ ({ \ - ((__typeof__(*(ptr)))cmpxchg64_local_size((ptr), (u64)(old), \ - (u64)(new))); \ + typeof(ptr) __ai_ptr = (ptr); \ + kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \ + arch_cmpxchg64_local(__ai_ptr, (old), (new)); \ }) /*