From patchwork Thu Jul 5 16:40:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 141213 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp2154556ljj; Thu, 5 Jul 2018 09:41:29 -0700 (PDT) X-Google-Smtp-Source: AAOMgpehuS3ydoWV9PVD8Lv0n6WYtrDNcY9gEcR7gwyqHfUu27eY6e7RjR8DdN8GxxhIzk0hN8tP X-Received: by 2002:a62:591a:: with SMTP id n26-v6mr7243292pfb.94.1530808889751; Thu, 05 Jul 2018 09:41:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1530808889; cv=none; d=google.com; s=arc-20160816; b=08wxgSkVCFn1bWD2lZu9AM5EpdZKOFQze3K7XSUpqdEHyQFCIaAypYwlfzeqLLi/af AIl2nZkLmeNWYZ1A1I0Pf//FZ0wz1jKMF/uJql89WYmOPkqWOgi6zt8sBlP5l5G098Sf As7zpuP2BsPMDnOCeMABH4aYeaJXS/UTzZIqRQzkKVy5cQsgTBE1h5SHR8B5ex9FjpO+ X14YxEUcZGKeeZupPg0qpCH8V6uwSw7oUDmwlty2x0Yb2BlTvrDqDqg3EWAMil8Jtw82 8gg1Mtvx9MqF1zVwrBmsTvRHP1O+exySE5HdN0bwlADavvT326uypehndxjfkqktmniK VFsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=SGXYqKfzheuXqG/15fZkE48eUWGbVKiaTrFwwZrmyKI=; b=kyNgEBji882eRpEfXq3JVpagXsZWJSb7w8geB+g7cBi1+23auDJa19MH9D0yzFsV5P 3zfAnAEu4PWwMbTItg24xAUyfdp3pp0Fv57187dlQC6r7C6vdSar0IQnN1FLRZicZTi9 1s++2y2LZpAclGqohlDmJSppU73KWmoEJwyd2nHyfGPN+PIcrm/AXHprTX0BZumfx7qX 3NGGFc66SWByuoPunc6tyVW05Fhi73bHbjsTau/euw+MXngGDzjhhchUEMON54rpyWKR i6hhngpnv/V7/io50+AU0b6sHXOd6uDwO99ESYd1/kMpJBM3nL2KLrIbXJx+0PqEgB8V /5eg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d5-v6si6107225plo.3.2018.07.05.09.41.29; Thu, 05 Jul 2018 09:41:29 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754073AbeGEQl0 (ORCPT + 31 others); Thu, 5 Jul 2018 12:41:26 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:53380 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753483AbeGEQlY (ORCPT ); Thu, 5 Jul 2018 12:41:24 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BBA2B1650; Thu, 5 Jul 2018 09:41:23 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6F90E3F5BA; Thu, 5 Jul 2018 09:41:20 -0700 (PDT) From: Mark Rutland To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, peterz@infradead.org, will.deacon@arm.com Cc: andy.shevchenko@gmail.com, arnd@arndb.de, aryabinin@virtuozzo.com, boqun.feng@gmail.com, catalin.marinas@arm.com, dvyukov@google.com, glider@google.com, hpa@zytor.com, linuxdrivers@attotech.com, mark.rutland@arm.com, mingo@kernel.org, mingo@redhat.com, parri.andrea@gmail.com, peter@hurleysoftware.com, tglx@linutronix.de Subject: [PATCHv3 03/12] atomics: simplify cmpxchg() instrumentation Date: Thu, 5 Jul 2018 17:40:44 +0100 Message-Id: <20180705164053.10073-4-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180705164053.10073-1-mark.rutland@arm.com> References: <20180705164053.10073-1-mark.rutland@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Currently we define some fairly verbose wrappers for the cmpxchg() family so that we can pass a pointer and size into kasan_check_write(). The wrappers duplicate the size-switching logic necessary in arch code, and only work for scalar types. On some architectures, (cmp)xchg are used on non-scalar types, and thus the instrumented wrappers need to be able to handle this. We could take the type-punning logic from {READ,WRITE}_ONCE(), but this makes the wrappers even more verbose, and requires several local variables in the macros. Instead, let's simplify the wrappers into simple macros which: * snapshot the pointer into a single local variable, called __ai_ptr to avoid conflicts with variables in the scope of the caller. * call kasan_check_write() on __ai_ptr. * invoke the relevant arch_*() function, passing the original arguments, bar __ai_ptr being substituted for ptr. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Acked-by: Peter Zijlstra (Intel) Cc: Boqun Feng Cc: Dmitry Vyukov Cc: Ingo Molnar Cc: Will Deacon --- include/asm-generic/atomic-instrumented.h | 100 +++++------------------------- 1 file changed, 15 insertions(+), 85 deletions(-) -- 2.11.0 diff --git a/include/asm-generic/atomic-instrumented.h b/include/asm-generic/atomic-instrumented.h index 3c64e95d5ed0..c7c3e4cdd942 100644 --- a/include/asm-generic/atomic-instrumented.h +++ b/include/asm-generic/atomic-instrumented.h @@ -408,109 +408,39 @@ static __always_inline bool atomic64_add_negative(s64 i, atomic64_t *v) } #endif -static __always_inline unsigned long -cmpxchg_size(volatile void *ptr, unsigned long old, unsigned long new, int size) -{ - kasan_check_write(ptr, size); - switch (size) { - case 1: - return arch_cmpxchg((u8 *)ptr, (u8)old, (u8)new); - case 2: - return arch_cmpxchg((u16 *)ptr, (u16)old, (u16)new); - case 4: - return arch_cmpxchg((u32 *)ptr, (u32)old, (u32)new); - case 8: - BUILD_BUG_ON(sizeof(unsigned long) != 8); - return arch_cmpxchg((u64 *)ptr, (u64)old, (u64)new); - } - BUILD_BUG(); - return 0; -} - #define cmpxchg(ptr, old, new) \ ({ \ - ((__typeof__(*(ptr)))cmpxchg_size((ptr), (unsigned long)(old), \ - (unsigned long)(new), sizeof(*(ptr)))); \ + typeof(ptr) __ai_ptr = (ptr); \ + kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \ + arch_cmpxchg(__ai_ptr, (old), (new)); \ }) -static __always_inline unsigned long -sync_cmpxchg_size(volatile void *ptr, unsigned long old, unsigned long new, - int size) -{ - kasan_check_write(ptr, size); - switch (size) { - case 1: - return arch_sync_cmpxchg((u8 *)ptr, (u8)old, (u8)new); - case 2: - return arch_sync_cmpxchg((u16 *)ptr, (u16)old, (u16)new); - case 4: - return arch_sync_cmpxchg((u32 *)ptr, (u32)old, (u32)new); - case 8: - BUILD_BUG_ON(sizeof(unsigned long) != 8); - return arch_sync_cmpxchg((u64 *)ptr, (u64)old, (u64)new); - } - BUILD_BUG(); - return 0; -} - #define sync_cmpxchg(ptr, old, new) \ ({ \ - ((__typeof__(*(ptr)))sync_cmpxchg_size((ptr), \ - (unsigned long)(old), (unsigned long)(new), \ - sizeof(*(ptr)))); \ + typeof(ptr) __ai_ptr = (ptr); \ + kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \ + arch_sync_cmpxchg(__ai_ptr, (old), (new)); \ }) -static __always_inline unsigned long -cmpxchg_local_size(volatile void *ptr, unsigned long old, unsigned long new, - int size) -{ - kasan_check_write(ptr, size); - switch (size) { - case 1: - return arch_cmpxchg_local((u8 *)ptr, (u8)old, (u8)new); - case 2: - return arch_cmpxchg_local((u16 *)ptr, (u16)old, (u16)new); - case 4: - return arch_cmpxchg_local((u32 *)ptr, (u32)old, (u32)new); - case 8: - BUILD_BUG_ON(sizeof(unsigned long) != 8); - return arch_cmpxchg_local((u64 *)ptr, (u64)old, (u64)new); - } - BUILD_BUG(); - return 0; -} - #define cmpxchg_local(ptr, old, new) \ ({ \ - ((__typeof__(*(ptr)))cmpxchg_local_size((ptr), \ - (unsigned long)(old), (unsigned long)(new), \ - sizeof(*(ptr)))); \ + typeof(ptr) __ai_ptr = (ptr); \ + kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \ + arch_cmpxchg_local(__ai_ptr, (old), (new)); \ }) -static __always_inline u64 -cmpxchg64_size(volatile u64 *ptr, u64 old, u64 new) -{ - kasan_check_write(ptr, sizeof(*ptr)); - return arch_cmpxchg64(ptr, old, new); -} - #define cmpxchg64(ptr, old, new) \ ({ \ - ((__typeof__(*(ptr)))cmpxchg64_size((ptr), (u64)(old), \ - (u64)(new))); \ + typeof(ptr) __ai_ptr = (ptr); \ + kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \ + arch_cmpxchg64(__ai_ptr, (old), (new)); \ }) -static __always_inline u64 -cmpxchg64_local_size(volatile u64 *ptr, u64 old, u64 new) -{ - kasan_check_write(ptr, sizeof(*ptr)); - return arch_cmpxchg64_local(ptr, old, new); -} - #define cmpxchg64_local(ptr, old, new) \ ({ \ - ((__typeof__(*(ptr)))cmpxchg64_local_size((ptr), (u64)(old), \ - (u64)(new))); \ + typeof(ptr) __ai_ptr = (ptr); \ + kasan_check_write(__ai_ptr, sizeof(*__ai_ptr)); \ + arch_cmpxchg64_local(__ai_ptr, (old), (new)); \ }) /*