From patchwork Wed Oct 18 13:32:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 116296 Delivered-To: patch@linaro.org Received: by 10.140.22.163 with SMTP id 32csp6074174qgn; Wed, 18 Oct 2017 06:32:52 -0700 (PDT) X-Google-Smtp-Source: ABhQp+SdHEdfI3iEWCwOMYzeEmWFCS4m/hkFQOkNQuGZ0i8APB+yfdpLJ6EqNfjPqYQEvhAAN+Ro X-Received: by 10.84.128.73 with SMTP id 67mr5051960pla.96.1508333572362; Wed, 18 Oct 2017 06:32:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1508333572; cv=none; d=google.com; s=arc-20160816; b=Vr9RpZDbhwfXNvT84IR+UcaLWzMVs+LPQ1FQ+emQDFYKIaFwlQT5R3eU8cx9Hzbqig BtEIoGWN+eDJL7/LwjKtcuM+drJrM15ZyfbQm3fDrvtmPWqhlLue3Vx2xcRSFrb2220U yuZBYGlOgcDg4vH/Q15rIvL2MCPnC6pG3m1m/hTBaAYlIMraRlM+jMQ51rUNwIf7pJzu hOXJmdW6FaRAiKTbej298Wh9R2y2U4NmMG3hj+aenUhUJo+pvw5Kutc9L5xMCHrALSen TpV7VXMln4tF7aSShqoCd+uxJal8kNzzQjkoNYgPJPUtCAxL3leQaC9gct5WrW/370uF BVxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :arc-authentication-results; bh=cXf6iAjhkCCubG6hJIza2F5/YLldnjsYZ5L4zHumtfI=; b=Er4grKif341ACMuhgcc51D+/Lgn5OxKEzIPmVFXyFeDX8dZFQNhQcbqfReAXipF7rV i4/KyhOHY2KJukzcA+Ywmf10N+P7l3cgQ1kFP1sKqE9+GP2OBPJqeAwLTcH7gwC4K4Ya xv0kQDY8ww95ty4osuOeCbZLn5uYdFxrMGvFCQNtTbNxrQ0OSd+9EizPoA2JkA0Jd4Rj +aG8FIGNMxYtTH0bb3R9T6RycY+O3GJTCaKXXAYdR8hm+G+wF0zE9koTVQ9RLzWop3dq tfcvdbCXK8y+SRl5Ypq4QV9Klv89+cpQj3I76iXuTFdEtnocD1Sn4xxRGtIRvPRY3iIq iLlg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g34si7865717pld.207.2017.10.18.06.32.52; Wed, 18 Oct 2017 06:32:52 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754118AbdJRNcu (ORCPT + 9 others); Wed, 18 Oct 2017 09:32:50 -0400 Received: from foss.arm.com ([217.140.101.70]:40634 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753871AbdJRNcq (ORCPT ); Wed, 18 Oct 2017 09:32:46 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2FB20F; Wed, 18 Oct 2017 06:32:46 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 85F533F5A1; Wed, 18 Oct 2017 06:32:44 -0700 (PDT) From: Mark Rutland To: stable@vger.kernel.org Cc: Mark Rutland , Arnd Bergmann , Christoph Lameter , Peter Zijlstra , Pranith Kumar , Tejun Heo , Thomas Gleixner , linux-arch@vger.kernel.org Subject: [PATCH v4.4.y] percpu: make this_cpu_generic_read() atomic w.r.t. interrupts Date: Wed, 18 Oct 2017 14:32:37 +0100 Message-Id: <20171018133237.55218-1-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org Commit e88d62cd4b2f0b1ae55e9008e79c2794b1fc914d upstream. As raw_cpu_generic_read() is a plain read from a raw_cpu_ptr() address, it's possible (albeit unlikely) that the compiler will split the access across multiple instructions. In this_cpu_generic_read() we disable preemption but not interrupts before calling raw_cpu_generic_read(). Thus, an interrupt could be taken in the middle of the split load instructions. If a this_cpu_write() or RMW this_cpu_*() op is made to the same variable in the interrupt handling path, this_cpu_read() will return a torn value. For native word types, we can avoid tearing using READ_ONCE(), but this won't work in all cases (e.g. 64-bit types on most 32-bit platforms). This patch reworks this_cpu_generic_read() to use READ_ONCE() where possible, otherwise falling back to disabling interrupts. Signed-off-by: Mark Rutland Cc: Arnd Bergmann Cc: Christoph Lameter Cc: Peter Zijlstra Cc: Pranith Kumar Cc: Tejun Heo Cc: Thomas Gleixner Cc: linux-arch@vger.kernel.org Cc: stable@vger.kernel.org Signed-off-by: Tejun Heo [Mark: backport to v4.4.y] Signed-off-by: Mark Rutland --- include/asm-generic/percpu.h | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) -- 2.11.0 diff --git a/include/asm-generic/percpu.h b/include/asm-generic/percpu.h index 4d9f233c4ba8..7d58ffdacd62 100644 --- a/include/asm-generic/percpu.h +++ b/include/asm-generic/percpu.h @@ -105,15 +105,35 @@ do { \ (__ret); \ }) -#define this_cpu_generic_read(pcp) \ +#define __this_cpu_generic_read_nopreempt(pcp) \ ({ \ typeof(pcp) __ret; \ preempt_disable(); \ - __ret = *this_cpu_ptr(&(pcp)); \ + __ret = READ_ONCE(*raw_cpu_ptr(&(pcp))); \ preempt_enable(); \ __ret; \ }) +#define __this_cpu_generic_read_noirq(pcp) \ +({ \ + typeof(pcp) __ret; \ + unsigned long __flags; \ + raw_local_irq_save(__flags); \ + __ret = *raw_cpu_ptr(&(pcp)); \ + raw_local_irq_restore(__flags); \ + __ret; \ +}) + +#define this_cpu_generic_read(pcp) \ +({ \ + typeof(pcp) __ret; \ + if (__native_word(pcp)) \ + __ret = __this_cpu_generic_read_nopreempt(pcp); \ + else \ + __ret = __this_cpu_generic_read_noirq(pcp); \ + __ret; \ +}) + #define this_cpu_generic_to_op(pcp, val, op) \ do { \ unsigned long __flags; \