From patchwork Tue Jan 3 18:27:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 89715 Delivered-To: patch@linaro.org Received: by 10.140.20.101 with SMTP id 92csp8138863qgi; Tue, 3 Jan 2017 10:29:16 -0800 (PST) X-Received: by 10.98.137.21 with SMTP id v21mr59740874pfd.48.1483468156413; Tue, 03 Jan 2017 10:29:16 -0800 (PST) Return-Path: Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id k79si69937502pfj.2.2017.01.03.10.29.16 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 03 Jan 2017 10:29:16 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) smtp.mailfrom=linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1cOTox-00063I-5H; Tue, 03 Jan 2017 18:28:27 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1cOTor-0005xo-S7 for linux-arm-kernel@lists.infradead.org; Tue, 03 Jan 2017 18:28:23 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 96A5C707; Tue, 3 Jan 2017 10:28:01 -0800 (PST) Received: from leverpostej.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 90CF63F24D; Tue, 3 Jan 2017 10:28:00 -0800 (PST) From: Mark Rutland To: catalin.marinas@arm.com Subject: [PATCH] arm64: restore get_current() optimisation Date: Tue, 3 Jan 2017 18:27:01 +0000 Message-Id: <1483468021-8237-1-git-send-email-mark.rutland@arm.com> X-Mailer: git-send-email 1.9.1 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170103_102822_101399_534CB83F X-CRM114-Status: GOOD ( 10.45 ) X-Spam-Score: -8.3 (--------) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-8.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high trust [217.140.101.70 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -1.4 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , dave@stgolabs.net, Will Deacon , linux-arm-kernel@lists.infradead.org, dbueso@suse.de MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org Hi Catalin, My THREAD_INFO_IN_TASK series had an unintended performance regression in get_current() / current_thread_info(). Could you please take the below as a fix for the next rc? Thanks, Mark. ---->8---- Commit c02433dd6de32f04 ("arm64: split thread_info from task stack") inverted the relationship between get_current() and current_thread_info(), with sp_el0 now holding the current task_struct rather than the current thead_info. The new implementation of get_current() prevents the compiler from being able to optimize repeated calls to either, resulting in a noticeable penalty in some microbenchmarks. This patch restores the previous optimisation by implementing get_current() in the same way as our old current_thread_info(), using a non-volatile asm statement. Signed-off-by: Mark Rutland Cc: Will Deacon Cc: Catalin Marinas Reported-by: Davidlohr Bueso --- arch/arm64/include/asm/current.h | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) -- 1.9.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel Acked-by: Will Deacon diff --git a/arch/arm64/include/asm/current.h b/arch/arm64/include/asm/current.h index f2bcbe2..86c4041 100644 --- a/arch/arm64/include/asm/current.h +++ b/arch/arm64/include/asm/current.h @@ -9,9 +9,17 @@ struct task_struct; +/* + * We don't use read_sysreg() as we want the compiler to cache the value where + * possible. + */ static __always_inline struct task_struct *get_current(void) { - return (struct task_struct *)read_sysreg(sp_el0); + unsigned long sp_el0; + + asm ("mrs %0, sp_el0" : "=r" (sp_el0)); + + return (struct task_struct *)sp_el0; } #define current get_current()