From patchwork Thu Jul 6 09:49:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dietmar Eggemann X-Patchwork-Id: 107102 Delivered-To: patch@linaro.org Received: by 10.140.101.44 with SMTP id t41csp1881957qge; Thu, 6 Jul 2017 02:51:00 -0700 (PDT) X-Received: by 10.99.104.129 with SMTP id d123mr25110937pgc.236.1499334660579; Thu, 06 Jul 2017 02:51:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1499334660; cv=none; d=google.com; s=arc-20160816; b=IXHjvFlU0weFtLw0LM8XJcmD5+pLZcPexLr+IthDe0x835sk43Qw8CFYN/7An0UlB3 Th1Bnh6hLgx9E3DotHkem9K/Spxx5FTNpRv8B8O5RDOQdzZV/pVdhqkOiCuFXDJAHAsK bOHfDYsnhvUdNUKgS5HXh8NIHw9SEtcQtKfqfIDRO378JmXqN+4LsmFH7ap+VJLAQIys V2keI+wSjNLtiUEEwS9hjSsevvRKwfN2S8Y7qYGC5GmvbsHqqVACrIpvIIf/TBdFMUPL DnzL6AAQubdJ1EmoxSMfKWU1+igUB57HW9O+cwYpbYNrGM6/fX/AoopPdl5W3DQ/f0Bv U9RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=YDqUfXiaPN59xNQG588fNaqwtRUqZ5XPZCQMj9tvcyg=; b=trtopR3JUyDX3PpaX89vF83K8yCyE8JAHWeerMsxmL7GMabV8ISek+5cnhowEc/fge 7vjikHPySzEEd0yPw6tw1kTN/qUFba3o7nG2RPHPsTK+i/omB2G+aNpBEskWN2j7gLTo dqFzTgM7KglOfsdVYzPkWTolisvBY5hGsc5OJaJ7VrxgzL6Cg9uOrtco9NVaTinDhgG3 ffdA8HBkkjj+757qirbO4PT7/PXjfzCqkFOBHUa9NVIYM8u3e0XW5Hb5xKuHmYLoi/Y3 xaZrrjdrQEjy49SHOQbu2UQiOWbOfDp1FK30lt6kNRS45GZy6/9bzKeEG1MDNaKsgodP h/3g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d20si1351456plj.429.2017.07.06.02.51.00; Thu, 06 Jul 2017 02:51:00 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-pm-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752149AbdGFJuy (ORCPT + 14 others); Thu, 6 Jul 2017 05:50:54 -0400 Received: from foss.arm.com ([217.140.101.70]:36192 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751059AbdGFJuw (ORCPT ); Thu, 6 Jul 2017 05:50:52 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3CFF21713; Thu, 6 Jul 2017 02:50:38 -0700 (PDT) Received: from e107985-lin.cambridge.arm.com (e107985-lin.cambridge.arm.com [10.1.210.41]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3C5AB3F3E1; Thu, 6 Jul 2017 02:50:36 -0700 (PDT) From: Dietmar Eggemann To: linux-kernel@vger.kernel.org Cc: linux-pm@vger.kernel.org, linux@arm.linux.org.uk, Greg Kroah-Hartman , Russell King , Catalin Marinas , Will Deacon , Juri Lelli , Vincent Guittot , Peter Zijlstra , Morten Rasmussen , Viresh Kumar , "Rafael J . Wysocki" Subject: [PATCH v2 10/10] drivers base/arch_topology: inline cpu- and frequency-invariant accounting Date: Thu, 6 Jul 2017 10:49:48 +0100 Message-Id: <20170706094948.8779-11-dietmar.eggemann@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170706094948.8779-1-dietmar.eggemann@arm.com> References: <20170706094948.8779-1-dietmar.eggemann@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org To speed up the cpu- and frequency-invariant accounting of the task scheduler make sure that the CIE (topology_get_cpu_scale()) and FIE (topology_get_freq_scale() get completely inlined into the task scheduler consumer functions (e.g. __update_load_avg_se()). This patch-set changes the interface for CIE and FIE from: drivers/base/arch_topology.c: static DEFINE_PER_CPU(unsigned long, item); unsigned long topology_get_item_scale(...) { return per_cpu(item, cpu) } include/linux/arch_topology.h: unsigned long topology_get_item_scale(...); to: drivers/base/arch_topology.c: DEFINE_PER_CPU(unsigned long, item); include/linux/arch_topology.h: DECLARE_PER_CPU(unsigned long, item); static inline unsigned long topology_get_item_scale(...) { return per_cpu(item, cpu) } An uplift in performance could be detected running the kernel with the following test patch on top (on JUNO R0 (arm64)): @@ -2812,10 +2812,18 @@ accumulate_sum(u64 delta, int cpu, struct sched_avg *sa, unsigned long scale_freq, scale_cpu; u32 contrib = (u32)delta; /* p == 0 -> delta < 1024 */ u64 periods; + u64 t1, t2; + + t1 = sched_clock_cpu(cpu); scale_freq = arch_scale_freq_capacity(NULL, cpu); scale_cpu = arch_scale_cpu_capacity(NULL, cpu); + t2 = sched_clock_cpu(cpu); + + trace_printk("cpu=%d t1=%llu t2=%llu diff=%llu\n", + cpu, t1, t2, t2 - t1); + delta += sa->period_contrib; periods = delta / 1024; /* A period is * 1024us * (~1ms) */ The following test results (3 test runs each) have been obtained by tracing this trace printk (diff=x) for Cortex A-53 (LITTLE) and Cortex A-57 (big) cpus w/ (inline) and w/o (non-inline) this patch. mean max min A-57 inline: 119.6 300 60 96.8 280 60 110.2 660 60 A-57 non-inline: 142.8 460 80 157.6 680 80 153.4 720 80 A-53 inline: 141.6 360 100 118.8 500 100 148.6 380 100 A-53 non-inline: 293 840 120 253.2 840 120 299.6 1060 140 Cc: Greg Kroah-Hartman Cc: Juri Lelli Signed-off-by: Dietmar Eggemann --- drivers/base/arch_topology.c | 14 ++------------ include/linux/arch_topology.h | 15 +++++++++++++-- 2 files changed, 15 insertions(+), 14 deletions(-) -- 2.11.0 diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index 63fb3f945d21..b4481cff14bf 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -22,12 +22,7 @@ #include #include -static DEFINE_PER_CPU(unsigned long, freq_scale) = SCHED_CAPACITY_SCALE; - -unsigned long topology_get_freq_scale(struct sched_domain *sd, int cpu) -{ - return per_cpu(freq_scale, cpu); -} +DEFINE_PER_CPU(unsigned long, freq_scale) = SCHED_CAPACITY_SCALE; void topology_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq, unsigned long max_freq) @@ -43,12 +38,7 @@ void topology_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq, static DEFINE_MUTEX(cpu_scale_mutex); -static DEFINE_PER_CPU(unsigned long, cpu_scale) = SCHED_CAPACITY_SCALE; - -unsigned long topology_get_cpu_scale(struct sched_domain *sd, int cpu) -{ - return per_cpu(cpu_scale, cpu); -} +DEFINE_PER_CPU(unsigned long, cpu_scale) = SCHED_CAPACITY_SCALE; void topology_set_cpu_scale(unsigned int cpu, unsigned long capacity) { diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h index 168104d2d2cf..361e85a30151 100644 --- a/include/linux/arch_topology.h +++ b/include/linux/arch_topology.h @@ -11,12 +11,23 @@ void topology_normalize_cpu_scale(void); struct device_node; int topology_parse_cpu_capacity(struct device_node *cpu_node, int cpu); +DECLARE_PER_CPU(unsigned long, cpu_scale); +DECLARE_PER_CPU(unsigned long, freq_scale); + struct sched_domain; -unsigned long topology_get_cpu_scale(struct sched_domain *sd, int cpu); +static inline +unsigned long topology_get_cpu_scale(struct sched_domain *sd, int cpu) +{ + return per_cpu(cpu_scale, cpu); +} void topology_set_cpu_scale(unsigned int cpu, unsigned long capacity); -unsigned long topology_get_freq_scale(struct sched_domain *sd, int cpu); +static inline +unsigned long topology_get_freq_scale(struct sched_domain *sd, int cpu) +{ + return per_cpu(freq_scale, cpu); +} void topology_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq, unsigned long max_freq);