From patchwork Wed Mar 27 14:35:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ulf Hansson X-Patchwork-Id: 161263 Delivered-To: patches@linaro.org Received: by 2002:a02:c6d8:0:0:0:0:0 with SMTP id r24csp6561610jan; Wed, 27 Mar 2019 07:36:01 -0700 (PDT) X-Received: by 2002:ac2:4825:: with SMTP id 5mr18776561lft.64.1553697361537; Wed, 27 Mar 2019 07:36:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553697361; cv=none; d=google.com; s=arc-20160816; b=xXIWrpeZX/POvg8F/AurjaSiE8zO0xttPVuwK6d4jimXSeTB/UTqvYkEkc2lKcK/ny rLJF+bntOhJ43ZFbE47WRrgSIV95vwbI03Ye0fmzfN3ACNMfBOcGcrApQ9oOe7T0aEME XgmNB1nM+kvHDY9VMc7kXcJFwdIe2CWpdiyeXe46ImbUlNvul82ewaAm7MjuAuxo7+aT aejtv02BRyeVWxLyG5p4aO9l3polP5VIuLJg9jtad+JafVHqvX4PVx7F5MuEScR664Yg feAh0uN5NHjtCTSwUlEKyltAi+JN8KHFx3jZxKEVXNt/p5PV1yEDmVk7TH4NB72c5uux GE/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5vc0Fh7WVBEgxYINTNsSBAHF75OIY/EqVtxxKOnOMeU=; b=yJa6Ibtv4JtZF2Yye3DaQSsTVlPqvdEEZ4kPFUvRlkljv0sY04+x9ssua9aWyYFqH1 9SJ4FE6VptAHEgy1LMN6uLAJF914Q4b3/U7cACJ6in57ZR9u/7qX93npQFNhkSBCynut bhO7K/0Mr0bXntiB1SIoMD5AJIFSEwAx5z5w6yU6jxvQpdF+c1KVYkZrkoS9o7OgXFQi 6LnapF8LxTp2g+YTQjQpEepV3znP6qgONNCkWHhMWLG7Sgb7s+WUUXG6tyHP4MI+2YDx VW3W46HqiMd3Z/UPAEPSSayjFFHxIDEktDFsv740qCiQ54n0aFZwHvA6BLf7dI4Fr1Sj 352w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MTrdssA0; spf=pass (google.com: domain of ulf.hansson@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=ulf.hansson@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y9sor8753742ljj.19.2019.03.27.07.36.01 for (Google Transport Security); Wed, 27 Mar 2019 07:36:01 -0700 (PDT) Received-SPF: pass (google.com: domain of ulf.hansson@linaro.org designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MTrdssA0; spf=pass (google.com: domain of ulf.hansson@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=ulf.hansson@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5vc0Fh7WVBEgxYINTNsSBAHF75OIY/EqVtxxKOnOMeU=; b=MTrdssA0TPWQa4lNDrtuZYmFjZtQEp9NuJXqUFJnOAP0JcSMEATBgM9kPNKibwinXU +Hd+Ht+XFjZr69eWzu/wdFQb1oaxzSM3Vqkz2IVL2dBjBEZ7pHDVPXOsOEBvd6DNWL7w SJzA23Z7Y/BhudD2sfsUomqLtvK9Id1KfkATcOD1992bWeZAI6VG41atqyO8lZll5aL2 7Mnp0mO3oM5kMbeq09mfXN+TdBg07KXEhJJ4V/z3osDE3X1S5YKv9uHpBJDHbWCoLxQx IYNKfg0a9BAn7WvEBCeEtFW4Jyh4sJGadUhX0hKXJtA4jJT9G1pIQZ7nvHwFWmBJvO6Z tbcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5vc0Fh7WVBEgxYINTNsSBAHF75OIY/EqVtxxKOnOMeU=; b=hqiK6cC/9gnghf64/X0/xedexAk1kuYISjcrObhfTNkiZ3PwBNSI95TwpU/xoXK2du LSyMjMkPPGwOUvb60slYPZ8dKGGx3BA/KEvZIOsV5nMIT2e/SslkAR8H8ozp9IgXQP/X aVx/SvJjJhsRbM15wNQmqdsUlQQaeR1tHIGTH3nydOhhqiiIu6K2dDlLYzN2r0gWTYxL UwoMEvpvGPednrCSHuq5igvn7kYzBrVFC3YUjZWcumzi6xyTpvywvXt6gwiZmB1UNY0/ lj/B80q9A9/smEGCcgGK1K/kgNj3TA1x0HyHA+BkZDYkkPFJT9VRn+YGZXl1tX8AwLkZ WBKQ== X-Gm-Message-State: APjAAAWaA4ulpxL2NIljGbkYCAhTpZ4rijeNghSK7nT1LIJNNWhUqaYv vSlxtr4OtpZzu/Il2IhzFJsY2wEL X-Google-Smtp-Source: APXvYqxM6qi/XDrPHpLp+RXLlmfNtCXJbY2FGNByAhSqp/7Ums1A1rJ2C5zXW0rMIhKX2jVXPoZkDA== X-Received: by 2002:a2e:6c0f:: with SMTP id h15mr19630105ljc.155.1553697361156; Wed, 27 Mar 2019 07:36:01 -0700 (PDT) Return-Path: Received: from localhost.localdomain (h-158-174-22-210.NA.cust.bahnhof.se. [158.174.22.210]) by smtp.gmail.com with ESMTPSA id q2sm4548789lfj.58.2019.03.27.07.35.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Mar 2019 07:35:59 -0700 (PDT) From: Ulf Hansson To: "Rafael J . Wysocki" , linux-pm@vger.kernel.org Cc: Frederic Weisbecker , Thomas Gleixner , Sudeep Holla , Lorenzo Pieralisi , Mark Rutland , Daniel Lezcano , "Raju P . L . S . S . S . N" , Stephen Boyd , Tony Lindgren , Kevin Hilman , Lina Iyer , Ulf Hansson , Viresh Kumar , Vincent Guittot , Geert Uytterhoeven , linux-arm-kernel@lists.infradead.org, linux-arm-msm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v13 3/4] cpuidle: Export the next timer/tick expiration for a CPU Date: Wed, 27 Mar 2019 15:35:47 +0100 Message-Id: <20190327143548.25305-4-ulf.hansson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190327143548.25305-1-ulf.hansson@linaro.org> References: <20190327143548.25305-1-ulf.hansson@linaro.org> To be able to predict the sleep duration for a CPU that is entering idle, knowing when the next timer/tick is going to expire, is extremely useful. Both the teo and the menu cpuidle governors already makes use of this information, while selecting an idle state. Moving forward, the similar prediction needs to be done, but for a group of idle CPUs rather than for a single idle CPU. Following changes implements a new genpd governor, which needs this. Support this, by sharing a new function called tick_nohz_get_next_hrtimer(), which returns the next hrtimer or the next tick, whatever that expires first. Additionally, when cpuidle is about to invoke the ->enter() callback, then call tick_nohz_get_next_hrtimer() and store its return value in the per CPU struct cpuidle_device, as to make it available outside cpuidle. Do note, at the point when cpuidle calls tick_nohz_get_next_hrtimer(), the governor's ->select() callback has already made a decision whether to stop the tick or not. In this way, tick_nohz_get_next_hrtimer() actually returns the next timer expiration, whatever origin. Cc: Lina Iyer Co-developed-by: Lina Iyer Co-developed-by: Daniel Lezcano Acked-by: Daniel Lezcano Signed-off-by: Ulf Hansson --- Changes in v13: - As suggested by Rafael, use WRITE|READ_ONCE when writing/reading the next_hrtimer variable and also avoid it from containing a stale value. --- drivers/cpuidle/cpuidle.c | 19 +++++++++++++++++-- include/linux/cpuidle.h | 1 + include/linux/tick.h | 7 ++++++- kernel/time/tick-sched.c | 12 ++++++++++++ 4 files changed, 36 insertions(+), 3 deletions(-) -- 2.17.1 diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c index 7f108309e871..0f4b7c45df3e 100644 --- a/drivers/cpuidle/cpuidle.c +++ b/drivers/cpuidle/cpuidle.c @@ -328,9 +328,23 @@ int cpuidle_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, int cpuidle_enter(struct cpuidle_driver *drv, struct cpuidle_device *dev, int index) { + int ret = 0; + + /* + * Store the next hrtimer, which becomes either next tick or the next + * timer event, whatever expires first. Additionally, to make this data + * useful for consumers outside cpuidle, we rely on that the governor's + * ->select() callback have decided, whether to stop the tick or not. + */ + WRITE_ONCE(dev->next_hrtimer, tick_nohz_get_next_hrtimer()); + if (cpuidle_state_is_coupled(drv, index)) - return cpuidle_enter_state_coupled(dev, drv, index); - return cpuidle_enter_state(dev, drv, index); + ret = cpuidle_enter_state_coupled(dev, drv, index); + else + ret = cpuidle_enter_state(dev, drv, index); + + WRITE_ONCE(dev->next_hrtimer, 0); + return ret; } /** @@ -511,6 +525,7 @@ static void __cpuidle_device_init(struct cpuidle_device *dev) { memset(dev->states_usage, 0, sizeof(dev->states_usage)); dev->last_residency = 0; + dev->next_hrtimer = 0; } /** diff --git a/include/linux/cpuidle.h b/include/linux/cpuidle.h index 3b39472324a3..bb9a0db89f1a 100644 --- a/include/linux/cpuidle.h +++ b/include/linux/cpuidle.h @@ -83,6 +83,7 @@ struct cpuidle_device { unsigned int use_deepest_state:1; unsigned int poll_time_limit:1; unsigned int cpu; + ktime_t next_hrtimer; int last_residency; struct cpuidle_state_usage states_usage[CPUIDLE_STATE_MAX]; diff --git a/include/linux/tick.h b/include/linux/tick.h index 55388ab45fd4..8891b5ac3e40 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -122,6 +122,7 @@ extern void tick_nohz_idle_enter(void); extern void tick_nohz_idle_exit(void); extern void tick_nohz_irq_exit(void); extern bool tick_nohz_idle_got_tick(void); +extern ktime_t tick_nohz_get_next_hrtimer(void); extern ktime_t tick_nohz_get_sleep_length(ktime_t *delta_next); extern unsigned long tick_nohz_get_idle_calls(void); extern unsigned long tick_nohz_get_idle_calls_cpu(int cpu); @@ -145,7 +146,11 @@ static inline void tick_nohz_idle_restart_tick(void) { } static inline void tick_nohz_idle_enter(void) { } static inline void tick_nohz_idle_exit(void) { } static inline bool tick_nohz_idle_got_tick(void) { return false; } - +static inline ktime_t tick_nohz_get_next_hrtimer(void) +{ + /* Next wake up is the tick period, assume it starts now */ + return ktime_add(ktime_get(), TICK_NSEC); +} static inline ktime_t tick_nohz_get_sleep_length(ktime_t *delta_next) { *delta_next = TICK_NSEC; diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 6fa52cd6df0b..8d18e03124ff 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1022,6 +1022,18 @@ bool tick_nohz_idle_got_tick(void) return false; } +/** + * tick_nohz_get_next_hrtimer - return the next expiration time for the hrtimer + * or the tick, whatever that expires first. Note that, if the tick has been + * stopped, it returns the next hrtimer. + * + * Called from power state control code with interrupts disabled + */ +ktime_t tick_nohz_get_next_hrtimer(void) +{ + return __this_cpu_read(tick_cpu_device.evtdev)->next_event; +} + /** * tick_nohz_get_sleep_length - return the expected length of the current sleep * @delta_next: duration until the next event if the tick cannot be stopped