From patchwork Sun Aug 20 21:06:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qais Yousef X-Patchwork-Id: 715633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1777CEE49AF for ; Sun, 20 Aug 2023 21:08:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231893AbjHTVIT (ORCPT ); Sun, 20 Aug 2023 17:08:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230169AbjHTVIS (ORCPT ); Sun, 20 Aug 2023 17:08:18 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71A384222 for ; Sun, 20 Aug 2023 14:08:05 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id ffacd0b85a97d-31c5a2e8501so8614f8f.0 for ; Sun, 20 Aug 2023 14:08:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20221208.gappssmtp.com; s=20221208; t=1692565684; x=1693170484; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c6qFJPzVrYvXdJFLmkCCK0atIYbjN/54u4wyc75/JBA=; b=HgG+fO1k7dm+jpnKvEM5X5dUvf//duLry6vdxj39GAjwCvPChkZJ8V6NrPGsycssq6 WBi4ahdeVIDEBq2LPdZCZKF8GX++0qJ/OwIiqTxyaKrRo9r4fCKc6TP36YZkImJBVoEJ dJz1P3os9ETaqjObgP/XD7xDMyZrScTkZQVQauNzJ8oDELiqWLY07ADiEQ46vCr1fb87 L/vQ0U8ue4POOVJwDdSSXO/LxX5frec8MHiD8zyKEak7BWWiBFy3X+dKxCBAI8TQCiRl DrdistocMSk3YQbjmIpESR5eLSXLmon5CDzSxue3x4M0klKy2wzg+Yp17lhADIZVFX8u X32w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692565684; x=1693170484; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c6qFJPzVrYvXdJFLmkCCK0atIYbjN/54u4wyc75/JBA=; b=UnpykcFJucGtCLmxowbzqez4zL/dKnwagOt7Bl6426XoDi5qVxsOmH/ZBcRo17698M JbUJ62DAACiXWdyL/LAc67cRRNSl/IbizlmPVUcRQFkPOaCaaj8MFHCWzgLVwaXEDDa1 EdD834/TBGhN0LBMcelC4mLZ9/pGaincaTUnaOBc12xP/udAMKSp6m4V2CweadOltV1u FhRS4PmiVD1bmnkZRVua1HDA/ZbbUCt4lbhhgXWZEZ1snpDrj89IQCPZVfQIF2ldApMp d/YR2o826Qzq4tS6irvVDoZmZHSRqS+LpRw8hpPE+ccZuFIgTzQhp9+IR4N7LK76pLSP 3w9A== X-Gm-Message-State: AOJu0YwzIPl+lIKbUEwu8Llqk39X7iHJNbNcLNYztKfscHm/OGNvaHnt wP9mdMAOlyoBH5T/JSFrVKX4LA== X-Google-Smtp-Source: AGHT+IGTizLO4t/BYhWwB0zqwdQLDBDyphgkRD7gCuJ1OhbkmLiRC3MOreod/y3MppTTTWxtqnCK6g== X-Received: by 2002:a5d:4fc6:0:b0:319:5e0d:9b0f with SMTP id h6-20020a5d4fc6000000b003195e0d9b0fmr3356883wrw.33.1692565684026; Sun, 20 Aug 2023 14:08:04 -0700 (PDT) Received: from airbuntu.. (host109-151-228-137.range109-151.btcentralplus.com. [109.151.228.137]) by smtp.gmail.com with ESMTPSA id j6-20020adfea46000000b003197a4b0f68sm10336800wrn.7.2023.08.20.14.08.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Aug 2023 14:08:03 -0700 (PDT) From: Qais Yousef To: "Rafael J. Wysocki" , Viresh Kumar , Ingo Molnar , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Vincent Guittot , Dietmar Eggemann , Lukasz Luba , Qais Yousef Subject: [PATCH 1/4] sched: cpufreq: Rename map_util_perf to apply_dvfs_headroom Date: Sun, 20 Aug 2023 22:06:37 +0100 Message-Id: <20230820210640.585311-2-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230820210640.585311-1-qyousef@layalina.io> References: <20230820210640.585311-1-qyousef@layalina.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org We are providing headroom for the utilization to grow until the next decision point to pick the next frequency. Give the function a better name and give it some documentation. It is not really mapping anything. Provide a dummy definition for !CONFIG_CPUFREQ which will be required for later patches. Signed-off-by: Qais Yousef (Google) --- include/linux/energy_model.h | 2 +- include/linux/sched/cpufreq.h | 27 ++++++++++++++++++++++++++- kernel/sched/cpufreq_schedutil.c | 6 +++--- 3 files changed, 30 insertions(+), 5 deletions(-) diff --git a/include/linux/energy_model.h b/include/linux/energy_model.h index b9caa01dfac4..6ebde4e69e98 100644 --- a/include/linux/energy_model.h +++ b/include/linux/energy_model.h @@ -243,7 +243,7 @@ static inline unsigned long em_cpu_energy(struct em_perf_domain *pd, scale_cpu = arch_scale_cpu_capacity(cpu); ps = &pd->table[pd->nr_perf_states - 1]; - max_util = map_util_perf(max_util); + max_util = apply_dvfs_headroom(max_util); max_util = min(max_util, allowed_cpu_cap); freq = map_util_freq(max_util, ps->frequency, scale_cpu); diff --git a/include/linux/sched/cpufreq.h b/include/linux/sched/cpufreq.h index bdd31ab93bc5..f0069b354ac8 100644 --- a/include/linux/sched/cpufreq.h +++ b/include/linux/sched/cpufreq.h @@ -29,10 +29,35 @@ static inline unsigned long map_util_freq(unsigned long util, return freq * util / cap; } -static inline unsigned long map_util_perf(unsigned long util) +/* + * DVFS decision are made at discrete points. If CPU stays busy, the util will + * continue to grow, which means it could need to run at a higher frequency + * before the next decision point was reached. IOW, we can't follow the util as + * it grows immediately, but there's a delay before we issue a request to go to + * higher frequency. The headroom caters for this delay so the system continues + * to run at adequate performance point. + * + * This function provides enough headroom to provide adequate performance + * assuming the CPU continues to be busy. + * + * At the moment it is a constant multiplication with 1.25. + * + * TODO: The headroom should be a function of the delay. 25% is too high + * especially on powerful systems. For example, if the delay is 500us, it makes + * more sense to give a small headroom as the next decision point is not far + * away and will follow the util if it continues to rise. On the other hand if + * the delay is 10ms, then we need a bigger headroom so the CPU won't struggle + * at a lower frequency if it never goes to idle until then. + */ +static inline unsigned long apply_dvfs_headroom(unsigned long util) { return util + (util >> 2); } +#else +static inline unsigned long apply_dvfs_headroom(unsigned long util) +{ + return util; +} #endif /* CONFIG_CPU_FREQ */ #endif /* _LINUX_SCHED_CPUFREQ_H */ diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index 4492608b7d7f..916c4d3d6192 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -143,7 +143,7 @@ static unsigned int get_next_freq(struct sugov_policy *sg_policy, unsigned int freq = arch_scale_freq_invariant() ? policy->cpuinfo.max_freq : policy->cur; - util = map_util_perf(util); + util = apply_dvfs_headroom(util); freq = map_util_freq(util, freq, max); if (freq == sg_policy->cached_raw_freq && !sg_policy->need_freq_update) @@ -406,8 +406,8 @@ static void sugov_update_single_perf(struct update_util_data *hook, u64 time, sugov_cpu_is_busy(sg_cpu) && sg_cpu->util < prev_util) sg_cpu->util = prev_util; - cpufreq_driver_adjust_perf(sg_cpu->cpu, map_util_perf(sg_cpu->bw_dl), - map_util_perf(sg_cpu->util), max_cap); + cpufreq_driver_adjust_perf(sg_cpu->cpu, apply_dvfs_headroom(sg_cpu->bw_dl), + apply_dvfs_headroom(sg_cpu->util), max_cap); sg_cpu->sg_policy->last_freq_update_time = time; } From patchwork Sun Aug 20 21:06:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qais Yousef X-Patchwork-Id: 715634 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E23A3EE49AA for ; Sun, 20 Aug 2023 21:08:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231271AbjHTVIU (ORCPT ); Sun, 20 Aug 2023 17:08:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231334AbjHTVIS (ORCPT ); Sun, 20 Aug 2023 17:08:18 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB3E149C0 for ; Sun, 20 Aug 2023 14:08:06 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-319e93a1594so2502178f8f.1 for ; Sun, 20 Aug 2023 14:08:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20221208.gappssmtp.com; s=20221208; t=1692565685; x=1693170485; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jIPeq0k+JV1/cJi07cu7HqrUCm18GlZjB51v4GbQCSk=; b=oitvAfMTAL2TAzCpYa9n1HXigQmMES6+P4QDHTX71Tt0ktjQnBddDi2qMOk+f7RKtp ftHLz7jUn22Q0Qi2lP5zE50K8EwsYA/iu/rI+MEBXwzkdlDx6+tH47K4jEBzf39iYX0f uUWONXWXUi0RSZox/2hr0t+D2+j8xBXoExHBcQCkApZJ7DVcjD4C27lTe/soYgGB1LpL QPbkvnQ3/RSVFu3HnJUIp42wHIHHoMqV1N3P0s5jixalznezKx2uIJ0MYBwH3NMY3RFi 29EJu0ZIg97o+uN1lU0i6EtiayWkizBVlggSfFJDE8MS1Z5NiMD+vEvUYxl0NS4cz9v2 Oygg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692565685; x=1693170485; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jIPeq0k+JV1/cJi07cu7HqrUCm18GlZjB51v4GbQCSk=; b=j2v6aKCFmjk6ewVNNufWfwB4gLdmFkoYoJUNIlt/J+CVb+pO5o7wzVrUGYj+iKAHS7 1jU6AvU0otNo9cfLgbphMqwoyU4fPwc7/bw5os2Vm+U0ABIACQI7lJDovi986U0Nx71x XnZn99YK7+813yfmCLNCqxw3I0+xD2B66bw78Wk9WNzjsEXlWWVVinfo9TZIH6K9mTMq fhcnfv1jpOo5fYLUMzHVHVoIeLRc+YON2Zn1c40XVOr19EI6FPTmOrRVT387L9CYtEM0 is0fR3oP+KxvtbArggAY4Dy8fTD128gwaD96NBgFLzDze4+9D/upSNaZzlhvhs5K+TdQ cJAQ== X-Gm-Message-State: AOJu0YyPzBF7ZE0wWNHHvzF6Bg6OrmhBF6Aemwf17sgIYEdTIJlAjjRJ M/dxpXDAsrTLX5vasrdGhH0D2Q== X-Google-Smtp-Source: AGHT+IFmFD8f6fkzSvJchuznRWY9tDHKmOC6fE/r10rQ8Sa0lLB9+P3BHUwp0xhJBhBHNvoAvdlxXg== X-Received: by 2002:adf:ee41:0:b0:31a:e772:ddf0 with SMTP id w1-20020adfee41000000b0031ae772ddf0mr3826175wro.53.1692565684931; Sun, 20 Aug 2023 14:08:04 -0700 (PDT) Received: from airbuntu.. (host109-151-228-137.range109-151.btcentralplus.com. [109.151.228.137]) by smtp.gmail.com with ESMTPSA id j6-20020adfea46000000b003197a4b0f68sm10336800wrn.7.2023.08.20.14.08.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Aug 2023 14:08:04 -0700 (PDT) From: Qais Yousef To: "Rafael J. Wysocki" , Viresh Kumar , Ingo Molnar , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Vincent Guittot , Dietmar Eggemann , Lukasz Luba , Qais Yousef Subject: [PATCH 2/4] sched: cpufreq: Fix apply_dvfs_headroom() escaping uclamp constraints Date: Sun, 20 Aug 2023 22:06:38 +0100 Message-Id: <20230820210640.585311-3-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230820210640.585311-1-qyousef@layalina.io> References: <20230820210640.585311-1-qyousef@layalina.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org DVFS headroom is applied after we calculate the effective_cpu_util() which is where we honour uclamp constraints. It makes more sense to apply the headroom there once and let all users naturally get the right thing without having to sprinkle the call around in various places. Before this fix running uclampset -M 800 cat /dev/zero > /dev/null Will cause the test system to run at max freq of 2.8GHz. After the fix it runs at 2.2GHz instead which is the correct value that matches the capacity of 800. Note that similar problem exist for uclamp_min. If util was 50, and uclamp_min is 100. Since we apply_dvfs_headroom() after apply uclamp constraints, we'll end up with util of 125 instead of 100. IOW, we get boosted twice, first time by uclamp_min, and second time by dvfs headroom. Signed-off-by: Qais Yousef (Google) --- include/linux/energy_model.h | 1 - kernel/sched/core.c | 11 ++++++++--- kernel/sched/cpufreq_schedutil.c | 5 ++--- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/include/linux/energy_model.h b/include/linux/energy_model.h index 6ebde4e69e98..adec808b371a 100644 --- a/include/linux/energy_model.h +++ b/include/linux/energy_model.h @@ -243,7 +243,6 @@ static inline unsigned long em_cpu_energy(struct em_perf_domain *pd, scale_cpu = arch_scale_cpu_capacity(cpu); ps = &pd->table[pd->nr_perf_states - 1]; - max_util = apply_dvfs_headroom(max_util); max_util = min(max_util, allowed_cpu_cap); freq = map_util_freq(max_util, ps->frequency, scale_cpu); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index efe3848978a0..441d433c83cd 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -7439,8 +7439,10 @@ unsigned long effective_cpu_util(int cpu, unsigned long util_cfs, * frequency will be gracefully reduced with the utilization decay. */ util = util_cfs + cpu_util_rt(rq); - if (type == FREQUENCY_UTIL) + if (type == FREQUENCY_UTIL) { + util = apply_dvfs_headroom(util); util = uclamp_rq_util_with(rq, util, p); + } dl_util = cpu_util_dl(rq); @@ -7471,9 +7473,12 @@ unsigned long effective_cpu_util(int cpu, unsigned long util_cfs, * max - irq * U' = irq + --------- * U * max + * + * We only need to apply dvfs headroom to irq part since the util part + * already had it applied. */ util = scale_irq_capacity(util, irq, max); - util += irq; + util += type == FREQUENCY_UTIL ? apply_dvfs_headroom(irq) : irq; /* * Bandwidth required by DEADLINE must always be granted while, for @@ -7486,7 +7491,7 @@ unsigned long effective_cpu_util(int cpu, unsigned long util_cfs, * an interface. So, we only do the latter for now. */ if (type == FREQUENCY_UTIL) - util += cpu_bw_dl(rq); + util += apply_dvfs_headroom(cpu_bw_dl(rq)); return min(max, util); } diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index 916c4d3d6192..0c7565ac31fb 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -143,7 +143,6 @@ static unsigned int get_next_freq(struct sugov_policy *sg_policy, unsigned int freq = arch_scale_freq_invariant() ? policy->cpuinfo.max_freq : policy->cur; - util = apply_dvfs_headroom(util); freq = map_util_freq(util, freq, max); if (freq == sg_policy->cached_raw_freq && !sg_policy->need_freq_update) @@ -406,8 +405,8 @@ static void sugov_update_single_perf(struct update_util_data *hook, u64 time, sugov_cpu_is_busy(sg_cpu) && sg_cpu->util < prev_util) sg_cpu->util = prev_util; - cpufreq_driver_adjust_perf(sg_cpu->cpu, apply_dvfs_headroom(sg_cpu->bw_dl), - apply_dvfs_headroom(sg_cpu->util), max_cap); + cpufreq_driver_adjust_perf(sg_cpu->cpu, sg_cpu->bw_dl, + sg_cpu->util, max_cap); sg_cpu->sg_policy->last_freq_update_time = time; } From patchwork Sun Aug 20 21:06:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qais Yousef X-Patchwork-Id: 715635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A00CEEE49A4 for ; Sun, 20 Aug 2023 21:08:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231837AbjHTVIU (ORCPT ); Sun, 20 Aug 2023 17:08:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41504 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231586AbjHTVIS (ORCPT ); Sun, 20 Aug 2023 17:08:18 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 507CB49C4 for ; Sun, 20 Aug 2023 14:08:07 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-3feef504ccbso5037135e9.2 for ; Sun, 20 Aug 2023 14:08:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20221208.gappssmtp.com; s=20221208; t=1692565686; x=1693170486; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FvS6i5MmlM83okIt3hu6QZE/yTaGAFd1OJzWlOJdeII=; b=eUJdEfU+wCh7GbOSrOcgXSU+omBihmSIkylrFWG+wpsvLidpTcilplQelHH1voLoO7 3fypW+Xof0JkFMUwbwJPZBcBbhasRAZIHXFoD6eHqTREGlT9HPDtVEpIG1XzUrJR07gA KqrDzd/ZhelmjWYKeYPF7d7FYr9JD8kkaOQIFd3+2GxGt2kfaso/vGEWk0lwMNpI/Fgy XIHKOBXpGWXDg6wU6AEqLvkTabTxbnjl5nTTAZbABRiR432mhjRpryHfra/Z6OcU76cF 6CyjNjBHbzzDcjjSIZC2Vn1Jlqrz7nf6Ry6HPF1rgcBbx/hB0s4Q/9pe2x8wfOIIT25N YYlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692565686; x=1693170486; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FvS6i5MmlM83okIt3hu6QZE/yTaGAFd1OJzWlOJdeII=; b=idLkoQRgeju7owkv9MLfDMG1YWCiNW2SpNA0qRrhek4cVWFXPilX+Q7Y3LiPMQV+1W yetIDQNjDOPcWc5Of7UZ5Jal8zeocYSB7+PtoB6yhgT08dV0VYNHrKwZRS4CDD7dMMGs v0z1UZ+N8/7JyGmHXIjxNcV3XUbLcjPrTXoABsEDNiluHuLxcfMDJDCCKRAOP3fD3JrC iTChSAtTddTMG9nqTzYZp1LJTzIaCdBp72kx6Q4DL3EwEU5kpVG/dpj9IW4qEriY9OBP +oNmVOOcjhf+srTGmVk8u7vlXl4VejTEprowBeSx1xZ7cKEDY8AhzFDx3ncLScVrgRKw Oqxw== X-Gm-Message-State: AOJu0YwPjPqXHgYls5fAh3L2cpalaabsgMqWp/ezd850Y1pWGtOHlI5c axnfpFn0WdzSfT48wTl8i1TmXA== X-Google-Smtp-Source: AGHT+IEjvAnle4Qh/tgCe9Abtc/2UPfgazY/8no6cp5UNC4rcPBG4Ofv/HaNWmre2NOn8zi6Sbi0Kg== X-Received: by 2002:adf:cf02:0:b0:317:6b0d:1b1 with SMTP id o2-20020adfcf02000000b003176b0d01b1mr3791525wrj.4.1692565685830; Sun, 20 Aug 2023 14:08:05 -0700 (PDT) Received: from airbuntu.. (host109-151-228-137.range109-151.btcentralplus.com. [109.151.228.137]) by smtp.gmail.com with ESMTPSA id j6-20020adfea46000000b003197a4b0f68sm10336800wrn.7.2023.08.20.14.08.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Aug 2023 14:08:05 -0700 (PDT) From: Qais Yousef To: "Rafael J. Wysocki" , Viresh Kumar , Ingo Molnar , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Vincent Guittot , Dietmar Eggemann , Lukasz Luba , Qais Yousef Subject: [PATCH 3/4] sched: cpufreq: Move apply_dvfs_headroom() to sched.h Date: Sun, 20 Aug 2023 22:06:39 +0100 Message-Id: <20230820210640.585311-4-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230820210640.585311-1-qyousef@layalina.io> References: <20230820210640.585311-1-qyousef@layalina.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org This function relies on updating util signal appropriately to give a headroom to grow. This is more of a scheduler functionality than cpufreq. Move it to sched.h where all the other util handling code belongs. Signed-off-by: Qais Yousef (Google) --- include/linux/sched/cpufreq.h | 30 ------------------------------ kernel/sched/sched.h | 24 ++++++++++++++++++++++++ 2 files changed, 24 insertions(+), 30 deletions(-) diff --git a/include/linux/sched/cpufreq.h b/include/linux/sched/cpufreq.h index f0069b354ac8..d01755d3142f 100644 --- a/include/linux/sched/cpufreq.h +++ b/include/linux/sched/cpufreq.h @@ -28,36 +28,6 @@ static inline unsigned long map_util_freq(unsigned long util, { return freq * util / cap; } - -/* - * DVFS decision are made at discrete points. If CPU stays busy, the util will - * continue to grow, which means it could need to run at a higher frequency - * before the next decision point was reached. IOW, we can't follow the util as - * it grows immediately, but there's a delay before we issue a request to go to - * higher frequency. The headroom caters for this delay so the system continues - * to run at adequate performance point. - * - * This function provides enough headroom to provide adequate performance - * assuming the CPU continues to be busy. - * - * At the moment it is a constant multiplication with 1.25. - * - * TODO: The headroom should be a function of the delay. 25% is too high - * especially on powerful systems. For example, if the delay is 500us, it makes - * more sense to give a small headroom as the next decision point is not far - * away and will follow the util if it continues to rise. On the other hand if - * the delay is 10ms, then we need a bigger headroom so the CPU won't struggle - * at a lower frequency if it never goes to idle until then. - */ -static inline unsigned long apply_dvfs_headroom(unsigned long util) -{ - return util + (util >> 2); -} -#else -static inline unsigned long apply_dvfs_headroom(unsigned long util) -{ - return util; -} #endif /* CONFIG_CPU_FREQ */ #endif /* _LINUX_SCHED_CPUFREQ_H */ diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 3a01b7a2bf66..56eeb5b05b50 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2997,6 +2997,30 @@ enum cpu_util_type { unsigned long effective_cpu_util(int cpu, unsigned long util_cfs, enum cpu_util_type type, struct task_struct *p); +/* + * DVFS decision are made at discrete points. If CPU stays busy, the util will + * continue to grow, which means it could need to run at a higher frequency + * before the next decision point was reached. IOW, we can't follow the util as + * it grows immediately, but there's a delay before we issue a request to go to + * higher frequency. The headroom caters for this delay so the system continues + * to run at adequate performance point. + * + * This function provides enough headroom to provide adequate performance + * assuming the CPU continues to be busy. + * + * At the moment it is a constant multiplication with 1.25. + * + * TODO: The headroom should be a function of the delay. 25% is too high + * especially on powerful systems. For example, if the delay is 500us, it makes + * more sense to give a small headroom as the next decision point is not far + * away and will follow the util if it continues to rise. On the other hand if + * the delay is 10ms, then we need a bigger headroom so the CPU won't struggle + * at a lower frequency if it never goes to idle until then. + */ +static inline unsigned long apply_dvfs_headroom(unsigned long util) +{ + return util + (util >> 2); +} /* * Verify the fitness of task @p to run on @cpu taking into account the From patchwork Sun Aug 20 21:06:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qais Yousef X-Patchwork-Id: 715314 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0E95EE49A8 for ; Sun, 20 Aug 2023 21:08:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232000AbjHTVIW (ORCPT ); Sun, 20 Aug 2023 17:08:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231710AbjHTVIS (ORCPT ); Sun, 20 Aug 2023 17:08:18 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 362EC49CB for ; Sun, 20 Aug 2023 14:08:08 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id ffacd0b85a97d-31aec0a1a8bso778488f8f.0 for ; Sun, 20 Aug 2023 14:08:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20221208.gappssmtp.com; s=20221208; t=1692565687; x=1693170487; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=P3gpb+XmkO3kHE5m1LvhKTjxDnutMdnBN1ptIU0mlos=; b=Vx7ub+UEHibwSm58rblsz2lbfLZYMFzAFCYEr4wy10Ysmns2ib1N5rU/Vo3rQ2V3LD Jcei/nJED5cDuWTudsVY9OR8FbJf80QsAktDBH/K/k6J/cyfMIDSDKR3eOUL6T9INIgz aG7BNn5GZaV6HIpyUVtt3glePhkW9IkK+M9Uqt/1W8oXsIU+RNmNdZgxxoloso8UaQx7 WUoGHXBYpXunJrf8zmDxjPSdhdiIpHj6Psg2/oTF2+ix7r1celKs0Ujtd7ApQqrp7L+e 8HaQA0QUuZy0ndUvkdEWzE/ziXX7wNq41QeL9uD43h1zi50LHDC1iVZ/mmZ4bV2qXYAN eR6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692565687; x=1693170487; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P3gpb+XmkO3kHE5m1LvhKTjxDnutMdnBN1ptIU0mlos=; b=a7h/E4GXjGpjvg1JcfP0xNC1/xCcEewMWBpAcJgdEzLpNVe1xSq/WcUZEWmNokDw+V iRQTSmd6JqDtVM7pUtiCf6VbNBz8Yv/Id6NLwOvzajwT4GZfDgkXz8nMmbyy+vcacpaV M3xnSn2V5tgJqnu85GWbJCh2vMC2PKOVcQLs4c7UmDxj6+KcSTIfyAG+bN2Z8iazQrFq 1KGlc+OnyR0Pn6Q/O8ZhPbXg5WQxYGeQaUZxNDSJUmIezjaX1dJVqBVmntuDErKZmuzh WzHQZ+ie9kg83wYTTioJsCVS5WlYy47jia473WDu5qGnTFsGM9EuNBtp8b+xSYLwldVW 80KQ== X-Gm-Message-State: AOJu0Yx3fXyfS6AymuB1KlsfMsMiVoKIzz/IR0POCw8Y00oon1gIuIKT Z9n18wO158jjty2CZ30qhfDbdA== X-Google-Smtp-Source: AGHT+IGfJM4ptG529Qy8vK3PBDRC2tNg/jFwDG2NVh2mgUavtRwvq099OxDsEoWdWeDZdcMPOsCn7Q== X-Received: by 2002:a5d:498e:0:b0:31a:e3df:f719 with SMTP id r14-20020a5d498e000000b0031ae3dff719mr3327502wrq.17.1692565686660; Sun, 20 Aug 2023 14:08:06 -0700 (PDT) Received: from airbuntu.. (host109-151-228-137.range109-151.btcentralplus.com. [109.151.228.137]) by smtp.gmail.com with ESMTPSA id j6-20020adfea46000000b003197a4b0f68sm10336800wrn.7.2023.08.20.14.08.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Aug 2023 14:08:06 -0700 (PDT) From: Qais Yousef To: "Rafael J. Wysocki" , Viresh Kumar , Ingo Molnar , Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Vincent Guittot , Dietmar Eggemann , Lukasz Luba , Qais Yousef Subject: [PATCH RFC 4/4] sched: cpufreq: Apply DVFS headroom to CFS only Date: Sun, 20 Aug 2023 22:06:40 +0100 Message-Id: <20230820210640.585311-5-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230820210640.585311-1-qyousef@layalina.io> References: <20230820210640.585311-1-qyousef@layalina.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org RT and Deadline have exact performance requirement when running. RT runs at max or a specific OPP defined by uclamp_min. Deadline's OPP is defined by its bandwidth. Both of which are known ahead of time and don't require a headroom to grow into. IRQs on the other hand have no specific performance requirement and cruises along at whatever the current OPP happens to be when they occur. Now they all have PELT pressure signals that does impact frequency selection and task placement. The question is do they need DVFS headroom? I think the answer is no because when CFS is not running at all, these pressure signal has no real impact on performance for RT, DL or IRQ. If CFS util is not zero, we already add their pressure as an *additional* headroom to account for the lost/stolen time. So I argue that the pressure are headroom themselves and shouldn't need an additional DVFS headroom applied on top. In summary final outcome should be: CFS + DVFS headroom + (RT, DT, IRQ) pressure headroom Signed-off-by: Qais Yousef (Google) --- kernel/sched/core.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 441d433c83cd..602e369753a3 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -7438,10 +7438,11 @@ unsigned long effective_cpu_util(int cpu, unsigned long util_cfs, * When there are no CFS RUNNABLE tasks, clamps are released and * frequency will be gracefully reduced with the utilization decay. */ - util = util_cfs + cpu_util_rt(rq); if (type == FREQUENCY_UTIL) { - util = apply_dvfs_headroom(util); + util = apply_dvfs_headroom(util_cfs) + cpu_util_rt(rq); util = uclamp_rq_util_with(rq, util, p); + } else { + util = util_cfs + cpu_util_rt(rq); } dl_util = cpu_util_dl(rq); @@ -7473,12 +7474,9 @@ unsigned long effective_cpu_util(int cpu, unsigned long util_cfs, * max - irq * U' = irq + --------- * U * max - * - * We only need to apply dvfs headroom to irq part since the util part - * already had it applied. */ util = scale_irq_capacity(util, irq, max); - util += type == FREQUENCY_UTIL ? apply_dvfs_headroom(irq) : irq; + util += irq; /* * Bandwidth required by DEADLINE must always be granted while, for @@ -7491,7 +7489,7 @@ unsigned long effective_cpu_util(int cpu, unsigned long util_cfs, * an interface. So, we only do the latter for now. */ if (type == FREQUENCY_UTIL) - util += apply_dvfs_headroom(cpu_bw_dl(rq)); + util += cpu_bw_dl(rq); return min(max, util); }