From patchwork Wed Nov 22 14:35:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 119494 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp254087qgn; Wed, 22 Nov 2017 06:36:34 -0800 (PST) X-Google-Smtp-Source: AGs4zMYKzAAYuNxOhOYBQvZ0mFn+hVOaHCpZbB/kSxyiaJAAV8zNNgaCYw83Mlcibjpx45j7BhW3 X-Received: by 10.98.33.203 with SMTP id o72mr19242063pfj.163.1511361394432; Wed, 22 Nov 2017 06:36:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511361394; cv=none; d=google.com; s=arc-20160816; b=R3mO9zXa4hEdFFnwdpsSgnTT4/Ntkug/ut1ijCgnXzKRFXJQ3Pm/cxaw0qEcfMBgKp Kpf3byvy3p0ksbZRyRR57haxJY8Xz5iMcZeI2IEew7PT7BxP047r88dv1odZ2/1JAZZE pPCuw6vVRCsL+EGfQ2OTTdF/eEJwr57iDN0DHyb7bqSb5+yC6G7OS1/yK0u2bMYsZY5O NozPDMasuDAh51tN0cvoHjVR2Jryxy2nJlFLU1dRhl7vUFqOLeIQiHJzOCloxUu8BX5F 9xpMeY9pyRZ0pmfRgq0pLUu2VYnq5OggBzJJh+tnndilAsaJH1PtX85gtsb91T75zodC LeXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=JpfvcPHa35i386+8TxxK0xdOEQR6x4zGL6DkJiJrAD8=; b=AesZS+ndWnCzLnpF5oD8ye21V9aeLjNB56OoUaPBx806r9bDqaAtV3O82MvigO/7Um zZDr8vxt6TxhIXu3LtVkJGppTbRWiQjJw3wRn/YbUmf3nyundqBGERz3OhF7noAY22tn XyRQ3g8cEgUKsYfp4cIyvGTTLdhv7HSj00HBXFi7vBuxceAv8GHSjS4CzUWr2F8nFUkg fFgWx3E7AZlid1Ms3gIX4eRADQhfJTSVEEoJ26Dt877Js4b/Kkkb8uvn7YwzLkUskj90 TBo4CV+1gaVjApR/GYDnX7pWv+73mzboGNf9gShOWxKfqxnPac5LpmUSWIa0rrMGHOiX BMVQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=h1tEIdoj; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id k10si13407686pgq.750.2017.11.22.06.36.33; Wed, 22 Nov 2017 06:36:34 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=h1tEIdoj; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751677AbdKVOgb (ORCPT + 28 others); Wed, 22 Nov 2017 09:36:31 -0500 Received: from mail-wr0-f194.google.com ([209.85.128.194]:33799 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751274AbdKVOgM (ORCPT ); Wed, 22 Nov 2017 09:36:12 -0500 Received: by mail-wr0-f194.google.com with SMTP id k18so9770562wre.1 for ; Wed, 22 Nov 2017 06:36:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JpfvcPHa35i386+8TxxK0xdOEQR6x4zGL6DkJiJrAD8=; b=h1tEIdoj19IY2Iwzy2LwqqNhktP+r7G+4K3UJWGmTf6/0yAPE+4+inA2hK3HZUGb0d HCDxktkpHlZumXZVLa3Dh3bn5uFFn+4W7Up2KwVoY8cP7kkztT8U5OSSoPloGsrjNg93 RpK0jy2uFuLFStU+lln/Xaj39k57pt8HF+ors= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JpfvcPHa35i386+8TxxK0xdOEQR6x4zGL6DkJiJrAD8=; b=r4lgXQiU5pQwgktKA32gG1CWX6VcuJwFmijTjB4btmxve7Go/m4yuO4Wb0wTQ7Kd2w kUyB+CPxKO+Kfite40rLBQnyiRjLbcrpQJCe4AP3E7CshRWBY/7vvLExY7bWGyN6FrSl jmY8M3ReuhhEOlWPeHB4OP0RO0CTajuLhg++Oz5i7V/uzjGXWIzXZHG7cZoM8/QOeCP1 Zvb4CR4lSG/NB19/bxEtVZL0fasdHliLPGK8jcqQ70hCgUupcMRiWVzkwB/JQTNK+l6w FajOuCmIsTeeJJNyTwj995PDRiE0GJBQA84ygJkrDnqW9zA+PVH9IEf/PLDLd4bugvh+ qY5Q== X-Gm-Message-State: AJaThX6KMxmxxklLpeuEg+4TxZgWHL6JtuzEwiKuof4+ZVaKRfKEsHkp geOM0RrQdej/rOO7O6DGHVIHfA== X-Received: by 10.223.176.150 with SMTP id i22mr19150221wra.257.1511361370875; Wed, 22 Nov 2017 06:36:10 -0800 (PST) Received: from localhost.localdomain ([2a01:e0a:f:6020:f445:1e6c:5367:6c59]) by smtp.gmail.com with ESMTPSA id v18sm7931087wrv.37.2017.11.22.06.36.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 22 Nov 2017 06:36:10 -0800 (PST) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, rjw@rjwysocki.net Cc: juri.lelli@arm.com, dietmar.eggemann@arm.com, Morten.Rasmussen@arm.com, viresh.kumar@linaro.org, Vincent Guittot Subject: [PATCH v3 2/3] sched/rt: add rt_rq utilization tracking Date: Wed, 22 Nov 2017 15:35:54 +0100 Message-Id: <1511361355-10715-3-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1511361355-10715-1-git-send-email-vincent.guittot@linaro.org> References: <1511361355-10715-1-git-send-email-vincent.guittot@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org schedutil governor relies on cfs_rq's util_avg to choose the OPP when cfs tasks are running. When the CPU is overloaded by cfs and rt tasks, cfs tasks are preempted by rt tasks and in this case util_avg reflects the remaining capacity that is used by cfs task but not what cfs want to use. In such case, schedutil can select a lower OPP whereas the CPU is overloaded. In order to have a more accurate view of the utilization of the CPU, we track the utilization that is "stolen" by RT tasks. Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 2 ++ kernel/sched/pelt.c | 23 +++++++++++++++++++++++ kernel/sched/pelt.h | 7 +++++++ kernel/sched/rt.c | 9 +++++++++ kernel/sched/sched.h | 1 + 5 files changed, 42 insertions(+) -- 2.7.4 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b88550e..57d486a 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7029,6 +7029,7 @@ static void update_blocked_averages(int cpu) if (cfs_rq_is_decayed(cfs_rq)) list_del_leaf_cfs_rq(cfs_rq); } + update_rt_rq_load_avg(rq_clock_task(rq), cpu, &rq->rt, 0); rq_unlock_irqrestore(rq, &rf); } @@ -7088,6 +7089,7 @@ static inline void update_blocked_averages(int cpu) rq_lock_irqsave(rq, &rf); update_rq_clock(rq); update_cfs_rq_load_avg(cfs_rq_clock_task(cfs_rq), cfs_rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu, &rq->rt, 0); rq_unlock_irqrestore(rq, &rf); } diff --git a/kernel/sched/pelt.c b/kernel/sched/pelt.c index da6d84f..c8b5d23 100644 --- a/kernel/sched/pelt.c +++ b/kernel/sched/pelt.c @@ -306,3 +306,26 @@ int __update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq) return 0; } + +/* + * rt_rq: + * + * util_sum = \Sum se->avg.util_sum but se->avg.util_sum is not tracked + * util_sum = cpu_scale * load_sum + * runnable_load_sum = load_sum + * + */ + +int update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running) +{ + if (___update_load_sum(now, cpu, &rt_rq->avg, + running, + running, + running)) { + + ___update_load_avg(&rt_rq->avg, 1, 1); + return 1; + } + + return 0; +} diff --git a/kernel/sched/pelt.h b/kernel/sched/pelt.h index c312d8c..78a2107 100644 --- a/kernel/sched/pelt.h +++ b/kernel/sched/pelt.h @@ -3,6 +3,7 @@ int __update_load_avg_blocked_se(u64 now, int cpu, struct sched_entity *se); int __update_load_avg_se(u64 now, int cpu, struct cfs_rq *cfs_rq, struct sched_entity *se); int __update_load_avg_cfs_rq(u64 now, int cpu, struct cfs_rq *cfs_rq); +int update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running); #else @@ -12,6 +13,12 @@ update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq) return 0; } +static inline int +update_rt_rq_load_avg(u64 now, int cpu, struct rt_rq *rt_rq, int running) +{ + return 0; +} + #endif diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index d8c43d7..2ddcc27 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -9,6 +9,8 @@ #include #include +#include "pelt.h" + int sched_rr_timeslice = RR_TIMESLICE; int sysctl_sched_rr_timeslice = (MSEC_PER_SEC / HZ) * RR_TIMESLICE; @@ -1569,6 +1571,10 @@ pick_next_task_rt(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) queue_push_tasks(rq); + if (p) + update_rt_rq_load_avg(rq_clock_task(rq), cpu_of(rq), rt_rq, + rq->curr->sched_class == &rt_sched_class); + return p; } @@ -1576,6 +1582,8 @@ static void put_prev_task_rt(struct rq *rq, struct task_struct *p) { update_curr_rt(rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu_of(rq), &rq->rt, 1); + /* * The previous task needs to be made eligible for pushing * if it is still active @@ -2283,6 +2291,7 @@ static void task_tick_rt(struct rq *rq, struct task_struct *p, int queued) struct sched_rt_entity *rt_se = &p->rt; update_curr_rt(rq); + update_rt_rq_load_avg(rq_clock_task(rq), cpu_of(rq), &rq->rt, 1); watchdog(rq, p); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 6fefef6..c7bd5dd 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -529,6 +529,7 @@ struct rt_rq { int overloaded; struct plist_head pushable_tasks; + struct sched_avg avg; #endif /* CONFIG_SMP */ int rt_queued;