From patchwork Sun Dec 6 11:17:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 338817 Delivered-To: patch@linaro.org Received: by 2002:a02:85a7:0:0:0:0:0 with SMTP id d36csp1955475jai; Sun, 6 Dec 2020 03:44:19 -0800 (PST) X-Google-Smtp-Source: ABdhPJyyF0kWr7jtnhxnXcXTcaiLqCHS+jSKBdQ2V6cj55/eeN03jDJ3Oa/ZkLj0nStz42CFgV2V X-Received: by 2002:a05:6402:1841:: with SMTP id v1mr15907398edy.194.1607254959438; Sun, 06 Dec 2020 03:42:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1607254959; cv=none; d=google.com; s=arc-20160816; b=oICH2Y3PDLNrLog++xi/rTkx8iSSdmYTCHc5z77Pzf3/+fXAQLYnDQjEAoWBg6vaBy DLUIamZ5e8H5tRpBhEVaJ9pmAYm/ML8Fu9hUPAaQKjmlPlBe7tkQF0blsevbrqM2ZxyG qXTwFGiuYcDByz15M7mnTgQL7K1BmWhfCNwNp23oXLg2Jd2RtGOlheJpBBDIGWIozAhJ 9C6L9FxTiYC3CY8DKxnkLvp970kTjRJoMCcGl9o4wS7Erpb0KSJDI4bp/+URSvbbmGrH 6/KBXU1CmyLrlR397eRr6WVQP60jzixYUriUdBYJQIyXnNIk2BFgPTyQNtNjCNWYGayo Ykbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from; bh=KRzW2GBM5tCsvSsYlQa5Y/qzSwu5uUHLjB6SrksfrCw=; b=kn0xtLhMhuX4biPyaHn1R0HFutv2q3pEW/tfBQ/H7+5VoOojgCzYhmFyefgCSRhIxu rZTGiepbmfwT3OOhiiC0mG/q2zuUkFURJDZfWglnN/0u0dZXb0ZphNilNy9bV7Oq70C2 XFXr/oV40Gb6hIoNrc51vXg0NdHY6wQEunWQ28cArdfN8CgmC1vwkxj7Gs+oJRhY0O/Z Zokgi94+r0FL9zV/jd914Esu935s73Zf5xNm7etNL2figMLOPv5CclVPai2FcKCR6ovW bmX5WkQH2fagjt59VLAZbEYJyXgGPM3k3s4h5yOMTqbMMQP1TEvNz581yRBb1otkR1He E6Ug== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id o19si6199165edw.119.2020.12.06.03.42.39; Sun, 06 Dec 2020 03:42:39 -0800 (PST) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728858AbgLFLlv (ORCPT + 14 others); Sun, 6 Dec 2020 06:41:51 -0500 Received: from mail.kernel.org ([198.145.29.99]:39504 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728866AbgLFLlp (ORCPT ); Sun, 6 Dec 2020 06:41:45 -0500 From: Greg Kroah-Hartman Authentication-Results: mail.kernel.org; dkim=permerror (bad message/signature format) To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Vincent Guittot , "Peter Zijlstra (Intel)" , Phil Auld , Ben Segall , Sasha Levin Subject: [PATCH 5.4 14/39] sched/fair: Fix unthrottle_cfs_rq() for leaf_cfs_rq list Date: Sun, 6 Dec 2020 12:17:18 +0100 Message-Id: <20201206111555.364264359@linuxfoundation.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201206111554.677764505@linuxfoundation.org> References: <20201206111554.677764505@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Vincent Guittot [ Upstream commit 39f23ce07b9355d05a64ae303ce20d1c4b92b957 ] Although not exactly identical, unthrottle_cfs_rq() and enqueue_task_fair() are quite close and follow the same sequence for enqueuing an entity in the cfs hierarchy. Modify unthrottle_cfs_rq() to use the same pattern as enqueue_task_fair(). This fixes a problem already faced with the latter and add an optimization in the last for_each_sched_entity loop. Fixes: fe61468b2cb (sched/fair: Fix enqueue_task_fair warning) Reported-by Tao Zhou Signed-off-by: Vincent Guittot Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Phil Auld Reviewed-by: Ben Segall Link: https://lkml.kernel.org/r/20200513135528.4742-1-vincent.guittot@linaro.org Signed-off-by: Sasha Levin --- kernel/sched/fair.c | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-) -- 2.27.0 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 200e121101097..3dd7c10d6a582 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4580,7 +4580,6 @@ void unthrottle_cfs_rq(struct cfs_rq *cfs_rq) struct rq *rq = rq_of(cfs_rq); struct cfs_bandwidth *cfs_b = tg_cfs_bandwidth(cfs_rq->tg); struct sched_entity *se; - int enqueue = 1; long task_delta, idle_task_delta; se = cfs_rq->tg->se[cpu_of(rq)]; @@ -4604,21 +4603,41 @@ void unthrottle_cfs_rq(struct cfs_rq *cfs_rq) idle_task_delta = cfs_rq->idle_h_nr_running; for_each_sched_entity(se) { if (se->on_rq) - enqueue = 0; + break; + cfs_rq = cfs_rq_of(se); + enqueue_entity(cfs_rq, se, ENQUEUE_WAKEUP); + + cfs_rq->h_nr_running += task_delta; + cfs_rq->idle_h_nr_running += idle_task_delta; + /* end evaluation on encountering a throttled cfs_rq */ + if (cfs_rq_throttled(cfs_rq)) + goto unthrottle_throttle; + } + + for_each_sched_entity(se) { cfs_rq = cfs_rq_of(se); - if (enqueue) - enqueue_entity(cfs_rq, se, ENQUEUE_WAKEUP); + cfs_rq->h_nr_running += task_delta; cfs_rq->idle_h_nr_running += idle_task_delta; + + /* end evaluation on encountering a throttled cfs_rq */ if (cfs_rq_throttled(cfs_rq)) - break; + goto unthrottle_throttle; + + /* + * One parent has been throttled and cfs_rq removed from the + * list. Add it back to not break the leaf list. + */ + if (throttled_hierarchy(cfs_rq)) + list_add_leaf_cfs_rq(cfs_rq); } - if (!se) - add_nr_running(rq, task_delta); + /* At this point se is NULL and we are at root level*/ + add_nr_running(rq, task_delta); +unthrottle_throttle: /* * The cfs_rq_throttled() breaks in the above iteration can result in * incomplete leaf list maintenance, resulting in triggering the @@ -4627,7 +4646,8 @@ void unthrottle_cfs_rq(struct cfs_rq *cfs_rq) for_each_sched_entity(se) { cfs_rq = cfs_rq_of(se); - list_add_leaf_cfs_rq(cfs_rq); + if (list_add_leaf_cfs_rq(cfs_rq)) + break; } assert_list_leaf_cfs_rq(rq);