From patchwork Tue Feb 13 20:32:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Poirier X-Patchwork-Id: 128298 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp33177ljc; Tue, 13 Feb 2018 12:34:19 -0800 (PST) X-Google-Smtp-Source: AH8x226iIE/ADUzUiLSO1xRGJ8OeiygNJhPqV5f6wdtze3gqdTuOwTNCCcNlMcQzIKS+Vo39Sc0X X-Received: by 2002:a17:902:bb8e:: with SMTP id m14-v6mr569604pls.423.1518554059740; Tue, 13 Feb 2018 12:34:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518554059; cv=none; d=google.com; s=arc-20160816; b=CxBGM6SLRs+Th/o3PMGuDSVhj+wMfNKRbs064uWfV+ii/0RfibMWwvpUOZO+iH7WJN jwepk3ACe5LCUpLXXB63NTcrLMxuGIzDnRhOLcAov4kSUjSIFSyzAsWzu5jRsdud1A2E k6AtmezVLu+CYOE3bzU3K3/6lSgOjrXTuau/deYnb9OfH0MakqZIC52SqCeTN/0E8G1G EJEk9jPgblvAom+bi3Uc8m12vU8aIu30wlQOVHs827R6gp7tgYQ4TDsJ8qpdhC/rODS7 QRg3tgw0IP4yDi/KX6PzAZ+Gb7KBYOBk15yU4akxs6TBfn0XGwHw+itboQ0oWRb64Iva jYfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=RpRuSxilgCH4LvoCdF6+8SofEdCVGn9nDBpdua5XPeg=; b=okE2JBmvANDQIydiLB/Sgw8Jr2kpytxB9j2MUS88GsCn0n3B+VfVcjDMtTVfV8gyCw 9MFxOlhvmVb0nsitH1undXHoMwu36oxR48VQOFYXmudreEaGds7J+Ky5y7dn9H9W9zoN iI0gBePPs6ouk480lMPeiDwf2meVp0HaxguGStxEsh/e106olocSpCJXpBQkSuW1KEiT CMSB2TD7lAzpaKf+q98gDt2VKksh5cThvX6n4YH4XCxnUAzWo0lVv1AEOPMtYhQCsjFm T4xyZ+omA3jPmusqEZtbaMHFWKS6exz2X7/XifaBfmboTklgC28nImG6KYCiPtZJvFmv 1jbQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZyuffiXf; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 69-v6si7891058plc.814.2018.02.13.12.34.19; Tue, 13 Feb 2018 12:34:19 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZyuffiXf; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965914AbeBMUeQ (ORCPT + 28 others); Tue, 13 Feb 2018 15:34:16 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:41843 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S965816AbeBMUc7 (ORCPT ); Tue, 13 Feb 2018 15:32:59 -0500 Received: by mail-pg0-f67.google.com with SMTP id t4so667062pgp.8 for ; Tue, 13 Feb 2018 12:32:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=RpRuSxilgCH4LvoCdF6+8SofEdCVGn9nDBpdua5XPeg=; b=ZyuffiXfo5iEz4uAcBgVh9AbjJH/5OUzhigoxN2CbbuxFHdrROm2U7MPr4IvGUJOk/ cTyZ7LxGXqGzrfnj4ITSxP8pzQb4/w4Ldb/Y9PLCnBEbcfNmliQ9PhH6wfw5+X4MXgKI lVY+kt6t1HwJFSh02q7haJn8rjheinsPeh9rk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=RpRuSxilgCH4LvoCdF6+8SofEdCVGn9nDBpdua5XPeg=; b=ehSUMK9h9M/SydciXR9COigyqn0fKZ74rfXllstajFoimSytxx/O7urZjhhpxDY6JL rE/FseKjUNqo4RIrpPvie2Y40FFq7tenGKDd5jFs/fL5BxEaky7WJFDPTt9TUh+8Hrch UavAbXDOMEre21dDrWOc7syilEa0R+F+vW19zncmNBzYM3VFkCpK+FSLouqjnPeRM0Q7 W4JfcUfi2gx1v8Iv0isnxVNFiucWGuU4sG6BD3Vl2eLtTahAIH/N70Z4ltP5truxPMdp 60o3HdZzPTNu+3y5WF/MUZ7/gNuRj2zQ1Q2k8+yqE4sXvjj2Te/05ACNfnstZN+EiSy6 g2SQ== X-Gm-Message-State: APf1xPDs3gui3NrtOq9vsbHN5AdadjXgKMs2WJuQBQlSTsi8wyHZGrFi AcIbqd/DV9gJZ7ZEohMsiTx+tQ== X-Received: by 10.98.103.83 with SMTP id b80mr2415502pfc.12.1518553978835; Tue, 13 Feb 2018 12:32:58 -0800 (PST) Received: from xps15.cg.shawcable.net (S0106002369de4dac.cg.shawcable.net. [68.147.8.254]) by smtp.gmail.com with ESMTPSA id o135sm35540873pfg.45.2018.02.13.12.32.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Feb 2018 12:32:57 -0800 (PST) From: Mathieu Poirier To: peterz@infradead.org Cc: lizefan@huawei.com, mingo@redhat.com, rostedt@goodmis.org, claudio@evidence.eu.com, bristot@redhat.com, tommaso.cucinotta@santannapisa.it, juri.lelli@redhat.com, luca.abeni@santannapisa.it, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 05/10] cpuset: Rebuild root domain deadline accounting information Date: Tue, 13 Feb 2018 13:32:42 -0700 Message-Id: <1518553967-20656-6-git-send-email-mathieu.poirier@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1518553967-20656-1-git-send-email-mathieu.poirier@linaro.org> References: <1518553967-20656-1-git-send-email-mathieu.poirier@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When the topology of root domains is modified by CPUset or CPUhotplug operations information about the current deadline bandwidth held in the root domain is lost. This patch address the issue by recalculating the lost deadline bandwidth information by circling through the deadline tasks held in CPUsets and adding their current load to the root domain they are associated with. Signed-off-by: Mathieu Poirier --- include/linux/sched.h | 5 ++++ include/linux/sched/deadline.h | 8 ++++++ kernel/cgroup/cpuset.c | 63 +++++++++++++++++++++++++++++++++++++++++- kernel/sched/deadline.c | 31 +++++++++++++++++++++ kernel/sched/sched.h | 3 -- kernel/sched/topology.c | 14 +++++++++- 6 files changed, 119 insertions(+), 5 deletions(-) -- 2.7.4 diff --git a/include/linux/sched.h b/include/linux/sched.h index b161ef8a902e..b9bd7da8dc85 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -239,6 +239,11 @@ struct vtime { u64 gtime; }; +#ifdef CONFIG_SMP +extern struct root_domain def_root_domain; +extern struct mutex sched_domains_mutex; +#endif + struct sched_info { #ifdef CONFIG_SCHED_INFO /* Cumulative counters: */ diff --git a/include/linux/sched/deadline.h b/include/linux/sched/deadline.h index a5bc8728ead7..050961659b1d 100644 --- a/include/linux/sched/deadline.h +++ b/include/linux/sched/deadline.h @@ -29,4 +29,12 @@ static inline bool dl_time_before(u64 a, u64 b) return (s64)(a - b) < 0; } +#ifdef CONFIG_SMP + +struct root_domain; +extern void dl_add_task_root_domain(struct task_struct *p); +extern void dl_clear_root_domain(struct root_domain *rd); + +#endif /* CONFIG_SMP */ + #endif /* _LINUX_SCHED_DEADLINE_H */ diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 41f8391640e6..d8108030b754 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -44,6 +44,7 @@ #include #include #include +#include #include #include #include @@ -812,6 +813,66 @@ static int generate_sched_domains(cpumask_var_t **domains, return ndoms; } +static void update_tasks_root_domain(struct cpuset *cs) +{ + struct css_task_iter it; + struct task_struct *task; + + css_task_iter_start(&cs->css, 0, &it); + + while ((task = css_task_iter_next(&it))) + dl_add_task_root_domain(task); + + css_task_iter_end(&it); +} + +/* + * Called with cpuset_mutex held (rebuild_sched_domains()) + * Called with hotplug lock held (rebuild_sched_domains_locked()) + * Called with sched_domains_mutex held (partition_and_rebuild_domains()) + */ +static void rebuild_root_domains(void) +{ + struct cpuset *cs = NULL; + struct cgroup_subsys_state *pos_css; + + rcu_read_lock(); + + /* + * Clear default root domain DL accounting, it will be computed again + * if a task belongs to it. + */ + dl_clear_root_domain(&def_root_domain); + + cpuset_for_each_descendant_pre(cs, pos_css, &top_cpuset) { + + if (cpumask_empty(cs->effective_cpus)) { + pos_css = css_rightmost_descendant(pos_css); + continue; + } + + css_get(&cs->css); + + rcu_read_unlock(); + + update_tasks_root_domain(cs); + + rcu_read_lock(); + css_put(&cs->css); + } + rcu_read_unlock(); +} + +static void +partition_and_rebuild_sched_domains(int ndoms_new, cpumask_var_t doms_new[], + struct sched_domain_attr *dattr_new) +{ + mutex_lock(&sched_domains_mutex); + partition_sched_domains_locked(ndoms_new, doms_new, dattr_new); + rebuild_root_domains(); + mutex_unlock(&sched_domains_mutex); +} + /* * Rebuild scheduler domains. * @@ -844,7 +905,7 @@ static void rebuild_sched_domains_locked(void) ndoms = generate_sched_domains(&doms, &attr); /* Have scheduler rebuild the domains */ - partition_sched_domains(ndoms, doms, attr); + partition_and_rebuild_sched_domains(ndoms, doms, attr); out: put_online_cpus(); } diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 9bb0e0c412ec..8eb508cf1990 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2269,6 +2269,37 @@ void __init init_sched_dl_class(void) GFP_KERNEL, cpu_to_node(i)); } +void dl_add_task_root_domain(struct task_struct *p) +{ + unsigned long flags; + struct rq_flags rf; + struct rq *rq; + struct dl_bw *dl_b; + + rq = task_rq_lock(p, &rf); + if (!dl_task(p)) + goto unlock; + + dl_b = &rq->rd->dl_bw; + raw_spin_lock_irqsave(&dl_b->lock, flags); + + dl_b->total_bw += p->dl.dl_bw; + + raw_spin_unlock_irqrestore(&dl_b->lock, flags); + +unlock: + task_rq_unlock(rq, p, &rf); +} + +void dl_clear_root_domain(struct root_domain *rd) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&rd->dl_bw.lock, flags); + rd->dl_bw.total_bw = 0; + raw_spin_unlock_irqrestore(&rd->dl_bw.lock, flags); +} + #endif /* CONFIG_SMP */ static void switched_from_dl(struct rq *rq, struct task_struct *p) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index fb5fc458547f..8aba192d4d17 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -685,9 +685,6 @@ struct root_domain { unsigned long max_cpu_capacity; }; -extern struct root_domain def_root_domain; -extern struct mutex sched_domains_mutex; - extern void init_defrootdomain(void); extern int sched_init_domains(const struct cpumask *cpu_map); extern void rq_attach_root(struct rq *rq, struct root_domain *rd); diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index eee721da40fb..3210ded386ba 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -3,6 +3,7 @@ * Scheduler topology setup/handling methods */ #include +#include #include #include #include @@ -1889,8 +1890,19 @@ void partition_sched_domains_locked(int ndoms_new, cpumask_var_t doms_new[], for (i = 0; i < ndoms_cur; i++) { for (j = 0; j < n && !new_topology; j++) { if (cpumask_equal(doms_cur[i], doms_new[j]) - && dattrs_equal(dattr_cur, i, dattr_new, j)) + && dattrs_equal(dattr_cur, i, dattr_new, j)) { + struct root_domain *rd; + + /* + * This domain won't be destroyed and as such + * its dl_bw->total_bw needs to be cleared. It + * will be recomputed in function + * update_tasks_root_domain(). + */ + rd = cpu_rq(cpumask_any(doms_cur[i]))->rd; + dl_clear_root_domain(rd); goto match1; + } } /* No match - a current sched domain not in new doms_new[] */ detach_destroy_domains(doms_cur[i]);