From patchwork Tue Feb 13 20:32:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Poirier X-Patchwork-Id: 128297 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp33129ljc; Tue, 13 Feb 2018 12:34:15 -0800 (PST) X-Google-Smtp-Source: AH8x226Nvz12ftb5cTSLvC39h2DirAyA22xwvwZ9VbYX9pxps01u1wjswHuStZQ3/uRRm5O4mWgn X-Received: by 10.99.145.199 with SMTP id l190mr1854504pge.397.1518554055558; Tue, 13 Feb 2018 12:34:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518554055; cv=none; d=google.com; s=arc-20160816; b=n3W9CQ0f1AYx2CSup5O39Du7z2H1ttbCctp+XYkSbDc4pOm6Ta1w/n3ELe9B8/z2y4 Bhi6BXCp9THIjh6x2AIj1jg01AKWh+16NWFodmHaDbYZ1cT4OBBzvl7WnJdtrBhJbUGG Yi8zU1qYD2LZtCxpVAcFt0Bp/+0WTBZPy4GjDbFqrAeVTPQUhaux0B4kJDukQUqMHsBS 9lNAU6IeRxNSdHxeuvm4XTryDJ1v+iNg7vMNQVStbgKEStIstjfKO7ksH/+uWEYLERU1 UdJsmpbelZz1BE8755hJdr633ZwrT3gb4cf4YJT4Ax957e4OOoZg8H1G7vTf1ep7Hum0 eZ5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=uq46gIPTMMHr/Kat4X5lgfFhxL9zwh8U+r4MofJlGJE=; b=lzHfjASx0qzu9vzyZx+nShuvFsBfjAXSuYVU904sNnpD1Ey/QeaIc29keCM7TLLjvq JgiNEQnc070wjDkm6v7IyELCep1HZI4Beo0lZ3kWmucs0XaXfDljvAjKMSmmIcx1RjWJ fSxTs85lw7zGKIZ1eTQcV1/lTFFda/ZZ/S4p7WaOsc6QPL9kIuy1uoADZft1p1QpER+c bfyapybbvhQj1xERAenHTTXKkv6O02wUnUgb7TLQk+J4MBAlw3t1BI+qurwvigod5/Vq PWuhHJwMr6BMaPkQV58cISCJBlc5QBIh41YEfJy2fdr7WQSIOPVfJC6wEoVcYMUpKNZR QQpQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BDHATBqF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id bj9-v6si342395plb.464.2018.02.13.12.33.59; Tue, 13 Feb 2018 12:34:15 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BDHATBqF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965906AbeBMUd4 (ORCPT + 28 others); Tue, 13 Feb 2018 15:33:56 -0500 Received: from mail-pg0-f68.google.com ([74.125.83.68]:34915 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S965835AbeBMUdD (ORCPT ); Tue, 13 Feb 2018 15:33:03 -0500 Received: by mail-pg0-f68.google.com with SMTP id l131so680400pga.2 for ; Tue, 13 Feb 2018 12:33:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=uq46gIPTMMHr/Kat4X5lgfFhxL9zwh8U+r4MofJlGJE=; b=BDHATBqFc0sPhc0T3pEQV4hcu+EyeElcrfCkPm8ZLX5bC+E8dC1qDaBXcPf3DIHcud Ncl6sQ6ipH+y05ptMPKTT0WUgMnsruggG5mWYb8VsWfFUkh9RHYV+qVuWzIZuL8Hx6EU f8iIOUx+ybCZkiFKDeOr2fIeQTGvsxYgvECeI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=uq46gIPTMMHr/Kat4X5lgfFhxL9zwh8U+r4MofJlGJE=; b=o4Ll36U6jCEM4zQPPtkFcZaeIyitTyaGbK58heHNyMIDQAv+/WK4o1z85mDPa6Jinb bhrksM1OiLgmG7iRoFsuk3iGt8gFdeezfxxqeNTHAGt0WdLJmkR2RI9IEeiOb/jvg5CX 8CFI4RG7K8lfvvnw5bpK0hmC1+45L4naA01H7VT0l+DmdDWPIG43sY0piXUDHOf23cRW uUOwFZrh5W8ePd4q+xCEEoix6qjdgGfcPEX1pngBGo7xv2owg3owApqld7/k3rmTjupb 96EAU+M9Y+IpprA5t1jpitOIKfUAgZU1Fdag5NY3aRmTYgI/oOH6OU96hzjSiL7kuzVs 1f/A== X-Gm-Message-State: APf1xPCO1tty/M82bJmZpRERKAfBADNbBoQHO1SKgGMhbhmoC75lcdLJ 2P60A7q528wo4/89ViDAaoRJLg== X-Received: by 10.98.204.75 with SMTP id a72mr2403665pfg.33.1518553982168; Tue, 13 Feb 2018 12:33:02 -0800 (PST) Received: from xps15.cg.shawcable.net (S0106002369de4dac.cg.shawcable.net. [68.147.8.254]) by smtp.gmail.com with ESMTPSA id o135sm35540873pfg.45.2018.02.13.12.33.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Feb 2018 12:33:01 -0800 (PST) From: Mathieu Poirier To: peterz@infradead.org Cc: lizefan@huawei.com, mingo@redhat.com, rostedt@goodmis.org, claudio@evidence.eu.com, bristot@redhat.com, tommaso.cucinotta@santannapisa.it, juri.lelli@redhat.com, luca.abeni@santannapisa.it, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 07/10] cgroup: Constrain 'sched_load_balance' flag when DL tasks are present Date: Tue, 13 Feb 2018 13:32:44 -0700 Message-Id: <1518553967-20656-8-git-send-email-mathieu.poirier@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1518553967-20656-1-git-send-email-mathieu.poirier@linaro.org> References: <1518553967-20656-1-git-send-email-mathieu.poirier@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch prevents the 'sched_load_balance' flag from being set to 0 when DL tasks are present in a CPUset. Otherwise we end up with the DL tasks using CPUs belonging to different root domains, something that breaks the mathematical model behind DL bandwidth management. For example on a 4 core system CPUset "set1" has been created and CPUs 0 and 1 assigned to it. A DL task has also been spun off. By default the DL task can use all the CPUs in the default CPUset. If we set the base CPUset's cpuset.sched_load_balance to 0, CPU 0 and 1 are added to a newly created root domain while CPU 2 and 3 endup in the default root domain. But the DL task is still part of the base CPUset and as such can use CPUs 0 to 3, spanning at the same time more than one root domain. Signed-off-by: Mathieu Poirier --- kernel/cgroup/cpuset.c | 104 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) -- 2.7.4 diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 45a5035ae601..4f5e8bac5337 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -458,6 +458,106 @@ static void free_trial_cpuset(struct cpuset *trial) kfree(trial); } +static bool cpuset_has_dl_tasks(struct cpuset *cs) +{ + bool dl_tasks = false; + struct css_task_iter it; + struct task_struct *task; + + /* Go through each task in @cs looking for a DL task */ + css_task_iter_start(&cs->css, 0, &it); + + while (!dl_tasks && (task = css_task_iter_next(&it))) { + if (dl_task(task)) + dl_tasks = true; + } + + css_task_iter_end(&it); + + return dl_tasks; +} + +/* + * Assumes RCU read lock and cpuset_mutex are held. + */ +static int +validate_change_load_balance(struct cpuset *cur, struct cpuset *trial) +{ + bool populated = false, dl_tasks = false; + int ret = -EBUSY; + struct cgroup_subsys_state *pos_css; + struct cpuset *cs; + + /* Bail out if nothing has changed. */ + if (is_sched_load_balance(cur) == + is_sched_load_balance(trial)) { + ret = 0; + goto out; + } + + /* + * First deal with the generic case that applies when + * cpuset.sched_load_balance gets flipped on a cpuset, + * regardless of the value. + */ + cpuset_for_each_descendant_pre(cs, pos_css, cur) { + if (cpuset_has_dl_tasks(cs)) + dl_tasks = true; + + /* Skip the top cpuset since it obviously exists */ + if (cs == cur) + continue; + + /* Children without CPUs are not important */ + if (cpumask_empty(cs->cpus_allowed)) { + pos_css = css_rightmost_descendant(pos_css); + continue; + } + + /* CPUs have been assigned to this cpuset. */ + populated = true; + + /* + * Go no further if both conditions are true so that we + * don't end up in a situation where a DL task is + * spanning more than one root domain or only assigned + * to a subset of the CPUs in a root domain. + */ + if (populated && dl_tasks) + goto out; + } + + /* + * Things get very complicated when dealing with children cpuset, + * resulting in hard to maintain code and low confidence that + * all cases are handled properly. As such prevent the + * cpuset.sched_load_balance from being modified on children cpuset + * where DL tasks have been assigned (or any of its children). + */ + if (dl_tasks && parent_cs(cur)) + goto out; + + ret = 0; +out: + return ret; +} + +/* + * Assumes RCU read lock and cpuset_mutex are held. + */ +static int +validate_dl_change(struct cpuset *cur, struct cpuset *trial) +{ + int ret = 0; + + /* Check if the sched_load_balance flag has been changed */ + ret = validate_change_load_balance(cur, trial); + if (ret) + return ret; + + return ret; +} + /* * validate_change() - Used to validate that any proposed cpuset change * follows the structural rules for cpusets. @@ -492,6 +592,10 @@ static int validate_change(struct cpuset *cur, struct cpuset *trial) if (!is_cpuset_subset(c, trial)) goto out; + /* Make sure changes are compatible with deadline scheduling class */ + if (validate_dl_change(cur, trial)) + goto out; + /* Remaining checks don't apply to root cpuset */ ret = 0; if (cur == &top_cpuset)