From patchwork Wed Aug 16 21:20:36 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Poirier X-Patchwork-Id: 110278 Delivered-To: patch@linaro.org Received: by 10.140.95.78 with SMTP id h72csp1323255qge; Wed, 16 Aug 2017 14:20:52 -0700 (PDT) X-Received: by 10.98.42.88 with SMTP id q85mr3006532pfq.85.1502918452445; Wed, 16 Aug 2017 14:20:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1502918452; cv=none; d=google.com; s=arc-20160816; b=OHHWehfkzp9qFm4xB60sYWHwft7XNp2EeLcjODicoAn89MpFzXq1xEUnZAtiXCRxqy r81oGedYSwHjIKQMt970I/z6lLZImK1rrrubeQuU/TMH0PiVCroxeJGUoE5whJeYQYGk lDA1HmpYcEbTG+8m7FZAckyzGM3OcBvSC3w3sbn7UqxzgxASB9ShxafEw2bsHvIiWcaE TsAav0suRIFvChGBrcK4kJ/XreDzWTOMa0L96EaZy4j88MoUjiTHs/IrkUJrvNZP0Ehr qSLqtozFkdYfCUROYHB7cpWQXF6Q57BfdTe53xD0cMZp/EzSxhMpwShtQ/KiDGKZYWx/ EAtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=DogM2cEXLCYyDD62O0RFI2eTcBXmy6GYvjND9qj5zjA=; b=k2MPw8I2f0QfcO8bCbx9/x01jleq6PZ+fPFDhu5OMXBn95R5kajShjfwq5VSGmSCXn 3PzX4VB2o4eKf5+DCWIAWz88BvAE8wgFLx3UVxJPlInUWC2BndL6YzHYrHen1j+D8ZYN pQfxgZTnHHFGGxpouo1MZ6jHkTai64+4uVMm1kMxzlaJreTbWoe6GX7haH6ToWgUKaVp tl+16QFP2LlD+xH725Z1bEP9qo2xTTi2rU3Yrq9vCg9Sz7NPgps+WJeYB3lwZpx5kU5P zbSLj8+o+VGZXIb3/BI0nhhPpV/XF5CnbReEpMH0n2HUuKg/fqDIM5SDodrVoCjfqcLb aZrQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=D3m6e65q; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 76si1027203pfw.0.2017.08.16.14.20.52; Wed, 16 Aug 2017 14:20:52 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=D3m6e65q; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752408AbdHPVUs (ORCPT + 26 others); Wed, 16 Aug 2017 17:20:48 -0400 Received: from mail-io0-f179.google.com ([209.85.223.179]:38449 "EHLO mail-io0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752105AbdHPVUr (ORCPT ); Wed, 16 Aug 2017 17:20:47 -0400 Received: by mail-io0-f179.google.com with SMTP id g71so17298718ioe.5 for ; Wed, 16 Aug 2017 14:20:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=DogM2cEXLCYyDD62O0RFI2eTcBXmy6GYvjND9qj5zjA=; b=D3m6e65qDiM70b0ukud29p3WcBfhcGj2w6eYcnuwmIShsGVXjc5+JeXwcS5p63VSz6 aUQ8jU2rz+pX9Hoi1rA9K9lztQGLqOGPUo3KsN1oSPOxKCOxKjd+ORv+HcJsAv5nMLJ1 /+SI8e9k9r/to9h2R64dMAqPTe2Jero/iDNmE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=DogM2cEXLCYyDD62O0RFI2eTcBXmy6GYvjND9qj5zjA=; b=r7BRFmGS6SOG/HJ3+KO/mNqsnT+mSzQalYkj6qh+c9qdNMQtVZvI7gd1CO/BxvPnLv /1fddFXmaIzYUxbWqII0BOnyHGJal3Mlr5kGOVLcu4Oh9jKRz1O8whDDsi66q3NoTKiQ 0SkI1omKVlzFl79melAWybUnIgj627IaZRkpTmOdlvZB3rZLKOqX5B3KhCN/gdzUfBE+ Fro6BnLfKUDigWPWPo8SbEFey6oWAunoktXQ1j2mq2TRaZ30XSrMxlOJL9+SB9nhlZV8 QW3mwC2UJ+VxCQYQB130QDY1NepIJypUZ7Zg5o1pHmaPBGYtJ+PKHuy2MmdHh41FDF2p Lc1w== X-Gm-Message-State: AHYfb5gfzzQQkBBRyCDFQ2Pw8CDT7652xwlgZLoPEAMNklXQYwnDMY0+ H7lhLNdXAZxHeuB/ X-Received: by 10.107.168.37 with SMTP id r37mr3112032ioe.274.1502918446333; Wed, 16 Aug 2017 14:20:46 -0700 (PDT) Received: from xps15.cg.shawcable.net (S0106002369de4dac.cg.shawcable.net. [68.147.8.254]) by smtp.gmail.com with ESMTPSA id 80sm918281itk.11.2017.08.16.14.20.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 16 Aug 2017 14:20:45 -0700 (PDT) From: Mathieu Poirier To: mingo@redhat.com, peterz@infradead.org Cc: tj@kernel.org, vbabka@suse.cz, lizefan@huawei.com, akpm@linux-foundation.org, weiyongjun1@huawei.com, juri.lelli@arm.com, rostedt@goodmis.org, claudio@evidence.eu.com, luca.abeni@santannapisa.it, bristot@redhat.com, linux-kernel@vger.kernel.org, mathieu.poirier@linaro.org Subject: [PATCH 0/7] sched/deadline: fix cpusets bandwidth accounting Date: Wed, 16 Aug 2017 15:20:36 -0600 Message-Id: <1502918443-30169-1-git-send-email-mathieu.poirier@linaro.org> X-Mailer: git-send-email 2.7.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This is a renewed attempt at fixing a problem reported by Steve Rostedt [1] where DL bandwidth accounting is not recomputed after CPUset and CPUhotplug operations. When CPUhotplug and some CUPset manipulation take place root domains are destroyed and new ones created, loosing at the same time DL accounting pertaining to utilisation. An earlier attempt by Juri [2] used the scheduling classes' rq_online() and rq_offline() methods, something that highlighted a problem with sleeping DL tasks. The email thread that followed envisioned creating a list of sleeping tasks to circle through when recomputing DL accounting. In this set the problem is addressed by relying on existing list of tasks (sleeping or not) already maintained by CPUsets. When CPUset or CPUhotplug operations have completed we circle through the list of tasks maintained by each CPUset looking for DL tasks. When a DL task is found its utilisation is added to the root domain it pertains to by way of its runqueue. The advantage of proceeding this way is that recomputing of DL accounting is done the same way for both active and inactive tasks, along with guaranteeing that DL accounting for tasks end up in the correct root domain regardless of the CPUset topology. The disadvantage is that circling through all the tasks in a CPUset can be time consuming. The counter argument is that both CPUset and CPUhotplug operations are time consuming in the first place. OPEN ISSUE: Regardless of how we proceed (using existing CPUset list or new ones) we need to deal with DL tasks that span more than one root domain, something that will typically happen after a CPUset operation. For example, if we split the number of available CPUs on a system in two CPUsets and then turn off the 'sched_load_balance' flag on the parent CPUset, DL tasks in the parent CPUset will end up spanning two root domains. One way to deal with this is to prevent CPUset operations from happening when such condition is detected, as enacted in this set. Although simple this approach feels brittle and akin to a "whack-a-mole" game. A better and more reliable approach would be to teach the DL scheduler to deal with tasks that span multiple root domains, a serious and substantial undertaking. I am sending this as a starting point for discussion. I would be grateful if you could take the time to comment on the approach and most importantly provide input on how to deal with the open issue underlined above. Many thanks, Mathieu [1]. https://lkml.org/lkml/2016/2/3/966 [2]. https://marc.info/?l=linux-kernel&m=145493552607388&w=2 Mathieu Poirier (7): sched/topology: Adding function partition_sched_domains_locked() cpuset: Rebuild root domain deadline accounting information sched/deadline: Keep new DL task within root domain's boundary cgroup: Constrain 'sched_load_balance' flag when DL tasks are present cgroup: Concentrate DL related validation code in one place cgroup: Constrain the addition of CPUs to a new CPUset sched/core: Don't change the affinity of DL tasks include/linux/sched.h | 3 + include/linux/sched/deadline.h | 8 ++ include/linux/sched/topology.h | 9 ++ kernel/cgroup/cpuset.c | 186 ++++++++++++++++++++++++++++++++++++++--- kernel/sched/core.c | 10 +-- kernel/sched/deadline.c | 47 ++++++++++- kernel/sched/sched.h | 3 - kernel/sched/topology.c | 31 +++++-- 8 files changed, 272 insertions(+), 25 deletions(-) -- 2.7.4