From patchwork Fri Oct 13 18:11:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 733796 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C823BCDB47E for ; Fri, 13 Oct 2023 18:12:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231248AbjJMSMl (ORCPT ); Fri, 13 Oct 2023 14:12:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229958AbjJMSMk (ORCPT ); Fri, 13 Oct 2023 14:12:40 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 186FFDC for ; Fri, 13 Oct 2023 11:11:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697220714; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=XIlRT028Oilu2T1armBjKk281LICHF+uokOf7ejKB88=; b=MSOlkvu4DB9ygxlyKDTnbpbBN2IohNktsA3gdwtVNzlrJwAtH9KY4acgEL4jy3Mp2l+GwQ CjsTSsDwhILcuV+KTbO/N2zhV1Ckpi/khPKlBC2OBO2jbS2s5e0gjgAV/a8ZMAYc/T4imP lMKxf1U1lHXai7N9yJuoBS4P5ETFsEA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-41-AhnJ3bsNPlSj1UTHtQz4mg-1; Fri, 13 Oct 2023 14:11:48 -0400 X-MC-Unique: AhnJ3bsNPlSj1UTHtQz4mg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3575F88B7A0; Fri, 13 Oct 2023 18:11:48 +0000 (UTC) Received: from llong.com (unknown [10.22.17.138]) by smtp.corp.redhat.com (Postfix) with ESMTP id 87F7C20296DB; Fri, 13 Oct 2023 18:11:47 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Lai Jiangshan , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Waiman Long Subject: [PATCH-cgroup 0/4] cgroup/cpuset: Improve CPU isolation in isolated partitions Date: Fri, 13 Oct 2023 14:11:18 -0400 Message-Id: <20231013181122.3518610-1-longman@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Isolated cpuset partition can currently be created to contain an exclusive set of CPUs not used in other cgroups and with load balancing disabled to reduce interference from the scheduler. The main purpose of this isolated partition type is to dynamically emulate what can be done via the "isolcpus" boot command line option, specifically the default domain flag. One effect of the "isolcpus" option is to remove the isolated CPUs from the cpumasks of unbound workqueues since running work functions in an isolated CPU can be a major source of interference. Changing the unbound workqueue cpumasks can be done at run time by writing an appropriate cpumask without the isolated CPUs to /sys/devices/virtual/workqueue/cpumask. So one can set up an isolated cpuset partition and then write to the cpumask sysfs file to achieve similar level of CPU isolation. However, this manual process can be error prone. This patch series implements automatic exclusion of isolated CPUs from unbound workqueue cpumasks when an isolated cpuset partition is created and then adds those CPUs back when the isolated partition is destroyed. There are also other places in the kernel that look at the HK_FLAG_DOMAIN cpumask or other HK_FLAG_* cpumasks and exclude the isolated CPUs from certain actions to further reduce interference. CPUs in an isolated cpuset partition will not be able to avoid those interferences yet. That may change in the future as the need arises. Waiman Long (4): workqueue: Add workqueue_unbound_exclude_cpumask() to exclude CPUs from wq_unbound_cpumask selftests/cgroup: Minor code cleanup and reorganization of test_cpuset_prs.sh cgroup/cpuset: Keep track of CPUs in isolated partitions cgroup/cpuset: Take isolated CPUs out of workqueue unbound cpumask Documentation/admin-guide/cgroup-v2.rst | 10 +- include/linux/workqueue.h | 2 +- kernel/cgroup/cpuset.c | 237 +++++++++++++----- kernel/workqueue.c | 42 +++- .../selftests/cgroup/test_cpuset_prs.sh | 209 +++++++++------ 5 files changed, 350 insertions(+), 150 deletions(-)