From patchwork Wed Jan 17 16:35:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 763493 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CAD8E1E536 for ; Wed, 17 Jan 2024 16:36:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509373; cv=none; b=kpiW+nOrnsx2icJfGFv8o4guO2RjfY51MEMuKXcC1uvT/L1n9pey0y1FX84p1DZr19yiOCae87bofk0mpGMzc2D8+YAODNUpMc6Xs46MJ+6zewaBOOfoGOME9Mu+NfJNufjlrPhWi7a0JjjHgwsIdMQynoOQMpaClRi+Q/w3AA0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509373; c=relaxed/simple; bh=+9dhGREDar7JnxPjUNvDr3ukqeiS2D+HMIM215GKB3o=; h=DKIM-Signature:Received:X-MC-Unique:Received:Received:From:To:Cc: Subject:Date:Message-Id:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding:X-Scanned-By; b=ozDLzKJ5Cc29lk9ExeboR717TQO1fI07aRvYqe6VsLIht71ZInJ413bC9zsYwbzGKFumi7ivmJ32g7IeKsEKbD0UU64CF2VJ0PQj0U9rkiPsygHNcrdOC1onDE0XCVfdMaaD1nfYjHMHOq5e9N6MXxINn98SfzqfTeJL+bKpIAw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=PyUxGLfo; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="PyUxGLfo" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705509369; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xvHDvVh6wugcYdMxmpgUJ/MFh5amT/B96L0R0KTNtzk=; b=PyUxGLfoEtTFm3rRNBXqB8puqUU8t4ZWDnEJM8CBDsv00chFO4mQ7kLzHOsPkNJV5M9Vei ymUEisOb3dPM3nUdJLGAQAXx1gt6j116F25DiiRDpEf58VyeGTvCuhqei8/lYB7P0xDurR F/znV9N6wW6JwgSHuKM1OysERwHMzxE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-473-xwCFczkROcG7hczghk4wLg-1; Wed, 17 Jan 2024 11:36:07 -0500 X-MC-Unique: xwCFczkROcG7hczghk4wLg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5E6E782A6C6; Wed, 17 Jan 2024 16:36:06 +0000 (UTC) Received: from llong.com (unknown [10.22.16.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id B9285111E402; Wed, 17 Jan 2024 16:36:04 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Frederic Weisbecker , Jonathan Corbet , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Davidlohr Bueso , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org, Mrunal Patel , Ryan Phillips , Brent Rowsell , Peter Hunt , Cestmir Kalina , Nicolas Saenz Julienne , Alex Gladkov , Marcelo Tosatti , Phil Auld , Paul Gortmaker , Daniel Bristot de Oliveira , Juri Lelli , Peter Zijlstra , Costa Shulyupin , Waiman Long Subject: [RFC PATCH 1/8] rcu/nocb: Pass a cpumask instead of a single CPU to offload/deoffload Date: Wed, 17 Jan 2024 11:35:04 -0500 Message-Id: <20240117163511.88173-2-longman@redhat.com> In-Reply-To: <20240117163511.88173-1-longman@redhat.com> References: <20240117163511.88173-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 From: Frederic Weisbecker Currently the interface to toggle callbacks offloading state only takes a single CPU per call. Now driving RCU NOCB through cpusets requires to be able to change the offloading state of a whole set of CPUs. To make it easier, extend the (de-)offloading interface to support a cpumask. Signed-off-by: Frederic Weisbecker Cc: Zefan Li Cc: Tejun Heo Cc: Johannes Weiner Cc: Paul E. McKenney Cc: Phil Auld Cc: Nicolas Saenz Julienne Cc: Marcelo Tosatti Cc: Paul Gortmaker Cc: Waiman Long Cc: Daniel Bristot de Oliveira Cc: Peter Zijlstra --- include/linux/rcupdate.h | 9 ++-- kernel/rcu/rcutorture.c | 4 +- kernel/rcu/tree_nocb.h | 102 ++++++++++++++++++++++++++------------- 3 files changed, 76 insertions(+), 39 deletions(-) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 0746b1b0b663..b649344075d2 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -142,13 +142,14 @@ static inline void rcu_irq_work_resched(void) { } #ifdef CONFIG_RCU_NOCB_CPU void rcu_init_nohz(void); -int rcu_nocb_cpu_offload(int cpu); -int rcu_nocb_cpu_deoffload(int cpu); +int rcu_nocb_cpumask_update(const struct cpumask *cpumask, bool offload); void rcu_nocb_flush_deferred_wakeup(void); #else /* #ifdef CONFIG_RCU_NOCB_CPU */ static inline void rcu_init_nohz(void) { } -static inline int rcu_nocb_cpu_offload(int cpu) { return -EINVAL; } -static inline int rcu_nocb_cpu_deoffload(int cpu) { return 0; } +static inline int rcu_nocb_cpumask_update(const struct cpumask *cpumask, bool offload) +{ + return -EINVAL; +} static inline void rcu_nocb_flush_deferred_wakeup(void) { } #endif /* #else #ifdef CONFIG_RCU_NOCB_CPU */ diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c index 7567ca8e743c..228a5488eb5e 100644 --- a/kernel/rcu/rcutorture.c +++ b/kernel/rcu/rcutorture.c @@ -2140,10 +2140,10 @@ static int rcu_nocb_toggle(void *arg) r = torture_random(&rand); cpu = (r >> 1) % (maxcpu + 1); if (r & 0x1) { - rcu_nocb_cpu_offload(cpu); + rcu_nocb_cpumask_update(cpumask_of(cpu), true); atomic_long_inc(&n_nocb_offload); } else { - rcu_nocb_cpu_deoffload(cpu); + rcu_nocb_cpumask_update(cpumask_of(cpu), false); atomic_long_inc(&n_nocb_deoffload); } toggle_delay = torture_random(&rand) % toggle_fuzz + toggle_interval; diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index 4efbf7333d4e..60b0a15ed6e2 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -1203,29 +1203,23 @@ static long rcu_nocb_rdp_deoffload(void *arg) return 0; } -int rcu_nocb_cpu_deoffload(int cpu) +static int rcu_nocb_cpu_deoffload(int cpu) { struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); int ret = 0; - cpus_read_lock(); - mutex_lock(&rcu_state.barrier_mutex); - if (rcu_rdp_is_offloaded(rdp)) { - if (cpu_online(cpu)) { - ret = work_on_cpu(cpu, rcu_nocb_rdp_deoffload, rdp); - if (!ret) - cpumask_clear_cpu(cpu, rcu_nocb_mask); - } else { - pr_info("NOCB: Cannot CB-deoffload offline CPU %d\n", rdp->cpu); - ret = -EINVAL; - } - } - mutex_unlock(&rcu_state.barrier_mutex); - cpus_read_unlock(); + if (cpu_is_offline(cpu)) + return -EINVAL; + + if (!rcu_rdp_is_offloaded(rdp)) + return 0; + + ret = work_on_cpu(cpu, rcu_nocb_rdp_deoffload, rdp); + if (!ret) + cpumask_clear_cpu(cpu, rcu_nocb_mask); return ret; } -EXPORT_SYMBOL_GPL(rcu_nocb_cpu_deoffload); static long rcu_nocb_rdp_offload(void *arg) { @@ -1236,12 +1230,6 @@ static long rcu_nocb_rdp_offload(void *arg) struct rcu_data *rdp_gp = rdp->nocb_gp_rdp; WARN_ON_ONCE(rdp->cpu != raw_smp_processor_id()); - /* - * For now we only support re-offload, ie: the rdp must have been - * offloaded on boot first. - */ - if (!rdp->nocb_gp_rdp) - return -EINVAL; if (WARN_ON_ONCE(!rdp_gp->nocb_gp_kthread)) return -EINVAL; @@ -1288,29 +1276,77 @@ static long rcu_nocb_rdp_offload(void *arg) return 0; } -int rcu_nocb_cpu_offload(int cpu) +static int rcu_nocb_cpu_offload(int cpu) { struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); - int ret = 0; + int ret; + + if (cpu_is_offline(cpu)) + return -EINVAL; + + if (rcu_rdp_is_offloaded(rdp)) + return 0; + + ret = work_on_cpu(cpu, rcu_nocb_rdp_offload, rdp); + if (!ret) + cpumask_set_cpu(cpu, rcu_nocb_mask); + + return ret; +} + +int rcu_nocb_cpumask_update(const struct cpumask *cpumask, bool offload) +{ + int cpu; + int err = 0; + int err_cpu; + cpumask_var_t saved_nocb_mask; + + if (!alloc_cpumask_var(&saved_nocb_mask, GFP_KERNEL)) + return -ENOMEM; + + cpumask_copy(saved_nocb_mask, rcu_nocb_mask); cpus_read_lock(); mutex_lock(&rcu_state.barrier_mutex); - if (!rcu_rdp_is_offloaded(rdp)) { - if (cpu_online(cpu)) { - ret = work_on_cpu(cpu, rcu_nocb_rdp_offload, rdp); - if (!ret) - cpumask_set_cpu(cpu, rcu_nocb_mask); + for_each_cpu(cpu, cpumask) { + if (offload) { + err = rcu_nocb_cpu_offload(cpu); + if (err < 0) { + err_cpu = cpu; + pr_err("NOCB: offload cpu %d failed (%d)\n", cpu, err); + break; + } } else { - pr_info("NOCB: Cannot CB-offload offline CPU %d\n", rdp->cpu); - ret = -EINVAL; + err = rcu_nocb_cpu_deoffload(cpu); + if (err < 0) { + err_cpu = cpu; + pr_err("NOCB: deoffload cpu %d failed (%d)\n", cpu, err); + break; + } + } + } + + /* Rollback in case of error */ + if (err < 0) { + err_cpu = cpu; + for_each_cpu(cpu, cpumask) { + if (err_cpu == cpu) + break; + if (cpumask_test_cpu(cpu, saved_nocb_mask)) + WARN_ON_ONCE(rcu_nocb_cpu_offload(cpu)); + else + WARN_ON_ONCE(rcu_nocb_cpu_deoffload(cpu)); } } + mutex_unlock(&rcu_state.barrier_mutex); cpus_read_unlock(); - return ret; + free_cpumask_var(saved_nocb_mask); + + return err; } -EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); +EXPORT_SYMBOL_GPL(rcu_nocb_cpumask_update); #ifdef CONFIG_RCU_LAZY static unsigned long From patchwork Wed Jan 17 16:35:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 763492 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B110B225AD for ; Wed, 17 Jan 2024 16:36:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509378; cv=none; b=Ne6GnruGynEmXW4FsW7V7kFO018PrwuAhIG95H8OnwrYBLCoYu/DU8bP2Di6E5lzjtEZg1pSN1Ktx4+omUU2fQ3WSTGZjCk60cwhM4TqHky3DaDLu2+UCCpBeXPQ2QO4K4hUdgdOKIoDK+lTyWWaS3nu8Fk3hQvu53ATQIbXRoQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509378; c=relaxed/simple; bh=T/btiksz2pjdc/htwRedVgNxpGRbSjdss4lHcMx0Ds4=; h=DKIM-Signature:Received:X-MC-Unique:Received:Received:From:To:Cc: Subject:Date:Message-Id:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding:X-Scanned-By; b=ATmf6zkEG/zJMlbzYLqgqohGEMT0dBaFmK0C152ZFjAUyJ+FwX7xJqwJdZHP2lYijWyl7B/X3HD4wsZ62k2UnEXDYU7NzghZjl4PYNzke4a1tnJWaWW2sFM6DvPVGlZPhyOpQ9s9ZuhzURJIv0MbxJhGTo7o7KSAjd9qUbsCNN4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=OqEFQ6WV; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OqEFQ6WV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705509375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hMKTEJCLgRjbNlvIEToGyQAyufjPtT3IoQqw3muUhg4=; b=OqEFQ6WVnJsCy2tz5ip4Ok9Qv4xlFajiy356iw+QWd/VbuWQ1JCDNXz4y3e3RDEHF2IeWr vtbKNCLFCZf1fUB6eEmpwu2Pjr9X1SwrFKjtZvq6geIHnsQ+ACh8jiP/G/bJkIrS/8p8zi ByrupW7FEgVbIR8zQLXWjcoWp/AR1kY= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-261-y1ui9Ik_OUyghLE7fiOplw-1; Wed, 17 Jan 2024 11:36:09 -0500 X-MC-Unique: y1ui9Ik_OUyghLE7fiOplw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 11A88383009D; Wed, 17 Jan 2024 16:36:08 +0000 (UTC) Received: from llong.com (unknown [10.22.16.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6F72D1121306; Wed, 17 Jan 2024 16:36:06 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Frederic Weisbecker , Jonathan Corbet , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Davidlohr Bueso , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org, Mrunal Patel , Ryan Phillips , Brent Rowsell , Peter Hunt , Cestmir Kalina , Nicolas Saenz Julienne , Alex Gladkov , Marcelo Tosatti , Phil Auld , Paul Gortmaker , Daniel Bristot de Oliveira , Juri Lelli , Peter Zijlstra , Costa Shulyupin , Waiman Long Subject: [RFC PATCH 2/8] rcu/nocb: Prepare to change nocb cpumask from CPU-hotplug protected cpuset caller Date: Wed, 17 Jan 2024 11:35:05 -0500 Message-Id: <20240117163511.88173-3-longman@redhat.com> In-Reply-To: <20240117163511.88173-1-longman@redhat.com> References: <20240117163511.88173-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 From: Frederic Weisbecker cpusets is going to use the NOCB (de-)offloading interface while holding hotplug lock. Therefore pull out the responsibility of protecting against concurrent CPU-hotplug changes to the callers of rcu_nocb_cpumask_update(). Signed-off-by: Frederic Weisbecker Cc: Zefan Li Cc: Tejun Heo Cc: Johannes Weiner Cc: Paul E. McKenney Cc: Phil Auld Cc: Nicolas Saenz Julienne Cc: Marcelo Tosatti Cc: Paul Gortmaker Cc: Waiman Long Cc: Daniel Bristot de Oliveira Cc: Peter Zijlstra --- kernel/rcu/rcutorture.c | 2 ++ kernel/rcu/tree_nocb.h | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c index 228a5488eb5e..e935152346ff 100644 --- a/kernel/rcu/rcutorture.c +++ b/kernel/rcu/rcutorture.c @@ -2139,6 +2139,7 @@ static int rcu_nocb_toggle(void *arg) do { r = torture_random(&rand); cpu = (r >> 1) % (maxcpu + 1); + cpus_read_lock(); if (r & 0x1) { rcu_nocb_cpumask_update(cpumask_of(cpu), true); atomic_long_inc(&n_nocb_offload); @@ -2146,6 +2147,7 @@ static int rcu_nocb_toggle(void *arg) rcu_nocb_cpumask_update(cpumask_of(cpu), false); atomic_long_inc(&n_nocb_deoffload); } + cpus_read_unlock(); toggle_delay = torture_random(&rand) % toggle_fuzz + toggle_interval; set_current_state(TASK_INTERRUPTIBLE); schedule_hrtimeout(&toggle_delay, HRTIMER_MODE_REL); diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index 60b0a15ed6e2..bbcf6f4152a3 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -1301,12 +1301,13 @@ int rcu_nocb_cpumask_update(const struct cpumask *cpumask, bool offload) int err_cpu; cpumask_var_t saved_nocb_mask; + lockdep_assert_cpus_held(); + if (!alloc_cpumask_var(&saved_nocb_mask, GFP_KERNEL)) return -ENOMEM; cpumask_copy(saved_nocb_mask, rcu_nocb_mask); - cpus_read_lock(); mutex_lock(&rcu_state.barrier_mutex); for_each_cpu(cpu, cpumask) { if (offload) { @@ -1340,7 +1341,6 @@ int rcu_nocb_cpumask_update(const struct cpumask *cpumask, bool offload) } mutex_unlock(&rcu_state.barrier_mutex); - cpus_read_unlock(); free_cpumask_var(saved_nocb_mask); From patchwork Wed Jan 17 16:35:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 764556 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71010225A4 for ; Wed, 17 Jan 2024 16:36:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509377; cv=none; b=Ijjbr52fUDsTd99eG/Sg+PHsg7wVzfqD0Hc+9y3GAzSTyUHkekWyZZkVt8up1zSrDZqgVdcZEpzsuIFKarfvLzSxuZHM2ynmco6mGKXM6Pgv6hOsWEB/94+RyGaPI+8I2WWZwvp7qkgXgKQc0OJgO7TBEnoLV5xDx+0CcMvWi7s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509377; c=relaxed/simple; bh=cqoNjlPc6YRXzLiB2bCt5YjvjoQjt75FOzIQ4Y3qvrA=; h=DKIM-Signature:Received:X-MC-Unique:Received:Received:From:To:Cc: Subject:Date:Message-Id:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding:X-Scanned-By; b=VP4umrV9BiHo0yK2DBtAQ7US8vgAr02RsncOhYuxysAC7W1vlolP7Mr/bTwENWmgGihAfvPiRZ8U9MQq7ba/VEZqjeMM79ZDma9zA3IBw6eSQcwTRd63IQv2BgR/g1kVb01y208yUKUdjyCSixill3oPOO68ONZxuKIuYI6j+xw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=EP2svvm2; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="EP2svvm2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705509375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v6l2z5hmcKTog3XiOuPlPdWnGHwPaZmCTdkb04dCE+c=; b=EP2svvm2TpP9kwnXZhjSw+DOoBhzf/bxMVe+uq8uAE1ptz4zIfCMBRqtMsQCtBnIjPmaMP 32fNp0OMB1gZyqDnwL/B8elPrCyCOpWyuMtEZ9UiCO2oloYUOmAYjRRbiCWoS2WtBME+ho OP2PHAm6AcHa+fl2BI0ntnCVSz6HiyY= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-292-NamU-_HuOXOPyQsDLH2XPg-1; Wed, 17 Jan 2024 11:36:12 -0500 X-MC-Unique: NamU-_HuOXOPyQsDLH2XPg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B0F6529AC003; Wed, 17 Jan 2024 16:36:09 +0000 (UTC) Received: from llong.com (unknown [10.22.16.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id 22AAD1121306; Wed, 17 Jan 2024 16:36:08 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Frederic Weisbecker , Jonathan Corbet , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Davidlohr Bueso , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org, Mrunal Patel , Ryan Phillips , Brent Rowsell , Peter Hunt , Cestmir Kalina , Nicolas Saenz Julienne , Alex Gladkov , Marcelo Tosatti , Phil Auld , Paul Gortmaker , Daniel Bristot de Oliveira , Juri Lelli , Peter Zijlstra , Costa Shulyupin , Waiman Long Subject: [RFC PATCH 3/8] rcu/no_cb: Add rcu_nocb_enabled() to expose the rcu_nocb state Date: Wed, 17 Jan 2024 11:35:06 -0500 Message-Id: <20240117163511.88173-4-longman@redhat.com> In-Reply-To: <20240117163511.88173-1-longman@redhat.com> References: <20240117163511.88173-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 Add a new rcu_nocb_enabled() helper to expose the rcu_nocb state to other kernel subsystems like cpuset. That will allow cpuset to determine if RCU no-callback can be enabled on isolated CPUs within isolated partitions. If so, the corresponding RCU functions can be called to enable it when full CPU isolation is requested. Signed-off-by: Waiman Long --- include/linux/rcupdate.h | 6 ++++++ kernel/rcu/tree_nocb.h | 12 ++++++++++++ 2 files changed, 18 insertions(+) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index b649344075d2..976d55a3e523 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -120,6 +120,12 @@ void rcu_init(void); extern int rcu_scheduler_active; void rcu_sched_clock_irq(int user); +#ifdef CONFIG_RCU_NOCB_CPU +int rcu_nocb_enabled(struct cpumask *out_mask); +#else +static inline int rcu_nocb_enabled(struct cpumask *out_mask) { return 0; } +#endif + #ifdef CONFIG_TASKS_RCU_GENERIC void rcu_init_tasks_generic(void); #else diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index bbcf6f4152a3..020a347ccd52 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -81,6 +81,18 @@ static int __init parse_rcu_nocb_poll(char *arg) } __setup("rcu_nocb_poll", parse_rcu_nocb_poll); +/* + * Return the rcu_nocb state & optionally copy out rcu_nocb_mask. + */ +int rcu_nocb_enabled(struct cpumask *out_mask) +{ + if (!rcu_state.nocb_is_setup) + return 0; + if (out_mask) + cpumask_copy(out_mask, rcu_nocb_mask); + return 1; +} + /* * Don't bother bypassing ->cblist if the call_rcu() rate is low. * After all, the main point of bypassing is to avoid lock contention From patchwork Wed Jan 17 16:35:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 764555 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8AA2225AF for ; Wed, 17 Jan 2024 16:36:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509378; cv=none; b=PfJKMGadb0AYUgCHMbcdZzst2H3xensZVJ7bwajisOYBbL9bCyqyTvGfU7Nac11GvgaFC0DSgDv8bSTbwDZXO7daO6NIkt6wt7WuzRSatgWvu7a2lm6p/xhJAKMUspIM8h9TukvQkEC5HMR1xfNC290SmDtDq54+P9FvK+7461I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509378; c=relaxed/simple; bh=5UEzyAawuulkn9wa4nP2VXfeCOs8DGc/KPmirNiRgUU=; h=DKIM-Signature:Received:X-MC-Unique:Received:Received:From:To:Cc: Subject:Date:Message-Id:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding:X-Scanned-By; b=MlUM4P8Tuxw0lGGGTNQZ1ccebfPIwU5TUa5b1dnUIoQH9rvT0MbH/EBiLvpTD1JOjr+ZKCGW+i3k6q0LV1ShIQDAP59txalW/6ijR96SnzhWlpKSsY53JlNApw8pjrHUa/naBlDBK/QaHGE4KL05PU8sAY3rvNSUIKo5ZL42HEQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ELgqeB4A; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ELgqeB4A" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705509375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/ssqmGGPuOAdNch15QLVD55rG/uP6CAKjxRipGrMtQM=; b=ELgqeB4Av28YVS29XnOEVFv4NvoGm2KVgueQHgubE68WVcftQ5DYN3DZOrn6CYjk730qV8 tZvvPO1KMVf0Chg29bGJoxhDDzI8uGXovC2EFT325x2teoYHEulWbPoG9JqL6UaZELPwK7 ceAnhqEnPQxgbNTA72PwkacGNyAm7Uk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-515-5gxLqypwMn2tdHIuj81JEQ-1; Wed, 17 Jan 2024 11:36:12 -0500 X-MC-Unique: 5gxLqypwMn2tdHIuj81JEQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 672EE81DA88; Wed, 17 Jan 2024 16:36:11 +0000 (UTC) Received: from llong.com (unknown [10.22.16.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id C42601121306; Wed, 17 Jan 2024 16:36:09 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Frederic Weisbecker , Jonathan Corbet , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Davidlohr Bueso , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org, Mrunal Patel , Ryan Phillips , Brent Rowsell , Peter Hunt , Cestmir Kalina , Nicolas Saenz Julienne , Alex Gladkov , Marcelo Tosatti , Phil Auld , Paul Gortmaker , Daniel Bristot de Oliveira , Juri Lelli , Peter Zijlstra , Costa Shulyupin , Waiman Long Subject: [RFC PATCH 4/8] cgroup/cpuset: Better tracking of addition/deletion of isolated CPUs Date: Wed, 17 Jan 2024 11:35:07 -0500 Message-Id: <20240117163511.88173-5-longman@redhat.com> In-Reply-To: <20240117163511.88173-1-longman@redhat.com> References: <20240117163511.88173-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 The process of updating workqueue unbound cpumask to exclude isolated CPUs in cpuset only requires the use of the aggregated isolated_cpus cpumask. Other types of CPU isolation, like the RCU no-callback CPU mode, may require knowing more granular addition and deletion of isolated CPUs. To enable these types of CPU isolation at run time, we need to provide better tracking of the addition and deletion of isolated CPUs. This patch adds a new isolated_cpus_modifier enum type for tracking the addition and deletion of isolated CPUs as well as renaming update_unbound_workqueue_cpumask() to update_isolation_cpumasks() to accommodate additional CPU isolation modes in the future. There is no functional change. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 113 +++++++++++++++++++++++++---------------- 1 file changed, 69 insertions(+), 44 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index dfbb16aca9f4..0479af76a5dc 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -206,6 +206,13 @@ struct cpuset { */ static cpumask_var_t subpartitions_cpus; +/* Enum types for possible changes to the set of isolated CPUs */ +enum isolated_cpus_modifiers { + ISOL_CPUS_NONE = 0, + ISOL_CPUS_ADD, + ISOL_CPUS_DELETE, +}; + /* * Exclusive CPUs in isolated partitions */ @@ -1446,14 +1453,14 @@ static void partition_xcpus_newstate(int old_prs, int new_prs, struct cpumask *x * @new_prs: new partition_root_state * @parent: parent cpuset * @xcpus: exclusive CPUs to be added - * Return: true if isolated_cpus modified, false otherwise + * Return: isolated_cpus modifier * * Remote partition if parent == NULL */ -static bool partition_xcpus_add(int new_prs, struct cpuset *parent, - struct cpumask *xcpus) +static int partition_xcpus_add(int new_prs, struct cpuset *parent, + struct cpumask *xcpus) { - bool isolcpus_updated; + int icpus_mod = ISOL_CPUS_NONE; WARN_ON_ONCE(new_prs < 0); lockdep_assert_held(&callback_lock); @@ -1464,13 +1471,14 @@ static bool partition_xcpus_add(int new_prs, struct cpuset *parent, if (parent == &top_cpuset) cpumask_or(subpartitions_cpus, subpartitions_cpus, xcpus); - isolcpus_updated = (new_prs != parent->partition_root_state); - if (isolcpus_updated) + if (new_prs != parent->partition_root_state) { partition_xcpus_newstate(parent->partition_root_state, new_prs, xcpus); - + icpus_mod = (new_prs == PRS_ISOLATED) + ? ISOL_CPUS_ADD : ISOL_CPUS_DELETE; + } cpumask_andnot(parent->effective_cpus, parent->effective_cpus, xcpus); - return isolcpus_updated; + return icpus_mod; } /* @@ -1478,14 +1486,14 @@ static bool partition_xcpus_add(int new_prs, struct cpuset *parent, * @old_prs: old partition_root_state * @parent: parent cpuset * @xcpus: exclusive CPUs to be removed - * Return: true if isolated_cpus modified, false otherwise + * Return: isolated_cpus modifier * * Remote partition if parent == NULL */ -static bool partition_xcpus_del(int old_prs, struct cpuset *parent, +static int partition_xcpus_del(int old_prs, struct cpuset *parent, struct cpumask *xcpus) { - bool isolcpus_updated; + int icpus_mod; WARN_ON_ONCE(old_prs < 0); lockdep_assert_held(&callback_lock); @@ -1495,27 +1503,40 @@ static bool partition_xcpus_del(int old_prs, struct cpuset *parent, if (parent == &top_cpuset) cpumask_andnot(subpartitions_cpus, subpartitions_cpus, xcpus); - isolcpus_updated = (old_prs != parent->partition_root_state); - if (isolcpus_updated) + if (old_prs != parent->partition_root_state) { partition_xcpus_newstate(old_prs, parent->partition_root_state, xcpus); - + icpus_mod = (old_prs == PRS_ISOLATED) + ? ISOL_CPUS_DELETE : ISOL_CPUS_ADD; + } cpumask_and(xcpus, xcpus, cpu_active_mask); cpumask_or(parent->effective_cpus, parent->effective_cpus, xcpus); - return isolcpus_updated; + return icpus_mod; } -static void update_unbound_workqueue_cpumask(bool isolcpus_updated) +/** + * update_isolation_cpumasks - Add or remove CPUs to/from full isolation state + * @mask: cpumask of the CPUs to be added or removed + * @modifier: enum isolated_cpus_modifiers + * Return: 0 if successful, error code otherwise + * + * Workqueue unbound cpumask update is applied irrespective of isolation_full + * state and the whole isolated_cpus is passed. Repeated calls with the same + * isolated_cpus will not cause further action other than a wasted mutex + * lock/unlock. + */ +static int update_isolation_cpumasks(struct cpumask *mask, int modifier) { - int ret; + int err; lockdep_assert_cpus_held(); - if (!isolcpus_updated) - return; + if (!modifier) + return 0; /* No change in isolated CPUs */ - ret = workqueue_unbound_exclude_cpumask(isolated_cpus); - WARN_ON_ONCE(ret < 0); + err = workqueue_unbound_exclude_cpumask(isolated_cpus); + WARN_ON_ONCE(err); + return err; } /** @@ -1577,7 +1598,7 @@ static inline bool is_local_partition(struct cpuset *cs) static int remote_partition_enable(struct cpuset *cs, int new_prs, struct tmpmasks *tmp) { - bool isolcpus_updated; + int icpus_mod; /* * The user must have sysadmin privilege. @@ -1600,7 +1621,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs, return 0; spin_lock_irq(&callback_lock); - isolcpus_updated = partition_xcpus_add(new_prs, NULL, tmp->new_cpus); + icpus_mod = partition_xcpus_add(new_prs, NULL, tmp->new_cpus); list_add(&cs->remote_sibling, &remote_children); if (cs->use_parent_ecpus) { struct cpuset *parent = parent_cs(cs); @@ -1609,7 +1630,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs, parent->child_ecpus_count--; } spin_unlock_irq(&callback_lock); - update_unbound_workqueue_cpumask(isolcpus_updated); + update_isolation_cpumasks(tmp->new_cpus, icpus_mod); /* * Proprogate changes in top_cpuset's effective_cpus down the hierarchy. @@ -1630,7 +1651,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs, */ static void remote_partition_disable(struct cpuset *cs, struct tmpmasks *tmp) { - bool isolcpus_updated; + int icpus_mod; compute_effective_exclusive_cpumask(cs, tmp->new_cpus); WARN_ON_ONCE(!is_remote_partition(cs)); @@ -1638,14 +1659,14 @@ static void remote_partition_disable(struct cpuset *cs, struct tmpmasks *tmp) spin_lock_irq(&callback_lock); list_del_init(&cs->remote_sibling); - isolcpus_updated = partition_xcpus_del(cs->partition_root_state, - NULL, tmp->new_cpus); + icpus_mod = partition_xcpus_del(cs->partition_root_state, NULL, + tmp->new_cpus); cs->partition_root_state = -cs->partition_root_state; if (!cs->prs_err) cs->prs_err = PERR_INVCPUS; reset_partition_data(cs); spin_unlock_irq(&callback_lock); - update_unbound_workqueue_cpumask(isolcpus_updated); + update_isolation_cpumasks(tmp->new_cpus, icpus_mod); /* * Proprogate changes in top_cpuset's effective_cpus down the hierarchy. @@ -1668,7 +1689,8 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *newmask, { bool adding, deleting; int prs = cs->partition_root_state; - int isolcpus_updated = 0; + int icpus_add_mod = ISOL_CPUS_NONE; + int icpus_del_mod = ISOL_CPUS_NONE; if (WARN_ON_ONCE(!is_remote_partition(cs))) return; @@ -1693,12 +1715,12 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *newmask, spin_lock_irq(&callback_lock); if (adding) - isolcpus_updated += partition_xcpus_add(prs, NULL, tmp->addmask); + icpus_add_mod = partition_xcpus_add(prs, NULL, tmp->addmask); if (deleting) - isolcpus_updated += partition_xcpus_del(prs, NULL, tmp->delmask); + icpus_del_mod = partition_xcpus_del(prs, NULL, tmp->delmask); spin_unlock_irq(&callback_lock); - update_unbound_workqueue_cpumask(isolcpus_updated); - + update_isolation_cpumasks(tmp->addmask, icpus_add_mod); + update_isolation_cpumasks(tmp->delmask, icpus_del_mod); /* * Proprogate changes in top_cpuset's effective_cpus down the hierarchy. */ @@ -1819,7 +1841,8 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd, int part_error = PERR_NONE; /* Partition error? */ int subparts_delta = 0; struct cpumask *xcpus; /* cs effective_xcpus */ - int isolcpus_updated = 0; + int icpus_add_mod = ISOL_CPUS_NONE; + int icpus_del_mod = ISOL_CPUS_NONE; bool nocpu; lockdep_assert_held(&cpuset_mutex); @@ -2052,22 +2075,23 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd, cs->nr_subparts = 0; } /* - * Adding to parent's effective_cpus means deletion CPUs from cs + * Adding to parent's effective_cpus means deleting CPUs from cs * and vice versa. */ if (adding) - isolcpus_updated += partition_xcpus_del(old_prs, parent, - tmp->addmask); + icpus_add_mod = partition_xcpus_del(old_prs, parent, + tmp->addmask); if (deleting) - isolcpus_updated += partition_xcpus_add(new_prs, parent, - tmp->delmask); + icpus_del_mod = partition_xcpus_add(new_prs, parent, + tmp->delmask); if (is_partition_valid(parent)) { parent->nr_subparts += subparts_delta; WARN_ON_ONCE(parent->nr_subparts < 0); } spin_unlock_irq(&callback_lock); - update_unbound_workqueue_cpumask(isolcpus_updated); + update_isolation_cpumasks(tmp->addmask, icpus_add_mod); + update_isolation_cpumasks(tmp->delmask, icpus_del_mod); if ((old_prs != new_prs) && (cmd == partcmd_update)) update_partition_exclusive(cs, new_prs); @@ -3044,7 +3068,7 @@ static int update_prstate(struct cpuset *cs, int new_prs) int err = PERR_NONE, old_prs = cs->partition_root_state; struct cpuset *parent = parent_cs(cs); struct tmpmasks tmpmask; - bool new_xcpus_state = false; + int icpus_mod = ISOL_CPUS_NONE; if (old_prs == new_prs) return 0; @@ -3096,7 +3120,8 @@ static int update_prstate(struct cpuset *cs, int new_prs) /* * A change in load balance state only, no change in cpumasks. */ - new_xcpus_state = true; + icpus_mod = (new_prs == PRS_ISOLATED) + ? ISOL_CPUS_ADD : ISOL_CPUS_DELETE; } else { /* * Switching back to member is always allowed even if it @@ -3128,10 +3153,10 @@ static int update_prstate(struct cpuset *cs, int new_prs) WRITE_ONCE(cs->prs_err, err); if (!is_partition_valid(cs)) reset_partition_data(cs); - else if (new_xcpus_state) + else if (icpus_mod) partition_xcpus_newstate(old_prs, new_prs, cs->effective_xcpus); spin_unlock_irq(&callback_lock); - update_unbound_workqueue_cpumask(new_xcpus_state); + update_isolation_cpumasks(cs->effective_xcpus, icpus_mod); /* Force update if switching back to member */ update_cpumasks_hier(cs, &tmpmask, !new_prs ? HIER_CHECKALL : 0); From patchwork Wed Jan 17 16:35:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 763491 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03492224E1 for ; Wed, 17 Jan 2024 16:36:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509379; cv=none; b=Toz3IU+BEi+fYrpDl2wIF3KX/NT+kQQ1BocGIfyua7rfv1BGJ+Z4QLjR2ajd4Sfpr84kKc4ODuTIzc29KQrmJOGCkxP1uDBNbaPB6T0inaHxthlKzYzRLUslspzMG+3oWk2cGQc8ZhhDKIYXQjaWd+YET1FMJCd1Lw2rdU1EFJs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509379; c=relaxed/simple; bh=8QWtVc8UZZD0QE6DxocDuZo346g6BcMbcuc1gxx+sDM=; h=DKIM-Signature:Received:X-MC-Unique:Received:Received:From:To:Cc: Subject:Date:Message-Id:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding:X-Scanned-By; b=WlvzdwYmyJttBCPuWoKByIhijqJdtcXIIhq3wRUOKwtxUl3Dfvoal21BAUWRG9yG6cguyoTeOVHMtVoz50q5bOZQ+iAnOG92ShCO1TTXd8xkLpTYA1LfVSZ0iLWVqf2QRlZBXCihxHeuRi/wKmfM5Y+roiK+SAzdSkBicyyVhpI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LxJj1sgc; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LxJj1sgc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705509376; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AOAtVa5plf2y0/XGqN0I4sCFdII80MtXXh53ltOWBp4=; b=LxJj1sgcQRTreRe8Yxy3OqspBorTYxmiMjA2S7mN+BF6YHyWddQX3uer5y7BSI3/uVmMTr aKErRtcaFnPBEYDgrBGXupk/eIz4exGfrxrlYEkDY1pmNsG+3jLqa9HQsCxcg4QsNXB5Hs jDueMmKN292bbgK95KOH17f+/TKaoMw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-94-idE8Qwz3MPmcTh9apb_XGw-1; Wed, 17 Jan 2024 11:36:14 -0500 X-MC-Unique: idE8Qwz3MPmcTh9apb_XGw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1452E85A58C; Wed, 17 Jan 2024 16:36:13 +0000 (UTC) Received: from llong.com (unknown [10.22.16.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7791B1121306; Wed, 17 Jan 2024 16:36:11 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Frederic Weisbecker , Jonathan Corbet , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Davidlohr Bueso , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org, Mrunal Patel , Ryan Phillips , Brent Rowsell , Peter Hunt , Cestmir Kalina , Nicolas Saenz Julienne , Alex Gladkov , Marcelo Tosatti , Phil Auld , Paul Gortmaker , Daniel Bristot de Oliveira , Juri Lelli , Peter Zijlstra , Costa Shulyupin , Waiman Long Subject: [RFC PATCH 5/8] cgroup/cpuset: Add cpuset.cpus.isolation_full Date: Wed, 17 Jan 2024 11:35:08 -0500 Message-Id: <20240117163511.88173-6-longman@redhat.com> In-Reply-To: <20240117163511.88173-1-longman@redhat.com> References: <20240117163511.88173-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 This patch adds a new root only cpuset.cpus.isolation_full control file for enabling or disabling full CPU isolation mode where additional CPU isolation methods available to be used by cpuset will be turned on or off for all the isolated CPUs within isolated partitions. On write, cpuset.cpus.isolation_full accepts any integer. A zero value will disable full CPU isolation while a non-zero value will enable it. On read, cpuset.cpus.isolation_full will return either "0" (disabled) or "1" (enabled) followed by a comma separated list of additional CPU isolation methods that are enabled. The list of these available isolation methods will depend on kernel configuration options used as well as the presence of some pre-conditions for some of them. This patch only provides the infrastructure code. The various isolation methods will be added later on. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 88 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 0479af76a5dc..d1d4ce213979 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -218,6 +218,30 @@ enum isolated_cpus_modifiers { */ static cpumask_var_t isolated_cpus; +/* + * Enable full CPU isolation in isolated partitions, if set. + */ +static bool isolation_full; + +/* + * A flag indicating what cpuset full isolation modes can be enabled. + */ +static int isolation_flags; + +enum cpuset_isolation_types { + ISOL_TYPE_MAX, +}; + +static const char * const isolation_type_names[ISOL_TYPE_MAX] = { +}; + +/* Detect the cpuset isolation modes that can be enabled */ +static __init int set_isolation_flags(void) +{ + return 0; +} +late_initcall(set_isolation_flags); + /* List of remote partition root children */ static struct list_head remote_children; @@ -1524,6 +1548,8 @@ static int partition_xcpus_del(int old_prs, struct cpuset *parent, * state and the whole isolated_cpus is passed. Repeated calls with the same * isolated_cpus will not cause further action other than a wasted mutex * lock/unlock. + * + * The other isolation modes will only be activated when isolation_full is set. */ static int update_isolation_cpumasks(struct cpumask *mask, int modifier) { @@ -1536,6 +1562,13 @@ static int update_isolation_cpumasks(struct cpumask *mask, int modifier) err = workqueue_unbound_exclude_cpumask(isolated_cpus); WARN_ON_ONCE(err); + + if (!isolation_flags || !isolation_full) + return err; + + if (WARN_ON_ONCE(cpumask_empty(mask))) + return -EINVAL; + return err; } @@ -3514,6 +3547,7 @@ typedef enum { FILE_EXCLUSIVE_CPULIST, FILE_EFFECTIVE_XCPULIST, FILE_ISOLATED_CPULIST, + FILE_ISOLATION_FULL, FILE_CPU_EXCLUSIVE, FILE_MEM_EXCLUSIVE, FILE_MEM_HARDWALL, @@ -3713,6 +3747,25 @@ static int cpuset_common_seq_show(struct seq_file *sf, void *v) case FILE_ISOLATED_CPULIST: seq_printf(sf, "%*pbl\n", cpumask_pr_args(isolated_cpus)); break; + case FILE_ISOLATION_FULL: + if (isolation_full) { + int i, cnt; + + /* Also print the isolation modes that are enabled */ + seq_puts(sf, "1"); + for (i = cnt = 0; i < ISOL_TYPE_MAX; i++) { + if (!(isolation_flags & BIT(i))) + continue; + + seq_printf(sf, "%c%s", cnt ? ',' : ' ', + isolation_type_names[i]); + cnt++; + } + seq_puts(sf, "\n"); + } else { + seq_puts(sf, "0\n"); + } + break; default: ret = -EINVAL; } @@ -3833,6 +3886,33 @@ static ssize_t sched_partition_write(struct kernfs_open_file *of, char *buf, return retval ?: nbytes; } +/* + * cpuset_write_isolfull - enable/disable cpuset isolation full mode + */ +static int cpuset_write_isolfull(struct cgroup_subsys_state *css, + struct cftype *cft, u64 val) +{ + struct cpuset *cs = css_cs(css); + int retval = 0; + + cpus_read_lock(); + mutex_lock(&cpuset_mutex); + if (!is_cpuset_online(cs)) { + retval = -ENODEV; + } else if (isolation_full != !!val) { + isolation_full = !!val; + if (!cpumask_empty(isolated_cpus)) { + int mod = isolation_full + ? ISOL_CPUS_ADD : ISOL_CPUS_DELETE; + + retval = update_isolation_cpumasks(isolated_cpus, mod); + } + } + mutex_unlock(&cpuset_mutex); + cpus_read_unlock(); + return retval; +} + /* * for the common functions, 'private' gives the type of file */ @@ -4013,6 +4093,14 @@ static struct cftype dfl_files[] = { .flags = CFTYPE_ONLY_ON_ROOT, }, + { + .name = "cpus.isolation_full", + .seq_show = cpuset_common_seq_show, + .write_u64 = cpuset_write_isolfull, + .private = FILE_ISOLATION_FULL, + .flags = CFTYPE_ONLY_ON_ROOT, + }, + { } /* terminate */ }; From patchwork Wed Jan 17 16:35:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 763490 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB68323749 for ; Wed, 17 Jan 2024 16:36:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509383; cv=none; b=cWxR27LWRBRsJgx7mtcYw/wAM+pCcK7jVW1j8eRaTUpPIIk5ebb2ANYIBUnm8F3QOM98sPiQtqts8gys5KP19WLdOwlx2Bsv2AGMfGNIwAB+XRYSjocTwlPY010pmsn8XhwzaqJyW9C30DNY1cdlkr67mdHrnVq5xOeNoia6o1w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509383; c=relaxed/simple; bh=RIu0wCoHEEwmwrWrI4fk2QwIf3Z4+DNKewqmZMfVEmc=; h=DKIM-Signature:Received:X-MC-Unique:Received:Received:From:To:Cc: Subject:Date:Message-Id:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding:X-Scanned-By; b=efmdyFYxrPMs/xFn1nVLkrX5eCFLSYtlZ3KGKodwy51Y62spX5TbI1WqJm6qFeN0UVP0NTQNgSeUPJoQl69zusN7mRLhBjQNJSdJHsdUCr8XiQv0H1IKBgNlr74eWOt68ltFFJ2bU4bHJXZMuOJ24Ttwm48OrOX+LIP1pNN61xU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=MU2/2Uik; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MU2/2Uik" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705509380; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pwiZxk2/LIns0OuqqkiNPgLv6/0v49jXf5KKeW3bftU=; b=MU2/2Uik/MtLx9/09yZwO5LWP4oLdiO6SJmjGgMUlt9SwvvGC7JSjnLHIxUvXF8Z20xyvV UG2Bcxvh/vf4xIqRZhlKAQhu+RAF4bCLrDZILhSj+vV5WofzDYOsUMUlEacINd0UgawYnC HIGDfSc///1Z4lvFQtqYYbq8/z3oAfY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-306-Ry1xxOizP6KjrjiwrnqQ_w-1; Wed, 17 Jan 2024 11:36:16 -0500 X-MC-Unique: Ry1xxOizP6KjrjiwrnqQ_w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BEFDB81DA88; Wed, 17 Jan 2024 16:36:14 +0000 (UTC) Received: from llong.com (unknown [10.22.16.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id 272321121306; Wed, 17 Jan 2024 16:36:13 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Frederic Weisbecker , Jonathan Corbet , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Davidlohr Bueso , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org, Mrunal Patel , Ryan Phillips , Brent Rowsell , Peter Hunt , Cestmir Kalina , Nicolas Saenz Julienne , Alex Gladkov , Marcelo Tosatti , Phil Auld , Paul Gortmaker , Daniel Bristot de Oliveira , Juri Lelli , Peter Zijlstra , Costa Shulyupin , Waiman Long Subject: [RFC PATCH 6/8] cgroup/cpuset: Enable dynamic rcu_nocb mode on isolated CPUs Date: Wed, 17 Jan 2024 11:35:09 -0500 Message-Id: <20240117163511.88173-7-longman@redhat.com> In-Reply-To: <20240117163511.88173-1-longman@redhat.com> References: <20240117163511.88173-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 The patch adds RCU no-callback isolation mode dynamically to isolated CPUs within isolated partitions when the full CPU isolation mode is enabled. This isolation feature will only be available for use by cpuset if the "rcu_nocb" boot command line option is specified in the kernel command line with or without the optional CPU list argument. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index d1d4ce213979..40bbb0a9cb84 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -218,6 +218,11 @@ enum isolated_cpus_modifiers { */ static cpumask_var_t isolated_cpus; +/* + * rcu_nocb_mask set up at boot time. + */ +static cpumask_var_t rcu_nocb_mask_preset; + /* * Enable full CPU isolation in isolated partitions, if set. */ @@ -229,15 +234,26 @@ static bool isolation_full; static int isolation_flags; enum cpuset_isolation_types { + ISOL_TYPE_RCU, /* RCU no-callback CPU mode */ ISOL_TYPE_MAX, }; +enum cpuset_isolation_flags { + ISOL_FLAG_RCU = BIT(ISOL_TYPE_RCU), +}; + static const char * const isolation_type_names[ISOL_TYPE_MAX] = { + [ISOL_TYPE_RCU] = "rcu_nocbs", }; /* Detect the cpuset isolation modes that can be enabled */ static __init int set_isolation_flags(void) { + if (rcu_nocb_enabled(NULL)) { + BUG_ON(!zalloc_cpumask_var(&rcu_nocb_mask_preset, GFP_KERNEL)); + (void)rcu_nocb_enabled(rcu_nocb_mask_preset); + isolation_flags |= ISOL_FLAG_RCU; + } return 0; } late_initcall(set_isolation_flags); @@ -1554,6 +1570,7 @@ static int partition_xcpus_del(int old_prs, struct cpuset *parent, static int update_isolation_cpumasks(struct cpumask *mask, int modifier) { int err; + bool enable = (modifier == ISOL_CPUS_ADD); lockdep_assert_cpus_held(); @@ -1569,6 +1586,25 @@ static int update_isolation_cpumasks(struct cpumask *mask, int modifier) if (WARN_ON_ONCE(cpumask_empty(mask))) return -EINVAL; + err = 0; + if (isolation_flags & ISOL_FLAG_RCU) { + /* + * When disabling rcu_nocb, make sure that we don't touch any + * CPUs that have already been set in rcu_nocb_mask_preset. + */ + if (!enable && cpumask_intersects(mask, rcu_nocb_mask_preset)) { + cpumask_var_t tmp_mask; + + if (WARN_ON_ONCE(!alloc_cpumask_var(&tmp_mask, GFP_KERNEL))) + return -ENOMEM; + if (cpumask_andnot(tmp_mask, mask, rcu_nocb_mask_preset)) + err = rcu_nocb_cpumask_update(tmp_mask, enable); + free_cpumask_var(tmp_mask); + } else { + err = rcu_nocb_cpumask_update(mask, enable); + } + } + WARN_ON_ONCE(err); return err; } From patchwork Wed Jan 17 16:35:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 763489 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0CA723768 for ; Wed, 17 Jan 2024 16:36:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509386; cv=none; b=gHQaWRRxlkgl3F4Hai/BNVp0G4wzw66nK2QBqVvpFE8nEjh1fdNototECI+rJopTyR4Widjkn6178xdvqhTptAoTEFvJWQLRQz7IFyMf5Jdy43Zgi3Po5KhbW78lIdH2tifFserlYSp7EYqfIvbiaqww42krZtGNCZPyELt2SsU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509386; c=relaxed/simple; bh=i0Ofa11RRO77lr53R6C5icaSmSfpdwG1dq7E5sm7z1M=; h=DKIM-Signature:Received:X-MC-Unique:Received:Received:From:To:Cc: Subject:Date:Message-Id:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding:X-Scanned-By; b=X3BiffW1b+nHRPN8UZHsMX3Gj+FnB0ut2WyNuQlxy62IJ4H0yXPcWmUqRuQRlctx8HeIl3I432IZYzmBSM+GRwy5Kdm3ZhFDrjMzdxM4eQssSsX18PZzXarwAuORbtFXZRZPYXnyQZGLo7yt0nYTCNxgtzFU8hElujcogthOC/Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Vr4Q12Fv; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Vr4Q12Fv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705509384; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KotaQECG6rJbRoU9m6xm3uHY0VUggl9xjizYviPB8yA=; b=Vr4Q12FvLeIGDRgD6f/rn2TcoweO9rn1+pGl7BHNRGPHIp8C8S7eN1+Cm5E3d1TWFJojmr 3l4nsv9mHu3ENjkPIvZZUaOYQqDC8+maVl24S+BWUyMe7m2TXstcN00KAzGeWQLPvx9Ncv 7JzDAvpQfAcOAZP/Y+6sx7Wejyq/SJw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-450-P98TlzovNymX4FjnA35Hcg-1; Wed, 17 Jan 2024 11:36:18 -0500 X-MC-Unique: P98TlzovNymX4FjnA35Hcg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7FDC4185A7A8; Wed, 17 Jan 2024 16:36:16 +0000 (UTC) Received: from llong.com (unknown [10.22.16.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id D1B9F1121306; Wed, 17 Jan 2024 16:36:14 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Frederic Weisbecker , Jonathan Corbet , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Davidlohr Bueso , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org, Mrunal Patel , Ryan Phillips , Brent Rowsell , Peter Hunt , Cestmir Kalina , Nicolas Saenz Julienne , Alex Gladkov , Marcelo Tosatti , Phil Auld , Paul Gortmaker , Daniel Bristot de Oliveira , Juri Lelli , Peter Zijlstra , Costa Shulyupin , Waiman Long Subject: [RFC PATCH 7/8] cgroup/cpuset: Document the new cpuset.cpus.isolation_full control file Date: Wed, 17 Jan 2024 11:35:10 -0500 Message-Id: <20240117163511.88173-8-longman@redhat.com> In-Reply-To: <20240117163511.88173-1-longman@redhat.com> References: <20240117163511.88173-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 Document the new cpuset.cpus.isolation_full control file. Currently only the rcu_nocbs flag is supported, but more will be added in the future. Signed-off-by: Waiman Long --- Documentation/admin-guide/cgroup-v2.rst | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 17e6e9565156..bbd066838f93 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2352,6 +2352,30 @@ Cpuset Interface Files isolated partitions. It will be empty if no isolated partition is created. + cpuset.cpus.isolation_full + A read multiple values and write single value file which exists + on root cgroup only. + + This file shows the state of full CPU isolation mode for isolated + CPUs in isolated partitions. It shows either "0' if full CPU + isolation mode is disabled, or "1" followed by a comma-separated + list of additional CPU isolation flags that are enabled. + The currently supported CPU isolation flag is: + + rcu_nocbs + RCU no-callback CPU mode, which prevents such CPUs' + callbacks from being invoked in softirq context. + Invocation of such CPUs' RCU callbacks will instead be + offloaded to "rcuox/N" kthreads created for that purpose. + It is similar in functionality to the "rcu_nocbs" + boot command line option, but for dynamically created + isolated CPUs in isolated partitions. This flag can + only be enabled if such a "rcu_nocbs" option is present + in the boot command line of the running kernel. + + Full CPU isolation mode is enabled by writing a non-zero value + to this file and disabled by writing a zero value to it. + cpuset.cpus.partition A read-write single value file which exists on non-root cpuset-enabled cgroups. This flag is owned by the parent cgroup From patchwork Wed Jan 17 16:35:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 764553 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B839A241E9 for ; Wed, 17 Jan 2024 16:36:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509385; cv=none; b=VSpuG5ByAvsLLMiC4cP7S8lR2Nq86oCRtu7El5lINxdV4xrM0MhBlMZsO+BaUAO9VtAK/5kXtWmFiSQIXPoWMUh8Lyou9nIO9u7we12thzIa2daiKCIe8UjZkM0omFblHaFS6puuMJ2GWrRl6jqsq9nAzPG8ywQ2zDb3G4r34/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705509385; c=relaxed/simple; bh=BaKtGrLRrsyMl7efMCkHzm2H6pXTqNSWhWDk1P/kMOg=; h=DKIM-Signature:Received:X-MC-Unique:Received:Received:From:To:Cc: Subject:Date:Message-Id:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding:X-Scanned-By; b=TA3etORnwJKFXWqhkdb8KUm6mgezUAT3LKAVK7P5xww6MIda6XLS6fsx2D/zZcbFt9mdM+P6jLdLeCkNaWkpUv2kVfCB7IJdM7cJVK3hwnmXphn3+KVj4vrsnsUzSU741CWqMcl5atXp71uqzUJNHnplEJfwENuauNc7XfoSLAs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=bayT4rGV; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bayT4rGV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705509382; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=55mOiYryAKVg43LXJjjfXDXa8cEy1gWQGp1PtwK9/yc=; b=bayT4rGV11oQdMPgQPxuctKIDg13JxuylE0eFGapHed99v+1FOYEK21lpR6iYo0omag5Cw UOudbZT5dremYYT4vmzpB20EQp3QXlYLYHaXwKqLSHS0PP5ZepUwO0YqnnAxc5So4p4wdB vR4gLIZaahoHETBKqAQn6Gnx2y5lONA= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-685-XuKyGuYONhafxTxTf8BZKg-1; Wed, 17 Jan 2024 11:36:19 -0500 X-MC-Unique: XuKyGuYONhafxTxTf8BZKg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3138C2813007; Wed, 17 Jan 2024 16:36:18 +0000 (UTC) Received: from llong.com (unknown [10.22.16.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id 915741121312; Wed, 17 Jan 2024 16:36:16 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Frederic Weisbecker , Jonathan Corbet , "Paul E. McKenney" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Davidlohr Bueso , Shuah Khan Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-kselftest@vger.kernel.org, Mrunal Patel , Ryan Phillips , Brent Rowsell , Peter Hunt , Cestmir Kalina , Nicolas Saenz Julienne , Alex Gladkov , Marcelo Tosatti , Phil Auld , Paul Gortmaker , Daniel Bristot de Oliveira , Juri Lelli , Peter Zijlstra , Costa Shulyupin , Waiman Long Subject: [RFC PATCH 8/8] cgroup/cpuset: Update test_cpuset_prs.sh to handle cpuset.cpus.isolation_full Date: Wed, 17 Jan 2024 11:35:11 -0500 Message-Id: <20240117163511.88173-9-longman@redhat.com> In-Reply-To: <20240117163511.88173-1-longman@redhat.com> References: <20240117163511.88173-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 Add a new "-F" option to cpuset.cpus.isolation_full to enable cpuset.cpus.isolation_full for trying out the effect of enabling full CPU isolation. Signed-off-by: Waiman Long --- .../selftests/cgroup/test_cpuset_prs.sh | 23 ++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh index b5eb1be2248c..2a8f0cb8d252 100755 --- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh +++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh @@ -32,6 +32,7 @@ NR_CPUS=$(lscpu | grep "^CPU(s):" | sed -e "s/.*:[[:space:]]*//") PROG=$1 VERBOSE=0 DELAY_FACTOR=1 +ISOLATION_FULL= SCHED_DEBUG= while [[ "$1" = -* ]] do @@ -44,7 +45,10 @@ do -d) DELAY_FACTOR=$2 shift ;; - *) echo "Usage: $PROG [-v] [-d " + -F) ISOLATION_FULL=1 + shift + ;; + *) echo "Usage: $PROG [-v] [-d ] [-F]" exit ;; esac @@ -108,6 +112,22 @@ console_msg() pause 0.01 } +setup_isolation_full() +{ + ISOL_FULL=${CGROUP2}/cpuset.cpus.isolation_full + if [[ -n "$ISOLATION_FULL" ]] + then + echo 1 > $ISOL_FULL + set -- $(cat $ISOL_FULL) + ISOLATION_FLAGS=$2 + [[ $VERBOSE -gt 0 ]] && { + echo "Full CPU isolation flags: $ISOLATION_FLAGS" + } + else + echo 0 > $ISOL_FULL + fi +} + test_partition() { EXPECTED_VAL=$1 @@ -930,6 +950,7 @@ test_inotify() } trap cleanup 0 2 3 6 +setup_isolation_full run_state_test TEST_MATRIX test_isolated test_inotify