| Message ID | 20240820195536.202066-2-longman@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | cgroup/cpuset: Account for boot time isolated CPUs | expand |
Hi. On Tue, Aug 20, 2024 at 03:55:35PM GMT, Waiman Long <longman@redhat.com> wrote: > The prstate_housekeeping_conflict() function does check the > HK_TYPE_DOMAIN housekeeping cpumask to make sure that CPUs outside of it > can only be used in isolated partition. > Given the fact that we are going to make housekeeping cpumasks > dynamic, the current check may not be right anymore. Save the boot > time HK_TYPE_DOMAIN cpumask and check against it instead of the > upcoming dynamic HK_TYPE_DOMAIN housekeeping cpumask. Why is (will be) checking against the stored HK_TYPE_DOMAIN mask correct when this mask becomes dynamic? Thanks, Michal
On 8/27/24 04:01, Michal Koutný wrote: > Hi. > > On Tue, Aug 20, 2024 at 03:55:35PM GMT, Waiman Long <longman@redhat.com> wrote: >> The prstate_housekeeping_conflict() function does check the >> HK_TYPE_DOMAIN housekeeping cpumask to make sure that CPUs outside of it >> can only be used in isolated partition. >> Given the fact that we are going to make housekeeping cpumasks >> dynamic, the current check may not be right anymore. Save the boot >> time HK_TYPE_DOMAIN cpumask and check against it instead of the >> upcoming dynamic HK_TYPE_DOMAIN housekeeping cpumask. > Why is (will be) checking against the stored HK_TYPE_DOMAIN mask correct > when this mask becomes dynamic? In term of isolated CPUs, there are 2 categories - static and dynamic. Statically isolated CPUs are those that are designed as isolated at boot time by "isolcpus". Other isolated CPUs created by the cpuset isolated partitions are considered dynamic in the sense that its state can change at run time. The degree of CPU isolation of dynamically isolated CPUs isn't as good as that of the statically isolated ones. So I want to handle them separately which is what the prstate_housekeeping_conflict() is intended to do. As it is my intention to make the housekeeping cpumasks dynamic, I need to keep a copy of the statically isolated CPUs and check against them. There is no point to check dynamically isolated CPUs as the test may produce a false positive result. In the future when dynamic CPU isolation is as almost as good as the static ones, we can get rid of this distinction and treat all of them as dynamic. Cheers, Longman
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 7db55eed63cf..8b40df89c3c1 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -224,6 +224,12 @@ static cpumask_var_t subpartitions_cpus; */ static cpumask_var_t isolated_cpus; +/* + * Housekeeping (HK_TYPE_DOMAIN) CPUs at boot + */ +static cpumask_var_t boot_hk_cpus; +static bool have_boot_isolcpus; + /* List of remote partition root children */ static struct list_head remote_children; @@ -1823,15 +1829,15 @@ static void remote_partition_check(struct cpuset *cs, struct cpumask *newmask, * @new_cpus: cpu mask * Return: true if there is conflict, false otherwise * - * CPUs outside of housekeeping_cpumask(HK_TYPE_DOMAIN) can only be used in - * an isolated partition. + * CPUs outside of boot_hk_cpus, if defined, can only be used in an + * isolated partition. */ static bool prstate_housekeeping_conflict(int prstate, struct cpumask *new_cpus) { - const struct cpumask *hk_domain = housekeeping_cpumask(HK_TYPE_DOMAIN); - bool all_in_hk = cpumask_subset(new_cpus, hk_domain); + if (!have_boot_isolcpus) + return false; - if (!all_in_hk && (prstate != PRS_ISOLATED)) + if ((prstate != PRS_ISOLATED) && !cpumask_subset(new_cpus, boot_hk_cpus)) return true; return false; @@ -4345,6 +4351,13 @@ int __init cpuset_init(void) BUG_ON(!alloc_cpumask_var(&cpus_attach, GFP_KERNEL)); + have_boot_isolcpus = housekeeping_enabled(HK_TYPE_DOMAIN); + if (have_boot_isolcpus) { + BUG_ON(!alloc_cpumask_var(&boot_hk_cpus, GFP_KERNEL)); + cpumask_copy(boot_hk_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN)); + cpumask_andnot(isolated_cpus, cpu_possible_mask, boot_hk_cpus); + } + return 0; }
With the "isolcpus" boot command line parameter, we are able to create isolated CPUs at boot time. These isolated CPUs aren't fully accounted for in the cpuset code. For instance, the root cgroup's "cpuset.cpus.isolated" control file does not include the boot time isolated CPUs. Fix that by looking for pre-isolated CPUs at init time. The prstate_housekeeping_conflict() function does check the HK_TYPE_DOMAIN housekeeping cpumask to make sure that CPUs outside of it can only be used in isolated partition. Given the fact that we are going to make housekeeping cpumasks dynamic, the current check may not be right anymore. Save the boot time HK_TYPE_DOMAIN cpumask and check against it instead of the upcoming dynamic HK_TYPE_DOMAIN housekeeping cpumask. Signed-off-by: Waiman Long <longman@redhat.com> --- kernel/cgroup/cpuset.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-)