| Message ID | 1469453670-2660-6-git-send-email-morten.rasmussen@arm.com |
|---|---|
| State | New |
| Headers | show |
On Mon, Aug 15, 2016 at 12:54:59PM +0200, Peter Zijlstra wrote: > On Mon, Jul 25, 2016 at 02:34:22PM +0100, Morten Rasmussen wrote: > > @@ -6336,14 +6338,16 @@ static int sched_domains_curr_level; > > * SD_NUMA - describes NUMA topologies > > * SD_SHARE_POWERDOMAIN - describes shared power domain > > * > > - * Odd one out: > > + * Odd ones out: > > * SD_ASYM_PACKING - describes SMT quirks > > + * SD_ASYM_CPUCAPACITY - describes mixed capacity topologies > > */ > > So I'm not sure the new CPUCAPACITY is 'odd'. > > That said, the comment is very terse and doesn't explain why PACKING is > odd. > > IIRC the distinction is that the 'normal' ones only describe topology, > while the ASYM_PACKING one also prescribes behaviour. It is odd in the > way that it doesn't only describe things. > > This ASYM_CPUCAPACITY otoh is purely descriptive, it doesn't prescribe > how to deal with it. I think I initially put it in as an 'odd' flag due to the somewhat strange semantics in the previous versions, but now that it is fixed I agree that it belongs together with purely descriptive flags. > > Does something like so clarify things? > > --- a/kernel/sched/core.c > +++ b/kernel/sched/core.c > @@ -6355,13 +6355,19 @@ static int sched_domains_curr_level; > /* > * SD_flags allowed in topology descriptions. > * > - * SD_SHARE_CPUCAPACITY - describes SMT topologies > - * SD_SHARE_PKG_RESOURCES - describes shared caches > - * SD_NUMA - describes NUMA topologies > - * SD_SHARE_POWERDOMAIN - describes shared power domain > + * These flags are purely descriptive of the topology and do not prescribe > + * behaviour. Behaviour is artificial and mapped in the below sd_init() > + * function: > * > - * Odd one out: > - * SD_ASYM_PACKING - describes SMT quirks > + * SD_SHARE_CPUCAPACITY - describes SMT topologies > + * SD_SHARE_PKG_RESOURCES - describes shared caches > + * SD_NUMA - describes NUMA topologies > + * SD_SHARE_POWERDOMAIN - describes shared power domain > + * > + * Odd one out, which beside describing the topology has a quirk also > + * prescribes the desired behaviour that goes along with it: > + * > + * SD_ASYM_PACKING - describes SMT quirks > */ > #define TOPOLOGY_SD_FLAGS \ > (SD_SHARE_CPUCAPACITY | \ I like it :) Morten
On Wed, Aug 17, 2016 at 04:42:36PM +0800, Wanpeng Li wrote: > 2016-07-25 21:34 GMT+08:00 Morten Rasmussen <morten.rasmussen@arm.com>: > > Add a topology flag to the sched_domain hierarchy indicating the lowest > > domain level where the full range of cpu capacities is represented by > > the domain members for asymmetric capacity topologies (e.g. ARM > > big.LITTLE). > > > > The flag is intended to indicate that extra care should be taken when > > placing tasks on cpus and this level spans all the different types of > > cpus found in the system (no need to look further up the domain > > hierarchy). This information is currently only available through > > iterating through the capacities of all the cpus at parent levels in the > > sched_domain hierarchy. > > > > SD 2 [ 0 1 2 3] SD_ASYM_CPUCAPACITY > > > > SD 1 [ 0 1] [ 2 3] !SD_ASYM_CPUCAPACITY > > > > cpu: 0 1 2 3 > > capacity: 756 756 1024 1024 > > > > If the topology in the example above is duplicated to create an eight > > cpu example with third sched_domain level on top (SD 3), this level > > should not have the flag set (!SD_ASYM_CPUCAPACITY) as its two group > > would both have all cpu capacities represented within them. > > I didn't find the place where set SD_ASYM_CPUCAPACITY to any SDs in > this patchset, but you have testing result in cover letter, where I > miss? The flag is supposed to be set by arch-specific code. I included a few patches in v1 and v2 that set the flag for arch/arm. However, since they are hopefully soon to be superseded by patches from Juri I dropped them from the v3 posting and provided a pointer to branch containing this patch set, Juri's patches, and few additional glue patches instead that enabled the flag when necessary for arch/arm and arch/arm64. Sorry for the confusion. Morten
diff --git a/include/linux/sched.h b/include/linux/sched.h index 930784dd7cd8..4e0c47af9b05 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1021,6 +1021,7 @@ extern void wake_up_q(struct wake_q_head *head); #define SD_BALANCE_FORK 0x0008 /* Balance on fork, clone */ #define SD_BALANCE_WAKE 0x0010 /* Balance on wakeup */ #define SD_WAKE_AFFINE 0x0020 /* Wake task to waking CPU */ +#define SD_ASYM_CPUCAPACITY 0x0040 /* Groups have different max cpu capacities */ #define SD_SHARE_CPUCAPACITY 0x0080 /* Domain members share cpu capacity */ #define SD_SHARE_POWERDOMAIN 0x0100 /* Domain members share power domain */ #define SD_SHARE_PKG_RESOURCES 0x0200 /* Domain members share cpu pkg resources */ diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 0a40beb46841..1a22fc9e3788 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5692,6 +5692,7 @@ static int sd_degenerate(struct sched_domain *sd) SD_BALANCE_FORK | SD_BALANCE_EXEC | SD_SHARE_CPUCAPACITY | + SD_ASYM_CPUCAPACITY | SD_SHARE_PKG_RESOURCES | SD_SHARE_POWERDOMAIN)) { if (sd->groups != sd->groups->next) @@ -5722,6 +5723,7 @@ sd_parent_degenerate(struct sched_domain *sd, struct sched_domain *parent) SD_BALANCE_NEWIDLE | SD_BALANCE_FORK | SD_BALANCE_EXEC | + SD_ASYM_CPUCAPACITY | SD_SHARE_CPUCAPACITY | SD_SHARE_PKG_RESOURCES | SD_PREFER_SIBLING | @@ -6336,14 +6338,16 @@ static int sched_domains_curr_level; * SD_NUMA - describes NUMA topologies * SD_SHARE_POWERDOMAIN - describes shared power domain * - * Odd one out: + * Odd ones out: * SD_ASYM_PACKING - describes SMT quirks + * SD_ASYM_CPUCAPACITY - describes mixed capacity topologies */ #define TOPOLOGY_SD_FLAGS \ (SD_SHARE_CPUCAPACITY | \ SD_SHARE_PKG_RESOURCES | \ SD_NUMA | \ SD_ASYM_PACKING | \ + SD_ASYM_CPUCAPACITY | \ SD_SHARE_POWERDOMAIN) static struct sched_domain *
Add a topology flag to the sched_domain hierarchy indicating the lowest domain level where the full range of cpu capacities is represented by the domain members for asymmetric capacity topologies (e.g. ARM big.LITTLE). The flag is intended to indicate that extra care should be taken when placing tasks on cpus and this level spans all the different types of cpus found in the system (no need to look further up the domain hierarchy). This information is currently only available through iterating through the capacities of all the cpus at parent levels in the sched_domain hierarchy. SD 2 [ 0 1 2 3] SD_ASYM_CPUCAPACITY SD 1 [ 0 1] [ 2 3] !SD_ASYM_CPUCAPACITY cpu: 0 1 2 3 capacity: 756 756 1024 1024 If the topology in the example above is duplicated to create an eight cpu example with third sched_domain level on top (SD 3), this level should not have the flag set (!SD_ASYM_CPUCAPACITY) as its two group would both have all cpu capacities represented within them. cc: Ingo Molnar <mingo@redhat.com> cc: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Morten Rasmussen <morten.rasmussen@arm.com> --- include/linux/sched.h | 1 + kernel/sched/core.c | 6 +++++- 2 files changed, 6 insertions(+), 1 deletion(-) -- 1.9.1