From patchwork Wed Mar 28 07:46:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 132515 Delivered-To: patch@linaro.org Received: by 10.46.84.29 with SMTP id i29csp152864ljb; Wed, 28 Mar 2018 00:47:09 -0700 (PDT) X-Google-Smtp-Source: AIpwx48gStB6+gkswIfiT8uU5urr5VBZMGnfJNjBnOTrjlE5ND3LMZYuO3IGt3CGYBXT+F+nQJNO X-Received: by 10.98.106.193 with SMTP id f184mr2105786pfc.201.1522223229362; Wed, 28 Mar 2018 00:47:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522223229; cv=none; d=google.com; s=arc-20160816; b=hyqnloHqcJsssRTPgJkDD45k0v3o+0j8VH5Ti7mEl3+5LylG3qf4f6Mgj3DLbOpiUh 7yzXajSyNejAumwUAJK7V7yCe1JxR21R24raxov4SxpOM9UMPotqOdM9hAla3Ry6u+mq 0Z2mVP/NALhetsX04bT/5WPj8Zqm4Jj+oOq5AxPPTxwC5l3Hunjv9KiD0Zf0hOGmyzJg VvXWWxHYICpynoIPFAcmytsK5yLvC5wO3f/GWPO/+15CTlYEg2oZug+kEQKsmRjCxSBO vrtOtNRnTXjyO7wzvIN/9G0TLx2bxNahZHtbeVW+KYJw1d4+InBIdb2/TQJ1wZA/6Xpb wOEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=N/NZAt9E9VbSBuQZaA+/cOvS1+1WjnfwpcgpGkW4YBs=; b=B8rkf7K7sOC/1iQsBjHTwara9h3fO3yZWrJFjocB0CcoLmjsFOMTyTZdBd7bCeBKuX K+0gBwvStOZxLiePNRqoaK+n6hFJX0vbTgQOUgVkRJ566X2q7XZ0Tir//iKfeZZ7NIU9 JT4+sEzAtXOy0zAdxx5hDaZlZqRGdtVZuX6Uh9NLoGtnVt5g52RxqEY1HX07SL+lBzqE H7Z5ZEgl+weKBpz9NZ33zO+DTR6q1vBcEj/1PukMnYy80t9B+Aaf3Tf2pvD79vp+fEKo ZtoV4o0OeJkE75A/x1t1xPMNdwpC9RM7kea0u/SCtz6LeiN38Goy2O7IgXD0XNLopQNs yyVg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AjD3uqTW; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c13si2141619pgn.696.2018.03.28.00.47.09; Wed, 28 Mar 2018 00:47:09 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AjD3uqTW; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752582AbeC1HrG (ORCPT + 28 others); Wed, 28 Mar 2018 03:47:06 -0400 Received: from mail-wr0-f195.google.com ([209.85.128.195]:41489 "EHLO mail-wr0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752159AbeC1HrF (ORCPT ); Wed, 28 Mar 2018 03:47:05 -0400 Received: by mail-wr0-f195.google.com with SMTP id f14so1263918wre.8 for ; Wed, 28 Mar 2018 00:47:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=N/NZAt9E9VbSBuQZaA+/cOvS1+1WjnfwpcgpGkW4YBs=; b=AjD3uqTW6J18t6zAOW0aEBmQxyY7Guuglew7GM4BXoLqfEU1+AruQ0VWrzHppFbFEh mFT3uOSQtVZcYXVeP7pdC+VrbvfWfSQXMkVjwqPoBKMqesxlb0T8AXU+pGUK0en623EN cKcFyJefLLhnJ5JOphcUdy191m/szNBhu1C8w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=N/NZAt9E9VbSBuQZaA+/cOvS1+1WjnfwpcgpGkW4YBs=; b=IIlmaJJFyKR2duvSq4P9Tcc7NHOvNlXl/MG8WsJeLn4Blkg+GFWxlTiBEQGwq9SxX8 1uZXuUt8Sl526MQxS/m/fcQSTyRMo2zs8Df/rIaUhw2NnrB47/gVo2r4QmK5HENyY8vh YjiP3G2fB91cBbcz5l48BttVKsX3gv5c02A98DsVVJgn7pSTVWZFVbQ7Gm4SazPhyzkY sq2Og1SBJ1WrPBmG3LTAO+QAXQ0Rrqy0ij/SzEvXZRrbax9XVfqaj8iZE6ufQQCbMOiF RpvuCGYJyAhdeK+cXQP7magycn2Dymr11NlnG6rsyzM37WU6JVq4TQ6XhXxJGmVl4c0+ q+Cg== X-Gm-Message-State: AElRT7GLUaw1GJ0slo3QP+lKQsSdUnRnp+98AGLJ+xqaWavo2TtP4zR6 jmDjl/oC+0CycRglnWftSPRDHA== X-Received: by 10.223.161.135 with SMTP id u7mr1991417wru.184.1522223223669; Wed, 28 Mar 2018 00:47:03 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:f:6020:a5f7:3db:a520:d281]) by smtp.gmail.com with ESMTPSA id k184sm3229251wmk.45.2018.03.28.00.47.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 28 Mar 2018 00:47:02 -0700 (PDT) From: Vincent Guittot To: catalin.marinas@arm.com, will.deacon@arm.com, linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, dietmar.eggemann@arm.com, Morten.Rasmussen@arm.com, chris.redpath@arm.com, Vincent Guittot Subject: [PATCH] sched: support dynamiQ cluster Date: Wed, 28 Mar 2018 09:46:55 +0200 Message-Id: <1522223215-23524-1-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.7.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Arm DynamiQ system can integrate cores with different micro architecture or max OPP under the same DSU so we can have cores with different compute capacity at the LLC (which was not the case with legacy big/LITTLE architecture). Such configuration is similar in some way to ITMT on intel platform which allows some cores to be boosted to higher turbo frequency than others and which uses SD_ASYM_PACKING feature to ensures that CPUs with highest capacity, will always be used in priortiy in order to provide maximum throughput. Add arch_asym_cpu_priority() for arm64 as this function is used to differentiate CPUs in the scheduler. The CPU's capacity is used to order CPUs in the same DSU. Create sched domain topolgy level for arm64 so we can set SD_ASYM_PACKING at MC level. Some tests have been done on a hikey960 platform (quad cortex-A53, quad cortex-A73). For the test purpose, the CPUs topology of the hikey960 has been modified so the 8 heterogeneous cores are described as being part of the same cluster and sharing resources (MC level) like with a DynamiQ DSU. Results below show the time in seconds to run sysbench --test=cpu with an increasing number of threads. The sysbench test run 32 times without patch with patch diff 1 threads 11.04(+/- 30%) 8.86(+/- 0%) -19% 2 threads 5.59(+/- 14%) 4.43(+/- 0%) -20% 3 threads 3.80(+/- 13%) 2.95(+/- 0%) -22% 4 threads 3.10(+/- 12%) 2.22(+/- 0%) -28% 5 threads 2.47(+/- 5%) 1.95(+/- 0%) -21% 6 threads 2.09(+/- 0%) 1.73(+/- 0%) -17% 7 threads 1.64(+/- 0%) 1.56(+/- 0%) - 7% 8 threads 1.42(+/- 0%) 1.42(+/- 0%) 0% Results show a better and stable results across iteration with the patch compared to mainline because we are always using big cores in priority whereas with mainline, the scheduler randomly choose a big or a little cores when there are more cores than number of threads. With 1 thread, the test duration varies in the range [8.85 .. 15.86] for mainline whereas it stays in the range [8.85..8.87] with the patch Signed-off-by: Vincent Guittot --- The SD_ASYM_PACKING flag is disabled by default and I'm preparing another patch to enable this dynamically at boot time by detecting the system topology. arch/arm64/kernel/topology.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 2186853..cb6705e5 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -296,6 +296,33 @@ static void __init reset_cpu_topology(void) } } +#ifdef CONFIG_SCHED_MC +unsigned int __read_mostly arm64_sched_asym_enabled; + +int arch_asym_cpu_priority(int cpu) +{ + return topology_get_cpu_scale(NULL, cpu); +} + +static inline int arm64_sched_dynamiq(void) +{ + return arm64_sched_asym_enabled ? SD_ASYM_PACKING : 0; +} + +static int arm64_core_flags(void) +{ + return cpu_core_flags() | arm64_sched_dynamiq(); +} +#endif + +static struct sched_domain_topology_level arm64_topology[] = { +#ifdef CONFIG_SCHED_MC + { cpu_coregroup_mask, arm64_core_flags, SD_INIT_NAME(MC) }, +#endif + { cpu_cpu_mask, SD_INIT_NAME(DIE) }, + { NULL, }, +}; + void __init init_cpu_topology(void) { reset_cpu_topology(); @@ -306,4 +333,7 @@ void __init init_cpu_topology(void) */ if (of_have_populated_dt() && parse_dt_topology()) reset_cpu_topology(); + + /* Set scheduler topology descriptor */ + set_sched_topology(arm64_topology); }