From patchwork Thu Jul 10 09:30:01 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 33411 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qc0-f198.google.com (mail-qc0-f198.google.com [209.85.216.198]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 7D74C203C0 for ; Thu, 10 Jul 2014 09:31:35 +0000 (UTC) Received: by mail-qc0-f198.google.com with SMTP id m20sf29605132qcx.5 for ; Thu, 10 Jul 2014 02:31:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=MS2TCbTtbmqR7Cz7XjQOoH4J2dtIsXTfG3zpJ3QQdAA=; b=LBBjbeWHPR7nLqNlQNjrpxmRbp9vWy/e80bJsYatKgujb76NCYuzerpxO5S/3V4GlA MG1LUmuVWtvUllegoeRogTtREocD9U1KccV2iROPd9HDIxO3VhdPyJ70WUj+ctkMDlBb kK2zgqZEX9cccQn9DVCE/IKJkyjPpB97rG5ThCdCZNCXJbjafSh2SNmEh93OhCl6Dsdu QBhbix1Gae8sohEmwOXPTaQ0XSWi7sX3nxOeJr3WgH8xGAu+9ZX8wnmPapzmA1AtDBiT 6PamVCv+S0UNCLNxYZNMdzva1DNu2CgUBAYS4TzxQcFdoMHjO0g0I+zPDFZ1+iwrS1U7 iDJQ== X-Gm-Message-State: ALoCoQlK05Amdw0PazfQwKL8m/TMt23IQgr+G95wm5QyWoG5sAnvyP1F7Tn9JiF+o4gmb39qZGEI X-Received: by 10.236.1.198 with SMTP id 46mr19370405yhd.16.1404984695258; Thu, 10 Jul 2014 02:31:35 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.41.100 with SMTP id y91ls10370qgy.60.gmail; Thu, 10 Jul 2014 02:31:35 -0700 (PDT) X-Received: by 10.52.252.4 with SMTP id zo4mr37457507vdc.20.1404984695181; Thu, 10 Jul 2014 02:31:35 -0700 (PDT) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by mx.google.com with ESMTPS id n9si22717339vcx.91.2014.07.10.02.31.35 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Jul 2014 02:31:35 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.178 as permitted sender) client-ip=209.85.220.178; Received: by mail-vc0-f178.google.com with SMTP id ij19so9940036vcb.37 for ; Thu, 10 Jul 2014 02:31:35 -0700 (PDT) X-Received: by 10.58.208.170 with SMTP id mf10mr44259920vec.22.1404984695103; Thu, 10 Jul 2014 02:31:35 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp117887vcb; Thu, 10 Jul 2014 02:31:34 -0700 (PDT) X-Received: by 10.68.171.193 with SMTP id aw1mr10917943pbc.117.1404984694114; Thu, 10 Jul 2014 02:31:34 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id st6si48164230pab.46.2014.07.10.02.31.33; Thu, 10 Jul 2014 02:31:33 -0700 (PDT) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752891AbaGJJbZ (ORCPT + 28 others); Thu, 10 Jul 2014 05:31:25 -0400 Received: from mail-wi0-f173.google.com ([209.85.212.173]:41765 "EHLO mail-wi0-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752423AbaGJJbW (ORCPT ); Thu, 10 Jul 2014 05:31:22 -0400 Received: by mail-wi0-f173.google.com with SMTP id cc10so4178149wib.6 for ; Thu, 10 Jul 2014 02:31:21 -0700 (PDT) X-Received: by 10.194.205.65 with SMTP id le1mr48809303wjc.67.1404984681547; Thu, 10 Jul 2014 02:31:21 -0700 (PDT) Received: from lmenx30s.lme.st.com (LPuteaux-656-01-48-212.w82-127.abo.wanadoo.fr. [82.127.83.212]) by mx.google.com with ESMTPSA id de5sm29646273wib.18.2014.07.10.02.31.18 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Jul 2014 02:31:20 -0700 (PDT) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, riel@redhat.com, linux-kernel@vger.kernel.org, preeti@linux.vnet.ibm.com, linux@arm.linux.org.uk, linux-arm-kernel@lists.infradead.org Cc: Morten.Rasmussen@arm.com, efault@gmx.de, nicolas.pitre@linaro.org, linaro-kernel@lists.linaro.org, daniel.lezcano@linaro.org, dietmar.eggemann@arm.com, Vincent Guittot Subject: [PATCH v4 ] sched: fix imbalance flag reset Date: Thu, 10 Jul 2014 11:30:01 +0200 Message-Id: <1404984601-7306-1-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: References: Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: vincent.guittot@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.178 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , The imbalance flag can stay set whereas there is no imbalance. Let assume that we have 3 tasks that run on a dual cores /dual cluster system. We will have some idle load balance which are triggered during tick. Unfortunately, the tick is also used to queue background work so we can reach the situation where short work has been queued on a CPU which already runs a task. The load balance will detect this imbalance (2 tasks on 1 CPU and an idle CPU) and will try to pull the waiting task on the idle CPU. The waiting task is a worker thread that is pinned on a CPU so an imbalance due to pinned task is detected and the imbalance flag is set. Then, we will not be able to clear the flag because we have at most 1 task on each CPU but the imbalance flag will trig to useless active load balance between the idle CPU and the busy CPU. We need to reset of the imbalance flag as soon as we have reached a balanced state. If all tasks are pinned, we don't consider that as a balanced state and let the imbalance flag set. Signed-off-by: Vincent Guittot Reviewed-by: Preeti U Murthy --- kernel/sched/fair.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d3c73122..a836198 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6615,10 +6615,8 @@ static int load_balance(int this_cpu, struct rq *this_rq, if (sd_parent) { int *group_imbalance = &sd_parent->groups->sgc->imbalance; - if ((env.flags & LBF_SOME_PINNED) && env.imbalance > 0) { + if ((env.flags & LBF_SOME_PINNED) && env.imbalance > 0) *group_imbalance = 1; - } else if (*group_imbalance) - *group_imbalance = 0; } /* All tasks on this runqueue were pinned by CPU affinity */ @@ -6629,7 +6627,7 @@ static int load_balance(int this_cpu, struct rq *this_rq, env.loop_break = sched_nr_migrate_break; goto redo; } - goto out_balanced; + goto out_all_pinned; } } @@ -6703,6 +6701,22 @@ static int load_balance(int this_cpu, struct rq *this_rq, goto out; out_balanced: + /* + * We reach balance although we may have faced some affinity + * constraints. Clear the imbalance flag if it was set. + */ + if (sd_parent) { + int *group_imbalance = &sd_parent->groups->sgc->imbalance; + if (*group_imbalance) + *group_imbalance = 0; + } + +out_all_pinned: + /* + * We reach balance because all tasks are pinned at this level so + * we can't migrate them. Let the imbalance flag set so parent level + * can try to migrate them. + */ schedstat_inc(sd, lb_balanced[idle]); sd->nr_balance_failed = 0;