From patchwork Mon Nov 28 23:32:58 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 5326 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id C849923E1F for ; Mon, 28 Nov 2011 23:33:22 +0000 (UTC) Received: from mail-lpp01m010-f52.google.com (mail-lpp01m010-f52.google.com [209.85.215.52]) by fiordland.canonical.com (Postfix) with ESMTP id ADD7AA18414 for ; Mon, 28 Nov 2011 23:33:22 +0000 (UTC) Received: by mail-lpp01m010-f52.google.com with SMTP id h2so1022790laa.11 for ; Mon, 28 Nov 2011 15:33:22 -0800 (PST) Received: by 10.152.105.226 with SMTP id gp2mr29589785lab.28.1322523202591; Mon, 28 Nov 2011 15:33:22 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.152.41.198 with SMTP id h6cs13055lal; Mon, 28 Nov 2011 15:33:22 -0800 (PST) Received: by 10.68.25.199 with SMTP id e7mr57540510pbg.123.1322523198712; Mon, 28 Nov 2011 15:33:18 -0800 (PST) Received: from e5.ny.us.ibm.com (e5.ny.us.ibm.com. [32.97.182.145]) by mx.google.com with ESMTPS id g9si6872742pbj.39.2011.11.28.15.33.17 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 28 Nov 2011 15:33:18 -0800 (PST) Received-SPF: pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.182.145 as permitted sender) client-ip=32.97.182.145; Authentication-Results: mx.google.com; spf=pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.182.145 as permitted sender) smtp.mail=paulmck@linux.vnet.ibm.com Received: from /spool/local by e5.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 28 Nov 2011 18:33:16 -0500 Received: from d01relay05.pok.ibm.com ([9.56.227.237]) by e5.ny.us.ibm.com ([192.168.1.105]) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 28 Nov 2011 18:33:14 -0500 Received: from d01av04.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by d01relay05.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id pASNXDBK244510; Mon, 28 Nov 2011 18:33:13 -0500 Received: from d01av04.pok.ibm.com (loopback [127.0.0.1]) by d01av04.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id pASNX9tP031579; Mon, 28 Nov 2011 18:33:13 -0500 Received: from paulmck-ThinkPad-W500 ([9.47.24.246]) by d01av04.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id pASNX99V031494; Mon, 28 Nov 2011 18:33:09 -0500 Received: by paulmck-ThinkPad-W500 (Postfix, from userid 1000) id A6011EAA0D; Mon, 28 Nov 2011 15:33:07 -0800 (PST) From: "Paul E. McKenney" To: linux-kernel@vger.kernel.org Cc: mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca, josh@joshtriplett.org, niv@us.ibm.com, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, Valdis.Kletnieks@vt.edu, dhowells@redhat.com, eric.dumazet@gmail.com, darren@dvhart.com, patches@linaro.org, "Paul E. McKenney" , "Paul E. McKenney" Subject: [PATCH RFC tip/core/rcu 06/13] rcu: Go dyntick-idle more quickly if CPU has serviced current grace period Date: Mon, 28 Nov 2011 15:32:58 -0800 Message-Id: <1322523185-456-6-git-send-email-paulmck@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.3.2 In-Reply-To: <20111128233212.GA32279@linux.vnet.ibm.com> References: <20111128233212.GA32279@linux.vnet.ibm.com> x-cbid: 11112823-5930-0000-0000-000002A601B1 From: Paul E. McKenney The earlier version would attempt to push callbacks through five times before going into dyntick-idle mode if callbacks remained, but the CPU had done all that it needed to do for the current RCU grace periods. This is wasteful: In most cases, once the CPU has done all that it needs to for the current RCU grace periods, it will make no further progress on the callbacks no matter how many times it loops through the RCU core processing and the idle-entry code. This commit therefore goes to dyntick-idle mode whenever the current CPU has done all it can for the current grace period. Signed-off-by: Paul E. McKenney Signed-off-by: Paul E. McKenney --- kernel/rcutree_plugin.h | 24 ++++++++++++++++++------ 1 files changed, 18 insertions(+), 6 deletions(-) diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index 6467f56..45790bf 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -2028,12 +2028,29 @@ static void rcu_prepare_for_idle(int cpu) { int c = 0; - /* If no callbacks or in the holdoff period, enter dyntick-idle. */ + /* + * If there are no callbacks on this CPU or if RCU has no further + * need for this CPU at the moment, enter dyntick-idle mode. + * Also reset state so as to not prejudice later attempts. + */ if (!rcu_cpu_has_callbacks(cpu)) { per_cpu(rcu_dyntick_holdoff, cpu) = jiffies - 1; + per_cpu(rcu_dyntick_drain, cpu) = 0; trace_rcu_prep_idle("No callbacks"); return; } + if (!rcu_pending(cpu)) { + trace_rcu_prep_idle("Dyntick with callbacks"); + per_cpu(rcu_dyntick_holdoff, cpu) = jiffies - 1; + per_cpu(rcu_dyntick_drain, cpu) = 0; + per_cpu(rcu_awake_at_gp_end, cpu) = 1; + return; /* Nothing to do immediately. */ + } + + /* + * If in holdoff mode, just return. We will presumably have + * refrained from disabling the scheduling-clock tick. + */ if (per_cpu(rcu_dyntick_holdoff, cpu) == jiffies) { trace_rcu_prep_idle("In holdoff"); return; @@ -2046,11 +2063,6 @@ static void rcu_prepare_for_idle(int cpu) } else if (--per_cpu(rcu_dyntick_drain, cpu) <= 0) { /* We have hit the limit, so time to give up. */ per_cpu(rcu_dyntick_holdoff, cpu) = jiffies; - if (!rcu_pending(cpu)) { - trace_rcu_prep_idle("Dyntick with callbacks"); - per_cpu(rcu_awake_at_gp_end, cpu) = 1; - return; /* Nothing to do immediately. */ - } trace_rcu_prep_idle("Begin holdoff"); invoke_rcu_core(); /* Force the CPU out of dyntick-idle. */ return;