From patchwork Sat Jan 5 17:49:01 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13829 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 27CF523E21 for ; Sat, 5 Jan 2013 17:49:22 +0000 (UTC) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by fiordland.canonical.com (Postfix) with ESMTP id C3132A191C4 for ; Sat, 5 Jan 2013 17:49:21 +0000 (UTC) Received: by mail-vc0-f178.google.com with SMTP id l6so3835917vcl.37 for ; Sat, 05 Jan 2013 09:49:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:x-forwarded-to:x-forwarded-for:delivered-to:x-received :received-spf:from:to:cc:subject:date:message-id:x-mailer :in-reply-to:references:x-content-scanned:x-cbid:x-gm-message-state; bh=9T5WbR9rF5Gi/GjL5G9HKU+OQXNFMP41SkuBI9gEYsQ=; b=f56UfXWpLhyN/Z4kFtQl9TUL2Gt/joYosyp8LHiwIm4mqn/nO3k7e1ylT2pMf34zm6 +Lq3w2LI+AI6OQwYVK70iQEtg/1R0AIlWu60/CdA6hwuz+Z+mT2m3+FgYHeN49ofvhMy +4VChz2tkdUPMFvZJbKxXul/O5VzpBma1Qwdx38/Gn2pBxl3vKxXJzmTLG85CmbqjERc Ca/HO+amxEtUVjAvHTN2+e0s/pS2G+OaXKg12KakQal6gihIKLiyV/fFE02dAhU0gFx9 o2oADu1Eh5nBOvBMzzilxh/4ejE/99jKOUXppwqDmATqVDt2xxwzHpWLBbpC5Dldnjn0 i3XQ== X-Received: by 10.58.210.65 with SMTP id ms1mr80274371vec.59.1357408161204; Sat, 05 Jan 2013 09:49:21 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.58.145.101 with SMTP id st5csp17431veb; Sat, 5 Jan 2013 09:49:20 -0800 (PST) X-Received: by 10.43.82.72 with SMTP id ab8mr32830220icc.33.1357408158633; Sat, 05 Jan 2013 09:49:18 -0800 (PST) Received: from e8.ny.us.ibm.com (e8.ny.us.ibm.com. [32.97.182.138]) by mx.google.com with ESMTPS id w10si42088851icv.94.2013.01.05.09.49.18 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 05 Jan 2013 09:49:18 -0800 (PST) Received-SPF: pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.182.138 as permitted sender) client-ip=32.97.182.138; Authentication-Results: mx.google.com; spf=pass (google.com: domain of paulmck@linux.vnet.ibm.com designates 32.97.182.138 as permitted sender) smtp.mail=paulmck@linux.vnet.ibm.com Received: from /spool/local by e8.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Sat, 5 Jan 2013 12:49:17 -0500 Received: from d01dlp03.pok.ibm.com (9.56.250.168) by e8.ny.us.ibm.com (192.168.1.108) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Sat, 5 Jan 2013 12:49:16 -0500 Received: from d01relay04.pok.ibm.com (d01relay04.pok.ibm.com [9.56.227.236]) by d01dlp03.pok.ibm.com (Postfix) with ESMTP id B80B8C90041; Sat, 5 Jan 2013 12:49:15 -0500 (EST) Received: from d03av02.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by d01relay04.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r05HnFnn284050; Sat, 5 Jan 2013 12:49:15 -0500 Received: from d03av02.boulder.ibm.com (loopback [127.0.0.1]) by d03av02.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r05HnCJt032371; Sat, 5 Jan 2013 10:49:15 -0700 Received: from paulmck-ThinkPad-W500 ([9.80.23.97]) by d03av02.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id r05HnAvt032185; Sat, 5 Jan 2013 10:49:10 -0700 Received: by paulmck-ThinkPad-W500 (Postfix, from userid 1000) id 706D4E528B; Sat, 5 Jan 2013 09:49:06 -0800 (PST) From: "Paul E. McKenney" To: linux-kernel@vger.kernel.org Cc: mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca, josh@joshtriplett.org, niv@us.ibm.com, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, Valdis.Kletnieks@vt.edu, dhowells@redhat.com, edumazet@google.com, darren@dvhart.com, fweisbec@gmail.com, sbw@mit.edu, patches@linaro.org, "Paul E. McKenney" , "Paul E. McKenney" Subject: [PATCH tip/core/rcu 11/14] rcu: Push lock release to rcu_start_gp()'s callers Date: Sat, 5 Jan 2013 09:49:01 -0800 Message-Id: <1357408144-15830-11-git-send-email-paulmck@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.8 In-Reply-To: <1357408144-15830-1-git-send-email-paulmck@linux.vnet.ibm.com> References: <20130105174844.GA14172@linux.vnet.ibm.com> <1357408144-15830-1-git-send-email-paulmck@linux.vnet.ibm.com> X-Content-Scanned: Fidelis XPS MAILER x-cbid: 13010517-9360-0000-0000-00000ECE5F90 X-Gm-Message-State: ALoCoQm1cvol5DDXFTn4+9AxkkUm7zf9LqUv5Vrg+ydhHdXcvCimtcEN5WM7crPmvNZuwzwbSBEo From: "Paul E. McKenney" If CPUs are to give prior notice of needed grace periods, it will be necessary to invoke rcu_start_gp() without dropping the root rcu_node structure's ->lock. This commit takes a second step in this direction by moving the release of this lock to rcu_start_gp()'s callers. Signed-off-by: Paul E. McKenney Signed-off-by: Paul E. McKenney --- kernel/rcutree.c | 24 ++++++++++-------------- kernel/rcutree_plugin.h | 5 ++--- 2 files changed, 12 insertions(+), 17 deletions(-) diff --git a/kernel/rcutree.c b/kernel/rcutree.c index 7207435..8ca18ec 100644 --- a/kernel/rcutree.c +++ b/kernel/rcutree.c @@ -1525,16 +1525,14 @@ static int __noreturn rcu_gp_kthread(void *arg) /* * Start a new RCU grace period if warranted, re-initializing the hierarchy * in preparation for detecting the next grace period. The caller must hold - * the root node's ->lock, which is released before return. Hard irqs must - * be disabled. + * the root node's ->lock and hard irqs must be disabled. * * Note that it is legal for a dying CPU (which is marked as offline) to * invoke this function. This can happen when the dying CPU reports its * quiescent state. */ static void -rcu_start_gp(struct rcu_state *rsp, unsigned long flags) - __releases(rcu_get_root(rsp)->lock) +rcu_start_gp(struct rcu_state *rsp) { struct rcu_data *rdp = this_cpu_ptr(rsp->rda); struct rcu_node *rnp = rcu_get_root(rsp); @@ -1548,15 +1546,13 @@ rcu_start_gp(struct rcu_state *rsp, unsigned long flags) */ rcu_advance_cbs(rsp, rnp, rdp); - if (!rsp->gp_kthread || - !cpu_needs_another_gp(rsp, rdp)) { + if (!rsp->gp_kthread || !cpu_needs_another_gp(rsp, rdp)) { /* * Either we have not yet spawned the grace-period * task, this CPU does not need another grace period, * or a grace period is already in progress. * Either way, don't start a new grace period. */ - raw_spin_unlock_irqrestore(&rnp->lock, flags); return; } rsp->gp_flags = RCU_GP_FLAG_INIT; @@ -1566,15 +1562,14 @@ rcu_start_gp(struct rcu_state *rsp, unsigned long flags) /* Wake up rcu_gp_kthread() to start the grace period. */ wake_up(&rsp->gp_wq); - raw_spin_unlock_irqrestore(&rnp->lock, flags); } /* * Report a full set of quiescent states to the specified rcu_state * data structure. This involves cleaning up after the prior grace * period and letting rcu_start_gp() start up the next grace period - * if one is needed. Note that the caller must hold rnp->lock, as - * required by rcu_start_gp(), which will release it. + * if one is needed. Note that the caller must hold rnp->lock, which + * is released before return. */ static void rcu_report_qs_rsp(struct rcu_state *rsp, unsigned long flags) __releases(rcu_get_root(rsp)->lock) @@ -2172,7 +2167,8 @@ __rcu_process_callbacks(struct rcu_state *rsp) local_irq_save(flags); if (cpu_needs_another_gp(rsp, rdp)) { raw_spin_lock(&rcu_get_root(rsp)->lock); /* irqs disabled. */ - rcu_start_gp(rsp, flags); /* releases above lock */ + rcu_start_gp(rsp); + raw_spin_unlock_irqrestore(&rcu_get_root(rsp)->lock, flags); } else { local_irq_restore(flags); } @@ -2252,11 +2248,11 @@ static void __call_rcu_core(struct rcu_state *rsp, struct rcu_data *rdp, /* Start a new grace period if one not already started. */ if (!rcu_gp_in_progress(rsp)) { - unsigned long nestflag; struct rcu_node *rnp_root = rcu_get_root(rsp); - raw_spin_lock_irqsave(&rnp_root->lock, nestflag); - rcu_start_gp(rsp, nestflag); /* rlses rnp_root->lock */ + raw_spin_lock(&rnp_root->lock); + rcu_start_gp(rsp); + raw_spin_lock(&rnp_root->lock); } else { /* Give the grace period a kick. */ rdp->blimit = LONG_MAX; diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h index d09acdf..736dd2c 100644 --- a/kernel/rcutree_plugin.h +++ b/kernel/rcutree_plugin.h @@ -2213,7 +2213,6 @@ static void rcu_nocb_wait_gp(struct rcu_data *rdp) unsigned long c; bool d; unsigned long flags; - unsigned long flags1; struct rcu_node *rnp = rdp->mynode; struct rcu_node *rnp_root = rcu_get_root(rdp->rsp); @@ -2275,8 +2274,8 @@ static void rcu_nocb_wait_gp(struct rcu_data *rdp) c, rnp->level, rnp->grplo, rnp->grphi, "Startedroot"); - local_save_flags(flags1); - rcu_start_gp(rdp->rsp, flags1); /* Rlses ->lock. */ + rcu_start_gp(rdp->rsp); + raw_spin_unlock(&rnp->lock); } /* Clean up locking and irq state. */