From patchwork Thu Sep 4 15:32:09 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Pitre X-Patchwork-Id: 36727 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ig0-f199.google.com (mail-ig0-f199.google.com [209.85.213.199]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 62CDF20CBB for ; Thu, 4 Sep 2014 15:48:40 +0000 (UTC) Received: by mail-ig0-f199.google.com with SMTP id l13sf49459675iga.10 for ; Thu, 04 Sep 2014 08:48:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe :content-transfer-encoding; bh=IndQTEDHe+xIAcSsouX7uTtqPpUNW9mPWPHEi5mQTCM=; b=GhTRV4L0ZOY8g3v3s1tVBhjkR6xGFNR/z5emwP9sfAPdYaNi3/KmpcY5U7YF5JfTZ3 0jpPUtPQ/e4KXvOamgHi5DEXLk3u5pAGoRZ2PiKf6c7+FQPFNew6BEak2NmAUH1COkdO oLMVNPsBQvOgJCZSIhzDnxznM9r8g72hdHiAa19J4tpoNkIZWM6wDCz6CiusYrvMcLVk tyHWyd/rduamPcthxI0EbXQbubpf/nyrsfI+KiIX/z8OIKOIrTStL61Xo2/mJDMX7S58 7KjE2nv1JISW0MtvbPGsSWMTylsfT5akFzaA5spvegNit4h//cIP6N2hMyOmp+GP3Agb rrOg== X-Gm-Message-State: ALoCoQnmhsGAMu7RogL3gKNjabQr7CzO6DAs/enbC0k2qTz2CiIRU0CqVMmaBkQkvyn7l/Bl0kgh X-Received: by 10.43.1.133 with SMTP id nq5mr2906703icb.21.1409845719986; Thu, 04 Sep 2014 08:48:39 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.44.119 with SMTP id f110ls335484qga.96.gmail; Thu, 04 Sep 2014 08:48:39 -0700 (PDT) X-Received: by 10.220.194.130 with SMTP id dy2mr2309301vcb.47.1409845719851; Thu, 04 Sep 2014 08:48:39 -0700 (PDT) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by mx.google.com with ESMTPS id eh8si3186615vdd.99.2014.09.04.08.48.39 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 04 Sep 2014 08:48:39 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.178 as permitted sender) client-ip=209.85.220.178; Received: by mail-vc0-f178.google.com with SMTP id la4so10853327vcb.37 for ; Thu, 04 Sep 2014 08:48:39 -0700 (PDT) X-Received: by 10.52.129.165 with SMTP id nx5mr3257441vdb.25.1409845719646; Thu, 04 Sep 2014 08:48:39 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.45.67 with SMTP id uj3csp859033vcb; Thu, 4 Sep 2014 08:48:39 -0700 (PDT) X-Received: by 10.42.80.70 with SMTP id u6mr5167742ick.93.1409845719059; Thu, 04 Sep 2014 08:48:39 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id km11si4204767pbd.215.2014.09.04.08.48.35 for ; Thu, 04 Sep 2014 08:48:39 -0700 (PDT) Received-SPF: none (google.com: linux-pm-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750744AbaIDPsb (ORCPT + 15 others); Thu, 4 Sep 2014 11:48:31 -0400 Received: from relais.videotron.ca ([24.201.245.36]:60017 "EHLO relais.videotron.ca" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751437AbaIDPr6 (ORCPT ); Thu, 4 Sep 2014 11:47:58 -0400 X-Greylist: delayed 918 seconds by postgrey-1.27 at vger.kernel.org; Thu, 04 Sep 2014 11:47:57 EDT Received: from yoda.home ([66.130.143.177]) by VL-VM-MR001.ip.videotron.ca (Oracle Communications Messaging Exchange Server 7u4-22.01 64bit (built Apr 21 2011)) with ESMTP id <0NBD00IA4V6ETKE0@VL-VM-MR001.ip.videotron.ca>; Thu, 04 Sep 2014 11:32:38 -0400 (EDT) Received: from xanadu.home (xanadu.home [192.168.2.2]) by yoda.home (Postfix) with ESMTP id 717EA2DA0768; Thu, 04 Sep 2014 11:32:38 -0400 (EDT) From: Nicolas Pitre To: Peter Zijlstra , Ingo Molnar Cc: Daniel Lezcano , "Rafael J. Wysocki" , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linaro-kernel@lists.linaro.org Subject: [PATCH v2 1/2] sched: let the scheduler see CPU idle states Date: Thu, 04 Sep 2014 11:32:09 -0400 Message-id: <1409844730-12273-2-git-send-email-nicolas.pitre@linaro.org> X-Mailer: git-send-email 1.8.4.108.g55ea5f6 In-reply-to: <1409844730-12273-1-git-send-email-nicolas.pitre@linaro.org> References: <1409844730-12273-1-git-send-email-nicolas.pitre@linaro.org> Sender: linux-pm-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: nicolas.pitre@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.178 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Content-transfer-encoding: 7BIT From: Daniel Lezcano When the cpu enters idle, it stores the cpuidle state pointer in its struct rq instance which in turn could be used to make a better decision when balancing tasks. As soon as the cpu exits its idle state, the struct rq reference is cleared. There are a couple of situations where the idle state pointer could be changed while it is being consulted: 1. For x86/acpi with dynamic c-states, when a laptop switches from battery to AC that could result on removing the deeper idle state. The acpi driver triggers: 'acpi_processor_cst_has_changed' 'cpuidle_pause_and_lock' 'cpuidle_uninstall_idle_handler' 'kick_all_cpus_sync'. All cpus will exit their idle state and the pointed object will be set to NULL. 2. The cpuidle driver is unloaded. Logically that could happen but not in practice because the drivers are always compiled in and 95% of them are not coded to unregister themselves. In any case, the unloading code must call 'cpuidle_unregister_device', that calls 'cpuidle_pause_and_lock' leading to 'kick_all_cpus_sync' as mentioned above. A race can happen if we use the pointer and then one of these two scenarios occurs at the same moment. In order to be safe, the idle state pointer stored in the rq must be used inside a rcu_read_lock section where we are protected with the 'rcu_barrier' in the 'cpuidle_uninstall_idle_handler' function. The idle_get_state() and idle_put_state() accessors should be used to that effect. Signed-off-by: Daniel Lezcano Signed-off-by: Nicolas Pitre --- drivers/cpuidle/cpuidle.c | 6 ++++++ kernel/sched/idle.c | 6 ++++++ kernel/sched/sched.h | 39 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 51 insertions(+) diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c index ee9df5e3f5..530e3055a2 100644 --- a/drivers/cpuidle/cpuidle.c +++ b/drivers/cpuidle/cpuidle.c @@ -225,6 +225,12 @@ void cpuidle_uninstall_idle_handler(void) initialized = 0; kick_all_cpus_sync(); } + + /* + * Make sure external observers (such as the scheduler) + * are done looking at pointed idle states. + */ + rcu_barrier(); } /** diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 11e7bc434f..c47fce75e6 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -147,6 +147,9 @@ use_default: clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &dev->cpu)) goto use_default; + /* Take note of the planned idle state. */ + idle_set_state(this_rq(), &drv->states[next_state]); + /* * Enter the idle state previously returned by the governor decision. * This function will block until an interrupt occurs and will take @@ -154,6 +157,9 @@ use_default: */ entered_state = cpuidle_enter(drv, dev, next_state); + /* The cpu is no longer idle or about to enter idle. */ + idle_set_state(this_rq(), NULL); + if (broadcast) clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &dev->cpu); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 579712f4e9..aea8baa7a5 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -14,6 +14,7 @@ #include "cpuacct.h" struct rq; +struct cpuidle_state; extern __read_mostly int scheduler_running; @@ -636,6 +637,11 @@ struct rq { #ifdef CONFIG_SMP struct llist_head wake_list; #endif + +#ifdef CONFIG_CPU_IDLE + /* Must be inspected within a rcu lock section */ + struct cpuidle_state *idle_state; +#endif }; static inline int cpu_of(struct rq *rq) @@ -1180,6 +1186,39 @@ static inline void idle_exit_fair(struct rq *rq) { } #endif +#ifdef CONFIG_CPU_IDLE +static inline void idle_set_state(struct rq *rq, + struct cpuidle_state *idle_state) +{ + rq->idle_state = idle_state; +} + +static inline struct cpuidle_state *idle_get_state(struct rq *rq) +{ + rcu_read_lock(); + return rq->idle_state; +} + +static inline void cpuidle_put_state(struct rq *rq) +{ + rcu_read_unlock(); +} +#else +static inline void idle_set_state(struct rq *rq, + struct cpuidle_state *idle_state) +{ +} + +static inline struct cpuidle_state *idle_get_state(struct rq *rq) +{ + return NULL; +} + +static inline void cpuidle_put_state(struct rq *rq) +{ +} +#endif + extern void sysrq_sched_debug_show(void); extern void sched_init_granularity(void); extern void update_max_interval(void);