From patchwork Mon Dec 19 15:14:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krzysztof Kozlowski X-Patchwork-Id: 635375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB53AC4332F for ; Mon, 19 Dec 2022 15:15:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232870AbiLSPPS (ORCPT ); Mon, 19 Dec 2022 10:15:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43024 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232829AbiLSPPP (ORCPT ); Mon, 19 Dec 2022 10:15:15 -0500 Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com [IPv6:2a00:1450:4864:20::135]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EB68B7C4 for ; Mon, 19 Dec 2022 07:15:13 -0800 (PST) Received: by mail-lf1-x135.google.com with SMTP id cf42so14152089lfb.1 for ; Mon, 19 Dec 2022 07:15:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5GU2NjmgE9pvJfcbP7Wbuf1L4Bg6ALcOGzwayLNEdCc=; b=M96PQCEKu708zYSAKfnI3vYYuOYQPIDYBpF3ko4XOFDTuP8CChmqa/eBMxnob9Xvvu 13bJnQo4DxOWhMM/jR1CrFAF5M9LqPtdN/7YNIKVERNdhI1SDlDl3cc86hykawp9nDlW A8LaZVRmgg3mME0zszFlmghrDv2CYA9UQGulYue18w+tfHQ41K2i09gLezGgefdt2Xat yA3iZWLVPKyr3leBBG1RmA/uIihnVnrApGFWHFy9C1HuHrxB4Ntk1TXktxvLrCwRmM4l HKEKzORBxoGPpeRsegGr5/AeKG3CT40OE9h/5COkm0Ke5wzV7SIHlnUWUrpTu1NnSrTp 0XRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5GU2NjmgE9pvJfcbP7Wbuf1L4Bg6ALcOGzwayLNEdCc=; b=wKzky5sTqHyymSEowcCzbTMtCwpz/Cy5xaAFjADv+DJr1/OfDrJMfwmlpJ7vdu1bi3 ny/Iw3fl0Ag7rnaD3x+KdG8weK0yHyWO3memkv8/H0bOjBBKlczGdM56kmhKix2ezV/l fh3bfSzOgtm2HbRQ6BObdt1bK6u7g8kyEVZKW3aKhp3eFb47ApCybiCfNkT1o3qki6mY C4RRKICXU8pDWjMLDgRSJddL/8wAGl4cGmZqZwl16L5/TuYpG+1Stufr6Ae3FLdHbF2z EPCCW2bnSA0a1HyRUM/AFiVtRkdrOfrFNPPY6OKt9eM1u6QQHfGScA7rWhYcr5vW5aXJ mTGw== X-Gm-Message-State: ANoB5pnEdw+6YskzqD2djX4n0Vg+awbbil5o8rPFWVijDyjDL1g5RJFQ czXtsSz+9JhwZbr4bPrXiwhDsQ== X-Google-Smtp-Source: AA0mqf5P0EXzSYerNvMnE1tqLXFMHI3GyGO5br1pN4dVVzBmBdh/UJ2E1Tze6Cn2gf/kxMeSGKg5YA== X-Received: by 2002:ac2:560a:0:b0:4b6:f0ea:4f49 with SMTP id v10-20020ac2560a000000b004b6f0ea4f49mr7591911lfd.9.1671462912024; Mon, 19 Dec 2022 07:15:12 -0800 (PST) Received: from krzk-bin.NAT.warszawa.vectranet.pl (088156142067.dynamic-2-waw-k-3-2-0.vectranet.pl. [88.156.142.67]) by smtp.gmail.com with ESMTPSA id e1-20020a05651236c100b004bd8534ebbcsm1109894lfs.37.2022.12.19.07.15.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 07:15:11 -0800 (PST) From: Krzysztof Kozlowski To: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Krzysztof Kozlowski , Adrien Thierry , Brian Masney , linux-rt-users@vger.kernel.org Subject: [PATCH v2 1/5] PM: domains: Add GENPD_FLAG_RT_SAFE for PREEMPT_RT Date: Mon, 19 Dec 2022 16:14:59 +0100 Message-Id: <20221219151503.385816-2-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org Realtime kernels with PREEMPT_RT must use raw_spinlock_t for domains which are invoked from CPU idle (thus from atomic section). Example is cpuidle PSCI domain driver which itself is PREEMPT_RT safe, but is being called as part of cpuidle. Add a flag allowing a power domain provider to indicate it is RT safe. The flag is supposed to be used with existing GENPD_FLAG_IRQ_SAFE. Cc: Adrien Thierry Cc: Brian Masney Cc: linux-rt-users@vger.kernel.org Signed-off-by: Krzysztof Kozlowski --- Independently from Adrien, I encountered the same problem around genpd when using PREEMPT_RT kernel. Previous patch by Adrien: https://lore.kernel.org/all/20220615203605.1068453-1-athierry@redhat.com/ --- drivers/base/power/domain.c | 59 +++++++++++++++++++++++++++++++++++-- include/linux/pm_domain.h | 13 ++++++++ 2 files changed, 70 insertions(+), 2 deletions(-) diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c index 967bcf9d415e..4dfce1d476f4 100644 --- a/drivers/base/power/domain.c +++ b/drivers/base/power/domain.c @@ -119,6 +119,48 @@ static const struct genpd_lock_ops genpd_spin_ops = { .unlock = genpd_unlock_spin, }; +static void genpd_lock_rawspin(struct generic_pm_domain *genpd) + __acquires(&genpd->rslock) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&genpd->rslock, flags); + genpd->rlock_flags = flags; +} + +static void genpd_lock_nested_rawspin(struct generic_pm_domain *genpd, + int depth) + __acquires(&genpd->rslock) +{ + unsigned long flags; + + raw_spin_lock_irqsave_nested(&genpd->rslock, flags, depth); + genpd->rlock_flags = flags; +} + +static int genpd_lock_interruptible_rawspin(struct generic_pm_domain *genpd) + __acquires(&genpd->rslock) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&genpd->rslock, flags); + genpd->rlock_flags = flags; + return 0; +} + +static void genpd_unlock_rawspin(struct generic_pm_domain *genpd) + __releases(&genpd->rslock) +{ + raw_spin_unlock_irqrestore(&genpd->rslock, genpd->rlock_flags); +} + +static const struct genpd_lock_ops genpd_rawspin_ops = { + .lock = genpd_lock_rawspin, + .lock_nested = genpd_lock_nested_rawspin, + .lock_interruptible = genpd_lock_interruptible_rawspin, + .unlock = genpd_unlock_rawspin, +}; + #define genpd_lock(p) p->lock_ops->lock(p) #define genpd_lock_nested(p, d) p->lock_ops->lock_nested(p, d) #define genpd_lock_interruptible(p) p->lock_ops->lock_interruptible(p) @@ -126,6 +168,8 @@ static const struct genpd_lock_ops genpd_spin_ops = { #define genpd_status_on(genpd) (genpd->status == GENPD_STATE_ON) #define genpd_is_irq_safe(genpd) (genpd->flags & GENPD_FLAG_IRQ_SAFE) +#define genpd_is_rt_safe(genpd) (genpd_is_irq_safe(genpd) && \ + (genpd->flags & GENPD_FLAG_RT_SAFE)) #define genpd_is_always_on(genpd) (genpd->flags & GENPD_FLAG_ALWAYS_ON) #define genpd_is_active_wakeup(genpd) (genpd->flags & GENPD_FLAG_ACTIVE_WAKEUP) #define genpd_is_cpu_domain(genpd) (genpd->flags & GENPD_FLAG_CPU_DOMAIN) @@ -1838,6 +1882,12 @@ static int genpd_add_subdomain(struct generic_pm_domain *genpd, return -EINVAL; } + if (!genpd_is_rt_safe(genpd) && genpd_is_rt_safe(subdomain)) { + WARN(1, "Parent %s of subdomain %s must be RT safe\n", + genpd->name, subdomain->name); + return -EINVAL; + } + link = kzalloc(sizeof(*link), GFP_KERNEL); if (!link) return -ENOMEM; @@ -2008,8 +2058,13 @@ static void genpd_free_data(struct generic_pm_domain *genpd) static void genpd_lock_init(struct generic_pm_domain *genpd) { if (genpd->flags & GENPD_FLAG_IRQ_SAFE) { - spin_lock_init(&genpd->slock); - genpd->lock_ops = &genpd_spin_ops; + if (genpd->flags & GENPD_FLAG_RT_SAFE) { + raw_spin_lock_init(&genpd->rslock); + genpd->lock_ops = &genpd_rawspin_ops; + } else { + spin_lock_init(&genpd->slock); + genpd->lock_ops = &genpd_spin_ops; + } } else { mutex_init(&genpd->mlock); genpd->lock_ops = &genpd_mtx_ops; diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h index 1cd41bdf73cf..0a1600244963 100644 --- a/include/linux/pm_domain.h +++ b/include/linux/pm_domain.h @@ -61,6 +61,14 @@ * GENPD_FLAG_MIN_RESIDENCY: Enable the genpd governor to consider its * components' next wakeup when determining the * optimal idle state. + * + * GENPD_FLAG_RT_SAFE: When used with GENPD_FLAG_IRQ_SAFE, this informs + * genpd that its backend callbacks, ->power_on|off(), + * do not use other spinlocks. They might use + * raw_spinlocks or other pre-emption-disable + * methods, all of which are PREEMPT_RT safe. Note + * that, a genpd having this flag set, requires its + * masterdomains to also have it set. */ #define GENPD_FLAG_PM_CLK (1U << 0) #define GENPD_FLAG_IRQ_SAFE (1U << 1) @@ -69,6 +77,7 @@ #define GENPD_FLAG_CPU_DOMAIN (1U << 4) #define GENPD_FLAG_RPM_ALWAYS_ON (1U << 5) #define GENPD_FLAG_MIN_RESIDENCY (1U << 6) +#define GENPD_FLAG_RT_SAFE (1U << 7) enum gpd_status { GENPD_STATE_ON = 0, /* PM domain is on */ @@ -164,6 +173,10 @@ struct generic_pm_domain { spinlock_t slock; unsigned long lock_flags; }; + struct { + raw_spinlock_t rslock; + unsigned long rlock_flags; + }; }; }; From patchwork Mon Dec 19 15:15:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krzysztof Kozlowski X-Patchwork-Id: 635560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B689FC10F1B for ; Mon, 19 Dec 2022 15:15:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232877AbiLSPPT (ORCPT ); Mon, 19 Dec 2022 10:15:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232851AbiLSPPR (ORCPT ); Mon, 19 Dec 2022 10:15:17 -0500 Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CBC90D2C8 for ; Mon, 19 Dec 2022 07:15:14 -0800 (PST) Received: by mail-lf1-x132.google.com with SMTP id q6so14075707lfm.10 for ; Mon, 19 Dec 2022 07:15:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=710npLST/xPzuXiih2c1d9pFv9jXZIz92buQVuG0oAE=; b=uEio8RqgLj7kCKnMrT/ndnvx/qtape0C21ameHDn5oxR1frx1ZVQ/UjjKEP/ibeWWh /FrAeFueaLx7BNbF218RzyfLQhs6ts56jbLBL2inGVZ8II/hhK38SdmB46mvrjRjuhWs LneaquS3DyNt64GkklXVgbkS2zL9bf6bEp74uvqATEfY07UyjJTqmZM+QFcOLM4uy8Bo CE/mOMKX0Cwc47pRYdikRdnVWrHoU5LoezIzgPZcUuo1I6KJTFkMPX9rvq5ht+fmfcMC ONvGwjkQVW0VtSI8uJhE12tdg/7q+CM2fWOMU6PmvL3H+i1ZDUpFhpWuX439KLEKV4MS LZ7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=710npLST/xPzuXiih2c1d9pFv9jXZIz92buQVuG0oAE=; b=dyNXdCmxBoppx+bqJMsjJrB0JKfEdDbBrsi5HnD2QIoL0UfwfQ+VtMuak8yLNCONTW wn4+STI48/Yfr5x9DlHNogL4hzCVE0rWRxpqO5XeSq3oig0w/HEvhxDIWMwHEg/Z53pl i/Q8bQ7d3UqAaG2NvAq6siHDBG862ZDbJn0MTH3CLlyXegpIXegqkKMTn7qvulM1quw8 MXEAkL5NqGse4dGMAX1RIk5XYXuNYgvqDRGMxnBjast76bXaya2QfY5wlkVEt6OnkNWz 3DjqlHhef0Hczlkxk6R/8Uo867U5A0KRU6QSSlOQjGy+IV05uQfWRAZ3hJx5CJ/kLgDl ebWA== X-Gm-Message-State: ANoB5pl874XSYVW81aduGBZ8qkjcclsxnoKIRF9VTeTnufyowo5ZKLpl B9paoxltgjFvagYm8/pZgPRKXQ== X-Google-Smtp-Source: AA0mqf5O5lOshUMBMDUAarE8CUlxxQDab0aOfGYCt75u7yth94O0C4yTULqeDUE5eHe18yeibJ7oYw== X-Received: by 2002:a05:6512:261b:b0:4b4:dd3d:4310 with SMTP id bt27-20020a056512261b00b004b4dd3d4310mr16841834lfb.19.1671462913205; Mon, 19 Dec 2022 07:15:13 -0800 (PST) Received: from krzk-bin.NAT.warszawa.vectranet.pl (088156142067.dynamic-2-waw-k-3-2-0.vectranet.pl. [88.156.142.67]) by smtp.gmail.com with ESMTPSA id e1-20020a05651236c100b004bd8534ebbcsm1109894lfs.37.2022.12.19.07.15.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 07:15:12 -0800 (PST) From: Krzysztof Kozlowski To: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Krzysztof Kozlowski , Adrien Thierry , Brian Masney , linux-rt-users@vger.kernel.org Subject: [PATCH v2 2/5] cpuidle: psci: Mark as PREEMPT_RT safe Date: Mon, 19 Dec 2022 16:15:00 +0100 Message-Id: <20221219151503.385816-3-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org The PSCI cpuidle power domain in power_off callback uses __this_cpu_write() so it is PREEMPT_RT safe. This allows to use it in Realtime kernels and solves errors like: BUG: scheduling while atomic: swapper/2/0/0x00000002 Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT) Call trace: dump_backtrace.part.0+0xe0/0xf0 show_stack+0x18/0x40 dump_stack_lvl+0x68/0x84 dump_stack+0x18/0x34 __schedule_bug+0x60/0x80 __schedule+0x628/0x800 schedule_rtlock+0x28/0x5c rtlock_slowlock_locked+0x360/0xd30 rt_spin_lock+0x88/0xb0 genpd_lock_nested_spin+0x1c/0x30 genpd_power_off.part.0.isra.0+0x20c/0x2a0 genpd_runtime_suspend+0x150/0x2bc __rpm_callback+0x48/0x170 rpm_callback+0x6c/0x7c rpm_suspend+0x108/0x660 __pm_runtime_suspend+0x4c/0x8c __psci_enter_domain_idle_state.constprop.0+0x54/0xe0 psci_enter_domain_idle_state+0x18/0x2c cpuidle_enter_state+0x8c/0x4e0 cpuidle_enter+0x38/0x50 do_idle+0x248/0x2f0 cpu_startup_entry+0x24/0x30 secondary_start_kernel+0x130/0x154 __secondary_switched+0xb0/0xb4 Cc: Adrien Thierry Cc: Brian Masney Cc: linux-rt-users@vger.kernel.org Signed-off-by: Krzysztof Kozlowski --- drivers/cpuidle/cpuidle-psci-domain.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/cpuidle/cpuidle-psci-domain.c b/drivers/cpuidle/cpuidle-psci-domain.c index c80cf9ddabd8..d15a91fb7048 100644 --- a/drivers/cpuidle/cpuidle-psci-domain.c +++ b/drivers/cpuidle/cpuidle-psci-domain.c @@ -62,7 +62,8 @@ static int psci_pd_init(struct device_node *np, bool use_osi) if (!pd_provider) goto free_pd; - pd->flags |= GENPD_FLAG_IRQ_SAFE | GENPD_FLAG_CPU_DOMAIN; + pd->flags |= GENPD_FLAG_IRQ_SAFE | GENPD_FLAG_RT_SAFE | \ + GENPD_FLAG_CPU_DOMAIN; /* Allow power off when OSI has been successfully enabled. */ if (use_osi) From patchwork Mon Dec 19 15:15:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krzysztof Kozlowski X-Patchwork-Id: 635374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A42C1C4167B for ; Mon, 19 Dec 2022 15:15:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232884AbiLSPPV (ORCPT ); Mon, 19 Dec 2022 10:15:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232861AbiLSPPR (ORCPT ); Mon, 19 Dec 2022 10:15:17 -0500 Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com [IPv6:2a00:1450:4864:20::12e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 046DB10571 for ; Mon, 19 Dec 2022 07:15:16 -0800 (PST) Received: by mail-lf1-x12e.google.com with SMTP id j4so14195853lfk.0 for ; Mon, 19 Dec 2022 07:15:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NLwTozcUqVuaex+an++z/NMr4xtzIfovpnza53ZqTDE=; b=Ph1MMT+VKo3WhiyGRG1FZKkXojebF95OOhoxbbAdyKuxvtcuJHNFcTz+GQryrGBEJH JgoEcPjmfRrTHRZ6gQrzc0a97FezxX/lSEA5WkJkWTxfOmkD7BWFipsPNylStoKPLclU 4juNW9JRwUbpvvXYTgXaUp4kKivBGFRoR05rQgeYzBZ5+4gdktxjA3Mp7NsfrUCuZiAn 271GvZZQJo0W+1njeKjlKWlG7nJIT4KFHgVz9W1wNIB5EsXZdM/BJwAYkkXEkioVughp VFbbXbXy939GC7TRbJxNCrsD7XBrkTuQQLbJyg0zNVx+iQq/cWbFGjEYLrgt05etQPnn gOkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NLwTozcUqVuaex+an++z/NMr4xtzIfovpnza53ZqTDE=; b=xD3gPep/0AMnyVyjBxeP3KxahoJwn/tRiXYEwxarL4QZlyrEwFIKaLaw3rrzBEQ0r+ suamyaRyOiG7jK0HT/mPughysmRxvaiMBRB63ZdGaMrjhUbuGyfx2XjCEnY0TipeaBNJ VTdbHrQ+Oo4+mBmNYm/+8eMXg7l3d5vqecBmT9pnBE9xGRcdjGNOgkCVWrRDlKTDQNfM lr8GPT5k9i1sdH3If9IzcxLIvbMKOZKNDL2pPATkYt0tOml7J0Sut4/tgXiZl90y60u6 UsndPVN8soOrRCTDCcy+ITFBC3jBgJytIJk/mN6vX6K9lsrlA9le/b5Ph3YaLzvDjFFu /59w== X-Gm-Message-State: AFqh2krVlwkJBjsP4lnWQ2DBFG8bNObxWtpbgpYuknvxxsldPMO6iARk Gxm8WZwMbkCLenqaBYb2SHY9Mg== X-Google-Smtp-Source: AMrXdXuGYlf+vpB46IweTHtie63ev7zPtyxOiEOJ2TTi6R+ghe6q6e1/IXIJXq9YScrmKCSb2fxeRg== X-Received: by 2002:a05:6512:695:b0:4c3:3a78:eab6 with SMTP id t21-20020a056512069500b004c33a78eab6mr4004090lfe.65.1671462914357; Mon, 19 Dec 2022 07:15:14 -0800 (PST) Received: from krzk-bin.NAT.warszawa.vectranet.pl (088156142067.dynamic-2-waw-k-3-2-0.vectranet.pl. [88.156.142.67]) by smtp.gmail.com with ESMTPSA id e1-20020a05651236c100b004bd8534ebbcsm1109894lfs.37.2022.12.19.07.15.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 07:15:14 -0800 (PST) From: Krzysztof Kozlowski To: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Krzysztof Kozlowski , Adrien Thierry , Brian Masney , linux-rt-users@vger.kernel.org Subject: [PATCH v2 3/5] cpuidle: psci: Do not suspend topology CPUs on PREEMPT_RT Date: Mon, 19 Dec 2022 16:15:01 +0100 Message-Id: <20221219151503.385816-4-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org Realtime kernels are not supposed to go into deep sleep modes (neither system nor CPUs) because the latencies might become unpredictable. Therefore actual power suspending of CPU topology is not that important. On the other hand, this runtime PM of CPU topology is not compatible with PREEMPT_RT: 1. Core cpuidle path disables IRQs. 2. Core cpuidle calls cpuidle-psci. 3. cpuidle-psci in __psci_enter_domain_idle_state() calls pm_runtime_put_sync_suspend() and pm_runtime_get_sync() which use spinlocks (which are sleeping on PREEMPT_RT): BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:46 in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 0, name: swapper/0 preempt_count: 1, expected: 0 RCU nest depth: 0, expected: 0 1 lock held by swapper/0/0: #0: ffff20524613c1a0 (&dev->power.lock){+.+.}-{3:3}, at: __pm_runtime_suspend+0x30/0xac irq event stamp: 18776 hardirqs last enabled at (18775): [] tick_nohz_idle_enter+0x7c/0x17c hardirqs last disabled at (18776): [] do_idle+0xe0/0x300 softirqs last enabled at (4310): [] __local_bh_enable_ip+0x98/0x280 softirqs last disabled at (4288): [] cgroup_idr_alloc.constprop.0+0x0/0xd0 Preemption disabled at: [] schedule_preempt_disabled+0x20/0x30 CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.0-rt5-00322-gb49b67f1d8dc #1 Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT) Call trace: dump_backtrace.part.0+0xe0/0xf0 show_stack+0x18/0x40 dump_stack_lvl+0x8c/0xb8 dump_stack+0x18/0x34 __might_resched+0x17c/0x214 rt_spin_lock+0x5c/0x100 __pm_runtime_suspend+0x30/0xac __psci_enter_domain_idle_state.constprop.0+0x60/0x104 psci_enter_domain_idle_state+0x18/0x2c cpuidle_enter_state+0x220/0x37c cpuidle_enter+0x38/0x50 do_idle+0x258/0x300 cpu_startup_entry+0x28/0x30 rest_init+0x104/0x180 arch_post_acpi_subsys_init+0x0/0x18 start_kernel+0x6fc/0x73c __primary_switched+0xbc/0xc4 Cc: Adrien Thierry Cc: Brian Masney Cc: linux-rt-users@vger.kernel.org Signed-off-by: Krzysztof Kozlowski --- drivers/cpuidle/cpuidle-psci.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index 57bc3e3ae391..9d971cc4b12b 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -72,7 +72,7 @@ static int __psci_enter_domain_idle_state(struct cpuidle_device *dev, ct_irq_enter_irqson(); if (s2idle) dev_pm_genpd_suspend(pd_dev); - else + else if (!IS_ENABLED(CONFIG_PREEMPT_RT)) pm_runtime_put_sync_suspend(pd_dev); ct_irq_exit_irqson(); @@ -85,7 +85,7 @@ static int __psci_enter_domain_idle_state(struct cpuidle_device *dev, ct_irq_enter_irqson(); if (s2idle) dev_pm_genpd_resume(pd_dev); - else + else if (!IS_ENABLED(CONFIG_PREEMPT_RT)) pm_runtime_get_sync(pd_dev); ct_irq_exit_irqson(); From patchwork Mon Dec 19 15:15:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krzysztof Kozlowski X-Patchwork-Id: 635559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43260C46467 for ; Mon, 19 Dec 2022 15:15:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232868AbiLSPPY (ORCPT ); Mon, 19 Dec 2022 10:15:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232854AbiLSPPR (ORCPT ); Mon, 19 Dec 2022 10:15:17 -0500 Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com [IPv6:2a00:1450:4864:20::130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EFC5EF580 for ; Mon, 19 Dec 2022 07:15:15 -0800 (PST) Received: by mail-lf1-x130.google.com with SMTP id cf42so14152328lfb.1 for ; Mon, 19 Dec 2022 07:15:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=y2D0MZ2hxPTHwXNnho8HOkobrXZQciZPG6h42YTOjRw=; b=ucUJppIgRBgtpDQOSNoFKkTpre/wj89djnkxbJXoXfrK0vOY/QJLX1iUpqT5cY5qV/ GF+Dl82w3xJ9mA3oFvYtRV0YhivF59Kq+GNvnC3Jqv4aY+ohv2cJJWZlvBeaEZEzK6ou HIlLEZWK5gd1rofobutljGLWfO3nULqGfRQ0ICl88S/eoknin4PISCvnL7lw7XSStvto AE4nkyesvYnJaizYJIHpktZOUcmnuiuncKvdsRPUYSWCKWWwaxvzUs4Jljp2D+ucASqX xQDAJUDjIsYIBgPxGqZ7U5LO8uF94J/koNWBdV2fQu+agiJ3nra/ype6SXUSompNmfk6 bBwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y2D0MZ2hxPTHwXNnho8HOkobrXZQciZPG6h42YTOjRw=; b=peg068dHw4G96Thmr0VHHe1/vqhtyjRX2GPB4fhGFz2qHRlYgw4oFbupWqdY4Czxoo ccjWXvhODtF+gWuVk2TBbQVgXGrFeEskn3uX72cSOsLzFi/NFs1PcsiCOCy1vtD1BZVP r6TTzxkmCQNji9lsE+45RWlVqfpO46Kso9RCfKF97oJMaoU8Lp4lNk3wlRm1FKEIyAbV CSBbp6er9WLL5Fw4xYhDYGtWF0M8dr/ju/C39TKv4NwdgR1X+BgQsGpK6S5BCE1BuEzr +xBCbn3RProiHaN3qyUNK4+CPcdh/Nlo6ed6O3gbzggrlD2ufFQTEUKMszDjhVa7R2Ob 90hw== X-Gm-Message-State: ANoB5pmA/K7ueJh6vxsl4nrJx+Om3bK4YMBsNeQc5yu2/tUX+pYNLNXN mY3C079c+MbviZNZtK2M7S/iiQ== X-Google-Smtp-Source: AA0mqf7n/NP8HRaYuYDbxI6GABLuMF7vUSdWaQUoDWCU0hAjFOQml5a/3nwbg+N3+37H31i1a6Yv2A== X-Received: by 2002:a05:6512:a94:b0:4b7:113:9296 with SMTP id m20-20020a0565120a9400b004b701139296mr14233163lfu.14.1671462915529; Mon, 19 Dec 2022 07:15:15 -0800 (PST) Received: from krzk-bin.NAT.warszawa.vectranet.pl (088156142067.dynamic-2-waw-k-3-2-0.vectranet.pl. [88.156.142.67]) by smtp.gmail.com with ESMTPSA id e1-20020a05651236c100b004bd8534ebbcsm1109894lfs.37.2022.12.19.07.15.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 07:15:15 -0800 (PST) From: Krzysztof Kozlowski To: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Krzysztof Kozlowski , Adrien Thierry , Brian Masney , linux-rt-users@vger.kernel.org Subject: [PATCH v2 4/5] PM: Allow calling dev_pm_domain_set() with raw spinlock Date: Mon, 19 Dec 2022 16:15:02 +0100 Message-Id: <20221219151503.385816-5-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org device_pm_check_callbacks() uses dev->power spinlock, which on PREEMPT_RT sleeps. However some PM domains on PREEMPT_RT might be using raw spinlocks as genpd_lock(), thus dev_pm_domain_set() must not call device_pm_check_callbacks(). In fact device_pm_check_callbacks() is not strictly related to dev_pm_domain_set() and calls for these two can be made separately. Add new helper dev_pm_domain_set_no_cb() which will only set PM domain but will not check the callbacks, leaving the checl to the caller. Cc: Adrien Thierry Cc: Brian Masney Cc: linux-rt-users@vger.kernel.org Signed-off-by: Krzysztof Kozlowski --- drivers/base/power/common.c | 27 +++++++++++++++++++++++++-- include/linux/pm_domain.h | 3 +++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/drivers/base/power/common.c b/drivers/base/power/common.c index 72115917e0bd..f81cab6990ad 100644 --- a/drivers/base/power/common.c +++ b/drivers/base/power/common.c @@ -218,6 +218,30 @@ EXPORT_SYMBOL_GPL(dev_pm_domain_start); * This function must be called with the device lock held. */ void dev_pm_domain_set(struct device *dev, struct dev_pm_domain *pd) +{ + if (dev->pm_domain == pd) + return; + + dev_pm_domain_set_no_cb(dev, pd); + device_pm_check_callbacks(dev); +} +EXPORT_SYMBOL_GPL(dev_pm_domain_set); + +/** + * dev_pm_domain_set_no_cb - Set PM domain of a device. + * @dev: Device whose PM domain is to be set. + * @pd: PM domain to be set, or NULL. + * + * Sets the PM domain the device belongs to. The PM domain of a device needs + * to be set before its probe finishes (it's bound to a driver). + * + * This is exactly like dev_pm_domain_set(), however device_pm_check_callbacks() + * is not called and the caller is responsible to invoke + * device_pm_check_callbacks() with device lock held. + * + * This function must be called with the device lock held. + */ +void dev_pm_domain_set_no_cb(struct device *dev, struct dev_pm_domain *pd) { if (dev->pm_domain == pd) return; @@ -225,6 +249,5 @@ void dev_pm_domain_set(struct device *dev, struct dev_pm_domain *pd) WARN(pd && device_is_bound(dev), "PM domains can only be changed for unbound devices\n"); dev->pm_domain = pd; - device_pm_check_callbacks(dev); } -EXPORT_SYMBOL_GPL(dev_pm_domain_set); +EXPORT_SYMBOL_GPL(dev_pm_domain_set_no_cb); diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h index 0a1600244963..352d0c76bfec 100644 --- a/include/linux/pm_domain.h +++ b/include/linux/pm_domain.h @@ -438,6 +438,7 @@ struct device *dev_pm_domain_attach_by_name(struct device *dev, void dev_pm_domain_detach(struct device *dev, bool power_off); int dev_pm_domain_start(struct device *dev); void dev_pm_domain_set(struct device *dev, struct dev_pm_domain *pd); +void dev_pm_domain_set_no_cb(struct device *dev, struct dev_pm_domain *pd); #else static inline int dev_pm_domain_attach(struct device *dev, bool power_on) { @@ -460,6 +461,8 @@ static inline int dev_pm_domain_start(struct device *dev) } static inline void dev_pm_domain_set(struct device *dev, struct dev_pm_domain *pd) {} +static inline void dev_pm_domain_set_no_cb(struct device *dev, + struct dev_pm_domain *pd) {} #endif #endif /* _LINUX_PM_DOMAIN_H */ From patchwork Mon Dec 19 15:15:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krzysztof Kozlowski X-Patchwork-Id: 635373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFA15C4332F for ; Mon, 19 Dec 2022 15:15:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232911AbiLSPP1 (ORCPT ); Mon, 19 Dec 2022 10:15:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43024 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232875AbiLSPPS (ORCPT ); Mon, 19 Dec 2022 10:15:18 -0500 Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26F7D11816 for ; Mon, 19 Dec 2022 07:15:17 -0800 (PST) Received: by mail-lf1-x134.google.com with SMTP id b13so14138215lfo.3 for ; Mon, 19 Dec 2022 07:15:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=m8Uj/f0x4H68pfp6EBuS49q8mUm/dd1ntZTx2c5gQRw=; b=joGdkJgxChVHrhqXxh4D8sJDaKpzxBgUcFgLtPczIcYGrPWnS0gYSfGGb/xwur1BZA HXUG1BhIHBctF7bFBpebP9+UCrd1/wX6sQ06q9+cIYwobrrc2Zzo3WsBKRD3558LWzFQ /N47j8fJwjN1I4eRjd0Qr07xS6EOzoIv8QhoPs4d+muMgyaWlNH3o7v1b1Xr6gxm8Bsn zsRaxMej9uybQ/82hjh0l+j0+akCnf8EYggcdPI5wQ4zTU9cGGyx+pH3DhxSdkmYX4n/ boQmP89KDmwLe8i18IoxJ121DOicW9mnYp04EIuXSuoqBPamMqwF9q353kDRDAlSKDOO ifvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=m8Uj/f0x4H68pfp6EBuS49q8mUm/dd1ntZTx2c5gQRw=; b=tYfOJjBhX4kWt6vWEfVuxF8m5j2HsQBhqsXwC1YlTa9VRqUByxtss5stKmSuYI/OwY BqpYZlOHwzk0HR49KRFnFp9bto4OiKaU/1th6w+Jz8omG85LUPe2QonZ8YrHRPPJ3tj0 gJac/BT8EuIVZHXF8ulkMKKfwuSykDo4dBkgqM5cX16Qf2gYHssLQD+nuJxj1jUDjvwy MppZMWRFfQv7hNIip/KYEAu2d+KpD7ItracIQ57LYwfIjxmI5Fqm9jUVTd1N0xi5RxC1 0wNOHDr61DNHGMPqv3LCff4+92HY02P8KpIrGCukwbvHmlC1Kx5kwIvY5GVFFQxLjeXL BQSw== X-Gm-Message-State: ANoB5plcx/Dcn3njTVyL1mWyw4BgzvSwc+7jSCUY4K8876T1vrmwGLzZ Ev67y2TuoVW2wu2E6UsTpzp9Rg== X-Google-Smtp-Source: AA0mqf6TIA7wSK78A1e7LxWcZY/ah0RMdr0IMbRzO6DwrSrVhPg3kDS+rfPE3kSDU9y+6k9YFRKeKw== X-Received: by 2002:a05:6512:2c85:b0:4a4:68b9:607f with SMTP id dw5-20020a0565122c8500b004a468b9607fmr11616697lfb.10.1671462916643; Mon, 19 Dec 2022 07:15:16 -0800 (PST) Received: from krzk-bin.NAT.warszawa.vectranet.pl (088156142067.dynamic-2-waw-k-3-2-0.vectranet.pl. [88.156.142.67]) by smtp.gmail.com with ESMTPSA id e1-20020a05651236c100b004bd8534ebbcsm1109894lfs.37.2022.12.19.07.15.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 07:15:16 -0800 (PST) From: Krzysztof Kozlowski To: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Krzysztof Kozlowski , Adrien Thierry , Brian Masney , linux-rt-users@vger.kernel.org Subject: [PATCH v2 5/5] PM: domains: Do not call device_pm_check_callbacks() when holding genpd_lock() Date: Mon, 19 Dec 2022 16:15:03 +0100 Message-Id: <20221219151503.385816-6-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rt-users@vger.kernel.org If PM domain on PREEMPT_RT is marked as GENPD_FLAG_RT_SAFE(), the genpd_lock() will be a raw spin lock, thus device_pm_check_callbacks() must be called outside of the domain lock. This solves on PREEMPT_RT: [ BUG: Invalid wait context ] 6.1.0-rt5-00325-g8a5f56bcfcca #8 Tainted: G W ----------------------------- swapper/0/1 is trying to lock: ffff76e045dec9a0 (&dev->power.lock){+.+.}-{3:3}, at: device_pm_check_callbacks+0x20/0xf0 other info that might help us debug this: context-{5:5} 3 locks held by swapper/0/1: #0: ffff76e045deb8e8 (&dev->mutex){....}-{4:4}, at: __device_attach+0x38/0x1c0 #1: ffffa92b81f825e0 (gpd_list_lock){+.+.}-{4:4}, at: __genpd_dev_pm_attach+0x7c/0x250 #2: ffff76e04105c7a0 (&genpd->rslock){....}-{2:2}, at: genpd_lock_rawspin+0x1c/0x30 stack backtrace: CPU: 5 PID: 1 Comm: swapper/0 Tainted: G W 6.1.0-rt5-00325-g8a5f56bcfcca #8 Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT) Call trace: dump_backtrace.part.0+0xe0/0xf0 show_stack+0x18/0x40 dump_stack_lvl+0x8c/0xb8 dump_stack+0x18/0x34 __lock_acquire+0x938/0x2100 lock_acquire.part.0+0x104/0x28c lock_acquire+0x68/0x84 rt_spin_lock+0x40/0x100 device_pm_check_callbacks+0x20/0xf0 dev_pm_domain_set+0x54/0x64 genpd_add_device+0x258/0x340 __genpd_dev_pm_attach+0xa8/0x250 genpd_dev_pm_attach_by_id+0xc4/0x190 genpd_dev_pm_attach_by_name+0x3c/0x60 dev_pm_domain_attach_by_name+0x20/0x30 dt_idle_attach_cpu+0x24/0x90 psci_cpuidle_probe+0x300/0x4b0 platform_probe+0x68/0xe0 really_probe+0xbc/0x2dc __driver_probe_device+0x78/0xe0 driver_probe_device+0x3c/0x160 __device_attach_driver+0xb8/0x140 bus_for_each_drv+0x78/0xd0 __device_attach+0xa8/0x1c0 device_initial_probe+0x14/0x20 bus_probe_device+0x9c/0xa4 device_add+0x3b4/0x8dc platform_device_add+0x114/0x234 platform_device_register_full+0x108/0x1a4 psci_idle_init+0x6c/0xb0 do_one_initcall+0x74/0x450 kernel_init_freeable+0x2e0/0x350 kernel_init+0x24/0x130 ret_from_fork+0x10/0x20 Cc: Adrien Thierry Cc: Brian Masney Cc: linux-rt-users@vger.kernel.org Signed-off-by: Krzysztof Kozlowski --- drivers/base/power/domain.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c index 4dfce1d476f4..db499ba40497 100644 --- a/drivers/base/power/domain.c +++ b/drivers/base/power/domain.c @@ -1666,10 +1666,14 @@ static int genpd_add_device(struct generic_pm_domain *genpd, struct device *dev, if (ret) goto out; + + /* PREEMPT_RT: Must be outside of genpd_lock */ + device_pm_check_callbacks(dev); + genpd_lock(genpd); genpd_set_cpumask(genpd, gpd_data->cpu); - dev_pm_domain_set(dev, &genpd->domain); + dev_pm_domain_set_no_cb(dev, &genpd->domain); genpd->device_count++; if (gd)