From patchwork Mon Nov 2 05:21:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 315244 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7C19C388F2 for ; Mon, 2 Nov 2020 05:21:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9B98C222BA for ; Mon, 2 Nov 2020 05:21:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="LcvlVVPc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727808AbgKBFVu (ORCPT ); Mon, 2 Nov 2020 00:21:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725208AbgKBFVt (ORCPT ); Mon, 2 Nov 2020 00:21:49 -0500 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AC72C061A04 for ; Sun, 1 Nov 2020 21:21:48 -0800 (PST) Received: by mail-pg1-x542.google.com with SMTP id h6so9784913pgk.4 for ; Sun, 01 Nov 2020 21:21:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ZLied8helMF+F7KhRJDmbzvMJ1y9a8NSxlUAHjNzrA4=; b=LcvlVVPcHl8HKxEsJOC3f/AskunDiuG6wO8JqVWzXUizFQzMUsGK1y3I6LiXYgWbdG UEJkXD4PSQoj33CalsPE9+ztINdFlLkt0ehHERU9GrSD/WJqIRSJfA0m6JNVm8dU+7TD Fs3sxoiXonqIyuwzW6u0lH71gcPQnsK/OglfTtikUebccZjHuk3Rq1ume3RT1ilBCR/u 3HBiAlrJzAwt9NY+78UeJyktKmyfM9Z3EjFb5wKu4jq49nOc118ZPdZe4veaZRHiacPs 8Swq8mmjiqL1JdMPm/b0LW5rGOcsJmGUa4U4TfevS3C8piJR5iWjR4foG/a5Gi9uSWgS q6ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ZLied8helMF+F7KhRJDmbzvMJ1y9a8NSxlUAHjNzrA4=; b=dg0HSg0ZmYZ8V3BrN48zza5u0u12gHRC6gSXYSQCZfRge8e+nBO6HOKXVN65T5vRCV Me8TytQSHI3aYU1s2HWG+hlAaRSCjlZikUk3PAwR1PuqW9bldAaZqD8mx78SHsf29wHO x+7ElMJwbLxo12WkIz5c4leA1kUSTkyK7WjCaLUcrqQuosPW8EoytEOKC14xG4gtX35b pvJ2meJub2G1C69TlwdmKKZZGqC4pNDTVoP1/+WcTA9z9/OfJOOMUsgWXADWqTCDHTqa gRJWZMtoG5v6MmC2z2JjDmmmeWquK7cUe+s3JKX8s3snZpJ+aW0xuTFc+GBIWT0zU+HW Fl5A== X-Gm-Message-State: AOAM530Y/yr96H11WqqCLnghJ01RhGKgu7Tl4GBUBxUI3kd8ck3hKfWs iRyE/vj2QX1zaQ30uex8MGOdtA== X-Google-Smtp-Source: ABdhPJyaQSnyKmfheFF8kxj1+XNkSRk5unrkzwuHuXuBwOQJneDdzvwSyxx8WZ9seXSSbIglizHbwg== X-Received: by 2002:a17:90a:ef81:: with SMTP id m1mr16029143pjy.212.1604294507492; Sun, 01 Nov 2020 21:21:47 -0800 (PST) Received: from localhost ([122.181.54.133]) by smtp.gmail.com with ESMTPSA id g17sm12310731pfu.130.2020.11.01.21.21.46 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 01 Nov 2020 21:21:46 -0800 (PST) From: Viresh Kumar To: Viresh Kumar , Nishanth Menon , Stephen Boyd Cc: Viresh Kumar , linux-pm@vger.kernel.org, Vincent Guittot , Rafael Wysocki , robdclark@gmail.com, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] opp: Always add entries in dev_list with opp_table->lock held Date: Mon, 2 Nov 2020 10:51:38 +0530 Message-Id: X-Mailer: git-send-email 2.25.0.rc1.19.g042ed3e048af MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The readers of dev_list expect the updates to it to take place from within the opp_table->lock and this is missing in the case where the dev_list is updated for already managed OPPs. Fix that by calling _add_opp_dev() from there and remove the now unused _add_opp_dev_unlocked() callback. While at it, also reduce the length of the critical section in _add_opp_dev(). Signed-off-by: Viresh Kumar --- drivers/opp/core.c | 20 +++++--------------- 1 file changed, 5 insertions(+), 15 deletions(-) diff --git a/drivers/opp/core.c b/drivers/opp/core.c index 0e0a5269dc82..84035ab8bb31 100644 --- a/drivers/opp/core.c +++ b/drivers/opp/core.c @@ -1036,8 +1036,8 @@ static void _remove_opp_dev(struct opp_device *opp_dev, kfree(opp_dev); } -static struct opp_device *_add_opp_dev_unlocked(const struct device *dev, - struct opp_table *opp_table) +struct opp_device *_add_opp_dev(const struct device *dev, + struct opp_table *opp_table) { struct opp_device *opp_dev; @@ -1048,7 +1048,9 @@ static struct opp_device *_add_opp_dev_unlocked(const struct device *dev, /* Initialize opp-dev */ opp_dev->dev = dev; + mutex_lock(&opp_table->lock); list_add(&opp_dev->node, &opp_table->dev_list); + mutex_unlock(&opp_table->lock); /* Create debugfs entries for the opp_table */ opp_debug_register(opp_dev, opp_table); @@ -1056,18 +1058,6 @@ static struct opp_device *_add_opp_dev_unlocked(const struct device *dev, return opp_dev; } -struct opp_device *_add_opp_dev(const struct device *dev, - struct opp_table *opp_table) -{ - struct opp_device *opp_dev; - - mutex_lock(&opp_table->lock); - opp_dev = _add_opp_dev_unlocked(dev, opp_table); - mutex_unlock(&opp_table->lock); - - return opp_dev; -} - static struct opp_table *_allocate_opp_table(struct device *dev, int index) { struct opp_table *opp_table; @@ -1148,7 +1138,7 @@ static struct opp_table *_opp_get_opp_table(struct device *dev, int index) opp_table = _managed_opp(dev, index); if (opp_table) { - if (!_add_opp_dev_unlocked(dev, opp_table)) { + if (!_add_opp_dev(dev, opp_table)) { dev_pm_opp_put_opp_table(opp_table); opp_table = ERR_PTR(-ENOMEM); } From patchwork Mon Nov 2 05:21:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 320185 Delivered-To: patch@linaro.org Received: by 2002:a92:7b12:0:0:0:0:0 with SMTP id w18csp3446980ilc; Sun, 1 Nov 2020 21:21:55 -0800 (PST) X-Google-Smtp-Source: ABdhPJzkC3NYyze6rbmZN/cQ7S7hdCWrMcpzd/0fz2TmyhVU4mXMHfWB37kZC7MHyfv+i3ADD4MQ X-Received: by 2002:aa7:d2d2:: with SMTP id k18mr8106543edr.290.1604294515749; Sun, 01 Nov 2020 21:21:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1604294515; cv=none; d=google.com; s=arc-20160816; b=qN+osN1kq+X9hRzy1l4KbOnQ11yywDkizg1P97RrSJyJY/6JZHgjSuNJGEsrkix1G8 Gq1fetFd97pVa/Zocbc7KVSIv+c9W0lOJ9CGIseNFKhbgB+1O4nCWWaxty0QQnsH6b1l R/+LwcUjlP6+F6hEncSgJeYOIexuVKKzNApczDlEzoYWjMJn6ps3vtjASM2wVfF+m+GA kSLGTJNC8bN1vop9DiHzub7KJAl/SwAuyLNi+Ec0SqidprIGuJTAvrw6kPXhCPy943Cw jdMalT13pE46AhgcDtqTzwPx4Yf+/vH35pL0l3QB4+GwE+aU3jC5DRQ6TKYg3yqr+2qH XEXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SN8bYxSdkigCEEdTUoVtcPHjy2/GSlL+rkbq+IMRWrA=; b=n5FKqGvyBnFuXQxhpY84q/Xi1wIF0fHQuCAy62FYa5uKdkcosPETd/hgikvPe5yvYI cR6h82VHk/gK/FEPUweDGNo0DGj+pNgIvu5vAuda2XJwUD+dYJ8JKoM7twHSSHCYxUMN DwVTdqP/zCnfIArI3m4bjBBJUb87DnLSJace7lEpTIww0bOrGp+WasVPylfYzrLx9Xcl wjCUM7LZK3Fd8ro8oLVgwYexbdJhqxp8h9pD7wlwLhNMvXzyTcGUlnPeAFSK+0o7TQkM 4D1b57YwrYdPQd7/r2VNHcHiSdBi6IgAudeKJ39ckXTuFpd6/DOZyCcdg4X/KQXJBE9+ Cwjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="SLJrR4/Z"; spf=pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f3si6088409edy.186.2020.11.01.21.21.55; Sun, 01 Nov 2020 21:21:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="SLJrR4/Z"; spf=pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727846AbgKBFVw (ORCPT + 9 others); Mon, 2 Nov 2020 00:21:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725208AbgKBFVv (ORCPT ); Mon, 2 Nov 2020 00:21:51 -0500 Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2A85C0617A6 for ; Sun, 1 Nov 2020 21:21:51 -0800 (PST) Received: by mail-pf1-x443.google.com with SMTP id z3so3696322pfz.6 for ; Sun, 01 Nov 2020 21:21:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SN8bYxSdkigCEEdTUoVtcPHjy2/GSlL+rkbq+IMRWrA=; b=SLJrR4/Zm0U+gkJ7TFiR91FqKxoLXv21ITOr2Uahf20EOjrOKJEhCcoskUmnvy+PCK R07y1lyS71ZJ189nLdieoQK+bRcOvpCAuXQnh+HTZTWOjF8eO1rQwQRFNjSEy2NYteqC NXkGUeTIycRLAHYuUiH0pC41frvt0sHa5mJgrJQxiy1m3Jj7RvUqvwhq8gDkmG7Dj7fq RP1KPe5O/qiihOaiUVwLOaYrDwF/z94V50qMg7X7DR2Te3QTeSiZ1CAUJJkdMq94dDaQ ktEZAbyBKwy0jTkT99+A7xI9k2qBWWvTxfce9h0pe7WN+VBjS2zcrlM5qQghx6EoePfK PhCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SN8bYxSdkigCEEdTUoVtcPHjy2/GSlL+rkbq+IMRWrA=; b=SOsBk36vRYIxISAMMf4F8iThrwL3nggioIYHg3WFIhROya4gSMLpNbygMg2Z0wuDGB C7L7H/ULcngNKnMtnvXhcBZpwC8kOUqXjDs2cneQyod1hV4pHuUsq5V+yifTrUbZP0Tq cfZ9lcNpTgBXfDHTPR9+tYpp04ZAb/YL9VO8SeZSJ8I74xkQ2iXPXL/8UmwPTi5OvT9f MnPqFIrOmHgE4bPZAa4D/UCNtPrOlVwncrvxrppOuSdDNyTNaDz1cziBLdYN4RfHpy+B Pt7FTT0MYKRzSr3+RRv5eYse9OTNx8pgJAbNVqVMBs0ThuZlkBgZwteadBTVCBw0tUWv C0cA== X-Gm-Message-State: AOAM532T6yKDEVvmxNsXdeEI5Y8vvz36bAkssd7cdxQoMKe2wNwxii+9 wrxyBVW4pnxxAQTJoE+8zrJZoA3JqhSgZg== X-Received: by 2002:a17:90a:5b02:: with SMTP id o2mr15112647pji.221.1604294511312; Sun, 01 Nov 2020 21:21:51 -0800 (PST) Received: from localhost ([122.181.54.133]) by smtp.gmail.com with ESMTPSA id 21sm3396343pfw.105.2020.11.01.21.21.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 01 Nov 2020 21:21:50 -0800 (PST) From: Viresh Kumar To: Viresh Kumar , Nishanth Menon , Stephen Boyd Cc: Viresh Kumar , linux-pm@vger.kernel.org, Vincent Guittot , Rafael Wysocki , robdclark@gmail.com, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] opp: Allocate the OPP table outside of opp_table_lock Date: Mon, 2 Nov 2020 10:51:39 +0530 Message-Id: <836b442002d598437e3e6b56d3e07087e5c08afd.1604294460.git.viresh.kumar@linaro.org> X-Mailer: git-send-email 2.25.0.rc1.19.g042ed3e048af In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org There is no critical section which needs protection with locks while allocating an OPP table, except while adding it to the opp_tables list. And taking the opp_table_lock for the entire duration causes circular dependency issues like the one mentioned below. This patch takes another approach to reduce the size of the critical section to avoid such issues, the details of that are present within the patch. ====================================================== WARNING: possible circular locking dependency detected 5.4.72 #14 Not tainted ------------------------------------------------------ chrome/1865 is trying to acquire lock: ffffffdd34921750 (opp_table_lock){+.+.}, at: _find_opp_table+0x34/0x74 but task is already holding lock: ffffff81f0fc71a8 (reservation_ww_class_mutex){+.+.}, at: submit_lock_objects+0x70/0x1ec fscrypt: AES-256-CTS-CBC using implementation "cts-cbc-aes-ce" which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #4 (reservation_ww_class_mutex){+.+.}: __mutex_lock_common+0xec/0xc0c ww_mutex_lock_interruptible+0x5c/0xc4 msm_gem_fault+0x2c/0x124 __do_fault+0x40/0x16c handle_mm_fault+0x7cc/0xd98 do_page_fault+0x230/0x3b4 do_translation_fault+0x5c/0x78 do_mem_abort+0x4c/0xb4 el0_da+0x1c/0x20 -> #3 (&mm->mmap_sem){++++}: __might_fault+0x70/0x98 compat_filldir+0xf8/0x48c dcache_readdir+0x70/0x1dc iterate_dir+0xd4/0x180 __arm64_compat_sys_getdents+0xa0/0x19c el0_svc_common+0xa8/0x178 el0_svc_compat_handler+0x2c/0x40 el0_svc_compat+0x8/0x10 -> #2 (&sb->s_type->i_mutex_key#3){++++}: down_write+0x54/0x16c start_creating+0x68/0x128 debugfs_create_dir+0x28/0x114 opp_debug_register+0x8c/0xc0 _add_opp_dev_unlocked+0x5c/0x70 _add_opp_dev+0x38/0x58 _opp_get_opp_table+0xdc/0x1ac dev_pm_opp_get_opp_table_indexed+0x24/0x30 dev_pm_opp_of_add_table_indexed+0x48/0x84 of_genpd_add_provider_onecell+0xc0/0x1b8 rpmhpd_probe+0x240/0x268 platform_drv_probe+0x90/0xb0 really_probe+0x134/0x2ec driver_probe_device+0x64/0xfc __device_attach_driver+0x8c/0xa4 bus_for_each_drv+0x90/0xd8 __device_attach+0xc0/0x148 device_initial_probe+0x20/0x2c bus_probe_device+0x34/0x94 device_add+0x1fc/0x3b0 of_device_add+0x3c/0x4c of_platform_device_create_pdata+0xb8/0xfc of_platform_bus_create+0x1e4/0x368 of_platform_populate+0x70/0xbc devm_of_platform_populate+0x58/0xa0 rpmh_rsc_probe+0x36c/0x3cc platform_drv_probe+0x90/0xb0 really_probe+0x134/0x2ec driver_probe_device+0x64/0xfc __device_attach_driver+0x8c/0xa4 bus_for_each_drv+0x90/0xd8 __device_attach+0xc0/0x148 device_initial_probe+0x20/0x2c bus_probe_device+0x34/0x94 device_add+0x1fc/0x3b0 of_device_add+0x3c/0x4c of_platform_device_create_pdata+0xb8/0xfc of_platform_bus_create+0x1e4/0x368 of_platform_bus_create+0x230/0x368 of_platform_populate+0x70/0xbc of_platform_default_populate_init+0xa8/0xc0 do_one_initcall+0x1c8/0x3fc do_initcall_level+0xb4/0x10c do_basic_setup+0x30/0x48 kernel_init_freeable+0x124/0x1a4 kernel_init+0x14/0x104 ret_from_fork+0x10/0x18 -> #1 (&opp_table->lock){+.+.}: __mutex_lock_common+0xec/0xc0c mutex_lock_nested+0x40/0x50 _add_opp_dev+0x2c/0x58 _opp_get_opp_table+0xdc/0x1ac dev_pm_opp_get_opp_table_indexed+0x24/0x30 dev_pm_opp_of_add_table_indexed+0x48/0x84 of_genpd_add_provider_onecell+0xc0/0x1b8 rpmhpd_probe+0x240/0x268 platform_drv_probe+0x90/0xb0 really_probe+0x134/0x2ec driver_probe_device+0x64/0xfc __device_attach_driver+0x8c/0xa4 bus_for_each_drv+0x90/0xd8 __device_attach+0xc0/0x148 device_initial_probe+0x20/0x2c bus_probe_device+0x34/0x94 device_add+0x1fc/0x3b0 of_device_add+0x3c/0x4c of_platform_device_create_pdata+0xb8/0xfc of_platform_bus_create+0x1e4/0x368 of_platform_populate+0x70/0xbc devm_of_platform_populate+0x58/0xa0 rpmh_rsc_probe+0x36c/0x3cc platform_drv_probe+0x90/0xb0 really_probe+0x134/0x2ec driver_probe_device+0x64/0xfc __device_attach_driver+0x8c/0xa4 bus_for_each_drv+0x90/0xd8 __device_attach+0xc0/0x148 device_initial_probe+0x20/0x2c bus_probe_device+0x34/0x94 device_add+0x1fc/0x3b0 of_device_add+0x3c/0x4c of_platform_device_create_pdata+0xb8/0xfc of_platform_bus_create+0x1e4/0x368 of_platform_populate+0x70/0xbc devm_of_platform_populate+0x58/0xa0 rpmh_rsc_probe+0x36c/0x3cc platform_drv_probe+0x90/0xb0 really_probe+0x134/0x2ec driver_probe_device+0x64/0xfc __device_attach_driver+0x8c/0xa4 bus_for_each_drv+0x90/0xd8 __device_attach+0xc0/0x148 device_initial_probe+0x20/0x2c bus_probe_device+0x34/0x94 device_add+0x1fc/0x3b0 of_device_add+0x3c/0x4c of_platform_device_create_pdata+0xb8/0xfc of_platform_bus_create+0x1e4/0x368 of_platform_bus_create+0x230/0x368 of_platform_populate+0x70/0xbc of_platform_default_populate_init+0xa8/0xc0 do_one_initcall+0x1c8/0x3fc do_initcall_level+0xb4/0x10c do_basic_setup+0x30/0x48 kernel_init_freeable+0x124/0x1a4 kernel_init+0x14/0x104 ret_from_fork+0x10/0x18 -> #0 (opp_table_lock){+.+.}: __lock_acquire+0xee4/0x2450 lock_acquire+0x1cc/0x210 __mutex_lock_common+0xec/0xc0c mutex_lock_nested+0x40/0x50 _find_opp_table+0x34/0x74 dev_pm_opp_find_freq_exact+0x2c/0xdc a6xx_gmu_resume+0xc8/0xecc a6xx_pm_resume+0x148/0x200 adreno_resume+0x28/0x34 pm_generic_runtime_resume+0x34/0x48 __rpm_callback+0x70/0x10c rpm_callback+0x34/0x8c rpm_resume+0x414/0x550 __pm_runtime_resume+0x7c/0xa0 msm_gpu_submit+0x60/0x1c0 msm_ioctl_gem_submit+0xadc/0xb60 drm_ioctl_kernel+0x9c/0x118 drm_ioctl+0x27c/0x408 drm_compat_ioctl+0xcc/0xdc __se_compat_sys_ioctl+0x100/0x206c __arm64_compat_sys_ioctl+0x20/0x2c el0_svc_common+0xa8/0x178 el0_svc_compat_handler+0x2c/0x40 el0_svc_compat+0x8/0x10 other info that might help us debug this: Chain exists of: opp_table_lock --> &mm->mmap_sem --> reservation_ww_class_mutex Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(reservation_ww_class_mutex); lock(&mm->mmap_sem); lock(reservation_ww_class_mutex); lock(opp_table_lock); *** DEADLOCK *** 3 locks held by chrome/1865: #0: ffffff81edecc0d8 (&dev->struct_mutex){+.+.}, at: msm_ioctl_gem_submit+0x264/0xb60 #1: ffffff81d0000870 (reservation_ww_class_acquire){+.+.}, at: msm_ioctl_gem_submit+0x8e8/0xb60 #2: ffffff81f0fc71a8 (reservation_ww_class_mutex){+.+.}, at: submit_lock_objects+0x70/0x1ec stack backtrace: CPU: 0 PID: 1865 Comm: chrome Not tainted 5.4.72 #14 Hardware name: Google Lazor (rev1+) with LTE (DT) Call trace: dump_backtrace+0x0/0x158 show_stack+0x20/0x2c dump_stack+0xc8/0x160 print_circular_bug+0x2c4/0x2c8 check_noncircular+0x1a8/0x1b0 __lock_acquire+0xee4/0x2450 lock_acquire+0x1cc/0x210 __mutex_lock_common+0xec/0xc0c mutex_lock_nested+0x40/0x50 _find_opp_table+0x34/0x74 dev_pm_opp_find_freq_exact+0x2c/0xdc a6xx_gmu_resume+0xc8/0xecc a6xx_pm_resume+0x148/0x200 adreno_resume+0x28/0x34 pm_generic_runtime_resume+0x34/0x48 __rpm_callback+0x70/0x10c rpm_callback+0x34/0x8c rpm_resume+0x414/0x550 __pm_runtime_resume+0x7c/0xa0 msm_gpu_submit+0x60/0x1c0 msm_ioctl_gem_submit+0xadc/0xb60 drm_ioctl_kernel+0x9c/0x118 drm_ioctl+0x27c/0x408 drm_compat_ioctl+0xcc/0xdc __se_compat_sys_ioctl+0x100/0x206c __arm64_compat_sys_ioctl+0x20/0x2c el0_svc_common+0xa8/0x178 el0_svc_compat_handler+0x2c/0x40 el0_svc_compat+0x8/0x10 Reported-by: Rob Clark Signed-off-by: Viresh Kumar --- drivers/opp/core.c | 47 +++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 5 deletions(-) -- 2.25.0.rc1.19.g042ed3e048af diff --git a/drivers/opp/core.c b/drivers/opp/core.c index 84035ab8bb31..6f4a73a6391f 100644 --- a/drivers/opp/core.c +++ b/drivers/opp/core.c @@ -29,6 +29,8 @@ LIST_HEAD(opp_tables); /* Lock to allow exclusive modification to the device and opp lists */ DEFINE_MUTEX(opp_table_lock); +/* Flag indicating that opp_tables list is being updated at the moment */ +static bool opp_tables_busy; static struct opp_device *_find_opp_dev(const struct device *dev, struct opp_table *opp_table) @@ -1111,8 +1113,6 @@ static struct opp_table *_allocate_opp_table(struct device *dev, int index) INIT_LIST_HEAD(&opp_table->opp_list); kref_init(&opp_table->kref); - /* Secure the device table modification */ - list_add(&opp_table->node, &opp_tables); return opp_table; err: @@ -1125,27 +1125,64 @@ void _get_opp_table_kref(struct opp_table *opp_table) kref_get(&opp_table->kref); } +/* + * We need to make sure that the OPP table for a device doesn't get added twice, + * if this routine gets called in parallel with the same device pointer. + * + * The simplest way to enforce that is to perform everything (find existing + * table and if not found, create a new one) under the opp_table_lock, so only + * one creator gets access to the same. But that expands the critical section + * under the lock and may end up causing circular dependencies with frameworks + * like debugfs, interconnect or clock framework as they may be direct or + * indirect users of OPP core. + * + * And for that reason we have to go for a bit tricky implementation here, which + * uses the opp_tables_busy flag to indicate if another creator is in the middle + * of adding an OPP table and others should wait for it to finish. + */ static struct opp_table *_opp_get_opp_table(struct device *dev, int index) { struct opp_table *opp_table; - /* Hold our table modification lock here */ +again: mutex_lock(&opp_table_lock); opp_table = _find_opp_table_unlocked(dev); if (!IS_ERR(opp_table)) goto unlock; + /* + * The opp_tables list or an OPP table's dev_list is getting updated by + * another user, wait for it to finish. + */ + if (unlikely(opp_tables_busy)) { + mutex_unlock(&opp_table_lock); + cpu_relax(); + goto again; + } + + opp_tables_busy = true; opp_table = _managed_opp(dev, index); + + /* Drop the lock to reduce the size of critical section */ + mutex_unlock(&opp_table_lock); + if (opp_table) { if (!_add_opp_dev(dev, opp_table)) { dev_pm_opp_put_opp_table(opp_table); opp_table = ERR_PTR(-ENOMEM); } - goto unlock; + + mutex_lock(&opp_table_lock); + } else { + opp_table = _allocate_opp_table(dev, index); + + mutex_lock(&opp_table_lock); + if (!IS_ERR(opp_table)) + list_add(&opp_table->node, &opp_tables); } - opp_table = _allocate_opp_table(dev, index); + opp_tables_busy = false; unlock: mutex_unlock(&opp_table_lock); From patchwork Mon Nov 2 05:21:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 315243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6442C388F9 for ; Mon, 2 Nov 2020 05:21:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7A9B42224E for ; Mon, 2 Nov 2020 05:21:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="NNUAW7kU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727880AbgKBFVz (ORCPT ); Mon, 2 Nov 2020 00:21:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727882AbgKBFVz (ORCPT ); Mon, 2 Nov 2020 00:21:55 -0500 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 23636C0617A6 for ; Sun, 1 Nov 2020 21:21:55 -0800 (PST) Received: by mail-pl1-x643.google.com with SMTP id t22so6256783plr.9 for ; Sun, 01 Nov 2020 21:21:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OvXFnt9hhX2LofCdOg6VIZ0RCEbm1xCOvYlBDofSUBs=; b=NNUAW7kUvw/9uReQqDk5oj1JzyggrFkEfRrlSEHfvjEKrm3PkmAMmm13GLImGPD9de aU1Mi4n4CG66oGFMZpZuslEro6Ipl14DMpEQ+DaovL85ZjFMTEwiuSqcgybS+Pu/ANN9 Ml1vtAmzfjM1X+8GxE4e7qUxL53QZXm1UMIsXjFmBnpqijQc2Rsn9WVc7hL9B6XCM9av djgsxNhn6wfeqZagfPew5s8Q6vMzFKxGEKYna/p+UnUzU0Kzj7MjMZdFR/BUUdSOO84j JnPXfOomFQrjCbPVyYOCPhPDYPfdKX9L/iMJ5AEXMM3QcRXgRHqgX5R23T6wHEm+5+XE kgxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OvXFnt9hhX2LofCdOg6VIZ0RCEbm1xCOvYlBDofSUBs=; b=DT0j6G3A9ttIme+q+HEYgRmFeLANMD+wbh6HoOYrFb5zjlLv4247TE4o2bYmn6dgpB oFJdXIMRHsyEmV0XfvQmPx/HB+elxb7P3pkgRWSWxxsmWv7XTwRgKPLTpgvrr1Xb4dSP fDlWsgoo5zjC8tnhHFr5ItyNtvRcb/kYwIGplGrgE+cPIoXxXxPq0PrmcQ8/FHi0VltL a+USln757WbQia7XU37aH00+8T9ockiZV2EGYwMUh7SEzsmO0U5FCMLOmN2UmqHG7iVa M9rUga9Lct98+e0/IUUTc3Dhg9VMFm3LwgpjfpgDmtg7+RvnqEYWQmD4F8+pPbLyyb1M hYNw== X-Gm-Message-State: AOAM531jWAykcmJqPqQlv3blOkxlm5o/8d3Q0fLHxBO089uvSeofR3tz 9dLwPNOv9zXYf4YOY/nNogJ9PlS7TlVy4g== X-Google-Smtp-Source: ABdhPJwPzQ2GGc6ql3unqtgFq7Rej7ZDNthA1XBf30TKCmGaSI9EkbPOwmQVMrAL+CfaY9cFASpbRA== X-Received: by 2002:a17:902:6943:b029:d6:bd35:c84b with SMTP id k3-20020a1709026943b02900d6bd35c84bmr7574255plt.53.1604294514719; Sun, 01 Nov 2020 21:21:54 -0800 (PST) Received: from localhost ([122.181.54.133]) by smtp.gmail.com with ESMTPSA id s38sm11375758pgm.62.2020.11.01.21.21.53 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 01 Nov 2020 21:21:54 -0800 (PST) From: Viresh Kumar To: Viresh Kumar , Nishanth Menon , Stephen Boyd Cc: Viresh Kumar , linux-pm@vger.kernel.org, Vincent Guittot , Rafael Wysocki , robdclark@gmail.com, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] opp: Don't return opp_dev from _find_opp_dev() Date: Mon, 2 Nov 2020 10:51:40 +0530 Message-Id: X-Mailer: git-send-email 2.25.0.rc1.19.g042ed3e048af In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The caller of _find_opp_dev() only needs to know if the opp_dev is there in the list or not. Signed-off-by: Viresh Kumar --- drivers/opp/core.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/drivers/opp/core.c b/drivers/opp/core.c index 6f4a73a6391f..9915e8487f0b 100644 --- a/drivers/opp/core.c +++ b/drivers/opp/core.c @@ -32,31 +32,29 @@ DEFINE_MUTEX(opp_table_lock); /* Flag indicating that opp_tables list is being updated at the moment */ static bool opp_tables_busy; -static struct opp_device *_find_opp_dev(const struct device *dev, - struct opp_table *opp_table) +static bool _find_opp_dev(const struct device *dev, struct opp_table *opp_table) { struct opp_device *opp_dev; + bool found = false; + mutex_lock(&opp_table->lock); list_for_each_entry(opp_dev, &opp_table->dev_list, node) - if (opp_dev->dev == dev) - return opp_dev; + if (opp_dev->dev == dev) { + found = true; + break; + } - return NULL; + mutex_unlock(&opp_table->lock); + return found; } static struct opp_table *_find_opp_table_unlocked(struct device *dev) { struct opp_table *opp_table; - bool found; list_for_each_entry(opp_table, &opp_tables, node) { - mutex_lock(&opp_table->lock); - found = !!_find_opp_dev(dev, opp_table); - mutex_unlock(&opp_table->lock); - - if (found) { + if (_find_opp_dev(dev, opp_table)) { _get_opp_table_kref(opp_table); - return opp_table; } }