From patchwork Wed Aug 17 15:14:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil P Oommen X-Patchwork-Id: 598166 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 936DAC00140 for ; Thu, 18 Aug 2022 04:31:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238358AbiHREbQ (ORCPT ); Thu, 18 Aug 2022 00:31:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242902AbiHREbP (ORCPT ); Thu, 18 Aug 2022 00:31:15 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 197EC9C214; Wed, 17 Aug 2022 21:31:15 -0700 (PDT) Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27I0BUEg019114; Thu, 18 Aug 2022 04:31:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=Tl3JCkcvrqyRysK5Yzl7CCNb1u6Nwn4OBsT5VI7+LxE=; b=MTrC9kVpHJoWtSDPAI8fvqRFLhW7DjNmJMP0+LGh0SOY+Ryjv2JOQg0YgGegl7Y9pbYk O+2cqjRP6ypq6sSjeK4c21oMm7KkLK0nDPzeiRkrNK3/xy2RTd0RF0cMw89iq0IXYzth BXuPgOv+rZZwAbF97xTHW6tm5FIocSGZjRg3uNFTQZ50tW9S7AyirGJplVwi5NFcMxc4 VjirOw7iXct+5DIeofEmtRdVWu3FJqEu+DRuSamZNx62AeiA4wYe2mPAzJjURUnryPcV eSTa4adpQVH3nfylegYwU7C4CvwQw1wOcG+vkEoSA/rFBAHEbZalPl+G0011YOF+AnNZ 2g== Received: from nalasppmta02.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3j0tqccq6q-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 18 Aug 2022 04:31:08 +0000 Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 27I4HJ0w012608 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 18 Aug 2022 04:17:21 GMT Received: from hyd-lnxbld559.qualcomm.com (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Wed, 17 Aug 2022 08:14:50 -0700 From: Akhil P Oommen To: freedreno , , , Rob Clark , Bjorn Andersson , "Dmitry Baryshkov" CC: Matthias Kaehlcke , Jonathan Marek , Jordan Crouse , Douglas Anderson , Akhil P Oommen , "Abhinav Kumar" , Chia-I Wu , "Daniel Vetter" , David Airlie , Sean Paul , Subject: [PATCH v4 4/7] drm/msm: Fix cx collapse issue during recovery Date: Wed, 17 Aug 2022 20:44:17 +0530 Message-ID: <20220817204224.v4.4.I4ac27a0b34ea796ce0f938bb509e257516bc6f57@changeid> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1660749261-7602-1-git-send-email-quic_akhilpo@quicinc.com> References: <1660749261-7602-1-git-send-email-quic_akhilpo@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 33G014pJvOZqS8z6Kx-J5848F7bKY9Lg X-Proofpoint-ORIG-GUID: 33G014pJvOZqS8z6Kx-J5848F7bKY9Lg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-18_02,2022-08-16_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 malwarescore=0 bulkscore=0 priorityscore=1501 impostorscore=0 phishscore=0 lowpriorityscore=0 adultscore=0 mlxlogscore=999 suspectscore=0 clxscore=1015 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208180014 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org There are some hardware logic under CX domain. For a successful recovery, we should ensure cx headswitch collapses to ensure all the stale states are cleard out. This is especially true to for a6xx family where we can GMU co-processor. Currently, cx doesn't collapse due to a devlink between gpu and its smmu. So the *struct gpu device* needs to be runtime suspended to ensure that the iommu driver removes its vote on cx gdsc. Signed-off-by: Akhil P Oommen --- Changes in v4: - Keep active_submit lock across the suspend & resume (Rob) - Clear gpu->active_submits to silence a WARN() during runpm suspend (Rob) Changes in v3: - Simplied the pm refcount drop since we have just a single refcount now for all active submits drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 32 +++++++++++++++++++++++++++++--- drivers/gpu/drm/msm/msm_gpu.c | 4 +--- 2 files changed, 30 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 42ed9a3..0c8f19e 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -1193,7 +1193,7 @@ static void a6xx_recover(struct msm_gpu *gpu) { struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); - int i; + int i, active_submits; adreno_dump_info(gpu); @@ -1210,8 +1210,34 @@ static void a6xx_recover(struct msm_gpu *gpu) */ gmu_write(&a6xx_gpu->gmu, REG_A6XX_GMU_GMU_PWR_COL_KEEPALIVE, 0); - gpu->funcs->pm_suspend(gpu); - gpu->funcs->pm_resume(gpu); + pm_runtime_dont_use_autosuspend(&gpu->pdev->dev); + + /* active_submit won't change until we make a submission */ + mutex_lock(&gpu->active_lock); + active_submits = gpu->active_submits; + + /* + * Temporarily clear active_submits count to silence a WARN() in the + * runtime suspend cb + */ + gpu->active_submits = 0; + + /* Drop the rpm refcount from active submits */ + if (active_submits) + pm_runtime_put(&gpu->pdev->dev); + + /* And the final one from recover worker */ + pm_runtime_put_sync(&gpu->pdev->dev); + + pm_runtime_use_autosuspend(&gpu->pdev->dev); + + if (active_submits) + pm_runtime_get(&gpu->pdev->dev); + + pm_runtime_get_sync(&gpu->pdev->dev); + + gpu->active_submits = active_submits; + mutex_unlock(&gpu->active_lock); msm_gpu_hw_init(gpu); } diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index 1945efb..07e55a6 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -426,9 +426,7 @@ static void recover_worker(struct kthread_work *work) /* retire completed submits, plus the one that hung: */ retire_submits(gpu); - pm_runtime_get_sync(&gpu->pdev->dev); gpu->funcs->recover(gpu); - pm_runtime_put_sync(&gpu->pdev->dev); /* * Replay all remaining submits starting with highest priority @@ -445,7 +443,7 @@ static void recover_worker(struct kthread_work *work) } } - pm_runtime_put_sync(&gpu->pdev->dev); + pm_runtime_put(&gpu->pdev->dev); mutex_unlock(&gpu->lock);