From patchwork Thu Mar 30 13:17:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jagan Teki X-Patchwork-Id: 669233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8401CC761A6 for ; Thu, 30 Mar 2023 13:19:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229724AbjC3NTv (ORCPT ); Thu, 30 Mar 2023 09:19:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231857AbjC3NTs (ORCPT ); Thu, 30 Mar 2023 09:19:48 -0400 Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5259975E for ; Thu, 30 Mar 2023 06:19:42 -0700 (PDT) Received: by mail-pl1-x629.google.com with SMTP id z19so18053235plo.2 for ; Thu, 30 Mar 2023 06:19:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amarulasolutions.com; s=google; t=1680182382; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zkeMr7lTFTJZ/1wYN/aWGJRLdHK6qyRIY62qhqd0xcs=; b=SrPP5+QQMAsX92varBQzlyFJkk677Jd4b475emJTx/GUqc+NOZHrOftAqMaBeM9pUO pnaSSS7ovAyDu8ugmvKUPvsrhPeC0XC1rNRkNKfq/oa4Obb8GXqvnsoEkBwoMC3JV0o4 QMitcmZev25KI2mu0HNpbJvqqu5ndb2X/ug7E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680182382; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zkeMr7lTFTJZ/1wYN/aWGJRLdHK6qyRIY62qhqd0xcs=; b=QEti7JWNtr6FXlWY8vOvsuSKTiBxLSFvuZzJB6UMUHOXsHwnih2xzSEs9Gan2wLGZp yXYYOOw7oT1tX8IgJ/W7/7+WFqLZVAphZFkND2ClgPVekGnT+4Or+v78Q1vBlpr+hiT1 hB3i39gGUQjTR2vFAmpZW2eWX73FWAgZs66GmITj7kJ+at4sEOFRk1V13VlKj24imqay W+YodiHXFZcyje5H2Kp6HXYu/q9dNrlQDnUal9QFq+yGOdtgjtxE+W01NveStIe7mNQV apdcRG9w6mgZ543sLVs4tL/4DFEIIjFGJgtK9sEFEuL6x9eYA6Ca9LXMorNfYXUvQ77G V5AA== X-Gm-Message-State: AO0yUKV9dngxJw5HpLYdSBh1uO0YFZD7umcEpDxLQIj5q+dh8Qvg41Ic unOeJKbzJBFxqVN6kbOIna5yFA== X-Google-Smtp-Source: AK7set82Aid4902HLlcH1chRztL6gakvowwhKIziFtuTd977Fa9pW7Vq2k/7HUNnKFPcgcOH7Nej0g== X-Received: by 2002:a05:6a20:4c10:b0:d5:73ad:87c2 with SMTP id fm16-20020a056a204c1000b000d573ad87c2mr16883923pzb.56.1680182382070; Thu, 30 Mar 2023 06:19:42 -0700 (PDT) Received: from localhost.localdomain ([2405:201:c00a:a047:8bc3:c2fc:8dab:d912]) by smtp.gmail.com with ESMTPSA id e15-20020a62aa0f000000b006259e883ee9sm22473622pff.189.2023.03.30.06.19.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Mar 2023 06:19:41 -0700 (PDT) From: Jagan Teki To: Heiko Stuebner , Joerg Roedel , Will Deacon , Robin Murphy , Rob Herring , Krzysztof Kozlowski Cc: iommu@lists.linux.dev, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, devicetree@vger.kernel.org, linux-amarula , Jagan Teki , Simon Xue Subject: [PATCH 2/3] iommu/rockchip: Disable the device link during resume Date: Thu, 30 Mar 2023 18:47:45 +0530 Message-Id: <20230330131746.1475514-2-jagan@amarulasolutions.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230330131746.1475514-1-jagan@amarulasolutions.com> References: <20230330131746.1475514-1-jagan@amarulasolutions.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Rockchip iommu is trying to enable the associated device at runtime resume however some devices might enable the iommu during their pm runtime resume operation which indeed leads iommu to use the wrong domain and this leads to device iommu page fault. An example of this behavior has been observed in Rockchip RK3328, where iommu stalls request timeout dring VOP device enablement. Here is the dmesg log for the same: rockchip-drm display-subsystem: bound ff370000.vop (ops vop_component_ops) dwhdmi-rockchip ff3c0000.hdmi: supply avdd-0v9 not found, using dummy regulator rk_iommu ff373f00.iommu: Enable stall request timed out, status: 0x00004b dwhdmi-rockchip ff3c0000.hdmi: supply avdd-1v8 not found, using dummy regulator rk_iommu ff373f00.iommu: Disable paging request timed out, status: 0x00004b dwhdmi-rockchip ff3c0000.hdmi: Detected HDMI TX controller v2.11a with HDCP (inno_dw_hdmi_phy2) dwhdmi-rockchip ff3c0000.hdmi: registered DesignWare HDMI I2C bus driver rockchip-drm display-subsystem: bound ff3c0000.hdmi (ops dw_hdmi_rockchip_ops) [drm] Initialized rockchip 1.0.0 20140818 for display-subsystem on minor 0 This issue is reproduced if we enable the display in U-Boot however U-Boot is not even touched any iommu register as the U-Boot display uses the simple frame buffer like other Rockchip platforms RK3399, and RK3328 do. When VOP is trying to enable the iommu using runtime resume call pm_runtime_resume_and_get from @vop_enable then the iommu runtime resume call @rk_iommu_resume will try to attach the VOP in the wrong domain via @rk_iommu_enable will lead to the vop iommu page fault. vop_enable() pm_runtime_resume_and_get() rk_iommu_resume() rk_iommu_enable() ... vop iommu page fault ... rk_iommu ff373f00.iommu: Enable stall request timed out, status: 0x00004b rk_iommu ff373f00.iommu: Disable paging request timed out, status: 0x00004b So, this patch is trying to disable the device link for those devices that are enabled rockchip,disable-device-link-resume flag assumes here VOP device. This makes the device enablement for that iommu domain ignored during the rk_iommu_resume call as it assumes it handled iommu device attachment in the associated device itself. vop_enable() pm_runtime_resume_and_get() rk_iommu_resume() ... ignore the device link ... rockchip_drm_dma_attach_device() iommu_attach_device() Here is the downstream patch for similar issue, https://github.com/rockchip-linux/kernel/commit/85959f645ba38617233fbf44f442f8a88875d765 Co-developed-by: Simon Xue Signed-off-by: Simon Xue Signed-off-by: Jagan Teki --- drivers/iommu/rockchip-iommu.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/iommu/rockchip-iommu.c b/drivers/iommu/rockchip-iommu.c index f30db22ea5d7..bcff0dc21223 100644 --- a/drivers/iommu/rockchip-iommu.c +++ b/drivers/iommu/rockchip-iommu.c @@ -111,6 +111,7 @@ struct rk_iommu { struct clk_bulk_data *clocks; int num_clocks; bool reset_disabled; + bool dlr_disable; /* avoid access iommu when runtime ops called */ struct iommu_device iommu; struct list_head node; /* entry in rk_iommu_domain.iommus */ struct iommu_domain *domain; /* domain to which iommu is attached */ @@ -1250,6 +1251,8 @@ static int rk_iommu_probe(struct platform_device *pdev) iommu->reset_disabled = device_property_read_bool(dev, "rockchip,disable-mmu-reset"); + iommu->dlr_disable = device_property_read_bool(dev, + "rockchip,disable-device-link-resume"); iommu->num_clocks = ARRAY_SIZE(rk_iommu_clocks); iommu->clocks = devm_kcalloc(iommu->dev, iommu->num_clocks, @@ -1346,6 +1349,9 @@ static int __maybe_unused rk_iommu_suspend(struct device *dev) if (!iommu->domain) return 0; + if (iommu->dlr_disable) + return 0; + rk_iommu_disable(iommu); return 0; } @@ -1357,6 +1363,9 @@ static int __maybe_unused rk_iommu_resume(struct device *dev) if (!iommu->domain) return 0; + if (iommu->dlr_disable) + return 0; + return rk_iommu_enable(iommu); } From patchwork Thu Mar 30 13:17:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jagan Teki X-Patchwork-Id: 669232 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C05EBC77B60 for ; Thu, 30 Mar 2023 13:19:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229883AbjC3NTw (ORCPT ); Thu, 30 Mar 2023 09:19:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231967AbjC3NTu (ORCPT ); Thu, 30 Mar 2023 09:19:50 -0400 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9306E1721 for ; Thu, 30 Mar 2023 06:19:48 -0700 (PDT) Received: by mail-pj1-x102f.google.com with SMTP id p3-20020a17090a74c300b0023f69bc7a68so19534012pjl.4 for ; Thu, 30 Mar 2023 06:19:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amarulasolutions.com; s=google; t=1680182388; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sRHEgyUwDyzDOzARH/KbUIOIYBwk/M3j51CvYnIlMgs=; b=oRmW925Xm5WyLdVxo8qlbOXP2Zjh4XarIC07Gvgljqxr6pl+SS4Pwoy1OJI/Sto6hn 0G4Vhb2MCwW1GTKA238m/M+Wk0aNCnZOi6+W0YBHfzWnF2k3dn5TwmqNyVhhgiU5lvuj fA+2lzdvbcAIKVrmPaxAdVnbEveKjqhC5g188= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680182388; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sRHEgyUwDyzDOzARH/KbUIOIYBwk/M3j51CvYnIlMgs=; b=pg3DKbl4oTQQsbKpG42zGdTJxjnFKhsFytUs52d5mYwXpMYoUIeiHuQzrBC0LCeXoS 1Mlfn7O59F7U7YVD35v4m+jl0zNj+oig8SXQdBdp6J+Xx5Z5zJqTx0lyWSS0sAgM67sU 8HBD2pVi1tTSG58C/RH0Jc1Z0KMDzTs9daUvWvKbRjEh99xZRnuRfKA2IE7gIeHQzxZE 2cN6IHPF+qFv7hx7E0mDMit/DO4cEUIblSnZbxrJeCn67I5i1P89ApTn315LjdUSoh42 IiXfHrXSMO4y2bRc7Zb539dw1eszz1fAcQMqHiR7ME5IWxf6U4DeKWCssOCC4h9Ct7Vn 2KhA== X-Gm-Message-State: AO0yUKWk904bqhZSb4C5ZAXrkJ/rE6y1dt+YScl14ECOBQkmJ9YtvVrI xY0RKs3I2wG5L/vZV+R75HyxSQ== X-Google-Smtp-Source: AK7set/se186QNK4y8v2yF6M9tADOW5OdPNi17wkM/FKlIdKOpYWorRygvlG2Iu9dU18QwSIZE6joQ== X-Received: by 2002:a05:6a20:8b82:b0:d9:7fcf:1076 with SMTP id m2-20020a056a208b8200b000d97fcf1076mr22802629pzh.25.1680182388082; Thu, 30 Mar 2023 06:19:48 -0700 (PDT) Received: from localhost.localdomain ([2405:201:c00a:a047:8bc3:c2fc:8dab:d912]) by smtp.gmail.com with ESMTPSA id e15-20020a62aa0f000000b006259e883ee9sm22473622pff.189.2023.03.30.06.19.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Mar 2023 06:19:47 -0700 (PDT) From: Jagan Teki To: Heiko Stuebner , Joerg Roedel , Will Deacon , Robin Murphy , Rob Herring , Krzysztof Kozlowski Cc: iommu@lists.linux.dev, linux-rockchip@lists.infradead.org, linux-arm-kernel@lists.infradead.org, devicetree@vger.kernel.org, linux-amarula , Jagan Teki Subject: [PATCH 3/3] arm64: dts: rockchip: Disable device link for RK3328 VOP Date: Thu, 30 Mar 2023 18:47:46 +0530 Message-Id: <20230330131746.1475514-3-jagan@amarulasolutions.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230330131746.1475514-1-jagan@amarulasolutions.com> References: <20230330131746.1475514-1-jagan@amarulasolutions.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Rockchip iommu is trying to enable the VOP device at runtime resume however VOP device might enable the iommu during their pm runtime resume operation which indeed leads iommu to use the wrong domain and this leads to VOP iommu page fault. This issue is reproduced if we enable the display in U-Boot however U-Boot is not even touched any iommu register as the U-Boot display uses the simple frame buffer like other Rockchip platforms RK3399, and RK3328 do. rockchip-drm display-subsystem: bound ff370000.vop (ops vop_component_ops) dwhdmi-rockchip ff3c0000.hdmi: supply avdd-0v9 not found, using dummy regulator rk_iommu ff373f00.iommu: Enable stall request timed out, status: 0x00004b dwhdmi-rockchip ff3c0000.hdmi: supply avdd-1v8 not found, using dummy regulator rk_iommu ff373f00.iommu: Disable paging request timed out, status: 0x00004b dwhdmi-rockchip ff3c0000.hdmi: Detected HDMI TX controller v2.11a with HDCP (inno_dw_hdmi_phy2) dwhdmi-rockchip ff3c0000.hdmi: registered DesignWare HDMI I2C bus driver rockchip-drm display-subsystem: bound ff3c0000.hdmi (ops dw_hdmi_rockchip_ops) [drm] Initialized rockchip 1.0.0 20140818 for display-subsystem on minor 0 So, prevent this by enabling rockchip,disable-device-link-resume in VOP node so that VOP enablement for that iommu domain ignored during the rk_iommu_resume call as it assumes it handled iommu device attachment in the VOP itself. Signed-off-by: Jagan Teki --- arch/arm64/boot/dts/rockchip/rk3328.dtsi | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/boot/dts/rockchip/rk3328.dtsi b/arch/arm64/boot/dts/rockchip/rk3328.dtsi index 6d7a7bf72ac7..7ca83bc844c1 100644 --- a/arch/arm64/boot/dts/rockchip/rk3328.dtsi +++ b/arch/arm64/boot/dts/rockchip/rk3328.dtsi @@ -718,6 +718,7 @@ vop_mmu: iommu@ff373f00 { clocks = <&cru ACLK_VOP>, <&cru HCLK_VOP>; clock-names = "aclk", "iface"; #iommu-cells = <0>; + rockchip,disable-device-link-resume; status = "disabled"; };