From patchwork Mon Aug 16 10:59:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 497741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02616C4338F for ; Mon, 16 Aug 2021 11:00:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DF85B61283 for ; Mon, 16 Aug 2021 11:00:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235948AbhHPLA5 (ORCPT ); Mon, 16 Aug 2021 07:00:57 -0400 Received: from mailgw02.mediatek.com ([210.61.82.184]:41376 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S235941AbhHPLA4 (ORCPT ); Mon, 16 Aug 2021 07:00:56 -0400 X-UUID: 78217859e08049f8b6a909ef524d1f1a-20210816 X-UUID: 78217859e08049f8b6a909ef524d1f1a-20210816 Received: from mtkcas10.mediatek.inc [(172.21.101.39)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1652081063; Mon, 16 Aug 2021 19:00:19 +0800 Received: from MTKCAS06.mediatek.inc (172.21.101.30) by mtkexhb01.mediatek.inc (172.21.101.102) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Aug 2021 19:00:18 +0800 Received: from localhost.localdomain (10.17.3.153) by MTKCAS06.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 16 Aug 2021 19:00:17 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores Date: Mon, 16 Aug 2021 18:59:26 +0800 Message-ID: <20210816105934.28265-2-irui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20210816105934.28265-1-irui.wang@mediatek.com> References: <20210816105934.28265-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Enable MT8195 two H.264 venc cores, updates vcodec binding document. Signed-off-by: Irui Wang Acked-by: Rob Herring --- Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt index de961699ba0a..eb2e24c32426 100644 --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt @@ -11,6 +11,8 @@ Required properties: "mediatek,mt8173-vcodec-dec" for MT8173 decoder. "mediatek,mt8192-vcodec-enc" for MT8192 encoder. "mediatek,mt8195-vcodec-enc" for MT8195 encoder. + "mediatek,mtk-venc-core0" for MT8195 avc core0 device. + "mediatek,mtk-venc-core1" for MT8195 avc core1 device. - reg : Physical base address of the video codec registers and length of memory mapped region. - interrupts : interrupt number to the cpu. From patchwork Mon Aug 16 10:59:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 497740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3EAFC432BE for ; Mon, 16 Aug 2021 11:00:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D1F996320E for ; Mon, 16 Aug 2021 11:00:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236062AbhHPLBH (ORCPT ); Mon, 16 Aug 2021 07:01:07 -0400 Received: from mailgw01.mediatek.com ([60.244.123.138]:53198 "EHLO mailgw01.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S236032AbhHPLBE (ORCPT ); Mon, 16 Aug 2021 07:01:04 -0400 X-UUID: 1967078d2e28441aad6b1e1da8d0cb6f-20210816 X-UUID: 1967078d2e28441aad6b1e1da8d0cb6f-20210816 Received: from mtkcas10.mediatek.inc [(172.21.101.39)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1880302440; Mon, 16 Aug 2021 19:00:29 +0800 Received: from mtkmbs10n2.mediatek.inc (172.21.101.183) by mtkmbs06n1.mediatek.inc (172.21.101.129) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Aug 2021 19:00:27 +0800 Received: from MTKCAS06.mediatek.inc (172.21.101.30) by mtkmbs10n2.mediatek.inc (172.21.101.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.3; Mon, 16 Aug 2021 19:00:27 +0800 Received: from localhost.localdomain (10.17.3.153) by MTKCAS06.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 16 Aug 2021 19:00:26 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , "Tiffany Lin" , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH 3/9] media: mtk-vcodec: Rewrite venc power manage interface Date: Mon, 16 Aug 2021 18:59:28 +0800 Message-ID: <20210816105934.28265-4-irui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20210816105934.28265-1-irui.wang@mediatek.com> References: <20210816105934.28265-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org The args "struct mtk_vcodec_dev *" doesn't appropriate for init/release_enc_pm functions because of component devices. For master device, it has no "pm/clk" properties in dtsi, component devices will init their own "pm/clk" instead. So rewrite the pm interface with args "platform_device *" and "mtk_vcodec_pm*". Signed-off-by: Irui Wang --- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 6 ++-- .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c | 13 ++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 31 +++++++------------ .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 5 +-- 4 files changed, 30 insertions(+), 25 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c index f7538d4b5cbb..9c2224b199d0 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c @@ -341,7 +341,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev) dev->venc_pdata = of_device_get_match_data(&pdev->dev); if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) { - ret = mtk_vcodec_init_enc_pm(dev); + ret = mtk_vcodec_init_enc_pm(dev->plat_dev, &dev->pm); if (ret < 0) { dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!"); goto err_enc_pm; @@ -471,7 +471,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev) v4l2_device_unregister(&dev->v4l2_dev); err_res: if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) - mtk_vcodec_release_enc_pm(dev); + mtk_vcodec_release_enc_pm(&dev->pm); err_enc_pm: mtk_vcodec_fw_release(dev->fw_handler); return ret; @@ -568,7 +568,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev) v4l2_device_unregister(&dev->v4l2_dev); if (dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) - mtk_vcodec_release_enc_pm(dev); + mtk_vcodec_release_enc_pm(&dev->pm); mtk_vcodec_fw_release(dev->fw_handler); diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c index 4e6a8a81ff67..2a9112b0561d 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c @@ -10,6 +10,7 @@ #include #include "mtk_vcodec_enc_hw.h" +#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_enc.h" static void clean_irq_status(unsigned int irq_status, void __iomem *addr) @@ -105,6 +106,14 @@ static int mtk_venc_comp_probe(struct platform_device *pdev) comp_dev->plat_dev = pdev; + ret = mtk_vcodec_init_enc_pm(pdev, &comp_dev->pm); + if (ret < 0) { + dev_err(&pdev->dev, "Failed to get venc component clock source!"); + return ret; + } + + pm_runtime_enable(&pdev->dev); + comp_dev->core_id = (enum mtk_venc_hw_id)of_device_get_match_data(&pdev->dev); @@ -146,11 +155,15 @@ static int mtk_venc_comp_probe(struct platform_device *pdev) return 0; err: + mtk_vcodec_release_enc_pm(&comp_dev->pm); return ret; } static int mtk_venc_comp_remove(struct platform_device *pdev) { + struct mtk_venc_comp_dev *comp_dev = platform_get_drvdata(pdev); + + mtk_vcodec_release_enc_pm(&comp_dev->pm); component_del(&pdev->dev, &mtk_venc_component_ops); return 0; } diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c index 1b2e4930ed27..5edeb93fca84 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c @@ -13,38 +13,29 @@ #include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_util.h" -int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev) +int mtk_vcodec_init_enc_pm(struct platform_device *pdev, + struct mtk_vcodec_pm *pm) { struct device_node *node; - struct platform_device *pdev; - struct mtk_vcodec_pm *pm; + struct platform_device *larb_pdev; struct mtk_vcodec_clk *enc_clk; struct mtk_vcodec_clk_info *clk_info; int ret = 0, i = 0; - struct device *dev; - pdev = mtkdev->plat_dev; - pm = &mtkdev->pm; - memset(pm, 0, sizeof(struct mtk_vcodec_pm)); - pm->mtkdev = mtkdev; - pm->dev = &pdev->dev; - dev = &pdev->dev; - enc_clk = &pm->venc_clk; - - node = of_parse_phandle(dev->of_node, "mediatek,larb", 0); + node = of_parse_phandle(pdev->dev.of_node, "mediatek,larb", 0); if (!node) { mtk_v4l2_err("no mediatek,larb found"); return -ENODEV; } - pdev = of_find_device_by_node(node); + larb_pdev = of_find_device_by_node(node); of_node_put(node); - if (!pdev) { + if (!larb_pdev) { mtk_v4l2_err("no mediatek,larb device found"); return -ENODEV; } - pm->larbvenc = &pdev->dev; - pdev = mtkdev->plat_dev; + pm->larbvenc = &larb_pdev->dev; pm->dev = &pdev->dev; + enc_clk = &pm->venc_clk; enc_clk->clk_num = of_property_count_strings(pdev->dev.of_node, "clock-names"); @@ -87,10 +78,10 @@ int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev) return ret; } -void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *mtkdev) +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm) { - pm_runtime_disable(mtkdev->pm.dev); - put_device(mtkdev->pm.larbvenc); + pm_runtime_disable(pm->dev); + put_device(pm->larbvenc); } diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h index b7ecdfd74823..f4a4aa441185 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h @@ -9,8 +9,9 @@ #include "mtk_vcodec_drv.h" -int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *dev); -void mtk_vcodec_release_enc_pm(struct mtk_vcodec_dev *dev); +int mtk_vcodec_init_enc_pm(struct platform_device *pdev, + struct mtk_vcodec_pm *pm); +void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm); void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm); void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm); From patchwork Mon Aug 16 10:59:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 497739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CB88C4338F for ; Mon, 16 Aug 2021 11:00:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5045E61BE1 for ; Mon, 16 Aug 2021 11:00:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236237AbhHPLBM (ORCPT ); Mon, 16 Aug 2021 07:01:12 -0400 Received: from mailgw01.mediatek.com ([60.244.123.138]:53354 "EHLO mailgw01.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S236222AbhHPLBK (ORCPT ); Mon, 16 Aug 2021 07:01:10 -0400 X-UUID: f5233eb66ed64c1ebc9c4c2b6a271ffd-20210816 X-UUID: f5233eb66ed64c1ebc9c4c2b6a271ffd-20210816 Received: from mtkcas10.mediatek.inc [(172.21.101.39)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1983101265; Mon, 16 Aug 2021 19:00:35 +0800 Received: from mtkcas07.mediatek.inc (172.21.101.84) by mtkexhb01.mediatek.inc (172.21.101.102) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Aug 2021 19:00:34 +0800 Received: from MTKCAS06.mediatek.inc (172.21.101.30) by mtkcas07.mediatek.inc (172.21.101.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Aug 2021 19:00:33 +0800 Received: from localhost.localdomain (10.17.3.153) by MTKCAS06.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 16 Aug 2021 19:00:32 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , "Tiffany Lin" , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH 5/9] media: mtk-vcodec: Rewrite venc clock interface Date: Mon, 16 Aug 2021 18:59:30 +0800 Message-ID: <20210816105934.28265-6-irui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20210816105934.28265-1-irui.wang@mediatek.com> References: <20210816105934.28265-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Frame-racing mode need enable the specific core's clk, add an another param for clock_on/clock_off interface. Signed-off-by: Irui Wang --- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 104 ++++++++++++++---- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 4 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 10 +- 3 files changed, 86 insertions(+), 32 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c index d9df0e3701f1..bfb9e21bfd26 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c @@ -86,25 +86,50 @@ void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm) } -void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id) { - struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; - int ret, i = 0; + struct mtk_venc_comp_dev *venc; + struct mtk_vcodec_pm *enc_pm; + struct mtk_vcodec_clk *enc_clk; + struct clk *clk; + int ret, i; - for (i = 0; i < enc_clk->clk_num; i++) { - ret = clk_prepare_enable(enc_clk->clk_info[i].vcodec_clk); - if (ret) { - mtk_v4l2_err("venc clk_prepare_enable %d %s fail %d", i, - enc_clk->clk_info[i].clk_name, ret); - goto clkerr; + if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) { + venc = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[core_id]; + enc_pm = &venc->pm; + enc_clk = &enc_pm->venc_clk; + + for (i = 0; i < enc_clk->clk_num; i++) { + clk = enc_clk->clk_info[i].vcodec_clk; + ret = clk_enable(clk); + if (ret) { + mtk_v4l2_err("clk_enable %d %s fail %d", i, + enc_clk->clk_info[i].clk_name, + ret); + goto comp_clk_err; + } } - } + } else { + enc_pm = &dev->pm; + enc_clk = &enc_pm->venc_clk; - ret = mtk_smi_larb_get(pm->larbvenc); - if (ret) { - mtk_v4l2_err("mtk_smi_larb_get larb3 fail %d", ret); - goto clkerr; + for (i = 0; i < enc_clk->clk_num; i++) { + clk = enc_clk->clk_info[i].vcodec_clk; + ret = clk_prepare_enable(clk); + if (ret) { + mtk_v4l2_err("clk_prepare_enable %d %s fail %d", + i, enc_clk->clk_info[i].clk_name, + ret); + goto clkerr; + } + } } + + return; + +comp_clk_err: + for (i -= 1; i >= 0; i--) + clk_disable(enc_clk->clk_info[i].vcodec_clk); return; clkerr: @@ -112,20 +137,35 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); } -void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id) { - struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; - int i = 0; + struct mtk_venc_comp_dev *venc; + struct mtk_vcodec_pm *enc_pm; + struct mtk_vcodec_clk *enc_clk; + int i; - mtk_smi_larb_put(pm->larbvenc); - for (i = enc_clk->clk_num - 1; i >= 0; i--) - clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); + if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) { + venc = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[core_id]; + enc_pm = &venc->pm; + enc_clk = &enc_pm->venc_clk; + + for (i = enc_clk->clk_num - 1; i >= 0; i--) + clk_disable(enc_clk->clk_info[i].vcodec_clk); + } else { + enc_pm = &dev->pm; + enc_clk = &enc_pm->venc_clk; + + for (i = enc_clk->clk_num - 1; i >= 0; i--) + clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); + } } int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev) { int i, ret; struct mtk_venc_comp_dev *venc_comp; + struct mtk_vcodec_clk *enc_clk; + int j = 0; /* * frame_racing mode needs power on all available component devices. @@ -140,23 +180,38 @@ int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev) mtk_v4l2_err("power on core[%d] fail %d", i, ret); goto pw_err; } + + enc_clk = &venc_comp->pm.venc_clk; + for (j = 0; j < enc_clk->clk_num; j++) { + ret = clk_prepare(enc_clk->clk_info[j].vcodec_clk); + if (ret) { + mtk_v4l2_err("prepare clk [%s] fail %d", + enc_clk->clk_info[j].clk_name, + ret); + goto pw_err; + } + } } return 0; pw_err: for (i -= 1; i >= 0; i--) { venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i]; + enc_clk = &venc_comp->pm.venc_clk; + + for (j -= 1; j >= 0; j--) + clk_unprepare(enc_clk->clk_info[j].vcodec_clk); mtk_smi_larb_put(venc_comp->pm.larbvenc); } - return ret; } int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev) { - int i; + int i, j; struct mtk_venc_comp_dev *venc_comp; + struct mtk_vcodec_clk *enc_clk; /*power off all available component device*/ for (i = 0; i < MTK_VENC_HW_MAX; i++) { @@ -164,6 +219,11 @@ int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev) if (!venc_comp) return 0; + enc_clk = &venc_comp->pm.venc_clk; + + for (j = enc_clk->clk_num - 1; j >= 0; j--) + clk_unprepare(enc_clk->clk_info[j].vcodec_clk); + mtk_smi_larb_put(venc_comp->pm.larbvenc); } diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h index 2e76b858fed1..f3a819cc0e56 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h @@ -13,8 +13,8 @@ int mtk_vcodec_init_enc_pm(struct platform_device *pdev, struct mtk_vcodec_pm *pm); void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm); -void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm); -void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm); +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id); +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id); int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev); int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev); diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c index ce0bce811615..6cbdb7e30bb3 100644 --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c @@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) } mtk_venc_lock(ctx); - mtk_vcodec_enc_clock_on(&ctx->dev->pm); ret = ctx->enc_if->init(ctx); - mtk_vcodec_enc_clock_off(&ctx->dev->pm); mtk_venc_unlock(ctx); return ret; @@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx, int ret = 0; mtk_venc_lock(ctx); - mtk_vcodec_enc_clock_on(&ctx->dev->pm); ret = ctx->enc_if->set_param(ctx->drv_handle, type, in); - mtk_vcodec_enc_clock_off(&ctx->dev->pm); mtk_venc_unlock(ctx); return ret; @@ -68,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx, ctx->dev->curr_ctx = ctx; spin_unlock_irqrestore(&ctx->dev->irqlock, flags); - mtk_vcodec_enc_clock_on(&ctx->dev->pm); + mtk_vcodec_enc_clock_on(ctx->dev, 0); ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result); - mtk_vcodec_enc_clock_off(&ctx->dev->pm); + mtk_vcodec_enc_clock_off(ctx->dev, 0); spin_lock_irqsave(&ctx->dev->irqlock, flags); ctx->dev->curr_ctx = NULL; @@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx) return 0; mtk_venc_lock(ctx); - mtk_vcodec_enc_clock_on(&ctx->dev->pm); ret = ctx->enc_if->deinit(ctx->drv_handle); - mtk_vcodec_enc_clock_off(&ctx->dev->pm); mtk_venc_unlock(ctx); ctx->drv_handle = NULL; From patchwork Mon Aug 16 10:59:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 497738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9DBAC4320A for ; Mon, 16 Aug 2021 11:00:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A2BF561C12 for ; Mon, 16 Aug 2021 11:00:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235973AbhHPLB0 (ORCPT ); Mon, 16 Aug 2021 07:01:26 -0400 Received: from mailgw02.mediatek.com ([210.61.82.184]:41986 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S236315AbhHPLBT (ORCPT ); Mon, 16 Aug 2021 07:01:19 -0400 X-UUID: 9dce8888aaff44768683aa06e80821c3-20210816 X-UUID: 9dce8888aaff44768683aa06e80821c3-20210816 Received: from mtkmbs10n2.mediatek.inc [(172.21.101.183)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 478976115; Mon, 16 Aug 2021 19:00:44 +0800 Received: from mtkmbs10n1.mediatek.inc (172.21.101.34) by mtkmbs05n1.mediatek.inc (172.21.101.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Aug 2021 19:00:43 +0800 Received: from MTKCAS06.mediatek.inc (172.21.101.30) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.15; Mon, 16 Aug 2021 19:00:42 +0800 Received: from localhost.localdomain (10.17.3.153) by MTKCAS06.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 16 Aug 2021 19:00:41 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , "Tiffany Lin" , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH 8/9] media: mtk-vcodec: Return encode result to client Date: Mon, 16 Aug 2021 18:59:33 +0800 Message-ID: <20210816105934.28265-9-irui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20210816105934.28265-1-irui.wang@mediatek.com> References: <20210816105934.28265-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org For frame-racing mode encoding, there will no sync wait IRQ done function, so the encode result can't return to client in device_run. After device_run called to trigger encoding process, finish it, so another scheduled job can be ran. When encoding done, return the encode result to client in IRQ handler. Signed-off-by: Irui Wang --- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 6 ++ .../platform/mtk-vcodec/mtk_vcodec_enc.c | 61 ++++++++++++++++++- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 4 +- .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c | 35 +++++++++++ .../platform/mtk-vcodec/mtk_vcodec_util.h | 1 + .../platform/mtk-vcodec/venc/venc_common_if.c | 9 +++ .../media/platform/mtk-vcodec/venc_drv_if.h | 2 + 7 files changed, 115 insertions(+), 3 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h index 7a9c2dd52b91..c43321362896 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h @@ -274,6 +274,9 @@ struct vdec_pic_info { * @lock: protect variables accessed by V4L2 threads and worker thread such as * mtk_video_dec_buf. * @enc_idx: used to record encoded frame count + * @pfrm_buf: used to store current ctx's frame buffer information + * @pbs_buf: used to store current ctx's bitstream buffer information + * @hdr_size: used to store prepend header size */ struct mtk_vcodec_ctx { enum mtk_instance_type type; @@ -316,6 +319,9 @@ struct mtk_vcodec_ctx { int hw_id; int enc_idx; + struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_HW_MAX]; + struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_HW_MAX]; + unsigned int hdr_size; }; enum mtk_chip { diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c index 6b37bd092fdf..de8e012f1b0e 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c @@ -867,6 +867,8 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q) mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->id, q->type); + mtk_venc_lock_all(ctx); + if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) { dst_buf->vb2_buf.planes[0].bytesused = 0; @@ -1014,10 +1016,13 @@ static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx) ctx->id, vb2_v4l2->vb2_buf.index, mtk_buf->enc_params.force_intra); - if (mtk_buf->enc_params.force_intra) + if (mtk_buf->enc_params.force_intra) { ret |= venc_if_set_param(ctx, VENC_SET_PARAM_FORCE_INTRA, NULL); + + ctx->enc_idx = 0; + } } mtk_buf->param_change = MTK_ENCODE_PARAM_NONE; @@ -1068,9 +1073,12 @@ static void mtk_venc_worker(struct work_struct *work) frm_buf.fb_addr[i].size = (size_t)src_buf->vb2_buf.planes[i].length; } + frm_buf.src_addr = src_buf; + bs_buf.va = vb2_plane_vaddr(&dst_buf->vb2_buf, 0); bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0); bs_buf.size = (size_t)dst_buf->vb2_buf.planes[0].length; + bs_buf.buf = dst_buf; mtk_v4l2_debug(2, "Framebuf PA=%llx Size=0x%zx;PA=0x%llx Size=0x%zx;PA=0x%llx Size=%zu", @@ -1119,6 +1127,8 @@ static void m2mops_venc_device_run(void *priv) { struct mtk_vcodec_ctx *ctx = priv; + mtk_venc_param_change(ctx); + if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) && (ctx->state != MTK_STATE_HEADER)) { /* encode h264 sps/pps header */ @@ -1127,7 +1137,6 @@ static void m2mops_venc_device_run(void *priv) return; } - mtk_venc_param_change(ctx); queue_work(ctx->dev->encode_workqueue, &ctx->encode_work); } @@ -1310,6 +1319,33 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, return vb2_queue_init(dst_vq); } +void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id, + unsigned int bs_size, bool time_out, bool key_frame) +{ + struct vb2_v4l2_buffer *src_vb2_v4l2 = NULL; + struct vb2_v4l2_buffer *dst_vb2_v4l2 = NULL; + + /* + * the frm_buf(src_buf) and bs_buf(dst_buf) can be obtained from ctx, + * then put them to done list, user can get them by dqbuf call + */ + src_vb2_v4l2 = ctx->pfrm_buf[core_id]; + dst_vb2_v4l2 = ctx->pbs_buf[core_id]; + + if (src_vb2_v4l2 && dst_vb2_v4l2) { + dst_vb2_v4l2->vb2_buf.timestamp = + src_vb2_v4l2->vb2_buf.timestamp; + dst_vb2_v4l2->timecode = src_vb2_v4l2->timecode; + + if (key_frame) + dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME; + + v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE); + dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size; + v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE); + } +} + int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id) { struct mtk_vcodec_dev *dev = ctx->dev; @@ -1335,3 +1371,24 @@ void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx) ctx->state = MTK_STATE_FREE; } + +void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx) +{ + unsigned int i; + struct mtk_vcodec_dev *dev = ctx->dev; + + /* + * For frame-racing mode encoding, there are may be bufs being encoded + * when get the empty flush buffer or stop streaming, for example, the + * buffer with LAST flag will return to client before the encoding + * buffers, which will cause frame lost. + * The encoder device mutex will be locked during encoding process, + * when encode done, the mutex unlocked. So if all encoder device mutex + * can be locked, which means there are no bufs being encoded at this + * time, then the buffer with LAST flag can return to client properly. + */ + for (i = 0; i < MTK_VENC_HW_MAX; i++) { + mutex_lock(&dev->enc_mutex[i]); + mutex_unlock(&dev->enc_mutex[i]); + } +} diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h index 8d486f532025..1d603ccb2f3d 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h @@ -48,5 +48,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx); int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx); void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx); - +void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id, + unsigned int bs_size, bool time_out, bool key_frame); +void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx); #endif /* _MTK_VCODEC_ENC_H_ */ diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c index 2a9112b0561d..687b255c58e8 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c @@ -12,6 +12,17 @@ #include "mtk_vcodec_enc_hw.h" #include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_enc.h" +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098 +#define VENC_PIC_FRM_TYPE 0x0010 +#define VENC_PIC_KEY_FRM 0x2 + +/* Wake up core context wait_queue */ +static void mtk_venc_comp_wake_up_ctx(struct mtk_vcodec_ctx *ctx, + unsigned int hw_id) +{ + ctx->int_cond = 1; + wake_up_interruptible(&ctx->queue); +} static void clean_irq_status(unsigned int irq_status, void __iomem *addr) { @@ -40,6 +51,9 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv) struct mtk_vcodec_ctx *ctx; unsigned long flags; void __iomem *addr; + unsigned int bs_size; + unsigned int frm_type; + bool is_key_frame = 0; spin_lock_irqsave(&dev->master_dev->irqlock, flags); ctx = dev->curr_ctx; @@ -49,7 +63,28 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv) addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET; ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET); + bs_size = readl(dev->reg_base + VENC_PIC_BITSTREAM_BYTE_CNT); + frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE); clean_irq_status(ctx->irq_status, addr); + if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) { + if (ctx->hdr_size != 0) { + bs_size += ctx->hdr_size; + ctx->hdr_size = 0; + } + if (frm_type & VENC_PIC_KEY_FRM) + is_key_frame = 1; + + mtk_venc_buf_done(ctx, dev->core_id, bs_size, 0, is_key_frame); + mtk_vcodec_enc_clock_off(dev->master_dev, dev->core_id); + + spin_lock_irqsave(&dev->master_dev->irqlock, flags); + dev->curr_ctx = NULL; + spin_unlock_irqrestore(&dev->master_dev->irqlock, flags); + + mtk_venc_unlock(ctx, dev->core_id); + } else { + mtk_venc_comp_wake_up_ctx(ctx, dev->core_id); + } return IRQ_HANDLED; } diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h index 0027747c0a27..7321b7537f20 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h @@ -15,6 +15,7 @@ struct mtk_vcodec_mem { size_t size; void *va; dma_addr_t dma_addr; + void *buf; }; struct mtk_vcodec_fb { diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c index 1580bd1beaac..75b4e90232ae 100644 --- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c +++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c @@ -481,6 +481,13 @@ static int venc_encode(void *handle, unsigned int bs_size_hdr; unsigned int bs_size_frm; + /* + * the frm_buf and bs_buf need to be recorded into ctx, + * when encoding done, the target buffer can be obtained + */ + ctx->pfrm_buf[ctx->hw_id] = frm_buf->src_addr; + ctx->pbs_buf[ctx->hw_id] = bs_buf->buf; + if (!inst->prepend_hdr) { ret = venc_encode_frame(inst, frm_buf, bs_buf, &result->bs_size, ctx->hw_id); @@ -528,6 +535,8 @@ static int venc_encode(void *handle, result->bs_size = hdr_sz + filler_sz + bs_size_frm; + ctx->hdr_size = hdr_sz + filler_sz; + mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d", hdr_sz, filler_sz, bs_size_frm, result->bs_size); diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h index 7bd1f437d8e3..50824e718c91 100644 --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h @@ -108,9 +108,11 @@ struct venc_frame_info { /* * struct venc_frm_buf - frame buffer information used in venc_if_encode() * @fb_addr: plane frame buffer addresses + * @src_addr: vb2_v4l2_buffer src buffer address */ struct venc_frm_buf { struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES]; + void *src_addr; }; /* From patchwork Mon Aug 16 10:59:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 497737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30627C4320A for ; Mon, 16 Aug 2021 11:01:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 17F2A60EFF for ; Mon, 16 Aug 2021 11:01:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236534AbhHPLBf (ORCPT ); Mon, 16 Aug 2021 07:01:35 -0400 Received: from mailgw02.mediatek.com ([210.61.82.184]:42090 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S236376AbhHPLBW (ORCPT ); Mon, 16 Aug 2021 07:01:22 -0400 X-UUID: 0f203480291e4a2c8795eaf1ead89c8f-20210816 X-UUID: 0f203480291e4a2c8795eaf1ead89c8f-20210816 Received: from mtkcas10.mediatek.inc [(172.21.101.39)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1912319955; Mon, 16 Aug 2021 19:00:47 +0800 Received: from MTKCAS06.mediatek.inc (172.21.101.30) by mtkmbs07n2.mediatek.inc (172.21.101.141) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Aug 2021 19:00:45 +0800 Received: from localhost.localdomain (10.17.3.153) by MTKCAS06.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 16 Aug 2021 19:00:44 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH 9/9] media: mtk-vcodec: Add delayed worker for encode timeout Date: Mon, 16 Aug 2021 18:59:34 +0800 Message-ID: <20210816105934.28265-10-irui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20210816105934.28265-1-irui.wang@mediatek.com> References: <20210816105934.28265-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org When encoding timeout, a delayed timeout worker is needed because of asynchronous encoding process, restore encoding process in timeout worker. Signed-off-by: Irui Wang --- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 12 ++++-- .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c | 26 +++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h | 2 + .../platform/mtk-vcodec/venc/venc_common_if.c | 43 +++++++++++++++++-- 4 files changed, 77 insertions(+), 6 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c index de8e012f1b0e..bc635436aa90 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c @@ -1340,9 +1340,15 @@ void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id, if (key_frame) dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME; - v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE); - dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size; - v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE); + if (time_out) { + v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_ERROR); + dst_vb2_v4l2->vb2_buf.planes[0].bytesused = 0; + v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_ERROR); + } else { + v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE); + dst_vb2_v4l2->vb2_buf.planes[0].bytesused = bs_size; + v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE); + } } } diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c index 687b255c58e8..f6a5dfa6929f 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c @@ -16,6 +16,28 @@ #define VENC_PIC_FRM_TYPE 0x0010 #define VENC_PIC_KEY_FRM 0x2 +static void mtk_venc_timeout_worker(struct work_struct *work) +{ + struct mtk_venc_comp_dev *dev = container_of(work, + struct mtk_venc_comp_dev, + enc_timeout_worker.work); + + struct mtk_vcodec_ctx *ctx; + unsigned long flags; + + spin_lock_irqsave(&dev->master_dev->irqlock, flags); + ctx = dev->curr_ctx; + + dev_err(&dev->plat_dev->dev, "core: %d timeout\n", dev->core_id); + + mtk_venc_buf_done(ctx, dev->core_id, 0, 1, 0); + + mtk_vcodec_enc_clock_off(dev->master_dev, dev->core_id); + dev->curr_ctx = NULL; + spin_unlock_irqrestore(&dev->master_dev->irqlock, flags); + + mtk_venc_unlock(ctx, dev->core_id); +} /* Wake up core context wait_queue */ static void mtk_venc_comp_wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int hw_id) @@ -67,6 +89,7 @@ static irqreturn_t mtk_enc_comp_irq_handler(int irq, void *priv) frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE); clean_irq_status(ctx->irq_status, addr); if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) { + cancel_delayed_work(&dev->enc_timeout_worker); if (ctx->hdr_size != 0) { bs_size += ctx->hdr_size; ctx->hdr_size = 0; @@ -182,6 +205,9 @@ static int mtk_venc_comp_probe(struct platform_device *pdev) platform_set_drvdata(pdev, comp_dev); + INIT_DELAYED_WORK(&comp_dev->enc_timeout_worker, + mtk_venc_timeout_worker); + ret = component_add(&pdev->dev, &mtk_venc_component_ops); if (ret) { dev_err(&pdev->dev, "Failed to add component: %d\n", ret); diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h index f9ae97e252dc..e69344e38ca6 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h @@ -19,6 +19,7 @@ * @irq_status: venc hw irq status * @enc_irq: venc device irq * @core id: for venc core id: core#0, core#1... + * @enc_timeout_worker: venc timeout worker */ struct mtk_venc_comp_dev { struct platform_device *plat_dev; @@ -31,6 +32,7 @@ struct mtk_venc_comp_dev { unsigned int irq_status; int enc_irq; int core_id; + struct delayed_work enc_timeout_worker; }; #endif /* _MTK_VCODEC_ENC_HW_H_ */ diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c index 75b4e90232ae..1aab84faadf0 100644 --- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c +++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c @@ -10,11 +10,13 @@ #include "../mtk_vcodec_drv.h" #include "../mtk_vcodec_util.h" #include "../mtk_vcodec_enc_hw.h" +#include "../mtk_vcodec_enc_pm.h" static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc}; #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker) #define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098 +#define MTK_VENC_HW_TIMEOUT_MSEC 2000 enum venc_vpu_work_buf { VENC_VPU_WORK_BUF_RC_INFO_CORE0, @@ -413,6 +415,34 @@ static void h264_encode_filler(struct venc_common_inst *inst, void *buf, memset(p, 0xff, size); } +static void init_schedule_work(struct mtk_vcodec_ctx *ctx) +{ + unsigned long timeout; + struct mtk_venc_comp_dev *venc; + + venc = ctx->dev->enc_comp_dev[ctx->hw_id]; + timeout = msecs_to_jiffies(MTK_VENC_HW_TIMEOUT_MSEC); + schedule_delayed_work(&venc->enc_timeout_worker, timeout); +} + +static void cancel_schedule_work(struct mtk_vcodec_ctx *ctx) +{ + struct mtk_venc_comp_dev *venc; + unsigned long flags; + + venc = ctx->dev->enc_comp_dev[ctx->hw_id]; + cancel_delayed_work(&venc->enc_timeout_worker); + + mtk_venc_buf_done(ctx, ctx->hw_id, 0, 1, 0); + mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id); + + spin_lock_irqsave(&ctx->dev->irqlock, flags); + venc->curr_ctx = NULL; + spin_unlock_irqrestore(&ctx->dev->irqlock, flags); + + mtk_venc_unlock(ctx, ctx->hw_id); +} + static int venc_init(struct mtk_vcodec_ctx *ctx) { int i; @@ -489,12 +519,13 @@ static int venc_encode(void *handle, ctx->pbs_buf[ctx->hw_id] = bs_buf->buf; if (!inst->prepend_hdr) { + init_schedule_work(ctx); ret = venc_encode_frame(inst, frm_buf, bs_buf, &result->bs_size, ctx->hw_id); if (ret) { mtk_vcodec_err(inst, "encode frame failed: %d", ret); - return ret; + goto encode_err; } result->is_key_frm = inst->vpu_inst.is_key_frm; @@ -503,11 +534,13 @@ static int venc_encode(void *handle, mtk_vcodec_debug(inst, "venc_encode_frame prepend SPS/PPS"); + init_schedule_work(ctx); + ret = venc_encode_header(inst, bs_buf, &bs_size_hdr); if (ret) { mtk_vcodec_err(inst, "encode prepend hdr failed: %d", ret); - return ret; + goto encode_err; } hdr_sz = bs_size_hdr; @@ -530,7 +563,7 @@ static int venc_encode(void *handle, if (ret) { mtk_vcodec_err(inst, "encode hdr frame failed: %d", ret); - return ret; + goto encode_err; } result->bs_size = hdr_sz + filler_sz + bs_size_frm; @@ -552,6 +585,10 @@ static int venc_encode(void *handle, } return ret; + +encode_err: + cancel_schedule_work(ctx); + return ret; } static int venc_set_param(void *handle,