From patchwork Mon Jan 17 12:06:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 532742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63D9EC4332F for ; Mon, 17 Jan 2022 12:06:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239566AbiAQMGe (ORCPT ); Mon, 17 Jan 2022 07:06:34 -0500 Received: from mailgw02.mediatek.com ([210.61.82.184]:36694 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S239554AbiAQMGd (ORCPT ); Mon, 17 Jan 2022 07:06:33 -0500 X-UUID: 79eca5885b7c48bcbd8666ea03d3df5f-20220117 X-UUID: 79eca5885b7c48bcbd8666ea03d3df5f-20220117 Received: from mtkcas10.mediatek.inc [(172.21.101.39)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1089449713; Mon, 17 Jan 2022 20:06:30 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs07n2.mediatek.inc (172.21.101.141) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 17 Jan 2022 20:06:28 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 17 Jan 2022 20:06:27 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu , CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH v2, 02/10] media: mtk-vcodec: export encoder functions Date: Mon, 17 Jan 2022 20:06:07 +0800 Message-ID: <20220117120615.21687-3-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220117120615.21687-1-irui.wang@mediatek.com> References: <20220117120615.21687-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org mtk vcodec is built as a module, export some functions to make them visible by other modules. Signed-off-by: Irui Wang --- drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 2 ++ drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 3 +++ 2 files changed, 5 insertions(+) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c index ffb046eec610..134dc53e4855 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c @@ -1432,6 +1432,7 @@ int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx) mutex_unlock(&dev->enc_mutex); return 0; } +EXPORT_SYMBOL_GPL(mtk_venc_unlock); int mtk_venc_lock(struct mtk_vcodec_ctx *ctx) { @@ -1440,6 +1441,7 @@ int mtk_venc_lock(struct mtk_vcodec_ctx *ctx) mutex_lock(&dev->enc_mutex); return 0; } +EXPORT_SYMBOL_GPL(mtk_venc_lock); void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx) { diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c index 0825c6ec4eb7..7eba1a990ed7 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c @@ -85,6 +85,7 @@ int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev) put_device(pm->larbvenc); return ret; } +EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk); void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) { @@ -111,6 +112,7 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) for (i -= 1; i >= 0; i--) clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); } +EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_on); void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm) { @@ -121,3 +123,4 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm) for (i = enc_clk->clk_num - 1; i >= 0; i--) clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); } +EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_off); From patchwork Mon Jan 17 12:06:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 532741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5449BC433FE for ; Mon, 17 Jan 2022 12:06:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239603AbiAQMGk (ORCPT ); Mon, 17 Jan 2022 07:06:40 -0500 Received: from mailgw02.mediatek.com ([210.61.82.184]:36888 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S239606AbiAQMGj (ORCPT ); Mon, 17 Jan 2022 07:06:39 -0500 X-UUID: 10d3ae7e9f634065960712d5b7211fab-20220117 X-UUID: 10d3ae7e9f634065960712d5b7211fab-20220117 Received: from mtkexhb02.mediatek.inc [(172.21.101.103)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1050059200; Mon, 17 Jan 2022 20:06:35 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs10n2.mediatek.inc (172.21.101.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.3; Mon, 17 Jan 2022 20:06:33 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 17 Jan 2022 20:06:32 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , "Tiffany Lin" , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu , CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH v2, 04/10] media: mtk-vcodec: Enable venc dual core usage Date: Mon, 17 Jan 2022 20:06:09 +0800 Message-ID: <20220117120615.21687-5-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220117120615.21687-1-irui.wang@mediatek.com> References: <20220117120615.21687-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Adds new venc core mode to indicate different venc hardware mode: VENC_SINGLE_CORE_MODE means only one core, the device has its own power/clk/irq, init_clk/request_irq helper can be used. VENC_DUAL_CORE_MODE means more than one core inside, the core device can use the init_clk/request_irq helper to initialize their own power/clk/irq. And the main device doesn't need use these helper anymore. MT8195 has two H264 venc cores, enable dual_core_mode for it. Signed-off-by: Irui Wang --- drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 22 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 153 ++++++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 88 +++++----- 5 files changed, 266 insertions(+), 37 deletions(-) create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile index 93e7a343b5b0..c472b221bd6b 100644 --- a/drivers/media/platform/mtk-vcodec/Makefile +++ b/drivers/media/platform/mtk-vcodec/Makefile @@ -3,7 +3,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-enc.o \ mtk-vcodec-common.o \ - mtk-vcodec-dec-hw.o + mtk-vcodec-dec-hw.o \ + mtk-vcodec-enc-core.o mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \ @@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc_drv_if.o \ venc_vpu_if.o \ +mtk-vcodec-enc-core-y := mtk_vcodec_enc_core.o mtk-vcodec-common-y := mtk_vcodec_intr.o \ mtk_vcodec_util.o \ diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h index f78463ff4551..9e4e4290a69a 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h @@ -117,6 +117,23 @@ enum mtk_vdec_hw_count { MTK_VDEC_MAX_HW_COUNT, }; +/* + * enum mtk_venc_core_id -- encoder core id + */ +enum mtk_venc_core_id { + MTK_VENC_CORE0 = 0, + MTK_VENC_CORE1 = 1, + MTK_VENC_CORE_MAX, +}; + +/** + * enmu mtk_venc_core_mode - Used to indicate different encode mode + */ +enum mtk_venc_core_mode { + VENC_SINGLE_CORE_MODE = 0, + VENC_DUAL_CORE_MODE = 1, +}; + /* * struct mtk_video_fmt - Structure used to store information about pixelformats */ @@ -420,6 +437,7 @@ struct mtk_vcodec_dec_pdata { * @output_formats: array of supported output formats * @num_output_formats: number of entries in output_formats * @core_type: stand for h264 or vp8 encode + * @core_mode: indicate encode core mode */ struct mtk_vcodec_enc_pdata { bool uses_ext; @@ -430,6 +448,7 @@ struct mtk_vcodec_enc_pdata { const struct mtk_video_fmt *output_formats; size_t num_output_formats; int core_type; + enum mtk_venc_core_mode core_mode; }; #define MTK_ENC_CTX_IS_EXT(ctx) ((ctx)->dev->venc_pdata->uses_ext) @@ -479,6 +498,7 @@ struct mtk_vcodec_enc_pdata { * @subdev_dev: subdev hardware device * @subdev_prob_done: check whether all used hw device is prob done * @subdev_bitmap: used to record hardware is ready or not + * @enc_core_dev: used to store venc core device */ struct mtk_vcodec_dev { struct v4l2_device v4l2_dev; @@ -524,6 +544,8 @@ struct mtk_vcodec_dev { void *subdev_dev[MTK_VDEC_HW_MAX]; int (*subdev_prob_done)(struct mtk_vcodec_dev *vdec_dev); DECLARE_BITMAP(subdev_bitmap, MTK_VDEC_HW_MAX); + + void *enc_core_dev[MTK_VENC_CORE_MAX]; }; static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file mode 100644 index 000000000000..d84914f615a5 --- /dev/null +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c @@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2021 MediaTek Inc. + */ + +#include +#include +#include +#include +#include +#include + +#include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_core.h" + +static const struct of_device_id mtk_venc_core_ids[] = { + { + .compatible = "mediatek,mtk-venc-core0", + .data = (void *)MTK_VENC_CORE0, + }, + { + .compatible = "mediatek,mtk-venc-core1", + .data = (void *)MTK_VENC_CORE1, + }, + {}, +}; +MODULE_DEVICE_TABLE(of, mtk_venc_core_ids); + +static void clean_irq_status(unsigned int irq_status, void __iomem *addr) +{ + if (irq_status & MTK_VENC_IRQ_STATUS_PAUSE) + writel(MTK_VENC_IRQ_STATUS_PAUSE, addr); + + if (irq_status & MTK_VENC_IRQ_STATUS_SWITCH) + writel(MTK_VENC_IRQ_STATUS_SWITCH, addr); + + if (irq_status & MTK_VENC_IRQ_STATUS_DRAM) + writel(MTK_VENC_IRQ_STATUS_DRAM, addr); + + if (irq_status & MTK_VENC_IRQ_STATUS_SPS) + writel(MTK_VENC_IRQ_STATUS_SPS, addr); + + if (irq_status & MTK_VENC_IRQ_STATUS_PPS) + writel(MTK_VENC_IRQ_STATUS_PPS, addr); + + if (irq_status & MTK_VENC_IRQ_STATUS_FRM) + writel(MTK_VENC_IRQ_STATUS_FRM, addr); +} + +static irqreturn_t mtk_enc_core_irq_handler(int irq, void *priv) +{ + struct mtk_venc_core_dev *core = priv; + struct mtk_vcodec_ctx *ctx; + unsigned long flags; + void __iomem *addr; + + spin_lock_irqsave(&core->main_dev->irqlock, flags); + ctx = core->curr_ctx; + spin_unlock_irqrestore(&core->main_dev->irqlock, flags); + if (!ctx) + return IRQ_HANDLED; + + mtk_v4l2_debug(1, "id=%d core :%d", ctx->id, core->core_id); + + addr = core->reg_base + MTK_VENC_IRQ_ACK_OFFSET; + ctx->irq_status = readl(core->reg_base + MTK_VENC_IRQ_STATUS_OFFSET); + clean_irq_status(ctx->irq_status, addr); + + wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0); + return IRQ_HANDLED; +} + +static int mtk_venc_core_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct mtk_venc_core_dev *core; + struct mtk_vcodec_dev *main_dev; + int ret; + + if (!dev->parent) { + dev_err(dev, "No parent for venc core device\n"); + return -ENODEV; + } + + main_dev = dev_get_drvdata(dev->parent); + if (!main_dev) { + dev_err(dev, "Failed to get parent driver data"); + return -EINVAL; + } + + core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL); + if (!core) + return -ENOMEM; + + core->plat_dev = pdev; + + core->reg_base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(core->reg_base)) { + dev_err(&pdev->dev, "Failed to get reg base"); + ret = PTR_ERR(core->reg_base); + goto err; + } + + core->enc_irq = platform_get_irq(pdev, 0); + if (core->enc_irq < 0) { + dev_err(&pdev->dev, "Failed to get irq resource"); + ret = core->enc_irq; + goto err; + } + + ret = devm_request_irq(&pdev->dev, core->enc_irq, + mtk_enc_core_irq_handler, 0, + pdev->name, core); + if (ret) { + dev_err(&pdev->dev, + "Failed to install core->enc_irq %d (%d)", + core->enc_irq, ret); + ret = -EINVAL; + goto err; + } + + core->core_id = + (enum mtk_venc_core_id)of_device_get_match_data(&pdev->dev); + if (core->core_id < 0 || core->core_id >= MTK_VENC_CORE_MAX) { + ret = -EINVAL; + goto err; + } + + main_dev->enc_core_dev[core->core_id] = core; + core->main_dev = main_dev; + + platform_set_drvdata(pdev, core); + + dev_info(dev, "Venc core :%d probe done\n", core->core_id); + + return 0; + +err: + return ret; +} + +static struct platform_driver mtk_venc_core_driver = { + .probe = mtk_venc_core_probe, + .driver = { + .name = "mtk-venc-core", + .of_match_table = mtk_venc_core_ids, + }, +}; +module_platform_driver(mtk_venc_core_driver); + +MODULE_LICENSE("GPL v2"); +MODULE_DESCRIPTION("Mediatek video encoder core driver"); diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file mode 100644 index 000000000000..856681989869 --- /dev/null +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2021 MediaTek Inc. + */ + +#ifndef _MTK_VCODEC_ENC_CORE_H_ +#define _MTK_VCODEC_ENC_CORE_H_ + +#include +#include "mtk_vcodec_drv.h" + +/* + * struct mtk_venc_core_dev - driver data + * @plat_dev: platform_device + * @main_dev: main device + * @pm: power management data + * @curr_ctx: the context that is waiting for venc hardware + * @reg_base: mapped address of venc registers + * @irq_status: venc core irq status + * @enc_irq: venc device irq + * @core id: for venc core id: core#0, core#1... + */ +struct mtk_venc_core_dev { + struct platform_device *plat_dev; + struct mtk_vcodec_dev *main_dev; + + struct mtk_vcodec_pm pm; + struct mtk_vcodec_ctx *curr_ctx; + + void __iomem *reg_base; + unsigned int irq_status; + int enc_irq; + int core_id; +}; + +#endif /* _MTK_VCODEC_ENC_CORE_H_ */ diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c index 1f950310a3fb..9d48c604bc54 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c @@ -231,10 +231,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev) { struct mtk_vcodec_dev *dev; struct video_device *vfd_enc; - struct resource *res; phandle rproc_phandle; enum mtk_vcodec_fw_type fw_type; - int ret; + int ret, core_type; dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL); if (!dev) @@ -260,39 +259,37 @@ static int mtk_vcodec_probe(struct platform_device *pdev) return PTR_ERR(dev->fw_handler); dev->venc_pdata = of_device_get_match_data(&pdev->dev); - ret = mtk_vcodec_init_enc_clk(dev); - if (ret < 0) { - dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!"); - goto err_enc_pm; - } + core_type = dev->venc_pdata->core_type; - pm_runtime_enable(&pdev->dev); + if (dev->venc_pdata->core_mode == VENC_SINGLE_CORE_MODE) { + ret = mtk_vcodec_init_enc_clk(dev); + if (ret < 0) { + dev_err(&pdev->dev, + "Failed to get mtk vcodec clock source!"); + goto err_enc_pm; + } - dev->reg_base[dev->venc_pdata->core_type] = - devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(dev->reg_base[dev->venc_pdata->core_type])) { - ret = PTR_ERR(dev->reg_base[dev->venc_pdata->core_type]); - goto err_res; - } + pm_runtime_enable(&pdev->dev); - res = platform_get_resource(pdev, IORESOURCE_IRQ, 0); - if (res == NULL) { - dev_err(&pdev->dev, "failed to get irq resource"); - ret = -ENOENT; - goto err_res; - } + dev->reg_base[core_type] = + devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(dev->reg_base[core_type])) { + ret = PTR_ERR(dev->reg_base[core_type]); + goto err_res; + } - dev->enc_irq = platform_get_irq(pdev, 0); - irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN); - ret = devm_request_irq(&pdev->dev, dev->enc_irq, - mtk_vcodec_enc_irq_handler, - 0, pdev->name, dev); - if (ret) { - dev_err(&pdev->dev, - "Failed to install dev->enc_irq %d (%d) core_type (%d)", - dev->enc_irq, ret, dev->venc_pdata->core_type); - ret = -EINVAL; - goto err_res; + dev->enc_irq = platform_get_irq(pdev, 0); + irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN); + ret = devm_request_irq(&pdev->dev, dev->enc_irq, + mtk_vcodec_enc_irq_handler, + 0, pdev->name, dev); + if (ret) { + dev_err(&pdev->dev, + "Failed to install irq %d (%d) core_type (%d)", + dev->enc_irq, ret, core_type); + ret = -EINVAL; + goto err_res; + } } mutex_init(&dev->enc_mutex); @@ -358,8 +355,17 @@ static int mtk_vcodec_probe(struct platform_device *pdev) goto err_enc_reg; } + if (dev->venc_pdata->core_mode == VENC_DUAL_CORE_MODE) { + ret = of_platform_populate(pdev->dev.of_node, + NULL, NULL, &pdev->dev); + if (ret) { + mtk_v4l2_err("Venc main device populate failed"); + goto err_enc_reg; + } + } + mtk_v4l2_debug(0, "encoder %d registered as /dev/video%d", - dev->venc_pdata->core_type, vfd_enc->num); + core_type, vfd_enc->num); return 0; @@ -372,8 +378,10 @@ static int mtk_vcodec_probe(struct platform_device *pdev) err_enc_alloc: v4l2_device_unregister(&dev->v4l2_dev); err_res: - pm_runtime_disable(dev->pm.dev); - put_device(dev->pm.larbvenc); + if (dev->venc_pdata->core_mode == VENC_SINGLE_CORE_MODE) { + pm_runtime_disable(dev->pm.dev); + put_device(dev->pm.larbvenc); + } err_enc_pm: mtk_vcodec_fw_release(dev->fw_handler); return ret; @@ -387,6 +395,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_avc_pdata = { .min_bitrate = 64, .max_bitrate = 60000000, .core_type = VENC_SYS, + .core_mode = VENC_SINGLE_CORE_MODE, }; static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = { @@ -397,6 +406,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = { .min_bitrate = 64, .max_bitrate = 9000000, .core_type = VENC_LT_SYS, + .core_mode = VENC_SINGLE_CORE_MODE, }; static const struct mtk_vcodec_enc_pdata mt8183_pdata = { @@ -408,6 +418,7 @@ static const struct mtk_vcodec_enc_pdata mt8183_pdata = { .min_bitrate = 64, .max_bitrate = 40000000, .core_type = VENC_SYS, + .core_mode = VENC_SINGLE_CORE_MODE, }; static const struct mtk_vcodec_enc_pdata mt8192_pdata = { @@ -419,6 +430,7 @@ static const struct mtk_vcodec_enc_pdata mt8192_pdata = { .min_bitrate = 64, .max_bitrate = 100000000, .core_type = VENC_SYS, + .core_mode = VENC_SINGLE_CORE_MODE, }; static const struct mtk_vcodec_enc_pdata mt8195_pdata = { @@ -430,6 +442,7 @@ static const struct mtk_vcodec_enc_pdata mt8195_pdata = { .min_bitrate = 64, .max_bitrate = 100000000, .core_type = VENC_SYS, + .core_mode = VENC_DUAL_CORE_MODE, }; static const struct of_device_id mtk_vcodec_enc_match[] = { @@ -457,8 +470,11 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev) video_unregister_device(dev->vfd_enc); v4l2_device_unregister(&dev->v4l2_dev); - pm_runtime_disable(dev->pm.dev); - put_device(dev->pm.larbvenc); + + if (dev->venc_pdata->core_mode == VENC_SINGLE_CORE_MODE) { + pm_runtime_disable(dev->pm.dev); + put_device(dev->pm.larbvenc); + } mtk_vcodec_fw_release(dev->fw_handler); return 0; } From patchwork Mon Jan 17 12:06:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 532740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E917DC433EF for ; Mon, 17 Jan 2022 12:06:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239649AbiAQMGq (ORCPT ); Mon, 17 Jan 2022 07:06:46 -0500 Received: from mailgw02.mediatek.com ([210.61.82.184]:37034 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S239636AbiAQMGp (ORCPT ); Mon, 17 Jan 2022 07:06:45 -0500 X-UUID: 4231c94e96db4edf8439762bc09b432d-20220117 X-UUID: 4231c94e96db4edf8439762bc09b432d-20220117 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 813468978; Mon, 17 Jan 2022 20:06:41 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs07n2.mediatek.inc (172.21.101.141) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 17 Jan 2022 20:06:39 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 17 Jan 2022 20:06:38 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu , CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH v2, 07/10] media: mtk-vcodec: Rewrite venc clock interface Date: Mon, 17 Jan 2022 20:06:12 +0800 Message-ID: <20220117120615.21687-8-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220117120615.21687-1-irui.wang@mediatek.com> References: <20220117120615.21687-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Dual core mode need enable the specific core's clk, add an another param for clock_on/clock_off interface. Signed-off-by: Irui Wang --- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 96 ++++++++++++++++--- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 4 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 10 +- 3 files changed, 86 insertions(+), 24 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c index dfaef884e6e3..4e37d68f75dd 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c @@ -83,17 +83,42 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev, } EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk); -void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id) { - struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; + struct mtk_venc_core_dev *core; + struct mtk_vcodec_pm *enc_pm; + struct mtk_vcodec_clk *enc_clk; + struct clk *clk; int ret, i = 0; - for (i = 0; i < enc_clk->clk_num; i++) { - ret = clk_prepare_enable(enc_clk->clk_info[i].vcodec_clk); - if (ret) { - mtk_v4l2_err("venc clk_prepare_enable %d %s fail %d", i, - enc_clk->clk_info[i].clk_name, ret); - goto clkerr; + if (dev->venc_pdata->core_mode == VENC_DUAL_CORE_MODE) { + core = (struct mtk_venc_core_dev *)dev->enc_core_dev[core_id]; + enc_pm = &core->pm; + enc_clk = &enc_pm->venc_clk; + + for (i = 0; i < enc_clk->clk_num; i++) { + clk = enc_clk->clk_info[i].vcodec_clk; + ret = clk_enable(clk); + if (ret) { + mtk_v4l2_err("clk_enable %d %s fail %d", i, + enc_clk->clk_info[i].clk_name, + ret); + goto core_clk_err; + } + } + } else { + enc_pm = &dev->pm; + enc_clk = &enc_pm->venc_clk; + + for (i = 0; i < enc_clk->clk_num; i++) { + clk = enc_clk->clk_info[i].vcodec_clk; + ret = clk_prepare_enable(clk); + if (ret) { + mtk_v4l2_err("clk_prepare %d %s fail %d", + i, enc_clk->clk_info[i].clk_name, + ret); + goto clkerr; + } } } @@ -104,27 +129,49 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) } return; +core_clk_err: + for (i -= 1; i >= 0; i--) + clk_disable(enc_clk->clk_info[i].vcodec_clk); + + return; + clkerr: for (i -= 1; i >= 0; i--) clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); } EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_on); -void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id) { - struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; + struct mtk_venc_core_dev *core; + struct mtk_vcodec_pm *enc_pm; + struct mtk_vcodec_clk *enc_clk; int i = 0; mtk_smi_larb_put(pm->larbvenc); - for (i = enc_clk->clk_num - 1; i >= 0; i--) - clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); + + if (dev->venc_pdata->core_mode == VENC_DUAL_CORE_MODE) { + core = (struct mtk_venc_core_dev *)dev->enc_core_dev[core_id]; + enc_pm = &core->pm; + enc_clk = &enc_pm->venc_clk; + + for (i = enc_clk->clk_num - 1; i >= 0; i--) + clk_disable(enc_clk->clk_info[i].vcodec_clk); + } else { + enc_pm = &dev->pm; + enc_clk = &enc_pm->venc_clk; + + for (i = enc_clk->clk_num - 1; i >= 0; i--) + clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); + } } EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_off); int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev) { - int i, ret; + int i, ret, j = 0; struct mtk_venc_core_dev *core; + struct mtk_vcodec_clk *clk; /* power on all available venc cores */ for (i = 0; i < MTK_VENC_CORE_MAX; i++) { @@ -137,12 +184,28 @@ int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev) mtk_v4l2_err("power on core[%d] fail %d", i, ret); goto pw_on_fail; } + + clk = &core->pm.venc_clk; + for (j = 0; j < clk->clk_num; j++) { + ret = clk_prepare(clk->clk_info[j].vcodec_clk); + if (ret) { + mtk_v4l2_err("prepare clk [%s] fail %d", + clk->clk_info[j].clk_name, + ret); + goto pw_on_fail; + } + } } return 0; pw_on_fail: for (i -= 1; i >= 0; i--) { core = (struct mtk_venc_core_dev *)dev->enc_core_dev[i]; + + clk = &core->pm.venc_clk; + for (j -= 1; j >= 0; j--) + clk_unprepare(clk->clk_info[j].vcodec_clk); + pm_runtime_put_sync(&core->plat_dev->dev); } return ret; @@ -150,8 +213,9 @@ int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev) int mtk_venc_core_pw_off(struct mtk_vcodec_dev *dev) { - int i, ret; + int i, ret, j; struct mtk_venc_core_dev *core; + struct mtk_vcodec_clk *clk; /* power off all available venc cores */ for (i = 0; i < MTK_VENC_CORE_MAX; i++) { @@ -159,6 +223,10 @@ int mtk_venc_core_pw_off(struct mtk_vcodec_dev *dev) if (!core) return 0; + clk = &core->pm.venc_clk; + for (j = clk->clk_num - 1; j >= 0; j--) + clk_unprepare(clk->clk_info[j].vcodec_clk); + ret = pm_runtime_put_sync(&core->plat_dev->dev); if (ret < 0) mtk_v4l2_err("power off core[%d] fail %d", i, ret); diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h index 99b6b6e29e35..5113ed8a869e 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h @@ -11,8 +11,8 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev, struct mtk_vcodec_pm *pm); -void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm); -void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm); +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id); +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id); int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev); int mtk_venc_core_pw_off(struct mtk_vcodec_dev *dev); diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c index ce0bce811615..6cbdb7e30bb3 100644 --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c @@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) } mtk_venc_lock(ctx); - mtk_vcodec_enc_clock_on(&ctx->dev->pm); ret = ctx->enc_if->init(ctx); - mtk_vcodec_enc_clock_off(&ctx->dev->pm); mtk_venc_unlock(ctx); return ret; @@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx, int ret = 0; mtk_venc_lock(ctx); - mtk_vcodec_enc_clock_on(&ctx->dev->pm); ret = ctx->enc_if->set_param(ctx->drv_handle, type, in); - mtk_vcodec_enc_clock_off(&ctx->dev->pm); mtk_venc_unlock(ctx); return ret; @@ -68,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx, ctx->dev->curr_ctx = ctx; spin_unlock_irqrestore(&ctx->dev->irqlock, flags); - mtk_vcodec_enc_clock_on(&ctx->dev->pm); + mtk_vcodec_enc_clock_on(ctx->dev, 0); ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result); - mtk_vcodec_enc_clock_off(&ctx->dev->pm); + mtk_vcodec_enc_clock_off(ctx->dev, 0); spin_lock_irqsave(&ctx->dev->irqlock, flags); ctx->dev->curr_ctx = NULL; @@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx) return 0; mtk_venc_lock(ctx); - mtk_vcodec_enc_clock_on(&ctx->dev->pm); ret = ctx->enc_if->deinit(ctx->drv_handle); - mtk_vcodec_enc_clock_off(&ctx->dev->pm); mtk_venc_unlock(ctx); ctx->drv_handle = NULL; From patchwork Mon Jan 17 12:06:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 532738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CC0CC433FE for ; Mon, 17 Jan 2022 12:07:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239686AbiAQMG7 (ORCPT ); Mon, 17 Jan 2022 07:06:59 -0500 Received: from mailgw01.mediatek.com ([60.244.123.138]:48574 "EHLO mailgw01.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S239667AbiAQMGt (ORCPT ); Mon, 17 Jan 2022 07:06:49 -0500 X-UUID: 34f2768e457a4dc79579b52bfe8c1f1a-20220117 X-UUID: 34f2768e457a4dc79579b52bfe8c1f1a-20220117 Received: from mtkcas11.mediatek.inc [(172.21.101.40)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 633297809; Mon, 17 Jan 2022 20:06:46 +0800 Received: from mtkexhb01.mediatek.inc (172.21.101.102) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.15; Mon, 17 Jan 2022 20:06:45 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkexhb01.mediatek.inc (172.21.101.102) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 17 Jan 2022 20:06:44 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 17 Jan 2022 20:06:43 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , "Tiffany Lin" , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu , CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH v2, 09/10] media: mtk-vcodec: Add dual core mode encode process Date: Mon, 17 Jan 2022 20:06:14 +0800 Message-ID: <20220117120615.21687-10-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220117120615.21687-1-irui.wang@mediatek.com> References: <20220117120615.21687-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The dual core mode encoding is tries to uses the two venc cores: frame#0 uses core#0, frame#1 uses core#1, frame#2 uses core#0..., Lock the device and enable the clock by used core, for sequence header encoding, it always uses core#0. Signed-off-by: Irui Wang --- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 11 ++- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 23 +++--- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 4 +- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 7 +- .../platform/mtk-vcodec/venc/venc_h264_if.c | 16 ++-- .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 73 ++++++++++++++----- .../media/platform/mtk-vcodec/venc_drv_if.h | 5 ++ .../media/platform/mtk-vcodec/venc_vpu_if.c | 10 ++- .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- 10 files changed, 113 insertions(+), 42 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h index dc036279c42f..c3cf5904b4c6 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h @@ -308,6 +308,9 @@ struct vdec_pic_info { * @max_width: hardware supported max width * @max_height: hardware supported max height * @msg_queue: msg queue used to store lat buffer information. + * @q_mutex: src & dst vb2_queue mutex + * @enc_idx: used to record encoded frame count + * @core_id: used to reoord used core */ struct mtk_vcodec_ctx { enum mtk_instance_type type; @@ -356,6 +359,10 @@ struct mtk_vcodec_ctx { unsigned int max_width; unsigned int max_height; struct vdec_msg_queue msg_queue; + + struct mutex q_mutex; + int enc_idx; + int core_id; }; /* @@ -533,7 +540,9 @@ struct mtk_vcodec_dev { /* decoder hardware mutex lock */ struct mutex dec_mutex[MTK_VDEC_HW_MAX]; - struct mutex enc_mutex; + + /* encoder core mutex lock */ + struct mutex enc_mutex[MTK_VENC_CORE_MAX]; struct mtk_vcodec_pm pm; unsigned int dec_capability; diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c index df231e67cdb2..dc70d328fcd3 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c @@ -1198,14 +1198,17 @@ static void mtk_venc_worker(struct work_struct *work) vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0); v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR); mtk_v4l2_err("venc_if_encode failed=%d", ret); - } else { + } else if (ctx->dev->venc_pdata->core_mode == VENC_SINGLE_CORE_MODE) { v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE); - vb2_set_plane_payload(&dst_buf->vb2_buf, 0, enc_result.bs_size); + vb2_set_plane_payload(&dst_buf->vb2_buf, 0, + enc_result.bs_size); v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE); mtk_v4l2_debug(2, "venc_if_encode bs size=%d", - enc_result.bs_size); + enc_result.bs_size); } + ctx->enc_idx++; + v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx); mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>", @@ -1259,7 +1262,7 @@ void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx) { struct mtk_q_data *q_data; - ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex; + ctx->m2m_ctx->q_lock = &ctx->q_mutex; ctx->fh.m2m_ctx = ctx->m2m_ctx; ctx->fh.ctrl_handler = &ctx->ctrl_hdl; INIT_WORK(&ctx->encode_work, mtk_venc_worker); @@ -1391,7 +1394,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, src_vq->ops = &mtk_venc_vb2_ops; src_vq->mem_ops = &vb2_dma_contig_memops; src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; - src_vq->lock = &ctx->dev->dev_mutex; + src_vq->lock = &ctx->q_mutex; src_vq->dev = &ctx->dev->plat_dev->dev; ret = vb2_queue_init(src_vq); @@ -1405,26 +1408,26 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, dst_vq->ops = &mtk_venc_vb2_ops; dst_vq->mem_ops = &vb2_dma_contig_memops; dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; - dst_vq->lock = &ctx->dev->dev_mutex; + dst_vq->lock = &ctx->q_mutex; dst_vq->dev = &ctx->dev->plat_dev->dev; return vb2_queue_init(dst_vq); } -int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx) +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id) { struct mtk_vcodec_dev *dev = ctx->dev; - mutex_unlock(&dev->enc_mutex); + mutex_unlock(&dev->enc_mutex[core_id]); return 0; } EXPORT_SYMBOL_GPL(mtk_venc_unlock); -int mtk_venc_lock(struct mtk_vcodec_ctx *ctx) +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id) { struct mtk_vcodec_dev *dev = ctx->dev; - mutex_lock(&dev->enc_mutex); + mutex_lock(&dev->enc_mutex[core_id]); return 0; } EXPORT_SYMBOL_GPL(mtk_venc_lock); diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h index 513ee7993e34..434d91d36158 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h @@ -39,8 +39,8 @@ struct mtk_video_enc_buf { extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops; extern const struct v4l2_m2m_ops mtk_venc_m2m_ops; -int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx); -int mtk_venc_lock(struct mtk_vcodec_ctx *ctx); +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id); +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id); int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq); void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx); diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c index cef134bb6e83..cf42a5930b48 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c @@ -133,6 +133,7 @@ static int fops_vcodec_open(struct file *file) INIT_LIST_HEAD(&ctx->list); ctx->dev = dev; init_waitqueue_head(&ctx->queue[0]); + mutex_init(&ctx->q_mutex); ctx->type = MTK_INST_ENCODER; ret = mtk_vcodec_enc_ctrls_setup(ctx); @@ -240,7 +241,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev) struct video_device *vfd_enc; phandle rproc_phandle; enum mtk_vcodec_fw_type fw_type; - int ret, core_type; + int ret, core_type, i; dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL); if (!dev) @@ -299,7 +300,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev) } } - mutex_init(&dev->enc_mutex); + for (i = 0; i < MTK_VENC_CORE_MAX; i++) + mutex_init(&dev->enc_mutex[i]); + mutex_init(&dev->dev_mutex); spin_lock_init(&dev->irqlock); diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c index 023b6eb8b20c..fdecf1d24b25 100644 --- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c @@ -448,7 +448,8 @@ static int h264_encode_sps(struct venc_h264_inst *inst, mtk_vcodec_debug_enter(inst); - ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL, bs_buf, NULL); + ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, + NULL, bs_buf, NULL, MTK_VENC_CORE0); if (ret) return ret; @@ -474,7 +475,8 @@ static int h264_encode_pps(struct venc_h264_inst *inst, mtk_vcodec_debug_enter(inst); - ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL, bs_buf, NULL); + ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, + NULL, bs_buf, NULL, MTK_VENC_CORE0); if (ret) return ret; @@ -516,7 +518,8 @@ static int h264_encode_header(struct venc_h264_inst *inst, static int h264_encode_frame(struct venc_h264_inst *inst, struct venc_frm_buf *frm_buf, struct mtk_vcodec_mem *bs_buf, - unsigned int *bs_size) + unsigned int *bs_size, + int core_id) { int ret = 0; unsigned int irq_status; @@ -536,7 +539,8 @@ static int h264_encode_frame(struct venc_h264_inst *inst, mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n", frame_info.frm_count, frame_info.skip_frm_count, frame_info.frm_type); - ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf, bs_buf, &frame_info); + ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, + frm_buf, bs_buf, &frame_info, core_id); if (ret) return ret; @@ -664,7 +668,7 @@ static int h264_enc_encode(void *handle, if (!inst->prepend_hdr) { ret = h264_encode_frame(inst, frm_buf, bs_buf, - &result->bs_size); + &result->bs_size, ctx->core_id); if (ret) goto encode_err; result->is_key_frm = inst->vpu_inst.is_key_frm; @@ -692,7 +696,7 @@ static int h264_enc_encode(void *handle, tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz); ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf, - &bs_size_frm); + &bs_size_frm, ctx->core_id); if (ret) goto encode_err; diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c index 56ce58f761f1..3b3471d90e21 100644 --- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c +++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c @@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst, mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt); - ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, NULL); + ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, + NULL, MTK_VENC_CORE0); if (ret) return ret; diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c index 6cbdb7e30bb3..5c5013d0f32e 100644 --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c @@ -14,6 +14,7 @@ #include "venc_drv_if.h" #include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_core.h" #include "mtk_vcodec_enc_pm.h" int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) @@ -31,9 +32,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) return -EINVAL; } - mtk_venc_lock(ctx); + mtk_venc_lock(ctx, ctx->core_id); ret = ctx->enc_if->init(ctx); - mtk_venc_unlock(ctx); + mtk_venc_unlock(ctx, ctx->core_id); return ret; } @@ -43,9 +44,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx, { int ret = 0; - mtk_venc_lock(ctx); + mtk_venc_lock(ctx, ctx->core_id); ret = ctx->enc_if->set_param(ctx->drv_handle, type, in); - mtk_venc_unlock(ctx); + mtk_venc_unlock(ctx, ctx->core_id); return ret; } @@ -56,24 +57,14 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx, struct venc_done_result *result) { int ret = 0; - unsigned long flags; - - mtk_venc_lock(ctx); - spin_lock_irqsave(&ctx->dev->irqlock, flags); - ctx->dev->curr_ctx = ctx; - spin_unlock_irqrestore(&ctx->dev->irqlock, flags); + venc_encode_prepare(ctx, opt); - mtk_vcodec_enc_clock_on(ctx->dev, 0); ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result); - mtk_vcodec_enc_clock_off(ctx->dev, 0); - spin_lock_irqsave(&ctx->dev->irqlock, flags); - ctx->dev->curr_ctx = NULL; - spin_unlock_irqrestore(&ctx->dev->irqlock, flags); + venc_encode_unprepare(ctx, opt); - mtk_venc_unlock(ctx); return ret; } @@ -84,11 +75,57 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx) if (!ctx->drv_handle) return 0; - mtk_venc_lock(ctx); + mtk_venc_lock(ctx, ctx->core_id); ret = ctx->enc_if->deinit(ctx->drv_handle); - mtk_venc_unlock(ctx); + mtk_venc_unlock(ctx, ctx->core_id); ctx->drv_handle = NULL; return ret; } + +void venc_encode_prepare(struct mtk_vcodec_ctx *ctx, + enum venc_start_opt opt) +{ + unsigned long flags; + struct mtk_venc_core_dev *core; + + if (ctx->dev->venc_pdata->core_mode == VENC_DUAL_CORE_MODE) { + if (ctx->enc_idx & 0x01) + ctx->core_id = MTK_VENC_CORE1; + else + ctx->core_id = MTK_VENC_CORE0; + } else { + ctx->core_id = MTK_VENC_CORE0; + } + mtk_venc_lock(ctx, ctx->core_id); + + spin_lock_irqsave(&ctx->dev->irqlock, flags); + + if (ctx->dev->venc_pdata->core_mode == VENC_DUAL_CORE_MODE) { + core = ctx->dev->enc_core_dev[ctx->core_id]; + + core->curr_ctx = ctx; + } else { + ctx->dev->curr_ctx = ctx; + } + + spin_unlock_irqrestore(&ctx->dev->irqlock, flags); + + mtk_vcodec_enc_clock_on(ctx->dev, ctx->core_id); +} + +void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx, + enum venc_start_opt opt) +{ + unsigned long flags; + + if (ctx->dev->venc_pdata->core_mode == VENC_SINGLE_CORE_MODE || + opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) { + mtk_vcodec_enc_clock_off(ctx->dev, ctx->core_id); + spin_lock_irqsave(&ctx->dev->irqlock, flags); + ctx->dev->curr_ctx = NULL; + spin_unlock_irqrestore(&ctx->dev->irqlock, flags); + mtk_venc_unlock(ctx, ctx->core_id); + } +} diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h index 0b04a1020873..15e9a2ab9cda 100644 --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h @@ -167,4 +167,9 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx, struct mtk_vcodec_mem *bs_buf, struct venc_done_result *result); +void venc_encode_prepare(struct mtk_vcodec_ctx *ctx, + enum venc_start_opt opt); +void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx, + enum venc_start_opt opt); + #endif /* _VENC_DRV_IF_H_ */ diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c index d3570c4c177d..a7219c10013b 100644 --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c @@ -225,9 +225,11 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu, int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode, struct venc_frm_buf *frm_buf, struct mtk_vcodec_mem *bs_buf, - struct venc_frame_info *frame_info) + struct venc_frame_info *frame_info, + int core_id) { const bool is_ext = MTK_ENC_CTX_IS_EXT(vpu->ctx); + const int core_mode = MTK_ENC_CORE_MODE(vpu->ctx); size_t msg_size = is_ext ? sizeof(struct venc_ap_ipi_msg_enc_ext) : sizeof(struct venc_ap_ipi_msg_enc); @@ -261,6 +263,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode, out.data[1] = frame_info->skip_frm_count; out.data[2] = frame_info->frm_type; } + + if (core_mode == VENC_DUAL_CORE_MODE) { + out.data_item = 4; + out.data[3] = core_id; + } + if (vpu_enc_send_msg(vpu, &out, msg_size)) { mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail", bs_mode); diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h index f83bc1b3f2bf..a8055eb62f7c 100644 --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h @@ -45,7 +45,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu, int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode, struct venc_frm_buf *frm_buf, struct mtk_vcodec_mem *bs_buf, - struct venc_frame_info *frame_info); + struct venc_frame_info *frame_info, + int core_id); int vpu_enc_deinit(struct venc_vpu_inst *vpu); #endif From patchwork Mon Jan 17 12:06:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 532739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E014C433F5 for ; Mon, 17 Jan 2022 12:06:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239661AbiAQMGz (ORCPT ); Mon, 17 Jan 2022 07:06:55 -0500 Received: from mailgw02.mediatek.com ([210.61.82.184]:37174 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S239654AbiAQMGv (ORCPT ); Mon, 17 Jan 2022 07:06:51 -0500 X-UUID: b7876d18f76141a18be2e2925111f3bb-20220117 X-UUID: b7876d18f76141a18be2e2925111f3bb-20220117 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1176916195; Mon, 17 Jan 2022 20:06:48 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs07n1.mediatek.inc (172.21.101.16) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 17 Jan 2022 20:06:47 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 17 Jan 2022 20:06:45 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu , CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH v2, 10/10] media: mtk-vcodec: Done encode result to client Date: Mon, 17 Jan 2022 20:06:15 +0800 Message-ID: <20220117120615.21687-11-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220117120615.21687-1-irui.wang@mediatek.com> References: <20220117120615.21687-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org When enable dual core mode encoding, the wait IRQ done synchronous function should not be called, so the encode result can't return to client in device_run. device_run called to trigger encoding process, finish it, so another scheduled job can be ran. When encoding done, done the encode result to client in IRQ handler. Signed-off-by: Irui Wang --- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 6 ++ .../platform/mtk-vcodec/mtk_vcodec_enc.c | 64 ++++++++++++++++++- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 3 + .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 27 +++++++- .../platform/mtk-vcodec/mtk_vcodec_util.h | 1 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 22 ++++++- .../media/platform/mtk-vcodec/venc_drv_if.h | 2 + 7 files changed, 120 insertions(+), 5 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h index c3cf5904b4c6..5faeffd397db 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h @@ -311,6 +311,9 @@ struct vdec_pic_info { * @q_mutex: src & dst vb2_queue mutex * @enc_idx: used to record encoded frame count * @core_id: used to reoord used core + * @pfrm_buf: used to store current ctx's frame buffer + * @pbs_buf: used to store current ctx's bitstream buffer + * @hdr_size: used to store prepend header size */ struct mtk_vcodec_ctx { enum mtk_instance_type type; @@ -363,6 +366,9 @@ struct mtk_vcodec_ctx { struct mutex q_mutex; int enc_idx; int core_id; + struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_CORE_MAX]; + struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_CORE_MAX]; + unsigned int hdr_size; }; /* diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c index dc70d328fcd3..3056cbc4b3a3 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c @@ -924,6 +924,8 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q) mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->id, q->type); + mtk_venc_lock_all(ctx); + if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) { vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0); @@ -1103,10 +1105,13 @@ static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx) ctx->id, vb2_v4l2->vb2_buf.index, mtk_buf->enc_params.force_intra); - if (mtk_buf->enc_params.force_intra) + if (mtk_buf->enc_params.force_intra) { ret |= venc_if_set_param(ctx, VENC_SET_PARAM_FORCE_INTRA, NULL); + + ctx->enc_idx = 0; + } } mtk_buf->param_change = MTK_ENCODE_PARAM_NONE; @@ -1157,6 +1162,7 @@ static void mtk_venc_worker(struct work_struct *work) * is dequeued. */ if (src_buf == &ctx->empty_flush_buf.vb) { + mtk_venc_lock_all(ctx); vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0); dst_buf->flags |= V4L2_BUF_FLAG_LAST; v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE); @@ -1171,9 +1177,12 @@ static void mtk_venc_worker(struct work_struct *work) frm_buf.fb_addr[i].size = (size_t)src_buf->vb2_buf.planes[i].length; } + frm_buf.src_addr = src_buf; + bs_buf.va = vb2_plane_vaddr(&dst_buf->vb2_buf, 0); bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0); bs_buf.size = (size_t)dst_buf->vb2_buf.planes[0].length; + bs_buf.buf = dst_buf; mtk_v4l2_debug(2, "Framebuf PA=%llx Size=0x%zx;PA=0x%llx Size=0x%zx;PA=0x%llx Size=%zu", @@ -1220,6 +1229,8 @@ static void m2mops_venc_device_run(void *priv) { struct mtk_vcodec_ctx *ctx = priv; + mtk_venc_param_change(ctx); + if ((ctx->q_data[MTK_Q_DATA_DST].fmt->fourcc == V4L2_PIX_FMT_H264) && (ctx->state != MTK_STATE_HEADER)) { /* encode h264 sps/pps header */ @@ -1228,7 +1239,6 @@ static void m2mops_venc_device_run(void *priv) return; } - mtk_venc_param_change(ctx); queue_work(ctx->dev->encode_workqueue, &ctx->encode_work); } @@ -1414,6 +1424,34 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, return vb2_queue_init(dst_vq); } +void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id, + unsigned int bs_size, bool time_out, bool key_frame) +{ + struct vb2_v4l2_buffer *src_vb2_v4l2 = NULL; + struct vb2_v4l2_buffer *dst_vb2_v4l2 = NULL; + + /* + * the frm_buf(src_buf) and bs_buf(dst_buf) can be obtained from ctx, + * then put them to done list, user can get them by dqbuf call + */ + src_vb2_v4l2 = ctx->pfrm_buf[core_id]; + dst_vb2_v4l2 = ctx->pbs_buf[core_id]; + + if (src_vb2_v4l2 && dst_vb2_v4l2) { + dst_vb2_v4l2->vb2_buf.timestamp = + src_vb2_v4l2->vb2_buf.timestamp; + dst_vb2_v4l2->timecode = src_vb2_v4l2->timecode; + + if (key_frame) + dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME; + + v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE); + vb2_set_plane_payload(&dst_vb2_v4l2->vb2_buf, 0, bs_size); + v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE); + } +} +EXPORT_SYMBOL_GPL(mtk_venc_buf_done); + int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id) { struct mtk_vcodec_dev *dev = ctx->dev; @@ -1441,3 +1479,25 @@ void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx) ctx->state = MTK_STATE_FREE; } + +void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx) +{ + unsigned int i; + struct mtk_vcodec_dev *dev = ctx->dev; + + /* + * For frame-racing mode encoding, there are may be bufs being encoded + * when get the empty flush buffer or stop streaming, for example, the + * buffer with LAST flag will return to client before the encoding + * buffers, which will cause frame lost. + * The encoder device mutex will be locked during encoding process, + * when encode done, the mutex unlocked. So if all encoder device mutex + * can be locked, which means there are no bufs being encoded at this + * time, then the buffer with LAST flag can return to client properly. + */ + for (i = 0; i < MTK_VENC_CORE_MAX; i++) { + mutex_lock(&dev->enc_mutex[i]); + mutex_unlock(&dev->enc_mutex[i]); + } +} +EXPORT_SYMBOL_GPL(mtk_venc_lock_all); diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h index 434d91d36158..1e22c77a2760 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h @@ -46,5 +46,8 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx); int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx); void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx); +void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id, + unsigned int bs_size, bool time_out, bool key_frame); +void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx); #endif /* _MTK_VCODEC_ENC_H_ */ diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c index 3835b9f458bf..6140b9bd4dd0 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c @@ -14,6 +14,11 @@ #include "mtk_vcodec_enc.h" #include "mtk_vcodec_enc_core.h" #include "mtk_vcodec_enc_pm.h" +#include "mtk_vcodec_intr.h" + +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098 +#define VENC_PIC_FRM_TYPE 0x0010 +#define VENC_PIC_KEY_FRM 0x2 static const struct of_device_id mtk_venc_core_ids[] = { { @@ -47,6 +52,7 @@ static void clean_irq_status(unsigned int irq_status, void __iomem *addr) if (irq_status & MTK_VENC_IRQ_STATUS_FRM) writel(MTK_VENC_IRQ_STATUS_FRM, addr); + } static irqreturn_t mtk_enc_core_irq_handler(int irq, void *priv) @@ -55,6 +61,9 @@ static irqreturn_t mtk_enc_core_irq_handler(int irq, void *priv) struct mtk_vcodec_ctx *ctx; unsigned long flags; void __iomem *addr; + unsigned int bs_size; + unsigned int frm_type; + bool is_key_frame = 0; spin_lock_irqsave(&core->main_dev->irqlock, flags); ctx = core->curr_ctx; @@ -66,9 +75,25 @@ static irqreturn_t mtk_enc_core_irq_handler(int irq, void *priv) addr = core->reg_base + MTK_VENC_IRQ_ACK_OFFSET; ctx->irq_status = readl(core->reg_base + MTK_VENC_IRQ_STATUS_OFFSET); + bs_size = readl(core->reg_base + VENC_PIC_BITSTREAM_BYTE_CNT); + frm_type = readl(core->reg_base + VENC_PIC_FRM_TYPE); clean_irq_status(ctx->irq_status, addr); - wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0); + if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) { + if (ctx->hdr_size != 0) { + bs_size += ctx->hdr_size; + ctx->hdr_size = 0; + } + if (frm_type & VENC_PIC_KEY_FRM) + is_key_frame = 1; + + mtk_venc_buf_done(ctx, core->core_id, bs_size, 0, is_key_frame); + mtk_vcodec_enc_clock_off(core->main_dev, core->core_id); + mtk_venc_unlock(ctx, core->core_id); + } else { + wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0); + } + return IRQ_HANDLED; } diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h index de9e18688842..4436d8bc60a0 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h @@ -15,6 +15,7 @@ struct mtk_vcodec_mem { size_t size; void *va; dma_addr_t dma_addr; + void *buf; }; struct mtk_vcodec_fb { diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c index fdecf1d24b25..733dbf431aae 100644 --- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c @@ -558,6 +558,12 @@ static int h264_encode_frame(struct venc_h264_inst *inst, return ret; } + if (is_dual_core) { + ++inst->frm_cnt; + mtk_vcodec_debug(inst, "dual core : frm %d <-", inst->frm_cnt); + return ret; + } + irq_status = h264_enc_wait_venc_done(inst); if (irq_status != MTK_VENC_IRQ_STATUS_FRM) { mtk_vcodec_err(inst, "irq_status=%d failed", irq_status); @@ -639,10 +645,12 @@ static int h264_enc_encode(void *handle, int ret = 0; struct venc_h264_inst *inst = (struct venc_h264_inst *)handle; struct mtk_vcodec_ctx *ctx = inst->ctx; + bool is_single_core = (MTK_ENC_CORE_MODE(ctx) == VENC_SINGLE_CORE_MODE); mtk_vcodec_debug(inst, "opt %d ->", opt); - enable_irq(ctx->dev->enc_irq); + if (is_single_core) + enable_irq(ctx->dev->enc_irq); switch (opt) { case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: { @@ -666,6 +674,13 @@ static int h264_enc_encode(void *handle, unsigned int bs_size_hdr; unsigned int bs_size_frm; + /* + * the frm_buf and bs_buf need to be recorded into ctx, + * when encoding done, the target buffer can be obtained + */ + ctx->pfrm_buf[ctx->core_id] = frm_buf->src_addr; + ctx->pbs_buf[ctx->core_id] = bs_buf->buf; + if (!inst->prepend_hdr) { ret = h264_encode_frame(inst, frm_buf, bs_buf, &result->bs_size, ctx->core_id); @@ -702,6 +717,8 @@ static int h264_enc_encode(void *handle, result->bs_size = hdr_sz + filler_sz + bs_size_frm; + ctx->hdr_size = hdr_sz + filler_sz; + mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d", hdr_sz, filler_sz, bs_size_frm, result->bs_size); @@ -718,8 +735,9 @@ static int h264_enc_encode(void *handle, } encode_err: + if (is_single_core) + disable_irq(ctx->dev->enc_irq); - disable_irq(ctx->dev->enc_irq); mtk_vcodec_debug(inst, "opt %d <-", opt); return ret; diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h index 15e9a2ab9cda..3ad31abb7290 100644 --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h @@ -108,9 +108,11 @@ struct venc_frame_info { /* * struct venc_frm_buf - frame buffer information used in venc_if_encode() * @fb_addr: plane frame buffer addresses + * @src_addr: vb2_v4l2_buffer src buffer address */ struct venc_frm_buf { struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES]; + void *src_addr; }; /*