From patchwork Mon Jan 17 12:06:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 532743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBEDBC433EF for ; Mon, 17 Jan 2022 12:06:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239542AbiAQMG0 (ORCPT ); Mon, 17 Jan 2022 07:06:26 -0500 Received: from mailgw02.mediatek.com ([210.61.82.184]:36526 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S236706AbiAQMGZ (ORCPT ); Mon, 17 Jan 2022 07:06:25 -0500 X-UUID: 0308f1e1ef3345339cb562872c6f7905-20220117 X-UUID: 0308f1e1ef3345339cb562872c6f7905-20220117 Received: from mtkexhb01.mediatek.inc [(172.21.101.102)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1375964619; Mon, 17 Jan 2022 20:06:22 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs07n2.mediatek.inc (172.21.101.141) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 17 Jan 2022 20:06:20 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 17 Jan 2022 20:06:19 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu , CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH v2, 00/10] Enable two H264 encoder cores on MT8195 Date: Mon, 17 Jan 2022 20:06:05 +0800 Message-ID: <20220117120615.21687-1-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org MT8195 has two H264 encoder cores, they have their own power-domains, clocks, interrupts, register base. The two H264 encoder cores can work together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve 4K@60fps. The two encoder core encoding process looks like this: VENC Core0: frm#0....frm#2....frm#4.... VENC Core1: ..frm#1....frm#3....frm#5.... This series of patches are used to enable the two H264 encoder cores, encoding process will be changed: As-Is: Synchronous V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish ... To-Be: Asynchronous V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish (venc core0 may encode done here, done the encoding result to client) V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish. There is no "wait encoder IRQ" synchronous call during frame-racing mode encoding process, it can full use the two encoder cores to achieve higher performance. --- This series patches dependent on: [1]: the latest linux stage tree: https://git.linuxtv.org/media_stage.git mtk decoder patches [2]: https://patchwork.linuxtv.org/project/linux-media/list/?series=7105 [3]: https://patchwork.linuxtv.org/project/linux-media/list/?series=7131 new yaml included files [4]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=551641 [5]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579 --- --- changes compared with v1: - of_platform_populate was used in place of the component framework. - new yaml file for venc cores. - some modifications for patch v1's review comments. --- Irui Wang (10): media: mtk-vcodec: Use core type to indicate h264 and vp8 enc media: mtk-vcodec: export encoder functions dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195 media: mtk-vcodec: Enable venc dual core usage media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface media: mtk-vcodec: Add venc power on/off interface media: mtk-vcodec: Rewrite venc clock interface media: mtk-vcodec: Add more extra processing for dual-core mode media: mtk-vcodec: Add dual core mode encode process media: mtk-vcodec: Done encode result to client .../media/mediatek,vcodec-encoder-core.yaml | 214 +++++++++++++++++ drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 44 +++- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 109 ++++++--- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 7 +- .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 187 +++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 118 ++++++---- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 187 +++++++++++++-- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 11 +- .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 ++ .../platform/mtk-vcodec/mtk_vcodec_util.h | 5 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 216 +++++++++++++++--- .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 79 +++++-- .../media/platform/mtk-vcodec/venc_drv_if.h | 7 + .../media/platform/mtk-vcodec/venc_vpu_if.c | 10 +- .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- 18 files changed, 1097 insertions(+), 162 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h