| Message ID | 20220117120615.21687-1-irui.wang@mediatek.com |
|---|---|
| Headers | show |
| Series | Enable two H264 encoder cores on MT8195 | expand |
Dear Rob, Many thanks for your comments. On Mon, 2022-01-17 at 09:35 -0600, Rob Herring wrote: > On Mon, 17 Jan 2022 20:06:08 +0800, Irui Wang wrote: > > Adds encoder cores dt-bindings for mt8195 > > > > Signed-off-by: Irui Wang <irui.wang@mediatek.com> > > --- > > .../media/mediatek,vcodec-encoder-core.yaml | 214 > > ++++++++++++++++++ > > 1 file changed, 214 insertions(+) > > create mode 100644 > > Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- > > core.yaml > > > > My bot found errors running 'make DT_CHECKER_FLAGS=-m > dt_binding_check' > on your patch (DT_CHECKER_FLAGS is new in v5.13): > > yamllint warnings/errors: > > dtschema/dtc warnings/errors: > Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- > core.example.dts:20:18: fatal error: dt-bindings/memory/mt8195- > memory-port.h: No such file or directory > 20 | #include <dt-bindings/memory/mt8195-memory-port.h> > | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > compilation terminated. > make[1]: *** [scripts/Makefile.lib:373: > Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- > core.example.dt.yaml] Error 1 > make[1]: *** Waiting for unfinished jobs.... > make: *** [Makefile:1413: dt_binding_check] Error 2 > > doc reference errors (make refcheckdocs): > > See https://patchwork.ozlabs.org/patch/1580741 > > This check can fail if there are any dependencies. The base for a > patch > series is generally the most recent rc1. > > If you already ran 'make dt_binding_check' and didn't see the above > error(s), then make sure 'yamllint' is installed and dt-schema is up > to > date: > > pip3 install dtschema --upgrade > > Please check and re-submit. This new encoder yaml includes mt8195 iommu and power&clock node, so it depends on patches: iommu: https://patchwork.kernel.org/project/linux-mediatek/list/?series=551641 power: https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579 which are not accept yet. But IOMMU and power already send new version series patches, I will rebase this series in next version. Thanks BRs >
Dear all maintainers, Gently ping. Could you help to review this series of patches? I would be very grateful for any of your comments. Thanks Best Regards On Mon, 2022-01-17 at 20:06 +0800, Irui Wang wrote: > MT8195 has two H264 encoder cores, they have their own power-domains, > clocks, interrupts, register base. The two H264 encoder cores can > work > together to achieve higher performance, it's a core mode called > frame-racing, one core has 4K@30fps performance, two cores can > achieve > 4K@60fps. > The two encoder core encoding process looks like this: > > VENC Core0: frm#0....frm#2....frm#4.... > VENC Core1: ..frm#1....frm#3....frm#5.... > > This series of patches are used to enable the two H264 encoder cores, > encoding process will be changed: > As-Is: Synchronous > V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder > IRQ --> > encoding done with result --> job_finish > V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder > IRQ --> > encoding done with result --> job_finish > ... > > To-Be: Asynchronous > V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish > ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish > (venc core0 may encode done here, done the encoding result to client) > V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish. > > There is no "wait encoder IRQ" synchronous call during frame-racing > mode > encoding process, it can full use the two encoder cores to achieve > higher > performance. > > --- > This series patches dependent on: > [1]: the latest linux stage tree: > https://git.linuxtv.org/media_stage.git > > mtk decoder patches > [2]: > https://patchwork.linuxtv.org/project/linux-media/list/?series=7105 > [3]: > https://patchwork.linuxtv.org/project/linux-media/list/?series=7131 > > new yaml included files > [4]: > https://patchwork.kernel.org/project/linux-mediatek/list/?series=551641 > [5]: > https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579 > > --- > --- > changes compared with v1: > - of_platform_populate was used in place of the component framework. > - new yaml file for venc cores. > - some modifications for patch v1's review comments. > --- > > Irui Wang (10): > media: mtk-vcodec: Use core type to indicate h264 and vp8 enc > media: mtk-vcodec: export encoder functions > dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for > mt8195 > media: mtk-vcodec: Enable venc dual core usage > media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface > media: mtk-vcodec: Add venc power on/off interface > media: mtk-vcodec: Rewrite venc clock interface > media: mtk-vcodec: Add more extra processing for dual-core mode > media: mtk-vcodec: Add dual core mode encode process > media: mtk-vcodec: Done encode result to client > > .../media/mediatek,vcodec-encoder-core.yaml | 214 > +++++++++++++++++ > drivers/media/platform/mtk-vcodec/Makefile | 4 +- > .../platform/mtk-vcodec/mtk_vcodec_drv.h | 44 +++- > .../platform/mtk-vcodec/mtk_vcodec_enc.c | 109 ++++++--- > .../platform/mtk-vcodec/mtk_vcodec_enc.h | 7 +- > .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 187 +++++++++++++++ > .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++ > .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 118 ++++++---- > .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 187 +++++++++++++-- > .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 11 +- > .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 ++ > .../platform/mtk-vcodec/mtk_vcodec_util.h | 5 + > .../platform/mtk-vcodec/venc/venc_h264_if.c | 216 +++++++++++++++- > -- > .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- > .../media/platform/mtk-vcodec/venc_drv_if.c | 79 +++++-- > .../media/platform/mtk-vcodec/venc_drv_if.h | 7 + > .../media/platform/mtk-vcodec/venc_vpu_if.c | 10 +- > .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- > 18 files changed, 1097 insertions(+), 162 deletions(-) > create mode 100644 > Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- > core.yaml > create mode 100644 drivers/media/platform/mtk- > vcodec/mtk_vcodec_enc_core.c > create mode 100644 drivers/media/platform/mtk- > vcodec/mtk_vcodec_enc_core.h >
Il 17/01/22 13:06, Irui Wang ha scritto: > Adds encoder cores dt-bindings for mt8195 > > Signed-off-by: Irui Wang <irui.wang@mediatek.com> > --- > .../media/mediatek,vcodec-encoder-core.yaml | 214 ++++++++++++++++++ > 1 file changed, 214 insertions(+) > create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml > > diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml > new file mode 100644 > index 000000000000..d1e7bfa50bce > --- /dev/null > +++ b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml > @@ -0,0 +1,214 @@ > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > + > +%YAML 1.2 > +--- > +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#" > +$schema: "http://devicetree.org/meta-schemas/core.yaml#" > + > +title: Mediatek Video Encoder Accelerator With Multi Core > + > +maintainers: > + - Irui Wang <irui.wang@mediatek.com> > + > +description: | > + Mediatek Video Encode is the video encode hardware present in Mediatek > + SoCs which supports high resolution encoding functionalities. Required > + parent and child device node. > + > +properties: > + compatible: > + const: mediatek,mt8195-vcodec-enc > + > + mediatek,scp: > + $ref: /schemas/types.yaml#/definitions/phandle > + maxItems: 1 > + description: | > + The node of system control processor (SCP), using > + the remoteproc & rpmsg framework. > + > + iommus: > + minItems: 1 > + maxItems: 32 > + description: | > + List of the hardware port in respective IOMMU block for current Socs. > + Refer to bindings/iommu/mediatek,iommu.yaml. > + > + dma-ranges: > + maxItems: 1 > + description: | > + Describes the physical address space of IOMMU maps to memory. > + > + "#address-cells": > + const: 1 > + > + "#size-cells": > + const: 1 > + > + ranges: true > + > +# Required child node: > +patternProperties: > + "venc_core0@1a020000": This should be a pattern, venc-coreN where N is a number between 0 and 9, and any iostart should be allowed, not just 1a020000; doing it that way allows you to do a spec for all the core subnodes in one. > + type: object > + > + properties: > + compatible: > + const: mediatek,mtk-venc-core0 > + ..snip.. > + > +required: > + - compatible > + - mediatek,scp > + - iommus > + - dma-ranges > + - ranges > + > +additionalProperties: false > + > +examples: > + - | > + #include <dt-bindings/interrupt-controller/arm-gic.h> > + #include <dt-bindings/memory/mt8195-memory-port.h> > + #include <dt-bindings/interrupt-controller/irq.h> > + #include <dt-bindings/clock/mt8195-clk.h> > + #include <dt-bindings/power/mt8195-power.h> > + > + venc { > + compatible = "mediatek,mt8195-vcodec-enc"; > + mediatek,scp = <&scp>; > + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>; > + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; > + #address-cells = <1>; > + #size-cells = <1>; > + ranges; What about doing instead... #address-cells = <2>; #size-cells = <2>; (or fix your dma-ranges property) and... ranges = <0 0 0 0x1a020000 0 0x1020000>; venc-core0@0 { compatible = "mediatek,mtk-venc-hw"; reg = <0 0 0 0x10000>; mediatek,hw-leader; ..... other properties ..... }; venc-core1@1000000 { compatible = "mediatek,mtk-venc-hw"; reg = <0 0x1000000 0 0x10000>; ..... other properties ..... } Regards, Angelo > + > + venc_core0@1a020000 { > + compatible = "mediatek,mtk-venc-core0"; > + reg = <0x1a020000 0x10000>; > + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, > + <&iommu_vdo M4U_PORT_L19_VENC_REC>, > + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, > + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, > + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, > + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, > + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, > + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, > + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; > + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; > + clocks = <&vencsys CLK_VENC_VENC>; > + clock-names = "MT_CG_VENC0"; > + assigned-clocks = <&topckgen CLK_TOP_VENC>; > + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; > + power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; > + }; > + > + venc_core1@1b020000 { > + compatible = "mediatek,mtk-venc-core1"; > + reg = <0x1b020000 0x10000>; > + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, > + <&iommu_vpp M4U_PORT_L20_VENC_REC>, > + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, > + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, > + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, > + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, > + <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>, > + <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, > + <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>; > + interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; > + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; > + clock-names = "MT_CG_VENC1"; > + assigned-clocks = <&topckgen CLK_TOP_VENC>; > + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; > + power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>; > + }; > + };
MT8195 has two H264 encoder cores, they have their own power-domains, clocks, interrupts, register base. The two H264 encoder cores can work together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve 4K@60fps. The two encoder core encoding process looks like this: VENC Core0: frm#0....frm#2....frm#4.... VENC Core1: ..frm#1....frm#3....frm#5.... This series of patches are used to enable the two H264 encoder cores, encoding process will be changed: As-Is: Synchronous V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish ... To-Be: Asynchronous V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish (venc core0 may encode done here, done the encoding result to client) V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish. There is no "wait encoder IRQ" synchronous call during frame-racing mode encoding process, it can full use the two encoder cores to achieve higher performance. --- This series patches dependent on: [1]: the latest linux stage tree: https://git.linuxtv.org/media_stage.git mtk decoder patches [2]: https://patchwork.linuxtv.org/project/linux-media/list/?series=7105 [3]: https://patchwork.linuxtv.org/project/linux-media/list/?series=7131 new yaml included files [4]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=551641 [5]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579 --- --- changes compared with v1: - of_platform_populate was used in place of the component framework. - new yaml file for venc cores. - some modifications for patch v1's review comments. --- Irui Wang (10): media: mtk-vcodec: Use core type to indicate h264 and vp8 enc media: mtk-vcodec: export encoder functions dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195 media: mtk-vcodec: Enable venc dual core usage media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface media: mtk-vcodec: Add venc power on/off interface media: mtk-vcodec: Rewrite venc clock interface media: mtk-vcodec: Add more extra processing for dual-core mode media: mtk-vcodec: Add dual core mode encode process media: mtk-vcodec: Done encode result to client .../media/mediatek,vcodec-encoder-core.yaml | 214 +++++++++++++++++ drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 44 +++- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 109 ++++++--- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 7 +- .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 187 +++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 118 ++++++---- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 187 +++++++++++++-- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 11 +- .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 ++ .../platform/mtk-vcodec/mtk_vcodec_util.h | 5 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 216 +++++++++++++++--- .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 79 +++++-- .../media/platform/mtk-vcodec/venc_drv_if.h | 7 + .../media/platform/mtk-vcodec/venc_vpu_if.c | 10 +- .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- 18 files changed, 1097 insertions(+), 162 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h