From patchwork Tue Apr 27 11:15:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Courbot X-Patchwork-Id: 428131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBD94C433B4 for ; Tue, 27 Apr 2021 11:15:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 96225613E8 for ; Tue, 27 Apr 2021 11:15:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235809AbhD0LQa (ORCPT ); Tue, 27 Apr 2021 07:16:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39786 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230270AbhD0LQ0 (ORCPT ); Tue, 27 Apr 2021 07:16:26 -0400 Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1C09C061760 for ; Tue, 27 Apr 2021 04:15:42 -0700 (PDT) Received: by mail-pl1-x62d.google.com with SMTP id 20so26734687pll.7 for ; Tue, 27 Apr 2021 04:15:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4qZsQRsGuCyk4j1C/e+zC2HYXSpPpt7+LRIVR6EvA1Q=; b=MmztzcydDiCmIGZ5rBt4LY8NA386OT8abcppdLkMCrtROeyrELhMc+YEI6p+MbHyqw skitkOULGsK8x+AZwLLnSb46mFvajphr/pzpLnRrnVkBADs881RIBMsjsFebdSzIdka1 wHteEOG6wARXDxF9nYgYratiLYwoewW65fmzc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4qZsQRsGuCyk4j1C/e+zC2HYXSpPpt7+LRIVR6EvA1Q=; b=RDuTDvB13huLkCmsJoBxE7bxdnFaCLnH7wc39OcDTPJyMOteVmQZLe7xJ+Iw2/NeB2 M4OjENDxi9DQndLN8fdNyJE83M9VSFXn5uzN4KqZdlDM2u2ODH3D0/X9xXTt5Fb6BOZF 820/Y4vgAQouCgRXCHT5+0qV33BhApSDoE6dqLEPFq9G/yuq2h6jEDZKStUwjRhqvu1O iRYTqVeX+IiWZuwptX2zu73Rezez0/dfkYjO7YWIFiGUUxfTzioxcKtIM8hU1SzrijvV aDXrilFADzNytp/w/078s+ni4xEtNVub/iqq7k3ELRIlKz3KI7fjWcvQwOcFw1YwPCVt RzzA== X-Gm-Message-State: AOAM530NCmjju0pnNJTqJp2Sj+Lb+/4yI/ow7hidb97txG9m+2gHlcLU 3Rhb9cgaL58JryRO2bf0T5uHVg== X-Google-Smtp-Source: ABdhPJx1C2iIDlsCOU3glRSJBL8B286p5e+lxgbowpxikjRFr5/iSS83yGB8JZ9mAXiUCoL9ig6FmA== X-Received: by 2002:a17:90b:190d:: with SMTP id mp13mr5060186pjb.156.1619522142415; Tue, 27 Apr 2021 04:15:42 -0700 (PDT) Received: from acourbot.tok.corp.google.com ([2401:fa00:8f:203:b182:d363:32d0:53a2]) by smtp.gmail.com with ESMTPSA id c21sm2347210pfc.165.2021.04.27.04.15.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Apr 2021 04:15:42 -0700 (PDT) From: Alexandre Courbot To: Tiffany Lin , Andrew-CT Chen , Rob Herring , Yunfei Dong Cc: Mauro Carvalho Chehab , Hans Verkuil , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mediatek@lists.infradead.org, Alexandre Courbot Subject: [PATCH v4 02/15] media: mtk-vcodec: vdec: handle firmware version field Date: Tue, 27 Apr 2021 20:15:13 +0900 Message-Id: <20210427111526.1772293-3-acourbot@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210427111526.1772293-1-acourbot@chromium.org> References: <20210427111526.1772293-1-acourbot@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Firmwares for decoders newer than MT8173 will include an ABI version number in their initialization ack message. Add the capacity to manage it and make initialization fail if the firmware ABI is of a version that we don't support. For MT8173, this ABI version field does not exist ; thus ignore it on this chip. There should only be one firmware version available for it anyway. Signed-off-by: Alexandre Courbot --- .../mtk-vcodec/mtk_vcodec_dec_stateful.c | 1 + .../platform/mtk-vcodec/mtk_vcodec_drv.h | 4 ++++ .../media/platform/mtk-vcodec/vdec_ipi_msg.h | 5 +++++ .../media/platform/mtk-vcodec/vdec_vpu_if.c | 21 +++++++++++++++++-- 4 files changed, 29 insertions(+), 2 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c index f31f4f94001d..c5538fcfc9a5 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c @@ -640,6 +640,7 @@ static struct vb2_ops mtk_vdec_frame_vb2_ops = { }; const struct mtk_vcodec_dec_pdata mtk_vdec_8173_pdata = { + .chip = MTK_MT8173, .init_vdec_params = mtk_init_vdec_params, .ctrls_setup = mtk_vcodec_dec_ctrls_setup, .vdec_vb2_ops = &mtk_vdec_frame_vb2_ops, diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h index 85af0bffc7be..e5b8309785e1 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h @@ -321,6 +321,8 @@ enum mtk_chip { * @vdec_framesizes: supported video decoder frame sizes * @num_framesizes: count of video decoder frame sizes * + * @chip: chip this decoder is compatible with + * * @uses_stateless_api: whether the decoder uses the stateless API with requests */ @@ -340,6 +342,8 @@ struct mtk_vcodec_dec_pdata { const struct mtk_codec_framesizes *vdec_framesizes; const int num_framesizes; + enum mtk_chip chip; + bool uses_stateless_api; }; diff --git a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h index 47a1c1c0fd04..eb66729fda63 100644 --- a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h +++ b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h @@ -83,12 +83,17 @@ struct vdec_ap_ipi_dec_start { * @status : VPU exeuction result * @ap_inst_addr : AP vcodec_vpu_inst instance address * @vpu_inst_addr : VPU decoder instance address + * @vdec_abi_version: ABI version of the firmware. Kernel can use it to + * ensure that it is compatible with the firmware. + * This field is not valid for MT8173 and must not be + * accessed for this chip. */ struct vdec_vpu_ipi_init_ack { uint32_t msg_id; int32_t status; uint64_t ap_inst_addr; uint32_t vpu_inst_addr; + uint32_t vdec_abi_version; }; #endif diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c index 58b0e6fa8fd2..203089213e67 100644 --- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c +++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c @@ -24,6 +24,22 @@ static void handle_init_ack_msg(const struct vdec_vpu_ipi_init_ack *msg) vpu->inst_addr = msg->vpu_inst_addr; mtk_vcodec_debug(vpu, "- vpu_inst_addr = 0x%x", vpu->inst_addr); + + /* Firmware version field does not exist on MT8173. */ + if (vpu->ctx->dev->vdec_pdata->chip == MTK_MT8173) + return; + + /* Check firmware version. */ + mtk_vcodec_debug(vpu, "firmware version 0x%x\n", msg->vdec_abi_version); + switch (msg->vdec_abi_version) { + case 1: + break; + default: + mtk_vcodec_err(vpu, "unhandled firmware version 0x%x\n", + msg->vdec_abi_version); + vpu->failure = 1; + break; + } } /* @@ -44,6 +60,9 @@ static void vpu_dec_ipi_handler(void *data, unsigned int len, void *priv) mtk_vcodec_debug(vpu, "+ id=%X", msg->msg_id); + vpu->failure = msg->status; + vpu->signaled = 1; + if (msg->status == 0) { switch (msg->msg_id) { case VPU_IPIMSG_DEC_INIT_ACK: @@ -63,8 +82,6 @@ static void vpu_dec_ipi_handler(void *data, unsigned int len, void *priv) } mtk_vcodec_debug(vpu, "- id=%X", msg->msg_id); - vpu->failure = msg->status; - vpu->signaled = 1; } static int vcodec_vpu_send_msg(struct vdec_vpu_inst *vpu, void *msg, int len) From patchwork Tue Apr 27 11:15:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Courbot X-Patchwork-Id: 428130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 981E8C43460 for ; Tue, 27 Apr 2021 11:15:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 684B9613E5 for ; Tue, 27 Apr 2021 11:15:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235889AbhD0LQd (ORCPT ); Tue, 27 Apr 2021 07:16:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39794 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235835AbhD0LQ2 (ORCPT ); Tue, 27 Apr 2021 07:16:28 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7EF58C061574 for ; Tue, 27 Apr 2021 04:15:45 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id h20so30686493plr.4 for ; Tue, 27 Apr 2021 04:15:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8c3lQ6Ig+7BLf+08Bn0c/e0llvAY8LJA+7xZwUYw4VU=; b=BhjwcZH+gtk+Sx5PoZFo2H1P7vIWXRstinsxLjmJk90j3lmzIrSfAc6djN7YKmxNV0 5wcg8jq6Uyz2ASKGQJV6BRDDHV/oUZdcEhm7E0OdQoTbjIe9jZxL2rAPN8u4elnAlT8F OTE+OnZZcqY8TXQujM9TDqnEqTHzS9xndn8Ls= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8c3lQ6Ig+7BLf+08Bn0c/e0llvAY8LJA+7xZwUYw4VU=; b=A4bkbkwMxib4n85a/BVhjJZ69HfhdENdIvLbBeg6v+Zm5v5gOJJ8H5TpsPueKOxRcW NyvSjMWWpcnA7Ay3erj+LIiBsZHhbBn1IDN+6kvKBWhHifqGEYhKuwyyavquXhReIaj2 TbpaEJIo948PsAyBJuwhZX4sC+Tepoeuk2rZr4YMU3+tU1kiWuY2y/8lZrYpGKhytXYf OnPOOal1emkKuaJCF+sgv1/N+CcHLS3pJNfypgK+K7ajk98VtggS+OKhuU7iWxcjng/5 NkT11h5wgWxjdj+LkKcTxrnb/ZVWL7oOpnc/4IOjPcpgITMRz0M2ROkXkzOlEhKraBam sZ2Q== X-Gm-Message-State: AOAM533geb9s9yeW+b7vxK9s7rf2Gan++Z+x7HjtxYCYZ5bVfKr0o6vb pNhwTAvO3GDb6mTrWrEJ0o88ug== X-Google-Smtp-Source: ABdhPJxQ/s/IDGETgz+RzGhkqjW4D9BbGwOcOqhMFHXFHYPnvqa3s7P75umaBg0ty+lSHJnGLjPz6g== X-Received: by 2002:a17:90a:8a12:: with SMTP id w18mr18652526pjn.26.1619522145117; Tue, 27 Apr 2021 04:15:45 -0700 (PDT) Received: from acourbot.tok.corp.google.com ([2401:fa00:8f:203:b182:d363:32d0:53a2]) by smtp.gmail.com with ESMTPSA id c21sm2347210pfc.165.2021.04.27.04.15.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Apr 2021 04:15:44 -0700 (PDT) From: Alexandre Courbot To: Tiffany Lin , Andrew-CT Chen , Rob Herring , Yunfei Dong Cc: Mauro Carvalho Chehab , Hans Verkuil , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mediatek@lists.infradead.org, Alexandre Courbot Subject: [PATCH v4 03/15] media: mtk-vcodec: support version 2 of decoder firmware ABI Date: Tue, 27 Apr 2021 20:15:14 +0900 Message-Id: <20210427111526.1772293-4-acourbot@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210427111526.1772293-1-acourbot@chromium.org> References: <20210427111526.1772293-1-acourbot@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add support for decoder firmware version 2, which makes the kernel responsible for managing the VSI context and is used for stateless codecs. Signed-off-by: Alexandre Courbot --- .../media/platform/mtk-vcodec/vdec_ipi_msg.h | 18 +++++++++--- .../media/platform/mtk-vcodec/vdec_vpu_if.c | 28 +++++++++++++++---- .../media/platform/mtk-vcodec/vdec_vpu_if.h | 5 ++++ 3 files changed, 42 insertions(+), 9 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h index eb66729fda63..a0e773ae3ab3 100644 --- a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h +++ b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h @@ -29,11 +29,15 @@ enum vdec_ipi_msgid { /** * struct vdec_ap_ipi_cmd - generic AP to VPU ipi command format * @msg_id : vdec_ipi_msgid - * @vpu_inst_addr : VPU decoder instance address + * @vpu_inst_addr : VPU decoder instance address. Used if ABI version < 2. + * @inst_id : instance ID. Used if the ABI version >= 2. */ struct vdec_ap_ipi_cmd { uint32_t msg_id; - uint32_t vpu_inst_addr; + union { + uint32_t vpu_inst_addr; + uint32_t inst_id; + }; }; /** @@ -63,7 +67,8 @@ struct vdec_ap_ipi_init { /** * struct vdec_ap_ipi_dec_start - for AP_IPIMSG_DEC_START * @msg_id : AP_IPIMSG_DEC_START - * @vpu_inst_addr : VPU decoder instance address + * @vpu_inst_addr : VPU decoder instance address. Used if ABI version < 2. + * @inst_id : instance ID. Used if the ABI version >= 2. * @data : Header info * H264 decoder [0]:buf_sz [1]:nal_start * VP8 decoder [0]:width/height @@ -72,7 +77,10 @@ struct vdec_ap_ipi_init { */ struct vdec_ap_ipi_dec_start { uint32_t msg_id; - uint32_t vpu_inst_addr; + union { + uint32_t vpu_inst_addr; + uint32_t inst_id; + }; uint32_t data[3]; uint32_t reserved; }; @@ -87,6 +95,7 @@ struct vdec_ap_ipi_dec_start { * ensure that it is compatible with the firmware. * This field is not valid for MT8173 and must not be * accessed for this chip. + * @inst_id : instance ID. Valid only if the ABI version >= 2. */ struct vdec_vpu_ipi_init_ack { uint32_t msg_id; @@ -94,6 +103,7 @@ struct vdec_vpu_ipi_init_ack { uint64_t ap_inst_addr; uint32_t vpu_inst_addr; uint32_t vdec_abi_version; + uint32_t inst_id; }; #endif diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c index 203089213e67..5dffc459a33d 100644 --- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c +++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c @@ -25,18 +25,30 @@ static void handle_init_ack_msg(const struct vdec_vpu_ipi_init_ack *msg) mtk_vcodec_debug(vpu, "- vpu_inst_addr = 0x%x", vpu->inst_addr); + /* Set default ABI version if dealing with unversioned firmware. */ + vpu->fw_abi_version = 0; + /* + * Instance ID is only used if ABI version >= 2. Initialize it with + * garbage by default. + */ + vpu->inst_id = 0xdeadbeef; + /* Firmware version field does not exist on MT8173. */ if (vpu->ctx->dev->vdec_pdata->chip == MTK_MT8173) return; /* Check firmware version. */ - mtk_vcodec_debug(vpu, "firmware version 0x%x\n", msg->vdec_abi_version); - switch (msg->vdec_abi_version) { + vpu->fw_abi_version = msg->vdec_abi_version; + mtk_vcodec_debug(vpu, "firmware version 0x%x\n", vpu->fw_abi_version); + switch (vpu->fw_abi_version) { case 1: break; + case 2: + vpu->inst_id = msg->inst_id; + break; default: mtk_vcodec_err(vpu, "unhandled firmware version 0x%x\n", - msg->vdec_abi_version); + vpu->fw_abi_version); vpu->failure = 1; break; } @@ -113,7 +125,10 @@ static int vcodec_send_ap_ipi(struct vdec_vpu_inst *vpu, unsigned int msg_id) memset(&msg, 0, sizeof(msg)); msg.msg_id = msg_id; - msg.vpu_inst_addr = vpu->inst_addr; + if (vpu->fw_abi_version < 2) + msg.vpu_inst_addr = vpu->inst_addr; + else + msg.inst_id = vpu->inst_id; err = vcodec_vpu_send_msg(vpu, &msg, sizeof(msg)); mtk_vcodec_debug(vpu, "- id=%X ret=%d", msg_id, err); @@ -163,7 +178,10 @@ int vpu_dec_start(struct vdec_vpu_inst *vpu, uint32_t *data, unsigned int len) memset(&msg, 0, sizeof(msg)); msg.msg_id = AP_IPIMSG_DEC_START; - msg.vpu_inst_addr = vpu->inst_addr; + if (vpu->fw_abi_version < 2) + msg.vpu_inst_addr = vpu->inst_addr; + else + msg.inst_id = vpu->inst_id; for (i = 0; i < len; i++) msg.data[i] = data[i]; diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h index 85224eb7e34b..c2ed5b6cab8b 100644 --- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h +++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h @@ -18,6 +18,9 @@ struct mtk_vcodec_ctx; * for control and info share * @failure : VPU execution result status, 0: success, others: fail * @inst_addr : VPU decoder instance address + * @fw_abi_version : ABI version of the firmware. + * @inst_id : if fw_abi_version >= 2, contains the instance ID to be given + * in place of inst_addr in messages. * @signaled : 1 - Host has received ack message from VPU, 0 - not received * @ctx : context for v4l2 layer integration * @dev : platform device of VPU @@ -29,6 +32,8 @@ struct vdec_vpu_inst { void *vsi; int32_t failure; uint32_t inst_addr; + uint32_t fw_abi_version; + uint32_t inst_id; unsigned int signaled; struct mtk_vcodec_ctx *ctx; wait_queue_head_t wq; From patchwork Tue Apr 27 11:15:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Courbot X-Patchwork-Id: 428129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4E21C433ED for ; Tue, 27 Apr 2021 11:16:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B05FF613DE for ; Tue, 27 Apr 2021 11:16:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235942AbhD0LQp (ORCPT ); Tue, 27 Apr 2021 07:16:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39840 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235906AbhD0LQi (ORCPT ); Tue, 27 Apr 2021 07:16:38 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BDF8C061756 for ; Tue, 27 Apr 2021 04:15:54 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id v13so17086617ple.9 for ; Tue, 27 Apr 2021 04:15:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=32Rb3qxtDBiHk1MQmf6EtybjRS+50jHcEMgcLRPgxfc=; b=P4KWBGYolxKKFIg+qx2Wd9R9PjowUrNKq6VzCAo63CmCdOenCLV3dD1ND0fW+Sx+TX zNIi9M++EwQIbLx7OoYM3uMGUnxiOzUY0Fo4ynd8spTfRpbxEaH7p6Yls9fBV/OAw+jq vYY/DRxvalO2ITKMlVJNPxDFtHZNAEtixUWnU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=32Rb3qxtDBiHk1MQmf6EtybjRS+50jHcEMgcLRPgxfc=; b=bb45JZ9Bq734Bhz6N6wXoLJOGD+e3xx0C+ebxjZWxXDfuKkd8K0j2mJPoeSCIgBwS+ 8ZzqZ+nFT2Z8MFi7Qbkfo3QBCdjgzSeu9zLPb/nOB5yP5y5FlXZiTZqP4ixALymsQD5v BlVa6n1ER9PudF1W6mlw6blhb0LwaHSdRx1AADkOvXTg6IoNG/KgRPDcNaN4ztLDk2C8 /6aSzTgtsxzU+IlHfvPr9XgJL2jsnP787DbuumGIppWR9hA9327vroLPXXQat0ktCJx0 O9qeYoSpO93oK4LpGnbGNPKCnXcSvJGbsiKbYZ/5k9D67Cbu+swzn10iWJCf7+DQL19H Memg== X-Gm-Message-State: AOAM533IZsLPTQosOkq5mWSev8ZWWpjbiGPyaN8PZ3o1Al+DEuzFADJV arHXzOhP70VF2p6Tx+JB6Ro85A== X-Google-Smtp-Source: ABdhPJyShK7VK3qeiPfpKiObkB4XPkSzOAsQ0vzdRvTX3w7SEx1fnaTRjcIqmiZIKFKbsVRJCdVJEA== X-Received: by 2002:a17:902:8688:b029:ea:f782:5eae with SMTP id g8-20020a1709028688b02900eaf7825eaemr23595536plo.15.1619522153523; Tue, 27 Apr 2021 04:15:53 -0700 (PDT) Received: from acourbot.tok.corp.google.com ([2401:fa00:8f:203:b182:d363:32d0:53a2]) by smtp.gmail.com with ESMTPSA id c21sm2347210pfc.165.2021.04.27.04.15.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Apr 2021 04:15:53 -0700 (PDT) From: Alexandre Courbot To: Tiffany Lin , Andrew-CT Chen , Rob Herring , Yunfei Dong Cc: Mauro Carvalho Chehab , Hans Verkuil , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mediatek@lists.infradead.org, Alexandre Courbot Subject: [PATCH v4 06/15] media: mtk-vcodec: vdec: support stateless H.264 decoding Date: Tue, 27 Apr 2021 20:15:17 +0900 Message-Id: <20210427111526.1772293-7-acourbot@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210427111526.1772293-1-acourbot@chromium.org> References: <20210427111526.1772293-1-acourbot@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Yunfei Dong Add support for H.264 decoding using the stateless API, as supported by MT8183. This support takes advantage of the V4L2 H.264 reference list builders. Signed-off-by: Yunfei Dong [acourbot: refactor, cleanup and split] Co-developed-by: Alexandre Courbot Signed-off-by: Alexandre Courbot --- drivers/media/platform/Kconfig | 1 + drivers/media/platform/mtk-vcodec/Makefile | 1 + .../mtk-vcodec/vdec/vdec_h264_req_if.c | 780 ++++++++++++++++++ .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 + .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 + 5 files changed, 786 insertions(+) create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index 157c924686e4..ae1468aa1b4e 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -314,6 +314,7 @@ config VIDEO_MEDIATEK_VCODEC select V4L2_MEM2MEM_DEV select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP + select V4L2_H264 help Mediatek video codec driver provides HW capability to encode and decode in a range of video formats on MT8173 diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile index 4ba93d838ab6..ca8e9e7a9c4e 100644 --- a/drivers/media/platform/mtk-vcodec/Makefile +++ b/drivers/media/platform/mtk-vcodec/Makefile @@ -7,6 +7,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \ vdec/vdec_vp9_if.o \ + vdec/vdec_h264_req_if.o \ mtk_vcodec_dec_drv.o \ vdec_drv_if.o \ vdec_vpu_if.o \ diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c new file mode 100644 index 000000000000..8ec8da469ef3 --- /dev/null +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c @@ -0,0 +1,780 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include +#include +#include + +#include "../vdec_drv_if.h" +#include "../mtk_vcodec_util.h" +#include "../mtk_vcodec_dec.h" +#include "../mtk_vcodec_intr.h" +#include "../vdec_vpu_if.h" +#include "../vdec_drv_base.h" + +#define BUF_PREDICTION_SZ (64 * 4096) +#define MB_UNIT_LEN 16 + +/* get used parameters for sps/pps */ +#define GET_MTK_VDEC_FLAG(cond, flag) \ + { dst_param->cond = ((src_param->flags & flag) ? (1) : (0)); } +#define GET_MTK_VDEC_PARAM(param) \ + { dst_param->param = src_param->param; } +/* motion vector size (bytes) for every macro block */ +#define HW_MB_STORE_SZ 64 + +#define H264_MAX_FB_NUM 17 +#define H264_MAX_MV_NUM 32 +#define HDR_PARSING_BUF_SZ 1024 + +/** + * struct mtk_h264_dpb_info - h264 dpb information + * @y_dma_addr: Y bitstream physical address + * @c_dma_addr: CbCr bitstream physical address + * @reference_flag: reference picture flag (short/long term reference picture) + * @field: field picture flag + */ +struct mtk_h264_dpb_info { + dma_addr_t y_dma_addr; + dma_addr_t c_dma_addr; + int reference_flag; + int field; +}; + +/** + * struct mtk_h264_sps_param - parameters for sps + */ +struct mtk_h264_sps_param { + unsigned char chroma_format_idc; + unsigned char bit_depth_luma_minus8; + unsigned char bit_depth_chroma_minus8; + unsigned char log2_max_frame_num_minus4; + unsigned char pic_order_cnt_type; + unsigned char log2_max_pic_order_cnt_lsb_minus4; + unsigned char max_num_ref_frames; + unsigned char separate_colour_plane_flag; + unsigned short pic_width_in_mbs_minus1; + unsigned short pic_height_in_map_units_minus1; + unsigned int max_frame_nums; + unsigned char qpprime_y_zero_transform_bypass_flag; + unsigned char delta_pic_order_always_zero_flag; + unsigned char frame_mbs_only_flag; + unsigned char mb_adaptive_frame_field_flag; + unsigned char direct_8x8_inference_flag; + unsigned char reserved[3]; +}; + +/** + * struct mtk_h264_pps_param - parameters for pps + */ +struct mtk_h264_pps_param { + unsigned char num_ref_idx_l0_default_active_minus1; + unsigned char num_ref_idx_l1_default_active_minus1; + unsigned char weighted_bipred_idc; + char pic_init_qp_minus26; + char chroma_qp_index_offset; + char second_chroma_qp_index_offset; + unsigned char entropy_coding_mode_flag; + unsigned char pic_order_present_flag; + unsigned char deblocking_filter_control_present_flag; + unsigned char constrained_intra_pred_flag; + unsigned char weighted_pred_flag; + unsigned char redundant_pic_cnt_present_flag; + unsigned char transform_8x8_mode_flag; + unsigned char scaling_matrix_present_flag; + unsigned char reserved[2]; +}; + +struct slice_api_h264_scaling_matrix { + unsigned char scaling_list_4x4[6][16]; + unsigned char scaling_list_8x8[6][64]; +}; + +struct slice_h264_dpb_entry { + unsigned long long reference_ts; + unsigned short frame_num; + unsigned short pic_num; + /* Note that field is indicated by v4l2_buffer.field */ + int top_field_order_cnt; + int bottom_field_order_cnt; + unsigned int flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */ +}; + +/** + * struct slice_api_h264_decode_param - parameters for decode. + */ +struct slice_api_h264_decode_param { + struct slice_h264_dpb_entry dpb[16]; + unsigned short num_slices; + unsigned short nal_ref_idc; + unsigned char ref_pic_list_p0[32]; + unsigned char ref_pic_list_b0[32]; + unsigned char ref_pic_list_b1[32]; + int top_field_order_cnt; + int bottom_field_order_cnt; + unsigned int flags; /* V4L2_H264_DECODE_PARAM_FLAG_* */ +}; + +/** + * struct mtk_h264_dec_slice_param - parameters for decode current frame + */ +struct mtk_h264_dec_slice_param { + struct mtk_h264_sps_param sps; + struct mtk_h264_pps_param pps; + struct slice_api_h264_scaling_matrix scaling_matrix; + struct slice_api_h264_decode_param decode_params; + struct mtk_h264_dpb_info h264_dpb_info[16]; +}; + +/** + * struct h264_fb - h264 decode frame buffer information + * @vdec_fb_va : virtual address of struct vdec_fb + * @y_fb_dma : dma address of Y frame buffer (luma) + * @c_fb_dma : dma address of C frame buffer (chroma) + * @poc : picture order count of frame buffer + * @reserved : for 8 bytes alignment + */ +struct h264_fb { + uint64_t vdec_fb_va; + uint64_t y_fb_dma; + uint64_t c_fb_dma; + int32_t poc; + uint32_t reserved; +}; + +/** + * struct vdec_h264_dec_info - decode information + * @dpb_sz : decoding picture buffer size + * @resolution_changed : resoltion change happen + * @realloc_mv_buf : flag to notify driver to re-allocate mv buffer + * @cap_num_planes : number planes of capture buffer + * @bs_dma : Input bit-stream buffer dma address + * @y_fb_dma : Y frame buffer dma address + * @c_fb_dma : C frame buffer dma address + * @vdec_fb_va : VDEC frame buffer struct virtual address + */ +struct vdec_h264_dec_info { + uint32_t dpb_sz; + uint32_t resolution_changed; + uint32_t realloc_mv_buf; + uint32_t cap_num_planes; + uint64_t bs_dma; + uint64_t y_fb_dma; + uint64_t c_fb_dma; + uint64_t vdec_fb_va; +}; + +/** + * struct vdec_h264_vsi - shared memory for decode information exchange + * between VPU and Host. + * The memory is allocated by VPU then mapping to Host + * in vpu_dec_init() and freed in vpu_dec_deinit() + * by VPU. + * AP-W/R : AP is writer/reader on this item + * VPU-W/R: VPU is write/reader on this item + * @pred_buf_dma : HW working predication buffer dma address (AP-W, VPU-R) + * @mv_buf_dma : HW working motion vector buffer dma address (AP-W, VPU-R) + * @dec : decode information (AP-R, VPU-W) + * @pic : picture information (AP-R, VPU-W) + * @crop : crop information (AP-R, VPU-W) + */ +struct vdec_h264_vsi { + uint64_t pred_buf_dma; + uint64_t mv_buf_dma[H264_MAX_MV_NUM]; + struct vdec_h264_dec_info dec; + struct vdec_pic_info pic; + struct v4l2_rect crop; + struct mtk_h264_dec_slice_param h264_slice_params; +}; + +/** + * struct vdec_h264_slice_inst - h264 decoder instance + * @num_nalu : how many nalus be decoded + * @ctx : point to mtk_vcodec_ctx + * @pred_buf : HW working predication buffer + * @mv_buf : HW working motion vector buffer + * @vpu : VPU instance + * @vsi_ctx : Local VSI data for this decoding context + */ +struct vdec_h264_slice_inst { + unsigned int num_nalu; + struct mtk_vcodec_ctx *ctx; + struct mtk_vcodec_mem pred_buf; + struct mtk_vcodec_mem mv_buf[H264_MAX_MV_NUM]; + struct vdec_vpu_inst vpu; + struct vdec_h264_vsi vsi_ctx; + struct mtk_h264_dec_slice_param h264_slice_param; + + struct v4l2_h264_dpb_entry dpb[16]; +}; + +static void *get_ctrl_ptr(struct mtk_vcodec_ctx *ctx, + int id) +{ + struct v4l2_ctrl *ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, id); + + return ctrl->p_cur.p; +} + +static void get_h264_dpb_list(struct vdec_h264_slice_inst *inst, + struct mtk_h264_dec_slice_param *slice_param) +{ + struct vb2_queue *vq; + struct vb2_buffer *vb; + struct vb2_v4l2_buffer *vb2_v4l2; + u64 index; + + vq = v4l2_m2m_get_vq(inst->ctx->m2m_ctx, + V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + + for (index = 0; index < ARRAY_SIZE(slice_param->decode_params.dpb); index++) { + const struct slice_h264_dpb_entry *dpb; + int vb2_index; + + dpb = &slice_param->decode_params.dpb[index]; + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) { + slice_param->h264_dpb_info[index].reference_flag = 0; + continue; + } + + vb2_index = vb2_find_timestamp(vq, dpb->reference_ts, 0); + if (vb2_index < 0) { + mtk_vcodec_err(inst, "Reference invalid: dpb_index(%lld) reference_ts(%lld)", + index, dpb->reference_ts); + continue; + } + /* 1 for short term reference, 2 for long term reference */ + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)) + slice_param->h264_dpb_info[index].reference_flag = 1; + else + slice_param->h264_dpb_info[index].reference_flag = 2; + + vb = vq->bufs[vb2_index]; + vb2_v4l2 = container_of(vb, struct vb2_v4l2_buffer, vb2_buf); + slice_param->h264_dpb_info[index].field = vb2_v4l2->field; + + slice_param->h264_dpb_info[index].y_dma_addr = + vb2_dma_contig_plane_dma_addr(vb, 0); + if (inst->ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 2) { + slice_param->h264_dpb_info[index].c_dma_addr = + vb2_dma_contig_plane_dma_addr(vb, 1); + } + } +} + +static void get_h264_sps_parameters(struct mtk_h264_sps_param *dst_param, + const struct v4l2_ctrl_h264_sps *src_param) +{ + GET_MTK_VDEC_PARAM(chroma_format_idc); + GET_MTK_VDEC_PARAM(bit_depth_luma_minus8); + GET_MTK_VDEC_PARAM(bit_depth_chroma_minus8); + GET_MTK_VDEC_PARAM(log2_max_frame_num_minus4); + GET_MTK_VDEC_PARAM(pic_order_cnt_type); + GET_MTK_VDEC_PARAM(log2_max_pic_order_cnt_lsb_minus4); + GET_MTK_VDEC_PARAM(max_num_ref_frames); + GET_MTK_VDEC_PARAM(pic_width_in_mbs_minus1); + GET_MTK_VDEC_PARAM(pic_height_in_map_units_minus1); + + GET_MTK_VDEC_FLAG(separate_colour_plane_flag, + V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE); + GET_MTK_VDEC_FLAG(qpprime_y_zero_transform_bypass_flag, + V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS); + GET_MTK_VDEC_FLAG(delta_pic_order_always_zero_flag, + V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO); + GET_MTK_VDEC_FLAG(frame_mbs_only_flag, + V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY); + GET_MTK_VDEC_FLAG(mb_adaptive_frame_field_flag, + V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD); + GET_MTK_VDEC_FLAG(direct_8x8_inference_flag, + V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE); +} + +static void get_h264_pps_parameters(struct mtk_h264_pps_param *dst_param, + const struct v4l2_ctrl_h264_pps *src_param) +{ + GET_MTK_VDEC_PARAM(num_ref_idx_l0_default_active_minus1); + GET_MTK_VDEC_PARAM(num_ref_idx_l1_default_active_minus1); + GET_MTK_VDEC_PARAM(weighted_bipred_idc); + GET_MTK_VDEC_PARAM(pic_init_qp_minus26); + GET_MTK_VDEC_PARAM(chroma_qp_index_offset); + GET_MTK_VDEC_PARAM(second_chroma_qp_index_offset); + + GET_MTK_VDEC_FLAG(entropy_coding_mode_flag, + V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE); + GET_MTK_VDEC_FLAG(pic_order_present_flag, + V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT); + GET_MTK_VDEC_FLAG(weighted_pred_flag, + V4L2_H264_PPS_FLAG_WEIGHTED_PRED); + GET_MTK_VDEC_FLAG(deblocking_filter_control_present_flag, + V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT); + GET_MTK_VDEC_FLAG(constrained_intra_pred_flag, + V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED); + GET_MTK_VDEC_FLAG(redundant_pic_cnt_present_flag, + V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT); + GET_MTK_VDEC_FLAG(transform_8x8_mode_flag, + V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE); + GET_MTK_VDEC_FLAG(scaling_matrix_present_flag, + V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT); +} + +static void +get_h264_scaling_matrix(struct slice_api_h264_scaling_matrix *dst_matrix, + const struct v4l2_ctrl_h264_scaling_matrix *src_matrix) +{ + memcpy(dst_matrix->scaling_list_4x4, src_matrix->scaling_list_4x4, + sizeof(dst_matrix->scaling_list_4x4)); + + memcpy(dst_matrix->scaling_list_8x8, src_matrix->scaling_list_8x8, + sizeof(dst_matrix->scaling_list_8x8)); +} + +static void get_h264_decode_parameters( + struct slice_api_h264_decode_param *dst_params, + const struct v4l2_ctrl_h264_decode_params *src_params, + const struct v4l2_h264_dpb_entry dpb[V4L2_H264_NUM_DPB_ENTRIES]) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(dst_params->dpb); i++) { + struct slice_h264_dpb_entry *dst_entry = &dst_params->dpb[i]; + const struct v4l2_h264_dpb_entry *src_entry = &dpb[i]; + + dst_entry->reference_ts = src_entry->reference_ts; + dst_entry->frame_num = src_entry->frame_num; + dst_entry->pic_num = src_entry->pic_num; + dst_entry->top_field_order_cnt = src_entry->top_field_order_cnt; + dst_entry->bottom_field_order_cnt = + src_entry->bottom_field_order_cnt; + dst_entry->flags = src_entry->flags; + } + + /* + * num_slices is a leftover from the old H.264 support and is ignored + * by the firmware. + */ + dst_params->num_slices = 0; + dst_params->nal_ref_idc = src_params->nal_ref_idc; + dst_params->top_field_order_cnt = src_params->top_field_order_cnt; + dst_params->bottom_field_order_cnt = src_params->bottom_field_order_cnt; + dst_params->flags = src_params->flags; +} + +static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a, + const struct v4l2_h264_dpb_entry *b) +{ + return a->top_field_order_cnt == b->top_field_order_cnt && + a->bottom_field_order_cnt == b->bottom_field_order_cnt; +} + +/* + * Move DPB entries of dec_param that refer to a frame already existing in dpb + * into the already existing slot in dpb, and move other entries into new slots. + * + * This function is an adaptation of the similarly-named function in + * hantro_h264.c. + */ +static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param, + struct v4l2_h264_dpb_entry *dpb) +{ + DECLARE_BITMAP(new, ARRAY_SIZE(dec_param->dpb)) = { 0, }; + DECLARE_BITMAP(in_use, ARRAY_SIZE(dec_param->dpb)) = { 0, }; + DECLARE_BITMAP(used, ARRAY_SIZE(dec_param->dpb)) = { 0, }; + unsigned int i, j; + + /* Disable all entries by default, and mark the ones in use. */ + for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + set_bit(i, in_use); + dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE; + } + + /* Try to match new DPB entries with existing ones by their POCs. */ + for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { + const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i]; + + if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) + continue; + + /* + * To cut off some comparisons, iterate only on target DPB + * entries were already used. + */ + for_each_set_bit(j, in_use, ARRAY_SIZE(dec_param->dpb)) { + struct v4l2_h264_dpb_entry *cdpb; + + cdpb = &dpb[j]; + if (!dpb_entry_match(cdpb, ndpb)) + continue; + + *cdpb = *ndpb; + set_bit(j, used); + /* Don't reiterate on this one. */ + clear_bit(j, in_use); + break; + } + + if (j == ARRAY_SIZE(dec_param->dpb)) + set_bit(i, new); + } + + /* For entries that could not be matched, use remaining free slots. */ + for_each_set_bit(i, new, ARRAY_SIZE(dec_param->dpb)) { + const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i]; + struct v4l2_h264_dpb_entry *cdpb; + + /* + * Both arrays are of the same sizes, so there is no way + * we can end up with no space in target array, unless + * something is buggy. + */ + j = find_first_zero_bit(used, ARRAY_SIZE(dec_param->dpb)); + if (WARN_ON(j >= ARRAY_SIZE(dec_param->dpb))) + return; + + cdpb = &dpb[j]; + *cdpb = *ndpb; + set_bit(j, used); + } +} + +/* + * The firmware expects unused reflist entries to have the value 0x20. + */ +static void fixup_ref_list(u8 *ref_list, size_t num_valid) +{ + memset(&ref_list[num_valid], 0x20, 32 - num_valid); +} + +static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) +{ + const struct v4l2_ctrl_h264_decode_params *dec_params = + get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_DECODE_PARAMS); + const struct v4l2_ctrl_h264_sps *sps = + get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_SPS); + const struct v4l2_ctrl_h264_pps *pps = + get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_PPS); + const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix = + get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_SCALING_MATRIX); + struct mtk_h264_dec_slice_param *slice_param = &inst->h264_slice_param; + struct v4l2_h264_reflist_builder reflist_builder; + u8 *p0_reflist = slice_param->decode_params.ref_pic_list_p0; + u8 *b0_reflist = slice_param->decode_params.ref_pic_list_b0; + u8 *b1_reflist = slice_param->decode_params.ref_pic_list_b1; + + update_dpb(dec_params, inst->dpb); + + get_h264_sps_parameters(&slice_param->sps, sps); + get_h264_pps_parameters(&slice_param->pps, pps); + get_h264_scaling_matrix(&slice_param->scaling_matrix, scaling_matrix); + get_h264_decode_parameters(&slice_param->decode_params, dec_params, + inst->dpb); + get_h264_dpb_list(inst, slice_param); + + /* Build the reference lists */ + v4l2_h264_init_reflist_builder(&reflist_builder, dec_params, sps, + inst->dpb); + v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist); + v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist); + /* Adapt the built lists to the firmware's expectations */ + fixup_ref_list(p0_reflist, reflist_builder.num_valid); + fixup_ref_list(b0_reflist, reflist_builder.num_valid); + fixup_ref_list(b1_reflist, reflist_builder.num_valid); + + memcpy(&inst->vsi_ctx.h264_slice_params, slice_param, + sizeof(inst->vsi_ctx.h264_slice_params)); +} + +static unsigned int get_mv_buf_size(unsigned int width, unsigned int height) +{ + int unit_size = (width / MB_UNIT_LEN) * (height / MB_UNIT_LEN) + 8; + + return HW_MB_STORE_SZ * unit_size; +} + +static int allocate_predication_buf(struct vdec_h264_slice_inst *inst) +{ + int err = 0; + + inst->pred_buf.size = BUF_PREDICTION_SZ; + err = mtk_vcodec_mem_alloc(inst->ctx, &inst->pred_buf); + if (err) { + mtk_vcodec_err(inst, "failed to allocate ppl buf"); + return err; + } + + inst->vsi_ctx.pred_buf_dma = inst->pred_buf.dma_addr; + return 0; +} + +static void free_predication_buf(struct vdec_h264_slice_inst *inst) +{ + struct mtk_vcodec_mem *mem = NULL; + + mtk_vcodec_debug_enter(inst); + + inst->vsi_ctx.pred_buf_dma = 0; + mem = &inst->pred_buf; + if (mem->va) + mtk_vcodec_mem_free(inst->ctx, mem); +} + +static int alloc_mv_buf(struct vdec_h264_slice_inst *inst, + struct vdec_pic_info *pic) +{ + int i; + int err; + struct mtk_vcodec_mem *mem = NULL; + unsigned int buf_sz = get_mv_buf_size(pic->buf_w, pic->buf_h); + + mtk_v4l2_debug(3, "size = 0x%lx", buf_sz); + for (i = 0; i < H264_MAX_MV_NUM; i++) { + mem = &inst->mv_buf[i]; + if (mem->va) + mtk_vcodec_mem_free(inst->ctx, mem); + mem->size = buf_sz; + err = mtk_vcodec_mem_alloc(inst->ctx, mem); + if (err) { + mtk_vcodec_err(inst, "failed to allocate mv buf"); + return err; + } + inst->vsi_ctx.mv_buf_dma[i] = mem->dma_addr; + } + + return 0; +} + +static void free_mv_buf(struct vdec_h264_slice_inst *inst) +{ + int i; + struct mtk_vcodec_mem *mem = NULL; + + for (i = 0; i < H264_MAX_MV_NUM; i++) { + inst->vsi_ctx.mv_buf_dma[i] = 0; + mem = &inst->mv_buf[i]; + if (mem->va) + mtk_vcodec_mem_free(inst->ctx, mem); + } +} + +static void get_pic_info(struct vdec_h264_slice_inst *inst, + struct vdec_pic_info *pic) +{ + struct mtk_vcodec_ctx *ctx = inst->ctx; + + ctx->picinfo.buf_w = (ctx->picinfo.pic_w + 15) & 0xFFFFFFF0; + ctx->picinfo.buf_h = (ctx->picinfo.pic_h + 31) & 0xFFFFFFE0; + ctx->picinfo.fb_sz[0] = ctx->picinfo.buf_w * ctx->picinfo.buf_h; + ctx->picinfo.fb_sz[1] = ctx->picinfo.fb_sz[0] >> 1; + inst->vsi_ctx.dec.cap_num_planes = + ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes; + + *pic = ctx->picinfo; + mtk_vcodec_debug(inst, "pic(%d, %d), buf(%d, %d)", + ctx->picinfo.pic_w, ctx->picinfo.pic_h, + ctx->picinfo.buf_w, ctx->picinfo.buf_h); + mtk_vcodec_debug(inst, "Y/C(%d, %d)", ctx->picinfo.fb_sz[0], + ctx->picinfo.fb_sz[1]); + + if ((ctx->last_decoded_picinfo.pic_w != ctx->picinfo.pic_w) || + (ctx->last_decoded_picinfo.pic_h != ctx->picinfo.pic_h)) { + inst->vsi_ctx.dec.resolution_changed = true; + if ((ctx->last_decoded_picinfo.buf_w != ctx->picinfo.buf_w) || + (ctx->last_decoded_picinfo.buf_h != ctx->picinfo.buf_h)) + inst->vsi_ctx.dec.realloc_mv_buf = true; + + mtk_v4l2_debug(1, "ResChg: (%d %d) : old(%d, %d) -> new(%d, %d)", + inst->vsi_ctx.dec.resolution_changed, + inst->vsi_ctx.dec.realloc_mv_buf, + ctx->last_decoded_picinfo.pic_w, + ctx->last_decoded_picinfo.pic_h, + ctx->picinfo.pic_w, ctx->picinfo.pic_h); + } +} + +static void get_crop_info(struct vdec_h264_slice_inst *inst, + struct v4l2_rect *cr) +{ + cr->left = inst->vsi_ctx.crop.left; + cr->top = inst->vsi_ctx.crop.top; + cr->width = inst->vsi_ctx.crop.width; + cr->height = inst->vsi_ctx.crop.height; + + mtk_vcodec_debug(inst, "l=%d, t=%d, w=%d, h=%d", + cr->left, cr->top, cr->width, cr->height); +} + +static void get_dpb_size(struct vdec_h264_slice_inst *inst, + unsigned int *dpb_sz) +{ + *dpb_sz = inst->vsi_ctx.dec.dpb_sz; + mtk_vcodec_debug(inst, "sz=%d", *dpb_sz); +} + +static int vdec_h264_slice_init(struct mtk_vcodec_ctx *ctx) +{ + struct vdec_h264_slice_inst *inst = NULL; + int err; + + inst = kzalloc(sizeof(*inst), GFP_KERNEL); + if (!inst) + return -ENOMEM; + + inst->ctx = ctx; + + inst->vpu.id = SCP_IPI_VDEC_H264; + inst->vpu.ctx = ctx; + + err = vpu_dec_init(&inst->vpu); + if (err) { + mtk_vcodec_err(inst, "vdec_h264 init err=%d", err); + goto error_free_inst; + } + + memcpy(&inst->vsi_ctx, inst->vpu.vsi, sizeof(inst->vsi_ctx)); + inst->vsi_ctx.dec.resolution_changed = true; + inst->vsi_ctx.dec.realloc_mv_buf = true; + + err = allocate_predication_buf(inst); + if (err) + goto error_deinit; + + mtk_vcodec_debug(inst, "struct size = %d,%d,%d,%d\n", + sizeof(struct mtk_h264_sps_param), + sizeof(struct mtk_h264_pps_param), + sizeof(struct mtk_h264_dec_slice_param), + sizeof(struct mtk_h264_dpb_info)); + + mtk_vcodec_debug(inst, "H264 Instance >> %p", inst); + + ctx->drv_handle = inst; + return 0; + +error_deinit: + vpu_dec_deinit(&inst->vpu); + +error_free_inst: + kfree(inst); + return err; +} + +static void vdec_h264_slice_deinit(void *h_vdec) +{ + struct vdec_h264_slice_inst *inst = + (struct vdec_h264_slice_inst *)h_vdec; + + mtk_vcodec_debug_enter(inst); + + vpu_dec_deinit(&inst->vpu); + free_predication_buf(inst); + free_mv_buf(inst); + + kfree(inst); +} + +static int vdec_h264_slice_decode(void *h_vdec, struct mtk_vcodec_mem *bs, + struct vdec_fb *fb, bool *res_chg) +{ + struct vdec_h264_slice_inst *inst = + (struct vdec_h264_slice_inst *)h_vdec; + const struct v4l2_ctrl_h264_decode_params *dec_params = + get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_DECODE_PARAMS); + struct vdec_vpu_inst *vpu = &inst->vpu; + uint32_t data[2]; + uint64_t y_fb_dma; + uint64_t c_fb_dma; + int err; + + mtk_vcodec_debug(inst, "+ [%d] FB y_dma=%llx c_dma=%llx va=%p", + ++inst->num_nalu, y_fb_dma, c_fb_dma, fb); + + /* bs NULL means flush decoder */ + if (bs == NULL) + return vpu_dec_reset(vpu); + + y_fb_dma = fb ? (u64)fb->base_y.dma_addr : 0; + c_fb_dma = fb ? (u64)fb->base_c.dma_addr : 0; + + inst->vsi_ctx.dec.bs_dma = (uint64_t)bs->dma_addr; + inst->vsi_ctx.dec.y_fb_dma = y_fb_dma; + inst->vsi_ctx.dec.c_fb_dma = c_fb_dma; + inst->vsi_ctx.dec.vdec_fb_va = (u64)(uintptr_t)fb; + + get_vdec_decode_parameters(inst); + data[0] = bs->size; + /* + * Reconstruct the first byte of the NAL unit, as the firmware requests + * that information to be passed even though it is present in the stream + * itself... + */ + data[1] = (dec_params->nal_ref_idc << 5) | + ((dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_IDR_PIC) + ? 0x5 : 0x1); + + *res_chg = inst->vsi_ctx.dec.resolution_changed; + if (*res_chg) { + mtk_vcodec_debug(inst, "- resolution changed -"); + if (inst->vsi_ctx.dec.realloc_mv_buf) { + err = alloc_mv_buf(inst, &(inst->ctx->picinfo)); + inst->vsi_ctx.dec.realloc_mv_buf = false; + if (err) + goto err_free_fb_out; + } + *res_chg = false; + } + + memcpy(inst->vpu.vsi, &inst->vsi_ctx, sizeof(inst->vsi_ctx)); + err = vpu_dec_start(vpu, data, 2); + if (err) + goto err_free_fb_out; + + /* wait decoder done interrupt */ + err = mtk_vcodec_wait_for_done_ctx(inst->ctx, + MTK_INST_IRQ_RECEIVED, + WAIT_INTR_TIMEOUT_MS); + if (err) + goto err_free_fb_out; + vpu_dec_end(vpu); + + memcpy(&inst->vsi_ctx, inst->vpu.vsi, sizeof(inst->vsi_ctx)); + mtk_vcodec_debug(inst, "\n - NALU[%d] -\n", inst->num_nalu, nal_type); + return 0; + +err_free_fb_out: + mtk_vcodec_err(inst, "\n - NALU[%d] err=%d -\n", inst->num_nalu, err); + return err; +} + +static int vdec_h264_slice_get_param(void *h_vdec, + enum vdec_get_param_type type, void *out) +{ + struct vdec_h264_slice_inst *inst = + (struct vdec_h264_slice_inst *)h_vdec; + + switch (type) { + case GET_PARAM_PIC_INFO: + get_pic_info(inst, out); + break; + + case GET_PARAM_DPB_SIZE: + get_dpb_size(inst, out); + break; + + case GET_PARAM_CROP_INFO: + get_crop_info(inst, out); + break; + + default: + mtk_vcodec_err(inst, "invalid get parameter type=%d", type); + return -EINVAL; + } + + return 0; +} + +const struct vdec_common_if vdec_h264_slice_if = { + .init = vdec_h264_slice_init, + .decode = vdec_h264_slice_decode, + .get_param = vdec_h264_slice_get_param, + .deinit = vdec_h264_slice_deinit, +}; diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c index b18743b906ea..42008243ceac 100644 --- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c @@ -19,6 +19,9 @@ int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) int ret = 0; switch (fourcc) { + case V4L2_PIX_FMT_H264_SLICE: + ctx->dec_if = &vdec_h264_slice_if; + break; case V4L2_PIX_FMT_H264: ctx->dec_if = &vdec_h264_if; break; diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h index ec8f4e8d3d23..d467e8af4a84 100644 --- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h @@ -55,6 +55,7 @@ struct vdec_fb_node { }; extern const struct vdec_common_if vdec_h264_if; +extern const struct vdec_common_if vdec_h264_slice_if; extern const struct vdec_common_if vdec_vp8_if; extern const struct vdec_common_if vdec_vp9_if; From patchwork Tue Apr 27 11:15:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Courbot X-Patchwork-Id: 428128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66033C433B4 for ; Tue, 27 Apr 2021 11:16:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37A51613CA for ; Tue, 27 Apr 2021 11:16:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235954AbhD0LQt (ORCPT ); Tue, 27 Apr 2021 07:16:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235929AbhD0LQm (ORCPT ); Tue, 27 Apr 2021 07:16:42 -0400 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6115DC061763 for ; Tue, 27 Apr 2021 04:15:59 -0700 (PDT) Received: by mail-pj1-x102e.google.com with SMTP id gc22-20020a17090b3116b02901558435aec1so3294645pjb.4 for ; Tue, 27 Apr 2021 04:15:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=po7QYMX2bjP6lmDmZxPgPCLAaN+WesS3W5xew9880lo=; b=Ofb4NdEX3S2iytgtS3/pPlP1K99zNiG+f8OLW8mkhYjYDGZz/hzd/XxTQ0h8To6ezU VQTSTTXKkTfKAenI3j6SvdSEPj5FT1ax9kcmCUzO1URw+894GTp83c3LCv8ZfazowJDO kggiebZoWJX0/NWqlDyl7MRgc7kRa+3cLS6pA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=po7QYMX2bjP6lmDmZxPgPCLAaN+WesS3W5xew9880lo=; b=GzJC2d8DhaaRKwysFKM5M8ke8KVk5WfdPOOWtjkoowOnexLuy09TOaZX77l454XKO+ xWkclRy0YDGwPZIJHuzCfARcdzFB+099fjPikJsxI5yhJJk9lY5LmCQR/1R1WeCsdzZK M02r2mhod7yiXSCMzq2I2gZvHNXvuCzoOT3b5VpeHfTE7M16EpCm8OOohBbXPjCpYBrc sxJlip+FqYekNWcxq5y8DKCOl7rCJcyhb2Diq5Ik5bbBYFhg1ZOK5/dXJteLGeJ23bxo GKt2K3KgkWpZfeVTw/gRhCY9q8ip6hZxzJiGzGC40wALB24JO76hIgJnmYsBRuE1P/qo AYfA== X-Gm-Message-State: AOAM531ktawnbhB+jRE4mDWzMPRnBCcNqa+9gKdOkD4YNcTFK3FBOiz7 4AemPZdbgaAQOFy0+GZkx6MWKQ== X-Google-Smtp-Source: ABdhPJyXeuIoiIttCVmVyb9b58xtxgJ5aD/gCgZSHhItF1eClG5XVDPSa2tdqRuGiCSwTCqpRJoSAQ== X-Received: by 2002:a17:902:9685:b029:e9:abc1:7226 with SMTP id n5-20020a1709029685b02900e9abc17226mr24369400plp.64.1619522159035; Tue, 27 Apr 2021 04:15:59 -0700 (PDT) Received: from acourbot.tok.corp.google.com ([2401:fa00:8f:203:b182:d363:32d0:53a2]) by smtp.gmail.com with ESMTPSA id c21sm2347210pfc.165.2021.04.27.04.15.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Apr 2021 04:15:58 -0700 (PDT) From: Alexandre Courbot To: Tiffany Lin , Andrew-CT Chen , Rob Herring , Yunfei Dong Cc: Mauro Carvalho Chehab , Hans Verkuil , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mediatek@lists.infradead.org, Alexandre Courbot Subject: [PATCH v4 08/15] dt-bindings: media: document mediatek, mt8183-vcodec-dec Date: Tue, 27 Apr 2021 20:15:19 +0900 Message-Id: <20210427111526.1772293-9-acourbot@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210427111526.1772293-1-acourbot@chromium.org> References: <20210427111526.1772293-1-acourbot@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org MT8183's decoder is instantiated similarly to MT8173's. Signed-off-by: Alexandre Courbot Acked-by: Rob Herring --- Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt index 06db6837cefd..a2a2a9e2ade3 100644 --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt @@ -9,6 +9,7 @@ Required properties: "mediatek,mt8173-vcodec-enc" for mt8173 avc encoder. "mediatek,mt8183-vcodec-enc" for MT8183 encoder. "mediatek,mt8173-vcodec-dec" for MT8173 decoder. + "mediatek,mt8183-vcodec-dec" for MT8183 decoder. - reg : Physical base address of the video codec registers and length of memory mapped region. - interrupts : interrupt number to the cpu. From patchwork Tue Apr 27 11:15:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Courbot X-Patchwork-Id: 428127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F0C3C43461 for ; Tue, 27 Apr 2021 11:16:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DB63F613C0 for ; Tue, 27 Apr 2021 11:16:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236039AbhD0LQz (ORCPT ); Tue, 27 Apr 2021 07:16:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235428AbhD0LQs (ORCPT ); Tue, 27 Apr 2021 07:16:48 -0400 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD9F5C061574 for ; Tue, 27 Apr 2021 04:16:04 -0700 (PDT) Received: by mail-pf1-x42f.google.com with SMTP id y62so8494004pfg.4 for ; Tue, 27 Apr 2021 04:16:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qDhPIhfqI4Pq+xMHlyEIR9SMexQlN4dYaagLiPYwmXw=; b=oCxdPjDxqgobH5jkDDoNia3CKUV62rbY0dfh44gq35HsSMuaWJlr7F7TsvdG5aln89 uH1/2zlA2BbYG7ca2eTjKcf+Zgx+Jsi1QXv4yKGn5lIrSXaVvO67wQ3TlvbBUdngMyaR vrJ0MO5ui2qHu/dLUbKRXQtfcWpYoxmDpPKBg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qDhPIhfqI4Pq+xMHlyEIR9SMexQlN4dYaagLiPYwmXw=; b=QCjMjWwnz0sUtxsSJt8T/ZZopxPpWySRSGjw38LoRze9EuF7KGAM47I4vHRp+r+5yh 8p/+ygmHOKY6+yNwgNyriX8VzLJeiEI7zNwMCGXjK8jGKWw3FHAFe1O+6hGg7K59qhhV FRgTI7gRsk3KWHMAjoaCHGRXcbNSQ0WONN63LcT+3sB+LoprFj+pS6EkxGX9x3AzMMgf oc5+VjZTTswdVzmD4I+7vKU/8H0VWsfiSHgGVW5nRAmbsosJBPP4cHyzWnL6rx4RWVxM fbsdljWuJdyVbwrJpDDfDXEBzzubJidrHeQrv1bv/i6/5eSa6rAF03zOkLD5y/5NucCA rNcw== X-Gm-Message-State: AOAM531D+mUSnmWTVj0x11eD/DMaCuvtzmmmchA1ALdX60wuInvNKfHg +kDmz8JnrgmM0PyOK4g0nqB8VA== X-Google-Smtp-Source: ABdhPJxqw98nbi47JE/oWyswUWPhOTXxfI7Or9Fl7giJxqzcHMD94e1TOTYdUKyFM6zwAZpP78OejQ== X-Received: by 2002:a63:e347:: with SMTP id o7mr4815571pgj.184.1619522164479; Tue, 27 Apr 2021 04:16:04 -0700 (PDT) Received: from acourbot.tok.corp.google.com ([2401:fa00:8f:203:b182:d363:32d0:53a2]) by smtp.gmail.com with ESMTPSA id c21sm2347210pfc.165.2021.04.27.04.16.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Apr 2021 04:16:04 -0700 (PDT) From: Alexandre Courbot To: Tiffany Lin , Andrew-CT Chen , Rob Herring , Yunfei Dong Cc: Mauro Carvalho Chehab , Hans Verkuil , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mediatek@lists.infradead.org, Alexandre Courbot Subject: [PATCH v4 10/15] media: mtk-vcodec: vdec: use helpers in VIDIOC_(TRY_)DECODER_CMD Date: Tue, 27 Apr 2021 20:15:21 +0900 Message-Id: <20210427111526.1772293-11-acourbot@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210427111526.1772293-1-acourbot@chromium.org> References: <20210427111526.1772293-1-acourbot@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Let's use the dedicated helpers to make sure we get the expected behavior on stateful decoders as well. Signed-off-by: Alexandre Courbot --- .../media/platform/mtk-vcodec/mtk_vcodec_dec.c | 16 ++-------------- 1 file changed, 2 insertions(+), 14 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c index 01c5333d6cff..4ef4b68ec9bd 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c @@ -52,22 +52,10 @@ static int vidioc_try_decoder_cmd(struct file *file, void *priv, if (ctx->dev->vdec_pdata->uses_stateless_api) return v4l2_m2m_ioctl_stateless_try_decoder_cmd(file, priv, cmd); - - switch (cmd->cmd) { - case V4L2_DEC_CMD_STOP: - case V4L2_DEC_CMD_START: - if (cmd->flags != 0) { - mtk_v4l2_err("cmd->flags=%u", cmd->flags); - return -EINVAL; - } - break; - default: - return -EINVAL; - } - return 0; + else + return v4l2_m2m_ioctl_try_decoder_cmd(file, priv, cmd); } - static int vidioc_decoder_cmd(struct file *file, void *priv, struct v4l2_decoder_cmd *cmd) { From patchwork Tue Apr 27 11:15:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Courbot X-Patchwork-Id: 428126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29CE3C433B4 for ; Tue, 27 Apr 2021 11:16:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8AFA613D0 for ; Tue, 27 Apr 2021 11:16:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237842AbhD0LRI (ORCPT ); Tue, 27 Apr 2021 07:17:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235915AbhD0LQy (ORCPT ); Tue, 27 Apr 2021 07:16:54 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA291C061761 for ; Tue, 27 Apr 2021 04:16:10 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id t2-20020a17090a0242b0290155433387beso1194244pje.1 for ; Tue, 27 Apr 2021 04:16:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NeuaYr+AFNS6zFyhjEclvaB+RiQlE5bnT3MqlgX4tYM=; b=NM92KqfkwK8D3MAj2A2IAOebWPxBDpy9nSUs6JX+LRjwpxE0EIEV6xkPFinBPMwPcJ 7z4RGCGDQ1uTLDWFAbWnbv131GzqbJY3okRp8m3cMGKPOVIsWF3TlS8P+ko8Pe8XwWgf 4/Ju9NNy1J/ngY8tL+zqPRbuxYBveekRdsRnA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NeuaYr+AFNS6zFyhjEclvaB+RiQlE5bnT3MqlgX4tYM=; b=PpAkHTPme+OagLwVkXrFGtA13GJ9wr0YJYxF6FVaL6VP3ewGqajXac6T70sRxu/Ntu P3dWWa2Vl0yg3Lw2GzHI3zlaTm+Vbdo9xmAE03fhKydgblpysk924o+lAoaeo2Lriaby N4z2pk8Dsxg45UwjdCN5UnhQu+DK2DLBYUj02V02KVxk1QCjkWdMwyxPNguGugpunOze Wz+GY2zbwviUKl7NPFxLboh8K/b0xUnZk5/DQr7PNw34x+x7RB3rnM9FKrqzE3pG8+Hh qCTuOX7JEadGcR5xUXPtJI2zNiVJSnOJfOkINyoCnp50/GGdmcarsvKeWlVBQLMbBmgr oJTg== X-Gm-Message-State: AOAM532mk4BGoLg6ZaPuMQnbrJqNzcZR31sQMs1UAD/zyHqsrd+D5r7c 97oQ+OgkkSZJOdaBNw8WAhPAkrFeennYLg== X-Google-Smtp-Source: ABdhPJyutim3aaPgcxoEFFLupS3n22kco1aCh6AI2C6Fs/kJN99bNeQBf5nXs1moTycGo7zNU8lGPg== X-Received: by 2002:a17:902:9a0a:b029:e6:bf00:8a36 with SMTP id v10-20020a1709029a0ab02900e6bf008a36mr24255695plp.51.1619522170256; Tue, 27 Apr 2021 04:16:10 -0700 (PDT) Received: from acourbot.tok.corp.google.com ([2401:fa00:8f:203:b182:d363:32d0:53a2]) by smtp.gmail.com with ESMTPSA id c21sm2347210pfc.165.2021.04.27.04.16.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Apr 2021 04:16:09 -0700 (PDT) From: Alexandre Courbot To: Tiffany Lin , Andrew-CT Chen , Rob Herring , Yunfei Dong Cc: Mauro Carvalho Chehab , Hans Verkuil , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mediatek@lists.infradead.org, Alexandre Courbot Subject: [PATCH v4 12/15] media: mtk-vcodec: vdec: clamp OUTPUT resolution to hardware limits Date: Tue, 27 Apr 2021 20:15:23 +0900 Message-Id: <20210427111526.1772293-13-acourbot@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210427111526.1772293-1-acourbot@chromium.org> References: <20210427111526.1772293-1-acourbot@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Calling S_FMT or TRY_FMT on the OUTPUT queue should adjust the resolution to the limits supported by the hardware. Until now this was only done on the CAPTURE queue, which could make clients believe that unsupported resolutions can be used when they set the coded size on the OUTPUT queue. In the case of the stateless decoder, the problem was even bigger since subsequently calling G_FMT on the CAPTURE queue would result in the unclamped resolution being returned, further inducing the client into error. Signed-off-by: Alexandre Courbot --- drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c index 4ef4b68ec9bd..0fafe8f9e1b9 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c @@ -223,19 +223,19 @@ static int vidioc_try_fmt(struct v4l2_format *f, pix_fmt_mp->field = V4L2_FIELD_NONE; + pix_fmt_mp->width = clamp(pix_fmt_mp->width, + MTK_VDEC_MIN_W, + MTK_VDEC_MAX_W); + pix_fmt_mp->height = clamp(pix_fmt_mp->height, + MTK_VDEC_MIN_H, + MTK_VDEC_MAX_H); + if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { pix_fmt_mp->num_planes = 1; pix_fmt_mp->plane_fmt[0].bytesperline = 0; } else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { int tmp_w, tmp_h; - pix_fmt_mp->height = clamp(pix_fmt_mp->height, - MTK_VDEC_MIN_H, - MTK_VDEC_MAX_H); - pix_fmt_mp->width = clamp(pix_fmt_mp->width, - MTK_VDEC_MIN_W, - MTK_VDEC_MAX_W); - /* * Find next closer width align 64, heign align 64, size align * 64 rectangle From patchwork Tue Apr 27 11:15:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Courbot X-Patchwork-Id: 428125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84ED0C43460 for ; Tue, 27 Apr 2021 11:16:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5005B6101D for ; Tue, 27 Apr 2021 11:16:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238051AbhD0LR2 (ORCPT ); Tue, 27 Apr 2021 07:17:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237807AbhD0LRE (ORCPT ); Tue, 27 Apr 2021 07:17:04 -0400 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 303A8C061345 for ; Tue, 27 Apr 2021 04:16:16 -0700 (PDT) Received: by mail-pj1-x1034.google.com with SMTP id m6-20020a17090a8586b02901507e1acf0fso7020677pjn.3 for ; Tue, 27 Apr 2021 04:16:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y245BjsBBoBDMhp2Hq7RuZw66md7jkGOZPH/qXHFLTc=; b=Ll7JiSsyTowxg9GU0xEzlSZqZ5CShh41qc1CI04juF3EIUHDkz1fEngw+5y0rfBfce lI53h2LDp6hJiz4sg1hPxQ7qeA8PHRD7NPH3oM7fXhoaCzLHZiey0+tDs3WWH6yjDLHd qOUvzx5mwX5KgFcyOSPZ+dI1xN+OQTiMy/emo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y245BjsBBoBDMhp2Hq7RuZw66md7jkGOZPH/qXHFLTc=; b=NQpu2W3BUKOhr9zm/NJ5hvw47N0aqRfGQ2YSZfZ45fUB+wQKbUsCqDQNSH5it8MbFH 1tDTu0h4kjz9DE0HV05Lwqg4THeIC+EroGP5EVbbFlvdr3RH58l9xaS5o0Vc5IU7rSuu tjBmNpXYQdR9KncM4pV7C6oFyngOePIVqFEWNZzUZjmyP1uWKzIX0kvAl3cvbi28JTjG 6sde6ruLb7Z0vURB5mnxI0nluJA6zOryOJes/RWBZ9t1T6BmuZfQRJW0KTj67p8sq/3v btfjmSiznmJU/gUBL/jYtkaxXQSlx0WK17qzRz+X7es9nffR/FRaHGWs0kChqx9TsSsn s4fw== X-Gm-Message-State: AOAM532wWAOwK7MKrlpo3ON23+rQVsJMmyKW+H+dfvq2rIjDUjwM7cj0 6usC1BhQo4SCvmI706EUknQAkQ== X-Google-Smtp-Source: ABdhPJzNo6PhHgHEhisSVqz6a9twVXvVYg1xIl0eLawZniTXM93PXxY+b89jNm4VDZ8DCJKUiJy2uA== X-Received: by 2002:a17:902:8685:b029:e6:5ff6:f7df with SMTP id g5-20020a1709028685b02900e65ff6f7dfmr23693092plo.40.1619522175746; Tue, 27 Apr 2021 04:16:15 -0700 (PDT) Received: from acourbot.tok.corp.google.com ([2401:fa00:8f:203:b182:d363:32d0:53a2]) by smtp.gmail.com with ESMTPSA id c21sm2347210pfc.165.2021.04.27.04.16.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Apr 2021 04:16:15 -0700 (PDT) From: Alexandre Courbot To: Tiffany Lin , Andrew-CT Chen , Rob Herring , Yunfei Dong Cc: Mauro Carvalho Chehab , Hans Verkuil , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mediatek@lists.infradead.org, Alexandre Courbot Subject: [PATCH v4 14/15] media: mtk-vcodec: venc: support START and STOP commands Date: Tue, 27 Apr 2021 20:15:25 +0900 Message-Id: <20210427111526.1772293-15-acourbot@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210427111526.1772293-1-acourbot@chromium.org> References: <20210427111526.1772293-1-acourbot@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The V4L2 encoder specification requires encoders to support the V4L2_ENC_CMD_START and V4L2_ENC_CMD_STOP commands. Add support for these to the mtk-vcodec encoder by reusing the same flush buffer as used by the decoder driver. Signed-off-by: Alexandre Courbot --- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 2 + .../platform/mtk-vcodec/mtk_vcodec_enc.c | 123 +++++++++++++++++- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 4 + 3 files changed, 122 insertions(+), 7 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h index c2f4cad6cfc2..1b61d722d313 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h @@ -252,6 +252,7 @@ struct vdec_pic_info { * @last_decoded_picinfo: pic information get from latest decode * @empty_flush_buf: a fake size-0 capture buffer that indicates flush. Only * to be used with encoder and stateful decoder. + * @is_flushing: set to true if flushing is in progress. * @current_codec: current set input codec, in V4L2 pixel format * * @colorspace: enum v4l2_colorspace; supplemental to pixelformat @@ -291,6 +292,7 @@ struct mtk_vcodec_ctx { struct work_struct encode_work; struct vdec_pic_info last_decoded_picinfo; struct v4l2_m2m_buffer empty_flush_buf; + bool is_flushing; u32 current_codec; diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c index 4831052f475d..8058331dc553 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c @@ -659,6 +659,7 @@ static int vidioc_venc_dqbuf(struct file *file, void *priv, struct v4l2_buffer *buf) { struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv); + int ret; if (ctx->state == MTK_STATE_ABORT) { mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error", @@ -666,7 +667,77 @@ static int vidioc_venc_dqbuf(struct file *file, void *priv, return -EIO; } - return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf); + ret = v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf); + if (ret) + return ret; + + /* + * Complete flush if the user dequeued the 0-payload LAST buffer. + * We check the payload because a buffer with the LAST flag can also + * be seen during resolution changes. If we happen to be flushing at + * that time, the last buffer before the resolution changes could be + * misinterpreted for the buffer generated by the flush and terminate + * it earlier than we want. + */ + if (!V4L2_TYPE_IS_OUTPUT(buf->type) && + buf->flags & V4L2_BUF_FLAG_LAST && + buf->m.planes[0].bytesused == 0 && + ctx->is_flushing) { + /* + * Last CAPTURE buffer is dequeued, we can allow another flush + * to take place. + */ + ctx->is_flushing = false; + } + + return 0; +} + +static int vidioc_encoder_cmd(struct file *file, void *priv, + struct v4l2_encoder_cmd *cmd) +{ + struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv); + struct vb2_queue *src_vq, *dst_vq; + int ret; + + ret = v4l2_m2m_ioctl_try_encoder_cmd(file, priv, cmd); + if (ret) + return ret; + + /* Calling START or STOP is invalid if a flush is in progress */ + if (ctx->is_flushing) + return -EBUSY; + + mtk_v4l2_debug(1, "encoder cmd=%u", cmd->cmd); + + dst_vq = v4l2_m2m_get_vq(ctx->m2m_ctx, + V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + switch (cmd->cmd) { + case V4L2_ENC_CMD_STOP: + src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (!vb2_is_streaming(src_vq)) { + mtk_v4l2_debug(1, "Output stream is off. No need to flush."); + return 0; + } + if (!vb2_is_streaming(dst_vq)) { + mtk_v4l2_debug(1, "Capture stream is off. No need to flush."); + return 0; + } + ctx->is_flushing = true; + v4l2_m2m_buf_queue(ctx->m2m_ctx, &ctx->empty_flush_buf.vb); + v4l2_m2m_try_schedule(ctx->m2m_ctx); + break; + + case V4L2_ENC_CMD_START: + vb2_clear_last_buffer_dequeued(dst_vq); + break; + + default: + return -EINVAL; + } + + return 0; } const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = { @@ -702,6 +773,9 @@ const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = { .vidioc_g_selection = vidioc_venc_g_selection, .vidioc_s_selection = vidioc_venc_s_selection, + + .vidioc_encoder_cmd = vidioc_encoder_cmd, + .vidioc_try_encoder_cmd = v4l2_m2m_ioctl_try_encoder_cmd, }; static int vb2ops_venc_queue_setup(struct vb2_queue *vq, @@ -869,9 +943,27 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q) dst_buf->vb2_buf.planes[0].bytesused = 0; v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR); } + /* STREAMOFF on the CAPTURE queue completes any ongoing flush */ + if (ctx->is_flushing) { + mtk_v4l2_debug(1, "STREAMOFF called while flushing"); + v4l2_m2m_buf_remove_by_buf(&ctx->m2m_ctx->out_q_ctx, + &ctx->empty_flush_buf.vb); + ctx->is_flushing = false; + } } else { - while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx))) - v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR); + while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx))) { + if (src_buf != &ctx->empty_flush_buf.vb) + v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR); + } + if (ctx->is_flushing) { + /* + * If we are in the middle of a flush, put the flush + * buffer back into the queue so the next CAPTURE + * buffer gets returned with the LAST flag set. + */ + v4l2_m2m_buf_queue(ctx->m2m_ctx, + &ctx->empty_flush_buf.vb); + } } if ((q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && @@ -971,12 +1063,15 @@ static int mtk_venc_param_change(struct mtk_vcodec_ctx *ctx) { struct venc_enc_param enc_prm; struct vb2_v4l2_buffer *vb2_v4l2 = v4l2_m2m_next_src_buf(ctx->m2m_ctx); - struct mtk_video_enc_buf *mtk_buf = - container_of(vb2_v4l2, struct mtk_video_enc_buf, - m2m_buf.vb); - + struct mtk_video_enc_buf *mtk_buf; int ret = 0; + /* Don't upcast the empty flush buffer */ + if (vb2_v4l2 == &ctx->empty_flush_buf.vb) + return 0; + + mtk_buf = container_of(vb2_v4l2, struct mtk_video_enc_buf, m2m_buf.vb); + memset(&enc_prm, 0, sizeof(enc_prm)); if (mtk_buf->param_change == MTK_ENCODE_PARAM_NONE) return 0; @@ -1062,6 +1157,20 @@ static void mtk_venc_worker(struct work_struct *work) } src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx); + + /* + * If we see the flush buffer, send an empty buffer with the LAST flag + * to the client. is_flushing will be reset at the time the buffer + * is dequeued. + */ + if (src_buf == &ctx->empty_flush_buf.vb) { + vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0); + dst_buf->flags |= V4L2_BUF_FLAG_LAST; + v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE); + v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx); + return; + } + memset(&frm_buf, 0, sizeof(frm_buf)); for (i = 0; i < src_buf->vb2_buf.num_planes ; i++) { frm_buf.fb_addr[i].dma_addr = diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c index 7d7b8cfc2cc5..2dd6fef896df 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c @@ -131,6 +131,7 @@ static int fops_vcodec_open(struct file *file) struct mtk_vcodec_dev *dev = video_drvdata(file); struct mtk_vcodec_ctx *ctx = NULL; int ret = 0; + struct vb2_queue *src_vq; ctx = kzalloc(sizeof(*ctx), GFP_KERNEL); if (!ctx) @@ -164,6 +165,9 @@ static int fops_vcodec_open(struct file *file) ret); goto err_m2m_ctx_init; } + src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + ctx->empty_flush_buf.vb.vb2_buf.vb2_queue = src_vq; mtk_vcodec_enc_set_default_params(ctx); if (v4l2_fh_is_singular(&ctx->fh)) {