From patchwork Thu Mar 31 19:37:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2FEAC433EF for ; Thu, 31 Mar 2022 19:37:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240795AbiCaTjc (ORCPT ); Thu, 31 Mar 2022 15:39:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240780AbiCaTj2 (ORCPT ); Thu, 31 Mar 2022 15:39:28 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE4351F160C; Thu, 31 Mar 2022 12:37:37 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id E0A2B1F4724B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755456; bh=RzISFmgzdgSdtW0+/gjxICIzsCqQWUTem2X0Qdl07rI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P8JxZJhCDdx9Qxk6/3g0ArBQAL6zq/4nDv4o4se2D/r8x9FqDSTG1Gmclg/ipJvOd QVlZiS09IZSS6knovbP88HF2IV7HrZE1F3rhavI2rX4E0451TzvLMBrUwygzg1GjGI irNuFE3v7ose8uwUbKL2ZQSLKbjzcElTY/HxaeP2BDXzjK85i+gTlt8+YOVA47lGj6 7d+S7kJOkvLXM5m7rWGukhvKJ9TAGX/mCC6sFzvZCD+1PsYD2j/I2gdHM/oH4ueAsy aKVOeo7LNLz2HOR4LEw51uhN5GKXsZa77yUZ14AZ1Bgg11PvsIE/kVhKqYEApsz9Cg DE6LjQf5fnoCw== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 02/23] media: v4l2-mem2mem: Trace on implicit un-hold Date: Thu, 31 Mar 2022 15:37:04 -0400 Message-Id: <20220331193726.289559-3-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org If the timestamp of the src buffer differs from the timestamp of a held dst buffer, the held buffer is implicitly removed and marked as done. Add a trace to help debugging if someone hits that case. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-mem2mem.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c index 675e22895ebe..fbcd79763e8f 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -336,6 +336,7 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev, if (src && dst && dst->is_held && dst->vb2_buf.copied_timestamp && dst->vb2_buf.timestamp != src->vb2_buf.timestamp) { + dprintk("src and dst timestamp mismatch, removing held capture buffer.\n"); dst->is_held = false; v4l2_m2m_dst_buf_remove(m2m_ctx); v4l2_m2m_buf_done(dst, VB2_BUF_STATE_DONE); From patchwork Thu Mar 31 19:37:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 047D3C433F5 for ; Thu, 31 Mar 2022 19:37:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240815AbiCaTjf (ORCPT ); Thu, 31 Mar 2022 15:39:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240781AbiCaTj2 (ORCPT ); Thu, 31 Mar 2022 15:39:28 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70D3222451D; Thu, 31 Mar 2022 12:37:40 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id A47591F47247 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755459; bh=XWSCMrc+NrLm1p3fYYJooFLeLhu8K+wqylIt0iyj7Vc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ci2A27WG3nn3hVwxc9jwIR3E4CPWTQcVTJX69lc7p4gFqCHtA6JBUl2Zc403FpXq5 LkQ1jV3EkM9Dn964TVruUiU4v9lkrRH9Fb7J0VIytyFFZtFeOIktrZzNfcxOPviNDp o0jMoLjQioZWZclguu1x7RvVwgXbWK1O6VGJRcseqdEWPy8sEYg/hqZUdFQ2x9J9oq g5RbA6c821YYmIh0tZ95dqRwo/xZCZW2B5tcxonaUAFoGB/d/vG4+jCtUYEX65iuB6 1l7ebu99zlOQkx1eBuyjPSfSIpK3Oy/QWqai+EVzKRIlJnavqmkwAlyP/y7mQs+0mr IYSHXnNduxHAw== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 04/23] media: h264: Avoid wrapping long_term_frame_idx Date: Thu, 31 Mar 2022 15:37:06 -0400 Message-Id: <20220331193726.289559-5-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org For long term references, frame_num is set to long_term_frame_idx which does not require wrapping. This is fixed by observation, no directly related issue have been found yet. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 5633a242520a..ac47519a9fbe 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -57,8 +57,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * '8.2.4.1 Decoding process for picture numbers' of the spec. * TODO: This logic will have to be adjusted when we start * supporting interlaced content. + * For long term references, frame_num is set to + * long_term_frame_idx which requires no wrapping. */ - if (dpb[i].frame_num > cur_frame_num) + if (!b->refs[i].longterm && dpb[i].frame_num > cur_frame_num) b->refs[i].frame_num = (int)dpb[i].frame_num - max_frame_num; else From patchwork Thu Mar 31 19:37:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65C53C4332F for ; Thu, 31 Mar 2022 19:37:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240858AbiCaTjn (ORCPT ); Thu, 31 Mar 2022 15:39:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240805AbiCaTjd (ORCPT ); Thu, 31 Mar 2022 15:39:33 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1C3923B3E8; Thu, 31 Mar 2022 12:37:45 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 4C41B1F4724D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755464; bh=VjX+liG+SlI3iuDWs50MzK4K6rmwEP4Ktzmj7Beu+Mo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i1Up/5emswsszUr/IL82EU9CA2ojTsAugslRJT9YMSbyEf30Ti5BXZws52rF6LNiV y5+cLWg7C5SqyiIXBAY2YUtyVYPQVQ0q/bQalp4u+V1GMeduCKVwTa1UYz2JGXe2/C WafKxJ5Y4tsT9UbfNjuahk3E1SccJN388ZRPx+KytlH1ngeNwZ2lLs6XZ8Risfa/FK qBtN4be/+0m8Hzg3xmNvmH9zRQPftUBsc+WvrSTyMOyB/+WlCd0Eofe5CV2R77f6yN F7nLcd0c4r8+gO7wek8hFboqHt5R4ZY5gR22Z5omOh24PqR6akhbS2/G+USGX5dCq0 xfEvo4aX5tcTg== From: Nicolas Dufresne To: Mauro Carvalho Chehab , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v2 06/23] media: h264: Increase reference lists size to 32 Date: Thu, 31 Mar 2022 15:37:08 -0400 Message-Id: <20220331193726.289559-7-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This is to accommodate support for field decoding, which splits the top and the bottom references into the reference list. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 6 +++--- drivers/staging/media/hantro/hantro_hw.h | 6 +++--- drivers/staging/media/rkvdec/rkvdec-h264.c | 6 +++--- include/media/v4l2-h264.h | 8 ++++---- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index afbfcf78efe4..4b46b36526c0 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -212,7 +212,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in @@ -235,9 +235,9 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 2bc6b8f088f5..292aaaabaf24 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; }; /** diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 3c7f3d87fab4..dff89732ddd0 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index ef9a894e3c32..e282fb16ac58 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; @@ -51,9 +51,9 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described @@ -70,7 +70,7 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in From patchwork Thu Mar 31 19:37:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555442 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BDAEC433EF for ; Thu, 31 Mar 2022 19:38:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240943AbiCaTjy (ORCPT ); Thu, 31 Mar 2022 15:39:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240876AbiCaTjv (ORCPT ); Thu, 31 Mar 2022 15:39:51 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C25423D5A7; Thu, 31 Mar 2022 12:37:49 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id EDC501F47255 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755468; bh=CCCLzz5lUZJ9ab0CExOOh36ojMkZJq6/0fFDYiktq9Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GfK+mou3ezaFhxxu8/w8QbXkSgyKRVrLhN8TxjPchHNoIzz8UPuu3BOrImGYWjp5+ AVkO7OlfcbSm4bLrpRdYp239qZWS8qPgVLBwDYB8D4jQg7l2HZSLgqbiFsDgJ35VdJ Dmhev3I0nQq4tL4A4f0rVlKuskyPGproDz/lCkZQ8P9Xjpzo/g7h4+TBiE18t7UizI k6eX8P1BhtbLKt5L+PJ2876KCQsIuBwwrfmwx/2K6mt6LPQdPFFFMPYLVaWHzLoWoc vwoLJxuVO0RgJdSiCiGyD5Cu8ieL7vcteIvN2Vs4f+LQ1ujCm1CIA0qLzfLDCG4u15 soQpkjVpVqFIg== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 09/23] media: v4l2: Trace calculated p/b0/b1 initial reflist Date: Thu, 31 Mar 2022 15:37:11 -0400 Message-Id: <20220331193726.289559-10-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add debug print statements to print the content of P & B reference lists, to verify that the ordering of the generated reference lists is correct. This is especially important for the field decoding mode, where sorting is more complex. Signed-off-by: Nicolas Dufresne Tested-by: Sebastian Fricke Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 86 +++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 38d8dbda0045..bcf9b7774560 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -241,6 +241,87 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return poca < pocb ? -1 : 1; } +static char ref_type_to_char (u8 ref_type) +{ + switch (ref_type) { + case V4L2_H264_FRAME_REF: + return 'f'; + case V4L2_H264_TOP_FIELD_REF: + return 't'; + case V4L2_H264_BOTTOM_FIELD_REF: + return 'b'; + } + + return '?'; +} + +static const char *format_ref_list_p(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, + char *out_str, const int len) +{ + int n = 0, i; + + n += snprintf(out_str + n, len - n, "|"); + + for (i = 0; i < builder->num_valid; i++) { + /* this is pic_num for frame and frame_num (wrapped) for field, + * but for frame pic_num is equal to frame_num (wrapped). + */ + int frame_num = builder->refs[reflist[i].index].frame_num; + bool longterm = builder->refs[reflist[i].index].longterm; + + n += scnprintf(out_str + n, len - n, "%i%c%c|", + frame_num, longterm ? 'l' : 's', + ref_type_to_char (reflist[i].fields)); + } + + return out_str; +} + +static void print_ref_list_p(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist) +{ + char buf[1024]; + + pr_debug("ref_pic_list_p (cur_poc %u%c) %s\n", + builder->cur_pic_order_count, + ref_type_to_char(builder->cur_pic_fields), + format_ref_list_p(builder, reflist, buf, sizeof(buf))); +} + +static const char *format_ref_list_b(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, + char *out_str, const int len) +{ + int n = 0, i; + + n += snprintf(out_str + n, len - n, "|"); + + for (i = 0; i < builder->num_valid; i++) { + int frame_num = builder->refs[reflist[i].index].frame_num; + u32 poc = v4l2_h264_get_poc(builder, reflist + i); + bool longterm = builder->refs[reflist[i].index].longterm; + + n += scnprintf(out_str + n, len - n, "%i%c%c|", + longterm ? frame_num : poc, + longterm ? 'l' : 's', + ref_type_to_char(reflist[i].fields)); + } + + return out_str; +} + +static void print_ref_list_b(const struct v4l2_h264_reflist_builder *builder, + struct v4l2_h264_reference *reflist, u8 list_num) +{ + char buf[1024]; + + pr_debug("ref_pic_list_b%u (cur_poc %u%c) %s", + list_num, builder->cur_pic_order_count, + ref_type_to_char (builder->cur_pic_fields), + format_ref_list_b(builder, reflist, buf, sizeof(buf))); +} + /** * v4l2_h264_build_p_ref_list() - Build the P reference list * @@ -261,6 +342,8 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, sizeof(builder->unordered_reflist[0]) * builder->num_valid); sort_r(reflist, builder->num_valid, sizeof(*reflist), v4l2_h264_p_ref_list_cmp, NULL, builder); + + print_ref_list_p(builder, reflist); } EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); @@ -296,6 +379,9 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, if (builder->num_valid > 1 && !memcmp(b1_reflist, b0_reflist, builder->num_valid)) swap(b1_reflist[0], b1_reflist[1]); + + print_ref_list_b(builder, b0_reflist, 0); + print_ref_list_b(builder, b1_reflist, 1); } EXPORT_SYMBOL_GPL(v4l2_h264_build_b_ref_lists); From patchwork Thu Mar 31 19:37:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9824C433F5 for ; Thu, 31 Mar 2022 19:38:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240990AbiCaTj5 (ORCPT ); Thu, 31 Mar 2022 15:39:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240897AbiCaTjw (ORCPT ); Thu, 31 Mar 2022 15:39:52 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89F3223D770; Thu, 31 Mar 2022 12:37:52 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id A6A121F47253 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755471; bh=SqPD7oTR+3FuYunP0khMcdvMcGz4bOvgAm9j9vzmvSk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iv777jy1D9CAI9gb1UAFfxR2k7QirrCUKilgDl8VU9WTv+FByMkFfG5Ua4MFWhsU5 YIcdCPJVM1KT7RTtBvuPriBFyRWCt4ldLG7HF/rCh/Opjtuz/5iFKnfoU+Ug1aBTps 3IQXnC0bPymS3XjP0Z9kkPLaVrLcWASjASJ19lZwTyJjs62ZLlJbJr1UZ+0HeUHJ33 QcBUZf3My8+xWSH0UCh1z2dAfrehA1KnPAVK20mYreggE5bl8QTW722Ph9n0SM+VeS ayvPRtQvJNqe7QGbh0TUS+2uWMKjA46XMmBCP8KrWvhmZpozHjRQ/EB9VRBXrg1UYb LoFeblkB9CqVg== From: Nicolas Dufresne To: Mauro Carvalho Chehab Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 11/23] media: v4l2: Reorder field reflist Date: Thu, 31 Mar 2022 15:37:13 -0400 Message-Id: <20220331193726.289559-12-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org As per spec, the field reflist requires interleaving top and bottom field in a specific way that does not fit inside the sort operation. The process consist of alternating references parity, starting with a reference of the same parity as the current picture. This processs is done twice, once for short term references and a second time for the long term references. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/media/v4l2-core/v4l2-h264.c | 42 +++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index 7e1eba03099a..d2d1f3bed7e8 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -244,6 +244,40 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, return poca < pocb ? -1 : 1; } +/* + * The references need to be reordered so that references are alternating + * between top and bottom field references starting with the current picture + * parity. This has to be * done for short term and long term references + * separately. + */ +static void reorder_field_reflist(const struct v4l2_h264_reflist_builder *b, + struct v4l2_h264_reference *reflist) +{ + struct v4l2_h264_reference tmplist[V4L2_H264_REF_LIST_LEN]; + u8 lt, i = 0, j = 0, k = 0; + + memcpy(tmplist, reflist, sizeof(tmplist[0]) * b->num_valid); + + for (lt = 0; lt <= 1; lt++) { + do { + for (; i < b->num_valid && b->refs[tmplist[i].index].longterm == lt; i++) { + if (tmplist[i].fields == b->cur_pic_fields) { + reflist[k++] = tmplist[i++]; + break; + } + } + + for (; j < b->num_valid && b->refs[tmplist[j].index].longterm == lt; j++) { + if (tmplist[j].fields != b->cur_pic_fields) { + reflist[k++] = tmplist[j++]; + break; + } + } + } while ((i < b->num_valid && b->refs[tmplist[i].index].longterm == lt) || + (j < b->num_valid && b->refs[tmplist[j].index].longterm == lt)); + } +} + static char ref_type_to_char (u8 ref_type) { switch (ref_type) { @@ -346,6 +380,9 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, sort_r(reflist, builder->num_valid, sizeof(*reflist), v4l2_h264_p_ref_list_cmp, NULL, builder); + if (builder->cur_pic_fields != V4L2_H264_FRAME_REF) + reorder_field_reflist(builder, reflist); + print_ref_list_p(builder, reflist); } EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); @@ -379,6 +416,11 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, sort_r(b1_reflist, builder->num_valid, sizeof(*b1_reflist), v4l2_h264_b1_ref_list_cmp, NULL, builder); + if (builder->cur_pic_fields != V4L2_H264_FRAME_REF) { + reorder_field_reflist(builder, b0_reflist); + reorder_field_reflist(builder, b1_reflist); + } + if (builder->num_valid > 1 && !memcmp(b1_reflist, b0_reflist, builder->num_valid)) swap(b1_reflist[0], b1_reflist[1]); From patchwork Thu Mar 31 19:37:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8D17C433FE for ; Thu, 31 Mar 2022 19:38:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240747AbiCaTkC (ORCPT ); Thu, 31 Mar 2022 15:40:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240905AbiCaTjw (ORCPT ); Thu, 31 Mar 2022 15:39:52 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4859223DE96; Thu, 31 Mar 2022 12:37:54 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id E968B1F47255 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755473; bh=jjxmbWJ+OUjcuFBbFVuZBkezVddHPzPh9O3QIqvsczU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RwGxqDjb0z4nPRaKXOOXYxg8RTqfoEsVbLNoGTKoOzPLv3hNGVnc6HLxlP8mSG/at AG+QmqMCODgEoABnKum4nYnaByGjFVNYW9dBw7IJgMjmfBhD0NhgMaJ8pAlPcT82zP Lz6kwWHhO7YdLQTjpqVWuhsRwi8Rr6Kjs+wCzmoD0UcTv4MHn5PhDy+oDM6nGn0+V/ 7OmMUW8v0byfOjG9iiJxlykixdc7+5gDkU7MslxNYsodoOqffxM6OlJAO1R0yAIqbt jSXOE3Vpj9rhLG/RDjJMIHnxY/W5m2zWng9BLDkBqpkfSBD3ZApYKpfDusZaPnYtcs hjn1fjNYR0I2A== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 12/23] media: rkvdec: Stop overclocking the decoder Date: Thu, 31 Mar 2022 15:37:14 -0400 Message-Id: <20220331193726.289559-13-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org While this overclock hack seems to work on some implementations (some ChromeBooks, RockPi4) it also causes instability on other implementations (notably LibreComputer Renegade, but there were more reports in the LibreELEC project, where this has been removed). While performance is indeed affected (tested with GStreamer), 4K playback still works as long as you don't operate in lock step and keep at least 1 frame ahead of time in the decode queue. After discussion with ChromeOS members, it would seem that their implementation indeed used to synchronously decode each frames, so this hack was simply compensating for their code being less efficient. In my opinion, this hack should not have been included upstream. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index c0cf3488f970..2df8cf4883e2 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev) if (ret) return ret; - /* - * Bump ACLK to max. possible freq. (500 MHz) to improve performance - * When 4k video playback. - */ - clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000); - rkvdec->regs = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(rkvdec->regs)) return PTR_ERR(rkvdec->regs); From patchwork Thu Mar 31 19:37:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C0ABC433EF for ; Thu, 31 Mar 2022 19:38:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241021AbiCaTkG (ORCPT ); Thu, 31 Mar 2022 15:40:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240933AbiCaTjy (ORCPT ); Thu, 31 Mar 2022 15:39:54 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD03423F3A6; Thu, 31 Mar 2022 12:37:57 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 538C01F4725C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755476; bh=s3TCwg9L0AAOtv/6Hb2IdzZi6+nwv8qK9bwTi/Shc+k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jfGbAELsD56jkeFy74WurVTn19hU0NwWH0mqm7jGdhva3aRlObO+CSaQ05wC3Hl9b IJQP7wTfoGK5IW9uOdJIaRJ+a/Sza1KTG7p7mjnINjGQY8W9oPoW60r3ggaWLSGvc2 7I1uswmzE5ZPvQ5z0eFTKiqmH73zPPp02kgpQ2MsIVe7XC8WEhSijExUFXdxuOrfHO aFPDih9mmHf5LouncQ1S0Bktk5Pcg5ivSp6CWHMCpIr/GKuznpcvrAb7Gl0nXrb9KL HpAWLFCO7LniNhwiGb9LtS6yp8pTqgzEKDczRcS1fCLZ/uuGIb4FmNuO5sgnfTU6N7 9rRK4jd5T4Vnw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , Ezequiel Garcia , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 14/23] media: rkvdec: h264: Fix bit depth wrap in pps packet Date: Thu, 31 Mar 2022 15:37:16 -0400 Message-Id: <20220331193726.289559-15-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The luma and chroma bit depth fields in the pps packet are 3 bits wide. 8 is wrongly added to the bit depth values written to these 3 bit fields. Because only the 3 LSB are written, the hardware was configured correctly. Correct this by not adding 8 to the luma and chroma bit depth value. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index bcde37d72244..8d44a884a52e 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -662,8 +662,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, WRITE_PPS(0xff, PROFILE_IDC); WRITE_PPS(1, CONSTRAINT_SET3_FLAG); WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC); - WRITE_PPS(sps->bit_depth_luma_minus8 + 8, BIT_DEPTH_LUMA); - WRITE_PPS(sps->bit_depth_chroma_minus8 + 8, BIT_DEPTH_CHROMA); + WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA); + WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA); WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG); WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4); WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES); From patchwork Thu Mar 31 19:37:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555438 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A00BC433EF for ; Thu, 31 Mar 2022 19:38:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241074AbiCaTkW (ORCPT ); Thu, 31 Mar 2022 15:40:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240889AbiCaTjz (ORCPT ); Thu, 31 Mar 2022 15:39:55 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 883CA23FF01; Thu, 31 Mar 2022 12:38:01 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 08D9D1F4725F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755480; bh=HmMBg5b8YH4LzW0sBzAl4u/8f+BCfQenxWj/fruvkP8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KbVTEhI17KkBlxrzHM4EuGkf7QIYl4hZ/hOfcIcrzBe13cx9Xk5TLnQilPRIkCk/u +RAY0swuwiXIxoC7+k0vRmZm036mvACh+quaET/cKJOI1pwgQV9em1k9c0K7ymuKsf /Q7biHye8EnmdIsduVy61i1tLrDezx2l9quW2m4+1MKZOo/JAiSUAXxQnx3emifXBL 8COXL3t4Q7KArtzscrvkyquK+0+O2W+4FznSZTULfF5040+lllV7hyjwDLsz/z77P2 O3gWwpXQDdPv8zkLNX/NAufkBj5R0d8RXWFUa8vfYn7pR9ZDFaRx+fBm4Bc5cxyEGx fuwF0A10DIMuA== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , Ezequiel Garcia , Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 16/23] media: rkvdec: h264: Fix reference frame_num wrap for second field Date: Thu, 31 Mar 2022 15:37:18 -0400 Message-Id: <20220331193726.289559-17-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman When decoding the second field in a complementary field pair the second field is sharing the same frame_num with the first field. Currently the frame_num for the first field is wrapped when it matches the field being decoded, this cause issues to decode the second field in a complementary field pair. Fix this by using inclusive comparison, less than or equal. Signed-off-by: Jonas Karlman Reviewed-by: Ezequiel Garcia Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index a42cf19bcc6d..730f8ebf7f58 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -773,7 +773,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, continue; if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num < dec_params->frame_num) { + dpb[i].frame_num <= dec_params->frame_num) { p[i] = dpb[i].frame_num; continue; } From patchwork Thu Mar 31 19:37:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C07EBC4332F for ; Thu, 31 Mar 2022 19:38:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241000AbiCaTkZ (ORCPT ); Thu, 31 Mar 2022 15:40:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240982AbiCaTj4 (ORCPT ); Thu, 31 Mar 2022 15:39:56 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EE0D23D46B; Thu, 31 Mar 2022 12:38:04 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id A94421F47263 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755483; bh=77EXBZHZKJfw+bMSetijgv88NZsVwqAJYYAOhg0Ep10=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nTv+anF6T6CZSDOWJpi/e7JMPAiQ+OiqDq+zoXcivpfWj7RiIAT/QPwgP5BPFCfv5 gsIO6FqhhxY2Ww7s8nunjbRmJ7/l517nuWJls4QZgKjjY0P/hRvGxLO/F7Te92Lst6 FWM6R7qfFqcEtMT2V1TzcnkeNbQHvtGGd35gaKbqaGQlZgehqCyFaENvVuQLvYE5RI sEx0D+LS5k7zGfAk96yg3bm//7/ufqz6jkvHRMxAezteFD/K3ZJc97vMbwMXsC2TmH awsoiK0OybnrqBRvZLR/Wbl8SNZ7MUkVd79MskUJ7f0OjxV4xGxeg/3iiPepvnZ0mG UihnBY7ifC5cQ== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 18/23] media: rkvdec: Ensure decoded resolution fit coded resolution Date: Thu, 31 Mar 2022 15:37:20 -0400 Message-Id: <20220331193726.289559-19-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman Ensure decoded CAPTURE buffer resolution is larger or equal to the coded OUTPUT buffer resolution. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index b6376eaa92d7..21b6e7dc5181 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -279,6 +279,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, pix_mp->pixelformat = coded_desc->decoded_fmts[0]; /* Always apply the frmsize constraint of the coded end. */ + pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width); + pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height); v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height, &coded_desc->frmsize); From patchwork Thu Mar 31 19:37:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555436 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 777F4C433FE for ; Thu, 31 Mar 2022 19:38:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240929AbiCaTk3 (ORCPT ); Thu, 31 Mar 2022 15:40:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240919AbiCaTkF (ORCPT ); Thu, 31 Mar 2022 15:40:05 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC68823D776; Thu, 31 Mar 2022 12:38:08 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 3B01D1F47267 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755487; bh=OT93d1YFbCsPl7b/q8Q9Ng7KBwuAX7su/L0Pt2JnS6E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BLW245FpFm1U2UG9Q60ps8Hy5eaR5MstdHCKZC9iaLqioT1JxP0Ik2h47Oz0cI5q+ nymV4c/TmmiMxn74T88xYmfybIfMYgJkv7o77E3jpk70cL23DhF7aRtguFzaLrnqoh dHiZNVYcZUOHsGjrnr6NITnlqQ6ObBxMZw3arQiNXg8cbPw49iMuSJwXkUE08cy0/+ wZyuoDSxWdCWvM3BZvad4t/+1W0pN6HEs7UWwzbMTzr4PkUwwp9hyRbQP1JYDxc0GM a04j+gWFgGlMgLmR/4lgoYf0zjTgy5IjANtDtQFDLV5J+K13RE4wF6N8SQaxZ8sHkO diWgQU/zMSnQQ== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Sebastian Fricke , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 20/23] media: hantro: Enable HOLD_CAPTURE_BUF for H.264 Date: Thu, 31 Mar 2022 15:37:22 -0400 Message-Id: <20220331193726.289559-21-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This is needed to optimize field decoding. Each field will be decoded into the same capture buffer. To be able to queue multiple buffers, we need to be able to ask the driver to hold the capture buffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_v4l2.c | 25 ++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c index 67148ba346f5..50d636678ff3 100644 --- a/drivers/staging/media/hantro/hantro_v4l2.c +++ b/drivers/staging/media/hantro/hantro_v4l2.c @@ -409,6 +409,30 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc) } } +static void +hantro_update_requires_hold_capture_buf(struct hantro_ctx *ctx, u32 fourcc) +{ + struct vb2_queue *vq; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + switch (fourcc) { + case V4L2_PIX_FMT_JPEG: + case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_VP8_FRAME: + case V4L2_PIX_FMT_HEVC_SLICE: + case V4L2_PIX_FMT_VP9_FRAME: + vq->subsystem_flags &= ~(VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF); + break; + case V4L2_PIX_FMT_H264_SLICE: + vq->subsystem_flags |= VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF; + break; + default: + break; + } +} + static int hantro_set_fmt_out(struct hantro_ctx *ctx, struct v4l2_pix_format_mplane *pix_mp) { @@ -472,6 +496,7 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx, ctx->dst_fmt.quantization = pix_mp->quantization; hantro_update_requires_request(ctx, pix_mp->pixelformat); + hantro_update_requires_hold_capture_buf(ctx, pix_mp->pixelformat); vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode); vpu_debug(0, "fmt - w: %d, h: %d\n", From patchwork Thu Mar 31 19:37:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 555435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E83CC433EF for ; Thu, 31 Mar 2022 19:38:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240949AbiCaTkj (ORCPT ); Thu, 31 Mar 2022 15:40:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241060AbiCaTkP (ORCPT ); Thu, 31 Mar 2022 15:40:15 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 413492414D1; Thu, 31 Mar 2022 12:38:12 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id C572A1F47264 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1648755491; bh=XEgENTYjCMT2Anghn2qsNUIMLssrmzXSWUAtGOk2EIY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E2gxoqJYVViah1bnCh0Ns/I6N5AWWkUGJ5pwWF1M2iGOkatQdu7HYSfXKLCPR6Vjs CIOdDSZTD8YGL/O0L0HIIUraQV/W+PS2aZqG0ebyX3gYmT0hnDmMfbqpGI4DIX6HgC +CXc0sS+dZYb/4sDd8TzDpDaTyiBaREh+L3XbxjKYDjZTHJGx8q+UWB290VIDhPwvO bqKYSgcQr5KZ7ycx3sB7ZR4Cp+0a9HHvql1YJ7trdLRBLGv6ZvzHqSXVGH3FI4qSNC kq2I4PWNBc6HPvyPsvZvKya5e2900uSShQ/0cLwh2RVgLekIsUANMF8sKfvUT1yfBK HeSQf8SQMqmMg== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: kernel@collabora.com, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 22/23] media: hantro: h264: Make dpb entry management more robust Date: Thu, 31 Mar 2022 15:37:24 -0400 Message-Id: <20220331193726.289559-23-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220331193726.289559-1-nicolas.dufresne@collabora.com> References: <20220331193726.289559-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman The driver maintains stable slot locations for reference pictures. This change makes the code more robust by using the reference_ts as key and by marking all entries invalid right from the start. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 228629fb3cdf..7377fc26f780 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -258,8 +258,7 @@ static void prepare_table(struct hantro_ctx *ctx) static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a, const struct v4l2_h264_dpb_entry *b) { - return a->top_field_order_cnt == b->top_field_order_cnt && - a->bottom_field_order_cnt == b->bottom_field_order_cnt; + return a->reference_ts == b->reference_ts; } static void update_dpb(struct hantro_ctx *ctx) @@ -273,13 +272,13 @@ static void update_dpb(struct hantro_ctx *ctx) /* Disable all entries by default. */ for (i = 0; i < ARRAY_SIZE(ctx->h264_dec.dpb); i++) - ctx->h264_dec.dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE; + ctx->h264_dec.dpb[i].flags = 0; /* Try to match new DPB entries with existing ones by their POCs. */ for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i]; - if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) + if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) continue; /* @@ -290,8 +289,7 @@ static void update_dpb(struct hantro_ctx *ctx) struct v4l2_h264_dpb_entry *cdpb; cdpb = &ctx->h264_dec.dpb[j]; - if (cdpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE || - !dpb_entry_match(cdpb, ndpb)) + if (!dpb_entry_match(cdpb, ndpb)) continue; *cdpb = *ndpb;