From patchwork Wed Mar 11 17:42:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 210671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1F8CC2BB1D for ; Wed, 11 Mar 2020 17:47:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7075C2073E for ; Wed, 11 Mar 2020 17:47:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730649AbgCKRrP (ORCPT ); Wed, 11 Mar 2020 13:47:15 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:33054 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730487AbgCKRrP (ORCPT ); Wed, 11 Mar 2020 13:47:15 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id 639D72912DD From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , Nicolas Dufresne , kernel@collabora.com, Jonas Karlman , Heiko Stuebner , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Ezequiel Garcia Subject: [PATCH 1/6] v4l2-mem2mem: return CAPTURE buffer first Date: Wed, 11 Mar 2020 14:42:55 -0300 Message-Id: <20200311174300.19407-2-ezequiel@collabora.com> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200311174300.19407-1-ezequiel@collabora.com> References: <20200311174300.19407-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org When the request API is used, typically an OUTPUT (src) buffer will be part of a request. A userspace process will be typically blocked, waiting on the request file descriptor. Returning the OUTPUT (src) buffer will wake-up such processes, who will immediately attempt to dequeue the CAPTURE buffer, only to find it's still unavailable. Therefore, change v4l2_m2m_buf_done_and_job_finish returning the CAPTURE (dst) buffer first, to avoid signalling the request file descriptor prematurely, i.e. before the CAPTURE buffer is done. When the request API is not used, this change should have no impact. Signed-off-by: Ezequiel Garcia --- drivers/media/v4l2-core/v4l2-mem2mem.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c index 8986c31176e9..62ac9424c92a 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -504,12 +504,21 @@ void v4l2_m2m_buf_done_and_job_finish(struct v4l2_m2m_dev *m2m_dev, if (WARN_ON(!src_buf || !dst_buf)) goto unlock; - v4l2_m2m_buf_done(src_buf, state); dst_buf->is_held = src_buf->flags & V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF; if (!dst_buf->is_held) { v4l2_m2m_dst_buf_remove(m2m_ctx); v4l2_m2m_buf_done(dst_buf, state); } + /* + * If the request API is being used, returning the OUTPUT + * (src) buffer will wake-up any process waiting on the + * request file descriptor. + * + * Therefore, return the CAPTURE (dst) buffer first, + * to avoid signalling the request file descriptor + * before the CAPTURE buffer is done. + */ + v4l2_m2m_buf_done(src_buf, state); schedule_next = _v4l2_m2m_job_finish(m2m_dev, m2m_ctx); unlock: spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags); From patchwork Wed Mar 11 17:42:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 210670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF7D0C0044D for ; Wed, 11 Mar 2020 17:47:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 47EA420734 for ; Wed, 11 Mar 2020 17:47:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730700AbgCKRrX (ORCPT ); Wed, 11 Mar 2020 13:47:23 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:33080 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730645AbgCKRrW (ORCPT ); Wed, 11 Mar 2020 13:47:22 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id C9A522912DD From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , Nicolas Dufresne , kernel@collabora.com, Jonas Karlman , Heiko Stuebner , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Ezequiel Garcia Subject: [PATCH 3/6] hantro: Use v4l2_m2m_buf_done_and_job_finish Date: Wed, 11 Mar 2020 14:42:57 -0300 Message-Id: <20200311174300.19407-4-ezequiel@collabora.com> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200311174300.19407-1-ezequiel@collabora.com> References: <20200311174300.19407-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Let the core sort out the nuances of returning buffers to userspace, by using the v4l2_m2m_buf_done_and_job_finish helper. This change also removes usage of buffer sequence fields, which shouldn't have any meaning for stateless decoders. Signed-off-by: Ezequiel Garcia --- drivers/staging/media/hantro/hantro_drv.c | 27 ++++++++--------------- 1 file changed, 9 insertions(+), 18 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c index 0b1200fc0e1a..ec889d755cd6 100644 --- a/drivers/staging/media/hantro/hantro_drv.c +++ b/drivers/staging/media/hantro/hantro_drv.c @@ -94,32 +94,23 @@ static void hantro_job_finish(struct hantro_dev *vpu, unsigned int bytesused, enum vb2_buffer_state result) { - struct vb2_v4l2_buffer *src, *dst; int ret; pm_runtime_mark_last_busy(vpu->dev); pm_runtime_put_autosuspend(vpu->dev); clk_bulk_disable(vpu->variant->num_clocks, vpu->clocks); - src = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); - dst = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx); - - if (WARN_ON(!src)) - return; - if (WARN_ON(!dst)) - return; - - src->sequence = ctx->sequence_out++; - dst->sequence = ctx->sequence_cap++; - - ret = ctx->buf_finish(ctx, &dst->vb2_buf, bytesused); - if (ret) - result = VB2_BUF_STATE_ERROR; + if (ctx->buf_finish) { + struct vb2_v4l2_buffer *dst; - v4l2_m2m_buf_done(src, result); - v4l2_m2m_buf_done(dst, result); + dst = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx); + ret = ctx->buf_finish(ctx, &dst->vb2_buf, bytesused); + if (ret) + result = VB2_BUF_STATE_ERROR; + } - v4l2_m2m_job_finish(vpu->m2m_dev, ctx->fh.m2m_ctx); + v4l2_m2m_buf_done_and_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx, + result); } void hantro_irq_done(struct hantro_dev *vpu, unsigned int bytesused, From patchwork Wed Mar 11 17:42:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 210669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E20BAC0044D for ; Wed, 11 Mar 2020 17:47:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 76A7B20746 for ; Wed, 11 Mar 2020 17:47:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730728AbgCKRra (ORCPT ); Wed, 11 Mar 2020 13:47:30 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:33110 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730645AbgCKRra (ORCPT ); Wed, 11 Mar 2020 13:47:30 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id 4B97A2912E8 From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , Nicolas Dufresne , kernel@collabora.com, Jonas Karlman , Heiko Stuebner , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Ezequiel Garcia Subject: [PATCH 5/6] hantro: Move H264 motion vector calculation to a helper Date: Wed, 11 Mar 2020 14:42:59 -0300 Message-Id: <20200311174300.19407-6-ezequiel@collabora.com> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200311174300.19407-1-ezequiel@collabora.com> References: <20200311174300.19407-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Move the extra bytes calculation that are needed for H264 motion vector to a helper. This is just a cosmetic cleanup. Signed-off-by: Ezequiel Garcia --- drivers/staging/media/hantro/hantro.h | 4 --- drivers/staging/media/hantro/hantro_hw.h | 31 ++++++++++++++++++++++ drivers/staging/media/hantro/hantro_v4l2.c | 25 ++--------------- 3 files changed, 33 insertions(+), 27 deletions(-) diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h index 327ddef45345..2089f88a44a2 100644 --- a/drivers/staging/media/hantro/hantro.h +++ b/drivers/staging/media/hantro/hantro.h @@ -26,10 +26,6 @@ #include "hantro_hw.h" -#define MB_DIM 16 -#define MB_WIDTH(w) DIV_ROUND_UP(w, MB_DIM) -#define MB_HEIGHT(h) DIV_ROUND_UP(h, MB_DIM) - struct hantro_ctx; struct hantro_codec_ops; diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 2398d4c1f207..435f30ae89fd 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -18,6 +18,10 @@ #define DEC_8190_ALIGN_MASK 0x07U +#define MB_DIM 16 +#define MB_WIDTH(w) DIV_ROUND_UP(w, MB_DIM) +#define MB_HEIGHT(h) DIV_ROUND_UP(h, MB_DIM) + struct hantro_dev; struct hantro_ctx; struct hantro_buf; @@ -175,6 +179,33 @@ void hantro_g1_h264_dec_run(struct hantro_ctx *ctx); int hantro_h264_dec_init(struct hantro_ctx *ctx); void hantro_h264_dec_exit(struct hantro_ctx *ctx); +static inline size_t +hantro_h264_mv_size(unsigned int width, unsigned int height) +{ + /* + * A decoded 8-bit 4:2:0 NV12 frame may need memory for up to + * 448 bytes per macroblock with additional 32 bytes on + * multi-core variants. + * + * The H264 decoder needs extra space on the output buffers + * to store motion vectors. This is needed for reference + * frames and only if the format is non-post-processed NV12. + * + * Memory layout is as follow: + * + * +---------------------------+ + * | Y-plane 256 bytes x MBs | + * +---------------------------+ + * | UV-plane 128 bytes x MBs | + * +---------------------------+ + * | MV buffer 64 bytes x MBs | + * +---------------------------+ + * | MC sync 32 bytes | + * +---------------------------+ + */ + return 64 * MB_WIDTH(width) * MB_WIDTH(height) + 32; +} + void hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx); void rk3399_vpu_mpeg2_dec_run(struct hantro_ctx *ctx); void hantro_mpeg2_dec_copy_qtable(u8 *qtable, diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c index 3142ab6697d5..458b502ff01b 100644 --- a/drivers/staging/media/hantro/hantro_v4l2.c +++ b/drivers/staging/media/hantro/hantro_v4l2.c @@ -273,32 +273,11 @@ static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f, /* Fill remaining fields */ v4l2_fill_pixfmt_mp(pix_mp, fmt->fourcc, pix_mp->width, pix_mp->height); - /* - * A decoded 8-bit 4:2:0 NV12 frame may need memory for up to - * 448 bytes per macroblock with additional 32 bytes on - * multi-core variants. - * - * The H264 decoder needs extra space on the output buffers - * to store motion vectors. This is needed for reference - * frames and only if the format is non-post-processed NV12. - * - * Memory layout is as follow: - * - * +---------------------------+ - * | Y-plane 256 bytes x MBs | - * +---------------------------+ - * | UV-plane 128 bytes x MBs | - * +---------------------------+ - * | MV buffer 64 bytes x MBs | - * +---------------------------+ - * | MC sync 32 bytes | - * +---------------------------+ - */ if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE && !hantro_needs_postproc(ctx, fmt)) pix_mp->plane_fmt[0].sizeimage += - 64 * MB_WIDTH(pix_mp->width) * - MB_WIDTH(pix_mp->height) + 32; + hantro_h264_mv_size(pix_mp->width, + pix_mp->height); } else if (!pix_mp->plane_fmt[0].sizeimage) { /* * For coded formats the application can specify