From patchwork Wed Aug 2 21:45:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 709878 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D3FCC04FE0 for ; Wed, 2 Aug 2023 21:47:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229894AbjHBVrL (ORCPT ); Wed, 2 Aug 2023 17:47:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233610AbjHBVqn (ORCPT ); Wed, 2 Aug 2023 17:46:43 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7B0A4224 for ; Wed, 2 Aug 2023 14:46:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691012768; x=1722548768; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jCvzY1zp5KE9yonAiKuUCTNHerxKkvy7vnFRprLXYAI=; b=JJXHXpobkCXcjkQdVSWyh7DzxV//QumugI1x5qBXhD7W8peiZvI6ZEoV zV3BURgOEwluJ/MKRzmZ59bviwbdgu3JfX5jGiU35145OuQHKgUrrpqcg ENEWkwD1yjdXQ0mxLOQc0b0m1gn7B0vlHUgvSwomQb1js0KBq9isomHZa sspirj9IbG5/bDc8FZr+uzW6lKwwncqWi1B2HjtRGFI066mHtn74z20YB oKGNjIqKKMMPCHOOKVT7lCN3oRec5XANJz5VfVBkaDTGc6uERjZe+4glV gRXduGwxT8SggLGo2cVR2zNwBdQBKRgGPLBZyI8tfuXXmXksQE2Bt4ghq A==; X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="372442044" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="372442044" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="853009277" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="853009277" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:01 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id DB2D8120AB8; Thu, 3 Aug 2023 00:45:57 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v2 1/9] media: Documentation: Align numbered list Date: Thu, 3 Aug 2023 00:45:48 +0300 Message-Id: <20230802214556.180589-2-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230802214556.180589-1-sakari.ailus@linux.intel.com> References: <20230802214556.180589-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Align lines for numbered list so that Sphinx produces an uniform output for all list entries. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-subdev.rst | 27 +++++++++---------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index a4f1df7093e8..d737c9530dbd 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -579,20 +579,19 @@ is started. There are three steps in configuring the streams: -1) Set up links. Connect the pads between sub-devices using the :ref:`Media -Controller API ` - -2) Streams. Streams are declared and their routing is configured by -setting the routing table for the sub-device using -:ref:`VIDIOC_SUBDEV_S_ROUTING ` ioctl. Note that -setting the routing table will reset formats and selections in the -sub-device to default values. - -3) Configure formats and selections. Formats and selections of each stream -are configured separately as documented for plain sub-devices in -:ref:`format-propagation`. The stream ID is set to the same stream ID -associated with either sink or source pads of routes configured using the -:ref:`VIDIOC_SUBDEV_S_ROUTING ` ioctl. +1) Set up links. Connect the pads between sub-devices using the + :ref:`Media Controller API ` + +2) Streams. Streams are declared and their routing is configured by setting the + routing table for the sub-device using :ref:`VIDIOC_SUBDEV_S_ROUTING + ` ioctl. Note that setting the routing table will + reset formats and selections in the sub-device to default values. + +3) Configure formats and selections. Formats and selections of each stream are + configured separately as documented for plain sub-devices in + :ref:`format-propagation`. The stream ID is set to the same stream ID + associated with either sink or source pads of routes configured using the + :ref:`VIDIOC_SUBDEV_S_ROUTING ` ioctl. Multiplexed streams setup example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ From patchwork Wed Aug 2 21:45:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 709879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D487C04A94 for ; Wed, 2 Aug 2023 21:47:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233879AbjHBVrJ (ORCPT ); Wed, 2 Aug 2023 17:47:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233828AbjHBVqm (ORCPT ); Wed, 2 Aug 2023 17:46:42 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7F084225 for ; Wed, 2 Aug 2023 14:46:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691012768; x=1722548768; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=x+LP+4lfNYxrrkLWfRafgulLicRZ0Ly2LjAWyJO66Mc=; b=isZrcCKy1iitCwVzOylsWwED1hTyaYZQ4Q9YN71I/KD07rKWoDPw5RfB uwnQlt/SyvG5Q56IB88jsRZ4Avm2oJpXtP6RK7T2EmlrqdwEKW4sMxoxe AJL5QOBeyyF6w3mzmaygBFUSnjVDDvPDVe8hEPOoD/2Lxy2ozRbq514ag 5wCvdYrqDGViVa7YxY8nCW1Z3in/cLpOL9I1hhd1pvuH7BQuXHLbfAw77 hSfRXCw+5pD+8f3qrgzLqYK/6b2409+1sO8odz5KKNpBgc/K026GpAb8i IVmkFQbbFQhU2HzfDHFWWV+ctBvxrhO08jbU5mLaipBuCEnEsVwUObPsw Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="372442049" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="372442049" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="853009295" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="853009295" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:02 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 45450120BBA; Thu, 3 Aug 2023 00:45:59 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v2 2/9] media: mc: Check pad flag validity Date: Thu, 3 Aug 2023 00:45:49 +0300 Message-Id: <20230802214556.180589-3-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230802214556.180589-1-sakari.ailus@linux.intel.com> References: <20230802214556.180589-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Check the validity of pad flags on entity init. Exactly one of the flags must be set. Signed-off-by: Sakari Ailus --- drivers/media/mc/mc-entity.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 83468d4a440b..4991281dcccc 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -197,6 +197,7 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, struct media_device *mdev = entity->graph_obj.mdev; struct media_pad *iter; unsigned int i = 0; + int ret = 0; if (num_pads >= MEDIA_ENTITY_MAX_PADS) return -E2BIG; @@ -210,6 +211,14 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, media_entity_for_each_pad(entity, iter) { iter->entity = entity; iter->index = i++; + + if (hweight32(iter->flags & (MEDIA_PAD_FL_SINK | + MEDIA_PAD_FL_SOURCE)) + != 1) { + ret = -EINVAL; + break; + } + if (mdev) media_gobj_create(mdev, MEDIA_GRAPH_PAD, &iter->graph_obj); @@ -218,7 +227,7 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, if (mdev) mutex_unlock(&mdev->graph_mutex); - return 0; + return ret; } EXPORT_SYMBOL_GPL(media_entity_pads_init); From patchwork Wed Aug 2 21:45:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 709440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E55AC001E0 for ; Wed, 2 Aug 2023 21:47:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233807AbjHBVra (ORCPT ); Wed, 2 Aug 2023 17:47:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233820AbjHBVrI (ORCPT ); Wed, 2 Aug 2023 17:47:08 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 193643A86 for ; Wed, 2 Aug 2023 14:46:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691012791; x=1722548791; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=r/SIjrwUReb/l+aDUKCinWddolNalKr3CmKEm3N7G3M=; b=XpYIASzBv1foEU3LRJQ8ZsilEW0xJZuzGjTMUGte3wnrlbOdWFE4HgjJ BuNYdqmlb9/1SQaFzJlhz0ZeuygsmRw3cON3n1x9tGQ6HeUjO6YimLAdh kQzUyYahDFK2Dq459D0l5l20D6XmAJDXTVAAz3g1/Tr8WbOYuac+DTKCX I6cUe58TqqKpRGz3j93YGFSsJHDezDAsP8iQB7g9FkYkqucfRx2o6UBnt 41V1zxDlhkqy8TUuou7Xd50//+gstGlIU/WggYTK0Oe71kQtGDeYJCLqI eYV9YwG/bvLR/8ZlcyUqx67B42E7ys7Qs02hiJP0J/uPM99ufMeWXx774 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="372442055" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="372442055" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="853009323" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="853009323" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:04 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id E17FC120BBB; Thu, 3 Aug 2023 00:46:00 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v2 3/9] media: mc: Add INTERNAL pad flag Date: Thu, 3 Aug 2023 00:45:50 +0300 Message-Id: <20230802214556.180589-4-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230802214556.180589-1-sakari.ailus@linux.intel.com> References: <20230802214556.180589-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Internal source pads will be used as routing endpoints in V4L2 [GS]_ROUTING IOCTLs, to indicate that the stream begins in the entity. Also prevent creating links to pads that have been flagged as internal. Signed-off-by: Sakari Ailus --- Documentation/userspace-api/media/glossary.rst | 6 ++++++ Documentation/userspace-api/media/mediactl/media-types.rst | 6 ++++++ drivers/media/mc/mc-entity.c | 6 +++++- include/uapi/linux/media.h | 1 + 4 files changed, 18 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/glossary.rst b/Documentation/userspace-api/media/glossary.rst index 96a360edbf3b..f7b99a4527c7 100644 --- a/Documentation/userspace-api/media/glossary.rst +++ b/Documentation/userspace-api/media/glossary.rst @@ -173,6 +173,12 @@ Glossary An integrated circuit that integrates all components of a computer or other electronic systems. +_media-glossary-stream: + Stream + A distinct flow of data (image data or metadata) over a media pipeline + from source to sink. A source may be e.g. an image sensor and a sink + e.g. a memory buffer. + V4L2 API **V4L2 userspace API** diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/Documentation/userspace-api/media/mediactl/media-types.rst index 0ffeece1e0c8..28941da27790 100644 --- a/Documentation/userspace-api/media/mediactl/media-types.rst +++ b/Documentation/userspace-api/media/mediactl/media-types.rst @@ -361,6 +361,7 @@ Types and flags used to represent the media graph elements .. _MEDIA-PAD-FL-SINK: .. _MEDIA-PAD-FL-SOURCE: .. _MEDIA-PAD-FL-MUST-CONNECT: +.. _MEDIA-PAD-FL-INTERNAL: .. flat-table:: Media pad flags :header-rows: 0 @@ -382,6 +383,11 @@ Types and flags used to represent the media graph elements when this flag isn't set; the absence of the flag doesn't imply there is none. + * - ``MEDIA_PAD_FL_INTERNAL`` + - The internal flag indicates an internal pad that has no external + connections. Such a pad shall not be connected with a link. The + internal flag indicates that the :ref:``stream + `` either starts or ends in the entity. One and only one of ``MEDIA_PAD_FL_SINK`` and ``MEDIA_PAD_FL_SOURCE`` must be set for every pad. diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 4991281dcccc..03a9e0b8ebab 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1071,7 +1071,8 @@ int media_get_pad_index(struct media_entity *entity, u32 pad_type, for (i = 0; i < entity->num_pads; i++) { if ((entity->pads[i].flags & - (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE)) != pad_type) + (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE | + MEDIA_PAD_FL_INTERNAL)) != pad_type) continue; if (entity->pads[i].sig_type == sig_type) @@ -1094,6 +1095,9 @@ media_create_pad_link(struct media_entity *source, u16 source_pad, return -EINVAL; if (WARN_ON(!(source->pads[source_pad].flags & MEDIA_PAD_FL_SOURCE))) return -EINVAL; + if (WARN_ON(source->pads[source_pad].flags & MEDIA_PAD_FL_SOURCE && + source->pads[source_pad].flags & MEDIA_PAD_FL_INTERNAL)) + return -EINVAL; if (WARN_ON(!(sink->pads[sink_pad].flags & MEDIA_PAD_FL_SINK))) return -EINVAL; diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h index 1c80b1d6bbaf..80cfd12a43fc 100644 --- a/include/uapi/linux/media.h +++ b/include/uapi/linux/media.h @@ -208,6 +208,7 @@ struct media_entity_desc { #define MEDIA_PAD_FL_SINK (1U << 0) #define MEDIA_PAD_FL_SOURCE (1U << 1) #define MEDIA_PAD_FL_MUST_CONNECT (1U << 2) +#define MEDIA_PAD_FL_INTERNAL (1U << 3) struct media_pad_desc { __u32 entity; /* entity ID */ From patchwork Wed Aug 2 21:45:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 709439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20F98C00528 for ; Wed, 2 Aug 2023 21:47:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233884AbjHBVre (ORCPT ); Wed, 2 Aug 2023 17:47:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233880AbjHBVrJ (ORCPT ); Wed, 2 Aug 2023 17:47:09 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C48E3A89 for ; Wed, 2 Aug 2023 14:46:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691012791; x=1722548791; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a3O9/XhkuF/Pz3RnPPviZ8nyts04CWuGNflXANcunag=; b=en2VPKy8bXbfbbnEGavMWoh1nxiZXJ0fu5s+iEuBcC5BF2J4rxyB3Zj7 df5t/oSO/222u6krdxdlbAXxzYuGvWIgo0iAwzo8ViyyXtT6UpfaVQ1Rm zhaVUVhdu9lrrCCBzlrB62wmuMBzeH9I1pNn8X+pSxqIKKmOBcl7CUdfx 4BUZ6fHcp+E0OFa9VY5yDYCbp/6jGen7Vxhx5jbLN5kxHh3FUKi4STQsy 7ggoyR3ROoBsZhboROjDz871F+rb+yOvnGtvJE+jHm03m9fXFeCA8Cy5l AA4QzRu1hBtuZswbVpWyRMZvOu9DLqR5rnm7VwEOwVdnfvAfB0y+uE2qW g==; X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="372442061" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="372442061" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="853009327" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="853009327" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:05 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 45422122177; Thu, 3 Aug 2023 00:46:02 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v2 4/9] media: uapi: v4l: Document source routes Date: Thu, 3 Aug 2023 00:45:51 +0300 Message-Id: <20230802214556.180589-5-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230802214556.180589-1-sakari.ailus@linux.intel.com> References: <20230802214556.180589-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Document how internal pads are used on source routes. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-subdev.rst | 117 ++++++++++++++++++ 1 file changed, 117 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index d737c9530dbd..0e3ca6d6d81f 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -551,6 +551,27 @@ A stream at a specific point in the media pipeline is identified by the sub-device and a (pad, stream) pair. For sub-devices that do not support multiplexed streams the 'stream' field is always 0. +.. _v4l2-subdev-source-routes: + +Internal pads and source routes +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Cases where a single sub-device source pad is traversed by multiple streams one +or more of which originate from within the sub-device itself are special as +there is no external sink pad for such routes. In those cases, the sources of +the internally generated streams are represented by internal sink pads, which +are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL ` +pad flag set. + +Internal pads have all the properties of an external pad, including formats and +selections. The format in this case is the source format of the stream. An +internal pad always has a single stream only (0). + +/Source routes/ are routes from an internal sink pad to a(n external) source +pad. Generally source routes are not modifiable but they can be activated and +deactivated using the :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE +` flag, depending on driver capabilities. + Interaction between routes, streams, formats and selections ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -666,3 +687,99 @@ A common way to accomplish this is to start from the sensors and propagate the configurations along the stream towards the receiver, using :ref:`VIDIOC_SUBDEV_S_FMT ` ioctls to configure each stream endpoint in each sub-device. + +Internal pads setup example +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A simple example of a multiplexed stream setup might be as follows: + +- A camera sensor, with one source pad (0) and two internal sink pads (source of + image data (1) and source of embedded data (2)) and two source routes, one + from each internal sink pad towards the (external) source pad. The embedded + data stream needs to be enabled by activating the related route. + +- Multiplexer bridge (Bridge). The bridge has one sink pad, connected to the + sensor (pad 0), and one source pad (pad 1), which outputs two streams. + +- Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0), + connected to the bridge, and two source pads (pads 1-2), going to the DMA + engine. The receiver demultiplexes the incoming streams to the source pads. + +- DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is + connected to a single source pad in the receiver. + +The sensor, the bridge and the receiver are modeled as V4L2 sub-devices, +exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are +modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes. + +To configure this pipeline, the userspace must take the following steps: + +1) Set up media links between entities: connect the sensors to the bridge, +bridge to the receiver, and the receiver to the DMA engines. This step does +not differ from normal non-multiplexed media controller setup. + +2) Configure routing + +.. flat-table:: Camera sensor + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 1/0 + - 0/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from internal sink pad + * - 2/0 + - 0/1 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Metadata stream from internal sink pad + +.. flat-table:: Bridge routing table + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 0/0 + - 1/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from camera sensor + * - 0/1 + - 1/1 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Metadata stream from camera sensor + +.. flat-table:: Receiver routing table + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 0/0 + - 1/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from camera sensor + * - 0/1 + - 2/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Metadata stream from camera sensor + +3) Configure formats and selections + +After configuring routing, the next step is configuring the formats and +selections for the streams. This is similar to performing this step without +streams, with just one exception: the ``stream`` field needs to be assigned +to the value of the stream ID. + +A common way to accomplish this is to start from the sensors and propagate the +configurations along the stream towards the receiver, +using :ref:`VIDIOC_SUBDEV_S_FMT ` ioctls to configure each +stream endpoint in each sub-device. + +The options available in sensor's routing configuration are dictated by hardware +capabilities: typically camera sensors always produce image data stream while +the embedded data stream typically can be either enabled or disabled. From patchwork Wed Aug 2 21:45:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 709877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37DD2C001E0 for ; Wed, 2 Aug 2023 21:47:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233847AbjHBVrb (ORCPT ); Wed, 2 Aug 2023 17:47:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233870AbjHBVrI (ORCPT ); Wed, 2 Aug 2023 17:47:08 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4CE1F3A90 for ; Wed, 2 Aug 2023 14:46:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691012791; x=1722548791; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=baraVIKViWXHBmyB8wvxlZVfLAf9ztDNDwMbJv8RESc=; b=Vg4nPoLoaqhq7jHU5G2W5e8zqgO+fNOWWdEpJLFYhdY+Pe621m79LqP9 uQOVh0cJusrfQRcxTatFgpLdgbR00DBdyCJmabveoR1tCTt4QPqAGNyV8 iV6qSQfqThRO5b2r9hGR7ZVfGM31vD98Tye0Fe6TY9TPQYKnA1EvqL7kW yiNUtX7KaxjNnmNgBH4iRnHEY1dYyZ8nE5v/Lfm9O0o2bGW5Jiy8VVI8V KmdQ3TS4u2QFJmM8AK7zu4iInv/wisheUI5jw//0dUkhP0xBnqgAzQ2M2 wQXZLLMPl5f7d4UANpe9aN/qCvZbf1pVnGWc7728+uAZl/piifZUuatid w==; X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="372442067" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="372442067" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="853009336" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="853009336" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:07 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id EF2AC1204CF; Thu, 3 Aug 2023 00:46:03 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v2 5/9] media: uapi: Add generic serial metadata mbus formats Date: Thu, 3 Aug 2023 00:45:52 +0300 Message-Id: <20230802214556.180589-6-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230802214556.180589-1-sakari.ailus@linux.intel.com> References: <20230802214556.180589-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add generic serial metadata mbus formats. These formats describe data width and packing but not the content itself. The reason for specifying such formats is that the formats as such are fairly device specific but they are still handled by CSI-2 receiver drivers that should not be aware of device specific formats. What makes generic metadata formats possible is that these formats are parsed by software only, after capturing the data to system memory. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-subdev.rst | 8 +- .../media/v4l/subdev-formats.rst | 257 ++++++++++++++++++ include/uapi/linux/media-bus-format.h | 9 + 3 files changed, 270 insertions(+), 4 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index 0e3ca6d6d81f..8197100663b9 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -639,8 +639,8 @@ modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes. To configure this pipeline, the userspace must take the following steps: 1) Set up media links between entities: connect the sensors to the bridge, -bridge to the receiver, and the receiver to the DMA engines. This step does -not differ from normal non-multiplexed media controller setup. + bridge to the receiver, and the receiver to the DMA engines. This step does + not differ from normal non-multiplexed media controller setup. 2) Configure routing @@ -715,8 +715,8 @@ modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes. To configure this pipeline, the userspace must take the following steps: 1) Set up media links between entities: connect the sensors to the bridge, -bridge to the receiver, and the receiver to the DMA engines. This step does -not differ from normal non-multiplexed media controller setup. + bridge to the receiver, and the receiver to the DMA engines. This step does + not differ from normal non-multiplexed media controller setup. 2) Configure routing diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index a3a35eeed708..c615da08502d 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -8234,3 +8234,260 @@ The following table lists the existing metadata formats. both sides of the link and the bus format is a fixed metadata format that is not configurable from userspace. Width and height will be set to 0 for this format. + +Generic Serial Metadata Formats +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Generic serial metadata formats are used on serial busses where the actual data +content is more or less device specific but the data is transmitted and received +by multiple devices that do not process the data in any way, simply writing +it to system memory for processing in software at the end of the pipeline. + +The more specific variant describing the actual data is used on the internal +source pad of the originating sub-device. + +"b" in an array cell signifies a byte of data, followed by the number of byte +and finally the bit number in subscript. "p" indicates a padding bit. + +.. _media-bus-format-generic-meta: + +.. cssclass: longtable + +.. flat-table:: Generic Serial Metadata Formats + :header-rows: 2 + :stub-columns: 0 + + * - Identifier + - Code + - + - :cspan:`23` Data organization + * - + - + - Bit + - 23 + - 22 + - 21 + - 20 + - 19 + - 18 + - 17 + - 16 + - 15 + - 14 + - 13 + - 12 + - 11 + - 10 + - 9 + - 8 + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + * .. _MEDIA-BUS-FMT-META-8: + + - MEDIA_BUS_FMT_META_8 + - 0x8001 + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + * .. _MEDIA-BUS-FMT-META-10: + + - MEDIA_BUS_FMT_META_10 + - 0x8002 + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + * .. _MEDIA-BUS-FMT-META-12: + + - MEDIA_BUS_FMT_META_12 + - 0x8003 + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-14: + + - MEDIA_BUS_FMT_META_14 + - 0x8004 + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-16: + + - MEDIA_BUS_FMT_META_16 + - 0x8005 + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-20: + + - MEDIA_BUS_FMT_META_20 + - 0x8007 + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-24: + + - MEDIA_BUS_FMT_META_24 + - 0x8009 + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index a03c543cb072..9ee031397372 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -173,4 +173,13 @@ */ #define MEDIA_BUS_FMT_METADATA_FIXED 0x7001 +/* Generic line based metadata formats for serial buses. Next is 0x8008. */ +#define MEDIA_BUS_FMT_META_8 0x8001 +#define MEDIA_BUS_FMT_META_10 0x8002 +#define MEDIA_BUS_FMT_META_12 0x8003 +#define MEDIA_BUS_FMT_META_14 0x8004 +#define MEDIA_BUS_FMT_META_16 0x8005 +#define MEDIA_BUS_FMT_META_20 0x8006 +#define MEDIA_BUS_FMT_META_24 0x8007 + #endif /* __LINUX_MEDIA_BUS_FORMAT_H */ From patchwork Wed Aug 2 21:45:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 709876 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C44CFC04A94 for ; Wed, 2 Aug 2023 21:47:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233754AbjHBVrv (ORCPT ); Wed, 2 Aug 2023 17:47:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44028 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232776AbjHBVr0 (ORCPT ); Wed, 2 Aug 2023 17:47:26 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDA243C20 for ; Wed, 2 Aug 2023 14:46:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691012807; x=1722548807; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mtd2OlaD2uF9uQfv/VWnnRlsGNgouPl7W3/VQMYJfmg=; b=PRayyqKfAeR9jNhbd8Bqtqm/T2oVmAX8uKuxBAHKExpQlEp51omaRfwj 68AW60Zxc+ReYWroZeQmOcsuSomtE0fGV9StXXeoVFuYEEclelSKEZxCM FyXFZzBbHyEiI4rrbHulbh7/wIi0eeF4vmv0W5znOQL/zXl8bjLR4Qbkt kKNWKwm7lfuUORahq9muukHfmWMnljtgZpXZ48hyChs/XhxcSCC/1qn4R yAvRgrq+dSAweec1mJdXKxf/fh0dKGfeOzxTvF5DNqfY3evIllM5XAXip X+rCHCzMLd86nwMDF2BJAT2L1AtocghFurtkOSN0bde9X8bJQzGYIoBwP g==; X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="372442071" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="372442071" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="853009354" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="853009354" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:08 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 4FDE7120AB8; Thu, 3 Aug 2023 00:46:05 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v2 6/9] media: uapi: Document which mbus format fields are valid for metadata Date: Thu, 3 Aug 2023 00:45:53 +0300 Message-Id: <20230802214556.180589-7-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230802214556.180589-1-sakari.ailus@linux.intel.com> References: <20230802214556.180589-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Now that metadata mbus formats have been added, it is necessary to define which fields in struct v4l2_mbus_format are applicable to them (not many). Signed-off-by: Sakari Ailus --- include/uapi/linux/v4l2-mediabus.h | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/include/uapi/linux/v4l2-mediabus.h b/include/uapi/linux/v4l2-mediabus.h index 6b07b73473b5..3cadb3b58b85 100644 --- a/include/uapi/linux/v4l2-mediabus.h +++ b/include/uapi/linux/v4l2-mediabus.h @@ -19,12 +19,18 @@ * @width: image width * @height: image height * @code: data format code (from enum v4l2_mbus_pixelcode) - * @field: used interlacing type (from enum v4l2_field) - * @colorspace: colorspace of the data (from enum v4l2_colorspace) - * @ycbcr_enc: YCbCr encoding of the data (from enum v4l2_ycbcr_encoding) - * @hsv_enc: HSV encoding of the data (from enum v4l2_hsv_encoding) - * @quantization: quantization of the data (from enum v4l2_quantization) - * @xfer_func: transfer function of the data (from enum v4l2_xfer_func) + * @field: used interlacing type (from enum v4l2_field), not applicable + * to metadata mbus codes + * @colorspace: colorspace of the data (from enum v4l2_colorspace), zero on + * metadata mbus codes + * @ycbcr_enc: YCbCr encoding of the data (from enum v4l2_ycbcr_encoding), zero + * on metadata mbus codes + * @hsv_enc: HSV encoding of the data (from enum v4l2_hsv_encoding), zero on + * metadata mbus codes + * @quantization: quantization of the data (from enum v4l2_quantization), zero + * on metadata mbus codes + * @xfer_func: transfer function of the data (from enum v4l2_xfer_func), zero + * on metadata mbus codes * @flags: flags (V4L2_MBUS_FRAMEFMT_*) * @reserved: reserved bytes that can be later used */ From patchwork Wed Aug 2 21:45:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 709438 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93937C00528 for ; Wed, 2 Aug 2023 21:47:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233793AbjHBVrw (ORCPT ); Wed, 2 Aug 2023 17:47:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233757AbjHBVr1 (ORCPT ); Wed, 2 Aug 2023 17:47:27 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 979A13C25 for ; Wed, 2 Aug 2023 14:46:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691012808; x=1722548808; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zLZsEP9m7c0aA3XJAE3/+rd++V0bB/KkiYVa0+jwjCY=; b=fU+rS6IMMkvrOE2E/9MMLsnaVUMS9IZ+lM0obna9KUcT1kx3r/KW0tBU c2ID/hzAoGwRRDD2v4jc712aOQ6dqBekia+Zu6k+xsTplfwAfyBnP7V4B +NNQaIjRah0O9Rh9kBc9CLA4H8QnuDfiD/zlkEamAYO9G8B9QNlfi+uqX E5+eXVmb8Er9e7uxGGM4ryRMwrcqA90Jy6iD/E0k7Es/3fIZc2osnRhII vVwF8XwlCfwElBzzmiY7vS7WCivyYzgl3Krz9xiz8PCyM61CuLXzRvLEv 7MqJQF0MWdyO/MPFLi4SBMmjnlfyW95gLUTYUx2g/kRmcGhUWr676vfBu Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="372442076" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="372442076" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="853009376" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="853009376" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:09 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 872B7120BBA; Thu, 3 Aug 2023 00:46:06 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v2 7/9] media: uapi: Add a macro to tell whether an mbus code is metadata Date: Thu, 3 Aug 2023 00:45:54 +0300 Message-Id: <20230802214556.180589-8-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230802214556.180589-1-sakari.ailus@linux.intel.com> References: <20230802214556.180589-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add a macro to tell whether a given mbus code is metadata. Signed-off-by: Sakari Ailus --- include/uapi/linux/media-bus-format.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index 9ee031397372..2486b4178c5f 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -182,4 +182,7 @@ #define MEDIA_BUS_FMT_META_20 0x8006 #define MEDIA_BUS_FMT_META_24 0x8007 +#define MEDIA_BUS_FMT_IS_META(code) \ + ((code) & 0xf000 == 0x7000 || (code) & 0xf000 == 0x8000) + #endif /* __LINUX_MEDIA_BUS_FORMAT_H */ From patchwork Wed Aug 2 21:45:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 709875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8241CC001E0 for ; Wed, 2 Aug 2023 21:47:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233757AbjHBVry (ORCPT ); Wed, 2 Aug 2023 17:47:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232769AbjHBVr2 (ORCPT ); Wed, 2 Aug 2023 17:47:28 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC86D3C28 for ; Wed, 2 Aug 2023 14:46:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691012808; x=1722548808; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6MHSV6Nf1CkmOvAz1SebL+rUFkx/VIefq0W7LAJjSyE=; b=Ujxs93MQZfeozS9wZk+x/r7FeDiNVz94U5mNcIgwCVAdx8XOcTY8inCU YiZGWymjJ4IV2cHGeP8omREuu3JFkkxgdfxThIJH9NtBM1xtofQBwoNd6 f/8/qSOTZtx2/I51cmsfWYIGfxdAq2AzdF2pyVv0logNFm9EgWKt8f85j W5WEXGG/PBHcEe2+CF8MEHJ0Ol9ZbEIqkuxc4vMFvLc5uq8d97LSoQ+IA cxW5yrtI3pJMrZRVWeO2TErE3ELb72kS97tsWyx+ekJArYeLfRayZCpR5 GVmtse9SdFukWzh+SH8kRccPpyRhA/gv+5ZGmrEAknQaJjCllHag0Jc/L w==; X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="372442096" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="372442096" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="853009421" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="853009421" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:12 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id AA968120BBB; Thu, 3 Aug 2023 00:46:07 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v2 8/9] media: uapi: Add generic 8-bit metadata format definitions Date: Thu, 3 Aug 2023 00:45:55 +0300 Message-Id: <20230802214556.180589-9-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230802214556.180589-1-sakari.ailus@linux.intel.com> References: <20230802214556.180589-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Generic 8-bit metadata formats define the in-memory data layout but not the format of the data itself. The reasoning for having such formats is to allow CSI-2 receiver drivers to receive and DMA drivers to write the data to memory without knowing a large number of device specific formats. These formats may be used only in conjunction of a Media controller pipeline where the internal pad of the source sub-device defines the specific format of the data (using an mbus code). Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/meta-formats.rst | 1 + .../media/v4l/metafmt-generic.rst | 331 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 8 + include/uapi/linux/videodev2.h | 9 + 4 files changed, 349 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/metafmt-generic.rst diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst index 0bb61fc5bc00..919f595576b9 100644 --- a/Documentation/userspace-api/media/v4l/meta-formats.rst +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst @@ -19,3 +19,4 @@ These formats are used for the :ref:`metadata` interface only. metafmt-vsp1-hgo metafmt-vsp1-hgt metafmt-vivid + metafmt-generic diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst new file mode 100644 index 000000000000..a27bfc721edf --- /dev/null +++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst @@ -0,0 +1,331 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later + +************************************************************************************************************************************************************************************************************************************************************************************************************************** +V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_META_FMT_GENERIC_CSI2_12 ('MC1C'), V4L2_META_FMT_GENERIC_CSI2_14 ('MC1E'), V4L2_META_FMT_GENERIC_CSI2_16 ('MC1G'), V4L2_META_FMT_GENERIC_CSI2_20 ('MC1K'), V4L2_META_FMT_GENERIC_CSI2_24 ('MC1O'), V4L2_META_FMT_GENERIC_CSI2_2_24 ('MC2O') +************************************************************************************************************************************************************************************************************************************************************************************************************************** + + +Generic line-based metadata formats + + +Description +=========== + +These generic line-based metadata formats define the memory layout of the data +without defining the format or meaning of the metadata itself. These formats may +only be used with a Media controller pipeline where the more specific format is +defined in an :ref:`internal source pad ` of the source +sub-device. See also :ref:`source routes `. + +.. _v4l2-meta-fmt-generic-8: + +V4L2_META_FMT_GENERIC_8 +----------------------- + +The V4L2_META_FMT_GENERIC_8 format is a plain 8-bit metadata format. + +This format is also used on CSI-2 on both 8 bits per sample as well as on +16 bits per sample when two bytes of metadata are packed into one sample. + +**Byte Order Of V4L2_META_FMT_GENERIC_8.** +Each cell is one byte. "M" denotes a byte of metadata. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + * - start + 4: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + +.. _v4l2-meta-fmt-generic-csi2-10: + +V4L2_META_FMT_GENERIC_CSI2_10 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_10 contains packed 8-bit generic metadata, 10 bits +for each 8 bits of data. Every four bytes of metadata is followed by a single +byte of padding. The way the data is stored follows the CSI-2 specification. + +This format is also used on CSI-2 on 20 bits per sample format that packs two +bytes of metadata into one sample. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_10.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + * - start + 5: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + +.. _v4l2-meta-fmt-generic-csi2-12: + +V4L2_META_FMT_GENERIC_CSI2_12 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_12 contains packed 8-bit generic metadata, 12 bits +for each 8 bits of data. Every four bytes of metadata is followed by two bytes +of padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_12.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + - p + * - start + 6: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-14: + +V4L2_META_FMT_GENERIC_CSI2_14 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_14 contains packed 8-bit generic metadata, 14 bits +for each 8 bits of data. Every four bytes of metadata is followed by three +bytes of padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_14.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + - p + - p + * - start + 7: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-16: + +V4L2_META_FMT_GENERIC_CSI2_16 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_16 contains packed 8-bit generic metadata, 16 bits +for each 8 bits of data. Every byte of metadata is followed by one byte of +padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_16.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - M\ :sub:`10` + - p + - M\ :sub:`20` + - p + - M\ :sub:`30` + - p + * - start + 8: + - M\ :sub:`01` + - p + - M\ :sub:`11` + - p + - M\ :sub:`21` + - p + - M\ :sub:`31` + - p + +.. _v4l2-meta-fmt-generic-csi2-20: + +V4L2_META_FMT_GENERIC_CSI2_20 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_20 contains packed 8-bit generic metadata, 20 bits +for each 8 bits of data. Every byte of metadata is followed by alternating one +and two bytes of padding. The way the data is stored follows the CSI-2 +specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_20.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - M\ :sub:`10` + - p + - p + - M\ :sub:`20` + - p + - M\ :sub:`30` + - p + - p + * - start + 10: + - M\ :sub:`01` + - p + - M\ :sub:`11` + - p + - p + - M\ :sub:`21` + - p + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-24: + +V4L2_META_FMT_GENERIC_CSI2_24 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_24 contains packed 8-bit generic metadata, 24 bits +for each 8 bits of data. Every byte of metadata is followed by two bytes of +padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_24.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - p + - M\ :sub:`10` + - p + - p + - M\ :sub:`20` + - p + - p + - M\ :sub:`30` + - p + - p + * - start + 12: + - M\ :sub:`01` + - p + - p + - M\ :sub:`11` + - p + - p + - M\ :sub:`21` + - p + - p + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-2-24: + +V4L2_META_FMT_GENERIC_CSI2_2_24 +------------------------------- + +V4L2_META_FMT_GENERIC_CSI2_2_24 contains packed 8-bit generic metadata, 24 bits +for each two times 8 bits of data. Every two bytes of metadata are followed by +one byte of padding. The way the data is stored follows the CSI-2 +specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_2_24.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - p + - M\ :sub:`20` + - M\ :sub:`30` + - p + * - start + 6: + - M\ :sub:`01` + - M\ :sub:`11` + - p + - M\ :sub:`21` + - M\ :sub:`31` + - p + diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 01ba27f2ef87..7174a590428f 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1452,6 +1452,14 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; + case V4L2_META_FMT_GENERIC_8: descr = "8-bit Generic Metadata"; break; + case V4L2_META_FMT_GENERIC_CSI2_10: descr = "8b Generic Meta, 10b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_12: descr = "8b Generic Meta, 12b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_14: descr = "8b Generic Meta, 14b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_16: descr = "8b Generic Meta, 16b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_20: descr = "8b Generic Meta, 20b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_24: descr = "8b Generic Meta, 24b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_2_24: descr = "2x8b Generic Meta, 24b CSI-2"; break; default: /* Compressed formats */ diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 3af6a82d0cad..b890b99d6146 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -836,6 +836,15 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ +#define V4L2_META_FMT_GENERIC_8 v4l2_fourcc('M', 'E', 'T', '8') /* Generic 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_10 v4l2_fourcc('M', 'C', '1', 'A') /* 10-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_12 v4l2_fourcc('M', 'C', '1', 'C') /* 12-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_14 v4l2_fourcc('M', 'C', '1', 'E') /* 14-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_16 v4l2_fourcc('M', 'C', '1', 'G') /* 16-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_20 v4l2_fourcc('M', 'C', '1', 'K') /* 20-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_24 v4l2_fourcc('M', 'C', '1', 'O') /* 24-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_2_24 v4l2_fourcc('M', 'C', '2', 'O') /* 2 bytes of 8-bit metadata, 24-bit CSI-2 packed */ + /* priv field value to indicates that subsequent fields are valid. */ #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe From patchwork Wed Aug 2 21:45:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 709437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D33F9C001E0 for ; Wed, 2 Aug 2023 21:48:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229713AbjHBVsT (ORCPT ); Wed, 2 Aug 2023 17:48:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233483AbjHBVrt (ORCPT ); Wed, 2 Aug 2023 17:47:49 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E8A144BD for ; Wed, 2 Aug 2023 14:47:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691012820; x=1722548820; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=j/FhNB9GQTqscFHP++EqCTzRgo1b4t+aQsBmTdrG1RM=; b=jna0kNZsYC3qT1S8Q1BywtqaG9Qz81c+vBCBaS8SVbspTuODMSugcGuh vRKtslXIKCjTWWmzHxdTi+bP6Xf/N08V9fenvtwXGUBWdA3DIlHO8DGZI WQUAzJimUn8ic/RCwp+M7SGqojijiJNd3iO4OZafzSSOAjNnOp+0iVT3T rKfu+DBCr+D5+kdITVbvp8nxlv++2PibbSJEsE3v6OGc1SKCA3xawPLhh Dcb0GkBptl/sWu5zVTr+j4yeIhwBZten0Ku2biVfcat2qz/HhgrWlVkHm ST3JU36qX7xR0/EdNGV5/MEDV6fiNdGIFDjgpFt/BRZE1JhumvpEF2nrF A==; X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="372442111" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="372442111" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10790"; a="853009428" X-IronPort-AV: E=Sophos;i="6.01,250,1684825200"; d="scan'208";a="853009428" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Aug 2023 14:46:15 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 527181204CF; Thu, 3 Aug 2023 00:46:10 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v2 9/9] media: v4l: Support line-based metadata capture Date: Thu, 3 Aug 2023 00:45:56 +0300 Message-Id: <20230802214556.180589-10-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230802214556.180589-1-sakari.ailus@linux.intel.com> References: <20230802214556.180589-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org many camera sensors, among other devices, transmit embedded data and image data for each CSI-2 frame. This embedded data typically contains register configuration of the sensor that has been used to capture the image data of the same frame. The embedded data is received by the CSI-2 receiver and has the same properties as the image data, including that it is line based: it has width, height and bytesperline (stride). Add these fields to struct v4l2_meta_format and document them. Also add V4L2_FMT_FLAG_META_LINE_BASED to tell a given format is line-based i.e. these fields of struct v4l2_meta_format are valid for it. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-meta.rst | 15 +++++++++++++++ .../userspace-api/media/v4l/vidioc-enum-fmt.rst | 7 +++++++ .../media/videodev2.h.rst.exceptions | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 5 +++-- include/uapi/linux/videodev2.h | 10 ++++++++++ 5 files changed, 36 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/dev-meta.rst b/Documentation/userspace-api/media/v4l/dev-meta.rst index 0e7e1ee1471a..4b24bae6e171 100644 --- a/Documentation/userspace-api/media/v4l/dev-meta.rst +++ b/Documentation/userspace-api/media/v4l/dev-meta.rst @@ -65,3 +65,18 @@ to 0. - ``buffersize`` - Maximum buffer size in bytes required for data. The value is set by the driver. + * - __u32 + - ``width`` + - Width of a line of metadata in samples. Valid when :c:type`v4l2_fmtdesc` + flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``height`` + - Number of rows of metadata. Valid when :c:type`v4l2_fmtdesc` flag + ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``bytesperline`` + - Offset in bytes between the beginning of two consecutive lines. Valid + when :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is + set, otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. diff --git a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst index 000c154b0f98..6d7664345a4e 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst @@ -227,6 +227,13 @@ the ``mbus_code`` field is handled differently: The application can ask to configure the quantization of the capture device when calling the :ref:`VIDIOC_S_FMT ` ioctl with :ref:`V4L2_PIX_FMT_FLAG_SET_CSC ` set. + * - ``V4L2_FMT_FLAG_META_LINE_BASED`` + - 0x0200 + - The metadata format is line-based. In this case the ``width``, + ``height`` and ``bytesperline`` fields of :c:type:`v4l2_meta_format` are + valid. The buffer consists of ``height`` lines, each having ``width`` + bytes of data and offset between the beginning of each two consecutive + lines is ``bytesperline``. Return Value ============ diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions index 3e58aac4ef0b..bdc628e8c1d6 100644 --- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions +++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions @@ -215,6 +215,7 @@ replace define V4L2_FMT_FLAG_CSC_XFER_FUNC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_YCBCR_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_HSV_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_QUANTIZATION fmtdesc-flags +replace define V4L2_FMT_FLAG_META_LINE_BASED fmtdesc-flags # V4L2 timecode types replace define V4L2_TC_TYPE_24FPS timecode-type diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 7174a590428f..cb4eb99a94ab 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -343,8 +343,9 @@ static void v4l_print_format(const void *arg, bool write_only) case V4L2_BUF_TYPE_META_OUTPUT: meta = &p->fmt.meta; pixelformat = meta->dataformat; - pr_cont(", dataformat=%p4cc, buffersize=%u\n", - &pixelformat, meta->buffersize); + pr_cont(", dataformat=%p4cc, buffersize=%u, width=%u, height=%u, bytesperline=%u\n", + &pixelformat, meta->buffersize, meta->width, + meta->height, meta->bytesperline); break; } } diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index b890b99d6146..9cadcf97886c 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -875,6 +875,7 @@ struct v4l2_fmtdesc { #define V4L2_FMT_FLAG_CSC_YCBCR_ENC 0x0080 #define V4L2_FMT_FLAG_CSC_HSV_ENC V4L2_FMT_FLAG_CSC_YCBCR_ENC #define V4L2_FMT_FLAG_CSC_QUANTIZATION 0x0100 +#define V4L2_FMT_FLAG_META_LINE_BASED 0x0200 /* Frame Size and frame rate enumeration */ /* @@ -2418,10 +2419,19 @@ struct v4l2_sdr_format { * struct v4l2_meta_format - metadata format definition * @dataformat: little endian four character code (fourcc) * @buffersize: maximum size in bytes required for data + * @width: number of bytes of data per line (valid for line based + * formats only, see format documentation) + * @height: number of lines of data per buffer (valid for line based + * formats only) + * @bytesperline: offset between the beginnings of two adjacent lines in + * bytes (valid for line based formats only) */ struct v4l2_meta_format { __u32 dataformat; __u32 buffersize; + __u32 width; + __u32 height; + __u32 bytesperline; } __attribute__ ((packed)); /**