From patchwork Tue Aug 8 07:55:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 711789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73C3EC001DB for ; Tue, 8 Aug 2023 16:44:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233360AbjHHQoc (ORCPT ); Tue, 8 Aug 2023 12:44:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232457AbjHHQni (ORCPT ); Tue, 8 Aug 2023 12:43:38 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AF44919E for ; Tue, 8 Aug 2023 08:55:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691510152; x=1723046152; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=faWPaX7FCNBhL/5vyVMl/Wwy9zb2mjg+L/MsKx0doC0=; b=I1oZpVSZpF4G2JP4b90JUxqq0aohRb1GtOGP1amCEzin2qV6wqhec5eO SuiPiM2HKXiVGrZC5emaiUl/Te6Gm7Kl/gUjyWv3NOvh3ktwTC7S1MxJm s3BZUOZyqlmcWFFVKlWaEl6fyF2PUF6vwGNfNjlDEITwZFJKzELah7MDu aNu8NxxMZENnLtUGobHiY7l5Bsqb3iYOgBoGASSvjqGsLXaHfCXOOzRrI nIdPGnt4rE78MaQipCgWRiq6ZhuJ8nOOMcfw2SPwgYMeFRfegS8Mvx8lT 25io4NBlVyW2Oh2ifZ53KN79BYFsbCRyylz3RFWpZtfcm4yPWMbEyNYXA Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="369648844" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="369648844" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="874627374" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:06 -0700 Received: from punajuuri.localdomain (punajuuri.localdomain [192.168.240.130]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 8F9E311FB5E; Tue, 8 Aug 2023 10:56:00 +0300 (EEST) Received: from sailus by punajuuri.localdomain with local (Exim 4.96) (envelope-from ) id 1qTHZE-00ClsH-0V; Tue, 08 Aug 2023 10:55:48 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v3 01/10] media: Documentation: Align numbered list Date: Tue, 8 Aug 2023 10:55:29 +0300 Message-Id: <20230808075538.3043934-2-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> References: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Align lines for numbered list so that Sphinx produces an uniform output for all list entries. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-subdev.rst | 27 +++++++++---------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index a4f1df7093e8..b023918177b5 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -579,20 +579,19 @@ is started. There are three steps in configuring the streams: -1) Set up links. Connect the pads between sub-devices using the :ref:`Media -Controller API ` +1) Set up links. Connect the pads between sub-devices using the + :ref:`Media Controller API ` -2) Streams. Streams are declared and their routing is configured by -setting the routing table for the sub-device using -:ref:`VIDIOC_SUBDEV_S_ROUTING ` ioctl. Note that -setting the routing table will reset formats and selections in the -sub-device to default values. +2) Streams. Streams are declared and their routing is configured by setting the + routing table for the sub-device using :ref:`VIDIOC_SUBDEV_S_ROUTING + ` ioctl. Note that setting the routing table will + reset formats and selections in the sub-device to default values. -3) Configure formats and selections. Formats and selections of each stream -are configured separately as documented for plain sub-devices in -:ref:`format-propagation`. The stream ID is set to the same stream ID -associated with either sink or source pads of routes configured using the -:ref:`VIDIOC_SUBDEV_S_ROUTING ` ioctl. +3) Configure formats and selections. Formats and selections of each stream are + configured separately as documented for plain sub-devices in + :ref:`format-propagation`. The stream ID is set to the same stream ID + associated with either sink or source pads of routes configured using the + :ref:`VIDIOC_SUBDEV_S_ROUTING ` ioctl. Multiplexed streams setup example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -619,8 +618,8 @@ modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes. To configure this pipeline, the userspace must take the following steps: 1) Set up media links between entities: connect the sensors to the bridge, -bridge to the receiver, and the receiver to the DMA engines. This step does -not differ from normal non-multiplexed media controller setup. + bridge to the receiver, and the receiver to the DMA engines. This step does + not differ from normal non-multiplexed media controller setup. 2) Configure routing From patchwork Tue Aug 8 07:55:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 711788 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEE1FC001B0 for ; Tue, 8 Aug 2023 16:45:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230196AbjHHQp1 (ORCPT ); Tue, 8 Aug 2023 12:45:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232513AbjHHQoe (ORCPT ); Tue, 8 Aug 2023 12:44:34 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E99E16BDB for ; Tue, 8 Aug 2023 08:56:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691510168; x=1723046168; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=x+LP+4lfNYxrrkLWfRafgulLicRZ0Ly2LjAWyJO66Mc=; b=bjGmftVL5GLSqV9y+8Zjd9g475+speulmYnGv6XlnNHITLftc1816Zo6 ixVVDFy+o4eIKjHG13GW3f0GCTXSWEfNdpZK3wxyiA+5f8UUYVR1WGL+m QTKLP/GK4lutI4oxr/TanR9tYBqIf21rfnXRsvP8MJU28Pe1J6cXZpVpo l77/rnyjZiaEFhJqcKIBDLBK504hRwcSimTJBTzUM90d8MlDl2mjRq9ji KBfK1DhHlIpvtXUYkVjqdJIHimFz3kFIbfblsrUgATjkVm9DdkW56A2wr S5gdiVmMh8BWwslwdJOYJM/4YIvLt880yDorfshPaNV1O7sfZhfxUoYT6 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="369648855" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="369648855" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="874627378" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:06 -0700 Received: from punajuuri.localdomain (punajuuri.localdomain [192.168.240.130]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 912D912023A; Tue, 8 Aug 2023 10:56:00 +0300 (EEST) Received: from sailus by punajuuri.localdomain with local (Exim 4.96) (envelope-from ) id 1qTHZE-00ClsM-0b; Tue, 08 Aug 2023 10:55:48 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v3 02/10] media: mc: Check pad flag validity Date: Tue, 8 Aug 2023 10:55:30 +0300 Message-Id: <20230808075538.3043934-3-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> References: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Check the validity of pad flags on entity init. Exactly one of the flags must be set. Signed-off-by: Sakari Ailus --- drivers/media/mc/mc-entity.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 83468d4a440b..4991281dcccc 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -197,6 +197,7 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, struct media_device *mdev = entity->graph_obj.mdev; struct media_pad *iter; unsigned int i = 0; + int ret = 0; if (num_pads >= MEDIA_ENTITY_MAX_PADS) return -E2BIG; @@ -210,6 +211,14 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, media_entity_for_each_pad(entity, iter) { iter->entity = entity; iter->index = i++; + + if (hweight32(iter->flags & (MEDIA_PAD_FL_SINK | + MEDIA_PAD_FL_SOURCE)) + != 1) { + ret = -EINVAL; + break; + } + if (mdev) media_gobj_create(mdev, MEDIA_GRAPH_PAD, &iter->graph_obj); @@ -218,7 +227,7 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, if (mdev) mutex_unlock(&mdev->graph_mutex); - return 0; + return ret; } EXPORT_SYMBOL_GPL(media_entity_pads_init); From patchwork Tue Aug 8 07:55:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 712183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07D97C001DE for ; Tue, 8 Aug 2023 16:45:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231620AbjHHQpT (ORCPT ); Tue, 8 Aug 2023 12:45:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231514AbjHHQoX (ORCPT ); Tue, 8 Aug 2023 12:44:23 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCF0F16BC8 for ; Tue, 8 Aug 2023 08:56:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691510164; x=1723046164; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=r/SIjrwUReb/l+aDUKCinWddolNalKr3CmKEm3N7G3M=; b=fvwhAbcjeHQzP3krZVrQ7Wy4T3eBoBnMUpWycEEmZn5wSiS+RDv7YdCG R7KsDgujVA9yNXUMb6k8f4JBb/xceF59aRwdaCAbbxKyc2SQiLOVa4ZSi X67q+9+Zh6LH0IvHb3M9G/WUhHbS4oyIAM10jAmy7l/tiOOJmdH6bOErH ekZoJgDzL2PBTN5byRoHTCrQhgSa/I7uCrMbk2sVZ4JM/aHKGhhWD5Nip GEUZ5BKW9q5pATJc/qyEESEMGnNcQRrJw4DMf+R3pScll0nimlhhq2xkK oDCfazZMTUFEV5gCUIbdAgih2XbkmHVH9MJuV+xgzAx3ClQYaRytw6B3k g==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="369648851" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="369648851" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="874627376" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:06 -0700 Received: from punajuuri.localdomain (punajuuri.localdomain [192.168.240.130]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 94292120279; Tue, 8 Aug 2023 10:56:00 +0300 (EEST) Received: from sailus by punajuuri.localdomain with local (Exim 4.96) (envelope-from ) id 1qTHZE-00ClsQ-0d; Tue, 08 Aug 2023 10:55:48 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v3 03/10] media: mc: Add INTERNAL pad flag Date: Tue, 8 Aug 2023 10:55:31 +0300 Message-Id: <20230808075538.3043934-4-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> References: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Internal source pads will be used as routing endpoints in V4L2 [GS]_ROUTING IOCTLs, to indicate that the stream begins in the entity. Also prevent creating links to pads that have been flagged as internal. Signed-off-by: Sakari Ailus --- Documentation/userspace-api/media/glossary.rst | 6 ++++++ Documentation/userspace-api/media/mediactl/media-types.rst | 6 ++++++ drivers/media/mc/mc-entity.c | 6 +++++- include/uapi/linux/media.h | 1 + 4 files changed, 18 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/glossary.rst b/Documentation/userspace-api/media/glossary.rst index 96a360edbf3b..f7b99a4527c7 100644 --- a/Documentation/userspace-api/media/glossary.rst +++ b/Documentation/userspace-api/media/glossary.rst @@ -173,6 +173,12 @@ Glossary An integrated circuit that integrates all components of a computer or other electronic systems. +_media-glossary-stream: + Stream + A distinct flow of data (image data or metadata) over a media pipeline + from source to sink. A source may be e.g. an image sensor and a sink + e.g. a memory buffer. + V4L2 API **V4L2 userspace API** diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/Documentation/userspace-api/media/mediactl/media-types.rst index 0ffeece1e0c8..28941da27790 100644 --- a/Documentation/userspace-api/media/mediactl/media-types.rst +++ b/Documentation/userspace-api/media/mediactl/media-types.rst @@ -361,6 +361,7 @@ Types and flags used to represent the media graph elements .. _MEDIA-PAD-FL-SINK: .. _MEDIA-PAD-FL-SOURCE: .. _MEDIA-PAD-FL-MUST-CONNECT: +.. _MEDIA-PAD-FL-INTERNAL: .. flat-table:: Media pad flags :header-rows: 0 @@ -382,6 +383,11 @@ Types and flags used to represent the media graph elements when this flag isn't set; the absence of the flag doesn't imply there is none. + * - ``MEDIA_PAD_FL_INTERNAL`` + - The internal flag indicates an internal pad that has no external + connections. Such a pad shall not be connected with a link. The + internal flag indicates that the :ref:``stream + `` either starts or ends in the entity. One and only one of ``MEDIA_PAD_FL_SINK`` and ``MEDIA_PAD_FL_SOURCE`` must be set for every pad. diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 4991281dcccc..03a9e0b8ebab 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1071,7 +1071,8 @@ int media_get_pad_index(struct media_entity *entity, u32 pad_type, for (i = 0; i < entity->num_pads; i++) { if ((entity->pads[i].flags & - (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE)) != pad_type) + (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE | + MEDIA_PAD_FL_INTERNAL)) != pad_type) continue; if (entity->pads[i].sig_type == sig_type) @@ -1094,6 +1095,9 @@ media_create_pad_link(struct media_entity *source, u16 source_pad, return -EINVAL; if (WARN_ON(!(source->pads[source_pad].flags & MEDIA_PAD_FL_SOURCE))) return -EINVAL; + if (WARN_ON(source->pads[source_pad].flags & MEDIA_PAD_FL_SOURCE && + source->pads[source_pad].flags & MEDIA_PAD_FL_INTERNAL)) + return -EINVAL; if (WARN_ON(!(sink->pads[sink_pad].flags & MEDIA_PAD_FL_SINK))) return -EINVAL; diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h index 1c80b1d6bbaf..80cfd12a43fc 100644 --- a/include/uapi/linux/media.h +++ b/include/uapi/linux/media.h @@ -208,6 +208,7 @@ struct media_entity_desc { #define MEDIA_PAD_FL_SINK (1U << 0) #define MEDIA_PAD_FL_SOURCE (1U << 1) #define MEDIA_PAD_FL_MUST_CONNECT (1U << 2) +#define MEDIA_PAD_FL_INTERNAL (1U << 3) struct media_pad_desc { __u32 entity; /* entity ID */ From patchwork Tue Aug 8 07:55:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 711790 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DB3DC001B0 for ; Tue, 8 Aug 2023 16:43:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232488AbjHHQni (ORCPT ); Tue, 8 Aug 2023 12:43:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39246 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232490AbjHHQms (ORCPT ); Tue, 8 Aug 2023 12:42:48 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3EBB216568 for ; Tue, 8 Aug 2023 08:55:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691510126; x=1723046126; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=B/UTzExT+AA5PHDSza/sBvtJLnsA/4Al/m8pE6Z6jdk=; b=Lpj25K0W9flM9N8blocPzltjb2KPOtjffm4J6T7pt/tqBwASx+XBWqKu 44KHxCdA5MVQkDaqnt4NyzFUM0sAwu70FHJllDnC8+KHpOH/FFbOfWMKY OOkRm6MVnD048cyAdeKswspEGvtKM4IJaMgPlhrGwICSQUVc4BHuLxref CtNV5u0sN+aGsr1Hrwx4uu2txqijNTjSJ7HBszsBQNqyN3tHZ3dajbPJy ffSGgiqDZc+QzJ+DRT/i9kpUK1Wg2ysmAxZFeZCd6/vXQHFIyGGfQ95NZ weGtImK1/hN3COl1R2mS5XQrKCBYl8jeYT6MdHzMAaapposBBCxKUt4kM Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="369648840" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="369648840" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="874627372" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:06 -0700 Received: from punajuuri.localdomain (punajuuri.localdomain [192.168.240.130]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 970B512033A; Tue, 8 Aug 2023 10:56:00 +0300 (EEST) Received: from sailus by punajuuri.localdomain with local (Exim 4.96) (envelope-from ) id 1qTHZE-00ClsX-0g; Tue, 08 Aug 2023 10:55:48 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v3 04/10] media: uapi: Add generic serial metadata mbus formats Date: Tue, 8 Aug 2023 10:55:32 +0300 Message-Id: <20230808075538.3043934-5-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> References: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add generic serial metadata mbus formats. These formats describe data width and packing but not the content itself. The reason for specifying such formats is that the formats as such are fairly device specific but they are still handled by CSI-2 receiver drivers that should not be aware of device specific formats. What makes generic metadata formats possible is that these formats are parsed by software only, after capturing the data to system memory. Signed-off-by: Sakari Ailus --- .../media/v4l/subdev-formats.rst | 257 ++++++++++++++++++ include/uapi/linux/media-bus-format.h | 9 + 2 files changed, 266 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index a3a35eeed708..c615da08502d 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -8234,3 +8234,260 @@ The following table lists the existing metadata formats. both sides of the link and the bus format is a fixed metadata format that is not configurable from userspace. Width and height will be set to 0 for this format. + +Generic Serial Metadata Formats +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Generic serial metadata formats are used on serial busses where the actual data +content is more or less device specific but the data is transmitted and received +by multiple devices that do not process the data in any way, simply writing +it to system memory for processing in software at the end of the pipeline. + +The more specific variant describing the actual data is used on the internal +source pad of the originating sub-device. + +"b" in an array cell signifies a byte of data, followed by the number of byte +and finally the bit number in subscript. "p" indicates a padding bit. + +.. _media-bus-format-generic-meta: + +.. cssclass: longtable + +.. flat-table:: Generic Serial Metadata Formats + :header-rows: 2 + :stub-columns: 0 + + * - Identifier + - Code + - + - :cspan:`23` Data organization + * - + - + - Bit + - 23 + - 22 + - 21 + - 20 + - 19 + - 18 + - 17 + - 16 + - 15 + - 14 + - 13 + - 12 + - 11 + - 10 + - 9 + - 8 + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + * .. _MEDIA-BUS-FMT-META-8: + + - MEDIA_BUS_FMT_META_8 + - 0x8001 + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + * .. _MEDIA-BUS-FMT-META-10: + + - MEDIA_BUS_FMT_META_10 + - 0x8002 + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + * .. _MEDIA-BUS-FMT-META-12: + + - MEDIA_BUS_FMT_META_12 + - 0x8003 + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-14: + + - MEDIA_BUS_FMT_META_14 + - 0x8004 + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-16: + + - MEDIA_BUS_FMT_META_16 + - 0x8005 + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-20: + + - MEDIA_BUS_FMT_META_20 + - 0x8007 + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-24: + + - MEDIA_BUS_FMT_META_24 + - 0x8009 + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index a03c543cb072..9ee031397372 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -173,4 +173,13 @@ */ #define MEDIA_BUS_FMT_METADATA_FIXED 0x7001 +/* Generic line based metadata formats for serial buses. Next is 0x8008. */ +#define MEDIA_BUS_FMT_META_8 0x8001 +#define MEDIA_BUS_FMT_META_10 0x8002 +#define MEDIA_BUS_FMT_META_12 0x8003 +#define MEDIA_BUS_FMT_META_14 0x8004 +#define MEDIA_BUS_FMT_META_16 0x8005 +#define MEDIA_BUS_FMT_META_20 0x8006 +#define MEDIA_BUS_FMT_META_24 0x8007 + #endif /* __LINUX_MEDIA_BUS_FORMAT_H */ From patchwork Tue Aug 8 07:55:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 712181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33D97C001B0 for ; Tue, 8 Aug 2023 16:47:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231910AbjHHQrV (ORCPT ); Tue, 8 Aug 2023 12:47:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232322AbjHHQqV (ORCPT ); Tue, 8 Aug 2023 12:46:21 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A97F170E0 for ; Tue, 8 Aug 2023 08:56:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691510195; x=1723046195; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mtd2OlaD2uF9uQfv/VWnnRlsGNgouPl7W3/VQMYJfmg=; b=JLt7fqfJT/8ci6VpxMOpdj7z8xzZMQu7rMrhHRaYxstgFWHx1fZw1n1I 7e2tQQoWg1ApVbfmdq9BqTm/ecu0mec4tC7/3eeTbEaBks0j1HbcKNgz7 yGeGpnn9btosbaQ5q7A9s1UjUfB2GTqnH4spNjfbNJBuiwt8UlwQxuauF cuOr+Uk/hmultvyl76sFmkfrFIXjwzHoyq9UwjDRMNkN5CNh5Oxsjwivv LR3YR0cfmX8crWYRvF0BlLa7928erMwG7Uz7l6WylgZiJjc2wDGXcnKKm ly2OP8leD9tr0gFB4EvA7lko4nceSpFcLaM6M6d+hlZsYX4KEkdbjqqEl A==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="369648895" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="369648895" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="874627397" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:10 -0700 Received: from punajuuri.localdomain (punajuuri.localdomain [192.168.240.130]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 99FF0120356; Tue, 8 Aug 2023 10:56:00 +0300 (EEST) Received: from sailus by punajuuri.localdomain with local (Exim 4.96) (envelope-from ) id 1qTHZE-00Clsc-0i; Tue, 08 Aug 2023 10:55:48 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v3 05/10] media: uapi: Document which mbus format fields are valid for metadata Date: Tue, 8 Aug 2023 10:55:33 +0300 Message-Id: <20230808075538.3043934-6-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> References: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Now that metadata mbus formats have been added, it is necessary to define which fields in struct v4l2_mbus_format are applicable to them (not many). Signed-off-by: Sakari Ailus --- include/uapi/linux/v4l2-mediabus.h | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/include/uapi/linux/v4l2-mediabus.h b/include/uapi/linux/v4l2-mediabus.h index 6b07b73473b5..3cadb3b58b85 100644 --- a/include/uapi/linux/v4l2-mediabus.h +++ b/include/uapi/linux/v4l2-mediabus.h @@ -19,12 +19,18 @@ * @width: image width * @height: image height * @code: data format code (from enum v4l2_mbus_pixelcode) - * @field: used interlacing type (from enum v4l2_field) - * @colorspace: colorspace of the data (from enum v4l2_colorspace) - * @ycbcr_enc: YCbCr encoding of the data (from enum v4l2_ycbcr_encoding) - * @hsv_enc: HSV encoding of the data (from enum v4l2_hsv_encoding) - * @quantization: quantization of the data (from enum v4l2_quantization) - * @xfer_func: transfer function of the data (from enum v4l2_xfer_func) + * @field: used interlacing type (from enum v4l2_field), not applicable + * to metadata mbus codes + * @colorspace: colorspace of the data (from enum v4l2_colorspace), zero on + * metadata mbus codes + * @ycbcr_enc: YCbCr encoding of the data (from enum v4l2_ycbcr_encoding), zero + * on metadata mbus codes + * @hsv_enc: HSV encoding of the data (from enum v4l2_hsv_encoding), zero on + * metadata mbus codes + * @quantization: quantization of the data (from enum v4l2_quantization), zero + * on metadata mbus codes + * @xfer_func: transfer function of the data (from enum v4l2_xfer_func), zero + * on metadata mbus codes * @flags: flags (V4L2_MBUS_FRAMEFMT_*) * @reserved: reserved bytes that can be later used */ From patchwork Tue Aug 8 07:55:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 712182 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4766C001B0 for ; Tue, 8 Aug 2023 16:46:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232586AbjHHQqR (ORCPT ); Tue, 8 Aug 2023 12:46:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233351AbjHHQpW (ORCPT ); Tue, 8 Aug 2023 12:45:22 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E61CA44452 for ; Tue, 8 Aug 2023 08:56:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691510181; x=1723046181; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zLZsEP9m7c0aA3XJAE3/+rd++V0bB/KkiYVa0+jwjCY=; b=lav3FDk5fkbgipFOb275CmH5ekKZrNM5KUbTW9VP5phtyjch8+dzp9o0 p6l+6cDfeZ7id+SZyLB98W5sPTcpWj1JaQNvqJHLoHUcw2NKA0hmUwrTR AbMHtd7qgsbeRj4cFtG7zzZwONtRvg0iLb9wUWlgUObuQcIQAlDuBSccx aUwo9pNd4X8bAkw1/rIU3msQZuNXXewpK1YSn1rbIcPwsLZ5M8ZPYza9P eosixhGU1TIPWvg1h78mMoEBp5t4j4sTHQI3BVUCNMUz+zDO87YY1kN4q nXCyTrS1XPcS8EvLwsUF9iu5+CO+vjNYH0rL5NJu4d9+jKTZPn1Rh3eWW A==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="369648883" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="369648883" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="874627392" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:10 -0700 Received: from punajuuri.localdomain (punajuuri.localdomain [192.168.240.130]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 9E112120AB3; Tue, 8 Aug 2023 10:56:00 +0300 (EEST) Received: from sailus by punajuuri.localdomain with local (Exim 4.96) (envelope-from ) id 1qTHZE-00Clsh-0l; Tue, 08 Aug 2023 10:55:48 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v3 06/10] media: uapi: Add a macro to tell whether an mbus code is metadata Date: Tue, 8 Aug 2023 10:55:34 +0300 Message-Id: <20230808075538.3043934-7-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> References: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add a macro to tell whether a given mbus code is metadata. Signed-off-by: Sakari Ailus --- include/uapi/linux/media-bus-format.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index 9ee031397372..2486b4178c5f 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -182,4 +182,7 @@ #define MEDIA_BUS_FMT_META_20 0x8006 #define MEDIA_BUS_FMT_META_24 0x8007 +#define MEDIA_BUS_FMT_IS_META(code) \ + ((code) & 0xf000 == 0x7000 || (code) & 0xf000 == 0x8000) + #endif /* __LINUX_MEDIA_BUS_FORMAT_H */ From patchwork Tue Aug 8 07:55:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 711785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81F0FC001DE for ; Tue, 8 Aug 2023 16:48:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233510AbjHHQso (ORCPT ); Tue, 8 Aug 2023 12:48:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39214 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233393AbjHHQrw (ORCPT ); Tue, 8 Aug 2023 12:47:52 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54B3E46FFA for ; Tue, 8 Aug 2023 08:56:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691510218; x=1723046218; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dv89ZDHuR4D/9qzx98ivrVxTonxgJezEyRXem4PNETo=; b=Vkq6kkQkWy5qYrsQDr920vvMYcbRYnrJN5NW/T20sngstdm0o2rURJxs PP4Ar8RbKDQxAdkKcGI3AZZF/eMSDGcppj55FGl8VkTWcnEfp+p6bUZB0 FEwvWj/yNjgkI9pMn67/BxF3kFjdTnxNLEGd6UKnHw0qzF3ubsIhe9OL1 /Q2xJ8o4icNUfc6FVyTavBTL2kvMRZso1dz1cNoV5m+8NjGm7cXxvWGlu x9+55O1loPK6TgbztyZWMMBpZ306rLoCJIuEPsVGcGwgcwPvegi4zgVUb ViYM3pfwiUH178Z5frH4psqjaH2zakxNSannhLj+SQS1QAwEtonFB4L7f w==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="369648908" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="369648908" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="874627402" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:10 -0700 Received: from punajuuri.localdomain (punajuuri.localdomain [192.168.240.130]) by kekkonen.fi.intel.com (Postfix) with ESMTP id A2859120ECF; Tue, 8 Aug 2023 10:56:00 +0300 (EEST) Received: from sailus by punajuuri.localdomain with local (Exim 4.96) (envelope-from ) id 1qTHZE-00Clsm-0o; Tue, 08 Aug 2023 10:55:48 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v3 07/10] media: uapi: Add generic 8-bit metadata format definitions Date: Tue, 8 Aug 2023 10:55:35 +0300 Message-Id: <20230808075538.3043934-8-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> References: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Generic 8-bit metadata formats define the in-memory data layout but not the format of the data itself. The reasoning for having such formats is to allow CSI-2 receiver drivers to receive and DMA drivers to write the data to memory without knowing a large number of device specific formats. These formats may be used only in conjunction of a Media controller pipeline where the internal pad of the source sub-device defines the specific format of the data (using an mbus code). Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/meta-formats.rst | 1 + .../media/v4l/metafmt-generic.rst | 331 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 8 + include/uapi/linux/videodev2.h | 9 + 4 files changed, 349 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/metafmt-generic.rst diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst index 0bb61fc5bc00..919f595576b9 100644 --- a/Documentation/userspace-api/media/v4l/meta-formats.rst +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst @@ -19,3 +19,4 @@ These formats are used for the :ref:`metadata` interface only. metafmt-vsp1-hgo metafmt-vsp1-hgt metafmt-vivid + metafmt-generic diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst new file mode 100644 index 000000000000..a27bfc721edf --- /dev/null +++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst @@ -0,0 +1,331 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later + +************************************************************************************************************************************************************************************************************************************************************************************************************************** +V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_META_FMT_GENERIC_CSI2_12 ('MC1C'), V4L2_META_FMT_GENERIC_CSI2_14 ('MC1E'), V4L2_META_FMT_GENERIC_CSI2_16 ('MC1G'), V4L2_META_FMT_GENERIC_CSI2_20 ('MC1K'), V4L2_META_FMT_GENERIC_CSI2_24 ('MC1O'), V4L2_META_FMT_GENERIC_CSI2_2_24 ('MC2O') +************************************************************************************************************************************************************************************************************************************************************************************************************************** + + +Generic line-based metadata formats + + +Description +=========== + +These generic line-based metadata formats define the memory layout of the data +without defining the format or meaning of the metadata itself. These formats may +only be used with a Media controller pipeline where the more specific format is +defined in an :ref:`internal source pad ` of the source +sub-device. See also :ref:`source routes `. + +.. _v4l2-meta-fmt-generic-8: + +V4L2_META_FMT_GENERIC_8 +----------------------- + +The V4L2_META_FMT_GENERIC_8 format is a plain 8-bit metadata format. + +This format is also used on CSI-2 on both 8 bits per sample as well as on +16 bits per sample when two bytes of metadata are packed into one sample. + +**Byte Order Of V4L2_META_FMT_GENERIC_8.** +Each cell is one byte. "M" denotes a byte of metadata. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + * - start + 4: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + +.. _v4l2-meta-fmt-generic-csi2-10: + +V4L2_META_FMT_GENERIC_CSI2_10 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_10 contains packed 8-bit generic metadata, 10 bits +for each 8 bits of data. Every four bytes of metadata is followed by a single +byte of padding. The way the data is stored follows the CSI-2 specification. + +This format is also used on CSI-2 on 20 bits per sample format that packs two +bytes of metadata into one sample. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_10.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + * - start + 5: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + +.. _v4l2-meta-fmt-generic-csi2-12: + +V4L2_META_FMT_GENERIC_CSI2_12 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_12 contains packed 8-bit generic metadata, 12 bits +for each 8 bits of data. Every four bytes of metadata is followed by two bytes +of padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_12.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + - p + * - start + 6: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-14: + +V4L2_META_FMT_GENERIC_CSI2_14 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_14 contains packed 8-bit generic metadata, 14 bits +for each 8 bits of data. Every four bytes of metadata is followed by three +bytes of padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_14.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + - p + - p + * - start + 7: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-16: + +V4L2_META_FMT_GENERIC_CSI2_16 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_16 contains packed 8-bit generic metadata, 16 bits +for each 8 bits of data. Every byte of metadata is followed by one byte of +padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_16.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - M\ :sub:`10` + - p + - M\ :sub:`20` + - p + - M\ :sub:`30` + - p + * - start + 8: + - M\ :sub:`01` + - p + - M\ :sub:`11` + - p + - M\ :sub:`21` + - p + - M\ :sub:`31` + - p + +.. _v4l2-meta-fmt-generic-csi2-20: + +V4L2_META_FMT_GENERIC_CSI2_20 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_20 contains packed 8-bit generic metadata, 20 bits +for each 8 bits of data. Every byte of metadata is followed by alternating one +and two bytes of padding. The way the data is stored follows the CSI-2 +specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_20.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - M\ :sub:`10` + - p + - p + - M\ :sub:`20` + - p + - M\ :sub:`30` + - p + - p + * - start + 10: + - M\ :sub:`01` + - p + - M\ :sub:`11` + - p + - p + - M\ :sub:`21` + - p + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-24: + +V4L2_META_FMT_GENERIC_CSI2_24 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_24 contains packed 8-bit generic metadata, 24 bits +for each 8 bits of data. Every byte of metadata is followed by two bytes of +padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_24.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - p + - M\ :sub:`10` + - p + - p + - M\ :sub:`20` + - p + - p + - M\ :sub:`30` + - p + - p + * - start + 12: + - M\ :sub:`01` + - p + - p + - M\ :sub:`11` + - p + - p + - M\ :sub:`21` + - p + - p + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-2-24: + +V4L2_META_FMT_GENERIC_CSI2_2_24 +------------------------------- + +V4L2_META_FMT_GENERIC_CSI2_2_24 contains packed 8-bit generic metadata, 24 bits +for each two times 8 bits of data. Every two bytes of metadata are followed by +one byte of padding. The way the data is stored follows the CSI-2 +specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_2_24.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - p + - M\ :sub:`20` + - M\ :sub:`30` + - p + * - start + 6: + - M\ :sub:`01` + - M\ :sub:`11` + - p + - M\ :sub:`21` + - M\ :sub:`31` + - p + diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index f4d9d6279094..fbbddc333a30 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1452,6 +1452,14 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; + case V4L2_META_FMT_GENERIC_8: descr = "8-bit Generic Metadata"; break; + case V4L2_META_FMT_GENERIC_CSI2_10: descr = "8b Generic Meta, 10b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_12: descr = "8b Generic Meta, 12b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_14: descr = "8b Generic Meta, 14b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_16: descr = "8b Generic Meta, 16b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_20: descr = "8b Generic Meta, 20b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_24: descr = "8b Generic Meta, 24b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_2_24: descr = "2x8b Generic Meta, 24b CSI-2"; break; default: /* Compressed formats */ diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 78260e5d9985..b4284a564025 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -838,6 +838,15 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ +#define V4L2_META_FMT_GENERIC_8 v4l2_fourcc('M', 'E', 'T', '8') /* Generic 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_10 v4l2_fourcc('M', 'C', '1', 'A') /* 10-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_12 v4l2_fourcc('M', 'C', '1', 'C') /* 12-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_14 v4l2_fourcc('M', 'C', '1', 'E') /* 14-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_16 v4l2_fourcc('M', 'C', '1', 'G') /* 16-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_20 v4l2_fourcc('M', 'C', '1', 'K') /* 20-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_24 v4l2_fourcc('M', 'C', '1', 'O') /* 24-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_2_24 v4l2_fourcc('M', 'C', '2', 'O') /* 2 bytes of 8-bit metadata, 24-bit CSI-2 packed */ + /* priv field value to indicates that subsequent fields are valid. */ #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe From patchwork Tue Aug 8 07:55:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 711786 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00356C001B0 for ; Tue, 8 Aug 2023 16:47:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232829AbjHHQrk (ORCPT ); Tue, 8 Aug 2023 12:47:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233019AbjHHQqe (ORCPT ); Tue, 8 Aug 2023 12:46:34 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C2AA170EE for ; Tue, 8 Aug 2023 08:56:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691510199; x=1723046199; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LfYG4/WgY5vf74TcXoCxK73LBYGoRuwmS0Eiwlzfx6o=; b=eFLVVGiKiMS+5eABBXRdIuhz3UgIc7thsIWtoiQJUgDnmUCDAgGMoGlw 88TipZplf9psI7afG9IE91wekmOFpB4+MhdBJvEfw26tnYySBqQEQPz0N PXMnUiRoOdoUSKoe7k2gwLApQ9S0wPyO44DLR5yqcNhpjjNcb+gXEry6i W1CTU9ZG0F1ho7v5N9hKZn3UY4+kbMFuKTpMKX0HWxIOYlZpPvdr9Ndeg 9DLnKgnEuynMdCtM8tK9S+DdToSVZNGkMQXK/XocYYVd/ExhdoO+GegnL qisL0N5LbKwy17cQGxnW6Rlrzxctt6orHles1aZItvErD37Dn3RFiakZz Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="369648902" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="369648902" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="874627398" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:10 -0700 Received: from punajuuri.localdomain (punajuuri.localdomain [192.168.240.130]) by kekkonen.fi.intel.com (Postfix) with ESMTP id A742C12193B; Tue, 8 Aug 2023 10:56:00 +0300 (EEST) Received: from sailus by punajuuri.localdomain with local (Exim 4.96) (envelope-from ) id 1qTHZE-00Clsr-0s; Tue, 08 Aug 2023 10:55:48 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v3 08/10] media: v4l: Support line-based metadata capture Date: Tue, 8 Aug 2023 10:55:36 +0300 Message-Id: <20230808075538.3043934-9-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> References: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org many camera sensors, among other devices, transmit embedded data and image data for each CSI-2 frame. This embedded data typically contains register configuration of the sensor that has been used to capture the image data of the same frame. The embedded data is received by the CSI-2 receiver and has the same properties as the image data, including that it is line based: it has width, height and bytesperline (stride). Add these fields to struct v4l2_meta_format and document them. Also add V4L2_FMT_FLAG_META_LINE_BASED to tell a given format is line-based i.e. these fields of struct v4l2_meta_format are valid for it. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-meta.rst | 15 +++++++++++++++ .../userspace-api/media/v4l/vidioc-enum-fmt.rst | 7 +++++++ .../media/videodev2.h.rst.exceptions | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 5 +++-- include/uapi/linux/videodev2.h | 10 ++++++++++ 5 files changed, 36 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/dev-meta.rst b/Documentation/userspace-api/media/v4l/dev-meta.rst index 0e7e1ee1471a..4b24bae6e171 100644 --- a/Documentation/userspace-api/media/v4l/dev-meta.rst +++ b/Documentation/userspace-api/media/v4l/dev-meta.rst @@ -65,3 +65,18 @@ to 0. - ``buffersize`` - Maximum buffer size in bytes required for data. The value is set by the driver. + * - __u32 + - ``width`` + - Width of a line of metadata in samples. Valid when :c:type`v4l2_fmtdesc` + flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``height`` + - Number of rows of metadata. Valid when :c:type`v4l2_fmtdesc` flag + ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``bytesperline`` + - Offset in bytes between the beginning of two consecutive lines. Valid + when :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is + set, otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. diff --git a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst index 000c154b0f98..6d7664345a4e 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst @@ -227,6 +227,13 @@ the ``mbus_code`` field is handled differently: The application can ask to configure the quantization of the capture device when calling the :ref:`VIDIOC_S_FMT ` ioctl with :ref:`V4L2_PIX_FMT_FLAG_SET_CSC ` set. + * - ``V4L2_FMT_FLAG_META_LINE_BASED`` + - 0x0200 + - The metadata format is line-based. In this case the ``width``, + ``height`` and ``bytesperline`` fields of :c:type:`v4l2_meta_format` are + valid. The buffer consists of ``height`` lines, each having ``width`` + bytes of data and offset between the beginning of each two consecutive + lines is ``bytesperline``. Return Value ============ diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions index 3e58aac4ef0b..bdc628e8c1d6 100644 --- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions +++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions @@ -215,6 +215,7 @@ replace define V4L2_FMT_FLAG_CSC_XFER_FUNC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_YCBCR_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_HSV_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_QUANTIZATION fmtdesc-flags +replace define V4L2_FMT_FLAG_META_LINE_BASED fmtdesc-flags # V4L2 timecode types replace define V4L2_TC_TYPE_24FPS timecode-type diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index fbbddc333a30..971d784e7429 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -343,8 +343,9 @@ static void v4l_print_format(const void *arg, bool write_only) case V4L2_BUF_TYPE_META_OUTPUT: meta = &p->fmt.meta; pixelformat = meta->dataformat; - pr_cont(", dataformat=%p4cc, buffersize=%u\n", - &pixelformat, meta->buffersize); + pr_cont(", dataformat=%p4cc, buffersize=%u, width=%u, height=%u, bytesperline=%u\n", + &pixelformat, meta->buffersize, meta->width, + meta->height, meta->bytesperline); break; } } diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index b4284a564025..d26c0650c6a7 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -877,6 +877,7 @@ struct v4l2_fmtdesc { #define V4L2_FMT_FLAG_CSC_YCBCR_ENC 0x0080 #define V4L2_FMT_FLAG_CSC_HSV_ENC V4L2_FMT_FLAG_CSC_YCBCR_ENC #define V4L2_FMT_FLAG_CSC_QUANTIZATION 0x0100 +#define V4L2_FMT_FLAG_META_LINE_BASED 0x0200 /* Frame Size and frame rate enumeration */ /* @@ -2420,10 +2421,19 @@ struct v4l2_sdr_format { * struct v4l2_meta_format - metadata format definition * @dataformat: little endian four character code (fourcc) * @buffersize: maximum size in bytes required for data + * @width: number of bytes of data per line (valid for line based + * formats only, see format documentation) + * @height: number of lines of data per buffer (valid for line based + * formats only) + * @bytesperline: offset between the beginnings of two adjacent lines in + * bytes (valid for line based formats only) */ struct v4l2_meta_format { __u32 dataformat; __u32 buffersize; + __u32 width; + __u32 height; + __u32 bytesperline; } __attribute__ ((packed)); /** From patchwork Tue Aug 8 07:55:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 711787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BB16C001B0 for ; Tue, 8 Aug 2023 16:46:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232829AbjHHQqZ (ORCPT ); Tue, 8 Aug 2023 12:46:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233363AbjHHQpc (ORCPT ); Tue, 8 Aug 2023 12:45:32 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4613F4445D for ; Tue, 8 Aug 2023 08:56:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691510184; x=1723046184; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=64z0iHES/dFeITo3Vn7fWATVi0C19w3f1FwZVWt4p00=; b=jqgf8qPxUaD09PAlAEfXAeMo2EU3BNx73XW9ln2I2vL54lASOoX0vKHV A1q0UGOikVA2pSTxqreEMicq7KpGRONkciuKXwwM79d1oHNxHxxek2Clr R1laHegDFCQde2BN2/mDbpWuKHOzcJ+/suYJduyRDl9kREJxGWnVticGD 0U7aHN25GxugqPgfelgCgxBQEuzt2pLX2Cx+xSUAaFzNQ9xwrZNVRb5E0 VPKhg09M1/V60IbPDzyA2nVeVF1NVy8KOWulJiS/eMJ85Ds9hmJcMYgf/ lzRVxCZG1IvhYcHWlHi2K7j5DZYew4e2uBhQxA3fOuwtphjF0S9ZTnMuM w==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="369648890" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="369648890" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="874627394" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:10 -0700 Received: from punajuuri.localdomain (punajuuri.localdomain [192.168.240.130]) by kekkonen.fi.intel.com (Postfix) with ESMTP id AB3B6122265; Tue, 8 Aug 2023 10:56:00 +0300 (EEST) Received: from sailus by punajuuri.localdomain with local (Exim 4.96) (envelope-from ) id 1qTHZE-00Clsw-0v; Tue, 08 Aug 2023 10:55:48 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v3 09/10] media: Add media bus codes for MIPI CCS embedded data Date: Tue, 8 Aug 2023 10:55:37 +0300 Message-Id: <20230808075538.3043934-10-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> References: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add new MIPI CCS embedded data media bus formats. Signed-off-by: Sakari Ailus --- .../media/v4l/subdev-formats.rst | 32 +++++++++++++++++++ include/uapi/linux/media-bus-format.h | 10 +++++- 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index c615da08502d..5d5407738af9 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -8491,3 +8491,35 @@ and finally the bit number in subscript. "p" indicates a padding bit. - p - p - p + +MIPI CCS Embedded Data Formats +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +`MIPI CCS `_ defines an +metadata format for sensor embedded data, which is used to store the register +configuration used for capturing a given frame. The format is defined in the CCS +specification. + +The bit depth of the CCS embedded data matches the pixel data bit depth +configured on the sensor. The formats used and their corresponding generic +formats are listed in the table below. + +.. flat-table: CCS embedded data mbus formats and corresponding generic formats + :header-rows: 1 + + * - CCS embedded data mbus format + - Generic metadata format + * - MEDIA_BUS_FMT_CCS_EMBEDDED_8 + - MEDIA_BUS_FMT_META_8 + * - MEDIA_BUS_FMT_CCS_EMBEDDED_10 + - MEDIA_BUS_FMT_META_10 + * - MEDIA_BUS_FMT_CCS_EMBEDDED_12 + - MEDIA_BUS_FMT_META_12 + * - MEDIA_BUS_FMT_CCS_EMBEDDED_14 + - MEDIA_BUS_FMT_META_14 + * - MEDIA_BUS_FMT_CCS_EMBEDDED_16 + - MEDIA_BUS_FMT_META_16 + * - MEDIA_BUS_FMT_CCS_EMBEDDED_20 + - MEDIA_BUS_FMT_META_20 + * - MEDIA_BUS_FMT_CCS_EMBEDDED_24 + - MEDIA_BUS_FMT_META_24 diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index 2486b4178c5f..31d532446b23 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -182,7 +182,15 @@ #define MEDIA_BUS_FMT_META_20 0x8006 #define MEDIA_BUS_FMT_META_24 0x8007 +/* Specific metadata formats. Next is 0x9008. */ +#define MEDIA_BUS_FMT_CCS_EMBEDDED_8 0x9001 +#define MEDIA_BUS_FMT_CCS_EMBEDDED_10 0x9002 +#define MEDIA_BUS_FMT_CCS_EMBEDDED_12 0x9003 +#define MEDIA_BUS_FMT_CCS_EMBEDDED_14 0x9004 +#define MEDIA_BUS_FMT_CCS_EMBEDDED_16 0x9005 +#define MEDIA_BUS_FMT_CCS_EMBEDDED_20 0x9006 +#define MEDIA_BUS_FMT_CCS_EMBEDDED_24 0x9007 + #define MEDIA_BUS_FMT_IS_META(code) \ ((code) & 0xf000 == 0x7000 || (code) & 0xf000 == 0x8000) - #endif /* __LINUX_MEDIA_BUS_FORMAT_H */ From patchwork Tue Aug 8 07:55:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 712180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F106C001B0 for ; Tue, 8 Aug 2023 16:48:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233498AbjHHQs3 (ORCPT ); Tue, 8 Aug 2023 12:48:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231569AbjHHQr0 (ORCPT ); Tue, 8 Aug 2023 12:47:26 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2031446FE7 for ; Tue, 8 Aug 2023 08:56:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691510214; x=1723046214; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JgZDU5kCi3na3iWCzv8U0ILdVQ6j4OMLAwGTmEastxE=; b=NJM+jNZucqkXpCBr80Y3nOb/BASVQ4m91Fj495hqMBy1FG/Cl6OMOxbx jtGSQslDFmSSWidZQtdZkds7dngs1OE+zWFvBa8y1qRuBEGPP2UxyHAIp cbC1lQtzz0PCIllBFHyOhPiBLV8xvYUMHEMcOU+yUC6O5uuIKyNYkOtr1 ljfrY/+bmPt/Qjz5/l2GcFSJluCD1YOvxfst7u/uR2QxmggXhoQbSRMY3 WD+Gv/phTrGXXGRgfDJs3tKqUwCMidKdihQ9HupjnCus5IBVpnG0esvcD LRg24J5bs2eHZhix0hI56vA0es05LKL/kRhvjxi3TTlexS4GtiAU7PSsD Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="369648905" X-IronPort-AV: E=Sophos;i="6.01,263,1684825200"; d="scan'208";a="369648905" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="874627400" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 00:56:10 -0700 Received: from punajuuri.localdomain (punajuuri.localdomain [192.168.240.130]) by kekkonen.fi.intel.com (Postfix) with ESMTP id AEBD8122299; Tue, 8 Aug 2023 10:56:00 +0300 (EEST) Received: from sailus by punajuuri.localdomain with local (Exim 4.96) (envelope-from ) id 1qTHZE-00Clt1-0y; Tue, 08 Aug 2023 10:55:48 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov Subject: [PATCH v3 10/10] media: uapi: v4l: Document source routes Date: Tue, 8 Aug 2023 10:55:38 +0300 Message-Id: <20230808075538.3043934-11-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> References: <20230808075538.3043934-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Document how internal pads are used on source routes. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-subdev.rst | 179 ++++++++++++++++++ 1 file changed, 179 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index b023918177b5..27b0fe2dc83a 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -551,6 +551,27 @@ A stream at a specific point in the media pipeline is identified by the sub-device and a (pad, stream) pair. For sub-devices that do not support multiplexed streams the 'stream' field is always 0. +.. _v4l2-subdev-source-routes: + +Internal pads and source routes +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Cases where a single sub-device source pad is traversed by multiple streams one +or more of which originate from within the sub-device itself are special as +there is no external sink pad for such routes. In those cases, the sources of +the internally generated streams are represented by internal sink pads, which +are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL ` +pad flag set. + +Internal pads have all the properties of an external pad, including formats and +selections. The format in this case is the source format of the stream. An +internal pad always has a single stream only (0). + +/Source routes/ are routes from an internal sink pad to a(n external) source +pad. Generally source routes are not modifiable but they can be activated and +deactivated using the :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE +` flag, depending on driver capabilities. + Interaction between routes, streams, formats and selections ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -666,3 +687,161 @@ A common way to accomplish this is to start from the sensors and propagate the configurations along the stream towards the receiver, using :ref:`VIDIOC_SUBDEV_S_FMT ` ioctls to configure each stream endpoint in each sub-device. + +Internal pads setup example +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A simple example of a multiplexed stream setup might be as follows: + +- A CCS camera sensor source sub-device, with one sink pad (0), one source pad + (1), an internal sink pad (2) that represents the source of embedded + data. There are two routes, one from the sink pad to the source, and another + from the internal sink pad to the source pad. The embedded data stream needs + to be enabled by activating the related route. The configuration of the rest + of the CCS sub-devices is omitted from this example. + +- Multiplexer bridge (Bridge). The bridge has one sink pad, connected to the + sensor (pad 0), and one source pad (pad 1), which outputs two streams. + +- Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0), + connected to the bridge, and two source pads (pads 1-2), going to the DMA + engine. The receiver demultiplexes the incoming streams to the source pads. + +- DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is + connected to a single source pad in the receiver. + +The sensor, the bridge and the receiver are modeled as V4L2 sub-devices, +exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are +modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes. + +To configure this pipeline, the userspace must take the following steps: + +1) Set up media links between entities: connect the sensors to the bridge, + bridge to the receiver, and the receiver to the DMA engines. This step does + not differ from normal non-multiplexed media controller setup. + +2) Configure routing + +.. flat-table:: Camera sensor + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 0/0 + - 1/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from the sink pad + * - 2/0 + - 1/1 + - **V4L2_SUBDEV_ROUTE_FL_ACTIVE** + - Metadata stream from the internal sink pad + +.. flat-table:: Bridge routing table + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 0/0 + - 1/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from camera sensor + * - 0/1 + - 1/1 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Metadata stream from camera sensor + +.. flat-table:: Receiver routing table + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 0/0 + - 1/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from camera sensor + * - 0/1 + - 2/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Metadata stream from camera sensor + +The options available in sensor's routing configuration are dictated by hardware +capabilities: typically camera sensors always produce image data stream while +the embedded data stream typically can be either enabled or disabled. + +3) Configure formats and selections + +This example assumes that the formats are propagated from sink pad to the source +pad as-is. The tables contain fields of both struct v4l2_subdev_format and +struct v4l2_mbus_framefmt. The full configuration of CCS camera sensor is out of +scope of this example. + +.. flat-table:: Formats set on the sub-devices. Bold values are set, others are + static or propagated. + :header-rows: 1 + :fill-cells: + + * - Sub-device + - Pad/Stream + - Width + - Height + - Code + * - :rspan:`3` Camera sensor sub-device (CCS source sub-device) + - 0/0 + - 640 + - 480 + - MEDIA_BUS_FMT_SGRBG10 + * - 1/0 + - 640 + - 480 + - **MEDIA_BUS_FMT_SGRBG10** + * - 2/0 + - 640 + - 2 + - MEDIA_BUS_FMT_CCS_EMBEDDED_10 + * - 1/1 + - 640 + - 2 + - MEDIA_BUS_FMT_META_10 + * - :rspan:`3` Bridge + - 0/0 + - **640** + - **480** + - **MEDIA_BUS_FMT_SGRBG10** + * - 1/0 + - 640 + - 480 + - MEDIA_BUS_FMT_SGRBG10 + * - 0/1 + - **640** + - **2** + - **MEDIA_BUS_FMT_META_10** + * - 1/1 + - 640 + - 2 + - MEDIA_BUS_FMT_META_10 + * - :rspan:`3` Receiver + - 0/0 + - **640** + - **480** + - **MEDIA_BUS_FMT_SGRBG10** + * - 1/0 + - 640 + - 480 + - MEDIA_BUS_FMT_SGRBG10 + * - 0/1 + - **640** + - **2** + - **MEDIA_BUS_FMT_META_10** + * - 2/0 + - 640 + - 2 + - MEDIA_BUS_FMT_META_10 + +The embedded data format does not need to be configured as the format is +dictated by the pixel data format in this case.