From patchwork Fri May 5 21:52:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 679838 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1758C77B7C for ; Fri, 5 May 2023 21:53:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233502AbjEEVxI (ORCPT ); Fri, 5 May 2023 17:53:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52522 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231783AbjEEVxG (ORCPT ); Fri, 5 May 2023 17:53:06 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D5E34C29 for ; Fri, 5 May 2023 14:53:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683323585; x=1714859585; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NP1l/tnRjqy3OjQxoOH+KSpm+18ukdCogG2Ir80SSkI=; b=T50WOirzLWwgaqlWT4//fx3EQIEVHQ2qLQnAAB1iHGe8E/NXREflh/5i XTjQc37vLF5cIPMVZIfZbAOcQVgR8By4H9R3xCK8OSazTLi5z4KS3PdUK WAP0X4coKMVef55S6YCsgFkHerut7U50m+iPiXNUUD0u9xh/X5WTuZ44l TLKtTWzAfy8VTYETpWNQXc/kxKIe0Uj4dVvp4zXEbo2gJjewkmO389n3k 4hShMXSexFUL6qX7TEbyPAWbYYejGF2vkZsebJXC36COsEInvzvTTYAO5 m0ejX4u0RNRlm/yDapM/DJz7iRrYDp78yIWV4T360rk3pjPjHsbeO1e2X A==; X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="328925833" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="328925833" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="700626124" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="700626124" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:02 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 81D5312244C; Sat, 6 May 2023 00:52:59 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: laurent.pinchart@ideasonboard.com, tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com Subject: [RFC 1/7] media: mc: Add INTERNAL_SOURCE pad type flag Date: Sat, 6 May 2023 00:52:51 +0300 Message-Id: <20230505215257.60704-2-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230505215257.60704-1-sakari.ailus@linux.intel.com> References: <20230505215257.60704-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Internal source pads will be used as routing endpoints in V4L2 [GS]_ROUTING IOCTLs, to indicate that the stream begins in the entity. Also prevent creating links to pads that have been flagged as internal. Signed-off-by: Sakari Ailus --- .../userspace-api/media/mediactl/media-types.rst | 7 +++++++ drivers/media/mc/mc-entity.c | 8 +++++++- include/uapi/linux/media.h | 1 + 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/Documentation/userspace-api/media/mediactl/media-types.rst index 0ffeece1e0c8..c724139ad46c 100644 --- a/Documentation/userspace-api/media/mediactl/media-types.rst +++ b/Documentation/userspace-api/media/mediactl/media-types.rst @@ -361,6 +361,7 @@ Types and flags used to represent the media graph elements .. _MEDIA-PAD-FL-SINK: .. _MEDIA-PAD-FL-SOURCE: .. _MEDIA-PAD-FL-MUST-CONNECT: +.. _MEDIA-PAD-FL-INTERNAL-SOURCE: .. flat-table:: Media pad flags :header-rows: 0 @@ -382,6 +383,12 @@ Types and flags used to represent the media graph elements when this flag isn't set; the absence of the flag doesn't imply there is none. + * - ``MEDIA_PAD_FL_INTERNAL_SOURCE`` + - This flag indicates an internal pad that has no external + connections. Such a pad may not be connected with a link. The internal + flag indicates that the stream either starts or ends in the + entity. For a given pad, the INTERNAL_SOURCE flag may not be set if + either SINK or SOURCE flags is set. One and only one of ``MEDIA_PAD_FL_SINK`` and ``MEDIA_PAD_FL_SOURCE`` must be set for every pad. diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index ef34ddd715c9..ed99160a2487 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1062,7 +1062,8 @@ int media_get_pad_index(struct media_entity *entity, u32 pad_type, for (i = 0; i < entity->num_pads; i++) { if ((entity->pads[i].flags & - (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE)) != pad_type) + (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE | + MEDIA_PAD_FL_INTERNAL_SOURCE)) != pad_type) continue; if (entity->pads[i].sig_type == sig_type) @@ -1087,6 +1088,11 @@ media_create_pad_link(struct media_entity *source, u16 source_pad, return -EINVAL; if (WARN_ON(!(sink->pads[sink_pad].flags & MEDIA_PAD_FL_SINK))) return -EINVAL; + if (WARN_ON(sink->pads[sink_pad].flags & + MEDIA_PAD_FL_INTERNAL_SOURCE) || + WARN_ON(source->pads[source_pad].flags & + MEDIA_PAD_FL_INTERNAL_SOURCE)) + return -EINVAL; link = media_add_link(&source->links); if (link == NULL) diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h index edb8dfef9eba..0e2577e8b425 100644 --- a/include/uapi/linux/media.h +++ b/include/uapi/linux/media.h @@ -208,6 +208,7 @@ struct media_entity_desc { #define MEDIA_PAD_FL_SINK (1U << 0) #define MEDIA_PAD_FL_SOURCE (1U << 1) #define MEDIA_PAD_FL_MUST_CONNECT (1U << 2) +#define MEDIA_PAD_FL_INTERNAL_SOURCE (1U << 3) struct media_pad_desc { __u32 entity; /* entity ID */ From patchwork Fri May 5 21:52:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 679474 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14785C77B7F for ; Fri, 5 May 2023 21:53:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233184AbjEEVxH (ORCPT ); Fri, 5 May 2023 17:53:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230028AbjEEVxF (ORCPT ); Fri, 5 May 2023 17:53:05 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 229954C2B for ; Fri, 5 May 2023 14:53:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683323585; x=1714859585; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yUogmbGdY1chA+JB40wTsn4Xaw+wDBpW7L652XZeqHE=; b=hI/LL2INHtHLyz8z+mySy/PZ4GyC/5oP/okqtn/yqavWatbwqdbPaeNG 2Ii3B9wHCk23Y7JC8jUQNVqInpmEBarj6cTg8YC7WlaDjp8xh47bqnlAL GzIayhaUL6d8663aI6F6HxMao8nvkNFKAqY2zAg/zoxbSb+u878OOSriz 4RL0QulxUEjeqOeZSw0ZacIal2abl3fY90dyP+XjJMxZun9+H0ABuiXfS N90tO8sqPSAWJ28tTgzkh7dMy0FU/aDuCPMUiI1axqhc62e23tBVWs2MN rqt8vq1AGLz55b0zy4y6C8qDRsG2UZsfguHkdPEWj9IV2Psf00J9k2s8x g==; X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="328925837" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="328925837" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="700626134" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="700626134" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:03 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id B615E12245D; Sat, 6 May 2023 00:53:00 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: laurent.pinchart@ideasonboard.com, tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com Subject: [RFC 2/7] media: v4l: subdev: Support INTERNAL_SOURCE pads in routing IOCTLs Date: Sat, 6 May 2023 00:52:52 +0300 Message-Id: <20230505215257.60704-3-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230505215257.60704-1-sakari.ailus@linux.intel.com> References: <20230505215257.60704-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Take the new INTERNAL_SOURCE pad flag into account in validating routing IOCTL argument. Effectively this is a SINK pad in this respect. Due to the union there's no need to check for the particular name. Signed-off-by: Sakari Ailus --- drivers/media/v4l2-core/v4l2-subdev.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 2ec179cd1264..6312fc2bf1f0 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -906,7 +906,8 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, return -EINVAL; if (!(pads[route->sink_pad].flags & - MEDIA_PAD_FL_SINK)) + (MEDIA_PAD_FL_SINK | + MEDIA_PAD_FL_INTERNAL_SOURCE))) return -EINVAL; if (route->source_pad >= sd->entity.num_pads) @@ -1787,7 +1788,8 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd, /* Validate the sink and source pad numbers. */ if (route->sink_pad >= sd->entity.num_pads || - !(sd->entity.pads[route->sink_pad].flags & MEDIA_PAD_FL_SINK)) { + !(sd->entity.pads[route->sink_pad].flags & + (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_INTERNAL_SOURCE))) { dev_dbg(sd->dev, "route %u sink (%u) is not a sink pad\n", i, route->sink_pad); goto out; From patchwork Fri May 5 21:52:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 679473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55B22C77B75 for ; Fri, 5 May 2023 21:53:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233398AbjEEVxJ (ORCPT ); Fri, 5 May 2023 17:53:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229622AbjEEVxG (ORCPT ); Fri, 5 May 2023 17:53:06 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C467A4C20 for ; Fri, 5 May 2023 14:53:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683323585; x=1714859585; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EcSyvEaM+tWi+Tf3yEYgO9K5CPFL7iyJCMdJ029ORDM=; b=g8cOhOqxnCwTqpPDYXBkTlW69eXkOrFsVO4uX0YVNMyFD7w3Pd9EkmQm VXQEjN+N05rW5zfRUITNV/lIfJ1R916F7MiryI/n02Hi05X4G7IaXVnFs 5s+T7yiYqgwDi6rU4g3poC0Dtgii8eHLgxido+OUjiJvKTITe1mioCgRC 6WdcQp5Rsxk00mq3xG7QsjqZSb6gKH92JGc1yr5dcKkmqNgtBUtnSH1Qs xYouIl/5nMrSK+XddP8r0WUoPaRjGqgO4VPqwoIVHuLVEXoCc6bz1cRs1 eF9ZbZB/YCoRF0yaP9Zt0JNAgRlcVjmD23ZoEYgcC1qG23cKoZSMqRnwj A==; X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="328925839" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="328925839" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="700626137" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="700626137" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:04 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 818311224B5; Sat, 6 May 2023 00:53:01 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: laurent.pinchart@ideasonboard.com, tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com Subject: [RFC 3/7] media: uapi: v4l: Document source routes Date: Sat, 6 May 2023 00:52:53 +0300 Message-Id: <20230505215257.60704-4-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230505215257.60704-1-sakari.ailus@linux.intel.com> References: <20230505215257.60704-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Document how internal pads are used on source routes. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-subdev.rst | 18 ++++++++++++++++++ .../media/v4l/vidioc-subdev-g-routing.rst | 5 +++++ include/uapi/linux/v4l2-subdev.h | 6 +++++- 3 files changed, 28 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index a4f1df7093e8..395e06d2f0f2 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -551,6 +551,24 @@ A stream at a specific point in the media pipeline is identified by the sub-device and a (pad, stream) pair. For sub-devices that do not support multiplexed streams the 'stream' field is always 0. +.. _v4l2-subdev-source-routes: + +Source routes +^^^^^^^^^^^^^ + +Cases where a single sub-device pad is a source of multiple streams are special +as there is no sink pad for such a route. In those cases, an internal pad is +used as the sink pad. Such pads have the :ref:`MEDIA_PAD_FL_INTERNAL_SOURCE +` flag set. + +Internal pads have all the properties of a sink pad in such case, including +formats and selections. The format in this case is the source format of the +stream. An internal pad always has a single stream only (0). + +Generally source routes are not modifiable but they can be activated and +inactivated using the :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE +` flag, depending on driver capabilities. + Interaction between routes, streams, formats and selections ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst index 68ca343c3b44..e00622071b64 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst @@ -94,6 +94,11 @@ for all the route entries and call ``VIDIOC_SUBDEV_G_ROUTING`` again. * - __u32 - ``sink_pad`` - Sink pad number. + * - __u32 + - ``internal_source_pad`` + - Internal source pad number. For pads with :ref:`internal source pad flag + `, for use with :ref:`source routes + `. * - __u32 - ``sink_stream`` - Sink pad stream number. diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h index 4a195b68f28f..703e3a1f199b 100644 --- a/include/uapi/linux/v4l2-subdev.h +++ b/include/uapi/linux/v4l2-subdev.h @@ -203,6 +203,7 @@ struct v4l2_subdev_capability { * struct v4l2_subdev_route - A route inside a subdev * * @sink_pad: the sink pad index + * @internal_source_pad: the internal source pad * @sink_stream: the sink stream identifier * @source_pad: the source pad index * @source_stream: the source stream identifier @@ -210,7 +211,10 @@ struct v4l2_subdev_capability { * @reserved: drivers and applications must zero this array */ struct v4l2_subdev_route { - __u32 sink_pad; + union { + __u32 sink_pad; + __u32 internal_source_pad; + }; __u32 sink_stream; __u32 source_pad; __u32 source_stream; From patchwork Fri May 5 21:52:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 679837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82A79C7EE24 for ; Fri, 5 May 2023 21:53:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233551AbjEEVxK (ORCPT ); Fri, 5 May 2023 17:53:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230028AbjEEVxH (ORCPT ); Fri, 5 May 2023 17:53:07 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9725C4C28 for ; Fri, 5 May 2023 14:53:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683323586; x=1714859586; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fNdRQN/ML/8GxcbXUmF20S5kNZdqQ0oPBrnqJLIEGAk=; b=b2eYmtErHpNrpbPTQwz/ebIJVi89gU4Mi7LiZtrcB1Gszt8eSMjFWKAm t6cQOLcJh5ugTYAot9qoQn5IcBlpOLt5stl3yb1zPZYysPUNDeloZaonN gBb/efIAhqJz2eREg3h6vfkolsf86JjkWpkR1SDUsEEuH9ypSSVpUxziA q3xcD3mv+9LAgMyWjltAf+wDRhK3nksD1CD1OlzaLRD2Oj2rhon9sH9QS rPKlgoE1CaK4/+PhVLthe5k5p5h16BUqGXKLiqaR+UL3ynmfCia9Yrxi0 BfVWlv+Rgp2oGNDFgsNJefwVKOtPc49iqLOx8fsvIjiK2lP6zCfZQraUH Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="328925842" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="328925842" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="700626140" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="700626140" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:05 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 60B25122707; Sat, 6 May 2023 00:53:02 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: laurent.pinchart@ideasonboard.com, tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com Subject: [RFC 4/7] media: mc: Check pad flag validity Date: Sat, 6 May 2023 00:52:54 +0300 Message-Id: <20230505215257.60704-5-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230505215257.60704-1-sakari.ailus@linux.intel.com> References: <20230505215257.60704-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Check the validity of pad flags on entity init. Exactly one of the flags must be set. Signed-off-by: Sakari Ailus --- drivers/media/mc/mc-entity.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index ed99160a2487..c1bc48c4d239 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -197,6 +197,7 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, struct media_device *mdev = entity->graph_obj.mdev; struct media_pad *iter; unsigned int i = 0; + int ret = 0; if (num_pads >= MEDIA_ENTITY_MAX_PADS) return -E2BIG; @@ -210,6 +211,15 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, media_entity_for_each_pad(entity, iter) { iter->entity = entity; iter->index = i++; + + if (hweight32(iter->flags & (MEDIA_PAD_FL_SINK | + MEDIA_PAD_FL_SOURCE | + MEDIA_PAD_FL_INTERNAL_SOURCE)) + != 1) { + ret = -EINVAL; + break; + } + if (mdev) media_gobj_create(mdev, MEDIA_GRAPH_PAD, &iter->graph_obj); @@ -218,7 +228,7 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, if (mdev) mutex_unlock(&mdev->graph_mutex); - return 0; + return ret; } EXPORT_SYMBOL_GPL(media_entity_pads_init); From patchwork Fri May 5 21:52:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 679472 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61F10C77B7C for ; Fri, 5 May 2023 21:53:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233577AbjEEVxL (ORCPT ); Fri, 5 May 2023 17:53:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233505AbjEEVxI (ORCPT ); Fri, 5 May 2023 17:53:08 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82A2D4C37 for ; Fri, 5 May 2023 14:53:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683323587; x=1714859587; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xDj0tJLUEPW8cSNaxQvpW6xpEtqbvA5sN/gZkh5ZYpE=; b=ekDkUZozbnxz0K0wRfVybnB9fVTktongwZhtM/uH1I/XzdCJUJmMN3Ug I95ephM0NcqBJNcVQN1Jan5TcfPYzGjIf0M/U6thfea1wMmGSI/TEp6DE 4HLZHbfaxksDHRqNoIOfWH/NqaQuNqGYsjyD+BLFD2dZusq5UgqM90sXX fNhNkPHXFq82sUuBcSRQjZsTD4GIBIXgFUIk+7VGeV/1x05ODRSfT0RYC Ze3xFCSbDXxGipaJFrvZG4hw/IUdovjf1K8UhlYnyIqOBYmfpbOTyQzei PX+UJlQv+EiE6baEGRu6yCjLXiSxn85litpEs82o83z7QS3NIqli+dS8k w==; X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="328925844" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="328925844" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="700626145" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="700626145" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:05 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 27F8C1227C1; Sat, 6 May 2023 00:53:03 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: laurent.pinchart@ideasonboard.com, tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com Subject: [RFC 5/7] media: uapi: Add generic serial metadata mbus formats Date: Sat, 6 May 2023 00:52:55 +0300 Message-Id: <20230505215257.60704-6-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230505215257.60704-1-sakari.ailus@linux.intel.com> References: <20230505215257.60704-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add generic serial metadata mbus formats. These formats describe data width and packing but not the content itself. The reason for specifying such formats is that the formats as such are fairly device specific but they are still handled by CSI-2 receiver drivers that should not be aware of device specific formats. What makes generic metadata formats possible is that these formats are parsed by software only, after capturing the data to system memory. Signed-off-by: Sakari Ailus --- .../media/v4l/subdev-formats.rst | 257 ++++++++++++++++++ include/uapi/linux/media-bus-format.h | 9 + 2 files changed, 266 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index a3a35eeed708..1492fff58426 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -8234,3 +8234,260 @@ The following table lists the existing metadata formats. both sides of the link and the bus format is a fixed metadata format that is not configurable from userspace. Width and height will be set to 0 for this format. + +Generic Serial Metadata Formats +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Generic serial metadata formats are used on serial busses where the actual data +content is more or less device specific but the data is transmitted and received +by multiple devices that do not process the data in any way, simply writing +it to system memory for processing in software at the end of the pipeline. + +The more specific variant describing the actual data is used on the internal +source pad of the originating sub-device. + +"b" in an array cell signifies a byte of data, followed by the number of byte +and finally the bit number in subscript. "p" indicates a padding bit. + +.. _media-bus-format-generic-meta: + +.. cssclass: longtable + +.. flat-table:: Generic Serial Metadata Formats + :header-rows: 2 + :stub-columns: 0 + + * - Identifier + - Code + - + - :cspan:`23` Data organization + * - + - + - Bit + - 23 + - 22 + - 21 + - 20 + - 19 + - 18 + - 17 + - 16 + - 15 + - 14 + - 13 + - 12 + - 11 + - 10 + - 9 + - 8 + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + * .. _MEDIA-BUS-FMT-META-1X8-8: + + - MEDIA_BUS_FMT_META_1X8_8 + - 0x8001 + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + * .. _MEDIA-BUS-FMT-META-1X8-10: + + - MEDIA_BUS_FMT_META_1X8_10 + - 0x8002 + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + * .. _MEDIA-BUS-FMT-META-1X8-12: + + - MEDIA_BUS_FMT_META_1X8_12 + - 0x8003 + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-1X8-14: + + - MEDIA_BUS_FMT_META_1X8_14 + - 0x8004 + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-1X8-16: + + - MEDIA_BUS_FMT_META_1X8_16 + - 0x8005 + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-1X8-20: + + - MEDIA_BUS_FMT_META_1X8_20 + - 0x8007 + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-1X8-24: + + - MEDIA_BUS_FMT_META_1X8_24 + - 0x8009 + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index a03c543cb072..722463523bbd 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -173,4 +173,13 @@ */ #define MEDIA_BUS_FMT_METADATA_FIXED 0x7001 +/* Generic line based metadata formats for serial buses. Next is 0x800b. */ +#define MEDIA_BUS_FMT_META_1X8_8 0x8001 +#define MEDIA_BUS_FMT_META_1X8_10 0x8002 +#define MEDIA_BUS_FMT_META_1X8_12 0x8003 +#define MEDIA_BUS_FMT_META_1X8_14 0x8004 +#define MEDIA_BUS_FMT_META_1X8_16 0x8005 +#define MEDIA_BUS_FMT_META_1X8_20 0x8007 +#define MEDIA_BUS_FMT_META_1X8_24 0x8009 + #endif /* __LINUX_MEDIA_BUS_FORMAT_H */ From patchwork Fri May 5 21:52:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 679471 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98C4DC77B7F for ; Fri, 5 May 2023 21:53:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233651AbjEEVxM (ORCPT ); Fri, 5 May 2023 17:53:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52602 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229886AbjEEVxK (ORCPT ); Fri, 5 May 2023 17:53:10 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F37A4C28 for ; Fri, 5 May 2023 14:53:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683323588; x=1714859588; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4CnlJpZBmv2rnJJ9HG6MtY2RliaXbbG6t1V0p6kUcJ0=; b=g4MDhZ1+t74rqnf3wdkT8lHZl4YEsHYl72B07Imjeknnur27iRWHq2Xw Njt+XDSaG+ybwGWxXzXkoF1dpqj9P70S9fZeHAeoQBaHGemlpdXtwXWIb fxat3isvzuU2H+yOEAjsRI3tashX0L0mkV+xDCwyZS/t5qsXRZ/WV9sVS HDa2pCna1bTfq47MzHK678RT8ypjZee3yJMaE6jkmCT1QaxUKWE24a1V9 PHLnpg2LGW/UaqOeqfpEUIH9KGOZYpRKsHUjLuUtoRcWPyOoAWNz0kLCx GRCdVh8tACeX4ttUbhqJsEDrbwKJx09mH2Q+PNlfLjMBX9AYmcDhCn3DC A==; X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="328925847" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="328925847" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="700626150" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="700626150" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:06 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id E0A27121354; Sat, 6 May 2023 00:53:03 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: laurent.pinchart@ideasonboard.com, tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com Subject: [RFC 6/7] media: uapi: Add generic 8-bit metadata format definitions Date: Sat, 6 May 2023 00:52:56 +0300 Message-Id: <20230505215257.60704-7-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230505215257.60704-1-sakari.ailus@linux.intel.com> References: <20230505215257.60704-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Generic 8-bit metadata formats define the in-memory data layout but not the format of the data itself. The reasoning for having such formats is to allow CSI-2 receiver drivers to receive and DMA drivers to write the data to memory without knowing a large number of device specific formats. These formats may be used only in conjunction of a Media controller pipeline where the internal pad of the source sub-device defines the specific format of the data (using an mbus code). Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/meta-formats.rst | 1 + .../media/v4l/metafmt-generic.rst | 317 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 8 + include/uapi/linux/videodev2.h | 9 + 4 files changed, 335 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/metafmt-generic.rst diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst index 0bb61fc5bc00..919f595576b9 100644 --- a/Documentation/userspace-api/media/v4l/meta-formats.rst +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst @@ -19,3 +19,4 @@ These formats are used for the :ref:`metadata` interface only. metafmt-vsp1-hgo metafmt-vsp1-hgt metafmt-vivid + metafmt-generic diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst new file mode 100644 index 000000000000..09d3fabb1e94 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst @@ -0,0 +1,317 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later + +************************************************************************************************************************************************************************************************************************************************************************************************************************** +V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_META_FMT_GENERIC_CSI2_12 ('MC1C'), V4L2_META_FMT_GENERIC_CSI2_14 ('MC1E'), V4L2_META_FMT_GENERIC_CSI2_16 ('MC1G'), V4L2_META_FMT_GENERIC_CSI2_20 ('MC1K'), V4L2_META_FMT_GENERIC_CSI2_24 ('MC1O'), V4L2_META_FMT_GENERIC_CSI2_2_24 ('MC2O') +************************************************************************************************************************************************************************************************************************************************************************************************************************** + + +Generic line-based metadata formats + + +Description +=========== + +These generic line-based metadata formats define the memory layout of the data +without defining the format or meaning of the metadata itself. These formats may +only be used with a Media controller pipeline where the more specific format is +defined in an :ref:`internal source pad ` of the +source sub-device. See also :ref:`source routes `. + +.. _v4l2-meta-fmt-generic-8: + +V4L2_META_FMT_GENERIC_8 +----------------------- + +The V4L2_META_FMT_GENERIC_8 format is a plain 8-bit metadata format whereas the +rest of the formats have CSI-2 packing that are typically used on camera sensors +in conjunction with pixel data of the same bit depth. + +This format is also used on CSI-2 on both 8 bits per sample as well as on +16 bits per sample when two bytes of metadata are packed into one sample. + +**Byte Order Of V4L2_META_FMT_GENERIC_8.** +Each cell is one byte. "M" denotes a byte of metadata. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + * - start + 4: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + +.. _v4l2-meta-fmt-generic-csi2-10: + +V4L2_META_FMT_GENERIC_CSI2_10 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_10 contains 8-bit generic metadata with CSI-2 +packing, 10 bits for each 8 bits of data. Every four bytes of metadata is +followed by a single byte of padding. + +This format is also used on CSI2 on 20 bits per sample format that packs two +bytes of metadata into one sample. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_10.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + * - start + 5: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + +.. _v4l2-meta-fmt-generic-csi2-12: + +V4L2_META_FMT_GENERIC_CSI2_12 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_12 contains 8-bit generic metadata with CSI-2 +packing, 12 bits for each 8 bits of data. Every four bytes of metadata is +followed by two bytes of padding. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_12.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + - p + * - start + 6: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-14: + +V4L2_META_FMT_GENERIC_CSI2_14 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_14 contains 8-bit generic metadata with CSI-2 +packing, 14 bits for each 8 bits of data. Every four bytes of metadata is +followed by three bytes of padding. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_14.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + - p + - p + * - start + 7: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-16: + +V4L2_META_FMT_GENERIC_CSI2_16 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_16 contains 8-bit generic metadata with CSI-2 +packing, 16 bits for each 8 bits of data. Every byte of metadata is +followed by one byte of padding. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_16.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - M\ :sub:`10` + - p + - M\ :sub:`20` + - p + - M\ :sub:`30` + - p + * - start + 8: + - M\ :sub:`01` + - p + - M\ :sub:`11` + - p + - M\ :sub:`21` + - p + - M\ :sub:`31` + - p + +.. _v4l2-meta-fmt-generic-csi2-20: + +V4L2_META_FMT_GENERIC_CSI2_20 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_20 contains 8-bit generic metadata with CSI-2 +packing, 20 bits for each 8 bits of data. Every byte of metadata is followed by +alternating one and two bytes of padding. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_20.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - M\ :sub:`10` + - p + - p + - M\ :sub:`20` + - p + - M\ :sub:`30` + - p + - p + * - start + 10: + - M\ :sub:`01` + - p + - M\ :sub:`11` + - p + - p + - M\ :sub:`21` + - p + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-24: + +V4L2_META_FMT_GENERIC_CSI2_24 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_24 contains 8-bit generic metadata with CSI-2 +packing, 24 bits for each 8 bits of data. Every byte of metadata is followed by +alternating one and two bytes of padding. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_24.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - p + - M\ :sub:`10` + - p + - p + - M\ :sub:`20` + - p + - p + - M\ :sub:`30` + - p + - p + * - start + 12: + - M\ :sub:`01` + - p + - p + - M\ :sub:`11` + - p + - p + - M\ :sub:`21` + - p + - p + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-2-24: + +V4L2_META_FMT_GENERIC_CSI2_2_24 +------------------------------- + +V4L2_META_FMT_GENERIC_CSI2_2_24 contains 8-bit generic metadata with CSI-2 +packing, 24 bits for each two times 8 bits of data. Every two bytes of metadata +are followed by one byte of padding. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_2_24.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - p + - M\ :sub:`20` + - M\ :sub:`30` + - p + * - start + 6: + - M\ :sub:`01` + - M\ :sub:`11` + - p + - M\ :sub:`21` + - M\ :sub:`31` + - p + diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index a858acea6547..7781b0bd3e7d 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1451,6 +1451,14 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; + case V4L2_META_FMT_GENERIC_8: descr = "8-bit Generic Metadata"; break; + case V4L2_META_FMT_GENERIC_CSI2_10: descr = "8b Generic Meta, 10b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_12: descr = "8b Generic Meta, 12b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_14: descr = "8b Generic Meta, 14b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_16: descr = "8b Generic Meta, 16b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_20: descr = "8b Generic Meta, 20b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_24: descr = "8b Generic Meta, 24b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_2_24: descr = "2x8b Generic Meta, 24b CSI-2"; break; default: /* Compressed formats */ diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index aee75eb9e686..adcbdc15dcdb 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -834,6 +834,15 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ +#define V4L2_META_FMT_GENERIC_8 v4l2_fourcc('M', 'E', 'T', '8') /* Generic 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_10 v4l2_fourcc('M', 'C', '1', 'A') /* 10-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_12 v4l2_fourcc('M', 'C', '1', 'C') /* 12-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_14 v4l2_fourcc('M', 'C', '1', 'E') /* 14-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_16 v4l2_fourcc('M', 'C', '1', 'G') /* 16-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_20 v4l2_fourcc('M', 'C', '1', 'K') /* 20-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_24 v4l2_fourcc('M', 'C', '1', 'O') /* 24-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_2_24 v4l2_fourcc('M', 'C', '2', 'O') /* 2 bytes of 8-bit metadata, 24-bit CSI-2 packed */ + /* priv field value to indicates that subsequent fields are valid. */ #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe From patchwork Fri May 5 21:52:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 679836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31741C77B75 for ; Fri, 5 May 2023 21:53:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233666AbjEEVxN (ORCPT ); Fri, 5 May 2023 17:53:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233627AbjEEVxK (ORCPT ); Fri, 5 May 2023 17:53:10 -0400 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 034984ECF for ; Fri, 5 May 2023 14:53:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683323588; x=1714859588; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PuBUFxBezyIHCdrKqd1+uqhhvDLY//GiI77j0EPWuxo=; b=JXSni1FqptfXaixav7uJzC4mM9mUF33dfM6kkttdDknDaaiFpiQJqbWG QxK/UKGWxz9i2I2Mh8xy+k9w4IGxBQdHAvyZ+yng5KZgu5kustbVxB7Zl e+vhEbmHgvWFT8IMc4XX36aHRCfm9b4BZXB7PTvE2V5bkiUZ/6MbJ2RoP h2PpoJZ2nS9DcyMf9vhlNKTKHiDMRnm7iMKUs0s16wxCxab1i/xBlgJht dl2fD6ldkrzJb+1KVbLu7nAcNym+cTtvARzY5O05JKXK0AQzXcIXBYXAS cmGZpZW1biGftZSWOYMNYnmxG4Akg+vyp+H1j4K/ffJNqrzu/jgS5lC7Y Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="328925850" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="328925850" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10701"; a="700626155" X-IronPort-AV: E=Sophos;i="5.99,253,1677571200"; d="scan'208";a="700626155" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2023 14:53:07 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id C2B3212244C; Sat, 6 May 2023 00:53:04 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: laurent.pinchart@ideasonboard.com, tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com Subject: [RFC 7/7] media: v4l: Support line-based metadata capture Date: Sat, 6 May 2023 00:52:57 +0300 Message-Id: <20230505215257.60704-8-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230505215257.60704-1-sakari.ailus@linux.intel.com> References: <20230505215257.60704-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org many camera sensors, among other devices, transmit embedded data and image data for each CSI-2 frame. This embedded data typically contains register configuration of the sensor that has been used to capture the image data of the same frame. The embedded data is received by the CSI-2 receiver and has the same properties as the image data, including that it is line based: it has width, height and bytesperline (stride). Add these fields to struct v4l2_meta_format and document them. Also add V4L2_FMT_FLAG_META_LINE_BASED to tell a given format is line-based i.e. these fields of struct v4l2_meta_format are valid for it. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-meta.rst | 15 +++++++++++++++ .../userspace-api/media/v4l/vidioc-enum-fmt.rst | 7 +++++++ include/uapi/linux/videodev2.h | 10 ++++++++++ 3 files changed, 32 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/dev-meta.rst b/Documentation/userspace-api/media/v4l/dev-meta.rst index 0e7e1ee1471a..7d3a64514db0 100644 --- a/Documentation/userspace-api/media/v4l/dev-meta.rst +++ b/Documentation/userspace-api/media/v4l/dev-meta.rst @@ -65,3 +65,18 @@ to 0. - ``buffersize`` - Maximum buffer size in bytes required for data. The value is set by the driver. + * - __u32 + - ``width`` + - Width of a line of metadata in bytes. Valid when :c:type`v4l2_fmtdesc` + flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``height`` + - Height of a line of metadata in bytes. Valid when :c:type`v4l2_fmtdesc` + flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``bytesperlines`` + - Offset in bytes between the beginning of two consecutive lines. Valid + when :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is + set, otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. diff --git a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst index 000c154b0f98..6d7664345a4e 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst @@ -227,6 +227,13 @@ the ``mbus_code`` field is handled differently: The application can ask to configure the quantization of the capture device when calling the :ref:`VIDIOC_S_FMT ` ioctl with :ref:`V4L2_PIX_FMT_FLAG_SET_CSC ` set. + * - ``V4L2_FMT_FLAG_META_LINE_BASED`` + - 0x0200 + - The metadata format is line-based. In this case the ``width``, + ``height`` and ``bytesperline`` fields of :c:type:`v4l2_meta_format` are + valid. The buffer consists of ``height`` lines, each having ``width`` + bytes of data and offset between the beginning of each two consecutive + lines is ``bytesperline``. Return Value ============ diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index adcbdc15dcdb..3681b2c15901 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -873,6 +873,7 @@ struct v4l2_fmtdesc { #define V4L2_FMT_FLAG_CSC_YCBCR_ENC 0x0080 #define V4L2_FMT_FLAG_CSC_HSV_ENC V4L2_FMT_FLAG_CSC_YCBCR_ENC #define V4L2_FMT_FLAG_CSC_QUANTIZATION 0x0100 +#define V4L2_FMT_FLAG_META_LINE_BASED 0x0200 /* Frame Size and frame rate enumeration */ /* @@ -2407,10 +2408,19 @@ struct v4l2_sdr_format { * struct v4l2_meta_format - metadata format definition * @dataformat: little endian four character code (fourcc) * @buffersize: maximum size in bytes required for data + * @width: number of bytes of data per line (valid for line based + * formats only, see format documentation) + * @height: number of lines of data per buffer (valid for line based + * formats only) + * @bytesperline: offset between the beginnings of two adjacent lines in + * bytes (valid for line based formats only) */ struct v4l2_meta_format { __u32 dataformat; __u32 buffersize; + __u32 width; + __u32 height; + __u32 bytesperline; } __attribute__ ((packed)); /**