From patchwork Tue Oct 3 11:52:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B0B9E7544F for ; Tue, 3 Oct 2023 11:52:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240493AbjJCLwx (ORCPT ); Tue, 3 Oct 2023 07:52:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231770AbjJCLww (ORCPT ); Tue, 3 Oct 2023 07:52:52 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 506A1A6 for ; Tue, 3 Oct 2023 04:52:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696333969; x=1727869969; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kn0wzA+uAt7xFQzvaxyZOJ5t8GgKMQyISUwz6L8y9gw=; b=g40qcmfOT6HJdxA9Hkeqw83byeDAUjb1W51R8Us372WqMIcDFBXU7Bgr ycAmCEU2lllEFtpN629koNC1l61gGcEudGMq2EKE5mk0a+xfJVDzyStol YnS8SwbQHz1FqftDJZC0YmgTiqtX+PzH0l6LnUYjL5lwcPtsVC9WOjqvU 0yCXnMYIWjRlAaIoosSJFMwN9CrktcWN/PTag63BRPBZyj5H2cHlrmPA3 OQEP3o8sFDdYXngkGuD58zh6W9QlQW8EpEcZEtn8kScuaVESDYzPWMBRR og5c8e0nOfBiXeuuEOB57JTOsqFrKejZBVC1UJ2i76V50TryZ5962nXNR w==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="413767097" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="413767097" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="816650485" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="816650485" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:42 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 788AB120A2E; Tue, 3 Oct 2023 14:52:39 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 01/28] media: mc: Add INTERNAL pad flag Date: Tue, 3 Oct 2023 14:52:29 +0300 Message-Id: <20231003115237.76828-2-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Internal source pads will be used as routing endpoints in V4L2 [GS]_ROUTING IOCTLs, to indicate that the stream begins in the entity. Also prevent creating links to pads that have been flagged as internal and initialising source pads with internal flag set. Signed-off-by: Sakari Ailus --- Documentation/userspace-api/media/glossary.rst | 6 ++++++ .../userspace-api/media/mediactl/media-types.rst | 6 ++++++ drivers/media/mc/mc-entity.c | 10 ++++++++-- include/uapi/linux/media.h | 1 + 4 files changed, 21 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/glossary.rst b/Documentation/userspace-api/media/glossary.rst index 96a360edbf3b..f7b99a4527c7 100644 --- a/Documentation/userspace-api/media/glossary.rst +++ b/Documentation/userspace-api/media/glossary.rst @@ -173,6 +173,12 @@ Glossary An integrated circuit that integrates all components of a computer or other electronic systems. +_media-glossary-stream: + Stream + A distinct flow of data (image data or metadata) over a media pipeline + from source to sink. A source may be e.g. an image sensor and a sink + e.g. a memory buffer. + V4L2 API **V4L2 userspace API** diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/Documentation/userspace-api/media/mediactl/media-types.rst index 0ffeece1e0c8..28941da27790 100644 --- a/Documentation/userspace-api/media/mediactl/media-types.rst +++ b/Documentation/userspace-api/media/mediactl/media-types.rst @@ -361,6 +361,7 @@ Types and flags used to represent the media graph elements .. _MEDIA-PAD-FL-SINK: .. _MEDIA-PAD-FL-SOURCE: .. _MEDIA-PAD-FL-MUST-CONNECT: +.. _MEDIA-PAD-FL-INTERNAL: .. flat-table:: Media pad flags :header-rows: 0 @@ -382,6 +383,11 @@ Types and flags used to represent the media graph elements when this flag isn't set; the absence of the flag doesn't imply there is none. + * - ``MEDIA_PAD_FL_INTERNAL`` + - The internal flag indicates an internal pad that has no external + connections. Such a pad shall not be connected with a link. The + internal flag indicates that the :ref:``stream + `` either starts or ends in the entity. One and only one of ``MEDIA_PAD_FL_SINK`` and ``MEDIA_PAD_FL_SOURCE`` must be set for every pad. diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 543a392f8635..f5f290781021 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -213,7 +213,9 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, iter->index = i++; if (hweight32(iter->flags & (MEDIA_PAD_FL_SINK | - MEDIA_PAD_FL_SOURCE)) != 1) { + MEDIA_PAD_FL_SOURCE)) != 1 || + (iter->flags & MEDIA_PAD_FL_INTERNAL && + !(iter->flags & MEDIA_PAD_FL_SINK))) { ret = -EINVAL; break; } @@ -1075,7 +1077,8 @@ int media_get_pad_index(struct media_entity *entity, u32 pad_type, for (i = 0; i < entity->num_pads; i++) { if ((entity->pads[i].flags & - (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE)) != pad_type) + (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE | + MEDIA_PAD_FL_INTERNAL)) != pad_type) continue; if (entity->pads[i].sig_type == sig_type) @@ -1100,6 +1103,9 @@ media_create_pad_link(struct media_entity *source, u16 source_pad, return -EINVAL; if (WARN_ON(!(sink->pads[sink_pad].flags & MEDIA_PAD_FL_SINK))) return -EINVAL; + if (WARN_ON(source->pads[source_pad].flags & MEDIA_PAD_FL_INTERNAL) || + WARN_ON(source->pads[sink_pad].flags & MEDIA_PAD_FL_INTERNAL)) + return -EINVAL; link = media_add_link(&source->links); if (link == NULL) diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h index 1c80b1d6bbaf..80cfd12a43fc 100644 --- a/include/uapi/linux/media.h +++ b/include/uapi/linux/media.h @@ -208,6 +208,7 @@ struct media_entity_desc { #define MEDIA_PAD_FL_SINK (1U << 0) #define MEDIA_PAD_FL_SOURCE (1U << 1) #define MEDIA_PAD_FL_MUST_CONNECT (1U << 2) +#define MEDIA_PAD_FL_INTERNAL (1U << 3) struct media_pad_desc { __u32 entity; /* entity ID */ From patchwork Tue Oct 3 11:52:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FDEEE75452 for ; Tue, 3 Oct 2023 11:52:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231849AbjJCLwx (ORCPT ); Tue, 3 Oct 2023 07:52:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232058AbjJCLww (ORCPT ); Tue, 3 Oct 2023 07:52:52 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 624DCA7 for ; Tue, 3 Oct 2023 04:52:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696333969; x=1727869969; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PrIYsWsx/AHSo5nDA3v2jd2fa1WTup9gwXhDzEGlrV8=; b=PUzc1IJm9ugWxufzMfBNfOCuNkyINoBOhbmvVnQpO6uKYUbahJDZ+xKr koWArE3jaC2V/5cYS0wmBCXGAToADBSCfJ/aDrBnIkfvAzAdaAMtspwQa ZNX9o26JUK8yhDmfUEq6ZjrJDjuJSCvaGx07GjBUfQQIe2kLk4wxH+MqN nEkf+FVyczqX9dz0xb9lVXiyTSDmEnbXOFREPb6jPZI/dK2hgwARwlB45 f9Ln0SJVA2APEDaqVIDk365HOIdrCOohNchc4vK4okTEAvzLwG/kWOtyo KLgU9Y5v7MXu2qLGu+lKwXuo6krL1KydDrcMkrWtNAFwB9HCKF0J37Gq/ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="413767109" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="413767109" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="816650495" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="816650495" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:44 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id B654B120A56; Tue, 3 Oct 2023 14:52:40 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 02/28] media: uapi: Add generic serial metadata mbus formats Date: Tue, 3 Oct 2023 14:52:30 +0300 Message-Id: <20231003115237.76828-3-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add generic serial metadata mbus formats. These formats describe data width and packing but not the content itself. The reason for specifying such formats is that the formats as such are fairly device specific but they are still handled by CSI-2 receiver drivers that should not be aware of device specific formats. What makes generic metadata formats possible is that these formats are parsed by software only, after capturing the data to system memory. Also add a definition for "Data unit" to cover what is essentially a pixel but is not image data. Signed-off-by: Sakari Ailus --- .../userspace-api/media/glossary.rst | 8 + .../media/v4l/subdev-formats.rst | 258 ++++++++++++++++++ include/uapi/linux/media-bus-format.h | 9 + 3 files changed, 275 insertions(+) diff --git a/Documentation/userspace-api/media/glossary.rst b/Documentation/userspace-api/media/glossary.rst index f7b99a4527c7..65217b8a44cc 100644 --- a/Documentation/userspace-api/media/glossary.rst +++ b/Documentation/userspace-api/media/glossary.rst @@ -25,6 +25,14 @@ Glossary See :ref:`cec`. +.. _media-glossary-data-unit: + + Data unit + + Unit of data transported by a bus. On parallel buses, this is called a + sample while on serial buses the data unit is logical. If the data unit + is image data, it may also be called a pixel. + Device Driver Part of the Linux Kernel that implements support for a hardware component. diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index a3a35eeed708..c54bf834d839 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -8234,3 +8234,261 @@ The following table lists the existing metadata formats. both sides of the link and the bus format is a fixed metadata format that is not configurable from userspace. Width and height will be set to 0 for this format. + +Generic Serial Metadata Formats +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Generic serial metadata formats are used on serial buses where the actual data +content is more or less device specific but the data is transmitted and received +by multiple devices that do not process the data in any way, simply writing +it to system memory for processing in software at the end of the pipeline. + +The more specific variant describing the actual data is used on the internal +source pad of the originating sub-device. + +"b" in an array cell signifies a byte of data, followed by the number of the bit +and finally the bit number in subscript. "X" indicates a padding bit. + +.. _media-bus-format-generic-meta: + +.. cssclass: longtable + +.. flat-table:: Generic Serial Metadata Formats + :header-rows: 2 + :stub-columns: 0 + + * - Identifier + - Code + - + - :cspan:`23` Data organization within bus ``Data unit + `` + * - + - + - Bit + - 23 + - 22 + - 21 + - 20 + - 19 + - 18 + - 17 + - 16 + - 15 + - 14 + - 13 + - 12 + - 11 + - 10 + - 9 + - 8 + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + * .. _MEDIA-BUS-FMT-META-8: + + - MEDIA_BUS_FMT_META_8 + - 0x8001 + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + * .. _MEDIA-BUS-FMT-META-10: + + - MEDIA_BUS_FMT_META_10 + - 0x8002 + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - X + - X + * .. _MEDIA-BUS-FMT-META-12: + + - MEDIA_BUS_FMT_META_12 + - 0x8003 + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - X + - X + - X + - X + * .. _MEDIA-BUS-FMT-META-14: + + - MEDIA_BUS_FMT_META_14 + - 0x8004 + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - X + - X + - X + - X + - X + - X + * .. _MEDIA-BUS-FMT-META-16: + + - MEDIA_BUS_FMT_META_16 + - 0x8005 + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - X + - X + - X + - X + - X + - X + - X + - X + * .. _MEDIA-BUS-FMT-META-20: + + - MEDIA_BUS_FMT_META_20 + - 0x8006 + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - X + - X + - X + - X + - X + - X + - X + - X + - X + - X + - X + - X + * .. _MEDIA-BUS-FMT-META-24: + + - MEDIA_BUS_FMT_META_24 + - 0x8007 + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - X + - X + - X + - X + - X + - X + - X + - X + - X + - X + - X + - X + - X + - X + - X + - X diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index a03c543cb072..9ee031397372 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -173,4 +173,13 @@ */ #define MEDIA_BUS_FMT_METADATA_FIXED 0x7001 +/* Generic line based metadata formats for serial buses. Next is 0x8008. */ +#define MEDIA_BUS_FMT_META_8 0x8001 +#define MEDIA_BUS_FMT_META_10 0x8002 +#define MEDIA_BUS_FMT_META_12 0x8003 +#define MEDIA_BUS_FMT_META_14 0x8004 +#define MEDIA_BUS_FMT_META_16 0x8005 +#define MEDIA_BUS_FMT_META_20 0x8006 +#define MEDIA_BUS_FMT_META_24 0x8007 + #endif /* __LINUX_MEDIA_BUS_FORMAT_H */ From patchwork Tue Oct 3 11:52:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E96BFE75454 for ; Tue, 3 Oct 2023 11:52:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240607AbjJCLwy (ORCPT ); Tue, 3 Oct 2023 07:52:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239993AbjJCLww (ORCPT ); Tue, 3 Oct 2023 07:52:52 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44808A3 for ; Tue, 3 Oct 2023 04:52:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696333970; x=1727869970; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mtd2OlaD2uF9uQfv/VWnnRlsGNgouPl7W3/VQMYJfmg=; b=EhWpv6iV0TsweJJqYKZ3yqyHsWPey9JxB1tinPXLwk3/CK3SkIano8r2 Ub0sy1FWiZZq0k84xHHtbmsl3wablcLY/gjz8xxwNQfQ7Pz1ulPWUSkGa 7zYe/p8Xy8+SVKkSuv6B0GRJTcPnWZsA2b0QyPKAeHATRuDFetY1Tov1Q v1ATe48e8++1h74neVpn4hrCAvwKxsg5TL/eA8ffD6NW8rNy76KwgcKTY AbMhhMGx3uO3FELMyT8hpOTrZtYVTzAtuuq0pcXqJVLaNe7T7ouRD2dcW Ojuv9CFz5qUVJ9iebDz9nPULy7TDo6sPh4UPlE40d4ox+ZjzNL4SNyrCA A==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="413767115" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="413767115" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="816650502" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="816650502" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:46 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 34D4C120A57; Tue, 3 Oct 2023 14:52:42 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 03/28] media: uapi: Document which mbus format fields are valid for metadata Date: Tue, 3 Oct 2023 14:52:31 +0300 Message-Id: <20231003115237.76828-4-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Now that metadata mbus formats have been added, it is necessary to define which fields in struct v4l2_mbus_format are applicable to them (not many). Signed-off-by: Sakari Ailus --- include/uapi/linux/v4l2-mediabus.h | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/include/uapi/linux/v4l2-mediabus.h b/include/uapi/linux/v4l2-mediabus.h index 6b07b73473b5..3cadb3b58b85 100644 --- a/include/uapi/linux/v4l2-mediabus.h +++ b/include/uapi/linux/v4l2-mediabus.h @@ -19,12 +19,18 @@ * @width: image width * @height: image height * @code: data format code (from enum v4l2_mbus_pixelcode) - * @field: used interlacing type (from enum v4l2_field) - * @colorspace: colorspace of the data (from enum v4l2_colorspace) - * @ycbcr_enc: YCbCr encoding of the data (from enum v4l2_ycbcr_encoding) - * @hsv_enc: HSV encoding of the data (from enum v4l2_hsv_encoding) - * @quantization: quantization of the data (from enum v4l2_quantization) - * @xfer_func: transfer function of the data (from enum v4l2_xfer_func) + * @field: used interlacing type (from enum v4l2_field), not applicable + * to metadata mbus codes + * @colorspace: colorspace of the data (from enum v4l2_colorspace), zero on + * metadata mbus codes + * @ycbcr_enc: YCbCr encoding of the data (from enum v4l2_ycbcr_encoding), zero + * on metadata mbus codes + * @hsv_enc: HSV encoding of the data (from enum v4l2_hsv_encoding), zero on + * metadata mbus codes + * @quantization: quantization of the data (from enum v4l2_quantization), zero + * on metadata mbus codes + * @xfer_func: transfer function of the data (from enum v4l2_xfer_func), zero + * on metadata mbus codes * @flags: flags (V4L2_MBUS_FRAMEFMT_*) * @reserved: reserved bytes that can be later used */ From patchwork Tue Oct 3 11:52:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 731DEE75451 for ; Tue, 3 Oct 2023 11:52:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240616AbjJCLwz (ORCPT ); Tue, 3 Oct 2023 07:52:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240523AbjJCLwy (ORCPT ); Tue, 3 Oct 2023 07:52:54 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC094A6 for ; Tue, 3 Oct 2023 04:52:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696333970; x=1727869970; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IUm42fmekmMuZUlwhbf+2Nf/J7pOVxmL53D0Qvv7xWk=; b=OwyZM0FmDT6oJHTPPAFncFyxH+jqFEZyGeA98L/9XQCpPBkmzW+KrNEX ma6iPKuc8igqpbtWwQq8S2FvRrhokoHfZvERu8VkoivFSLLmG74Oos20a gzMUc398qi1Nfyy8Rn3wvD6Uoufd1Yu1OO1POBHLVAovtpmdXWL1vi88P +j8AgkkgidvIRIl45/PnoAuzZ2j1allMw2mbFPI14Fi4lBpq2NjjTtS/i 9HMMl8Fr2TlibXPnmVeqgTGw3zRjcWzOa0g6bVxvpbhdvnR4H2h/Xe9UH NTAwGHNjr3sOXfTdqN5Ia3ZwHsYLe8suU1pGcwBLWBEcx2Go7OR8j/+hd g==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="413767123" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="413767123" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="816650513" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="816650513" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:47 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 97BA5120A58; Tue, 3 Oct 2023 14:52:43 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 04/28] media: uapi: Add generic 8-bit metadata format definitions Date: Tue, 3 Oct 2023 14:52:32 +0300 Message-Id: <20231003115237.76828-5-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Generic 8-bit metadata formats define the in-memory data layout but not the format of the data itself. The reasoning for having such formats is to allow CSI-2 receiver drivers to receive and DMA drivers to write the data to memory without knowing a large number of device specific formats. These formats may be used only in conjunction of a Media controller pipeline where the internal pad of the source sub-device defines the specific format of the data (using an mbus code). Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-subdev.rst | 2 + .../userspace-api/media/v4l/meta-formats.rst | 1 + .../media/v4l/metafmt-generic.rst | 304 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 7 + include/uapi/linux/videodev2.h | 8 + 5 files changed, 322 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/metafmt-generic.rst diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index 43988516acdd..f375b820ab68 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -506,6 +506,8 @@ source pads. subdev-formats +.. _subdev-routing: + Streams, multiplexed media pads and internal routing ---------------------------------------------------- diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst index 0bb61fc5bc00..919f595576b9 100644 --- a/Documentation/userspace-api/media/v4l/meta-formats.rst +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst @@ -19,3 +19,4 @@ These formats are used for the :ref:`metadata` interface only. metafmt-vsp1-hgo metafmt-vsp1-hgt metafmt-vivid + metafmt-generic diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst new file mode 100644 index 000000000000..2ebab1f895e0 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst @@ -0,0 +1,304 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later + +************************************************************************************************************************************************************************************************************************************************************************************************************************** +V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_META_FMT_GENERIC_CSI2_12 ('MC1C'), V4L2_META_FMT_GENERIC_CSI2_14 ('MC1E'), V4L2_META_FMT_GENERIC_CSI2_16 ('MC1G'), V4L2_META_FMT_GENERIC_CSI2_20 ('MC1K'), V4L2_META_FMT_GENERIC_CSI2_24 ('MC1O') +************************************************************************************************************************************************************************************************************************************************************************************************************************** + + +Generic line-based metadata formats + + +Description +=========== + +These generic line-based metadata formats define the memory layout of the data +without defining the format or meaning of the metadata itself. These formats may +only be used with a Media controller pipeline where the more specific format is +defined in an :ref:`internal source pad ` of the source +sub-device. See also :ref:`source routes `. + +.. _v4l2-meta-fmt-generic-8: + +V4L2_META_FMT_GENERIC_8 +----------------------- + +The V4L2_META_FMT_GENERIC_8 format is a plain 8-bit metadata format. + +This format is also used on CSI-2 for both 8 bits per ``Data unit +`` as well as for 16 bits per Data unit when two bytes +of metadata are packed into one 16-bit Data unit. + +**Byte Order Of V4L2_META_FMT_GENERIC_8.** +Each cell is one byte. "M" denotes a byte of metadata. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + * - start + 4: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + +.. _v4l2-meta-fmt-generic-csi2-10: + +V4L2_META_FMT_GENERIC_CSI2_10 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_10 contains packed 8-bit generic metadata, 10 bits +for each 8 bits of data. Every four bytes of metadata is followed by a single +byte of padding. The way the data is packed follows the MIPI CSI-2 specification +and the padding is defined in the MIPI CCS specification. + +This format is also used in conjunction with 20 bits per ``Data unit +`` formats that pack two bytes of metadata into one +Data unit. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_10.** +Each cell is one byte. "M" denotes a byte of metadata and "X" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - X + * - start + 5: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - X + +.. _v4l2-meta-fmt-generic-csi2-12: + +V4L2_META_FMT_GENERIC_CSI2_12 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_12 contains packed 8-bit generic metadata, 12 bits +for each 8 bits of data. Every four bytes of metadata is followed by a single +byte of padding. The way the data is packed follows the MIPI CSI-2 specification +and the padding is defined in the MIPI CCS specification. + +This format is also used in conjunction with 24 bits per ``Data unit +`` formats that pack two bytes of metadata into one +Data unit. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_12.** +Each cell is one byte. "M" denotes a byte of metadata and "X" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - X + - M\ :sub:`20` + - M\ :sub:`30` + - X + * - start + 6: + - M\ :sub:`01` + - M\ :sub:`11` + - X + - M\ :sub:`21` + - M\ :sub:`31` + - X + +.. _v4l2-meta-fmt-generic-csi2-14: + +V4L2_META_FMT_GENERIC_CSI2_14 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_14 contains packed 8-bit generic metadata, 14 bits +for each 8 bits of data. Every four bytes of metadata is followed by three bytes +of padding. The way the data is packed follows the MIPI CSI-2 specification and +the padding is defined in the MIPI CCS specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_14.** +Each cell is one byte. "M" denotes a byte of metadata and "X" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - X + - X + - X + * - start + 7: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - X + - X + - X + +.. _v4l2-meta-fmt-generic-csi2-16: + +V4L2_META_FMT_GENERIC_CSI2_16 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_16 contains packed 8-bit generic metadata, 16 bits +for each 8 bits of data. Every byte of metadata is followed by one byte of +padding. The way the data is packed follows the MIPI CSI-2 specification and the +padding is defined in the MIPI CCS specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_16.** +Each cell is one byte. "M" denotes a byte of metadata and "X" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - X + - M\ :sub:`10` + - X + - M\ :sub:`20` + - X + - M\ :sub:`30` + - X + * - start + 8: + - M\ :sub:`01` + - X + - M\ :sub:`11` + - X + - M\ :sub:`21` + - X + - M\ :sub:`31` + - X + +.. _v4l2-meta-fmt-generic-csi2-20: + +V4L2_META_FMT_GENERIC_CSI2_20 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_20 contains packed 8-bit generic metadata, 20 bits +for each 8 bits of data. Every byte of metadata is followed by alternating one +and two bytes of padding. The way the data is packed follows the MIPI CSI-2 +specification and the padding is defined in the MIPI CCS specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_20.** +Each cell is one byte. "M" denotes a byte of metadata and "X" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - X + - M\ :sub:`10` + - X + - X + - M\ :sub:`20` + - X + - M\ :sub:`30` + - X + - X + * - start + 10: + - M\ :sub:`01` + - X + - M\ :sub:`11` + - X + - X + - M\ :sub:`21` + - X + - M\ :sub:`31` + - X + - X + +.. _v4l2-meta-fmt-generic-csi2-24: + +V4L2_META_FMT_GENERIC_CSI2_24 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_24 contains packed 8-bit generic metadata, 24 bits +for each 8 bits of data. Every byte of metadata is followed by two bytes of +padding. The way the data is packed follows the MIPI CSI-2 specification and the +padding is defined in the MIPI CCS specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_24.** +Each cell is one byte. "M" denotes a byte of metadata and "X" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - X + - X + - M\ :sub:`10` + - X + - X + - M\ :sub:`20` + - X + - X + - M\ :sub:`30` + - X + - X + * - start + 12: + - M\ :sub:`01` + - X + - X + - M\ :sub:`11` + - X + - X + - M\ :sub:`21` + - X + - X + - M\ :sub:`31` + - X + - X diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 9b1de54ce379..ce4b3929ff5f 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1452,6 +1452,13 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; + case V4L2_META_FMT_GENERIC_8: descr = "8-bit Generic Metadata"; break; + case V4L2_META_FMT_GENERIC_CSI2_10: descr = "8b Generic Meta, 10b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_12: descr = "8b Generic Meta, 12b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_14: descr = "8b Generic Meta, 14b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_16: descr = "8b Generic Meta, 16b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_20: descr = "8b Generic Meta, 20b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_24: descr = "8b Generic Meta, 24b CSI-2"; break; default: /* Compressed formats */ diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index c3d4e490ce7c..2b16b06ad278 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -839,6 +839,14 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ +#define V4L2_META_FMT_GENERIC_8 v4l2_fourcc('M', 'E', 'T', '8') /* Generic 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_10 v4l2_fourcc('M', 'C', '1', 'A') /* 10-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_12 v4l2_fourcc('M', 'C', '1', 'C') /* 12-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_14 v4l2_fourcc('M', 'C', '1', 'E') /* 14-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_16 v4l2_fourcc('M', 'C', '1', 'G') /* 16-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_20 v4l2_fourcc('M', 'C', '1', 'K') /* 20-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_24 v4l2_fourcc('M', 'C', '1', 'O') /* 24-bit CSI-2 packed 8-bit metadata */ + /* priv field value to indicates that subsequent fields are valid. */ #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe From patchwork Tue Oct 3 11:52:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729442 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D663EE75453 for ; Tue, 3 Oct 2023 11:52:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240617AbjJCLw4 (ORCPT ); Tue, 3 Oct 2023 07:52:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240614AbjJCLwz (ORCPT ); Tue, 3 Oct 2023 07:52:55 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 848A4A3 for ; Tue, 3 Oct 2023 04:52:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696333971; x=1727869971; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=u7KawUKPiQSEjWm/faf7YsODCZ4pxBMjgviiwVBeQ/Y=; b=EA1BqwxabxfG+n+v8LkUgsWs/W5EzqJNAVJXYH2/5Mw1ueJuXjlVrA6D 6r4pQvQI2kcAGn039RB7nMwj776sXOKsZ6lCQmixvUGm8KDgt7OrIIcwK X0UWxa6VXU+QNsP89c+sCuA65nIG3+bUeg3hUq8hMT3yxLm+NlnFOIpGc +CdFRS1CZT0ViaFLQrQno3jeGk2qyXLcWExAmaYn6RuRKWZSVKuQa0jU+ HlvCVI5L/zX3HLvZTrqg+03NktDNDkbDIAlz8gQznPMRw9ENXtmxoLi3G 0v5tDaT7rfH1FgtaEV8dniH4Z+qgCA2GISEYQqGansTLybV0lry7y3Nod A==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="413767129" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="413767129" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="816650518" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="816650518" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:48 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 306E4120A5B; Tue, 3 Oct 2023 14:52:45 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 05/28] media: v4l: Support line-based metadata capture Date: Tue, 3 Oct 2023 14:52:33 +0300 Message-Id: <20231003115237.76828-6-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org many camera sensors, among other devices, transmit embedded data and image data for each CSI-2 frame. This embedded data typically contains register configuration of the sensor that has been used to capture the image data of the same frame. The embedded data is received by the CSI-2 receiver and has the same properties as the image data, including that it is line based: it has width, height and bytesperline (stride). Add these fields to struct v4l2_meta_format and document them. Also add V4L2_FMT_FLAG_META_LINE_BASED to tell a given format is line-based i.e. these fields of struct v4l2_meta_format are valid for it. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-meta.rst | 15 +++++++++++++++ .../userspace-api/media/v4l/vidioc-enum-fmt.rst | 7 +++++++ .../media/videodev2.h.rst.exceptions | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 5 +++-- include/uapi/linux/videodev2.h | 10 ++++++++++ 5 files changed, 36 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/dev-meta.rst b/Documentation/userspace-api/media/v4l/dev-meta.rst index 0e7e1ee1471a..4dfd79e0a705 100644 --- a/Documentation/userspace-api/media/v4l/dev-meta.rst +++ b/Documentation/userspace-api/media/v4l/dev-meta.rst @@ -65,3 +65,18 @@ to 0. - ``buffersize`` - Maximum buffer size in bytes required for data. The value is set by the driver. + * - __u32 + - ``width`` + - Width of a line of metadata in Data units. Valid when + :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, + otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``height`` + - Number of rows of metadata. Valid when :c:type`v4l2_fmtdesc` flag + ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``bytesperline`` + - Offset in bytes between the beginning of two consecutive lines. Valid + when :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is + set, otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. diff --git a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst index 000c154b0f98..a79abf4428c8 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst @@ -227,6 +227,13 @@ the ``mbus_code`` field is handled differently: The application can ask to configure the quantization of the capture device when calling the :ref:`VIDIOC_S_FMT ` ioctl with :ref:`V4L2_PIX_FMT_FLAG_SET_CSC ` set. + * - ``V4L2_FMT_FLAG_META_LINE_BASED`` + - 0x0200 + - The metadata format is line-based. In this case the ``width``, + ``height`` and ``bytesperline`` fields of :c:type:`v4l2_meta_format` are + valid. The buffer consists of ``height`` lines, each having ``width`` + Data units of data and offset (in bytes) between the beginning of each + two consecutive lines is ``bytesperline``. Return Value ============ diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions index 3e58aac4ef0b..bdc628e8c1d6 100644 --- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions +++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions @@ -215,6 +215,7 @@ replace define V4L2_FMT_FLAG_CSC_XFER_FUNC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_YCBCR_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_HSV_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_QUANTIZATION fmtdesc-flags +replace define V4L2_FMT_FLAG_META_LINE_BASED fmtdesc-flags # V4L2 timecode types replace define V4L2_TC_TYPE_24FPS timecode-type diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index ce4b3929ff5f..fb453b7d0c91 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -343,8 +343,9 @@ static void v4l_print_format(const void *arg, bool write_only) case V4L2_BUF_TYPE_META_OUTPUT: meta = &p->fmt.meta; pixelformat = meta->dataformat; - pr_cont(", dataformat=%p4cc, buffersize=%u\n", - &pixelformat, meta->buffersize); + pr_cont(", dataformat=%p4cc, buffersize=%u, width=%u, height=%u, bytesperline=%u\n", + &pixelformat, meta->buffersize, meta->width, + meta->height, meta->bytesperline); break; } } diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 2b16b06ad278..7b0781a20dbe 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -877,6 +877,7 @@ struct v4l2_fmtdesc { #define V4L2_FMT_FLAG_CSC_YCBCR_ENC 0x0080 #define V4L2_FMT_FLAG_CSC_HSV_ENC V4L2_FMT_FLAG_CSC_YCBCR_ENC #define V4L2_FMT_FLAG_CSC_QUANTIZATION 0x0100 +#define V4L2_FMT_FLAG_META_LINE_BASED 0x0200 /* Frame Size and frame rate enumeration */ /* @@ -2420,10 +2421,19 @@ struct v4l2_sdr_format { * struct v4l2_meta_format - metadata format definition * @dataformat: little endian four character code (fourcc) * @buffersize: maximum size in bytes required for data + * @width: number of data units of data per line (valid for line + * based formats only, see format documentation) + * @height: number of lines of data per buffer (valid for line based + * formats only) + * @bytesperline: offset between the beginnings of two adjacent lines in + * bytes (valid for line based formats only) */ struct v4l2_meta_format { __u32 dataformat; __u32 buffersize; + __u32 width; + __u32 height; + __u32 bytesperline; } __attribute__ ((packed)); /** From patchwork Tue Oct 3 11:52:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7DCAE7544F for ; Tue, 3 Oct 2023 11:52:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240631AbjJCLw5 (ORCPT ); Tue, 3 Oct 2023 07:52:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240622AbjJCLw4 (ORCPT ); Tue, 3 Oct 2023 07:52:56 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3CC7A7 for ; Tue, 3 Oct 2023 04:52:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696333972; x=1727869972; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1eRhR3vdQbwE+zqQIbgaOvRBqmRRrtHraOJQuVIaw5A=; b=UYZpioXf/r9QDB01UC8T86RSj2upmPOYd+3aXFjjX5OnT2wRz/pe7b9A oMalZWbjpJOYWLDwxNdDlY8gRI8UNqdTstzJ9XeDSihgFtOQ1KX1/V9OV 92rsVb6tLyC5KuonQ46Z29iPhwIcR1fxOP2WHkFAHehD7yA/rBxuj7L2n hScQ12Da9DaM5tQ12yCJVFOxe701m0bfyJePOAimxxx9V8jWF0SoO0ZCk f4r8Z+YMMvaCWXTAU4xwZyvdusE5jkiQyUdVA1yVFWTHTDnT/EqDdAL8x qGgmglbfCyvXxPyCiYtbXRvdBt1YudX7nszwQHa4cobS+em9pY7g5pi/K g==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="413767136" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="413767136" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="816650525" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="816650525" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:50 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id A0CCD120797; Tue, 3 Oct 2023 14:52:46 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 06/28] media: uapi: ccs: Add media bus code for MIPI CCS embedded data Date: Tue, 3 Oct 2023 14:52:34 +0300 Message-Id: <20231003115237.76828-7-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add new MIPI CCS embedded data media bus code (MEDIA_BUS_FMT_CCS_EMBEDDED). Signed-off-by: Sakari Ailus --- .../media/v4l/subdev-formats.rst | 28 +++++++++++++++++++ include/uapi/linux/media-bus-format.h | 3 ++ 2 files changed, 31 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index c54bf834d839..70cad81bf862 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -8492,3 +8492,31 @@ and finally the bit number in subscript. "X" indicates a padding bit. - X - X - X + +.. _MEDIA-BUS-FMT-CCS-EMBEDDED: + +MIPI CCS Embedded Data Formats +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +`MIPI CCS `_ defines a +metadata format for sensor embedded data, which is used to store the register +configuration used for capturing a given frame. The format is defined in the CCS +specification. The media bus code for this format is +``MEDIA_BUS_FMT_CCS_EMBEDDED``. + +The CCS embedded data format definition includes three levels: + +1. Padding within CSI-2 bus :ref:`Data unit ` as + documented in the MIPI CCS specification. + +2. The tagged data format as documented in the MIPI CCS specification. + +3. Register addresses and register documentation as documented in the MIPI CCS + specification. + +The format definition shall be used only by devices that fulfill all three +levels above. + +This mbus code are only used for "2-byte simplified tagged data format" (code +0xa) but their use may be extended further in the future, to cover other CCS +embedded data format codes. diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index 9ee031397372..9805f8e73e48 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -182,4 +182,7 @@ #define MEDIA_BUS_FMT_META_20 0x8006 #define MEDIA_BUS_FMT_META_24 0x8007 +/* Specific metadata formats. Next is 0x9002. */ +#define MEDIA_BUS_FMT_CCS_EMBEDDED 0x9001 + #endif /* __LINUX_MEDIA_BUS_FORMAT_H */ From patchwork Tue Oct 3 11:52:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AF9EE75454 for ; Tue, 3 Oct 2023 11:52:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240622AbjJCLw5 (ORCPT ); Tue, 3 Oct 2023 07:52:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240630AbjJCLw5 (ORCPT ); Tue, 3 Oct 2023 07:52:57 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65415A3 for ; Tue, 3 Oct 2023 04:52:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696333974; x=1727869974; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=RudmQC+2iXiULLXvR1EEr7/PIesrdyhgrIStxBi1e1k=; b=Rg8J2LK7bpPrUlglARkNoqyo2uxBGhB8J2dyKY9P/pRDAZt2Izjg4/Nw L8srFiBUKGWtMYam7Vo/kwodGCHTYKXHe5JWCY91D8gmcXEZckXw8IhnQ YvJghPPDfrPBqnHO9EauXGpVIb64UeGE/0EmZfvxTMpNXu50vx8LP7f2K uHwtiElfg4TSUJF3bHPSfI3ovfinGvap3NIo0+6kSi+gxCoilPzoGmsHc CdANFYq7tRPNLM0p+Pp9tl0BhYZ0yteSVd6Ya+PhoVNwrO68aXINeF8BM pcxrE2ewaBEkfN3G88fx64vpt6GySo7Ps8fvh6bSRl/I1HhpPY8S4ymcg g==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="413767142" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="413767142" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="816650539" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="816650539" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:51 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 18684120A2E; Tue, 3 Oct 2023 14:52:48 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 07/28] media: Documentation: ccs: Document routing Date: Tue, 3 Oct 2023 14:52:35 +0300 Message-Id: <20231003115237.76828-8-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Document which routes are available for the CCS driver (source) sub-device and what configuration are possible. Also update copyright. Signed-off-by: Sakari Ailus --- .../userspace-api/media/drivers/ccs.rst | 34 ++++++++++++++++++- .../media/v4l/subdev-formats.rst | 2 ++ 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/drivers/ccs.rst b/Documentation/userspace-api/media/drivers/ccs.rst index 161cb65f4d98..ad8615233bae 100644 --- a/Documentation/userspace-api/media/drivers/ccs.rst +++ b/Documentation/userspace-api/media/drivers/ccs.rst @@ -107,4 +107,36 @@ than in the centre. Shading correction needs to be enabled for luminance correction level to have an effect. -**Copyright** |copy| 2020 Intel Corporation +.. _media-ccs-routes: + +Routes +------ + +The CCS driver implements one or two :ref:`routes ` in +its source sub-device (scaler sub-device if exists for the device, otherwise +binner) depending on whether the sensor supports embedded data. (All CCS +compliant sensors do but the CCS driver supports preceding standards that did +not require embedded data support, too.) + +The first route of the CCS source sub-device is for pixel data (internal pad +1/stream 0 -> pad 0/stream 0) and the second one is for embedded data (internal +pad 2/stream 0 -> pad 0/stream 1). + +Embedded data +~~~~~~~~~~~~~ + +MIPI CCS supports generation of camera sensor embedded data. The media bus code +used for this format is :ref:`MEDIA_BUS_FMT_CCS_EMBEDDDED +`. + +The bit depth of the CCS pixel data affects how the sensor will output the +embedded data, adding padding to align with CSI-2 bus :ref:`Data units +` for that particular bit depth. This is indicated by +the generic metadata format on the sensor's source sub-device's source pad. + +Embedded data for bit depths greater than or equal to 16 may support more dense +packing or legacy single metadata byte per data unit, or both of these, +depending on the device. The supported embedded data formats can be enumerated +and configured on stream 1 of the source pad (1) of the CCS source sub-device. + +**Copyright** |copy| 2020, 2023 Intel Corporation diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index 70cad81bf862..62396a47af4d 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -8520,3 +8520,5 @@ levels above. This mbus code are only used for "2-byte simplified tagged data format" (code 0xa) but their use may be extended further in the future, to cover other CCS embedded data format codes. + +Also see :ref:`CCS driver documentation `. From patchwork Tue Oct 3 11:52:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5F2CE75452 for ; Tue, 3 Oct 2023 11:52:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240638AbjJCLw7 (ORCPT ); Tue, 3 Oct 2023 07:52:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40788 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240630AbjJCLw6 (ORCPT ); Tue, 3 Oct 2023 07:52:58 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AABD8A3 for ; Tue, 3 Oct 2023 04:52:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696333975; x=1727869975; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NCBA2ZWrxrOSJHNzrnnv7zJUFHNyA34Wx+SZLYkoGpY=; b=RBWAVXlV9NnLgCFoGRUh3ATCYp8JKFB66SZgJD1woyoNBqd7+/d4ne3G NOoWFohg3OYatVVHU6PSx0OhkogOpGWW0QONDR3MRxzCut/Ec4o8DTkLU 2T7Doyu1pc0gFouXjjU617SUdBNjOpEw8qMajwYEqXeqba37ognvy/ujg RMonI2+VV8355FFaAuG4uorBGYXgKFobd8leO/DLFV9SvRydAM3JCswjj 1dTMjOwqudTt4cPJCZcglSLPe405EXMu2h9CDmhsKr8RiAdA60sOsszDC 7oefDP0R05feUzTeuEx54BXYFuEoMLu3TES0kqZHWnqOL5QZxjQZe9tT/ A==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="413767149" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="413767149" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="816650549" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="816650549" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:52 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 86775120A56; Tue, 3 Oct 2023 14:52:49 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 08/28] media: Documentation: Additional streams generally don't harm capture Date: Tue, 3 Oct 2023 14:52:36 +0300 Message-Id: <20231003115237.76828-9-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Having extra streams on the source end of the link that cannot be captured by the sink sub-device generally are not an issue, at least not on CSI-2 bus. Still document that there may be hardware specific limitations. For example on parallel bus this might not work on all cases. Signed-off-by: Sakari Ailus --- Documentation/userspace-api/media/v4l/dev-subdev.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index f375b820ab68..a387e8a15b8d 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -529,9 +529,9 @@ the its sink pad and allows to route them individually to one of its source pads. Subdevice drivers that support multiplexed streams are compatible with -non-multiplexed subdev drivers, but, of course, require a routing configuration -where the link between those two types of drivers contains only a single -stream. +non-multiplexed subdev drivers. However, if the driver at the sink end of a link +does not support streams, then only the stream 0 on source end may be +captured. There may be additional hardware specific limitations. Understanding streams ^^^^^^^^^^^^^^^^^^^^^ From patchwork Tue Oct 3 11:52:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B95EE75452 for ; Tue, 3 Oct 2023 11:53:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240659AbjJCLxC (ORCPT ); Tue, 3 Oct 2023 07:53:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40820 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240654AbjJCLxB (ORCPT ); Tue, 3 Oct 2023 07:53:01 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AEB8AA6 for ; Tue, 3 Oct 2023 04:52:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696333977; x=1727869977; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9HgH2x6KTlGIuYfK9xL0HwkOulI2QMkfpdkpQqRZHhQ=; b=hh5gqYAqAi9p1yrLfo8SwAob2m6m1/DcdBykJ3o4t0NqqkA6IF8AW6vT NIOsmcYlucywTcwQJhtkmfsJ0nkDYm9o8qQrOt8sjVvs/ZlfxhFDGrhrd FThnEdZGJS+T8xHqKyHmZXOwJvssqtmKzJ1M0bWJuSyw+yUnAC/Zstvh8 KIfQJ1y+cFXpujmqE6orPATQ3JyEHzaSFt90So02CRxYh2e0VxoORg5/n EETkBPEwcjf970YihdwJye8fCBsud6E9mt+z33ojCwg8T54OPS79zwkGQ GAQmDbCz2OhWWbp+HzhMmAI2soYhyQEVth5adS1syadEyFY/mPXZCU5vp Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="413767153" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="413767153" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="816650559" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="816650559" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 04:52:54 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id D8F22120A57; Tue, 3 Oct 2023 14:52:50 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 09/28] media: Documentation: Document embedded data guidelines for camera sensors Date: Tue, 3 Oct 2023 14:52:37 +0300 Message-Id: <20231003115237.76828-10-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Document how embedded data support should be implemented for camera sensors, and when and how CCS embedded data format should be referenced. Signed-off-by: Sakari Ailus --- .../media/drivers/camera-sensor.rst | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst index 919a50e8b9d9..308f391c5ca1 100644 --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst @@ -102,3 +102,31 @@ register programming sequences shall initialize the :ref:`V4L2_CID_HFLIP values programmed by the register sequences. The default values of these controls shall be 0 (disabled). Especially these controls shall not be inverted, independently of the sensor's mounting rotation. + +Embedded data +------------- + +Many sensors, mostly raw sensors, support embedded data which is used to convey +the sensor configuration for the captured frame back to the host. While CSI-2 is +the most common bus used by such sensors, embedded data is not entirely limited +to CSI-2 bus due to e.g. bridge devices. + +Embedded data support should use an internal source pad and route to the +external pad. If embedded data output can be disabled in hardware, it should be +possible to disable the embedded data route via ``VIDIOC_SUBDEV_S_ROUTING`` +IOCTL. + +In general, changing the embedded data format from the driver-configured values +is not supported. The height of the metadata is hardware specific and the width +is that (or less of that) of the image width, as configured on the pixel data +stream. + +CCS and non-CCS embedded data +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Embedded data which is compliant with CCS definitions shall use ``CCS embedded +data format ``. Device specific embedded data which +is compliant up to MIPI CCS embedded data levels 1 or 2 only shall refer to CCS +embedded data formats and document the level of conformance. The rest of the +device specific embedded data format shall be documented in the context of the +data format itself. From patchwork Tue Oct 3 12:07:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70174E75426 for ; Tue, 3 Oct 2023 12:08:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231944AbjJCMIY (ORCPT ); Tue, 3 Oct 2023 08:08:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231127AbjJCMIY (ORCPT ); Tue, 3 Oct 2023 08:08:24 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 040DAA6 for ; Tue, 3 Oct 2023 05:08:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334901; x=1727870901; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GwjsOAEJYrWWLIuAaHpO2bTZMa77lfX+gV/cof4TEO0=; b=bLl2i0KLxpK+tB8tOJw5KX/VkUgGBmmJ0Fow99uVUfsWVpSJFRO6FTpP q7v0ZiPTd6bpsG/uU5V2u30Wpa4mRQJoOZI+f6i1UOMY/OoO8V5yQzuE7 M8Y0289NRy6klyBVVHgthQyzzVH5bzzwiAwiWSHtuvhTvm6xpqhHRPVsW Ub1x7koF1iW0TFell7DYVmof1g84BU6dbPKsY6hSJo7/OGrn4ox/L28yi VqyGhBGN99Zr0uvlSC8ao7aLpncyaQBQA1vaWFB48uvm7TQHPxQWx33OU 1gR4Ncd1fe+5opQnvu544mlNlvchtVHV6cDCV3no/G2CyYv9qcP+2gV/J A==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681487" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681487" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285896" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285896" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:17 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 23B05120797; Tue, 3 Oct 2023 15:08:14 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 10/28] media: Documentation: v4l: Document source routes Date: Tue, 3 Oct 2023 15:07:55 +0300 Message-Id: <20231003120813.77726-1-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Document how internal pads are used on source routes. Use the IMX219 camera sensor as an example. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-subdev.rst | 179 ++++++++++++++++++ 1 file changed, 179 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index a387e8a15b8d..fb73a95401c3 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the sub-device and a (pad, stream) pair. For sub-devices that do not support multiplexed streams the 'stream' field is always 0. +.. _v4l2-subdev-source-routes: + +Internal pads and source routes +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Cases where a single sub-device source pad is traversed by multiple streams one +or more of which originate from within the sub-device itself are special as +there is no external sink pad for such routes. In those cases, the sources of +the internally generated streams are represented by internal sink pads, which +are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL ` +pad flag set. + +Internal pads have all the properties of an external pad, including formats and +selections. The format in this case is the source format of the stream. An +internal pad always has a single stream only (0). + +*Source routes* are routes from an internal sink pad to an external source +pad. In most cases source routes are not modifiable but they can be activated +and deactivated using the :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE +` flag, depending on driver capabilities. + Interaction between routes, streams, formats and selections ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -668,3 +689,161 @@ To configure this pipeline, the userspace must take the following steps: the configurations along the stream towards the receiver, using :ref:`VIDIOC_SUBDEV_S_FMT ` ioctls to configure each stream endpoint in each sub-device. + +Internal pads setup example +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A simple example of a multiplexed stream setup might be as follows: + +- A CCS camera sensor source sub-device, with one sink pad (0), one source pad + (1), an internal sink pad (2) that represents the source of embedded + data. There are two routes, one from the sink pad to the source, and another + from the internal sink pad to the source pad. The embedded data stream needs + to be enabled by activating the related route. The configuration of the rest + of the CCS sub-devices is omitted from this example. + +- Multiplexer bridge (Bridge). The bridge has one sink pad, connected to the + sensor (pad 0), and one source pad (pad 1), which outputs two streams. + +- Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0), + connected to the bridge, and two source pads (pads 1-2), going to the DMA + engine. The receiver demultiplexes the incoming streams to the source pads. + +- DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is + connected to a single source pad in the receiver. + +The sensor, the bridge and the receiver are modeled as V4L2 sub-devices, +exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are +modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes. + +To configure this pipeline, the userspace must take the following steps: + +1) Set up media links between entities: connect the sensors to the bridge, + bridge to the receiver, and the receiver to the DMA engines. This step does + not differ from normal non-multiplexed media controller setup. + +2) Configure routing + +.. flat-table:: Camera sensor + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 0/0 + - 1/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from the sink pad + * - 2/0 + - 1/1 + - **V4L2_SUBDEV_ROUTE_FL_ACTIVE** + - Metadata stream from the internal sink pad + +.. flat-table:: Bridge routing table + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 0/0 + - 1/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from camera sensor + * - 0/1 + - 1/1 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Metadata stream from camera sensor + +.. flat-table:: Receiver routing table + :header-rows: 1 + + * - Sink Pad/Stream + - Source Pad/Stream + - Routing Flags + - Comments + * - 0/0 + - 1/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Pixel data stream from camera sensor + * - 0/1 + - 2/0 + - V4L2_SUBDEV_ROUTE_FL_ACTIVE + - Metadata stream from camera sensor + +The options available in sensor's routing configuration are dictated by +hardware capabilities: typically camera sensors always produce image data +stream while the embedded data stream typically can be either enabled or +disabled. + +3) Configure formats and selections + + This example assumes that the formats are propagated from sink pad to the + source pad as-is. The tables contain fields of both struct v4l2_subdev_format + and struct v4l2_mbus_framefmt. + +.. flat-table:: Formats set on the sub-devices. Bold values are set, others are + static or propagated. + :header-rows: 1 + :fill-cells: + + * - Sub-device + - Pad/Stream + - Width + - Height + - Code + * - :rspan:`3` Camera sensor sub-device (IMX219) + - 1/0 + - 3296 + - 2480 + - MEDIA_BUS_FMT_SRGGB10 + * - 0/0 + - **3296** + - **2480** + - **MEDIA_BUS_FMT_SRGGB10** + * - 2/0 + - 3296 + - 2 + - MEDIA_BUS_FMT_IMX219_EMBEDDED + * - 1/1 + - 3296 + - 2 + - MEDIA_BUS_FMT_META_10 + * - :rspan:`3` Bridge + - 0/0 + - **3296** + - **2480** + - **MEDIA_BUS_FMT_SRGGB10** + * - 1/0 + - 3296 + - 2480 + - MEDIA_BUS_FMT_SRGGB10 + * - 0/1 + - **3296** + - **2** + - **MEDIA_BUS_FMT_META_10** + * - 1/1 + - 3296 + - 2 + - MEDIA_BUS_FMT_META_10 + * - :rspan:`3` Receiver + - 0/0 + - **3296** + - **2480** + - **MEDIA_BUS_FMT_SRGGB10** + * - 1/0 + - 3296 + - 2480 + - MEDIA_BUS_FMT_SRGGB10 + * - 0/1 + - **3296** + - **2** + - **MEDIA_BUS_FMT_META_10** + * - 2/0 + - 3296 + - 2 + - MEDIA_BUS_FMT_META_10 + +The embedded data format does not need to be configured as the format is +dictated by the pixel data format in this case. From patchwork Tue Oct 3 12:07:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED598E7544F for ; Tue, 3 Oct 2023 12:08:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231849AbjJCMIZ (ORCPT ); Tue, 3 Oct 2023 08:08:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40324 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231942AbjJCMIY (ORCPT ); Tue, 3 Oct 2023 08:08:24 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5570DAC for ; Tue, 3 Oct 2023 05:08:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334902; x=1727870902; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ksPIgIvx7MLM9V1Y4A3HIcSSQO26VJP2cm3CDBzi34o=; b=Cwmecp8VNnGNbRs/itmoyUi5412OXhmF7biMnzK4uGhvy121y63eizBu NxDjgFvjWQPgCxhwBVVz4mkuN7/4SLyp2+tvb5Sttg+UenRnZ1a5Oc/hT oemXEzDvobvR31CpDfmQsS+He24JFOg6+8E4gyt3LDMGr+88lR1rIMZVR 9WTz9XhwieiUSf4/9w5bOLkYHqVyfnpbUCpQTTx2f4TbYjFe7WhPhW+gK vo5DyZcZ9dKXUDGi4WQ/ZIJeTeuXE0RV1EGH6PskA0s814o76OX2pCfds QzhYHGDMUksKvLFAB9EjAU6isx5bBWAELDukcGDTTEuZU4sxlxBIsLNBc A==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681491" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681491" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285900" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285900" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:18 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 57F3A120A2E; Tue, 3 Oct 2023 15:08:15 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 11/28] media: Documentation: Document S_ROUTING behaviour Date: Tue, 3 Oct 2023 15:07:56 +0300 Message-Id: <20231003120813.77726-2-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Document S_ROUTING behaviour for different devices. Generally in devices that produce streams the streams are static and some can be enabled and disabled, whereas in devices that just transport them or write them to memory, more configurability is allowed. Document this. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-subdev.rst | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index fb73a95401c3..83993775237f 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -593,6 +593,27 @@ Any configurations of a stream within a pad, such as format or selections, are independent of similar configurations on other streams. This is subject to change in the future. +Device types and routing setup +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Different kinds of sub-devices have differing behaviour for route inactivation, +depending on the hardware. Devices generating the streams may often allow +enabling and disabling some of these or the configuration is fixed. If these +routes can be disabled, not declaring these routes (or declaring them without +``VIDIOC_SUBDEV_STREAM_FL_ACTIVE`` flag set) in ``VIDIOC_SUBDEV_S_ROUTING`` will +disable the routes while the sub-device driver retain the streams and their +configuration. The ``VIDIOC_SUBDEV_S_ROUTING`` will still return such routes +back to the user in the routes array, with the ``V4L2_SUBDEV_STREAM_FL_ACTIVE`` +flag unset. + +Devices transporting the streams almost always have more configurability with +respect to routing. Typically any route between the sub-device's sink and source +pads is possible, and multiple routes (usually up to certain limited number) may +be active simultaneously. For such devices, no routes are created by the driver +and user-created routes are fully replaced when ``VIDIOC_SUBDEV_S_ROUTING`` is +called on the sub-device. Such newly created routes have the device's default +configuration for format and selection rectangles. + Configuring streams ^^^^^^^^^^^^^^^^^^^ From patchwork Tue Oct 3 12:07:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67A27E75452 for ; Tue, 3 Oct 2023 12:08:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231982AbjJCMI0 (ORCPT ); Tue, 3 Oct 2023 08:08:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231942AbjJCMI0 (ORCPT ); Tue, 3 Oct 2023 08:08:26 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FFD8A6 for ; Tue, 3 Oct 2023 05:08:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334903; x=1727870903; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fKrqt/oVHxujyEDtQ/2VaWPFvNlfU4S6H9l/Pn0mQsY=; b=HfgpEeX4hEQTVhD4cNyYp4N1p1J8n+ODqGsJww1Eb+8dnSg4A98kts5s AAM2zyi0vRKgrJG7L5Wy+90YB+72QIGT99iMvA07t1h2FxzqBHdhuJwXa FxYB4t5JVpzkmt4+yC0GORPErSbxf8iQQxRUDF7+qOkyINsLRTEbW4JJ7 HBShDdJeYN8LUq1cYg4fzRFIrllIt3zJgDCm7uFbLEpSWXXOTERhLQrJZ 6mUJ4sR4j67skWCMepsIFvdPlIdsKtFD8wPEjaRmTwPICOClO8vjRmBLA Mzrp69xaDTdo/9Tw51aFlQ6kDj5JaRBLF43mejNrPHX1qJyJGm6KYmj65 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681498" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681498" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285903" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285903" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:20 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 97017120A56; Tue, 3 Oct 2023 15:08:16 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 12/28] media: v4l: subdev: Add helpers for format, crop and compose pointers Date: Tue, 3 Oct 2023 15:07:57 +0300 Message-Id: <20231003120813.77726-3-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add a helper for obtaining format, crop and compose pointers. These are convenient for drivers, independently of the driver uses streams or not. Signed-off-by: Sakari Ailus --- drivers/media/v4l2-core/v4l2-subdev.c | 54 ++++++++++++++++++++++---- include/media/v4l2-subdev.h | 56 +++++++++++++++++++++++++++ 2 files changed, 102 insertions(+), 8 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index d295a4e87b66..854f9d4db923 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -1573,19 +1573,57 @@ v4l2_subdev_init_stream_configs(struct v4l2_subdev_stream_configs *stream_config return 0; } +struct v4l2_mbus_framefmt +*v4l2_subdev_get_fmt_ptr(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, unsigned int pad, + unsigned int stream) +{ + if (sd->flags & V4L2_SUBDEV_FL_STREAMS) + return v4l2_subdev_state_get_stream_format(state, pad, stream); + + if (pad < sd->entity.num_pads && stream == 0) + return v4l2_subdev_get_pad_format(sd, state, pad); + + return NULL; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_fmt_ptr); + +struct v4l2_rect +*v4l2_subdev_get_crop_ptr(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, unsigned int pad, + unsigned int stream) +{ + if (sd->flags & V4L2_SUBDEV_FL_STREAMS) + return v4l2_subdev_state_get_stream_crop(state, pad, stream); + + if (pad < sd->entity.num_pads && stream == 0) + return v4l2_subdev_get_pad_crop(sd, state, pad); + + return NULL; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_crop_ptr); + +struct v4l2_rect +*v4l2_subdev_get_compose_ptr(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, unsigned int pad, + unsigned int stream) +{ + if (sd->flags & V4L2_SUBDEV_FL_STREAMS) + return v4l2_subdev_state_get_stream_compose(state, pad, stream); + + if (pad < sd->entity.num_pads && stream == 0) + return v4l2_subdev_get_pad_compose(sd, state, pad); + + return NULL; +} +EXPORT_SYMBOL_GPL(v4l2_subdev_get_compose_ptr); + int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state, struct v4l2_subdev_format *format) { struct v4l2_mbus_framefmt *fmt; - if (sd->flags & V4L2_SUBDEV_FL_STREAMS) - fmt = v4l2_subdev_state_get_stream_format(state, format->pad, - format->stream); - else if (format->pad < sd->entity.num_pads && format->stream == 0) - fmt = v4l2_subdev_get_pad_format(sd, state, format->pad); - else - fmt = NULL; - + fmt = v4l2_subdev_get_fmt_ptr(sd, state, format->pad, format->stream); if (!fmt) return -EINVAL; diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index 5f59ff0796b7..7c34243ffed9 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -1479,6 +1479,62 @@ v4l2_subdev_lock_and_get_active_state(struct v4l2_subdev *sd) #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API) +/** + * v4l2_subdev_get_fmt_ptr - Obtain a pointer to V4L2 sub-device format for pad + * and stream + * @sd: subdevice + * @state: subdevice state + * @pad: the pad on the sub-device + * @stream: stream in the pad + * + * For given pad and stream, obtain a pointer to the mbus format from the + * sub-device. + * + * Returns NULL if the format is not found or the parameters are invalid. + */ +struct v4l2_mbus_framefmt * +v4l2_subdev_get_fmt_ptr(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, unsigned int pad, + unsigned int stream); + +/** + * v4l2_subdev_get_crop_ptr - Obtain a pointer to V4L2 sub-device crop + * rectangle for pad and stream + * @sd: subdevice + * @state: subdevice state + * @pad: the pad on the sub-device + * @stream: stream in the pad + * + * For given pad and stream, obtain a pointer to the crop selection rectangle + * from the sub-device. + * + * Returns NULL if the selection rectangle is not found or the parameters are + * invalid. + */ +struct v4l2_rect * +v4l2_subdev_get_crop_ptr(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, unsigned int pad, + unsigned int stream); + +/** + * v4l2_subdev_get_compose_ptr - Obtain a pointer to V4L2 sub-device compose + * rectangle for pad and stream + * @sd: subdevice + * @state: subdevice state + * @pad: the pad on the sub-device + * @stream: stream in the pad + * + * For given pad and stream, obtain a pointer to the compose selection rectangle + * from the sub-device. + * + * Returns NULL if the selection rectangle is not found or the parameters are + * invalid. + */ +struct v4l2_rect * +v4l2_subdev_get_compose_ptr(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, unsigned int pad, + unsigned int stream); + /** * v4l2_subdev_get_fmt() - Fill format based on state * @sd: subdevice From patchwork Tue Oct 3 12:07:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729438 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8234BE75426 for ; Tue, 3 Oct 2023 12:08:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232080AbjJCMI1 (ORCPT ); Tue, 3 Oct 2023 08:08:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40364 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232060AbjJCMI1 (ORCPT ); Tue, 3 Oct 2023 08:08:27 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C2A0A9 for ; Tue, 3 Oct 2023 05:08:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334904; x=1727870904; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hFROCNkmsn1XKTVWSYkM1LA3nbVxvI0qIVVkLgkbOLY=; b=Hq3AS3CtECGSk7/7jJhNUKAPi4p8/WXDuKUdGPC8NoKWpKHcBWXxt/+W X9Ty7BXCSkF/k+KN26X7KcryVF/AHEG5612kuE+D13r3lgjjM+nG/Mkua 0Z8xMoeVWCggNhuNKVvmq+cxpjITbLvAdJN+etIbERrDJ4DoMq+2SQWKR ibR4jYu2jvDrYaF1vPJ52dI315ofrLl+KzhMPq/Ni+IaRvY6G/dZV5pJM ahNvXIP6Ya2CH6iuv2OOAuKD1KDKRxHgNbq82eMPXBaxiESPGYHeIFLHd sremQvtTvDW9d1KVxCUx7Mi3g9ixEwjv+vRDkbkmwLgC2nPkRDCizEQnj Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681504" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681504" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285906" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285906" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:21 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id F0BB3120A57; Tue, 3 Oct 2023 15:08:17 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 13/28] media: v4l: subdev: Add a function to lock two sub-device states, use it Date: Tue, 3 Oct 2023 15:07:58 +0300 Message-Id: <20231003120813.77726-4-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add two new functions, v4l2_subdev_lock_states() and v4l2_subdev_unclock_states(), to acquire and release the state of two sub-devices. They differ from calling v4l2_subdev_{un,}lock_state() so that if the two states share the same lock, the lock is acquired only once. Also use the new functions in v4l2_subdev_link_validate(). Signed-off-by: Sakari Ailus --- drivers/media/v4l2-core/v4l2-subdev.c | 12 +++----- include/media/v4l2-subdev.h | 40 +++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 8 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 854f9d4db923..df9a1ae65410 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -1377,17 +1377,13 @@ int v4l2_subdev_link_validate(struct media_link *link) states_locked = sink_state && source_state; - if (states_locked) { - v4l2_subdev_lock_state(sink_state); - v4l2_subdev_lock_state(source_state); - } + if (states_locked) + v4l2_subdev_lock_states(sink_state, source_state); ret = v4l2_subdev_link_validate_locked(link, states_locked); - if (states_locked) { - v4l2_subdev_unlock_state(sink_state); - v4l2_subdev_unlock_state(source_state); - } + if (states_locked) + v4l2_subdev_unlock_states(sink_state, source_state); return ret; } diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index 7c34243ffed9..e49e8af2fb52 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -1418,6 +1418,46 @@ static inline void v4l2_subdev_unlock_state(struct v4l2_subdev_state *state) mutex_unlock(state->lock); } +/** + * v4l2_subdev_lock_states - Lock two sub-device states + * @state1: One subdevice state + * @state2: The other subdevice state + * + * Locks the state of two sub-devices. + * + * The states must be unlocked with v4l2_subdev_unlock_states() after use. + * + * This differs from calling v4l2_subdev_lock_state() on both states so that if + * the states share the same lock, the lock is acquired only once (so no + * deadlock occurs). Note that it must be ensured the locks must always be + * acquired in the same order. + */ +static inline void v4l2_subdev_lock_states(struct v4l2_subdev_state *state1, + struct v4l2_subdev_state *state2) +{ + mutex_lock(state1->lock); + if (state1->lock != state2->lock) + mutex_lock(state2->lock); +} + +/** + * v4l2_subdev_unlock_states() - Unlock two sub-device states + * @state1: One subdevice state + * @state2: The other subdevice state + * + * Unlocks the state of two sub-devices. + * + * This differs from calling v4l2_subdev_unlock_state() on both states so that if + * the states share the same lock, the lock is released only once. + */ +static inline void v4l2_subdev_unlock_states(struct v4l2_subdev_state *state1, + struct v4l2_subdev_state *state2) +{ + mutex_unlock(state1->lock); + if (state1->lock != state2->lock) + mutex_unlock(state2->lock); +} + /** * v4l2_subdev_get_unlocked_active_state() - Checks that the active subdev state * is unlocked and returns it From patchwork Tue Oct 3 12:07:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAD3EE75453 for ; Tue, 3 Oct 2023 12:08:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232082AbjJCMIa (ORCPT ); Tue, 3 Oct 2023 08:08:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232110AbjJCMI3 (ORCPT ); Tue, 3 Oct 2023 08:08:29 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B5B7A9 for ; Tue, 3 Oct 2023 05:08:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334906; x=1727870906; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=35JYVf8F5uxK3Q/i4iuOMCiky5gw/qs2pQ50bIhjrXU=; b=DxEcSdpyJQj5U+1FtOSd785ogkE7iPZbfzWWZPzVC1Gb/5v7Ib4JFfO9 yy1yehNnHDQObBmV0w0kQ1TtunDR1VEhx+HI/gn73cheK1jXYIIkU6Bik L2V7fSCWKCG4GG3XAt2+rSR0GkhewaFTewS7NuBj6hBs3xPBOvBxeIvWN U7MxI/0kV7lpTIkQ5xT6i9IDB/kRISKSi6M4V0xtoDorj1ogQpDvCpNQI gVIoI8f0I+VwKPIgsdvRV1ci0vYnyQuYsU41Yb7isMgZqPRLsIuJZe9jW aCLwwJhbvUz9BG/NvQTyt9ryCZVEvEXwhP3tO3Ep0KS6XtgU7Ig+ahpKH w==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681513" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681513" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285910" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285910" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:23 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 6815C120A58; Tue, 3 Oct 2023 15:08:19 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 14/28] media: v4l: subdev: Move G_ROUTING handling below S_ROUTING Date: Tue, 3 Oct 2023 15:07:59 +0300 Message-Id: <20231003120813.77726-5-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Move G_ROUTING IOCTL handling below that of S_ROUTING. G_ROUTING implementation will soon needed in handling S_ROUTING as well. Signed-off-by: Sakari Ailus --- drivers/media/v4l2-core/v4l2-subdev.c | 54 +++++++++++++-------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index df9a1ae65410..614ff0031831 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -887,33 +887,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, case VIDIOC_SUBDEV_QUERYSTD: return v4l2_subdev_call(sd, video, querystd, arg); - case VIDIOC_SUBDEV_G_ROUTING: { - struct v4l2_subdev_routing *routing = arg; - struct v4l2_subdev_krouting *krouting; - - if (!v4l2_subdev_enable_streams_api) - return -ENOIOCTLCMD; - - if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) - return -ENOIOCTLCMD; - - memset(routing->reserved, 0, sizeof(routing->reserved)); - - krouting = &state->routing; - - if (routing->num_routes < krouting->num_routes) { - routing->num_routes = krouting->num_routes; - return -ENOSPC; - } - - memcpy((struct v4l2_subdev_route *)(uintptr_t)routing->routes, - krouting->routes, - krouting->num_routes * sizeof(*krouting->routes)); - routing->num_routes = krouting->num_routes; - - return 0; - } - case VIDIOC_SUBDEV_S_ROUTING: { struct v4l2_subdev_routing *routing = arg; struct v4l2_subdev_route *routes = @@ -962,6 +935,33 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, routing->which, &krouting); } + case VIDIOC_SUBDEV_G_ROUTING: { + struct v4l2_subdev_routing *routing = arg; + struct v4l2_subdev_krouting *krouting; + + if (!v4l2_subdev_enable_streams_api) + return -ENOIOCTLCMD; + + if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) + return -ENOIOCTLCMD; + + memset(routing->reserved, 0, sizeof(routing->reserved)); + + krouting = &state->routing; + + if (routing->num_routes < krouting->num_routes) { + routing->num_routes = krouting->num_routes; + return -ENOSPC; + } + + memcpy((struct v4l2_subdev_route *)(uintptr_t)routing->routes, + krouting->routes, + krouting->num_routes * sizeof(*krouting->routes)); + routing->num_routes = krouting->num_routes; + + return 0; + } + case VIDIOC_SUBDEV_G_CLIENT_CAP: { struct v4l2_subdev_client_capability *client_cap = arg; From patchwork Tue Oct 3 12:08:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E744E75426 for ; Tue, 3 Oct 2023 12:08:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232124AbjJCMIb (ORCPT ); Tue, 3 Oct 2023 08:08:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57840 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232110AbjJCMIb (ORCPT ); Tue, 3 Oct 2023 08:08:31 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB7BDA6 for ; Tue, 3 Oct 2023 05:08:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334907; x=1727870907; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kVOii1f/5s2PJ8KVti6PipXSxyHHcLq5+5oOMw3hR+A=; b=Jo7TfE5J7l5cmPG9aEedYhGcU2yze8SGRF5kDJ7+CC5pS1LnDafUWzAs Ww4W8zeBUenfmleeTu88RmicM6F8Bjq05Xx8D5qDajDmrBhDekTq1/p8p waOtp8kXRRs7u5MRPqrDPPDkI6Vg/+oGcY7kU0Ul37vHO0Ic6u21n09/Y JxFja5RS4jrtjYRvDkRh4JDn8Jo0ioUiXGRVxVGLQrevV8R4YV5a1hoF7 KA6a5UhjisI+Mkqp9X/A3KW6IBZiuUWjh7pZPptiRQptVTmwPc8PCrPRB YcVQ0MrpEcV6CrAkxN7aK6OVheZPmqxzSGo8g6irGbveleb4A+1GSbwNv Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681520" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681520" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285914" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285914" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:24 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 13C12120797; Tue, 3 Oct 2023 15:08:20 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 15/28] media: v4l: subdev: Copy argument back to user also for S_ROUTING Date: Tue, 3 Oct 2023 15:08:00 +0300 Message-Id: <20231003120813.77726-6-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org As the user needs to know what went wrong for S_ROUTING, copy array arguments back to the user. Signed-off-by: Sakari Ailus --- drivers/media/v4l2-core/v4l2-ioctl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index fb453b7d0c91..6921a72566df 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -3419,7 +3419,8 @@ video_usercopy(struct file *file, unsigned int orig_cmd, unsigned long arg, * in case of failure, but it is not defined here as part of the * 'v4l2_ioctls' array, insert an ad-hoc check to address that. */ - if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING) + if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING && + cmd != VIDIOC_SUBDEV_S_ROUTING) goto out; if (has_array_args) { From patchwork Tue Oct 3 12:08:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC766E75452 for ; Tue, 3 Oct 2023 12:08:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232133AbjJCMId (ORCPT ); Tue, 3 Oct 2023 08:08:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232110AbjJCMIc (ORCPT ); Tue, 3 Oct 2023 08:08:32 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53B63AC for ; Tue, 3 Oct 2023 05:08:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334909; x=1727870909; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rsiwEInmtlzxD+1LZV2zIKl2BC0GdQE2Tr/WKOaHPv4=; b=JB2QWTfxIUrtg+QK5iNuF9nk7MyLlDd2Yw8P/q1Q4mmYgxRWWBzyA1aj WKvykm4ZXmswYxu6NGDfe+/W4k/44WhJ/0ZRnH+gEr4ntR5DBKp07L3wK Md99WhT1QXML7xJ6UGzmVx9zWfkRGA630P7zQLwgF89Al2jTwc+YOQnez vA6AXq8jthd+uDyhpG4X3qp0QrTToJDALVE3QzrRMmFQajWfUU9KLLVJO z13HxugrdVco37uqJefr0v376l5QIF4u/Q5JbNZiVxwo6YpsiU2M5RsZ4 xbpS1yijvF1tECFD4B0ocXKP/v78FMe7Ucgul8aXYEukASxaGoC2Lgv9w Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681524" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681524" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285917" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285917" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:26 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id E7D68120A2E; Tue, 3 Oct 2023 15:08:22 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 16/28] media: v4l: subdev: Add len_routes field to struct v4l2_subdev_routing Date: Tue, 3 Oct 2023 15:08:01 +0300 Message-Id: <20231003120813.77726-7-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The len_routes field is used to tell the size of the routes array in struct v4l2_subdev_routing. This way the number of routes returned from S_ROUTING IOCTL may be larger than the number of routes provided, in case there are more routes returned by the driver. Note that this changes the (now-disabled) UAPI, users must be updated. Signed-off-by: Sakari Ailus --- .../media/v4l/vidioc-subdev-g-routing.rst | 31 ++++++++++++------- drivers/media/v4l2-core/v4l2-ioctl.c | 4 +-- drivers/media/v4l2-core/v4l2-subdev.c | 6 +++- include/media/v4l2-subdev.h | 2 ++ include/uapi/linux/v4l2-subdev.h | 8 +++-- 5 files changed, 34 insertions(+), 17 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst index 72677a280cd6..9a9765ddc316 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst @@ -46,20 +46,26 @@ with the ``VIDIOC_SUBDEV_S_ROUTING`` ioctl, by adding or removing routes and setting or clearing flags of the ``flags`` field of a struct :c:type:`v4l2_subdev_route`. -All stream configurations are reset when ``VIDIOC_SUBDEV_S_ROUTING`` is called. This -means that the userspace must reconfigure all streams after calling the ioctl -with e.g. ``VIDIOC_SUBDEV_S_FMT``. +All stream configurations are reset when ``VIDIOC_SUBDEV_S_ROUTING`` is +called. This means that the userspace must reconfigure all streams after calling +the ioctl with e.g. ``VIDIOC_SUBDEV_S_FMT``. Only subdevices which have both sink and source pads can support routing. -When inspecting routes through ``VIDIOC_SUBDEV_G_ROUTING`` and the application -provided ``num_routes`` is not big enough to contain all the available routes -the subdevice exposes, drivers return the ENOSPC error code and adjust the -value of the ``num_routes`` field. Application should then reserve enough memory -for all the route entries and call ``VIDIOC_SUBDEV_G_ROUTING`` again. +The ``num_routes`` field is used to denote the number of routes set (set by user +space on ``VIDIOC_SUBDEV_S_ROUTING`` argument) on the routing table as well as +the number of routes returned back from both IOCTLs. The ``len_routes`` +signifies the number of routes that can fit into the ``routes`` array. The +userspace shall set ``len_routes`` for both IOCTLs and ``num_routes`` for +``VIDIOC_SUBDEV_S_ROUTING``. -On a successful ``VIDIOC_SUBDEV_G_ROUTING`` call the driver updates the -``num_routes`` field to reflect the actual number of routes returned. +On a ``VIDIOC_SUBDEV_G_ROUTING`` call the driver updates the ``num_routes`` +field to reflect the actual number of routes known by the driver. ``num_routes`` +larger than ``len_routes`` in both IOCTLs. In this ``len_routes`` were returned +back to the userspace. This is not an error. + +Also ``VIDIOC_SUBDEV_S_ROUTING`` may return more route than the user provided in +``num_routes`` field due to e.g. hardware properties. .. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}| @@ -74,6 +80,9 @@ On a successful ``VIDIOC_SUBDEV_G_ROUTING`` call the driver updates the - ``which`` - Format to modified, from enum :ref:`v4l2_subdev_format_whence `. + * - __u32 + - ``len_routes`` + - The length of the array (as in memory reserved for the array) * - struct :c:type:`v4l2_subdev_route` - ``routes[]`` - Array of struct :c:type:`v4l2_subdev_route` entries @@ -81,7 +90,7 @@ On a successful ``VIDIOC_SUBDEV_G_ROUTING`` call the driver updates the - ``num_routes`` - Number of entries of the routes array * - __u32 - - ``reserved``\ [5] + - ``reserved``\ [11] - Reserved for future extensions. Applications and drivers must set the array to zero. diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 6921a72566df..1e3da9d64958 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -3155,13 +3155,13 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size, case VIDIOC_SUBDEV_S_ROUTING: { struct v4l2_subdev_routing *routing = parg; - if (routing->num_routes > 256) + if (routing->len_routes > 256) return -E2BIG; *user_ptr = u64_to_user_ptr(routing->routes); *kernel_ptr = (void **)&routing->routes; *array_size = sizeof(struct v4l2_subdev_route) - * routing->num_routes; + * routing->len_routes; ret = 1; break; } diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 614ff0031831..bd1e8205913c 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -903,6 +903,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, if (routing->which != V4L2_SUBDEV_FORMAT_TRY && ro_subdev) return -EPERM; + if (routing->num_routes > routing->len_routes) + return -EINVAL; + memset(routing->reserved, 0, sizeof(routing->reserved)); for (i = 0; i < routing->num_routes; ++i) { @@ -929,6 +932,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, } krouting.num_routes = routing->num_routes; + krouting.len_routes = routing->len_routes; krouting.routes = routes; return v4l2_subdev_call(sd, pad, set_routing, state, @@ -949,7 +953,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, krouting = &state->routing; - if (routing->num_routes < krouting->num_routes) { + if (routing->len_routes < krouting->num_routes) { routing->num_routes = krouting->num_routes; return -ENOSPC; } diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h index e49e8af2fb52..baaa81a9497e 100644 --- a/include/media/v4l2-subdev.h +++ b/include/media/v4l2-subdev.h @@ -741,12 +741,14 @@ struct v4l2_subdev_stream_configs { /** * struct v4l2_subdev_krouting - subdev routing table * + * @len_routes: length of routes array, in routes * @num_routes: number of routes * @routes: &struct v4l2_subdev_route * * This structure contains the routing table for a subdev. */ struct v4l2_subdev_krouting { + unsigned int len_routes; unsigned int num_routes; struct v4l2_subdev_route *routes; }; diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h index 4a195b68f28f..b57fb89caa9e 100644 --- a/include/uapi/linux/v4l2-subdev.h +++ b/include/uapi/linux/v4l2-subdev.h @@ -222,15 +222,17 @@ struct v4l2_subdev_route { * struct v4l2_subdev_routing - Subdev routing information * * @which: configuration type (from enum v4l2_subdev_format_whence) - * @num_routes: the total number of routes in the routes array + * @len_routes: the length of the routes array, in routes * @routes: pointer to the routes array + * @num_routes: the total number of routes in the routes array * @reserved: drivers and applications must zero this array */ struct v4l2_subdev_routing { __u32 which; - __u32 num_routes; + __u32 len_routes; __u64 routes; - __u32 reserved[6]; + __u32 num_routes; + __u32 reserved[11]; }; /* From patchwork Tue Oct 3 12:08:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729436 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89760E75453 for ; Tue, 3 Oct 2023 12:08:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232134AbjJCMIe (ORCPT ); Tue, 3 Oct 2023 08:08:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232136AbjJCMId (ORCPT ); Tue, 3 Oct 2023 08:08:33 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54CDCA6 for ; Tue, 3 Oct 2023 05:08:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334910; x=1727870910; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=M5kxrDp/+fX1OAgJYc03JU+o2Vmo5zfugbw5MfAby1s=; b=S2lHP/8HdOBj/VclwViKnG82UXTWHhP478hx2XrIJc/cdLFfojKEjlw0 1DymhqRJlV5jG2j/n0a+U+AHqzEkYaqv1U8qmWh+97o9cIKq2c88VoQjF tLXnJ5AkMcfUyA/r0oDxwo//NvrsuORk7a72AIuUZeY1lNQDDzDAqRuLR 29DqXEYw9f0jt/0TfB+qWBBHE2HBLffGbVUtu5NECS0GY0ajJo8vB/2Fi e1+njOTAUpUfWAxyLoxqL6iqBgtWw5d90oKewAuY+OBk/xywJv4+XFrMw 1PWOQYWfUgepTY7U0qTgEEitDuYXH4EJ6LjcjxTiK9LfzvrnGfC2nyhL3 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681529" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681529" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285921" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285921" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:27 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 3541C120A56; Tue, 3 Oct 2023 15:08:24 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 17/28] media: v4l: subdev: Return routes set using S_ROUTING Date: Tue, 3 Oct 2023 15:08:02 +0300 Message-Id: <20231003120813.77726-8-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Return the routes set using S_ROUTING back to the user. Also reflect this in documentation. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/vidioc-subdev-g-routing.rst | 5 +++-- drivers/media/v4l2-core/v4l2-subdev.c | 5 ++++- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst index 9a9765ddc316..ced53ea5f23c 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst @@ -43,8 +43,9 @@ The routing configuration determines the flows of data inside an entity. Drivers report their current routing tables using the ``VIDIOC_SUBDEV_G_ROUTING`` ioctl and application may enable or disable routes with the ``VIDIOC_SUBDEV_S_ROUTING`` ioctl, by adding or removing routes and -setting or clearing flags of the ``flags`` field of a -struct :c:type:`v4l2_subdev_route`. +setting or clearing flags of the ``flags`` field of a struct +:c:type:`v4l2_subdev_route`. Similarly to ``VIDIOC_SUBDEV_G_ROUTING``, also +``VIDIOC_SUBDEV_S_ROUTING`` returns the routes back to the user. All stream configurations are reset when ``VIDIOC_SUBDEV_S_ROUTING`` is called. This means that the userspace must reconfigure all streams after calling diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index bd1e8205913c..9a34e13dfd96 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -935,9 +935,12 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, krouting.len_routes = routing->len_routes; krouting.routes = routes; - return v4l2_subdev_call(sd, pad, set_routing, state, + rval = v4l2_subdev_call(sd, pad, set_routing, state, routing->which, &krouting); + if (rval < 0) + return rval; } + fallthrough; case VIDIOC_SUBDEV_G_ROUTING: { struct v4l2_subdev_routing *routing = arg; From patchwork Tue Oct 3 12:08:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729122 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B406E7544F for ; Tue, 3 Oct 2023 12:08:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232139AbjJCMIf (ORCPT ); Tue, 3 Oct 2023 08:08:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232136AbjJCMIf (ORCPT ); Tue, 3 Oct 2023 08:08:35 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E0BAA6 for ; Tue, 3 Oct 2023 05:08:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334912; x=1727870912; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4U3b8j9YsSgrHGAWBd9E2mlW5dgfuStmvj2uthNMFm4=; b=nJtDxSAdygQBGLkyLcuM3PVK1eilw7WQhnhsIa3vxTe0KH+ZCnTLfPVu Btv2/L52UPqv7CHdCDSiCAC7UORgsdCqkp0eE5MdrYtHHyQtao4I7sBJ5 QOaRYmwJwgJrVCA7U2xKNhm3bvzi9vnnAM8Ev4aIODJdJG0cKPV+XsPGh QvqY0AdI0HkEQAiC7KwS0Uu5doxRkxMGjF5LyGPprTDQl6auHg2eS9rYe Tosu1ULObDX7hQy9D5PTRXPlwe5FBo/ZpOfzT5KdMXghhkrbrTp5EigPl 2F/EeQnsbhxsCbityJR1kdx8rUFJE6qS0VP+NTSJQi5Ey5xpmK2qDhMu/ Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681535" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681535" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285924" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285924" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:29 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id AB1A2120A57; Tue, 3 Oct 2023 15:08:25 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 18/28] media: uapi: Allow a larger number of routes than there's room for Date: Tue, 3 Oct 2023 15:08:03 +0300 Message-Id: <20231003120813.77726-9-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org On VIDIOC_SUBDEV_[GS]_ROUTING, only return as many routes back to the user as there's room. Do not consider it an error if more routes existed. Simply inform the user there are more routes. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/vidioc-subdev-g-routing.rst | 4 ---- drivers/media/v4l2-core/v4l2-subdev.c | 8 ++------ 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst index ced53ea5f23c..99d3c15fd759 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst @@ -145,10 +145,6 @@ On success 0 is returned, on error -1 and the ``errno`` variable is set appropriately. The generic error codes are described at the :ref:`Generic Error Codes ` chapter. -ENOSPC - The application provided ``num_routes`` is not big enough to contain - all the available routes the subdevice exposes. - EINVAL The sink or source pad identifiers reference a non-existing pad, or reference pads of different types (ie. the sink_pad identifiers refers to a source pad). diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 9a34e13dfd96..dd48e7e549fb 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -956,14 +956,10 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, krouting = &state->routing; - if (routing->len_routes < krouting->num_routes) { - routing->num_routes = krouting->num_routes; - return -ENOSPC; - } - memcpy((struct v4l2_subdev_route *)(uintptr_t)routing->routes, krouting->routes, - krouting->num_routes * sizeof(*krouting->routes)); + min(krouting->num_routes, routing->len_routes) * + sizeof(*krouting->routes)); routing->num_routes = krouting->num_routes; return 0; From patchwork Tue Oct 3 12:08:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A313E75452 for ; Tue, 3 Oct 2023 12:08:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232163AbjJCMIh (ORCPT ); Tue, 3 Oct 2023 08:08:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57906 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232141AbjJCMIh (ORCPT ); Tue, 3 Oct 2023 08:08:37 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ADF84A6 for ; Tue, 3 Oct 2023 05:08:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334913; x=1727870913; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=arXNYG/ECUwMl5Pg60n2hCKRDtenMSoSi8QL+LfWOmg=; b=kVe9hV17hEIOVSnWFVLtd8dGklPAi2/MedwVNcYf8axn+Wlo5YTTSdgr 5XKfrnFYm4ou+pMcx9sir4bKnEk1Zl4VScqpQrdWJo3Ymq/2AEfkw+ROh qXLcy641rqVGY/qW/lynGDPX8hofnvIg60St0YfMsbd337Ux5+UU98Fqm jeSiC9yuQpyCcYA1RJv+q09z195+r8chzFgyO/zRWyeLEnSPUoIdN52/5 l6At6I/i/eaRrTCtlammPCM6wLyK1eDmCEAN6FRdpcMaz89bL1awLt0F/ YMUt1xhUcAOI/zRM58nyIL9K1TsQdx1Z1KHvIjCdqyNJyPpOPe7L+SdQi Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681541" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681541" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285928" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285928" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:30 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 70EE0120A58; Tue, 3 Oct 2023 15:08:27 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 19/28] media: v4l: subdev: Add trivial set_routing support Date: Tue, 3 Oct 2023 15:08:04 +0300 Message-Id: <20231003120813.77726-10-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add trivial S_ROUTING IOCTL support for drivers where routing is static. Essentially this means returning the same information G_ROUTING call would have done. Signed-off-by: Sakari Ailus --- drivers/media/v4l2-core/v4l2-subdev.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index dd48e7e549fb..7d7028de581a 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -894,6 +894,9 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, struct v4l2_subdev_krouting krouting = {}; unsigned int i; + if (!v4l2_subdev_has_op(sd, pad, set_routing)) + goto do_vidioc_subdev_g_routing; + if (!v4l2_subdev_enable_streams_api) return -ENOIOCTLCMD; @@ -939,6 +942,8 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, routing->which, &krouting); if (rval < 0) return rval; +do_vidioc_subdev_g_routing: + ; } fallthrough; From patchwork Tue Oct 3 12:08:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 147E8E75426 for ; Tue, 3 Oct 2023 12:08:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232168AbjJCMIi (ORCPT ); Tue, 3 Oct 2023 08:08:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232136AbjJCMIh (ORCPT ); Tue, 3 Oct 2023 08:08:37 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3125BB8 for ; Tue, 3 Oct 2023 05:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334915; x=1727870915; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JUyhVFl4xUivp4dSpuOIWT5iT7J/6k8bI34Dl5If6vo=; b=NYMph/daMJH5Z2yjpUneGzISE1yWCPW/6KAwWhUvBTJeAxAdCX7o7VNK 0goni80+Q63N3yBBQvPc2lu8KTN53r8Xb4KRjoesFDFvFWVyem7ypZAl9 A5py8dFSlpuSJDu2joTjT/eEqs1tm3BYpWIzoabwbG8mhDmxy8LPqlHrz Z0emGdHYfLDlJFf2NvgjxKD9htYyUdqH3D+NZbIAjSQx9rwfo1yg462mE +Nla+w6U7A0VPVNnUUP90nuirOHL1B3OQC5ba4fBs34dlawnB9OfeuZ62 04D+Z0vMnWoYZ5BcjL3U//ep5+mqu5CCnK4GzEJjmoeHTMrpPTqK4RXY7 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681546" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681546" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285930" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285930" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:32 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id DFC8C120797; Tue, 3 Oct 2023 15:08:28 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 20/28] media: uapi: v4l: subdev: Enable streams API Date: Tue, 3 Oct 2023 15:08:05 +0300 Message-Id: <20231003120813.77726-11-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Remove v4l2_subdev_enable_streams_api variable that was used to easily enable streams API for development, and conditions that use the variable. This patch enables the streams API for V4L2 sub-device interface which allows transporting multiple streams on a single MC link. Signed-off-by: Sakari Ailus --- drivers/media/v4l2-core/v4l2-subdev.c | 23 ----------------------- 1 file changed, 23 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 7d7028de581a..5e24a638bdba 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -26,15 +26,6 @@ #include #include -#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API) -/* - * The Streams API is an experimental feature. To use the Streams API, set - * 'v4l2_subdev_enable_streams_api' to 1 below. - */ - -static bool v4l2_subdev_enable_streams_api; -#endif - /* * Maximum stream ID is 63 for now, as we use u64 bitmask to represent a set * of streams. @@ -897,9 +888,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, if (!v4l2_subdev_has_op(sd, pad, set_routing)) goto do_vidioc_subdev_g_routing; - if (!v4l2_subdev_enable_streams_api) - return -ENOIOCTLCMD; - if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) return -ENOIOCTLCMD; @@ -951,9 +939,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, struct v4l2_subdev_routing *routing = arg; struct v4l2_subdev_krouting *krouting; - if (!v4l2_subdev_enable_streams_api) - return -ENOIOCTLCMD; - if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) return -ENOIOCTLCMD; @@ -981,14 +966,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg, case VIDIOC_SUBDEV_S_CLIENT_CAP: { struct v4l2_subdev_client_capability *client_cap = arg; - /* - * Clear V4L2_SUBDEV_CLIENT_CAP_STREAMS if streams API is not - * enabled. Remove this when streams API is no longer - * experimental. - */ - if (!v4l2_subdev_enable_streams_api) - client_cap->capabilities &= ~V4L2_SUBDEV_CLIENT_CAP_STREAMS; - /* Filter out unsupported capabilities */ client_cap->capabilities &= V4L2_SUBDEV_CLIENT_CAP_STREAMS; From patchwork Tue Oct 3 12:08:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729434 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BA30E75453 for ; Tue, 3 Oct 2023 12:08:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232173AbjJCMIj (ORCPT ); Tue, 3 Oct 2023 08:08:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232136AbjJCMIj (ORCPT ); Tue, 3 Oct 2023 08:08:39 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BF1FB0 for ; Tue, 3 Oct 2023 05:08:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334916; x=1727870916; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bF/BcJQaDDRuRcpdS4pVMfaIZJcjCXS5pYDwceWseeA=; b=O/ze4aQ0wojwcYTaswT3zCs/R7xZexJ0ExBmA/Nblo5B5uZ8nwgnI/Sr 4znwqZ3fWRiivUQx9frp1/gBZA0AT3i17LQsz10CSvdIIHT33Hr2MDIL2 4SwS1009MFJdGoFCvpZzOf1CCWL6rdPc30U3PkPcH3IiMM1NBzRmNKHVs kB3l7AknyZA24gM1DkmtUWx0/aprZJu4yKpxiXn4iaZBtvaP2pFSSORqv IXOrDfuc9IasSq1oxQGnWassRawfHlQNEScRyFtxTsdLYnasPW83/NRsk 6fBI7X/h6c3iZVUMSjQiIbU1K19rPMB0aJZcq7A3EHPAgKsGxaLzQM9LC Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681553" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681553" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285934" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285934" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:33 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 2916F120A2E; Tue, 3 Oct 2023 15:08:30 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 21/28] media: ccs: No need to set streaming to false in power off Date: Tue, 3 Oct 2023 15:08:06 +0300 Message-Id: <20231003120813.77726-12-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Streaming will have been stopped by the sensor is powered off, and so sensor->streaming is also false already. Do not set it as part of the runtime suspend callback. Signed-off-by: Sakari Ailus --- drivers/media/i2c/ccs/ccs-core.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c index 2abfd5932e02..0c518145b2c8 100644 --- a/drivers/media/i2c/ccs/ccs-core.c +++ b/drivers/media/i2c/ccs/ccs-core.c @@ -1700,7 +1700,6 @@ static int ccs_power_off(struct device *dev) usleep_range(5000, 5000); regulator_bulk_disable(ARRAY_SIZE(ccs_regulators), sensor->regulators); - sensor->streaming = false; return 0; } From patchwork Tue Oct 3 12:08:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40993E7544F for ; Tue, 3 Oct 2023 12:08:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232208AbjJCMIn (ORCPT ); Tue, 3 Oct 2023 08:08:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232136AbjJCMIl (ORCPT ); Tue, 3 Oct 2023 08:08:41 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3053BB3 for ; Tue, 3 Oct 2023 05:08:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334918; x=1727870918; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OaJpNe+tEK/yaoQ1BvTwGBi0fduXK+ePMAJgM7P+Aug=; b=kwQD5xb4aebGNvPV9UotidzorgP+lqGWzOo/gs7x+3hdZWfpqoBz1/1G 2ByOrBsLkGxmqtS8mCkXRwPGTYi3Vhk5jDoC3FCRzebNtQuQH1u8la+HR xuS2ezWyVmBe3t3F11f65n0rqa6nsoMlHVUt8QoCQv+Ys/nrKMr5K/o3e iTRaucquSe6OjtF2iZSEO0o/9uIqkqNmZ39o8cYFlnjheYx0jCiX4MyCL pOVUsz6FlTr3E7DxZa23Yxq4xObUrjcU5zoRol/1Rpp5Fo11hbw/2xUCz uDz47vwRXSAUvxFW4zj9J/dqwXjbjGJPZfjc2SktLOrWJEymhGSOJnC97 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681557" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681557" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285946" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285946" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:34 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 512BE120A56; Tue, 3 Oct 2023 15:08:31 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 22/28] media: ccs: Use {enable,disable}_streams operations Date: Tue, 3 Oct 2023 15:08:07 +0300 Message-Id: <20231003120813.77726-13-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Switch from s_stream() video op to enable_streams() and disable_streams() pad operations. They are preferred and required for streams support. Signed-off-by: Sakari Ailus --- drivers/media/i2c/ccs/ccs-core.c | 183 +++++++++++++++---------------- 1 file changed, 86 insertions(+), 97 deletions(-) diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c index 0c518145b2c8..70e7582bbf5f 100644 --- a/drivers/media/i2c/ccs/ccs-core.c +++ b/drivers/media/i2c/ccs/ccs-core.c @@ -1705,22 +1705,64 @@ static int ccs_power_off(struct device *dev) } /* ----------------------------------------------------------------------------- - * Video stream management + * V4L2 subdev video operations */ -static int ccs_start_streaming(struct ccs_sensor *sensor) +static int ccs_pm_get_init(struct ccs_sensor *sensor) { + struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd); + int rval; + + /* + * It can't use pm_runtime_resume_and_get() here, as the driver + * relies at the returned value to detect if the device was already + * active or not. + */ + rval = pm_runtime_get_sync(&client->dev); + if (rval < 0) + goto error; + + /* Device was already active, so don't set controls */ + if (rval == 1) + return 0; + + /* Restore V4L2 controls to the previously suspended device */ + rval = __v4l2_ctrl_handler_setup(&sensor->pixel_array->ctrl_handler); + if (rval) + goto error; + + rval = __v4l2_ctrl_handler_setup(&sensor->src->ctrl_handler); + if (rval) + goto error; + + /* Keep PM runtime usage_count incremented on success */ + return 0; +error: + pm_runtime_put(&client->dev); + return rval; +} + +static int ccs_enable_streams(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *state, u32 pad, + u64 streams_mask) +{ + struct ccs_sensor *sensor = to_ccs_sensor(subdev); struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd); unsigned int binning_mode; int rval; - mutex_lock(&sensor->mutex); + if (pad != CCS_PAD_SRC) + return -EINVAL; + + rval = ccs_pm_get_init(sensor); + if (rval) + return rval; rval = ccs_write(sensor, CSI_DATA_FORMAT, (sensor->csi_format->width << 8) | sensor->csi_format->compressed); if (rval) - goto out; + goto err_pm_put; /* Binning configuration */ if (sensor->binning_horizontal == 1 && @@ -1733,38 +1775,38 @@ static int ccs_start_streaming(struct ccs_sensor *sensor) rval = ccs_write(sensor, BINNING_TYPE, binning_type); if (rval < 0) - goto out; + goto err_pm_put; binning_mode = 1; } rval = ccs_write(sensor, BINNING_MODE, binning_mode); if (rval < 0) - goto out; + goto err_pm_put; /* Set up PLL */ rval = ccs_pll_configure(sensor); if (rval) - goto out; + goto err_pm_put; /* Analog crop start coordinates */ rval = ccs_write(sensor, X_ADDR_START, sensor->pa_src.left); if (rval < 0) - goto out; + goto err_pm_put; rval = ccs_write(sensor, Y_ADDR_START, sensor->pa_src.top); if (rval < 0) - goto out; + goto err_pm_put; /* Analog crop end coordinates */ rval = ccs_write(sensor, X_ADDR_END, sensor->pa_src.left + sensor->pa_src.width - 1); if (rval < 0) - goto out; + goto err_pm_put; rval = ccs_write(sensor, Y_ADDR_END, sensor->pa_src.top + sensor->pa_src.height - 1); if (rval < 0) - goto out; + goto err_pm_put; /* * Output from pixel array, including blanking, is set using @@ -1777,22 +1819,22 @@ static int ccs_start_streaming(struct ccs_sensor *sensor) rval = ccs_write(sensor, DIGITAL_CROP_X_OFFSET, sensor->scaler_sink.left); if (rval < 0) - goto out; + goto err_pm_put; rval = ccs_write(sensor, DIGITAL_CROP_Y_OFFSET, sensor->scaler_sink.top); if (rval < 0) - goto out; + goto err_pm_put; rval = ccs_write(sensor, DIGITAL_CROP_IMAGE_WIDTH, sensor->scaler_sink.width); if (rval < 0) - goto out; + goto err_pm_put; rval = ccs_write(sensor, DIGITAL_CROP_IMAGE_HEIGHT, sensor->scaler_sink.height); if (rval < 0) - goto out; + goto err_pm_put; } /* Scaling */ @@ -1800,20 +1842,20 @@ static int ccs_start_streaming(struct ccs_sensor *sensor) != CCS_SCALING_CAPABILITY_NONE) { rval = ccs_write(sensor, SCALING_MODE, sensor->scaling_mode); if (rval < 0) - goto out; + goto err_pm_put; rval = ccs_write(sensor, SCALE_M, sensor->scale_m); if (rval < 0) - goto out; + goto err_pm_put; } /* Output size from sensor */ rval = ccs_write(sensor, X_OUTPUT_SIZE, sensor->src_src.width); if (rval < 0) - goto out; + goto err_pm_put; rval = ccs_write(sensor, Y_OUTPUT_SIZE, sensor->src_src.height); if (rval < 0) - goto out; + goto err_pm_put; if (CCS_LIM(sensor, FLASH_MODE_CAPABILITY) & (CCS_FLASH_MODE_CAPABILITY_SINGLE_STROBE | @@ -1822,109 +1864,52 @@ static int ccs_start_streaming(struct ccs_sensor *sensor) sensor->hwcfg.strobe_setup->trigger != 0) { rval = ccs_setup_flash_strobe(sensor); if (rval) - goto out; + goto err_pm_put; } rval = ccs_call_quirk(sensor, pre_streamon); if (rval) { dev_err(&client->dev, "pre_streamon quirks failed\n"); - goto out; + goto err_pm_put; } rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_STREAMING); -out: - mutex_unlock(&sensor->mutex); - - return rval; -} - -static int ccs_stop_streaming(struct ccs_sensor *sensor) -{ - struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd); - int rval; - - mutex_lock(&sensor->mutex); - rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_SOFTWARE_STANDBY); - if (rval) - goto out; - - rval = ccs_call_quirk(sensor, post_streamoff); - if (rval) - dev_err(&client->dev, "post_streamoff quirks failed\n"); - -out: - mutex_unlock(&sensor->mutex); - return rval; -} - -/* ----------------------------------------------------------------------------- - * V4L2 subdev video operations - */ - -static int ccs_pm_get_init(struct ccs_sensor *sensor) -{ - struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd); - int rval; - - /* - * It can't use pm_runtime_resume_and_get() here, as the driver - * relies at the returned value to detect if the device was already - * active or not. - */ - rval = pm_runtime_get_sync(&client->dev); - if (rval < 0) - goto error; - - /* Device was already active, so don't set controls */ - if (rval == 1) - return 0; + sensor->streaming = true; - /* Restore V4L2 controls to the previously suspended device */ - rval = v4l2_ctrl_handler_setup(&sensor->pixel_array->ctrl_handler); - if (rval) - goto error; + return 0; - rval = v4l2_ctrl_handler_setup(&sensor->src->ctrl_handler); - if (rval) - goto error; +err_pm_put: + pm_runtime_mark_last_busy(&client->dev); + pm_runtime_put_autosuspend(&client->dev); - /* Keep PM runtime usage_count incremented on success */ - return 0; -error: - pm_runtime_put(&client->dev); return rval; } -static int ccs_set_stream(struct v4l2_subdev *subdev, int enable) +static int ccs_disable_streams(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *state, u32 pad, + u64 streams_mask) { struct ccs_sensor *sensor = to_ccs_sensor(subdev); struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd); int rval; - if (!enable) { - ccs_stop_streaming(sensor); - sensor->streaming = false; - pm_runtime_mark_last_busy(&client->dev); - pm_runtime_put_autosuspend(&client->dev); - - return 0; - } + if (pad != CCS_PAD_SRC) + return -EINVAL; - rval = ccs_pm_get_init(sensor); + rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_SOFTWARE_STANDBY); if (rval) return rval; - sensor->streaming = true; + rval = ccs_call_quirk(sensor, post_streamoff); + if (rval) + dev_err(&client->dev, "post_streamoff quirks failed\n"); - rval = ccs_start_streaming(sensor); - if (rval < 0) { - sensor->streaming = false; - pm_runtime_mark_last_busy(&client->dev); - pm_runtime_put_autosuspend(&client->dev); - } + sensor->streaming = false; + pm_runtime_mark_last_busy(&client->dev); + pm_runtime_put_autosuspend(&client->dev); - return rval; + return 0; } static int ccs_pre_streamon(struct v4l2_subdev *subdev, u32 flags) @@ -1950,7 +1935,9 @@ static int ccs_pre_streamon(struct v4l2_subdev *subdev, u32 flags) } } + mutex_lock(&sensor->mutex); rval = ccs_pm_get_init(sensor); + mutex_unlock(&sensor->mutex); if (rval) return rval; @@ -3033,7 +3020,7 @@ static int ccs_init_cfg(struct v4l2_subdev *sd, } static const struct v4l2_subdev_video_ops ccs_video_ops = { - .s_stream = ccs_set_stream, + .s_stream = v4l2_subdev_s_stream_helper, .pre_streamon = ccs_pre_streamon, .post_streamoff = ccs_post_streamoff, }; @@ -3045,6 +3032,8 @@ static const struct v4l2_subdev_pad_ops ccs_pad_ops = { .set_fmt = ccs_set_format, .get_selection = ccs_get_selection, .set_selection = ccs_set_selection, + .enable_streams = ccs_enable_streams, + .disable_streams = ccs_disable_streams, }; static const struct v4l2_subdev_sensor_ops ccs_sensor_ops = { From patchwork Tue Oct 3 12:08:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADE04E75426 for ; Tue, 3 Oct 2023 12:08:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232198AbjJCMIo (ORCPT ); Tue, 3 Oct 2023 08:08:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232193AbjJCMIm (ORCPT ); Tue, 3 Oct 2023 08:08:42 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 038F1BB for ; Tue, 3 Oct 2023 05:08:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334919; x=1727870919; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rTofabdLgGNT+FGxr0MzNByKxARc+ID4rVgkhnh3neM=; b=e2sUR93cfVnIGDO5UdKwCJNmd/rL6aM9da+BKhdRnaBbfR0KByPPzLqF zCeL9Wklk9ThQd9HCgoU/ZRLzBXEGv61d3htlhntenQF8IBOIzIVrjwgT r/20yC6r8+acz7OokKjq3jgzzdVNTcOINz/8V59VmaEvmsQZTOPwq9Pb0 PmLYije3PB+GBdQ9rxpIymsAf4f7yb61KV+cxyIznoVhldfL3lRXVuMes d7LbDOpcGhRLYTbWowYHSx+wbEJSp4it3FDOa2CiOHQNCwcd8Ma9jjKX0 FOSU8xJlbONFMdOg6xNT9V/Jkm/4HeSsGciL5ms36XEyu91TECSxLm80B w==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681562" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681562" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285949" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285949" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:35 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id A6E91120A57; Tue, 3 Oct 2023 15:08:32 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 23/28] media: ccs: Track streaming state Date: Tue, 3 Oct 2023 15:08:08 +0300 Message-Id: <20231003120813.77726-14-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org With enable_streams and disable_streams, the driver for a device where streams are not independently started and stopped needs to maintain state information on streams that have been requested to be started. Do that now. In the future, a helper function in the framework is a desirable way to do this instead. Signed-off-by: Sakari Ailus --- drivers/media/i2c/ccs/ccs-core.c | 13 ++++++++++--- drivers/media/i2c/ccs/ccs.h | 2 +- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c index 70e7582bbf5f..27ecf8688658 100644 --- a/drivers/media/i2c/ccs/ccs-core.c +++ b/drivers/media/i2c/ccs/ccs-core.c @@ -1754,6 +1754,11 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, if (pad != CCS_PAD_SRC) return -EINVAL; + if (sensor->streaming) { + sensor->streaming |= streams_mask; + return 0; + } + rval = ccs_pm_get_init(sensor); if (rval) return rval; @@ -1875,7 +1880,7 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_STREAMING); - sensor->streaming = true; + sensor->streaming |= streams_mask; return 0; @@ -1897,6 +1902,10 @@ static int ccs_disable_streams(struct v4l2_subdev *subdev, if (pad != CCS_PAD_SRC) return -EINVAL; + sensor->streaming &= ~streams_mask; + if (sensor->streaming) + return 0; + rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_SOFTWARE_STANDBY); if (rval) return rval; @@ -1905,7 +1914,6 @@ static int ccs_disable_streams(struct v4l2_subdev *subdev, if (rval) dev_err(&client->dev, "post_streamoff quirks failed\n"); - sensor->streaming = false; pm_runtime_mark_last_busy(&client->dev); pm_runtime_put_autosuspend(&client->dev); @@ -3500,7 +3508,6 @@ static int ccs_probe(struct i2c_client *client) goto out_cleanup; } - sensor->streaming = false; sensor->dev_init_done = true; rval = ccs_write_msr_regs(sensor); diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h index 9c3587b2fbe7..043a679e115c 100644 --- a/drivers/media/i2c/ccs/ccs.h +++ b/drivers/media/i2c/ccs/ccs.h @@ -234,7 +234,7 @@ struct ccs_sensor { u16 image_start; /* image data start line */ u16 visible_pixel_start; /* start pixel of the visible image */ - bool streaming; + u8 streaming; bool dev_init_done; u8 compressed_min_bpp; From patchwork Tue Oct 3 12:08:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729432 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA83AE75452 for ; Tue, 3 Oct 2023 12:08:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232206AbjJCMIo (ORCPT ); Tue, 3 Oct 2023 08:08:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232197AbjJCMIm (ORCPT ); Tue, 3 Oct 2023 08:08:42 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1258FB3 for ; Tue, 3 Oct 2023 05:08:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334920; x=1727870920; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WF7R6WCcF16DzqQ4oKIzXua2fsuAALVhBA+QmOF5vqA=; b=lZWdhLRcAjiqGrM8CRElMoTWD4nhmvQmTine61FWSiwnoyo3rSOU+MfR 9Kpe7a2k8sCFkwNrZkPc0EKjAhigGyKKtLf1SQXbe11u9KJHYlUpqiDAc LXn75N2yXPay0DAY9weAPZzVO12SWTegXMU2P4go9qLm2R/sr/mZ88nqR i6QanGqUhlISjyQqcY0DRK6v0w4iUywqZBuBDLjoMKf/ruD39Pcg6qoWj t7uVU0FL5wBp4wW4guaoaXe0SXw9FYqnyANjKaIqSYcDBsdiZ72U0Sz9W x1fmGzN3SPJk94zdk22zMkOZVRo6EWLB8ydFbZlY7Jd0XiH2J4L+ygVQy g==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681567" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681567" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285951" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285951" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:37 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id E428A120A58; Tue, 3 Oct 2023 15:08:33 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 24/28] media: ccs: Move ccs_validate_csi_data_format up Date: Tue, 3 Oct 2023 15:08:09 +0300 Message-Id: <20231003120813.77726-15-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org ccs_validate_csi_data_format() will soon be needed elsewhere, above its current location. Move it up. Signed-off-by: Sakari Ailus --- drivers/media/i2c/ccs/ccs-core.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c index 27ecf8688658..4924ec460d81 100644 --- a/drivers/media/i2c/ccs/ccs-core.c +++ b/drivers/media/i2c/ccs/ccs-core.c @@ -1967,6 +1967,20 @@ static int ccs_post_streamoff(struct v4l2_subdev *subdev) return pm_runtime_put(&client->dev); } +static const struct ccs_csi_data_format +*ccs_validate_csi_data_format(struct ccs_sensor *sensor, u32 code) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(ccs_csi_data_formats); i++) { + if (sensor->mbus_frame_fmts & (1 << i) && + ccs_csi_data_formats[i].code == code) + return &ccs_csi_data_formats[i]; + } + + return sensor->csi_format; +} + static int ccs_enum_mbus_code(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, struct v4l2_subdev_mbus_code_enum *code) @@ -2098,20 +2112,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev, } } -static const struct ccs_csi_data_format -*ccs_validate_csi_data_format(struct ccs_sensor *sensor, u32 code) -{ - unsigned int i; - - for (i = 0; i < ARRAY_SIZE(ccs_csi_data_formats); i++) { - if (sensor->mbus_frame_fmts & (1 << i) && - ccs_csi_data_formats[i].code == code) - return &ccs_csi_data_formats[i]; - } - - return sensor->csi_format; -} - static int ccs_set_format_source(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, struct v4l2_subdev_format *fmt) From patchwork Tue Oct 3 12:08:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE76AE75453 for ; Tue, 3 Oct 2023 12:08:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232197AbjJCMIq (ORCPT ); Tue, 3 Oct 2023 08:08:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232177AbjJCMIo (ORCPT ); Tue, 3 Oct 2023 08:08:44 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BFAABB8 for ; Tue, 3 Oct 2023 05:08:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334921; x=1727870921; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ao5c1JwIhBY/yJ0XvNMhgn73MFLT3TaB3IaF8xIA95E=; b=TddZHBgKk2MWjIaelQ2F6muG8e2Z/ZyKyjG7dlORimbWk6sfx/I++kDK F6UV1A44e96Fiuu6M5Y4AMzE0Z93woVad6+BSojxMzbgUVaMfqfNdpwYK s3QHRkrS8wDRQK/FehlP/x585Y7JRTttJdxVMGDp+5P324Sr2cn2uF6C3 eBvthHxp44CLFaMODTcFa6DtSJwohs12V719QezvhqZqb+Usw0ungANMb t4+SNpxL+7EVdF4KZqT8VsEzpgv1LKo694UTUNpiWg4oVoCa+Qn0ZWRk6 2xor1SebQLez5mJJxg56SxTCtOOkMVjEcGR74p1Sqfd50UJtyVvcWd90d Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681578" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681578" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285956" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285956" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:38 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 30D58120797; Tue, 3 Oct 2023 15:08:35 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 25/28] media: ccs: Support frame descriptors Date: Tue, 3 Oct 2023 15:08:10 +0300 Message-Id: <20231003120813.77726-16-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Provide information on the frame layout using frame descriptors. Signed-off-by: Sakari Ailus --- drivers/media/i2c/ccs/ccs-core.c | 60 +++++++++++++++++++++++++++++++ drivers/media/i2c/ccs/ccs-quirk.h | 7 ++++ drivers/media/i2c/ccs/ccs.h | 4 +++ 3 files changed, 71 insertions(+) diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c index 4924ec460d81..75abb18ae847 100644 --- a/drivers/media/i2c/ccs/ccs-core.c +++ b/drivers/media/i2c/ccs/ccs-core.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -235,6 +236,33 @@ static int ccs_read_all_limits(struct ccs_sensor *sensor) return ret; } +static u8 ccs_mipi_csi2_data_type(unsigned int bpp) +{ + switch (bpp) { + case 6: + return MIPI_CSI2_DT_RAW6; + case 7: + return MIPI_CSI2_DT_RAW7; + case 8: + return MIPI_CSI2_DT_RAW8; + case 10: + return MIPI_CSI2_DT_RAW10; + case 12: + return MIPI_CSI2_DT_RAW12; + case 14: + return MIPI_CSI2_DT_RAW14; + case 16: + return MIPI_CSI2_DT_RAW16; + case 20: + return MIPI_CSI2_DT_RAW20; + case 24: + return MIPI_CSI2_DT_RAW24; + default: + WARN_ON(1); + return 0; + } +} + static int ccs_read_frame_fmt(struct ccs_sensor *sensor) { struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd); @@ -2616,6 +2644,37 @@ static int ccs_set_selection(struct v4l2_subdev *subdev, return ret; } +static int ccs_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad, + struct v4l2_mbus_frame_desc *desc) +{ + struct ccs_sensor *sensor = to_ccs_sensor(subdev); + struct v4l2_mbus_frame_desc_entry *entry = desc->entry; + + if (ccs_has_quirk(sensor, frame_desc)) + return ccs_call_quirk(sensor, frame_desc, desc); + + switch (sensor->hwcfg.csi_signalling_mode) { + case CCS_CSI_SIGNALING_MODE_CSI_2_DPHY: + case CCS_CSI_SIGNALING_MODE_CSI_2_CPHY: + desc->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2; + break; + default: + /* FIXME: CCP2 support */ + return -EINVAL; + } + + entry->pixelcode = sensor->csi_format->code; + entry->stream = CCS_STREAM_PIXEL; + entry->bus.csi2.dt = + sensor->csi_format->width == sensor->csi_format->compressed ? + ccs_mipi_csi2_data_type(sensor->csi_format->compressed) : + CCS_DEFAULT_COMPRESSED_DT; + entry++; + desc->num_entries++; + + return 0; +} + static int ccs_get_skip_frames(struct v4l2_subdev *subdev, u32 *frames) { struct ccs_sensor *sensor = to_ccs_sensor(subdev); @@ -3042,6 +3101,7 @@ static const struct v4l2_subdev_pad_ops ccs_pad_ops = { .set_selection = ccs_set_selection, .enable_streams = ccs_enable_streams, .disable_streams = ccs_disable_streams, + .get_frame_desc = ccs_get_frame_desc, }; static const struct v4l2_subdev_sensor_ops ccs_sensor_ops = { diff --git a/drivers/media/i2c/ccs/ccs-quirk.h b/drivers/media/i2c/ccs/ccs-quirk.h index 0b1a64958d71..13f334fe0ac8 100644 --- a/drivers/media/i2c/ccs/ccs-quirk.h +++ b/drivers/media/i2c/ccs/ccs-quirk.h @@ -36,6 +36,7 @@ struct ccs_sensor; * access may be done by the caller (default read * value is zero), else negative error code on error * @flags: Quirk flags + * @frame_desc: Obtain the frame descriptor */ struct ccs_quirk { int (*limits)(struct ccs_sensor *sensor); @@ -46,6 +47,8 @@ struct ccs_quirk { int (*init)(struct ccs_sensor *sensor); int (*reg_access)(struct ccs_sensor *sensor, bool write, u32 *reg, u32 *val); + int (*frame_desc)(struct ccs_sensor *sensor, + struct v4l2_mbus_frame_desc *desc); unsigned long flags; }; @@ -62,6 +65,10 @@ struct ccs_reg_8 { .val = _val, \ } +#define ccs_has_quirk(sensor, _quirk) \ + ((sensor)->minfo.quirk && \ + (sensor)->minfo.quirk->_quirk) + #define ccs_call_quirk(sensor, _quirk, ...) \ ((sensor)->minfo.quirk && \ (sensor)->minfo.quirk->_quirk ? \ diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h index 043a679e115c..7fd9b6493d2b 100644 --- a/drivers/media/i2c/ccs/ccs.h +++ b/drivers/media/i2c/ccs/ccs.h @@ -45,6 +45,8 @@ #define CCS_COLOUR_COMPONENTS 4 +#define CCS_DEFAULT_COMPRESSED_DT 0x30 + #define SMIAPP_NAME "smiapp" #define CCS_NAME "ccs" @@ -174,6 +176,8 @@ struct ccs_csi_data_format { #define CCS_PAD_SRC 1 #define CCS_PADS 2 +#define CCS_STREAM_PIXEL 0 + struct ccs_binning_subtype { u8 horizontal:4; u8 vertical:4; From patchwork Tue Oct 3 12:08:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EBA2E75452 for ; Tue, 3 Oct 2023 12:08:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232228AbjJCMIu (ORCPT ); Tue, 3 Oct 2023 08:08:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39646 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232243AbjJCMIt (ORCPT ); Tue, 3 Oct 2023 08:08:49 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CE27B0 for ; Tue, 3 Oct 2023 05:08:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334923; x=1727870923; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3/VxT8OKT9DNjLoMAcpb0qFZW8B1gySmyxu9FMXRgpc=; b=BEH7+keylxVKC4JMBjpC6BkBjMSoG6KrhiV8lkVAyEyJXQcpFzItNm6K cJttoVna3AReXe7PHSBkTXIZ9koR3sopa8soY/Nzo6/+7eOv7GIPd7TUG BiO5WLG/ZvfwIv1EcsD7PlZs8OYCIBGE4zDMwDZ7e+HfjKTTAV7AHQXIe Fb/GgcvfJ4bESmiPVLACbFKQGPH2HNjpbrs7M6P2d94R1yWeuw5PI+pwl CEL0zOtizP344Squ2YFYP+Iuj/K+CBbRih6TZpkpjromwJqGrcRndpP8T d8P8OS470ZKxbsjAKPHc11vEGJJ4P1aTbflo9fLEf3I5Uu8vzxc5TikqN Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681584" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681584" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285966" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285966" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:39 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 6FA13120A2E; Tue, 3 Oct 2023 15:08:36 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 26/28] media: ccs: Add support for embedded data stream Date: Tue, 3 Oct 2023 15:08:11 +0300 Message-Id: <20231003120813.77726-17-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add support for embedded data stream, in UAPI and frame descriptor. This patch adds also a new embedded data pad (2) to the source sub-device. Signed-off-by: Sakari Ailus --- drivers/media/i2c/ccs/ccs-core.c | 337 +++++++++++++++++++++++++++++-- drivers/media/i2c/ccs/ccs.h | 18 +- 2 files changed, 336 insertions(+), 19 deletions(-) diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c index 75abb18ae847..2c8ff7cb63a5 100644 --- a/drivers/media/i2c/ccs/ccs-core.c +++ b/drivers/media/i2c/ccs/ccs-core.c @@ -1890,6 +1890,13 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, if (rval < 0) goto err_pm_put; + /* Configure embedded data */ + if (sensor->csi_format->compressed >= 16) { + rval = ccs_write(sensor, EMB_DATA_CTRL, sensor->emb_data_ctrl); + if (rval < 0) + goto err_pm_put; + } + if (CCS_LIM(sensor, FLASH_MODE_CAPABILITY) & (CCS_FLASH_MODE_CAPABILITY_SINGLE_STROBE | SMIAPP_FLASH_MODE_CAPABILITY_MULTIPLE_STROBE) && @@ -2009,6 +2016,57 @@ static const struct ccs_csi_data_format return sensor->csi_format; } +#define CCS_EMBEDDED_CODE_DEPTH(depth, half_depth) \ + depth, \ + CCS_EMB_DATA_CAPABILITY_TWO_BYTES_PER_RAW##depth, \ + CCS_EMB_DATA_CAPABILITY_NO_ONE_BYTE_PER_RAW##depth, \ + CCS_EMB_DATA_CTRL_RAW##half_depth##_PACKING_FOR_RAW##depth, \ + MEDIA_BUS_FMT_META_##half_depth, \ + MEDIA_BUS_FMT_META_##depth, \ + +static const struct ccs_embedded_code { + u8 depth; + u8 cap_two_bytes_per_sample; + u8 cap_no_legacy; + u8 ctrl; + u32 code_two_bytes; + u32 code_legacy; +} ccs_embedded_codes[] = { + { CCS_EMBEDDED_CODE_DEPTH(16, 8) }, + { CCS_EMBEDDED_CODE_DEPTH(20, 10) }, + { CCS_EMBEDDED_CODE_DEPTH(24, 12) }, +}; + +static const struct ccs_embedded_code *ccs_embedded_code(unsigned int bpp) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(ccs_embedded_codes); i++) + if (ccs_embedded_codes[i].depth == bpp) + return ccs_embedded_codes + i; + + WARN_ON(1); + + return ccs_embedded_codes; +} + +static u32 +ccs_default_embedded_code(struct ccs_sensor *sensor, + const struct ccs_embedded_code *embedded_code) +{ + if (CCS_LIM(sensor, EMB_DATA_CAPABILITY) & + BIT(embedded_code->cap_two_bytes_per_sample)) + return embedded_code->code_two_bytes; + + if (!(CCS_LIM(sensor, EMB_DATA_CAPABILITY) & + BIT(embedded_code->cap_no_legacy))) + return embedded_code->code_legacy; + + WARN_ON(1); + + return embedded_code->code_legacy; +} + static int ccs_enum_mbus_code(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, struct v4l2_subdev_mbus_code_enum *code) @@ -2024,6 +2082,69 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev, dev_err(&client->dev, "subdev %s, pad %u, index %u\n", subdev->name, code->pad, code->index); + if (subdev == &sensor->src->sd) { + if (code->pad == CCS_PAD_META) { + if (code->index) + goto out; + + code->code = MEDIA_BUS_FMT_CCS_EMBEDDED; + + rval = 0; + goto out; + } + if (code->stream == CCS_STREAM_META) { + struct v4l2_mbus_framefmt *pix_fmt = + v4l2_subdev_state_get_stream_format(sd_state, + CCS_PAD_SRC, + CCS_STREAM_PIXEL); + const struct ccs_csi_data_format *csi_format = + ccs_validate_csi_data_format(sensor, + pix_fmt->code); + unsigned int i = 0; + u32 codes[2]; + + switch (csi_format->compressed) { + case 8: + codes[i++] = MEDIA_BUS_FMT_META_8; + break; + case 10: + codes[i++] = MEDIA_BUS_FMT_META_10; + break; + case 12: + codes[i++] = MEDIA_BUS_FMT_META_12; + break; + case 14: + codes[i++] = MEDIA_BUS_FMT_META_14; + break; + case 16: + case 20: + case 24: { + const struct ccs_embedded_code *embedded_code = + ccs_embedded_code(csi_format->compressed); + + if (CCS_LIM(sensor, EMB_DATA_CAPABILITY) & + BIT(embedded_code->cap_two_bytes_per_sample)) + codes[i++] = + embedded_code->code_two_bytes; + + if (!(CCS_LIM(sensor, EMB_DATA_CAPABILITY) & + BIT(embedded_code->cap_no_legacy))) + codes[i++] = embedded_code->code_legacy; + break; + } + default: + WARN_ON(1); + } + + if (WARN_ON(i > ARRAY_SIZE(codes)) || code->index >= i) + goto out; + + code->code = codes[code->index]; + rval = 0; + goto out; + } + } + if (subdev != &sensor->src->sd || code->pad != CCS_PAD_SRC) { if (code->index) goto out; @@ -2066,8 +2187,11 @@ static int __ccs_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, struct v4l2_subdev_format *fmt) { - fmt->format = *v4l2_subdev_get_pad_format(subdev, sd_state, fmt->pad); - fmt->format.code = __ccs_get_mbus_code(subdev, fmt->pad); + fmt->format = *v4l2_subdev_get_fmt_ptr(subdev, sd_state, fmt->pad, + fmt->stream); + + if (fmt->pad != CCS_PAD_META && fmt->stream != CCS_STREAM_META) + fmt->format.code = __ccs_get_mbus_code(subdev, fmt->pad); return 0; } @@ -2097,10 +2221,12 @@ static void ccs_get_crop_compose(struct v4l2_subdev *subdev, if (crops) for (i = 0; i < subdev->entity.num_pads; i++) crops[i] = - v4l2_subdev_get_pad_crop(subdev, sd_state, i); + v4l2_subdev_get_crop_ptr(subdev, sd_state, i, + CCS_STREAM_PIXEL); if (comps) - *comps = v4l2_subdev_get_pad_compose(subdev, sd_state, - ssd->sink_pad); + *comps = v4l2_subdev_get_compose_ptr(subdev, sd_state, + ssd->sink_pad, + CCS_STREAM_PIXEL); } /* Changes require propagation only on sink pad. */ @@ -2193,6 +2319,83 @@ static int ccs_set_format_source(struct v4l2_subdev *subdev, return ccs_pll_update(sensor); } +static int ccs_set_format_meta(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *sd_state, + struct v4l2_mbus_framefmt *fmt) +{ + struct ccs_sensor *sensor = to_ccs_sensor(subdev); + const struct ccs_csi_data_format *csi_format; + struct v4l2_mbus_framefmt *pix_fmt; + struct v4l2_mbus_framefmt *meta_fmt; + struct v4l2_mbus_framefmt *meta_out_fmt; + u32 code; + + pix_fmt = v4l2_subdev_state_get_stream_format(sd_state, CCS_PAD_SRC, + CCS_STREAM_PIXEL); + meta_fmt = v4l2_subdev_state_get_stream_format(sd_state, CCS_PAD_META, + 0); + meta_out_fmt = v4l2_subdev_state_get_stream_format(sd_state, + CCS_PAD_SRC, + CCS_STREAM_META); + + code = fmt ? fmt->code : meta_out_fmt->code; + + meta_out_fmt->width = meta_fmt->width = pix_fmt->width; + meta_out_fmt->height = meta_fmt->height = + sensor->embedded_end - sensor->embedded_start; + meta_fmt->code = MEDIA_BUS_FMT_CCS_EMBEDDED; + + csi_format = ccs_validate_csi_data_format(sensor, pix_fmt->code); + + switch (csi_format->compressed) { + case 8: + meta_out_fmt->code = MEDIA_BUS_FMT_META_8; + break; + case 10: + meta_out_fmt->code = MEDIA_BUS_FMT_META_10; + break; + case 12: + meta_out_fmt->code = MEDIA_BUS_FMT_META_12; + break; + case 14: + meta_out_fmt->code = MEDIA_BUS_FMT_META_14; + break; + case 16: + case 20: + case 24: { + const struct ccs_embedded_code *embedded_code; + + embedded_code = ccs_embedded_code(csi_format->compressed); + meta_out_fmt->code = + ccs_default_embedded_code(sensor, embedded_code); + + if (!(CCS_LIM(sensor, EMB_DATA_CAPABILITY) & + BIT(embedded_code->cap_no_legacy)) && + code == embedded_code->code_legacy) { + meta_out_fmt->code = embedded_code->code_legacy; + sensor->emb_data_ctrl = 0; + } + + if (CCS_LIM(sensor, EMB_DATA_CAPABILITY) & + BIT(embedded_code->cap_two_bytes_per_sample) && + code == embedded_code->code_two_bytes) { + meta_out_fmt->code = embedded_code->code_two_bytes; + sensor->emb_data_ctrl = embedded_code->ctrl; + } + + break; + } + default: + WARN_ON(1); + return 0; + } + + if (fmt) + *fmt = *meta_out_fmt; + + return 0; +} + static int ccs_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, struct v4l2_subdev_format *fmt) @@ -2201,12 +2404,24 @@ static int ccs_set_format(struct v4l2_subdev *subdev, struct ccs_subdev *ssd = to_ccs_subdev(subdev); struct v4l2_rect *crops[CCS_PADS]; + if (subdev == &sensor->src->sd && fmt->pad == CCS_PAD_META) + return ccs_get_format(subdev, sd_state, fmt); + mutex_lock(&sensor->mutex); + if (subdev == &sensor->src->sd && fmt->stream == CCS_STREAM_META) { + ccs_set_format_meta(subdev, sd_state, &fmt->format); + + mutex_unlock(&sensor->mutex); + + return 0; + } + if (fmt->pad == ssd->source_pad) { int rval; rval = ccs_set_format_source(subdev, sd_state, fmt); + ccs_set_format_meta(subdev, sd_state, NULL); mutex_unlock(&sensor->mutex); @@ -2481,6 +2696,12 @@ static int ccs_sel_supported(struct v4l2_subdev *subdev, struct ccs_sensor *sensor = to_ccs_sensor(subdev); struct ccs_subdev *ssd = to_ccs_subdev(subdev); + if (sel->stream != CCS_STREAM_PIXEL) + return -EINVAL; + + if (sel->pad == CCS_PAD_META) + return -EINVAL; + /* We only implement crop in three places. */ switch (sel->target) { case V4L2_SEL_TGT_CROP: @@ -2525,7 +2746,8 @@ static int ccs_set_crop(struct v4l2_subdev *subdev, if (sel->pad == ssd->sink_pad) { struct v4l2_mbus_framefmt *mfmt = - v4l2_subdev_get_pad_format(subdev, sd_state, sel->pad); + v4l2_subdev_get_fmt_ptr(subdev, sd_state, sel->pad, + CCS_STREAM_PIXEL); src_size.width = mfmt->width; src_size.height = mfmt->height; @@ -2585,8 +2807,10 @@ static int ccs_get_selection(struct v4l2_subdev *subdev, ccs_get_native_size(ssd, &sel->r); } else if (sel->pad == ssd->sink_pad) { struct v4l2_mbus_framefmt *sink_fmt = - v4l2_subdev_get_pad_format(subdev, sd_state, - ssd->sink_pad); + v4l2_subdev_get_fmt_ptr(subdev, sd_state, + ssd->sink_pad, + CCS_STREAM_PIXEL); + sel->r.top = sel->r.left = 0; sel->r.width = sink_fmt->width; sel->r.height = sink_fmt->height; @@ -2672,6 +2896,14 @@ static int ccs_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad, entry++; desc->num_entries++; + if (sensor->embedded_start != sensor->embedded_end) { + entry->pixelcode = MEDIA_BUS_FMT_CCS_EMBEDDED; + entry->stream = CCS_STREAM_META; + entry->bus.csi2.dt = MIPI_CSI2_DT_EMBEDDED_8B; + entry++; + desc->num_entries++; + } + return 0; } @@ -3020,7 +3252,8 @@ static int ccs_init_subdev(struct ccs_sensor *sensor, ssd->sensor = sensor; ssd->npads = num_pads; - ssd->source_pad = num_pads - 1; + ssd->source_pad = + ssd == sensor->pixel_array ? CCS_PA_PAD_SRC : CCS_PAD_SRC; v4l2_i2c_subdev_set_name(&ssd->sd, client, sensor->minfo.name, name); @@ -3034,6 +3267,10 @@ static int ccs_init_subdev(struct ccs_sensor *sensor, ssd->sd.owner = THIS_MODULE; ssd->sd.dev = &client->dev; v4l2_set_subdevdata(&ssd->sd, client); + } else { + ssd->sd.flags |= V4L2_SUBDEV_FL_STREAMS; + ssd->pads[CCS_PAD_META].flags = + MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_INTERNAL; } rval = media_entity_pads_init(&ssd->sd.entity, ssd->npads, ssd->pads); @@ -3062,11 +3299,16 @@ static int ccs_init_cfg(struct v4l2_subdev *sd, for (i = 0; i < ssd->npads; i++) { struct v4l2_mbus_framefmt *fmt = - v4l2_subdev_get_pad_format(sd, sd_state, i); + v4l2_subdev_get_fmt_ptr(sd, sd_state, i, + CCS_STREAM_PIXEL); struct v4l2_rect *crop = - v4l2_subdev_get_pad_crop(sd, sd_state, i); + v4l2_subdev_get_crop_ptr(sd, sd_state, i, + CCS_STREAM_PIXEL); struct v4l2_rect *comp; + if (!fmt) + continue; + ccs_get_native_size(ssd, crop); fmt->width = crop->width; @@ -3077,7 +3319,8 @@ static int ccs_init_cfg(struct v4l2_subdev *sd, if (ssd == sensor->pixel_array) continue; - comp = v4l2_subdev_get_pad_compose(sd, sd_state, i); + comp = v4l2_subdev_get_compose_ptr(sd, sd_state, i, + CCS_STREAM_PIXEL); *comp = *crop; } @@ -3086,6 +3329,47 @@ static int ccs_init_cfg(struct v4l2_subdev *sd, return 0; } +static int ccs_src_init_cfg(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state) +{ + struct v4l2_subdev_route routes[] = { + { + .sink_pad = CCS_PAD_SINK, + .source_pad = CCS_PAD_SRC, + .source_stream = CCS_STREAM_PIXEL, + .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE, + }, { + .sink_pad = CCS_PAD_META, + .source_pad = CCS_PAD_SRC, + .source_stream = CCS_STREAM_META, + .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE, + } + }; + struct v4l2_subdev_krouting routing = { + .routes = routes, + .num_routes = 1, + }; + struct ccs_subdev *ssd = to_ccs_subdev(sd); + struct ccs_sensor *sensor = ssd->sensor; + int rval; + + if (sensor->embedded_start != sensor->embedded_end) + routing.num_routes++; + + rval = v4l2_subdev_set_routing(sd, sd_state, &routing); + if (rval) + return rval; + + rval = ccs_init_cfg(sd, sd_state); + if (rval) + return rval; + + if (sensor->embedded_start != sensor->embedded_end) + ccs_set_format_meta(sd, sd_state, NULL); + + return 0; +} + static const struct v4l2_subdev_video_ops ccs_video_ops = { .s_stream = v4l2_subdev_s_stream_helper, .pre_streamon = ccs_pre_streamon, @@ -3099,6 +3383,15 @@ static const struct v4l2_subdev_pad_ops ccs_pad_ops = { .set_fmt = ccs_set_format, .get_selection = ccs_get_selection, .set_selection = ccs_set_selection, +}; + +static const struct v4l2_subdev_pad_ops ccs_src_pad_ops = { + .init_cfg = ccs_src_init_cfg, + .enum_mbus_code = ccs_enum_mbus_code, + .get_fmt = ccs_get_format, + .set_fmt = ccs_set_format, + .get_selection = ccs_get_selection, + .set_selection = ccs_set_selection, .enable_streams = ccs_enable_streams, .disable_streams = ccs_disable_streams, .get_frame_desc = ccs_get_frame_desc, @@ -3115,6 +3408,12 @@ static const struct v4l2_subdev_ops ccs_ops = { .sensor = &ccs_sensor_ops, }; +static const struct v4l2_subdev_ops ccs_src_ops = { + .video = &ccs_video_ops, + .pad = &ccs_src_pad_ops, + .sensor = &ccs_sensor_ops, +}; + static const struct media_entity_operations ccs_entity_ops = { .link_validate = v4l2_subdev_link_validate, }; @@ -3268,7 +3567,7 @@ static int ccs_probe(struct i2c_client *client) sensor->src = &sensor->ssds[sensor->ssds_used]; - v4l2_i2c_subdev_init(&sensor->src->sd, client, &ccs_ops); + v4l2_i2c_subdev_init(&sensor->src->sd, client, &ccs_src_ops); sensor->src->sd.internal_ops = &ccs_internal_src_ops; sensor->regulators = devm_kcalloc(&client->dev, @@ -3533,11 +3832,19 @@ static int ccs_probe(struct i2c_client *client) goto out_cleanup; } - rval = ccs_init_subdev(sensor, sensor->scaler, " scaler", 2, + rval = ccs_init_subdev(sensor, sensor->scaler, " scaler", + sensor->ssds_used != CCS_SUBDEVS ? + CCS_PADS_NOMETA : + sensor->embedded_start == sensor->embedded_end ? + CCS_PADS_NOMETA : CCS_PADS, MEDIA_ENT_F_PROC_VIDEO_SCALER); if (rval) goto out_cleanup; - rval = ccs_init_subdev(sensor, sensor->binner, " binner", 2, + rval = ccs_init_subdev(sensor, sensor->binner, " binner", + sensor->ssds_used == CCS_SUBDEVS ? + CCS_PADS_NOMETA : + sensor->embedded_start == sensor->embedded_end ? + CCS_PADS_NOMETA : CCS_PADS, MEDIA_ENT_F_PROC_VIDEO_SCALER); if (rval) goto out_cleanup; diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h index 7fd9b6493d2b..5e561cc91717 100644 --- a/drivers/media/i2c/ccs/ccs.h +++ b/drivers/media/i2c/ccs/ccs.h @@ -172,11 +172,18 @@ struct ccs_csi_data_format { #define CCS_SUBDEVS 3 #define CCS_PA_PAD_SRC 0 -#define CCS_PAD_SINK 0 -#define CCS_PAD_SRC 1 -#define CCS_PADS 2 +enum { + CCS_PAD_SINK, + CCS_PAD_SRC, + CCS_PAD_META, + CCS_PADS_NOMETA = CCS_PAD_META, + CCS_PADS, +}; -#define CCS_STREAM_PIXEL 0 +enum { + CCS_STREAM_PIXEL, + CCS_STREAM_META, +}; struct ccs_binning_subtype { u8 horizontal:4; @@ -226,6 +233,9 @@ struct ccs_sensor { int default_pixel_order; struct ccs_data_container sdata, mdata; + u32 embedded_mbus_code; + u8 emb_data_ctrl; + u8 binning_horizontal; u8 binning_vertical; From patchwork Tue Oct 3 12:08:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0476AE7544F for ; Tue, 3 Oct 2023 12:08:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232222AbjJCMIu (ORCPT ); Tue, 3 Oct 2023 08:08:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232177AbjJCMIt (ORCPT ); Tue, 3 Oct 2023 08:08:49 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 318DBBB for ; Tue, 3 Oct 2023 05:08:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334924; x=1727870924; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fktGfw+HsjdV4keozBIEfasQQ0QPYp+WtEV8odPHHeU=; b=huZ+fL310vYP9zNZqrvSieZOxzmTJ1svtrjvLKmrpCXe4RujlxtrWroI B3F8ZdOd5z8BvpzmY+qGR33NZRit5EHaVHn1YWOIEJEntNFHUI57mwpsD cacIO3eAT8+DNY69ObuDzfEfBA7/WMsNOC74UfF193oU968WODjiW1Ab0 l0VdiDrKTz2WtT3q27S4sjjLythiKZ1g/TxB4rB962HFkv1tgB9YvbBAv fJ7H0ZPMxDbAzD0YvbLgywbhB7vnGJOCGts4NCRmUCkt7qBwIUA6oa8MR byjQmTK3nYsoUuodPWQFNwaeGQOtUHU1yUAXGLty3GvXryW7Nel0egBgU A==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681589" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681589" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285970" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285970" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:41 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id DB58C120A56; Tue, 3 Oct 2023 15:08:37 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 27/28] media: ccs: Remove ccs_get_crop_compose helper Date: Tue, 3 Oct 2023 15:08:12 +0300 Message-Id: <20231003120813.77726-18-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org As it's now easier to obtain the necessary information on crop and compose rectangles after moving to sub-device state, remove the ccs_get_crop_compose helper. Signed-off-by: Sakari Ailus --- drivers/media/i2c/ccs/ccs-core.c | 120 ++++++++++++++----------------- 1 file changed, 53 insertions(+), 67 deletions(-) diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c index 2c8ff7cb63a5..9ebf6063150f 100644 --- a/drivers/media/i2c/ccs/ccs-core.c +++ b/drivers/media/i2c/ccs/ccs-core.c @@ -2210,25 +2210,6 @@ static int ccs_get_format(struct v4l2_subdev *subdev, return rval; } -static void ccs_get_crop_compose(struct v4l2_subdev *subdev, - struct v4l2_subdev_state *sd_state, - struct v4l2_rect **crops, - struct v4l2_rect **comps) -{ - struct ccs_subdev *ssd = to_ccs_subdev(subdev); - unsigned int i; - - if (crops) - for (i = 0; i < subdev->entity.num_pads; i++) - crops[i] = - v4l2_subdev_get_crop_ptr(subdev, sd_state, i, - CCS_STREAM_PIXEL); - if (comps) - *comps = v4l2_subdev_get_compose_ptr(subdev, sd_state, - ssd->sink_pad, - CCS_STREAM_PIXEL); -} - /* Changes require propagation only on sink pad. */ static void ccs_propagate(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, int which, @@ -2237,9 +2218,14 @@ static void ccs_propagate(struct v4l2_subdev *subdev, struct ccs_sensor *sensor = to_ccs_sensor(subdev); struct ccs_subdev *ssd = to_ccs_subdev(subdev); struct v4l2_rect *comp, *crops[CCS_PADS]; + unsigned int i; - ccs_get_crop_compose(subdev, sd_state, crops, &comp); + for (i = 0; i < subdev->entity.num_pads; i++) + crops[i] = v4l2_subdev_get_crop_ptr(subdev, sd_state, i, + CCS_STREAM_PIXEL); + comp = v4l2_subdev_get_compose_ptr(subdev, sd_state, + ssd->sink_pad, CCS_STREAM_PIXEL); switch (target) { case V4L2_SEL_TGT_CROP: comp->width = crops[CCS_PAD_SINK]->width; @@ -2402,7 +2388,7 @@ static int ccs_set_format(struct v4l2_subdev *subdev, { struct ccs_sensor *sensor = to_ccs_sensor(subdev); struct ccs_subdev *ssd = to_ccs_subdev(subdev); - struct v4l2_rect *crops[CCS_PADS]; + struct v4l2_rect *crop; if (subdev == &sensor->src->sd && fmt->pad == CCS_PAD_META) return ccs_get_format(subdev, sd_state, fmt); @@ -2443,12 +2429,13 @@ static int ccs_set_format(struct v4l2_subdev *subdev, CCS_LIM(sensor, MIN_Y_OUTPUT_SIZE), CCS_LIM(sensor, MAX_Y_OUTPUT_SIZE)); - ccs_get_crop_compose(subdev, sd_state, crops, NULL); + crop = v4l2_subdev_get_crop_ptr(subdev, sd_state, ssd->sink_pad, + CCS_STREAM_PIXEL); - crops[ssd->sink_pad]->left = 0; - crops[ssd->sink_pad]->top = 0; - crops[ssd->sink_pad]->width = fmt->format.width; - crops[ssd->sink_pad]->height = fmt->format.height; + crop->left = 0; + crop->top = 0; + crop->width = fmt->format.width; + crop->height = fmt->format.height; ccs_propagate(subdev, sd_state, fmt->which, V4L2_SEL_TGT_CROP); mutex_unlock(&sensor->mutex); @@ -2503,24 +2490,23 @@ static int scaling_goodness(struct v4l2_subdev *subdev, int w, int ask_w, static void ccs_set_compose_binner(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, struct v4l2_subdev_selection *sel, - struct v4l2_rect **crops, + struct v4l2_rect *sink_crop, struct v4l2_rect *comp) { struct ccs_sensor *sensor = to_ccs_sensor(subdev); unsigned int i; unsigned int binh = 1, binv = 1; int best = scaling_goodness( - subdev, - crops[CCS_PAD_SINK]->width, sel->r.width, - crops[CCS_PAD_SINK]->height, sel->r.height, sel->flags); + subdev, sink_crop->width, sel->r.width, + sink_crop->height, sel->r.height, sel->flags); for (i = 0; i < sensor->nbinning_subtypes; i++) { int this = scaling_goodness( subdev, - crops[CCS_PAD_SINK]->width + sink_crop->width / sensor->binning_subtypes[i].horizontal, sel->r.width, - crops[CCS_PAD_SINK]->height + sink_crop->height / sensor->binning_subtypes[i].vertical, sel->r.height, sel->flags); @@ -2535,8 +2521,8 @@ static void ccs_set_compose_binner(struct v4l2_subdev *subdev, sensor->binning_horizontal = binh; } - sel->r.width = (crops[CCS_PAD_SINK]->width / binh) & ~1; - sel->r.height = (crops[CCS_PAD_SINK]->height / binv) & ~1; + sel->r.width = (sink_crop->width / binh) & ~1; + sel->r.height = (sink_crop->height / binv) & ~1; } /* @@ -2551,7 +2537,7 @@ static void ccs_set_compose_binner(struct v4l2_subdev *subdev, static void ccs_set_compose_scaler(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, struct v4l2_subdev_selection *sel, - struct v4l2_rect **crops, + struct v4l2_rect *sink_crop, struct v4l2_rect *comp) { struct i2c_client *client = v4l2_get_subdevdata(subdev); @@ -2564,17 +2550,12 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev, unsigned int i; int best = INT_MIN; - sel->r.width = min_t(unsigned int, sel->r.width, - crops[CCS_PAD_SINK]->width); - sel->r.height = min_t(unsigned int, sel->r.height, - crops[CCS_PAD_SINK]->height); - - a = crops[CCS_PAD_SINK]->width - * CCS_LIM(sensor, SCALER_N_MIN) / sel->r.width; - b = crops[CCS_PAD_SINK]->height - * CCS_LIM(sensor, SCALER_N_MIN) / sel->r.height; - max_m = crops[CCS_PAD_SINK]->width - * CCS_LIM(sensor, SCALER_N_MIN) + sel->r.width = min_t(unsigned int, sel->r.width, sink_crop->width); + sel->r.height = min_t(unsigned int, sel->r.height, sink_crop->height); + + a = sink_crop->width * CCS_LIM(sensor, SCALER_N_MIN) / sel->r.width; + b = sink_crop->height * CCS_LIM(sensor, SCALER_N_MIN) / sel->r.height; + max_m = sink_crop->width * CCS_LIM(sensor, SCALER_N_MIN) / CCS_LIM(sensor, MIN_X_OUTPUT_SIZE); a = clamp(a, CCS_LIM(sensor, SCALER_M_MIN), @@ -2607,10 +2588,10 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev, for (i = 0; i < ntry; i++) { int this = scaling_goodness( subdev, - crops[CCS_PAD_SINK]->width + sink_crop->width / try[i] * CCS_LIM(sensor, SCALER_N_MIN), sel->r.width, - crops[CCS_PAD_SINK]->height, + sink_crop->height, sel->r.height, sel->flags); @@ -2627,12 +2608,10 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev, continue; this = scaling_goodness( - subdev, crops[CCS_PAD_SINK]->width - / try[i] + subdev, sink_crop->width / try[i] * CCS_LIM(sensor, SCALER_N_MIN), sel->r.width, - crops[CCS_PAD_SINK]->height - / try[i] + sink_crop->height / try[i] * CCS_LIM(sensor, SCALER_N_MIN), sel->r.height, sel->flags); @@ -2645,17 +2624,15 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev, } sel->r.width = - (crops[CCS_PAD_SINK]->width - / scale_m + (sink_crop->width / scale_m * CCS_LIM(sensor, SCALER_N_MIN)) & ~1; if (mode == SMIAPP_SCALING_MODE_BOTH) sel->r.height = - (crops[CCS_PAD_SINK]->height - / scale_m + (sink_crop->height / scale_m * CCS_LIM(sensor, SCALER_N_MIN)) & ~1; else - sel->r.height = crops[CCS_PAD_SINK]->height; + sel->r.height = sink_crop->height; if (sel->which == V4L2_SUBDEV_FORMAT_ACTIVE) { sensor->scale_m = scale_m; @@ -2669,17 +2646,20 @@ static int ccs_set_compose(struct v4l2_subdev *subdev, { struct ccs_sensor *sensor = to_ccs_sensor(subdev); struct ccs_subdev *ssd = to_ccs_subdev(subdev); - struct v4l2_rect *comp, *crops[CCS_PADS]; + struct v4l2_rect *comp, *sink_crop; - ccs_get_crop_compose(subdev, sd_state, crops, &comp); + sink_crop = v4l2_subdev_get_crop_ptr(subdev, sd_state, CCS_PAD_SINK, + CCS_STREAM_PIXEL); + comp = v4l2_subdev_get_compose_ptr(subdev, sd_state, ssd->sink_pad, + CCS_STREAM_PIXEL); sel->r.top = 0; sel->r.left = 0; if (ssd == sensor->binner) - ccs_set_compose_binner(subdev, sd_state, sel, crops, comp); + ccs_set_compose_binner(subdev, sd_state, sel, sink_crop, comp); else - ccs_set_compose_scaler(subdev, sd_state, sel, crops, comp); + ccs_set_compose_scaler(subdev, sd_state, sel, sink_crop, comp); *comp = sel->r; ccs_propagate(subdev, sd_state, sel->which, V4L2_SEL_TGT_COMPOSE); @@ -2740,9 +2720,12 @@ static int ccs_set_crop(struct v4l2_subdev *subdev, { struct ccs_sensor *sensor = to_ccs_sensor(subdev); struct ccs_subdev *ssd = to_ccs_subdev(subdev); - struct v4l2_rect src_size = { 0 }, *crops[CCS_PADS], *comp; + struct v4l2_rect src_size = { 0 }, *crop, *comp; - ccs_get_crop_compose(subdev, sd_state, crops, &comp); + crop = v4l2_subdev_get_crop_ptr(subdev, sd_state, sel->pad, + CCS_STREAM_PIXEL); + comp = v4l2_subdev_get_compose_ptr(subdev, sd_state, ssd->sink_pad, + CCS_STREAM_PIXEL); if (sel->pad == ssd->sink_pad) { struct v4l2_mbus_framefmt *mfmt = @@ -2766,7 +2749,7 @@ static int ccs_set_crop(struct v4l2_subdev *subdev, sel->r.left = min_t(int, sel->r.left, src_size.width - sel->r.width); sel->r.top = min_t(int, sel->r.top, src_size.height - sel->r.height); - *crops[sel->pad] = sel->r; + *crop = sel->r; if (ssd != sensor->pixel_array && sel->pad == CCS_PAD_SINK) ccs_propagate(subdev, sd_state, sel->which, V4L2_SEL_TGT_CROP); @@ -2791,14 +2774,17 @@ static int ccs_get_selection(struct v4l2_subdev *subdev, { struct ccs_sensor *sensor = to_ccs_sensor(subdev); struct ccs_subdev *ssd = to_ccs_subdev(subdev); - struct v4l2_rect *comp, *crops[CCS_PADS]; + struct v4l2_rect *crop, *comp; int ret; ret = ccs_sel_supported(subdev, sel); if (ret) return ret; - ccs_get_crop_compose(subdev, sd_state, crops, &comp); + crop = v4l2_subdev_get_crop_ptr(subdev, sd_state, sel->pad, + CCS_STREAM_PIXEL); + comp = v4l2_subdev_get_compose_ptr(subdev, sd_state, ssd->sink_pad, + CCS_STREAM_PIXEL); switch (sel->target) { case V4L2_SEL_TGT_CROP_BOUNDS: @@ -2820,7 +2806,7 @@ static int ccs_get_selection(struct v4l2_subdev *subdev, break; case V4L2_SEL_TGT_CROP: case V4L2_SEL_TGT_COMPOSE_BOUNDS: - sel->r = *crops[sel->pad]; + sel->r = *crop; break; case V4L2_SEL_TGT_COMPOSE: sel->r = *comp; From patchwork Tue Oct 3 12:08:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 729117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F246E75426 for ; Tue, 3 Oct 2023 12:08:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232224AbjJCMIv (ORCPT ); Tue, 3 Oct 2023 08:08:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232216AbjJCMIu (ORCPT ); Tue, 3 Oct 2023 08:08:50 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05FC8CE for ; Tue, 3 Oct 2023 05:08:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696334926; x=1727870926; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5Qc7c9hLKvrfaUEMxNionXwYY4FgDovSx6j6eLs6vSI=; b=b+ijsmrwSuYNv/jXdOdZ5T1g7xVGVVYfY4mUUYswfEp5UFQfUGQ9cwsq mB9u3I0SUXJ/LUiNpb8EqtfBvy2Sj3c7+VTIWsj3iHRWz9cTQEsesTlxi WyitKt6QwZVE8GVE9jf/07zZwIdsUqVQpGmsWvSxtoqfOnRTaJyFMoQEn X8BzDuS03Xe9RzJE1x9jDF7mCAV5AV2vGblK2bhc39X5kUrkovxHDl5zy awZ0a3k51otcUl7WLn9HUsl4OIuwf1tvGtAFsVj20MSEnC1kKm/R5aHKp jM5g/NfmV1mkVZXqTiuh6T/oC88C7EIzaL/Vddz3muF4b+nsNjdSdlRL2 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="385681594" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="385681594" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10851"; a="780285980" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="780285980" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 05:08:42 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 5B253120A57; Tue, 3 Oct 2023 15:08:39 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" Subject: [PATCH v6 28/28] media: ccs: Rely on sub-device state locking Date: Tue, 3 Oct 2023 15:08:13 +0300 Message-Id: <20231003120813.77726-19-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231003115237.76828-1-sakari.ailus@linux.intel.com> References: <20231003115237.76828-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Rely on sub-device state locking to serialise access to driver's data structures. The driver-provided mutex is used as the state lock for all driver sub-devices. Signed-off-by: Sakari Ailus --- drivers/media/i2c/ccs/ccs-core.c | 163 +++++++++++++++---------------- drivers/media/i2c/ccs/ccs.h | 1 - 2 files changed, 77 insertions(+), 87 deletions(-) diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c index 9ebf6063150f..89eda03a99a1 100644 --- a/drivers/media/i2c/ccs/ccs-core.c +++ b/drivers/media/i2c/ccs/ccs-core.c @@ -531,12 +531,13 @@ static int ccs_pll_update(struct ccs_sensor *sensor) * */ -static void __ccs_update_exposure_limits(struct ccs_sensor *sensor) +static void __ccs_update_exposure_limits(struct ccs_sensor *sensor, + struct v4l2_rect *pa_src) { struct v4l2_ctrl *ctrl = sensor->exposure; int max; - max = sensor->pa_src.height + sensor->vblank->val - + max = pa_src->height + sensor->vblank->val - CCS_LIM(sensor, COARSE_INTEGRATION_TIME_MAX_MARGIN); __v4l2_ctrl_modify_range(ctrl, ctrl->minimum, max, ctrl->step, max); @@ -639,12 +640,21 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl) container_of(ctrl->handler, struct ccs_subdev, ctrl_handler) ->sensor; struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd); + struct v4l2_subdev_state *state; + struct v4l2_rect *pa_src; int pm_status; u32 orient = 0; unsigned int i; int exposure; int rval; + if (ctrl->id == V4L2_CID_VBLANK || ctrl->id == V4L2_CID_HBLANK) { + state = v4l2_subdev_get_locked_active_state(&sensor->pixel_array->sd); + pa_src = v4l2_subdev_get_crop_ptr(&sensor->pixel_array->sd, + state, CCS_PA_PAD_SRC, + CCS_STREAM_PIXEL); + } + switch (ctrl->id) { case V4L2_CID_HFLIP: case V4L2_CID_VFLIP: @@ -663,7 +673,7 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl) case V4L2_CID_VBLANK: exposure = sensor->exposure->val; - __ccs_update_exposure_limits(sensor); + __ccs_update_exposure_limits(sensor, pa_src); if (exposure > sensor->exposure->maximum) { sensor->exposure->val = sensor->exposure->maximum; @@ -755,12 +765,12 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl) break; case V4L2_CID_VBLANK: rval = ccs_write(sensor, FRAME_LENGTH_LINES, - sensor->pa_src.height + ctrl->val); + pa_src->height + ctrl->val); break; case V4L2_CID_HBLANK: rval = ccs_write(sensor, LINE_LENGTH_PCK, - sensor->pa_src.width + ctrl->val); + pa_src->width + ctrl->val); break; case V4L2_CID_TEST_PATTERN: @@ -1215,7 +1225,8 @@ static int ccs_get_mbus_formats(struct ccs_sensor *sensor) return 0; } -static void ccs_update_blanking(struct ccs_sensor *sensor) +static void ccs_update_blanking(struct ccs_sensor *sensor, + struct v4l2_rect *pa_src) { struct v4l2_ctrl *vblank = sensor->vblank; struct v4l2_ctrl *hblank = sensor->hblank; @@ -1238,21 +1249,26 @@ static void ccs_update_blanking(struct ccs_sensor *sensor) min = max_t(int, CCS_LIM(sensor, MIN_FRAME_BLANKING_LINES), - min_fll - sensor->pa_src.height); - max = max_fll - sensor->pa_src.height; + min_fll - pa_src->height); + max = max_fll - pa_src->height; __v4l2_ctrl_modify_range(vblank, min, max, vblank->step, min); - min = max_t(int, min_llp - sensor->pa_src.width, min_lbp); - max = max_llp - sensor->pa_src.width; + min = max_t(int, min_llp - pa_src->width, min_lbp); + max = max_llp - pa_src->width; __v4l2_ctrl_modify_range(hblank, min, max, hblank->step, min); - __ccs_update_exposure_limits(sensor); + __ccs_update_exposure_limits(sensor, pa_src); } static int ccs_pll_blanking_update(struct ccs_sensor *sensor) { + struct v4l2_subdev_state *state = + v4l2_subdev_get_locked_active_state(&sensor->pixel_array->sd); + struct v4l2_rect *pa_src = + v4l2_subdev_get_crop_ptr(&sensor->pixel_array->sd, state, + CCS_PA_PAD_SRC, CCS_STREAM_PIXEL); struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd); int rval; @@ -1261,15 +1277,15 @@ static int ccs_pll_blanking_update(struct ccs_sensor *sensor) return rval; /* Output from pixel array, including blanking */ - ccs_update_blanking(sensor); + ccs_update_blanking(sensor, pa_src); dev_dbg(&client->dev, "vblank\t\t%d\n", sensor->vblank->val); dev_dbg(&client->dev, "hblank\t\t%d\n", sensor->hblank->val); dev_dbg(&client->dev, "real timeperframe\t100/%d\n", sensor->pll.pixel_rate_pixel_array / - ((sensor->pa_src.width + sensor->hblank->val) * - (sensor->pa_src.height + sensor->vblank->val) / 100)); + ((pa_src->width + sensor->hblank->val) * + (pa_src->height + sensor->vblank->val) / 100)); return 0; } @@ -1775,6 +1791,16 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, u64 streams_mask) { struct ccs_sensor *sensor = to_ccs_sensor(subdev); + struct v4l2_subdev_state *pa_state = + v4l2_subdev_get_locked_active_state(&sensor->pixel_array->sd); + struct v4l2_subdev_state *src_state = + v4l2_subdev_get_locked_active_state(&sensor->src->sd); + struct v4l2_rect *pa_src = + v4l2_subdev_get_crop_ptr(&sensor->pixel_array->sd, pa_state, + CCS_PA_PAD_SRC, CCS_STREAM_PIXEL); + struct v4l2_rect *src_src = + v4l2_subdev_get_crop_ptr(&sensor->src->sd, src_state, + CCS_PAD_SRC, CCS_STREAM_PIXEL); struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd); unsigned int binning_mode; int rval; @@ -1822,22 +1848,20 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, goto err_pm_put; /* Analog crop start coordinates */ - rval = ccs_write(sensor, X_ADDR_START, sensor->pa_src.left); + rval = ccs_write(sensor, X_ADDR_START, pa_src->left); if (rval < 0) goto err_pm_put; - rval = ccs_write(sensor, Y_ADDR_START, sensor->pa_src.top); + rval = ccs_write(sensor, Y_ADDR_START, pa_src->top); if (rval < 0) goto err_pm_put; /* Analog crop end coordinates */ - rval = ccs_write(sensor, X_ADDR_END, - sensor->pa_src.left + sensor->pa_src.width - 1); + rval = ccs_write(sensor, X_ADDR_END, pa_src->left + pa_src->width - 1); if (rval < 0) goto err_pm_put; - rval = ccs_write(sensor, Y_ADDR_END, - sensor->pa_src.top + sensor->pa_src.height - 1); + rval = ccs_write(sensor, Y_ADDR_END, pa_src->top + pa_src->height - 1); if (rval < 0) goto err_pm_put; @@ -1849,23 +1873,31 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, /* Digital crop */ if (CCS_LIM(sensor, DIGITAL_CROP_CAPABILITY) == CCS_DIGITAL_CROP_CAPABILITY_INPUT_CROP) { + struct v4l2_subdev_state *scaler_state = + v4l2_subdev_get_locked_active_state(&sensor->scaler->sd); + struct v4l2_rect *scaler_sink = + v4l2_subdev_get_crop_ptr(&sensor->scaler->sd, + scaler_state, + sensor->scaler->sink_pad, + CCS_STREAM_PIXEL); + rval = ccs_write(sensor, DIGITAL_CROP_X_OFFSET, - sensor->scaler_sink.left); + scaler_sink->left); if (rval < 0) goto err_pm_put; rval = ccs_write(sensor, DIGITAL_CROP_Y_OFFSET, - sensor->scaler_sink.top); + scaler_sink->top); if (rval < 0) goto err_pm_put; rval = ccs_write(sensor, DIGITAL_CROP_IMAGE_WIDTH, - sensor->scaler_sink.width); + scaler_sink->width); if (rval < 0) goto err_pm_put; rval = ccs_write(sensor, DIGITAL_CROP_IMAGE_HEIGHT, - sensor->scaler_sink.height); + scaler_sink->height); if (rval < 0) goto err_pm_put; } @@ -1883,10 +1915,10 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev, } /* Output size from sensor */ - rval = ccs_write(sensor, X_OUTPUT_SIZE, sensor->src_src.width); + rval = ccs_write(sensor, X_OUTPUT_SIZE, src_src->width); if (rval < 0) goto err_pm_put; - rval = ccs_write(sensor, Y_OUTPUT_SIZE, sensor->src_src.height); + rval = ccs_write(sensor, Y_OUTPUT_SIZE, src_src->height); if (rval < 0) goto err_pm_put; @@ -2075,9 +2107,6 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev, struct ccs_sensor *sensor = to_ccs_sensor(subdev); unsigned int i; int idx = -1; - int rval = -EINVAL; - - mutex_lock(&sensor->mutex); dev_err(&client->dev, "subdev %s, pad %u, index %u\n", subdev->name, code->pad, code->index); @@ -2085,12 +2114,11 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev, if (subdev == &sensor->src->sd) { if (code->pad == CCS_PAD_META) { if (code->index) - goto out; + return -EINVAL; code->code = MEDIA_BUS_FMT_CCS_EMBEDDED; - rval = 0; - goto out; + return 0; } if (code->stream == CCS_STREAM_META) { struct v4l2_mbus_framefmt *pix_fmt = @@ -2137,21 +2165,21 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev, } if (WARN_ON(i > ARRAY_SIZE(codes)) || code->index >= i) - goto out; + return -EINVAL; code->code = codes[code->index]; - rval = 0; - goto out; + + return 0; } } if (subdev != &sensor->src->sd || code->pad != CCS_PAD_SRC) { if (code->index) - goto out; + return -EINVAL; code->code = sensor->internal_csi_format->code; - rval = 0; - goto out; + + return 0; } for (i = 0; i < ARRAY_SIZE(ccs_csi_data_formats); i++) { @@ -2162,18 +2190,14 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev, code->code = ccs_csi_data_formats[i].code; dev_err(&client->dev, "found index %u, i %u, code %x\n", code->index, i, code->code); - rval = 0; - break; + return 0; } } -out: - mutex_unlock(&sensor->mutex); - - return rval; + return -EINVAL; } -static u32 __ccs_get_mbus_code(struct v4l2_subdev *subdev, unsigned int pad) +static u32 ccs_get_mbus_code(struct v4l2_subdev *subdev, unsigned int pad) { struct ccs_sensor *sensor = to_ccs_sensor(subdev); @@ -2183,33 +2207,19 @@ static u32 __ccs_get_mbus_code(struct v4l2_subdev *subdev, unsigned int pad) return sensor->internal_csi_format->code; } -static int __ccs_get_format(struct v4l2_subdev *subdev, - struct v4l2_subdev_state *sd_state, - struct v4l2_subdev_format *fmt) +static int ccs_get_format(struct v4l2_subdev *subdev, + struct v4l2_subdev_state *sd_state, + struct v4l2_subdev_format *fmt) { fmt->format = *v4l2_subdev_get_fmt_ptr(subdev, sd_state, fmt->pad, fmt->stream); if (fmt->pad != CCS_PAD_META && fmt->stream != CCS_STREAM_META) - fmt->format.code = __ccs_get_mbus_code(subdev, fmt->pad); + fmt->format.code = ccs_get_mbus_code(subdev, fmt->pad); return 0; } -static int ccs_get_format(struct v4l2_subdev *subdev, - struct v4l2_subdev_state *sd_state, - struct v4l2_subdev_format *fmt) -{ - struct ccs_sensor *sensor = to_ccs_sensor(subdev); - int rval; - - mutex_lock(&sensor->mutex); - rval = __ccs_get_format(subdev, sd_state, fmt); - mutex_unlock(&sensor->mutex); - - return rval; -} - /* Changes require propagation only on sink pad. */ static void ccs_propagate(struct v4l2_subdev *subdev, struct v4l2_subdev_state *sd_state, int which, @@ -2235,7 +2245,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev, sensor->scale_m = CCS_LIM(sensor, SCALER_N_MIN); sensor->scaling_mode = CCS_SCALING_MODE_NO_SCALING; - sensor->scaler_sink = *comp; } else if (ssd == sensor->binner) { sensor->binning_horizontal = 1; sensor->binning_vertical = 1; @@ -2244,8 +2253,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev, fallthrough; case V4L2_SEL_TGT_COMPOSE: *crops[CCS_PAD_SRC] = *comp; - if (which == V4L2_SUBDEV_FORMAT_ACTIVE && ssd == sensor->src) - sensor->src_src = *crops[CCS_PAD_SRC]; break; default: WARN_ON_ONCE(1); @@ -2264,7 +2271,7 @@ static int ccs_set_format_source(struct v4l2_subdev *subdev, unsigned int i; int rval; - rval = __ccs_get_format(subdev, sd_state, fmt); + rval = ccs_get_format(subdev, sd_state, fmt); if (rval) return rval; @@ -2393,13 +2400,9 @@ static int ccs_set_format(struct v4l2_subdev *subdev, if (subdev == &sensor->src->sd && fmt->pad == CCS_PAD_META) return ccs_get_format(subdev, sd_state, fmt); - mutex_lock(&sensor->mutex); - if (subdev == &sensor->src->sd && fmt->stream == CCS_STREAM_META) { ccs_set_format_meta(subdev, sd_state, &fmt->format); - mutex_unlock(&sensor->mutex); - return 0; } @@ -2409,13 +2412,12 @@ static int ccs_set_format(struct v4l2_subdev *subdev, rval = ccs_set_format_source(subdev, sd_state, fmt); ccs_set_format_meta(subdev, sd_state, NULL); - mutex_unlock(&sensor->mutex); - return rval; } /* Sink pad. Width and height are changeable here. */ - fmt->format.code = __ccs_get_mbus_code(subdev, fmt->pad); + fmt->format.code = ccs_get_mbus_code(subdev, fmt->pad); + fmt->format.width &= ~1; fmt->format.height &= ~1; fmt->format.field = V4L2_FIELD_NONE; @@ -2438,8 +2440,6 @@ static int ccs_set_format(struct v4l2_subdev *subdev, crop->height = fmt->format.height; ccs_propagate(subdev, sd_state, fmt->which, V4L2_SEL_TGT_CROP); - mutex_unlock(&sensor->mutex); - return 0; } @@ -2753,9 +2753,6 @@ static int ccs_set_crop(struct v4l2_subdev *subdev, if (ssd != sensor->pixel_array && sel->pad == CCS_PAD_SINK) ccs_propagate(subdev, sd_state, sel->which, V4L2_SEL_TGT_CROP); - else if (sel->which == V4L2_SUBDEV_FORMAT_ACTIVE && - ssd == sensor->pixel_array) - sensor->pa_src = sel->r; return 0; } @@ -2827,8 +2824,6 @@ static int ccs_set_selection(struct v4l2_subdev *subdev, if (ret) return ret; - mutex_lock(&sensor->mutex); - sel->r.left = max(0, sel->r.left & ~1); sel->r.top = max(0, sel->r.top & ~1); sel->r.width = CCS_ALIGN_DIM(sel->r.width, sel->flags); @@ -2850,7 +2845,6 @@ static int ccs_set_selection(struct v4l2_subdev *subdev, ret = -EINVAL; } - mutex_unlock(&sensor->mutex); return ret; } @@ -3235,6 +3229,7 @@ static int ccs_init_subdev(struct ccs_sensor *sensor, ssd->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; ssd->sd.entity.function = function; + ssd->sd.state_lock = &sensor->mutex; ssd->sensor = sensor; ssd->npads = num_pads; @@ -3281,8 +3276,6 @@ static int ccs_init_cfg(struct v4l2_subdev *sd, struct ccs_sensor *sensor = ssd->sensor; unsigned int i; - mutex_lock(&sensor->mutex); - for (i = 0; i < ssd->npads; i++) { struct v4l2_mbus_framefmt *fmt = v4l2_subdev_get_fmt_ptr(sd, sd_state, i, @@ -3310,8 +3303,6 @@ static int ccs_init_cfg(struct v4l2_subdev *sd, *comp = *crop; } - mutex_unlock(&sensor->mutex); - return 0; } diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h index 5e561cc91717..47c77f3284c8 100644 --- a/drivers/media/i2c/ccs/ccs.h +++ b/drivers/media/i2c/ccs/ccs.h @@ -228,7 +228,6 @@ struct ccs_sensor { u32 mbus_frame_fmts; const struct ccs_csi_data_format *csi_format; const struct ccs_csi_data_format *internal_csi_format; - struct v4l2_rect pa_src, scaler_sink, src_src; u32 default_mbus_frame_fmts; int default_pixel_order; struct ccs_data_container sdata, mdata;