From patchwork Thu Jul 27 07:15:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bingbu Cao X-Patchwork-Id: 710013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DE0CEB64DD for ; Thu, 27 Jul 2023 07:12:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233262AbjG0HMq (ORCPT ); Thu, 27 Jul 2023 03:12:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232688AbjG0HMY (ORCPT ); Thu, 27 Jul 2023 03:12:24 -0400 Received: from mgamail.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57F7959D3 for ; Thu, 27 Jul 2023 00:07:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690441621; x=1721977621; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nfnosXBr8oiyDYpL9QCqNCugs2jC/dM7h4AXByU/dRk=; b=b7kqBeeXdS1DvxGoHMp/4W+Bg3T6jIWKUgzxDsgG8e6dJ0M7PLyxh4ce OprQrcExuRweMacM7gfZS+E7GCeW82+PTZxuPy9UFW4hed05oKbUUu9lE x2gZvsVDa6tBeLohrSaDpU6m/aR20zBIrQXvcXJLRO3YYjUjPBPI3e4Ma zIpW2eSnHEXLgy03LOizo0NH8F1LfACh0+IddyyeaMoOmVEope3fdgRer ajm1fTx4wj1r7oRVd1j5ebEnU1WEB4FA7k80BBc8kDM1IB59ogKUDja2c 7PzfGkSGivthFKsB8+OU5WhboAg3ry4OGcOLLQxgx+5WynhMaAzuFSlcn w==; X-IronPort-AV: E=McAfee;i="6600,9927,10783"; a="370901040" X-IronPort-AV: E=Sophos;i="6.01,234,1684825200"; d="scan'208";a="370901040" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jul 2023 00:05:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10783"; a="704073359" X-IronPort-AV: E=Sophos;i="6.01,234,1684825200"; d="scan'208";a="704073359" Received: from icg-kernel3.bj.intel.com ([172.16.126.100]) by orsmga006.jf.intel.com with ESMTP; 27 Jul 2023 00:05:02 -0700 From: bingbu.cao@intel.com To: linux-media@vger.kernel.org, sakari.ailus@linux.intel.com, laurent.pinchart@ideasonboard.com Cc: ilpo.jarvinen@linux.intel.com, tfiga@chromium.org, senozhatsky@chromium.org, andriy.shevchenko@linux.intel.com, hdegoede@redhat.com, tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, bingbu.cao@linux.intel.com, tian.shu.qiu@intel.com, hongju.wang@intel.com Subject: [PATCH 14/15] Documentation: add Intel IPU6 ISYS driver admin-guide doc Date: Thu, 27 Jul 2023 15:15:57 +0800 Message-Id: <20230727071558.1148653-15-bingbu.cao@intel.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230727071558.1148653-1-bingbu.cao@intel.com> References: <20230727071558.1148653-1-bingbu.cao@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Bingbu Cao This document mainly describe the functionality of IPU6 and IPU6 isys driver, and gives an example that how user can do imaging capture with tools. Signed-off-by: Bingbu Cao --- Documentation/admin-guide/media/ipu6-isys.rst | 138 ++ .../admin-guide/media/ipu6_isys_graph.svg | 338 +++++ .../admin-guide/media/ipu6_isys_multi.svg | 1124 +++++++++++++++++ .../admin-guide/media/v4l-drivers.rst | 1 + 4 files changed, 1601 insertions(+) create mode 100644 Documentation/admin-guide/media/ipu6-isys.rst create mode 100644 Documentation/admin-guide/media/ipu6_isys_graph.svg create mode 100644 Documentation/admin-guide/media/ipu6_isys_multi.svg diff --git a/Documentation/admin-guide/media/ipu6-isys.rst b/Documentation/admin-guide/media/ipu6-isys.rst new file mode 100644 index 000000000000..ad1dd9ab892a --- /dev/null +++ b/Documentation/admin-guide/media/ipu6-isys.rst @@ -0,0 +1,138 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. include:: + +=============================================================== +Intel Image Processing Unit 6 (IPU6) Input System driver +=============================================================== + +Copyright |copy| 2023 Intel Corporation + +Introduction +============ + +This file documents the Intel IPU6 (6th generation Image Processing Unit) +Input System (MIPI CSI2 receiver) drivers located under +drivers/media/pci/intel/ipu6. + +The Intel IPU6 can be found in certain Intel Chipsets but not in all SKUs: + +* TigerLake +* JasperLake +* AlderLake +* RaptorLake +* MeteorLake + +Intel IPU6 is made up of two components - Input System (ISYS) and Processing +System (PSYS). + +The Input System mainly works as MIPI CSI2 receiver which receives and +processes the imaging data from the sensors and outputs the frames to memory. + +There are 2 driver modules - intel_ipu6 and intel_ipu6_isys. intel_ipu6 is an +IPU6 common driver which does PCI configuration, firmware loading and parsing, +firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit) +configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2 +sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected +to the IPU6 ISYS through V4L2 sub-device sensor drivers. + +.. Note:: See Documentation/driver-api/media/drivers/ipu6.rst for more + information about the IPU6 hardware. + + +Input system driver +=================== + +The input System driver mainly configures CSI2 DPHY, constructs the firmware +stream configuration, sends commands to firmware, gets response from hardware +and firmware and then returns buffers to user. +The ISYS is represented as several V4L2 sub-devices - 'Intel IPU6 CSI2 $port', +which provide V4L2 subdev interfaces to the user space, there are also several +video nodes for each CSI-2 stream capture - 'Intel IPU6 ISYS capture $num' which +provide interface to user to set formats, queue buffers and streaming. + +.. kernel-figure:: ipu6_isys_graph.svg + :alt: ipu6 isys media graph without multiple streams support + + ipu6 isys media graph without multiple streams support + +.. kernel-figure:: ipu6_isys_multi.svg + :alt: ipu6 isys media graph with multiple streams support + + ipu6 isys media graph with multiple streams support + +Capturing frames by IPU6 ISYS +------------------------------------ + +IPU6 ISYS is used to capture frames from the camera sensors connected to the +CSI2 ports. The supported input formats of ISYS are listed in table below: + +.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}| + +.. flat-table:: + :header-rows: 1 + + * - IPU6 ISYS supported input formats + + * - RGB565, RGB888 + + * - UYVY8, YUYV8 + + * - RAW8, RAW10, RAW12 + +Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this +machine, ov01a10 sensor is connected to IPU ISYS CSI2 port 2, which can +generate images at sBGGR10 with resolution 1280x800. + +Using the media controller APIs, we can configure ov01a10 sensor by +media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS. + +.. code-block:: none + + # This example assumes /dev/media0 as the IPU ISYS media device + export MDEV=/dev/media0 + + # Establish the link for the media devices using media-ctl + media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" + + # Set the format for the media devices + media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]" + media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]" + media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]" + + # Establish the link for the media devices using media-ctl + media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" + media-ctl -d $MDEV -l "\"Intel IPU6 CSI2 2\":1 ->\"Intel IPU6 ISYS Capture 0\":0[5]" + +Once the media pipeline is configured, desired sensor specific settings +(such as exposure and gain settings) can be set, using the yavta tool. + +e.g + +.. code-block:: none + + # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36 + export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036") + + yavta -w 0x009e0903 400 $SDEV + yavta -w 0x009e0913 1000 $SDEV + yavta -w 0x009e0911 2000 $SDEV + +Once the desired sensor settings are set, frame captures can be done as below. + +e.g + +.. code-block:: none + + yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \ + -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0") + +With the above command, 10 frames are captured at 1280x800 resolution with +sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files. + + +References +========== + +.. [#f1] https://git.ideasonboard.org/?p=media-ctl.git;a=summary +.. [#f2] https://git.ideasonboard.org/yavta.git diff --git a/Documentation/admin-guide/media/ipu6_isys_graph.svg b/Documentation/admin-guide/media/ipu6_isys_graph.svg new file mode 100644 index 000000000000..661aee18dbe2 --- /dev/null +++ b/Documentation/admin-guide/media/ipu6_isys_graph.svg @@ -0,0 +1,338 @@ + + + + + + +board + + + +n00000001 + +Intel IPU6 ISYS Capture 0 +/dev/video0 + + + +n00000005 + +Intel IPU6 ISYS Capture 1 +/dev/video1 + + + +n00000009 + +Intel IPU6 ISYS Capture 2 +/dev/video2 + + + +n0000000d + +Intel IPU6 ISYS Capture 3 +/dev/video3 + + + +n00000011 + +0 + +Intel IPU6 CSI2 0 +/dev/v4l-subdev0 + +1 + + + +n00000011:port1->n00000001 + + + + + +n00000011:port1->n00000005 + + + + + +n00000011:port1->n00000009 + + + + + +n00000011:port1->n0000000d + + + + + +n00000014 + +0 + +Intel IPU6 CSI2 1 +/dev/v4l-subdev1 + +1 + + + +n00000014:port1->n00000001 + + + + + +n00000014:port1->n00000005 + + + + + +n00000014:port1->n00000009 + + + + + +n00000014:port1->n0000000d + + + + + +n00000017 + +0 + +Intel IPU6 CSI2 2 +/dev/v4l-subdev2 + +1 + + + +n00000017:port1->n00000001 + + + + + +n00000017:port1->n00000005 + + + + + +n00000017:port1->n00000009 + + + + + +n00000017:port1->n0000000d + + + + + +n0000001a + +0 + +Intel IPU6 CSI2 3 +/dev/v4l-subdev3 + +1 + + + +n0000001a:port1->n00000001 + + + + + +n0000001a:port1->n00000005 + + + + + +n0000001a:port1->n00000009 + + + + + +n0000001a:port1->n0000000d + + + + + +n0000001d + +0 + +Intel IPU6 CSI2 4 +/dev/v4l-subdev4 + +1 + + + +n0000001d:port1->n00000001 + + + + + +n0000001d:port1->n00000005 + + + + + +n0000001d:port1->n00000009 + + + + + +n0000001d:port1->n0000000d + + + + + +n00000020 + +0 + +Intel IPU6 CSI2 5 +/dev/v4l-subdev5 + +1 + + + +n00000020:port1->n00000001 + + + + + +n00000020:port1->n00000005 + + + + + +n00000020:port1->n00000009 + + + + + +n00000020:port1->n0000000d + + + + + +n00000023 + +0 + +Intel IPU6 CSI2 6 +/dev/v4l-subdev6 + +1 + + + +n00000023:port1->n00000001 + + + + + +n00000023:port1->n00000005 + + + + + +n00000023:port1->n00000009 + + + + + +n00000023:port1->n0000000d + + + + + +n00000026 + +0 + +Intel IPU6 CSI2 7 +/dev/v4l-subdev7 + +1 + + + +n00000026:port1->n00000001 + + + + + +n00000026:port1->n00000005 + + + + + +n00000026:port1->n00000009 + + + + + +n00000026:port1->n0000000d + + + + + +n00000069 + + + +ov01a10 3-0036 +/dev/v4l-subdev8 + +0 + + + +n00000069:port0->n00000017:port0 + + + + + diff --git a/Documentation/admin-guide/media/ipu6_isys_multi.svg b/Documentation/admin-guide/media/ipu6_isys_multi.svg new file mode 100644 index 000000000000..5bd748d15c70 --- /dev/null +++ b/Documentation/admin-guide/media/ipu6_isys_multi.svg @@ -0,0 +1,1124 @@ + + + + + + +board + + + +n00000001 + +Intel IPU6 ISYS Capture 0 +/dev/video0 + + + +n00000005 + +Intel IPU6 ISYS Capture 1 +/dev/video1 + + + +n00000009 + +Intel IPU6 ISYS Capture ... + + + +n00000075 + +Intel IPU6 ISYS Capture 29 +/dev/video29 + + + +n00000079 + +Intel IPU6 ISYS Capture 30 +/dev/video30 + + + +n0000007d + +0 + +Intel IPU6 CSI2 0 +/dev/v4l-subdev0 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + + +n0000007d:port1->n00000001 + + + + + +n0000007d:port2->n00000001 + + + + + +n0000007d:port3->n00000001 + + + + + +n0000007d:port4->n00000001 + + + + + +n0000007d:port5->n00000001 + + + + + +n0000007d:port6->n00000001 + + + + + +n0000007d:port7->n00000001 + + + + + +n0000007d:port8->n00000001 + + + + + +n0000007d:port1->n00000005 + + + + + +n0000007d:port2->n00000005 + + + + + +n0000007d:port3->n00000005 + + + + + +n0000007d:port4->n00000005 + + + + + +n0000007d:port5->n00000005 + + + + + +n0000007d:port6->n00000005 + + + + + +n0000007d:port7->n00000005 + + + + + +n0000007d:port8->n00000005 + + + + + +n0000007d:port1->n00000009 + + + + + +n0000007d:port2->n00000009 + + + + + +n0000007d:port3->n00000009 + + + + + +n0000007d:port4->n00000009 + + + + + +n0000007d:port5->n00000009 + + + + + +n0000007d:port6->n00000009 + + + + + +n0000007d:port7->n00000009 + + + + + +n0000007d:port8->n00000009 + + + + + +n0000007d:port1->n00000075 + + + + + +n0000007d:port2->n00000075 + + + + + +n0000007d:port3->n00000075 + + + + + +n0000007d:port4->n00000075 + + + + + +n0000007d:port5->n00000075 + + + + + +n0000007d:port6->n00000075 + + + + + +n0000007d:port7->n00000075 + + + + + +n0000007d:port8->n00000075 + + + + + +n0000007d:port1->n00000079 + + + + + +n0000007d:port2->n00000079 + + + + + +n0000007d:port3->n00000079 + + + + + +n0000007d:port4->n00000079 + + + + + +n0000007d:port5->n00000079 + + + + + +n0000007d:port6->n00000079 + + + + + +n0000007d:port7->n00000079 + + + + + +n0000007d:port8->n00000079 + + + + + +n00000087 + +0 + +Intel IPU6 CSI2 1 +/dev/v4l-subdev1 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + + +n00000087:port1->n00000001 + + + + + +n00000087:port2->n00000001 + + + + + +n00000087:port3->n00000001 + + + + + +n00000087:port4->n00000001 + + + + + +n00000087:port5->n00000001 + + + + + +n00000087:port6->n00000001 + + + + + +n00000087:port7->n00000001 + + + + + +n00000087:port8->n00000001 + + + + + +n00000087:port1->n00000005 + + + + + +n00000087:port2->n00000005 + + + + + +n00000087:port3->n00000005 + + + + + +n00000087:port4->n00000005 + + + + + +n00000087:port5->n00000005 + + + + + +n00000087:port6->n00000005 + + + + + +n00000087:port7->n00000005 + + + + + +n00000087:port8->n00000005 + + + + + +n00000087:port1->n00000009 + + + + + +n00000087:port2->n00000009 + + + + + +n00000087:port3->n00000009 + + + + + +n00000087:port4->n00000009 + + + + + +n00000087:port5->n00000009 + + + + + +n00000087:port6->n00000009 + + + + + +n00000087:port7->n00000009 + + + + + +n00000087:port8->n00000009 + + + + + +n00000087:port1->n00000075 + + + + + +n00000087:port2->n00000075 + + + + + +n00000087:port3->n00000075 + + + + + +n00000087:port4->n00000075 + + + + + +n00000087:port5->n00000075 + + + + + +n00000087:port6->n00000075 + + + + + +n00000087:port7->n00000075 + + + + + +n00000087:port8->n00000075 + + + + + +n00000087:port1->n00000079 + + + + + +n00000087:port2->n00000079 + + + + + +n00000087:port3->n00000079 + + + + + +n00000087:port4->n00000079 + + + + + +n00000087:port5->n00000079 + + + + + +n00000087:port6->n00000079 + + + + + +n00000087:port7->n00000079 + + + + + +n00000087:port8->n00000079 + + + + + +n00000091 + +0 + +Intel IPU6 CSI2 2 +/dev/v4l-subdev2 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + + +n00000091:port1->n00000001 + + + + + +n00000091:port2->n00000001 + + + + + +n00000091:port3->n00000001 + + + + + +n00000091:port4->n00000001 + + + + + +n00000091:port5->n00000001 + + + + + +n00000091:port6->n00000001 + + + + + +n00000091:port7->n00000001 + + + + + +n00000091:port8->n00000001 + + + + + +n00000091:port1->n00000005 + + + + + +n00000091:port2->n00000005 + + + + + +n00000091:port3->n00000005 + + + + + +n00000091:port4->n00000005 + + + + + +n00000091:port5->n00000005 + + + + + +n00000091:port6->n00000005 + + + + + +n00000091:port7->n00000005 + + + + + +n00000091:port8->n00000005 + + + + + +n00000091:port1->n00000009 + + + + + +n00000091:port2->n00000009 + + + + + +n00000091:port3->n00000009 + + + + + +n00000091:port4->n00000009 + + + + + +n00000091:port5->n00000009 + + + + + +n00000091:port6->n00000009 + + + + + +n00000091:port7->n00000009 + + + + + +n00000091:port8->n00000009 + + + + + +n00000091:port1->n00000075 + + + + + +n00000091:port2->n00000075 + + + + + +n00000091:port3->n00000075 + + + + + +n00000091:port4->n00000075 + + + + + +n00000091:port5->n00000075 + + + + + +n00000091:port6->n00000075 + + + + + +n00000091:port7->n00000075 + + + + + +n00000091:port8->n00000075 + + + + + +n00000091:port1->n00000079 + + + + + +n00000091:port2->n00000079 + + + + + +n00000091:port3->n00000079 + + + + + +n00000091:port4->n00000079 + + + + + +n00000091:port5->n00000079 + + + + + +n00000091:port6->n00000079 + + + + + +n00000091:port7->n00000079 + + + + + +n00000091:port8->n00000079 + + + + + +n0000009b + +0 + +Intel IPU6 CSI2 3 +/dev/v4l-subdev3 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + + +n0000009b:port1->n00000001 + + + + + +n0000009b:port2->n00000001 + + + + + +n0000009b:port3->n00000001 + + + + + +n0000009b:port4->n00000001 + + + + + +n0000009b:port5->n00000001 + + + + + +n0000009b:port6->n00000001 + + + + + +n0000009b:port7->n00000001 + + + + + +n0000009b:port8->n00000001 + + + + + +n0000009b:port1->n00000005 + + + + + +n0000009b:port2->n00000005 + + + + + +n0000009b:port3->n00000005 + + + + + +n0000009b:port4->n00000005 + + + + + +n0000009b:port5->n00000005 + + + + + +n0000009b:port6->n00000005 + + + + + +n0000009b:port7->n00000005 + + + + + +n0000009b:port8->n00000005 + + + + + +n0000009b:port1->n00000009 + + + + + +n0000009b:port2->n00000009 + + + + + +n0000009b:port3->n00000009 + + + + + +n0000009b:port4->n00000009 + + + + + +n0000009b:port5->n00000009 + + + + + +n0000009b:port6->n00000009 + + + + + +n0000009b:port7->n00000009 + + + + + +n0000009b:port8->n00000009 + + + + + +n0000009b:port1->n00000075 + + + + + +n0000009b:port2->n00000075 + + + + + +n0000009b:port3->n00000075 + + + + + +n0000009b:port4->n00000075 + + + + + +n0000009b:port5->n00000075 + + + + + +n0000009b:port6->n00000075 + + + + + +n0000009b:port7->n00000075 + + + + + +n0000009b:port8->n00000075 + + + + + +n0000009b:port1->n00000079 + + + + + +n0000009b:port2->n00000079 + + + + + +n0000009b:port3->n00000079 + + + + + +n0000009b:port4->n00000079 + + + + + +n0000009b:port5->n00000079 + + + + + +n0000009b:port6->n00000079 + + + + + +n0000009b:port7->n00000079 + + + + + +n0000009b:port8->n00000079 + + + + + +n00000865 + + + +ov01a10 3-0036 +/dev/v4l-subdev4 + +0 + + + +n00000865:port0->n00000091:port0 + + + + + diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index 1c41f87c3917..f6328a242cbe 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -16,6 +16,7 @@ Video4Linux (V4L) driver-specific documentation imx imx7 ipu3 + ipu6-isys ivtv omap3isp omap4_camera