From patchwork Thu Apr 13 10:04:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bingbu Cao X-Patchwork-Id: 674368 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E60AC77B6C for ; Thu, 13 Apr 2023 09:55:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230115AbjDMJzu (ORCPT ); Thu, 13 Apr 2023 05:55:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229722AbjDMJzt (ORCPT ); Thu, 13 Apr 2023 05:55:49 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7AB998A6A for ; Thu, 13 Apr 2023 02:55:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681379730; x=1712915730; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wBHU6pWtqCcZ+MLmEquS+uZP2DaNANFrhgCqLDUpPzs=; b=awNfQ4iNpFZB904/wKYsMgJyzERgaMSm1zYK/mg+O9+qYL3iNZYOYK0n i1hY1p6mRe36IvtcjXkTz7cXAI1gJuRgsDZzzwypz2z1bwyGx5q728Wcj 0wwGdAUvRffwysoIMoCW7rxDMn3ItZHbqI1X+UicTLFVWHqkHFUkaFKnx PofLV8Z8Mp5KlKp44HhxSGeM+MydwIJmyxAChJRlJX5CBzYY4X5gf70Ig PoOpRsD6KV/2CK8OBZNeos4j5woUlVLEbUKhqHWdQktXcAAkvcn91DfJP sfFUq9V8hvid4FOYJP4DASnvkuz2rcBz2kZa+VL2a43uzE7jsUtkUvUc/ g==; X-IronPort-AV: E=McAfee;i="6600,9927,10678"; a="371993135" X-IronPort-AV: E=Sophos;i="5.98,341,1673942400"; d="scan'208";a="371993135" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Apr 2023 02:55:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10678"; a="639600233" X-IronPort-AV: E=Sophos;i="5.98,341,1673942400"; d="scan'208";a="639600233" Received: from icg-kernel3.bj.intel.com ([172.16.126.100]) by orsmga003.jf.intel.com with ESMTP; 13 Apr 2023 02:55:18 -0700 From: bingbu.cao@intel.com To: linux-media@vger.kernel.org, sakari.ailus@linux.intel.com, laurent.pinchart@ideasonboard.com, ilpo.jarvinen@linux.intel.com Cc: tfiga@chromium.org, senozhatsky@chromium.org, hdegoede@redhat.com, bingbu.cao@intel.com, bingbu.cao@linux.intel.com, tian.shu.qiu@intel.com, hongju.wang@intel.com, daniel.h.kang@intel.com Subject: [RFC PATCH 13/14] Documentation: add Intel IPU6 ISYS driver admin-guide doc Date: Thu, 13 Apr 2023 18:04:28 +0800 Message-Id: <20230413100429.919622-14-bingbu.cao@intel.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230413100429.919622-1-bingbu.cao@intel.com> References: <20230413100429.919622-1-bingbu.cao@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Bingbu Cao This document mainly describe the functionality of IPU6 and IPU6 isys driver, and gives an example that how user can do imaging capture with tools. Signed-off-by: Bingbu Cao --- Documentation/admin-guide/media/ipu6-isys.rst | 128 +++++++ .../admin-guide/media/ipu6_isys_graph.svg | 338 ++++++++++++++++++ .../admin-guide/media/v4l-drivers.rst | 1 + 3 files changed, 467 insertions(+) create mode 100644 Documentation/admin-guide/media/ipu6-isys.rst create mode 100644 Documentation/admin-guide/media/ipu6_isys_graph.svg diff --git a/Documentation/admin-guide/media/ipu6-isys.rst b/Documentation/admin-guide/media/ipu6-isys.rst new file mode 100644 index 000000000000..083b00449b86 --- /dev/null +++ b/Documentation/admin-guide/media/ipu6-isys.rst @@ -0,0 +1,128 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. include:: + +=============================================================== +Intel Image Processing Unit 6 (IPU6) Input System driver +=============================================================== + +Copyright |copy| 2023 Intel Corporation + +Introduction +============ + +This file documents the Intel IPU6 (6th generation Image Processing Unit) +Input System (MIPI CSI2 receiver) drivers located under +drivers/media/pci/intel/ipu6. + + +The Intel IPU6 can be found in certain Intel Chipsets but not in all SKUs: + +* TigerLake +* JasperLake +* AlderLake +* RaptorLake +* MeteorLake + +Intel IPU6 is made up of two components - Input System (ISYS) and Processing +System (PSYS). + +The Input System mainly work as MIPI CSI2 receiver which receive and process the +imaging data from the sensors and outputs the frames to memory. + +There are 2 driver modules - intel_ipu6 and intel_ipu6_isys. intel_ipu6 is IPU6 +common driver which does PCI configuration, firmware loading and parsing, +firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit) +configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2 +sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected +to the IPU6 ISYS through V4L2 sub-device sensor drivers. + +Input system driver +=================== + +Input System driver mainly configure CSI2 DPHY, construct the firmware stream +configuration and send commands to firmware and get response from hardware and +firmware and then return buffers to user. +The ISYS is represented as several V4L2 sub-devices - 'Intel IPU6 CSI2 $port', +which provide V4L2 subdev interfaces to the user space, there are also several +video nodes for each CSI-2 stream capture - 'Intel IPU6 ISYS capture $num' which +provide interface to user to set formats, queue buffers and streaming. + +.. kernel-figure:: ipu6_isys_graph.svg + :alt: ipu6 isys media graph + +Capturing frames by IPU6 ISYS +------------------------------------ + +IPU6 ISYS is used to capture frames from the camera sensors connected to the +CSI2 port. The supported input formats of ISYS are listed in table below: + +.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}| + +.. flat-table:: + :header-rows: 1 + + * - IPU6 ISYS supported input formats + + * - RGB565, RGB888 + + * - UYVY8, YUYV8 + + * - RAW8, RAW10, RAW12 + +Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this +machine, ov01a10 sensor is connected to IPU ISYS CSI2 port 2, which can +generate images at sBGGR10 with resolution 1280x800. + +Using the media controller APIs, we can configure ov01a10 sensor by +media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS. + +.. code-block:: none + + # This example assumes /dev/media0 as the IPU ISYS media device + export MDEV=/dev/media0 + + # Establish the link for the media devices using media-ctl + media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" + + # Set the format for the media devices + media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]" + media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]" + media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]" + + # Establish the link for the media devices using media-ctl + media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" + media-ctl -d $MDEV -l "\"Intel IPU6 CSI2 2\":1 ->\"Intel IPU6 ISYS Capture 0\":0[5]" + +Once the media pipeline is configured, desired sensor specific settings +(such as exposure and gain settings) can be set, using the yavta tool. + +e.g + +.. code-block:: none + + # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36 + export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036") + + yavta -w 0x009e0903 400 $SDEV + yavta -w 0x009e0913 1000 $SDEV + yavta -w 0x009e0911 2000 $SDEV + +Once the desired sensor settings are set, frame captures can be done as below. + +e.g + +.. code-block:: none + + yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \ + -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0") + +With the above command, 10 frames are captured at 1280x800 resolution with +sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files. + + +References +========== + +.. [#f1] https://git.ideasonboard.org/?p=media-ctl.git;a=summary +.. [#f2] https://git.ideasonboard.org/yavta.git diff --git a/Documentation/admin-guide/media/ipu6_isys_graph.svg b/Documentation/admin-guide/media/ipu6_isys_graph.svg new file mode 100644 index 000000000000..661aee18dbe2 --- /dev/null +++ b/Documentation/admin-guide/media/ipu6_isys_graph.svg @@ -0,0 +1,338 @@ + + + + + + +board + + + +n00000001 + +Intel IPU6 ISYS Capture 0 +/dev/video0 + + + +n00000005 + +Intel IPU6 ISYS Capture 1 +/dev/video1 + + + +n00000009 + +Intel IPU6 ISYS Capture 2 +/dev/video2 + + + +n0000000d + +Intel IPU6 ISYS Capture 3 +/dev/video3 + + + +n00000011 + +0 + +Intel IPU6 CSI2 0 +/dev/v4l-subdev0 + +1 + + + +n00000011:port1->n00000001 + + + + + +n00000011:port1->n00000005 + + + + + +n00000011:port1->n00000009 + + + + + +n00000011:port1->n0000000d + + + + + +n00000014 + +0 + +Intel IPU6 CSI2 1 +/dev/v4l-subdev1 + +1 + + + +n00000014:port1->n00000001 + + + + + +n00000014:port1->n00000005 + + + + + +n00000014:port1->n00000009 + + + + + +n00000014:port1->n0000000d + + + + + +n00000017 + +0 + +Intel IPU6 CSI2 2 +/dev/v4l-subdev2 + +1 + + + +n00000017:port1->n00000001 + + + + + +n00000017:port1->n00000005 + + + + + +n00000017:port1->n00000009 + + + + + +n00000017:port1->n0000000d + + + + + +n0000001a + +0 + +Intel IPU6 CSI2 3 +/dev/v4l-subdev3 + +1 + + + +n0000001a:port1->n00000001 + + + + + +n0000001a:port1->n00000005 + + + + + +n0000001a:port1->n00000009 + + + + + +n0000001a:port1->n0000000d + + + + + +n0000001d + +0 + +Intel IPU6 CSI2 4 +/dev/v4l-subdev4 + +1 + + + +n0000001d:port1->n00000001 + + + + + +n0000001d:port1->n00000005 + + + + + +n0000001d:port1->n00000009 + + + + + +n0000001d:port1->n0000000d + + + + + +n00000020 + +0 + +Intel IPU6 CSI2 5 +/dev/v4l-subdev5 + +1 + + + +n00000020:port1->n00000001 + + + + + +n00000020:port1->n00000005 + + + + + +n00000020:port1->n00000009 + + + + + +n00000020:port1->n0000000d + + + + + +n00000023 + +0 + +Intel IPU6 CSI2 6 +/dev/v4l-subdev6 + +1 + + + +n00000023:port1->n00000001 + + + + + +n00000023:port1->n00000005 + + + + + +n00000023:port1->n00000009 + + + + + +n00000023:port1->n0000000d + + + + + +n00000026 + +0 + +Intel IPU6 CSI2 7 +/dev/v4l-subdev7 + +1 + + + +n00000026:port1->n00000001 + + + + + +n00000026:port1->n00000005 + + + + + +n00000026:port1->n00000009 + + + + + +n00000026:port1->n0000000d + + + + + +n00000069 + + + +ov01a10 3-0036 +/dev/v4l-subdev8 + +0 + + + +n00000069:port0->n00000017:port0 + + + + + diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index 1c41f87c3917..f6328a242cbe 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -16,6 +16,7 @@ Video4Linux (V4L) driver-specific documentation imx imx7 ipu3 + ipu6-isys ivtv omap3isp omap4_camera