From patchwork Fri Feb 9 16:48:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 771446 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8D4412837F for ; Fri, 9 Feb 2024 16:48:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707497334; cv=none; b=KBGXNlK2Z6LYS9MjYKc10Cfkxmu9Q47uIKb3LEPMNXlVxekr1XTmgg5dMUz/zAYK+A2aGb+vWEsBn4y+XTWXu0J0GynOhqEZ5Q9/oxJx7rlFCjTlA3ZRc1oBJ1Yk38xBGGWNDo87UWG1yc1CRf7hShWPvO3/qHnQuYUFTlXg3Vw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707497334; c=relaxed/simple; bh=/DFZpj8a/u0yBiSoHCVkkTn7ISv+BZvmglqOpsMlz1E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BhYNSWvUy/Rzg+xi7lmhH05tLexm5/60qxUPthj5ZkLciGQJYxIt8xkOTl6bXUQNHU559FeVACppvwdW9/DJa4YMgCAaw1+7qjep4wUj1vcrkohNRoaSPmGw6OR8Uvpmw+oRQdrVF6XeRVYonX3w5ZXoKi7mr62ZW7LteP7VtPU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=ktkwLIst; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ktkwLIst" Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5D8BF22DD; Fri, 9 Feb 2024 17:47:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1707497241; bh=/DFZpj8a/u0yBiSoHCVkkTn7ISv+BZvmglqOpsMlz1E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ktkwLIstDs1DM48zOFTRIFjqKKp1Niyjir0qzRQwbRT5oPo6eBJBH5IpBXfuqzIvb 4QIdWZv+V47phDx3rdOvI2NB7JZ3dGYP9kbSGPAuShWZ2XKKq8shA/GsK4o0D++02M wQ6ko0NO10WAM9PDaCXit1FDrNUVMkkHG2ZuFa7g= From: Jacopo Mondi To: Linux Media Mailing List Cc: Jacopo Mondi , David Plowman , Naushir Patuck , Nick Hollinghurst , Dave Stevenson , Tomi Valkeinen , Laurent Pinchart , Kieran Bingham , Sakari Ailus , Hans Verkuil , Mauro Carvalho Chehab Subject: [PATCH 3/8] media: Add a pixel format for BRG48 and RGB48 Date: Fri, 9 Feb 2024 17:48:18 +0100 Message-ID: <20240209164825.166800-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240209164825.166800-1-jacopo.mondi@ideasonboard.com> References: <20240209164825.166800-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add BGR48 and RGB48 16-bit per component image formats. Signed-off-by: Jacopo Mondi Reviewed-by: Naushir Patuck --- .../userspace-api/media/v4l/pixfmt-rgb.rst | 39 +++++++++++++++++++ drivers/media/v4l2-core/v4l2-common.c | 2 + drivers/media/v4l2-core/v4l2-ioctl.c | 2 + include/uapi/linux/videodev2.h | 4 ++ 4 files changed, 47 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/pixfmt-rgb.rst b/Documentation/userspace-api/media/v4l/pixfmt-rgb.rst index b71b80d634d6..458308ae4eb8 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-rgb.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-rgb.rst @@ -996,6 +996,45 @@ arranged in little endian order. \normalsize +16 Bits Per Component +===================== + +These formats store an RGB triplet in six bytes, with 16 bits per component. + +.. raw:: latex + + \small + +.. flat-table:: RGB Formats With 16 Bits Per Component + :header-rows: 1 + + * - Identifier + - Code + - Byte 1-0 + - Byte 3-2 + - Byte 5-4 + * .. _V4L2-PIX-FMT-BGR48: + + - ``V4L2_PIX_FMT_BGR48`` + - 'BGR6' + + - B\ :sub:`15-0` + - G\ :sub:`15-0` + - R\ :sub:`15-0` + - + * .. _V4L2-PIX-FMT-RGB48: + + - ``V4L2_PIX_FMT_RGB48`` + - 'RGB6' + + - R\ :sub:`15-0` + - G\ :sub:`15-0` + - B\ :sub:`15-0` + +.. raw:: latex + + \normalsize + Deprecated RGB Formats ====================== diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c index d34d210908d9..ff4b4d2de9d2 100644 --- a/drivers/media/v4l2-core/v4l2-common.c +++ b/drivers/media/v4l2-core/v4l2-common.c @@ -253,6 +253,8 @@ const struct v4l2_format_info *v4l2_format_info(u32 format) { .format = V4L2_PIX_FMT_RGB555, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_BGR666, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_BGR48_12, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 6, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, + { .format = V4L2_PIX_FMT_BGR48, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 6, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, + { .format = V4L2_PIX_FMT_RGB48, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 6, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_ABGR64_12, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 8, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_RGBA1010102, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, { .format = V4L2_PIX_FMT_RGBX1010102, .pixel_enc = V4L2_PIXEL_ENC_RGB, .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 }, diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 24f52485e59c..3c9b5b2f456c 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1298,6 +1298,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_RGBX1010102: descr = "32-bit RGBX 10-10-10-2"; break; case V4L2_PIX_FMT_RGBA1010102: descr = "32-bit RGBA 10-10-10-2"; break; case V4L2_PIX_FMT_ARGB2101010: descr = "32-bit ARGB 2-10-10-10"; break; + case V4L2_PIX_FMT_BGR48: descr = "48-bit BGR 16-16-16"; break; + case V4L2_PIX_FMT_RGB48: descr = "48-bit RGB 16-16-16"; break; case V4L2_PIX_FMT_BGR48_12: descr = "12-bit Depth BGR"; break; case V4L2_PIX_FMT_ABGR64_12: descr = "12-bit Depth BGRA"; break; case V4L2_PIX_FMT_GREY: descr = "8-bit Greyscale"; break; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 94a0373e8234..2782c7962974 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -584,6 +584,10 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_BGR48_12 v4l2_fourcc('B', '3', '1', '2') /* 48 BGR 12-bit per component */ #define V4L2_PIX_FMT_ABGR64_12 v4l2_fourcc('B', '4', '1', '2') /* 64 BGRA 12-bit per component */ +/* RGB formats (6 bytes per pixel) */ +#define V4L2_PIX_FMT_BGR48 v4l2_fourcc('B', 'G', 'R', '6') /* 16 BGR-16-16-16 */ +#define V4L2_PIX_FMT_RGB48 v4l2_fourcc('R', 'G', 'B', '6') /* 16 RGB-16-16-16 */ + /* Grey formats */ #define V4L2_PIX_FMT_GREY v4l2_fourcc('G', 'R', 'E', 'Y') /* 8 Greyscale */ #define V4L2_PIX_FMT_Y4 v4l2_fourcc('Y', '0', '4', ' ') /* 4 Greyscale */ From patchwork Fri Feb 9 16:48:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 771445 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B07E7EEFD for ; Fri, 9 Feb 2024 16:48:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707497338; cv=none; b=NlEIGyJOcN28RWYeKI8FD3KqPc1pTQH3Q4xKiiAd/0IC96JZS2F5mTDkj+iVFfmMb61lHg0otuzfqCyZLFkQi7V7B9yPTlPrEdnmTpfsz/IPDjYxUG1QB4y1tbjh9L6xBjMmfcC8mKZzQaIBSviGVud5egmUCpc9Sij6pbCpdME= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707497338; c=relaxed/simple; bh=GNkO1f13TvBPdvtgoqETf3TRV7PKZmkAvp8W3Oba1eg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DuVLW+b2Mt0mbyVYlYS52brNR4/1f7X9KzVe1lEd6guCOXNg8rAUqFF6Pgfy5TnPfEY62jcEvESpHDlwmiz34Eh89Ikp1mGdn2qXq8kx/fFd1Bg0EW3ulsAv3imYASTRAwQNnm4K4g9WBA+TNYTuso+tQmBmGfB2AOO4xQbigYo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=ovqbq8Eh; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ovqbq8Eh" Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D981351A4; Fri, 9 Feb 2024 17:47:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1707497243; bh=GNkO1f13TvBPdvtgoqETf3TRV7PKZmkAvp8W3Oba1eg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ovqbq8EhJOy0FyqmPN5nu+9cCQRbrZ0XpTropCWgQgNn5M48OJ5y8Ncckb+w1FAxd ehUVyhfFcOi2WC1D7E+UrFrrryC4OR/ARhm4v0/cO+K+OQg1H8Gx8CYNaV8ooN3nk0 AsXmJJTWkaSrYGA07JaXdwBwidnJtUIIzs/qLGiM= From: Jacopo Mondi To: Linux Media Mailing List Cc: Jacopo Mondi , David Plowman , Naushir Patuck , Nick Hollinghurst , Dave Stevenson , Tomi Valkeinen , Laurent Pinchart , Kieran Bingham , Sakari Ailus , Hans Verkuil , Mauro Carvalho Chehab Subject: [PATCH 5/8] media: Add PiSP Compressed RAW Bayer formats Date: Fri, 9 Feb 2024 17:48:20 +0100 Message-ID: <20240209164825.166800-6-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240209164825.166800-1-jacopo.mondi@ideasonboard.com> References: <20240209164825.166800-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add Raspberry Pi compressed RAW Bayer formats. The compression algorithm description is provided by Nick Hollinghurst from Raspberry Pi. Signed-off-by: Jacopo Mondi --- .../userspace-api/media/v4l/pixfmt-bayer.rst | 1 + .../media/v4l/pixfmt-pisp-comp-rggb.rst | 70 +++++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 10 +++ include/uapi/linux/videodev2.h | 12 ++++ 4 files changed, 93 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-pisp-comp-rggb.rst diff --git a/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst b/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst index 2500413e5f43..81b69ea4084a 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst @@ -30,3 +30,4 @@ orders. See also `the Wikipedia article on Bayer filter pixfmt-srggb14 pixfmt-srggb14p pixfmt-srggb16 + pixfmt-pisp-comp-rggb diff --git a/Documentation/userspace-api/media/v4l/pixfmt-pisp-comp-rggb.rst b/Documentation/userspace-api/media/v4l/pixfmt-pisp-comp-rggb.rst new file mode 100644 index 000000000000..7be1b25b2241 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/pixfmt-pisp-comp-rggb.rst @@ -0,0 +1,70 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _v4l2-pix-fmt-pisp-comp1-rggb: +.. _v4l2-pix-fmt-pisp-comp1-grbg: +.. _v4l2-pix-fmt-pisp-comp1-gbrg: +.. _v4l2-pix-fmt-pisp-comp1-bggr: +.. _v4l2-pix-fmt-pisp-comp1-mono: +.. _v4l2-pix-fmt-pisp-comp2-rggb: +.. _v4l2-pix-fmt-pisp-comp2-grbg: +.. _v4l2-pix-fmt-pisp-comp2-gbrg: +.. _v4l2-pix-fmt-pisp-comp2-bggr: +.. _v4l2-pix-fmt-pisp-comp2-mono: + +================================================ +Raspberry Pi PiSP compressed 8-bit Bayer formats +================================================ + +Description +=========== + +Raspberry Pi ISP uses a family of three fixed-rate compressed Bayer formats. +A black-level offset may be subtracted to improve compression efficiency; +the nominal black level and amount of offset must be signalled out of band. +Each scanline is padded to a multiple of 8 pixels wide, and each block of 8 +horizontally-contiguous pixels is coded using 8 bytes. + +Mode 1 uses a quantization and delta-based coding scheme which preserves up to +12 significant bits. Mode 2 is a simple sqrt-like companding scheme with 6 PWL +chords, preserving up to 12 significant bits. Mode 3 combines both companding +(with 4 chords) and the delta scheme, preserving up to 14 significant bits. + +The remainder of this description applies to Modes 1 and 3. + +Each block of 8 pixels is separated into even and odd phases of 4 pixels, +coded independently by 32-bit words at successive locations in memory. +The two LS bits of each 32-bit word give its "quantization mode". + +In quantization mode 0, the lowest 321 quantization levels are multiples of +FSD/4096 and the remaining levels are successive multiples of FSD/2048. +Quantization modes 1 and 2 use linear quantization with step sizes of +FSD/1024 and FSD/512 respectively. Each of the four pixels is quantized +independently, with rounding to the nearest level. +In quantization mode 2 where the middle two samples have quantized values +(q1,q2) both in the range [384..511], they are coded using 9 bits for q1 +followed by 7 bits for (q2 & 127). Otherwise, for quantization modes +0, 1 and 2: a 9-bit field encodes MIN(q1,q2) which must be in the range +[0..511] and a 7-bit field encodes (q2-q1+64) which must be in [0..127]. + +Each of the outer samples (q0,q3) is encoded using a 7-bit field based +on its inner neighbour q1 or q2. In quantization mode 2 where the inner +sample has a quantized value in the range [448..511], the field value is +(q0-384). Otherwise for quantization modes 0, 1 and 2: The outer sample +is encoded as (q0-MAX(0,q1-64)). q3 is likewise coded based on q2. +Each of these values must be in the range [0..127]. All these fields +of 2, 9, 7, 7, 7 bits respectively are packed in little-endian order +to give a 32-bit word with LE byte order. + +Quantization mode 3 has a "7.5-bit" escape, used when none of the above +encodings will fit. Each pixel value is quantized to the nearest of 176 +levels, where the lowest 48 levels are multiples of FSD/256 and the +remaining levels are multiples of FSD/512 (level 175 represents values +very close to FSD and may require saturating arithmetic to decode). + +Each pair of quantized pixels (q0,q1) or (q2,q3) is jointly coded +by a 15-bit field: 2816*(q0>>4) + 16*q1 + (q0&15). +Three fields of 2, 15, 15 bits are packed in LE order {15,15,2}. + +An implementation of a software decoder of compressed formats is available +in `Raspberry Pi camera applications code base +_`. diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 1720c268fb30..2c396ba6196e 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1516,6 +1516,16 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_MT2110T: descr = "Mediatek 10bit Tile Mode"; break; case V4L2_PIX_FMT_MT2110R: descr = "Mediatek 10bit Raster Mode"; break; case V4L2_PIX_FMT_HEXTILE: descr = "Hextile Compressed Format"; break; + case V4L2_PIX_FMT_PISP_COMP1_RGGB: descr = "PiSP 8b RGRG/GBGB mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_GRBG: descr = "PiSP 8b GRGR/BGBG mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_GBRG: descr = "PiSP 8b GBGB/RGRG mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_BGGR: descr = "PiSP 8b BGBG/GRGR mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_MONO: descr = "PiSP 8b monochrome mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_RGGB: descr = "PiSP 8b RGRG/GBGB mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_GRBG: descr = "PiSP 8b GRGR/BGBG mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_GBRG: descr = "PiSP 8b GBGB/RGRG mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_BGGR: descr = "PiSP 8b BGBG/GRGR mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_MONO: descr = "PiSP 8b monochrome mode2 compr"; break; default: if (fmt->description[0]) return; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index ed72afd0e209..dbe1829a2875 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -818,6 +818,18 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_IPU3_SGRBG10 v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */ #define V4L2_PIX_FMT_IPU3_SRGGB10 v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */ +/* Raspberry Pi PiSP compressed formats. */ +#define V4L2_PIX_FMT_PISP_COMP1_RGGB v4l2_fourcc('P', 'C', '1', 'R') /* PiSP 8-bit mode 1 compressed RGGB bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_GRBG v4l2_fourcc('P', 'C', '1', 'G') /* PiSP 8-bit mode 1 compressed GRBG bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_GBRG v4l2_fourcc('P', 'C', '1', 'g') /* PiSP 8-bit mode 1 compressed GBRG bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_BGGR v4l2_fourcc('P', 'C', '1', 'B') /* PiSP 8-bit mode 1 compressed BGGR bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_MONO v4l2_fourcc('P', 'C', '1', 'M') /* PiSP 8-bit mode 1 compressed monochrome */ +#define V4L2_PIX_FMT_PISP_COMP2_RGGB v4l2_fourcc('P', 'C', '2', 'R') /* PiSP 8-bit mode 2 compressed RGGB bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_GRBG v4l2_fourcc('P', 'C', '2', 'G') /* PiSP 8-bit mode 2 compressed GRBG bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_GBRG v4l2_fourcc('P', 'C', '2', 'g') /* PiSP 8-bit mode 2 compressed GBRG bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_BGGR v4l2_fourcc('P', 'C', '2', 'B') /* PiSP 8-bit mode 2 compressed BGGR bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_MONO v4l2_fourcc('P', 'C', '2', 'M') /* PiSP 8-bit mode 2 compressed monochrome */ + /* SDR formats - used only for Software Defined Radio devices */ #define V4L2_SDR_FMT_CU8 v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */ #define V4L2_SDR_FMT_CU16LE v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */ From patchwork Fri Feb 9 16:48:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 771444 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7FA417E115 for ; Fri, 9 Feb 2024 16:49:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707497344; cv=none; b=CSey46W0p34iXHvdkd3+NiZUoUG7Wq6BrLHbugU2Rmj64Voxv5XEklUYaW2wK6DZnW49kZ9LcUlpMy2iADX/3Nv8NYfxEhEQWmGgCaCVkkzFYsvzXbj3GW0BTf2JupvPAHZzCheC+Zd2v7sAyIalDBP/5An7UBHw4SbMeHtbNqw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707497344; c=relaxed/simple; bh=0HtMe6hNYfpY/mewRhsoX6CI2BMKIa0cQQLfu40DBb4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ui5IWKgtRvKjSWVx8W1753g5yA6e+udvKTxVGuYVsEkuSo1ui4VLBz6Ahzev5Ppzxe7o3+MGh0df/V+16Xmb65wu4f6LgCOWQaQZCtOCzzqSWW346T+7fY87J85wqWDpUjs44o5yV4IMmZ5v7r5icSIdMpCsHo8d4e9CWl9Fdxg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=vtQB7j7S; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vtQB7j7S" Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 96CF66054; Fri, 9 Feb 2024 17:47:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1707497246; bh=0HtMe6hNYfpY/mewRhsoX6CI2BMKIa0cQQLfu40DBb4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vtQB7j7SyHJARlTwiYpxKYg8U5zPH+v9OnAoAwXDzf+Te/a2FG4OxtYwTuwa8bDVG UCof4VLdq1ZSOl8UCotCZE4v0iyZrp0RiRmmwUylHCqb+Dak/Z4+wi0hIoWMSeQelU BEutlYtoqnXHXvki7qZS4KteHmjFP5Pwd85UwHeg= From: Jacopo Mondi To: Linux Media Mailing List Cc: Jacopo Mondi , David Plowman , Naushir Patuck , Nick Hollinghurst , Dave Stevenson , Tomi Valkeinen , Laurent Pinchart , Kieran Bingham , Sakari Ailus , Hans Verkuil , Mauro Carvalho Chehab Subject: [PATCH 8/8] media: admin-guide: Document the Raspberry Pi PiSP BE Date: Fri, 9 Feb 2024 17:48:23 +0100 Message-ID: <20240209164825.166800-9-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240209164825.166800-1-jacopo.mondi@ideasonboard.com> References: <20240209164825.166800-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add documentation for the PiSP Back End memory-to-memory ISP. Signed-off-by: Jacopo Mondi --- Documentation/admin-guide/media/pisp-be.dot | 22 ++++ Documentation/admin-guide/media/pisp-be.rst | 117 ++++++++++++++++++ .../admin-guide/media/v4l-drivers.rst | 1 + 3 files changed, 140 insertions(+) create mode 100644 Documentation/admin-guide/media/pisp-be.dot create mode 100644 Documentation/admin-guide/media/pisp-be.rst diff --git a/Documentation/admin-guide/media/pisp-be.dot b/Documentation/admin-guide/media/pisp-be.dot new file mode 100644 index 000000000000..001b2c16e232 --- /dev/null +++ b/Documentation/admin-guide/media/pisp-be.dot @@ -0,0 +1,22 @@ +digraph board { + rankdir=TB + n00000001 [label="{{ 0 | 1 | 2 | 8} | pispbe\n | { 3 | 4 | 5 | 6 | 7}}", shape=Mrecord, style=filled, fillcolor=green] + n00000001:port3 -> n0000001d [style=bold] + n00000001:port4 -> n00000023 [style=bold] + n00000001:port5 -> n00000029 [style=bold] + n00000001:port6 -> n0000002f [style=bold] + n00000001:port7 -> n00000035 [style=bold] + n0000000b [label="pispbe-input\n/dev/video0", shape=box, style=filled, fillcolor=yellow] + n0000000b -> n00000001:port0 [style=bold] + n00000011 [label="pispbe-tdn_input\n/dev/video1", shape=box, style=filled, fillcolor=yellow] + n00000011 -> n00000001:port1 [style=bold] + n00000017 [label="pispbe-stitch_input\n/dev/video2", shape=box, style=filled, fillcolor=yellow] + n00000017 -> n00000001:port2 [style=bold] + n0000001d [label="pispbe-hog_output\n/dev/video3", shape=box, style=filled, fillcolor=yellow] + n00000023 [label="pispbe-output0\n/dev/video4", shape=box, style=filled, fillcolor=yellow] + n00000029 [label="pispbe-output1\n/dev/video5", shape=box, style=filled, fillcolor=yellow] + n0000002f [label="pispbe-tdn_output\n/dev/video6", shape=box, style=filled, fillcolor=yellow] + n00000035 [label="pispbe-stitch_output\n/dev/video7", shape=box, style=filled, fillcolor=yellow] + n0000003b [label="pispbe-config\n/dev/video8", shape=box, style=filled, fillcolor=yellow] + n0000003b -> n00000001:port8 [style=bold] +} diff --git a/Documentation/admin-guide/media/pisp-be.rst b/Documentation/admin-guide/media/pisp-be.rst new file mode 100644 index 000000000000..871d44c3de04 --- /dev/null +++ b/Documentation/admin-guide/media/pisp-be.rst @@ -0,0 +1,117 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================================= +Raspberry Pi PiSP Back End Memory-to-Memory ISP (pisp-be) +========================================================= + +The PiSP Back End +================= + +The PiSP Back End is a memory-to-memory Image Signal Processor (ISP) which reads +image data from DRAM memory and performs image processing as specified by the +application through the parameters in a configuration buffer, before writing +pixel data back to memory through two distinct output channels. + +The ISP registers a programming model are documented in the `Raspberry Pi Image +Signal Processor (ISP) Specification document +`_ + +The PiSP Back End ISP processes images in tiles to allow concurrent processing +of different images. The handling of image tassellation and the computation of +low-level configuration parameters is realized by a free software library +called `libpisp `_. + +The full image processing pipeline, which involves capturing RAW Bayer data from +an image sensor through a MIPI CSI-2 compatible capture interface, storing them +in DRAM memory and processing them in the PiSP Back to obtain images usable by +an application is implemented in `libcamera `_ as part of the +Raspberry Pi platform support. + +The pisp-be driver +================== + +The Raspberry Pi PiSP Back End (pisp-be) driver is located under +drivers/media/platform/raspberrypi/pisp-be. It uses the `V4L2 API` to register +a number of video capture and output devices, the `V4L2 subdev API` to register +a subdevice for the ISP that connects the video devices in a single media graph +realized using the `Media Controller (MC) API`. + +As the PiSP Back End device processes images in tiles, it can be multiplexed +between several different execution contexts. The driver registers two +`contexts` represented as two distinct media graph instances but could +potentially register more of them. + +The media topology registered by the `pisp-be` driver is the following one: + +.. _pips-be-topology: + +.. kernel-figure:: pisp-be.dot + :alt: Diagram of the default media pipeline topology + :align: center + + +Each media graph instance registers the following video device nodes: + +- pispbe-input: output device for images to be submitted to the ISP for + processing. +- pispbe-tdn_input: output device for temporal denoise. +- pispbe-stitch_input: output device for image stitching (HDR). +- pipsbe-hog_output: capture device for HOG (histogram of oriented gradients). +- pispbe-output0: first capture device for processed images. +- pispbe-output1: second capture device for processed images. +- pispbe-tdn_output: capture device for temporal denoise. +- pispbe-stitch_output: capture device for image stitching (HDR). +- pispbe-config: output device for ISP configuration parameters. + +pispbe-input +------------ + +Images to be processed by the ISP are queued to the `pispbe-input` output device +node. For a list of image formats supported as input format by the ISP refer to +the PiSP specification document. + +pispbe-tdn_input, pispbe-tdn_output +----------------------------------- + +The `pispbe-tdn_input` output video device receives images to be processed by +the temporal denoise block which are captured from the `pispbe-tdn_output` +capture video device. Userspace should maintain a queue of buffers as captured +from the TDN output and input to the TDN input device. + +pispbe-stitch_input, pispbe-tdn_output +-------------------------------------- + +To realize HDR (high dynamic range) image processing the image stitching and +tonemapping blocks are used. The `pispbe-stitch_output` writes images to memory +and the `pispbe-stitch_input` receives the previously written frame to process +it along with the current input image. Userspace should maintain a queue of +buffers as captured from the stitch output and input to the stitch input device. + +pipsbe-hog_output +----------------- + +The `pipsbe-hog_output` video capture device produces image statistics in the +form of 'Histogram of Oriented Gradients'. + +pispbe-output0, pispbe-output1 +------------------------------ + +The two capture devices write to memory the pixel data as processed by the ISP. + +pispbe-config +------------- + +The `pispbe-config` output video devices receives a buffer of configuration +parameters that define the desired image processing to be performed by the ISP. + +The format of the ISP configuration parameter is defined by +:c:type:`pisp_be_tiles_config` C structure and the meaning of each parameter is +described in the `PiSP specification` document. + +ISP configuration +================= + +The ISP configuration is described solely by the content of the parameters +buffer. The only parameter that userspace needs to configure using the V4L2 API +is the image format on the output and capture video devices for validation of +the content of the parameters buffer. diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index f4bb2605f07e..1a491a74a272 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -21,6 +21,7 @@ Video4Linux (V4L) driver-specific documentation omap3isp omap4_camera philips + pisp-be qcom_camss rcar-fdp1 rkisp1