From patchwork Tue Aug 23 17:18:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 599922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 915F0C32789 for ; Tue, 23 Aug 2022 18:54:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233001AbiHWSyD (ORCPT ); Tue, 23 Aug 2022 14:54:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233288AbiHWSxp (ORCPT ); Tue, 23 Aug 2022 14:53:45 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D377205D8 for ; Tue, 23 Aug 2022 10:18:49 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E430B484; Tue, 23 Aug 2022 19:18:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275127; bh=qc4dX5X6pcswRfuYM0u0Kk5nQTzPLRvQQ3wiK+TRz2g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jm/670bQrEC9gfWreLhXIviKU0Q3dz7ikUl+rxizGDmJIE6C4W7wVvlD53NaXThxB BSCeafu7i0BGJFJYSKIv9nh5D3IMA6gRBxFXf5waTwYxlhwY0ztYRKUvVoiImsjyVF 1uxyXj1zDkdXLQyLpnyEBih3mINTT4ULdRR48jbM= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 1/9] media: rkisp1: Initialize color space on ISP sink and source pads Date: Tue, 23 Aug 2022 20:18:32 +0300 Message-Id: <20220823171840.8958-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Initialize the four color space fields on the sink and source video pads of the ISP in the .init_cfg() operation. As the main use case for the ISP is to convert Bayer data to YUV, select a raw color space on the sink pad and a limited range quantization of SYCC on the source pad by default. Signed-off-by: Laurent Pinchart Reviewed-by: Dafna Hirschfeld Reviewed-by: Paul Elder --- Changes since v1: - Mention ISP in the subject line --- drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c index a3c7d4d88387..a52b22824739 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c @@ -553,12 +553,17 @@ static int rkisp1_isp_init_config(struct v4l2_subdev *sd, struct v4l2_mbus_framefmt *sink_fmt, *src_fmt; struct v4l2_rect *sink_crop, *src_crop; + /* Video. */ sink_fmt = v4l2_subdev_get_try_format(sd, sd_state, RKISP1_ISP_PAD_SINK_VIDEO); sink_fmt->width = RKISP1_DEFAULT_WIDTH; sink_fmt->height = RKISP1_DEFAULT_HEIGHT; sink_fmt->field = V4L2_FIELD_NONE; sink_fmt->code = RKISP1_DEF_SINK_PAD_FMT; + sink_fmt->colorspace = V4L2_COLORSPACE_RAW; + sink_fmt->xfer_func = V4L2_XFER_FUNC_NONE; + sink_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601; + sink_fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE; sink_crop = v4l2_subdev_get_try_crop(sd, sd_state, RKISP1_ISP_PAD_SINK_VIDEO); @@ -571,11 +576,16 @@ static int rkisp1_isp_init_config(struct v4l2_subdev *sd, RKISP1_ISP_PAD_SOURCE_VIDEO); *src_fmt = *sink_fmt; src_fmt->code = RKISP1_DEF_SRC_PAD_FMT; + src_fmt->colorspace = V4L2_COLORSPACE_SRGB; + src_fmt->xfer_func = V4L2_XFER_FUNC_SRGB; + src_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601; + src_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE; src_crop = v4l2_subdev_get_try_crop(sd, sd_state, RKISP1_ISP_PAD_SOURCE_VIDEO); *src_crop = *sink_crop; + /* Parameters and statistics. */ sink_fmt = v4l2_subdev_get_try_format(sd, sd_state, RKISP1_ISP_PAD_SINK_PARAMS); src_fmt = v4l2_subdev_get_try_format(sd, sd_state, From patchwork Tue Aug 23 17:18:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 599920 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E50CDC32772 for ; Tue, 23 Aug 2022 18:54:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233604AbiHWSyc (ORCPT ); Tue, 23 Aug 2022 14:54:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230472AbiHWSyJ (ORCPT ); Tue, 23 Aug 2022 14:54:09 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14B3AEA8B8 for ; Tue, 23 Aug 2022 10:19:46 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DDC849C4; Tue, 23 Aug 2022 19:18:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275131; bh=WeVhIIrOGSxD0QDS3twAok28MZFQgk9/L8LNeg/uP5M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mnZgWudn2X4/jhB3tTQP6nmjtN4RFvG5ZawL822nzIhxNhpQHTv5fZvH6mp7HpQvO v+tpePb29+2rUm4Zg01xtd2n0gfSlAh8t9iSWlkO5BNiEbrPO02b1yoO13FoHpWBzP F+68JAO0w/JnwHhZmVGCSEoO59zcu9E1dpKH6Y0M= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 4/9] media: rkisp1: Allow setting all color space fields on ISP source pad Date: Tue, 23 Aug 2022 20:18:35 +0300 Message-Id: <20220823171840.8958-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The ISP output color space is configured through the ISP source pad. At the moment, only the quantization can be set. Extend it to the three other color space fields: - The ycbcr_enc field will be used to configure the RGB to YUV matrix (currently hardcoded to Rec. 601). - The colorspace (which controls the color primaries) and xfer_func fields will not be used to configure the ISP, as the corresponding hardware blocks (the cross-talk RGB to RGB matrix and the tone mapping curve) are programmed directly by applications through ISP parameters. Nonetheless, those two fields should be set by applications to match the ISP configuration, in order to propagate the correct color space down the pipeline up to the capture video nodes. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Dafna Hirschfeld --- Changes since v1: - Fix quantization setting that was set on sink pad by mistake --- .../platform/rockchip/rkisp1/rkisp1-isp.c | 55 ++++++++++++++++--- 1 file changed, 48 insertions(+), 7 deletions(-) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c index d34f32271d74..7869f0cced62 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c @@ -609,6 +609,7 @@ static void rkisp1_isp_set_src_fmt(struct rkisp1_isp *isp, struct v4l2_mbus_framefmt *sink_fmt; struct v4l2_mbus_framefmt *src_fmt; const struct v4l2_rect *src_crop; + bool set_csc; sink_fmt = rkisp1_isp_get_pad_fmt(isp, sd_state, RKISP1_ISP_PAD_SINK_VIDEO, which); @@ -645,20 +646,60 @@ static void rkisp1_isp_set_src_fmt(struct rkisp1_isp *isp, src_fmt->height = src_crop->height; /* - * The CSC API is used to allow userspace to force full - * quantization on YUV formats. + * Copy the color space for the sink pad. When converting from Bayer to + * YUV, default to a limited quantization range. */ - if (format->flags & V4L2_MBUS_FRAMEFMT_SET_CSC && - format->quantization == V4L2_QUANTIZATION_FULL_RANGE && + src_fmt->colorspace = sink_fmt->colorspace; + src_fmt->xfer_func = sink_fmt->xfer_func; + src_fmt->ycbcr_enc = sink_fmt->ycbcr_enc; + + if (sink_info->pixel_enc == V4L2_PIXEL_ENC_BAYER && src_info->pixel_enc == V4L2_PIXEL_ENC_YUV) - src_fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE; - else if (src_info->pixel_enc == V4L2_PIXEL_ENC_YUV) src_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE; else - src_fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE; + src_fmt->quantization = sink_fmt->quantization; + + /* + * Allow setting the source color space fields when the SET_CSC flag is + * set and the source format is YUV. If the sink format is YUV, don't + * set the color primaries, transfer function or YCbCr encoding as the + * ISP is bypassed in that case and passes YUV data through without + * modifications. + * + * The color primaries and transfer function are configured through the + * cross-talk matrix and tone curve respectively. Settings for those + * hardware blocks are conveyed through the ISP parameters buffer, as + * they need to combine color space information with other image tuning + * characteristics and can't thus be computed by the kernel based on the + * color space. The source pad colorspace and xfer_func fields are thus + * ignored by the driver, but can be set by userspace to propagate + * accurate color space information down the pipeline. + */ + set_csc = !!(format->flags & V4L2_MBUS_FRAMEFMT_SET_CSC); + + if (set_csc && src_info->pixel_enc == V4L2_PIXEL_ENC_YUV) { + if (sink_info->pixel_enc == V4L2_PIXEL_ENC_BAYER) { + if (format->colorspace != V4L2_COLORSPACE_DEFAULT) + src_fmt->colorspace = format->colorspace; + if (format->xfer_func != V4L2_XFER_FUNC_DEFAULT) + src_fmt->xfer_func = format->xfer_func; + if (format->ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT) + src_fmt->ycbcr_enc = format->ycbcr_enc; + } + + if (format->quantization != V4L2_QUANTIZATION_DEFAULT) + src_fmt->quantization = format->quantization; + } *format = *src_fmt; + /* + * Restore the SET_CSC flag if it was set to indicate support for the + * CSC setting API. + */ + if (set_csc) + format->flags |= V4L2_MBUS_FRAMEFMT_SET_CSC; + /* Store the source format info when setting the active format. */ if (which == V4L2_SUBDEV_FORMAT_ACTIVE) isp->src_fmt = src_info; From patchwork Tue Aug 23 17:18:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 599919 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D799C32774 for ; Tue, 23 Aug 2022 18:54:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232587AbiHWSym (ORCPT ); Tue, 23 Aug 2022 14:54:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232447AbiHWSyU (ORCPT ); Tue, 23 Aug 2022 14:54:20 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01F7211D535 for ; Tue, 23 Aug 2022 10:20:01 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B226FE1E; Tue, 23 Aug 2022 19:18:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275134; bh=ei00uxRrPfVtT4n0RYTb5ZUOk9kkdzGSUKVGJZnvSoM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OdD9eqVLle7qyTj+HNqmKTt53/iGk9JFY0/VJk/sg2HMvTe9M5H/V1FvHxbo7axBl k/XaXG92Bl5yL28R1CItiGUDAIm8mLmr84UfwQezseSGpR0TYwn1MmUqpKKkmmV6OP Pk6pqn4bu25nvrMZm9jTBMXpz3Ei1SlJT4vUYoyw= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 6/9] media: rkisp1: Don't pass the quantization to rkisp1_csm_config() Date: Tue, 23 Aug 2022 20:18:37 +0300 Message-Id: <20220823171840.8958-7-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The rkisp1_csm_config() function takes a pointer to the rkisp1_params structure which contains the quantization value. There's no need to pass it separately to the function. Drop it from the function parameters. Signed-off-by: Laurent Pinchart Reviewed-by: Dafna Hirschfeld Reviewed-by: Paul Elder --- drivers/media/platform/rockchip/rkisp1/rkisp1-params.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c index 8b4eea77af0d..163419624370 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c @@ -1076,7 +1076,7 @@ static void rkisp1_ie_enable(struct rkisp1_params *params, bool en) } } -static void rkisp1_csm_config(struct rkisp1_params *params, bool full_range) +static void rkisp1_csm_config(struct rkisp1_params *params) { static const u16 full_range_coeff[] = { 0x0026, 0x004b, 0x000f, @@ -1090,7 +1090,7 @@ static void rkisp1_csm_config(struct rkisp1_params *params, bool full_range) }; unsigned int i; - if (full_range) { + if (params->quantization == V4L2_QUANTIZATION_FULL_RANGE) { for (i = 0; i < ARRAY_SIZE(full_range_coeff); i++) rkisp1_write(params->rkisp1, RKISP1_CIF_ISP_CC_COEFF_0 + i * 4, @@ -1562,11 +1562,7 @@ static void rkisp1_params_config_parameter(struct rkisp1_params *params) rkisp1_param_set_bits(params, RKISP1_CIF_ISP_HIST_PROP_V10, rkisp1_hst_params_default_config.mode); - /* set the range */ - if (params->quantization == V4L2_QUANTIZATION_FULL_RANGE) - rkisp1_csm_config(params, true); - else - rkisp1_csm_config(params, false); + rkisp1_csm_config(params); spin_lock_irq(¶ms->config_lock); From patchwork Tue Aug 23 17:18:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 599918 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BC90C32772 for ; Tue, 23 Aug 2022 18:54:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233595AbiHWSyz (ORCPT ); Tue, 23 Aug 2022 14:54:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231603AbiHWSyb (ORCPT ); Tue, 23 Aug 2022 14:54:31 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C0E565540 for ; Tue, 23 Aug 2022 10:20:18 -0700 (PDT) Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8387B10CD; Tue, 23 Aug 2022 19:18:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1661275136; bh=e9dDoI7js6GAclo5QIceH/HdFmi9cGjnukCULiCyneE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Slke7OUD7FbrSdZFj81M/mBKdABPf/5k+HpkHY9ASG1pWVacc8K+e9YIMJGX+3ZvQ vR7DTrXG/uISbi1PPykZJpOoAZITKxonwaKDsGIdBaVOsHMl/LEBKmEXOADmpam/uv 2LQgR74uSFQGir8B5gWc3/eGOZbdkgyq2LQprGX0= From: Laurent Pinchart To: linux-media@vger.kernel.org Cc: Dafna Hirschfeld , Heiko Stuebner , Florian Sylvestre , Paul Elder Subject: [PATCH v2 8/9] media: rkisp1: Initialize color space on resizer sink and source pads Date: Tue, 23 Aug 2022 20:18:39 +0300 Message-Id: <20220823171840.8958-9-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> References: <20220823171840.8958-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Initialize the four color space fields on the sink and source video pads of the resizer in the .init_cfg() operation. The resizer can't perform any color space conversion, so set the sink and source color spaces to the same defaults, which match the ISP source video pad default. Signed-off-by: Laurent Pinchart Reviewed-by: Paul Elder Reviewed-by: Dafna Hirschfeld --- drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c index becef04fdf2d..6f6ec00b63b8 100644 --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c @@ -430,6 +430,10 @@ static int rkisp1_rsz_init_config(struct v4l2_subdev *sd, sink_fmt->height = RKISP1_DEFAULT_HEIGHT; sink_fmt->field = V4L2_FIELD_NONE; sink_fmt->code = RKISP1_DEF_FMT; + sink_fmt->colorspace = V4L2_COLORSPACE_SRGB; + sink_fmt->xfer_func = V4L2_XFER_FUNC_SRGB; + sink_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601; + sink_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE; sink_crop = v4l2_subdev_get_try_crop(sd, sd_state, RKISP1_RSZ_PAD_SINK);