From patchwork Wed Mar 9 16:37:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eugen Hristev X-Patchwork-Id: 549850 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3488C433FE for ; Wed, 9 Mar 2022 16:45:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236121AbiCIQqi (ORCPT ); Wed, 9 Mar 2022 11:46:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235022AbiCIQpA (ORCPT ); Wed, 9 Mar 2022 11:45:00 -0500 Received: from esa.microchip.iphmx.com (esa.microchip.iphmx.com [68.232.154.123]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79A6A2B260; Wed, 9 Mar 2022 08:38:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=microchip.com; i=@microchip.com; q=dns/txt; s=mchp; t=1646843916; x=1678379916; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8pGmp/P6DW09QNCZnxhFBr88c90cP0y+5hBJacUv2zo=; b=QJi/L/nVNgQAO0qoNUs5NjI3haPM6ajfVE0RxFzeKu0tkXhHqyAlEekH /jW1oJvyld/yHZUAzVl4NyXPMnr9OPUpkX6z46rVTtx3m5w5ryrVdnDF4 IoV+LbbX3h8FrNHr8VSCzPyVvZP1fxAFLb2uNEfTEkRlp1zr4f4PU7X3W z+IMkKNw+4BghpBz4uu76tV5oL6VoHqH7cs1jFzJrg7i/jBqzC0riz5PT tepNCDeStE7BkytjbxkXOS9NFOtwqjECbPxTnHxkq7Rp3IuFxpR6ebmXB b9PIxPwNzvjx+Desp0eKV7KOTrMTESSmlAvQE5JDGbF2kKHdba7JKsjhc w==; X-IronPort-AV: E=Sophos;i="5.90,167,1643698800"; d="scan'208";a="151431959" Received: from smtpout.microchip.com (HELO email.microchip.com) ([198.175.253.82]) by esa2.microchip.iphmx.com with ESMTP/TLS/AES256-SHA256; 09 Mar 2022 09:38:26 -0700 Received: from chn-vm-ex03.mchp-main.com (10.10.85.151) by chn-vm-ex02.mchp-main.com (10.10.85.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Wed, 9 Mar 2022 09:38:25 -0700 Received: from ROB-ULT-M18282.microchip.com (10.10.115.15) by chn-vm-ex03.mchp-main.com (10.10.85.151) with Microsoft SMTP Server id 15.1.2375.17 via Frontend Transport; Wed, 9 Mar 2022 09:38:12 -0700 From: Eugen Hristev To: , CC: , , , , , , Eugen Hristev Subject: [PATCH v8 01/13] media: atmel: atmel-isc-base: use streaming status when queueing buffers Date: Wed, 9 Mar 2022 18:37:46 +0200 Message-ID: <20220309163758.2672727-2-eugen.hristev@microchip.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220309163758.2672727-1-eugen.hristev@microchip.com> References: <20220309163758.2672727-1-eugen.hristev@microchip.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org During experiments with libcamera, it looks like vb2_is_streaming returns true before our start streaming is called. Order of operations is streamon -> queue -> start_streaming ISC would have started the DMA immediately when a buffer is being added to the vbqueue if the queue is streaming. It is more safe to start the DMA after the start streaming of the driver is called. Thus, even if vb2queue is streaming, add the buffer to the dma queue of the driver instead of actually starting the DMA process, if the start streaming has not been called yet. Tho achieve this, we have to use vb2_start_streaming_called instead of vb2_is_streaming. Signed-off-by: Eugen Hristev Reviewed-by: Jacopo Mondi --- drivers/media/platform/atmel/atmel-isc-base.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c index db15770d5b88..d2cc6c99984f 100644 --- a/drivers/media/platform/atmel/atmel-isc-base.c +++ b/drivers/media/platform/atmel/atmel-isc-base.c @@ -442,7 +442,7 @@ static void isc_buffer_queue(struct vb2_buffer *vb) spin_lock_irqsave(&isc->dma_queue_lock, flags); if (!isc->cur_frm && list_empty(&isc->dma_queue) && - vb2_is_streaming(vb->vb2_queue)) { + vb2_start_streaming_called(vb->vb2_queue)) { isc->cur_frm = buf; isc_start_dma(isc); } else From patchwork Wed Mar 9 16:37:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eugen Hristev X-Patchwork-Id: 549849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06480C433F5 for ; Wed, 9 Mar 2022 16:46:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235860AbiCIQrU (ORCPT ); Wed, 9 Mar 2022 11:47:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235833AbiCIQpo (ORCPT ); Wed, 9 Mar 2022 11:45:44 -0500 Received: from esa.microchip.iphmx.com (esa.microchip.iphmx.com [68.232.154.123]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62D6860DBF; Wed, 9 Mar 2022 08:39:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=microchip.com; i=@microchip.com; q=dns/txt; s=mchp; t=1646843944; x=1678379944; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2jhYb3qEhZlt64XJu+carXPbikihThqhivb9u1q5IWI=; b=1hhwpYnnuQ93kBT2AsxBpGwLGmOx6Orn6sSyzlQ4YPZNDVqkPCQMxFk0 37ZKND5yU8vkiU+ECtikgUicbUUTGaCSm3WlgAlqQILLRYUz+78HZCOmX lvSmJURxjk3/xFlM8bMJIHYYPhCVqgNidsuFJOUI9I8Mx5m/pSi0TWit0 +Vg29zV35WzfjqcSZYmlbMvrgjGlpOntp2ZkL6wNpgT9op0aatszJ6YWJ DMiGH9YW+34MjwQQcdkwQNZxFM2qCpCnaTdXmswZbT95emYRLH8WiL4/O RDoqZt/yl86AmmETRLNaw4GeppxK+CjCaqklp+5xEIFIjZBQwRHI3imOF g==; X-IronPort-AV: E=Sophos;i="5.90,167,1643698800"; d="scan'208";a="151432042" Received: from smtpout.microchip.com (HELO email.microchip.com) ([198.175.253.82]) by esa2.microchip.iphmx.com with ESMTP/TLS/AES256-SHA256; 09 Mar 2022 09:38:56 -0700 Received: from chn-vm-ex03.mchp-main.com (10.10.85.151) by chn-vm-ex01.mchp-main.com (10.10.85.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Wed, 9 Mar 2022 09:38:51 -0700 Received: from ROB-ULT-M18282.microchip.com (10.10.115.15) by chn-vm-ex03.mchp-main.com (10.10.85.151) with Microsoft SMTP Server id 15.1.2375.17 via Frontend Transport; Wed, 9 Mar 2022 09:38:43 -0700 From: Eugen Hristev To: , CC: , , , , , , Eugen Hristev Subject: [PATCH v8 04/13] media: atmel: atmel-isc: implement media controller Date: Wed, 9 Mar 2022 18:37:49 +0200 Message-ID: <20220309163758.2672727-5-eugen.hristev@microchip.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220309163758.2672727-1-eugen.hristev@microchip.com> References: <20220309163758.2672727-1-eugen.hristev@microchip.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Implement the support for media-controller. This means that the capabilities of the driver have changed and now it also advertises the IO_MC . The driver will register its media device, and add the video entity to this media device. The subdevices are registered to the same media device. The ISC will have a base entity which is auto-detected as atmel_isc_base. It will also register a subdevice that allows cropping of the incoming frame to the maximum frame size supported by the ISC. The ISC will create a link between the subdevice that is asynchronously registered and the atmel_isc_scaler entity. Then, the atmel_isc_scaler and atmel_isc_base are connected through another link. Signed-off-by: Eugen Hristev Reviewed-by: Jacopo Mondi --- Changes in v8: - use source format size as bounds always Changes in v7: - use maximum isc frame size as bounds always Changes in v6: - reworked a bit as suggested by Jacopo - add try crops Changes in v5: - reworked s_fmt to pass the same format from sink to source - simplified enum_mbus_code - separated tgt and bounds to report correctly in g_sel Changes in v4: As suggested by Jacopo: - renamed atmel_isc_mc to atmel_isc_scaler.c - moved init_mc/clean_mc to isc_base file Changes in v2: - implement try formats drivers/media/platform/atmel/Makefile | 2 +- drivers/media/platform/atmel/atmel-isc-base.c | 75 ++++- .../media/platform/atmel/atmel-isc-scaler.c | 262 ++++++++++++++++++ drivers/media/platform/atmel/atmel-isc.h | 37 +++ .../media/platform/atmel/atmel-sama5d2-isc.c | 14 +- .../media/platform/atmel/atmel-sama7g5-isc.c | 12 +- 6 files changed, 394 insertions(+), 8 deletions(-) create mode 100644 drivers/media/platform/atmel/atmel-isc-scaler.c diff --git a/drivers/media/platform/atmel/Makefile b/drivers/media/platform/atmel/Makefile index 794e8f739287..f02d03df89d6 100644 --- a/drivers/media/platform/atmel/Makefile +++ b/drivers/media/platform/atmel/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only atmel-isc-objs = atmel-sama5d2-isc.o atmel-xisc-objs = atmel-sama7g5-isc.o -atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o +atmel-isc-common-objs = atmel-isc-base.o atmel-isc-clk.o atmel-isc-scaler.o obj-$(CONFIG_VIDEO_ATMEL_ISI) += atmel-isi.o obj-$(CONFIG_VIDEO_ATMEL_ISC_BASE) += atmel-isc-common.o diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c index 67b4a2323fed..448bf281c61a 100644 --- a/drivers/media/platform/atmel/atmel-isc-base.c +++ b/drivers/media/platform/atmel/atmel-isc-base.c @@ -1712,6 +1712,7 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier, struct isc_device, v4l2_dev); struct isc_subdev_entity *subdev_entity = container_of(notifier, struct isc_subdev_entity, notifier); + int pad; if (video_is_registered(&isc->video_dev)) { v4l2_err(&isc->v4l2_dev, "only supports one sub-device.\n"); @@ -1720,6 +1721,16 @@ static int isc_async_bound(struct v4l2_async_notifier *notifier, subdev_entity->sd = subdev; + pad = media_entity_get_fwnode_pad(&subdev->entity, asd->match.fwnode, + MEDIA_PAD_FL_SOURCE); + if (pad < 0) { + v4l2_err(&isc->v4l2_dev, "failed to find pad for %s\n", + subdev->name); + return pad; + } + + isc->remote_pad = pad; + return 0; } @@ -1734,8 +1745,8 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier, v4l2_ctrl_handler_free(&isc->ctrls.handler); } -static struct isc_format *find_format_by_code(struct isc_device *isc, - unsigned int code, int *index) +struct isc_format *isc_find_format_by_code(struct isc_device *isc, + unsigned int code, int *index) { struct isc_format *fmt = &isc->formats_list[0]; unsigned int i; @@ -1751,6 +1762,7 @@ static struct isc_format *find_format_by_code(struct isc_device *isc, return NULL; } +EXPORT_SYMBOL_GPL(isc_find_format_by_code); static int isc_formats_init(struct isc_device *isc) { @@ -1767,7 +1779,7 @@ static int isc_formats_init(struct isc_device *isc) NULL, &mbus_code)) { mbus_code.index++; - fmt = find_format_by_code(isc, mbus_code.code, &i); + fmt = isc_find_format_by_code(isc, mbus_code.code, &i); if (!fmt) { v4l2_warn(&isc->v4l2_dev, "Mbus code %x not supported\n", mbus_code.code); @@ -1893,7 +1905,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier) vdev->queue = q; vdev->lock = &isc->lock; vdev->ctrl_handler = &isc->ctrls.handler; - vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE; + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE | + V4L2_CAP_IO_MC; video_set_drvdata(vdev, isc); ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); @@ -1903,8 +1916,19 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier) goto isc_async_complete_err; } + ret = isc_scaler_link(isc); + if (ret < 0) + goto isc_async_complete_unregister_device; + + ret = media_device_register(&isc->mdev); + if (ret < 0) + goto isc_async_complete_unregister_device; + return 0; +isc_async_complete_unregister_device: + video_unregister_device(vdev); + isc_async_complete_err: mutex_destroy(&isc->lock); return ret; @@ -1971,6 +1995,49 @@ int isc_pipeline_init(struct isc_device *isc) } EXPORT_SYMBOL_GPL(isc_pipeline_init); +int isc_mc_init(struct isc_device *isc, u32 ver) +{ + const struct of_device_id *match; + int ret; + + isc->video_dev.entity.function = MEDIA_ENT_F_IO_V4L; + isc->video_dev.entity.flags = MEDIA_ENT_FL_DEFAULT; + isc->pads[ISC_PAD_SINK].flags = MEDIA_PAD_FL_SINK; + + ret = media_entity_pads_init(&isc->video_dev.entity, ISC_PADS_NUM, + isc->pads); + if (ret < 0) { + dev_err(isc->dev, "media entity init failed\n"); + return ret; + } + + isc->mdev.dev = isc->dev; + + match = of_match_node(isc->dev->driver->of_match_table, + isc->dev->of_node); + + strscpy(isc->mdev.driver_name, KBUILD_MODNAME, + sizeof(isc->mdev.driver_name)); + strscpy(isc->mdev.model, match->compatible, sizeof(isc->mdev.model)); + snprintf(isc->mdev.bus_info, sizeof(isc->mdev.bus_info), "platform:%s", + isc->v4l2_dev.name); + isc->mdev.hw_revision = ver; + + media_device_init(&isc->mdev); + + isc->v4l2_dev.mdev = &isc->mdev; + + return isc_scaler_init(isc); +} +EXPORT_SYMBOL_GPL(isc_mc_init); + +void isc_mc_cleanup(struct isc_device *isc) +{ + media_entity_cleanup(&isc->video_dev.entity); + media_device_cleanup(&isc->mdev); +} +EXPORT_SYMBOL_GPL(isc_mc_cleanup); + /* regmap configuration */ #define ATMEL_ISC_REG_MAX 0xd5c const struct regmap_config isc_regmap_config = { diff --git a/drivers/media/platform/atmel/atmel-isc-scaler.c b/drivers/media/platform/atmel/atmel-isc-scaler.c new file mode 100644 index 000000000000..a1ca4633787c --- /dev/null +++ b/drivers/media/platform/atmel/atmel-isc-scaler.c @@ -0,0 +1,262 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Microchip Image Sensor Controller (ISC) Scaler entity support + * + * Copyright (C) 2022 Microchip Technology, Inc. + * + * Author: Eugen Hristev + * + */ + +#include +#include +#include +#include + +#include "atmel-isc-regs.h" +#include "atmel-isc.h" + +static void isc_scaler_prepare_fmt(struct v4l2_mbus_framefmt *framefmt) +{ + framefmt->colorspace = V4L2_COLORSPACE_SRGB; + framefmt->field = V4L2_FIELD_NONE; + framefmt->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + framefmt->quantization = V4L2_QUANTIZATION_DEFAULT; + framefmt->xfer_func = V4L2_XFER_FUNC_DEFAULT; +}; + +static int isc_scaler_get_fmt(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state, + struct v4l2_subdev_format *format) +{ + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd); + struct v4l2_mbus_framefmt *v4l2_try_fmt; + + if (format->which == V4L2_SUBDEV_FORMAT_TRY) { + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state, + format->pad); + format->format = *v4l2_try_fmt; + + return 0; + } + + format->format = isc->scaler_format[format->pad]; + + return 0; +} + +static int isc_scaler_set_fmt(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state, + struct v4l2_subdev_format *req_fmt) +{ + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd); + struct v4l2_mbus_framefmt *v4l2_try_fmt; + struct isc_format *fmt; + unsigned int i; + + /* Source format is fixed, we cannot change it */ + if (req_fmt->pad == ISC_SCALER_PAD_SOURCE) { + req_fmt->format = isc->scaler_format[ISC_SCALER_PAD_SOURCE]; + return 0; + } + + /* There is no limit on the frame size on the sink pad */ + v4l_bound_align_image(&req_fmt->format.width, 16, UINT_MAX, 0, + &req_fmt->format.height, 16, UINT_MAX, 0, 0); + + isc_scaler_prepare_fmt(&req_fmt->format); + + fmt = isc_find_format_by_code(isc, req_fmt->format.code, &i); + + if (!fmt) + fmt = &isc->formats_list[0]; + + req_fmt->format.code = fmt->mbus_code; + + if (req_fmt->which == V4L2_SUBDEV_FORMAT_TRY) { + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state, + req_fmt->pad); + *v4l2_try_fmt = req_fmt->format; + /* Trying on the sink pad makes the source pad change too */ + v4l2_try_fmt = v4l2_subdev_get_try_format(sd, sd_state, + ISC_SCALER_PAD_SOURCE); + *v4l2_try_fmt = req_fmt->format; + + v4l_bound_align_image(&v4l2_try_fmt->width, + 16, isc->max_width, 0, + &v4l2_try_fmt->height, + 16, isc->max_height, 0, 0); + /* if we are just trying, we are done */ + return 0; + } + + isc->scaler_format[ISC_SCALER_PAD_SINK] = req_fmt->format; + + /* The source pad is the same as the sink, but we have to crop it */ + isc->scaler_format[ISC_SCALER_PAD_SOURCE] = + isc->scaler_format[ISC_SCALER_PAD_SINK]; + v4l_bound_align_image + (&isc->scaler_format[ISC_SCALER_PAD_SOURCE].width, 16, + isc->max_width, 0, + &isc->scaler_format[ISC_SCALER_PAD_SOURCE].height, 16, + isc->max_height, 0, 0); + + return 0; +} + +static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state, + struct v4l2_subdev_mbus_code_enum *code) +{ + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd); + + /* + * All formats supported by the ISC are supported by the scaler. + * Advertise the formats which the ISC can take as input, as the scaler + * entity cropping is part of the PFE module (parallel front end) + */ + if (code->index < isc->formats_list_size) { + code->code = isc->formats_list[code->index].mbus_code; + return 0; + } + + return -EINVAL; +} + +static int isc_scaler_g_sel(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state, + struct v4l2_subdev_selection *sel) +{ + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd); + + if (sel->pad == ISC_SCALER_PAD_SOURCE) + return -EINVAL; + + if (sel->target != V4L2_SEL_TGT_CROP_BOUNDS && + sel->target != V4L2_SEL_TGT_CROP) + return -EINVAL; + + sel->r.height = isc->scaler_format[ISC_SCALER_PAD_SOURCE].height; + sel->r.width = isc->scaler_format[ISC_SCALER_PAD_SOURCE].width; + + sel->r.left = 0; + sel->r.top = 0; + + return 0; +} + +static int isc_scaler_init_cfg(struct v4l2_subdev *sd, + struct v4l2_subdev_state *sd_state) +{ + struct v4l2_mbus_framefmt *v4l2_try_fmt = + v4l2_subdev_get_try_format(sd, sd_state, 0); + struct v4l2_rect *try_crop; + struct isc_device *isc = container_of(sd, struct isc_device, scaler_sd); + + *v4l2_try_fmt = isc->scaler_format[ISC_SCALER_PAD_SOURCE]; + + try_crop = v4l2_subdev_get_try_crop(sd, sd_state, 0); + + try_crop->top = 0; + try_crop->left = 0; + try_crop->width = v4l2_try_fmt->width; + try_crop->height = v4l2_try_fmt->height; + + return 0; +} + +static const struct v4l2_subdev_pad_ops isc_scaler_pad_ops = { + .enum_mbus_code = isc_scaler_enum_mbus_code, + .set_fmt = isc_scaler_set_fmt, + .get_fmt = isc_scaler_get_fmt, + .get_selection = isc_scaler_g_sel, + .init_cfg = isc_scaler_init_cfg, +}; + +static const struct v4l2_subdev_ops xisc_scaler_subdev_ops = { + .pad = &isc_scaler_pad_ops, +}; + +int isc_scaler_init(struct isc_device *isc) +{ + int ret; + + v4l2_subdev_init(&isc->scaler_sd, &xisc_scaler_subdev_ops); + + isc->scaler_sd.owner = THIS_MODULE; + isc->scaler_sd.dev = isc->dev; + snprintf(isc->scaler_sd.name, sizeof(isc->scaler_sd.name), + "atmel_isc_scaler"); + + isc->scaler_sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; + isc->scaler_sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER; + isc->scaler_pads[ISC_SCALER_PAD_SINK].flags = MEDIA_PAD_FL_SINK; + isc->scaler_pads[ISC_SCALER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE; + + isc_scaler_prepare_fmt(&isc->scaler_format[ISC_SCALER_PAD_SOURCE]); + isc->scaler_format[ISC_SCALER_PAD_SOURCE].height = isc->max_height; + isc->scaler_format[ISC_SCALER_PAD_SOURCE].width = isc->max_width; + isc->scaler_format[ISC_SCALER_PAD_SOURCE].code = + isc->formats_list[0].mbus_code; + + isc->scaler_format[ISC_SCALER_PAD_SINK] = + isc->scaler_format[ISC_SCALER_PAD_SOURCE]; + + ret = media_entity_pads_init(&isc->scaler_sd.entity, + ISC_SCALER_PADS_NUM, + isc->scaler_pads); + if (ret < 0) { + dev_err(isc->dev, "scaler sd media entity init failed\n"); + return ret; + } + + ret = v4l2_device_register_subdev(&isc->v4l2_dev, &isc->scaler_sd); + if (ret < 0) { + dev_err(isc->dev, "scaler sd failed to register subdev\n"); + return ret; + } + + return ret; +} +EXPORT_SYMBOL_GPL(isc_scaler_init); + +int isc_scaler_link(struct isc_device *isc) +{ + int ret; + + ret = media_create_pad_link(&isc->current_subdev->sd->entity, + isc->remote_pad, &isc->scaler_sd.entity, + ISC_SCALER_PAD_SINK, + MEDIA_LNK_FL_ENABLED | + MEDIA_LNK_FL_IMMUTABLE); + + if (ret < 0) { + dev_err(isc->dev, "Failed to create pad link: %s to %s\n", + isc->current_subdev->sd->entity.name, + isc->scaler_sd.entity.name); + return ret; + } + + dev_dbg(isc->dev, "link with %s pad: %d\n", + isc->current_subdev->sd->name, isc->remote_pad); + + ret = media_create_pad_link(&isc->scaler_sd.entity, + ISC_SCALER_PAD_SOURCE, + &isc->video_dev.entity, ISC_PAD_SINK, + MEDIA_LNK_FL_ENABLED | + MEDIA_LNK_FL_IMMUTABLE); + + if (ret < 0) { + dev_err(isc->dev, "Failed to create pad link: %s to %s\n", + isc->scaler_sd.entity.name, + isc->video_dev.entity.name); + return ret; + } + + dev_dbg(isc->dev, "link with %s pad: %d\n", isc->scaler_sd.name, + ISC_SCALER_PAD_SOURCE); + + return ret; +} +EXPORT_SYMBOL_GPL(isc_scaler_link); + diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h index f9ad7ec6bd13..9cc69c3ae26d 100644 --- a/drivers/media/platform/atmel/atmel-isc.h +++ b/drivers/media/platform/atmel/atmel-isc.h @@ -183,6 +183,17 @@ struct isc_reg_offsets { u32 his_entry; }; +enum isc_mc_pads { + ISC_PAD_SINK = 0, + ISC_PADS_NUM = 1, +}; + +enum isc_scaler_pads { + ISC_SCALER_PAD_SINK = 0, + ISC_SCALER_PAD_SOURCE = 1, + ISC_SCALER_PADS_NUM = 2, +}; + /* * struct isc_device - ISC device driver data/config struct * @regmap: Register map @@ -258,6 +269,12 @@ struct isc_reg_offsets { * be used as an input to the controller * @controller_formats_size: size of controller_formats array * @formats_list_size: size of formats_list array + * @pads: media controller pads for isc video entity + * @mdev: media device that is registered by the isc + * @remote_pad: remote pad on the connected subdevice + * @scaler_sd: subdevice for the scaler that isc registers + * @scaler_pads: media controller pads for the scaler subdevice + * @scaler_format: current format for the scaler subdevice */ struct isc_device { struct regmap *regmap; @@ -346,6 +363,19 @@ struct isc_device { struct isc_format *formats_list; u32 controller_formats_size; u32 formats_list_size; + + struct { + struct media_pad pads[ISC_PADS_NUM]; + struct media_device mdev; + + u32 remote_pad; + }; + + struct { + struct v4l2_subdev scaler_sd; + struct media_pad scaler_pads[ISC_SCALER_PADS_NUM]; + struct v4l2_mbus_framefmt scaler_format[ISC_SCALER_PADS_NUM]; + }; }; extern const struct regmap_config isc_regmap_config; @@ -357,4 +387,11 @@ int isc_clk_init(struct isc_device *isc); void isc_subdev_cleanup(struct isc_device *isc); void isc_clk_cleanup(struct isc_device *isc); +int isc_scaler_link(struct isc_device *isc); +int isc_scaler_init(struct isc_device *isc); +int isc_mc_init(struct isc_device *isc, u32 ver); +void isc_mc_cleanup(struct isc_device *isc); + +struct isc_format *isc_find_format_by_code(struct isc_device *isc, + unsigned int code, int *index); #endif diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c index c5b9563e36cb..c244682ea22f 100644 --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c @@ -553,6 +553,12 @@ static int atmel_isc_probe(struct platform_device *pdev) break; } + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver); + + ret = isc_mc_init(isc, ver); + if (ret < 0) + goto isc_probe_mc_init_err; + pm_runtime_set_active(dev); pm_runtime_enable(dev); pm_request_idle(dev); @@ -562,7 +568,7 @@ static int atmel_isc_probe(struct platform_device *pdev) ret = clk_prepare_enable(isc->ispck); if (ret) { dev_err(dev, "failed to enable ispck: %d\n", ret); - goto cleanup_subdev; + goto isc_probe_mc_init_err; } /* ispck should be greater or equal to hclock */ @@ -572,7 +578,6 @@ static int atmel_isc_probe(struct platform_device *pdev) goto unprepare_clk; } - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver); dev_info(dev, "Microchip ISC version %x\n", ver); return 0; @@ -580,6 +585,9 @@ static int atmel_isc_probe(struct platform_device *pdev) unprepare_clk: clk_disable_unprepare(isc->ispck); +isc_probe_mc_init_err: + isc_mc_cleanup(isc); + cleanup_subdev: isc_subdev_cleanup(isc); @@ -600,6 +608,8 @@ static int atmel_isc_remove(struct platform_device *pdev) pm_runtime_disable(&pdev->dev); + isc_mc_cleanup(isc); + isc_subdev_cleanup(isc); v4l2_device_unregister(&isc->v4l2_dev); diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c index 07a80b08bc54..9dc75eed0098 100644 --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c @@ -547,15 +547,23 @@ static int microchip_xisc_probe(struct platform_device *pdev) break; } + regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver); + + ret = isc_mc_init(isc, ver); + if (ret < 0) + goto isc_probe_mc_init_err; + pm_runtime_set_active(dev); pm_runtime_enable(dev); pm_request_idle(dev); - regmap_read(isc->regmap, ISC_VERSION + isc->offsets.version, &ver); dev_info(dev, "Microchip XISC version %x\n", ver); return 0; +isc_probe_mc_init_err: + isc_mc_cleanup(isc); + cleanup_subdev: isc_subdev_cleanup(isc); @@ -576,6 +584,8 @@ static int microchip_xisc_remove(struct platform_device *pdev) pm_runtime_disable(&pdev->dev); + isc_mc_cleanup(isc); + isc_subdev_cleanup(isc); v4l2_device_unregister(&isc->v4l2_dev); From patchwork Wed Mar 9 16:37:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eugen Hristev X-Patchwork-Id: 549845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B6FDC433F5 for ; Wed, 9 Mar 2022 16:46:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236357AbiCIQrc (ORCPT ); Wed, 9 Mar 2022 11:47:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37232 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235869AbiCIQpo (ORCPT ); Wed, 9 Mar 2022 11:45:44 -0500 Received: from esa.microchip.iphmx.com (esa.microchip.iphmx.com [68.232.154.123]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFA73A1BCF; Wed, 9 Mar 2022 08:39:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=microchip.com; i=@microchip.com; q=dns/txt; s=mchp; t=1646843958; x=1678379958; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LThCuAfW7xET2MZo5q9C4gm+wpOhS9QVzkmQqBCA9Y8=; b=e8HmjTm2A5yYx+3mzJ/a/erAsG233l3StTZdsdIUDGRmv2XE8HmuY3r2 lWmovkBbi87ynAJd6WJHPEyp4yn329iTVUbaZMjKAvFLHlK2NzMByteHB nX8GBJny2X9W/LC8mu57arUN1ZU8kEiyXmV2in8+s5xa6A6wPSRHG4cb7 ddN1TeBvBOMzw0Be0oKPi015POKgnFTPXXj9l+l/MoS0ARaBYtzZyjprf 7N0SqvpLnHZg4JrAtUBAOn2gzx8kt6/AXGD0aoUpNtnfSSdaeE3XAK8WP 0bJ9jdr5KsuRxhA7B5iIhFJZfY3DMowFZWz/YBbxmr7rwv233zehk7U0Q A==; X-IronPort-AV: E=Sophos;i="5.90,167,1643698800"; d="scan'208";a="148642762" Received: from smtpout.microchip.com (HELO email.microchip.com) ([198.175.253.82]) by esa4.microchip.iphmx.com with ESMTP/TLS/AES256-SHA256; 09 Mar 2022 09:39:18 -0700 Received: from chn-vm-ex03.mchp-main.com (10.10.85.151) by chn-vm-ex02.mchp-main.com (10.10.85.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Wed, 9 Mar 2022 09:39:16 -0700 Received: from ROB-ULT-M18282.microchip.com (10.10.115.15) by chn-vm-ex03.mchp-main.com (10.10.85.151) with Microsoft SMTP Server id 15.1.2375.17 via Frontend Transport; Wed, 9 Mar 2022 09:39:07 -0700 From: Eugen Hristev To: , CC: , , , , , , Eugen Hristev Subject: [PATCH v8 06/13] media: atmel: atmel-isc-base: use mutex to lock awb workqueue from streaming Date: Wed, 9 Mar 2022 18:37:51 +0200 Message-ID: <20220309163758.2672727-7-eugen.hristev@microchip.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220309163758.2672727-1-eugen.hristev@microchip.com> References: <20220309163758.2672727-1-eugen.hristev@microchip.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org The AWB workqueue runs in a kernel thread and needs to be synchronized w.r.t. the streaming status. It is possible that streaming is stopped while the AWB workq is running. In this case it is likely that the check for vb2_start_streaming_called is done at one point in time, but the AWB computations are done later, including a call to isc_update_profile, which requires streaming to be started. Thus , isc_update_profile will fail if during this operation sequence the streaming was stopped. To solve this issue, a mutex is added, that will serialize the awb work and streaming stopping, with the mention that either streaming is stopped completely including termination of the last frame is done, and after that the AWB work can check stream status and stop; either first AWB work is completed and after that the streaming can stop correctly. The awb spin lock cannot be used since this spinlock is taken in the same context and using it in the stop streaming will result in a recursion BUG. Signed-off-by: Eugen Hristev Reviewed-by: Jacopo Mondi --- drivers/media/platform/atmel/atmel-isc-base.c | 29 ++++++++++++++++--- drivers/media/platform/atmel/atmel-isc.h | 2 ++ 2 files changed, 27 insertions(+), 4 deletions(-) diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c index 448bf281c61a..ee1dda6707a0 100644 --- a/drivers/media/platform/atmel/atmel-isc-base.c +++ b/drivers/media/platform/atmel/atmel-isc-base.c @@ -401,6 +401,7 @@ static void isc_stop_streaming(struct vb2_queue *vq) struct isc_buffer *buf; int ret; + mutex_lock(&isc->awb_mutex); v4l2_ctrl_activate(isc->do_wb_ctrl, false); isc->stop = true; @@ -410,6 +411,8 @@ static void isc_stop_streaming(struct vb2_queue *vq) v4l2_err(&isc->v4l2_dev, "Timeout waiting for end of the capture\n"); + mutex_unlock(&isc->awb_mutex); + /* Disable DMA interrupt */ regmap_write(isc->regmap, ISC_INTDIS, ISC_INT_DDONE); @@ -1397,10 +1400,6 @@ static void isc_awb_work(struct work_struct *w) u32 min, max; int ret; - /* streaming is not active anymore */ - if (isc->stop) - return; - if (ctrls->hist_stat != HIST_ENABLED) return; @@ -1455,7 +1454,24 @@ static void isc_awb_work(struct work_struct *w) } regmap_write(regmap, ISC_HIS_CFG + isc->offsets.his, hist_id | baysel | ISC_HIS_CFG_RAR); + + /* + * We have to make sure the streaming has not stopped meanwhile. + * ISC requires a frame to clock the internal profile update. + * To avoid issues, lock the sequence with a mutex + */ + mutex_lock(&isc->awb_mutex); + + /* streaming is not active anymore */ + if (isc->stop) { + mutex_unlock(&isc->awb_mutex); + return; + }; + isc_update_profile(isc); + + mutex_unlock(&isc->awb_mutex); + /* if awb has been disabled, we don't need to start another histogram */ if (ctrls->awb) regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ); @@ -1548,6 +1564,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl) */ v4l2_ctrl_activate(isc->do_wb_ctrl, false); } + mutex_unlock(&isc->awb_mutex); /* if we have autowhitebalance on, start histogram procedure */ if (ctrls->awb == ISC_WB_AUTO && @@ -1740,6 +1757,7 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier, { struct isc_device *isc = container_of(notifier->v4l2_dev, struct isc_device, v4l2_dev); + mutex_destroy(&isc->awb_mutex); cancel_work_sync(&isc->awb_work); video_unregister_device(&isc->video_dev); v4l2_ctrl_handler_free(&isc->ctrls.handler); @@ -1850,6 +1868,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier) isc->current_subdev = container_of(notifier, struct isc_subdev_entity, notifier); mutex_init(&isc->lock); + mutex_init(&isc->awb_mutex); + init_completion(&isc->comp); /* Initialize videobuf2 queue */ @@ -1930,6 +1950,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier) video_unregister_device(vdev); isc_async_complete_err: + mutex_destroy(&isc->awb_mutex); mutex_destroy(&isc->lock); return ret; } diff --git a/drivers/media/platform/atmel/atmel-isc.h b/drivers/media/platform/atmel/atmel-isc.h index 9cc69c3ae26d..f98f25a55e73 100644 --- a/drivers/media/platform/atmel/atmel-isc.h +++ b/drivers/media/platform/atmel/atmel-isc.h @@ -229,6 +229,7 @@ enum isc_scaler_pads { * * @lock: lock for serializing userspace file operations * with ISC operations + * @awb_mutex: serialize access to streaming status from awb work queue * @awb_lock: lock for serializing awb work queue operations * with DMA/buffer operations * @@ -307,6 +308,7 @@ struct isc_device { struct work_struct awb_work; struct mutex lock; + struct mutex awb_mutex; spinlock_t awb_lock; struct regmap_field *pipeline[ISC_PIPE_LINE_NODE_NUM]; From patchwork Wed Mar 9 16:37:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eugen Hristev X-Patchwork-Id: 549847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0080BC4167B for ; Wed, 9 Mar 2022 16:46:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235964AbiCIQrX (ORCPT ); Wed, 9 Mar 2022 11:47:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235884AbiCIQpq (ORCPT ); Wed, 9 Mar 2022 11:45:46 -0500 Received: from esa.microchip.iphmx.com (esa.microchip.iphmx.com [68.232.154.123]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D6941C7C3A; Wed, 9 Mar 2022 08:39:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=microchip.com; i=@microchip.com; q=dns/txt; s=mchp; t=1646843980; x=1678379980; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=inbCfOApjHcZj6rPnStLuoDA+QTiQF8O+NIL6iLwJf0=; b=grnzG/Ao73HwBdY+9V3WesUA3BeSLrDfozJYUPNBrQLEDWlHUctArzGI a9R3n6lCGkYgv+S0xHYghy5mxeaWC8NDIQMorLgTM+53BvCj7Cg8nuL/0 95g5mv7I/OGFb62595C87go2oDUv5TJpJtT4YYrzAcrmRH4WIR+Gw2f8u 2b4Ev8HkBtJWIY6ejtUt7tlqBbAc9SXinn0PfsOspCzR5k22nGeqT6axn FfQ7Bp0Kt2ePD7m2eOhII96Oc57Ha5Kcj3DbKK62GZ4VO7jhANetT8ag5 TZQI8FYO48hcHWkaQrcYthLIBDHhpDVdqPcr1JNoE83F7EY0lsTEuf9I2 w==; X-IronPort-AV: E=Sophos;i="5.90,167,1643698800"; d="scan'208";a="148642846" Received: from smtpout.microchip.com (HELO email.microchip.com) ([198.175.253.82]) by esa4.microchip.iphmx.com with ESMTP/TLS/AES256-SHA256; 09 Mar 2022 09:39:34 -0700 Received: from chn-vm-ex03.mchp-main.com (10.10.85.151) by chn-vm-ex01.mchp-main.com (10.10.85.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Wed, 9 Mar 2022 09:39:29 -0700 Received: from ROB-ULT-M18282.microchip.com (10.10.115.15) by chn-vm-ex03.mchp-main.com (10.10.85.151) with Microsoft SMTP Server id 15.1.2375.17 via Frontend Transport; Wed, 9 Mar 2022 09:39:18 -0700 From: Eugen Hristev To: , CC: , , , , , , Eugen Hristev , Jacopo Mondi Subject: [PATCH v8 07/13] media: atmel: atmel-isc: compact the controller formats list Date: Wed, 9 Mar 2022 18:37:52 +0200 Message-ID: <20220309163758.2672727-8-eugen.hristev@microchip.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220309163758.2672727-1-eugen.hristev@microchip.com> References: <20220309163758.2672727-1-eugen.hristev@microchip.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Compact the list array to be more readable. No other changes, only cosmetic. Signed-off-by: Eugen Hristev Reviewed-by: Jacopo Mondi --- .../media/platform/atmel/atmel-sama5d2-isc.c | 51 ++++++---------- .../media/platform/atmel/atmel-sama7g5-isc.c | 60 +++++++------------ 2 files changed, 37 insertions(+), 74 deletions(-) diff --git a/drivers/media/platform/atmel/atmel-sama5d2-isc.c b/drivers/media/platform/atmel/atmel-sama5d2-isc.c index 025c3e8a7e95..d96ee3373889 100644 --- a/drivers/media/platform/atmel/atmel-sama5d2-isc.c +++ b/drivers/media/platform/atmel/atmel-sama5d2-isc.c @@ -60,56 +60,39 @@ static const struct isc_format sama5d2_controller_formats[] = { { .fourcc = V4L2_PIX_FMT_ARGB444, - }, - { + }, { .fourcc = V4L2_PIX_FMT_ARGB555, - }, - { + }, { .fourcc = V4L2_PIX_FMT_RGB565, - }, - { + }, { .fourcc = V4L2_PIX_FMT_ABGR32, - }, - { + }, { .fourcc = V4L2_PIX_FMT_XBGR32, - }, - { + }, { .fourcc = V4L2_PIX_FMT_YUV420, - }, - { + }, { .fourcc = V4L2_PIX_FMT_YUYV, - }, - { + }, { .fourcc = V4L2_PIX_FMT_YUV422P, - }, - { + }, { .fourcc = V4L2_PIX_FMT_GREY, - }, - { + }, { .fourcc = V4L2_PIX_FMT_Y10, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SBGGR8, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SGBRG8, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SGRBG8, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SRGGB8, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SBGGR10, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SGBRG10, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SGRBG10, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SRGGB10, }, }; diff --git a/drivers/media/platform/atmel/atmel-sama7g5-isc.c b/drivers/media/platform/atmel/atmel-sama7g5-isc.c index 9dc75eed0098..e07ae188c15f 100644 --- a/drivers/media/platform/atmel/atmel-sama7g5-isc.c +++ b/drivers/media/platform/atmel/atmel-sama7g5-isc.c @@ -63,65 +63,45 @@ static const struct isc_format sama7g5_controller_formats[] = { { .fourcc = V4L2_PIX_FMT_ARGB444, - }, - { + }, { .fourcc = V4L2_PIX_FMT_ARGB555, - }, - { + }, { .fourcc = V4L2_PIX_FMT_RGB565, - }, - { + }, { .fourcc = V4L2_PIX_FMT_ABGR32, - }, - { + }, { .fourcc = V4L2_PIX_FMT_XBGR32, - }, - { + }, { .fourcc = V4L2_PIX_FMT_YUV420, - }, - { + }, { .fourcc = V4L2_PIX_FMT_UYVY, - }, - { + }, { .fourcc = V4L2_PIX_FMT_VYUY, - }, - { + }, { .fourcc = V4L2_PIX_FMT_YUYV, - }, - { + }, { .fourcc = V4L2_PIX_FMT_YUV422P, - }, - { + }, { .fourcc = V4L2_PIX_FMT_GREY, - }, - { + }, { .fourcc = V4L2_PIX_FMT_Y10, - }, - { + }, { .fourcc = V4L2_PIX_FMT_Y16, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SBGGR8, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SGBRG8, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SGRBG8, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SRGGB8, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SBGGR10, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SGBRG10, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SGRBG10, - }, - { + }, { .fourcc = V4L2_PIX_FMT_SRGGB10, }, }; From patchwork Wed Mar 9 16:37:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eugen Hristev X-Patchwork-Id: 549846 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71B6CC433F5 for ; Wed, 9 Mar 2022 16:46:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236340AbiCIQr0 (ORCPT ); Wed, 9 Mar 2022 11:47:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235946AbiCIQpq (ORCPT ); Wed, 9 Mar 2022 11:45:46 -0500 Received: from esa.microchip.iphmx.com (esa.microchip.iphmx.com [68.232.154.123]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C18A9EDF28; Wed, 9 Mar 2022 08:40:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=microchip.com; i=@microchip.com; q=dns/txt; s=mchp; t=1646844021; x=1678380021; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LPz0O/j6T/R5IOZ/0HdGMFHbVlsAXUy78/qaH+crieA=; b=dx5F9qc1ro7xXhENcK3+nLT0hRREb2N67Yj30Q5ooaBf0XLzZMLJGnIL qSQYb2Er3brkGhS2QUBOezC6x7uOE0H2/ihI6tSXJP0axa6f8BH4cJVvk YfFGfhdCFgI69F1/MgXJqFDt9yJNpz4tc3cNzsctf1C8M2YuKuxXIQlMN wrTFAoSYqg24+b5HkjDqeLsHg5kn+R2hALojZBUR8bFX/2IG9lk0IJqd2 GpDn3rmnVt7lgk84HA4OH350dqKObOYXyd8tXKQKQCcukstkzu4O+zkcr 8kWXguu37vK8pWvc35OYlafIeXlLR/hOZS+g4MBj66dbgEmHJur+O7xlB Q==; X-IronPort-AV: E=Sophos;i="5.90,167,1643698800"; d="scan'208";a="148643235" Received: from smtpout.microchip.com (HELO email.microchip.com) ([198.175.253.82]) by esa4.microchip.iphmx.com with ESMTP/TLS/AES256-SHA256; 09 Mar 2022 09:40:20 -0700 Received: from chn-vm-ex03.mchp-main.com (10.10.85.151) by chn-vm-ex01.mchp-main.com (10.10.85.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Wed, 9 Mar 2022 09:40:19 -0700 Received: from ROB-ULT-M18282.microchip.com (10.10.115.15) by chn-vm-ex03.mchp-main.com (10.10.85.151) with Microsoft SMTP Server id 15.1.2375.17 via Frontend Transport; Wed, 9 Mar 2022 09:40:11 -0700 From: Eugen Hristev To: , CC: , , , , , , Eugen Hristev Subject: [PATCH v8 11/13] ARM: dts: at91: sama7g5: add nodes for video capture Date: Wed, 9 Mar 2022 18:37:56 +0200 Message-ID: <20220309163758.2672727-12-eugen.hristev@microchip.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220309163758.2672727-1-eugen.hristev@microchip.com> References: <20220309163758.2672727-1-eugen.hristev@microchip.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Add node for the XISC (eXtended Image Sensor Controller) and CSI2DC (csi2 demux controller). These nodes represent the top level of the video capture hardware pipeline and are directly connected in hardware. Signed-off-by: Eugen Hristev --- arch/arm/boot/dts/sama7g5.dtsi | 49 ++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/arch/arm/boot/dts/sama7g5.dtsi b/arch/arm/boot/dts/sama7g5.dtsi index eddcfbf4d223..de43f854ce47 100644 --- a/arch/arm/boot/dts/sama7g5.dtsi +++ b/arch/arm/boot/dts/sama7g5.dtsi @@ -266,6 +266,55 @@ sdmmc2: mmc@e120c000 { status = "disabled"; }; + csi2dc: csi2dc@e1404000 { + compatible = "microchip,sama7g5-csi2dc"; + reg = <0xe1404000 0x500>; + clocks = <&pmc PMC_TYPE_PERIPHERAL 34>, <&xisc>; + clock-names = "pclk", "scck"; + assigned-clocks = <&xisc>; + assigned-clock-rates = <266000000>; + + ports { + #address-cells = <1>; + #size-cells = <0>; + port@0 { + reg = <0>; + csi2dc_in: endpoint { + }; + }; + + port@1 { + reg = <1>; + csi2dc_out: endpoint { + bus-width = <14>; + hsync-active = <1>; + vsync-active = <1>; + remote-endpoint = <&xisc_in>; + }; + }; + }; + }; + + xisc: xisc@e1408000 { + compatible = "microchip,sama7g5-isc"; + reg = <0xe1408000 0x2000>; + interrupts = ; + clocks = <&pmc PMC_TYPE_PERIPHERAL 56>; + clock-names = "hclock"; + #clock-cells = <0>; + clock-output-names = "isc-mck"; + + port { + xisc_in: endpoint { + bus-type = <5>; /* Parallel */ + bus-width = <14>; + hsync-active = <1>; + vsync-active = <1>; + remote-endpoint = <&csi2dc_out>; + }; + }; + }; + pwm: pwm@e1604000 { compatible = "microchip,sama7g5-pwm", "atmel,sama5d2-pwm"; reg = <0xe1604000 0x4000>; From patchwork Wed Mar 9 16:37:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eugen Hristev X-Patchwork-Id: 549848 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE1C7C4167D for ; Wed, 9 Mar 2022 16:46:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236097AbiCIQrW (ORCPT ); Wed, 9 Mar 2022 11:47:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235920AbiCIQpq (ORCPT ); Wed, 9 Mar 2022 11:45:46 -0500 Received: from esa.microchip.iphmx.com (esa.microchip.iphmx.com [68.232.153.233]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ACE45F4D27; Wed, 9 Mar 2022 08:40:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=microchip.com; i=@microchip.com; q=dns/txt; s=mchp; t=1646844033; x=1678380033; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DYznwx5dXH097XmlK1oyi9a51is+vbtNveAKiaCJeUg=; b=ILNZBkxhOM8VrU0Roq7LFNQfMwYEKyr+vYzV3gN0I00g1DAZJCRJysMy 4PSb0C120ptqTryCY7X0BoCI7KazCCOLSKwihRkg8NcMJTYo4aQr6+IEe OZhD+NJKKbxI9pxT55Ke+EEsCdoPAYcYmsdEXlxmKvvjx+ajGTkgM0dBX uhNpfnr/T1CfkkHcsXaa1ndiV3oyUVPwdqvNQHHt+fFSZjLxrsTjpe+gl jfbqGb2Nb/lOmiHc2u8W2vjlnUhBoKw8cZYGXVnJOiTFdzn26iQchsap8 ACTZltkp1HV7fSEJbJCwebWl5UzEBt8My5Ud661Bk7s0mzTGpX2gt+Uh0 Q==; X-IronPort-AV: E=Sophos;i="5.90,167,1643698800"; d="scan'208";a="165123130" Received: from smtpout.microchip.com (HELO email.microchip.com) ([198.175.253.82]) by esa1.microchip.iphmx.com with ESMTP/TLS/AES256-SHA256; 09 Mar 2022 09:40:33 -0700 Received: from chn-vm-ex03.mchp-main.com (10.10.85.151) by chn-vm-ex04.mchp-main.com (10.10.85.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.17; Wed, 9 Mar 2022 09:40:31 -0700 Received: from ROB-ULT-M18282.microchip.com (10.10.115.15) by chn-vm-ex03.mchp-main.com (10.10.85.151) with Microsoft SMTP Server id 15.1.2375.17 via Frontend Transport; Wed, 9 Mar 2022 09:40:20 -0700 From: Eugen Hristev To: , CC: , , , , , , Eugen Hristev Subject: [PATCH v8 12/13] ARM: configs: at91: sama7: add xisc and csi2dc Date: Wed, 9 Mar 2022 18:37:57 +0200 Message-ID: <20220309163758.2672727-13-eugen.hristev@microchip.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220309163758.2672727-1-eugen.hristev@microchip.com> References: <20220309163758.2672727-1-eugen.hristev@microchip.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Enable XISC and CSI2DC drivers. Signed-off-by: Eugen Hristev --- arch/arm/configs/sama7_defconfig | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm/configs/sama7_defconfig b/arch/arm/configs/sama7_defconfig index 0368068e04d9..9918cff93e5b 100644 --- a/arch/arm/configs/sama7_defconfig +++ b/arch/arm/configs/sama7_defconfig @@ -127,6 +127,8 @@ CONFIG_MEDIA_SUPPORT_FILTER=y CONFIG_MEDIA_CAMERA_SUPPORT=y CONFIG_MEDIA_PLATFORM_SUPPORT=y CONFIG_V4L_PLATFORM_DRIVERS=y +CONFIG_VIDEO_ATMEL_XISC=y +CONFIG_VIDEO_MICROCHIP_CSI2DC=y CONFIG_VIDEO_IMX219=m CONFIG_VIDEO_IMX274=m CONFIG_VIDEO_OV5647=m