From patchwork Mon Feb 26 08:28:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shengjiu Wang X-Patchwork-Id: 776118 Received: from inva020.nxp.com (inva020.nxp.com [92.121.34.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6FB9A4D5AC; Mon, 26 Feb 2024 08:44:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=92.121.34.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937059; cv=none; b=ealoJIOXkcS1CWooUFGAHBMOe6UqxfjvaFijMPh8HdLJxvsEuznSMIKJcaXm3rz0yuRS4qbsHAXZvRF/YMlaZaEQeDGNEDarAhWOmbO3UyJkgbswzRn4QL0m19MjWieFfk7ze3B62SL0oDqkrOreameAY34nsF2mfJRfnxWSj6E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937059; c=relaxed/simple; bh=GgIga9Nj+62L3GQWRmi334z4f2Zo3wnisgcFIUt1rAo=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References; b=HTJQYM5wKP/9eM0N2Sc8jxxN/ygA/HAz8Dyd3L+EA9rvwxpKYa3KyfaP+xA6bllH6ZYezKFozf/ftuLlS8WNrUF/QQ1hIXSceIw/b2INlCdMARMz0sfA3LxM32ygvU+NfoJATqNbL4MDKg1y4djB38Vch5hNF2Yn2oSQnrVomus= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com; spf=pass smtp.mailfrom=nxp.com; arc=none smtp.client-ip=92.121.34.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nxp.com Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 8D4731A0134; Mon, 26 Feb 2024 09:44:10 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 27D5E1A010D; Mon, 26 Feb 2024 09:44:10 +0100 (CET) Received: from localhost.localdomain (shlinux2.ap.freescale.net [10.192.224.44]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 0D5B4180222A; Mon, 26 Feb 2024 16:44:07 +0800 (+08) From: Shengjiu Wang To: hverkuil@xs4all.nl, sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, shengjiu.wang@gmail.com, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v13 01/16] media: v4l2-ctrls: add support for fraction_bits Date: Mon, 26 Feb 2024 16:28:14 +0800 Message-Id: <1708936109-11587-2-git-send-email-shengjiu.wang@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> References: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Hans Verkuil This adds support for the fraction_bits field, used with integer controls. This allows fixed point formats to be described. The fraction_bits field is only exposed through VIDIOC_QUERY_EXT_CTRL. For a given signed two's complement Qf fixed point value 'f' equals fraction_bits. Signed-off-by: Hans Verkuil --- .../media/v4l/vidioc-queryctrl.rst | 11 ++- drivers/media/v4l2-core/v4l2-ctrls-api.c | 1 + drivers/media/v4l2-core/v4l2-ctrls-core.c | 93 +++++++++++++++---- include/media/v4l2-ctrls.h | 7 +- include/uapi/linux/videodev2.h | 3 +- 5 files changed, 95 insertions(+), 20 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst index 4d38acafe8e1..e65c7e5d78ec 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst @@ -267,7 +267,16 @@ See also the examples in :ref:`control`. - The size of each dimension. The first ``nr_of_dims`` elements of this array must be non-zero, all remaining elements must be zero. * - __u32 - - ``reserved``\ [32] + - ``fraction_bits`` + - The number of least significant bits of the control value that + form the fraction of the fixed point value. This is 0 if the value + is a regular integer. This can be used with all integer control types + (``INTEGER``, ``INTEGER64``, ``U8``, ``U16`` and ``U32``). + For the signed types the signed two's complement representation is used. + This field applies to the control value as well as the ``minimum``, + ``maximum``, ``step`` and ``default_value`` fields. + * - __u32 + - ``reserved``\ [31] - Reserved for future extensions. Applications and drivers must set the array to zero. diff --git a/drivers/media/v4l2-core/v4l2-ctrls-api.c b/drivers/media/v4l2-core/v4l2-ctrls-api.c index d9a422017bd9..ef16b00421ec 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-api.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-api.c @@ -1101,6 +1101,7 @@ int v4l2_query_ext_ctrl(struct v4l2_ctrl_handler *hdl, struct v4l2_query_ext_ctr qc->elems = ctrl->elems; qc->nr_of_dims = ctrl->nr_of_dims; memcpy(qc->dims, ctrl->dims, qc->nr_of_dims * sizeof(qc->dims[0])); + qc->fraction_bits = ctrl->fraction_bits; qc->minimum = ctrl->minimum; qc->maximum = ctrl->maximum; qc->default_value = ctrl->default_value; diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c index c4d995f32191..d83a37198bb5 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-core.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c @@ -252,12 +252,61 @@ void v4l2_ctrl_type_op_init(const struct v4l2_ctrl *ctrl, u32 from_idx, } EXPORT_SYMBOL(v4l2_ctrl_type_op_init); +static void v4l2_ctrl_log_fp(s64 v, unsigned int fraction_bits) +{ + s64 i, f, mask; + + if (!fraction_bits) { + pr_cont("%lld", v); + return; + } + + mask = (1ULL << fraction_bits) - 1; + + /* + * Note: this function does not support fixed point u64 with + * fraction_bits set to 64. At the moment there is no U64 + * control type, but if that is added, then this code will have + * to add support for it. + */ + if (fraction_bits >= 63) + i = v < 0 ? -1 : 0; + else + i = div64_s64(v, 1LL << fraction_bits); + + f = v < 0 ? -((-v) & mask) : (v & mask); + + if (!f) { + pr_cont("%lld", i); + } else if (fraction_bits < 20) { + u64 div = 1ULL << fraction_bits; + + if (!i && f < 0) + pr_cont("-%lld/%llu", -f, div); + else if (!i) + pr_cont("%lld/%llu", f, div); + else if (i < 0 || f < 0) + pr_cont("-%lld-%llu/%llu", -i, -f, div); + else + pr_cont("%lld+%llu/%llu", i, f, div); + } else { + if (!i && f < 0) + pr_cont("-%lld/(2^%u)", -f, fraction_bits); + else if (!i) + pr_cont("%lld/(2^%u)", f, fraction_bits); + else if (i < 0 || f < 0) + pr_cont("-%lld-%llu/(2^%u)", -i, -f, fraction_bits); + else + pr_cont("%lld+%llu/(2^%u)", i, f, fraction_bits); + } +} + void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl) { union v4l2_ctrl_ptr ptr = ctrl->p_cur; if (ctrl->is_array) { - unsigned i; + unsigned int i; for (i = 0; i < ctrl->nr_of_dims; i++) pr_cont("[%u]", ctrl->dims[i]); @@ -266,7 +315,7 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl) switch (ctrl->type) { case V4L2_CTRL_TYPE_INTEGER: - pr_cont("%d", *ptr.p_s32); + v4l2_ctrl_log_fp(*ptr.p_s32, ctrl->fraction_bits); break; case V4L2_CTRL_TYPE_BOOLEAN: pr_cont("%s", *ptr.p_s32 ? "true" : "false"); @@ -281,19 +330,19 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl) pr_cont("0x%08x", *ptr.p_s32); break; case V4L2_CTRL_TYPE_INTEGER64: - pr_cont("%lld", *ptr.p_s64); + v4l2_ctrl_log_fp(*ptr.p_s64, ctrl->fraction_bits); break; case V4L2_CTRL_TYPE_STRING: pr_cont("%s", ptr.p_char); break; case V4L2_CTRL_TYPE_U8: - pr_cont("%u", (unsigned)*ptr.p_u8); + v4l2_ctrl_log_fp(*ptr.p_u8, ctrl->fraction_bits); break; case V4L2_CTRL_TYPE_U16: - pr_cont("%u", (unsigned)*ptr.p_u16); + v4l2_ctrl_log_fp(*ptr.p_u16, ctrl->fraction_bits); break; case V4L2_CTRL_TYPE_U32: - pr_cont("%u", (unsigned)*ptr.p_u32); + v4l2_ctrl_log_fp(*ptr.p_u32, ctrl->fraction_bits); break; case V4L2_CTRL_TYPE_H264_SPS: pr_cont("H264_SPS"); @@ -1753,11 +1802,12 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, u32 id, const char *name, enum v4l2_ctrl_type type, s64 min, s64 max, u64 step, s64 def, const u32 dims[V4L2_CTRL_MAX_DIMS], u32 elem_size, - u32 flags, const char * const *qmenu, + u32 fraction_bits, u32 flags, const char * const *qmenu, const s64 *qmenu_int, const union v4l2_ctrl_ptr p_def, void *priv) { struct v4l2_ctrl *ctrl; + unsigned int max_fraction_bits = 0; unsigned sz_extra; unsigned nr_of_dims = 0; unsigned elems = 1; @@ -1779,20 +1829,28 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, /* Prefill elem_size for all types handled by std_type_ops */ switch ((u32)type) { + case V4L2_CTRL_TYPE_INTEGER: + elem_size = sizeof(s32); + max_fraction_bits = 31; + break; case V4L2_CTRL_TYPE_INTEGER64: elem_size = sizeof(s64); + max_fraction_bits = 63; break; case V4L2_CTRL_TYPE_STRING: elem_size = max + 1; break; case V4L2_CTRL_TYPE_U8: elem_size = sizeof(u8); + max_fraction_bits = 8; break; case V4L2_CTRL_TYPE_U16: elem_size = sizeof(u16); + max_fraction_bits = 16; break; case V4L2_CTRL_TYPE_U32: elem_size = sizeof(u32); + max_fraction_bits = 32; break; case V4L2_CTRL_TYPE_MPEG2_SEQUENCE: elem_size = sizeof(struct v4l2_ctrl_mpeg2_sequence); @@ -1876,10 +1934,10 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, } /* Sanity checks */ - if (id == 0 || name == NULL || !elem_size || - id >= V4L2_CID_PRIVATE_BASE || - (type == V4L2_CTRL_TYPE_MENU && qmenu == NULL) || - (type == V4L2_CTRL_TYPE_INTEGER_MENU && qmenu_int == NULL)) { + if (id == 0 || !name || !elem_size || + fraction_bits > max_fraction_bits || id >= V4L2_CID_PRIVATE_BASE || + (type == V4L2_CTRL_TYPE_MENU && !qmenu) || + (type == V4L2_CTRL_TYPE_INTEGER_MENU && !qmenu_int)) { handler_set_err(hdl, -ERANGE); return NULL; } @@ -1940,6 +1998,7 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, ctrl->name = name; ctrl->type = type; ctrl->flags = flags; + ctrl->fraction_bits = fraction_bits; ctrl->minimum = min; ctrl->maximum = max; ctrl->step = step; @@ -2038,7 +2097,7 @@ struct v4l2_ctrl *v4l2_ctrl_new_custom(struct v4l2_ctrl_handler *hdl, ctrl = v4l2_ctrl_new(hdl, cfg->ops, cfg->type_ops, cfg->id, name, type, min, max, is_menu ? cfg->menu_skip_mask : step, def, - cfg->dims, cfg->elem_size, + cfg->dims, cfg->elem_size, cfg->fraction_bits, flags, qmenu, qmenu_int, cfg->p_def, priv); if (ctrl) ctrl->is_private = cfg->is_private; @@ -2063,7 +2122,7 @@ struct v4l2_ctrl *v4l2_ctrl_new_std(struct v4l2_ctrl_handler *hdl, return NULL; } return v4l2_ctrl_new(hdl, ops, NULL, id, name, type, - min, max, step, def, NULL, 0, + min, max, step, def, NULL, 0, 0, flags, NULL, NULL, ptr_null, NULL); } EXPORT_SYMBOL(v4l2_ctrl_new_std); @@ -2096,7 +2155,7 @@ struct v4l2_ctrl *v4l2_ctrl_new_std_menu(struct v4l2_ctrl_handler *hdl, return NULL; } return v4l2_ctrl_new(hdl, ops, NULL, id, name, type, - 0, max, mask, def, NULL, 0, + 0, max, mask, def, NULL, 0, 0, flags, qmenu, qmenu_int, ptr_null, NULL); } EXPORT_SYMBOL(v4l2_ctrl_new_std_menu); @@ -2128,7 +2187,7 @@ struct v4l2_ctrl *v4l2_ctrl_new_std_menu_items(struct v4l2_ctrl_handler *hdl, return NULL; } return v4l2_ctrl_new(hdl, ops, NULL, id, name, type, - 0, max, mask, def, NULL, 0, + 0, max, mask, def, NULL, 0, 0, flags, qmenu, NULL, ptr_null, NULL); } @@ -2150,7 +2209,7 @@ struct v4l2_ctrl *v4l2_ctrl_new_std_compound(struct v4l2_ctrl_handler *hdl, return NULL; } return v4l2_ctrl_new(hdl, ops, NULL, id, name, type, - min, max, step, def, NULL, 0, + min, max, step, def, NULL, 0, 0, flags, NULL, NULL, p_def, NULL); } EXPORT_SYMBOL(v4l2_ctrl_new_std_compound); @@ -2174,7 +2233,7 @@ struct v4l2_ctrl *v4l2_ctrl_new_int_menu(struct v4l2_ctrl_handler *hdl, return NULL; } return v4l2_ctrl_new(hdl, ops, NULL, id, name, type, - 0, max, 0, def, NULL, 0, + 0, max, 0, def, NULL, 0, 0, flags, NULL, qmenu_int, ptr_null, NULL); } EXPORT_SYMBOL(v4l2_ctrl_new_int_menu); diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h index 59679a42b3e7..c35514c5bf88 100644 --- a/include/media/v4l2-ctrls.h +++ b/include/media/v4l2-ctrls.h @@ -211,7 +211,8 @@ typedef void (*v4l2_ctrl_notify_fnc)(struct v4l2_ctrl *ctrl, void *priv); * except for dynamic arrays. In that case it is in the range of * 1 to @p_array_alloc_elems. * @dims: The size of each dimension. - * @nr_of_dims:The number of dimensions in @dims. + * @nr_of_dims: The number of dimensions in @dims. + * @fraction_bits: The number of fraction bits for fixed point values. * @menu_skip_mask: The control's skip mask for menu controls. This makes it * easy to skip menu items that are not valid. If bit X is set, * then menu item X is skipped. Of course, this only works for @@ -228,6 +229,7 @@ typedef void (*v4l2_ctrl_notify_fnc)(struct v4l2_ctrl *ctrl, void *priv); * :math:`ceil(\frac{maximum - minimum}{step}) + 1`. * Used only if the @type is %V4L2_CTRL_TYPE_INTEGER_MENU. * @flags: The control's flags. + * @fraction_bits: The number of fraction bits for fixed point values. * @priv: The control's private pointer. For use by the driver. It is * untouched by the control framework. Note that this pointer is * not freed when the control is deleted. Should this be needed @@ -286,6 +288,7 @@ struct v4l2_ctrl { u32 new_elems; u32 dims[V4L2_CTRL_MAX_DIMS]; u32 nr_of_dims; + u32 fraction_bits; union { u64 step; u64 menu_skip_mask; @@ -426,6 +429,7 @@ struct v4l2_ctrl_handler { * @dims: The size of each dimension. * @elem_size: The size in bytes of the control. * @flags: The control's flags. + * @fraction_bits: The number of fraction bits for fixed point values. * @menu_skip_mask: The control's skip mask for menu controls. This makes it * easy to skip menu items that are not valid. If bit X is set, * then menu item X is skipped. Of course, this only works for @@ -455,6 +459,7 @@ struct v4l2_ctrl_config { u32 dims[V4L2_CTRL_MAX_DIMS]; u32 elem_size; u32 flags; + u32 fraction_bits; u64 menu_skip_mask; const char * const *qmenu; const s64 *qmenu_int; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index a8015e5e7fa4..b8573e9ccde6 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -1947,7 +1947,8 @@ struct v4l2_query_ext_ctrl { __u32 elems; __u32 nr_of_dims; __u32 dims[V4L2_CTRL_MAX_DIMS]; - __u32 reserved[32]; + __u32 fraction_bits; + __u32 reserved[31]; }; /* Used in the VIDIOC_QUERYMENU ioctl for querying menu items */ From patchwork Mon Feb 26 08:28:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shengjiu Wang X-Patchwork-Id: 776116 Received: from inva020.nxp.com (inva020.nxp.com [92.121.34.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EB8654BCB; Mon, 26 Feb 2024 08:44:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=92.121.34.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937062; cv=none; b=Es9EtL9A3NmsThY2sR1OnMu1e7zDEBbNr2JmlHnLCQhF8q0sM4BTuLHD37To25VlE0zIaSwRS4Vz26uxwwRTKuLwZEnA3Lhx477JSlMXOUg+UUlrFuMy7Mr99yA8oEuSmVLNGqp5gwTfX3VJFr1KJKB22DI4ZtsF+EirkiRaaOM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937062; c=relaxed/simple; bh=av6fOLV/++JwoVuZ28V9t3kDf6O1gW/IMhILJHw/04U=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References; b=EdbKhS/husZ6XkXmG/LjjDEpfsra9jj8ZND6h7D3brYsOYdBJUhlQ2pfdfNXGS9J37FlnqCUI5xjgdUCbIJ8ZwT/grg5LlKyUAxPeqQ4Bt3S06xqv9jB/qXfTmb0FFas8WrIzAOOX8GmMz5bNrlGkRgrFM09pUX7JZ6jmDyzhNY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com; spf=pass smtp.mailfrom=nxp.com; arc=none smtp.client-ip=92.121.34.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nxp.com Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 624751A010D; Mon, 26 Feb 2024 09:44:13 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id F0DE31A0146; Mon, 26 Feb 2024 09:44:12 +0100 (CET) Received: from localhost.localdomain (shlinux2.ap.freescale.net [10.192.224.44]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 4C0B51820F56; Mon, 26 Feb 2024 16:44:11 +0800 (+08) From: Shengjiu Wang To: hverkuil@xs4all.nl, sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, shengjiu.wang@gmail.com, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v13 03/16] ASoC: fsl_easrc: define functions for memory to memory usage Date: Mon, 26 Feb 2024 16:28:16 +0800 Message-Id: <1708936109-11587-4-git-send-email-shengjiu.wang@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> References: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: ASRC can be used on memory to memory case, define several functions for m2m usage and export them as function pointer. Signed-off-by: Shengjiu Wang Acked-by: Mark Brown --- sound/soc/fsl/fsl_easrc.c | 214 ++++++++++++++++++++++++++++++++++++++ sound/soc/fsl/fsl_easrc.h | 4 + 2 files changed, 218 insertions(+) diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c index ec53bda46a46..cf7ad30a323b 100644 --- a/sound/soc/fsl/fsl_easrc.c +++ b/sound/soc/fsl/fsl_easrc.c @@ -1861,6 +1861,211 @@ static int fsl_easrc_get_fifo_addr(u8 dir, enum asrc_pair_index index) return REG_EASRC_FIFO(dir, index); } +/* Get sample numbers in FIFO */ +static unsigned int fsl_easrc_get_output_fifo_size(struct fsl_asrc_pair *pair) +{ + struct fsl_asrc *asrc = pair->asrc; + enum asrc_pair_index index = pair->index; + u32 val; + + regmap_read(asrc->regmap, REG_EASRC_SFS(index), &val); + val &= EASRC_SFS_NSGO_MASK; + + return val >> EASRC_SFS_NSGO_SHIFT; +} + +static int fsl_easrc_m2m_prepare(struct fsl_asrc_pair *pair) +{ + struct fsl_easrc_ctx_priv *ctx_priv = pair->private; + struct fsl_asrc *asrc = pair->asrc; + struct device *dev = &asrc->pdev->dev; + int ret; + + ctx_priv->in_params.sample_rate = pair->rate[IN]; + ctx_priv->in_params.sample_format = pair->sample_format[IN]; + ctx_priv->out_params.sample_rate = pair->rate[OUT]; + ctx_priv->out_params.sample_format = pair->sample_format[OUT]; + + ctx_priv->in_params.fifo_wtmk = FSL_EASRC_INPUTFIFO_WML; + ctx_priv->out_params.fifo_wtmk = FSL_EASRC_OUTPUTFIFO_WML; + /* Fill the right half of the re-sampler with zeros */ + ctx_priv->rs_init_mode = 0x2; + /* Zero fill the right half of the prefilter */ + ctx_priv->pf_init_mode = 0x2; + + ret = fsl_easrc_set_ctx_format(pair, + &ctx_priv->in_params.sample_format, + &ctx_priv->out_params.sample_format); + if (ret) { + dev_err(dev, "failed to set context format: %d\n", ret); + return ret; + } + + ret = fsl_easrc_config_context(asrc, pair->index); + if (ret) { + dev_err(dev, "failed to config context %d\n", ret); + return ret; + } + + ctx_priv->in_params.iterations = 1; + ctx_priv->in_params.group_len = pair->channels; + ctx_priv->in_params.access_len = pair->channels; + ctx_priv->out_params.iterations = 1; + ctx_priv->out_params.group_len = pair->channels; + ctx_priv->out_params.access_len = pair->channels; + + ret = fsl_easrc_set_ctx_organziation(pair); + if (ret) { + dev_err(dev, "failed to set fifo organization\n"); + return ret; + } + + /* The context start flag */ + pair->first_convert = 1; + return 0; +} + +static int fsl_easrc_m2m_start(struct fsl_asrc_pair *pair) +{ + /* start context once */ + if (pair->first_convert) { + fsl_easrc_start_context(pair); + pair->first_convert = 0; + } + + return 0; +} + +static int fsl_easrc_m2m_stop(struct fsl_asrc_pair *pair) +{ + /* Stop pair/context */ + if (!pair->first_convert) { + fsl_easrc_stop_context(pair); + pair->first_convert = 1; + } + + return 0; +} + +/* calculate capture data length according to output data length and sample rate */ +static int fsl_easrc_m2m_calc_out_len(struct fsl_asrc_pair *pair, int input_buffer_length) +{ + struct fsl_asrc *easrc = pair->asrc; + struct fsl_easrc_priv *easrc_priv = easrc->private; + struct fsl_easrc_ctx_priv *ctx_priv = pair->private; + unsigned int in_rate = ctx_priv->in_params.norm_rate; + unsigned int out_rate = ctx_priv->out_params.norm_rate; + unsigned int channels = pair->channels; + unsigned int in_samples, out_samples; + unsigned int in_width, out_width; + unsigned int out_length; + unsigned int frac_bits; + u64 val1, val2; + + switch (easrc_priv->rs_num_taps) { + case EASRC_RS_32_TAPS: + /* integer bits = 5; */ + frac_bits = 39; + break; + case EASRC_RS_64_TAPS: + /* integer bits = 6; */ + frac_bits = 38; + break; + case EASRC_RS_128_TAPS: + /* integer bits = 7; */ + frac_bits = 37; + break; + default: + return -EINVAL; + } + + val1 = (u64)in_rate << frac_bits; + do_div(val1, out_rate); + val1 += (s64)ctx_priv->ratio_mod << (frac_bits - 31); + + in_width = snd_pcm_format_physical_width(ctx_priv->in_params.sample_format) / 8; + out_width = snd_pcm_format_physical_width(ctx_priv->out_params.sample_format) / 8; + + ctx_priv->in_filled_len += input_buffer_length; + if (ctx_priv->in_filled_len <= ctx_priv->in_filled_sample * in_width * channels) { + out_length = 0; + } else { + in_samples = ctx_priv->in_filled_len / (in_width * channels) - + ctx_priv->in_filled_sample; + + /* right shift 12 bit to make ratio in 32bit space */ + val2 = (u64)in_samples << (frac_bits - 12); + val1 = val1 >> 12; + do_div(val2, val1); + out_samples = val2; + + out_length = out_samples * out_width * channels; + ctx_priv->in_filled_len = ctx_priv->in_filled_sample * in_width * channels; + } + + return out_length; +} + +static int fsl_easrc_m2m_get_maxburst(u8 dir, struct fsl_asrc_pair *pair) +{ + struct fsl_easrc_ctx_priv *ctx_priv = pair->private; + + if (dir == IN) + return ctx_priv->in_params.fifo_wtmk * pair->channels; + else + return ctx_priv->out_params.fifo_wtmk * pair->channels; +} + +static int fsl_easrc_m2m_pair_suspend(struct fsl_asrc_pair *pair) +{ + fsl_easrc_stop_context(pair); + + return 0; +} + +static int fsl_easrc_m2m_pair_resume(struct fsl_asrc_pair *pair) +{ + struct fsl_easrc_ctx_priv *ctx_priv = pair->private; + + pair->first_convert = 1; + ctx_priv->in_filled_len = 0; + + return 0; +} + +/* val is Q31 */ +static int fsl_easrc_m2m_set_ratio_mod(struct fsl_asrc_pair *pair, int val) +{ + struct fsl_easrc_ctx_priv *ctx_priv = pair->private; + struct fsl_asrc *easrc = pair->asrc; + struct fsl_easrc_priv *easrc_priv = easrc->private; + unsigned int frac_bits; + + ctx_priv->ratio_mod += val; + + switch (easrc_priv->rs_num_taps) { + case EASRC_RS_32_TAPS: + /* integer bits = 5; */ + frac_bits = 39; + break; + case EASRC_RS_64_TAPS: + /* integer bits = 6; */ + frac_bits = 38; + break; + case EASRC_RS_128_TAPS: + /* integer bits = 7; */ + frac_bits = 37; + break; + default: + return -EINVAL; + } + + val <<= (frac_bits - 31); + regmap_write(easrc->regmap, REG_EASRC_RUC(pair->index), EASRC_RSUC_RS_RM(val)); + + return 0; +} + static const struct of_device_id fsl_easrc_dt_ids[] = { { .compatible = "fsl,imx8mn-easrc",}, {} @@ -1926,6 +2131,15 @@ static int fsl_easrc_probe(struct platform_device *pdev) easrc->release_pair = fsl_easrc_release_context; easrc->get_fifo_addr = fsl_easrc_get_fifo_addr; easrc->pair_priv_size = sizeof(struct fsl_easrc_ctx_priv); + easrc->m2m_prepare = fsl_easrc_m2m_prepare; + easrc->m2m_start = fsl_easrc_m2m_start; + easrc->m2m_stop = fsl_easrc_m2m_stop; + easrc->get_output_fifo_size = fsl_easrc_get_output_fifo_size; + easrc->m2m_calc_out_len = fsl_easrc_m2m_calc_out_len; + easrc->m2m_get_maxburst = fsl_easrc_m2m_get_maxburst; + easrc->m2m_pair_suspend = fsl_easrc_m2m_pair_suspend; + easrc->m2m_pair_resume = fsl_easrc_m2m_pair_resume; + easrc->m2m_set_ratio_mod = fsl_easrc_m2m_set_ratio_mod; easrc_priv->rs_num_taps = EASRC_RS_32_TAPS; easrc_priv->const_coeff = 0x3FF0000000000000; diff --git a/sound/soc/fsl/fsl_easrc.h b/sound/soc/fsl/fsl_easrc.h index 7c70dac52713..c9f770862662 100644 --- a/sound/soc/fsl/fsl_easrc.h +++ b/sound/soc/fsl/fsl_easrc.h @@ -601,6 +601,8 @@ struct fsl_easrc_slot { * @out_missed_sample: sample missed in output * @st1_addexp: exponent added for stage1 * @st2_addexp: exponent added for stage2 + * @ratio_mod: update ratio + * @in_filled_len: input filled length */ struct fsl_easrc_ctx_priv { struct fsl_easrc_io_params in_params; @@ -618,6 +620,8 @@ struct fsl_easrc_ctx_priv { int out_missed_sample; int st1_addexp; int st2_addexp; + int ratio_mod; + unsigned int in_filled_len; }; /** From patchwork Mon Feb 26 08:28:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shengjiu Wang X-Patchwork-Id: 776117 Received: from inva021.nxp.com (inva021.nxp.com [92.121.34.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16F6054BC8; Mon, 26 Feb 2024 08:44:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=92.121.34.21 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937061; cv=none; b=UvrjiYMoes+oL/MXYb6EVMUjsdww+wdyFnXwo79Yv0LOOFndjmZUSqeTApNLCt/PudZwFu2j8YOJamWhwbedh5DrlDvW2tfouGnX5164h2wTomlYK7Wi17otAb6JD2glMyo81xCGWNlFwkiWUh/KZwi8R8N8Z4oNvaMqRwy8Fjo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937061; c=relaxed/simple; bh=CNk2VPYgaHZVUWLkXbMZyzxsjX0YYQCq1bBFNAfMI2M=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References; b=bZMl23Z5zyX/7vb6Hfj6eDyP5wecSatMjRsBLU6Ja4wA74ovPbqM0cEEuHzA02wMI6Qohofu4SiCvdO2IuacoOoFh8jPRRmHfh3AzntT4M4ZxaxvyQKhdlvQ6aef9SnjLiefMgriVomxpAm46MywNhVz1qpWTJtpiWW/ryYCcBk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com; spf=pass smtp.mailfrom=nxp.com; arc=none smtp.client-ip=92.121.34.21 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nxp.com Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 756EF200467; Mon, 26 Feb 2024 09:44:18 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 3C1D120008E; Mon, 26 Feb 2024 09:44:18 +0100 (CET) Received: from localhost.localdomain (shlinux2.ap.freescale.net [10.192.224.44]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 309A6180031D; Mon, 26 Feb 2024 16:44:16 +0800 (+08) From: Shengjiu Wang To: hverkuil@xs4all.nl, sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, shengjiu.wang@gmail.com, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v13 06/16] ASoC: fsl_easrc: register m2m platform device Date: Mon, 26 Feb 2024 16:28:19 +0800 Message-Id: <1708936109-11587-7-git-send-email-shengjiu.wang@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> References: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Register m2m platform device,that user can use M2M feature. Signed-off-by: Shengjiu Wang Acked-by: Mark Brown --- sound/soc/fsl/fsl_easrc.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c index cf7ad30a323b..ccbf45c7abf4 100644 --- a/sound/soc/fsl/fsl_easrc.c +++ b/sound/soc/fsl/fsl_easrc.c @@ -2075,6 +2075,7 @@ MODULE_DEVICE_TABLE(of, fsl_easrc_dt_ids); static int fsl_easrc_probe(struct platform_device *pdev) { struct fsl_easrc_priv *easrc_priv; + struct fsl_asrc_m2m_pdata m2m_pdata; struct device *dev = &pdev->dev; struct fsl_asrc *easrc; struct resource *res; @@ -2190,6 +2191,19 @@ static int fsl_easrc_probe(struct platform_device *pdev) goto err_pm_disable; } + m2m_pdata.asrc = easrc; + m2m_pdata.fmt_in = FSL_EASRC_FORMATS; + m2m_pdata.fmt_out = FSL_EASRC_FORMATS | SNDRV_PCM_FMTBIT_IEC958_SUBFRAME_LE; + m2m_pdata.rate_min = 8000; + m2m_pdata.rate_max = 768000; + m2m_pdata.chan_min = 1; + m2m_pdata.chan_max = 32; + easrc->m2m_pdev = platform_device_register_data(&pdev->dev, + M2M_DRV_NAME, + PLATFORM_DEVID_AUTO, + &m2m_pdata, + sizeof(m2m_pdata)); + return 0; err_pm_disable: @@ -2199,6 +2213,11 @@ static int fsl_easrc_probe(struct platform_device *pdev) static void fsl_easrc_remove(struct platform_device *pdev) { + struct fsl_asrc *easrc = dev_get_drvdata(&pdev->dev); + + if (easrc->m2m_pdev && !IS_ERR(easrc->m2m_pdev)) + platform_device_unregister(easrc->m2m_pdev); + pm_runtime_disable(&pdev->dev); } From patchwork Mon Feb 26 08:28:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shengjiu Wang X-Patchwork-Id: 776115 Received: from inva021.nxp.com (inva021.nxp.com [92.121.34.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5197155C28; Mon, 26 Feb 2024 08:44:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=92.121.34.21 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937066; cv=none; b=ccVGgR6mvKx9sNI83O6Ar2DTpngSYni6reCgwfu2TFgc40omCkCFnSdlZMYFk1bxo/Y0N1CUzTmJVuW8F+MoSFJ8hWmv02UnTT/yS70Liy3px5Tu8fcbBdTkTVYhSr/XO59GFBEutEryOPmBIIZldvryYf9wDKhtIQNYveOxwyU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937066; c=relaxed/simple; bh=ADNH7F5Y6dN85s8foi3HYM1ep3fE6hhGj06MswlNLjw=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References; b=Y6tJHfyi9kaHwdUYXwhxNvIfcdG1gUh9LJ1jZEPJBA0BLb9LQqEn+APkOQ4TCnTsheAAvxbiUwOrBnkVqhrB6GBWWzoh7t6bPuQYh9WYLsXOFFfET+lrTyDthEPugW7s5qOfe+VKYiLCAraMR1LmMsr9zPOILyhcYJvNgUKRpXQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com; spf=pass smtp.mailfrom=nxp.com; arc=none smtp.client-ip=92.121.34.21 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nxp.com Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id C05CA20011D; Mon, 26 Feb 2024 09:44:21 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 58A13201109; Mon, 26 Feb 2024 09:44:21 +0100 (CET) Received: from localhost.localdomain (shlinux2.ap.freescale.net [10.192.224.44]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 4B986183AD16; Mon, 26 Feb 2024 16:44:19 +0800 (+08) From: Shengjiu Wang To: hverkuil@xs4all.nl, sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, shengjiu.wang@gmail.com, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v13 08/16] media: v4l2: Add audio capture and output support Date: Mon, 26 Feb 2024 16:28:21 +0800 Message-Id: <1708936109-11587-9-git-send-email-shengjiu.wang@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> References: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Audio signal processing has the requirement for memory to memory similar as Video. This patch is to add this support in v4l2 framework, defined new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format for audio case usage. The created audio device is named "/dev/v4l-audioX". Signed-off-by: Shengjiu Wang --- .../userspace-api/media/v4l/buffer.rst | 6 ++ .../media/v4l/dev-audio-mem2mem.rst | 71 +++++++++++++++++++ .../userspace-api/media/v4l/devices.rst | 1 + .../media/v4l/vidioc-enum-fmt.rst | 2 + .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++ .../media/videodev2.h.rst.exceptions | 2 + .../media/common/videobuf2/videobuf2-v4l2.c | 4 ++ drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++ drivers/media/v4l2-core/v4l2-dev.c | 17 +++++ drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++++++++++++++ include/media/v4l2-dev.h | 2 + include/media/v4l2-ioctl.h | 34 +++++++++ include/uapi/linux/videodev2.h | 17 +++++ 13 files changed, 222 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst diff --git a/Documentation/userspace-api/media/v4l/buffer.rst b/Documentation/userspace-api/media/v4l/buffer.rst index 52bbee81c080..a3754ca6f0d6 100644 --- a/Documentation/userspace-api/media/v4l/buffer.rst +++ b/Documentation/userspace-api/media/v4l/buffer.rst @@ -438,6 +438,12 @@ enum v4l2_buf_type * - ``V4L2_BUF_TYPE_META_OUTPUT`` - 14 - Buffer for metadata output, see :ref:`metadata`. + * - ``V4L2_BUF_TYPE_AUDIO_CAPTURE`` + - 15 + - Buffer for audio capture, see :ref:`audio`. + * - ``V4L2_BUF_TYPE_AUDIO_OUTPUT`` + - 16 + - Buffer for audio output, see :ref:`audio`. .. _buffer-flags: diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst new file mode 100644 index 000000000000..54cc2abb6c04 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst @@ -0,0 +1,71 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _audiomem2mem: + +******************************** +Audio Memory-To-Memory Interface +******************************** + +An audio memory-to-memory device can compress, decompress, transform, or +otherwise convert audio data from one format into another format, in memory. +Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability. +Examples of memory-to-memory devices are audio codecs, audio preprocessing, +audio postprocessing. + +A memory-to-memory audio node supports both output (sending audio frames from +memory to the hardware) and capture (receiving the processed audio frames +from the hardware into memory) stream I/O. An application will have to +setup the stream I/O for both sides and finally call +:ref:`VIDIOC_STREAMON ` for both capture and output to +start the hardware. + +Memory-to-memory devices function as a shared resource: you can +open the audio node multiple times, each application setting up their +own properties that are local to the file handle, and each can use +it independently from the others. The driver will arbitrate access to +the hardware and reprogram it whenever another file handler gets access. + +Audio memory-to-memory devices are accessed through character device +special files named ``/dev/v4l-audio`` + +Querying Capabilities +===================== + +Device nodes supporting the audio memory-to-memory interface set the +``V4L2_CAP_AUDIO_M2M`` flag in the ``device_caps`` field of the +:c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP` +ioctl. + +Data Format Negotiation +======================= + +The audio device uses the :ref:`format` ioctls to select the capture format. +The audio buffer content format is bound to that selected format. In addition +to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must be +supported as well. + +To use the :ref:`format` ioctls applications set the ``type`` field of the +:c:type:`v4l2_format` structure to ``V4L2_BUF_TYPE_AUDIO_CAPTURE`` or to +``V4L2_BUF_TYPE_AUDIO_OUTPUT``. Both drivers and applications must set the +remainder of the :c:type:`v4l2_format` structure to 0. + +.. c:type:: v4l2_audio_format + +.. tabularcolumns:: |p{1.4cm}|p{2.4cm}|p{13.5cm}| + +.. flat-table:: struct v4l2_audio_format + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u32 + - ``audioformat`` + - The sample format, set by the application. see :ref:`pixfmt-audio` + * - __u32 + - ``channels`` + - The channel number, set by the application. channel number range is + [1, 32]. + * - __u32 + - ``buffersize`` + - Maximum buffer size in bytes required for data. The value is set by the + driver. diff --git a/Documentation/userspace-api/media/v4l/devices.rst b/Documentation/userspace-api/media/v4l/devices.rst index 8bfbad65a9d4..758bd90f1c26 100644 --- a/Documentation/userspace-api/media/v4l/devices.rst +++ b/Documentation/userspace-api/media/v4l/devices.rst @@ -24,3 +24,4 @@ Interfaces dev-event dev-subdev dev-meta + dev-audio-mem2mem diff --git a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst index 000c154b0f98..42deb07f4ff4 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst @@ -96,6 +96,8 @@ the ``mbus_code`` field is handled differently: ``V4L2_BUF_TYPE_VIDEO_OVERLAY``, ``V4L2_BUF_TYPE_SDR_CAPTURE``, ``V4L2_BUF_TYPE_SDR_OUTPUT``, + ``V4L2_BUF_TYPE_AUDIO_CAPTURE``, + ``V4L2_BUF_TYPE_AUDIO_OUTPUT``, ``V4L2_BUF_TYPE_META_CAPTURE`` and ``V4L2_BUF_TYPE_META_OUTPUT``. See :c:type:`v4l2_buf_type`. diff --git a/Documentation/userspace-api/media/v4l/vidioc-g-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-g-fmt.rst index 675c385e5aca..528fd9df41aa 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-g-fmt.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-g-fmt.rst @@ -130,6 +130,10 @@ The format as returned by :ref:`VIDIOC_TRY_FMT ` must be identical - ``meta`` - Definition of a metadata format, see :ref:`meta-formats`, used by metadata capture devices. + * - struct :c:type:`v4l2_audio_format` + - ``audio`` + - Definition of a audio data format, see :ref:`audiomem2mem`, used by + audio memory-to-memory devices * - __u8 - ``raw_data``\ [200] - Place holder for future extensions. diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions index da6d0b8e4c2c..e61152bb80d1 100644 --- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions +++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions @@ -29,6 +29,8 @@ replace symbol V4L2_FIELD_SEQ_TB :c:type:`v4l2_field` replace symbol V4L2_FIELD_TOP :c:type:`v4l2_field` # Documented enum v4l2_buf_type +replace symbol V4L2_BUF_TYPE_AUDIO_CAPTURE :c:type:`v4l2_buf_type` +replace symbol V4L2_BUF_TYPE_AUDIO_OUTPUT :c:type:`v4l2_buf_type` replace symbol V4L2_BUF_TYPE_META_CAPTURE :c:type:`v4l2_buf_type` replace symbol V4L2_BUF_TYPE_META_OUTPUT :c:type:`v4l2_buf_type` replace symbol V4L2_BUF_TYPE_SDR_CAPTURE :c:type:`v4l2_buf_type` diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c index c575198e8354..0738f45b2341 100644 --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c @@ -789,6 +789,10 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create) case V4L2_BUF_TYPE_META_OUTPUT: requested_sizes[0] = f->fmt.meta.buffersize; break; + case V4L2_BUF_TYPE_AUDIO_CAPTURE: + case V4L2_BUF_TYPE_AUDIO_OUTPUT: + requested_sizes[0] = f->fmt.audio.buffersize; + break; default: return -EINVAL; } diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c index 8c07400bd280..5e94db8dfdae 100644 --- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c +++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c @@ -101,6 +101,7 @@ struct v4l2_format32 { struct v4l2_sliced_vbi_format sliced; struct v4l2_sdr_format sdr; struct v4l2_meta_format meta; + struct v4l2_audio_format audio; __u8 raw_data[200]; /* user-defined */ } fmt; }; @@ -166,6 +167,10 @@ static int get_v4l2_format32(struct v4l2_format *p64, case V4L2_BUF_TYPE_META_OUTPUT: return copy_from_user(&p64->fmt.meta, &p32->fmt.meta, sizeof(p64->fmt.meta)) ? -EFAULT : 0; + case V4L2_BUF_TYPE_AUDIO_CAPTURE: + case V4L2_BUF_TYPE_AUDIO_OUTPUT: + return copy_from_user(&p64->fmt.audio, &p32->fmt.audio, + sizeof(p64->fmt.audio)) ? -EFAULT : 0; default: return -EINVAL; } @@ -216,6 +221,10 @@ static int put_v4l2_format32(struct v4l2_format *p64, case V4L2_BUF_TYPE_META_OUTPUT: return copy_to_user(&p32->fmt.meta, &p64->fmt.meta, sizeof(p64->fmt.meta)) ? -EFAULT : 0; + case V4L2_BUF_TYPE_AUDIO_CAPTURE: + case V4L2_BUF_TYPE_AUDIO_OUTPUT: + return copy_to_user(&p32->fmt.audio, &p64->fmt.audio, + sizeof(p64->fmt.audio)) ? -EFAULT : 0; default: return -EINVAL; } diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index d13954bd31fd..bac008fcedc6 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -553,6 +553,7 @@ static void determine_valid_ioctls(struct video_device *vdev) bool is_tch = vdev->vfl_type == VFL_TYPE_TOUCH; bool is_meta = vdev->vfl_type == VFL_TYPE_VIDEO && (vdev->device_caps & meta_caps); + bool is_audio = vdev->vfl_type == VFL_TYPE_AUDIO; bool is_rx = vdev->vfl_dir != VFL_DIR_TX; bool is_tx = vdev->vfl_dir != VFL_DIR_RX; bool is_io_mc = vdev->device_caps & V4L2_CAP_IO_MC; @@ -666,6 +667,19 @@ static void determine_valid_ioctls(struct video_device *vdev) SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_meta_out); } + if (is_audio && is_rx) { + /* audio capture specific ioctls */ + SET_VALID_IOCTL(ops, VIDIOC_ENUM_FMT, vidioc_enum_fmt_audio_cap); + SET_VALID_IOCTL(ops, VIDIOC_G_FMT, vidioc_g_fmt_audio_cap); + SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_audio_cap); + SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_audio_cap); + } else if (is_audio && is_tx) { + /* audio output specific ioctls */ + SET_VALID_IOCTL(ops, VIDIOC_ENUM_FMT, vidioc_enum_fmt_audio_out); + SET_VALID_IOCTL(ops, VIDIOC_G_FMT, vidioc_g_fmt_audio_out); + SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_audio_out); + SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_audio_out); + } if (is_vbi) { /* vbi specific ioctls */ if ((is_rx && (ops->vidioc_g_fmt_vbi_cap || @@ -929,6 +943,9 @@ int __video_register_device(struct video_device *vdev, case VFL_TYPE_TOUCH: name_base = "v4l-touch"; break; + case VFL_TYPE_AUDIO: + name_base = "v4l-audio"; + break; default: pr_err("%s called with unknown type: %d\n", __func__, type); diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 6e7b8b682d13..961abcdf7290 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -188,6 +188,8 @@ const char *v4l2_type_names[] = { [V4L2_BUF_TYPE_SDR_OUTPUT] = "sdr-out", [V4L2_BUF_TYPE_META_CAPTURE] = "meta-cap", [V4L2_BUF_TYPE_META_OUTPUT] = "meta-out", + [V4L2_BUF_TYPE_AUDIO_CAPTURE] = "audio-cap", + [V4L2_BUF_TYPE_AUDIO_OUTPUT] = "audio-out", }; EXPORT_SYMBOL(v4l2_type_names); @@ -276,6 +278,7 @@ static void v4l_print_format(const void *arg, bool write_only) const struct v4l2_sliced_vbi_format *sliced; const struct v4l2_window *win; const struct v4l2_meta_format *meta; + const struct v4l2_audio_format *audio; u32 pixelformat; u32 planes; unsigned i; @@ -346,6 +349,13 @@ static void v4l_print_format(const void *arg, bool write_only) pr_cont(", dataformat=%p4cc, buffersize=%u\n", &pixelformat, meta->buffersize); break; + case V4L2_BUF_TYPE_AUDIO_CAPTURE: + case V4L2_BUF_TYPE_AUDIO_OUTPUT: + audio = &p->fmt.audio; + pixelformat = audio->audioformat; + pr_cont(", format=%p4cc, channels=%u, buffersize=%u\n", + &pixelformat, audio->channels, audio->buffersize); + break; } } @@ -927,6 +937,7 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) bool is_tch = vfd->vfl_type == VFL_TYPE_TOUCH; bool is_meta = vfd->vfl_type == VFL_TYPE_VIDEO && (vfd->device_caps & meta_caps); + bool is_audio = vfd->vfl_type == VFL_TYPE_AUDIO; bool is_rx = vfd->vfl_dir != VFL_DIR_TX; bool is_tx = vfd->vfl_dir != VFL_DIR_RX; @@ -992,6 +1003,14 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type) if (is_meta && is_tx && ops->vidioc_g_fmt_meta_out) return 0; break; + case V4L2_BUF_TYPE_AUDIO_CAPTURE: + if (is_audio && is_rx && ops->vidioc_g_fmt_audio_cap) + return 0; + break; + case V4L2_BUF_TYPE_AUDIO_OUTPUT: + if (is_audio && is_tx && ops->vidioc_g_fmt_audio_out) + return 0; + break; default: break; } @@ -1597,6 +1616,16 @@ static int v4l_enum_fmt(const struct v4l2_ioctl_ops *ops, break; ret = ops->vidioc_enum_fmt_meta_out(file, fh, arg); break; + case V4L2_BUF_TYPE_AUDIO_CAPTURE: + if (unlikely(!ops->vidioc_enum_fmt_audio_cap)) + break; + ret = ops->vidioc_enum_fmt_audio_cap(file, fh, arg); + break; + case V4L2_BUF_TYPE_AUDIO_OUTPUT: + if (unlikely(!ops->vidioc_enum_fmt_audio_out)) + break; + ret = ops->vidioc_enum_fmt_audio_out(file, fh, arg); + break; } if (ret == 0) v4l_fill_fmtdesc(p); @@ -1673,6 +1702,10 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops, return ops->vidioc_g_fmt_meta_cap(file, fh, arg); case V4L2_BUF_TYPE_META_OUTPUT: return ops->vidioc_g_fmt_meta_out(file, fh, arg); + case V4L2_BUF_TYPE_AUDIO_CAPTURE: + return ops->vidioc_g_fmt_audio_cap(file, fh, arg); + case V4L2_BUF_TYPE_AUDIO_OUTPUT: + return ops->vidioc_g_fmt_audio_out(file, fh, arg); } return -EINVAL; } @@ -1784,6 +1817,16 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops, break; memset_after(p, 0, fmt.meta); return ops->vidioc_s_fmt_meta_out(file, fh, arg); + case V4L2_BUF_TYPE_AUDIO_CAPTURE: + if (unlikely(!ops->vidioc_s_fmt_audio_cap)) + break; + memset_after(p, 0, fmt.audio); + return ops->vidioc_s_fmt_audio_cap(file, fh, arg); + case V4L2_BUF_TYPE_AUDIO_OUTPUT: + if (unlikely(!ops->vidioc_s_fmt_audio_out)) + break; + memset_after(p, 0, fmt.audio); + return ops->vidioc_s_fmt_audio_out(file, fh, arg); } return -EINVAL; } @@ -1892,6 +1935,16 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops, break; memset_after(p, 0, fmt.meta); return ops->vidioc_try_fmt_meta_out(file, fh, arg); + case V4L2_BUF_TYPE_AUDIO_CAPTURE: + if (unlikely(!ops->vidioc_try_fmt_audio_cap)) + break; + memset_after(p, 0, fmt.audio); + return ops->vidioc_try_fmt_audio_cap(file, fh, arg); + case V4L2_BUF_TYPE_AUDIO_OUTPUT: + if (unlikely(!ops->vidioc_try_fmt_audio_out)) + break; + memset_after(p, 0, fmt.audio); + return ops->vidioc_try_fmt_audio_out(file, fh, arg); } return -EINVAL; } diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index d82dfdbf6e58..82b63f82d43f 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -30,6 +30,7 @@ * @VFL_TYPE_SUBDEV: for V4L2 subdevices * @VFL_TYPE_SDR: for Software Defined Radio tuners * @VFL_TYPE_TOUCH: for touch sensors + * @VFL_TYPE_AUDIO: for audio memory-to-memory devices * @VFL_TYPE_MAX: number of VFL types, must always be last in the enum */ enum vfl_devnode_type { @@ -39,6 +40,7 @@ enum vfl_devnode_type { VFL_TYPE_SUBDEV, VFL_TYPE_SDR, VFL_TYPE_TOUCH, + VFL_TYPE_AUDIO, VFL_TYPE_MAX /* Shall be the last one */ }; diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index edb733f21604..f840cf740ce1 100644 --- a/include/media/v4l2-ioctl.h +++ b/include/media/v4l2-ioctl.h @@ -45,6 +45,12 @@ struct v4l2_fh; * @vidioc_enum_fmt_meta_out: pointer to the function that implements * :ref:`VIDIOC_ENUM_FMT ` ioctl logic * for metadata output + * @vidioc_enum_fmt_audio_cap: pointer to the function that implements + * :ref:`VIDIOC_ENUM_FMT ` ioctl logic + * for audio capture + * @vidioc_enum_fmt_audio_out: pointer to the function that implements + * :ref:`VIDIOC_ENUM_FMT ` ioctl logic + * for audio output * @vidioc_g_fmt_vid_cap: pointer to the function that implements * :ref:`VIDIOC_G_FMT ` ioctl logic for video capture * in single plane mode @@ -79,6 +85,10 @@ struct v4l2_fh; * :ref:`VIDIOC_G_FMT ` ioctl logic for metadata capture * @vidioc_g_fmt_meta_out: pointer to the function that implements * :ref:`VIDIOC_G_FMT ` ioctl logic for metadata output + * @vidioc_g_fmt_audio_cap: pointer to the function that implements + * :ref:`VIDIOC_G_FMT ` ioctl logic for audio capture + * @vidioc_g_fmt_audio_out: pointer to the function that implements + * :ref:`VIDIOC_G_FMT ` ioctl logic for audio output * @vidioc_s_fmt_vid_cap: pointer to the function that implements * :ref:`VIDIOC_S_FMT ` ioctl logic for video capture * in single plane mode @@ -113,6 +123,10 @@ struct v4l2_fh; * :ref:`VIDIOC_S_FMT ` ioctl logic for metadata capture * @vidioc_s_fmt_meta_out: pointer to the function that implements * :ref:`VIDIOC_S_FMT ` ioctl logic for metadata output + * @vidioc_s_fmt_audio_cap: pointer to the function that implements + * :ref:`VIDIOC_S_FMT ` ioctl logic for audio capture + * @vidioc_s_fmt_audio_out: pointer to the function that implements + * :ref:`VIDIOC_S_FMT ` ioctl logic for audio output * @vidioc_try_fmt_vid_cap: pointer to the function that implements * :ref:`VIDIOC_TRY_FMT ` ioctl logic for video capture * in single plane mode @@ -149,6 +163,10 @@ struct v4l2_fh; * :ref:`VIDIOC_TRY_FMT ` ioctl logic for metadata capture * @vidioc_try_fmt_meta_out: pointer to the function that implements * :ref:`VIDIOC_TRY_FMT ` ioctl logic for metadata output + * @vidioc_try_fmt_audio_cap: pointer to the function that implements + * :ref:`VIDIOC_TRY_FMT ` ioctl logic for audio capture + * @vidioc_try_fmt_audio_out: pointer to the function that implements + * :ref:`VIDIOC_TRY_FMT ` ioctl logic for audio output * @vidioc_reqbufs: pointer to the function that implements * :ref:`VIDIOC_REQBUFS ` ioctl * @vidioc_querybuf: pointer to the function that implements @@ -315,6 +333,10 @@ struct v4l2_ioctl_ops { struct v4l2_fmtdesc *f); int (*vidioc_enum_fmt_meta_out)(struct file *file, void *fh, struct v4l2_fmtdesc *f); + int (*vidioc_enum_fmt_audio_cap)(struct file *file, void *fh, + struct v4l2_fmtdesc *f); + int (*vidioc_enum_fmt_audio_out)(struct file *file, void *fh, + struct v4l2_fmtdesc *f); /* VIDIOC_G_FMT handlers */ int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh, @@ -345,6 +367,10 @@ struct v4l2_ioctl_ops { struct v4l2_format *f); int (*vidioc_g_fmt_meta_out)(struct file *file, void *fh, struct v4l2_format *f); + int (*vidioc_g_fmt_audio_cap)(struct file *file, void *fh, + struct v4l2_format *f); + int (*vidioc_g_fmt_audio_out)(struct file *file, void *fh, + struct v4l2_format *f); /* VIDIOC_S_FMT handlers */ int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh, @@ -375,6 +401,10 @@ struct v4l2_ioctl_ops { struct v4l2_format *f); int (*vidioc_s_fmt_meta_out)(struct file *file, void *fh, struct v4l2_format *f); + int (*vidioc_s_fmt_audio_cap)(struct file *file, void *fh, + struct v4l2_format *f); + int (*vidioc_s_fmt_audio_out)(struct file *file, void *fh, + struct v4l2_format *f); /* VIDIOC_TRY_FMT handlers */ int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh, @@ -405,6 +435,10 @@ struct v4l2_ioctl_ops { struct v4l2_format *f); int (*vidioc_try_fmt_meta_out)(struct file *file, void *fh, struct v4l2_format *f); + int (*vidioc_try_fmt_audio_cap)(struct file *file, void *fh, + struct v4l2_format *f); + int (*vidioc_try_fmt_audio_out)(struct file *file, void *fh, + struct v4l2_format *f); /* Buffer handlers */ int (*vidioc_reqbufs)(struct file *file, void *fh, diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 5cc2a978fd9c..2c03d2dfadbe 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -153,6 +153,8 @@ enum v4l2_buf_type { V4L2_BUF_TYPE_SDR_OUTPUT = 12, V4L2_BUF_TYPE_META_CAPTURE = 13, V4L2_BUF_TYPE_META_OUTPUT = 14, + V4L2_BUF_TYPE_AUDIO_CAPTURE = 15, + V4L2_BUF_TYPE_AUDIO_OUTPUT = 16, /* Deprecated, do not use */ V4L2_BUF_TYPE_PRIVATE = 0x80, }; @@ -169,6 +171,7 @@ enum v4l2_buf_type { || (type) == V4L2_BUF_TYPE_VBI_OUTPUT \ || (type) == V4L2_BUF_TYPE_SLICED_VBI_OUTPUT \ || (type) == V4L2_BUF_TYPE_SDR_OUTPUT \ + || (type) == V4L2_BUF_TYPE_AUDIO_OUTPUT \ || (type) == V4L2_BUF_TYPE_META_OUTPUT) #define V4L2_TYPE_IS_CAPTURE(type) (!V4L2_TYPE_IS_OUTPUT(type)) @@ -2423,6 +2426,18 @@ struct v4l2_meta_format { __u32 buffersize; } __attribute__ ((packed)); +/** + * struct v4l2_audio_format - audio data format definition + * @audioformat: little endian four character code (fourcc) + * @channels: channel numbers + * @buffersize: maximum size in bytes required for data + */ +struct v4l2_audio_format { + __u32 audioformat; + __u32 channels; + __u32 buffersize; +} __attribute__ ((packed)); + /** * struct v4l2_format - stream data format * @type: enum v4l2_buf_type; type of the data stream @@ -2431,6 +2446,7 @@ struct v4l2_meta_format { * @fmt.win: definition of an overlaid image * @fmt.vbi: raw VBI capture or output parameters * @fmt.sliced: sliced VBI capture or output parameters + * @fmt.audio: definition of an audio format * @fmt.raw_data: placeholder for future extensions and custom formats * @fmt: union of @pix, @pix_mp, @win, @vbi, @sliced, @sdr, * @meta and @raw_data @@ -2445,6 +2461,7 @@ struct v4l2_format { struct v4l2_sliced_vbi_format sliced; /* V4L2_BUF_TYPE_SLICED_VBI_CAPTURE */ struct v4l2_sdr_format sdr; /* V4L2_BUF_TYPE_SDR_CAPTURE */ struct v4l2_meta_format meta; /* V4L2_BUF_TYPE_META_CAPTURE */ + struct v4l2_audio_format audio; /* V4L2_BUF_TYPE_AUDIO_CAPTURE */ __u8 raw_data[200]; /* user-defined */ } fmt; }; From patchwork Mon Feb 26 08:28:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shengjiu Wang X-Patchwork-Id: 776114 Received: from inva021.nxp.com (inva021.nxp.com [92.121.34.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1016556473; Mon, 26 Feb 2024 08:44:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=92.121.34.21 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937067; cv=none; b=c9HvemhcI97AaM4t5Ka4JaA2vs5zo9A68g1hVmAbTCfilUGtXRZ8UxKUeCZLethZ4BfboM5FRU4SDTvV/YbJn4twhexAieNW9N9+4LITfoUYBulKYOL7zMvUajpnUXZ2ARL7L5CcsAhULkoTPaxw5NrB83yKkK+5efYwO0BX3ZI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937067; c=relaxed/simple; bh=8XxItNrGIbtnyv5TKd9ahQKgCqL4nVgFNt/oH0YDQ2s=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References; b=IQPapqgohUIn8h6Y5yQcdwLQsdMeOPrADkhEEgnY0hjMut6aYZxlcat5eTaBGeMN75X95b7p6W5xxzJApB3CtT9ecbj4eh3Z8XyVfxggvR8U57yFRhnalnEzRQ/msunnzCCHWNUWEOXCBl+JYpafDa84SOFuVMNoXF4dlZW6zdE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com; spf=pass smtp.mailfrom=nxp.com; arc=none smtp.client-ip=92.121.34.21 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nxp.com Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 7EDCD201100; Mon, 26 Feb 2024 09:44:24 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 18DBF20014B; Mon, 26 Feb 2024 09:44:24 +0100 (CET) Received: from localhost.localdomain (shlinux2.ap.freescale.net [10.192.224.44]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 66975183AD17; Mon, 26 Feb 2024 16:44:22 +0800 (+08) From: Shengjiu Wang To: hverkuil@xs4all.nl, sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, shengjiu.wang@gmail.com, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v13 10/16] media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO Date: Mon, 26 Feb 2024 16:28:23 +0800 Message-Id: <1708936109-11587-11-git-send-email-shengjiu.wang@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> References: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The Audio M2M class includes controls for audio memory-to-memory use cases. The controls can be used for audio codecs, audio preprocessing, audio postprocessing. Signed-off-by: Shengjiu Wang --- .../userspace-api/media/v4l/common.rst | 1 + .../media/v4l/ext-ctrls-audio-m2m.rst | 21 +++++++++++++++++++ .../media/v4l/vidioc-g-ext-ctrls.rst | 4 ++++ drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4 ++++ include/uapi/linux/v4l2-controls.h | 4 ++++ 5 files changed, 34 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst diff --git a/Documentation/userspace-api/media/v4l/common.rst b/Documentation/userspace-api/media/v4l/common.rst index ea0435182e44..d5366e96a596 100644 --- a/Documentation/userspace-api/media/v4l/common.rst +++ b/Documentation/userspace-api/media/v4l/common.rst @@ -52,6 +52,7 @@ applicable to all devices. ext-ctrls-fm-rx ext-ctrls-detect ext-ctrls-colorimetry + ext-ctrls-audio-m2m fourcc format planar-apis diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst new file mode 100644 index 000000000000..82d2ecedbfee --- /dev/null +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst @@ -0,0 +1,21 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _audiom2m-controls: + +*************************** +Audio M2M Control Reference +*************************** + +The Audio M2M class includes controls for audio memory-to-memory +use cases. The controls can be used for audio codecs, audio +preprocessing, audio postprocessing. + +Audio M2M Control IDs +----------------------- + +.. _audiom2m-control-id: + +``V4L2_CID_M2M_AUDIO_CLASS (class)`` + The Audio M2M class descriptor. Calling + :ref:`VIDIOC_QUERYCTRL` for this control will + return a description of this control class. diff --git a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst index 4d56c0528ad7..aeb1ad8e7d29 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst @@ -488,6 +488,10 @@ still cause this situation. - 0xa50000 - The class containing colorimetry controls. These controls are described in :ref:`colorimetry-controls`. + * - ``V4L2_CTRL_CLASS_M2M_AUDIO`` + - 0xa60000 + - The class containing audio m2m controls. These controls are + described in :ref:`audiom2m-controls`. Return Value ============ diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c index 8696eb1cdd61..2a85ea3dc92f 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c @@ -1242,6 +1242,9 @@ const char *v4l2_ctrl_get_name(u32 id) case V4L2_CID_COLORIMETRY_CLASS: return "Colorimetry Controls"; case V4L2_CID_COLORIMETRY_HDR10_CLL_INFO: return "HDR10 Content Light Info"; case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY: return "HDR10 Mastering Display"; + + /* Audio M2M controls */ + case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls"; default: return NULL; } @@ -1451,6 +1454,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, case V4L2_CID_DETECT_CLASS: case V4L2_CID_CODEC_STATELESS_CLASS: case V4L2_CID_COLORIMETRY_CLASS: + case V4L2_CID_M2M_AUDIO_CLASS: *type = V4L2_CTRL_TYPE_CTRL_CLASS; /* You can neither read nor write these */ *flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_WRITE_ONLY; diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h index 99c3f5e99da7..a8b4b830c757 100644 --- a/include/uapi/linux/v4l2-controls.h +++ b/include/uapi/linux/v4l2-controls.h @@ -30,6 +30,7 @@ #define V4L2_CTRL_CLASS_DETECT 0x00a30000 /* Detection controls */ #define V4L2_CTRL_CLASS_CODEC_STATELESS 0x00a40000 /* Stateless codecs controls */ #define V4L2_CTRL_CLASS_COLORIMETRY 0x00a50000 /* Colorimetry controls */ +#define V4L2_CTRL_CLASS_M2M_AUDIO 0x00a60000 /* Audio M2M controls */ /* User-class control IDs */ @@ -3491,6 +3492,9 @@ struct v4l2_ctrl_av1_film_grain { __u8 reserved[4]; }; +#define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900) +#define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1) + /* MPEG-compression definitions kept for backwards compatibility */ #ifndef __KERNEL__ #define V4L2_CTRL_CLASS_MPEG V4L2_CTRL_CLASS_CODEC From patchwork Mon Feb 26 08:28:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shengjiu Wang X-Patchwork-Id: 776113 Received: from inva020.nxp.com (inva020.nxp.com [92.121.34.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8385B57892; Mon, 26 Feb 2024 08:44:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=92.121.34.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937071; cv=none; b=l/Tsmrees2QZiJqZ7KNN65vl7SLqANtBBx9xewCJREemhR4woc5s++m/8VIooCs716S9r0CLY0STydt8kGhQZsF0uQdV3scHR6VGy+cbEhYv5CX2K/n4Jcxw5zL5pX+K1v8eeZazBNs6fO1VEQOcw/FqhCXhhdw3YEl9PfjHu3I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937071; c=relaxed/simple; bh=fgYQVMxpAmTgYDCZIFPEaCK+FRLJh6UXAq9wyXsuRvM=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References; b=HeDlH8pXMtw1Y+q5XcI/ANuA+M76WXa93yySRMDvxPaoIeoMGAlDnzvcJ1cpsERbGe5WDvOsDV8NGAPdLwfFmabk7w4U7qMmP47wY0DjLT+VzG3N2miwECbua5yA5+mfCwh7CoW5l1BafCdA29f2PuXNZUN1WJevSnBM8XSsoCw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com; spf=pass smtp.mailfrom=nxp.com; arc=none smtp.client-ip=92.121.34.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nxp.com Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id F01A31A06F5; Mon, 26 Feb 2024 09:44:27 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 8BF4E1A0138; Mon, 26 Feb 2024 09:44:27 +0100 (CET) Received: from localhost.localdomain (shlinux2.ap.freescale.net [10.192.224.44]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 811F8183AD17; Mon, 26 Feb 2024 16:44:25 +0800 (+08) From: Shengjiu Wang To: hverkuil@xs4all.nl, sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, shengjiu.wang@gmail.com, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v13 12/16] media: uapi: Declare interface types for Audio Date: Mon, 26 Feb 2024 16:28:25 +0800 Message-Id: <1708936109-11587-13-git-send-email-shengjiu.wang@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> References: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Declare the interface types that will be used by Audio. The type is MEDIA_INTF_T_V4L_AUDIO. Signed-off-by: Shengjiu Wang --- .../userspace-api/media/mediactl/media-types.rst | 5 +++++ drivers/media/v4l2-core/v4l2-dev.c | 4 ++++ drivers/media/v4l2-core/v4l2-mem2mem.c | 13 +++++++++---- include/uapi/linux/media.h | 1 + 4 files changed, 19 insertions(+), 4 deletions(-) diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/Documentation/userspace-api/media/mediactl/media-types.rst index 6332e8395263..adfb37430f8e 100644 --- a/Documentation/userspace-api/media/mediactl/media-types.rst +++ b/Documentation/userspace-api/media/mediactl/media-types.rst @@ -265,6 +265,7 @@ Types and flags used to represent the media graph elements .. _MEDIA-INTF-T-V4L-SUBDEV: .. _MEDIA-INTF-T-V4L-SWRADIO: .. _MEDIA-INTF-T-V4L-TOUCH: +.. _MEDIA-INTF-T-V4L-AUDIO: .. _MEDIA-INTF-T-ALSA-PCM-CAPTURE: .. _MEDIA-INTF-T-ALSA-PCM-PLAYBACK: .. _MEDIA-INTF-T-ALSA-CONTROL: @@ -322,6 +323,10 @@ Types and flags used to represent the media graph elements - Device node interface for Touch device (V4L) - typically, /dev/v4l-touch? + * - ``MEDIA_INTF_T_V4L_AUDIO`` + - Device node interface for Audio device (V4L) + - typically, /dev/v4l-audio? + * - ``MEDIA_INTF_T_ALSA_PCM_CAPTURE`` - Device node interface for ALSA PCM Capture - typically, /dev/snd/pcmC?D?c diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index bac008fcedc6..ca8462a61e1f 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -844,6 +844,10 @@ static int video_register_media_controller(struct video_device *vdev) intf_type = MEDIA_INTF_T_V4L_SUBDEV; /* Entity will be created via v4l2_device_register_subdev() */ break; + case VFL_TYPE_AUDIO: + intf_type = MEDIA_INTF_T_V4L_AUDIO; + /* Entity will be created via v4l2_device_register_subdev() */ + break; default: return 0; } diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c index 75517134a5e9..cda5e255305f 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -1143,10 +1143,15 @@ int v4l2_m2m_register_media_controller(struct v4l2_m2m_dev *m2m_dev, if (ret) goto err_rm_links0; - /* Create video interface */ - m2m_dev->intf_devnode = media_devnode_create(mdev, - MEDIA_INTF_T_V4L_VIDEO, 0, - VIDEO_MAJOR, vdev->minor); + if (vdev->vfl_type == VFL_TYPE_AUDIO) + m2m_dev->intf_devnode = media_devnode_create(mdev, + MEDIA_INTF_T_V4L_AUDIO, 0, + VIDEO_MAJOR, vdev->minor); + else + /* Create video interface */ + m2m_dev->intf_devnode = media_devnode_create(mdev, + MEDIA_INTF_T_V4L_VIDEO, 0, + VIDEO_MAJOR, vdev->minor); if (!m2m_dev->intf_devnode) { ret = -ENOMEM; goto err_rm_links1; diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h index 1c80b1d6bbaf..9ff6dec7393a 100644 --- a/include/uapi/linux/media.h +++ b/include/uapi/linux/media.h @@ -260,6 +260,7 @@ struct media_links_enum { #define MEDIA_INTF_T_V4L_SUBDEV (MEDIA_INTF_T_V4L_BASE + 3) #define MEDIA_INTF_T_V4L_SWRADIO (MEDIA_INTF_T_V4L_BASE + 4) #define MEDIA_INTF_T_V4L_TOUCH (MEDIA_INTF_T_V4L_BASE + 5) +#define MEDIA_INTF_T_V4L_AUDIO (MEDIA_INTF_T_V4L_BASE + 6) #define MEDIA_INTF_T_ALSA_BASE 0x00000300 #define MEDIA_INTF_T_ALSA_PCM_CAPTURE (MEDIA_INTF_T_ALSA_BASE) From patchwork Mon Feb 26 08:28:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shengjiu Wang X-Patchwork-Id: 776112 Received: from inva021.nxp.com (inva021.nxp.com [92.121.34.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9EE258239; Mon, 26 Feb 2024 08:44:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=92.121.34.21 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937074; cv=none; b=b1KGOsyZd0zryu0tJfGjMm03gAF5uKYLQUMdaQuMFNODx4rrs+DYSG77Q+fiix1zINZJIZEi9kbgMrVKakZznAgTqXCgtfhePXGoKE4Gy7Fq6LaXYe5rmJb1scBn56gJz1byOQyNNS6wI0ubSWabHCIx2PPKK8+KzM474whpJyU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937074; c=relaxed/simple; bh=zEmkkA+yi7ga6ejdhZg4Dk4xxa9NT4XDTpiEUfaLUFc=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References; b=mGGWtZKA3urmKfeOPBlo9LLZ8Kn7ZqSxfu/gPNqhI8ck3JiPEs0uO56Ysv9ff14zqYs9o7GOpVm+i7iRhvGsKCpytiQeOpXcu4r8HKbAJeWjbAqdh5Ur+B5xqJyLYbtIiOoab1WSHkEKcCyVAgufejvlISvDZMxUQB5I4dvbX+Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com; spf=pass smtp.mailfrom=nxp.com; arc=none smtp.client-ip=92.121.34.21 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nxp.com Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 13F1F200B23; Mon, 26 Feb 2024 09:44:31 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id A270A201104; Mon, 26 Feb 2024 09:44:30 +0100 (CET) Received: from localhost.localdomain (shlinux2.ap.freescale.net [10.192.224.44]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id EEE67180222A; Mon, 26 Feb 2024 16:44:28 +0800 (+08) From: Shengjiu Wang To: hverkuil@xs4all.nl, sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, shengjiu.wang@gmail.com, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v13 14/16] media: vivid: add fixed point test controls Date: Mon, 26 Feb 2024 16:28:27 +0800 Message-Id: <1708936109-11587-15-git-send-email-shengjiu.wang@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> References: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Add fixed point test controls, one is for Q4.16 format another one is for Q63 format. Signed-off-by: Shengjiu Wang --- drivers/media/test-drivers/vivid/vivid-core.h | 2 ++ .../media/test-drivers/vivid/vivid-ctrls.c | 26 +++++++++++++++++++ include/media/v4l2-ctrls.h | 6 +++++ 3 files changed, 34 insertions(+) diff --git a/drivers/media/test-drivers/vivid/vivid-core.h b/drivers/media/test-drivers/vivid/vivid-core.h index cfb8e66083f6..f65465191bc9 100644 --- a/drivers/media/test-drivers/vivid/vivid-core.h +++ b/drivers/media/test-drivers/vivid/vivid-core.h @@ -222,6 +222,8 @@ struct vivid_dev { struct v4l2_ctrl *boolean; struct v4l2_ctrl *int32; struct v4l2_ctrl *int64; + struct v4l2_ctrl *int32_q16; + struct v4l2_ctrl *int64_q63; struct v4l2_ctrl *menu; struct v4l2_ctrl *string; struct v4l2_ctrl *bitmask; diff --git a/drivers/media/test-drivers/vivid/vivid-ctrls.c b/drivers/media/test-drivers/vivid/vivid-ctrls.c index f2b20e25a7a4..2444ea95b285 100644 --- a/drivers/media/test-drivers/vivid/vivid-ctrls.c +++ b/drivers/media/test-drivers/vivid/vivid-ctrls.c @@ -38,6 +38,8 @@ #define VIVID_CID_U8_PIXEL_ARRAY (VIVID_CID_CUSTOM_BASE + 14) #define VIVID_CID_S32_ARRAY (VIVID_CID_CUSTOM_BASE + 15) #define VIVID_CID_S64_ARRAY (VIVID_CID_CUSTOM_BASE + 16) +#define VIVID_CID_INT_Q4_16 (VIVID_CID_CUSTOM_BASE + 17) +#define VIVID_CID_INT64_Q63 (VIVID_CID_CUSTOM_BASE + 18) #define VIVID_CID_VIVID_BASE (0x00f00000 | 0xf000) #define VIVID_CID_VIVID_CLASS (0x00f00000 | 1) @@ -182,6 +184,28 @@ static const struct v4l2_ctrl_config vivid_ctrl_int64 = { .step = 1, }; +static const struct v4l2_ctrl_config vivid_ctrl_int32_q16 = { + .ops = &vivid_user_gen_ctrl_ops, + .id = VIVID_CID_INT_Q4_16, + .name = "Integer 32 Bits Q4.16", + .type = V4L2_CTRL_TYPE_INTEGER, + .min = v4l2_ctrl_fp_compose(-16, 0, 16), + .max = v4l2_ctrl_fp_compose(15, 0xffff, 16), + .step = 1, + .fraction_bits = 16, +}; + +static const struct v4l2_ctrl_config vivid_ctrl_int64_q63 = { + .ops = &vivid_user_gen_ctrl_ops, + .id = VIVID_CID_INT64_Q63, + .name = "Integer 64 Bits Q63", + .type = V4L2_CTRL_TYPE_INTEGER64, + .min = v4l2_ctrl_fp_compose(-1, 0, 63), + .max = v4l2_ctrl_fp_compose(0, LLONG_MAX, 63), + .step = 1, + .fraction_bits = 63, +}; + static const struct v4l2_ctrl_config vivid_ctrl_u32_array = { .ops = &vivid_user_gen_ctrl_ops, .id = VIVID_CID_U32_ARRAY, @@ -1670,6 +1694,8 @@ int vivid_create_controls(struct vivid_dev *dev, bool show_ccs_cap, dev->button = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_button, NULL); dev->int32 = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_int32, NULL); dev->int64 = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_int64, NULL); + dev->int32_q16 = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_int32_q16, NULL); + dev->int64_q63 = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_int64_q63, NULL); dev->boolean = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_boolean, NULL); dev->menu = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_menu, NULL); dev->string = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_string, NULL); diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h index c35514c5bf88..197d8b67ac13 100644 --- a/include/media/v4l2-ctrls.h +++ b/include/media/v4l2-ctrls.h @@ -1593,4 +1593,10 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl); */ int v4l2_ctrl_type_op_validate(const struct v4l2_ctrl *ctrl, union v4l2_ctrl_ptr ptr); +/* + * Fixed point compose helper define. This helper maps to the value + * i + f / (1 << fraction_bits). + */ +#define v4l2_ctrl_fp_compose(i, f, fraction_bits) (((s64)(i) << fraction_bits) + (f)) + #endif From patchwork Mon Feb 26 08:28:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shengjiu Wang X-Patchwork-Id: 776111 Received: from inva021.nxp.com (inva021.nxp.com [92.121.34.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E992159176; Mon, 26 Feb 2024 08:44:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=92.121.34.21 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937079; cv=none; b=XIqipDZZlRLro0zWOk78ZBnwLjFvhtk5/3263XzvxcOp+FqTLlEnjTpoLYw1BQnq/hAqRUl3d9W4WArlnYHkdHOqAzA36x/WE++GffPVEJn2DFgKM4va+z5MrkpJwSRHdkclv/84FNTdfnQvhbf7L43qGa2Hiajo01dyj2M1awY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708937079; c=relaxed/simple; bh=wxMRKG+UGk9dzZ9+AdwbpDfcwr+3kQggZ7FEiB1vDL0=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References; b=tOezqc/EnC3F/gtxsC/5d9JGHwqCskV5UD8CtmKYHnMhnnoa908BhTEynLk88RN6F56/qwC/9mYlI6eHUo7ZwRTOdDQAcj158E6EwlqsKWE4jRz12cTUmGfOLdDZODIoCtt05YRZ2bdOTanRO6k2tXWSrfOtT6bFtkqTcDAb1hQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com; spf=pass smtp.mailfrom=nxp.com; arc=none smtp.client-ip=92.121.34.21 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nxp.com Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 5D8D9201101; Mon, 26 Feb 2024 09:44:35 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id EABF42010FC; Mon, 26 Feb 2024 09:44:34 +0100 (CET) Received: from localhost.localdomain (shlinux2.ap.freescale.net [10.192.224.44]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 8C7D1183AC0B; Mon, 26 Feb 2024 16:44:32 +0800 (+08) From: Shengjiu Wang To: hverkuil@xs4all.nl, sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, shengjiu.wang@gmail.com, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v13 16/16] media: vim2m-audio: add virtual driver for audio memory to memory Date: Mon, 26 Feb 2024 16:28:29 +0800 Message-Id: <1708936109-11587-17-git-send-email-shengjiu.wang@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> References: <1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Audio memory to memory virtual driver use video memory to memory virtual driver vim2m.c as example. The main difference is device type is VFL_TYPE_AUDIO and device cap type is V4L2_CAP_AUDIO_M2M. The device_run function is a dummy function, which is simply copy the data from input buffer to output buffer. Signed-off-by: Shengjiu Wang --- drivers/media/test-drivers/Kconfig | 10 + drivers/media/test-drivers/Makefile | 1 + drivers/media/test-drivers/vim2m-audio.c | 793 +++++++++++++++++++++++ 3 files changed, 804 insertions(+) create mode 100644 drivers/media/test-drivers/vim2m-audio.c diff --git a/drivers/media/test-drivers/Kconfig b/drivers/media/test-drivers/Kconfig index 5a5379524bde..b6b52a7ca042 100644 --- a/drivers/media/test-drivers/Kconfig +++ b/drivers/media/test-drivers/Kconfig @@ -16,6 +16,16 @@ config VIDEO_VIM2M This is a virtual test device for the memory-to-memory driver framework. +config VIDEO_VIM2M_AUDIO + tristate "Virtual Memory-to-Memory Driver For Audio" + depends on VIDEO_DEV + select VIDEOBUF2_VMALLOC + select V4L2_MEM2MEM_DEV + select MEDIA_CONTROLLER + help + This is a virtual audio test device for the memory-to-memory driver + framework. + source "drivers/media/test-drivers/vicodec/Kconfig" source "drivers/media/test-drivers/vimc/Kconfig" source "drivers/media/test-drivers/vivid/Kconfig" diff --git a/drivers/media/test-drivers/Makefile b/drivers/media/test-drivers/Makefile index 740714a4584d..0c61c9ada3e1 100644 --- a/drivers/media/test-drivers/Makefile +++ b/drivers/media/test-drivers/Makefile @@ -10,6 +10,7 @@ obj-$(CONFIG_DVB_VIDTV) += vidtv/ obj-$(CONFIG_VIDEO_VICODEC) += vicodec/ obj-$(CONFIG_VIDEO_VIM2M) += vim2m.o +obj-$(CONFIG_VIDEO_VIM2M_AUDIO) += vim2m-audio.o obj-$(CONFIG_VIDEO_VIMC) += vimc/ obj-$(CONFIG_VIDEO_VIVID) += vivid/ obj-$(CONFIG_VIDEO_VISL) += visl/ diff --git a/drivers/media/test-drivers/vim2m-audio.c b/drivers/media/test-drivers/vim2m-audio.c new file mode 100644 index 000000000000..6361df6320b3 --- /dev/null +++ b/drivers/media/test-drivers/vim2m-audio.c @@ -0,0 +1,793 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * A virtual v4l2-mem2mem example for audio device. + */ + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +MODULE_DESCRIPTION("Virtual device for audio mem2mem testing"); +MODULE_LICENSE("GPL"); + +static unsigned int debug; +module_param(debug, uint, 0644); +MODULE_PARM_DESC(debug, "debug level"); + +#define MEM2MEM_NAME "vim2m-audio" + +#define dprintk(dev, lvl, fmt, arg...) \ + v4l2_dbg(lvl, debug, &(dev)->v4l2_dev, "%s: " fmt, __func__, ## arg) + +#define SAMPLE_NUM 4096 + +static void audm2m_dev_release(struct device *dev) +{} + +static struct platform_device audm2m_pdev = { + .name = MEM2MEM_NAME, + .dev.release = audm2m_dev_release, +}; + +static u32 formats[] = { + V4L2_AUDIO_FMT_S16_LE, +}; + +#define NUM_FORMATS ARRAY_SIZE(formats) + +/* Per-queue, driver-specific private data */ +struct audm2m_q_data { + unsigned int rate; + unsigned int channels; + unsigned int buffersize; + unsigned int sequence; + u32 fourcc; +}; + +enum { + V4L2_M2M_SRC = 0, + V4L2_M2M_DST = 1, +}; + +static snd_pcm_format_t find_format(u32 fourcc) +{ + snd_pcm_format_t fmt; + unsigned int k; + + for (k = 0; k < NUM_FORMATS; k++) { + if (formats[k] == fourcc) + break; + } + + if (k == NUM_FORMATS) + return 0; + + fmt = v4l2_fourcc_to_audfmt(formats[k]); + + return fmt; +} + +struct audm2m_dev { + struct v4l2_device v4l2_dev; + struct video_device vfd; + + struct mutex dev_mutex; + + struct v4l2_m2m_dev *m2m_dev; +#ifdef CONFIG_MEDIA_CONTROLLER + struct media_device mdev; +#endif +}; + +struct audm2m_ctx { + struct v4l2_fh fh; + struct v4l2_ctrl_handler ctrl_handler; + struct audm2m_dev *dev; + + struct mutex vb_mutex; + + /* Source and destination queue data */ + struct audm2m_q_data q_data[2]; +}; + +static inline struct audm2m_ctx *file2ctx(struct file *file) +{ + return container_of(file->private_data, struct audm2m_ctx, fh); +} + +static struct audm2m_q_data *get_q_data(struct audm2m_ctx *ctx, + enum v4l2_buf_type type) +{ + if (type == V4L2_BUF_TYPE_AUDIO_OUTPUT) + return &ctx->q_data[V4L2_M2M_SRC]; + return &ctx->q_data[V4L2_M2M_DST]; +} + +static const char *type_name(enum v4l2_buf_type type) +{ + if (type == V4L2_BUF_TYPE_AUDIO_OUTPUT) + return "Output"; + return "Capture"; +} + +/* + * mem2mem callbacks + */ + +/* + * device_run() - prepares and starts the device + */ +static void device_run(void *priv) +{ + struct audm2m_ctx *ctx = priv; + struct audm2m_dev *audm2m_dev; + struct vb2_v4l2_buffer *src_buf, *dst_buf; + struct audm2m_q_data *q_data_src, *q_data_dst; + int src_size, dst_size = 0; + short *src_addr, *dst_addr; + int i; + + audm2m_dev = ctx->dev; + + q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_AUDIO_OUTPUT); + if (!q_data_src) + return; + + q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_AUDIO_CAPTURE); + if (!q_data_dst) + return; + + src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx); + dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx); + src_buf->sequence = q_data_src->sequence++; + dst_buf->sequence = q_data_dst->sequence++; + v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, false); + + /* Process the conversion */ + src_size = vb2_get_plane_payload(&src_buf->vb2_buf, 0); + + src_addr = vb2_plane_vaddr(&src_buf->vb2_buf, 0); + dst_addr = vb2_plane_vaddr(&dst_buf->vb2_buf, 0); + + if (q_data_src->rate == q_data_dst->rate) { + memcpy(dst_addr, src_addr, src_size); + dst_size = src_size; + } else if (q_data_src->rate == 2 * q_data_dst->rate) { + /* 8k to 16k */ + for (i = 0; i < src_size / 2; i++) { + *dst_addr++ = *src_addr++; + src_addr++; + } + + dst_size = src_size / 2; + } else if (q_data_src->rate * 2 == q_data_dst->rate) { + /* 16k to 8k */ + for (i = 0; i < src_size / 2; i++) { + *dst_addr++ = *src_addr; + *dst_addr++ = *src_addr++; + } + + dst_size = src_size * 2; + } + + vb2_set_plane_payload(&dst_buf->vb2_buf, 0, dst_size); + + src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); + dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx); + + v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE); + v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE); + v4l2_m2m_job_finish(audm2m_dev->m2m_dev, ctx->fh.m2m_ctx); +} + +static int audm2m_querycap(struct file *file, void *priv, + struct v4l2_capability *cap) +{ + strscpy(cap->driver, MEM2MEM_NAME, sizeof(cap->driver)); + strscpy(cap->card, MEM2MEM_NAME, sizeof(cap->card)); + + return 0; +} + +static int enum_fmt(struct v4l2_fmtdesc *f) +{ + int i, num; + + num = 0; + + for (i = 0; i < NUM_FORMATS; ++i) { + if (num == f->index) + break; + /* + * Correct type but haven't reached our index yet, + * just increment per-type index + */ + ++num; + } + + if (i < NUM_FORMATS) { + /* Format found */ + f->pixelformat = formats[i]; + return 0; + } + + /* Format not found */ + return -EINVAL; +} + +static int audm2m_enum_fmt_audio_cap(struct file *file, void *priv, + struct v4l2_fmtdesc *f) +{ + return enum_fmt(f); +} + +static int audm2m_enum_fmt_audio_out(struct file *file, void *priv, + struct v4l2_fmtdesc *f) +{ + return enum_fmt(f); +} + +static int audm2m_g_fmt(struct audm2m_ctx *ctx, struct v4l2_format *f) +{ + struct vb2_queue *vq; + struct audm2m_q_data *q_data; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type); + if (!vq) + return -EINVAL; + + q_data = get_q_data(ctx, f->type); + if (!q_data) + return -EINVAL; + + f->fmt.audio.audioformat = q_data->fourcc; + f->fmt.audio.channels = q_data->channels; + f->fmt.audio.buffersize = q_data->buffersize; + + return 0; +} + +static int audm2m_g_fmt_audio_out(struct file *file, void *priv, + struct v4l2_format *f) +{ + return audm2m_g_fmt(file2ctx(file), f); +} + +static int audm2m_g_fmt_audio_cap(struct file *file, void *priv, + struct v4l2_format *f) +{ + return audm2m_g_fmt(file2ctx(file), f); +} + +static int audm2m_try_fmt(struct v4l2_format *f, snd_pcm_format_t fmt) +{ + f->fmt.audio.channels = 1; + f->fmt.audio.buffersize = f->fmt.audio.channels * + snd_pcm_format_physical_width(fmt) * + SAMPLE_NUM; + return 0; +} + +static int audm2m_try_fmt_audio_cap(struct file *file, void *priv, + struct v4l2_format *f) +{ + snd_pcm_format_t fmt; + + fmt = find_format(f->fmt.audio.audioformat); + if (!fmt) { + f->fmt.audio.audioformat = formats[0]; + fmt = find_format(f->fmt.audio.audioformat); + } + + return audm2m_try_fmt(f, fmt); +} + +static int audm2m_try_fmt_audio_out(struct file *file, void *priv, + struct v4l2_format *f) +{ + snd_pcm_format_t fmt; + + fmt = find_format(f->fmt.audio.audioformat); + if (!fmt) { + f->fmt.audio.audioformat = formats[0]; + fmt = find_format(f->fmt.audio.audioformat); + } + + return audm2m_try_fmt(f, fmt); +} + +static int audm2m_s_fmt(struct audm2m_ctx *ctx, struct v4l2_format *f) +{ + struct audm2m_q_data *q_data; + struct vb2_queue *vq; + snd_pcm_format_t fmt; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type); + if (!vq) + return -EINVAL; + + q_data = get_q_data(ctx, f->type); + if (!q_data) + return -EINVAL; + + if (vb2_is_busy(vq)) { + v4l2_err(&ctx->dev->v4l2_dev, "%s queue busy\n", __func__); + return -EBUSY; + } + + q_data->fourcc = f->fmt.audio.audioformat; + q_data->channels = f->fmt.audio.channels; + + fmt = find_format(f->fmt.audio.audioformat); + q_data->buffersize = q_data->channels * + snd_pcm_format_physical_width(fmt) * + SAMPLE_NUM; + + dprintk(ctx->dev, 1, + "Format for type %s: %d/%d, fmt: %c%c%c%c\n", + type_name(f->type), q_data->rate, + q_data->channels, + (q_data->fourcc & 0xff), + (q_data->fourcc >> 8) & 0xff, + (q_data->fourcc >> 16) & 0xff, + (q_data->fourcc >> 24) & 0xff); + + return 0; +} + +static int audm2m_s_fmt_audio_cap(struct file *file, void *priv, + struct v4l2_format *f) +{ + int ret; + + ret = audm2m_try_fmt_audio_cap(file, priv, f); + if (ret) + return ret; + + return audm2m_s_fmt(file2ctx(file), f); +} + +static int audm2m_s_fmt_audio_out(struct file *file, void *priv, + struct v4l2_format *f) +{ + int ret; + + ret = audm2m_try_fmt_audio_out(file, priv, f); + if (ret) + return ret; + + return audm2m_s_fmt(file2ctx(file), f); +} + +static const struct v4l2_ioctl_ops audm2m_ioctl_ops = { + .vidioc_querycap = audm2m_querycap, + + .vidioc_enum_fmt_audio_cap = audm2m_enum_fmt_audio_cap, + .vidioc_g_fmt_audio_cap = audm2m_g_fmt_audio_cap, + .vidioc_try_fmt_audio_cap = audm2m_try_fmt_audio_cap, + .vidioc_s_fmt_audio_cap = audm2m_s_fmt_audio_cap, + + .vidioc_enum_fmt_audio_out = audm2m_enum_fmt_audio_out, + .vidioc_g_fmt_audio_out = audm2m_g_fmt_audio_out, + .vidioc_try_fmt_audio_out = audm2m_try_fmt_audio_out, + .vidioc_s_fmt_audio_out = audm2m_s_fmt_audio_out, + + .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, + .vidioc_querybuf = v4l2_m2m_ioctl_querybuf, + .vidioc_qbuf = v4l2_m2m_ioctl_qbuf, + .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, + .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, + .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, + .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, + + .vidioc_streamon = v4l2_m2m_ioctl_streamon, + .vidioc_streamoff = v4l2_m2m_ioctl_streamoff, + + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, + .vidioc_unsubscribe_event = v4l2_event_unsubscribe, +}; + +/* + * Queue operations + */ +static int audm2m_queue_setup(struct vb2_queue *vq, + unsigned int *nbuffers, + unsigned int *nplanes, + unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct audm2m_ctx *ctx = vb2_get_drv_priv(vq); + struct audm2m_q_data *q_data; + + q_data = get_q_data(ctx, vq->type); + + if (*nplanes) + return sizes[0] < q_data->buffersize ? -EINVAL : 0; + + *nplanes = 1; + sizes[0] = q_data->buffersize; + + dprintk(ctx->dev, 1, "%s: get %d buffer(s) of size %d each.\n", + type_name(vq->type), *nplanes, sizes[0]); + + return 0; +} + +static void audm2m_buf_queue(struct vb2_buffer *vb) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct audm2m_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue); + + v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf); +} + +static int audm2m_start_streaming(struct vb2_queue *q, unsigned int count) +{ + struct audm2m_ctx *ctx = vb2_get_drv_priv(q); + struct audm2m_q_data *q_data = get_q_data(ctx, q->type); + + q_data->sequence = 0; + return 0; +} + +static void audm2m_stop_streaming(struct vb2_queue *q) +{ + struct audm2m_ctx *ctx = vb2_get_drv_priv(q); + struct vb2_v4l2_buffer *vbuf; + + for (;;) { + if (V4L2_TYPE_IS_OUTPUT(q->type)) + vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx); + else + vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx); + if (!vbuf) + return; + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR); + } +} + +static const struct vb2_ops audm2m_qops = { + .queue_setup = audm2m_queue_setup, + .buf_queue = audm2m_buf_queue, + .start_streaming = audm2m_start_streaming, + .stop_streaming = audm2m_stop_streaming, + .wait_prepare = vb2_ops_wait_prepare, + .wait_finish = vb2_ops_wait_finish, +}; + +static int queue_init(void *priv, struct vb2_queue *src_vq, + struct vb2_queue *dst_vq) +{ + struct audm2m_ctx *ctx = priv; + int ret; + + src_vq->type = V4L2_BUF_TYPE_AUDIO_OUTPUT; + src_vq->io_modes = VB2_MMAP | VB2_DMABUF; + src_vq->drv_priv = ctx; + src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer); + src_vq->ops = &audm2m_qops; + src_vq->mem_ops = &vb2_vmalloc_memops; + src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + src_vq->lock = &ctx->vb_mutex; + + ret = vb2_queue_init(src_vq); + if (ret) + return ret; + + dst_vq->type = V4L2_BUF_TYPE_AUDIO_CAPTURE; + dst_vq->io_modes = VB2_MMAP | VB2_DMABUF; + dst_vq->drv_priv = ctx; + dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer); + dst_vq->ops = &audm2m_qops; + dst_vq->mem_ops = &vb2_vmalloc_memops; + dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + dst_vq->lock = &ctx->vb_mutex; + + return vb2_queue_init(dst_vq); +} + +static const s64 audm2m_rates[] = { + 8000, 16000, +}; + +static int audm2m_op_s_ctrl(struct v4l2_ctrl *ctrl) +{ + struct audm2m_ctx *ctx = + container_of(ctrl->handler, struct audm2m_ctx, ctrl_handler); + int ret = 0; + + switch (ctrl->id) { + case V4L2_CID_M2M_AUDIO_SOURCE_RATE: + ctx->q_data[V4L2_M2M_SRC].rate = ctrl->qmenu_int[ctrl->val]; + break; + case V4L2_CID_M2M_AUDIO_DEST_RATE: + ctx->q_data[V4L2_M2M_DST].rate = ctrl->qmenu_int[ctrl->val]; + break; + default: + ret = -EINVAL; + break; + } + + return ret; +} + +static const struct v4l2_ctrl_ops audm2m_ctrl_ops = { + .s_ctrl = audm2m_op_s_ctrl, +}; + +/* + * File operations + */ +static int audm2m_open(struct file *file) +{ + struct audm2m_dev *dev = video_drvdata(file); + struct audm2m_ctx *ctx = NULL; + snd_pcm_format_t fmt; + int width; + int rc = 0; + + if (mutex_lock_interruptible(&dev->dev_mutex)) + return -ERESTARTSYS; + ctx = kzalloc(sizeof(*ctx), GFP_KERNEL); + if (!ctx) { + rc = -ENOMEM; + goto open_unlock; + } + + v4l2_fh_init(&ctx->fh, video_devdata(file)); + file->private_data = &ctx->fh; + ctx->dev = dev; + + ctx->q_data[V4L2_M2M_SRC].fourcc = formats[0]; + ctx->q_data[V4L2_M2M_SRC].rate = 8000; + ctx->q_data[V4L2_M2M_SRC].channels = 1; + + /* Fix to 4096 samples */ + fmt = find_format(formats[0]); + width = snd_pcm_format_physical_width(fmt); + ctx->q_data[V4L2_M2M_SRC].buffersize = SAMPLE_NUM * width; + ctx->q_data[V4L2_M2M_DST] = ctx->q_data[V4L2_M2M_SRC]; + + ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx, &queue_init); + + mutex_init(&ctx->vb_mutex); + + if (IS_ERR(ctx->fh.m2m_ctx)) { + rc = PTR_ERR(ctx->fh.m2m_ctx); + + v4l2_fh_exit(&ctx->fh); + kfree(ctx); + goto open_unlock; + } + + v4l2_fh_add(&ctx->fh); + + dprintk(dev, 1, "Created instance: %p, m2m_ctx: %p\n", + ctx, ctx->fh.m2m_ctx); + + v4l2_ctrl_handler_init(&ctx->ctrl_handler, 2); + + v4l2_ctrl_new_int_menu(&ctx->ctrl_handler, &audm2m_ctrl_ops, + V4L2_CID_M2M_AUDIO_SOURCE_RATE, + ARRAY_SIZE(audm2m_rates) - 1, 0, audm2m_rates); + v4l2_ctrl_new_int_menu(&ctx->ctrl_handler, &audm2m_ctrl_ops, + V4L2_CID_M2M_AUDIO_DEST_RATE, + ARRAY_SIZE(audm2m_rates) - 1, 0, audm2m_rates); + + if (ctx->ctrl_handler.error) { + rc = ctx->ctrl_handler.error; + v4l2_ctrl_handler_free(&ctx->ctrl_handler); + goto err_ctrl_handler; + } + + ctx->fh.ctrl_handler = &ctx->ctrl_handler; + + mutex_unlock(&dev->dev_mutex); + + return 0; + +err_ctrl_handler: + v4l2_m2m_ctx_release(ctx->fh.m2m_ctx); +open_unlock: + mutex_unlock(&dev->dev_mutex); + return rc; +} + +static int audm2m_release(struct file *file) +{ + struct audm2m_dev *dev = video_drvdata(file); + struct audm2m_ctx *ctx = file2ctx(file); + + dprintk(dev, 1, "Releasing instance %p\n", ctx); + + v4l2_ctrl_handler_free(&ctx->ctrl_handler); + v4l2_fh_del(&ctx->fh); + v4l2_fh_exit(&ctx->fh); + mutex_lock(&dev->dev_mutex); + v4l2_m2m_ctx_release(ctx->fh.m2m_ctx); + mutex_unlock(&dev->dev_mutex); + kfree(ctx); + + return 0; +} + +static void audm2m_device_release(struct video_device *vdev) +{ + struct audm2m_dev *dev = container_of(vdev, struct audm2m_dev, vfd); + + v4l2_device_unregister(&dev->v4l2_dev); + v4l2_m2m_release(dev->m2m_dev); + +#ifdef CONFIG_MEDIA_CONTROLLER + media_device_cleanup(&dev->mdev); +#endif + kfree(dev); +} + +static const struct v4l2_file_operations audm2m_fops = { + .owner = THIS_MODULE, + .open = audm2m_open, + .release = audm2m_release, + .poll = v4l2_m2m_fop_poll, + .unlocked_ioctl = video_ioctl2, + .mmap = v4l2_m2m_fop_mmap, +}; + +static const struct video_device audm2m_videodev = { + .name = MEM2MEM_NAME, + .vfl_dir = VFL_DIR_M2M, + .fops = &audm2m_fops, + .ioctl_ops = &audm2m_ioctl_ops, + .minor = -1, + .release = audm2m_device_release, + .device_caps = V4L2_CAP_AUDIO_M2M | V4L2_CAP_STREAMING, +}; + +static const struct v4l2_m2m_ops m2m_ops = { + .device_run = device_run, +}; + +static const struct media_device_ops audm2m_media_ops = { + .req_validate = vb2_request_validate, + .req_queue = v4l2_m2m_request_queue, +}; + +static int audm2m_probe(struct platform_device *pdev) +{ + struct audm2m_dev *dev; + struct video_device *vfd; + int ret; + + dev = kzalloc(sizeof(*dev), GFP_KERNEL); + if (!dev) + return -ENOMEM; + + ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev); + if (ret) + goto error_free; + + mutex_init(&dev->dev_mutex); + + dev->vfd = audm2m_videodev; + vfd = &dev->vfd; + vfd->lock = &dev->dev_mutex; + vfd->v4l2_dev = &dev->v4l2_dev; + + video_set_drvdata(vfd, dev); + platform_set_drvdata(pdev, dev); + + dev->m2m_dev = v4l2_m2m_init(&m2m_ops); + if (IS_ERR(dev->m2m_dev)) { + v4l2_err(&dev->v4l2_dev, "Failed to init mem2mem device\n"); + ret = PTR_ERR(dev->m2m_dev); + dev->m2m_dev = NULL; + goto error_dev; + } + +#ifdef CONFIG_MEDIA_CONTROLLER + dev->mdev.dev = &pdev->dev; + strscpy(dev->mdev.model, MEM2MEM_NAME, sizeof(dev->mdev.model)); + media_device_init(&dev->mdev); + dev->mdev.ops = &audm2m_media_ops; + dev->v4l2_dev.mdev = &dev->mdev; +#endif + + ret = video_register_device(vfd, VFL_TYPE_AUDIO, 0); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to register video device\n"); + goto error_m2m; + } + +#ifdef CONFIG_MEDIA_CONTROLLER + ret = v4l2_m2m_register_media_controller(dev->m2m_dev, vfd, + MEDIA_ENT_F_PROC_AUDIO_RESAMPLER); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to init mem2mem media controller\n"); + goto error_v4l2; + } + + ret = media_device_register(&dev->mdev); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to register mem2mem media device\n"); + goto error_m2m_mc; + } +#endif + + v4l2_info(&dev->v4l2_dev, + "Device registered as /dev/v4l-audio%d\n", vfd->num); + + return 0; + +#ifdef CONFIG_MEDIA_CONTROLLER +error_m2m_mc: + v4l2_m2m_unregister_media_controller(dev->m2m_dev); +#endif +error_v4l2: + video_unregister_device(&dev->vfd); + /* audm2m_device_release called by video_unregister_device to release various objects */ + return ret; +error_m2m: + v4l2_m2m_release(dev->m2m_dev); +error_dev: + v4l2_device_unregister(&dev->v4l2_dev); +error_free: + kfree(dev); + + return ret; +} + +static void audm2m_remove(struct platform_device *pdev) +{ + struct audm2m_dev *dev = platform_get_drvdata(pdev); + + v4l2_info(&dev->v4l2_dev, "Removing " MEM2MEM_NAME); + +#ifdef CONFIG_MEDIA_CONTROLLER + media_device_unregister(&dev->mdev); + v4l2_m2m_unregister_media_controller(dev->m2m_dev); +#endif + video_unregister_device(&dev->vfd); +} + +static struct platform_driver audm2m_pdrv = { + .probe = audm2m_probe, + .remove_new = audm2m_remove, + .driver = { + .name = MEM2MEM_NAME, + }, +}; + +static void __exit audm2m_exit(void) +{ + platform_driver_unregister(&audm2m_pdrv); + platform_device_unregister(&audm2m_pdev); +} + +static int __init audm2m_init(void) +{ + int ret; + + ret = platform_device_register(&audm2m_pdev); + if (ret) + return ret; + + ret = platform_driver_register(&audm2m_pdrv); + if (ret) + platform_device_unregister(&audm2m_pdev); + + return ret; +} + +module_init(audm2m_init); +module_exit(audm2m_exit);