| Message ID | 1708936109-11587-1-git-send-email-shengjiu.wang@nxp.com |
|---|---|
| Headers | show |
| Series | Add audio support in v4l2 framework | expand |
Hi Shengjiu, After thinking it over I think this patch can be improved: On 26/02/2024 9:28 am, Shengjiu Wang wrote: > The audio sample format definition is from alsa, > the header file is include/uapi/sound/asound.h, but > don't include this header file directly, because in > user space, there is another copy in alsa-lib. > There will be conflict in userspace for include > videodev2.h & asound.h and asoundlib.h > > Here still use the fourcc format. > > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> > --- > .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ > .../userspace-api/media/v4l/pixfmt.rst | 1 + > drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ > include/uapi/linux/videodev2.h | 23 +++++ > 4 files changed, 124 insertions(+) > create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > new file mode 100644 > index 000000000000..04b4a7fbd8f4 > --- /dev/null > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > @@ -0,0 +1,87 @@ > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later > + > +.. _pixfmt-audio: > + > +************* > +Audio Formats > +************* > + > +These formats are used for :ref:`audiomem2mem` interface only. Here you should also document that all these fourccs start with 'AU' and are reserved for mappings of the snd_pcm_format_t type. Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc define (see also below). > + > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| > + > +.. cssclass:: longtable > + > +.. flat-table:: Audio Format > + :header-rows: 1 > + :stub-columns: 0 > + :widths: 3 1 4 > + > + * - Identifier > + - Code > + - Details > + * .. _V4L2-AUDIO-FMT-S8: > + > + - ``V4L2_AUDIO_FMT_S8`` > + - 'S8' > + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA > + * .. _V4L2-AUDIO-FMT-S16-LE: > + > + - ``V4L2_AUDIO_FMT_S16_LE`` > + - 'S16_LE' > + - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA > + * .. _V4L2-AUDIO-FMT-U16-LE: > + > + - ``V4L2_AUDIO_FMT_U16_LE`` > + - 'U16_LE' > + - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA > + * .. _V4L2-AUDIO-FMT-S24-LE: > + > + - ``V4L2_AUDIO_FMT_S24_LE`` > + - 'S24_LE' > + - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA > + * .. _V4L2-AUDIO-FMT-U24-LE: > + > + - ``V4L2_AUDIO_FMT_U24_LE`` > + - 'U24_LE' > + - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA > + * .. _V4L2-AUDIO-FMT-S32-LE: > + > + - ``V4L2_AUDIO_FMT_S32_LE`` > + - 'S32_LE' > + - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA > + * .. _V4L2-AUDIO-FMT-U32-LE: > + > + - ``V4L2_AUDIO_FMT_U32_LE`` > + - 'U32_LE' > + - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA > + * .. _V4L2-AUDIO-FMT-FLOAT-LE: > + > + - ``V4L2_AUDIO_FMT_FLOAT_LE`` > + - 'FLOAT_LE' > + - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA > + * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE: > + > + - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE`` > + - 'IEC958_SUBFRAME_LE' > + - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA > + * .. _V4L2-AUDIO-FMT-S24-3LE: > + > + - ``V4L2_AUDIO_FMT_S24_3LE`` > + - 'S24_3LE' > + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA > + * .. _V4L2-AUDIO-FMT-U24-3LE: > + > + - ``V4L2_AUDIO_FMT_U24_3LE`` > + - 'U24_3LE' > + - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA > + * .. _V4L2-AUDIO-FMT-S20-3LE: > + > + - ``V4L2_AUDIO_FMT_S20_3LE`` > + - 'S20_3LE' > + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA > + * .. _V4L2-AUDIO-FMT-U20-3LE: > + > + - ``V4L2_AUDIO_FMT_U20_3LE`` > + - 'U20_3LE' > + - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA > diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst > index 11dab4a90630..2eb6fdd3b43d 100644 > --- a/Documentation/userspace-api/media/v4l/pixfmt.rst > +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst > @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.) > colorspaces > colorspaces-defs > colorspaces-details > + pixfmt-audio > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c > index 961abcdf7290..be229c69e991 100644 > --- a/drivers/media/v4l2-core/v4l2-ioctl.c > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) > case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; > case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; > case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; > + case V4L2_AUDIO_FMT_S8: descr = "8-bit Signed"; break; > + case V4L2_AUDIO_FMT_S16_LE: descr = "16-bit Signed LE"; break; > + case V4L2_AUDIO_FMT_U16_LE: descr = "16-bit Unsigned LE"; break; > + case V4L2_AUDIO_FMT_S24_LE: descr = "24(32)-bit Signed LE"; break; > + case V4L2_AUDIO_FMT_U24_LE: descr = "24(32)-bit Unsigned LE"; break; > + case V4L2_AUDIO_FMT_S32_LE: descr = "32-bit Signed LE"; break; > + case V4L2_AUDIO_FMT_U32_LE: descr = "32-bit Unsigned LE"; break; > + case V4L2_AUDIO_FMT_FLOAT_LE: descr = "32-bit Float LE"; break; > + case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break; > + case V4L2_AUDIO_FMT_S24_3LE: descr = "24(24)-bit Signed LE"; break; > + case V4L2_AUDIO_FMT_U24_3LE: descr = "24(24)-bit Unsigned LE"; break; > + case V4L2_AUDIO_FMT_S20_3LE: descr = "20(24)-bit Signed LE"; break; > + case V4L2_AUDIO_FMT_U20_3LE: descr = "20(24)-bit Unsigned LE"; break; > > default: > /* Compressed formats */ > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > index 2c03d2dfadbe..673a6235a029 100644 > --- a/include/uapi/linux/videodev2.h > +++ b/include/uapi/linux/videodev2.h > @@ -843,6 +843,29 @@ struct v4l2_pix_format { > #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ > #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ > > +/* > + * Audio-data formats > + * All these audio formats use a fourcc starting with 'AU' > + * followed by the SNDRV_PCM_FORMAT_ value from asound.h. Also document here that fourccs starting with 'AU' are reserved for the snd_pcm_format_t to fourcc mappings. That to avoid that someone adds a 'AUXX' fourcc later. > + */ > +#define V4L2_AUDIO_FMT_S8 v4l2_fourcc('A', 'U', '0', '0') > +#define V4L2_AUDIO_FMT_S16_LE v4l2_fourcc('A', 'U', '0', '2') > +#define V4L2_AUDIO_FMT_U16_LE v4l2_fourcc('A', 'U', '0', '4') > +#define V4L2_AUDIO_FMT_S24_LE v4l2_fourcc('A', 'U', '0', '6') > +#define V4L2_AUDIO_FMT_U24_LE v4l2_fourcc('A', 'U', '0', '8') > +#define V4L2_AUDIO_FMT_S32_LE v4l2_fourcc('A', 'U', '1', '0') > +#define V4L2_AUDIO_FMT_U32_LE v4l2_fourcc('A', 'U', '1', '2') > +#define V4L2_AUDIO_FMT_FLOAT_LE v4l2_fourcc('A', 'U', '1', '4') > +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE v4l2_fourcc('A', 'U', '1', '8') > +#define V4L2_AUDIO_FMT_S24_3LE v4l2_fourcc('A', 'U', '3', '2') > +#define V4L2_AUDIO_FMT_U24_3LE v4l2_fourcc('A', 'U', '3', '4') > +#define V4L2_AUDIO_FMT_S20_3LE v4l2_fourcc('A', 'U', '3', '6') > +#define V4L2_AUDIO_FMT_U20_3LE v4l2_fourcc('A', 'U', '3', '8') > + > +#define v4l2_fourcc_to_audfmt(fourcc) \ > + (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10 \ > + + ((((fourcc) >> 24) & 0xff) - '0')) > + As I suggested in an earlier reply, add this: #define v4l2_audfmt_to_fourcc(audfmt) \ v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10) Even though it is not used in the drivers, since this is a public header used by drivers and applications, it makes sense to provide the reverse mapping as well. Please test it in actual code to make sure there are no compilation warnings. Regards, Hans > /* priv field value to indicates that subsequent fields are valid. */ > #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe >
On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil <hverkuil@xs4all.nl> wrote: > > Hi Shengjiu, > > After thinking it over I think this patch can be improved: > > On 26/02/2024 9:28 am, Shengjiu Wang wrote: > > The audio sample format definition is from alsa, > > the header file is include/uapi/sound/asound.h, but > > don't include this header file directly, because in > > user space, there is another copy in alsa-lib. > > There will be conflict in userspace for include > > videodev2.h & asound.h and asoundlib.h > > > > Here still use the fourcc format. > > > > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> > > --- > > .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ > > .../userspace-api/media/v4l/pixfmt.rst | 1 + > > drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ > > include/uapi/linux/videodev2.h | 23 +++++ > > 4 files changed, 124 insertions(+) > > create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > new file mode 100644 > > index 000000000000..04b4a7fbd8f4 > > --- /dev/null > > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > > @@ -0,0 +1,87 @@ > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later > > + > > +.. _pixfmt-audio: > > + > > +************* > > +Audio Formats > > +************* > > + > > +These formats are used for :ref:`audiomem2mem` interface only. > > Here you should also document that all these fourccs start with 'AU' and are > reserved for mappings of the snd_pcm_format_t type. > > Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc > define (see also below). How about below description? ' All these fourccs starting with 'AU' are reserved for mappings of the snd_pcm_format_t type. The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t type to fourcc. The first character is 'A', the second character is 'U', the third character is ten's digit of snd_pcm_format_t, the fourth character is unit's digit of snd_pcm_format_t. The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to snd_pcm_format_t type. ' Best regards Shengjiu Wang > > > + > > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: Audio Format > > + :header-rows: 1 > > + :stub-columns: 0 > > + :widths: 3 1 4 > > + > > + * - Identifier > > + - Code > > + - Details > > + * .. _V4L2-AUDIO-FMT-S8: > > + > > + - ``V4L2_AUDIO_FMT_S8`` > > + - 'S8' > > + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA > > + * .. _V4L2-AUDIO-FMT-S16-LE: > > + > > + - ``V4L2_AUDIO_FMT_S16_LE`` > > + - 'S16_LE' > > + - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA > > + * .. _V4L2-AUDIO-FMT-U16-LE: > > + > > + - ``V4L2_AUDIO_FMT_U16_LE`` > > + - 'U16_LE' > > + - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA > > + * .. _V4L2-AUDIO-FMT-S24-LE: > > + > > + - ``V4L2_AUDIO_FMT_S24_LE`` > > + - 'S24_LE' > > + - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA > > + * .. _V4L2-AUDIO-FMT-U24-LE: > > + > > + - ``V4L2_AUDIO_FMT_U24_LE`` > > + - 'U24_LE' > > + - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA > > + * .. _V4L2-AUDIO-FMT-S32-LE: > > + > > + - ``V4L2_AUDIO_FMT_S32_LE`` > > + - 'S32_LE' > > + - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA > > + * .. _V4L2-AUDIO-FMT-U32-LE: > > + > > + - ``V4L2_AUDIO_FMT_U32_LE`` > > + - 'U32_LE' > > + - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA > > + * .. _V4L2-AUDIO-FMT-FLOAT-LE: > > + > > + - ``V4L2_AUDIO_FMT_FLOAT_LE`` > > + - 'FLOAT_LE' > > + - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA > > + * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE: > > + > > + - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE`` > > + - 'IEC958_SUBFRAME_LE' > > + - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA > > + * .. _V4L2-AUDIO-FMT-S24-3LE: > > + > > + - ``V4L2_AUDIO_FMT_S24_3LE`` > > + - 'S24_3LE' > > + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA > > + * .. _V4L2-AUDIO-FMT-U24-3LE: > > + > > + - ``V4L2_AUDIO_FMT_U24_3LE`` > > + - 'U24_3LE' > > + - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA > > + * .. _V4L2-AUDIO-FMT-S20-3LE: > > + > > + - ``V4L2_AUDIO_FMT_S20_3LE`` > > + - 'S20_3LE' > > + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA > > + * .. _V4L2-AUDIO-FMT-U20-3LE: > > + > > + - ``V4L2_AUDIO_FMT_U20_3LE`` > > + - 'U20_3LE' > > + - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst > > index 11dab4a90630..2eb6fdd3b43d 100644 > > --- a/Documentation/userspace-api/media/v4l/pixfmt.rst > > +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst > > @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.) > > colorspaces > > colorspaces-defs > > colorspaces-details > > + pixfmt-audio > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c > > index 961abcdf7290..be229c69e991 100644 > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > > @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) > > case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; > > case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; > > case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; > > + case V4L2_AUDIO_FMT_S8: descr = "8-bit Signed"; break; > > + case V4L2_AUDIO_FMT_S16_LE: descr = "16-bit Signed LE"; break; > > + case V4L2_AUDIO_FMT_U16_LE: descr = "16-bit Unsigned LE"; break; > > + case V4L2_AUDIO_FMT_S24_LE: descr = "24(32)-bit Signed LE"; break; > > + case V4L2_AUDIO_FMT_U24_LE: descr = "24(32)-bit Unsigned LE"; break; > > + case V4L2_AUDIO_FMT_S32_LE: descr = "32-bit Signed LE"; break; > > + case V4L2_AUDIO_FMT_U32_LE: descr = "32-bit Unsigned LE"; break; > > + case V4L2_AUDIO_FMT_FLOAT_LE: descr = "32-bit Float LE"; break; > > + case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break; > > + case V4L2_AUDIO_FMT_S24_3LE: descr = "24(24)-bit Signed LE"; break; > > + case V4L2_AUDIO_FMT_U24_3LE: descr = "24(24)-bit Unsigned LE"; break; > > + case V4L2_AUDIO_FMT_S20_3LE: descr = "20(24)-bit Signed LE"; break; > > + case V4L2_AUDIO_FMT_U20_3LE: descr = "20(24)-bit Unsigned LE"; break; > > > > default: > > /* Compressed formats */ > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > > index 2c03d2dfadbe..673a6235a029 100644 > > --- a/include/uapi/linux/videodev2.h > > +++ b/include/uapi/linux/videodev2.h > > @@ -843,6 +843,29 @@ struct v4l2_pix_format { > > #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ > > #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ > > > > +/* > > + * Audio-data formats > > + * All these audio formats use a fourcc starting with 'AU' > > + * followed by the SNDRV_PCM_FORMAT_ value from asound.h. > > Also document here that fourccs starting with 'AU' are reserved for > the snd_pcm_format_t to fourcc mappings. > > That to avoid that someone adds a 'AUXX' fourcc later. > > > + */ > > +#define V4L2_AUDIO_FMT_S8 v4l2_fourcc('A', 'U', '0', '0') > > +#define V4L2_AUDIO_FMT_S16_LE v4l2_fourcc('A', 'U', '0', '2') > > +#define V4L2_AUDIO_FMT_U16_LE v4l2_fourcc('A', 'U', '0', '4') > > +#define V4L2_AUDIO_FMT_S24_LE v4l2_fourcc('A', 'U', '0', '6') > > +#define V4L2_AUDIO_FMT_U24_LE v4l2_fourcc('A', 'U', '0', '8') > > +#define V4L2_AUDIO_FMT_S32_LE v4l2_fourcc('A', 'U', '1', '0') > > +#define V4L2_AUDIO_FMT_U32_LE v4l2_fourcc('A', 'U', '1', '2') > > +#define V4L2_AUDIO_FMT_FLOAT_LE v4l2_fourcc('A', 'U', '1', '4') > > +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE v4l2_fourcc('A', 'U', '1', '8') > > +#define V4L2_AUDIO_FMT_S24_3LE v4l2_fourcc('A', 'U', '3', '2') > > +#define V4L2_AUDIO_FMT_U24_3LE v4l2_fourcc('A', 'U', '3', '4') > > +#define V4L2_AUDIO_FMT_S20_3LE v4l2_fourcc('A', 'U', '3', '6') > > +#define V4L2_AUDIO_FMT_U20_3LE v4l2_fourcc('A', 'U', '3', '8') > > + > > +#define v4l2_fourcc_to_audfmt(fourcc) \ > > + (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10 \ > > + + ((((fourcc) >> 24) & 0xff) - '0')) > > + > > As I suggested in an earlier reply, add this: > > #define v4l2_audfmt_to_fourcc(audfmt) \ > v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10) > > Even though it is not used in the drivers, since this is a public header used > by drivers and applications, it makes sense to provide the reverse mapping as well. > > Please test it in actual code to make sure there are no compilation warnings. > > Regards, > > Hans > > > /* priv field value to indicates that subsequent fields are valid. */ > > #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe > > >
On 08/03/2024 12:52 pm, Shengjiu Wang wrote: > On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil <hverkuil@xs4all.nl> wrote: >> >> Hi Shengjiu, >> >> After thinking it over I think this patch can be improved: >> >> On 26/02/2024 9:28 am, Shengjiu Wang wrote: >>> The audio sample format definition is from alsa, >>> the header file is include/uapi/sound/asound.h, but >>> don't include this header file directly, because in >>> user space, there is another copy in alsa-lib. >>> There will be conflict in userspace for include >>> videodev2.h & asound.h and asoundlib.h >>> >>> Here still use the fourcc format. >>> >>> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> >>> --- >>> .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ >>> .../userspace-api/media/v4l/pixfmt.rst | 1 + >>> drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ >>> include/uapi/linux/videodev2.h | 23 +++++ >>> 4 files changed, 124 insertions(+) >>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>> >>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>> new file mode 100644 >>> index 000000000000..04b4a7fbd8f4 >>> --- /dev/null >>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>> @@ -0,0 +1,87 @@ >>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later >>> + >>> +.. _pixfmt-audio: >>> + >>> +************* >>> +Audio Formats >>> +************* >>> + >>> +These formats are used for :ref:`audiomem2mem` interface only. >> >> Here you should also document that all these fourccs start with 'AU' and are >> reserved for mappings of the snd_pcm_format_t type. >> >> Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc >> define (see also below). > > How about below description? > ' > All these fourccs starting with 'AU' are reserved for mappings All these fourccs -> All FourCCs > of the snd_pcm_format_t type. > > The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t convert -> convert the > type to fourcc. The first character is 'A', the second character fourcc -> a FourCC > is 'U', the third character is ten's digit of snd_pcm_format_t, > the fourth character is unit's digit of snd_pcm_format_t. "is 'U', and the remaining two characters is the snd_pcm_format_t value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2) maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps to 'AU32'." > > The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to fourccs -> FourCCs > snd_pcm_format_t type. BTW, given the way snd_pcm_format_t is defined I am fairly certain some casts are needed for the v4l2_audfmt_to_fourcc define. Regards, Hans > ' > Best regards > Shengjiu Wang >> >>> + >>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| >>> + >>> +.. cssclass:: longtable >>> + >>> +.. flat-table:: Audio Format >>> + :header-rows: 1 >>> + :stub-columns: 0 >>> + :widths: 3 1 4 >>> + >>> + * - Identifier >>> + - Code >>> + - Details >>> + * .. _V4L2-AUDIO-FMT-S8: >>> + >>> + - ``V4L2_AUDIO_FMT_S8`` >>> + - 'S8' >>> + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA >>> + * .. _V4L2-AUDIO-FMT-S16-LE: >>> + >>> + - ``V4L2_AUDIO_FMT_S16_LE`` >>> + - 'S16_LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-U16-LE: >>> + >>> + - ``V4L2_AUDIO_FMT_U16_LE`` >>> + - 'U16_LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-S24-LE: >>> + >>> + - ``V4L2_AUDIO_FMT_S24_LE`` >>> + - 'S24_LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-U24-LE: >>> + >>> + - ``V4L2_AUDIO_FMT_U24_LE`` >>> + - 'U24_LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-S32-LE: >>> + >>> + - ``V4L2_AUDIO_FMT_S32_LE`` >>> + - 'S32_LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-U32-LE: >>> + >>> + - ``V4L2_AUDIO_FMT_U32_LE`` >>> + - 'U32_LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-FLOAT-LE: >>> + >>> + - ``V4L2_AUDIO_FMT_FLOAT_LE`` >>> + - 'FLOAT_LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE: >>> + >>> + - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE`` >>> + - 'IEC958_SUBFRAME_LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-S24-3LE: >>> + >>> + - ``V4L2_AUDIO_FMT_S24_3LE`` >>> + - 'S24_3LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-U24-3LE: >>> + >>> + - ``V4L2_AUDIO_FMT_U24_3LE`` >>> + - 'U24_3LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-S20-3LE: >>> + >>> + - ``V4L2_AUDIO_FMT_S20_3LE`` >>> + - 'S20_3LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA >>> + * .. _V4L2-AUDIO-FMT-U20-3LE: >>> + >>> + - ``V4L2_AUDIO_FMT_U20_3LE`` >>> + - 'U20_3LE' >>> + - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA >>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst >>> index 11dab4a90630..2eb6fdd3b43d 100644 >>> --- a/Documentation/userspace-api/media/v4l/pixfmt.rst >>> +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst >>> @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.) >>> colorspaces >>> colorspaces-defs >>> colorspaces-details >>> + pixfmt-audio >>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c >>> index 961abcdf7290..be229c69e991 100644 >>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c >>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c >>> @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) >>> case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; >>> case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; >>> case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; >>> + case V4L2_AUDIO_FMT_S8: descr = "8-bit Signed"; break; >>> + case V4L2_AUDIO_FMT_S16_LE: descr = "16-bit Signed LE"; break; >>> + case V4L2_AUDIO_FMT_U16_LE: descr = "16-bit Unsigned LE"; break; >>> + case V4L2_AUDIO_FMT_S24_LE: descr = "24(32)-bit Signed LE"; break; >>> + case V4L2_AUDIO_FMT_U24_LE: descr = "24(32)-bit Unsigned LE"; break; >>> + case V4L2_AUDIO_FMT_S32_LE: descr = "32-bit Signed LE"; break; >>> + case V4L2_AUDIO_FMT_U32_LE: descr = "32-bit Unsigned LE"; break; >>> + case V4L2_AUDIO_FMT_FLOAT_LE: descr = "32-bit Float LE"; break; >>> + case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break; >>> + case V4L2_AUDIO_FMT_S24_3LE: descr = "24(24)-bit Signed LE"; break; >>> + case V4L2_AUDIO_FMT_U24_3LE: descr = "24(24)-bit Unsigned LE"; break; >>> + case V4L2_AUDIO_FMT_S20_3LE: descr = "20(24)-bit Signed LE"; break; >>> + case V4L2_AUDIO_FMT_U20_3LE: descr = "20(24)-bit Unsigned LE"; break; >>> >>> default: >>> /* Compressed formats */ >>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h >>> index 2c03d2dfadbe..673a6235a029 100644 >>> --- a/include/uapi/linux/videodev2.h >>> +++ b/include/uapi/linux/videodev2.h >>> @@ -843,6 +843,29 @@ struct v4l2_pix_format { >>> #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ >>> #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ >>> >>> +/* >>> + * Audio-data formats >>> + * All these audio formats use a fourcc starting with 'AU' >>> + * followed by the SNDRV_PCM_FORMAT_ value from asound.h. >> >> Also document here that fourccs starting with 'AU' are reserved for >> the snd_pcm_format_t to fourcc mappings. >> >> That to avoid that someone adds a 'AUXX' fourcc later. >> >>> + */ >>> +#define V4L2_AUDIO_FMT_S8 v4l2_fourcc('A', 'U', '0', '0') >>> +#define V4L2_AUDIO_FMT_S16_LE v4l2_fourcc('A', 'U', '0', '2') >>> +#define V4L2_AUDIO_FMT_U16_LE v4l2_fourcc('A', 'U', '0', '4') >>> +#define V4L2_AUDIO_FMT_S24_LE v4l2_fourcc('A', 'U', '0', '6') >>> +#define V4L2_AUDIO_FMT_U24_LE v4l2_fourcc('A', 'U', '0', '8') >>> +#define V4L2_AUDIO_FMT_S32_LE v4l2_fourcc('A', 'U', '1', '0') >>> +#define V4L2_AUDIO_FMT_U32_LE v4l2_fourcc('A', 'U', '1', '2') >>> +#define V4L2_AUDIO_FMT_FLOAT_LE v4l2_fourcc('A', 'U', '1', '4') >>> +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE v4l2_fourcc('A', 'U', '1', '8') >>> +#define V4L2_AUDIO_FMT_S24_3LE v4l2_fourcc('A', 'U', '3', '2') >>> +#define V4L2_AUDIO_FMT_U24_3LE v4l2_fourcc('A', 'U', '3', '4') >>> +#define V4L2_AUDIO_FMT_S20_3LE v4l2_fourcc('A', 'U', '3', '6') >>> +#define V4L2_AUDIO_FMT_U20_3LE v4l2_fourcc('A', 'U', '3', '8') >>> + >>> +#define v4l2_fourcc_to_audfmt(fourcc) \ >>> + (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10 \ >>> + + ((((fourcc) >> 24) & 0xff) - '0')) >>> + >> >> As I suggested in an earlier reply, add this: >> >> #define v4l2_audfmt_to_fourcc(audfmt) \ >> v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10) >> >> Even though it is not used in the drivers, since this is a public header used >> by drivers and applications, it makes sense to provide the reverse mapping as well. >> >> Please test it in actual code to make sure there are no compilation warnings. >> >> Regards, >> >> Hans >> >>> /* priv field value to indicates that subsequent fields are valid. */ >>> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe >>> >>
On Fri, Mar 8, 2024 at 8:06 PM Hans Verkuil <hverkuil@xs4all.nl> wrote: > > On 08/03/2024 12:52 pm, Shengjiu Wang wrote: > > On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil <hverkuil@xs4all.nl> wrote: > >> > >> Hi Shengjiu, > >> > >> After thinking it over I think this patch can be improved: > >> > >> On 26/02/2024 9:28 am, Shengjiu Wang wrote: > >>> The audio sample format definition is from alsa, > >>> the header file is include/uapi/sound/asound.h, but > >>> don't include this header file directly, because in > >>> user space, there is another copy in alsa-lib. > >>> There will be conflict in userspace for include > >>> videodev2.h & asound.h and asoundlib.h > >>> > >>> Here still use the fourcc format. > >>> > >>> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> > >>> --- > >>> .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ > >>> .../userspace-api/media/v4l/pixfmt.rst | 1 + > >>> drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ > >>> include/uapi/linux/videodev2.h | 23 +++++ > >>> 4 files changed, 124 insertions(+) > >>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst > >>> > >>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > >>> new file mode 100644 > >>> index 000000000000..04b4a7fbd8f4 > >>> --- /dev/null > >>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst > >>> @@ -0,0 +1,87 @@ > >>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later > >>> + > >>> +.. _pixfmt-audio: > >>> + > >>> +************* > >>> +Audio Formats > >>> +************* > >>> + > >>> +These formats are used for :ref:`audiomem2mem` interface only. > >> > >> Here you should also document that all these fourccs start with 'AU' and are > >> reserved for mappings of the snd_pcm_format_t type. > >> > >> Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc > >> define (see also below). > > > > How about below description? > > ' > > All these fourccs starting with 'AU' are reserved for mappings > > All these fourccs -> All FourCCs > > > of the snd_pcm_format_t type. > > > > The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t > > convert -> convert the > > > type to fourcc. The first character is 'A', the second character > > fourcc -> a FourCC > > > is 'U', the third character is ten's digit of snd_pcm_format_t, > > the fourth character is unit's digit of snd_pcm_format_t. > > "is 'U', and the remaining two characters is the snd_pcm_format_t > value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2) > maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps > to 'AU32'." > > > > > The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to > > fourccs -> FourCCs > > > snd_pcm_format_t type. > > BTW, given the way snd_pcm_format_t is defined I am fairly certain > some casts are needed for the v4l2_audfmt_to_fourcc define. > > Regards, > > Hans > > > ' > > Best regards > > Shengjiu Wang > >> > >>> + > >>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| > >>> + > >>> +.. cssclass:: longtable > >>> + > >>> +.. flat-table:: Audio Format > >>> + :header-rows: 1 > >>> + :stub-columns: 0 > >>> + :widths: 3 1 4 > >>> + > >>> + * - Identifier > >>> + - Code > >>> + - Details > >>> + * .. _V4L2-AUDIO-FMT-S8: > >>> + > >>> + - ``V4L2_AUDIO_FMT_S8`` > >>> + - 'S8' > >>> + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA > >>> + * .. _V4L2-AUDIO-FMT-S16-LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_S16_LE`` > >>> + - 'S16_LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-U16-LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_U16_LE`` > >>> + - 'U16_LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-S24-LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_S24_LE`` > >>> + - 'S24_LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-U24-LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_U24_LE`` > >>> + - 'U24_LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-S32-LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_S32_LE`` > >>> + - 'S32_LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-U32-LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_U32_LE`` > >>> + - 'U32_LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-FLOAT-LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_FLOAT_LE`` > >>> + - 'FLOAT_LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE`` > >>> + - 'IEC958_SUBFRAME_LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-S24-3LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_S24_3LE`` > >>> + - 'S24_3LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-U24-3LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_U24_3LE`` > >>> + - 'U24_3LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-S20-3LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_S20_3LE`` > >>> + - 'S20_3LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA > >>> + * .. _V4L2-AUDIO-FMT-U20-3LE: > >>> + > >>> + - ``V4L2_AUDIO_FMT_U20_3LE`` > >>> + - 'U20_3LE' > >>> + - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA > >>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst > >>> index 11dab4a90630..2eb6fdd3b43d 100644 > >>> --- a/Documentation/userspace-api/media/v4l/pixfmt.rst > >>> +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst > >>> @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.) > >>> colorspaces > >>> colorspaces-defs > >>> colorspaces-details > >>> + pixfmt-audio > >>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c > >>> index 961abcdf7290..be229c69e991 100644 > >>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c > >>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > >>> @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) > >>> case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; > >>> case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; > >>> case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; > >>> + case V4L2_AUDIO_FMT_S8: descr = "8-bit Signed"; break; > >>> + case V4L2_AUDIO_FMT_S16_LE: descr = "16-bit Signed LE"; break; > >>> + case V4L2_AUDIO_FMT_U16_LE: descr = "16-bit Unsigned LE"; break; > >>> + case V4L2_AUDIO_FMT_S24_LE: descr = "24(32)-bit Signed LE"; break; > >>> + case V4L2_AUDIO_FMT_U24_LE: descr = "24(32)-bit Unsigned LE"; break; > >>> + case V4L2_AUDIO_FMT_S32_LE: descr = "32-bit Signed LE"; break; > >>> + case V4L2_AUDIO_FMT_U32_LE: descr = "32-bit Unsigned LE"; break; > >>> + case V4L2_AUDIO_FMT_FLOAT_LE: descr = "32-bit Float LE"; break; > >>> + case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break; > >>> + case V4L2_AUDIO_FMT_S24_3LE: descr = "24(24)-bit Signed LE"; break; > >>> + case V4L2_AUDIO_FMT_U24_3LE: descr = "24(24)-bit Unsigned LE"; break; > >>> + case V4L2_AUDIO_FMT_S20_3LE: descr = "20(24)-bit Signed LE"; break; > >>> + case V4L2_AUDIO_FMT_U20_3LE: descr = "20(24)-bit Unsigned LE"; break; > >>> > >>> default: > >>> /* Compressed formats */ > >>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > >>> index 2c03d2dfadbe..673a6235a029 100644 > >>> --- a/include/uapi/linux/videodev2.h > >>> +++ b/include/uapi/linux/videodev2.h > >>> @@ -843,6 +843,29 @@ struct v4l2_pix_format { > >>> #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ > >>> #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ > >>> > >>> +/* > >>> + * Audio-data formats > >>> + * All these audio formats use a fourcc starting with 'AU' > >>> + * followed by the SNDRV_PCM_FORMAT_ value from asound.h. > >> > >> Also document here that fourccs starting with 'AU' are reserved for > >> the snd_pcm_format_t to fourcc mappings. > >> > >> That to avoid that someone adds a 'AUXX' fourcc later. > >> > >>> + */ > >>> +#define V4L2_AUDIO_FMT_S8 v4l2_fourcc('A', 'U', '0', '0') > >>> +#define V4L2_AUDIO_FMT_S16_LE v4l2_fourcc('A', 'U', '0', '2') > >>> +#define V4L2_AUDIO_FMT_U16_LE v4l2_fourcc('A', 'U', '0', '4') > >>> +#define V4L2_AUDIO_FMT_S24_LE v4l2_fourcc('A', 'U', '0', '6') > >>> +#define V4L2_AUDIO_FMT_U24_LE v4l2_fourcc('A', 'U', '0', '8') > >>> +#define V4L2_AUDIO_FMT_S32_LE v4l2_fourcc('A', 'U', '1', '0') > >>> +#define V4L2_AUDIO_FMT_U32_LE v4l2_fourcc('A', 'U', '1', '2') > >>> +#define V4L2_AUDIO_FMT_FLOAT_LE v4l2_fourcc('A', 'U', '1', '4') > >>> +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE v4l2_fourcc('A', 'U', '1', '8') > >>> +#define V4L2_AUDIO_FMT_S24_3LE v4l2_fourcc('A', 'U', '3', '2') > >>> +#define V4L2_AUDIO_FMT_U24_3LE v4l2_fourcc('A', 'U', '3', '4') > >>> +#define V4L2_AUDIO_FMT_S20_3LE v4l2_fourcc('A', 'U', '3', '6') > >>> +#define V4L2_AUDIO_FMT_U20_3LE v4l2_fourcc('A', 'U', '3', '8') > >>> + > >>> +#define v4l2_fourcc_to_audfmt(fourcc) \ > >>> + (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10 \ > >>> + + ((((fourcc) >> 24) & 0xff) - '0')) > >>> + > >> > >> As I suggested in an earlier reply, add this: > >> > >> #define v4l2_audfmt_to_fourcc(audfmt) \ > >> v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10) > >> > >> Even though it is not used in the drivers, since this is a public header used > >> by drivers and applications, it makes sense to provide the reverse mapping as well. > >> > >> Please test it in actual code to make sure there are no compilation warnings. I test this definition, the compiler doesn't report warning. best regards Shengjiu Wang > >> > >> Regards, > >> > >> Hans > >> > >>> /* priv field value to indicates that subsequent fields are valid. */ > >>> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe > >>> > >> >
On 08/03/2024 2:52 pm, Shengjiu Wang wrote: > On Fri, Mar 8, 2024 at 8:06 PM Hans Verkuil <hverkuil@xs4all.nl> wrote: >> >> On 08/03/2024 12:52 pm, Shengjiu Wang wrote: >>> On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil <hverkuil@xs4all.nl> wrote: >>>> >>>> Hi Shengjiu, >>>> >>>> After thinking it over I think this patch can be improved: >>>> >>>> On 26/02/2024 9:28 am, Shengjiu Wang wrote: >>>>> The audio sample format definition is from alsa, >>>>> the header file is include/uapi/sound/asound.h, but >>>>> don't include this header file directly, because in >>>>> user space, there is another copy in alsa-lib. >>>>> There will be conflict in userspace for include >>>>> videodev2.h & asound.h and asoundlib.h >>>>> >>>>> Here still use the fourcc format. >>>>> >>>>> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> >>>>> --- >>>>> .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++++++++++++++++++ >>>>> .../userspace-api/media/v4l/pixfmt.rst | 1 + >>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++ >>>>> include/uapi/linux/videodev2.h | 23 +++++ >>>>> 4 files changed, 124 insertions(+) >>>>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>>>> >>>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>>>> new file mode 100644 >>>>> index 000000000000..04b4a7fbd8f4 >>>>> --- /dev/null >>>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst >>>>> @@ -0,0 +1,87 @@ >>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later >>>>> + >>>>> +.. _pixfmt-audio: >>>>> + >>>>> +************* >>>>> +Audio Formats >>>>> +************* >>>>> + >>>>> +These formats are used for :ref:`audiomem2mem` interface only. >>>> >>>> Here you should also document that all these fourccs start with 'AU' and are >>>> reserved for mappings of the snd_pcm_format_t type. >>>> >>>> Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc >>>> define (see also below). >>> >>> How about below description? >>> ' >>> All these fourccs starting with 'AU' are reserved for mappings >> >> All these fourccs -> All FourCCs >> >>> of the snd_pcm_format_t type. >>> >>> The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t >> >> convert -> convert the >> >>> type to fourcc. The first character is 'A', the second character >> >> fourcc -> a FourCC >> >>> is 'U', the third character is ten's digit of snd_pcm_format_t, >>> the fourth character is unit's digit of snd_pcm_format_t. >> >> "is 'U', and the remaining two characters is the snd_pcm_format_t >> value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2) >> maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps >> to 'AU32'." >> >>> >>> The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to >> >> fourccs -> FourCCs >> >>> snd_pcm_format_t type. >> >> BTW, given the way snd_pcm_format_t is defined I am fairly certain >> some casts are needed for the v4l2_audfmt_to_fourcc define. >> >> Regards, >> >> Hans >> >>> ' >>> Best regards >>> Shengjiu Wang >>>> >>>>> + >>>>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}| >>>>> + >>>>> +.. cssclass:: longtable >>>>> + >>>>> +.. flat-table:: Audio Format >>>>> + :header-rows: 1 >>>>> + :stub-columns: 0 >>>>> + :widths: 3 1 4 >>>>> + >>>>> + * - Identifier >>>>> + - Code >>>>> + - Details >>>>> + * .. _V4L2-AUDIO-FMT-S8: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_S8`` >>>>> + - 'S8' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-S16-LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_S16_LE`` >>>>> + - 'S16_LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-U16-LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_U16_LE`` >>>>> + - 'U16_LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-S24-LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_S24_LE`` >>>>> + - 'S24_LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-U24-LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_U24_LE`` >>>>> + - 'U24_LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-S32-LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_S32_LE`` >>>>> + - 'S32_LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-U32-LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_U32_LE`` >>>>> + - 'U32_LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-FLOAT-LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_FLOAT_LE`` >>>>> + - 'FLOAT_LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE`` >>>>> + - 'IEC958_SUBFRAME_LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-S24-3LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_S24_3LE`` >>>>> + - 'S24_3LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-U24-3LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_U24_3LE`` >>>>> + - 'U24_3LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-S20-3LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_S20_3LE`` >>>>> + - 'S20_3LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA >>>>> + * .. _V4L2-AUDIO-FMT-U20-3LE: >>>>> + >>>>> + - ``V4L2_AUDIO_FMT_U20_3LE`` >>>>> + - 'U20_3LE' >>>>> + - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA >>>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst >>>>> index 11dab4a90630..2eb6fdd3b43d 100644 >>>>> --- a/Documentation/userspace-api/media/v4l/pixfmt.rst >>>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst >>>>> @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.) >>>>> colorspaces >>>>> colorspaces-defs >>>>> colorspaces-details >>>>> + pixfmt-audio >>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c >>>>> index 961abcdf7290..be229c69e991 100644 >>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c >>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c >>>>> @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) >>>>> case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; >>>>> case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; >>>>> case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; >>>>> + case V4L2_AUDIO_FMT_S8: descr = "8-bit Signed"; break; >>>>> + case V4L2_AUDIO_FMT_S16_LE: descr = "16-bit Signed LE"; break; >>>>> + case V4L2_AUDIO_FMT_U16_LE: descr = "16-bit Unsigned LE"; break; >>>>> + case V4L2_AUDIO_FMT_S24_LE: descr = "24(32)-bit Signed LE"; break; >>>>> + case V4L2_AUDIO_FMT_U24_LE: descr = "24(32)-bit Unsigned LE"; break; >>>>> + case V4L2_AUDIO_FMT_S32_LE: descr = "32-bit Signed LE"; break; >>>>> + case V4L2_AUDIO_FMT_U32_LE: descr = "32-bit Unsigned LE"; break; >>>>> + case V4L2_AUDIO_FMT_FLOAT_LE: descr = "32-bit Float LE"; break; >>>>> + case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break; >>>>> + case V4L2_AUDIO_FMT_S24_3LE: descr = "24(24)-bit Signed LE"; break; >>>>> + case V4L2_AUDIO_FMT_U24_3LE: descr = "24(24)-bit Unsigned LE"; break; >>>>> + case V4L2_AUDIO_FMT_S20_3LE: descr = "20(24)-bit Signed LE"; break; >>>>> + case V4L2_AUDIO_FMT_U20_3LE: descr = "20(24)-bit Unsigned LE"; break; >>>>> >>>>> default: >>>>> /* Compressed formats */ >>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h >>>>> index 2c03d2dfadbe..673a6235a029 100644 >>>>> --- a/include/uapi/linux/videodev2.h >>>>> +++ b/include/uapi/linux/videodev2.h >>>>> @@ -843,6 +843,29 @@ struct v4l2_pix_format { >>>>> #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ >>>>> #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ >>>>> >>>>> +/* >>>>> + * Audio-data formats >>>>> + * All these audio formats use a fourcc starting with 'AU' >>>>> + * followed by the SNDRV_PCM_FORMAT_ value from asound.h. >>>> >>>> Also document here that fourccs starting with 'AU' are reserved for >>>> the snd_pcm_format_t to fourcc mappings. >>>> >>>> That to avoid that someone adds a 'AUXX' fourcc later. >>>> >>>>> + */ >>>>> +#define V4L2_AUDIO_FMT_S8 v4l2_fourcc('A', 'U', '0', '0') >>>>> +#define V4L2_AUDIO_FMT_S16_LE v4l2_fourcc('A', 'U', '0', '2') >>>>> +#define V4L2_AUDIO_FMT_U16_LE v4l2_fourcc('A', 'U', '0', '4') >>>>> +#define V4L2_AUDIO_FMT_S24_LE v4l2_fourcc('A', 'U', '0', '6') >>>>> +#define V4L2_AUDIO_FMT_U24_LE v4l2_fourcc('A', 'U', '0', '8') >>>>> +#define V4L2_AUDIO_FMT_S32_LE v4l2_fourcc('A', 'U', '1', '0') >>>>> +#define V4L2_AUDIO_FMT_U32_LE v4l2_fourcc('A', 'U', '1', '2') >>>>> +#define V4L2_AUDIO_FMT_FLOAT_LE v4l2_fourcc('A', 'U', '1', '4') >>>>> +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE v4l2_fourcc('A', 'U', '1', '8') >>>>> +#define V4L2_AUDIO_FMT_S24_3LE v4l2_fourcc('A', 'U', '3', '2') >>>>> +#define V4L2_AUDIO_FMT_U24_3LE v4l2_fourcc('A', 'U', '3', '4') >>>>> +#define V4L2_AUDIO_FMT_S20_3LE v4l2_fourcc('A', 'U', '3', '6') >>>>> +#define V4L2_AUDIO_FMT_U20_3LE v4l2_fourcc('A', 'U', '3', '8') >>>>> + >>>>> +#define v4l2_fourcc_to_audfmt(fourcc) \ >>>>> + (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10 \ >>>>> + + ((((fourcc) >> 24) & 0xff) - '0')) >>>>> + >>>> >>>> As I suggested in an earlier reply, add this: >>>> >>>> #define v4l2_audfmt_to_fourcc(audfmt) \ >>>> v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10) >>>> >>>> Even though it is not used in the drivers, since this is a public header used >>>> by drivers and applications, it makes sense to provide the reverse mapping as well. >>>> >>>> Please test it in actual code to make sure there are no compilation warnings. > > I test this definition, the compiler doesn't report warning. typedef int __bitwise snd_pcm_format_t; And __bitwise is apparently a sparse static analyzer attribute, so I suspect that the v4l2_audfmt_to_fourcc definition will cause a sparse warning. So you need to check with sparse. Regards, Hans > > best regards > Shengjiu Wang > >>>> >>>> Regards, >>>> >>>> Hans >>>> >>>>> /* priv field value to indicates that subsequent fields are valid. */ >>>>> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe >>>>> >>>> >>
Audio signal processing also has the requirement for memory to memory similar as Video. This asrc memory to memory (memory ->asrc->memory) case is a non real time use case. User fills the input buffer to the asrc module, after conversion, then asrc sends back the output buffer to user. So it is not a traditional ALSA playback and capture case. It is a specific use case, there is no reference in current kernel. v4l2 memory to memory is the closed implementation, v4l2 current support video, image, radio, tuner, touch devices, so it is not complicated to add support for this specific audio case. Because we had implemented the "memory -> asrc ->i2s device-> codec" use case in ALSA. Now the "memory->asrc->memory" needs to reuse the code in asrc driver, so the first 3 patches is for refining the code to make it can be shared by the "memory->asrc->memory" driver. The main change is in the v4l2 side, A /dev/vl4-audioX will be created, user applications only use the ioctl of v4l2 framework. Other change is to add memory to memory support for two kinds of i.MX ASRC module. changes in v13 - change 'pixelformat' to 'audioformat' in dev-audio-mem2mem.rst - add more description for clock drift in ext-ctrls-audio-m2m.rst - Add "media: v4l2-ctrls: add support for fraction_bits" from Hans to avoid build issue for kernel test robot changes in v12 - minor changes according to comments - drop min_buffers_needed = 1 and V4L2_CTRL_FLAG_UPDATE flag - drop bus_info changes in v11 - add add-fixed-point-test-controls in vivid. - add v4l2_ctrl_fp_compose() helper function for min and max changes in v10 - remove FIXED_POINT type - change code base on media: v4l2-ctrls: add support for fraction_bits - fix issue reported by kernel test robot - remove module_alias changes in v9: - add MEDIA_ENT_F_PROC_AUDIO_RESAMPLER. - add MEDIA_INTF_T_V4L_AUDIO - add media controller support - refine the vim2m-audio to support 8k<->16k conversion. changes in v8: - refine V4L2_CAP_AUDIO_M2M to be 0x00000008 - update doc for FIXED_POINT - address comments for imx-asrc changes in v7: - add acked-by from Mark - separate commit for fixed point, m2m audio class, audio rate controls - use INTEGER_MENU for rate, FIXED_POINT for rate offset - remove used fmts - address other comments for Hans changes in v6: - use m2m_prepare/m2m_unprepare/m2m_start/m2m_stop to replace m2m_start_part_one/m2m_stop_part_one, m2m_start_part_two/m2m_stop_part_two. - change V4L2_CTRL_TYPE_ASRC_RATE to V4L2_CTRL_TYPE_FIXED_POINT - fix warning by kernel test rebot - remove some unused format V4L2_AUDIO_FMT_XX - Get SNDRV_PCM_FORMAT from V4L2_AUDIO_FMT in driver. - rename audm2m to viaudm2m. changes in v5: - remove V4L2_AUDIO_FMT_LPCM - define audio pixel format like V4L2_AUDIO_FMT_S8... - remove rate and format in struct v4l2_audio_format. - Add V4L2_CID_ASRC_SOURCE_RATE and V4L2_CID_ASRC_DEST_RATE controls - updata document accordingly. changes in v4: - update document style - separate V4L2_AUDIO_FMT_LPCM and V4L2_CAP_AUDIO_M2M in separate commit changes in v3: - Modify documents for adding audio m2m support - Add audio virtual m2m driver - Defined V4L2_AUDIO_FMT_LPCM format type for audio. - Defined V4L2_CAP_AUDIO_M2M capability type for audio m2m case. - with modification in v4l-utils, pass v4l2-compliance test. changes in v2: - decouple the implementation in v4l2 and ALSA - implement the memory to memory driver as a platfrom driver and move it to driver/media - move fsl_asrc_common.h to include/sound folder Hans Verkuil (1): media: v4l2-ctrls: add support for fraction_bits Shengjiu Wang (15): ASoC: fsl_asrc: define functions for memory to memory usage ASoC: fsl_easrc: define functions for memory to memory usage ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound ASoC: fsl_asrc: register m2m platform device ASoC: fsl_easrc: register m2m platform device media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag media: v4l2: Add audio capture and output support media: uapi: Define audio sample format fourcc type media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO media: uapi: Add audio rate controls support media: uapi: Declare interface types for Audio media: uapi: Add an entity type for audio resampler media: vivid: add fixed point test controls media: imx-asrc: Add memory to memory driver media: vim2m-audio: add virtual driver for audio memory to memory .../media/mediactl/media-types.rst | 11 + .../userspace-api/media/v4l/buffer.rst | 6 + .../userspace-api/media/v4l/common.rst | 1 + .../media/v4l/dev-audio-mem2mem.rst | 71 + .../userspace-api/media/v4l/devices.rst | 1 + .../media/v4l/ext-ctrls-audio-m2m.rst | 59 + .../userspace-api/media/v4l/pixfmt-audio.rst | 87 ++ .../userspace-api/media/v4l/pixfmt.rst | 1 + .../media/v4l/vidioc-enum-fmt.rst | 2 + .../media/v4l/vidioc-g-ext-ctrls.rst | 4 + .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 + .../media/v4l/vidioc-querycap.rst | 3 + .../media/v4l/vidioc-queryctrl.rst | 11 +- .../media/videodev2.h.rst.exceptions | 3 + .../media/common/videobuf2/videobuf2-v4l2.c | 4 + drivers/media/platform/nxp/Kconfig | 13 + drivers/media/platform/nxp/Makefile | 1 + drivers/media/platform/nxp/imx-asrc.c | 1256 +++++++++++++++++ drivers/media/test-drivers/Kconfig | 10 + drivers/media/test-drivers/Makefile | 1 + drivers/media/test-drivers/vim2m-audio.c | 793 +++++++++++ drivers/media/test-drivers/vivid/vivid-core.h | 2 + .../media/test-drivers/vivid/vivid-ctrls.c | 26 + drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 + drivers/media/v4l2-core/v4l2-ctrls-api.c | 1 + drivers/media/v4l2-core/v4l2-ctrls-core.c | 93 +- drivers/media/v4l2-core/v4l2-ctrls-defs.c | 10 + drivers/media/v4l2-core/v4l2-dev.c | 21 + drivers/media/v4l2-core/v4l2-ioctl.c | 66 + drivers/media/v4l2-core/v4l2-mem2mem.c | 13 +- include/media/v4l2-ctrls.h | 13 +- include/media/v4l2-dev.h | 2 + include/media/v4l2-ioctl.h | 34 + .../fsl => include/sound}/fsl_asrc_common.h | 60 + include/uapi/linux/media.h | 2 + include/uapi/linux/v4l2-controls.h | 9 + include/uapi/linux/videodev2.h | 44 +- sound/soc/fsl/fsl_asrc.c | 144 ++ sound/soc/fsl/fsl_asrc.h | 4 +- sound/soc/fsl/fsl_asrc_dma.c | 2 +- sound/soc/fsl/fsl_easrc.c | 233 +++ sound/soc/fsl/fsl_easrc.h | 6 +- 42 files changed, 3109 insertions(+), 27 deletions(-) create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst create mode 100644 Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst create mode 100644 drivers/media/platform/nxp/imx-asrc.c create mode 100644 drivers/media/test-drivers/vim2m-audio.c rename {sound/soc/fsl => include/sound}/fsl_asrc_common.h (60%)