diff mbox series

[v2,14/17] media: atomisp: Fix atomisp_try_fmt_cap() always returning YUV420 pixelformat

Message ID 20221020195533.114049-15-hdegoede@redhat.com
State Accepted
Commit 57e1222e8f9b08058bfe487478c14c624d03fd00
Headers show
Series media: atomisp: Convert to videobuf2 | expand

Commit Message

Hans de Goede Oct. 20, 2022, 7:55 p.m. UTC 
The atomisp_try_fmt() call in atomisp_try_fmt_cap() replaces
the pixelformat passed by userspace with the sensors native pixelformat.

Which always gets replaced by V4L2_PIX_FMT_YUV420 by atomisp_adjust_fmt()
because raw sensor formats are not supported.

This needs to be fixed so that userspace which does a try_fmt call before
s_fmt does not end up always getting YUV420 even if it passed something
else into the try_fmt call.

To fix this restore the userspace requested pixelformat before
the atomisp_adjust_fmt() call. atomisp_adjust_fmt() will replace this
with V4L2_PIX_FMT_YUV420 in case an unsupported format is requested.

Note this relies on the "media: atomisp: Refactor atomisp_adjust_fmt()"
change, before that atomisp_adjust_fmt() would return -EINVAL for
unsupported pixelformats.

Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/staging/media/atomisp/pci/atomisp_ioctl.c | 7 +++++++
 1 file changed, 7 insertions(+)
diff mbox series

Patch

diff --git a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
index 5034a5509a3b..384b231de8bc 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
@@ -922,6 +922,7 @@  static int atomisp_try_fmt_cap(struct file *file, void *fh,
 			       struct v4l2_format *f)
 {
 	struct video_device *vdev = video_devdata(file);
+	u32 pixfmt = f->fmt.pix.pixelformat;
 	int ret;
 
 	/*
@@ -935,6 +936,12 @@  static int atomisp_try_fmt_cap(struct file *file, void *fh,
 	if (ret)
 		return ret;
 
+	/*
+	 * atomisp_try_fmt() replaces pixelformat with the sensors native
+	 * format, restore the actual format requested by userspace.
+	 */
+	f->fmt.pix.pixelformat = pixfmt;
+
 	return atomisp_adjust_fmt(f);
 }