v4l2 encoder ignores input_format specification

[leezu]

Summary of the bug: With a video capture device providing both compressed Motion-JPEG and uncompressed YUYV, ffmpeg always chooses YUYV even if MJPEG is requested.

How to reproduce:

% ~/Downloads/ffmpeg -v 9 -loglevel 99 -i /dev/video0 -input_format mjpeg
ffmpeg version N-107804-gaa9eabb7a5-https://www.martin-riedl.de Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)                                   
  configuration: --prefix=/home/ffmpegBuild/out --enable-gpl --pkg-config-flags=--static --extra-version='https://www.martin-riedl.de' --ena
ble-gray --enable-libxml2 --enable-libfreetype --enable-fontconfig --enable-libbluray --enable-libass --enable-libaom --enable-libopenh264 -
-enable-libsvtav1 --enable-libvpx --enable-libx264 --enable-libx265 --enable-libmp3lame --enable-libopus --enable-libvorbis                 
  libavutil      57. 33.101 / 57. 33.101                                                                                                    
  libavcodec     59. 42.101 / 59. 42.101                                                                                                    
  libavformat    59. 30.100 / 59. 30.100                                                                                                    
  libavdevice    59.  8.101 / 59.  8.101                                                                                                    
  libavfilter     8. 46.103 /  8. 46.103                                                                                                    
  libswscale      6.  8.102 /  6.  8.102                                                                                                    
  libswresample   4.  8.100 /  4.  8.100                                                                                                    
  libpostproc    56.  7.100 / 56.  7.100                                                                                                    
Splitting the commandline.                                            
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-i' ... matched as input url with argument '/dev/video0'.                                                                   
Reading option '-input_format' ... matched as AVOption 'input_format' with argument 'mjpeg'.
Trailing option(s) found in the command: may be ignored.                                                                                    
Finished splitting the commandline.                                                                                                         
Parsing a group of options: global .                                                                                                        
Applying option v (set logging level) with argument 9.                                                                                      
Successfully parsed a group of options.                                                                                                     
Parsing a group of options: input url /dev/video0.                                                                                          
Successfully parsed a group of options.                                                                                                     
Opening an input file: /dev/video0.                                                                                                         
Probing video4linux2,v4l2 score:99 size:0                                                                                                   
[video4linux2,v4l2 @ 0x5d5a711060] fd:3 capabilities:84a00001                                                                               
[video4linux2,v4l2 @ 0x5d5a711060] Current input_channel: 0, input_name: Camera 1, input_std: 0
[video4linux2,v4l2 @ 0x5d5a711060] Querying the device for the current frame size
[video4linux2,v4l2 @ 0x5d5a711060] Setting frame size to 1280x720                                                                           
[video4linux2,v4l2 @ 0x5d5a711060] The V4L2 driver changed the pixel format from 0x32315559 to 0x47504A4D
[video4linux2,v4l2 @ 0x5d5a711060] Trying to set codec:rawvideo pix_fmt:yuv420p                                                             
[video4linux2,v4l2 @ 0x5d5a711060] The V4L2 driver changed the pixel format from 0x32315559 to 0x47504A4D
[video4linux2,v4l2 @ 0x5d5a711060] Trying to set codec:rawvideo pix_fmt:yuv420p                                                             
[video4linux2,v4l2 @ 0x5d5a711060] The V4L2 driver changed the pixel format from 0x32315659 to 0x47504A4D
[video4linux2,v4l2 @ 0x5d5a711060] Trying to set codec:rawvideo pix_fmt:yuv422p                                                             
[video4linux2,v4l2 @ 0x5d5a711060] The V4L2 driver changed the pixel format from 0x50323234 to 0x47504A4D
[video4linux2,v4l2 @ 0x5d5a711060] Trying to set codec:rawvideo pix_fmt:yuyv422                                                             
[video4linux2,v4l2 @ 0x5d5a711060] All info found                                                                                           
[video4linux2,v4l2 @ 0x5d5a711060] stream 0: start_time: 639.743 duration: NOPTS                                                            
[video4linux2,v4l2 @ 0x5d5a711060] format: start_time: 639.743 duration: NOPTS (estimate from bit rate) bitrate=147456 kb/s
Input #0, video4linux2,v4l2, from '/dev/video0':                                                                                            
  Duration: N/A, start: 639.742977, bitrate: 147456 kb/s                                                                                    
  Stream #0:0, 1, 1/1000000: Video: rawvideo, 1 reference frame (YUY2 / 0x32595559), yuyv422, 1280x720, 0/1, 147456 kb/s, 10 fps, 10 tbr, 10
00k tbn

Note that ffmpeg -list_formats all does list the mjpeg:

% ffmpeg -f video4linux2 -list_formats all -i /dev/video0
ffmpeg version 5.1-2+b1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12 (Debian 12.1.0-7)
  configuration: --prefix=/usr --extra-version=2+b1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-pocketsphinx --enable-librsvg --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libplacebo --enable-libx264 --enable-shared
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
[video4linux2,v4l2 @ 0x5694e62800] Compressed:       mjpeg :          Motion-JPEG : 1280x720 320x240 640x360 640x480
[video4linux2,v4l2 @ 0x5694e62800] Raw       :     yuyv422 :           YUYV 4:2:2 : 1280x720 320x240 640x360 640x480
/dev/video0: Immediate exit requested

v4l2-ctl provides further details on supported fps

% v4l2-ctl --list-formats-ext --device /dev/video0
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'MJPG' (Motion-JPEG, compressed)
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 320x240
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x360
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
        [1]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 1280x720
                        Interval: Discrete 0.100s (10.000 fps)
                Size: Discrete 320x240
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x360
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)

This bug was observed on a Snapdragon 7c arm64 based system with the Adreno 618 GPU.

[Balling]

Motion-JPEG and uncompressed YUYV, 

First of all, MJPEG is lossy. Without c copy there will be genration loss. Second of all, YUYV is 422, not 420, but MJPEG supports 422 and there are very different quality jpeg encoders, classic example being mozjpeg or even some 2000-s encoder done as a PhD, it is even better.

So I think it is all good.

[leezu]

yuyv422 at 1280x720 here only provides 10 fps (see the v4l2-ctl output above), whereas Motion-JPEG provides 30 fps.

So I think it is all good.

How do you suggest to capture 1280x720 resolution video at 30 fps with ffmpeg on this hardware? I assume the lossy MJPEG at 1280x720 will still be better than 640x360 YUYV. Even with -input_format mjpeg -c copy, yuyv422 is used.