Opened 4 years ago

Last modified 7 weeks ago

#7596 new enhancement

ffmpeg ignores v4l2 device prefs and selects a non-optimal format

Reported by: Cole Mickens Owned by:
Priority: wish Component: avdevice
Version: git-master Keywords: v4l2
Cc: leezu Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary:

  1. Set a v4l2 format preference.
  2. Use ffmpeg or mpv to open the v4l2 device.
  3. ffmpeg will use the correct resolution, but incorrect pixelformat

This results in a sub-optimal format being used, resulting in potentially highly unfortunate end-results for the user.

Ideally:

  1. ffmpeg would respect the preferences set for the device in v4l2
  2. ffmpeg would choose the more optimal resolution/format for the best result

In my case, since ffmpeg respects the resolution, but not the pixelformat, it winds up operating at the best 1080p YUYV profile, which is 5FPS. (compare to 1080p MJPEG which is 30FPS).

$ v4l2-ctl -d 4 --set-fmt-video=width=1920,height=1080,pixelformat=1

$ v4l2-ctl -d 4 --get-fmt-video
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'MJPG' (Motion-JPEG)
	Field             : None
	Bytes per Line    : 0
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Full Range)
	Flags             : 

$ mpv av://v4l2:/dev/video4
Playing: av://v4l2:/dev/video4
 (+) Video --vid=1 (rawvideo 1920x1080 5.000fps)
[autoconvert] Converting yuyv422 -> yuv422p
VO: [gpu] 1920x1080 yuv422p
V: 00:00:02 / 00:00:03 (87%)

$ ffmpeg -f v4l2 -i /dev/video4 -c:v copy out.mkv

...
Input #0, video4linux2,v4l2, from '/dev/video4':
  Duration: N/A, start: 23757.859585, bitrate: 165888 kb/s
    Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 1920x1080, 165888 kb/s, 5 fps, 5 tbr, 1000k tbn, 1000k tbc
...

$ ffmpeg -f v4l2 -input_format mjpeg -i /dev/video4 -c:v copy out.mkv

Input #0, video4linux2,v4l2, from '/dev/video4':
  Duration: N/A, start: 23792.047911, bitrate: N/A
    Stream #0:0: Video: mjpeg, yuvj422p(pc, bt470bg/unknown/unknown), 1920x1080, 30 fps, 30 tbr, 1000k tbn, 1000k tbc

Change History (2)

comment:1 by Carl Eugen Hoyos, 4 years ago

Component: undeterminedavdevice
Keywords: v4l2 added
Priority: normalwish
Type: defectenhancement

To make this a valid ticket please test current FFmpeg git head and provide the FFmpeg command line you tested together with the complete, uncut console output.

comment:2 by leezu, 7 weeks ago

Cc: leezu added
Version: unspecifiedgit-master

Reproduced on git master

% v4l2-ctl -d 0 --get-fmt-video                                                                                                 ~/Downloads                                                                       
Format Video Capture:                                                                                    
        Width/Height      : 1280/720                                                                                                                                                                              
        Pixel Format      : 'YUYV' (YUYV 4:2:2)                                                                                                                                                                   
        Field             : None                    
        Bytes per Line    : 2560                                                                         
        Size Image        : 1843200                                                                      
        Colorspace        : sRGB                    
        Transfer Function : Rec. 709                                                                     
        YCbCr/HSV Encoding: ITU-R 601                                                                    
        Quantization      : Default (maps to Limited Range)                                              
        Flags             :                                                                              
% v4l2-ctl --list-formats-ext --device /dev/video0                                                                              ~/Downloads
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'MJPG' (Motion-JPEG, compressed)
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 320x240
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x360
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
        [1]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 1280x720
                        Interval: Discrete 0.100s (10.000 fps)
                Size: Discrete 320x240
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x360
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
% v4l2-ctl -d 0 --set-fmt-video=width=1280,height=720,pixelformat=MJPG                                                          ~/Downloads
% v4l2-ctl -d 0 --get-fmt-video                                                                                                 ~/Downloads
Format Video Capture:
        Width/Height      : 1280/720
        Pixel Format      : 'MJPG' (Motion-JPEG)
        Field             : None
        Bytes per Line    : 0
        Size Image        : 1843200
        Colorspace        : sRGB
        Transfer Function : Rec. 709
        YCbCr/HSV Encoding: ITU-R 601
        Quantization      : Default (maps to Full Range)
        Flags             : 
% ~/Downloads/ffmpeg -v 9 -loglevel 99 -i /dev/video0 -input_format mjpeg                                                       ~/Downloads
ffmpeg version N-107804-gaa9eabb7a5-https://www.martin-riedl.de Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --prefix=/home/ffmpegBuild/out --enable-gpl --pkg-config-flags=--static --extra-version='https://www.martin-riedl.de' --enable-gray --enable-libxml2 --enable-libfreetype --enable-fontconfig --enable-libbluray --enable-libass --enable-libaom --enable-libopenh264 --enable-libsvtav1 --enable-libvpx --enable-libx264 --enable-libx265 --enable-libmp3lame --enable-libopus --enable-libvorbis
  libavutil      57. 33.101 / 57. 33.101
  libavcodec     59. 42.101 / 59. 42.101
  libavformat    59. 30.100 / 59. 30.100
  libavdevice    59.  8.101 / 59.  8.101
  libavfilter     8. 46.103 /  8. 46.103
  libswscale      6.  8.102 /  6.  8.102
  libswresample   4.  8.100 /  4.  8.100
  libpostproc    56.  7.100 / 56.  7.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-i' ... matched as input url with argument '/dev/video0'.
Reading option '-input_format' ... matched as AVOption 'input_format' with argument 'mjpeg'.
Trailing option(s) found in the command: may be ignored.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Successfully parsed a group of options.
Parsing a group of options: input url /dev/video0.
Successfully parsed a group of options.
Opening an input file: /dev/video0.
Probing video4linux2,v4l2 score:99 size:0
[video4linux2,v4l2 @ 0x5e06df3060] fd:3 capabilities:84a00001
[video4linux2,v4l2 @ 0x5e06df3060] Current input_channel: 0, input_name: Camera 1, input_std: 0
[video4linux2,v4l2 @ 0x5e06df3060] Querying the device for the current frame size
[video4linux2,v4l2 @ 0x5e06df3060] Setting frame size to 1280x720
[video4linux2,v4l2 @ 0x5e06df3060] The V4L2 driver changed the pixel format from 0x32315559 to 0x47504A4D
[video4linux2,v4l2 @ 0x5e06df3060] Trying to set codec:rawvideo pix_fmt:yuv420p
[video4linux2,v4l2 @ 0x5e06df3060] The V4L2 driver changed the pixel format from 0x32315559 to 0x47504A4D
[video4linux2,v4l2 @ 0x5e06df3060] Trying to set codec:rawvideo pix_fmt:yuv420p
[video4linux2,v4l2 @ 0x5e06df3060] The V4L2 driver changed the pixel format from 0x32315659 to 0x47504A4D
[video4linux2,v4l2 @ 0x5e06df3060] Trying to set codec:rawvideo pix_fmt:yuv422p
[video4linux2,v4l2 @ 0x5e06df3060] The V4L2 driver changed the pixel format from 0x50323234 to 0x47504A4D
[video4linux2,v4l2 @ 0x5e06df3060] Trying to set codec:rawvideo pix_fmt:yuyv422
[video4linux2,v4l2 @ 0x5e06df3060] All info found
[video4linux2,v4l2 @ 0x5e06df3060] stream 0: start_time: 1491.3 duration: NOPTS
[video4linux2,v4l2 @ 0x5e06df3060] format: start_time: 1491.3 duration: NOPTS (estimate from bit rate) bitrate=147456 kb/s
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 1491.299662, bitrate: 147456 kb/s
  Stream #0:0, 1, 1/1000000: Video: rawvideo, 1 reference frame (YUY2 / 0x32595559), yuyv422, 1280x720, 0/1, 147456 kb/s, 10 fps, 10 tbr, 1000k tbn
Successfully opened the file.
At least one output file must be specified
[video4linux2,v4l2 @ 0x5e06df3060] Some buffers are still owned by the caller on close.
ioctl(VIDIOC_QBUF): Bad file descriptor

This issue is related to https://trac.ffmpeg.org/ticket/9878 which tracks ffmpeg ignored input_format specification (not just the defaults as in the current issue)

Note: See TracTickets for help on using tickets.