v4l2 user control configuration mechanism

[Adam Clarke]

avdevice/v4l2.c does not provide a v4l2 user control ( V4L2_CID_* ) listing and configuration mechanism, see ​http://linuxtv.org/downloads/v4l-dvb-apis/control.html#control-id

something similar to the capabilities offered by v4l2-ctl --set-ctrl=<ctrl>=<val>[,<ctrl>=<val>] is needed

[Carl Eugen Hoyos]

As-is, this is an invalid ticket.

Please describe in more detail which v4l2 feature FFmpeg is missing.

[Adam Clarke]

I apologize if I am not using the correct format to report this issue.
Please correct me so I might learn the appropriate approach.

In ​https://lists.ffmpeg.org/pipermail/ffmpeg-user/2012-May/006601.html Geotz Dapp asks for an option to ioctl V4L2_CID_AUTO_WHITE_BALANCE false, and an option to ioctl
(from /usr/include/linux/videodev2.h) V4L2_CID_FOCUS_AUTO to V4L2_FOCUS_MANUAL (0).

In addition would very much like to have options to ioctl V4L2_CID_EXPOSURE to zero, V4L2_CID_AUTOGAIN to false, V4L2_CID_CHROMA_AGC to false, V4L2_CID_HUE_AUTO to false, V4L2_CID_AUTOBRIGHTNESS to false.

[Adam Clarke]

Replying to cehoyos:

As-is, this is an invalid ticket.

Please describe in more detail which v4l2 feature FFmpeg is missing.

Does the comment above provide sufficient detail to make this a valid ticket?

[dronus]

I think using v4l2-ctl before ffmpeg would be ok if it would work.

Currently ffmpeg does a mess on the settings made by v4l2-ctl. The pixel format is not honored, but must be set by -pixel_format. Doing so it seems tend to reset other parameters, eg. exposure settings or something on some cameras. As those cannot be controlled by ffmpeg, they must be reapplied by v4l2-ctl after starting the capture, a need which renders the first seconds of the capture useless.

So ffmpeg should either provide the full range of controls like cehoyos suggested (much work to do) or just keep whats set up and honor it. As v4l2.c matches up libav and v4l2 pixel formats it may be tricky to keep the already set v4l2 format and honor it, however ffmpeg could fail if no support is possible and suggest a compatible format to be set with v4l2-ctl.

[Elliott]

Here is a list of controls which could be added. This list is from Raspbian Buster, so it includes some private controls such as white_balance_auto_preset = greyworld which is important for Pi NoIR Camera.

Documentation and background info: ​https://www.kernel.org/doc/html/latest/media/uapi/v4l/control.html

$ v4l2-ctl -L

User Controls

                     brightness 0x00980900 (int)    : min=0 max=100 step=1 default=50 value=50 flags=slider
                       contrast 0x00980901 (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                     saturation 0x00980902 (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                    red_balance 0x0098090e (int)    : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
                   blue_balance 0x0098090f (int)    : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
           power_line_frequency 0x00980918 (menu)   : min=0 max=3 default=1 value=1
				0: Disabled
				1: 50 Hz
				2: 60 Hz
				3: Auto
                      sharpness 0x0098091b (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                  color_effects 0x0098091f (menu)   : min=0 max=15 default=0 value=0
				0: None
				1: Black & White
				2: Sepia
				3: Negative
				4: Emboss
				5: Sketch
				6: Sky Blue
				7: Grass Green
				8: Skin Whiten
				9: Vivid
				10: Aqua
				11: Art Freeze
				12: Silhouette
				13: Solarization
				14: Antique
				15: Set Cb/Cr
                         rotate 0x00980922 (int)    : min=0 max=360 step=90 default=0 value=0 flags=modify-layout
             color_effects_cbcr 0x0098092a (int)    : min=0 max=65535 step=1 default=32896 value=32896

Codec Controls

             video_bitrate_mode 0x009909ce (menu)   : min=0 max=1 default=0 value=0 flags=update
				0: Variable Bitrate
				1: Constant Bitrate
                  video_bitrate 0x009909cf (int)    : min=25000 max=25000000 step=25000 default=10000000 value=5000000
         repeat_sequence_header 0x009909e2 (bool)   : default=0 value=0
            h264_i_frame_period 0x00990a66 (int)    : min=0 max=2147483647 step=1 default=60 value=60
                     h264_level 0x00990a67 (menu)   : min=0 max=11 default=11 value=11
				0: 1
				1: 1b
				2: 1.1
				3: 1.2
				4: 1.3
				5: 2
				6: 2.1
				7: 2.2
				8: 3
				9: 3.1
				10: 3.2
				11: 4
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=4 value=4
				0: Baseline
				1: Constrained Baseline
				2: Main
				4: High

Camera Controls

                  auto_exposure 0x009a0901 (menu)   : min=0 max=3 default=0 value=0
				0: Auto Mode
				1: Manual Mode
         exposure_time_absolute 0x009a0902 (int)    : min=1 max=10000 step=1 default=1000 value=1000
     exposure_dynamic_framerate 0x009a0903 (bool)   : default=0 value=0
             auto_exposure_bias 0x009a0913 (intmenu): min=0 max=24 default=12 value=12
				0: -4000 (0xfffffffffffff060)
				1: -3667 (0xfffffffffffff1ad)
				2: -3333 (0xfffffffffffff2fb)
				3: -3000 (0xfffffffffffff448)
				4: -2667 (0xfffffffffffff595)
				5: -2333 (0xfffffffffffff6e3)
				6: -2000 (0xfffffffffffff830)
				7: -1667 (0xfffffffffffff97d)
				8: -1333 (0xfffffffffffffacb)
				9: -1000 (0xfffffffffffffc18)
				10: -667 (0xfffffffffffffd65)
				11: -333 (0xfffffffffffffeb3)
				12: 0 (0x0)
				13: 333 (0x14d)
				14: 667 (0x29b)
				15: 1000 (0x3e8)
				16: 1333 (0x535)
				17: 1667 (0x683)
				18: 2000 (0x7d0)
				19: 2333 (0x91d)
				20: 2667 (0xa6b)
				21: 3000 (0xbb8)
				22: 3333 (0xd05)
				23: 3667 (0xe53)
				24: 4000 (0xfa0)
      white_balance_auto_preset 0x009a0914 (menu)   : min=0 max=10 default=1 value=10
				0: Manual
				1: Auto
				2: Incandescent
				3: Fluorescent
				4: Fluorescent H
				5: Horizon
				6: Daylight
				7: Flash
				8: Cloudy
				9: Shade
				10: Greyworld
            image_stabilization 0x009a0916 (bool)   : default=0 value=0
                iso_sensitivity 0x009a0917 (intmenu): min=0 max=4 default=0 value=0
				0: 0 (0x0)
				1: 100000 (0x186a0)
				2: 200000 (0x30d40)
				3: 400000 (0x61a80)
				4: 800000 (0xc3500)
           iso_sensitivity_auto 0x009a0918 (menu)   : min=0 max=1 default=1 value=1
				0: Manual
				1: Auto
         exposure_metering_mode 0x009a0919 (menu)   : min=0 max=2 default=0 value=0
				0: Average
				1: Center Weighted
				2: Spot
                     scene_mode 0x009a091a (menu)   : min=0 max=13 default=0 value=0
				0: None
				8: Night
				11: Sports

JPEG Compression Controls

            compression_quality 0x009d0903 (int)    : min=1 max=100 step=1 default=30 value=30

[John Boiles]

+1 it'd be nice for either ffmpeg to have full control over v4l2 devices or no control at all. Perhaps there could be a new ffmpeg flag to tell it to not change v4l2 settings.

As it is currently, you may need to specify some settings twice -- once because sometimes v4l2 needs to switch to the right pixelformat before it can use pixelformat-related controls, then again for ffmpeg so that ffmpeg doesn't switch your settings around. E.g.:

v4l2-ctl -d /dev/video0 -v pixelformat=H264,width=1920,height=1080
v4l2-ctl -d /dev/video0 -c video_bitrate_mode=1,video_bitrate=2000000,h264_profile=0,h264_i_frame_period=60
v4l2-ctl -d /dev/video0 -p 30
ffmpeg -pix_fmt h264 -video_size 1920x1080 -i /dev/video0 ....