[ffmpeg][avc/hevc encode][regression]A commit caused all avc/hevc encode cases in ffmpeg-vaapi fail!

[yefeng]

Summary of the bug:
How to reproduce:

ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -v verbose -f rawvideo -pix_fmt yuv420p -s:v 320x240 -r:v 25 -i <input> -vf 'format=nv12,hwupload' -c:v h264_vaapi -profile:v high -rc_mode CBR -g 30 -slices 1 -bf 0 -b:v 500k -maxrate 500k -low_power 0 -y <output>

ffmpeg version N-99800-g7369595c55
commit 7369595c556516b90bf3acdf85043dc11add7d89 caused this issue.
Output:

[h264_vaapi @ 0x55f5559b0800] colour_primaries does not match inferred value: 0, but should be 2.
[h264_vaapi @ 0x55f5559b0800] Failed to write unit 0 (type 7).
[h264_vaapi @ 0x55f5559b0800] Failed to write packed header.
[h264_vaapi @ 0x55f5559b0800] Failed to write per-sequence header: -1094995529.
[h264_vaapi @ 0x55f5559b0800] Encode failed: -1094995529.
Video encoding failed
[AVIOContext @ 0x55f5559bbc40] Statistics: 0 seeks, 0 writeouts
[AVIOContext @ 0x55f5559ac000] Statistics: 6220800 bytes read, 0 seeks
 Conversion failed!

[jeeb]

Repro'd locally on my X230 Thinkpad. The input appears to not have anything set as far as colorspace information is concerned, but I will check what libavfilter actually returns :) .

./ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -v verbose -f rawvideo -pix_fmt yuv420p -s:v 320x240 -r:v "24000/1001" -i out.yuv -vf 'scale,format=nv12,hwupload' -c:v h264_vaapi -movflags faststart out.mp4
ffmpeg version N-99816-g3da35b7cc7 Copyright (c) 2000-2020 the FFmpeg developers
  built with clang version 11.0.0 (Fedora 11.0.0-1.fc33)
  configuration: --disable-autodetect --cc=clang --cxx=clang++ --enable-zlib --enable-gpl --enable-libx264 --prefix=/home/jeeb/ownapps/encoding_prefix/ --enable-pic --enable-libxml2 --enable-gnutls --disable-stripping --extra-cflags=-fno-omit-frame-pointer --extra-cxxflags=-fno-omit-frame-pointer --enable-version3 --enable-gnutls --enable-vaapi --enable-libzvbi --enable-version3 --enable-libaribb24 --enable-libdav1d --enable-libopus --enable-pic
  libavutil      56. 60.100 / 56. 60.100
  libavcodec     58.112.100 / 58.112.100
  libavformat    58. 64.100 / 58. 64.100
  libavdevice    58. 11.102 / 58. 11.102
  libavfilter     7. 88.100 /  7. 88.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
[AVHWDeviceContext @ 0x3721dc0] libva: VA-API version 1.9.0
[AVHWDeviceContext @ 0x3721dc0] libva: Trying to open /usr/lib64/dri/iHD_drv_video.so
[AVHWDeviceContext @ 0x3721dc0] libva: va_openDriver() returns -1
[AVHWDeviceContext @ 0x3721dc0] libva: Trying to open /usr/lib64/dri/i965_drv_video.so
[AVHWDeviceContext @ 0x3721dc0] libva: Found init function __vaDriverInit_1_7
[AVHWDeviceContext @ 0x3721dc0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x3721dc0] Initialised VAAPI connection: version 1.9
[AVHWDeviceContext @ 0x3721dc0] VAAPI driver: Intel i965 driver for Intel(R) Ivybridge Mobile - 2.4.0.
[AVHWDeviceContext @ 0x3721dc0] Driver not found in known nonstandard list, using standard behaviour.
[rawvideo @ 0x3733080] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from 'out.yuv':
  Duration: 00:00:01.04, start: 0.000000, bitrate: 22096 kb/s
    Stream #0:0: Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p, 320x240, 22096 kb/s, 23.98 tbr, 23.98 tbn, 23.98 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_vaapi))
Press [q] to stop, [?] for help
[Parsed_scale_0 @ 0x376aec0] w:iw h:ih flags:'bicubic' interl:0
[graph 0 input from stream 0:0 @ 0x376d200] w:320 h:240 pixfmt:yuv420p tb:1001/24000 fr:24000/1001 sar:0/1
[Parsed_scale_0 @ 0x376aec0] w:320 h:240 fmt:yuv420p sar:0/1 -> w:320 h:240 fmt:nv12 sar:0/1 flags:0x4
[h264_vaapi @ 0x375ce40] Using input frames context (format vaapi_vld) with h264_vaapi encoder.
[h264_vaapi @ 0x375ce40] Input surface format is nv12.
[h264_vaapi @ 0x375ce40] Using VAAPI profile VAProfileH264High (7).
[h264_vaapi @ 0x375ce40] Using VAAPI entrypoint VAEntrypointEncSlice (6).
[h264_vaapi @ 0x375ce40] Using VAAPI render target format YUV420 (0x1).
[h264_vaapi @ 0x375ce40] No quality level set; using default (20).
[h264_vaapi @ 0x375ce40] RC mode: CQP.
[h264_vaapi @ 0x375ce40] RC quality: 20.
[h264_vaapi @ 0x375ce40] RC framerate: 24000/1001 (23.98 fps).
[h264_vaapi @ 0x375ce40] Using intra, P- and B-frames (supported references: 1 / 1).
[h264_vaapi @ 0x375ce40] All wanted packed headers available (wanted 0xd, found 0x1f).
[h264_vaapi @ 0x375ce40] Using level 1.3.
[h264_vaapi @ 0x375ce40] colour_primaries does not match inferred value: 0, but should be 2.
[h264_vaapi @ 0x375ce40] Failed to write unit 0 (type 7).
[h264_vaapi @ 0x375ce40] Failed to write packed header.
[h264_vaapi @ 0x375ce40] Failed to write sequence header for extradata: -1094995529.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[AVIOContext @ 0x3767500] Statistics: 0 seeks, 0 writeouts
[AVIOContext @ 0x3733d40] Statistics: 115200 bytes read, 0 seeks
Conversion failed!

[jeeb]

[Parsed_scale_0 @ 0x3585ec0] w:320 h:240 fmt:yuv420p sar:0/1 -> w:320 h:240 fmt:nv12 sar:0/1 flags:0x4
[h264_vaapi @ 0x3c2de40] range: tv, primaries: unknown (2), trc: unknown, colorspace: unknown, chroma sample location: unspecified

So swscale specifically notes that it is limited range, which is why vf_scale appears to set that value to the AVFrame, and thus that is the only field set in the AVFrame as there is no further color information available.

Edit:
The primaries value passed to the h264 VAAPI encoder is seemingly properly 2.

[jeeb]

Seems like this is due to the VAAPI H.264 and HEVC encoders handling the VUI flags incorrectly. Mark was nice enough to come up with a patch which seems to have worked. Hopefully he posts it on the ML soon, HEVC I unfortunately cannot myself test but the logic should be the same.

As the result of the patch, the colour values are not left to possibly illegal values (the structure is zero-initialized)

[jkqxz]

​https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2020-November/271869.html

[yefeng]

the patches fix this issue: ​https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=2682

[U. Artie Eoff]

Thanks @yefeng for confirming the patches fix this without new regressions.

Thanks @jkqxz for the patches. Can these be merged ASAP?

[jeeb]

LGTM'd the set, as @jkqxz wakes up he can push it in :) . Tested the H.264 and MPEG-2 Video encoders with my X230 with and without additional color metadata.

[jeeb]

Patch set has been pulled in as:
9faf4dcf234dbc34f05cd2d973473df2eece4881
486817e1aedda759bd21287b462ee0bb6e100949
755203c2d270e1334987b462f80151eb88e30121