QSV: [M]JPEG encode "Uninitialized pict_type" regression

[U. Artie Eoff]

Summary of the bug:

QSV [M]JPEG encode fails with Assertion !"Uninitialized pict_type!" failed at libavcodec/qsvenc.c:1341 since commit:

commit 67cdfcf694f840d215be940f82545c45c9be193a
Author: Linjie Fu <linjie.fu@intel.com>
Date:   Wed Nov 28 10:41:55 2018 +0800

    lavc/qsvenc: assert uninitialized pict_type
    
    Assert in function ff_qsv_encode to avoid using uninitialized value:
    
    FF_DISABLE_DEPRECATION_WARNINGS
            avctx->coded_frame->pict_type = pict_type;
    FF_ENABLE_DEPRECATION_WARNINGS
    
    Signed-off-by: Linjie Fu <linjie.fu@intel.com>
    Signed-off-by: Zhong Li <zhong.li@intel.com>

How to reproduce:

% LIBVA_DRIVER_NAME=iHD ffmpeg -init_hw_device qsv=qsv:hw -hwaccel qsv -filter_hw_device qsv \
  -v verbose -f rawvideo -pix_fmt yuv420p -s:v 1280x720 \
  -i ./720p5994_parkrun_ter_1280x720_I420.yuv -vf 'format=nv12,hwupload=extra_hw_frames=64' \
  -c:v mjpeg_qsv -global_quality 100 -vframes 10 -y test.mjpeg

ffmpeg version 4.1.git Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 8 (GCC)
  configuration: --prefix=/home/uaeoff/Work/workspace/media/install --disable-static --enable-shared --enable-libdrm --enable-vaapi --enable-libmfx --disable-amf --disable-audiotoolbox --disable-cuda --disable-cuda-sdk --disable-cuvid --disable-d3d11va --disable-dxva2 --disable-libnpp --disable-mmal --disable-nvdec --disable-nvenc --disable-omx --disable-omx-rpi --disable-rkmpp --disable-v4l2-m2m --disable-vdpau --disable-videotoolbox --enable-nonfree --enable-gpl
  libavutil      56. 24.101 / 56. 24.101
  libavcodec     58. 41.101 / 58. 41.101
  libavformat    58. 23.102 / 58. 23.102
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 46.101 /  7. 46.101
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
[AVHWDeviceContext @ 0xe7d900] Opened VA display via X11 display :0.
[AVHWDeviceContext @ 0xe7d900] libva: VA-API version 1.4.0
[AVHWDeviceContext @ 0xe7d900] libva: va_getDriverName() returns 0
[AVHWDeviceContext @ 0xe7d900] libva: User requested driver 'iHD'
[AVHWDeviceContext @ 0xe7d900] libva: Trying to open /home/uaeoff/Work/workspace/media/install/lib/dri/iHD_drv_video.so
[AVHWDeviceContext @ 0xe7d900] libva: Found init function __vaDriverInit_1_4
[AVHWDeviceContext @ 0xe7d900] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0xe7d900] Initialised VAAPI connection: version 1.4
[AVHWDeviceContext @ 0xe7d900] VAAPI driver: Intel iHD driver - 1.0.0.
[AVHWDeviceContext @ 0xe7d900] Driver not found in known nonstandard list, using standard behaviour.
[AVHWDeviceContext @ 0xe7d4c0] Initialize MFX session: API version is 1.28, implementation version is 1.28
[AVHWDeviceContext @ 0xe7d4c0] MFX compile/runtime API: 1.28/1.28
[rawvideo @ 0xed4f40] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from '/home/uaeoff/Work/workspace/media/src/otc-media/assets/yuv/720p5994_parkrun_ter_1280x720_I420.yuv':
  Duration: 00:00:24.16, start: 0.000000, bitrate: 276480 kb/s
    Stream #0:0: Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p, 1280x720, 276480 kb/s, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> mjpeg (mjpeg_qsv))
Press [q] to stop, [?] for help
[graph 0 input from stream 0:0 @ 0xef4440] w:1280 h:720 pixfmt:yuv420p tb:1/25 fr:25/1 sar:0/1 sws_param:flags=2
[auto_scaler_0 @ 0xef7140] w:iw h:ih flags:'bicubic' interl:0
[Parsed_format_0 @ 0xef3540] auto-inserting filter 'auto_scaler_0' between the filter 'graph 0 input from stream 0:0' and the filter 'Parsed_format_0'
[auto_scaler_0 @ 0xef7140] w:1280 h:720 fmt:yuv420p sar:0/1 -> w:1280 h:720 fmt:nv12 sar:0/1 flags:0x4
[AVHWDeviceContext @ 0xef7ac0] VAAPI driver: Intel iHD driver - 1.0.0.
[AVHWDeviceContext @ 0xef7ac0] Driver not found in known nonstandard list, using standard behaviour.
Output #0, mjpeg, to 'test.mjpeg':
  Metadata:
    encoder         : Lavf58.23.102
    Stream #0:0: Video: mjpeg (mjpeg_qsv), 1 reference frame, qsv, 1280x720, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.41.101 mjpeg_qsv
Assertion !"Uninitialized pict_type!" failed at libavcodec/qsvenc.c:1341
Aborted (core dumped)

libva (master) heads/master-0-gcf11abe5e1b9  https://github.com/01org/libva
gmmlib (master) heads/master-0-g10ad15adc3a3  https://github.com/intel/gmmlib
intel-media-driver (master) heads/master-0-g09a7bb4793c9  https://github.com/intel/media-driver
msdk (master) heads/master-0-gf443f554d496  https://github.com/Intel-Media-SDK/MediaSDK
ffmpeg (master) heads/master-0-g5487560acfc4  https://git.ffmpeg.org/ffmpeg

[Carl Eugen Hoyos]

Please try to use binaries with sane version numbers (do not use github), not every report is as easy to understand as this one.

[U. Artie Eoff]

@cehoyos, can you give an example of a sane version number? Are you only concerned with ffmpeg version number? Generally, in our environment, we catch these regressions via our CI which follows mainline ToT commits and does out of tree builds. For ffmpeg, this results in .git to be used in the version since it's no longer in a git tree at configure/compile time.

[Carl Eugen Hoyos]

ffmpeg version N-92618-ged5680f
Out-of-tree builds generate the same version string, please elaborate.
If you mean exported trees, run ffbuild/version.sh . libavutil/ffversion.h before exporting the tree to get a useful version string.

[U. Artie Eoff]

Yes, I mean exported tree (i.e. all files, minus scm files, are copied to separate build directory). I will add your recommended extra steps to our process to ensure we generate a useful version string. Thanks!

[Carl Eugen Hoyos]

Replying to uartie:

Yes, I mean exported tree (i.e. all files, minus scm files, are copied to separate build directory). I will add your recommended extra steps to our process to ensure we generate a useful version string.

Please test it first, it's some time since I last had to use this!

[U. Artie Eoff]

Replying to cehoyos:

Replying to uartie:

Yes, I mean exported tree (i.e. all files, minus scm files, are copied to separate build directory). I will add your recommended extra steps to our process to ensure we generate a useful version string.

Please test it first, it's some time since I last had to use this!

Yes it works with minor addition: ffbuild/version.sh . libavutil/ffversion.h && touch .version

[U. Artie Eoff]

FYI... all qsv jpeg encode fails regardless of input file. That is, input file in reported description is not required to reproduce.

[Linjie.Fu]

Tracing into this regression:

In mjpeg_qsv:
pict_type was decided by Frame Type of bs and it equals AV_PICTURE_TYPE_NONE.

In native mjpeg encode:
pict_type was set to AV_PICTURE_TYPE_I in avcodec_encode_video2() after calling ff_mpv_encode_picture();

So, I think it may be better to set bs Frame Type to MFX_FRAMETYPE_I in MSDK than setting pict_type to AV_PICTURE_TYPE_NONE in ff_qsv_encode()?

[Zhong,Li]

Yes, I believe it is MSDK issue. QSV decoder has similar issue (hasn't provided frame type): ​https://github.com/Intel-Media-SDK/MediaSDK/issues/333

However, it will still be broken with old version MSDK even fixed in MSDK. And I prefer to set the picture type to be AV_PICTURE_TYPE_NONE when MSDK return an unknow frame type.

[Zhong,Li]

Fixed as commit 1c96d2e3998b9a2a97447547cb2d688ec0ce09ed