[ffmpeg-qsv][hevc] some cases decode failed on iHD driver.

[a]

Summary of the bug:

hevc ffmpeg/MSFT/JVT/BBC/container/allegro10/allegro10_25 decode failed

environment

SKL OS: ubuntu 16.04 kernel: 4.15.0 
libva b6c50dad4d7d14c507108d9f468662e2d7ae1c4d ​https://github.com/01org/libva.git 
iHD_Driver ed04556a6676fd77ff845508bc54bfc60915450c ​https://github.com/intel/media-driver
libva-utils 9a10ad663349732decd668a426cfc349b0010d1d ​https://github.com/01org/libva-utils.git
Mediasdk 7ea683585f0f84f1bea035a405855312c92c46c0 ​https://github.com/Intel-Media-SDK/MediaSDK
FFMPEG 56f68a099cc607658118e00cad30569103ae3751 ​https://git.ffmpeg.org/ffmpeg.git

How to reproduce:

1.build env as above lists
2.ffmpeg -hwaccel qsv -hwaccel_device /dev/dri/renderD128 -v verbose -y -c:v hevc_qsv -load_plugin hevc_hw -i ./DBLK_G_VIXS_1.bit -vf hwdownload,format=nv12 -pix_fmt yuv420p -f md5 qsv_out.md5

error message:

[hevc_qsv @ 0xdba740] Error initializing the MFX video decoder: invalid video parameters (-15)
Error while decoding stream #0:0: Invalid argument
[AVHWDeviceContext @ 0xea4c00] Unknown driver "Intel iHD driver - 2.0.0", assuming standard behaviour.
[hevc_qsv @ 0xdba740] Error initializing the MFX video decoder: invalid video parameters (-15)
Error while decoding stream #0:0: Invalid argument
[AVHWDeviceContext @ 0x1159900] Unknown driver "Intel iHD driver - 2.0.0", assuming standard behaviour.
[hevc_qsv @ 0xdba740] Error initializing the MFX video decoder: invalid video parameters (-15)
Error while decoding stream #0:0: Invalid argument
[AVHWDeviceContext @ 0x111e7c0] Unknown driver "Intel iHD driver - 2.0.0", assuming standard behaviour.
[hevc_qsv @ 0xdba740] Error initializing the MFX video decoder: invalid video parameters (-15)
Error while decoding stream #0:0: Invalid argument
[AVHWDeviceContext @ 0xf27cc0] Unknown driver "Intel iHD driver - 2.0.0", assuming standard behaviour.
[hevc_qsv @ 0xdba740] Error initializing the MFX video decoder: invalid video parameters (-15)
Error while decoding stream #0:0: Invalid argument
[AVHWDeviceContext @ 0x13b6980] Unknown driver "Intel iHD driver - 2.0.0", assuming standard behaviour.
[hevc_qsv @ 0xdba740] Error initializing the MFX video decoder: invalid video parameters (-15)
Error while decoding stream #0:0: Invalid argument
[AVHWDeviceContext @ 0x11230c0] Unknown driver "Intel iHD driver - 2.0.0", assuming standard behaviour.
[hevc_qsv @ 0xdba740] Error initializing the MFX video decoder: invalid video parameters (-15)
Error while decoding stream #0:0: Invalid argument
[AVHWDeviceContext @ 0xe9af80] Unknown driver "Intel iHD driver - 2.0.0", assuming standard behaviour.
[hevc_qsv @ 0xdba740] Error initializing the MFX video decoder: invalid video parameters (-15)
Error while decoding stream #0:0: Invalid argument
[hevc_qsv @ 0xdba740] video_get_buffer: image parameters invalid
[hevc_qsv @ 0xdba740] get_buffer() failed

failed case

DBLK_D_VIXS_1.bit
DBLK_E_VIXS_1.bit
DBLK_F_VIXS_1.bit
DBLK_G_VIXS_1.bit
ENTP_A_LG_2.bit
ENTP_B_LG_2.bit
IPRED_C_Mitsubishi_2.bit
RAP_B_Bossen_1.bit
TMVP_A_MS_2.bit
TSKIP_A_MS_2.bit
ENTP_C_LG_3.bit 
MSFT/MSHDRef_Texture_SkinTones_08_320x240p24f_randomaccess_main.bin
JVT/PICSIZE_D_Bossen_1.bin
BBC/SteamLocomotiveTrain_2560x1600_60_10bit_crop_qp32.bin

[jkqxz]

RAP_B_Bossen_1.bit works for me with libmfx.

It looks like libmfx barfs on files with missing profile information? All of DBLK_*_VIXS_1.bit, ENTP_*_LG_*.bit, IPRED_C_Mitsubishi_2.bit, TMVP_A_MS_2.bit and TSKIP_A_MS_2.bit have this problem and are therefore technically invalid, but the libavcodec decoder handles this case so they do work fine with VAAPI using the same hardware.

PICSIZE_D_Bossen_1.bin is 4216 samples wide, maybe libmfx has some problem with that? There isn't anything else interesting about it as far as I can tell (and it is intended to be a picture size test). It does work file with VAAPI, so it's not a hardware issue.

Can you share the files "MSFT/MSHDRef_Texture_SkinTones_08_320x240p24f_randomaccess_main.bin" and "BBC/SteamLocomotiveTrain_2560x1600_60_10bit_crop_qp32.bin"?

[Linjie.Fu]

I tried the attached clip only and find something about this issue.

ff_qsv_profile_to_mfx will return directly if the profile info can not be found, thus causing the invalid video parameters issues.

If set profile to FF_PROFILE_HEVC_MAIN when the profile is missing or cannot be found, the attached clip can be decoded successfully.

 libavcodec/qsv.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c
index bb0d79588c..f0ccd5d294 100644
--- a/libavcodec/qsv.c
+++ b/libavcodec/qsv.c
@@ -73,7 +73,11 @@ int ff_qsv_profile_to_mfx(enum AVCodecID codec_id, int profile)
         return MFX_PROFILE_UNKNOWN;
     switch (codec_id) {
     case AV_CODEC_ID_H264:
+        return profile;
     case AV_CODEC_ID_HEVC:
+        if (profile != FF_PROFILE_HEVC_MAIN && profile != FF_PROFILE_HEVC_MAIN_10 &&
+            profile != FF_PROFILE_HEVC_MAIN_STILL_PICTURE && profile != FF_PROFILE_HEVC_REXT)
+            profile = FF_PROFILE_HEVC_MAIN;
         return profile;
     case AV_CODEC_ID_VC1:
         return 4 * profile + 1;

[jkqxz]

That change seems like it will fail in an even worse way if the stream does not actually conform to Main profile?

For the streams with missing profile information I think there are three plausible options to choose from:

• Do nothing. The streams in question are technically invalid for decoding because they don't conform to any profile, and the specific examples here were superseded for conformance purposes by valid streams anyway (see e.g. DBLK_D_VIXS_2.bit, etc.).
• Add additional parsing to determine a real profile that an unknown stream conforms to. That could probably be done with CBS as a BSF, though checking all the section A.3 conditions is quite tedious.
• Modify libmfx to act like libavcodec and accept streams which don't have a profile set. I have no idea how difficult this would be.

Do we expect there to be a nontrivial number of streams in the wild without profiles set? If not, I think the first of these is the right option to choose.

[Zhong,Li]

Using MSDK parser which works closer with MSDK decoder, and can handle such clips well.
I've tested ​https://patchwork.ffmpeg.org/patch/12260/ can fix this issue.
Patch is still under review, will close this ticket once merged.

[Balling]

Replying to lizhong1008:

Using MSDK parser which works closer with MSDK decoder, and can handle such clips well.
I've tested ​https://patchwork.ffmpeg.org/patch/12260/ can fix this issue.
Patch is still under review, will close this ticket once merged.

Should work after this 00d0a4aa9eda8553113e51556123c46648a5f31b right?