Opened 17 months ago

Last modified 15 months ago

#9095 new defect

h264_qsv DECODER is corrupted after flush [WINDOWS]

Reported by: teslan Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: qsv
Cc: wenbin.chen@intel.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Unlike any other h264 decoder, h264_qsv is corrupted during a standard seeking procedure - specifically after avcodec_flush_buffers(ctx) and/or avcodec_send_packet(ctx, NULL) is called.

First, avcodec_send_packet() yields AVERROR_BUG a couple of times and then only AVERROR_EINVAL is being returned until I stop the decoding.

Errors Error initializing the MFX video decoder: undefined behavior (-16) and Unsupported pixel format. are being logged in great quantity while still trying to decode.

After flushing, no valid frames are returned from the decoder.

Note that when none of these flush functions are called after a seek, the decoder continues to work without any logged errors (but annoying artifacts are present, as expected)

Platforms tested:
Intel Ivy Bridge (i7-3632QM)
Intel Ice Lake (i5-1035G4)
So this spans almost through all the generations

How to reproduce:
Despite the fact that I need this in my application, the defect demonstrates itself also in ffplay:

% .\ffplay.exe -codec:v h264_qsv .\video-001.ts
ffplay version 4.3.2-2021-02-02-full_build-www.gyan.dev Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 10.2.0 (Rev6, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '.\video-001.ts':q=    0KB sq=    0B f=0/0
  Duration: 00:39:14.28, start: 0.000000, bitrate: 2199 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 60 fps, 60 tbr, 90k tbn, 120 tbc

......... PRESS RIGHT ARROW HERE (SEEK).............

[h264_qsv @ 00000211e46eb7c0] Error initializing the MFX video decoder: undefined behavior (-16)
    Last message repeated 3 times
[h264_qsv @ 00000211e46eb7c0] Error initializing the MFX video decoder: undefined behavior (-16)
    Last message repeated 1 times
[h264_qsv @ 00000211e46eb7c0] Unsupported pixel format.
    Last message repeated 8 times
[h264_qsv @ 00000211e46eb7c0] Unsupported pixel format.  0B f=0/0
    Last message repeated 11 times
[h264_qsv @ 00000211e46eb7c0] Unsupported pixel format.  0B f=0/0
    Last message repeated 11 times
[h264_qsv @ 00000211e46eb7c0] Unsupported pixel format.  0B f=0/0
    Last message repeated 15 times
[h264_qsv @ 00000211e46eb7c0] Unsupported pixel format.  0B f=0/0
    Last message repeated 15 times
[h264_qsv @ 00000211e46eb7c0] Unsupported pixel format.  0B f=0/0
    Last message repeated 14 times
[h264_qsv @ 00000211e46eb7c0] Unsupported pixel format.  0B f=0/0

......... UNTIL I STOP IT

Attachments (1)

0001-libavcodec-qsvdec-skip-non-key-frame-after-seek-func.patch (1.0 KB ) - added by wenbin,chen 17 months ago.

Download all attachments as: .zip

Change History (12)

comment:1 by teslan, 17 months ago

The bug does not demonstrate itself on ffplay always - especially if the video is in some "proper" container. But it does with all the transport streams. Example video:

https://mega.nz/file/FMoUhSSJ#I4yQmQfEtJkXP057v8iHg3G7YKNkpbuEnbfJS0vQwzQ

comment:2 by Carl Eugen Hoyos, 17 months ago

Keywords: h264_qsv flush seek avcodec_flush_buffers avcodec_send_packet removed
Priority: importantnormal

comment:3 by Carl Eugen Hoyos, 17 months ago

Please test current FFmpeg git head, the only version supported on this bug tracker.

comment:4 by teslan, 17 months ago

Tested right now on 5c8d4c4 (BtbN build)- same behaviour. No relevant commit was pushed since. But if this is too strict, I can wait for another build to test it :)

Last edited 17 months ago by teslan (previous) (diff)

comment:5 by teslan, 17 months ago

Ok, So after another testing the decoder seems to be corrupted only after it is flushed (by one of these functions or both) and is then fed by a non-key frame. Now it makes sense that with a "proper" container this issue does not manifest itself.

I worked around this by simply throwing away all non-key packets after a seek.

The issue is not fixed. qsv_h264 in ffplay corrupts and provides no output after a seek in .ts and the behaviour is not consistent with h264, h264_cuvid or h264 with any tested hwaccel (vaapi, vdpau, dxva2, d3d11va)

Last edited 17 months ago by teslan (previous) (diff)

in reply to:  5 comment:6 by Carl Eugen Hoyos, 17 months ago

Replying to teslan:

ffplay still crashes with qsv_h264 after a seek in .ts

You originally did not report a crash and the console output you provided did not indicate a crash.
Is there a crash?

comment:7 by teslan, 17 months ago

I'm very sorry for blatant imprecision of my expression. The comment is edited - i meant a corruption of decoder to the level that it does not provide any output frame, not a crash.

There is also another, quite interesting part: you basically can not flush it. Does not matter what you do after a seek - It will always output exactly two more frames from a previously decoded "chunk".

Trying to gather residual frames immediately after a flush with avcodec_receive_frame does not help as it will output invalid frames (with negative pts) indefinitely - without any error. The residual frames are outputted only after fresh packets are fed to it.

comment:8 by wenbin,chen, 17 months ago

Cc: wenbin.chen@intel.com added
Reproduced by developer: set

Hi teslan

I reproduced this error. I think the cause is that qsv decoder assume that after seek(), the first frame should be key frame. However this is not true when playing your video. I am working on it and try to submit a patch to fix this.

Thanks for your submission.

comment:9 by wenbin,chen, 17 months ago

you can try this patch. I already submit it to our internal review and if it is ok I will submit it to community.

Thanks

comment:11 by teslan, 15 months ago

Sorry for the late answer. Yes that works fine, thank You!

Note: See TracTickets for help on using tickets.