h264_qsv DECODER eventually freezes after many seeks [WINDOWS]

[teslan]

Summary of the bug:
The h264_qsv decoder freezes after random number of seeks with flushes (most of the times you need max 100 seeks). Does not happen when standard h264 or h264_cuvid decoder is used.

Note that this might be a bug not in FFMPEG, but in Intel Media SDK.
I am not sure on which grave should I cry, but because there are developers from Intel actively enhancing the QSV support, i'll describe the issue both here and at Intel Media SDK github.

Tested with both Windows-provided libmfxhw64.dll (File Version 11.21.4.14) and from the latest driver (File Version 21.11.3.320)

This is the debug call stack on my decoding thread when the freeze happens:

1 ZwWaitForAlertByThreadId ntdll 0x7fff7a530764
2 RtlSleepConditionVariableSRW ntdll 0x7fff7a4f4021
3 SleepConditionVariableSRW KERNELBASE 0x7fff77ecce89
4 MFXVideoVPP_GetVPPStat libmfxhw64 0x7fff027ef8cc
5 MFXVideoVPP_GetVPPStat libmfxhw64 0x7fff027ec7f4
6 MFXVideoVPP_GetVPPStat libmfxhw64 0x7fff027ec69f
7 MFXVideoVPP_GetVPPStat libmfxhw64 0x7fff023575b0
8 MFXVideoCORE_SyncOperation libmfxhw64 0x7fff02337a4d
9 av_qsv_alloc_context avcodec_59 0x7fff10eb4c06
10 av_qsv_alloc_context avcodec_59 0x7fff10eb573d
11 av_qsv_alloc_context avcodec_59 0x7fff10eb59c7
12 avcodec_default_get_buffer2 avcodec_59 0x7fff10a9f9b1
13 avcodec_send_packet avcodec_59 0x7fff10aa0590
14 libavDecoder::playLoopPrivate libavdecoder.cpp 724 0x7ff78094f6da

I am using FFMPEG builds from BtbN - confirmed with the latest master update (2022-02-15 12:35)

How to reproduce:
You need to flush the decoder and seek the video many times. Then after some decoded frames the decoder freezes. These are important excerpts from my code to hint how am I using it - in my use case I need to send an exact amount of frames from a given PTS with the need to consider the GOP interval:

Seek:

...........................
err = av_seek_frame(m.fmtctx, m.videoStream, [random-pts-in-media minus GOP interval], 0);
        if(err != 0) {
                av_strerror(err, errbuff, 256);
                std::cerr << "Error Seeking. AVERROR: " << errbuff  << std::endl;
        }

        ptsToFind = referencePts;
        seekMode = true;
        seekModeKeyframeFound = false;
        seekModeFlushed = false;

.........................

Read&decode:

...........................
 //Reads the right amount of data from stream
        err = av_read_frame(m.fmtctx, pPacket);
        if(err < 0) {
            av_strerror(err, errbuff, 256);
            std::cerr << "AV READ FRAME error or EOF " << errbuff << " " << m.lastPts[m.videoStream] << std::endl;
            if(err == AVERROR_EOF) {
                emit endOfFile(m.lastPts[m.videoStream]);
            }
            av_packet_unref(pPacket);
            break;
        }

        //If we are seeking, flush the codec and reset the GOP counter (as we automatically found nearest I frame)
        if(seekMode) {

            if(!seekModeFlushed) {
                if(!(pPacket->flags & AV_PKT_FLAG_KEY)){
                    av_packet_unref(pPacket);
                    continue;
                }
                avcodec_send_packet(avcc, NULL); //flush
                avcodec_flush_buffers(avcc);
                actual_gop_counter = 0;
                seekModeFlushed = true;
            }
        }

        //Send the "right amount of data" to the decoder
        err = avcodec_send_packet(avcc, pPacket);
        if(err != 0) {
            av_strerror(err, errbuff, 256);
            std::cerr << "Error sending packet. AVERROR: " << errbuff << std::endl;
            av_packet_unref(pPacket);
            continue;
        }


        //Gather first ready frame from the internal queue
        err = avcodec_receive_frame(avcc, pFrame);
        if(err != 0) {
            if(err != AVERROR(EAGAIN)) { //resource can be unavailable pretty often, do not write out anything in such case
                av_strerror(err, errbuff, 256);
                std::cerr << "Error receiving frame. AVERROR: " << errbuff  << std::endl;
            }
            av_packet_unref(pPacket);
            av_frame_unref(pFrame);
            continue;
        }

        //Save data for average GOP computation
        if(pFrame->pict_type == AV_PICTURE_TYPE_I && actual_gop_counter != 0) {
            if(actual_gop_counter > max_gop) {
                max_gop = actual_gop_counter;
            }
            actual_gop_counter = 0;
        }
        ++actual_gop_counter;

        if(seekMode) {
            if(pFrame->pts >= ptsToFind && pFrame->pts <= ptsToFind + avg_pts*MAX_ALLOWED_SKIP_FRAMES) {
                seekMode = false;
                //Because QSV is a fucked-up decoder, it always returns exactly TWO frames from the previous
                //decoding chunk. And you can not flush it, it will only return them after you provide
                //two fresh packets. Really fucked-up. Really. Therefore we can not stay with
                //the check that if the pts is already the same or greater that the one we need to find.
                //We also have to introduce if it is not too far away. That is defined in MAX_ALLOWED_SKIP_FRAMES
            }
            else {
                av_frame_unref(pFrame);
                av_packet_unref(pPacket);
                continue; //do not proceed if still in seek
            }
        }

.............................

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.