Gray frames while decoding when HEVC codec is used and frames with missing POC reference exist

[dhirendvs]

Summary of the bug:
When a H265 .mp4 file contains missing/corrupt frames, and this file is decoded for playback using the HEVC codec, gray frames are visible in case of frames with missing POC reference i.e. when the error "Could not find ref with POC XX" is displayed.

I have verified that after calling avcodec_send_packet() and avcodec_receive_frame() methods, no decoding warning or error was generated in any of the cases of missing POC reference, and the AV_FRAME_FLAG_CORRUPT or AV_FRAME_FLAG_DISCARD flags were not set for AVFrame::flags, and also the FF_DECODE_ERROR_MISSING_REFERENCE flag was not set for AVFrame::decode_error_flags.

I had also tried setting the following AVCodecContext members:

strict_std_compliance = FF_COMPLIANCE_VERY_STRICT;
err_recognition = AV_EF_EXPLODE;
skip_frame = AVDISCARD_NONREF;

But even with the above settings, no decoding warning/error was generated while decoding with the HEVC codec when gray frames were visible during playback.

However, when the HEVC Intel Quick Sync codec is used, gray frames are not visible in case of the 8 instances of the missing POC reference error, but a much better playback result is generated with only 3 instances of garbled frames.

Is this because HEVC Intel Quick Sync codec decodes frames with a non-zero latency whereas the latency is 0 in case of the default HEVC codec? Is it possible to produce a similar result without gray frames when the default HEVC codec is used for playback of such a file with missing/corrupt frames?

When the VLC media player (tested with Ver 3.0.8) is used for playback of this file, the same result as HEVC Intel Quick Sync codec is generated without gray frames, and when Tools --> Preferences --> Input / Codecs --> Hardware-accelerated decoding --> Disable is used instead of default value of Automatic, then the result is the same as use of HEVC codec.

Link to the H265.mp4 file (valid for 30 days) for testing: ​https://easyupload.io/wrg357

How to reproduce:
A) Gray frames:
ffplay -vcodec hevc H265.mp4

B) Garbled frames in a few instances but no gray frames:
ffplay -vcodec hevc_qsv H265.mp4

Built on:
FFmpeg 5.0

Reference links to bugs with similar errors:
​https://github.com/mpv-player/mpv/issues/3440
​https://github.com/mpv-player/mpv/issues/6218
​https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/10bc4c3a7df7bb26303067b97311b7eeedfd453e
​https://stackoverflow.com/questions/51668066/detect-frame-with-incorrect-poc
​https://trac.ffmpeg.org/ticket/7950
​https://trac.ffmpeg.org/ticket/8732

[Balling]

Does not happen for me on the file. VLC does not open it at all with HW turned off. Please check with ​https://github.com/BtbN/FFmpeg-Builds

it is usually only bad on ts files. See: #9412

This is a bug in our ts demuxer, hevc is good.

[dhirendvs]

Replying to Balling:

Does not happen for me on the file. VLC does not open it at all with HW turned off. Please check with ​https://github.com/BtbN/FFmpeg-Builds

it is usually only bad on ts files. See: #9412

This is a bug in our ts demuxer, hevc is good.

I tried using the latest FFmpeg build: ​​https://github.com/BtbN/FFmpeg-Builds on 64-bit Windows 10 (Ver 1803) with Intel i5-8400 CPU. While using ffmpeg-master-latest-win64-gpl-shared.zip or ffmpeg-master-latest-win64-lgpl-shared.zip, 8 instances of gray frames are still displayed when the following command is used:

ffplay -vcodec hevc H265.mp4

and the following warning messages are displayed:

[hevc @ 000001e0129bcfc0] Could not find ref with POC 13 0B f=0/0
[hevc @ 000001e019e625c0] Could not find ref with POC 25 0B f=0/0
[hevc @ 000001e0129f2600] Could not find ref with POC 18 0B f=0/0
[hevc @ 000001e019bb16c0] Could not find ref with POC 4 0B f=0/0
[hevc @ 000001e0129f2600] Could not find ref with POC 44 0B f=0/0
[hevc @ 000001e012a027c0] Could not find ref with POC 31 0B f=0/0
[hevc @ 000001e01a07dc80] Could not find ref with POC 38 0B f=0/0
[hevc @ 000001e0129bcfc0] Could not find ref with POC 35 0B f=0/0

I've attached a screenshot (ffplay - Gray frame.png) of one of the instances of gray frames at time 00:32 during ffplay playback.

[Balling]

Oh, okay! This is not due to seeking, this is due to corrupt bitstream. Indeed you can extract Annex B stream:

ffmpeg -i H265.mp4 -c copy cnqwaucqa.265

Same POC 13, POC 25, POC 18, etc happen.

This has nothing to do with seeking though, since there is no mp4 container to seek here.

[dhirendvs]

Replying to Balling:

Oh, okay! This is not due to seeking, this is due to corrupt bitstream. Indeed you can extract Annex B stream:

ffmpeg -i H265.mp4 -c copy cnqwaucqa.265

Same POC 13, POC 25, POC 18, etc happen.

This has nothing to do with seeking though, since there is no mp4 container to seek here.

Is there any way to detect the instances of missing POC reference warnings for e.g. by using any of the flags that I had tested in the bug description?

[Balling]

For me green/red frames happen on -vcodec hevc_qsv (green is 0, 0, 0 YCbCr) and some artefacts in motion happen there on hevc_cuvid as expected.

[Marton Balint]

Not a regression or a crash, so does not qualify as important.