avcodec: h.264 decoder flags result as ER error_occurred with frame threading

[jeeb]

Summary of the bug:

With sample ​https://megumin.fushizen.eu/samples/2024-04-15-framethread_causes_er_errors.264 , in case of frame threading being enabled, the H.264 decoder flags multiple output frames with decode_error_flags: FF_DECODE_ERROR_DECODE_SLICES (8) before the sudden end at the end of the sample (where there are actual MB decoding errors), while disabling threading leads to this flagging not happening

The data seems to match between these two cases and there is no visual corruption, so either both are fine or both are in some manner broken. I have not compared against the JM reference decoder yet.

How to reproduce:

% ffmpeg -threads:v 2 -i "https://megumin.fushizen.eu/samples/2024-04-15-framethread_causes_er_errors.264" -f null pipe:
ffmpeg version N-114837-g257bc2a82a Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 14 (GCC)
  configuration: --prefix=/home/jeeb/ownapps/encoding_prefix/ --enable-static --disable-doc --disable-stripping --enable-gpl --enable-version3 --enable-vaapi --enable-zlib --enable-libdav1d --enable-libx264 --enable-libsvtav1 --enable-libx265 --enable-libzimg --enable-gnutls --enable-libplacebo
  libavutil      59. 15.100 / 59. 15.100
  libavcodec     61.  5.103 / 61.  5.103
  libavformat    61.  3.100 / 61.  3.100
  libavdevice    61.  2.100 / 61.  2.100
  libavfilter    10.  2.101 / 10.  2.101
  libswscale      8.  2.100 /  8.  2.100
  libswresample   5.  2.100 /  5.  2.100
  libpostproc    58.  2.100 / 58.  2.100
[h264 @ 0x5475300] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, h264, from 'https://megumin.fushizen.eu/samples/2024-04-15-framethread_causes_er_errors.264':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 1200k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf61.3.100
  Stream #0:0: Video: wrapped_avframe, yuv420p(tv, bt709, top coded first (swapped)), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn
      Metadata:
        encoder         : Lavc61.5.103 wrapped_avframe
[vist#0:0/h264 @ 0x6144c80] [dec:h264 @ 0x551c4c0] corrupt decoded frame
    Last message repeated 7 times
[h264 @ 0x60bee40] error while decoding MB 64 10, bytestream -16 speed=1.92x    
[h264 @ 0x60bee40] concealing 6945 DC, 6945 AC, 6945 MV errors in P frame
[vist#0:0/h264 @ 0x6144c80] [dec:h264 @ 0x551c4c0] corrupt decoded frame
[out#0/null @ 0x5517800] video:18KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
frame=   42 fps=0.0 q=-0.0 Lsize=N/A time=00:00:01.68 bitrate=N/A speed=2.51x

Comparison with threading disabled:

ffmpeg -threads:v 1 -i "https://megumin.fushizen.eu/samples/2024-04-15-framethread_causes_er_errors.264" -f null pipe:
ffmpeg version N-114837-g257bc2a82a Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 14 (GCC)
  configuration: --prefix=/home/jeeb/ownapps/encoding_prefix/ --enable-static --disable-doc --disable-stripping --enable-gpl --enable-version3 --enable-vaapi --enable-zlib --enable-libdav1d --enable-libx264 --enable-libsvtav1 --enable-libx265 --enable-libzimg --enable-gnutls --enable-libplacebo
  libavutil      59. 15.100 / 59. 15.100
  libavcodec     61.  5.103 / 61.  5.103
  libavformat    61.  3.100 / 61.  3.100
  libavdevice    61.  2.100 / 61.  2.100
  libavfilter    10.  2.101 / 10.  2.101
  libswscale      8.  2.100 /  8.  2.100
  libswresample   5.  2.100 /  5.  2.100
  libpostproc    58.  2.100 / 58.  2.100
[h264 @ 0x605b300] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, h264, from 'https://megumin.fushizen.eu/samples/2024-04-15-framethread_causes_er_errors.264':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 1200k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf61.3.100
  Stream #0:0: Video: wrapped_avframe, yuv420p(tv, bt709, top coded first (swapped)), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn
      Metadata:
        encoder         : Lavc61.5.103 wrapped_avframe
[h264 @ 0x6de1580] error while decoding MB 64 10, bytestream -16 speed=1.16x    
[h264 @ 0x6de1580] concealing 6945 DC, 6945 AC, 6945 MV errors in P frame
[vist#0:0/h264 @ 0x6d2ac80] [dec:h264 @ 0x61024c0] corrupt decoded frame
[out#0/null @ 0x60fd800] video:18KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
frame=   42 fps= 34 q=-0.0 Lsize=N/A time=00:00:01.68 bitrate=N/A speed=1.37x

Some changes I have applied to check values:

diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index 70de151301..5be43910ad 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -731,7 +731,8 @@ static int packet_decode(DecoderPriv *dp, AVPacket *pkt, AVFrame *frame)
 
         if (frame->decode_error_flags || (frame->flags & AV_FRAME_FLAG_CORRUPT)) {
             av_log(dp, exit_on_error ? AV_LOG_FATAL : AV_LOG_WARNING,
-                   "corrupt decoded frame\n");
+                   "corrupt decoded frame. decode_error_flags: %d, AV_FRAME_FLAG_CORRUPT: %d\n",
+                   frame->decode_error_flags, (frame->flags & AV_FRAME_FLAG_CORRUPT));
             if (exit_on_error)
                 return AVERROR_INVALIDDATA;
         }
diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c
index 727dc1a662..f31c936595 100644
--- a/libavcodec/h264dec.c
+++ b/libavcodec/h264dec.c
@@ -756,6 +756,9 @@ static int decode_nal_units(H264Context *h, const uint8_t *buf, int buf_size)
         if (h->cur_pic_ptr->decode_error_flags) {
             /* Frame-threading in use */
             atomic_int *decode_error = h->cur_pic_ptr->decode_error_flags;
+            av_log(h->avctx, AV_LOG_WARNING,
+                   "threaded error case: ret: %d, ER error_occurred: %d, decode_error_flags : %d\n",
+                   ret, h->er.error_occurred, *h->cur_pic_ptr->decode_error_flags);
             /* Using atomics here is not supposed to provide syncronisation;
              * they are merely used to allow to set decode_error from both
              * decoding threads in case of coded slices. */

Example of checking that the output is the same between the two cases:

ffmpeg -threads:v 1 -i "https://megumin.fushizen.eu/samples/2024-04-15-framethread_causes_er_errors.264" -f framehash pipe: > 01-hashes_threadless.txt
ffmpeg -threads:v 2 -i "https://megumin.fushizen.eu/samples/2024-04-15-framethread_causes_er_errors.264" -f framehash pipe: > 02-hashes_threaded.txt
git diff --no-index 01-hashes_threadless.txt 02-hashes_threaded.txt