mpeg2video decode "end mismatch" errors and corrupted image output

[Aman]

Summary of the bug:

I have an mpeg2 720p sample which fails to decode correctly with avcodec/mpeg12dec.c, producing visible corruption on the bottom of generated images. Using other hardware decoders, no such corruption is visible.

How to reproduce:

ffmpeg version 4.3 Copyright (c) 2000-2020 the FFmpeg developers
  built with Apple clang version 11.0.3 (clang-1103.0.32.62)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.3_3 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from 'https://tmm1.s3.amazonaws.com/mpeg12dec-err.ts':
  Duration: 00:00:00.52, start: 32081.276567, bitrate: 5539 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
    Side data:
      cpb: bitrate max/min/avg: 80000000/0/0 buffer size: 9781248 vbv_delay: N/A
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to '/dev/null':
  Metadata:
    encoder         : Lavf58.45.100
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 59.94 fps, 59.94 tbn, 59.94 tbc
    Metadata:
      encoder         : Lavc58.91.100 wrapped_avframe
[mpeg2video @ 0x7fd90200b400] end mismatch left=29 2 at 0 45
[mpeg2video @ 0x7fd90200b400] Warning MVs not available
[mpeg2video @ 0x7fd90200b400] concealing 80 DC, 80 AC, 80 MV errors in B frame
https://tmm1.s3.amazonaws.com/mpeg12dec-err.ts: corrupt decoded frame in stream 0
[mpeg2video @ 0x7fd90200b400] end mismatch left=30 3E at 0 45
[mpeg2video @ 0x7fd90200b400] Warning MVs not available
[mpeg2video @ 0x7fd90200b400] concealing 80 DC, 80 AC, 80 MV errors in B frame
https://tmm1.s3.amazonaws.com/mpeg12dec-err.ts: corrupt decoded frame in stream 0
frame=   31 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.51 bitrate=N/A speed=0.826x
video:16kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

The visual corruption can be seen in the bottom 16 pixels of mpeg12decframe-005.png as generated by this command:

ffmpeg -i mpeg12dec-err.ts -map v -vframes 5 -y mpeg12decframe-%03d.png

A workaround is to disable the error as follows, which fixes the generated images:

--- a/libavcodec/mpeg12dec.c
+++ b/libavcodec/mpeg12dec.c
@@ -1889,9 +1889,12 @@ static int mpeg_decode_slice(MpegEncContext *s, int mb_y,
                 if (left < 0 ||
                     (left && show_bits(&s->gb, FFMIN(left, 23)) && !is_d10) ||
                     ((avctx->err_recognition & (AV_EF_BITSTREAM | AV_EF_AGGRESSIVE)) && left > 8)) {
+                    /*
                     av_log(avctx, AV_LOG_ERROR, "end mismatch left=%d %0X at %d %d\n",
                            left, left>0 ? show_bits(&s->gb, FFMIN(left, 23)) : 0, s->mb_x, s->mb_y);
                     return AVERROR_INVALIDDATA;
+                    */
+                    goto eos;
                 } else
                     goto eos;
             }

[Aman]

Note that the sample provided is a small ~500kb slice of a much larger file, and this issue occurs frequently both in the larger file and in the source feed where the file was recorded from.

For example, in a longer 2 minute sample:

$ ffmpeg -t 120 -i sample.ts -map v -f null -y /dev/null 2>&1 | grep concealing | sort | uniq -c | sort -n
  32 [mpeg2video @ 0x7fb2b501c400] concealing 80 DC, 80 AC, 80 MV errors in I frame
 194 [mpeg2video @ 0x7fb2b501c400] concealing 80 DC, 80 AC, 80 MV errors in P frame
 437 [mpeg2video @ 0x7fb2b501c400] concealing 80 DC, 80 AC, 80 MV errors in B frame

$ ffmpeg -nostats -t 120 -i sample.ts -map v -f null -y /dev/null 2>&1 | grep -Eo "end mismatch left=\d+" | sort | uniq -c | sort -n
   1 end mismatch left=41
   1 end mismatch left=42
   1 end mismatch left=49
   1 end mismatch left=51
   1 end mismatch left=54
   1 end mismatch left=56
   1 end mismatch left=58
   1 end mismatch left=61
   1 end mismatch left=63
   8 end mismatch left=13
   8 end mismatch left=15
  10 end mismatch left=12
  11 end mismatch left=14
  12 end mismatch left=8
  13 end mismatch left=10
  13 end mismatch left=11
  17 end mismatch left=9
  24 end mismatch left=21
  25 end mismatch left=19
  25 end mismatch left=20
  25 end mismatch left=30
  29 end mismatch left=18
  30 end mismatch left=16
  31 end mismatch left=22
  32 end mismatch left=31
  34 end mismatch left=23
  36 end mismatch left=27
  38 end mismatch left=25
  38 end mismatch left=29
  42 end mismatch left=28
  47 end mismatch left=24
  52 end mismatch left=26
  54 end mismatch left=17

[Carl Eugen Hoyos]

Is the issue reproducible with current FFmpeg git head?

Please provide a sample that was not created with FFmpeg (but dd).

[Aman]

Yes it reproduces with HEAD

Here's the same sample before feeding into ffmpeg to filter out extra tracks: ​https://tmm1.s3.amazonaws.com/avcodec-mpeg2-err.ts

[Marton Balint]

You can use -ec 0 as well to disable error concealment which is causing the corruption. However this looks like faulty encoder. Maybe error concealmnet should be smarter about this?

[Aman]

You're right. With -ec 0 the "end mismatch" is still present but the video corruption goes away.

Do you know where in the EC code this could be improved?