Ffmpeg duplicates and drops frames when given a .ts stream.

[Devon Vitkovsky]

Summary of the bug: I have a .ts video from a capture device. When I attempt to encode it in a less-bloated format, the resulting output has an incorrect framerate. Upon closer examination it appears that almost every frame has been dropped and then previous frames duplicated, resulting in video that not only plays at 15 FPS, but has many duplicate frames that make it appear to play at 10 FPS.

The source video file is running at 59.94 FPS. Specifying -r (both before and after -i, or any combination) does not solve the problem.

How to reproduce:

ffmpeg -i foo.ts -c:v libx264 -an outbar.mp4
ffmpeg version N-53528-g160ea26
built on May 27 2013 15:25:58 with gcc 4.7.3 (GCC)

Other video codecs and containers produce the same results. It is possible to take the resulting footage apart with "ffmpeg -i foo.mp4 frame_%d.png" and see the problematic frames.

Please note that the counter in the bottom right corner of the screen goes up by 3 every 3 frames (rather than 1 every frame), making it an unreliable way to test for the bug's presence.

I have uploaded a sample from the .ts file to upload.mplayerhq.hu/incoming/ under the filename "Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream.ts", along with a .txt file with the same name that contains a copy of this bug and the output of "ffmpeg -v 9 -loglevel 99 -i CROP_B.ts -c:v libx264 -an test.mp4". The log has an identical name to the .ts file, except with .txt at the end.

The sample is a little strange and may be related to the problem, but even with its defects ffmpeg is still producing very odd output.

[Carl Eugen Hoyos]

Please provide your failing command line (without using external libraries if they are not needed to reproduce the problem) together with the complete, uncut console output.

[Devon Vitkovsky]

Output of -v 9 -loglevel 99.

[Devon Vitkovsky]

My bad, forgot to attach that here.

I should also mention that this happens regardless of the codec or container I'm trying to encode to. I forgot to mention that, too.

[Carl Eugen Hoyos]

Is libx264 needed to reproduce the problem?
If it is not needed, please provide a command line that does not use an external library together with the console output.

[Clément Bœsch]

The codec doesn't seem to be related, but the behaviour within container looks slightly different.

AVI, dup=0 drop=806:

☭ ./ffmpeg -i ~/Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream.ts -c:v mpeg4 -an -y outbar.avi 
ffmpeg version N-53573-g3ed3aaa Copyright (c) 2000-2013 the FFmpeg developers
  built on May 28 2013 12:46:13 with gcc 4.8.0 (GCC) 20130502 (prerelease)
  configuration: --enable-gpl --enable-libx264 --enable-libmp3lame --enable-x11grab --enable-libvorbis --enable-libvpx --enable-libopenjpeg
  libavutil      52. 34.100 / 52. 34.100
  libavcodec     55. 12.100 / 55. 12.100
  libavformat    55.  7.100 / 55.  7.100
  libavdevice    55.  1.101 / 55.  1.101
  libavfilter     3. 72.100 /  3. 72.100
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mpegts, from '/home/ubitux/Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream.ts':
  Duration: 00:00:20.78, start: 2000.488000, bitrate: 25666 kb/s
  Program 2 
    Stream #0:0[0x64]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
    Stream #0:1[0x65](USA): Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 241 kb/s
Output #0, avi, to 'outbar.avi':
  Metadata:
    ISFT            : Lavf55.7.100
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 59.94 tbn, 59.94 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg4)
Press [q] to stop, [?] for help
error while decoding MB 73 16, bytestream (-80)=00:00:19.83 bitrate= 914.3kbits/s dup=0 drop=787    
[h264 @ 0x32540c0] concealing 2296 DC, 2296 AC, 2296 MV errors in I frame
frame=  410 fps= 39 q=24.8 Lsize=    2300kB time=00:00:20.38 bitrate= 924.1kbits/s dup=0 drop=806    
video:2265kB audio:0kB subtitle:0 global headers:0kB muxing overhead 1.522031%

MKV, same:

☭ ./ffmpeg -i ~/Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream.ts -c:v mpeg4 -an -y outbar.mkv
ffmpeg version N-53573-g3ed3aaa Copyright (c) 2000-2013 the FFmpeg developers
  built on May 28 2013 12:46:13 with gcc 4.8.0 (GCC) 20130502 (prerelease)
  configuration: --enable-gpl --enable-libx264 --enable-libmp3lame --enable-x11grab --enable-libvorbis --enable-libvpx --enable-libopenjpeg
  libavutil      52. 34.100 / 52. 34.100
  libavcodec     55. 12.100 / 55. 12.100
  libavformat    55.  7.100 / 55.  7.100
  libavdevice    55.  1.101 / 55.  1.101
  libavfilter     3. 72.100 /  3. 72.100
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mpegts, from '/home/ubitux/Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream.ts':
  Duration: 00:00:20.78, start: 2000.488000, bitrate: 25666 kb/s
  Program 2 
    Stream #0:0[0x64]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
    Stream #0:1[0x65](USA): Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 241 kb/s
Output #0, matroska, to 'outbar.mkv':
  Metadata:
    encoder         : Lavf55.7.100
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 1k tbn, 59.94 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg4)
Press [q] to stop, [?] for help
error while decoding MB 73 16, bytestream (-80)=00:00:20.03 bitrate= 908.8kbits/s dup=0 drop=795    
[h264 @ 0x2a410c0] concealing 2296 DC, 2296 AC, 2296 MV errors in I frame
frame=  410 fps= 39 q=24.8 Lsize=    2268kB time=00:00:20.38 bitrate= 911.4kbits/s dup=0 drop=806    
video:2264kB audio:0kB subtitle:0 global headers:0kB muxing overhead 0.198051%

MP4, dup=409 drop=403:

☭ ./ffmpeg -i ~/Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream.ts -c:v mpeg4 -an -y outbar.mp4
ffmpeg version N-53573-g3ed3aaa Copyright (c) 2000-2013 the FFmpeg developers
  built on May 28 2013 12:46:13 with gcc 4.8.0 (GCC) 20130502 (prerelease)
  configuration: --enable-gpl --enable-libx264 --enable-libmp3lame --enable-x11grab --enable-libvorbis --enable-libvpx --enable-libopenjpeg
  libavutil      52. 34.100 / 52. 34.100
  libavcodec     55. 12.100 / 55. 12.100
  libavformat    55.  7.100 / 55.  7.100
  libavdevice    55.  1.101 / 55.  1.101
  libavfilter     3. 72.100 /  3. 72.100
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mpegts, from '/home/ubitux/Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream.ts':
  Duration: 00:00:20.78, start: 2000.488000, bitrate: 25666 kb/s
  Program 2 
    Stream #0:0[0x64]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
    Stream #0:1[0x65](USA): Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 241 kb/s
Output #0, mp4, to 'outbar.mp4':
  Metadata:
    encoder         : Lavf55.7.100
    Stream #0:0: Video: mpeg4 ( [0][0][0] / 0x0020), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 60k tbn, 59.94 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg4)
Press [q] to stop, [?] for help
error while decoding MB 73 16, bytestream (-80)=00:00:19.68 bitrate=1801.3kbits/s dup=394 drop=390    
[h264 @ 0x3071480] concealing 2296 DC, 2296 AC, 2296 MV errors in I frame
frame= 1222 fps= 86 q=31.0 Lsize=    4517kB time=00:00:20.38 bitrate=1814.9kbits/s dup=409 drop=403    
video:4511kB audio:0kB subtitle:0 global headers:0kB muxing overhead 0.133645%

@Nofew: you're ticket is mentioning that you tried extracting the frames into images. If you do so to compare every frame, make sure you use the -vsync vfr switch (See #1644 for more info).

[Devon Vitkovsky]

Replying to ubitux:
ubitux: Thanks for that tip, though it doesn't change what happens. When I do it on the .ts video I get every frame as expected, but the .mp4 (or any other video encoded by ffmpeg) gives two correct frames followed by a duplicated frame, with the pattern repeating through all the frames.

[Carl Eugen Hoyos]

Which capture device produced the input video?

I tested the following command lines:
$ ffmpeg -i Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream.ts -qscale 2 -mbd 2 -vcodec mpeg2video out.ts
$ ffmpeg -i Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream.ts -qscale 2 -mbd 2 -vcodec msmpeg4 out.avi
$ ffmpeg -i Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream.ts -vcodec mpeg4 -acodec aac -qscale 2 -strict -2 out.mp4
Imo, the three output files play smooth with WMP but the dropped frames are visible with frame-stepping. I suspect the problem is a non-constant pkt_dts_time in the input file.

[Devon Vitkovsky]

I use an "Elgato Game Capture HD".

I've talked about it over IRC a bit, and people do agree that the DTS in the .ts file is weird. They say ffmpeg shouldn't be handling it the way it does, though, especially since other software (any other software, avisynth, Elgato's included editor and others) handle it without error. They even correct the DTS signatures and output video ffmpeg can process properly.

The duplicated frames become more noticeable later on since the game speeds up significantly. I've uploaded a second file named "Ffmpeg_duplicates_and_drops_frames_when_given_a_ts_stream_(faster_video).ts" that can be used to demonstrate this.

[Carl Eugen Hoyos]

This was fixed in 1e5271a9fd6ddcceb083f2185a4bbd8d44c9a813 (2.1), related to ticket #2143.