Transcoding a transport stream with a timestamp wrap does not work with -copyts

[Aurélien Nephtali]

Hello,

I have a sample I specialy crafted to reproduce a problem I'm facing with some streams I'm using.

I split an MPEG-TS file in half and put the second part before the first so timestamps suddenly jumps to a very old value (or very far in the future if we consider there was a timestamp wrap).

When transcoded with ffmpeg it gets stuck just like in ticket #3176 (nb_frames in do_video_out() is huge and ffmpeg duplicates a frame "forever").

If I remove -copyts it does not get stuck.

I uploaded the sample copyts_concatenated_stream.ts on the FTP.

How to reproduce:

% ffmpeg -copyts -i /dump/copyts_concatenated_stream.ts -vcodec libx264 -profile:v baseline -r 12 -g 36 -preset ultrafast -filter:v yadif -f mpegts -y /dev/null
ffmpeg version N-58512-g2459318 Copyright (c) 2000-2013 the FFmpeg developers
  built on Nov 28 2013 12:27:19 with gcc 4.7 (Debian 4.7.2-5)
  configuration: --prefix=/home/aurelien/local --enable-postproc --enable-nonfree --enable-gpl --enable-libfaac --enable-shared --enable-libx264 --extra-cflags=-I/home/aurelien/local/include --extra-ldflags=-L/home/aurelien/local/lib --enable-version3 --disable-optimizations --enable-libfreetype --disable-stripping --enable-libaacplus --enable-libfdk-aac
  libavutil      52. 55.100 / 52. 55.100
  libavcodec     55. 44.100 / 55. 44.100
  libavformat    55. 21.102 / 55. 21.102
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     3. 91.100 /  3. 91.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
[mp3 @ 0x18135c0] Header missing
[mpeg2video @ 0x1812b60] Invalid frame dimensions 0x0.
[mp3 @ 0x1814a80] Header missing
[mpeg2video @ 0x1812b60] Invalid frame dimensions 0x0.
    Last message repeated 4 times
[mpegts @ 0x180eb00] PES packet size mismatch
Input #0, mpegts, from '/dump/copyts_concatenated_stream.ts':
  Duration: 26:28:51.62, start: 863.036433, bitrate: 0 kb/s
  Program 1230
    Stream #0:0[0x8aa]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 544x576 [SAR 32:17 DAR 16:9], max. 5200 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x90e](fra): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:2[0x972](fra): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
    Stream #0:3[0x9d6](qad): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
[libx264 @ 0x1815c80] using SAR=32/17
[libx264 @ 0x1815c80] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
[libx264 @ 0x1815c80] profile Constrained Baseline, level 2.2
Output #0, mpegts, to '/dev/null':
  Metadata:
    encoder         : Lavf55.21.102
    Stream #0:0: Video: h264 (libx264), yuv420p, 544x576 [SAR 32:17 DAR 16:9], q=-1--1, 90k tbn, 12 tbc
    Stream #0:1(fra): Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video -> libx264)
  Stream #0:1 -> #0:1 (mp2 -> mp2)
Press [q] to stop, [?] for help   
[mp2 @ 0x18135c0] Header missing
Error while decoding stream #0:1: Invalid data found when processing input
[mp2 @ 0x18135c0] Header missing
Error while decoding stream #0:1: Invalid data found when processing input
[mpeg2video @ 0x1812b60] 00 motion_type at 0 28
[mpeg2video @ 0x1812b60] 00 motion_type at 0 29
[mpeg2video @ 0x1812b60] 00 motion_type at 18 24
[mpeg2video @ 0x1812b60] invalid mb type in B Frame at 3 32
[mpeg2video @ 0x1812b60] 00 motion_type at 1 33
[mpeg2video @ 0x1812b60] 00 motion_type at 0 34
[mpeg2video @ 0x1812b60] ac-tex damaged at 1 35
[mpeg2video @ 0x1812b60] 00 motion_type at 19 10
[mpeg2video @ 0x1812b60] 00 motion_type at 0 11
[mpeg2video @ 0x1812b60] 00 motion_type at 0 12
[mpeg2video @ 0x1812b60] 00 motion_type at 7 13
[mpeg2video @ 0x1812b60] invalid mb type in B Frame at 1 30
[mpeg2video @ 0x1812b60] 00 motion_type at 0 31
[mpeg2video @ 0x1812b60] 00 motion_type at 15 14
[mpeg2video @ 0x1812b60] invalid cbp -1 at 3 15
[mpeg2video @ 0x1812b60] slice mismatch
[mpeg2video @ 0x1812b60] 00 motion_type at 1 17
[mpeg2video @ 0x1812b60] invalid cbp -1 at 0 18
[mpeg2video @ 0x1812b60] ac-tex damaged at 0 19
[mpeg2video @ 0x1812b60] invalid cbp 0 at 0 20
[mpeg2video @ 0x1812b60] invalid mb type in B Frame at 4 21
[mpeg2video @ 0x1812b60] 00 motion_type at 5 22
[mpeg2video @ 0x1812b60] 00 motion_type at 0 23
[mpeg2video @ 0x1812b60] 00 motion_type at 1 24
[mpeg2video @ 0x1812b60] 00 motion_type at 2 25
[mpeg2video @ 0x1812b60] 00 motion_type at 0 26
[mpeg2video @ 0x1812b60] 00 motion_type at 0 27
[mpeg2video @ 0x1812b60] Warning MVs not available
[mpeg2video @ 0x1812b60] concealing 884 DC, 884 AC, 884 MV errors in B frame

<<< need to Ctrl-C twice to kill ffmpeg from here >>>

[Carl Eugen Hoyos]

Are you sure this is a real-world test-case?
While DVB streams may contain timestamp wraps, ffmpeg (the application) will not see them afair because the demuxer takes care of them.

[Aurélien Nephtali]

Yes, I see this kind of behaviour in some DVB-S streams (probably due to encoders being restarted or the live feed coming from another source, I don't really know).
The sample is not a real-life example but based on the debug I saw I was able to craft one that makes ffmpeg behave the same way.
I don't know if I will be able to capture a real sample since I have a lot of streams...

[Carl Eugen Hoyos]

I uploaded a sample to ​http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket3179/
Transcoding drops the beginning of the video, leading to massive A/V desync.

$ ffmpeg -copyts -i wrap.ts -qscale 10 out.ts
ffmpeg version N-58684-gf068aed Copyright (c) 2000-2013 the FFmpeg developers
  built on Dec  1 2013 11:25:37 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      52. 56.100 / 52. 56.100
  libavcodec     55. 44.100 / 55. 44.100
  libavformat    55. 21.102 / 55. 21.102
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     3. 91.100 /  3. 91.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
[mpeg2video @ 0x2e0cdc0] Invalid frame dimensions 0x0.
    Last message repeated 13 times
[NULL @ 0x2e15560] start time is not set in estimate_timings_from_pts
[NULL @ 0x2e16fa0] start time is not set in estimate_timings_from_pts
[mpegts @ 0x2de7900] PES packet size mismatch
    Last message repeated 3 times
[mpegts @ 0x2de7900] Could not find codec parameters for stream 5 (Unknown: none ([6][0][0][0] / 0x0006)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x2de7900] Could not find codec parameters for stream 6 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mpegts, from 'wrap.ts':
  Duration: 00:00:35.32, start: -21.053622, bitrate: 4750 kb/s
  Program 1001
    Stream #0:0[0x7d1]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 720x576 [SAR 64:45 DAR 16:9], max. 3500 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x7db](ger): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 160 kb/s (clean effects)
    Stream #0:2[0x7dc](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 160 kb/s (clean effects)
    Stream #0:3[0x7f9](ger): Audio: ac3 ([6][0][0][0] / 0x0006), 48000 Hz, stereo, fltp, 448 kb/s (clean effects)
    Stream #0:4[0x835](ger): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
    Stream #0:5[0x836]: Unknown: none ([6][0][0][0] / 0x0006)
    Stream #0:6[0x1f41]: Unknown: none ([5][0][0][0] / 0x0005)
  Program 1002
  Program 1003
  Program 1004
  Program 1005
  Program 1006
  Program 1007
  Program 1008
  Program 1009
  Program 1011
  Program 1012
  Program 1101
  Program 1102
  Program 1103
  Program 1105
  Program 1010
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf55.21.102
    Stream #0:0: Video: mpeg2video, yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1(ger): Audio: mp2, 48000 Hz, stereo, s16, 128 kb/s (clean effects)
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video -> mpeg2video)
  Stream #0:1 -> #0:1 (mp2 -> mp2)
Press [q] to stop, [?] for help
[mpegts @ 0x2de7900] PES packet size mismatchme=00:00:07.88 bitrate=2109.2kbits/s dup=0 drop=495
[mpeg2video @ 0x2e0cdc0] invalid cbp -1 at 25 12
[mpeg2video @ 0x2e0cdc0] Warning MVs not available
[mpeg2video @ 0x2e0cdc0] concealing 1080 DC, 1080 AC, 1080 MV errors in B frame
[mp2 @ 0x2e0e840] incomplete frame
Error while decoding stream #0:1: Invalid data found when processing input
frame=  360 fps=0.0 q=10.0 Lsize=    3766kB time=00:00:14.36 bitrate=2148.2kbits/s dup=0 drop=495
video:2891kB audio:543kB subtitle:0 global headers:0kB muxing overhead 9.658197%

I cannot reproduce the problem with a bigger sample.

[Aurélien Nephtali]

I don't have any A/V desync with your sample.
The beginning of the video is missing but when the woman talks it's OK.

ffmpeg version N-58684-gf068aed Copyright (c) 2000-2013 the FFmpeg developers

or

ffmpeg version N-58738-gf3008f3 Copyright (c) 2000-2013 the FFmpeg developers