MPEG TS: A/V async issue when remuxing. Plays fine in MPV

[Peter B.]

Summary of the bug:

I have off-the-air recordings: H.264/AAC in an MPEG-TS (.ts) file.
For archiving, I uncompress the audio to pcm_s16le, and the container to MKV.

I have a file where audio/video goes completely out of sync after a certain position in the video.
I've isolated the culprit part of the video using "dd".

I've tried the following commands:

$FFMPEG -i $IN -c copy ${OUT}-copy.ts
$FFMPEG -i $IN -c copy -vsync passthrough ${OUT}-vsync_pass.ts
$FFMPEG -i $IN -c copy -vsync cfr ${OUT}-vsync_cfr.ts
$FFMPEG -i $IN -c copy -vsync vfr ${OUT}-vsync_vfr.ts
$FFMPEG -i $IN -c copy -vsync drop ${OUT}-vsync_drop.ts

$FFMPEG -i $IN -c:v copy -c:a pcm_s16le ${OUT}-pcm_s16le.ts
$FFMPEG -i $IN -c:v copy -c:a pcm_s16le -af "aresample=async=1000" ${OUT}-aresample.mkv

None of them produce a clean output :(

The commercial clip starting at offset 26s seems to be defect.
The file plays fine in ​MPV Media Player, so I guess there must be a way to fix the timestamps somehow?

Here's the complete, uncut (*) console output from the first of the above commands I've tried:

$ ffmpeg -i nd_vs_v-cut2.ts -c copy out.ts

(*) I've removed a long number of repeating "decode_slice_header error" messages, since they were identical to the first one.

ffmpeg version N-84661-g66c1c9b Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --prefix=/usr/local --enable-gpl --enable-nonfree --enable-version3 --enable-postproc --enable-swscale --enable-avfilter --enable-pthreads --enable-bzlib --enable-zlib --enable-decoder=png --enable-encoder=png --samples=../fate-suite --enable-libfreetype --enable-libschroedinger --enable-libopenjpeg --disable-decoder=jpeg2000 --enable-libvpx --enable-libvorbis --enable-libx264
  libavutil      55. 51.100 / 55. 51.100
  libavcodec     57. 86.103 / 57. 86.103
  libavformat    57. 67.100 / 57. 67.100
  libavdevice    57.  3.101 / 57.  3.101
  libavfilter     6. 78.100 /  6. 78.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
[h264 @ 0x3f6e040] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x3f6e040] decode_slice_header error
[h264 @ 0x3f6e040] no frame!
[h264 @ 0x3f6e040] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x3f6e040] decode_slice_header error
[h264 @ 0x3f6e040] no frame!
[h264 @ 0x3f6e040] non-existing PPS 0 referenced
...
...
...
Input #0, mpegts, from 'nd_vs_v-cut2.ts':
  Duration: 00:03:18.03, start: 434.773333, bitrate: 1440 kb/s
  Program 1 
    Stream #0:0[0x101]: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 59 kb/s
    Stream #0:1[0x102]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720, 30 fps, 29.97 tbr, 90k tbn, 60 tbc
    Stream #0:2[0x103]: Data: timed_id3 (ID3  / 0x20334449)
Output #0, mpegts, to 'out/out.ts':
  Metadata:
    encoder         : Lavf57.67.100
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720, q=2-31, 30 fps, 29.97 tbr, 90k tbn, 90k tbc
    Stream #0:1: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 59 kb/s
Stream mapping:
  Stream #0:1 -> #0:0 (copy)
  Stream #0:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
[mpegts @ 0x3f68360] DTS 8589934592 < 8590832488 out of orderitrate=4346.8kbits/s speed=30.8x    
[mpegts @ 0x3f68360] DTS 55209870 < 8592634286 out of order9 bitrate=2633.7kbits/s speed=46.9x    
frame= 2589 fps=1058 q=-1.0 Lsize=   35793kB time=00:01:37.25 bitrate=3014.9kbits/s speed=39.7x    
video:32062kB audio:838kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 8.791791%

[Peter B.]

Uploaded the sample here:
​http://download.das-werkstatt.com/pb/contribs/files/nd_vs_v-cut2.ts
A/V sync problems appear in the commercial break starting around 26s.

(Tried "​ftp://upload.ffmpeg.org", but it returned a timeout...)

[Carl Eugen Hoyos]

Improves with -async 1.
Are you able to test the stream with a stream analyzer?

[Peter B.]

Stream analyzer:
Used ffprobe (show_frames/show_packets), but didn't really know what to look for.

Wanted to try "​dvbsnoop", but the documentation I could find seems to already require too much in-depth knowledge of DVB-streams in order to make any sense of how to use it.

Tried "-async 1", but the commercial A/V of the commercial is out of sync until "he's a captain!" (ca. 1m55s). That behavior is as close as I get with ffmpeg.

Sorry to post it as bugreport, but if MPV is able to make sense of the stream, I guess it means that ffmpeg can somehow be able too. Especially since ffmpeg libs are the base of MPV.

Thank you very much for looking into this!
These A/V async issues are quite common with MPEG-TS recordings, and it would be great to use ffmpeg to fix them :)

[Carl Eugen Hoyos]

Replying to peter_b:

Stream analyzer:

My question was: Do you have access to a (possibly very expensive, most likely proprietary) soft- or hardware that can tell you if the stream in question is valid (and can be played in-sync without changing video speed)?
(I thought you have.)

These A/V async issues are quite common with MPEG-TS recordings, and it would be great to use ffmpeg to fix them :)

I wonder if the sample you provided is a good example for such common issues (it could of course be).
My suspicion is that even if ffmpeg would be better with audio holes this sample would still be problematic (and needs at least a heuristic).

[Carl Eugen Hoyos]

A naive analysis seems to indicate there is an issue if two timestamp resets are too close to each-other.

[Peter B.]

Stream analyzer:
Understood :)
Unfortunately, no: I don't have access to them.
I'm pretty sure that data in the commercial which is non-valid.

I assumed it might just be broken beyond repair, but it "can be played in-syn without changing video speed" (at least not that I'm aware of) - by MPV.

btw: File also doesn't play properly in any version of VLC I've tried:
Linux: v2.2.5.1 Umbrella / v3.0.0 (​from Videolan's PPA)
Win7: v2.2.4

[Peter B.]

Regarding the sample being problematic:
I thought it might be good to have problematic sample ;)

I've got 3 other recordings that have similar issues. Just haven't been able to isolate the culprit-position because these streams cannot properly be seeked/cut at all. And the whole recordings are >3GB each...

[Peter B.]

You think it might be worth trying ​Videolan's "multicat" on these files to provide more info on their quirks?

[Dennis E. Mungai]

Hello there,

If this is not too late, could you please try:

$FFMPEG -vsync passthrough -copyts -i $IN -c copy ${OUT}-vsync_pass.ts

Then report back on findings?

[Carl Eugen Hoyos]

Replying to Brainiarc7:

If this is not too late, could you please try:

$FFMPEG -vsync passthrough -copyts -i $IN -c copy ${OUT}-vsync_pass.ts

Then report back on findings?

Why don’t you test yourself?