Opened 6 months ago

Last modified 6 months ago

#6263 open defect

MPEG TS: A/V async issue when remuxing. Plays fine in MPV

Reported by: peter_b Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: mpegts async
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:

I have off-the-air recordings: H.264/AAC in an MPEG-TS (.ts) file.
For archiving, I uncompress the audio to pcm_s16le, and the container to MKV.

I have a file where audio/video goes completely out of sync after a certain position in the video.
I've isolated the culprit part of the video using "dd".

I've tried the following commands:

$FFMPEG -i $IN -c copy ${OUT}-copy.ts
$FFMPEG -i $IN -c copy -vsync passthrough ${OUT}-vsync_pass.ts
$FFMPEG -i $IN -c copy -vsync cfr ${OUT}-vsync_cfr.ts
$FFMPEG -i $IN -c copy -vsync vfr ${OUT}-vsync_vfr.ts
$FFMPEG -i $IN -c copy -vsync drop ${OUT}-vsync_drop.ts

$FFMPEG -i $IN -c:v copy -c:a pcm_s16le ${OUT}-pcm_s16le.ts
$FFMPEG -i $IN -c:v copy -c:a pcm_s16le -af "aresample=async=1000" ${OUT}-aresample.mkv

None of them produce a clean output :(

The commercial clip starting at offset 26s seems to be defect.
The file plays fine in MPV Media Player, so I guess there must be a way to fix the timestamps somehow?

Here's the complete, uncut (*) console output from the first of the above commands I've tried:

$ ffmpeg -i nd_vs_v-cut2.ts -c copy out.ts

(*) I've removed a long number of repeating "decode_slice_header error" messages, since they were identical to the first one.

ffmpeg version N-84661-g66c1c9b Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --prefix=/usr/local --enable-gpl --enable-nonfree --enable-version3 --enable-postproc --enable-swscale --enable-avfilter --enable-pthreads --enable-bzlib --enable-zlib --enable-decoder=png --enable-encoder=png --samples=../fate-suite --enable-libfreetype --enable-libschroedinger --enable-libopenjpeg --disable-decoder=jpeg2000 --enable-libvpx --enable-libvorbis --enable-libx264
  libavutil      55. 51.100 / 55. 51.100
  libavcodec     57. 86.103 / 57. 86.103
  libavformat    57. 67.100 / 57. 67.100
  libavdevice    57.  3.101 / 57.  3.101
  libavfilter     6. 78.100 /  6. 78.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
[h264 @ 0x3f6e040] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x3f6e040] decode_slice_header error
[h264 @ 0x3f6e040] no frame!
[h264 @ 0x3f6e040] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x3f6e040] decode_slice_header error
[h264 @ 0x3f6e040] no frame!
[h264 @ 0x3f6e040] non-existing PPS 0 referenced
...
...
...
Input #0, mpegts, from 'nd_vs_v-cut2.ts':
  Duration: 00:03:18.03, start: 434.773333, bitrate: 1440 kb/s
  Program 1 
    Stream #0:0[0x101]: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 59 kb/s
    Stream #0:1[0x102]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720, 30 fps, 29.97 tbr, 90k tbn, 60 tbc
    Stream #0:2[0x103]: Data: timed_id3 (ID3  / 0x20334449)
Output #0, mpegts, to 'out/out.ts':
  Metadata:
    encoder         : Lavf57.67.100
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720, q=2-31, 30 fps, 29.97 tbr, 90k tbn, 90k tbc
    Stream #0:1: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 59 kb/s
Stream mapping:
  Stream #0:1 -> #0:0 (copy)
  Stream #0:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
[mpegts @ 0x3f68360] DTS 8589934592 < 8590832488 out of orderitrate=4346.8kbits/s speed=30.8x    
[mpegts @ 0x3f68360] DTS 55209870 < 8592634286 out of order9 bitrate=2633.7kbits/s speed=46.9x    
frame= 2589 fps=1058 q=-1.0 Lsize=   35793kB time=00:01:37.25 bitrate=3014.9kbits/s speed=39.7x    
video:32062kB audio:838kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 8.791791%

Change History (8)

comment:1 Changed 6 months ago by peter_b

Uploaded the sample here:
http://download.das-werkstatt.com/pb/contribs/files/nd_vs_v-cut2.ts
A/V sync problems appear in the commercial break starting around 26s.

(Tried "ftp://upload.ffmpeg.org", but it returned a timeout...)

Last edited 6 months ago by peter_b (previous) (diff)

comment:2 Changed 6 months ago by cehoyos

  • Keywords vsync removed
  • Version changed from unspecified to git-master

Improves with -async 1.
Are you able to test the stream with a stream analyzer?

Last edited 6 months ago by cehoyos (previous) (diff)

comment:3 follow-up: Changed 6 months ago by peter_b

Stream analyzer:
Used ffprobe (show_frames/show_packets), but didn't really know what to look for.

Wanted to try "dvbsnoop", but the documentation I could find seems to already require too much in-depth knowledge of DVB-streams in order to make any sense of how to use it.

Tried "-async 1", but the commercial A/V of the commercial is out of sync until "he's a captain!" (ca. 1m55s). That behavior is as close as I get with ffmpeg.

Sorry to post it as bugreport, but if MPV is able to make sense of the stream, I guess it means that ffmpeg can somehow be able too. Especially since ffmpeg libs are the base of MPV.

Thank you very much for looking into this!
These A/V async issues are quite common with MPEG-TS recordings, and it would be great to use ffmpeg to fix them :)

comment:4 in reply to: ↑ 3 Changed 6 months ago by cehoyos

Replying to peter_b:

Stream analyzer:

My question was: Do you have access to a (possibly very expensive, most likely proprietary) soft- or hardware that can tell you if the stream in question is valid (and can be played in-sync without changing video speed)?
(I thought you have.)

These A/V async issues are quite common with MPEG-TS recordings, and it would be great to use ffmpeg to fix them :)

I wonder if the sample you provided is a good example for such common issues (it could of course be).
My suspicion is that even if ffmpeg would be better with audio holes this sample would still be problematic (and needs at least a heuristic).

comment:5 Changed 6 months ago by cehoyos

  • Component changed from undetermined to avformat
  • Reproduced by developer set
  • Status changed from new to open

A naive analysis seems to indicate there is an issue if two timestamp resets are too close to each-other.

comment:6 Changed 6 months ago by peter_b

Stream analyzer:
Understood :)
Unfortunately, no: I don't have access to them.
I'm pretty sure that data in the commercial which is non-valid.

I assumed it might just be broken beyond repair, but it "can be played in-syn without changing video speed" (at least not that I'm aware of) - by MPV.

btw: File also doesn't play properly in any version of VLC I've tried:
Linux: v2.2.5.1 Umbrella / v3.0.0 (from Videolan's PPA)
Win7: v2.2.4

comment:7 Changed 6 months ago by peter_b

Regarding the sample being problematic:
I thought it might be good to have problematic sample ;)

I've got 3 other recordings that have similar issues. Just haven't been able to isolate the culprit-position because these streams cannot properly be seeked/cut at all. And the whole recordings are >3GB each...

Last edited 6 months ago by peter_b (previous) (diff)

comment:8 Changed 6 months ago by peter_b

You think it might be worth trying Videolan's "multicat" on these files to provide more info on their quirks?

Note: See TracTickets for help on using tickets.