Opened 5 years ago

Closed 15 months ago

#424 closed defect (fixed)

A/V desync on transport stream

Reported by: cehoyos Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: h264 mpegts videolan desync
Cc: mjs973@optonline.net Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: yes

Description

http://streams.videolan.org/issues/5203/ts720p_i_fr.ts

The stream contains 720p60 H264 video and mp2 stereo sound. When transcoded with FFmpeg, the resulting file is non-interleaved, i.e. the 15 seconds of sound come first (without picture). MPlayer's mpegts demuxer indicates that the stream does start with mostly identical timestamps for audio and video (but plays the video too fast).
FFmpeg seems to believe that audio starts with timestamps ~130 and video with ~260.

$ ffmpeg -i ts720p_i_fr.ts -qscale 5 out.avi
ffmpeg version N-32263-g3584b01, Copyright (c) 2000-2011 the FFmpeg developers
  built on Sep  1 2011 12:14:35 with gcc 4.5.3
  configuration: --cc='/usr/local/gcc-4.5.3/bin/gcc -m32'
  libavutil    51. 14. 0 / 51. 14. 0
  libavcodec   53. 12. 0 / 53. 12. 0
  libavformat  53. 10. 0 / 53. 10. 0
  libavdevice  53.  3. 0 / 53.  3. 0
  libavfilter   2. 37. 0 /  2. 37. 0
  libswscale    2.  0. 0 /  2.  0. 0
[mpegts @ 0x8c963a0] Continuity Check Failed
[mpegts @ 0x8c963a0] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x8c963a0] PES packet size mismatch
Input #0, mpegts, from 'ts720p_i_fr.ts':
  Duration: 00:00:14.61, start: 130.272000, bitrate: 9237 kb/s
  Program 16
    Metadata:
      service_name    : PMR COBRA 1213
      service_provider: PMR
    Stream #0.0[0x109]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 60 fps, 60 tbr, 90k tbn, 120 tbc
    Stream #0.1[0x103]: Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16, 128 kb/s
[buffer @ 0x8d86a00] w:1280 h:720 pixfmt:yuv420p tb:1/1000000 sar:1/1 sws_param:
Incompatible sample format 's16' for codec 'ac3', auto-selecting format 'flt'
Output #0, avi, to 'out.avi':
  Metadata:
    ISFT            : Lavf53.10.0
    Stream #0.0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 60 tbn, 60 tbc
    Stream #0.1: Audio: ac3 ([0] [0][0] / 0x2000), 48000 Hz, stereo, flt, 128 kb/s
Stream mapping:
  Stream #0.0 -> #0.0 (h264 -> mpeg4)
  Stream #0.1 -> #0.1 (mp2 -> ac3)
Press [q] to stop, [?] for help
[mpegts @ 0x8c963a0] Continuity Check Failed
    Last message repeated 1 times   236kB time=00:00:14.27 bitrate= 135.6kbits/s
[mpegts @ 0x8c963a0] PES packet size mismatch
[h264 @ 0x8c9b6e0] error while decoding MB 76 41, bytestream (-5)
[h264 @ 0x8c9b6e0] concealing 293 DC, 293 AC, 293 MV errors
frame=  852 fps= 31 q=5.0 Lsize=  177161kB time=00:00:14.62 bitrate=99241.4kbits/s
video:12002kB audio:228kB global headers:0kB muxing overhead 1348.534109%

Change History (12)

comment:1 in reply to: ↑ description Changed 5 years ago by wg

Replying to cehoyos:

The stream contains 720p60 H264 video and mp2 stereo sound. When transcoded with FFmpeg, the resulting file is non-interleaved, i.e. the 15 seconds of sound come first (without picture). MPlayer's mpegts demuxer indicates that the stream does start with mostly identical timestamps for audio and video (but plays the video too fast).
FFmpeg seems to believe that audio starts with timestamps ~130 and video with ~260.

ffprobe shows this file to have no video timestamps at all, so what should ffmpeg do except waiting for such a timestamp?
Please try

ffmpeg_g -i ts720p_i_fr.ts -qscale 5 out.avi -map 0.0:0.1 -map 0.1

maybe it does more like what you want. I could hear no sound neither in the original nor in the transcoded file.

comment:2 Changed 5 years ago by cehoyos

The sample does contain 14.6 seconds of sound. This is nearly exactly the intended video playtime.

comment:3 Changed 5 years ago by mjs973

Hi Carl. The bitstream looks good to me. The transport stream DTS/PTS are all good. The video has no B frames (I,P only), and uses PTS==DTS (which is legal).

This appears to be the same bug I already fixed in trac #88. My own media player, which includes the trac #88 patch, plays this stream fine. I hear audio immediately at the start.

I don't have time to look at latest git today, but my guess is that compute_pkt_fields() is corrupting the perfectly-good DTS/PTS timestamps passed to it.

Mike

P.S. the audio volume is really, really low in this clip. Peaks are at only 5% of full-scale.

comment:4 Changed 5 years ago by mjs973

  • Status changed from new to open

I looked at this sample running the latest videolan git code, downloaded 2011-09-10. For me, ffprobe does show dts/pts timestamps in the file, but the PTS timestamps shown by ffprobe are wrong. The PTS values displayed are multiplied by approximately 2.

I confirmed this pts corruption is being done by compute_pkt_fields(), as my previous comment guessed. The line that changes pts to the wrong value is line 1044 in libavformat/utils.c:

pkt->pts = pkt->dts + pc->pts_dts_delta * num / den;

No time to debug this further right now.

comment:5 Changed 5 years ago by cehoyos

I can confirm that changing that line into "pkt->pts = pkt->dts" fixes transoding / playing this sample.

Forcing low_delay (and your patch from ticket #88) does not help, so this seems not to be the reason for the problem.

comment:6 Changed 5 years ago by mjs973

  • Analyzed by developer set
  • Cc mjs973@optonline.net added
  • Component changed from undetermined to avcodec
  • Reproduced by developer set

I analyzed this file in greater depth. The ffmpeg h264 decoder is doing the right thing, it is the bitstream that is wrong. I see two problems, which are completely unrelated.

  1. The video stream was re-muxed at the the wrong frame rate. The original h264 elementary stream is 60.0 frames per second, but the .ts container was created using 30.0 frames per second.
  1. This h264 elementary stream contains "buffering period SEI" and "pic timing SEI". The value in the pic timing dpb_output_delay field is completely wrong. It is typically a small interger value like "2", but in this sample it is 15628 (and it increments by 2 for each frame.) I'd guess this is a firmware error in the camcorder that created the compressed video stream.

These SEI are optional in h264, and they are not needed for this stream. So, a workaround for this camcorder bug would be to re-mux the stream and tell the re-muxer to remove all the pic timing SEI NALs.

I don't see an easy way for the h264 decoder to work around this bug. The way I read the h.264 spec, such a large value for dpb_output_buffer is legal, but it is very, very unusual.

Version 0, edited 5 years ago by mjs973 (next)

comment:7 Changed 5 years ago by cehoyos

The stream plays fine with WMP 12 (and mplayer -demuxer mpegts -mc 100), so I believe decoding should be improved.

comment:8 Changed 5 years ago by cehoyos

  • Keywords H264 TS added

comment:9 Changed 4 years ago by cehoyos

  • Keywords h264 mpegts videolan added; H264 TS removed

Ticket #1715 may be related.

comment:10 Changed 4 years ago by cehoyos

Possibly related to ticket #1866.

comment:11 Changed 4 years ago by cehoyos

  • Keywords desync added

comment:12 Changed 15 months ago by cehoyos

  • Resolution set to fixed
  • Status changed from open to closed

Fixed by Michael in 1e5271a9fd6ddcceb083f2185a4bbd8d44c9a813 (FFmpeg 2.1), see also ticket #2143.

Note: See TracTickets for help on using tickets.