Opened 12 years ago

Last modified 9 years ago

#933 reopened defect

A/V desync when transcoding A/V files with timestamps non starting from 0 with -copyts

Reported by: Stefano Sabatini Owned by:
Priority: normal Component: ffmpeg
Version: git-master Keywords: timestamp desync
Cc: rogerdpack@gmail.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Using the command:

ffmpeg -vsync 0 -copyts -i columbia-intro.ts -acodec libfaac -y columbia-intro.out.ts

(same without -vsync) the A/V streams in the output file result desynched, and will result in the video being unplayable. This seems due to the first audio packet timestamp information not being correctly propagated to the encoder (and to the muxer), and being set to 0, so that ffmpeg will fill the first part of the file with audio packets, and the probe will fail since ffmpeg can't detect video frames in the probed initial part of the file.

Same result in case I use the -copyts with no -vsync. If the output file only contains video (e.g. if the -an switch is used) the output file seems to be correctly generated (playable and with correct timestamp information).

I'm attaching a sample file and log obtained by adding debugging print in ffmpeg.c.

Attachments (2)

columbia-intro.ts (985.5 KB ) - added by Stefano Sabatini 12 years ago.
log.gz (19.4 KB ) - added by Stefano Sabatini 12 years ago.
log file containing packets/frames timestamp information at various transcoding stages

Download all attachments as: .zip

Change History (17)

by Stefano Sabatini, 12 years ago

Attachment: columbia-intro.ts added

by Stefano Sabatini, 12 years ago

Attachment: log.gz added

log file containing packets/frames timestamp information at various transcoding stages

comment:1 by Stefano Sabatini, 12 years ago

Added the same file two times, please remove the duplicate.

comment:2 by Carl Eugen Hoyos, 12 years ago

Component: FFmpegundetermined

Please provide the complete, uncut console output together with your command line.

in reply to:  2 comment:3 by Stefano Sabatini, 12 years ago

Replying to cehoyos:

Please provide the complete, uncut console output together with your command line.

Not that I think that the console output gives more hints, but here it is:

$ ffmpeg -copyts -vsync 0 -an -i columbia-intro.ts -acodec libfaac -y columbia-intro.out.ts -nostats
ffmpeg version N-37013-ge7321ef Copyright (c) 2000-2012 the FFmpeg developers
  built on Jan 22 2012 17:39:37 with gcc 4.6.1
  configuration: --enable-version3 --enable-avconv --prefix=/home/stefano --enable-librtmp --enable-libx264 --enable-libfaac --disable-shared --enable-static --enable-debug=3 --enable-pthreads --enable-libvorbis --enable-gpl --enable-nonfree --enable-libmp3lame --enable-libtheora --enable-gpl --enable-x11grab --enable-frei0r --disable-optimizations --disable-mmx --enable-libfreetype --enable-libopencv --enable-libopencore-amrnb --enable-libopencore-amrnb --enable-libass
  libavutil      51. 34.101 / 51. 34.101
  libavcodec     53. 57.105 / 53. 57.105
  libavformat    53. 30.100 / 53. 30.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 59.101 /  2. 59.101
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  6.100 /  0.  6.100
  libpostproc    52.  0.100 / 52.  0.100
[mpegts @ 0x1a5dd80] max_analyze_duration 5000000 reached at 5000000
Input #0, mpegts, from 'columbia-intro.ts':
  Duration: 00:00:08.03, start: 12.928000, bitrate: 1005 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, s16, 176 kb/s
    Stream #0:2[0x102]: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, s16, 12 kb/s
[buffer @ 0x1a641d0] w:720 h:576 pixfmt:yuv420p tb:1/1000000 sar:64/45 sws_param:
[mpegts @ 0x1a633d0] muxrate VBR, pcr every 2 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'columbia-intro.out.ts':
  Metadata:
    encoder         : Lavf53.30.100
    Stream #0:0: Video: mpeg2video, yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1: Audio: aac, 48000 Hz, 5.1, s16, 128 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg2video)
  Stream #0:1 -> #0:1 (aac -> libfaac)
Press [q] to stop, [?] for help
frame=  200 fps= 44 q=31.0 Lsize=     717kB time=00:00:00.00 bitrate=   0.0kbits/s    
video:642kB audio:0kB global headers:0kB muxing overhead 11.598826%

comment:4 by Carl Eugen Hoyos, 12 years ago

Is this only reproducible using an external library (libfaac) or also with one of the native audio encoders?

What happens without -copyts? (Why are you using -copyts?)

If -vsync is not needed to reproduce the problem, please remove it from your command line.

comment:5 by Mike, 12 years ago

From what I can see in the code, -copyts does the right thing only when you also use '-acodec copy'. This command line works for me:

./ffmpeg -vsync 0 -copyts -i columbia-intro.ts -vcodec mpeg2video -b:v 3000000 -acodec copy out.ts

...and the proper time offset between the first audio and video packets is maintained.

So, the short answer is that ffmpeg (the app) doesn't yet support what you want to do.

The more detailed answer is that, if you try to actually encode the audio stream, ffmpeg.c looks like it always calls avcodec_encode_audio*() with the AVFrame pts timestamp set to AV_NOPTS_VALUE. Later on, this will cause av_interleaved_write_frame() to make up timestamps, starting at 0.

Last edited 12 years ago by Mike (previous) (diff)

comment:6 by Mike, 12 years ago

One other thing to be aware of: the mpegts mux does not honor the -copyts flag, and adds it's own constant offset to every timestamp.

comment:7 by Stefano Sabatini, 12 years ago

First of all thanks for the reply, I confirm that -copyts -acodec copy works just fine, and fixes my particular problem.

Still the problem stands when audio transcoding is required, from my analysis it looks like a problem in the ffmpeg.c code (decoded audio frame timestamp is ignored).

comment:8 by Carl Eugen Hoyos, 11 years ago

Resolution: worksforme
Status: newclosed

I was unable to reproduce the original problem - iiuc that -vsync 0 together with a start time >0 leads to unplayable output files because audio timestamps are incorrect - with both current FFmpeg and an ancient version. Testing the same version as the OP is impossible, "e7321ef" does not exist.

comment:9 by Stefano Sabatini, 11 years ago

Resolution: worksforme
Status: closedreopened

Command:

ffmpeg -copyts -i ~/s/columbia-intro.ts -acodec libfaac -y columbia-intro.out.ts
ffmpeg version N-46969-g64dc383 Copyright (c) 2000-2012 the FFmpeg developers
  built on Nov 21 2012 20:32:53 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
  configuration: --enable-pic --enable-fontconfig --enable-libschroedinger --enable-libass --enable-version3 --prefix=/home/stefano --enable-librtmp --enable-libx264 --enable-libfaac --disable-shared --enable-static --enable-debug=3 --enable-pthreads --enable-libvorbis --enable-gpl --enable-nonfree --enable-libmp3lame --enable-libtheora --enable-gpl --enable-x11grab --enable-frei0r --enable-libspeex --enable-libcaca --enable-libflite --enable-libfreetype --enable-libopencv --enable-libopencore-amrnb --enable-libopencore-amrnb --disable-optimizations --disable-mmx
  libavutil      52.  8.100 / 52.  8.100
  libavcodec     54. 74.100 / 54. 74.100
  libavformat    54. 37.100 / 54. 37.100
  libavdevice    54.  3.100 / 54.  3.100
  libavfilter     3. 23.101 /  3. 23.101
  libswscale      2.  1.102 /  2.  1.102
  libswresample   0. 17.101 /  0. 17.101
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x1ba1950] max_analyze_duration 5000000 reached at 5000000
Input #0, mpegts, from '/home/stefano/s/columbia-intro.ts':
  Duration: 00:00:08.03, start: 12.928000, bitrate: 1005 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, s16, 176 kb/s
    Stream #0:2[0x102]: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, s16, 12 kb/s
Output #0, mpegts, to 'columbia-intro.out.ts':
  Metadata:
    encoder         : Lavf54.37.100
    Stream #0:0: Video: mpeg2video, yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1: Audio: aac, 48000 Hz, 5.1, s16, 128 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg2video)
  Stream #0:1 -> #0:1 (aac -> libfaac)
Press [q] to stop, [?] for help
frame=  525 fps= 55 q=31.0 Lsize=    1545kB time=00:00:20.96 bitrate= 604.0kbits/s dup=325 drop=0    
video:1172kB audio:188kB subtitle:0 global headers:0kB muxing overhead 13.591660%

Now if you ffprobe the file with:

ffprobe columbia-intro.out.ts -show_packets -of compact | less

you see there are a lot of video packets at the beginning, and the first audio packet is in:

packet|codec_type=audio|stream_index=1|pts=1287600|pts_time=14.306667|dts=1287600|dts_time=14.306667|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=448|pos=910108|flags=K

The file is now playable with ffplay (I remember it wasn't), but if you ffprobe the file you get:

ffprobe columbia-intro.out.ts 
ffprobe version N-46969-g64dc383 Copyright (c) 2007-2012 the FFmpeg developers
  built on Nov 21 2012 20:32:53 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
  configuration: --enable-pic --enable-fontconfig --enable-libschroedinger --enable-libass --enable-version3 --prefix=/home/stefano --enable-librtmp --enable-libx264 --enable-libfaac --disable-shared --enable-static --enable-debug=3 --enable-pthreads --enable-libvorbis --enable-gpl --enable-nonfree --enable-libmp3lame --enable-libtheora --enable-gpl --enable-x11grab --enable-frei0r --enable-libspeex --enable-libcaca --enable-libflite --enable-libfreetype --enable-libopencv --enable-libopencore-amrnb --enable-libopencore-amrnb --disable-optimizations --disable-mmx
  libavutil      52.  8.100 / 52.  8.100
  libavcodec     54. 74.100 / 54. 74.100
  libavformat    54. 37.100 / 54. 37.100
  libavdevice    54.  3.100 / 54.  3.100
  libavfilter     3. 23.101 /  3. 23.101
  libswscale      2.  1.102 /  2.  1.102
  libswresample   0. 17.101 /  0. 17.101
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x34fe270] max_analyze_duration 5000000 reached at 5000000
[mpegts @ 0x34fe270] Could not find codec parameters for stream 1 (Audio: aac ([15][0][0][0] / 0x000F), 0 channels, s16): unspecified sample rate
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[NULL @ 0x35027e0] start time is not set in estimate_timings_from_pts
Input #0, mpegts, from 'columbia-intro.out.ts':
  Duration: 00:00:20.96, start: 1.400000, bitrate: 604 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 0 channels, s16

so the problem is still there.

Last edited 11 years ago by Stefano Sabatini (previous) (diff)

comment:10 by Carl Eugen Hoyos, 11 years ago

Is this only reproducible with -acodec libfaac or also with -strict -2 -acodec aac ?

in reply to:  10 ; comment:11 by Stefano Sabatini, 11 years ago

Replying to cehoyos:

Is this only reproducible with -acodec libfaac or also with -strict -2 -acodec aac ?

Tracked the problem to do_video_out() in ffmpeg.c, if vsync != passthrough the input value is ignored (this is significantly different from what happened 10 months ago), with the command:

ffmpeg -vsync passthrough -copyts -i columbia-intro.ts -acodec libfaac -y columbia-intro.out.ts

I got synched timestamps, yet there is a mismatch of around ~1s between input and output timestamps.
Also documentation for -copyts should be fixed (it only disables input timestamps processing).

Last edited 11 years ago by Stefano Sabatini (previous) (diff)

comment:12 by Stefano Sabatini, 11 years ago

Component: undeterminedFFmpeg
Keywords: mpegts removed

in reply to:  11 comment:13 by Carl Eugen Hoyos, 11 years ago

Replying to saste:

ffmpeg -vsync passthrough -copyts -i columbia-intro.ts -acodec libfaac -y columbia-intro.out.ts

Is this only reproducible with -acodec libfaac or also with -strict -2 -acodec aac ?

comment:14 by Michael Niedermayer, 10 years ago

Keywords: desync added; desynch removed

comment:15 by Roger Pack, 9 years ago

Cc: rogerdpack@gmail.com added
Summary: A/V desynch when transcoding A/V files with timestamps non starting from 0 with -copytsA/V desync when transcoding A/V files with timestamps non starting from 0 with -copyts

Is this still a problem?

Note: See TracTickets for help on using tickets.