Opened 5 years ago

Last modified 15 months ago

#933 reopened defect

A/V desync when transcoding A/V files with timestamps non starting from 0 with -copyts

Reported by: saste Owned by:
Priority: normal Component: ffmpeg
Version: git-master Keywords: timestamp desync
Cc: rogerdpack@gmail.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Using the command:

ffmpeg -vsync 0 -copyts -i columbia-intro.ts -acodec libfaac -y columbia-intro.out.ts

(same without -vsync) the A/V streams in the output file result desynched, and will result in the video being unplayable. This seems due to the first audio packet timestamp information not being correctly propagated to the encoder (and to the muxer), and being set to 0, so that ffmpeg will fill the first part of the file with audio packets, and the probe will fail since ffmpeg can't detect video frames in the probed initial part of the file.

Same result in case I use the -copyts with no -vsync. If the output file only contains video (e.g. if the -an switch is used) the output file seems to be correctly generated (playable and with correct timestamp information).

I'm attaching a sample file and log obtained by adding debugging print in ffmpeg.c.

Attachments (2)

columbia-intro.ts (985.5 KB) - added by saste 5 years ago.
log.gz (19.4 KB) - added by saste 5 years ago.
log file containing packets/frames timestamp information at various transcoding stages

Download all attachments as: .zip

Change History (17)

Changed 5 years ago by saste

Changed 5 years ago by saste

log file containing packets/frames timestamp information at various transcoding stages

comment:1 Changed 5 years ago by saste

Added the same file two times, please remove the duplicate.

comment:2 follow-up: Changed 5 years ago by cehoyos

  • Component changed from FFmpeg to undetermined

Please provide the complete, uncut console output together with your command line.

comment:3 in reply to: ↑ 2 Changed 5 years ago by saste

Replying to cehoyos:

Please provide the complete, uncut console output together with your command line.

Not that I think that the console output gives more hints, but here it is:

$ ffmpeg -copyts -vsync 0 -an -i columbia-intro.ts -acodec libfaac -y columbia-intro.out.ts -nostats
ffmpeg version N-37013-ge7321ef Copyright (c) 2000-2012 the FFmpeg developers
  built on Jan 22 2012 17:39:37 with gcc 4.6.1
  configuration: --enable-version3 --enable-avconv --prefix=/home/stefano --enable-librtmp --enable-libx264 --enable-libfaac --disable-shared --enable-static --enable-debug=3 --enable-pthreads --enable-libvorbis --enable-gpl --enable-nonfree --enable-libmp3lame --enable-libtheora --enable-gpl --enable-x11grab --enable-frei0r --disable-optimizations --disable-mmx --enable-libfreetype --enable-libopencv --enable-libopencore-amrnb --enable-libopencore-amrnb --enable-libass
  libavutil      51. 34.101 / 51. 34.101
  libavcodec     53. 57.105 / 53. 57.105
  libavformat    53. 30.100 / 53. 30.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 59.101 /  2. 59.101
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  6.100 /  0.  6.100
  libpostproc    52.  0.100 / 52.  0.100
[mpegts @ 0x1a5dd80] max_analyze_duration 5000000 reached at 5000000
Input #0, mpegts, from 'columbia-intro.ts':
  Duration: 00:00:08.03, start: 12.928000, bitrate: 1005 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, s16, 176 kb/s
    Stream #0:2[0x102]: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, s16, 12 kb/s
[buffer @ 0x1a641d0] w:720 h:576 pixfmt:yuv420p tb:1/1000000 sar:64/45 sws_param:
[mpegts @ 0x1a633d0] muxrate VBR, pcr every 2 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'columbia-intro.out.ts':
  Metadata:
    encoder         : Lavf53.30.100
    Stream #0:0: Video: mpeg2video, yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1: Audio: aac, 48000 Hz, 5.1, s16, 128 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg2video)
  Stream #0:1 -> #0:1 (aac -> libfaac)
Press [q] to stop, [?] for help
frame=  200 fps= 44 q=31.0 Lsize=     717kB time=00:00:00.00 bitrate=   0.0kbits/s    
video:642kB audio:0kB global headers:0kB muxing overhead 11.598826%

comment:4 Changed 5 years ago by cehoyos

Is this only reproducible using an external library (libfaac) or also with one of the native audio encoders?

What happens without -copyts? (Why are you using -copyts?)

If -vsync is not needed to reproduce the problem, please remove it from your command line.

comment:5 Changed 5 years ago by mjs973

From what I can see in the code, -copyts does the right thing only when you also use '-acodec copy'. This command line works for me:

./ffmpeg -vsync 0 -copyts -i columbia-intro.ts -vcodec mpeg2video -b:v 3000000 -acodec copy out.ts

...and the proper time offset between the first audio and video packets is maintained.

So, the short answer is that ffmpeg (the app) doesn't yet support what you want to do.

The more detailed answer is that, if you try to actually encode the audio stream, ffmpeg.c looks like it always calls avcodec_encode_audio*() with the AVFrame pts timestamp set to AV_NOPTS_VALUE. Later on, this will cause av_interleaved_write_frame() to make up timestamps, starting at 0.

Last edited 5 years ago by mjs973 (previous) (diff)

comment:6 Changed 5 years ago by mjs973

One other thing to be aware of: the mpegts mux does not honor the -copyts flag, and adds it's own constant offset to every timestamp.

comment:7 Changed 5 years ago by saste

First of all thanks for the reply, I confirm that -copyts -acodec copy works just fine, and fixes my particular problem.

Still the problem stands when audio transcoding is required, from my analysis it looks like a problem in the ffmpeg.c code (decoded audio frame timestamp is ignored).

comment:8 Changed 4 years ago by cehoyos

  • Resolution set to worksforme
  • Status changed from new to closed

I was unable to reproduce the original problem - iiuc that -vsync 0 together with a start time >0 leads to unplayable output files because audio timestamps are incorrect - with both current FFmpeg and an ancient version. Testing the same version as the OP is impossible, "e7321ef" does not exist.

comment:9 Changed 4 years ago by saste

  • Resolution worksforme deleted
  • Status changed from closed to reopened

Command:

ffmpeg -copyts -i ~/s/columbia-intro.ts -acodec libfaac -y columbia-intro.out.ts
ffmpeg version N-46969-g64dc383 Copyright (c) 2000-2012 the FFmpeg developers
  built on Nov 21 2012 20:32:53 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
  configuration: --enable-pic --enable-fontconfig --enable-libschroedinger --enable-libass --enable-version3 --prefix=/home/stefano --enable-librtmp --enable-libx264 --enable-libfaac --disable-shared --enable-static --enable-debug=3 --enable-pthreads --enable-libvorbis --enable-gpl --enable-nonfree --enable-libmp3lame --enable-libtheora --enable-gpl --enable-x11grab --enable-frei0r --enable-libspeex --enable-libcaca --enable-libflite --enable-libfreetype --enable-libopencv --enable-libopencore-amrnb --enable-libopencore-amrnb --disable-optimizations --disable-mmx
  libavutil      52.  8.100 / 52.  8.100
  libavcodec     54. 74.100 / 54. 74.100
  libavformat    54. 37.100 / 54. 37.100
  libavdevice    54.  3.100 / 54.  3.100
  libavfilter     3. 23.101 /  3. 23.101
  libswscale      2.  1.102 /  2.  1.102
  libswresample   0. 17.101 /  0. 17.101
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x1ba1950] max_analyze_duration 5000000 reached at 5000000
Input #0, mpegts, from '/home/stefano/s/columbia-intro.ts':
  Duration: 00:00:08.03, start: 12.928000, bitrate: 1005 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, s16, 176 kb/s
    Stream #0:2[0x102]: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, 5.1, s16, 12 kb/s
Output #0, mpegts, to 'columbia-intro.out.ts':
  Metadata:
    encoder         : Lavf54.37.100
    Stream #0:0: Video: mpeg2video, yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 25 tbc
    Stream #0:1: Audio: aac, 48000 Hz, 5.1, s16, 128 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg2video)
  Stream #0:1 -> #0:1 (aac -> libfaac)
Press [q] to stop, [?] for help
frame=  525 fps= 55 q=31.0 Lsize=    1545kB time=00:00:20.96 bitrate= 604.0kbits/s dup=325 drop=0    
video:1172kB audio:188kB subtitle:0 global headers:0kB muxing overhead 13.591660%

Now if you ffprobe the file with:

ffprobe columbia-intro.out.ts -show_packets -of compact | less

you see there are a lot of video packets at the beginning, and the first audio packet is in:

packet|codec_type=audio|stream_index=1|pts=1287600|pts_time=14.306667|dts=1287600|dts_time=14.306667|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=448|pos=910108|flags=K

The file is now playable with ffplay (I remember it wasn't), but if you ffprobe the file you get:

ffprobe columbia-intro.out.ts 
ffprobe version N-46969-g64dc383 Copyright (c) 2007-2012 the FFmpeg developers
  built on Nov 21 2012 20:32:53 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
  configuration: --enable-pic --enable-fontconfig --enable-libschroedinger --enable-libass --enable-version3 --prefix=/home/stefano --enable-librtmp --enable-libx264 --enable-libfaac --disable-shared --enable-static --enable-debug=3 --enable-pthreads --enable-libvorbis --enable-gpl --enable-nonfree --enable-libmp3lame --enable-libtheora --enable-gpl --enable-x11grab --enable-frei0r --enable-libspeex --enable-libcaca --enable-libflite --enable-libfreetype --enable-libopencv --enable-libopencore-amrnb --enable-libopencore-amrnb --disable-optimizations --disable-mmx
  libavutil      52.  8.100 / 52.  8.100
  libavcodec     54. 74.100 / 54. 74.100
  libavformat    54. 37.100 / 54. 37.100
  libavdevice    54.  3.100 / 54.  3.100
  libavfilter     3. 23.101 /  3. 23.101
  libswscale      2.  1.102 /  2.  1.102
  libswresample   0. 17.101 /  0. 17.101
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x34fe270] max_analyze_duration 5000000 reached at 5000000
[mpegts @ 0x34fe270] Could not find codec parameters for stream 1 (Audio: aac ([15][0][0][0] / 0x000F), 0 channels, s16): unspecified sample rate
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[NULL @ 0x35027e0] start time is not set in estimate_timings_from_pts
Input #0, mpegts, from 'columbia-intro.out.ts':
  Duration: 00:00:20.96, start: 1.400000, bitrate: 604 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 0 channels, s16

so the problem is still there.

Last edited 4 years ago by saste (previous) (diff)

comment:10 follow-up: Changed 4 years ago by cehoyos

Is this only reproducible with -acodec libfaac or also with -strict -2 -acodec aac ?

comment:11 in reply to: ↑ 10 ; follow-up: Changed 4 years ago by saste

Replying to cehoyos:

Is this only reproducible with -acodec libfaac or also with -strict -2 -acodec aac ?

Tracked the problem to do_video_out() in ffmpeg.c, if vsync != passthrough the input value is ignored (this is significantly different from what happened 10 months ago), with the command:

ffmpeg -vsync passthrough -copyts -i ~/s/columbia-intro.ts -acodec libfaac -y columbia-intro.out.ts

I got synched timestamps, yet there is a mismatch of around ~1s between input and output timestamps.
Also documentation for -copyts should be fixed (it only disables input timestamps processing).

Version 1, edited 4 years ago by saste (previous) (next) (diff)

comment:12 Changed 4 years ago by saste

  • Component changed from undetermined to FFmpeg
  • Keywords mpegts removed

comment:13 in reply to: ↑ 11 Changed 4 years ago by cehoyos

Replying to saste:

ffmpeg -vsync passthrough -copyts -i columbia-intro.ts -acodec libfaac -y columbia-intro.out.ts

Is this only reproducible with -acodec libfaac or also with -strict -2 -acodec aac ?

comment:14 Changed 3 years ago by michael

  • Keywords desync added; desynch removed

comment:15 Changed 15 months ago by rogerdpack

  • Cc rogerdpack@gmail.com added
  • Summary changed from A/V desynch when transcoding A/V files with timestamps non starting from 0 with -copyts to A/V desync when transcoding A/V files with timestamps non starting from 0 with -copyts

Is this still a problem?

Note: See TracTickets for help on using tickets.