Opened 12 years ago

Last modified 8 years ago

#2400 open defect

A/V desync for separate video and aac in mp4 input files

Reported by: chinshou Owned by:
Priority: important Component: undetermined
Version: git-master Keywords: mov aac regression desync
Cc: Michael Niedermayer Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

executing following command with the two videos I attached.

ffmpeg -i aaa.mp4 -i bbb.mp4 test.mp4

C:\Conversion>ffmpeg -i aaa.mp4 -i bbb.mp4 test.mp4
ffmpeg version N-51211-ge0e8c20 Copyright (c) 2000-2013 the FFmpeg developers

built on Mar 22 2013 08:56:38 with gcc 4.7.2 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av

isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libg
sm --enable-libilbc --enable-libmp3lame --enable-libopencore-amrnb --enable-libo
pencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-li
bschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-lib
twolame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enabl
e-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib

libavutil 52. 22.101 / 52. 22.101
libavcodec 55. 1.100 / 55. 1.100
libavformat 55. 0.100 / 55. 0.100
libavdevice 55. 0.100 / 55. 0.100
libavfilter 3. 48.100 / 3. 48.100
libswscale 2. 2.100 / 2. 2.100
libswresample 0. 17.102 / 0. 17.102
libpostproc 52. 2.100 / 52. 2.100

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'aaa.mp4':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf54.59.102

Duration: 00:00:08.71, start: 0.000000, bitrate: 245 kb/s

Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1366x768,

241 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc

Metadata:

handler_name : VideoHandler

Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb.mp4':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: isomiso2mp41
encoder : Lavf54.59.102

Duration: 00:00:00.13, start: 0.023220, bitrate: 9029 kb/s

Stream #1:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 89

40 kb/s

Metadata:

handler_name : SoundHandler

File 'test.mp4' already exists. Overwrite ? [y/N] y
using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 024adac0] profile High, level 3.2
[libx264 @ 024adac0] 264 - core 130 r2274 c832fe9 - H.264/MPEG-4 AVC codec - Cop
yleft 2003-2013 - ​http://www.videolan.org/x264.html - options: cabac=1 ref=3 deb
lock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 m
e_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chro
ma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 i
nterlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1
b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenec
ut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=
0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf55.0.100
Stream #0:0(und): Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1366x768, q

=-1--1, 30k tbn, 29.97 tbc

Metadata:

handler_name : VideoHandler

Stream #0:1(und): Audio: aac ([64][0][0][0] / 0x0040), 44100 Hz, stereo, s16

, 128 kb/s

Metadata:

handler_name : SoundHandler

Stream mapping:

Stream #0:0 -> #0:0 (h264 -> libx264)
Stream #1:0 -> #0:1 (aac -> libvo_aacenc)

Press [q] to stop, ? for help
frame= 54 fps=0.0 q=29.0 size= 0kB time=00:00:00.06 bitrate= 5.8kbits/
frame= 80 fps= 76 q=29.0 size= 219kB time=00:00:00.93 bitrate=1917.7kbits/
frame= 114 fps= 72 q=29.0 size= 219kB time=00:00:02.06 bitrate= 866.1kbits/
frame= 148 fps= 71 q=29.0 size= 219kB time=00:00:03.20 bitrate= 559.3kbits/
frame= 187 fps= 72 q=29.0 size= 219kB time=00:00:04.50 bitrate= 397.7kbits/
frame= 226 fps= 72 q=29.0 size= 219kB time=00:00:05.80 bitrate= 308.6kbits/
frame= 261 fps= 63 q=2686559.0 Lsize= 390kB time=00:00:08.64 bitrate= 369.9
kbits/s
video:251kB audio:133kB subtitle:0 global headers:0kB muxing overhead 1.517758%
[libx264 @ 024adac0] frame I:2 Avg QP:20.77 size: 84310
[libx264 @ 024adac0] frame P:68 Avg QP:24.08 size: 1098
[libx264 @ 024adac0] frame B:191 Avg QP:33.16 size: 70
[libx264 @ 024adac0] consecutive B-frames: 1.9% 1.5% 0.0% 96.6%
[libx264 @ 024adac0] mb I I16..4: 65.2% 4.7% 30.2%
[libx264 @ 024adac0] mb P I16..4: 0.8% 0.0% 0.3% P16..4: 0.6% 0.1% 0.0%

0.0% 0.0% skip:98.1%

[libx264 @ 024adac0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.7% 0.0% 0.0%

direct: 0.0% skip:99.2% L0:48.3% L1:51.6% BI: 0.1%

[libx264 @ 024adac0] 8x8 transform intra:3.7% inter:46.9%
[libx264 @ 024adac0] coded y,uvDC,uvAC intra: 20.3% 19.0% 16.6% inter: 0.1% 0.1%

0.0%

[libx264 @ 024adac0] i16 v,h,dc,p: 62% 37% 0% 0%
[libx264 @ 024adac0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 74% 12% 12% 0% 0% 0% 0%

0% 1%

[libx264 @ 024adac0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 34% 39% 12% 2% 2% 2% 3%

2% 3%

[libx264 @ 024adac0] i8c dc,h,v,p: 67% 23% 9% 0%
[libx264 @ 024adac0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 024adac0] ref P L0: 79.1% 12.4% 6.1% 2.4%
[libx264 @ 024adac0] ref B L0: 60.7% 38.5% 0.8%
[libx264 @ 024adac0] ref B L1: 95.4% 4.6%
[libx264 @ 024adac0] kb/s:235.67

then executing ffplay test.mp4 to view the result mp4, you will notice that audio not syndicate with video from the pronunciation of "ffffff" and input "fffffff" via keyboard .

But if we merge the audio and video to a avi file , the audio/video syndicated correctly.

ffmpeg -i aaa.mp4 -i bbb.mp4 test.avi.

I tested with old release and found that ​http://ffmpeg.zeranoe.com/builds/win32/dev/ffmpeg-git-e01f478-win32-dev.7z built on 2012/03/20 is still ok. From ​http://ffmpeg.zeranoe.com/builds/win32/dev/ffmpeg-git-4082198-win32-dev.7z built on 2012/03/25 the problem was introduced.

I compared the source code between 03/20 and 03/25 and suspect the audio encode api in codec switch from encode to encode2 interface caused the problem.


Attachments (4)

aaa.mp4​ (261.0 KB ) - added by chinshou 12 years ago.
bbb.mp4​ (137.8 KB ) - added by chinshou 12 years ago.
test.mp4​ (390.2 KB ) - added by chinshou 12 years ago.
test.avi​ (2.0 MB ) - added by chinshou 12 years ago.

Change History (12)

by chinshou, 12 years ago

Attachment: aaa.mp4​ added

by chinshou, 12 years ago

Attachment: bbb.mp4​ added

by chinshou, 12 years ago

Attachment: test.mp4​ added

by chinshou, 12 years ago

Attachment: test.avi​ added

comment:1 by Carl Eugen Hoyos, 12 years ago

Is this problem only reproducible using external libraries or also using "-vcodec mpeg4 -strict -2 -acodec aac"?

comment:2 by chinshou, 12 years ago

Hello,

same problem with the internal codec.

C:\Conversion>ffmpeg -i aaa.mp4 -i bbb.mp4 -vcodec mpeg4 -strict -2 -acodec aac
test.mp4
ffmpeg version N-51211-ge0e8c20 Copyright (c) 2000-2013 the FFmpeg developers
  built on Mar 22 2013 08:56:38 with gcc 4.7.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc --enable-libmp3lame --enable-libopencore-amrnb --enable-ibopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-ibschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      52. 22.101 / 52. 22.101
  libavcodec     55.  1.100 / 55.  1.100
  libavformat    55.  0.100 / 55.  0.100
  libavdevice    55.  0.100 / 55.  0.100
  libavfilter     3. 48.100 /  3. 48.100
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'aaa.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf54.59.102
  Duration: 00:00:08.71, start: 0.000000, bitrate: 245 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1366x768, 241 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      handler_name    : VideoHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf54.59.102
  Duration: 00:00:00.13, start: 0.023220, bitrate: 9029 kb/s
    Stream #1:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 8940 kb/s
    Metadata:
      handler_name    : SoundHandler
File 'test.mp4' already exists. Overwrite ? [y/N] y
Output #0, mp4, to 'test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.0.100
    Stream #0:0(und): Video: mpeg4 ( [0][0][0] / 0x0020), yuv420p, 1366x768, q=2-31, 200 kb/s, 30k tbn, 29.97 tbc
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac ([64][0][0][0] / 0x0040), 44100 Hz, stereo, fltp, 128 kb/s
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> mpeg4)
  Stream #1:0 -> #0:1 (aac -> aac)
Press [q] to stop, [?] for help
frame=   10 fps=0.0 q=6.2 size=     343kB time=00:00:00.33 bitrate=8410.3kbits/s
frame=   93 fps= 93 q=31.0 size=     343kB time=00:00:03.10 bitrate= 904.3kbits/s
frame=  180 fps=120 q=31.0 size=     343kB time=00:00:06.00 bitrate= 467.2kbits/s
frame=  261 fps=130 q=31.0 Lsize=    1986kB time=00:00:08.70 bitrate=1868.6kbits/s
video:1859kB audio:124kB subtitle:0 global headers:0kB muxing overhead 0.196587%
Last edited 12 years ago by Carl Eugen Hoyos (previous) (diff)

comment:3 by Carl Eugen Hoyos, 12 years ago

Keywords: regression added
Priority: normal → important
Reproduced by developer: set
Status: new → open

From a user's perspective, this is a regression since 967facb (depending on the used audio encoder), the original file bbb.mp4 shows no useful timestamps with MPlayer and ffplay:

$ time ffplay bbb.mp4
ffplay version N-51293-g67607e2 Copyright (c) 2003-2013 the FFmpeg developers
  built on Mar 25 2013 19:24:09 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 22.101 / 52. 22.101
  libavcodec     55.  1.100 / 55.  1.100
  libavformat    55.  0.100 / 55.  0.100
  libavdevice    55.  0.100 / 55.  0.100
  libavfilter     3. 48.102 /  3. 48.102
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb.mp4':sq=    0B f=0/0
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf54.59.102
  Duration: 00:00:00.13, start: 0.023220, bitrate: 9029 kb/s
    Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 8940 kb/s
    Metadata:
      handler_name    : SoundHandler
   0.07 A-V:  0.000 fd=   0 aq=    3KB vq=    0KB sq=    0B f=0/0

real    0m5.974s
user    0m0.127s
sys     0m0.090s
Last edited 12 years ago by Carl Eugen Hoyos (previous) (diff)

comment:4 by Carl Eugen Hoyos, 12 years ago

Keywords: mov aac added

comment:5 by Carl Eugen Hoyos, 12 years ago

Keywords: desync added
Summary: when merging audio and video to mp4 , the audio / video not syncdiated correctly → A/V desync for separate video and aac in mp4 input files

comment:6 by feelart, 11 years ago

FYI: Still buggy with latest build

C:\Users\Angel\Downloads>ffmpeg -i aaa.mp4 -i bbb.mp4 test.mp4
ffmpeg version N-54143-gb6ce50a Copyright (c) 2000-2013 the FFmpeg developers
  built on Jun 22 2013 15:32:43 with gcc 4.7.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --
enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enab
le-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --e
nable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-lib
theora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264
 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      52. 37.101 / 52. 37.101
  libavcodec     55. 16.100 / 55. 16.100
  libavformat    55.  9.100 / 55.  9.100
  libavdevice    55.  2.100 / 55.  2.100
  libavfilter     3. 77.101 /  3. 77.101
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'aaa.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf54.59.102
  Duration: 00:00:08.71, start: 0.000000, bitrate: 245 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1366x768, 241 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59
.94 tbc
    Metadata:
      handler_name    : VideoHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf54.59.102
  Duration: 00:00:00.13, start: 0.023220, bitrate: 9029 kb/s
    Stream #1:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 8940 kb/s
    Metadata:
      handler_name    : SoundHandler
[libx264 @ 00000000003bf2c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 00000000003bf2c0] profile High, level 3.2
[libx264 @ 00000000003bf2c0] 264 - core 133 r2334 a3ac64b - H.264/MPEG-4 AVC codec - Copyleft 2003-2013 - http://www.videola
n.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 m
e_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=
1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0
 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=
1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.9.100
    Stream #0:0(und): Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 1366x768, q=-1--1, 30k tbn, 29.97 tbc
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (libvo_aacenc) ([64][0][0][0] / 0x0040), 44100 Hz, stereo, s16, 128 kb/s
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> libx264)
  Stream #1:0 -> #0:1 (aac -> libvo_aacenc)
Press [q] to stop, [?] for help
frame=  261 fps= 59 q=-1.0 Lsize=     390kB time=00:00:08.64 bitrate= 369.9kbits/s
video:251kB audio:133kB subtitle:0 global headers:0kB muxing overhead 1.517858%
[libx264 @ 00000000003bf2c0] frame I:2     Avg QP:20.77  size: 84312
[libx264 @ 00000000003bf2c0] frame P:68    Avg QP:24.08  size:  1098
[libx264 @ 00000000003bf2c0] frame B:191   Avg QP:33.13  size:    69
[libx264 @ 00000000003bf2c0] consecutive B-frames:  1.9%  1.5%  0.0% 96.6%
[libx264 @ 00000000003bf2c0] mb I  I16..4: 65.1%  4.7% 30.2%
[libx264 @ 00000000003bf2c0] mb P  I16..4:  0.8%  0.0%  0.3%  P16..4:  0.6%  0.1%  0.0%  0.0%  0.0%    skip:98.1%
[libx264 @ 00000000003bf2c0] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.7%  0.0%  0.0%  direct: 0.0%  skip:99.2%  L0:47.9%
L1:52.0% BI: 0.1%
[libx264 @ 00000000003bf2c0] 8x8 transform intra:3.7% inter:46.7%
[libx264 @ 00000000003bf2c0] coded y,uvDC,uvAC intra: 20.2% 19.0% 16.6% inter: 0.1% 0.1% 0.0%
[libx264 @ 00000000003bf2c0] i16 v,h,dc,p: 63% 37%  0%  0%
[libx264 @ 00000000003bf2c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 74% 13% 11%  0%  0%  0%  0%  0%  1%
[libx264 @ 00000000003bf2c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 34% 39% 12%  2%  2%  2%  3%  2%  3%
[libx264 @ 00000000003bf2c0] i8c dc,h,v,p: 67% 23%  9%  0%
[libx264 @ 00000000003bf2c0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 00000000003bf2c0] ref P L0: 78.3% 12.8%  6.6%  2.4%
[libx264 @ 00000000003bf2c0] ref B L0: 60.1% 39.0%  0.8%
[libx264 @ 00000000003bf2c0] ref B L1: 95.5%  4.5%
[libx264 @ 00000000003bf2c0] kb/s:235.65

C:\Users\Angel\Downloads>

comment:7 by Michael Niedermayer, 8 years ago

The STTS atom of bbb.mp4 looks broken
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2fbeb40] track[0].stts.entries = 3
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2fbeb40] sample_count=4, sample_duration=1024
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2fbeb40] sample_count=360, sample_duration=1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2fbeb40] sample_count=1, sample_duration=1024

This results in bad timestamps for the audio

comment:8 by Michael Niedermayer, 8 years ago

Cc: Michael Niedermayer added
Note: See TracTickets for help on using tickets.