Opened 7 years ago

Closed 10 months ago

#6664 closed defect (invalid)

ffmpeg concat audio not in sync

Reported by: Tony Cast Owned by:
Priority: normal Component: undetermined
Version: unspecified Keywords: concat
Cc: tony@tonycast.com Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug: when using concat in 3.3.4 the audio is out of sync.
How to reproduce:

% ffmpeg -threads 0 -f concat -safe 0 -i /tmp/files.txt -vf "scale=iw*min(560/iw\,320/ih):ih*min(560/iw\,320/ih), pad=560:320:(560-iw*min(560/iw\,320/ih))/2:(320-ih*min(560/iw\,320/ih))/2" -preset veryslow -crf 15 -acodec libmp3lame -af aresample=resampler=soxr -ar 44100 -aspect 16:9 -async 1 -qp 20 -pix_fmt yuv420p -framerate 30 -ab 128k -ac 1 -vcodec libx264 -max_muxing_queue_size 9999 -movflags +faststart /tmp/final_output_video.mp4 -y
ffmpeg version 3.3.4
built on CentOS

Please refer to https://stackoverflow.com/questions/45969083/ffmpeg-concat-with-video-using-image-background for complete description of bug.

Attachments (4)

files.txt (59 bytes ) - added by Tony Cast 7 years ago.
Change directory for files.
new_image_video.mp4 (241.0 KB ) - added by Tony Cast 7 years ago.
Video created previously with audio track combined with image background
main_video.mp4 (483.8 KB ) - added by Tony Cast 7 years ago.
Main Video to attach at the end if image background video
final_output_video.mp4 (536.6 KB ) - added by Tony Cast 7 years ago.
Final Output after concat

Download all attachments as: .zip

Change History (15)

comment:1 by Carl Eugen Hoyos, 7 years ago

To make this a valid ticket, please test current FFmpeg git head and post the command line you tested together with the complete, uncut console output here on the bug tracker, do not use external resources.

comment:2 by Tony Cast, 7 years ago

Output below:

ffmpeg -threads 0 -f concat -safe 0 -i /tmp/files.txt -vf "scale=iw*min(560/iw\,320/ih):ih*min(560/iw\,320/ih), pad=560:320:(560-iw*min(560/iw\,320/ih))/2:(320-ih*min(560/iw\,320/ih))/2" -preset veryslow -crf 15 -acodec libmp3lame -af aresample=resampler=soxr -ar 44100 -aspect 16:9 -qp 20  -framerate 30 -ab 128k -ac 1 -vcodec libx264 -max_muxing_queue_size 9999 -movflags +faststart /tmp/final_output_video.mp4 -y
ffmpeg version 3.3.4-static http://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 6.4.0 (Debian 6.4.0-4) 20170820
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gray --enable-libfribidi --enable-libass --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-librtmp --enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libzimg
  libavutil      55. 58.100 / 55. 58.100
  libavcodec     57. 89.100 / 57. 89.100
  libavformat    57. 71.100 / 57. 71.100
  libavdevice    57.  6.100 / 57.  6.100
  libavfilter     6. 82.100 /  6. 82.100
  libswscale      4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
  libpostproc    54.  5.100 / 54.  5.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x4c7c9c0] Auto-inserting h264_mp4toannexb bitstream filter
Input #0, concat, from '/tmp/files.txt':
  Duration: N/A, start: -0.025057, bitrate: 146 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc), 560x320 [SAR 1280:1281 DAR 320:183], 18 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 44100 Hz, mono, s16p, 128 kb/s
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (mp3 (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
No pixel format specified, yuvj420p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 @ 0x4c8e100] using SAR=64/63
[libx264 @ 0x4c8e100] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x4c8e100] profile High, level 3.1
[libx264 @ 0x4c8e100] 264 - core 152 r19 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=16 deblock=1:0:0 analyse=0x3:0x133 me=umh subme=10 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=24 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=1 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=8 b_pyramid=2 b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=60 rc=crf mbtree=1 crf=15.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/tmp/final_output_video.mp4':
  Metadata:
    encoder         : Lavf57.71.100
    Stream #0:0(und): Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuvj420p(pc), 560x320 [SAR 64:63 DAR 16:9], q=-1--1, 30 fps, 15360 tbn, 30 tbc
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc57.89.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(und): Audio: mp3 (libmp3lame) (i[0][0][0] / 0x0069), 44100 Hz, mono, s16p, 128 kb/s
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc57.89.100 libmp3lame
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x51fdf60] Auto-inserting h264_mp4toannexb bitstream filter1 drop=0 speed=1.71x
[mp4 @ 0x4c800a0] Starting second pass: moving the moov atom to the beginning of the filerop=69 speed=0.421x
frame= 1910 fps= 10 q=21.0 Lsize=    4967kB time=00:01:16.41 bitrate= 532.5kbits/s dup=1 drop=69 speed=0.41x
video:3738kB audio:1169kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.229286%
[libx264 @ 0x4c8e100] frame I:15    Avg QP:10.22  size: 39698
[libx264 @ 0x4c8e100] frame P:601   Avg QP:16.86  size:  3558
[libx264 @ 0x4c8e100] frame B:1294  Avg QP:18.66  size:   845
[libx264 @ 0x4c8e100] consecutive B-frames: 10.4%  9.3%  8.8% 41.5%  4.5%  2.5%  1.1%  1.3% 20.7%
[libx264 @ 0x4c8e100] mb I  I16..4: 17.9% 41.8% 40.3%
[libx264 @ 0x4c8e100] mb P  I16..4:  0.2%  1.8%  0.7%  P16..4: 17.7%  8.8%  8.8%  0.5%  0.2%    skip:61.3%
[libx264 @ 0x4c8e100] mb B  I16..4:  0.0%  0.2%  0.2%  B16..8: 13.3%  3.6%  1.4%  direct: 1.4%  skip:79.9%  L0:38.7% L1:41.8% BI:19.6%
[libx264 @ 0x4c8e100] 8x8 transform intra:54.5% inter:52.1%
[libx264 @ 0x4c8e100] direct mvs  spatial:99.9% temporal:0.1%
[libx264 @ 0x4c8e100] coded y,uvDC,uvAC intra: 83.9% 89.4% 68.2% inter: 8.1% 9.7% 2.1%
[libx264 @ 0x4c8e100] i16 v,h,dc,p: 35% 32%  6% 27%
[libx264 @ 0x4c8e100] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 19%  8%  7%  9%  9% 10% 11% 12%
[libx264 @ 0x4c8e100] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 17%  8%  7% 11% 10% 11% 10% 13%
[libx264 @ 0x4c8e100] i8c dc,h,v,p: 41% 27% 16% 17%
[libx264 @ 0x4c8e100] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x4c8e100] ref P L0: 67.5% 13.5% 10.5%  3.0%  1.5%  0.9%  0.7%  0.4%  0.4%  0.3%  0.3%  0.2%  0.2%  0.2%  0.2%  0.2%
[libx264 @ 0x4c8e100] ref B L0: 85.1%  8.5%  3.1%  1.1%  0.6%  0.4%  0.3%  0.2%  0.2%  0.2%  0.1%  0.1%  0.1%  0.1%  0.0%
[libx264 @ 0x4c8e100] ref B L1: 97.4%  2.6%
[libx264 @ 0x4c8e100] kb/s:480.88

comment:3 by Carl Eugen Hoyos, 7 years ago

Keywords: ffmpeg removed
Reproduced by developer: unset

Is the issue reproducible with current FFmpeg git head (the only version supported here)?
Is the issue also reproducible if you don't use audio or video filters?
Why are you specifying an output frame rate?
Please provide the input files.

comment:4 by Tony Cast, 7 years ago

Yes I am able to reproduce this issue. I was doing testing with the frame rate, if the output frame rate is not set the track combined is still out of sync. I will attach the files used.

comment:5 by Tony Cast, 7 years ago

Yes I am able to reproduce this issue. I was doing testing with the frame rate, if the output frame rate is not set the track combined is still out of sync. I will attach the files used.

by Tony Cast, 7 years ago

Attachment: files.txt added

Change directory for files.

by Tony Cast, 7 years ago

Attachment: new_image_video.mp4 added

Video created previously with audio track combined with image background

comment:6 by Zhichao Liu, 7 years ago

It will be more convenient if you can upload your final_output_video.mp4 so I can make sure I successfully reproduce this issue on my OS.

by Tony Cast, 7 years ago

Attachment: main_video.mp4 added

Main Video to attach at the end if image background video

by Tony Cast, 7 years ago

Attachment: final_output_video.mp4 added

Final Output after concat

comment:7 by Tony Cast, 7 years ago

I added the files, also replaced main_video.mp4 there was something wrong with the previous file when I tested again.. this one is the actual one I tested with just shortened to be within the file size limit.

in reply to:  7 comment:8 by Zhichao Liu, 7 years ago

Replying to tcwebadmin:

I added the files, also replaced main_video.mp4 there was something wrong with the previous file when I tested again.. this one is the actual one I tested with just shortened to be within the file size limit.

Can you reproduce the problem when using two exactly same videos as the input files?

in reply to:  6 ; comment:9 by Carl Eugen Hoyos, 7 years ago

Replying to liuzceecs:

It will be more convenient if you can upload your final_output_video.mp4 so I can make sure I successfully reproduce this issue on my OS.

If you know of any operation with FFmpeg that produces different output on different systems (including different operating systems), please let us know, this could indicate a serious issue. This of course does not include different output due to floating point operations or inexact asm optimizations. Neither should be an issue for the issue described here.
Generally, do not ask for output files unless you cannot reproduce the issue: Were you unable to reproduce?

in reply to:  9 comment:10 by Zhichao Liu, 7 years ago

Replying to cehoyos:

Replying to liuzceecs:

It will be more convenient if you can upload your final_output_video.mp4 so I can make sure I successfully reproduce this issue on my OS.

If you know of any operation with FFmpeg that produces different output on different systems (including different operating systems), please let us know, this could indicate a serious issue. This of course does not include different output due to floating point operations or inexact asm optimizations. Neither should be an issue for the issue described here.
Generally, do not ask for output files unless you cannot reproduce the issue: Were you unable to reproduce?

Thank you for your advice. Actually, I have tried the newest ffmpeg package on Windows 8.1 and compiled the newest source code from git on Ubuntu 15.04. Neither can reproduce the issue. Actually, both two ways generate the same error log messages and that is different from what is provided by tcwebadmin. I will paste the log below (just a few lines since) and to your question, I cannot reproduce the issue.

 ~/bin/ffmpeg -threads 0 -f concat -safe 0 -i files.txt -vf "scale=iw*min(560/iw\,320/ih):ih*min(560/iw\,320/ih), pad=560:320:(560-iw*min(560/iw\,320/ih))/2:(320-ih*min(560/iw\,320/ih))/2" -preset veryslow -crf 15 -acodec libmp3lame -af aresample=resampler=soxr -ar 44100 -aspect 16:9 -qp 20 -framerate 30 -ab 128k -ac 1 -vcodec libx264 -max_muxing_queue_size 9999 -movflags +faststart final_output_video.mp4 -y 2> log

ffmpeg version N-87389-g5d07275 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu13)
  configuration: --prefix=/home/lzc/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/lzc/ffmpeg_build/include --extra-ldflags=-L/home/lzc/ffmpeg_build/lib --bindir=/home/lzc/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-libsoxr
  libavutil      55. 76.100 / 55. 76.100
  libavcodec     57.106.101 / 57.106.101
  libavformat    57. 82.101 / 57. 82.101
  libavdevice    57.  8.101 / 57.  8.101
  libavfilter     6.106.100 /  6.106.100
  libswscale      4.  7.103 /  4.  7.103
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2557520] Auto-inserting h264_mp4toannexb bitstream filter
Input #0, concat, from 'files.txt':
  Duration: N/A, start: -0.046440, bitrate: 145 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc), 560x320 [SAR 1280:1281 DAR 320:183], 17 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 128 kb/s
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
[libx264 @ 0x25fdae0] using SAR=64/63
[libx264 @ 0x25fdae0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
[libx264 @ 0x25fdae0] profile High, level 3.1
[libx264 @ 0x25fdae0] 264 - core 142 r2495 6a301b6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=16 deblock=1:0:0 analyse=0x3:0x133 me=umh subme=10 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=24 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=8 b_pyramid=2 b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=60 rc=crf mbtree=1 crf=15.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'final_output_video.mp4':
  Metadata:
    encoder         : Lavf57.82.101
    Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc), 560x320 [SAR 64:63 DAR 16:9], q=-1--1, 30 fps, 15360 tbn, 30 tbc
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc57.106.101 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(und): Audio: mp3 (libmp3lame) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 128 kb/s
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc57.106.101 libmp3lame
frame=  368 fps=0.0 q=21.0 size=       0kB time=00:00:12.35 bitrate=   0.0kbits/s dup=1 drop=0 speed=24.7x    
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2ea83a0] Auto-inserting h264_mp4toannexb bitstream filter
[aac @ 0x2569c20] Error decoding AAC frame header.
Error while decoding stream #0:1: Error number -67308554 occurred
[aac @ 0x2569c20] Multiple frames in a packet.
[aac @ 0x2569c20] channel element 2.15 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] More than one AAC RDB per ADTS frame is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x2569c20] channel element 2.0 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] channel element 3.12 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] invalid band type
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] Reserved bit set.
[aac @ 0x2569c20] TNS filter order 21 is greater than maximum 12.
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] channel element 2.6 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] Reserved bit set.
[aac @ 0x2569c20] Number of bands (29) exceeds limit (4).
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] channel element 3.2 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x2569c20] Inconsistent channel configuration.
[aac @ 0x2569c20] get_buffer() failed
Error while decoding stream #0:1: Invalid argument
[aac @ 0x2569c20] channel element 3.2 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x2569c20] Inconsistent channel configuration.
[aac @ 0x2569c20] get_buffer() failed
Error while decoding stream #0:1: Invalid argument
[aac @ 0x2569c20] channel element 3.0 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] Reserved bit set.
[aac @ 0x2569c20] TNS filter order 21 is greater than maximum 12.
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] channel element 2.14 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] channel element 3.10 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] channel element 3.8 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] Reserved bit set.
[aac @ 0x2569c20] SSR is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x2569c20] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
Error while decoding stream #0:1: Not yet implemented in FFmpeg, patches welcome
[aac @ 0x2569c20] channel element 2.10 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] channel element 1.6 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] channel element 3.15 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] channel element 1.9 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 0x2569c20] channel element 3.11 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input

comment:11 by Elon Musk, 10 months ago

Resolution: invalid
Status: newclosed

Concatenating mp3 stream in one file with aac stream in another at demuxer level is not possible.
Use concat filter instead for audio.

Note: See TracTickets for help on using tickets.