Opened 7 years ago

Closed 7 years ago

#6566 closed defect (fixed)

Overlay drops frames when audio is also processed

Reported by: Gyan Owned by:
Priority: important Component: undetermined
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

When performing an overlay of one video over the other, ffmpeg reports Buffer queue overflow and the overlay freezes at multiple points. Adding fifo filters to the two video feeds doesn't help. However, omitting the audio, resolves the issue.

# 1 Generate source files

ffmpeg -f lavfi -i testsrc2=d=55 -f lavfi -i testsrc=d=10 \
       -f lavfi -i sine=d=55 -f lavfi -i sine=640:d=10 \
        -map 0 -map 2 in.mp4 \
        -map 1 -map 3 ol.mp4

(The behaviour doesn't occur with direct lavfi source feeds)

# 2 Overlay with audio - overlay video freezes

ffmpeg -i in.mp4 -i ol.mp4 -strict -2 -filter_complex \
"[0:v]setpts=PTS-STARTPTS[v0];[1:v]setpts=PTS-STARTPTS+5/TB[v1]; \
[v0][v1]overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2:eof_action=pass[outv]; \
[1:a]adelay=5000[a1];[0:a][a1]amix[outa]" \
 -map [outv] -map [outa] out.mp4
ffmpeg version N-86920-g1193301 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.1.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
  libavutil      55. 69.100 / 55. 69.100
  libavcodec     57.102.100 / 57.102.100
  libavformat    57. 76.100 / 57. 76.100
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 95.100 /  6. 95.100
  libswscale      4.  7.101 /  4.  7.101
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.76.100
  Duration: 00:00:55.02, start: 0.000000, bitrate: 354 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 278 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 69 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'ol.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.76.100
  Duration: 00:00:10.02, start: 0.000000, bitrate: 111 kb/s
    Stream #1:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p, 320x240 [SAR 1:1 DAR 4:3], 34 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #1:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 69 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 (h264) -> setpts
  Stream #0:1 (aac) -> amix:input0
  Stream #1:0 (h264) -> setpts
  Stream #1:1 (aac) -> adelay
  overlay -> Stream #0:0 (libx264)
  amix -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
[libx264 @ 0000000002a41da0] using SAR=1/1
[libx264 @ 0000000002a41da0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000000002a41da0] profile High, level 1.3
[libx264 @ 0000000002a41da0] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'out.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.76.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.102.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 69 kb/s (default)
    Metadata:
      encoder         : Lavc57.102.100 aac


[Parsed_overlay_2 @ 00000000003cd5a0] [framesync @ 00000000047c0328] Buffer queue overflow, dropping.
    Last message repeated 115 times


frame=  328 fps=0.0 q=28.0 size=     256kB time=00:00:13.07 bitrate= 160.4kbits/s speed=26.1x    
frame=  709 fps=709 q=28.0 size=     768kB time=00:00:28.32 bitrate= 222.1kbits/s speed=28.3x    
frame= 1054 fps=703 q=28.0 size=    1280kB time=00:00:42.16 bitrate= 248.7kbits/s speed=28.1x    
frame= 1375 fps=695 q=-1.0 Lsize=    2007kB time=00:00:55.00 bitrate= 298.8kbits/s speed=27.8x    
video:1496kB audio:464kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.348816%
[libx264 @ 0000000002a41da0] frame I:7     Avg QP:17.46  size:  5572
[libx264 @ 0000000002a41da0] frame P:391   Avg QP:23.73  size:  1686
[libx264 @ 0000000002a41da0] frame B:977   Avg QP:28.18  size:   853
[libx264 @ 0000000002a41da0] consecutive B-frames:  2.0%  7.1%  8.3% 82.6%
[libx264 @ 0000000002a41da0] mb I  I16..4: 44.1% 27.7% 28.2%
[libx264 @ 0000000002a41da0] mb P  I16..4:  2.1%  3.9%  1.0%  P16..4: 14.1%  8.5%  5.9%  0.0%  0.0%    skip:64.4%
[libx264 @ 0000000002a41da0] mb B  I16..4:  0.2%  0.1%  0.1%  B16..8: 17.2%  5.4%  1.5%  direct: 2.2%  skip:73.4%  L0:50.9% L1:44.2% BI: 4.9%
[libx264 @ 0000000002a41da0] 8x8 transform intra:48.4% inter:34.6%
[libx264 @ 0000000002a41da0] coded y,uvDC,uvAC intra: 12.0% 35.6% 29.4% inter: 4.3% 13.3% 10.9%
[libx264 @ 0000000002a41da0] i16 v,h,dc,p: 72% 20%  6%  2%
[libx264 @ 0000000002a41da0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 10% 78%  1%  0%  0%  0%  0%  1%
[libx264 @ 0000000002a41da0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 38% 22% 31%  1%  1%  1%  2%  1%  2%
[libx264 @ 0000000002a41da0] i8c dc,h,v,p: 47% 18% 33%  2%
[libx264 @ 0000000002a41da0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000000002a41da0] ref P L0: 59.3%  8.1% 20.0% 12.6%
[libx264 @ 0000000002a41da0] ref B L0: 80.6% 15.8%  3.6%
[libx264 @ 0000000002a41da0] ref B L1: 94.3%  5.7%
[libx264 @ 0000000002a41da0] kb/s:222.77
[aac @ 000000000031d160] Qavg: 1015.382

# 3 Overlay with no audio - output normal

ffmpeg -i in.mp4 -i ol.mp4 -strict -2 -filter_complex \
"[0:v]setpts=PTS-STARTPTS[v0];[1:v]setpts=PTS-STARTPTS+5/TB[v1]; \
[v0][v1]overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2:eof_action=pass[outv]" \
-map [outv] out-noaud.mp4
ffmpeg version N-86920-g1193301 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.1.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
  libavutil      55. 69.100 / 55. 69.100
  libavcodec     57.102.100 / 57.102.100
  libavformat    57. 76.100 / 57. 76.100
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 95.100 /  6. 95.100
  libswscale      4.  7.101 /  4.  7.101
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.76.100
  Duration: 00:00:55.02, start: 0.000000, bitrate: 354 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 278 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 69 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'ol.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.76.100
  Duration: 00:00:10.02, start: 0.000000, bitrate: 111 kb/s
    Stream #1:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p, 320x240 [SAR 1:1 DAR 4:3], 34 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #1:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 69 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 (h264) -> setpts
  Stream #1:0 (h264) -> setpts
  overlay -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[libx264 @ 0000000000547f60] using SAR=1/1
[libx264 @ 0000000000547f60] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000000000547f60] profile High, level 1.3
[libx264 @ 0000000000547f60] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'out-noaud.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.76.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.102.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame=  614 fps=0.0 q=28.0 size=     256kB time=00:00:22.44 bitrate=  93.5kbits/s speed=44.9x    
frame= 1143 fps=1143 q=28.0 size=    1024kB time=00:00:43.60 bitrate= 192.4kbits/s speed=43.6x    
frame= 1375 fps=1092 q=-1.0 Lsize=    1515kB time=00:00:54.88 bitrate= 226.2kbits/s speed=43.6x    
video:1499kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.111186%
[libx264 @ 0000000000547f60] frame I:6     Avg QP:16.11  size:  5481
[libx264 @ 0000000000547f60] frame P:394   Avg QP:23.82  size:  1700
[libx264 @ 0000000000547f60] frame B:975   Avg QP:28.20  size:   853
[libx264 @ 0000000000547f60] consecutive B-frames:  2.1%  7.3%  8.3% 82.3%
[libx264 @ 0000000000547f60] mb I  I16..4: 51.7% 21.3% 27.1%
[libx264 @ 0000000000547f60] mb P  I16..4:  2.2%  3.8%  1.0%  P16..4: 14.7%  8.9%  6.1%  0.0%  0.0%    skip:63.4%
[libx264 @ 0000000000547f60] mb B  I16..4:  0.2%  0.1%  0.1%  B16..8: 17.4%  5.3%  1.5%  direct: 2.2%  skip:73.2%  L0:50.9% L1:44.1% BI: 4.9%
[libx264 @ 0000000000547f60] 8x8 transform intra:47.0% inter:34.2%
[libx264 @ 0000000000547f60] coded y,uvDC,uvAC intra: 11.4% 35.0% 28.3% inter: 4.3% 13.5% 11.1%
[libx264 @ 0000000000547f60] i16 v,h,dc,p: 73% 20%  5%  2%
[libx264 @ 0000000000547f60] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  8% 10% 79%  1%  0%  0%  0%  0%  1%
[libx264 @ 0000000000547f60] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 38% 23% 32%  1%  1%  1%  2%  1%  2%
[libx264 @ 0000000000547f60] i8c dc,h,v,p: 46% 18% 34%  2%
[libx264 @ 0000000000547f60] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000000000547f60] ref P L0: 59.5%  8.3% 20.2% 12.0%
[libx264 @ 0000000000547f60] ref B L0: 80.6% 15.8%  3.5%
[libx264 @ 0000000000547f60] ref B L1: 94.2%  5.8%
[libx264 @ 0000000000547f60] kb/s:223.11

Change History (1)

comment:1 by Elon Musk, 7 years ago

Resolution: fixed
Status: newclosed

Can not reproduce anymore.

Note: See TracTickets for help on using tickets.