"Buffer queue overflow, dropping." when splitting input into thumbnail and overlay filters.

[Vadim]

Summary of the bug:
I want to overlay an image over my video, and simultaneously produce a thumbnail from the video without the overlay. I see nothing in the docs saying I shouldn't attempt this with a single command.
When I do so, ffmpeg logs "Buffer queue overflow, dropping." 34 times, and the output mp4 has a segment about a second long where the video is frozen and the audio keeps running (at seconds 2..3 of playback); the output thumbnail contains a frame that would have been somewhere in that frozen segment.

Removing either of the filters removes the problem. Reordering the filter chains has no effect.

Versions: this bug was first noticed in ffmpeg release 2.6.2, and is still reproducible with Zeranoe git-1bf76cd (2015-08-23).

How to reproduce:

ffmpeg.exe -i Hangover3.mp4 -i Qbert.png -filter_complex '[0:v]split[ovl1],thumbnail[outThumb1];[ovl1][1:v]overlay[out1]' -strict -2 -map '[out1]' -map 0:a 'H3-avout.mp4' -an -vframes 1 -map '[outThumb1]' 'H3-thumb.jpeg'
ffmpeg version N-74600-g1bf76cd Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.9.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 58.100 / 56. 58.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 37.100 /  5. 37.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Hangover3.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2013-10-10 21:32:20
    artist          :
    description     :
    title           :
  Duration: 00:00:15.02, start: 0.000000, bitrate: 676 kb/s
    Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv), 480x270 [SAR 1:1 DAR 16:9], 619 kb/s, 24 fps, 24 tbr, 24k tbn, 48 tbc (default)
    Metadata:
      creation_time   : 2013-10-10 21:32:21
      handler_name    : Mainconcept MP4 Video Media Handler
      encoder         : AVC Coding
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 53 kb/s (default)
    Metadata:
      creation_time   : 2013-10-10 21:32:21
      handler_name    : Mainconcept MP4 Sound Media Handler
Input #1, png_pipe, from 'Qbert.png':
  Duration: N/A, bitrate: N/A
    Stream #1:0: Video: png, rgba(pc), 524x740, 25 tbr, 25 tbn, 25 tbc
[swscaler @ 0000000004f8cec0] deprecated pixel format used, make sure you did set range correctly
[libx264 @ 0000000005012f00] using SAR=1/1
[libx264 @ 0000000005012f00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0000000005012f00] profile High, level 2.1
[libx264 @ 0000000005012f00] 264 - core 148 r2579 73ae2d1 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=8 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'H3-avout.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    title           :
    artist          :
    description     :
    encoder         : Lavf56.40.101
    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 480x270 [SAR 1:1 DAR 16:9], q=-1--1, 24 fps, 12288 tbn, 24 tbc (default)
    Metadata:
      encoder         : Lavc56.58.100 libx264
    Stream #0:1(eng): Audio: aac (libvo_aacenc) ([64][0][0][0] / 0x0040), 44100 Hz, stereo, s16, 128 kb/s (default)
    Metadata:
      creation_time   : 2013-10-10 21:32:21
      handler_name    : Mainconcept MP4 Sound Media Handler
      encoder         : Lavc56.58.100 libvo_aacenc
Output #1, image2, to 'H3-thumb.jpeg':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    title           :
    artist          :
    description     :
    encoder         : Lavf56.40.101
    Stream #1:0: Video: mjpeg, yuvj444p(pc), 480x270 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 24 fps, 24 tbn, 24 tbc (default)
    Metadata:
      encoder         : Lavc56.58.100 mjpeg
Stream mapping:
  Stream #0:0 (h264) -> split (graph 0)
  Stream #1:0 (png) -> overlay:overlay (graph 0)
  overlay (graph 0) -> Stream #0:0 (libx264)
  Stream #0:1 -> #0:1 (aac (native) -> aac (libvo_aacenc))
  thumbnail (graph 0) -> Stream #1:0 (mjpeg)
Press [q] to stop, [?] for help
[Parsed_overlay_2 @ 0000000005015340] [framesync @ 00000000003bdaa8] Buffer queue overflow, dropping.
    Last message repeated 33 times
[Parsed_thumbnail_1 @ 000000000034ab20] frame id #72 (pts_time=3.000000) selected from a set of 100 images
[Parsed_thumbnail_1 @ 000000000034ab20] frame id #48 (pts_time=6.166667) selected from a set of 100 images
[Parsed_thumbnail_1 @ 000000000034ab20] frame id #15 (pts_time=8.958333) selected from a set of 100 images
[Parsed_thumbnail_1 @ 000000000034ab20] frame id #31 (pts_time=13.791667) selected from a set of 60 images
frame=  360 fps=344 q=-1.0 Lq=3.9 size=     952kB time=00:00:15.03 bitrate= 518.5kbits/s dup=35 drop=4
video:717kB audio:236kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0000000005012f00] frame I:5     Avg QP:17.78  size: 15102
[libx264 @ 0000000005012f00] frame P:214   Avg QP:23.26  size:  2777
[libx264 @ 0000000005012f00] frame B:141   Avg QP:22.93  size:   357
[libx264 @ 0000000005012f00] consecutive B-frames: 37.5% 27.2% 10.8% 24.4%
[libx264 @ 0000000005012f00] mb I  I16..4: 30.8% 21.2% 48.0%
[libx264 @ 0000000005012f00] mb P  I16..4:  5.0%  5.6%  6.5%  P16..4: 20.5%  8.7%  3.3%  0.0%  0.0%    skip:50.6%
[libx264 @ 0000000005012f00] mb B  I16..4:  0.1%  0.2%  0.2%  B16..8: 19.5%  2.2%  0.5%  direct: 0.7%  skip:76.6%  L0:52.7% L1:40.2% BI: 7.1%
[libx264 @ 0000000005012f00] 8x8 transform intra:31.6% inter:39.8%
[libx264 @ 0000000005012f00] coded y,uvDC,uvAC intra: 56.8% 61.9% 30.6% inter: 9.6% 8.9% 1.6%
[libx264 @ 0000000005012f00] i16 v,h,dc,p: 26% 46%  8% 20%
[libx264 @ 0000000005012f00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 30% 17%  5%  5%  4%  6%  5%  7%
[libx264 @ 0000000005012f00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 32% 12%  6%  6%  6%  7%  5%  7%
[libx264 @ 0000000005012f00] i8c dc,h,v,p: 39% 34% 16% 10%
[libx264 @ 0000000005012f00] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000000005012f00] ref P L0: 74.6% 12.8%  8.9%  3.6%
[libx264 @ 0000000005012f00] ref B L0: 88.9% 10.6%  0.5%
[libx264 @ 0000000005012f00] ref B L1: 96.3%  3.7%
[libx264 @ 0000000005012f00] kb/s:384.12

The output files of ffmpeg -v 9 -loglevel 99 are binary identical to the above, but the log is 445KB. I will attach it along with my inputs and outputs, instead of pasting it here.

[Vadim]

input video file

[Vadim]

input image overlay

[Carl Eugen Hoyos]

Is the issue not reproducible with older versions of FFmpeg?

[Vadim]

log of ffmpeg -v 9 -loglevel 99

[Vadim]

output thumbnail (correct)

[Vadim]

Replying to cehoyos:

Is the issue not reproducible with older versions of FFmpeg?

It's reproducible with 2.6.2; are there particular earlier versions I should try?

[Carl Eugen Hoyos]

No, I just wondered what "first noticed" means.

[Vadim]

Ahh, sorry about the ambiguity. Now that you mentioned it, though, I did a binary search. The issue appears to have been introduced in revision dcc7686 (on the Windows build download page ​http://ffmpeg.zeranoe.com/builds/win64/static/, ffmpeg-20130922-git-21f6ff3-win64-static.7z works as expected and ffmpeg-20130924-git-dcc7686-win64-static.7z reproduces the issue).

[Carl Eugen Hoyos]

Does it work if you insert the fifo filter?

[Vadim]

Yes! Running the overlay input through fifo fixes it.
Breaks: ..."[ovl1][1:v]overlay[out1]"...
Works: ..."[ovl1]fifo[ovl2];[ovl2][1:v]overlay[out1]"...

Thanks, I will use this as a workaround! Unless... is there a downside? If I use this for long videos (and optionally tell thumbnail filter to stop processing after its first or second thumbnail), will frames accumulate in the fifo and fill up my RAM?

[Vadim]

output mp4 (with video freeze) (apparently I forgot this earlier)

[Misaki]

The root cause might be that the thumbnail filter is somehow marking the frames as unavailable to 'pass on' to a later filter (or in this case to be output to the encoder), even though the stream has supposedly been split. I have encountered a bug like this that I hopefully documented somewhere but would have to confirm whether it still exists, where some type of action on one output of a split filter was affecting the other outputs. Maybe trying to change the range from mpeg to pc/jpeg with a scale filter without actually modifying the pixels in any other way (probably so a lut filter could use that expanded range).

As a workaround, someone in this very specific situation could try just using separate filtergraphs, with a movie filter to add the png. '-vf movie=...,[in]overlay' before the mp4 output, and '-vf thumbnail' before the jpeg output. If this works, it suggests the cause is indeed in how the filtergraph and specifically the split filter works, and not because there is an actual conflict between the thumbnail graph's memory requirements and the limited (~64?) number of frames that the overlay filter will buffer.

[Vadim]

I am not sure I understand what you said, but I tried the following two command lines and found that order matters.

ffmpeg.exe -i Hangover3.mp4 -vf "thumbnail" -vframes 1 "H3-thumb.jpeg" -vf "movie=Qbert.png,[in]overlay" -strict -2 "H3-avout.mp4"

does not fix the problem. I am still getting the same error messages

...
Press [q] to stop, [?] for help
[Parsed_overlay_1 @ 004C2AE0] [framesync @ 040FB1E8] Buffer queue overflow, dropping.
    Last message repeated 33 times
[Parsed_thumbnail_0 @ 004C26E0] frame id #72 (pts_time=3.000000) selected from a set of 100 images
[Parsed_overlay_1 @ 004C2AE0] [framesync @ 040FB1E8] Buffer queue overflow, dropping.
[Parsed_thumbnail_0 @ 004C26E0] frame id #48 (pts_time=6.166667) selected from a set of 100 images
[Parsed_thumbnail_0 @ 004C26E0] frame id #15 (pts_time=8.958333) selected from a set of 100 images
...

and the output file still has the 1-second freeze, but the output is not binary identical to the output of the original command.

However,

ffmpeg.exe -i Hangover3.mp4 -vf "movie=Qbert.png,[in]overlay" -strict -2 "H3-avout.mp4" -vf "thumbnail" -vframes 1 "H3-thumb.jpeg"

appears to fix it. No buffer queue overflows, and the output video looks good.

...
Press [q] to stop, [?] for help
[Parsed_thumbnail_0 @ 01A02EE0] frame id #72 (pts_time=3.000000) selected from a set of 100 images
frame=  156 fps=0.0 q=28.0 q=3.9 size=     288kB time=00:00:06.64 bitrate= 354.7kbits/s dup=1 drop=0
[Parsed_thumbnail_0 @ 01A02EE0] frame id #48 (pts_time=6.166667) selected from a set of 100 images
[Parsed_thumbnail_0 @ 01A02EE0] frame id #15 (pts_time=8.958333) selected from a set of 100 images
...

(I tried this with ffmpeg version 2.6.2)
(Edit: removed the superfluous output label "[outThumb1]". This did not change the behavior.)

[Elon Musk]

Can not reproduce with latest master.

[Carl Eugen Hoyos]

The warning is not shown since 19804024d5b26e9568ce2f21f15c6664717006cd

[Vadim]

Confirm fixed as of ​http://ffmpeg.zeranoe.com/builds/win64/static/ffmpeg-20170830-2b9fd15-win64-static.zip .
(Still broken in the previous build, ​http://ffmpeg.zeranoe.com/builds/win64/static/ffmpeg-20170827-ef0c6d9-win64-static.zip .)
Thank you!