Opened 9 years ago

Closed 7 years ago

Last modified 7 years ago

#4804 closed defect (fixed)

"Buffer queue overflow, dropping." when splitting input into thumbnail and overlay filters.

Reported by: Vadim Owned by:
Priority: important Component: avfilter
Version: git-master Keywords: overlay regression
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
I want to overlay an image over my video, and simultaneously produce a thumbnail from the video without the overlay. I see nothing in the docs saying I shouldn't attempt this with a single command.
When I do so, ffmpeg logs "Buffer queue overflow, dropping." 34 times, and the output mp4 has a segment about a second long where the video is frozen and the audio keeps running (at seconds 2..3 of playback); the output thumbnail contains a frame that would have been somewhere in that frozen segment.

Removing either of the filters removes the problem. Reordering the filter chains has no effect.

Versions: this bug was first noticed in ffmpeg release 2.6.2, and is still reproducible with Zeranoe git-1bf76cd (2015-08-23).

How to reproduce:

ffmpeg.exe -i Hangover3.mp4 -i Qbert.png -filter_complex '[0:v]split[ovl1],thumbnail[outThumb1];[ovl1][1:v]overlay[out1]' -strict -2 -map '[out1]' -map 0:a 'H3-avout.mp4' -an -vframes 1 -map '[outThumb1]' 'H3-thumb.jpeg'
ffmpeg version N-74600-g1bf76cd Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.9.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 58.100 / 56. 58.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 37.100 /  5. 37.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Hangover3.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2013-10-10 21:32:20
    artist          :
    description     :
    title           :
  Duration: 00:00:15.02, start: 0.000000, bitrate: 676 kb/s
    Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv), 480x270 [SAR 1:1 DAR 16:9], 619 kb/s, 24 fps, 24 tbr, 24k tbn, 48 tbc (default)
    Metadata:
      creation_time   : 2013-10-10 21:32:21
      handler_name    : Mainconcept MP4 Video Media Handler
      encoder         : AVC Coding
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 53 kb/s (default)
    Metadata:
      creation_time   : 2013-10-10 21:32:21
      handler_name    : Mainconcept MP4 Sound Media Handler
Input #1, png_pipe, from 'Qbert.png':
  Duration: N/A, bitrate: N/A
    Stream #1:0: Video: png, rgba(pc), 524x740, 25 tbr, 25 tbn, 25 tbc
[swscaler @ 0000000004f8cec0] deprecated pixel format used, make sure you did set range correctly
[libx264 @ 0000000005012f00] using SAR=1/1
[libx264 @ 0000000005012f00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0000000005012f00] profile High, level 2.1
[libx264 @ 0000000005012f00] 264 - core 148 r2579 73ae2d1 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=8 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'H3-avout.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    title           :
    artist          :
    description     :
    encoder         : Lavf56.40.101
    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 480x270 [SAR 1:1 DAR 16:9], q=-1--1, 24 fps, 12288 tbn, 24 tbc (default)
    Metadata:
      encoder         : Lavc56.58.100 libx264
    Stream #0:1(eng): Audio: aac (libvo_aacenc) ([64][0][0][0] / 0x0040), 44100 Hz, stereo, s16, 128 kb/s (default)
    Metadata:
      creation_time   : 2013-10-10 21:32:21
      handler_name    : Mainconcept MP4 Sound Media Handler
      encoder         : Lavc56.58.100 libvo_aacenc
Output #1, image2, to 'H3-thumb.jpeg':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    title           :
    artist          :
    description     :
    encoder         : Lavf56.40.101
    Stream #1:0: Video: mjpeg, yuvj444p(pc), 480x270 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 24 fps, 24 tbn, 24 tbc (default)
    Metadata:
      encoder         : Lavc56.58.100 mjpeg
Stream mapping:
  Stream #0:0 (h264) -> split (graph 0)
  Stream #1:0 (png) -> overlay:overlay (graph 0)
  overlay (graph 0) -> Stream #0:0 (libx264)
  Stream #0:1 -> #0:1 (aac (native) -> aac (libvo_aacenc))
  thumbnail (graph 0) -> Stream #1:0 (mjpeg)
Press [q] to stop, [?] for help
[Parsed_overlay_2 @ 0000000005015340] [framesync @ 00000000003bdaa8] Buffer queue overflow, dropping.
    Last message repeated 33 times
[Parsed_thumbnail_1 @ 000000000034ab20] frame id #72 (pts_time=3.000000) selected from a set of 100 images
[Parsed_thumbnail_1 @ 000000000034ab20] frame id #48 (pts_time=6.166667) selected from a set of 100 images
[Parsed_thumbnail_1 @ 000000000034ab20] frame id #15 (pts_time=8.958333) selected from a set of 100 images
[Parsed_thumbnail_1 @ 000000000034ab20] frame id #31 (pts_time=13.791667) selected from a set of 60 images
frame=  360 fps=344 q=-1.0 Lq=3.9 size=     952kB time=00:00:15.03 bitrate= 518.5kbits/s dup=35 drop=4
video:717kB audio:236kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0000000005012f00] frame I:5     Avg QP:17.78  size: 15102
[libx264 @ 0000000005012f00] frame P:214   Avg QP:23.26  size:  2777
[libx264 @ 0000000005012f00] frame B:141   Avg QP:22.93  size:   357
[libx264 @ 0000000005012f00] consecutive B-frames: 37.5% 27.2% 10.8% 24.4%
[libx264 @ 0000000005012f00] mb I  I16..4: 30.8% 21.2% 48.0%
[libx264 @ 0000000005012f00] mb P  I16..4:  5.0%  5.6%  6.5%  P16..4: 20.5%  8.7%  3.3%  0.0%  0.0%    skip:50.6%
[libx264 @ 0000000005012f00] mb B  I16..4:  0.1%  0.2%  0.2%  B16..8: 19.5%  2.2%  0.5%  direct: 0.7%  skip:76.6%  L0:52.7% L1:40.2% BI: 7.1%
[libx264 @ 0000000005012f00] 8x8 transform intra:31.6% inter:39.8%
[libx264 @ 0000000005012f00] coded y,uvDC,uvAC intra: 56.8% 61.9% 30.6% inter: 9.6% 8.9% 1.6%
[libx264 @ 0000000005012f00] i16 v,h,dc,p: 26% 46%  8% 20%
[libx264 @ 0000000005012f00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 30% 17%  5%  5%  4%  6%  5%  7%
[libx264 @ 0000000005012f00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 32% 12%  6%  6%  6%  7%  5%  7%
[libx264 @ 0000000005012f00] i8c dc,h,v,p: 39% 34% 16% 10%
[libx264 @ 0000000005012f00] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000000005012f00] ref P L0: 74.6% 12.8%  8.9%  3.6%
[libx264 @ 0000000005012f00] ref B L0: 88.9% 10.6%  0.5%
[libx264 @ 0000000005012f00] ref B L1: 96.3%  3.7%
[libx264 @ 0000000005012f00] kb/s:384.12

The output files of ffmpeg -v 9 -loglevel 99 are binary identical to the above, but the log is 445KB. I will attach it along with my inputs and outputs, instead of pasting it here.

Attachments (5)

Hangover3.mp4 (1.2 MB ) - added by Vadim 9 years ago.
input video file
Qbert.png (164.8 KB ) - added by Vadim 9 years ago.
input image overlay
v9-ll9-out.txt (444.6 KB ) - added by Vadim 9 years ago.
log of ffmpeg -v 9 -loglevel 99
H3-thumb.jpeg (12.7 KB ) - added by Vadim 9 years ago.
output thumbnail (correct)
H3-avout.mp4 (952.6 KB ) - added by Vadim 9 years ago.
output mp4 (with video freeze) (apparently I forgot this earlier)

Change History (16)

by Vadim, 9 years ago

Attachment: Hangover3.mp4 added

input video file

by Vadim, 9 years ago

Attachment: Qbert.png added

input image overlay

comment:1 by Carl Eugen Hoyos, 9 years ago

Is the issue not reproducible with older versions of FFmpeg?

by Vadim, 9 years ago

Attachment: v9-ll9-out.txt added

log of ffmpeg -v 9 -loglevel 99

by Vadim, 9 years ago

Attachment: H3-thumb.jpeg added

output thumbnail (correct)

in reply to:  1 comment:2 by Vadim, 9 years ago

Replying to cehoyos:

Is the issue not reproducible with older versions of FFmpeg?

It's reproducible with 2.6.2; are there particular earlier versions I should try?

comment:3 by Carl Eugen Hoyos, 9 years ago

No, I just wondered what "first noticed" means.

comment:4 by Vadim, 9 years ago

Ahh, sorry about the ambiguity. Now that you mentioned it, though, I did a binary search. The issue appears to have been introduced in revision dcc7686 (on the Windows build download page http://ffmpeg.zeranoe.com/builds/win64/static/, ffmpeg-20130922-git-21f6ff3-win64-static.7z works as expected and ffmpeg-20130924-git-dcc7686-win64-static.7z reproduces the issue).

comment:5 by Carl Eugen Hoyos, 9 years ago

Keywords: regression added
Priority: normalimportant

Does it work if you insert the fifo filter?

comment:6 by Vadim, 9 years ago

Yes! Running the overlay input through fifo fixes it.
Breaks: ..."[ovl1][1:v]overlay[out1]"...
Works: ..."[ovl1]fifo[ovl2];[ovl2][1:v]overlay[out1]"...

Thanks, I will use this as a workaround! Unless... is there a downside? If I use this for long videos (and optionally tell thumbnail filter to stop processing after its first or second thumbnail), will frames accumulate in the fifo and fill up my RAM?

by Vadim, 9 years ago

Attachment: H3-avout.mp4 added

output mp4 (with video freeze) (apparently I forgot this earlier)

comment:7 by Misaki, 8 years ago

The root cause might be that the thumbnail filter is somehow marking the frames as unavailable to 'pass on' to a later filter (or in this case to be output to the encoder), even though the stream has supposedly been split. I have encountered a bug like this that I hopefully documented somewhere but would have to confirm whether it still exists, where some type of action on one output of a split filter was affecting the other outputs. Maybe trying to change the range from mpeg to pc/jpeg with a scale filter without actually modifying the pixels in any other way (probably so a lut filter could use that expanded range).

As a workaround, someone in this very specific situation could try just using separate filtergraphs, with a movie filter to add the png. '-vf movie=...,[in]overlay' before the mp4 output, and '-vf thumbnail' before the jpeg output. If this works, it suggests the cause is indeed in how the filtergraph and specifically the split filter works, and not because there is an actual conflict between the thumbnail graph's memory requirements and the limited (~64?) number of frames that the overlay filter will buffer.

Last edited 8 years ago by Misaki (previous) (diff)

comment:8 by Vadim, 8 years ago

I am not sure I understand what you said, but I tried the following two command lines and found that order matters.

ffmpeg.exe -i Hangover3.mp4 -vf "thumbnail" -vframes 1 "H3-thumb.jpeg" -vf "movie=Qbert.png,[in]overlay" -strict -2 "H3-avout.mp4"

does not fix the problem. I am still getting the same error messages

...
Press [q] to stop, [?] for help
[Parsed_overlay_1 @ 004C2AE0] [framesync @ 040FB1E8] Buffer queue overflow, dropping.
    Last message repeated 33 times
[Parsed_thumbnail_0 @ 004C26E0] frame id #72 (pts_time=3.000000) selected from a set of 100 images
[Parsed_overlay_1 @ 004C2AE0] [framesync @ 040FB1E8] Buffer queue overflow, dropping.
[Parsed_thumbnail_0 @ 004C26E0] frame id #48 (pts_time=6.166667) selected from a set of 100 images
[Parsed_thumbnail_0 @ 004C26E0] frame id #15 (pts_time=8.958333) selected from a set of 100 images
...

and the output file still has the 1-second freeze, but the output is not binary identical to the output of the original command.

However,

ffmpeg.exe -i Hangover3.mp4 -vf "movie=Qbert.png,[in]overlay" -strict -2 "H3-avout.mp4" -vf "thumbnail" -vframes 1 "H3-thumb.jpeg"

appears to fix it. No buffer queue overflows, and the output video looks good.

...
Press [q] to stop, [?] for help
[Parsed_thumbnail_0 @ 01A02EE0] frame id #72 (pts_time=3.000000) selected from a set of 100 images
frame=  156 fps=0.0 q=28.0 q=3.9 size=     288kB time=00:00:06.64 bitrate= 354.7kbits/s dup=1 drop=0
[Parsed_thumbnail_0 @ 01A02EE0] frame id #48 (pts_time=6.166667) selected from a set of 100 images
[Parsed_thumbnail_0 @ 01A02EE0] frame id #15 (pts_time=8.958333) selected from a set of 100 images
...

(I tried this with ffmpeg version 2.6.2)
(Edit: removed the superfluous output label "[outThumb1]". This did not change the behavior.)

Last edited 8 years ago by Vadim (previous) (diff)

comment:9 by Elon Musk, 7 years ago

Resolution: fixed
Status: newclosed

Can not reproduce with latest master.

comment:10 by Carl Eugen Hoyos, 7 years ago

Component: undeterminedavfilter
Keywords: overlay added

The warning is not shown since 19804024d5b26e9568ce2f21f15c6664717006cd

Note: See TracTickets for help on using tickets.