Opened 7 years ago
Last modified 7 years ago
#7101 new defect
Overlay wrongly enabled before specified time
| Reported by: | slhck | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | avfilter |
| Version: | git-master | Keywords: | overlay |
| Cc: | Blocked By: | ||
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description
I want to show a "loading" spinner at a given time for a given amount of time, overlaid on top of a video. The video should freeze underneath the spinner.
In the following case, I want to show the spinner as an "initial loading" for 4 seconds, then at 10 seconds (media time) for 0.5 s, etc. At the same time, the video is looped (size = 1 frame) for the same amount of time.
Consider the following command:
ffmpeg \
-y \
-f lavfi -i testsrc=duration=60:size=320x240:rate=60,format=pix_fmts=yuv420p \
-i click_and_count.m4a \
-filter_complex " \
[0:v] \
loop=loop=240:size=1:start=0, setpts=N/FRAME_RATE/TB, \
loop=loop=30:size=1:start=840, setpts=N/FRAME_RATE/TB, \
loop=loop=84:size=1:start=1140, setpts=N/FRAME_RATE/TB, \
loop=loop=48:size=1:start=1548, setpts=N/FRAME_RATE/TB \
[stallvid]; \
movie=filename=spinner-64-white.png:loop=0, setpts=N/(FRAME_RATE*TB)*2 \
[spinner]; \
[stallvid][spinner] \
overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2:shortest=1:\
enable='between(t,0,4.0)+between(t,14.0,14.5)+between(t,19.0,20.4)+between(t,25.8,26.6)' \
[outv];
[1:a] \
aloop=loop=192000:size=1:start=0, asetpts=N/SAMPLE_RATE/TB, \
aloop=loop=24000:size=1:start=672000, asetpts=N/SAMPLE_RATE/TB, \
aloop=loop=67200:size=1:start=912000, asetpts=N/SAMPLE_RATE/TB, \
aloop=loop=38400:size=1:start=1238399, asetpts=N/SAMPLE_RATE/TB, \
volume=0:enable='between(t,0,4.0)+between(t,14.0,14.5)+between(t,19.0,20.4)+between(t,25.8,26.6)' \
[outa] \
" -shortest -map "[outv]" -map "[outa]" output.mp4
The spinner is indeed shown exactly when the 10 appears in the video (at 00:00:14), but it is also shown for exactly one frame, four frames before the 10 appears.
This should of course not happen, as the overlay is only enabled between 14 and 14.5.
Attachments (2)
Change History (10)
by , 7 years ago
| Attachment: | spinner-64-white.png added |
|---|
by , 7 years ago
| Attachment: | click_and_count.m4a added |
|---|
comment:1 by , 7 years ago
comment:2 by , 7 years ago
Does the timing issue occur if you disable all audio processing or perform it in a separate graph?
comment:4 by , 7 years ago
When I handle audio in a separate filter graph, the issue persists:
ffmpeg \
-y \
-f lavfi -i testsrc=duration=60:size=320x240:rate=60,format=pix_fmts=yuv420p \
-i spinners/click_and_count.m4a \
-filter_complex " \
[0:v] \
loop=loop=240:size=1:start=0, setpts=N/FRAME_RATE/TB, \
loop=loop=30:size=1:start=840, setpts=N/FRAME_RATE/TB, \
loop=loop=84:size=1:start=1140, setpts=N/FRAME_RATE/TB, \
loop=loop=48:size=1:start=1548, setpts=N/FRAME_RATE/TB \
[stallvid]; \
movie=filename=spinners/spinner-64-white.png:loop=0, setpts=N/(FRAME_RATE*TB)*2 \
[spinner]; \
[stallvid][spinner] \
overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2:shortest=1:\
enable='between(t,0,4.0)+between(t,14.0,14.5)+between(t,19.0,20.4)+between(t,25.8,26.6)' \
[outv]
" \
-filter_complex " \
[1:a] \
aloop=loop=192000:size=1:start=0, asetpts=N/SAMPLE_RATE/TB, \
aloop=loop=24000:size=1:start=672000, asetpts=N/SAMPLE_RATE/TB, \
aloop=loop=67200:size=1:start=912000, asetpts=N/SAMPLE_RATE/TB, \
aloop=loop=38400:size=1:start=1238399, asetpts=N/SAMPLE_RATE/TB, \
volume=0:enable='between(t,0,4.0)+between(t,14.0,14.5)+between(t,19.0,20.4)+between(t,25.8,26.6)' \
[outa] \
" \
-shortest -map "[outv]" -map "[outa]" output.mp4
comment:5 by , 7 years ago
Actually, even without audio processing (i.e., when adding -an), the issue is the same – the spinner shortly flashes before the specified time.
~/Downloads/ffmpeg \
-y \
-f lavfi -i testsrc=duration=60:size=320x240:rate=60,format=pix_fmts=yuv420p \
-i spinners/click_and_count.m4a \
-filter_complex " \
[0:v] \
loop=loop=240:size=1:start=0, setpts=N/FRAME_RATE/TB, \
loop=loop=30:size=1:start=840, setpts=N/FRAME_RATE/TB, \
loop=loop=84:size=1:start=1140, setpts=N/FRAME_RATE/TB, \
loop=loop=48:size=1:start=1548, setpts=N/FRAME_RATE/TB \
[stallvid]; \
movie=filename=spinners/spinner-64-white.png:loop=0, setpts=N/(FRAME_RATE*TB)*2 \
[spinner]; \
[stallvid][spinner] \
overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2:shortest=1:\
enable='between(t,0,4.0)+between(t,14.0,14.5)+between(t,19.0,20.4)+between(t,25.8,26.6)' \
[outv]
" \
-map "[outv]" -an test/tmp.mp4
ffmpeg version N-90417-ga8c2d375ca-tessus Copyright (c) 2000-2018 the FFmpeg developers
built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libass --enable-libbluray --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay
libavutil 56. 11.100 / 56. 11.100
libavcodec 58. 15.100 / 58. 15.100
libavformat 58. 10.100 / 58. 10.100
libavdevice 58. 2.100 / 58. 2.100
libavfilter 7. 13.100 / 7. 13.100
libswscale 5. 0.102 / 5. 0.102
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100
Input #0, lavfi, from 'testsrc=duration=60:size=320x240:rate=60,format=pix_fmts=yuv420p':
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 60 tbr, 60 tbn, 60 tbc
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'spinners/click_and_count.m4a':
Metadata:
major_brand : M4A
minor_version : 512
compatible_brands: isomiso2
encoder : Lavf57.83.100
Duration: 00:01:01.08, start: 0.000000, bitrate: 134 kb/s
Stream #1:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 132 kb/s (default)
Metadata:
handler_name : SoundHandler
Stream mapping:
Stream #0:0 (rawvideo) -> loop
overlay -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[libx264 @ 0x7fdd8a008800] using SAR=1/1
[libx264 @ 0x7fdd8a008800] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fdd8a008800] profile High, level 2.1
[libx264 @ 0x7fdd8a008800] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=7 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test/tmp.mp4':
Metadata:
encoder : Lavf58.10.100
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 60 fps, 15360 tbn, 60 tbc
Metadata:
encoder : Lavc58.15.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Past duration 0.999992 too large
Last message repeated 21 times
Past duration 0.999992 too large 0kB time=00:00:17.16 bitrate= 0.0kbits/s speed=34.3x
Last message repeated 24 times
Past duration 0.999992 too large= 256kB time=00:00:50.86 bitrate= 41.2kbits/s speed=33.9x
Last message repeated 32 times
[Parsed_testsrc_0 @ 0x7fdd88c03b00] EOF timestamp not reliable
frame= 4002 fps=2009 q=-1.0 Lsize= 497kB time=00:01:06.65 bitrate= 61.1kbits/s speed=33.5x
video:457kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 8.911427%
[libx264 @ 0x7fdd8a008800] frame I:17 Avg QP:16.61 size: 3274
[libx264 @ 0x7fdd8a008800] frame P:1766 Avg QP:17.44 size: 200
[libx264 @ 0x7fdd8a008800] frame B:2219 Avg QP:16.02 size: 27
[libx264 @ 0x7fdd8a008800] consecutive B-frames: 25.5% 1.2% 1.0% 72.2%
[libx264 @ 0x7fdd8a008800] mb I I16..4: 65.0% 7.0% 27.9%
[libx264 @ 0x7fdd8a008800] mb P I16..4: 1.7% 0.4% 0.1% P16..4: 9.0% 2.5% 1.2% 0.0% 0.0% skip:85.0%
[libx264 @ 0x7fdd8a008800] mb B I16..4: 0.1% 0.0% 0.0% B16..8: 2.2% 0.0% 0.0% direct: 0.0% skip:97.6% L0:51.7% L1:47.4% BI: 0.9%
[libx264 @ 0x7fdd8a008800] 8x8 transform intra:13.5% inter:56.1%
[libx264 @ 0x7fdd8a008800] coded y,uvDC,uvAC intra: 5.6% 24.2% 11.7% inter: 0.5% 2.6% 1.0%
[libx264 @ 0x7fdd8a008800] i16 v,h,dc,p: 77% 5% 3% 15%
[libx264 @ 0x7fdd8a008800] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 12% 58% 0% 0% 0% 0% 0% 0%
[libx264 @ 0x7fdd8a008800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 47% 26% 18% 2% 2% 2% 1% 2% 0%
[libx264 @ 0x7fdd8a008800] i8c dc,h,v,p: 25% 14% 49% 12%
[libx264 @ 0x7fdd8a008800] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7fdd8a008800] ref P L0: 72.2% 5.6% 18.7% 3.5%
[libx264 @ 0x7fdd8a008800] ref B L0: 54.0% 40.8% 5.2%
[libx264 @ 0x7fdd8a008800] ref B L1: 97.6% 2.4%
[libx264 @ 0x7fdd8a008800] kb/s:56.01
comment:6 by , 7 years ago
Upon further investigation it seems to be caused only by the combination of looping the input video and adding the overlay. Adding the overlay on its own does not cause problems.
comment:7 by , 7 years ago
This doesn't seem to be related to looping the input video. If I output the looped testsrc to a file and use that, it still occurs.
But if I append to the PNG filters, fps=60, I don't get the premature overlay.
comment:8 by , 7 years ago
That seems to fix it, thanks!
So, I assume that since the PTS are generated for the PNG input, the overlay filter – when seeing two different frame rates – interpolates the timestamps incorrectly?
Note:
See TracTickets
for help on using tickets.
ffmpeg output:
ffmpeg version N-90389-g72bb955625-tessus Copyright (c) 2000-2018 the FFmpeg developers built with Apple LLVM version 9.0.0 (clang-900.0.39.2) configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libass --enable-libbluray --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay libavutil 56. 11.100 / 56. 11.100 libavcodec 58. 14.100 / 58. 14.100 libavformat 58. 10.100 / 58. 10.100 libavdevice 58. 2.100 / 58. 2.100 libavfilter 7. 13.100 / 7. 13.100 libswscale 5. 0.102 / 5. 0.102 libswresample 3. 0.101 / 3. 0.101 libpostproc 55. 0.100 / 55. 0.100 Input #0, lavfi, from 'testsrc=duration=60:size=320x240:rate=60,format=pix_fmts=yuv420p': Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 60 tbr, 60 tbn, 60 tbc Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'click_and_count.m4a': Metadata: major_brand : M4A minor_version : 512 compatible_brands: isomiso2 encoder : Lavf57.83.100 Duration: 00:01:01.08, start: 0.000000, bitrate: 134 kb/s Stream #1:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 132 kb/s (default) Metadata: handler_name : SoundHandler Stream mapping: Stream #0:0 (rawvideo) -> loop Stream #1:0 (aac) -> aloop overlay -> Stream #0:0 (libx264) volume -> Stream #0:1 (aac) Press [q] to stop, [?] for help [libx264 @ 0x7fe6f0816e00] using SAR=1/1 [libx264 @ 0x7fe6f0816e00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x7fe6f0816e00] profile High, level 2.1 [libx264 @ 0x7fe6f0816e00] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=7 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to 'output.mp4': Metadata: encoder : Lavf58.10.100 Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 60 fps, 15360 tbn, 60 tbc (default) Metadata: encoder : Lavc58.14.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1 Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default) Metadata: encoder : Lavc58.14.100 aac [out_0_1 @ 0x7fe6efc0f380] 100 buffers queued in out_0_1, something may be wrong. [out_0_1 @ 0x7fe6efc0f380] 1000 buffers queued in out_0_1, something may be wrong. Past duration 0.999992 too large 0kB time=00:00:01.70 bitrate= 0.2kbits/s speed=3.41x Past duration 0.999992 too large 0kB time=00:00:03.52 bitrate= 0.1kbits/s speed=3.52x Last message repeated 8 times Past duration 0.999992 too large 0kB time=00:00:14.14 bitrate= 0.0kbits/s speed= 9.4x Last message repeated 11 times Past duration 0.999992 too large 512kB time=00:00:31.55 bitrate= 132.9kbits/s speed=12.6x Last message repeated 24 times Past duration 0.999992 too large= 1280kB time=00:01:02.12 bitrate= 168.8kbits/s speed=17.7x Last message repeated 29 times [Parsed_testsrc_0 @ 0x7fe6efd02500] EOF timestamp not reliable Past duration 0.999992 too large Last message repeated 2 times frame= 4002 fps=1079 q=-1.0 Lsize= 1558kB time=00:01:07.71 bitrate= 188.5kbits/s speed=18.3x video:457kB audio:1003kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 6.756100% [libx264 @ 0x7fe6f0816e00] frame I:17 Avg QP:16.61 size: 3274 [libx264 @ 0x7fe6f0816e00] frame P:1766 Avg QP:17.44 size: 200 [libx264 @ 0x7fe6f0816e00] frame B:2219 Avg QP:16.02 size: 27 [libx264 @ 0x7fe6f0816e00] consecutive B-frames: 25.5% 1.2% 1.0% 72.2% [libx264 @ 0x7fe6f0816e00] mb I I16..4: 65.0% 7.0% 27.9% [libx264 @ 0x7fe6f0816e00] mb P I16..4: 1.7% 0.4% 0.1% P16..4: 9.0% 2.5% 1.2% 0.0% 0.0% skip:85.0% [libx264 @ 0x7fe6f0816e00] mb B I16..4: 0.1% 0.0% 0.0% B16..8: 2.2% 0.0% 0.0% direct: 0.0% skip:97.6% L0:51.7% L1:47.4% BI: 0.9% [libx264 @ 0x7fe6f0816e00] 8x8 transform intra:13.5% inter:56.1% [libx264 @ 0x7fe6f0816e00] coded y,uvDC,uvAC intra: 5.6% 24.2% 11.7% inter: 0.5% 2.6% 1.0% [libx264 @ 0x7fe6f0816e00] i16 v,h,dc,p: 77% 5% 3% 15% [libx264 @ 0x7fe6f0816e00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 12% 58% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fe6f0816e00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 47% 26% 18% 2% 2% 2% 1% 2% 0% [libx264 @ 0x7fe6f0816e00] i8c dc,h,v,p: 25% 14% 49% 12% [libx264 @ 0x7fe6f0816e00] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fe6f0816e00] ref P L0: 72.2% 5.6% 18.7% 3.5% [libx264 @ 0x7fe6f0816e00] ref B L0: 54.0% 40.8% 5.2% [libx264 @ 0x7fe6f0816e00] ref B L1: 97.6% 2.4% [libx264 @ 0x7fe6f0816e00] kb/s:56.01 [aac @ 0x7fe6f0814a00] Qavg: 8128.149