Opened 2 years ago

Closed 11 months ago

#9485 closed defect (fixed)

Video frame drops when composing video from multiple generative video filters

Reported by: megpoid Owned by:
Priority: normal Component: avfilter
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description (last modified by megpoid)

Summary of the bug:
When trying to use multiple audio visualizer filters and compose them together to a single video stream, some filters will fail to output at the requested frame rate and some images coming from the filter will be duplicated.

How to reproduce:

% ffmpeg -v verbose -i input.mp3 -filter_complex "[0:a]avectorscope=s=256x824:r=30[a]; [0:a]showcqt=s=1664x1080:text=0:r=30[b]; [0:a]showwaves=s=256x256:mode=line:draw=full:r=30[c]; [a][c]vstack[d]; [b][d]hstack[out]" -map "[out]" -map 0:a -c:a libopus -vcodec libx264 -preset ultrafast -pix_fmt yuv444p10le -t 60 -crf 23 -r 30 out0.mkv 2>log0.txt


When watching the output video, between timestamps 00:00.307 and 00:00:340, or timestamps 00:03:407 and 00:03.440, you can see that showcqt filter's output on the left has duplicate frames. Between timestamps 00:03.874 and 00:03.907, the showwaves filter's output in the bottom-right corner also froze for a frame.
ffmpeg version N-104443-gdd7a61f62f-g783935de18+1
built on 2021 October 29th

Pastebin of the error log file: https://pastebin.com/5RRWFjzm or https://crypt.is-by.us/?5e76d7f27a7e50a7#Eqmqrdaj2Q5hNuCOMnycs6jYx1KnYm5FniNE3B5OA1o=
I have uploaded a file named ticket9485out01.10mb.mkv that demonstrates the issues described above.

Known workaround
This issue can be worked around by first encoding each individual filter into a PNG image sequence, then composing them in a second run, then converting to the desired format in a third run, like so:

ffmpeg -i input.mp3 -filter_complex "[0:a]avectorscope=s=256x824:r=30[out]" -map "[out]" -t 60 -r 30 a_%05d.png

ffmpeg -i input.mp3 -filter_complex "[0:a]showcqt=s=1664x1080:text=0:r=30[out]" -map "[out]" -t 60 -r 30 b_%05d.png

ffmpeg -i input.mp3 -filter_complex "[0:a]showwaves=s=256x256:mode=line:draw=full:r=30[out]" -map "[out]" -t 60 -r 30 c_%05d.png

ffmpeg -r 30 -i a_%05d.png -r 30 -i b_%05d.png -r 30 -i c_%05d.png -filter_complex "[0:v][2:v]vstack[a]; [1:v][a]hstack[out]" -map "[out]" -t 60 -r 30 d_%05d.png

ffmpeg -r 30 -i d_%05d.png -i input.mp3 -acodec libopus -vcodec libx264 -preset ultrafast -pix_fmt yuv444p10le -t 60 -crf 23 -r 30 output.mkv

I have permission in writing to redistribute the attached audio file as long as credit is given: https://soundcloud.com/polyak

Attachments (2)

input.mp3 (1.8 MB ) - added by megpoid 2 years ago.
log01.txt (1.1 MB ) - added by megpoid 2 years ago.
Log file from encoding attempt

Change History (4)

by megpoid, 2 years ago

Attachment: input.mp3 added

by megpoid, 2 years ago

Attachment: log01.txt added

Log file from encoding attempt

comment:1 by megpoid, 2 years ago

Description: modified (diff)

comment:2 by Elon Musk, 11 months ago

Component: undeterminedavfilter
Reproduced by developer: set
Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.