scale2ref is one frame behind

[Jozef Chutka]

Summary of the bug:
When using scale2ref filter with dynamic ref size, the resized element does not match the ref exactly (lags 1 frame?)

In the following example [w]hite rect grows 1px every frame and [r]ed tries to match. However when rendered on top of each other, the red one is 1px smaller leaving 1px white border around.

ffmpeg -filter_complex "color=color=#000000:rate=20:size=400x400[b];color=color=#ffffff:rate=20:size=100x100,scale=width=iw+n:height=ih+n:eval=frame[w];color=color=#ff0000:rate=20[r];[r][w]scale2ref=eval=frame[r][w];[w][r]overlay[w];[b][w]overlay" -t 4 -y gif.mp4

ffmpeg version 2022-02-24-git-8ef03c2ff1-full_build-www.gyan.dev

[Gyan]

The issue is not with scale2ref. Insert a showinfo to check both outputs of scale2ref and compare sizes for frames with same PTS. They match. I haven't looked further but it's likely an issue with overlay framesync.

[Elon Musk]

Adding 1 or 2 null filters at right spot fixes issue for me:

ffplay -f lavfi -i "color=color=#000000:rate=25:size=400x400,format=yuv444p,setpts=N[b];color=color=#ffffff:rate=25:size=100x100,format=yuv444p,scale=width=iw+n:height=ih+n:eval=frame[w];color=color=#ff0000:rate=25,format=yuv444p[r];[r][w]scale2ref=eval=frame[r][w];[w]null[w];[r]null[r];[w][r]overlay=format=auto,setpts=N[w];[b][w]overlay=format=auto"

[Jozef Chutka]

Thanks for looking into it and proposing a workaround. Extra null works on my end as well.

Is there any work in progress (or will be) to fix overlay itself?

[Elon Musk]

Its not overlay filter or framesync bug, but how avfilter count frames, needs look for case where it will skip increasing it.

[Michael Koch]

When I saw this issue, I first thought it's because the inputs might be yuv420 and the size is incremented in steps of one. So I added a few "format=rgb24" and "format=auto".

ffmpeg -filter_complex "color=color=#000000:rate=20:size=400x400,format=rgb24[b];color=color=#ffffff:rate=20:size=100x100,scale=width=iw+n:height=ih+n:eval=frame,format=rgb24[w];color=color=#ff0000:rate=20,format=rgb24[r];[r][w]scale2ref=eval=frame[r][w];[w][r]overlay=format=auto[w];[b][w]overlay=format=auto" -t 4 -y gif.mp4

But now ffmpeg never stops. Is there an explanation for this behaviour?

Michael