Blend video filter *_expr options don't handle store and load functions correctly.

[Fallan]

For example make a local gradient over normalized height 0.2 starting at 0.5, which will be used to transition between 2 input videos. E.G. the inputs could be Big Buck Bunny variations, the first having been self-modified by blend-glow and the second by blend-burn:
ffmpeg -i B3.mov -vf "split[F][G];[F][G]blend=all_mode=glow" -c:v ffv1 -an glow.mkv
and similarly burn.mkv with blend=c0_mode=burn. Now I should be able to do
ffmpeg -i burn.mkv -i glow.mkv -filter_complex "[0:v][1:v]blend=all_expr=st(0,clip((Y/H-0.5)/0.2\,0.0\,1.0))\;A*ld(0)+B*(1.0-ld(0))" -c:v libx265 -crf 16 -preset slow -an output.mkv
But the result has random snow obscuring the image. It works fine if I omit st() and replace the 2 ld()'s with clip()'s but that takes longer to run.
I am using ffmpeg version "2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev" and it happens in older versions too.
I suspect this is known behavior which has been described in other context as "stored variables not having global scope" and they don't transmit between color planes but here we have the same color plane and frame, just different streams that are already in the same scope in avfilter. It sure would be beneficial if this worked.

[Michael Koch]

I can reproduce the problem.

This does not work:
ffmpeg -f lavfi -i color=red -f lavfi -i color=yellow -lavfi blend=all_expr='st(0,clip((Y/H-0.5)/0.2,0,1));A*ld(0)+B*(1.0-ld(0))' -t 5 -y out.mp4

But the same expression without ld() and st() does work:
ffmpeg -f lavfi -i color=red -f lavfi -i color=yellow -lavfi blend=all_expr='A*clip((Y/H-0.5)/0.2,0,1)+B*(1.0-clip((Y/H-0.5)/0.2,0,1))' -t 5 -y out.mp4

[Michael Koch]

Simplified examples for reproducing:

This does not work:
ffmpeg -f lavfi -i color=red -f lavfi -i color=yellow -lavfi blend=all_expr='st(0,Y/H);A*ld(0)+B*(1-ld(0))' -frames 1 -y out1.png

This does work:
ffmpeg -f lavfi -i color=red -f lavfi -i color=yellow -lavfi blend=all_expr='A*Y/H+B*(1-Y/H)' -frames 1 -y out2.png

It seems the memory for variable 0 is overwritten by something else in most cases (but not always).

[Fallan]

Some more information that might be useful: I ran Michael Koch's simplified example using st and ld (with "-frames=25" to avoid error message) for ffmpeg versions 6.1, 5.1 and 4.4 and found the same kind of problematic noisy result but not exactly the same. Also, I ran the recent version 7 master several times and got different random noise each time; feeding the outputs into hstack makes an easy comparison.

So the "overwriting" of the stored variable is not systematic, it's as if its memory is mistakenly freed and used for some other things at random.

[Marton Balint]

This is kind of expected if you are using multiple threads with the blend filter. If you specify threads=1 as an additional parameter of blend, it will work as expected.

[Michael Koch]

It does indeed work with threads=1 as an additional parameter of blend.

But this parameter is missing in official documentation.

Why doesn't it work if the option -filter_threads 1 is used instead? I did already try that.

Why does the geq filter work just fine with many ld() and st() in the expressions, without requiring any limitations to the number of threads?

[Marton Balint]

Fixed in 64330e365b97d9b8304a9d3d306581b6c14a773b.

Now blend filter will use a separate expression state for each slice thread, similar to the geq filter.

[Fallan]

Thank you for fixing this.