Wrong value is written to *.pgm file

[Michael Koch]

If a function in the geq filter evaluates to 'nan', this result is silently changed to zero when writing to a *.pgm file. That's not correct. Either there should be an error message, or the maximum possible value (65535) could be written to indicate that there was a problem.

Can be reproduced as follows:

ffmpeg -f lavfi -i nullsrc=size=2x2 -vf format=pix_fmts=gray16le,geq='sqrt(-1)' -frames 1 -y test.pgm

Check the output file with a hex editor. It contains '00 00' for the pixel values.

[Elon Musk]

Using 0 is better.

[Michael Koch]

Replying to Elon Musk:

Using 0 is better.

Why is 0 better? I'm making *.pgm files for the remap filter. In this case 0 is a valid output value. 65535 is not, and would force remap to use the "fill" color. It's very difficult to debug complicated functions if a wrong result is written to the output file and no error message is thrown.

[Elon Musk]

Well, you could write video filter that would map such float values to arbitrary values.

[Elon Musk]

Also, do your homework, geq can make use of isnan() function so you can do your own mapping inside geq filter.

[Michael Koch]

A good function should never have 'nan' as the result. But during debugging it sometimes happens that something is wrong in the function. In this case it's better to throw an error message, instead of silently converting 'nan' to zero.
For debugging it would also be helpful to print the geq filters input variables X and Y in the error message.