Invalid pal8 -> gbrp conversion

[Robin]

Summary of the bug:
ffmpeg is not correctly converting pal8 input to gbrp output. I discovered this when encoding optimized PNGs to UTVideo.

I have attached a .7z with two PNGs. One is original PNG from Elephants Dream Frame 13 in RGB24 and the other is optimized PNG in pal8.

How to reproduce:

1. Confirm both PNGs are identical:

ffmpeg -i ref-rgb24.png ref-rgb24.bmp
ffmpeg -i opt-pal8.png -vf format=bgr24 opt-pal8.bmp

You will have to force ffmpeg to convert to BGR24 to match the reference colorspace output. MD5 of both outputs should match 735ee7c8b493855480fd0f5847a66132.

This means pal8 -> bgr24 conversion is correct and matches the rgb24 -> bgr24 conversion output.

2. Force conversion to gbrp before final output:

ffmpeg -i ref-rgb24.png -vf format=gbrp ref-rgb24.bmp
ffmpeg -i opt-pal8.png -vf format=gbrp opt-pal8.bmp

MD5s of output BMP will no longer match. ref-rgb24.bmp will match original BMP output from step 1 and opt-pal8.bmp output in this step will be different.

3. Test alternate colorspace path:

ffmpeg -i opt-pal8.png -vf format=bgr0 opt-pal8.bmp
ffmpeg -i opt-pal8.png -vf format=bgr0,format=gbrp opt-pal8.bmp

Both these produce the correct output. Meaning pal -> bgr0 and bgr0 -> gbrp are correct.

Another way to confirm this is encoding to UTVideo which uses gbrp colorspace.

1. Encode frames to UTVideo:
ffmpeg -i ref-rgb24.png -c:v utvideo ref-rgb24.mkv
ffmpeg -i opt-pal8.png -c:v utvideo opt-pal8.mkv

This encodes the frames to gbrp colorspace.

2. Decode back to raw BMP:
ffmpeg -i ref-rgb24.mkv ref-rgb24.bmp
ffmpeg -i opt-pal8.mkv opt-pal8.bmp

Checksums are different. ref-rgb24.bmp is correct.

3. Test alternate colorspace path:
ffmpeg -i opt-pal8.png -vf format=bgr0 -c:v utvideo opt-pal8.mkv
ffmpeg -i opt-pal8.mkv opt-pal8.bmp

Output BMP now has correct checksum.

EDIT: Rewrote this whole ticket to simplify it.

[Balling]

There are some ​https://trac.ffmpeg.org/query?status=!closed&keywords=~pal8 open bugs about pal8 and some about alpha. Please check whether it is one of those.

[Robin]

Looks like it maybe related to #1286 which doesn't give me much hope this will ever be fixed since has been opened for a decade.

However that ticket is reporting color conversion errors TO pal8. This ticket is showing conversions errors are occurring FROM pal8 TO specific formats.

produces errors: pal8 -> gbrp
correct: pal8 -> bgr0
correct: pal8 -> bgr0 -> gbrp
correct: pal8 -> pal8

#1286:
produces errors: rgb24 -> pal8
correct: pal8 -> pal8

On a sidenote, it appears Irfanview also gives incorrect results converting to pal8. However, both ffmpeg and irfanview give different results when converting to pal8.

I confirmed by taking a bgr24 BMP and performing a down conversion to pal8 and then back to bgr24. The resulting pal8 BMPs were different from each other and the following bgr24 were also different, both from each other and from the reference bgr24 BMP.

Both Irfanview and ffmpeg give the same results up converting pal8 BMPs to bgr24 BMPs.

[Robin]

Rewrote this to be easier to follow and understand now.

[Carl Eugen Hoyos]

yuv444p is incorrectly used as an intermediate format.

[Niklas Haas]

Fix: https://ffmpeg.org//pipermail/ffmpeg-devel/2024-December/337639.html

[Niklas Haas <git@haasn.dev>]

In 8cf2d972/ffmpeg:

swscale/unscaled: add pal8 -> gbr(a)p special converter

Fixes: #9520
Signed-off-by: Niklas Haas <​git@haasn.dev>
Sponsored-by: Sovereign Tech Fund