libaom-av1 can encode rgb av1 webm files but not decode them (libdav1d decoder not affected)

[av1masterrace]

Summary of the bug:
If I use the newest version to encode a gif to an av1 webm it will choose gbrp which is great because it keeps the colours the same. It also decodes fine in Chromium and mpv.
However in VLC and ffplay white becomes pink and it all looks horrible. Same when decoding into another format with ffmpeg.
I don't know whether this also affects rgb av1 in other containers like mkv or mp4.

How to reproduce:

% ffmpeg -i input.gif -c:v av1 output.webm
% ffmpeg -i output.webm pink.gif

[av1masterrace]

Okay, I just tested it. It affects av1 no matter the container. I tested webm, mkv and mp4 with the same result. It all turns pink when viewing with ffplay.

[Carl Eugen Hoyos]

For future tickets: Please always test current FFmpeg git head and provide the command line together with the complete, uncut console output to make your tickets valid.

[pdr0]

Does AV1 support GBRP or any "RGB" derivative ?

ffmpeg -h encoder=librav1e
Supported pixel formats: yuv420p yuvj420p yuv420p10le yuv420p12le yuv422p yuvj422p yuv422p10le yuv422p12le yuv444p yuvj444p yuv444p10le yuv444p12le

ffmpeg -h encoder=libaom-av1
Supported pixel formats: yuv420p yuv422p yuv444p yuv420p10le yuv422p10le yuv444p10le yuv420p12le yuv422p12le yuv444p12le

aom aomenc.exe does not support rgb, gbrp either, only YUV formats

[Balling]

​https://patchwork.ffmpeg.org/project/ffmpeg/patch/CAB0OVGpWUwn=NZ27b0bqeDqwr5MErfaUhPVpV0GBp2bYRjZYUQ@mail.gmail.com/

[av1masterrace]

So is it the encoder or the decoder that is misbehaving?
I want to know before I start encoding lots of webms in rgb and have to do it all over again.

[Balling]

Replying to av1masterrace:

So is it the encoder or the decoder that is misbehaving?
I want to know before I start encoding lots of webms in rgb and have to do it all over again.

Decoder.

[av1masterrace]

Replying to Balling:

Decoder.

Thank you, now I can continue encoding stuff without worries!

[petr.diblik]

Replying to pdr0:

Does AV1 support GBRP or any "RGB" derivative ?

ffmpeg -h encoder=libaom-av1
Supported pixel formats: yuv420p yuv422p yuv444p yuv420p10le yuv422p10le yuv444p10le yuv420p12le yuv422p12le yuv444p12le

That command gives me:

ffmpeg version 2021-05-30-git-51f1194eda-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.3.0 (Rev2, Built by MSYS2 project)
Supported pixel formats: yuv420p yuv422p yuv444p gbrp yuv420p10le yuv422p10le yuv444p10le yuv420p12le yuv422p12le yuv444p12le gbrp10le gbrp12le gray gray10le gray12le

So gbrp is there.

[petr.diblik]

Replying to Balling:

​https://patchwork.ffmpeg.org/project/ffmpeg/patch/…

The patch seems OK. What has been preventing it from merging for 8 months?

[Balling]

Replying to petr.diblik:

Replying to Balling:

​https://patchwork.ffmpeg.org/project/ffmpeg/patch/…

The patch seems OK. What has been preventing it from merging for 8 months?

That is because Carl does not know the difference between AVCOL_TRC_IEC61966_2_1 transfer and primaries, which are indeed AVCOL_PRI_BT709 for sRGB.
The patch is also named wrong, what that patch does is that it allows sRGB (== AVCOL_TRC_IEC61966_2_1) not RGB to be decoded.
His patch should have tested for the only thing that says it is RGB, not YCbCr: AVCOL_SPC_RGB is set and AVCOL_PRI_SMPTE428 is not set.
YCbCr can also have AVCOL_TRC_IEC61966_2_1, that is WHAT jpeg actually IS.
So the patch is wrong.

VLC is just broken. Sigh. As for pink.gif I cannot reproduce it. Or ffplay for that matter.

[Balling]

Can you attach a input.gif sample? Samples i have I cannot reproduce upon.

[petr.diblik]

I attached 2 test files. This is how I converted the first one to the second one:

ffmpeg -i vid-h264-gbrp.mkv -c:v libaom-av1 -crf 0 vid-av1-gbrp.mkv

Then this plays incorrectly:

ffplay -vcodec libaom-av1 vid-av1-gbrp.mkv

And this plays correctly:

ffplay -vcodec libdav1d vid-av1-gbrp.mkv

[Balling]

Oh, I could not reproduce it because here libdav1d is the default decoder! Ha.

ffplay -vcodec libaom-av1 vid-av1-gbrp.mkv makes it pinky/green indeed (-c:v does not work in ffplay, WTF)!

[Balling]

I have sent a patch for this that will correctly fix this (Carl, maybe you will merge this ASAP).

https://ffmpeg.org/pipermail/ffmpeg-devel/2021-June/280964.html

[petr.diblik]

Replying to Balling:

I have sent a patch for this that will correctly fix this (Carl, maybe you will merge this ASAP).

This sounds promising, thanks.

BTW, OMG, YouTube is affected too…

[Balling]

Replying to petr.diblik:

Replying to Balling:

I have sent a patch for this that will correctly fix this (Carl, maybe you will merge this ASAP).

This sounds promising, thanks.

BTW, OMG, YouTube is affected too…

It never promised support for RGB anyway (and this is actually sRGB). Chrome does support bit perfect AV1 RGB decoding though. At least on this sample ​https://trac.ffmpeg.org/attachment/ticket/9167/av1_rgb.webm it works perfectly on Nvidia Turing (Windows, of course). It may work worse on previous to Turing (without software overlays from Turing).

[petr.diblik]

IMHO the support of RGB and its variants in AV1 codecs should depend on ​the AV1 spec. I checked it and it seems to me that it's not supported. Could someone clarify pls?

[Balling]

Replying to petr.diblik:

IMHO the support of RGB and its variants in AV1 codecs should depend on ​the AV1 spec. I checked it and it seems to me that it's not supported. Could someone clarify pls?

The keyword to search is MC_IDENTITY. MC_IDENTITY means RGB. Sigh. And no, Youtube always reencods since they are targeting not high quality with AV1 but lower bitrate. So RGB will have to be reencoded to YCbCr, I suppose.

v3 of my patch. ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210606051612.89211-1-val.zapod.vz@gmail.com/

[petr.diblik]

Replying to Balling:

Replying to petr.diblik:
The keyword to search is MC_IDENTITY. MC_IDENTITY means RGB. Sigh. And no, Youtube always reencods since they are targeting not high quality with AV1 but lower bitrate. So RGB will have to be reencoded to YCbCr, I suppose.

v3 of my patch. ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210606051612.89211-1-val.zapod.vz@gmail.com/

Thanks for the clarification and the patch.
By YouTube being affected I mean that an uploaded video (whatever it's reencoded to) has wrong colors.

[petr.diblik]

I also opened a bug report in aomedia issue tracker: ​https://bugs.chromium.org/p/aomedia/issues/detail?id=3109

[Balling]

I will just point out that AV1 spec does specify that it depends on H.273 and thus must support all in H.273, that includes other R'G'B' types besides sRGB, that is so comfortably mentioned in the spec, ffmpeg does support it all, so my PATCH IS CORRECT.

[Balling]

Finally fff4dbd90b87f5dc5298394cb0ab38d4aa147bda.