Swscale matrix is not autopropogated for types of YCbCr

[kvsico]

Summary of the bug:
I want to generate a video from an image, but the color in video is different from the color in image.

1.png: #ee4d2c
2.mp4: #fd5a2a

How to reproduce:

% ffmpeg -y -t 4 -loop 1 -i 1.png  2.mp4

[pdr0]

Not a bug.

8bit RGB vs. 8bit YUV

You need to encode either using 8bit RGB , or 10bit YUV

Neither are as compatible with most devices as 8bit YUV, only the latter is more common for distribution formats

ffmpeg -y -t 4 -loop 1 -i 1.png -c:v libx264rgb 2.mp4

[Balling]

Replying to pdr0:

Not a bug.

8bit RGB vs. 8bit YUV

No, it has nothing to do with this. Anyway, I could not even launch the command, the png is too big...

[pdr0]

Replying to Balling:

Replying to pdr0:

Not a bug.

8bit RGB vs. 8bit YUV

No, it has nothing to do with this. Anyway, I could not even launch the command, the png is too big...

It has something to with that

You can -vf scale...

You're assuming that he's using a different matrix for playback ?

One possible difference is the matrix used for YUV<=>RGB, but even if you use the correct matching matrix for both directions (and back to RGB for display), you will still get slightly off colors

You will usually not get exact colors from 8bit RGB with 8bit YUV because the color models do not overlap perfectly and you will get rounding errors +/-3, even with correct matching matrix transform YUV<=>RGB in either directions

ffmpeg -y -t 4 -loop 1 -i 1.png -vf scale=1302:2183:out_color_matrix=bt709 -x264opts colormatrix=bt709 3.mp4

It's closer, but still off

original RGB 238,77,45

3.mp4 (8bit YUV444) converted to YUV with 709 and back to 8bit RGB for display with 709 RGB 239,78,44

2.mp4 (libx264rgb) RGB 238,77,45

4.mp4 (10bit YUV) converted to YUV with 709 and back to 8bit RGB for display with 709 RGB 238,77,45

[Balling]

Replying to pdr0:

You can -vf scale...

No, you do not. The problem here is quite obvious: you need to remember what ffmpeg defaults to.

And now you need to remember what your source image is. The answer is quite obvious: it is sRGB, as everything PNG not tagged with chunks gAMA or cHRM or iCCP.

Now, what is the only matrix available for sRGB in the standards (iec61966_2_1 am. 1)? Right, BT.601 and such YCbCr colorspace is called sYCC.

Now, next. What is the color primaries of sRGB? Right: they are BT.709.

So your command (with smaller png, 1080p maybe, is this a regression that bigger files are not supported??) is:

ffmpeg -y -t 4 -loop 1 -i 1.png -color_primaries bt709 -color_trc iec61966_2_1 -colorspace smpte170m 2.mp4

So here it is. The color is the same (P.S. I was wrong, it is 0, 0, -1) as in source picture (I am not so sure it will be for all colors, though, because we now have limited range!!!). Of course you need a good player for this that supports sYCC:

mpv.com --no-config 2.mp4

The result is that the very same color is there.

Again, you can rescale all of it, however you wanna. Even change the lattice (chroma siting only applies to 4:2:0). Just be careful.

I am not setting full range, because I am sure mpv is broken in full range.

[Balling]

because the color models do not overlap perfectly

They do not overlap perfectly. Of course if you are doing limited vs. full range it is obvious. But even if you do full vs full, some YCbCr colors will be out-of-gamut for RGB. For example, YCbCr limited BT.709 values 139, 151, 24 R'G'B' is -21, 182, 181 (which is what is used for xvYCC, but that is another topic). The same about full range, of course.

You can use calculator here: ​https://res18h39.netlify.app/color It the only one I found that is correct including BT.1886/XYZ and other stuff. Of course no way to reshape BT.1886 which is not perfect 2.4 gamma.

Now with 4:2:0 you are able to do any color but if you fix one pixel of one color the other pixel neer if different color will have to be off-by-one or worse.

[pdr0]

Replying to Balling:

Replying to pdr0:

You can -vf scale...

No, you do not. The problem here is quite obvious: you need to remember what ffmpeg defaults to.

And now you need to remember what your source image is. The answer is quite obvious: it is sRGB, as everything PNG not tagged with chunks gAMA or cHRM or iCCP.

Now, what is the only matrix available for sRGB in the standards (iec61966_2_1 am. 1)? Right, BT.601 and such YCbCr colorspace is called sYCC.

Now, next. What is the color primaries of sRGB? Right: they are BT.709.

So your command (with smaller png, 1080p maybe, is this a regression that bigger files are not supported??) is:

ffmpeg -y -t 4 -loop 1 -i 1.png -color_primaries bt709 -color_trc iec61966_2_1 -colorspace smpte170m 2.mp4

So here it is. The color is the same as in source picture (I am not so sure it will be for all colors, though, because we now have limited range!!!). Of course you need a good player for this that supports sYCC:

mpv.com --no-config 2.mp4

The result is that the very same color is there.

Again, you can rescale all of it, however you wanna. Even change the lattice. Just be careful.

I am not setting full range, because I am sure mpv is broken in full range.

It's close, but the colors are not exactly the same

It's close in MPV, MPCHC, VLC, but with the expected +/- rounding errors.
Original 238,77,45
MPV 237,76,44
MPCHC 237,77,44
VLC 237,76,44
FFplay 252,90,41
Potplayer 251,89,39

FFplay and Potplayer are way off

If using 8bit YUV, I would use 709 and do it the other way with -vf scale, because more players will have similar close values +/-3 . Every player including FFplay, Potplayer, dozens others will look close, including browsers

Or if you want exact numbers, use 10bit YUV or RGB

[pdr0]

Replying to Balling:

because the color models do not overlap perfectly

They do not overlap perfectly. Of course if you are doing limited vs. full range it is obvious. But even if you do full vs full, some YCbCr colors will be out-of-gamut for RGB. For example, YCbCr limited BT.709 values 139, 151, 24 R'G'B' is -21, 182, 181 (which is what is used for xvYCC, but that is another topic). The same about full range, of course.

You can use calculator here: ​https://res18h39.netlify.app/color It the only one I found that is correct including BT.1886/XYZ and other stuff. Of course no way to reshape BT.1886 which is not perfect 2.4 gamma.

Now with 4:2:0 you are able to do any color but if you fix one pixel of one color the other pixel neer if different color will have to be off-by-one or worse.

Yes, he's using 4:4:4; we are assuming no subsampling, or things like lossy differences

For out of gamut - it does not apply in this situation because you're starting with RGB . All out of gamut values have already been eliminated by definition. (Assuming the absence of lossy encoding, chroma subsampling or other transforms applied)

All 8bit RGB colors are represented by 8bit YUV ; the problem is there are too many values; many YCbCr value map to the same RGB value. Hence the errors. 10bit YUV is required for the precision to specify exact 8bit RGB color

[Balling]

Replying to pdr0:

It's close, but the colors are not exactly the same

It's close in MPV, MPCHC, VLC, but with the expected +/- rounding errors.
Original 238,77,45
MPV 237,76,44
MPCHC 237,77,44
VLC 237,76,44
FFplay 252,90,41
Potplayer 251,89,39

FFplay and Potplayer are way off

If using 8bit YUV, I would use 709 and do it the other way with -vf scale, because more players will have similar close values +/-3 . Every player including FFplay, Potplayer, dozens others will look close, including browsers

Or if you want exact numbers, use 10bit YUV or RGB

MPC uses FFmpeg and does not really support sRGB in video. So does VLC (and very old version) and FFplay. FFplay does not support colorspaces good enough, because SDL library.

The only one player that supports those very non-standard for video sRGB is mpv.

I do not know what version of mpv you use, but here on Windows (Windows has much better pow function, just for example, LOL) mpv does give 238, 77, 44. Just saying, I will attach a sample.

will look close, including browsers

No. We in Chrome use sRGB EOTF for BT.709 OETF. So you are very off here. Unfortunately no choice here, everything else is done for sRGB ambient light, we cannot use BT.1886 as it is for dark ambient. Maybe when we will get TrueTone fully...

lossy differences

There are no lossy differences on one color. At least in x264. 4:2:0 also preserves all colors, on one color that is.

All out of gamut values have already been eliminated by definition

I would not be so sure about that, use --gamut-warning in mpv. You will be surprised. On default setting x264 does produce out-of-gamut (unless there is another bug in --gamut-warning). Oopsie. Use this bmp sample for example: ​https://video.stackexchange.com/questions/19944/ffmpeg-bmp-to-yuv-x264-color-shift It gives superblack.

[Balling]

sRGB with BT.601 a.k.a. sYCC

[pdr0]

Replying to Balling:

The only one player that supports those very non-standard for video sRGB is mpv.

Right, so why make a non standard video that only plays correctly in a small% of players ? The numbers are slightly off anyways

I do not know what version of mpv you use, but here on Windows (Windows has much better pow function, just for example, LOL) mpv does give 238, 77, 44. Just saying, I will attach a sample.

I'm using windows mpv-x86_64-20210316-git-5824d9f , and the color varies in each channel if you move a color picker slightly - it fluctuates a bit, almost as if output is dithered. Either way, it's always going to be off +/-3 when you check colors for 8bit YUV. It's mathematically not possible to have exact perfect colors for 8bit YUV from 8bit RGB for the majority of colors. You always expect +/-3 errors in each channel

It's just that more players assume "709" for "HD" resolutions, so you're going to have similar colors +/-3 more often in more players (instead of way off like +/-30). Some players do not even read flags at all. So it's smarter to do the actual conversion with 709 matrix and flag it 709.

No. We in Chrome use sRGB EOTF for BT.709 OETF.

Is that hardcoded, or can video metadata VUI flags influence that ?

lossy differences

There are no lossy differences on one color. At least in x264. 4:2:0 also preserves all colors, on one color that is.

Yes - that's what I'm saying; for the purposes of this math/conversion discussion , we're not using like crf 40 or something on a typical content (ie not single color), where you'd
contaminate the observations with lossy enocoding errors. You're not using subsampling, where the kernal used for resampling the U,V planes can and WILL generate YUV values that produce negative RGB values (out of gamut)

All out of gamut values have already been eliminated by definition

I would not be so sure about that, use --gamut-warning in mpv. You will be surprised. On default setting x264 does produce out-of-gamut (unless there is another bug in --gamut-warning). Oopsie. Use this bmp sample for example: ​https://video.stackexchange.com/questions/19944/ffmpeg-bmp-to-yuv-x264-color-shift It gives superblack.

I only see a PNG there, no BMP. Do you have the BMP, or should I convert PNG to BMP? Or just use those depicted RGB values?

Note when you use -pix_fmt yuv420p such as in that link, ffmpeg is using bicubic resampling for the chroma planes by default, so you would expect out of gamut values. Negative lobe and "sharper" kernals will generate more out of gamut values, right around where the color borders are adjacent. Especially colors against black and white (min and max) values.

[pdr0]

double post

[Balling]

Replying to pdr0:
Well, first of all I consider H.273 to be mandatory. If it is not supported, then it should be fixed. We consider to fix it in Chrome, already added YCgCo with SW overlays from Nvidia and what not. Time for ICtCp patches in AVIF and then to Chromium.

I also consider JPEG CMYK to be mandatory.

I'm using windows mpv-x86_64-20210316-git-5824d9f

Just updated to this version, I still have perfect results on Windows, the same color. VERY strange, but it is what it is. I mean here is the screenshot if do not believe me. ​https://fastpic.ru/view/114/2021/0331/_30edaccc5667f45bb40a59a57bbe7d0a.png.html

Is that hardcoded, or can video metadata VUI flags influence that ?

VUI flags are just an OETF for most cases, unless the EOTF is defined too in the standard, like for sRGB and SMPTE270M (here we do apply the needed transforms). In some cases like with xvYCC there is no EOTF at all. You can use BT.1886 for the BT.709 or BT.601 core of xvYCC but for extended gamut use of BT.1886 is inadequate, just like for HDR BT.1886 is not good enough, it was modified in BT.2390. So to answer your question, it is not hardcoded, just for the EOTF in case of BT.2020 and BT.709 SDR we use sRGB EOTF, not BT.1886 as specified by Rec. BT.2020 and BT.709. We are going to reconsider sooner or later, of course. We will need normal EOTFs for constant luminance stuff.

if you move a color picker slightly - it fluctuates a bit -- output is dithered

That is what MadVR does, I am not found of the idea. BT.601 is okay for that, because BT.601 uses different white point, so chromatic adaptation can be done by this hack.

I only see a PNG there, no BMP

Yes, just convert it. BTW, I need to update that question with sRGB video. Ha.

-pix_fmt yuv420p

I do not use it. It is still superblack, I will attach the sample. There are some values that are off-by-one.

[Balling]

another sYCC sample with superblack (use mpv --gamut-warning)

[Balling]

And, BTW, this sYCC color is opening correctly in Chrome browser. It is actually almost 238,77,45, it is 238, 75, 45 there, which may be due to wrong lattice a.k.a. chroma position (as you can see on top level of pixels that are now white and not red P.S. this was in source png file) in Microsoft Media Foundation Framework. (That is if you have Turing GPU.)

[pdr0]

Replying to Balling:

Just updated to this version, I still have perfect results on Windows, the same color. VERY strange, but it is what it is. I mean here is the screenshot if do not believe me. ​https://fastpic.ru/view/114/2021/0331/_30edaccc5667f45bb40a59a57bbe7d0a.png.html

I believe you, but not perfect as in off by -1 in B. I'm just seeing different results, if you want I can post screenshot too

I think there are some differences in mpv config locally with that build. I think there are differences in output display chain, gpu, or renderer setup. Mine has slight variations between pixels or if I move cursor. It's also reflected in 3rd party screenshot tools of mpv and color pickers

But if I use mpv.net (a GUI for mpv), it's a solid 238,77,44 (also off by -1 B) . I'll have to look into the configs , but I believe your version of mpv is setup like my version of mpv.net. By my mpv version has some weird config going on

(But you're still going to get +/- 3 values in all channels if you check more colors. If you come across a perfect round trip value, it will only be a few colors by chance. It's a 8bit YUV<=>RGB fact, regardless of transfer, matrix, or primaries, or full vs limited range)

I do not use it. It is still superblack, I will attach the sample. There are some values that are off-by-one.

How are you defining "superblack" ? <0 RGB in any channel ?

You started with values of 0 and 255 in each channel already

Negative RGB values will be clipped to 0,0,0 and >1 will be clipped to 255,255,255 in 8bit integer RGB. You'd have to check the conversion before the clipping

And you always expect off by +/-3 for 8bit RGB/YUV round trip conversions

If you want truly perfect values, you have to use 8bit RGB, or 10bit YUV.

[Balling]

Replying to pdr0:

I believe you, but not perfect as in off by -1 in B.

Oh, wow, I did not see that ;( I looked into topic starter hex values #ee4d2c which he gave wrong. Wow, sorry. Maybe it is RGBA issue? Will have to check.

your version of mpv is setup

I use --no-config :)

How are you defining "superblack" ? <0 RGB in any channel ?

No, standard definition: 16, 128, 128 in limited range YCbCr is black. If Y is less then 16 than it is superblack. 0 is reserved though, in HDMI interface in all components. Not in files AFAIK. So it is 15, 128, 128; 14, 128, 128; ... 1, 128, 128. If Cb or Cr are not 128 there, it is xvYCC already :) Not that superblack is even visable, of course. Only superwhite is, on high end devices.

Again, that is the point: RGB to YCbCr should not introduce superblack, IMHO. But that is just my opinion. D:) Sorry again about that Blue channel.

[pdr0]

Replying to Balling:

Replying to pdr0:

How are you defining "superblack" ? <0 RGB in any channel ?

No, standard definition: 16, 128, 128 in limited range YCbCr is black. If Y is less than 16 than it is superblack. 0 is reserved though, in HDMI interface in all components. Not in files AFAIK. So it is 15, 128, 128; 14, 128, 128; ... 1, 128, 128. If Cb or Cr are not 128 there, it is xvYCC already :) Not that superblack is even visible, of course. Only superwhite is, on high end devices.

But a studio range conversion is typically used on TV's (studio range RGB, Y 0-255 => RGB 0-255) . Reference black is RGB16,16,16 using studio range RGB , reference white is RGB 235,235,235. (this is also called "limited range RGB", or "full range equations") . So Y=14,CbCb128 <=> RGB 14,14,14 . If you look at broadcast standards, EBU r103 etc. this is what they are referring to. Consumer TV's vary widely on how they display, and qualtiy, but for the most part you can see much better superblack and superwhite tones than on a computer monitor

Computers and monitors typically use computer range RGB conversion (also called "full range RGB", or "limited range equations" , Y 16-235 => RGB 0-255 . Range expansion . Reference black is RGB 0,0,0 there. Indeed Y=14 and Y=16 both "map" to RGB 0,0,0

In broadcast and professional video distribution, 0 and 255 are reserved for sync. Y,R,G,B So that bmp is "illegal" too, you'd clip to [1,254] before you do anything; but preferred min/max is usually [5,246]

Again, that is the point: RGB to YCbCr should not introduce superblack, IMHO. But that is just my opinion.

Ok, but now you're mixing up some concepts :)

At one point you were talking about "out of gamut" . It means YCbCr values that "map" to negative RGB, or values >255 in 8bit RGB . (In broadcast their definition of this range is extended to <1 or >254)

You don't get "superblack" Y<16 from converting RGB 0,0,0 to YCbCr when you use standard range equations

Using standard range equations, computer 8bit RGB black (0,0,0) to 8bit YCbCr conversion should be Y=16,CbCr128

Using full range equations, studio RGB 8bit (16,16,16) to 8bit YCbCr conversion should also be Y=16,CbCr128 . But notice reference black is at 16. Black to white is RGB 16-235 in studio range RGB . Black to white is RGB 0-255 in computer RGB.

Note in both cases the YCbCr values are the same, reference black is Y=16, reference white is Y=235 in 8bit values

If you take that bmp , Ymin is 16, Ymax is 235 when you use standard range equations to convert RGB to YUV . No "superblack" by your definition

[Balling]

Replying to pdr0:

but for the most part you can see much better superblack and superwhite tones than on a computer monitor

Superblack must be invisible on default brightness (a.k.a. black level). It should be the same as black that is.

EBU r103

Yeah, I read it.

So Y=14,CbCb128 <=> RGB 14,14,14

I agree.

Consumer TV's vary widely on how they display

I have LG C9: in version 4 of firmware they output superwhite values even in internal player, you can calibrate 8.9% Superwhite overshot (that is as if 255 was not reserved). And it supports both sYCC and xvYCC digital formats (as can be forced by Nvidia control panel digital format option).

In broadcast and professional video distribution, 0 and 255 are reserved for sync.

In HDMI too, the HDMI spec is on archive.org. But not in files.

you'd clip to [1,254] before you do anything;

Frankly speaking Nvidia driver does it for you. Just like MFF can do it.

but preferred min/max is usually [5,246]

Yeah, that is also per EBU R103. But that is for HDR mainly.

No "superblack" by your definition

There is no superblack in bmp. It is full range, superblack does not exist in full range. But it does exist in avc stream. That is again, if there is no another bug in --gamut-warning. See ​https://github.com/mpv-player/mpv/issues/8161

[pdr0]

Replying to Balling:

No "superblack" by your definition

There is no superblack in bmp. It is full range, superblack does not exist in full range. But it does exist in avc stream. That is again, if there is no another bug in --gamut-warning. See ​https://github.com/mpv-player/mpv/issues/8161

I never said there was "superblack" in BMP . Your defintion of "superblack" was Y<16 . BMP is RGB not Y

I said: "If you take that bmp , Ymin is 16, Ymax is 235 when you use standard range equations to convert RGB to YUV . No "superblack" by your definition"

For 3.mp4, the mpv --gamut-warning is purely from lossy encoding. Try --qp 0 or --crf 0. "random" values that end up mapping to out of gamut RGB values.

[Balling]

"random" values that end up mapping to out of gamut RGB values.

Yeah. But lossless AVC is a little bit much for this.

[pdr0]

Replying to Balling:

"random" values that end up mapping to out of gamut RGB values.

Yeah. But lossless AVC is a little bit much for this.

Definitely; but that's just to demonstrate there is no problem with the actual math part of RGB<=>YCbCr conversion

When you're doing this to EBU specs, in addition to the errors/ spurious values from lossy encoding - you're generally subsampling too , which creates many more out of gamut values. Often you need to lowpass or use a professional legalizer to pass checks. But you're allowed 1% out of gamut of active picture area leeway in most places, and that's the recommendation in r103 too

[kvsico]

Replying to pdr0:

Not a bug.

8bit RGB vs. 8bit YUV

You need to encode either using 8bit RGB , or 10bit YUV

Neither are as compatible with most devices as 8bit YUV, only the latter is more common for distribution formats

ffmpeg -y -t 4 -loop 1 -i 1.png -c:v libx264rgb 2.mp4

After using libx264rgb, I have another problem: Chorome can't play the video file libx264rgb.mp4, it shows a black video, here is my cmd:

ffmpeg -y -t 4 -loop 1 -i 1.png -c:v libx264rgb libx264rgb.mp4.

[pdr0]

Replying to kvsico:

After using libx264rgb, I have another problem: Chorome can't play the video file libx264rgb.mp4, it shows a black video, here is my cmd:

ffmpeg -y -t 4 -loop 1 -i 1.png -c:v libx264rgb libx264rgb.mp4.

If it's for chrome, chrome does not have a 10bit pipeline yet either, so there is no benefit

You would have to either use Balling's suggestion , or use a "normal" 709 conversion for AVC. Eitherway, if you test a bunch of colors, they will be off slightly from expected rounding errors during the 8bit RGB=>YUV=>RGB round trip

"normal" 709
ffmpeg -y -t 4 -loop 1 -i 1.png -vf out_color_matrix=bt709 -x264opts colormatrix=bt709 709.mp4

Balling's suggestion
ffmpeg -y -t 4 -loop 1 -i 1.png -color_primaries bt709 -color_trc iec61966_2_1 -colorspace smpte170m 2.mp4

You might be able to use VP9 in RGB mode, not sure if chrome supports it, I'll check later

[Balling]

Replying to pdr0:

You might be able to use VP9 in RGB mode

There is no RGB support yet, at least in avc. There is 10 bit pipeline, but there may be some bugs in it, like HDR propogation on SDR 10 bit video. See ​https://bugs.chromium.org/p/chromium/issues/detail?id=1191915&sort=&groupby=&colspec=&x=&y=&mode=&cells=&num=&q=Hdr&can=1

[pdr0]

Replying to Balling:

Replying to pdr0:

You might be able to use VP9 in RGB mode

There is no RGB support yet, at least in avc. There is 10 bit pipeline, but there may be some bugs in it, like HDR propogation on SDR 10 bit video. See ​https://bugs.chromium.org/p/chromium/issues/detail?id=1191915&sort=&groupby=&colspec=&x=&y=&mode=&cells=&num=&q=Hdr&can=1

Nice to see some 10bit progress!

VP9 RGB doesn't seem to work properly in chrome either (but decodes ok with other decoders)

But interestingly AV1 works for RGB (gbrp), but chrome is +1 blue channel for some reason 238,77,46 . (my chrome is 89 and up to date)

ffmpeg -t 5 -r 1 -loop 1 -i 1280x720_RGB_238,77,45.png -c:v libaom-av1 -crf 18 av1_rgb.webm -y

Color space : RGB
Bit depth : 8 bits
Writing library : Lavc58.132.100 libaom-av1
Default : Yes
Forced : No
Color range : Full
Color primaries : BT.709
Transfer characteristics : sRGB/sYCC
Matrix coefficients : Identity

ffmpeg

Stream #0:0: Video: av1 (High), gbrp(pc, gbr/bt709/iec61966-2-1, progressive),

Other players display it correctly as 238,77,45, including firefox, ffplay, mpv

[kvsico]

Replying to pdr0:

Replying to kvsico:

After using libx264rgb, I have another problem: Chorome can't play the video file libx264rgb.mp4, it shows a black video, here is my cmd:

ffmpeg -y -t 4 -loop 1 -i 1.png -c:v libx264rgb libx264rgb.mp4.

If it's for chrome, chrome does not have a 10bit pipeline yet either, so there is no benefit

You would have to either use Balling's suggestion , or use a "normal" 709 conversion for AVC. Eitherway, if you test a bunch of colors, they will be off slightly from expected rounding errors during the 8bit RGB=>YUV=>RGB round trip

"normal" 709
ffmpeg -y -t 4 -loop 1 -i 1.png -vf out_color_matrix=bt709 -x264opts colormatrix=bt709 709.mp4

Balling's suggestion
ffmpeg -y -t 4 -loop 1 -i 1.png -color_primaries bt709 -color_trc iec61966_2_1 -colorspace smpte170m 2.mp4

You might be able to use VP9 in RGB mode, not sure if chrome supports it, I'll check later

ffmpeg -y -t 4 -loop 1 -i 1.png -vf out_color_matrix=bt709 -x264opts colormatrix=bt709 709.mp4, this command throw error No such filter: 'out_color_matrix' , so I changed to

ffmpeg -y -t 4 -loop 1 -i 1.png -vf scale=out_color_matrix=bt709 -x264opts colormatrix=bt709 709.mp4, then 709.mp4 generated successfully. I tested 709.mp4 and 2.mp4
from the two commands, their color is #ee4d2c and have same color as 1.png, this problem is solved, thanks to all of you.

[Balling]

Replying to pdr0:

Other players display it correctly as 238,77,45, including firefox, ffplay, mpv

That is a known bug. You do not have Turing, do you? Because that bug was "fixed" using SW overlays and I can see the color correctly, i.e. 238,77,45. See: ​https://fastpic.ru/view/114/2021/0331/_1aa32fbccd3d05c410b9f6344cdbb3ab.png.html The bug is this one:
​https://bugs.chromium.org/p/chromium/issues/detail?id=1068690

It just adds +1, 0, +1 (in RGB) pattern on all video if you use old approach. Very crazy, but it is what it is.

[Balling]

Replying to pdr0:
So, I did this
ffplay -i 2.mp4 -vf extractplanes=y is 120
ffplay -i 2.mp4 -vf extractplanes=u is 90
ffplay -i 2.mp4 -vf extractplanes=v is 201

And then checked with ​https://res18h39.netlify.app/color and yeah, 238, 77, 44; 238, 77, 45 convert to the same value after rounding and back it is 44 only! What is even more funny is that 238, 77, 46 converts to 120, 91, 201 but decodes as 238, 76, 46, which also encodes as 120, 91, 201. Just... Whatever. I mean, I knew it is what it is, just never had such experience.

P.S. I managed to confirm that superblack is indeed like that by using this very nice extractplanes.

[Balling]

Okay, so this is just a bug here. ​https://github.com/FFmpeg/FFmpeg/blob/573f05a7533cd9aed3ed895b4fa4ad8fcba4e56a/fftools/ffplay.c#L968

See ​https://github.com/libsdl-org/SDL/issues/2860

So frame->colorspace == AVCOL_SPC_SMPTE170M (SMPTE 170M matrix) should force that mode, but it does not.

Also why AVCOL_SPC_SMPTE240M matrix uses BT.601?? It is a different matrix as derived from SMPTE 170M primaries (yeah, SMPTE 170M matrix is not derived from SMPTE 170M primaries, I know, confusing).
P.S.
Obviously it is not primaries that define the matrix, even though it should be (not the case for JPEG (BT.601 a.k.a. SMPTE 170M matrix is derived from System M primaries ACTUALLY, see SMPTE 177 and ​https://github.com/AOMediaCodec/libavif/pull/430#discussion_r610920396) but BT.709 is the default in JPEG, since that is used in the default sRGB, also not the case for DCI-P3 in BT.2020 matrix as Netflix uses).

I think we should try to fix that. Or at least deprecate ffplaying sYCC files...

Since they migrated to github I will try to open an issue there too.

[Balling]

Sorry, wrong bug : )

[Balling]

VP9 RGB doesn't seem to work properly in chrome either

VP9 is now correctly decoded as being GBRP! (Sometimes it sees RGBP which is WRONG, but whatever, that part is fixed by -c copy.) P.S. RGBP part also fixed!

​https://bugs.chromium.org/p/chromium/issues/detail?id=1119260#c_ts1632145869

[mirh]

What's this hardon for seeing ffplay issues everywhere?
Yes there's possibly some issue with rounding in conversions (plenty of issues open for that already), and the simple portable media player that the project ships sucks a little, but that wasn't OP's main baffling problem.

As said in comment:5, non-color-managed PNGs assume a BT.601 color matrix.
Ffmpeg is more or less rightly "transparent" to the matter, and will just happily re-use it.

Problem is that whenever video players see something "untagged", if the image resolution is higher than ~PAL/NTSC dimensions they will infer that's some kind of BT.709 authored content and will use those curves.

Arguably ffmpeg should add the color space metadata it assumes to be using to all its encoded media.

[Balling]

As I said on github ​https://github.com/k4yt3x/video2x/issues/403#issuecomment-1013482958

there is no any bug here. Also PNG is only RGB, so no 601 matrix possible.

Ffplay is a separate bug.

There is no any issue with rounding, there is no way to preserve RGB 8 bit in YCbCr 8 bit.

Also the curve in 601 and 709 is the same.

[mirh]

The color matrix I meant, whatever the player uses to convert YUV back to RGB.

As I said thrice by now, you have ffmpeg that converts RGB data with the bt601 color space, but playback (without flags telling otherwise) will use bt709 above a certain resolution.

This is not right, and it should be pretty evident to anybody that just tried the simple command in the first post. Please stop to wander on other bugs that may or may not exist tangentially to this.

[Balling]

This bug is about the png converting to YCbCr in video and not tagging it. Yes, technically that should be fixed.

[mirh]

Thank you for your understanding.

[Balling]

API users also can trigger this insane bug. WTF is this. ​https://github.com/mpv-player/mpv/issues/9053#event-5970260312

[Balling]

So essentially the bug happens here ​https://github.com/FFmpeg/FFmpeg/commit/37f4aa133d0a0daa04661f65a016451525df7d0f

That fixme was deleted without actually fixing it ​https://github.com/k4yt3x/video2x/issues/403#issuecomment-1013705656

Jan, FYI, will assign you. Also note that openh264 should be also checked now that it can write matrix too.

Also your nice fix from chroma sample location broke JPEG sample when you go from 4:2:0 JPEG with thus center location to rgb h264 which is now tagged as center but RGB cannot be center chroma, it does not have chroma. 4:4:4 may be affected too.

[Balling]

Interestingly enough

So your command (with smaller png, 1080p maybe, is this a regression that bigger files are not supported??) is:

Appears it was a regression because the 1.png file of 2602x4366 [SAR 4366:4367 DAR 2602:4367] x264 encoding works now. HAHAHA. What a joke.

And of course libopenh264 prints this:

[libopenh264 @ 000002131adc6c80] [OpenH264] this = 0x000002131adb87b0, Error:ParamValidationExt(), width > 0, height > 0, width * height <= 9437184, invalid 2608 x 4368 in dependency layer settings!

Well by making png smaller I was able to encode openh264 file it is also not proper but at least it has range tagged. Other than that 601 matrix values are done correctly, same as in x264. Sigh.

In x264 only range of full is autowritten. Limited is ommited. Also should be fixed.

[Carl Eugen Hoyos]

The priority says all...

[mirh]

Lolwat?
You can even argue if priority should be "critical" or just "important" (please, understand all kind of software making videos out of HD PNGs is affected, from blender to opencv) but regardless the bug is real.