Wrong Chroma Location (Type 2, top left) for XDCAM Files (Type 0, left) in FFProbe

[Francesco Bucciantini]

Summary of the bug:

Whenever FFProbe tries to detect the chroma location of an XDCAM-50 file, it reports top left, namely 4:2:2 Type 2.
Not only top left is invalid for 4:2:2 files, but it has been standardized "recently" (2013) for UHD HDR PQ BD in H.265 which are indeed 4:2:0 Type2 (top left). The old MPEG-2 chroma placement is Type 0, namely "left" and it's the one being used in XDCAM-50 as it's an MPEG-2 standard defined in early 2003-2006 when Type 2 wasn't a thing and didn't even exist.

This affects FFProbe and, as result, Avisynth indexers like LWLibavVideoSource and FFVideoSource which populate the frameserver with the wrong frame property.

How to reproduce:

% ffmpeg -show_format -show_streams -i "input.mxf"


ffprobe version git-45dc668aea
built on 2021-11-07-git-45dc668aea

[STREAM]
index=0
codec_name=mpeg2video
codec_long_name=MPEG-2 video
profile=4:2:2
codec_type=video
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
width=1920
height=1080
coded_width=0
coded_height=0
closed_captions=0
film_grain=0
has_b_frames=1
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv422p
level=2
color_range=tv
color_space=bt709
color_transfer=bt709
color_primaries=bt709
chroma_location=topleft

This is yet a different XDCAM file I received from A&E:

index=0
codec_name=mpeg2video
codec_long_name=MPEG-2 video
profile=4:2:2
codec_type=video
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
width=1920
height=1080
coded_width=0
coded_height=0
closed_captions=0
film_grain=0
has_b_frames=1
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv422p
level=2
color_range=tv
color_space=unknown
color_transfer=bt709
color_primaries=unknown
chroma_location=topleft

and this is a movie I've got from Notorious Pictures and it says top left too:

[STREAM]
index=0
codec_name=mpeg2video
codec_long_name=MPEG-2 video
profile=4:2:2
codec_type=video
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
width=1920
height=1080
coded_width=0
coded_height=0
closed_captions=0
film_grain=0
has_b_frames=1
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv422p
level=2
color_range=tv
color_space=unknown
color_transfer=bt709
color_primaries=unknown
chroma_location=topleft

Sample (valid for 7 days): ​https://we.tl/t-MhjJWTj0lQ

This is reflected on Avisynth Indexers: ​https://i.imgur.com/f736Nld.png - ​https://i.imgur.com/igT1odL.png

[Balling]

I am sorry, what? 4:2:2 can only use one chroma location, while 4:2:0 can use 6 different types. Also top left is used in DV.

[Francesco Bucciantini]

@Balling, it all came down from this: ​https://forum.doom9.org/showthread.php?t=181351

If you have more info than I do after reading the topic, please reply either here or on Doom9

[Francesco Bucciantini]

Also what do you mean by "4:2:2 can use only one chroma location?"
If you mean that 4:2:2 only has left, so type 0, so the old MPEG-2 chroma location, that would be correct, in fact if I go through ffprobe with an XAVC Intra Class 300 file muxed in .mxf, it reports correctly "left", so Type 0:

[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=High 4:2:2 Intra
codec_type=video
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
width=3840
height=2160
coded_width=3840
coded_height=2160
closed_captions=0
has_b_frames=0
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv422p10le
level=52
color_range=tv
color_space=bt2020nc
color_transfer=arib-std-b67
color_primaries=bt2020
chroma_location=left

For other codecs like Apple ProRes 4:4:4 10bit, it says unspecified:

[STREAM]
index=0
codec_name=prores
codec_long_name=Apple ProRes (iCodec Pro)
profile=HQ
codec_type=video
codec_tag_string=apch
codec_tag=0x68637061
width=3840
height=2160
coded_width=3840
coded_height=2160
closed_captions=0
has_b_frames=0
sample_aspect_ratio=N/A
display_aspect_ratio=N/A
pix_fmt=yuv422p10le
level=-99
color_range=tv
color_space=unknown
color_transfer=unknown
color_primaries=unknown
chroma_location=unspecified

which is fine, but still, what I wonder is: why is it saying Type 2 Top Left for MPEG-2 50 Mbit/s files? Is it really top left type 2 or is it a bug?
'cause as far as I know top left type 2 can only be used for 4:2:0 files, hence the ticket in the bug tracker and the discussion in Doom9.

[Balling]

Well, actually, I just checked H.265 (page 327, Figure D.2 – Nominal vertical and horizontal sampling locations of 4:2:2 samples in top and bottom fields) for 4:2:2 fields (so may be interlaced) it supports 2 different chroma locations. But that is only for fields. I doubt ffmpeg will force some other chroma location for 4:2:2, since there is only one.

You can check it as I did for dolby vision! After all, the -vf scale commands are known.

[Francesco Bucciantini]

That's alright, but the thing is that I think XDCAM-50 is using Type 0, left. There isn't any reason why MPEG2 yv16 (4:2:2) shouldn't be using the MPEG2 chroma placement, so Type 0, left, so I don't think the value reported by FFProbe is right at all. Same goes for FFVideoSource and LWLibavVideoSource.

Just to align everyone, the discussion on Doom9 was:

Ferenc Pinter: Topleft is treated as invalid for 422, it's accepted only for 420

Dogway: looks MPEG2 to me

(he's referring to the MPEG2 Type 0 left chroma placement of the sample clip I posted, therefore confirming that FFProbe got it WRONG, just like ffms2 and LSMASH).

Reel.Deel: I was under the impression that 4:2:2 has always had a "left" (type 0) chroma location. My understanding is that the "left" chroma location was introduced in MPEG2, before then it was only centered since that was the spec on MPEG1. Also, it would not make much sense for 4:2:2 to be top left since the chroma has full vertical resolution. If that were to be the case it would mean shifting the chroma half a pixel down when converting to RGB, for no good reason. I've never seen any illustration/articles that show/mention a different chroma placement for 4:2:2.

..........

So, that being said, I'm under this impression too, 4:2:2 has always had left, type 0, MPEG-2 chroma placement, especially on MPEG-2 Streams like XDCAM-50 is!

We all seem to agree that FFProbe, LWLibavVideoSource and FFVideoSource all report the WRONG chroma location as it shouldn't be top left at all, but rather "left" and that's a bug and needs to be fixed.

[Hendrik]

If the chroma plane is both vertically and horizontally aligned with the Luma plane, which MPEG-2 4:2:2 is, then "topleft" actually seems like the correct value - because it indicates the position of the first chroma sample, as per the ffmpeg documentation of AVChromaLocation.

You could even argue that for 4:2:2 both "left" and "topleft" are supposed to be handled identically, due to the absence of vertical subsampling, so in practice it would make no difference. The only reason h264 would result in a different value is that the interpretation of different developers differed on what to call it, but neither mpeg2 or h264 even encode a chroma location value for 422, and always contain the same one.

The entire "type 0" and "type 2" thing only applies to 4:2:0, so its best to just not bring it into the discussion at all. 4:2:2 in reality only ever has one chroma position, and if you want to call that left or topleft, which for both would exist arguments, is rather irrelevant, because there should be no difference between them.

TL;DR
You might as well ignore chroma location for 4:2:2, since it never differs as per the specification of both mpeg2 and h264. Or at the very least instruct your code to handle left and topleft identically for 4:2:2, because they are. mpeg2, h264, and h265 for that matter, do not even encode the chroma location for 422 - therefor you cannot associate any "type X" with it, since no type number is even specified. Its just 422 chroma, no alternates are possible.

[Francesco Bucciantini]

I see!
So, in a nutshell, 4:2:2 doesn't have a chroma location, it's literally always the same. x262, x264, x265 don't encode any value for the chroma location and what ffprobe does (as well as the Avisynth Indexers) is reporting left for H.264 encoded streams and top left for MPEG-2 encoded streams, but even though the name is different, they're actually exactly the same.

In other words, in "Avisynth terms", it's always "left", so I think we have two choices:

1) We change this behavior in indexers so that they're gonna report "unknown" (or "left" eventually) every time we get a 4:2:2 stream

2) We make the internal resizers handle 4:2:2 anyway assuming the old MPEG-2 chroma location every time, no matter what the indexer says instead of throwing an error

Thank you for pointing this out, Hendrik.
I'm gonna go back to Doom9 in Avisynth-land, I'll let you FFMpeg guys the decision on what to do about this (like standardizing the naming convention for all 4:2:2 streams to avoid confusing people like it happened for me and other Avisynth developers or leave it as it is).

Cheers,
Frank

[Balling]

The only reason h264 would result in a different value is that the interpretation of different developers

Please fix it. Also JPEG 4:2:0 to 4:4:4 video propogation of chroma center sample location is quite strange.