defect : mpeg2 interlaced yuv420 chroma incorrectly decoded

[Cyril Lambin]

Summary of the bug:
When using an interlaced mpeg2 source, the yuv420 chroma is interpreted as progressive chroma instead of interlaced chroma.

This bug is present when decoding the following mpeg2 formats : VOB, MPEG-PS, MPEG-TS.

avcodec seems to decode interlaced videos using yuv420p pixel format, but it should instead default to any 422 format in this case (or a yuv420i format, but it looks to be unavailable in ffmpeg).

In addition, -pxl_fmt seems to be ineffective to force the input decode (ffmpeg fires an error : 'Option pixel_format not found'). Looks like it's unsupported by the mpeg2video decoder.

=> the bottomline is that yuv420 chroma coding is different in progressive and interlaced. In the latter pixel format, odd and even lines don't share the same chroma data.

This bug generates such chroma artifacts that the quality of the video is greatly degraded when it needs to be deinterlaced/scaled/transcoded after the ffmpeg conversion.

How to reproduce:

To be able to easily check for the bug, the odd and even fields should have a heavy chroma variation but a light luma variation (ie same luminosity but different colors). Blue and Red are the best candidates.
The bug is not dependant of the output pixel_format. Here I output a prores422 file but it's the same with a rawvideo avi.

NOTE : the mpeg2 files I used were correctly detected as interlaced by any other video converter/analyzer.

D:\>d:\ffmpeg\bin\ffmpeg -f mpeg -i bug.vob -c:v prores test.mov
ffmpeg version N-68289-g208c03a Copyright (c) 2000-2014 the FFmpeg developers
  built on Dec  7 2014 22:13:45 with gcc 4.9.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --
enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-lib
modplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrw
b --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinge
r --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --en
able-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis
 --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-
libx265 --enable-libxavs --enable-libxvid --enable-decklink --enable-zlib
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 14.100 / 56. 14.100
  libavformat    56. 15.102 / 56. 15.102
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  2.103 /  5.  2.103
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Guessed Channel Layout for  Input Stream #0.2 : stereo
Input #0, mpeg, from 'bug.vob':
  Duration: 00:04:11.26, start: 275.223311, bitrate: 9343 kb/s
    Stream #0:0[0x1bf]: Data: dvd_nav_packet
    Stream #0:1[0x1e0]: Video: mpeg2video (Main), yuv420p(tv, smpte170m), 720x48
0 [SAR 32:27 DAR 16:9], max. 9800 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

    Stream #0:2[0xa0]: Audio: pcm_dvd, 48000 Hz, 2 channels, s16, 1536 kb/s
[prores @ 0000000004422980] encoding with ProRes standard (apcn) profile
[prores @ 0000000004425160] encoding with ProRes standard (apcn) profile
[prores @ 0000000004427c00] encoding with ProRes standard (apcn) profile
[prores @ 0000000004428660] encoding with ProRes standard (apcn) profile
[prores @ 00000000043d9cc0] encoding with ProRes standard (apcn) profile
Output #0, mov, to 'test.mov':
  Metadata:
    encoder         : Lavf56.15.102
    Stream #0:0: Video: prores (apcn) (apcn / 0x6E637061), yuv422p10le, 720x480
[SAR 32:27 DAR 16:9], q=2-31, 200 kb/s, 29.97 fps, 30k tbn, 29.97 tbc
    Metadata:
      encoder         : Lavc56.14.100 prores
    Stream #0:1: Audio: aac (libvo_aacenc) (mp4a / 0x6134706D), 48000 Hz, stereo
, s16, 128 kb/s
    Metadata:
      encoder         : Lavc56.14.100 libvo_aacenc
Stream mapping:
  Stream #0:1 -> #0:0 (mpeg2video (native) -> prores (native))
  Stream #0:2 -> #0:1 (pcm_dvd (native) -> aac (libvo_aacenc))
Press [q] to stop, [?] for help
frame=   97 fps=0.0 q=0.0 size=   13010kB time=00:00:03.10 bitrate=34345.1kbits/
frame=  183 fps=182 q=0.0 size=   26269kB time=00:00:05.97 bitrate=36030.6kbits/
frame=  271 fps=180 q=0.0 size=   39098kB time=00:00:08.90 bitrate=35951.6kbits/
frame=  365 fps=181 q=0.0 size=   53793kB time=00:00:12.04 bitrate=36584.3kbits/
frame=  457 fps=182 q=0.0 size=   67180kB time=00:00:15.11 bitrate=36410.0kbits/
frame=  550 fps=182 q=0.0 size=   81079kB time=00:00:18.21 bitrate=36458.0kbits/
frame=  638 fps=181 q=0.0 size=   94735kB time=00:00:21.15 bitrate=36685.7kbits/
frame=  727 fps=181 q=0.0 size=  107704kB time=00:00:24.12 bitrate=36573.9kbits/
frame=  816 fps=180 q=0.0 size=  120764kB time=00:00:27.09 bitrate=36514.0kbits/
frame=  902 fps=179 q=0.0 size=  133535kB time=00:00:29.96 bitrate=36508.6kbits/
frame= 1000 fps=181 q=0.0 size=  148139kB time=00:00:33.23 bitrate=36516.4kbits/
frame= 1091 fps=181 q=0.0 size=  161676kB time=00:00:36.26 bitrate=36516.9kbits/
frame= 1185 fps=181 q=0.0 size=  175265kB time=00:00:39.40 bitrate=36435.3kbits/
frame= 1277 fps=181 q=0.0 size=  189336kB time=00:00:42.47 bitrate=36516.0kbits/
frame= 1360 fps=180 q=0.0 size=  201932kB time=00:00:45.24 bitrate=36561.4kbits/
frame= 1451 fps=180 q=0.0 size=  215693kB time=00:00:48.28 bitrate=36597.0kbits/
frame= 1524 fps=178 q=0.0 size=  227089kB time=00:00:50.71 bitrate=36680.1kbits/
frame= 1609 fps=178 q=0.0 size=  239458kB time=00:00:53.55 bitrate=36629.5kbits/
frame= 1701 fps=178 q=0.0 size=  253563kB time=00:00:56.62 bitrate=36684.4kbits/
frame= 1797 fps=179 q=0.0 size=  267792kB time=00:00:59.82 bitrate=36668.6kbits/
frame= 1894 fps=179 q=0.0 size=  283124kB time=00:01:03.06 bitrate=36778.3kbits/
frame= 1974 fps=178 q=0.0 size=  295109kB time=00:01:05.73 bitrate=36778.5kbits/
frame= 2064 fps=179 q=0.0 size=  307805kB time=00:01:08.73 bitrate=36684.8kbits/
frame= 2159 fps=179 q=0.0 size=  321994kB time=00:01:11.90 bitrate=36684.1kbits/
frame= 2256 fps=180 q=0.0 size=  336902kB time=00:01:15.14 bitrate=36729.2kbits/
frame= 2340 fps=179 q=0.0 size=  349615kB time=00:01:17.94 bitrate=36744.7kbits/
frame= 2431 fps=179 q=0.0 size=  363557kB time=00:01:20.98 bitrate=36777.3kbits/
frame= 2525 fps=179 q=0.0 size=  377814kB time=00:01:24.11 bitrate=36794.4kbits/
frame= 2613 fps=179 q=0.0 size=  390721kB time=00:01:27.05 bitrate=36768.0kbits/
frame= 2703 fps=179 q=0.0 size=  403783kB time=00:01:30.05 bitrate=36730.1kbits/
frame= 2796 fps=179 q=0.0 size=  417759kB time=00:01:33.15 bitrate=36735.7kbits/
frame= 2889 fps=180 q=0.0 size=  431245kB time=00:01:36.26 bitrate=36699.1kbits/
frame= 2985 fps=180 q=0.0 size=  446408kB time=00:01:39.46 bitrate=36766.0kbits/
frame= 3074 fps=180 q=0.0 size=  459475kB time=00:01:42.43 bitrate=36745.2kbits/
frame= 3166 fps=180 q=0.0 size=  473415kB time=00:01:45.50 bitrate=36758.4kbits/
frame= 3253 fps=180 q=0.0 size=  486306kB time=00:01:48.40 bitrate=36748.3kbits/
frame= 3358 fps=181 q=0.0 size=  501818kB time=00:01:51.91 bitrate=36733.3kbits/
frame= 3455 fps=181 q=0.0 size=  516493kB time=00:01:55.14 bitrate=36744.9kbits/
frame= 3549 fps=181 q=0.0 size=  530794kB time=00:01:58.28 bitrate=36761.0kbits/
frame= 3626 fps=180 q=0.0 size=  542060kB time=00:02:00.85 bitrate=36743.1kbits/
frame= 3704 fps=180 q=0.0 size=  554200kB time=00:02:03.45 bitrate=36774.1kbits/
frame= 3799 fps=180 q=0.0 size=  568720kB time=00:02:06.62 bitrate=36792.9kbits/
frame= 3891 fps=180 q=0.0 size=  582481kB time=00:02:09.69 bitrate=36791.2kbits/
frame= 3987 fps=180 q=0.0 size=  597045kB time=00:02:12.89 bitrate=36802.2kbits/
frame= 4080 fps=180 q=0.0 size=  611511kB time=00:02:16.00 bitrate=36833.9kbits/
frame= 4174 fps=180 q=0.0 size=  626288kB time=00:02:19.13 bitrate=36873.5kbits/
frame= 4258 fps=180 q=0.0 size=  638807kB time=00:02:21.94 bitrate=36868.0kbits/
frame= 4355 fps=180 q=0.0 size=  653064kB time=00:02:25.17 bitrate=36850.5kbits/
frame= 4452 fps=181 q=0.0 size=  667183kB time=00:02:28.41 bitrate=36826.3kbits/
frame= 4544 fps=181 q=0.0 size=  681130kB time=00:02:31.48 bitrate=36834.2kbits/
frame= 4640 fps=181 q=0.0 size=  696691kB time=00:02:34.68 bitrate=36895.6kbits/
frame= 4740 fps=181 q=0.0 size=  711711kB time=00:02:38.02 bitrate=36895.2kbits/
frame= 4834 fps=182 q=0.0 size=  726141kB time=00:02:41.16 bitrate=36910.6kbits/
frame= 4927 fps=182 q=0.0 size=  740218kB time=00:02:44.26 bitrate=36915.3kbits/
frame= 5009 fps=181 q=0.0 size=  752402kB time=00:02:47.00 bitrate=36908.2kbits/
frame= 5096 fps=181 q=0.0 size=  765328kB time=00:02:49.90 bitrate=36900.8kbits/
frame= 5192 fps=181 q=0.0 size=  779951kB time=00:02:53.10 bitrate=36910.0kbits/
frame= 5284 fps=181 q=0.0 size=  793501kB time=00:02:56.17 bitrate=36897.0kbits/
frame= 5379 fps=181 q=0.0 size=  807537kB time=00:02:59.34 bitrate=36886.0kbits/
frame= 5476 fps=182 q=0.0 size=  822034kB time=00:03:02.58 bitrate=36882.6kbits/
frame= 5567 fps=182 q=0.0 size=  835906kB time=00:03:05.61 bitrate=36891.4kbits/
frame= 5661 fps=182 q=0.0 size=  850075kB time=00:03:08.75 bitrate=36893.4kbits/
frame= 5761 fps=182 q=0.0 size=  864883kB time=00:03:12.09 bitrate=36884.0kbits/
frame= 5851 fps=182 q=0.0 size=  878796kB time=00:03:15.09 bitrate=36900.5kbits/
frame= 5951 fps=182 q=0.0 size=  894506kB time=00:03:18.43 bitrate=36928.6kbits/
frame= 6041 fps=182 q=0.0 size=  908028kB time=00:03:21.43 bitrate=36927.9kbits/
frame= 6134 fps=182 q=0.0 size=  922440kB time=00:03:24.53 bitrate=36944.9kbits/
frame= 6230 fps=182 q=0.0 size=  936754kB time=00:03:27.74 bitrate=36939.7kbits/
frame= 6319 fps=182 q=0.0 size=  949922kB time=00:03:30.71 bitrate=36931.1kbits/
frame= 6413 fps=182 q=0.0 size=  964988kB time=00:03:33.84 bitrate=36966.5kbits/
frame= 6500 fps=182 q=0.0 size=  977558kB time=00:03:36.74 bitrate=36946.5kbits/
frame= 6591 fps=182 q=0.0 size=  991774kB time=00:03:39.78 bitrate=36966.0kbits/
frame= 6690 fps=182 q=0.0 size= 1007205kB time=00:03:43.08 bitrate=36985.3kbits/
frame= 6787 fps=183 q=0.0 size= 1021390kB time=00:03:46.32 bitrate=36969.8kbits/
frame= 6886 fps=183 q=0.0 size= 1036841kB time=00:03:49.62 bitrate=36989.2kbits/
frame= 6977 fps=183 q=0.0 size= 1050526kB time=00:03:52.66 bitrate=36988.3kbits/
frame= 7059 fps=183 q=0.0 size= 1062827kB time=00:03:55.40 bitrate=36986.4kbits/
frame= 7142 fps=182 q=0.0 size= 1075130kB time=00:03:58.17 bitrate=36979.6kbits/
frame= 7236 fps=182 q=0.0 size= 1089818kB time=00:04:01.30 bitrate=36997.5kbits/
frame= 7332 fps=183 q=0.0 size= 1102861kB time=00:04:04.51 bitrate=36949.8kbits/
frame= 7430 fps=183 q=0.0 size= 1118039kB time=00:04:07.78 bitrate=36964.0kbits/
frame= 7536 fps=183 q=0.0 size= 1134283kB time=00:04:11.31 bitrate=36973.3kbits/
frame= 7538 fps=183 q=0.0 Lsize= 1136081kB time=00:04:11.51 bitrate=37002.4kbits
/s dup=8 drop=0
video:1131858kB audio:3927kB subtitle:0kB other streams:0kB global headers:0kB m
uxing overhead: 0.026072%

D:\>

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Cyril Lambin]

Chroma bug example

[Kieran Kunhya]

The "p" in yuv420p doesn't stand for progressive - it stands for planar and is unrelated to the issue at hand.

The bug you are reporting is incorrect chroma *upsampling* from 4:2:0 -> 4:2:2.

[Carl Eugen Hoyos]

Please point to an input sample.

[Cyril Lambin]

You're right, it's a chroma upsampling bug. Note that as far as I can remember, ffmpeg always had this issue. It looks like it also impacts VLC.

It also shows in FFmpegSource (source plugin for AviSynth based on libav) if the plugin is forced to a specific pixel format. The workaround for ffmpegsource is to change the pixel format with AviSynth builtin functions (ConvertTo...).

Here's a video that shows the bug in its full glory : ​https://www.dropbox.com/s/u2lpm6np9re57yi/ffmpeg_chromabug.mpg?dl=0

[Cyril Lambin]

I just found a workaround completely by chance -_-

The chroma upscale works fine if I call libswscale to activate the interlaced flag :

d:\ffmpeg\bin\ffmpeg -f mpeg -i bug.vob -vf scale=interl=1 -c:v prores test.mov

Looks like the interlacing information is lost somewhere between libavcodec and libswscale.

[Kieran Kunhya]

Yes, that will get you close to a decent result. I think it needs some correction for chroma plane offsets as well which I've been meaning to do for a while.

I've not decided whether to handle that in my code or to use vf_scale.

Kieran

[Cyril Lambin]

But isn't it supposed to be fixed by default ?

I mean, every single product that uses ffmpeg does have the same bug. I talked about VLC, but I know it's the same for XMedia Recode, and of course my own tools used in a broadcast environment. Some broadcast software solutions are also based on ffmpeg, like CasparCG.

[Kieran Kunhya]

It can't be fixed in swscale directly because swscale has no concept of interlaced video - as you say it assumes chroma is associated with the nearest line. vf_scale is a wrapper that basically calls swscale twice for interlaced video, one call per field - this is the simplest fix by far.

[Carl Eugen Hoyos]

Replying to clam:

Looks like the interlacing information is lost somewhere between libavcodec and libswscale.

I don't think this "information" exists at all: The interlaced flag in MPEG streams just tells you if the encoder was set to "interlaced" not if the content actually is interlaced. At least where I live, all dvb streams are encoded as "interlaced", both the progressive streams and the (live) interlaced streams, so the flag does not help in any way. We also have DVD sample streams where this flag does not help in any way.
FFmpeg contains a filter that detects if the content is interlaced if you don't know in advance.

[Kieran Kunhya]

Replying to cehoyos:

Replying to clam:

Looks like the interlacing information is lost somewhere between libavcodec and libswscale.

I don't think this "information" exists at all: The interlaced flag in MPEG streams just tells you if the encoder was set to "interlaced" not if the content actually is interlaced. At least where I live, all dvb streams are encoded as "interlaced", both the progressive streams and the (live) interlaced streams, so the flag does not help in any way. We also have DVD sample streams where this flag does not help in any way.
FFmpeg contains a filter that detects if the content is interlaced if you don't know in advance.

The DVB encoder should still perform an interlaced chroma downconversion irrespective of the content type (because the DVB encoder doesn't know either).

[Cyril Lambin]

Okay, I guess the fastest way to do this is to call vf_scale manually for now.

It's really strange that libswscale doesn't have any kind of support for interlaced 420 chroma. It's a very usual pixel format : DV/DVCPro, MPEG2, MPEG4, found in interlaced DVDs, BluRay, TV broadcasts, etc.

[Cyril Lambin]

Replying to kierank:

Replying to cehoyos:

Replying to clam:

Looks like the interlacing information is lost somewhere between libavcodec and libswscale.

I don't think this "information" exists at all: The interlaced flag in MPEG streams just tells you if the encoder was set to "interlaced" not if the content actually is interlaced.

The DVB encoder should still perform an interlaced chroma downconversion irrespective of the content type (because the DVB encoder doesn't know either).

Yes the actual content (interlaced/progressive) is irrelevant. The interlaced flag only indicates that the chroma is shared by two other lines (odd/odd or even/even) and not two consecutive lines (odd/even).

Progressive 4:2:0 chroma is a better choice for progressive content but when you mix interlaced & progressive sources, interlaced 4:2:0 chroma is an easier choice since it's compatible with both.

[Cyril Lambin]

I did a quick check this morning, the issue is the same for interlaced DV content. I wanted to make sure that it was not an issue with the mpeg decoder only.

IMO the real definitive answer to this would be to implement a mandatory conversion to a 4:2:2 pixel format (with correct upscale) while or just after decoding if the original pixel format is using interlaced 4:2:0 chroma.
That's actually how professionnal/broadcast conversion and editing tools works, except the chroma upscale is directly implemented into the various codecs. It's the best solution but it also means the upscale quality depends of the codec's builtin scale (hence the real quality difference between DV decoders like Microsoft, Apple, Mainconcept, Avid...). Implementing the upscale into the codec also mean you might break compatibility with software expecting a 4:2:0 output, and of course that kind of defeat the purpose of lbswscale.

Another way of doing this would be to detect if the destination format accepts chroma formats > 4:2:0 (like 4:2:2, 4:4:4, RGB) and then automatically upscale the chroma. If not, revert to the current behavior and log a warning that the chroma is going to be incorreclty handled.

For the record, AviSynth had the exact same kind of issue with pixel format conversion that was fixed very late.
​http://avisynth.nl/index.php/Interlaced_fieldbased
​http://avisynth.nl/index.php/Sampling

The source of this issue is that ffmpeg doesn't have an internal 4:2:0 interlaced pixel_format support (tagged incompatible with progressive 4:2:0). Which is not that surprising since it doesn't come from the broadcast world.

[Kieran Kunhya]

FFmpeg doesn't need an interlaced pixel format since the pixel format defines the structure in memory and not the chroma location.

You are right though that 4:2:0 interlaced content should be upscaled correctly.

Forcing a 4:2:2 conversion will break everything requiring 4:2:0 input (e.g x264)

[gjdfgh]

I don't know why you say that VLC has the same problem. VLC is a video player, and is supposed to display the video. So if you want meaningful output, you have to enable a deinterlacer (unless vlc does that by default). Also, VLC will probably upscale using the video output API, and not libswscale.

(Unless you mean the transcoder part of VLC.)

[Carl Eugen Hoyos]

Is there anything about this ticket that can be "fixed"?

[Kieran Kunhya]

I was hoping to find time to fix the chroma upsampling in vf_scale but that might not happen this week. I might be able to help our OPW student though with it.

[Cyril Lambin]

I said that VLC had the same issue to point that this problem impacts other project as well.
Yes as a video player you might expect that the user enables deinterlacing. But if there is a chroma scaling error, the deinterlace filter might end with an incorrect result (in this case, mixing chroma between frames).
And it was also to point out that it might use libswscale because the issue is exactly the same. In the past I reported various bugs to the VLC team, mostly all closed with "it's a bug in libav" -_-

In all those years what I learnt is that interlaced video is often not or badly supported in FLOSS projects. It's just unfortunate since it's part of the basics of video :/

As I said it's just as if the codecs were not able to send back the information that the chroma is interlaced. I misunderstood the meaning of yuv420p but the question I wanted to ask there is : why using the same pixel format if it can be interpreted in two (or more) different ways ? Isn't there something missing here ?

Now don't misunderstand me, I know ffmpeg is a huge project. I'm even sad I don't have enough time to bring my own fixes.

[Carl Eugen Hoyos]

Replying to clam:

As I said it's just as if the codecs were not able to send back the information that the chroma is interlaced.

I was hoping that we already agreed that this "information" does not exist in the video stream and therefore cannot be sent from the decoder to the filter.
Do you disagree?

[Cyril Lambin]

Yes, I disagree :)

A lot of codecs can compress videos with interlaced chroma or not. I don't speak of codecs like VC-3 (DNxHD) or ProRes : they're broadcast formats using 4:2:2 chroma so the chroma coding is the same for interlaced or progressive frames. For instance if you compress a 1080/50i or a 1080/25p sequence in ProRes, the video stream will be exactly the same but somewhere in the files there will be a flag telling than one is interlaced and the other progressive (often added by the editing software).

For MPEG2, DV and H264 it's a different matter since they use 4:2:0 chroma coding and they must indicate in the stream (or the file's header) that the content is interlaced or not. It also change the way macroblocks are coded.

MPEG2 is mostly used for interlaced videos (it's what you get on a DVD or a DVB broadcast), I only saw interlaced DV in my life, and H264 might be used for interlaced content using MBAFF.

The decoders must know they work on interlaced pictures since it changes the way macroblocks are encoded in the stream. A quick peek at mpeg12dec.c and dvdec.c confirms this.

[Carl Eugen Hoyos]

Replying to clam:

For MPEG2, DV and H264 it's a different matter since they use 4:2:0 chroma coding and they must indicate in the stream (or the file's header) that the content is interlaced or not.

As said, where I live, mpeg2video and h264 (1080) DVB streams are always flagged as interlaced, no matter if they are progressive or not. It was stated repeatedly on the user mailing that the same can be true for DVD streams, so I don't see how this flag can be used.

[Carl Eugen Hoyos]

Replying to clam:

MPEG2 is mostly used for interlaced videos

This is simply not true here: Except for news and sports, everything I watch is progressive (but is flagged as interlaced).

The decoders must know they work on interlaced pictures since it changes the way macroblocks are encoded in the stream.

That is why this flag is always set to interlaced for (many or most) dvb streams and at least for some progressive DVDs. As a result, the flag cannot be used to tell the scaler if the content is interlaced or not.

[Carl Eugen Hoyos]

Replying to clam:

I only saw interlaced DV in my life

​http://samples.ffmpeg.org/./archive/all/dv+dvvideo+pcm_s16le++imovie_dv_file_with_dup_audio_streams_segfaults_libfaac.dv

[Cyril Lambin]

Yes MPEG2 allows progressive encoding but it's never used (you can generate progressive MPEG2 though). This was actually the big advantage of MPEG2 when it was released, than there was - at least - a widely released format that handled interlacing.

The fact that the original content is progressive (for example 25p material broadcast as 50i) is not relevant for chroma decoding and chroma upscale. What's important is to know that the chroma is interlaced or not, and yes for MPEG2 it's mostly interlaced since DVD authoring software always generates interlaced TFF videos.
It's the same for broadcast, the live encoder will not switch back and forth between progressive and interlaced. Not that it wouldn't be possible, it's just that it would mean that the encoder is aware of the content (it's not) and that the receivers would be able to decode switching MPEG-TS streams (they're usually not).

You're actually talking about scaling the entire picture (like HD upscale / SD downscale). Indeed, if the original content is progressive and encoded as interlaced, it's a pity since a regular picture scaler will need to do an interlaced scale (deinterlace/scale/reinterlace) instead of just do a regular scaling. Unfortunately, it's the way it works, even in the broadcast world. The only way to avoid this is to visually inspect the stream and manually apply a progressive scale. You can also use some advanced algorithm to autodetect that both fields match the same original frame.
Note that this issue is not really new, in the PAL world it's been here for at least 30 years, even with Betacam/Digibeta/HDCAM tapes. In the NTSC world the issue also exists with pulldown sequences (especially 'broken' ones).

But it's not what we're talking here. The issue we're talking about is only about chroma coding :)

[Carl Eugen Hoyos]

Replying to clam:

Yes MPEG2 allows progressive encoding but it's never used

I thought progressive DVDs exist (and are not uncommon) but I don't think this matters:
If you are right, then please send the (I suspect trivial) patch to the development mailing list that makes the default for the software scaler interlaced. If you are wrong, I don't think this issue can be fixed.

The fact that the original content is progressive (for example 25p material broadcast as 50i) is not relevant for chroma decoding and chroma upscale. What's important is to know that the chroma is interlaced or not, and yes for MPEG2 it's mostly interlaced since DVD authoring software always generates interlaced TFF videos.

Sorry for my ignorance: How can chroma be interlaced for progressive video?

It's the same for broadcast, the live encoder will not switch back and forth between progressive and interlaced. Not that it wouldn't be possible, it's just that it would mean that the encoder is aware of the content (it's not) and that the receivers would be able to decode switching MPEG-TS streams (they're usually not).

That is exactly my point.

You're actually talking about scaling the entire picture (like HD upscale / SD downscale).

I was actually not talking about scaling or the software scaler at all: I just meant that I believe there is no flag in mpeg2video streams that tells you (reliably) if chroma is interlaced or not.

Indeed, if the original content is progressive and encoded as interlaced, it's a pity since a regular picture scaler will need to do an interlaced scale (deinterlace/scale/reinterlace) instead of just do a regular scaling.

This is not how FFmpeg works: It will default to a progressive scale, the user will have to specify if he wants an interlaced scale.

Unfortunately, it's the way it works, even in the broadcast world. The only way to avoid this is to visually inspect the stream and manually apply a progressive scale.

Yes (s/and do a progressive scale/and do an interlaced scale).

You can also use some advanced algorithm to autodetect that both fields match the same original frame.

Such a filter exists within FFmpeg, I believe it works well for many samples.

[Kieran Kunhya]

Carl, you are confusing the *coding method* of the picture with the *underlying picture format*. These are two separate things.

This discussion is related to the former.

[DonMoir]

Using ffmpeg_chromabug.mpg as a test case, this is not a bug in the decoding as the title of the ticket implies. It decodes fine, and the frames interlaced_frame flag is set to 1. It is up to you at this point to deinterlace or not and you would need to do it before calling into libswscale. libswscale has no notion of it.

It is probably such that the command line tools will not do it automatically.

You mentioned you may want to do something about it in your own code and this is what I do. I have since pulled the deinterlacing into my own code and use that and don't depend on ffmpeg for it. One problem is the interlace_flag does not contain the type of deinterlacing that should be performed. In all samples that I have and all that I have been able to find, a simple deinterlace such as avpicture_deinterlace works fine. It also works perfect for above sample. It is recommended you use avfilter and yadif for this now, but that can be way more overhead then you need. I do not use avfilter for anything.

So if you are using ffmpeg_chromabug.mpg as a sample to point to how this is a bug, then that is incorrect. It displays perfect for me using the following.

decode->deinterlace->scale not decode->scale

[DonMoir]

ffmpeg_deinterlace_correct.png​ is a screen shot of ffmpeg_chromabug.mpg decoded, deinterlaced, and scaled correctly.

[Cyril Lambin]

Replying to DonMoir:

So if you are using ffmpeg_chromabug.mpg as a sample to point to how this is a bug, then that is incorrect. It displays perfect for me using the following.

decode->deinterlace->scale not decode->scale

Your assumption that the user wants to deinterlace the video is incorrect. As a matter of fact, I use ffmpeg to convert videos without deinterlacing them. So it's "up to ffmpeg" to handle interlaced chroma correctly when deinterlacing is not required :)

Your comment about deinterlacing is interesting though, because you say that the user need to call a deinterlacing filter manually when the input video need to be deinterlaced. IMO this breaks the purpose of a video transcoder/converter : there should be a way to call the filter only if needed (like : deinterlace only if source is interlaced), else you need to analyze the video before calling ffmpeg. And as far as I'm concerned when I call deinterlacing I always want to get a doubled framerate to avoid discarding half of the video.
But this is a completely different matter, as I said we are talking here about format conversion without calling any filter.

[DonMoir]

Yeah, I see your point and just my 2 cents.

[Michael Niedermayer]

This ticket has become a bit messy, the decoder is fine, its not buggy. The related bug in the scale filter has been fixed by kierank.
What remains is that the scale filter does by default scale progressively.
So its really a issue with the default when the video indicates its interlaced.
IMHO this ticket should be closed and a new one opened that describes this remaining issue clearly and tersely so people dont have to read through this rather lengthly discussion to find out what it is and is not about

[Cyril Lambin]

I agree.

[Cyril Lambin]

I agree.

[Balling]

Were there any updates on this? That other sample is gone, but DV video is still there, but it uses 4:1:1 subsampling, which is kinda interesting...

Anyway, I agree this cannot be fixed simply. DVDs can have both hard telecines and progressive (soft telecine) and even native interlaced content. So it kinda complicates this crazy thing. And the flags may be not set correctly.

An example is Star Trek: Voyager

​https://superuser.com/questions/1611907/can-i-use-ffmpeg-to-convert-a-variable-frame-rate-video-to-constant-frame-rate-w

[Balling]

Interesting, did ​https://ffmpeg-devel.ffmpeg.narkive.com/Vv0QzsGZ/rfc-patch-use-correct-chroma-position-in-yuv420p-interlaced-conversions patch affect this or not? (1515bfb3132d9e040f856bc48fbe8a4f26b14c14)

Also, what the hell is "XXX: support other 4:2:0 pixel formats"?

[Balling]

Oh, and also 8efb7f5a26e7300e09ba01239328d1498238c532 (see ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/589892C6.9080608@m1stereo.tv/#8053)

[Balling]

Finally ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231013142205.60658-1-ffmpeg@haasn.xyz/

[Cyril Lambin]

Replying to Balling:

Finally ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231013142205.60658-1-ffmpeg@haasn.xyz/

About time :)

I stopped counting the quantity of video archives that are plagged with this bug since ffmpeg is used in so many software.
One recent example, a very rare archive by Daft Punk recently posted on their YouTube channel:
​https://shelter.moe/@fenarinarsa/111234979232590645

On my side, I do all video archives in 4:2:2 chroma (MP4 studio) as a workaround to this bug, else there's a 99.99% chance it gets badly decoded for deinterlacing/recompression.

The only software I know so far that compensate for this bug by relocating chroma data after decoding if it detects 4:2:0 video is the QTGMC filter for AviSynth.

[Balling]

Well, please test 2d1aeba8f26869dff6857087f48066f76dba8dc5