DNX120 Decoding is broken in FFMpeg and FFPlay (fine in AVID and Adobe Premiere)

[Francesco Bucciantini]

Summary of the bug:

DNX120 decoding/indexing is broken in:

• LWLibavVideoSource
• FFVideoSource
• DirectShowSource (LAV)
• FFMpeg
• Davinci Resolve

The following players also fail to decode it properly:

• MPV
• MPC-HC (LAV)
• PotPlayer

The following software decode it correctly:

• AVID Media Composer
• AVID Media Director
• Adobe Premiere
• EVS Hardware Playout Ports

Here's the mediainfo:

This is how it's decoded in FFMpeg / FFplay:

And this is how it should be instead (decoded by AVID):

Unfortunately, I cannot share the footage of the game (for obvious reasons, it's a copyrighted event), however I asked them to send me a non-sensitive content and they put a blue cloth on the camera and recorded a sample...
I know it kind of sucks as a sample, but here we go:

Sample - ​https://we.tl/t-SvCQRPGikp

How to reproduce:

ffmpeg -i "GreenFrames.mxf" -vcodec ffvhuff -f avi -y "test.avi"

or

ffplay -i "/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf"


ffmpeg version 2021-05-05-git-7c451b609c
built on 05/05/2021 12:29 PM UTC+01:00

and also

ffplay version 4.4

[FranceBB@router-localhost ~]$ ffplay -i "/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf"
ffplay version 4.4 Copyright (c) 2003-2021 the FFmpeg developers

built with gcc 11 (GCC)
configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libcdio --enable-libdrm --enable-libjack --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librav1e --enable-libsmbclient --enable-version3 --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libglslang --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzvbi --enable-lv2 --enable-avfilter --enable-avresample --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100

Input #0, mxf, from '/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf':

Metadata:

operational_pattern_ul: 060e2b34.04010101.0d010201.01010100
application_platform: Microsoft Windows 7 Professional Service Pack 1 (Build 7601)
uid : f3687fb0-a928-11eb-92b0-0cc47a8346df
generation_uid : f3687fb0-a928-11eb-92b1-0cc47a8346df
company_name : OpenCube
product_name : MXFTk Advanced
product_uid : 3a4fe380-0d01-11e4-869f-3cd92b5c1dfc
product_version : 2.10.4.20180828
product_version_num: 2.10.4.0.1
toolkit_version_num: 2.10.4.0.1
modification_date: 2021-04-29T21:25:14.052000Z
material_package_umid: 0x060A2B34010101050101052013000000F36858A0A92811EB92A40CC47A8346DF
timecode : 21:25:24:09

Duration: 00:00:35.68, start: 0.000000, bitrate: 121346 kb/s
Stream #0:0: Video: dnxhd (DNXHD), yuv422p(bt709/unknown/bt709, top first), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc

Metadata:

file_package_umid: 0x060A2B34010101050101052013000000F3683190A92811EB929C0CC47A8346DF
track_name : Picture

5.24 M-V: 0.045 fd= 0 aq= 0KB vq=15392KB sq= 0B f=0/0

[Marton Balint]

It seems -threads 1 or -thread_type slice is a viable workaround.

[Francesco Bucciantini]

Yep, I can confirm that

ffplay -i "/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf" -thread_type slice

works

So... where do we go from here?
I mean, for a quick encode I can easily put it there, but I think that there's something wrong with multithreading in the decoder at this point and it should be addressed in the future.

[Francesco Bucciantini]

There are also some DNX120 that play ok from some cameras and some other DNX120 that have green fields (unless we use the -thread_type slice flag) from some totally different cameras.
Going deeper in the analysis, those from the camera brands that are decoded correctly have:

[dnxhd @ 000001f4fd9d7c80] interlaced 3, cur field 1

those from the camera brands that are decoded incorrectly (unless -thread_type slice is specified) have:

[dnxhd @ 0000013fc0277c80] interlaced 2, cur field 0

the values are bits in the frame header, what i don't get is why we have the entries 2 times

[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 0000025bc07c7c80] Profile cid 1242.
[dnxhd @ 0000025bc07c7c80] 1920x1080, 4:2:2 8 bits, MBAFF=0 ACT=0
[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 0000025bc07c7c80] 1920x1080, 4:2:2 8 bits, MBAFF=0 ACT=0

in the cameras that produce the version with the decoding issue, it's 2 times "interlaced 2, cur field 0" while in the camera brands that produce a file correctly decoded without any flag, only the first entry is the same while the second entry is

[dnxhd @ 00000222415e7c80] interlaced 3, cur field 1

[Carl Eugen Hoyos]

I cannot reproduce the issue:
Could somebody provide a checksum for the test file?

[Marton Balint]

md5sum GreenFrames.mxf

5bcafcde36bd5a1052f23f65d264ceec GreenFrames.mxf

[Carl Eugen Hoyos]

Replying to Marton Balint:

md5sum GreenFrames.mxf

5bcafcde36bd5a1052f23f65d264ceec GreenFrames.mxf

I have this file but I only see blue frames, no rugby players, no matter the thread count.

[Francesco Bucciantini]

What do you mean?

Decoding with like:

ffplay -i "input.mxf"

leads to green fields, while

ffplay -i "input.mxf" -thread_type slice

works like a charm.

Here is another file: ​https://we.tl/t-7IU3JYToWT
MD5: 2E39838EF8455E4E15484782D5AC1375
SHA1: 438A3EAEDDCD554F0CCD4290E4A66C1647A86C0E
SHA256: B275FCC7E9D17E2EC2F8CB37086CD0B1D9C3A6B8B619A96598DE92D412732F82

It should be fairly easy to reproduce.

[Marton Balint]

Replying to Carl Eugen Hoyos:

Replying to Marton Balint:

md5sum GreenFrames.mxf

5bcafcde36bd5a1052f23f65d264ceec GreenFrames.mxf

I have this file but I only see blue frames, no rugby players, no matter the thread count.

It is a bit misleading, the screenshots are from a copyrighted material, but the sample contains blue frames. Green lines are also present there though here.

[Francesco Bucciantini]

Yeah, I shared one of the very short samples from the pitch as well.
That should help you figure it out.

[Francesco Bucciantini]

Very short pitch + players temporary clip: ​​https://we.tl/t-7IU3JYToWT

[Balling]

This looks the same as ​https://github.com/sekrit-twc/zimg/issues/135 and #9077. Does not (?) happen on windows ffplay in latest build here: ​https://github.com/BtbN/FFmpeg-Builds/

copyrighted event

I am prety sure this is fair use.

[Marton Balint]

Replying to Balling:

This looks the same as ​https://github.com/sekrit-twc/zimg/issues/135 and #9077. Does not (?) happen on windows ffplay in latest build here: ​https://github.com/BtbN/FFmpeg-Builds/

No, happens here on x86/linux.

[Balling]

In mpv that happens in full screen but does not in windowed (yes, double click breaks and fixes it, insanity). So that is just some kind of crazy interlace. Also, seeking is nuts there, it destroys interlace whatsover.

P.S. It does happen on windows ffplay too only on initial frames, which is IMHO just a corrupted frames issue (or maybe not, cause seeking back to the start works without any green things).

P.S. This looks like some kind of frame reordering for interlace and some kind of strange frame interpolation.

P.S. Yes, on windows interlace is indeed fixed with -thread_type slice, not that I know how to deinterlace this.

[Francesco Bucciantini]

@Balling... it is fixed with -thread_type slice on Linux (Fedora 34) as well.

Anyway, this is not an outlier, in the sense that every file we have been receiving from the pitch from this camera were like this, so this is not a corrupted file or anything, it's clearly a decoding error.

That being said, we've been testing several DNX120 and those that are decoded correctly in FFMpeg without the -thread_type slice have:

[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 00000222415e7c80] interlaced 3, cur field 1

while the ones that are decoded incorrectly and need the -thread_type slice in order to decode them correctly have:

[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0

Besides, if I use ffprobe to see what's going on in the files that are not decoded correctly I get:

ffprobe "test.dnxhd" -show_frames |findstr top_field_first

top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1

Now, according to the DNX standard, we have one frame header per field (unlike in other codecs like MPEG-2 where we only have one frame header for each pair of fields)

The frame header start sequence is 00 00 02 80 01 - the next byte tells us if we deal with the first or second field of an interlaced frame

00 00 02 80 01 02 indicates first field

00 00 02 80 01 03 indicates second field

If I take one of the DNX120 which is decoded correctly in ffmpeg by default, I have 02 and 03 that change with every frame header, like so:

frame 0, first field 00 00 02 80 01 02

frame 0, second field 00 00 02 80 01 03

00 00 02 80 01 02

00 00 02 80 01 03

00 00 02 80 01 02

00 00 02 80 01 03

but this is not the case for the DNX120 files we have received from the pitch in which we have:

00 00 02 80 01 02

00 00 02 80 01 02

00 00 02 80 01 03

00 00 02 80 01 03

00 00 02 80 01 02

00 00 02 80 01 02

00 00 02 80 01 03

00 00 02 80 01 03

See the issue here?

[Balling]

That is the point. What does it mean two top, two bottom fields? Is that how 4:2:2 is encoded or what?

[Francesco Bucciantini]

I don't know, but I mean, it shouldn't be the same, it should be different with one header per field rather than per frame I think. I've seen that kind of pattern in other codecs but not in DNX120, but this is also the reason I opened the bug in the first place. I don't know why it's behaving like this in the sense that I don't know why in an interlaced DNX we have the same header in both fields but that's the way it is, so... where do we go from here? I mean, can we work around this in the decoder so that it's gonna be aware of this possibility and decode it as such anyway? Also, why the heck is it not broken with -thread_type slice but it is for slice+frame? I'm a broadcast encoder, I work for Sky, but I'm mostly an Avisynth guy, so I don't know the inner working of the avcodec in FFMpeg...

[Francesco Bucciantini]

Using gsar to modify the header also works and makes the files possible to decode like so:

type "INPUTFILE" | C:\temp\gsar.exe -s:x83:x09:x40:x00:x00:x00:x02:x80:x01:x03 -r:x83:x09:x40:x00:x00:x00:x02:x80:x01:x02 -F | C:\temp\gsar.exe -s:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x02 -r:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x03 -F > "OUTPUTFILE"

Still, I believe this should be addressed anyway without using gsar.

[Francesco Bucciantini]

cat.exe "INPUTFILE" | gsar.exe -s:x83:x09:x40:x00:x00:x00:x02:x80:x01:x03 -r:x83:x09:x40:x00:x00:x00:x02:x80:x01:x02 -F | gsar.exe -s:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x02 -r:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x03 -F > "OUTPUTFILE"

this also fixes the decoding issue and it can be executed on Windows on the original file before decoding it.

[Francesco Bucciantini]

It's been three months now, any update on this...?

[Francesco Bucciantini]

By going on with the tests, I realized that this:

ffmpeg -thread_type slice -i "20210429WiganvHullFCX_C921h25m29s09.mxf" -c:v v210 -vf "format=yuv444p,format=yuv422p" "test.avi"

Fixed the decoding problem, so not only -thread_type slice is required, but also a conversion, despite the original file being yv16 (4:2:2 planar 8bit). If you don't put that in there, it will display green fields.

[Balling]

This was an actual bug in third party encoder, yet it is fixed in 507fdcd1b09deed0cfd274d6afb284a99963168f.

[Elon Musk]

This was never bug in ffmpeg encoder, i again ask you to stop commenting spam on this site with your entries with 0 gain.

[Balling]

Replying to richardpl:

This was never bug in ffmpeg encoder, i again ask you to stop commenting spam on this site with your entries with 0 gain.

It is a bug in AVID (or whatever 3rd party that was) encoder, not ffmpeg. Clarified what I meant.

[Francesco Bucciantini]

No it's not.
It comes from cameras deployed on the pitch and we've contacted the manufacturer and they said that the files have the listed differences, but are compliant with the DNX standard.
As such, a bunch of system support them without the blink of an eye like AVID Media Composer and AVID Media Director as well as Adobe Premiere and the EVS Studio Playout System generally deployed in OB Vans etc.
That being said, it's nice that Paul B Mahol actually submitted a patch that addresses the issue as it's the right thing to do.
@Balling please stop monkeying around over here unless you have some serious contributions and productive things to say. This is an open source community, everyone is welcome, but as long as their contribution is useful and doesn't make everyone lose time.
Please don't reply any further to this ticket.

Ticket: Closed.
Status: Fixed.

[Balling]

The rules on this bugtracker is that fixing commit should be mentioned when closing the ticket, I do not care personally, but not in this case as I wanted to test it. As for your "monkeying around" comment, as said in the commit message "Fixes decoding files produced by non-compliant encoders.", so your files were broken. I will continue "monkeying around" as I am one of key handlers and testers of this bugtracker, do not let richardpl a.k.a. Paul B Mahol confuse you. He is always attacking people on mailing list and here too, it is normal and I do care MUCH, since he is usually right and also one of key devs, so who cares.