Opened 14 months ago

Closed 10 months ago

Last modified 10 months ago

#9255 closed defect (fixed)

DNX120 Decoding is broken in FFMpeg and FFPlay (fine in AVID and Adobe Premiere)

Reported by: FranceBB Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: dnxhd race
Cc: FranceBB, Marton Balint Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by FranceBB)

Summary of the bug:

DNX120 decoding/indexing is broken in:

The following players also fail to decode it properly:

The following software decode it correctly:

  • AVID Media Composer
  • AVID Media Director
  • Adobe Premiere
  • EVS Hardware Playout Ports

Here's the mediainfo:

https://i.imgur.com/JFQWYwr.png
https://i.imgur.com/Jlr9sBX.png

This is how it's decoded in FFMpeg / FFplay:

https://i.imgur.com/UdEp4Nt.png

And this is how it should be instead (decoded by AVID):

https://i.imgur.com/dfPB74v.png

Unfortunately, I cannot share the footage of the game (for obvious reasons, it's a copyrighted event), however I asked them to send me a non-sensitive content and they put a blue cloth on the camera and recorded a sample...
I know it kind of sucks as a sample, but here we go:

Sample - https://we.tl/t-SvCQRPGikp

How to reproduce:

ffmpeg -i "GreenFrames.mxf" -vcodec ffvhuff -f avi -y "test.avi"

or

ffplay -i "/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf"


ffmpeg version 2021-05-05-git-7c451b609c
built on 05/05/2021 12:29 PM UTC+01:00

and also

ffplay version 4.4

[FranceBB@router-localhost ~]$ ffplay -i "/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf"
ffplay version 4.4 Copyright (c) 2003-2021 the FFmpeg developers

built with gcc 11 (GCC)
configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libcdio --enable-libdrm --enable-libjack --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librav1e --enable-libsmbclient --enable-version3 --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libglslang --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzvbi --enable-lv2 --enable-avfilter --enable-avresample --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100

Input #0, mxf, from '/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf':

Metadata:

operational_pattern_ul: 060e2b34.04010101.0d010201.01010100
application_platform: Microsoft Windows 7 Professional Service Pack 1 (Build 7601)
uid : f3687fb0-a928-11eb-92b0-0cc47a8346df
generation_uid : f3687fb0-a928-11eb-92b1-0cc47a8346df
company_name : OpenCube
product_name : MXFTk Advanced
product_uid : 3a4fe380-0d01-11e4-869f-3cd92b5c1dfc
product_version : 2.10.4.20180828
product_version_num: 2.10.4.0.1
toolkit_version_num: 2.10.4.0.1
modification_date: 2021-04-29T21:25:14.052000Z
material_package_umid: 0x060A2B34010101050101052013000000F36858A0A92811EB92A40CC47A8346DF
timecode : 21:25:24:09

Duration: 00:00:35.68, start: 0.000000, bitrate: 121346 kb/s
Stream #0:0: Video: dnxhd (DNXHD), yuv422p(bt709/unknown/bt709, top first), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc

Metadata:

file_package_umid: 0x060A2B34010101050101052013000000F3683190A92811EB929C0CC47A8346DF
track_name : Picture

5.24 M-V: 0.045 fd= 0 aq= 0KB vq=15392KB sq= 0B f=0/0

Attachments (2)

gsar.exe (21.0 KB ) - added by FranceBB 13 months ago.
cat.exe (16.0 KB ) - added by FranceBB 13 months ago.

Download all attachments as: .zip

Change History (29)

comment:1 by FranceBB, 14 months ago

Description: modified (diff)

comment:2 by Marton Balint, 14 months ago

Cc: Marton Balint added
Component: avformatavcodec
Keywords: dnxhd race added

It seems -threads 1 or -thread_type slice is a viable workaround.

comment:3 by FranceBB, 14 months ago

Yep, I can confirm that

ffplay -i "/home/FranceBB/Downloads/20210429WiganvHullFCX_C921h25m29s09.mxf" -thread_type slice

works

https://i.imgur.com/mhAMi0w.png

So... where do we go from here?
I mean, for a quick encode I can easily put it there, but I think that there's something wrong with multithreading in the decoder at this point and it should be addressed in the future.

comment:4 by FranceBB, 14 months ago

There are also some DNX120 that play ok from some cameras and some other DNX120 that have green fields (unless we use the -thread_type slice flag) from some totally different cameras.
Going deeper in the analysis, those from the camera brands that are decoded correctly have:

[dnxhd @ 000001f4fd9d7c80] interlaced 3, cur field 1

those from the camera brands that are decoded incorrectly (unless -thread_type slice is specified) have:

[dnxhd @ 0000013fc0277c80] interlaced 2, cur field 0

the values are bits in the frame header, what i don't get is why we have the entries 2 times

[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 0000025bc07c7c80] Profile cid 1242.
[dnxhd @ 0000025bc07c7c80] 1920x1080, 4:2:2 8 bits, MBAFF=0 ACT=0
[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 0000025bc07c7c80] 1920x1080, 4:2:2 8 bits, MBAFF=0 ACT=0

in the cameras that produce the version with the decoding issue, it's 2 times "interlaced 2, cur field 0" while in the camera brands that produce a file correctly decoded without any flag, only the first entry is the same while the second entry is

[dnxhd @ 00000222415e7c80] interlaced 3, cur field 1

Last edited 14 months ago by FranceBB (previous) (diff)

comment:5 by Carl Eugen Hoyos, 14 months ago

I cannot reproduce the issue:
Could somebody provide a checksum for the test file?

comment:6 by Marton Balint, 14 months ago

md5sum GreenFrames.mxf

5bcafcde36bd5a1052f23f65d264ceec GreenFrames.mxf

in reply to:  6 ; comment:7 by Carl Eugen Hoyos, 14 months ago

Replying to Marton Balint:

md5sum GreenFrames.mxf

5bcafcde36bd5a1052f23f65d264ceec GreenFrames.mxf

I have this file but I only see blue frames, no rugby players, no matter the thread count.

comment:8 by FranceBB, 14 months ago

What do you mean?

Decoding with like:

ffplay -i "input.mxf"

leads to green fields, while

ffplay -i "input.mxf" -thread_type slice

works like a charm.

Here is another file: https://we.tl/t-7IU3JYToWT
MD5: 2E39838EF8455E4E15484782D5AC1375
SHA1: 438A3EAEDDCD554F0CCD4290E4A66C1647A86C0E
SHA256: B275FCC7E9D17E2EC2F8CB37086CD0B1D9C3A6B8B619A96598DE92D412732F82

It should be fairly easy to reproduce.

in reply to:  7 comment:9 by Marton Balint, 14 months ago

Replying to Carl Eugen Hoyos:

Replying to Marton Balint:

md5sum GreenFrames.mxf

5bcafcde36bd5a1052f23f65d264ceec GreenFrames.mxf

I have this file but I only see blue frames, no rugby players, no matter the thread count.

It is a bit misleading, the screenshots are from a copyrighted material, but the sample contains blue frames. Green lines are also present there though here.

comment:10 by FranceBB, 14 months ago

Yeah, I shared one of the very short samples from the pitch as well.
That should help you figure it out.

comment:11 by FranceBB, 14 months ago

Very short pitch + players temporary clip: ​https://we.tl/t-7IU3JYToWT

comment:12 by Balling, 14 months ago

Status: newopen

This looks the same as https://github.com/sekrit-twc/zimg/issues/135 and #9077. Does not (?) happen on windows ffplay in latest build here: https://github.com/BtbN/FFmpeg-Builds/

copyrighted event

I am prety sure this is fair use.

Last edited 14 months ago by Balling (previous) (diff)

in reply to:  12 comment:13 by Marton Balint, 14 months ago

Replying to Balling:

This looks the same as https://github.com/sekrit-twc/zimg/issues/135 and #9077. Does not (?) happen on windows ffplay in latest build here: https://github.com/BtbN/FFmpeg-Builds/

No, happens here on x86/linux.

comment:14 by Balling, 14 months ago

In mpv that happens in full screen but does not in windowed (yes, double click breaks and fixes it, insanity). So that is just some kind of crazy interlace. Also, seeking is nuts there, it destroys interlace whatsover.

P.S. It does happen on windows ffplay too only on initial frames, which is IMHO just a corrupted frames issue (or maybe not, cause seeking back to the start works without any green things).

P.S. This looks like some kind of frame reordering for interlace and some kind of strange frame interpolation.

P.S. Yes, on windows interlace is indeed fixed with -thread_type slice, not that I know how to deinterlace this.

Last edited 14 months ago by Balling (previous) (diff)

comment:15 by FranceBB, 13 months ago

@Balling... it is fixed with -thread_type slice on Linux (Fedora 34) as well.

Anyway, this is not an outlier, in the sense that every file we have been receiving from the pitch from this camera were like this, so this is not a corrupted file or anything, it's clearly a decoding error.

That being said, we've been testing several DNX120 and those that are decoded correctly in FFMpeg without the -thread_type slice have:

[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 00000222415e7c80] interlaced 3, cur field 1

while the ones that are decoded incorrectly and need the -thread_type slice in order to decode them correctly have:

[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0
[dnxhd @ 0000025bc07c7c80] interlaced 2, cur field 0

Besides, if I use ffprobe to see what's going on in the files that are not decoded correctly I get:

ffprobe "test.dnxhd" -show_frames |findstr top_field_first

top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1
top_field_first=0
top_field_first=1

Now, according to the DNX standard, we have one frame header per field (unlike in other codecs like MPEG-2 where we only have one frame header for each pair of fields)

The frame header start sequence is 00 00 02 80 01 - the next byte tells us if we deal with the first or second field of an interlaced frame

00 00 02 80 01 02 indicates first field

00 00 02 80 01 03 indicates second field

If I take one of the DNX120 which is decoded correctly in ffmpeg by default, I have 02 and 03 that change with every frame header, like so:

frame 0, first field 00 00 02 80 01 02

frame 0, second field 00 00 02 80 01 03

00 00 02 80 01 02

00 00 02 80 01 03

00 00 02 80 01 02

00 00 02 80 01 03

but this is not the case for the DNX120 files we have received from the pitch in which we have:

00 00 02 80 01 02

00 00 02 80 01 02

00 00 02 80 01 03

00 00 02 80 01 03

00 00 02 80 01 02

00 00 02 80 01 02

00 00 02 80 01 03

00 00 02 80 01 03

See the issue here?

Last edited 13 months ago by FranceBB (previous) (diff)

comment:16 by Balling, 13 months ago

That is the point. What does it mean two top, two bottom fields? Is that how 4:2:2 is encoded or what?

comment:17 by FranceBB, 13 months ago

I don't know, but I mean, it shouldn't be the same, it should be different with one header per field rather than per frame I think. I've seen that kind of pattern in other codecs but not in DNX120, but this is also the reason I opened the bug in the first place. I don't know why it's behaving like this in the sense that I don't know why in an interlaced DNX we have the same header in both fields but that's the way it is, so... where do we go from here? I mean, can we work around this in the decoder so that it's gonna be aware of this possibility and decode it as such anyway? Also, why the heck is it not broken with -thread_type slice but it is for slice+frame? I'm a broadcast encoder, I work for Sky, but I'm mostly an Avisynth guy, so I don't know the inner working of the avcodec in FFMpeg...

by FranceBB, 13 months ago

Attachment: gsar.exe added

comment:18 by FranceBB, 13 months ago

Using gsar to modify the header also works and makes the files possible to decode like so:

type "INPUTFILE" | C:\temp\gsar.exe -s:x83:x09:x40:x00:x00:x00:x02:x80:x01:x03 -r:x83:x09:x40:x00:x00:x00:x02:x80:x01:x02 -F | C:\temp\gsar.exe -s:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x02 -r:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x03 -F > "OUTPUTFILE"

Still, I believe this should be addressed anyway without using gsar.

by FranceBB, 13 months ago

Attachment: cat.exe added

comment:19 by FranceBB, 13 months ago

cat.exe "INPUTFILE" | gsar.exe -s:x83:x09:x40:x00:x00:x00:x02:x80:x01:x03 -r:x83:x09:x40:x00:x00:x00:x02:x80:x01:x02 -F | gsar.exe -s:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x02 -r:x60:x0D:xC0:xDE:x00:x00:x02:x80:x01:x03 -F > "OUTPUTFILE"

this also fixes the decoding issue and it can be executed on Windows on the original file before decoding it.

comment:20 by FranceBB, 11 months ago

It's been three months now, any update on this...?

comment:21 by FranceBB, 11 months ago

By going on with the tests, I realized that this:

ffmpeg -thread_type slice -i "20210429WiganvHullFCX_C921h25m29s09.mxf" -c:v v210 -vf "format=yuv444p,format=yuv422p" "test.avi"

Fixed the decoding problem, so not only -thread_type slice is required, but also a conversion, despite the original file being yv16 (4:2:2 planar 8bit). If you don't put that in there, it will display green fields.

comment:22 by Elon Musk, 10 months ago

Resolution: fixed
Status: openclosed

comment:23 by Balling, 10 months ago

This was an actual bug in third party encoder, yet it is fixed in 507fdcd1b09deed0cfd274d6afb284a99963168f.

Last edited 10 months ago by Balling (previous) (diff)

comment:24 by Elon Musk, 10 months ago

This was never bug in ffmpeg encoder, i again ask you to stop commenting spam on this site with your entries with 0 gain.

in reply to:  24 comment:25 by Balling, 10 months ago

Replying to richardpl:

This was never bug in ffmpeg encoder, i again ask you to stop commenting spam on this site with your entries with 0 gain.

It is a bug in AVID (or whatever 3rd party that was) encoder, not ffmpeg. Clarified what I meant.

Last edited 10 months ago by Balling (previous) (diff)

comment:26 by FranceBB, 10 months ago

No it's not.
It comes from cameras deployed on the pitch and we've contacted the manufacturer and they said that the files have the listed differences, but are compliant with the DNX standard.
As such, a bunch of system support them without the blink of an eye like AVID Media Composer and AVID Media Director as well as Adobe Premiere and the EVS Studio Playout System generally deployed in OB Vans etc.
That being said, it's nice that Paul B Mahol actually submitted a patch that addresses the issue as it's the right thing to do.
@Balling please stop monkeying around over here unless you have some serious contributions and productive things to say. This is an open source community, everyone is welcome, but as long as their contribution is useful and doesn't make everyone lose time.
Please don't reply any further to this ticket.

Ticket: Closed.
Status: Fixed.

Last edited 10 months ago by FranceBB (previous) (diff)

comment:27 by Balling, 10 months ago

The rules on this bugtracker is that fixing commit should be mentioned when closing the ticket, I do not care personally, but not in this case as I wanted to test it. As for your "monkeying around" comment, as said in the commit message "Fixes decoding files produced by non-compliant encoders.", so your files were broken. I will continue "monkeying around" as I am one of key handlers and testers of this bugtracker, do not let richardpl a.k.a. Paul B Mahol confuse you. He is always attacking people on mailing list and here too, it is normal and I do care MUCH, since he is usually right and also one of key devs, so who cares.

Note: See TracTickets for help on using tickets.