mxf muxer writes the wrong bit depth and uses the CDCI essence descriptor UL instead of the RGBA essence descriptor UL for Full Range RGB48 MJPEG2000

[Francesco Bucciantini]

Summary of the bug:

When muxing a MJPEG2000 RGB Full Range stream, the mxf muxer is using the CDCI essence descriptor UL instead of the RGBA essence descriptor UL.
In other words, it's using

{ { 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x28,0x00 }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /* CDCI */

instead of

{ { 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x29,0x00 }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /* RGBA */

On top of this, the mxf muxer adds the color sampling descriptor as 4:4:4 which doesn't make sense as it's RGB, not YUV.

Last but not least, instead of writing 10bit for 10bit RGB sources and 12bit for 12bit RGB sources, it always writes 16bit, which of course is also wrong.
This results from the fact that FFMpeg is trying to the first compatible match when there is no colr atom.

How to reproduce:

1) Download the standard IMF evaluation package from here: ​https://aswf-dpel-assets.s3.amazonaws.com/asc-stem2/IMF/StEM2_TST_SDR_Rec709_IMF_App2e_Lossy.zip

2) Create the final mxf file by reading the CPL with FFMpeg like ffmpeg.exe -f imf -i "CPL.xml" -c:v copy -c:a copy -f mxf -y "IMF.mxf"

3) Check the resulting file: it was supposed to be rgb48le 10bit BT709, however the muxed file has all the above mentioned things wrong: wrong bit depth inside the container, CDCI wrongly populated instead of RGBA, causing it to be interpreted as YUV 4:4:4 16bit by other softwares other than ffmpeg and ffprobe themselves.

[Francesco Bucciantini]

​https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavformat/mxfdec.c#l3224

Check line 3224, that's CDCI, while line 3225 is RGBA

[Carl Eugen Hoyos]

Please provide the command line you tested together with the complete, uncut console output to make this a valid ticket.

[Tomas Härdin]

Just looking at the code it appears the muxer indeed always writes CDCIDescriptor, never RGBADescriptor. AV_PIX_FMT_FLAG_RGB needs to be checked for this, and the muxer is currently not really architectured for this. A stop gap would be to error out if the essence is RGB, until a proper solution is written.

I'm working on j2k stuff lately, so it's possible I may get some time to fix this.

[Francesco Bucciantini]

Hi Tom,
please don't make it error out as I'm currently using the remux after reading the CPL in an automated fashion as I have a workflow based on that.
It's basically FFMpeg that reads the CPL and remuxes to .mxf and then Avisynth indexes it and gets it ready to encode to XAVC with x264, so if it errors out it would break my workflow.

I would say to wait until a proper fix is implemented, but thank you for looking into that, Thomas! :)

[Jérôme Martinez]

RGBA example

[Jérôme Martinez]

Without having to download the whole huge package, I created a fake ("broken or empty index" warning must be ignored) small file "RGBA_example.mxf​".

> ffmpeg -i .\RGBA_example.mxf -c copy buggy.mxf
ffmpeg version 2022-09-19-git-4ba68639ca-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57. 36.102 / 57. 36.102
  libavcodec     59. 44.100 / 59. 44.100
  libavformat    59. 32.100 / 59. 32.100
  libavdevice    59.  8.101 / 59.  8.101
  libavfilter     8. 49.100 /  8. 49.100
  libswscale      6.  8.112 /  6.  8.112
  libswresample   4.  9.100 /  4.  9.100
  libpostproc    56.  7.100 / 56.  7.100
[mxf @ 000002054e07e480] broken or empty index
Input #0, mxf, from '.\RGBA_example.mxf':
  Metadata:
    operational_pattern_ul: 060e2b34.04010101.0d010201.01010100
    uid             : fac91339-668e-4d99-86ab-3273187e34b0
    generation_uid  : bc056b60-8d8b-4cb2-aaa8-2c5a978cf6b4
    company_name    : Colorfront
    product_name    : Transkoder
    product_version_num: 2020.0.0.51469.1
    product_version : 2020.51469.2.10.38
    product_uid     : 7d836e16-37c7-4c22-b2e0-46a717e84f42
    modification_date: 2022-06-09T18:31:39.000000Z
    toolkit_version_num: 2.10.38.27240.1
    application_platform: win32
    material_package_umid: 0x060A2B340101010501010F2013000000F62F901B2E8247A5935D0D50B05B89D4
    material_package_name: Material Package
  Duration: 00:17:26.42, start: 0.000000, bitrate: 0 kb/s
  Stream #0:0: Video: jpeg2000, rgb48le(10 bpc, bt709, progressive), 4096x1716, SAR 1:1 DAR 1024:429, 24 tbr, 24 tbn
    Metadata:
      file_package_umid: 0x060A2B340101010501010F201300000072DA4BAA76994154B11FFA0633762E39
      file_package_name: File Package: SMPTE ST 422 / ST 2067-5 frame wrapping of JPEG 2000 codestreams
      track_name      : Image Track
Output #0, mxf, to 'buggy.mxf':
  Metadata:
    operational_pattern_ul: 060e2b34.04010101.0d010201.01010100
    uid             : fac91339-668e-4d99-86ab-3273187e34b0
    generation_uid  : bc056b60-8d8b-4cb2-aaa8-2c5a978cf6b4
    material_package_name: Material Package
    material_package_umid: 0x060A2B340101010501010F2013000000F62F901B2E8247A5935D0D50B05B89D4
    product_version_num: 2020.0.0.51469.1
    application_platform: win32
    product_uid     : 7d836e16-37c7-4c22-b2e0-46a717e84f42
    modification_date: 2022-06-09T18:31:39.000000Z
    toolkit_version_num: 2.10.38.27240.1
    encoder         : Lavf59.32.100
  Stream #0:0: Video: jpeg2000, rgb48le(10 bpc, bt709, progressive), 4096x1716 [SAR 1:1 DAR 1024:429], q=2-31, 24 tbr, 24 tbn
    Metadata:
      file_package_umid: 0x060A2B340101010501010F201300000072DA4BAA76994154B11FFA0633762E39
      file_package_name: File Package: SMPTE ST 422 / ST 2067-5 frame wrapping of JPEG 2000 codestreams
      track_name      : Image Track
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=-1.0 Lsize=      61kB time=00:00:00.00 bitrate=N/A speed=   0x
video:53kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 15.153484%

But the issue is in the produced file, not visible in the FFmpeg output. With
mediainfo --Details=1 buggy.mxf:

12A8 CDCI Essence Descriptor (312 bytes)
12A8  Header (20 bytes)
12A8   Code (16 bytes)
12A8    Start (0x06):                        6 (0x06)
12A9    Length of the remaining key (0x0E):  14 (0x0E)
12AA    ISO, ORG (0x2B):                     43 (0x2B)
12AB    SMPTE (0x34):                        52 (0x34)
12AC    Category:                            2 (0x02) - Group (Set/Pack)
12AD    Registry:                            83 (0x53) - 2-byte tag, 2-byte length
12AE    Structure:                           1 (0x01) - 
12AF    Version:                             1 (0x01)
12B0    Item Designator:                     13 (0x0D) - User Organisation Registered For Public Use
12B1    Organization:                        1 (0x01) - AAF
12B2    Application:                         1 (0x01) - Structural Metadata Sets
12B3    Structure Version:                   1 (0x01) - Version 1
12B4    Structure Kind:                      1 (0x01) - MXF / AAF Association compatible sets & packs
12B5    Set Kind (1):                        1 (0x01)
12B6    Set Kind (2):                        40 (0x28)
12B7    Reserved:                            0 (0x00)
12B8   Length:                               131 (0x83)
12B9   Length:                               292 (0x000124)

It is the CDCI descriptor UL, it should be the RGBA descriptor label (byte at 12B6 should be 0x29).

1350  Horizontal colour subsampling - 1 (0x1) (8 bytes)
1350   Header (4 bytes)
1350    Code:                                13058 (0x3302)
1352    Length:                              4 (0x0004)
1354   Data:                                 1 (0x00000001)
1358  Vertical colour subsampling - 1 (0x1) (8 bytes)
1358   Header (4 bytes)
1358    Code:                                13064 (0x3308)
135A    Length:                              4 (0x0004)
135C   Data:                                 1 (0x00000001)

Chroma subsampling is not in the source file, it is there (legit but non sense), they should be removed when RGB.

1348  Active bits per sample - 16 (0x10) (8 bytes)
1348   Header (4 bytes)
1348    Code:                                13057 (0x3301)
134A    Length:                              4 (0x0004)
134C   Data:                                 16 (0x00000010)

Bit depth is wrong, even if it is well detected by FFmpeg (indicated as "rgb48 10 bpc"), byte at 134C should be 0x0A.

Initial check of the issue is from ​https://github.com/MediaArea/MediaInfo/issues/657 .

[Tomas Härdin]

Replying to Francesco Bucciantini:

Hi Tom,
please don't make it error out as I'm currently using the remux after reading the CPL in an automated fashion as I have a workflow based on that.
It's basically FFMpeg that reads the CPL and remuxes to .mxf and then Avisynth indexes it and gets it ready to encode to XAVC with x264, so if it errors out it would break my workflow.

I would say to wait until a proper fix is implemented, but thank you for looking into that, Thomas! :)

I'm having email problems at the moment, so my stop gap hasn't even made it to the list. But yeah RGBADescriptor should be used, not CDCI. Might as well implement all RGB(A) pixel formats while at it.

[Tomas Härdin]

Do you happen to know what the name of the JPEG2000 MXF wrapping spec is?

[Jérôme Martinez]

Do you happen to know what the name of the JPEG2000 MXF wrapping spec is?

SMPTE ST 422:2019
​https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8984770
(surprisingly it seems free of charge)

Note that the attached file RGBA_example.mxf is from an official IMF evaluation package, and it has good descriptors (essence descriptor with link to jp2k specific descriptor), it also has the J2CLayout ("The nature and order of the image components in the compressed domain as carried in the J2C codestream.") with the RGBALayout (from ST 377) part set to RGB, having the CDCI descriptor replaced by the RGBA descriptor would be a fix of the main issue (catching that this is RGB and not YUV), but having the J2CLayout (or the whole JP2k descriptor) transferred from the input file to the output file would be even better (more compliant to specs, but to be honest RGBALayout is optional so not the biggest problem, RGB can be guessed with RGBA descriptor UL alone and the readout of 3 JP2k components).