Preserve Dolby Vision metadata in mxf

[Elliott]

Summary of the bug:
I tried to remux a JPEG2000 stream with embedded Dolby Vision metadata in MXF Op1A container. The output is a valid J2K but the metadata is lost. I have included the result of Dolby's metafier validation tool below.

Sample clip:
​https://mega.nz/file/015GVDrT#0OR0TAhvHcWUfe2DFjMnY8zMtU_EMrBMSk56Uh89P7o

$ ffmpeg -i S002C003.mxf -c:v copy output.mxf
ffmpeg version N-99865-ga125e08130 Copyright (c) 2000-2020 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.21)
  configuration: --enable-nonfree --enable-gpl --enable-libx264 --enable-swscale --enable-avfilter --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-librsvg --enable-libtheora --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libsoxr --enable-libspeex --enable-libass --enable-libbluray --enable-lzma --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libfribidi --enable-opencl --enable-sdl2 --enable-libx265 --enable-libxvid --enable-libfdk-aac --extra-ldflags=-L/opt/local/lib --extra-cflags=-I/opt/local/include
  libavutil      56. 60.100 / 56. 60.100
  libavcodec     58.112.101 / 58.112.101
  libavformat    58. 64.100 / 58. 64.100
  libavdevice    58. 11.102 / 58. 11.102
  libavfilter     7. 89.100 /  7. 89.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
[mxf @ 0x7f8e9c811c00] wrapping of stream 0 is unknown
Input #0, mxf, from 'S002C003.mxf':
  Metadata:
    operational_pattern_ul: 060e2b34.04010101.0d010201.01010100
    uid             : d6ab93af-1c74-46bf-96d3-245a7367dcaf
    generation_uid  : 9fc540fb-28bc-4ef5-97b5-c4fdbbf506f1
    company_name    : Colorfront
    product_name    : Transkoder
    product_version : 2020.46529.2.10.32
    product_uid     : 7d836e16-37c7-4c22-b2e0-46a717e84f42
    modification_date: 2020-11-05T20:59:00.000000Z
    application_platform: win32
    material_package_umid: 0x060A2B340101010501010F20130000001EF41724BD8F4D1EB7B68E145240DC47
    material_package_name: Material Package
  Duration: 00:00:00.38, start: 0.000000, bitrate: 671190 kb/s
    Stream #0:0: Video: jpeg2000, yuv422p12le(tv, bt2020nc/bt2020/smpte2084, progressive), 3840x2160, SAR 1:1 DAR 16:9, 23.98 tbr, 23.98 tbn, 23.98 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010F2013000000F5184258A2A14676A775F4D6700CB01F
      file_package_name: File Package: PROTOTYPE SMPTE ST 422 / ST 2067-5 frame wrapping of JPEG 2000 codestreams with HDR metadata
      track_name      : PHDR Image Track
    Side data:
      Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.7080,0.2920) g(0.1700,0.7970) b(0.1310 0.0460) wp(0.3127, 0.3290) min_luminance=0.000100, max_luminance=1000.000000
Output #0, mxf, to 'output.mxf':
  Metadata:
    operational_pattern_ul: 060e2b34.04010101.0d010201.01010100
    uid             : d6ab93af-1c74-46bf-96d3-245a7367dcaf
    generation_uid  : 9fc540fb-28bc-4ef5-97b5-c4fdbbf506f1
    company_name    : Colorfront
    product_name    : Transkoder
    product_version : 2020.46529.2.10.32
    product_uid     : 7d836e16-37c7-4c22-b2e0-46a717e84f42
    modification_date: 2020-11-05T20:59:00.000000Z
    application_platform: win32
    material_package_umid: 0x060A2B340101010501010F20130000001EF41724BD8F4D1EB7B68E145240DC47
    material_package_name: Material Package
    encoder         : Lavf58.64.100
    Stream #0:0: Video: jpeg2000, yuv422p12le(tv, bt2020nc/bt2020/smpte2084, progressive), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 23.98 tbr, 23.98 tbn, 23.98 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010F2013000000F5184258A2A14676A775F4D6700CB01F
      file_package_name: File Package: PROTOTYPE SMPTE ST 422 / ST 2067-5 frame wrapping of JPEG 2000 codestreams with HDR metadata
      track_name      : PHDR Image Track
    Side data:
      Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.7080,0.2920) g(0.1700,0.7970) b(0.1310 0.0460) wp(0.3127, 0.3290) min_luminance=0.000100, max_luminance=1000.000000
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame=    9 fps=0.0 q=-1.0 Lsize=   30738kB time=00:00:00.37 bitrate=670801.3kbits/s speed= 7.4x    
video:30724kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.045708%

$ metafier --validate S002C003.mxf 
11/05/2020/16:23:07.719653019 metafier: INFO extracting Metadata, this might take a while...
11/05/2020/16:23:07.726957094 extract_metadata: INFO Metadata Extraction progress: 100%
11/05/2020/16:23:07.727231758 metafier: INFO metadata extraction is done
11/05/2020/16:23:07.735836167 Metadata: INFO Validation Summary:
=========================
Metadata version: "2.0.5"
Level254 CMVersion: 2 1
Aspect Ratios (Canvas - Image): 1.77778 - 1.77778
Frame Rate: 23.976fps
Mastering Monitor: 1000-nit, BT.2020, D65, ST.2084, Full (ID 21): Bit Depth: 16bit - Diagonal: 42.00in - Application Type: ALL - Color Info: "pq(0.0001,1000) rgb computer bt2020"
Mastering Monitor Validation Test: PASS
Target Displays Validation Test: PASS
Level6 (MaxFALL - MaxCLL): 32.00 - 255.00
Color Encoding: "u12 422 pq(0,10000) ycbcr_bt2020 video bt2020"
Color Encoding Validation Test: PASS
Overlapping Shots Validation Test: PASS
Gap between Shots Validation Test: PASS
Negative Shot duration Validation Test: PASS
Per-Frame Data out-of-range Validation Test: PASS
Number of Shots: 1
Frame Range: 0-8
L2 Trim Count (TargetID, count): (1, 1)
L1 Metadata Validation Test: PASS
L2 Metadata Validation Test: PASS
=========================
11/05/2020/16:23:07.735989360 metafier: INFO No issues found in the Metadata

$ metafier --validate output.mxf 
KLV format error, zero BER length not allowed.
Error initializing OP1a header packet.
11/05/2020/16:23:15.468424019 metafier: INFO extracting Metadata, this might take a while...
11/05/2020/16:23:15.468997729 FormatMXF: ERROR KLV format error, zero BER length not allowed.
11/05/2020/16:23:15.469039473 FormatMXF: ERROR Error initializing OP1a header packet.
11/05/2020/16:23:15.469091205 FormatMXF: ERROR Bad MXF file
11/05/2020/16:23:15.469681792 FormatMXF: ERROR BER Length decoding error
11/05/2020/16:23:15.469695405 FormatMXF: ERROR Failed to initialize RIP.
11/05/2020/16:23:15.469704566 FormatMXF: ERROR File contains no RIP
11/05/2020/16:23:15.469784860 FormatMXF: ERROR KLV format error, zero BER length not allowed.
11/05/2020/16:23:15.469796837 FormatMXF: ERROR Error initializing OP1a header packet.
11/05/2020/16:23:15.469826632 FormatMXF: ERROR TrackFileReader::OpenMXFRead, header init failed
11/05/2020/16:23:15.472073597 mezzanine: ERROR Unable to open MXF 'output.mxf' for read
11/05/2020/16:23:15.472762459 metafier: ERROR Unable to open file (check existence and permissions)

[Elliott]

And this command changed the SMPTE UL from UniTrack to MultiTrack.

[Tomas Härdin]

I took a quick look at both files using mxfdump. Some dark metadata indeed gets lost after remuxing. The original file also uses a CDCIEssenceDescriptor with a JPEG2000PictureSubDescriptor, while the output of ffmpeg has no JPEG2000PictureSubDescriptor. The errors about zero BER length also seem relevant.

Getting the output of mxfenc.c to be in line with what metafier expects is likely to take a bit of effort, since these analyzers tend to be picky. It is certainly doable however, I've done similar things in the past. Would you be willing to finance such an effort?

[Elliott]

I have no commercial interest in this, nor do I have the skills to implement myself. I only shared these details to help the community.

[Tomas Härdin]

Fair enough. I don't have the time to work on this at the moment, but maybe someone else does.

[Balling]

JPEG2000PictureSubDescriptor is done here: ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200913224218.60972-1-remiachard@gmail.com/

I did apply the copy paste in string for mxf_jp2k_rsiz, so will not apply cleanly.

BTW, does anything change with new parser fixes?