Opened 2 years ago

Last modified 3 months ago

#9996 open defect

Write joc_complexity_index to dec3 (EAC3SpecificBox), Windows and Android need it to play atmos

Reported by: BlueWindy Owned by:
Priority: normal Component: ffmpeg
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
Here is a Dolby Atmos sample file with an EAC3-JOC audio stream : https://we.tl/jxvi4Pgwz9
source : https://thedigitaltheater.com/dolby-trailers/

From this, if I demux the EAC3-JOC, as you can see the complexity index is set correctly.
<img width="346" alt="demuxed eac3-joc" src="https://user-images.githubusercontent.com/40682277/198607696-cace19e5-ea56-4544-a1a7-6767ffe5027d.png">

However, when I remux MKV to MP4, the complexity index is not present.
<img width="338" alt="MKV to MP4" src="https://user-images.githubusercontent.com/40682277/198609017-02fc30f6-d458-4412-81d5-2ba1bc04a44c.png">

The reason this bug needs to be resolved is that, in the case of a smartphone that supports Dolby Atmos, the Dolby Atmos virtualizer is automatically activated only when the complexity index is presented.

How to reproduce:

ffmpeg -i DD+Atmos.mkv -c copy DD+Atmos.mp4
ffmpeg version N-108886-g79508ee523-20221028 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.2.0 (crosstool-NG 1.25.0.90_cf9beb1)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --disable-libmfx --enable-libvpl --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20221028
  libavutil      57. 39.101 / 57. 39.101
  libavcodec     59. 51.100 / 59. 51.100
  libavformat    59. 34.101 / 59. 34.101
  libavdevice    59.  8.101 / 59.  8.101
  libavfilter     8. 49.101 /  8. 49.101
  libswscale      6.  8.112 /  6.  8.112
  libswresample   4.  9.100 /  4.  9.100
  libpostproc    56.  7.100 / 56.  7.100
Input #0, matroska,webm, from 'DD+Atmos.mkv':
  Metadata:
    encoder         : libebml v1.3.4 + libmatroska v1.4.5
    creation_time   : 2018-03-08T15:37:18.000000Z
  Duration: 00:01:51.03, start: 0.000000, bitrate: 9165 kb/s
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1k tbn (default)
    Metadata:
      BPS             : 8719852
      BPS-eng         : 8719852
      DURATION        : 00:01:50.966000000
      DURATION-eng    : 00:01:50.966000000
      NUMBER_OF_FRAMES: 3329
      NUMBER_OF_FRAMES-eng: 3329
      NUMBER_OF_BYTES : 120950899
      NUMBER_OF_BYTES-eng: 120950899
      _STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
      _STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1(side), fltp, 448 kb/s (default)
    Metadata:
      BPS             : 448000
      BPS-eng         : 448000
      DURATION        : 00:01:50.944000000
      DURATION-eng    : 00:01:50.944000000
      NUMBER_OF_FRAMES: 3467
      NUMBER_OF_FRAMES-eng: 3467
      NUMBER_OF_BYTES : 6212864
      NUMBER_OF_BYTES-eng: 6212864
      _STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
      _STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
[mp4 @ 000001d3e2ed6980] track 1: codec frame size is not set
Output #0, mp4, to 'DD+Atmos.mp4':
  Metadata:
    encoder         : Lavf59.34.101
  Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 30 tbr, 16k tbn (default)
    Metadata:
      BPS             : 8719852
      BPS-eng         : 8719852
      DURATION        : 00:01:50.966000000
      DURATION-eng    : 00:01:50.966000000
      NUMBER_OF_FRAMES: 3329
      NUMBER_OF_FRAMES-eng: 3329
      NUMBER_OF_BYTES : 120950899
      NUMBER_OF_BYTES-eng: 120950899
      _STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
      _STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:1(eng): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 448 kb/s (default)
    Metadata:
      BPS             : 448000
      BPS-eng         : 448000
      DURATION        : 00:01:50.944000000
      DURATION-eng    : 00:01:50.944000000
      NUMBER_OF_FRAMES: 3467
      NUMBER_OF_FRAMES-eng: 3467
      NUMBER_OF_BYTES : 6212864
      NUMBER_OF_BYTES-eng: 6212864
      _STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
      _STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 3329 fps=0.0 q=-1.0 Lsize=  124269kB time=00:01:50.93 bitrate=9176.7kbits/s speed= 995x
video:118116kB audio:6067kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.068845%

Change History (12)

comment:1 by Balling, 2 years ago

Status: newopen

Yes. I wanted to report that.

comment:2 by BlueWindy, 2 years ago

By the way, can I expect this bug to be fixed in an upcoming new ffmpeg version?

comment:3 by Balling, 2 years ago

Looks as follows: near AudioFormat = ec-3 there is Unknown_dec3, what is called EAC3SpecificBox dec3 there flag_ec3_extension_type_joc and joc_complexity_index. I would imagine this would require parsing of eac3.

0001BE       Audio (51 bytes)
0001BE        Header (8 bytes)
0001BE         Size:                           51 (0x00000033)
0001C2         Name:                           ec-3
0001C6        Reserved:                        0 (0x0000000000000000)
0001CC        Data reference index:            1 (0x0001)
0001CE        reserved (0):                    0 (0x00000000)
0001D2        reserved (0):                    0 (0x00000000)
0001D6        channelcount (2):                2 (0x0002)
0001D8        samplesize (16):                 16 (0x0010)
0001DA        pre_defined (0):                 0 (0x0000)
0001DC        reserved (0):                    0 (0x0000)
0001DE        samplerate:                      48000 (0xBB80)
0001E0        samplerate (0):                  0 (0x0000)
0001E2        EAC3SpecificBox (15 bytes)
0001E2         Header (8 bytes)
0001E2          Size:                          15 (0x0000000F)
0001E6          Name:                          dec3
0001EA         data_rate:                      768 (0x0300) - (13 bits)
0001EB         num_ind_sub:                    0 (0x0) - (3 bits)
0001EC         independent substream (3 bytes)
0001EC          fscod:                         0 (0x0) - (2 bits)
0001EC          bsid:                          16 (0x10) - (5 bits)
0001EC          reserved:                      No
0001ED          asvc:                          No
0001ED          bsmod:                         0 (0x0) - (3 bits)
0001ED          acmod:                         7 (0x7) - (3 bits)
0001ED          lfeon:                         Yes
0001EE          reserved:                      0 (0x0) - (3 bits)
0001EE          num_dep_sub:                   0 (0x0) - (4 bits)
0001EE          reserved:                      No
0001EF         reserved:                       0 (0x0) - (7 bits)
0001EF         flag_ec3_extension_type_joc (2 bytes)
0001EF          flag_ec3_extension_type_joc:   Yes
0001F0          joc_complexity_index:          16 (0x10) - (8 bits)
0001F1      Time to Sample (24 bytes)
Last edited 2 years ago by Balling (previous) (diff)

comment:4 by Balling, 23 months ago

This is worse problem than I thought: on Galaxy S22 when you disable Atmos in icon fast settings, you will have it activated (icon will be lighten blue) in Mx player or Samsung Player. But not without dec3. This is bad.

Last edited 22 months ago by Balling (previous) (diff)

comment:5 by Balling, 22 months ago

Now that #4488 is fixed, we should be able to autorecognize Atmos in THD and EAC3 and thus this allows to write code for this issue. Paul?

comment:7 by BlueWindy, 11 months ago

Can this be fixed in FFmpeg 7.0?

comment:8 by Balling, 3 months ago

So basically you need to change it from 646563330E00200F000000000014 (or whatever) to
646563331800200F00011000000018 or (last 0x18 byte is what signals joc_complexity_index, this means 0x10 is decimal 16, and that increases the size of the box from 14 to 15, size 15 is encoded with 0x0000000F) after ec-3 box.

So basically you need to find 64656333 + some bytes and modify it to be 646563331800200F00011000000018 then you need to change the size of that field from 0E to 0F (right before dec3), then hardest part, you go to next box (btrt, if it is not present -c copy the mp4 to new mp4 and ffmpeg will write btrt box), delete one of btrt box 0x00 bytes (62747274000000000 becomes 62747274000000) and then (most magical part) you decrease the size of btrt box to 0x13. BOOM!

Or you can -c copy it to mkv or to .ec3 those are both supported by new player: https://apps.microsoft.com/detail/9wzdncrfj3pt

Last edited 3 months ago by Balling (previous) (diff)

comment:9 by BlueWindy, 3 months ago

Oh nice! You threw a strike in 🎳. Haha.

comment:10 by Balling, 3 months ago

Summary: When remuxing dolby atmos mkv to mp4, complexity index is not present.Write joc_complexity_index to dec3 (EAC3SpecificBox), Windows and Android need it to play atmos
Version: unspecifiedgit-master

This is issue here: https://github.com/FFmpeg/FFmpeg/blob/a577d313b2c14c855ab8aa69bbe3527bd7727212/libavformat/movenc.c#L646

it needs to write one extra byte and set flag_ec3_extension_type_joc to true of this is atmos stream (nowadays we support Atmos recognition, but not decoding).

comment:11 by BlueWindy, 3 months ago

Hey,if you have ffmpeg.exe that reflects your modifications, could you please share it with me? I would like to have it.

comment:12 by Balling, 3 months ago

you have ffmpeg.exe that reflects your modifications

I just used a hex editor.

Note: See TracTickets for help on using tickets.