Opened 2 years ago
Last modified 8 weeks ago
#9996 open defect
Write joc_complexity_index to dec3 (EAC3SpecificBox), Windows and Android need it to play atmos
| Reported by: | BlueWindy | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | ffmpeg |
| Version: | git-master | Keywords: | |
| Cc: | Blocked By: | ||
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description
Summary of the bug:
Here is a Dolby Atmos sample file with an EAC3-JOC audio stream : https://we.tl/jxvi4Pgwz9
source : https://thedigitaltheater.com/dolby-trailers/
From this, if I demux the EAC3-JOC, as you can see the complexity index is set correctly.
<img width="346" alt="demuxed eac3-joc" src="https://user-images.githubusercontent.com/40682277/198607696-cace19e5-ea56-4544-a1a7-6767ffe5027d.png">
However, when I remux MKV to MP4, the complexity index is not present.
<img width="338" alt="MKV to MP4" src="https://user-images.githubusercontent.com/40682277/198609017-02fc30f6-d458-4412-81d5-2ba1bc04a44c.png">
The reason this bug needs to be resolved is that, in the case of a smartphone that supports Dolby Atmos, the Dolby Atmos virtualizer is automatically activated only when the complexity index is presented.
How to reproduce:
ffmpeg -i DD+Atmos.mkv -c copy DD+Atmos.mp4
ffmpeg version N-108886-g79508ee523-20221028 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 12.2.0 (crosstool-NG 1.25.0.90_cf9beb1)
configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --disable-libmfx --enable-libvpl --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20221028
libavutil 57. 39.101 / 57. 39.101
libavcodec 59. 51.100 / 59. 51.100
libavformat 59. 34.101 / 59. 34.101
libavdevice 59. 8.101 / 59. 8.101
libavfilter 8. 49.101 / 8. 49.101
libswscale 6. 8.112 / 6. 8.112
libswresample 4. 9.100 / 4. 9.100
libpostproc 56. 7.100 / 56. 7.100
Input #0, matroska,webm, from 'DD+Atmos.mkv':
Metadata:
encoder : libebml v1.3.4 + libmatroska v1.4.5
creation_time : 2018-03-08T15:37:18.000000Z
Duration: 00:01:51.03, start: 0.000000, bitrate: 9165 kb/s
Stream #0:0: Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1k tbn (default)
Metadata:
BPS : 8719852
BPS-eng : 8719852
DURATION : 00:01:50.966000000
DURATION-eng : 00:01:50.966000000
NUMBER_OF_FRAMES: 3329
NUMBER_OF_FRAMES-eng: 3329
NUMBER_OF_BYTES : 120950899
NUMBER_OF_BYTES-eng: 120950899
_STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
_STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1(side), fltp, 448 kb/s (default)
Metadata:
BPS : 448000
BPS-eng : 448000
DURATION : 00:01:50.944000000
DURATION-eng : 00:01:50.944000000
NUMBER_OF_FRAMES: 3467
NUMBER_OF_FRAMES-eng: 3467
NUMBER_OF_BYTES : 6212864
NUMBER_OF_BYTES-eng: 6212864
_STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
_STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
[mp4 @ 000001d3e2ed6980] track 1: codec frame size is not set
Output #0, mp4, to 'DD+Atmos.mp4':
Metadata:
encoder : Lavf59.34.101
Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 30 tbr, 16k tbn (default)
Metadata:
BPS : 8719852
BPS-eng : 8719852
DURATION : 00:01:50.966000000
DURATION-eng : 00:01:50.966000000
NUMBER_OF_FRAMES: 3329
NUMBER_OF_FRAMES-eng: 3329
NUMBER_OF_BYTES : 120950899
NUMBER_OF_BYTES-eng: 120950899
_STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
_STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:1(eng): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 448 kb/s (default)
Metadata:
BPS : 448000
BPS-eng : 448000
DURATION : 00:01:50.944000000
DURATION-eng : 00:01:50.944000000
NUMBER_OF_FRAMES: 3467
NUMBER_OF_FRAMES-eng: 3467
NUMBER_OF_BYTES : 6212864
NUMBER_OF_BYTES-eng: 6212864
_STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
_STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 3329 fps=0.0 q=-1.0 Lsize= 124269kB time=00:01:50.93 bitrate=9176.7kbits/s speed= 995x
video:118116kB audio:6067kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.068845%
Change History (12)
comment:1 by , 2 years ago
| Status: | new → open |
|---|
comment:2 by , 23 months ago
By the way, can I expect this bug to be fixed in an upcoming new ffmpeg version?
comment:3 by , 22 months ago
Looks as follow: near AudioFormat = ec-3 there is Unknown_dec3, what is called EAC3SpecificBox dec3 there flag_ec3_extension_type_joc and joc_complexity_index. I would imagine this would require parsing of eac3.
0001BE Audio (51 bytes) 0001BE Header (8 bytes) 0001BE Size: 51 (0x00000033) 0001C2 Name: ec-3 0001C6 Reserved: 0 (0x0000000000000000) 0001CC Data reference index: 1 (0x0001) 0001CE reserved (0): 0 (0x00000000) 0001D2 reserved (0): 0 (0x00000000) 0001D6 channelcount (2): 2 (0x0002) 0001D8 samplesize (16): 16 (0x0010) 0001DA pre_defined (0): 0 (0x0000) 0001DC reserved (0): 0 (0x0000) 0001DE samplerate: 48000 (0xBB80) 0001E0 samplerate (0): 0 (0x0000) 0001E2 EAC3SpecificBox (15 bytes) 0001E2 Header (8 bytes) 0001E2 Size: 15 (0x0000000F) 0001E6 Name: dec3 0001EA data_rate: 768 (0x0300) - (13 bits) 0001EB num_ind_sub: 0 (0x0) - (3 bits) 0001EC independent substream (3 bytes) 0001EC fscod: 0 (0x0) - (2 bits) 0001EC bsid: 16 (0x10) - (5 bits) 0001EC reserved: No 0001ED asvc: No 0001ED bsmod: 0 (0x0) - (3 bits) 0001ED acmod: 7 (0x7) - (3 bits) 0001ED lfeon: Yes 0001EE reserved: 0 (0x0) - (3 bits) 0001EE num_dep_sub: 0 (0x0) - (4 bits) 0001EE reserved: No 0001EF reserved: 0 (0x0) - (7 bits) 0001EF flag_ec3_extension_type_joc (2 bytes) 0001EF flag_ec3_extension_type_joc: Yes 0001F0 joc_complexity_index: 16 (0x10) - (8 bits) 0001F1 Time to Sample (24 bytes)
comment:4 by , 22 months ago
This is worse problem than I thought: on Galaxy S22 when you disable Atmos in icon fast settings, you will have it activated (icon will be lighten blue) in Mx player or Samsung Player. But not without dec3. This is bad.
comment:5 by , 21 months ago
Now that #4488 is fixed, we should be able to autorecognize Atmos in THD and EAC3 and thus this allows to write code for this issue. Paul?
comment:6 by , 21 months ago
comment:8 by , 2 months ago
So basically you need to change it from 646563330E00200F000000000014 (or whatever) to
646563331800200F00011000000018 or (last 0x18 byte is what signals joc_complexity_index, this means 0x10 is decimal 16, and that increases the size of the box from 14 to 15, size 15 is encoded with 0x0000000F) after ec-3 box.
So basically you need to find 64656333 + some bytes and modify it to be 646563331800200F00011000000018 then you need to change the size of that field from 0E to 0F (right before dec3), then hardest part, you go to next box (btrt, if it is not present -c copy the mp4 to new mp4 and ffmpeg will write btrt box), delete one of btrt box 0x00 bytes (62747274000000000 becomes 62747274000000) and then (most magical part) you decrease the size of btrt box to 0x13. BOOM!
Or you can -c copy it to mkv or to .ec3 those are both supported by new player: https://apps.microsoft.com/detail/9wzdncrfj3pt
comment:10 by , 2 months ago
| Summary: | When remuxing dolby atmos mkv to mp4, complexity index is not present. → Write joc_complexity_index to dec3 (EAC3SpecificBox), Windows and Android need it to play atmos |
|---|---|
| Version: | unspecified → git-master |
This is issue here: https://github.com/FFmpeg/FFmpeg/blob/a577d313b2c14c855ab8aa69bbe3527bd7727212/libavformat/movenc.c#L646
it needs to write one extra byte and set flag_ec3_extension_type_joc to true of this is atmos stream (nowadays we support Atmos recognition, but not decoding).
comment:11 by , 2 months ago
Hey,if you have ffmpeg.exe that reflects your modifications, could you please share it with me? I would like to have it.
comment:12 by , 8 weeks ago
you have ffmpeg.exe that reflects your modifications
I just used a hex editor.
Yes. I wanted to report that.