Opened 4 years ago
Closed 13 months ago
#9996 closed defect (fixed)
Write joc_complexity_index to dec3 (EAC3SpecificBox), Windows and Android need it to play atmos
| Reported by: | BlueWindy | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | avformat |
| Version: | git-master | Keywords: | |
| Cc: | Blocked By: | ||
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description
Summary of the bug:
Here is a Dolby Atmos sample file with an EAC3-JOC audio stream : https://we.tl/jxvi4Pgwz9
source : https://thedigitaltheater.com/dolby-trailers/
From this, if I demux the EAC3-JOC, as you can see the complexity index is set correctly.
<img width="346" alt="demuxed eac3-joc" src="https://user-images.githubusercontent.com/40682277/198607696-cace19e5-ea56-4544-a1a7-6767ffe5027d.png">
However, when I remux MKV to MP4, the complexity index is not present.
<img width="338" alt="MKV to MP4" src="https://user-images.githubusercontent.com/40682277/198609017-02fc30f6-d458-4412-81d5-2ba1bc04a44c.png">
The reason this bug needs to be resolved is that, in the case of a smartphone that supports Dolby Atmos, the Dolby Atmos virtualizer is automatically activated only when the complexity index is presented.
How to reproduce:
ffmpeg -i DD+Atmos.mkv -c copy DD+Atmos.mp4
ffmpeg version N-108886-g79508ee523-20221028 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 12.2.0 (crosstool-NG 1.25.0.90_cf9beb1)
configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --disable-libmfx --enable-libvpl --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20221028
libavutil 57. 39.101 / 57. 39.101
libavcodec 59. 51.100 / 59. 51.100
libavformat 59. 34.101 / 59. 34.101
libavdevice 59. 8.101 / 59. 8.101
libavfilter 8. 49.101 / 8. 49.101
libswscale 6. 8.112 / 6. 8.112
libswresample 4. 9.100 / 4. 9.100
libpostproc 56. 7.100 / 56. 7.100
Input #0, matroska,webm, from 'DD+Atmos.mkv':
Metadata:
encoder : libebml v1.3.4 + libmatroska v1.4.5
creation_time : 2018-03-08T15:37:18.000000Z
Duration: 00:01:51.03, start: 0.000000, bitrate: 9165 kb/s
Stream #0:0: Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1k tbn (default)
Metadata:
BPS : 8719852
BPS-eng : 8719852
DURATION : 00:01:50.966000000
DURATION-eng : 00:01:50.966000000
NUMBER_OF_FRAMES: 3329
NUMBER_OF_FRAMES-eng: 3329
NUMBER_OF_BYTES : 120950899
NUMBER_OF_BYTES-eng: 120950899
_STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
_STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1(side), fltp, 448 kb/s (default)
Metadata:
BPS : 448000
BPS-eng : 448000
DURATION : 00:01:50.944000000
DURATION-eng : 00:01:50.944000000
NUMBER_OF_FRAMES: 3467
NUMBER_OF_FRAMES-eng: 3467
NUMBER_OF_BYTES : 6212864
NUMBER_OF_BYTES-eng: 6212864
_STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
_STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
[mp4 @ 000001d3e2ed6980] track 1: codec frame size is not set
Output #0, mp4, to 'DD+Atmos.mp4':
Metadata:
encoder : Lavf59.34.101
Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 30 tbr, 16k tbn (default)
Metadata:
BPS : 8719852
BPS-eng : 8719852
DURATION : 00:01:50.966000000
DURATION-eng : 00:01:50.966000000
NUMBER_OF_FRAMES: 3329
NUMBER_OF_FRAMES-eng: 3329
NUMBER_OF_BYTES : 120950899
NUMBER_OF_BYTES-eng: 120950899
_STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
_STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:1(eng): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 448 kb/s (default)
Metadata:
BPS : 448000
BPS-eng : 448000
DURATION : 00:01:50.944000000
DURATION-eng : 00:01:50.944000000
NUMBER_OF_FRAMES: 3467
NUMBER_OF_FRAMES-eng: 3467
NUMBER_OF_BYTES : 6212864
NUMBER_OF_BYTES-eng: 6212864
_STATISTICS_WRITING_APP: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_APP-eng: mkvmerge v12.0.0 ('Trust / Lust') 64bit
_STATISTICS_WRITING_DATE_UTC: 2018-03-08 15:37:18
_STATISTICS_WRITING_DATE_UTC-eng: 2018-03-08 15:37:18
_STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 3329 fps=0.0 q=-1.0 Lsize= 124269kB time=00:01:50.93 bitrate=9176.7kbits/s speed= 995x
video:118116kB audio:6067kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.068845%
Change History (22)
comment:1 by , 4 years ago
| Status: | new → open |
|---|
comment:2 by , 4 years ago
By the way, can I expect this bug to be fixed in an upcoming new ffmpeg version?
comment:3 by , 3 years ago
Looks as follows: near AudioFormat = ec-3 there is Unknown_dec3, what is called EAC3SpecificBox dec3 there flag_ec3_extension_type_joc and joc_complexity_index. I would imagine this would require parsing of eac3.
0001BE Audio (51 bytes) 0001BE Header (8 bytes) 0001BE Size: 51 (0x00000033) 0001C2 Name: ec-3 0001C6 Reserved: 0 (0x0000000000000000) 0001CC Data reference index: 1 (0x0001) 0001CE reserved (0): 0 (0x00000000) 0001D2 reserved (0): 0 (0x00000000) 0001D6 channelcount (2): 2 (0x0002) 0001D8 samplesize (16): 16 (0x0010) 0001DA pre_defined (0): 0 (0x0000) 0001DC reserved (0): 0 (0x0000) 0001DE samplerate: 48000 (0xBB80) 0001E0 samplerate (0): 0 (0x0000) 0001E2 EAC3SpecificBox (15 bytes) 0001E2 Header (8 bytes) 0001E2 Size: 15 (0x0000000F) 0001E6 Name: dec3 0001EA data_rate: 768 (0x0300) - (13 bits) 0001EB num_ind_sub: 0 (0x0) - (3 bits) 0001EC independent substream (3 bytes) 0001EC fscod: 0 (0x0) - (2 bits) 0001EC bsid: 16 (0x10) - (5 bits) 0001EC reserved: No 0001ED asvc: No 0001ED bsmod: 0 (0x0) - (3 bits) 0001ED acmod: 7 (0x7) - (3 bits) 0001ED lfeon: Yes 0001EE reserved: 0 (0x0) - (3 bits) 0001EE num_dep_sub: 0 (0x0) - (4 bits) 0001EE reserved: No 0001EF reserved: 0 (0x0) - (7 bits) 0001EF flag_ec3_extension_type_joc (2 bytes) 0001EF flag_ec3_extension_type_joc: Yes 0001F0 joc_complexity_index: 16 (0x10) - (8 bits) 0001F1 Time to Sample (24 bytes)
comment:4 by , 3 years ago
This is worse problem than I thought: on Galaxy S22 when you disable Atmos in icon fast settings, you will have it activated (icon will be lighten blue) in Mx player or Samsung Player. But not without dec3. This is bad.
comment:5 by , 3 years ago
Now that #4488 is fixed, we should be able to autorecognize Atmos in THD and EAC3 and thus this allows to write code for this issue. Paul?
comment:6 by , 3 years ago
comment:8 by , 22 months ago
So basically you need to change it from 646563330E00200F000000000014 (or whatever) to
646563331800200F00011000000018 or (last 0x18 byte is what signals joc_complexity_index, this means 0x10 is decimal 16, and that increases the size of the box from 14 to 15, size 15 is encoded with 0x0000000F) after ec-3 box.
So basically you need to find 64656333 + some bytes and modify it to be 646563331800200F00011000000018 then you need to change the size of that field from 0E to 0F (right before dec3), then hardest part, you go to next box (btrt, if it is not present -c copy the mp4 to new mp4 and ffmpeg will write btrt box), delete one of btrt box 0x00 bytes (62747274000000000 becomes 62747274000000) and then (most magical part) you decrease the size of btrt box to 0x13. BOOM!
Or you can -c copy it to mkv or to .ec3 those are both supported by new player: https://apps.microsoft.com/detail/9wzdncrfj3pt
comment:10 by , 22 months ago
| Summary: | When remuxing dolby atmos mkv to mp4, complexity index is not present. → Write joc_complexity_index to dec3 (EAC3SpecificBox), Windows and Android need it to play atmos |
|---|---|
| Version: | unspecified → git-master |
This is issue here: https://github.com/FFmpeg/FFmpeg/blob/a577d313b2c14c855ab8aa69bbe3527bd7727212/libavformat/movenc.c#L646
it needs to write one extra byte and set flag_ec3_extension_type_joc to true of this is atmos stream (nowadays we support Atmos recognition, but not decoding).
comment:11 by , 22 months ago
Hey,if you have ffmpeg.exe that reflects your modifications, could you please share it with me? I would like to have it.
comment:12 by , 22 months ago
you have ffmpeg.exe that reflects your modifications
I just used a hex editor.
comment:13 by , 15 months ago
https://ffmpeg.org/pipermail/ffmpeg-devel/2025-March/341429.html
Did I understand correctly that FFMPEG 7.0 now supports decoding for TrueHD Atmos? If true, wouldn't that be huge news?
comment:14 by , 14 months ago
I'm looking into a patch for this for myself. I don't have enough dev experience to write an official patch.
As far as I know, this affects Atmos on iOS as well and causes MediaInfo to not show it as an Atmos track either.
mkv isn't an option, since AVPlayer (the only way to play Atmos on iOS) doesn't support mkv.
Upvoted this ticket.
follow-up: 17 comment:15 by , 14 months ago
causes MediaInfo to not show it as an Atmos track either.
That is false.
comment:16 by , 14 months ago
Hey 🎳, Could you possibly write this patch? I don't want to wait 10 years for someone to come along and write this patch...
follow-up: 18 comment:17 by , 14 months ago
Replying to Balling:
That is false.
Using my version of ffmpeg (in jellyfin server), it wasn't showing as Atmos at all. Maybe due to the other flag not being set either. Not sure.
Replying to BlueWindy:
Hey 🎳, Could you possibly write this patch? I don't want to wait 10 years for someone to come along and write this patch...
Patch here: https://github.com/jellyfin/jellyfin-ffmpeg/issues/584#issuecomment-2936063150, I tested and it's working properly
comment:18 by , 14 months ago
Replying to CosmicSkye:
Patch here: https://github.com/jellyfin/jellyfin-ffmpeg/issues/584#issuecomment-2936063150, I tested and it's working properly
OMG OMG OMG
Finally... someone finally provided a patch after 3 years of posting this ticket. It actually works!! Thank you so much for making this ticket closeable.
comment:19 by , 14 months ago
| Resolution: | → fixed |
|---|---|
| Status: | open → closed |
comment:20 by , 14 months ago
| Resolution: | fixed |
|---|---|
| Status: | closed → reopened |
The patch is not applied yet, so you cannot close the ticket.
comment:21 by , 14 months ago
Can you please test the patchset in https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=14696 ? It implements nyanmisaka's fix with some changes.
comment:22 by , 13 months ago
| Component: | ffmpeg → avformat |
|---|---|
| Resolution: | → fixed |
| Status: | reopened → closed |
Should be implemented in ebcf2dcb2c424da7084b18832043529c3dd62e0f.
Yes. I wanted to report that.