DolbyED2 track cannot be decoded

[Francesco Bucciantini]

Summary of the bug: DolbyED2 5.1.4 Atmos tracks cannot be decoded, unlike their DolbyE 5.1 or 5.1+2.0 counterparts which can be decoded.

Unlike DolbyE which is carried as 1 AES (i.e 1 fake stereo pair - so 1 .wav - or 2 fake mono pairs - so 2 .wav), DolbyED2 is carried as 2 AES (i.e 2 fake stereo pairs) so either 2 separate fake stereo .wav files that need to be merged together to be decoded or 1 fake quad .wav file.

If we use the same logic as DolbyE, however, the decoder doesn't recognize the file as a valid file and therefore doesn't decode it.

How to reproduce:

% 
rem first we merge the 2 fake stereo pairs (2 AES) in 1:

ffmpeg.exe -i "Y:\00_INGEST_MAM\DolbyED2\ED2A.wav" -i "Y:\00_INGEST_MAM\DolbyED2\ED2B.wav" -filter_complex "[0:a][1:a]join=inputs=2:channel_layout=stereo[a]" -map "[a]" -c:a pcm_s24le -ar 48000 -f wav -y "A:\MEDIA\temp\DolbyED2\DolbyED2.wav"


rem now we save the DolbyED2 file as .u8 to decode it

ffmpeg.exe -i "A:\MEDIA\temp\DolbyED2\DolbyED2.wav" -map 0:0 -acodec copy -f u8 -y "A:\MEDIA\temp\DolbyED2\stream1.u8"

rem now we cycle through the channels of the DolbyED2 to decode it, but while this works with DolbyE files, unfortunately DolbyED2 ones aren't recognized...

ffmpeg.exe  -i "A:\MEDIA\temp\DolbyED2\stream1.u8" -acodec pcm_s24le -ar 48000 -ac 1 -map_channel 0.0.0:0.0.0 -y "A:\MEDIA\temp\DolbyED2\Out1.wav"

ffmpeg.exe  -i "A:\MEDIA\temp\DolbyED2\stream1.u8" -acodec pcm_s24le -ar 48000 -ac 1 -map_channel 0.0.1:0.0.0 -y "A:\MEDIA\temp\DolbyED2\Out2.wav"

ffmpeg.exe  -i "A:\MEDIA\temp\DolbyED2\stream1.u8" -acodec pcm_s24le -ar 48000 -ac 1 -map_channel 0.0.2:0.0.0 -y "A:\MEDIA\temp\DolbyED2\Out3.wav"

ffmpeg.exe  -i "A:\MEDIA\temp\DolbyED2\stream1.u8" -acodec pcm_s24le -ar 48000 -ac 1 -map_channel 0.0.3:0.0.0 -y "A:\MEDIA\temp\DolbyED2\Out4.wav"

ffmpeg.exe  -i "A:\MEDIA\temp\DolbyED2\stream1.u8" -acodec pcm_s24le -ar 48000 -ac 1 -map_channel 0.0.4:0.0.0 -y "A:\MEDIA\temp\DolbyED2\Out5.wav"

ffmpeg.exe  -i "A:\MEDIA\temp\DolbyED2\stream1.u8" -acodec pcm_s24le -ar 48000 -ac 1 -map_channel 0.0.5:0.0.0 -y "A:\MEDIA\temp\DolbyED2\Out6.wav"

ffmpeg version n5.2-dev-2022-08-01
built on 2022-08-01

Expected result: .u8 is recognized and decoded
Actual result: A:\MEDIA\temp\DolbyED2\stream1.u8: Invalid data found when processing input

With normal DolbyE tracks decoding works: ​https://forum.doom9.org/showthread.php?t=176739

The sample in attachment is DolbyED2 2 AES already coupled together, ready to be decoded, lasting just 1 sec due to the bug tracker restrictions.

More info about DolbyED2 ​https://developer.dolby.com/globalassets/professional/whitepapers/dolby-ed2-whitepaper.pdf

[Francesco Bucciantini]

DolbyED2 Sample trimmed to 1 sec

[Balling]

Your file mostly consists of zero bytes.

[Francesco Bucciantini]

DolbyED2 Sample 2

[Francesco Bucciantini]

DolbyED2 Sample 3

[Francesco Bucciantini]

DolbyED2 Sample 4

[Francesco Bucciantini]

DolbyED2 Sample 5

[Francesco Bucciantini]

I would upload a bigger sample if the bug tracker allowed me.

Here are more samples trimmed at different points with FFMpeg, as requested:

ffmpeg.exe -i "DolbyED2.wav" -acodec copy -ss 1200 -t 1 -f wav -y "DolbyED2_trim_2.wav"
ffmpeg.exe -i "DolbyED2.wav" -acodec copy -ss 1900 -t 1 -f wav -y "DolbyED2_trim_3.wav"
ffmpeg.exe -i "DolbyED2.wav" -acodec copy -ss 2400 -t 1 -f wav -y "DolbyED2_trim_4.wav"
ffmpeg.exe -i "DolbyED2.wav" -acodec copy -ss 2401 -t 8 -f wav -y "DolbyED2_trim_5.wav"

And if you look with an hex editor, you can see data:

[Carl Eugen Hoyos]

Please provide longer samples.

[Francesco Bucciantini]

Hi Carl,
no problem. :)
I was trying to upload to your FTP, namely upload.ffmpeg.org but it fails to login and if I visit it with a browser ​https://upload.ffmpeg.org/ it says "The service has been decommissioned". :|
Is there any other official place where I could upload the sample?

In the meantime, here's a WeTransfer with the samples:

• DolbyED2 carried as 2 fake stereo tracks that need to be merged (2 AES)

​https://we.tl/t-ckWIdhYkW8

[Balling]

It is here: ​https://streams.videolan.org/upload/

Select ffmpeg issue number.

The service that was decommissioned is Jenkins. In fact you can see it on broken https warning on that site.

[Francesco Bucciantini]

Gotcha, thanks Balling.
I have uploaded the file called DolbyED2.rar at the address you provided, ​https://streams.videolan.org/upload/

Inside you're gonna find two files:

• DolbyED2_AES1.wav
• DolbyED2_AES2.wav

those are the two fake stereo pairs that carry the DolbyED2 Atmos 5.1.4, so ideally they should be merged together first to get 1 file and then decoded.

[Balling]

​https://streams.videolan.org/ffmpeg/incoming/9864/

[Francesco Bucciantini]

More about DolbyED2 can be found here: ​https://professionalsupport.dolby.com/s/article/Dolby-ED2-Technology-Brief?language=en_US

[Francesco Bucciantini]

Patch by Nicola Gaullier that allows ffmpeg to decode the provided samples

[Balling]

This should also be applied cause apparently even first version was not correct 100% ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220913213127.1756-1-nicolas.gaullier@cji.paris/

[Francesco Bucciantini]

Yes, that patch right there that Balling quoted ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220913213127.1756-1-nicolas.gaullier@cji.paris/ should really be merged upstream regardless of DolbyED2 'cause it's really really important. That bug affects the 20 bits section of standard DolbyE 5.1 + 2.0 files which raises an error “Read past end of channel XX” while decoding them with FFMpeg. That is bug 'cause it's not just a "simple warning" but rather a real error as it really truncates the audio, hence throwing away few ms. When I decode the very same file with the official Dolby DP600, however, there are no warnings and the decoding completes correctly. By applying Nicola's patch, there's gonna be a simple warning, but data will be preserved and we're gonna avoid the few ms of mute audio. This is really important as we use DolbyE 5.1 + 2.0 in production on a daily basis, therefore it should really be merged sooner rather than later.

[Francesco Bucciantini]

So... any chance of any of the patches being merged to master anytime soon?

[Elon Musk]

Nope

[Francesco Bucciantini]

Hi everyone,
happy new year.
As you can see this pull request has been stagnating for the last 6 months, forcing us at Sky to build our own FFMpeg with the patch applied.
We've been trying to contact the current maintainer via email on different occasions, but we never had an a reply.
Is there any other maintainer we can talk to?

Thank you in advance,
Frank

[Balling]

​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220913213127.1756-1-nicolas.gaullier@cji.paris/ your pull -- your ping. You should ping the mailing list every week or so.

[Francesco Bucciantini]

I can ping the mailing list, but who should I write to, Balling? I mean, the generic address or do you have someone in particular I can bother directly that might be able to take a look at these two pull requests and merge them?
Currently we compiled our own FFMpeg with the two patches and we're using it in production 'cause we needed it, but of course this is not how we roll and we'd like to have it merged rather than staying on our fork.

[Francesco Bucciantini]

Ok, Balling, I followed your suggestion.
I sent a second email to ​ffmpeg-devel@ffmpeg.org and I will follow your suggestion of sending out an email every week until the patches are merged.

Thanks.

[Balling]

Your ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230213180936.815-7-nicolas.gaullier@cji.paris/ is failing in make FATE, just as before: ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20201019081929.1926-9-nicolas.gaullier@cji.paris/

[Balling]

ffmpeg.exe -i msdvas.dat -strict experimental avcafva1.flac

will actually encode Dolby E to 32 bit integer flac. NUTS. Love it.

Anyway, Francesco Bucciantini, I have a question. What does this code do, 44800 Hz to be precise? ​https://github.com/FFmpeg/FFmpeg/commit/44e27d937d88f3e7cdc83f4de1b9d3a590bcaa04#diff-ec791aea00ab0f331287a3ffd7765c130b63f9b57d8bc62cd55be8611a5ab29aL85

I have a Dolby E file that is considered 44800 Hz, yet it should obviously be 48000 Hz, according to ​https://github.com/MediaArea/MediaInfoLib/pull/1299#issuecomment-847029150

I am confused about this.

I am still confused about 44800 Hz. ffmpeg produces 32 bit wav (default for dolby e)

Channel(s)                  : 8 channels
Channel layout              : L R C LFE Lb Rb Ls Rs
Sampling rate               : 44.8 kHz
Bit depth                   : 32 bits

and

Audio
Format                      : FLAC
Format/Info                 : Free Lossless Audio Codec
Duration                    : 5 s 0 ms
Bit rate mode               : Variable
Bit rate                    : 5 360 kb/s
Channel(s)                  : 8 channels
Channel layout              : L R C LFE Lb Rb Ls Rs
Sampling rate               : 44.8 kHz
Bit depth                   : 32 bits
Compression mode            : Lossless
Stream size                 : 3.19 MiB (100%)
Writing library             : Lavf60.2.100

[Balling]

[dolby_e @ 0000026ccff60d80] Read past end of channel 7ts/s speed

is now indeed

[dolby_e @ 000002534a95e5c0] Truncated mantissas @945, highest frequencies not recoverable

nice. LGTM.

[Francesco Bucciantini]

@Balling... about the 44800Hz, it's very simple really. So DolbyE spoofs itself as PCM 24bit 48'000Hz stereo when it's carried through SDI, however internally its sampling is different. When you encode in DolbyE, you have something called frame_loc, also referred to as "Line positioning". This value is adjusted according to the frame rate of the file and it dictates how many samples are inserted within a certain interval. Dolby set out some boundaries that your DolbyE file has to be within and they're highlighted here: ​https://developer.dolby.com/globalassets/professional/dolby-e/w08_20466_dolby_e_technical_paper.pdf
(check page 8)

Since it's an old document, stuff like UHD 50p is not listed, but I can tell you it's line 42.
Anyway, this means that DolbyE will have its own sampling which isn't 48'000Hz but rather something lower, internally. In this case, ffmpeg is detecting the real internal DolbyE sampling, which is 44'800Hz (very unusual, I know, given that people are used to see either 44'100Hz or 48'000Hz streams, but it's just the way it is). Of course, when for instance you play it on a DolbyE compatible hardware decoder, it will almost always be brought to 48'000Hz upon decoding by the internal circuitry, but since we're living in a software day and age, this will be FFMpeg's duty once the stream is decoded, however, the decoder won't do it automatically 'cause perhaps you might wanna have the real sampling without any resampling so it allows you to get it decoded at the original sampling (in this case 44'800Hz given that it's a 25i FULL HD stream on line 24).

[Balling]

​https://github.com/MediaArea/MediaInfoLib/pull/1299#issuecomment-1434471782

For 29.97 fps content, sampling rate is not an integer: 1792 * 30 / 1.001 = 53706.24 Hz

Is this true? cause if it is there is a bug in ffmpeg, it just uses 53706.

​https://github.com/FFmpeg/FFmpeg/commit/44e27d937d88f3e7cdc83f4de1b9d3a590bcaa04#diff-ec791aea00ab0f331287a3ffd7765c130b63f9b57d8bc62cd55be8611a5ab29aL85

you play it on a DolbyE compatible hardware decoder, it will almost always be brought to 48'000Hz upon decoding

Is that a good idea for 53706 Hz, e.g.? Are there garantees Dolby E encoder cannot accept >48000 Hz wav to encode? Also what is up with Dolby E decoder in ffmpeg decoding to 32 bit? Is that because lossy 24 bit already needs 32 bits to preserve itself? Does Dolby E encoders even allow 32 bit integer wavs?

[Francesco Bucciantini]

I live in PAL land, so I always had to deal with either 25i SD or FULL HD, so I don't know for sure about 29,970i. I'm just glad that as far as DolbyE is concerned for PAL 25i, we have 1792x25=44800Hz

The same is true for 50p in UHD for PAL, by the way, 'cause it's still 44800Hz, in fact it's still being made as if it was a 25fps stream, but with an higher line positioning.

I also gotta say that the hardware I have over here is pretty old and can only play a maximum of 48'000Hz, but given that as I said we're a European broadcaster it wouldn't make any difference anyway 'cause no matter if it's a 25i or 50p stream, it will always be 44800Hz upsampled to 48'000Hz upon playback. ;)

[Francesco Bucciantini]

Hi Balling,
I actually checked and yes, the internal sample rate of Dolby E varies depending on the associated video frame-rate between 42965Hz and 53760Hz, however such sample rate is ALWAYS converted back to 48kHz after decoding, doesn't matter if it's higher (like in NTSC) or lower (like in PAL).

[Francesco Bucciantini]

Hi there, everyone!
Nicolas added the required documentation in a/doc/decoders.texi about the DolbyE decoder and in a/doc/demuxers.texi about the Demuxer.
In particular, the patch now is fully completed including the required documentation part and you can find it here: ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230309175125.19691-2-nicolas.gaullier@cji.paris/

The Dolby E audio decoder is a raw decoder without any further processing, which means the output sample rate is different from the 48KHz input sample rate. A dolby E stream is always carried in a s337m muxer, as a raw file or muxed in any format that is able to transport an AES3 or an equivalent uncompressed audio stream (stereo, 48KHz, 16/20/24 bits). Currently, only wav and raw file formats are supported. Other formats require two passes of processing: a first one to extract the s337m stream and another to decode it. In case of AES3, or if the format itself supports it, the non-PCM mode may be signaled, but in a more general manner, a stream has to be probed to be handled as dolby E rather than pcm. By default, stream probing is enabled which forbids pass-through as no s337m muxer is implemented yet. In order to pass-through a dolby E stream, @code{--non_pcm_mode copy} must be specified (see input muxer options).

Please review the patch and finally merge it once and for all or at least give us some kind of feedback.

[Francesco Bucciantini]

Hi guys, it's been one year since the last version of the patch has been submitted, is there any chance to get this reviewed and merged once and for all? I'm literally begging you at this point as there's no point for us to continue building our own version of FFMpeg in FFAStrans if our contributions can't make decoding better for the wider community.