#1407 closed enhancement (fixed)
HE-AAC (v2): 960/120 MDCT window is not implemented
| Reported by: | Carl Eugen Hoyos | Owned by: | |
|---|---|---|---|
| Priority: | wish | Component: | avcodec |
| Version: | git-master | Keywords: | aac |
| Cc: | rafael@riseup.net, Maurits, basic.master@gmx.de | Blocked By: | |
| Blocking: | Reproduced by developer: | yes | |
| Analyzed by developer: | no |
Description
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/145562
An aac sample with 960/120 MDCT was uploaded by a user, mdct_error.flv is now in samples/A-codecs/AAC
$ ffmpeg -i mdct_error.flv
ffmpeg version N-41284-ge00b2d2 Copyright (c) 2000-2012 the FFmpeg developers
built on Jun 4 2012 08:52:52 with gcc 4.3.2
configuration: --cc=/usr/local/gcc-4.3.2/bin/gcc --enable-gpl --enable-libopenjpeg --enable-libvorbis --enable-libspeex --enable-libmp3lame --enable-libtheora --extra-ldflags=-lm --enable-libvpx --enable-libxavs
libavutil 51. 56.100 / 51. 56.100
libavcodec 54. 23.100 / 54. 23.100
libavformat 54. 6.101 / 54. 6.101
libavdevice 54. 0.100 / 54. 0.100
libavfilter 2. 78.100 / 2. 78.100
libswscale 2. 1.100 / 2. 1.100
libswresample 0. 15.100 / 0. 15.100
libpostproc 52. 0.100 / 52. 0.100
[aac @ 0x9018c00] 960/120 MDCT window is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x9018c00] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/MPlayer/incoming/ and contact the ffmpeg-devel mailing list.
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, flv, from 'mdct_error.flv':
Metadata:
fps : 25
SAR : 64:45
Duration: 00:00:31.04, start: 0.000000, bitrate: 1675 kb/s
Stream #0:0: Video: h264 (Constrained Baseline), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 tbr, 1k tbn, 50 tbc
Stream #0:1: Audio: aac, 48000 Hz, stereo
At least one output file must be specified
Attachments (5)
Change History (49)
comment:1 by , 13 years ago
comment:2 by , 11 years ago
Also Digital Radio Mondiale (a Digital Radio standard) uses AAC960.
I have some samples if needed.
comment:3 by , 11 years ago
Btw, a simple way to reintroduce support for AAC960 is re-add faad2 support compiled in 960 mode.
comment:4 by , 11 years ago
| Priority: | wish → minor |
|---|
I think the fact the ffmpeg does not support the playback of the audio from two major Digital Radio standards justify to raise the priority from wish to minor.
comment:5 by , 11 years ago
| Cc: | added |
|---|
comment:6 by , 11 years ago
| Priority: | minor → wish |
|---|
comment:7 by , 11 years ago
What is happening here, cehoyos?
According to the Issue Tracker manual[1]:
minor
(...)
Feature requests about things few people want or which do not make a big
difference.
wish
Something that is desirable to have but that there is no urgency at
all to implement, e.g. something completely cosmetic like a website
restyle or a personalized doxy template or the FFmpeg logo.
This priority is not valid for bugs.
Is it not clear that we have a minor bug here? I'll not change to priority higher, but I'd say that it's even a normal or important!
important
For feature requests this priority would be used for things many people
want.
[1] - http://git.videolan.org/?p=ffmpeg.git;a=blob_plain;f=doc/issue_tracker.txt;hb=HEAD
comment:9 by , 11 years ago
Sorry, but enhancements or feature requests have priority wish on this issue tracker.
comment:10 by , 11 years ago
| Priority: | minor → wish |
|---|
follow-up: 12 comment:11 by , 11 years ago
FYI, in case anyone wants to take this on, the people over at OpenDigitalRadio.org have patched their version of the Fraunhofer FDK AAC codec library to support the AAC 960 transform that is required for DAB+ digital radio.
The code can be found at: https://github.com/Opendigitalradio/fdk-aac-dabplus
comment:12 by , 11 years ago
| Cc: | added |
|---|
Replying to Maurits:
FYI, in case anyone wants to take this on, the people over at OpenDigitalRadio.org have patched their version of the Fraunhofer FDK AAC codec library to support the AAC 960 transform that is required for DAB+ digital radio.
Do I understand correctly that the people at "Open"DigitalRadio decided to patch a proprietary, non-free decoder instead of the one in libavcodec?
Do you agree that this is not exactly helpful for the issue described here?
comment:13 by , 11 years ago
The code he linked is for encoding DAB+ compatible audio, not decoding.
PS:
fdk-aac is free and open-source, just not in a way that is compatible with the GPL.
follow-up: 15 comment:14 by , 10 years ago
Is there still any link to an aac stream with 960 or 120 samples frame length ?
I tried to reproduce with:
wget http://www.dabmon.de/p/8002/playlist/1.m3u
cat 1.m3u -> http://www.dabmon.de/p/8002/stream/1.aac
ffplay http://www.dabmon.de/p/8002/stream/1.aac
and there is no odd factor length imdct used in the decoding process
comment:15 by , 10 years ago
Replying to DerouineauNicolas:
Is there still any link to an aac stream with 960 or 120 samples frame length ?
Try any of these: http://dabmonitor.nl
It is easier to hear the issue with human voices (they sound unnaturally deep), with music it's harder to distinguish as it just sounds slightly "slower". If you capture the stream I would expect you to see the difference, though.
comment:17 by , 9 years ago
| Cc: | added |
|---|
comment:19 by , 9 years ago
All over the air digital radio in the DAB+ and DRM standards use 960 transform AAC variation. With a cheap rtl2832U you can receive DAB+. You can easily create a 960 samples AAC with the modified fdk-aac which the link is posted in the thread.
comment:20 by , 9 years ago
How, I downloaded modified fdk-aac and failed to create sample. I downloaded odr-audioenc and that one produces gibberish that faad2 cant decode.
comment:21 by , 9 years ago
All that DABMON streams are still AAC with 960 samples/frame. The problem is that these streams use the ADTS format, which cannot incorporate the signalling whether 960 or 1024 was used. But this signalling is required, as all AAC decoders (I have seen so far) otherwise default to 1024 samples/frame and the audio sounds slow and low-pitched, as described!
The 960/1024 samples/frame signalling is done by the frameLengthFlag within the GASpecificConfig, which in turn is contained in the AudioSpecificConfig (all defined in ISO 14496-3). So for correct playback, such AAC content has to be stored in a format that contains this config. This may be e.g. just an MP4/ISOBMFF container or an LATM stream.
ODR-AudioEnc does not produce plain AAC frames, but the so-called Superframes. A Superframe is a special container defined by DAB+ (ETSI TS 102 563) that has a fixed length (for a given service). Within a Superframe, there are (depending on the usage of SBR, and of 48/32 kHz) 2 to 6 AAC frames of variable length. As this format is intended for DAB+ transmission, it is hardly surprising that it cannot be played without modification by usual players.
I will provide DAB+ samples in both LATM and MP4 format, with the necessary 960 signalling. Both are played correctly by VLC, but fail with ffplay and the mentioned error message.
by , 9 years ago
| Attachment: | aac_isobmff_960.m4a added |
|---|
HE-AAC in MP4/ISOBMFF format with 960 samples/frame
comment:22 by , 9 years ago
mdct_error.flv seems to expect the the full 128 coeffs for short windows using the top SFB which seems to be incompatible with the spec and FDK. Using this 128 limit would break conformant streams using the top SFB for non-PNS bands.
comment:23 by , 9 years ago
Meanwhile support for 960 samples/frame has been added:
https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/dbc9a8f21f92d0613142ea23bb836356fc41de38
(only discovered due to this issue: https://github.com/piratfm/eti-tools/issues/10)
While the support seems to be fine for the above AAC-LC sample, according to the console warnings there is no support (yet) for SBR (HE-AAC) and/or PS (HE-AAC v2) for the 960 case. Thus in such cases only the half-bandwidth respectively Mono core is decoded.
follow-up: 31 comment:24 by , 5 years ago
| Status: | new → open |
|---|---|
| Summary: | aac: 960/120 MDCT window is not implemented → HE-AAC (v2): 960/120 MDCT window is not implemented |
Where is HE-AAC v2 sample??
Also the mdct_error.flv now prints
[aac @ 000002142aaef380] Number of bands (15) exceeds limit (14).
[aac @ 000002142aaef380] invalid band type 200KB sq= 0B f=0/0
[aac @ 000002142aaef380] Scalefactor (-1) out of range.
[aac @ 000002142aaef380] invalid band type
Last message repeated 1 times
[aac @ 000002142aaef380] Number of bands (18) exceeds limit (14).
Last message repeated 1 times
[aac @ 000002142aaef380] Number of bands (17) exceeds limit (14).
[aac @ 000002142aaef380] Scalefactor (-1) out of range. 0B f=0/0
[aac @ 000002142aaef380] Number of bands (17) exceeds limit (14).
[aac @ 000002142aaef380] invalid band type 234KB sq= 0B f=0/0
[aac @ 000002142aaef380] Number of bands (15) exceeds limit (14).
[aac @ 000002142aaef380] invalid band type
[aac @ 000002142aaef380] Scalefactor (-1) out of range.
[aac @ 000002142aaef380] Number of bands (17) exceeds limit (14).
[aac @ 000002142aaef380] Number of bands (16) exceeds limit (14).
[aac @ 000002142aaef380] Number of bands (17) exceeds limit (14).
[aac @ 000002142aaef380] invalid band type
[aac @ 000002142aaef380] Number of bands (17) exceeds limit (14).
[aac @ 000002142aaef380] Pulse tool not allowed in eight short sequence.
***
[aac @ 000002142aaef380] Gain control is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
***
[aac @ 000002f5d27ff380] Sample rate index in program config element does not match the sample rate index configured by the container.
follow-up: 26 comment:25 by , 5 years ago
What do you mean by "Where is HE-AAC v2 sample??"
You can even create one with fdk-aac.
comment:26 by , 5 years ago
Replying to rafael2k:
What do you mean by "Where is HE-AAC v2 sample??"
You can even create one with fdk-aac.
I mean with 960 MDCT.
follow-up: 28 comment:27 by , 5 years ago
Do you want me to create one more 960 MDCT HE-AACv2 sample with latest fdk-aac v2?
comment:28 by , 5 years ago
Replying to rafael2k:
Do you want me to create one more 960 MDCT HE-AACv2 sample with latest fdk-aac v2?
Sure.
comment:29 by , 4 years ago
I am adding here a sample generated decoding a DAB+ radio. Latest ffprobe reports:
ffprobe version N-107228-g9fdfc74877 Copyright (c) 2007-2022 the FFmpeg developers built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1) configuration: libavutil 57. 27.100 / 57. 27.100 libavcodec 59. 33.100 / 59. 33.100 libavformat 59. 25.100 / 59. 25.100 libavdevice 59. 6.100 / 59. 6.100 libavfilter 8. 41.100 / 8. 41.100 libswscale 6. 6.100 / 6. 6.100 libswresample 4. 6.100 / 4. 6.100 [aac_latm @ 0x55b1c1a7bf40] SBR with 960 frame length is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented. [aac_latm @ 0x55b1c1a7bf40] SBR with 960 frame length is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented. Input #0, loas, from 'test.out': Duration: N/A, bitrate: N/A Stream #0:0: Audio: aac_latm (LC), 24000 Hz, mono, fltp [aac_latm @ 0x55b1c1a8e440] SBR with 960 frame length is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
It would be nice after 10 years to have ffmpeg capable of decoding DAB+ radios directly
comment:30 by , 4 years ago
I agree. Fdk-aac decoder does support 960 MDCT transform:
https://github.com/mstorsjo/fdk-aac/blob/master/libAACdec/include/aacdecoder_lib.h
comment:31 by , 4 years ago
Replying to Balling:
Where is HE-AAC v2 sample??
I uploaded a sample, is it possible to have this wish granted? It is quite unusual that ffmpeg can't decode the frames of the main Digital Radio format of Europe and Australia
In case you need to know how the samples have been generated, I used example 4 from https://github.com/JvanKatwijk/dab-cmdline/
comment:33 by , 3 years ago
comment:34 by , 3 years ago
This is needed not only for DAB+, but also for DRM (Digital Radio Mondiale) broadcast decoding.
comment:35 by , 7 months ago
There is currently an open WIP pull-request with a proposed fix for this problem on ffmpeg's forgejo instance: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20941
comment:36 by , 3 months ago
I have recorded a french DAB+ radio sample with dablast which produce a MPEG-TS file and keep the native codec audio track (DAB-FMU.ts) :
Format : AAC LC SBR Format/Info : Advanced Audio Codec Low Complexity with Spectral Band Replication Commercial name : HE-AAC Format settings : NBC Muxing mode : LATM
I have compiled the Lynne's ffmpeg version but the sound is distorded at playback.
When I open the waveform I can see the following:
each 40ms there is a nearly 3ms of zero samples (on both channels).
1 / 48000 = 0,00002083 sec/sample
0,040 / (1 / 48000) = 1920
128 * (1 / 48000) = 0,00266667 ms <-- length of zero samples ?
I don't have enought knowledge in he-aac to fix it by myself (i try !) but i'm really interested in this ticket.
follow-up: 38 comment:37 by , 3 months ago
have compiled the Lynne's ffmpeg version but the sound is distorded at playback.
Can you also test incomplete patch here
comment:38 by , 3 months ago
Replying to Balling:
have compiled the Lynne's ffmpeg version but the sound is distorded at playback.
Can you also test incomplete patch here
Ok i'm trying it. To apply the patch I have followed that doc. So :
git clone https://git.ffmpeg.org/ffmpeg.git && cd ffmpeg curl https://patchwork.ffmpeg.org/series/4252/mbox/ | git am -3
but it failed :
Applying: avcodec/aacdec: add support for AAC SBR with 960 frame length error: corrupt patch at line 25 error: could not build fake ancestor Patch failed at 0001 avcodec/aacdec: add support for AAC SBR with 960 frame length
- Is it because the patch is incomplete and we have to manually apply each diff ?
- Do I have to switch on a specific branch (release/6.1 ?)
- I didn't understand which commit is the direct ancestor.
Thanks for welcoming a newbie here :)
comment:40 by , 3 months ago
| Resolution: | → fixed |
|---|---|
| Status: | open → closed |
Issue fixed by c1b19ee69f2142bb4b098936cc7421d80b3db7e4
It's fully compliant support. Let me know if it works for you, I tested it on all the samples I could find.
comment:41 by , 3 months ago
Thank you Lynne ! I will test it on my system and give you feedback here in case of problem.
comment:42 by , 3 months ago
I noticed initial priming gap in bigger if you decode to 48000 Hz instead of 24000 Hz. I would imagine this is something very easy to fix, but there are many other AAC priming issues and one HE-AAC issue still open.
by , 3 months ago
| Attachment: | ffmpeg-initial-20ms-delay.png added |
|---|
20ms delay in pcm conversion between ffmpeg-master and ffmpeg-8.1
comment:43 by , 3 months ago
I can observe that for a same source file DAB-FMU.ts (48kHz stéréo), when I decode to PCM, there's a delay of 20ms between the file decoded by ffmpeg 8.1 release and the current ffmpeg master branch. Is that what you call "initial priming gap" Balling ?
ffmpeg-8.1 -i DAB-FMU.ts ffmpeg-8.1.wav ffmpeg-master -i DAB-FMU.ts ffmpeg-dab.wav
Some additional information.
While the 960 frame length is rare in standalone files, it is common in streams such as those used in DAB+ digital radio.
In case anyone needs more examples or test samples, this is a status overview of a typical German DAB+ digital radio multiplex: http://www.dabmon.de:8002/
Clicking on any of the streams after 'label:' will give you a AAC960 stream to test. While the stream will be played in most cases, it is clearly being played a bit too slow due to FFmpeg incorrectly treating it as AAC1024.