Opened 2 years ago

Last modified 5 months ago

#9667 open defect

HE-AAC (not in mp4) decode samples off by one sample-time

Reported by: Nick Owned by:
Priority: minor Component: avcodec
Version: git-master Keywords: aac
Cc: Nick Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Minor quirk - ffmpeg appears to decode HE-AACv1/v2 samples shifted in time (early) by 1 sample time at 48kHz.

1) Generate a WAV file, stereo 48kHz.

2) Use ffmpeg+libfdk_aac to encode the WAV file into AAC-LC, HE-AACv1, and HE-AACv2 ".aac" sample files (48kHz) with the -debug_ts option. The deubg_ts information reports an initial timestamp of -2048 samples for AAC-LC, -5058 samples for HE-AACv1. and -7106 samples for HE-AACv2. The HE-AAC values being a multiple of 2 at 48kHz makes sense as the underlying LC data will be 24kHz.

3) Use ffmpeg to decode the ".aac" sample files to WAV and compare the time-alignment of the resulting WAV file with the original WAV file.

AAC-LC - the output WAV file contains exactly 2048 samples of lead-in as expected.

HE-AACv1 - the output WAV file contains 5057 samples of lead-in where 5058 samples are expected. Being off by 1 at 48kHz (not a multiple of 2) suggests the off-by-one error may be on the decoder side.

HE-AACv2 - the output WAV file contains 7105 samples of lead-in where 7106 samples are expected.

Change History (13)

comment:1 by Balling, 2 years ago

Is native aac encoder affected too?

to decode the ".aac" sample files to WAV

You were supposed to use mp4 container and thus there should be 0 samples of lead-in due to an editlist.

Last edited 2 years ago by Balling (previous) (diff)

comment:2 by Nick, 2 years ago

Is native aac encoder affected too?

The problem is only seen with HE-AACv1/v2 which I don't think the native aac encoder will generate. Testing AAC-LC with the native aac encoder reports an initial timestamp of -1024. After converting back to WAV there are 1024 samples of lead-in as expected.

You were supposed to use mp4 container and thus there should be 0 samples of lead-in due to an editlist.

That is broken for HE-AAC. Encoding WAV to HE-AACv1 mp4 file reports an initial timestamp of -5058 as expected. Converting the mp4 to wav results in losing the first 5060 samples of real audio.

Repeating the same test with AAC-LC works correctly - perfectly aligned with no lead-in and no missing samples.

The symptom suggests the mp4 edit information may be based on 48kHz and may be being incorrectly applied to the 24kHz underlying AAC-LC data when handling HE-AAC.

Last edited 2 years ago by Nick (previous) (diff)

comment:3 by Balling, 2 years ago

I don't think the native aac encoder will generate.

No, it will not.

symptom suggests the mp4 edit information may be based on 48kHz and may be being incorrectly applied to the 24kHz

Just edit the media time in editlist from 5060 to 5058.

comment:4 by Nick, 2 years ago

Just edit the media time in editlist from 5060 to 5058.

Edit it to be 2529?

The encoder is adding 5058/48000 samples of lead-in.

FFmpeg applying the mp4 edit is dropping 10118 samples (5058+5060).

The resulting WAV file is correctly missing the 5058 samples of lead-in but is also missing the first 5060 samples of real audio.

comment:5 by Balling, 20 months ago

Resolution: fixed
Status: newclosed

This is now finally fixed in 5114ce1e2a4c71ddf4971ad3cf9bd43ae16571c3.

but is also missing the first 5060 samples of real audio.

Precisely the problem (it was eating 5058, not 5060 though, you misalligned samples somewhere), so it was eating not only priming but past it.

The sample can be generated by ffmpeg.exe -f lavfi -t 30 -i sine=frequency=1000 -codec:a libfdk_aac -profile:a aac_he testsionwave.mp4

and the problem could have been seen by then comparing with wav done with -ignore_editlist 1.

Of course there is still a problem that the remainder is not removed from the end (30 seconds +1094 samples, 1094 last samples must be removed, but they are not). Known bug.

Last edited 20 months ago by Balling (previous) (diff)

comment:6 by Balling, 20 months ago

the output WAV file contains 5057 samples of lead-in where 5058 samples are expected

This is wrong BTW, it has 5058 samples because you count from 0. So, 0, 1, 2, ... 5057.

Use ffmpeg to decode the ".aac" sample files to WAV and compare the time-alignment of the resulting WAV file with the original WAV file.

You could have just used -ignore_editlist 1 that produces bitperfect file with raw ADTS .aac file.

comment:7 by Balling, 20 months ago

Oh, BTW, -c:a libfdk_aac -i as decoder is still broken in the same way, though that one only has a mistake of 962 samples not 5058 (once again right after priming). 960 is SBR delay and there are indeed 2 additional samples.

Last edited 20 months ago by Balling (previous) (diff)

comment:8 by Carl Eugen Hoyos, 20 months ago

Keywords: aac added; he-aac removed
Version: unspecifiedgit-master

comment:9 by Balling, 20 months ago

Reproduced by developer: set
Resolution: fixed
Status: closedreopened
Summary: HE-AAC decode samples off by one sample-timeHE-AAC (not in mp4) decode samples off by one sample-time

Since it was bothering me, I checked 48k flac decoded to wav and then encoded that way to .aac and .mp4 with libfdk_aac. I then decoded both to wav with native decoder. mp4 is perfectly aligned in Audacity, so removal of initial priming is sample accurate. (Remainder is still not removed.)

Now, I check raw .aac file. Indeed if you will remove 5057 (counting from 0) in Audacity you need to remove one more sample to get in sync. So reopening.

AGAIN, mp4 editlist is not affected!

comment:11 by Balling, 20 months ago

qaac64.exe -gappless-mode 1 is not affected either (in mp4). --he is affected badly, it is not gapless at all.

Last edited 20 months ago by Balling (previous) (diff)

comment:12 by Balling, 5 months ago

Status: reopenedopen

https://patchwork.ffmpeg.org/project/ffmpeg/patch/NhyzMEL--3-9@lynne.ee/

Does it fix "HE-AAC (not in mp4) decode samples off by one sample-time"
Yes, through this mechanism, I add a sample back in, which I forgot to do for this patch, but >I've fixed it locally, and we're in sync:

comment:13 by Balling, 5 months ago

This appears to have regressed. The -c:a libfdk_aac -profile:a aac_he file.mp4

encoded file will be decoded off-by-one sample and it is bad, it eats one extra sample now. MP4 with editlist is affected. Not cool

Note: See TracTickets for help on using tickets.