FFmpeg fails to decode AAC with embedded tags

[shirt]

Summary of the bug: FFmpeg fails to decode AAC with embedded ID3v2 tags.
How to reproduce:
Use the attached aac file with ffmpeg. It works on ffmpeg 3.0.1 but fails on master. Also works after being processed with mkvmerge.

% ffmpeg -i test6000.aac -map 0:a -f null -
ffmpeg version git-2021-09-13-989febf-ffmpeg-windows-build-helpers Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.2.0 (GCC)
  configuration: --pkg-config=pkg-config --pkg-config-flags=--static --extra-version=ffmpeg-windows-build-helpers --enable-version3 --disable-debug --disable-w32threads --arch=x86_64 --target-os=mingw32 --cross-prefix=/home/runner/work/ffmpeg-autobuild/ffmpeg-autobuild/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32- --enable-libcaca --enable-gray --enable-libtesseract --enable-fontconfig --enable-gmp --enable-gnutls --enable-libass --enable-libbluray --enable-libbs2b --enable-libflite --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libvorbis --enable-libwebp --enable-libzimg --enable-libzvbi --enable-libmysofa --enable-libopenjpeg --enable-libopenh264 --enable-liblensfun --enable-libvmaf --enable-libsrt --enable-libxml2 --enable-opengl --enable-libdav1d --enable-cuda-llvm --enable-libsvthevc --enable-libsvtav1 --enable-libaom --enable-libvpx --enable-nvenc --enable-nvdec --extra-libs=-lharfbuzz --extra-libs=-lm --extra-libs=-lpthread --extra-cflags=-DLIBTWOLAME_STATIC --extra-cflags=-DMODPLUG_STATIC --extra-cflags=-DCACA_STATIC --enable-amf --enable-libmfx --enable-gpl --enable-frei0r --enable-librubberband --enable-libvidstab --enable-libx264 --enable-libx265 --enable-avisynth --enable-libaribb24 --enable-libxvid --enable-libdavs2 --enable-libxavs2 --enable-libxavs --extra-cflags='-mtune=generic' --extra-cflags=-O3 --enable-static --disable-shared --prefix=/home/runner/work/ffmpeg-autobuild/ffmpeg-autobuild/ffmpeg-source --enable-nonfree --enable-libfdk-aac --enable-decklink
  libavutil      57.  5.100 / 57.  5.100
  libavcodec     59.  7.103 / 59.  7.103
  libavformat    59.  5.100 / 59.  5.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8.  9.100 /  8.  9.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
[aac @ 000001b5e6f283c0] Estimating duration from bitrate, this may be inaccurate
Input #0, aac, from 'test6000.aac':
  Metadata:
    id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x00\x00\x00<
  Duration: 00:53:06.53, bitrate: 102 kb/s
  Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 102 kb/s
  Stream #0:1: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 576x432 [SAR 72:72 DAR 4:3], 90k tbr, 90k tbn (attached pic)
    Metadata:
      comment         : Movie/video screen capture
Stream mapping:
  Stream #0:0 -> #0:0 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x00\x00\x00<
    encoder         : Lavf59.5.100
  Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
    Metadata:
      encoder         : Lavc59.7.103 pcm_s16le
test6000.aac: Invalid data found when processing input
size=N/A time=00:05:40.35 bitrate=N/A speed=1.1e+03x
video:0kB audio:63816kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

Test file: ​https://cdn.discordapp.com/attachments/807246463808700487/890004176166338560/test6000.aac

[Balling]

Yet ffplay does work... RFC is here ​https://datatracker.ietf.org/doc/html/rfc8216#section-3.4

This is how it looks:

0000000 Id3v2 (73 bytes)
0000000  File Header (10 bytes)
0000000   identifier:                           ID3
0000003   version_major:                        4 (0x04)
0000004   version_revision:                     0 (0x00)
0000005   flags:                                0 (0x00)
0000006   Unsynchronisation:                    No
0000006   Extended header:                      No
0000006   Experimental indicator:               0 (0x0000000000000000)
0000006   Size:                                 63 (0x0000003F) - 63 (0x3F)
000000A  ---------------------------
000000A  ---   Id3v2, accepted   ---
000000A  ---------------------------
000000A  PRIV - Private frame (63 bytes)
000000A   Header (10 bytes)
000000A    Frame ID:                            PRIV
000000E    Size:                                53 (0x00000035) - 53 (0x35) bytes
0000012    Flags:                               0 (0x0000)
0000014    Tag alter preservation:              0 (0x0000000000000000)
0000014    File alter preservation:             0 (0x0000000000000000)
0000014    Read only:                           0 (0x0000000000000000)
0000014    Grouping identity:                   0 (0x0000000000000000)
0000014    Compression:                         0 (0x0000000000000000)
0000014    Encryption:                          0 (0x0000000000000000)
0000014    Unsynchronisation:                   No
0000014    Data length indicator:               No
0000014   Owner identifier:                     com.apple.streaming.transportStreamTimestamp
0000040   Null:                                 0 (0x00)
0000041   DTS:                                  60 (0x000000000000003C)
0000049  --------------------------
0000049  ---   Id3v2, filling   ---
0000049  --------------------------
0000049  ---------------------------
0000049  ---   Id3v2, finished   ---
0000049  ---------------------------
0000049 Id3v2 (25114 bytes)
0000049  File Header (10 bytes)
0000049   identifier:                           ID3
000004C   version_major:                        4 (0x04)
000004D   version_revision:                     0 (0x00)
000004E   flags:                                0 (0x00)
000004F   Unsynchronisation:                    No
000004F   Extended header:                      No
000004F   Experimental indicator:               0 (0x0000000000000000)
000004F   Size:                                 82960 (0x00014410) - 25104 (0x6210)
0000053  ---------------------------
0000053  ---   Id3v2, accepted   ---
0000053  ---------------------------
0000053  APIC - Attached picture - Screen capture (25104 bytes)
0000053   Header (10 bytes)
0000053    Frame ID:                            APIC
0000057    Size:                                82950 (0x00014406) - 25094 (0x6206) bytes
000005B    Flags:                               0 (0x0000)
000005D    Tag alter preservation:              0 (0x0000000000000000)
000005D    File alter preservation:             0 (0x0000000000000000)
000005D    Read only:                           0 (0x0000000000000000)
000005D    Grouping identity:                   0 (0x0000000000000000)
000005D    Compression:                         0 (0x0000000000000000)
000005D    Encryption:                          0 (0x0000000000000000)
000005D    Unsynchronisation:                   No
000005D    Data length indicator:               No
000005D   Text_encoding:                        3 (0x03)
000005E   MIME_type:                            image/jpeg
0000069   Picture_type:                         16 (0x10)
000006A   Description:                          
000006B   Data:                                 (25080 bytes)
0006263  --------------------------
0006263  ---   Id3v2, filling   ---
0006263  --------------------------
0006263  ---------------------------
0006263  ---   Id3v2, finished   ---
0006263  ---------------------------
0006263 adts_frame - 0 (0x0) (263 bytes)
0006263  adts_fixed_header (3 bytes)
0006263   syncword:                             4095 (0xFFF) - (12 bits)
0006264   id:                                   No - MPEG-4
0006264   layer:                                0 (0x0) - (2 bits)
0006264   protection_absent:                    Yes
0006265   profile_ObjectType:                   1 (0x1) - (2 bits) - AAC LC
0006265   sampling_frequency_index:             3 (0x3) - (4 bits) - 48000 (0xBB80) Hz
0006265   private:                              No
0006265   channel_configuration:                2 (0x2) - (3 bits)
0006266   original:                             No
0006266   home:                                 No
0006266  adts_variable_header (4 bytes)
0006266   copyright_id:                         No
0006266   copyright_id_start:                   No
0006266   aac_frame_length:                     263 (0x0107) - (13 bits)
0006268   adts_buffer_fullness:                 2047 (0x7FF) - (11 bits) - VBR
0006269   num_raw_data_blocks:                  0 (0x0) - (2 bits)
000626A  raw_data_block (263 bytes)
0006263   CPE - channel_pair_element (14 bytes)
000626A    id_syn_ele:                          1 (0x1) - (3 bits) - CPE - channel_pair_element
0006266   FIL - fill_element (259 bytes)
0006271    id_syn_ele:                          6 (0x6) - (3 bits) - FIL - fill_element
0006267   END - End (258 bytes)
0006369    id_syn_ele:                          7 (0x7) - (3 bits) - END - End
0006369   byte_alignment:                       0 (0x0) - (1 bits)
000636A adts_frame - 1 (0x1) (263 bytes)
000636A  adts_fixed_header (3 bytes)

[jeeb]

This seems to be a concatenated stream from a set of HLS segments.

A patch: ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210927213133.28258-1-jeebjp@gmail.com/ .

It would be very lovely if someone could figure out what those occasional padding bytes are and if they should even exist when such a raw ADTS stream is concatenated during ripping/dumping. As currently enabling resync to work just means that:

1. The ADTS reader is able to proceed (in other words, parsing of the audio bits of the ADTS stream should work).
2. You are most likely losing all of the following ID3 metadata as the resync will skip until the next valid ADTS start code.

So if the padding is not supposed to be there when the stream is ripped and concatenated, then the ripping application should be fixed to not have this stuff. Otherwise if there is a specification for this stuff, that should be noted so that proper improvements can be made if necessary.

In this sample it looks like this:

[ADTS packet]
[0x0b .... or so padding]
[ID3 magic]

Previously it would fail at the second set of 0x0bs, at offset 5053141 as that is more than the default probesize of 5000000.

[jeeb]

Resync enabled past probesize (and thus this issue fixed) with c20577806f0a161c6867e72f884d020a253de10a .

As for what that padding was, looking at ​https://github.com/yt-dlp/yt-dlp/commit/7687c8ac6e223a725b3ef8f56f04779bebdc86c5 it probably was padding for PKCS#7 encryption. And thus it should not have been included in the resulting data :) .