FFMPEG Audio decoding fails on audio packet (works using libfdk)

[ray]

Summary of the bug:

We use the FFMPEG default aac encoder to produce a 320k HLS stream:

ffmpeg -i graves-master-2.mp3 -ar 48000 -b:a 320k -hls_time 6 -hls_list_size 0 -hls_base_url segments/ -hls_segment_filename segments/segment%03d.ts -vn out.m3u8

On several occasions, we have observed behavior in Chrome where audio stops playing (PIPELINE DECODE ERROR) e.g.
(1) Visit ​https://audius.co/wearegraves/digital-mirage-full-set-60702
(2) Scrub to 13:20
(3) At 13:24, the audio stops

When inspecting the segment that causes issues, we observe error in spectral data, ESC overflow. This same behavior *does not* occur when using the libfdk aac.

How to reproduce:

ffmpeg -i ~/Downloads/QmUR6MeguuuvstqeK5q4q2EoroWQiY5d9bt1QqrzMEpQyd -vn -f wav -y /dev/null

I see

Input #0, mpegts, from '/Users/raymondjacobson/Downloads/QmUR6MeguuuvstqeK5q4q2EoroWQiY5d9bt1QqrzMEpQyd':
  Duration: 00:00:05.97, start: 805.410667, bitrate: 348 kb/s
  Program 1
    Metadata:
      service_name    : ekali set 9 shorter
      service_provider: FFmpeg
    Stream #0:0[0x100]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 319 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Output #0, wav, to '/dev/null':
  Metadata:
    ISFT            : Lavf58.42.100
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
    Metadata:
      encoder         : Lavc58.80.100 pcm_s16le
[aac @ 0x7fb0d000c400] error in spectral data, ESC overflow
Error while decoding stream #0:0: Invalid data found when processing input
size=    1120kB time=00:00:05.99 bitrate=1530.6kbits/s speed= 504x
video:0kB audio:1120kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.006801%

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

Attached is the segment that causes issues.
Here is a link to the "master" audio file: ​https://drive.google.com/file/d/1FziM3tAXvRzjA8XmxGUEan4Z0dtEPYez/view?usp=sharing

I was sent to the FFMPEG team from the Chromium team as they don't take ownership for this issue despite Firefox and Safari being able to handle the output correctly.

​https://bugs.chromium.org/p/chromium/issues/detail?id=1071899

[ray]

Segment causing issues

[Carl Eugen Hoyos]

Do you want to report an encoding or a decoding issue?
If you want to report an encoding issue, please provide the encoding command line that you tested together with the complete, uncut console output and an input file that allows to reproduce the issue to make this a valid ticket.
If you want to report a decoding issue: Is there an issue with the output wav file or are you reporting that an error is shown?

[ray]

This is an encoding issue.

Here's the source file
​https://drive.google.com/file/d/1FziM3tAXvRzjA8XmxGUEan4Z0dtEPYez/view?usp=sharing

Here's the command

ffmpeg -i graves-master-2.mp3 -ar 48000 -b:a 320k -hls_time 6 -hls_list_size 0 -hls_base_url segments/ -hls_segment_filename segments/segment%03d.ts -vn out.m3u8

Then run this:

ffmpeg -i segments/segment134.ts -vn -f wav -y /dev/null

[Carl Eugen Hoyos]

If you want to report an encoding issue, please provide the encoding command line that you tested together with the complete, uncut console output and an input file that allows to reproduce the issue to make this a valid ticket.

[ray]

Thanks for the reply!

Attached are the full logs from encoding and from probing.

Source file
​https://drive.google.com/file/d/1FziM3tAXvRzjA8XmxGUEan4Z0dtEPYez/view?usp=sharing

Encoding command:

mkdir segments
ffmpeg -i graves-master-2.mp3 -ar 48000 -b:a 320k -hls_time 6 -hls_list_size 0 -hls_base_url segments/ -hls_segment_filename segments/segment%03d.ts -vn out.m3u8

OUTPUT: encode.txt

Probing command:

ffmpeg -i segments/segment134.ts -vn -f wav -y /dev/null

OUTPUT: probe.txt

[ray]

You can see the error in the probe.txt output:

[aac @ 0x7fe73b802000] error in spectral data, ESC overflow
Error while decoding stream #0:0: Invalid data found when processing input
size= 1120kB time=00:00:05.99 bitrate=1530.6kbits/s speed= 503x
video:0kB audio:1120kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.006801%

[ricidina]

We seem to have encountered the same bug. We upgraded from ffmpeg 3.3 in Debian Stretch to a Docker container running Alpine Linux and ffmpeg 4.3. So far only one video that we know of has been affected thankfully.

I have tested this with ffmpeg 3.3, 3.4, 4.0, 4.1, and 4.3. It seems the bug is present ONLY in 4.0+ using the built-in aac.

Workaround is to use ffmpeg < 4.0 or libfdk_aac.

Here is an example command line we are using:

/usr/bin/env ffmpeg -i /app/scratch/tomars_theme_song_remix.mp4 -y -c:v libx264 -s 640x360 -preset veryfast -crf 21 -maxrate 2000000 -bufsize 2500000 -pix_fmt yuv420p -movflags +faststart -c:a aac -b:a 384000 -profile:v baseline /app/scratch/tomars_theme_song_remix.360p.mp4

Here is the build info:

ffmpeg version 3.4.8 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9.3.0 (Alpine 9.3.0)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libbluray --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100

ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9.3.0 (Alpine 9.3.0)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --
enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libbluray --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-libsrt --enable-libaribb24 --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100 

​Original video.

​Video encoded with 4.3.1. This freezes at around the 16 second mark in Chrome.

ffmpeg reports:

ffmpeg -i ffmpeg_bug/tomars_theme_song_remix.encoded_with_ffmpeg_4_3.mp4 -f null -

...
[aac @ 0x560f2f6dfe00] error in spectral data, ESC overflow
Error while decoding stream #0:1: Invalid data found when processing input

​Video encoded with 3.4.8. No issue with this one.

​Full encode log for 3.4.

​Full encode log for 4.3.

These logs are the full debug log for the project so contain ffprobe output and other stuff as well.

Let me know if I can provide anything else.

[Balling]

Real master is not a low quality mp3, it is bitrate: 348 kb/s aac.

youtube-dl cannot even download it, but yt-dlp can only mp3 (BTW, ​https://github.com/yt-dlp/yt-dlp/issues/40)

Apparently the sampels have such a not strange name because they are on DHT network (IFPS, not Mainline Kademlia that is).

​https://creatornode.audius.co/ipfs/QmVqz9qkTuWLJpmW5iNQooiqmoaoEUwmGWJXG8Kd2W4ZcE/150x150.jpg

This is nuts. libfdk_aac also does not like the file so I think it should be closed (libfdk_aac does not complain on other segments but does on that video from wikimedia). Also, this can be a bug in segmentation so how is it our fault?

ANYWAY, [libfdk_aac @ 000001bf9e33e880] aacDecoder_DecodeFrame() failed: 400c

have observed behavior in Chrome where audio stops playing

It does not stop playing forever, just for 0.3 seconds or so.

In fact in git-master aac encoder no longer is buggy with (though 384000 does not appear to apply) ffmpeg -i /app/scratch/tomars_theme_song_remix.mp4 -y -c:v libx264 -s 640x360 -preset veryfast -crf 21 -maxrate 2000000 -bufsize 2500000 -pix_fmt yuv420p -movflags +faststart -c:a aac -b:a 384000 -profile:v baseline /app/scratch/tomars_theme_song_remix.360p.mp4