Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#5973 closed defect (fixed)

Regression: aac_adtstoasc bitstream filter leads to malformed AAC stream

Reported by: zmwangx Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: aac_adtstoasc regression
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: yes

Description

Summary of the bug

There's a regression in FFmpeg 3.2 where applying the aac_adtstoasc bitstream filter could result in malformed AAC streams (with extra data). Specifically, when decoding the output streams, we get errors like

[aac @ 0x7fe0af012600] Number of scalefactor bands in group (45) exceeds limit (41).

The regression is caused by commit 5ef1959, "ffmpeg: switch to the new BSF API".

How to reproduce

% ffmpeg -v 9 -loglevel 99 -y -i aac_adtstoasc_bsf_breaks_aac_stream.m4a -c copy -bsf:a aac_adtstoasc filtered.m4a
ffmpeg version 3.2 Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.38)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-frei0r --enable-libass --enable-libcaca --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-opencl --enable-openssl --disable-lzma --enable-libopenjpeg --disable-decoder=jpeg2000 --extra-cflags=-I/usr/local/Cellar/openjpeg/2.1.2/include/openjpeg-2.1 --enable-nonfree --enable-vda
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.100 / 57. 64.100
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-i' ... matched as input file with argument 'aac_adtstoasc_bsf_breaks_aac_stream.m4a'.
Reading option '-c' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-bsf:a' ... matched as option 'bsf' (A comma-separated list of bitstream filters) with argument 'aac_adtstoasc'.
Reading option 'filtered.m4a' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Applying option loglevel (set logging level) with argument 99.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input file aac_adtstoasc_bsf_breaks_aac_stream.m4a.
Successfully parsed a group of options.
Opening an input file: aac_adtstoasc_bsf_breaks_aac_stream.m4a.
[file @ 0x7fa757600000] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fa759801000] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fa759801000] ISO: File Type Major Brand: M4A 
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fa759801000] Unknown dref type 0x08206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fa759801000] Processing st: 0, edit list 0 - media time: 0, duration: 10272
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fa759801000] Before avformat_find_stream_info() pos: 2499 bytes read:2499 seeks:0 nb_streams:1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fa759801000] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fa759801000] After avformat_find_stream_info() pos: 78 bytes read:2499 seeks:0 frames:1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'aac_adtstoasc_bsf_breaks_aac_stream.m4a':
  Metadata:
    major_brand     : M4A 
    minor_version   : 512
    compatible_brands: isomiso2
    encoder         : Lavf57.56.100
  Duration: 00:00:00.21, start: 0.000000, bitrate: 93 kb/s
    Stream #0:0(und), 1, 1/48000: Audio: aac (HE-AAC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 65 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Successfully opened the file.
Parsing a group of options: output file filtered.m4a.
Applying option c (codec name) with argument copy.
Applying option bsf:a (A comma-separated list of bitstream filters) with argument aac_adtstoasc.
Successfully parsed a group of options.
Opening an output file: filtered.m4a.
[file @ 0x7fa757405f60] Setting default whitelist 'file,crypto'
Successfully opened the file.
Output #0, ipod, to 'filtered.m4a':
  Metadata:
    major_brand     : M4A 
    minor_version   : 512
    compatible_brands: isomiso2
    encoder         : Lavf57.56.100
    Stream #0:0(und), 0, 1/48000: Audio: aac (HE-AAC) (mp4a / 0x6134706D), 48000 Hz, stereo, 65 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
No more output streams to write to, finishing.
size=       2kB time=00:00:00.17 bitrate= 117.0kbits/s speed= 749x    
video:0kB audio:2kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 43.721199%
Input file #0 (aac_adtstoasc_bsf_breaks_aac_stream.m4a):
  Input stream #0:0 (audio): 5 packets read (1736 bytes); 
  Total: 5 packets (1736 bytes) demuxed
Output file #0 (filtered.m4a):
  Output stream #0:0 (audio): 5 packets muxed (1736 bytes); 
  Total: 5 packets (1736 bytes) muxed
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x7fa757406080] Statistics: 30 seeks, 28 writeouts
[AVIOContext @ 0x7fa7576000c0] Statistics: 2499 bytes read, 0 seeks

The output stream is malformed, as revealed by ffprobe:

% ffprobe -v 9 -loglevel 99 -show_frames filtered.m4a
ffprobe version 3.2 Copyright (c) 2007-2016 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.38)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-frei0r --enable-libass --enable-libcaca --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-opencl --enable-openssl --disable-lzma --enable-libopenjpeg --disable-decoder=jpeg2000 --extra-cflags=-I/usr/local/Cellar/openjpeg/2.1.2/include/openjpeg-2.1 --enable-nonfree --enable-vda
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.100 / 57. 64.100
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[file @ 0x7fe0ae600320] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe0af003a00] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe0af003a00] ISO: File Type Major Brand: M4A 
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe0af003a00] Unknown dref type 0x08206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe0af003a00] Processing st: 0, edit list 0 - media time: 0, duration: 10272
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe0af003a00] Before avformat_find_stream_info() pos: 2495 bytes read:2495 seeks:0 nb_streams:1
[aac @ 0x7fe0af004600] noise_facs_q 254 is invalid
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe0af003a00] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe0af003a00] After avformat_find_stream_info() pos: 78 bytes read:2495 seeks:0 frames:1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'filtered.m4a':
  Metadata:
    major_brand     : M4A 
    minor_version   : 512
    compatible_brands: isomiso2
    encoder         : Lavf57.56.100
  Duration: 00:00:00.21, start: 0.000000, bitrate: 93 kb/s
    Stream #0:0(und), 1, 1/48000: Audio: aac (HE-AAC) (mp4a / 0x6134706D), 96000 Hz, stereo, fltp, 64 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Processing read interval id:0 start:N/A end:N/A
[aac @ 0x7fe0af012600] Number of scalefactor bands in group (45) exceeds limit (41).
[aac @ 0x7fe0af012600] Number of scalefactor bands in group (45) exceeds limit (41).
[aac @ 0x7fe0af012600] Number of scalefactor bands in group (45) exceeds limit (41).
[aac @ 0x7fe0af012600] Number of scalefactor bands in group (45) exceeds limit (41).
[aac @ 0x7fe0af012600] Number of scalefactor bands in group (14) exceeds limit (12).
[AVIOContext @ 0x7fe0ae40f340] Statistics: 2495 bytes read, 0 seeks

The same command line produces valid AAC output with FFmpeg 3.1.5.

Attachments (1)

aac_adtstoasc_bsf_breaks_aac_stream.m4a (2.4 KB ) - added by zmwangx 7 years ago.

Download all attachments as: .zip

Change History (9)

comment:1 by zmwangx, 7 years ago

By the way, the attached logs are from FFmpeg 3.2, but the behavior is exactly the same with current git head.

in reply to:  1 ; comment:2 by Carl Eugen Hoyos, 7 years ago

Keywords: regression added
Priority: normalimportant
Reproduced by developer: set
Status: newopen

Replying to zmwangx:

By the way, the attached logs are from FFmpeg 3.2, but the behavior is exactly the same with current git head.

Then why did you provide console output for an old version?

in reply to:  2 comment:3 by zmwangx, 7 years ago

Replying to cehoyos:

Then why did you provide console output for an old version?

Sorry, no particular reason, they're the same — the problematic part of ffmpeg.c hasn't changed since 5ef1959, and I forgot to use output from the head build when creating the ticket. Here they are, if you insist:

% ffmpeg -v 9 -loglevel 99 -y -i aac_adtstoasc_bsf_breaks_aac_stream.m4a -c copy -bsf:a aac_adtstoasc filtered.m4a
ffmpeg version N-82595-gf242e0a Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: 
  libavutil      55. 40.100 / 55. 40.100
  libavcodec     57. 66.106 / 57. 66.106
  libavformat    57. 58.100 / 57. 58.100
  libavdevice    57.  2.100 / 57.  2.100
  libavfilter     6. 67.100 /  6. 67.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-i' ... matched as input file with argument 'aac_adtstoasc_bsf_breaks_aac_stream.m4a'.
Reading option '-c' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-bsf:a' ... matched as option 'bsf' (A comma-separated list of bitstream filters) with argument 'aac_adtstoasc'.
Reading option 'filtered.m4a' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Applying option loglevel (set logging level) with argument 99.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input file aac_adtstoasc_bsf_breaks_aac_stream.m4a.
Successfully parsed a group of options.
Opening an input file: aac_adtstoasc_bsf_breaks_aac_stream.m4a.
[file @ 0x7f89ca700380] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f89cb800000] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f89cb800000] ISO: File Type Major Brand: M4A 
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f89cb800000] Unknown dref type 0x08206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f89cb800000] Processing st: 0, edit list 0 - media time: 0, duration: 10272
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f89cb800000] Before avformat_find_stream_info() pos: 2499 bytes read:2499 seeks:0 nb_streams:1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f89cb800000] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f89cb800000] After avformat_find_stream_info() pos: 78 bytes read:2499 seeks:0 frames:1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'aac_adtstoasc_bsf_breaks_aac_stream.m4a':
  Metadata:
    major_brand     : M4A 
    minor_version   : 512
    compatible_brands: isomiso2
    encoder         : Lavf57.56.100
  Duration: 00:00:00.21, start: 0.000000, bitrate: 93 kb/s
    Stream #0:0(und), 1, 1/48000: Audio: aac (HE-AAC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 65 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Successfully opened the file.
Parsing a group of options: output file filtered.m4a.
Applying option c (codec name) with argument copy.
Applying option bsf:a (A comma-separated list of bitstream filters) with argument aac_adtstoasc.
Successfully parsed a group of options.
Opening an output file: filtered.m4a.
[file @ 0x7f89ca40fe20] Setting default whitelist 'file,crypto'
Successfully opened the file.
Output #0, ipod, to 'filtered.m4a':
  Metadata:
    major_brand     : M4A 
    minor_version   : 512
    compatible_brands: isomiso2
    encoder         : Lavf57.58.100
    Stream #0:0(und), 0, 1/48000: Audio: aac (HE-AAC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 65 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
No more output streams to write to, finishing.
size=       2kB time=00:00:00.17 bitrate= 117.0kbits/s speed=34.9x    
video:0kB audio:2kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 43.721199%
Input file #0 (aac_adtstoasc_bsf_breaks_aac_stream.m4a):
  Input stream #0:0 (audio): 5 packets read (1736 bytes); 
  Total: 5 packets (1736 bytes) demuxed
Output file #0 (filtered.m4a):
  Output stream #0:0 (audio): 5 packets muxed (1736 bytes); 
  Total: 5 packets (1736 bytes) muxed
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x7f89ca40ff60] Statistics: 30 seeks, 28 writeouts
[AVIOContext @ 0x7f89ca7004c0] Statistics: 2499 bytes read, 0 seeks
% ffprobe -v 9 -loglevel 99 -show_frames filtered.m4a
ffprobe version N-82595-gf242e0a Copyright (c) 2007-2016 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: 
  libavutil      55. 40.100 / 55. 40.100
  libavcodec     57. 66.106 / 57. 66.106
  libavformat    57. 58.100 / 57. 58.100
  libavdevice    57.  2.100 / 57.  2.100
  libavfilter     6. 67.100 /  6. 67.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
[file @ 0x7f8070d00260] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f8071806600] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f8071806600] ISO: File Type Major Brand: M4A 
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f8071806600] Unknown dref type 0x08206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f8071806600] Processing st: 0, edit list 0 - media time: 0, duration: 10272
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f8071806600] Before avformat_find_stream_info() pos: 2495 bytes read:2495 seeks:0 nb_streams:1
[aac @ 0x7f8072000000] noise_facs_q 254 is invalid
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f8071806600] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f8071806600] After avformat_find_stream_info() pos: 78 bytes read:2495 seeks:0 frames:1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'filtered.m4a':
  Metadata:
    major_brand     : M4A 
    minor_version   : 512
    compatible_brands: isomiso2
    encoder         : Lavf57.58.100
  Duration: 00:00:00.21, start: 0.000000, bitrate: 93 kb/s
    Stream #0:0(und), 1, 1/48000: Audio: aac (HE-AAC) (mp4a / 0x6134706D), 96000 Hz, stereo, fltp, 64 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Processing read interval id:0 start:N/A end:N/A
[aac @ 0x7f8071001000] Number of scalefactor bands in group (45) exceeds limit (41).
[aac @ 0x7f8071001000] Number of scalefactor bands in group (45) exceeds limit (41).
[aac @ 0x7f8071001000] Number of scalefactor bands in group (45) exceeds limit (41).
[aac @ 0x7f8071001000] Number of scalefactor bands in group (45) exceeds limit (41).
[aac @ 0x7f8071001000] Number of scalefactor bands in group (14) exceeds limit (12).
[AVIOContext @ 0x7f8070d00360] Statistics: 2495 bytes read, 0 seeks
Last edited 7 years ago by zmwangx (previous) (diff)

comment:4 by James, 7 years ago

Analyzed by developer: set

This file doesn't even have adts encapsulated aac audio. Why are you trying to use the adtstoasc bsf? Just do "ffmpeg -y -i aac_adtstoasc_bsf_breaks_aac_stream.m4a -c copy filtered.m4a" and it will work.

In any case, looks like the old bsf API did some extradata copy that casually recovered the output extradata to its state before the av_bsf_init() call, something that was evidently convenient with this bsf for streams that were not ADTS encapsulated to begin with.
Since this bsf seems to have been written with the intention of passing the packet as is in case no adts header is found, i guess this can be considered a regression.

in reply to:  4 comment:5 by zmwangx, 7 years ago

Replying to jamrial:

This file doesn't even have adts encapsulated aac audio. Why are you trying to use the adtstoasc bsf?

Yes I know that. The reason for using the filter is that we're calling FFmpeg programmatically from you-get, where the data streams come from various sources, some of which are ADTS encapsulated. It's difficult for us to tell beforehand, so the filter is applied unconditionally, which has been working fine until FFmpeg 3.2.

comment:6 by James, 7 years ago

Resolution: fixed
Status: openclosed

Should be fixed in 6e1902bab4349a79c45807af18ebf5b50f7b436b

Will backport it to the 3.2 branch soon.

comment:7 by James, 7 years ago

Backported to 3.2 branch as commit c269c43a83166003ab6649263bc60634a6b7866f which will be included in ffmpeg 3.2.1

comment:8 by Carl Eugen Hoyos, 7 years ago

Component: ffmpegavcodec
Keywords: aac_adtstoasc added
Note: See TracTickets for help on using tickets.