Opened 2 years ago

Closed 17 months ago

#5162 closed defect (wontfix)

aac encoder adds extra frame(s)

Reported by: slhck Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: aac regression
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug: The aac encoder adds one extra audio frame when encoding

How to reproduce:

ffmpeg -i out.mp4 -c:v copy -c:a aac out2.mp4
ffmpeg version N-78077-g5989add-tessus Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
  configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --as=yasm --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libass --enable-libbluray --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzmq --enable-version3 --disable-ffplay --disable-indev=qtkit --disable-indev=x11grab_xcb
  libavutil      55. 13.100 / 55. 13.100
  libavcodec     57. 22.100 / 57. 22.100
  libavformat    57. 21.101 / 57. 21.101
  %                                                                                                                                                       !3491
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 25.100 /  6. 25.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
  Duration: 00:12:53.97, start: 0.000000, bitrate: 2180 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 1857 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[mp4 @ 0x7f8f8b842a00] Codec for stream 0 does not use global headers but container format requires global headers
Output #0, mp4, to 'out2.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.21.101
    Stream #0:0(eng): Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1920x1080, q=2-31, 1857 kb/s, 25 fps, 25 tbr, 12800 tbn, 12800 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: aac (LC) ([64][0][0][0] / 0x0040), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc57.22.100 aac
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
frame=19348 fps=353 q=-1.0 Lsize=  188726kB time=00:12:53.97 bitrate=1997.5kbits/s speed=14.1x
video:175481kB audio:12669kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.306213%
[aac @ 0x7f8f8b806600] Qavg: 691.497

The input file:

ffprobe -i out.mp4 -show_streams | grep nb_frames
…
nb_frames=36280

The output file has one extra (audio) frame:

ffprobe -i out2.mp4 -show_streams | grep nb_frames
…
nb_frames=36281

This means that if you re-encode a file multiple times, audio and video will become more and more asynchronous.

The same bug affects libfdk_aac as well, although it adds two frames. When I add -shortest, I get one frame less than the original file.

Change History (13)

comment:1 Changed 2 years ago by cehoyos

  • Keywords regression added
  • Reproduced by developer set
  • Status changed from new to open

Regression since 151ecc2a / 89eea6df

I can reproduce the issue with the following command lines:

$ ffmpeg -f lavfi -i sine=d=1 out.aac
$ ffmpeg -i out.aac out2.aac
$ ffmpeg -i out2.aac -acodec copy -map 0 -f image2 out%2d

comment:2 follow-up: Changed 2 years ago by heleppkes

This is normal and expected behavior of any AAC encoder. It produces "preroll" data that the decoder is supposed to skip.

comment:3 in reply to: ↑ 2 Changed 2 years ago by slhck

Replying to heleppkes:

This is normal and expected behavior of any AAC encoder. It produces "preroll" data that the decoder is supposed to skip.

How does the decoder know it's pre-roll data? Like I mentioned, if you re-encode the file multiple times it's not skipped, but added up.

comment:4 Changed 2 years ago by heleppkes

The encoder sets avctx->initial_padding, and the muxers are supposed to offset the timestamps in the file appropriately, so while yes it may get longer, it should not cause sync issues.

If this is a bug, its not a bug in aacenc.

comment:5 Changed 2 years ago by cehoyos

How is the adts muxer supposed to set preroll in the output file?

comment:6 Changed 2 years ago by heleppkes

raw files don't have timestamps, so all bets are off there.

comment:7 Changed 2 years ago by heleppkes

To be precise, the encoder sets an initial_padding and produces one frame with negative timestamps. *ideally* the decoder would skip these initial samples after decoding, however unless a container has a way to signal how much padding exists... it wouldnt know how much.

comment:8 Changed 2 years ago by cehoyos

Isn't it a bug that ffmpeg -i input.aac output.aac makes the output file longer (and longer). Or is this a property of aac?

comment:9 Changed 2 years ago by heleppkes

Its not a bug since raw aac files just don't have any way to signal how much data is to be skipped. Can't be fixed.

So yes, its a property of aac that it needs preroll data. Advanced containers could have ways to signal how much preroll exists and allow skipping it, but raw adts files of course don't have such metadata.

Last edited 2 years ago by heleppkes (previous) (diff)

comment:10 Changed 2 years ago by slhck

MP4 is not an advanced container then…?

Not sure how to work around this, I'm afraid.

comment:11 Changed 2 years ago by heleppkes

Just adding one more audio frame doesn't really harm, because as I said it'll offset the timestamps and maintain A/V sync - and you shouldn't be re-encoding in a loop to add more and more frames in any case. ;)

comment:12 Changed 2 years ago by slhck

I have a client who says they're experiencing a problem when encoding multiple times — I know it shouldn't be done for quality reasons, but the A/V sync is still lost.

comment:13 Changed 17 months ago by richardpl

  • Resolution set to wontfix
  • Status changed from open to closed
Note: See TracTickets for help on using tickets.