Crash with -acodec libfdk_aac -f latm

[Carl Eugen Hoyos]

FFmpeg crashes when trying to encode to format latm using libfdk-aac

(gdb) r -i fate-suite/svq3/Vertical400kbit.sorenson3.mov -latm 1 out.latm
Starting program: ffmpeg_g -i fate-suite/svq3/Vertical400kbit.sorenson3.mov -latm 1 out.latm
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ffmpeg version N-46646-g0e239b2 Copyright (c) 2000-2012 the FFmpeg developers
  built on Nov 14 2012 01:08:45 with gcc 4.7 (SUSE Linux)
  configuration: --enable-libfdk-aac --disable-indev=jack
  libavutil      52.  6.100 / 52.  6.100
  libavcodec     54. 71.100 / 54. 71.100
  libavformat    54. 36.100 / 54. 36.100
  libavdevice    54.  3.100 / 54.  3.100
  libavfilter     3. 22.101 /  3. 22.101
  libswscale      2.  1.102 /  2.  1.102
  libswresample   0. 16.100 /  0. 16.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x159d240] max_analyze_duration 5000000 reached at 5000998
Guessed Channel Layout for  Input Stream #0.1 : mono
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'fate-suite/svq3/Vertical400kbit.sorenson3.mov':
  Metadata:
    creation_time   : 2001-03-20 16:17:18
    title           : Vertical Online SV3 Demo
    title-eng       : Vertical Online SV3 Demo
    artist          : Logan Kelsey
    artist-eng      : Logan Kelsey
    copyright       : © Vertical Online 2001
    copyright-eng   : © Vertical Online 2001
    encoder         : Sorenson Video 3
    encoder-eng     : Sorenson Video 3
  Duration: 00:00:43.58, start: 0.000000, bitrate: 580 kb/s
    Stream #0:0(eng): Video: svq3 (SVQ3 / 0x33515653), yuvj420p, 320x240, 391 kb/s, 30.02 fps, 30 tbr, 600 tbn, 600 tbc
    Metadata:
      creation_time   : 2001-03-20 16:17:18
      handler_name    : Apple Alias Data Handler
    Stream #0:1(eng): Audio: adpcm_ima_qt (ima4 / 0x34616D69), 44100 Hz, mono, s16p, 176 kb/s
    Metadata:
      creation_time   : 2001-03-20 16:17:18
      handler_name    : Apple Alias Data Handler
Output #0, latm, to 'out.latm':
  Metadata:
    encoder-eng     : Sorenson Video 3
    title           : Vertical Online SV3 Demo
    title-eng       : Vertical Online SV3 Demo
    artist          : Logan Kelsey
    artist-eng      : Logan Kelsey
    copyright       : © Vertical Online 2001
    copyright-eng   : © Vertical Online 2001
    encoder         : Lavf54.36.100
    Stream #0:0(eng): Audio: aac, 44100 Hz, mono, s16, 96 kb/s
    Metadata:
      creation_time   : 2001-03-20 16:17:18
      handler_name    : Apple Alias Data Handler
Stream mapping:
  Stream #0:1 -> #0:0 (adpcm_ima_qt -> libfdk_aac)
Press [q] to stop, [?] for help
Multiple frames in a packet from stream 1

Program received signal SIGSEGV, Segmentation fault.
0x00000000005996d1 in avpriv_copy_bits (pb=pb@entry=0x7fffffffbf80, src=0x0,
    length=<optimized out>) at libavcodec/bitstream.c:79
79          put_bits(pb, bits, AV_RB16(src + 2*words)>>(16-bits));
(gdb) bt
#0  0x00000000005996d1 in avpriv_copy_bits (pb=pb@entry=0x7fffffffbf80, src=0x0,
    length=<optimized out>) at libavcodec/bitstream.c:79
#1  0x00000000004cb86d in latm_write_frame_header (bs=0x7fffffffbf80, s=<optimized out>)
    at libavformat/latmenc.c:123
#2  latm_write_packet (s=0x15a5000, pkt=0x7fffffffbff0) at libavformat/latmenc.c:164
#3  0x0000000000502358 in av_interleaved_write_frame (s=s@entry=0x15a5000,
    pkt=pkt@entry=0x7fffffffc310) at libavformat/mux.c:736
#4  0x000000000045be25 in write_frame (s=0x15a5000, pkt=0x7fffffffc310, ost=0x159de20)
    at ffmpeg.c:573
#5  0x000000000045d365 in do_audio_out (frame=<optimized out>, ost=<optimized out>,
    s=<optimized out>) at ffmpeg.c:647
#6  reap_filters () at ffmpeg.c:1038
#7  0x000000000044f809 in transcode_step () at ffmpeg.c:2933
#8  transcode () at ffmpeg.c:2976
#9  main (argc=<optimized out>, argv=<optimized out>) at ffmpeg.c:3160
(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0x5996b1 to 0x5996f1:
   0x00000000005996b1 <avpriv_copy_bits+145>:   sub    $0x10,%eax
   0x00000000005996b4 <avpriv_copy_bits+148>:   or     %r9d,%r8d
   0x00000000005996b7 <avpriv_copy_bits+151>:   cmp    %edi,%edx
   0x00000000005996b9 <avpriv_copy_bits+153>:   mov    %eax,0x4(%rbx)
   0x00000000005996bc <avpriv_copy_bits+156>:   mov    %r8d,(%rbx)
   0x00000000005996bf <avpriv_copy_bits+159>:   jg     0x599695 <avpriv_copy_bits+117>
   0x00000000005996c1 <avpriv_copy_bits+161>:   lea    (%rdx,%rdx,1),%r13d
   0x00000000005996c5 <avpriv_copy_bits+165>:   movslq %r13d,%r13
   0x00000000005996c8 <avpriv_copy_bits+168>:   and    $0xf,%r12d
   0x00000000005996cc <avpriv_copy_bits+172>:   mov    $0x10,%ecx
=> 0x00000000005996d1 <avpriv_copy_bits+177>:   movzwl 0x0(%rbp,%r13,1),%edx
   0x00000000005996d7 <avpriv_copy_bits+183>:   sub    %r12d,%ecx
   0x00000000005996da <avpriv_copy_bits+186>:   rol    $0x8,%dx
   0x00000000005996de <avpriv_copy_bits+190>:   movzwl %dx,%edx
   0x00000000005996e1 <avpriv_copy_bits+193>:   sar    %cl,%edx
   0x00000000005996e3 <avpriv_copy_bits+195>:   cmp    %eax,%r12d
   0x00000000005996e6 <avpriv_copy_bits+198>:   jl     0x5997d0 <avpriv_copy_bits+432>
   0x00000000005996ec <avpriv_copy_bits+204>:   mov    %eax,%ecx
   0x00000000005996ee <avpriv_copy_bits+206>:   mov    %edx,%esi
   0x00000000005996f0 <avpriv_copy_bits+208>:   shl    %cl,%r8d
End of assembler dump.
(gdb) info register
rax            0x10     16
rbx            0x7fffffffbf80   140737488338816
rcx            0x10     16
rdx            0x0      0
rsi            0x0      0
rdi            0x7fffffffbf80   140737488338816
rbp            0x0      0x0
rsp            0x7fffffffbf10   0x7fffffffbf10
r8             0x2000   8192
r9             0x0      0
r10            0x117    279
r11            0x7ffff67ed0de   140737328894174
r12            0x3      3
r13            0x0      0
r14            0x7fffffffbff0   140737488338928
r15            0x159ee40        22670912
rip            0x5996d1 0x5996d1 <avpriv_copy_bits+177>
eflags         0x10206  [ PF IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0

[jamal]

Try running "ffmpeg -i fate-suite/svq3/Vertical400kbit.sorenson3.mov -flags global_header -latm 1 out.latm". That prevents the crash for me and creates a seemingly working file.

Unless you set that flag, libfdk-aac does not store some extradata in the AVCodecContext that the latm muxer seems to be expecting unconditionally.

[Carl Eugen Hoyos]

I am not sure I understand: Can you reproduce the crash?

[jamal]

Replying to cehoyos:

I am not sure I understand: Can you reproduce the crash?

Yes, with "ffmpeg -i fate-suite/svq3/Vertical400kbit.sorenson3.mov -latm 1 out.latm" it crashes for me with the same gdb you posted above.

But with "ffmpeg -i fate-suite/svq3/Vertical400kbit.sorenson3.mov -flags global_header -latm 1 out.latm" or even "ffmpeg -i fate-suite/svq3/Vertical400kbit.sorenson3.mov -flags global_header out.latm" it doesn't, and it succeeds in creating a seemingly working file.

As i said, the LATM muxer always expects extradata inside the AVCodecContext, but the libfdk-aac encoder only sends such extradata if the global_header flag is enabled.

[Carl Eugen Hoyos]

Replying to jamal:

Replying to cehoyos:

I am not sure I understand: Can you reproduce the crash?

Yes, with "ffmpeg -i fate-suite/svq3/Vertical400kbit.sorenson3.mov -latm 1 out.latm" it crashes for me with the same gdb you posted above.

Thank you!

But with "ffmpeg -i fate-suite/svq3/Vertical400kbit.sorenson3.mov -flags global_header -latm 1 out.latm" or even "ffmpeg -i fate-suite/svq3/Vertical400kbit.sorenson3.mov -flags global_header out.latm" it doesn't, and it succeeds in creating a seemingly working file.

As i said, the LATM muxer always expects extradata inside the AVCodecContext, but the libfdk-aac encoder only sends such extradata if the global_header flag is enabled.

Good to know.

[Carl Eugen Hoyos]

A patch was sent:
​http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/157319/focus=157429

[Carl Eugen Hoyos]

Fixed by jamal.