ac3 encoder crash with low mem

[ami_stuff]

maybe only reproducable with --disable-asm build

​http://ffmpeg.org/trac/ffmpeg/raw-attachment/ticket/2722/divx.avi

knoppix@Microknoppix:/media/sdb1$ ulimit -Sv 18920
knoppix@Microknoppix:/media/sdb1$ ./ffmpeg4 -i o/divx.avi -y -vcodec msvideo1 out.avi 
ffmpeg version 1.1.git Copyright (c) 2000-2013 the FFmpeg developers 
  built on Jun 28 2013 16:46:26 with gcc 4.7 (Debian 4.7.2-4) 
  configuration: --disable-asm --enable-gpl --disable-ffprobe 
  libavutil      52. 37.101 / 52. 37.101 
  libavcodec     55. 17.100 / 55. 17.100 
  libavformat    55. 10.100 / 55. 10.100 
  libavdevice    55.  2.100 / 55.  2.100 
  libavfilter     3. 77.101 /  3. 77.101 
  libswscale      2.  3.100 /  2.  3.100 
  libswresample   0. 17.102 /  0. 17.102 
  libpostproc    52.  3.100 / 52.  3.100 
[mpeg4 @ 0x905d4e0] Invalid and inefficient vfw-avi packed B frames detected 
Input #0, avi, from 'o/divx.avi': 
  Duration: 00:00:12.64, start: 0.000000, bitrate: 963 kb/s 
    Stream #0:0: Video: mpeg4 (DX50 / 0x30355844), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 23.98 fps, 23.98 tbr, 23.97 tbn, 30k tbc 
    Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 44100 Hz, stereo, s16p, 128 kb/s 
[swscaler @ 0x9050e60] No accelerated colorspace conversion found from yuv420p to rgb555le. 
[swscaler @ 0x9056bf0] No accelerated colorspace conversion found from yuv420p to rgb555le. 
[swscaler @ 0x90790e0] No accelerated colorspace conversion found from yuv420p to rgb555le. 
Segmentation fault 
knoppix@Microknoppix:/media/sdb1$

[Carl Eugen Hoyos]

Is the crash also reproducible with -an?

[ami_stuff]

Replying to cehoyos:

Is the crash also reproducible with -an?

It doesn't crash with -an or -vn alone.

I have compiled --disable-asm and --disable-yasm builds. Only --disable-asm build crashes here.

[Carl Eugen Hoyos]

Replying to ami_stuff:

Replying to cehoyos:

Is the crash also reproducible with -an?

It doesn't crash with -an or -vn alone.

Not even with different ulimit values?

[ami_stuff]

got a bt:

knoppix@Microknoppix:/media/sdb1$ ./ffmpeg_g -i o/divx.avi -y -vcodec msvideo1 out.avi 
ffmpeg version 1.1.git Copyright (c) 2000-2013 the FFmpeg developers 
  built on Jun 30 2013 17:18:20 with gcc 4.7 (Debian 4.7.2-4) 
  configuration: --disable-asm --disable-ffprobe --disable-ffserver --enable-gpl 
  libavutil      52. 37.101 / 52. 37.101 
  libavcodec     55. 17.100 / 55. 17.100 
  libavformat    55. 10.100 / 55. 10.100 
  libavdevice    55.  2.100 / 55.  2.100 
  libavfilter     3. 77.101 /  3. 77.101 
  libswscale      2.  3.100 /  2.  3.100 
  libswresample   0. 17.102 /  0. 17.102 
  libpostproc    52.  3.100 / 52.  3.100 
[mpeg4 @ 0x905d4e0] Invalid and inefficient vfw-avi packed B frames detected 
Input #0, avi, from 'o/divx.avi': 
  Duration: 00:00:12.64, start: 0.000000, bitrate: 963 kb/s 
    Stream #0:0: Video: mpeg4 (DX50 / 0x30355844), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 23.98 fps, 23.98 tbr, 23.97 tbn, 30k tbc 
    Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 44100 Hz, stereo, s16p, 128 kb/s 
[swscaler @ 0x9050e60] No accelerated colorspace conversion found from yuv420p to rgb555le. 
[swscaler @ 0x9056bf0] No accelerated colorspace conversion found from yuv420p to rgb555le. 
[swscaler @ 0x90790e0] No accelerated colorspace conversion found from yuv420p to rgb555le. 
Segmentation fault (core dumped) 
knoppix@Microknoppix:/media/sdb1$ ulimit -c unlimited -Sv 199999000 
knoppix@Microknoppix:/media/sdb1$ ./gdb -c core ffmpeg_g 
warning: Can not parse XML syscalls information; XML support was disabled at compile time. 
GNU gdb (GDB) 7.0.1-debian 
Copyright (C) 2009 Free Software Foundation, Inc. 
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> 
This is free software: you are free to change and redistribute it. 
There is NO WARRANTY, to the extent permitted by law.  Type "show copying" 
and "show warranty" for details. 
This GDB was configured as "i486-linux-gnu". 
For bug reporting instructions, please see: 
<http://www.gnu.org/software/gdb/bugs/>... 
Reading symbols from /media/sdb1/ffmpeg_g...done. 

warning: Can't read pathname for load map: Input/output error. 
Reading symbols from /lib/i386-linux-gnu/libm.so.6...(no debugging symbols found)...done. 
Loaded symbols for /lib/i386-linux-gnu/libm.so.6 
Reading symbols from /lib/i386-linux-gnu/librt.so.1...(no debugging symbols found)...done. 
Loaded symbols for /lib/i386-linux-gnu/librt.so.1 
Reading symbols from /lib/i386-linux-gnu/libpthread.so.0...(no debugging symbols found)...done. 
Loaded symbols for /lib/i386-linux-gnu/libpthread.so.0 
Reading symbols from /lib/i386-linux-gnu/libc.so.6...(no debugging symbols found)...done. 
Loaded symbols for /lib/i386-linux-gnu/libc.so.6 
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. 
Loaded symbols for /lib/ld-linux.so.2 
Failed to read a valid object file image from memory. 
Core was generated by `./ffmpeg_g -i o/divx.avi -y -vcodec msvideo1 out.avi'. 
Program terminated with signal 11, Segmentation fault. 
#0  av_free (arg=0x0) at libavutil/mem.c:210 
210	    free(ptr); 
(gdb) bt 
#0  av_free (arg=0x0) at libavutil/mem.c:210 
#1  av_freep (arg=0x0) at libavutil/mem.c:217 
#2  0x0808d386 in ff_ac3_encode_close (avctx=0x906e480) 
    at libavcodec/ac3enc.c:2021 
#3  0x0808e46e in ff_ac3_encode_init (avctx=0x906e480) 
    at libavcodec/ac3enc.c:2488 
#4  0x08658dd9 in avcodec_open2 (avctx=0x906e480, codec=0x8a65a00, 
    options=0x906e998) at libavcodec/utils.c:1309 
#5  0x080b3478 in transcode_init () at ffmpeg.c:2439 
#6  0x0809d120 in transcode (argc=<value optimized out>, argv=0xbfe1b534) 
    at ffmpeg.c:3183 
#7  main (argc=<value optimized out>, argv=0xbfe1b534) at ffmpeg.c:3390 
(gdb) 

[Carl Eugen Hoyos]

I believe I have fixed this crash in 225f78b - I saw an additional output line here before the crash ("Cannot allocate memory.") and therefore suspected a different problem.
Please reopen if this is still reproducible.