Opened 13 years ago

Closed 13 years ago

#361 closed defect (invalid)

avcodec_decode_audio3 SEGFAULT while decoding certain codec types

Reported by: Danijel Owned by:
Priority: important Component: avcodec
Version: git Keywords: avcodec decode segfault float int conversion
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no


I am using libavcodec as a dynamically linked library under Linux loaded at runtime to decode audio from various codec/format combinations into raw 16-bit slinear 16kHz audio signal.

I downloaded the latest ffmpeg from git today and compiled it using the following configuration:

./configure --pkg-config=pkg-config --enable-shared --disable-static --enable-memalign-hack --enable-runtime-cpudetect --enable-debug=3 --disable-optimizations --disable-stripping

However, the same error happens with stock debian (wheezy) version of the library!

The versions I am using:

I try and decode a WMA encoded file like so:

Converting: input/test.asf.wav => output/test.asf.wav.raw
Format: WAV format
Codec: Windows Media Audio 2

And gdb displays the crash like so:

Program received signal SIGSEGV, Segmentation fault.
0xb7598400 in float_to_int16_interleave_sse2 (dst=0xb6f8b008, src=0xbffff3c8,
    len=2048, channels=2) at libavcodec/x86/fmtconvert_mmx.c:219
(gdb) bt
#0  0xb7598400 in float_to_int16_interleave_sse2 (dst=0xb6f8b008,
    src=0xbffff3c8, len=2048, channels=2)
    at libavcodec/x86/fmtconvert_mmx.c:219
#1  0xb755108d in wma_decode_frame (s=0xb6f34020, samples=0xb6f8b008)
    at libavcodec/wmadec.c:800
#2  0xb75513eb in wma_decode_superframe (avctx=0x805fb40, data=0xb6f8b008,
    data_size=0xbffff4b0, avpkt=0x8058118) at libavcodec/wmadec.c:887
#3  0xb74fc302 in avcodec_decode_audio3 (avctx=0x805fb40, samples=0xb6f8b008,
    frame_size_ptr=0xbffff4b0, avpkt=0x8058118) at libavcodec/utils.c:822
#4  0x08050e58 in FFmpegConverter::process (this=0x80580a8)
    at src/ffmpeg_converter.cpp:171
#5  0x0804ead9 in ConverterThread::Execute (this=0xbffff6bc)
    at src/converter_thread.cpp:91
#6  0x08052ddc in main (argc=2, argv=0xbffff7d4) at src/main.cpp:57

Curiously, the ffmpeg program doesn't seem to be affected by this bug, so maybe I am doing something wrong with the library in my program? In any case, this exact same program works fine when using the library on my Windows machine. The library is cross-compiled under Linux using mingw. I'm guessing it has something to do with CPU detection?

Here is cpuinfo on my Linux machine:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Pentium(R) Dual-Core  CPU      E5200  @ 2.50GHz
stepping        : 10
cpu MHz         : 2499.623
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arc h_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm xsave lahf_lm dts
bogomips        : 4999.24
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
(... and the rest of the file is similar as above ...)

Change History (2)

comment:1 by Danijel, 13 years ago

Ok, I found and fixed the problem. It was also fixed on an external forum:

The reason was a note in the documentation (note the bold):

You might have to align the input buffer avpkt->data and output buffer samples. The alignment requirements depend on the CPU: On some CPUs it isn't necessary at all, on others it won't work at all if not aligned and on others it will work but it will have an impact on performance.

Since I used packets decoded by the library I thought all is fine, but somehow missed that the output buffer also has to be aligned.

The error is kind of strange, since all the values are correct (all the arrays can be debugged and analyzed by GDB), but the system still throws out a SEGFAULT. It turns out that an SSE2 assembler command "MOVDQA" throws out a general protection fault if the source and destination is not aligned properly.

To fix he problem, it's enough to declare the output buffer correctly:


Or alternatively use av_malloc as described here:

And then, you can use the "decode_audio" method without any issues, like this:


I would also like to add that the "audio_decode_example" in libavcodec/api-example.c is incorrect in this fashion and should be updated. Just to make things clear, this code will work on some machines and compilers without any errors. Namely, if compiled using MinGW, it works fine, but under GCC it will bug out, since GCC doesn't automatically align its memory and MinGW does (source:

comment:2 by Carl Eugen Hoyos, 13 years ago

Resolution: invalid
Status: newclosed

Thank you for the explanation!

Note: See TracTickets for help on using tickets.