Opened 12 years ago

Closed 12 years ago

#1453 closed defect (invalid)

Segfault when decoding H264 video

Reported by: Kyle Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: h264 regression
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug: I'm still trying to track it down which git commit started it, but I now get a segfault whenever I decode h264 video. If I go back to the merge at commit c7b9eab2be7099b0d4f2fed4feaf69a7dda379f0 I no longer have the issue.

ffmpeg -i rtsp://(hidden)
ffmpeg version N-41634-gc7bdfbe Copyright (c) 2000-2012 the FFmpeg developers
  built on Jun 16 2012 02:58:25 with gcc 4.6.3
  configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-nonfree --enable-shared --enable-postproc --enable-libx264 --enable-frei0r --enable-librtmp --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libdc1394 --enable-libmp3lame --enable-libtheora --enable-libopenjpeg --enable-libvpx --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libvorbis --enable-libxvid --enable-libfaac --cpu=amdfam10 --arch=x86_64 --enable-x11grab --enable-libxavs --enable-libfreetype --e  libavutil      51. 58.100 / 51. 58.100
  libavcodec     54. 25.100 / 54. 25.100
  libavformat    54.  6.101 / 54.  6.101
  libavdevice    54.  0.100 / 54.  0.100
  libavfilter     2. 80.100 /  2. 80.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
Segmentation fault (core dumped)

gdb backtrace:

run -i rtsp://(hidden)
Starting program: /usr/src/ffmpeg/ffmpeg_g -i rtsp://(hidden)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
ffmpeg version N-41634-gc7bdfbe Copyright (c) 2000-2012 the FFmpeg developers
  built on Jun 16 2012 02:58:25 with gcc 4.6.3
  configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-nonfree --enable-shared --enable-postproc --enable-libx264 --enable-frei0r --enable-librtmp --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libdc1394 --enable-libmp3lame --enable-libtheora --enable-libopenjpeg --enable-libvpx --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libvorbis --enable-libxvid --enable-libfaac --cpu=amdfam10 --arch=x86_64 --enable-x11grab --enable-libxavs --enable-libfreetype --e  libavutil      51. 58.100 / 51. 58.100
  libavcodec     54. 25.100 / 54. 25.100
  libavformat    54.  6.101 / 54.  6.101
  libavdevice    54.  0.100 / 54.  0.100
  libavfilter     2. 80.100 /  2. 80.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff68e98b0 in ?? () from /usr/lib/libavcodec.so.54
(gdb) bt
#0  0x00007ffff68e98b0 in ?? () from /usr/lib/libavcodec.so.54
#1  0x00007ffff692f449 in ?? () from /usr/lib/libavcodec.so.54
#2  0x00007ffff6930245 in ?? () from /usr/lib/libavcodec.so.54
#3  0x00007ffff6953dd7 in ?? () from /usr/lib/libavcodec.so.54
#4  0x00007ffff6a9c1bd in av_parser_parse2 () from /usr/lib/libavcodec.so.54
#5  0x00007ffff76c0f83 in ?? () from /usr/lib/libavformat.so.54
#6  0x00007ffff76c143c in ?? () from /usr/lib/libavformat.so.54
#7  0x00007ffff76c2e71 in avformat_find_stream_info ()
   from /usr/lib/libavformat.so.54
#8  0x0000000000413156 in opt_input_file (o=0x7fffffffcf40, 
    opt=<optimized out>, filename=<optimized out>) at ffmpeg.c:4300
#9  0x0000000000419950 in parse_option (optctx=0x7fffffffcf40, 
    opt=0x7fffffffe3c3 "i", 
    arg=0x7fffffffe3c5 "rtsp://(hidden)", 
    options=0x6217a0) at cmdutils.c:311
#10 0x0000000000419af3 in parse_options (optctx=0x7fffffffcf40, argc=3, 
    argv=0x7fffffffe098, options=0x6217a0, 
    parse_arg_function=0x414780 <opt_output_file>) at cmdutils.c:344
#11 0x00000000004062d9 in main (argc=3, argv=0x7fffffffe098) at ffmpeg.c:589

disass

Dump of assembler code from 0x7ffff68e9890 to 0x7ffff68e98d0:
   0x00007ffff68e9890:	push   %r12
   0x00007ffff68e9892:	lea    0x1(%rsi),%r12
   0x00007ffff68e9896:	push   %rbp
   0x00007ffff68e9897:	mov    %rdx,%rbp
   0x00007ffff68e989a:	push   %rbx
   0x00007ffff68e989b:	mov    %rsi,%rbx
   0x00007ffff68e989e:	sub    $0x18,%rsp
   0x00007ffff68e98a2:	movzbl (%rsi),%eax
   0x00007ffff68e98a5:	mov    %rcx,0x8(%rsp)
   0x00007ffff68e98aa:	shr    $0x5,%al
   0x00007ffff68e98ad:	movzbl %al,%eax
=> 0x00007ffff68e98b0:	mov    %eax,0x4cb48(%rdi)
   0x00007ffff68e98b6:	movzbl (%rsi),%eax
   0x00007ffff68e98b9:	and    $0x1f,%eax
   0x00007ffff68e98bc:	cmp    $0x1,%r15d
   0x00007ffff68e98c0:	mov    %eax,0x4cb4c(%rdi)
   0x00007ffff68e98c6:	jle    0x7ffff68e996a
   0x00007ffff68e98cc:	movabs $0xfefffefffefffeff,%rdi
End of assembler dump.

info all-registers

rax            0x3	3
rbx            0x659984	6658436
rcx            0x7fffffffc4e8	140737488340200
rdx            0x7fffffffc4ec	140737488340204
rsi            0x659984	6658436
rdi            0x0	0
rbp            0x7fffffffc4ec	0x7fffffffc4ec
rsp            0x7fffffffc420	0x7fffffffc420
r8             0x27	39
r9             0x2b3	691
r10            0x8000000000000000	-9223372036854775808
r11            0x8000000000000000	-9223372036854775808
r12            0x659985	6658437
r13            0x0	0
r14            0x0	0
r15            0x26	38
rip            0x7ffff68e98b0	0x7ffff68e98b0
eflags         0x10216	[ PF AF IF RF ]
cs             0x33	51
ss             0x2b	43
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0
st0            0	(raw 0x00000000000000000000)
st1            0	(raw 0x00000000000000000000)
st2            0	(raw 0x00000000000000000000)
st3            0	(raw 0x00000000000000000000)
st4            0	(raw 0x00000000000000000000)
st5            0	(raw 0x00000000000000000000)
st6            -0.99999639682229436309264525295503745	(raw 0xbffeffffc38c783738b4)
st7            0.0026844631545961444225035895253320128	(raw 0x3ff6afedd174d0905b01)
fctrl          0x37f	895
fstat          0x20	32
ftag           0xffff	65535
fiseg          0x0	0
fioff          0x0	0
foseg          0x0	0
fooff          0x0	0
fop            0x0	0
xmm0           {v4_float = {0x2b020000, 0x0, 0x0, 0x0}, v2_double = {0x0, 
    0x0}, v16_int8 = {0xfc, 0xa9, 0xf1, 0xd2, 0x4d, 0x62, 0x10, 0x3f, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0xa9fc, 0xd2f1, 0x624d, 
    0x3f10, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0xd2f1a9fc, 0x3f10624d, 0x0, 
    0x0}, v2_int64 = {0x3f10624dd2f1a9fc, 0x0}, 
  uint128 = 0x00000000000000003f10624dd2f1a9fc}
xmm1           {v4_float = {0x0, 0x6, 0x0, 0x0}, v2_double = {0x3e80, 0x0}, 
  v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x40, 0xcf, 0x40, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0, 0x0, 0x4000, 0x40cf, 0x0, 0x0, 0x0, 
    0x0}, v4_int32 = {0x0, 0x40cf4000, 0x0, 0x0}, v2_int64 = {
    0x40cf400000000000, 0x0}, uint128 = 0x000000000000000040cf400000000000}
xmm2           {v4_float = {0x0, 0x1, 0x0, 0x1}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xef, 0x3f, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0xff, 0xef, 0x3f}, v8_int16 = {0x0, 0x0, 0xff00, 0x3fef, 0x0, 0x0, 
    0xff00, 0x3fef}, v4_int32 = {0x0, 0x3fefff00, 0x0, 0x3fefff00}, 
  v2_int64 = {0x3fefff0000000000, 0x3fefff0000000000}, 
  uint128 = 0x3fefff00000000003fefff0000000000}
xmm3           {v4_float = {0x0, 0x2, 0x0, 0x1}, v2_double = {0x2, 0x0}, 
  v16_int8 = {0x0, 0x0, 0x0, 0x10, 0xab, 0xef, 0x7, 0x40, 0x0, 0x0, 0x0, 0x20, 
    0xd6, 0xdf, 0xef, 0x3f}, v8_int16 = {0x0, 0x1000, 0xefab, 0x4007, 0x0, 
    0x2000, 0xdfd6, 0x3fef}, v4_int32 = {0x10000000, 0x4007efab, 0x20000000, 
    0x3fefdfd6}, v2_int64 = {0x4007efab10000000, 0x3fefdfd620000000}, 
  uint128 = 0x3fefdfd6200000004007efab10000000}
xmm4           {v4_float = {0x0, 0x2, 0x0, 0x1}, v2_double = {0x2, 0x0}, 
  v16_int8 = {0x0, 0x0, 0x0, 0x88, 0x75, 0xf7, 0x7, 0x40, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0xff, 0xef, 0x3f}, v8_int16 = {0x0, 0x8800, 0xf775, 0x4007, 0x0, 0x0, 
    0xff00, 0x3fef}, v4_int32 = {0x88000000, 0x4007f775, 0x0, 0x3fefff00}, 
  v2_int64 = {0x4007f77588000000, 0x3fefff0000000000}, 
  uint128 = 0x3fefff00000000004007f77588000000}
xmm5           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0x6d, 0xe9, 0x9d, 0x37, 0x0 <repeats 12 times>}, v8_int16 = {
    0xe96d, 0x379d, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x379de96d, 
    0x0, 0x0, 0x0}, v2_int64 = {0x379de96d, 0x0}, 
  uint128 = 0x000000000000000000000000379de96d}
xmm6           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0x23, 0xf0, 0x99, 0x3d, 0x0 <repeats 12 times>}, v8_int16 = {
    0xf023, 0x3d99, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x3d99f023, 
    0x0, 0x0, 0x0}, v2_int64 = {0x3d99f023, 0x0}, 
  uint128 = 0x0000000000000000000000003d99f023}
xmm7           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, 
  uint128 = 0x00000000000000000000000000000000}
xmm8           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, 
  uint128 = 0x00000000000000000000000000000000}
xmm9           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, 
  uint128 = 0x00000000000000000000000000000000}
xmm10          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x80, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0, 0x0, 0x0, 0x8000, 0x0, 0x0, 0x0, 
    0x0}, v4_int32 = {0x0, 0x80000000, 0x0, 0x0}, v2_int64 = {
    0x8000000000000000, 0x0}, uint128 = 0x00000000000000008000000000000000}
xmm11          {v4_float = {0xffffcfa4, 0x0, 0x0, 0x0}, v2_double = {0x0, 
    0x0}, v16_int8 = {0x16, 0x70, 0x41, 0xc6, 0x58, 0xac, 0x98, 0xb5, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x7016, 0xc641, 0xac58, 
    0xb598, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0xc6417016, 0xb598ac58, 0x0, 
    0x0}, v2_int64 = {0xb598ac58c6417016, 0x0}, 
  uint128 = 0x0000000000000000b598ac58c6417016}
xmm12          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x80, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0, 0x0, 0x0, 0x8000, 0x0, 0x0, 0x0, 
    0x0}, v4_int32 = {0x0, 0x80000000, 0x0, 0x0}, v2_int64 = {
    0x8000000000000000, 0x0}, uint128 = 0x00000000000000008000000000000000}
xmm13          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, 
  uint128 = 0x00000000000000000000000000000000}
xmm14          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0}, 
  uint128 = 0x00000000000000000000000000000000}
xmm15          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, 
  v16_int8 = {0xa0, 0x83, 0x47, 0x3, 0x1d, 0x3c, 0x8a, 0xb5, 0x0, 0x0, 0x0, 
    0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x83a0, 0x347, 0x3c1d, 0xb58a, 0x0, 
    0x0, 0x0, 0x0}, v4_int32 = {0x34783a0, 0xb58a3c1d, 0x0, 0x0}, v2_int64 = {
    0xb58a3c1d034783a0, 0x0}, uint128 = 0x0000000000000000b58a3c1d034783a0}
mxcsr          0x1fa0	[ PE IM DM ZM OM UM PM ]

Valgrind

valgrind ffmpeg -i rtsp://(hidden)
==31899== Memcheck, a memory error detector
==31899== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==31899== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==31899== Command: ffmpeg -i rtsp://(hidden)
==31899== 
ffmpeg version N-41634-gc7bdfbe Copyright (c) 2000-2012 the FFmpeg developers
  built on Jun 16 2012 02:58:25 with gcc 4.6.3
  configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-nonfree --enable-shared --enable-postproc --enable-libx264 --enable-frei0r --enable-librtmp --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libdc1394 --enable-libmp3lame --enable-libtheora --enable-libopenjpeg --enable-libvpx --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libvorbis --enable-libxvid --enable-libfaac --cpu=amdfam10 --arch=x86_64 --enable-x11grab --enable-libxavs --enable-libfreetype --e  libavutil      51. 58.100 / 51. 58.100
  libavcodec     54. 25.100 / 54. 25.100
  libavformat    54.  6.101 / 54.  6.101
  libavdevice    54.  0.100 / 54.  0.100
  libavfilter     2. 80.100 /  2. 80.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
==31899== Invalid write of size 4
==31899==    at 0x58808B0: ??? (in /usr/lib/libavcodec.so.54.25.100)
==31899==    by 0x58C6448: ??? (in /usr/lib/libavcodec.so.54.25.100)
==31899==    by 0x58C7244: ??? (in /usr/lib/libavcodec.so.54.25.100)
==31899==    by 0x58EADD6: ??? (in /usr/lib/libavcodec.so.54.25.100)
==31899==    by 0x5A331BC: av_parser_parse2 (in /usr/lib/libavcodec.so.54.25.100)
==31899==    by 0x53BCF82: ??? (in /usr/lib/libavformat.so.54.6.101)
==31899==    by 0x53BD43B: ??? (in /usr/lib/libavformat.so.54.6.101)
==31899==    by 0x53BEE70: avformat_find_stream_info (in /usr/lib/libavformat.so.54.6.101)
==31899==    by 0x413155: ??? (in /usr/bin/ffmpeg)
==31899==    by 0x41994F: ??? (in /usr/bin/ffmpeg)
==31899==    by 0x419AF2: ??? (in /usr/bin/ffmpeg)
==31899==    by 0x4062D8: ??? (in /usr/bin/ffmpeg)
==31899==  Address 0x4cb48 is not stack'd, malloc'd or (recently) free'd
==31899== 
==31899== 
==31899== Process terminating with default action of signal 11 (SIGSEGV)
==31899==  Access not within mapped region at address 0x4CB48
==31899==    at 0x58808B0: ??? (in /usr/lib/libavcodec.so.54.25.100)
==31899==    by 0x58C6448: ??? (in /usr/lib/libavcodec.so.54.25.100)
==31899==    by 0x58C7244: ??? (in /usr/lib/libavcodec.so.54.25.100)
==31899==    by 0x58EADD6: ??? (in /usr/lib/libavcodec.so.54.25.100)
==31899==    by 0x5A331BC: av_parser_parse2 (in /usr/lib/libavcodec.so.54.25.100)
==31899==    by 0x53BCF82: ??? (in /usr/lib/libavformat.so.54.6.101)
==31899==    by 0x53BD43B: ??? (in /usr/lib/libavformat.so.54.6.101)
==31899==    by 0x53BEE70: avformat_find_stream_info (in /usr/lib/libavformat.so.54.6.101)
==31899==    by 0x413155: ??? (in /usr/bin/ffmpeg)
==31899==    by 0x41994F: ??? (in /usr/bin/ffmpeg)
==31899==    by 0x419AF2: ??? (in /usr/bin/ffmpeg)
==31899==    by 0x4062D8: ??? (in /usr/bin/ffmpeg)
==31899==  If you believe this happened as a result of a stack
==31899==  overflow in your program's main thread (unlikely but
==31899==  possible), you can try to increase the size of the
==31899==  main thread stack using the --main-stacksize= flag.
==31899==  The main thread stack size used in this run was 8388608.
==31899== 
==31899== HEAP SUMMARY:
==31899==     in use at exit: 1,363,305 bytes in 2,866 blocks
==31899==   total heap usage: 4,461 allocs, 1,595 frees, 1,524,406 bytes allocated
==31899== 
==31899== LEAK SUMMARY:
==31899==    definitely lost: 61 bytes in 2 blocks
==31899==    indirectly lost: 336 bytes in 4 blocks
==31899==      possibly lost: 0 bytes in 0 blocks
==31899==    still reachable: 1,362,908 bytes in 2,860 blocks
==31899==         suppressed: 0 bytes in 0 blocks
==31899== Rerun with --leak-check=full to see details of leaked memory
==31899== 
==31899== For counts of detected and suppressed errors, rerun with: -v
==31899== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)
Segmentation fault (core dumped)

For some reason my build configuration is truncated by the version of libavutil, so here's the full line:

--prefix=/usr --enable-gpl --enable-version3 --enable-nonfree --enable-shared --enable-postproc --enable-libx264 --enable-frei0r --enable-librtmp --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libdc1394 --enable-libmp3lame --enable-libtheora --enable-libopenjpeg --enable-libvpx --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libvorbis --enable-libxvid --enable-libfaac --cpu=amdfam10 --arch=x86_64 --enable-x11grab --enable-libxavs --enable-libfreetype --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libcelt --enable-openal --enable-libcdio --enable-libaacplus --enable-libmodplug --enable-libpulse --enable-gnutls --enable-openssl --enable-libv4l2 --enable-libass --enable-static --enable-libbluray --enable-libutvideo --enable-avresample

For reference the stream is a rtsp url and the details of the streams in it from a working build:

Input #0, rtsp, from 'rtsp://(hidden)':
  Metadata:
    title           : (hidden)
    comment         : (hidden)
  Duration: N/A, start: 0.018000, bitrate: N/A
    Stream #0:0: Video: h264 (Constrained Baseline), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 14.99 fps, 30.08 tbr, 90k tbn, 29.97 tbc
    Stream #0:1: Audio: aac, 16000 Hz, mono, s16

Using Ubuntu 11.04 x86_64

Change History (12)

comment:1 by Carl Eugen Hoyos, 12 years ago

Priority: normalimportant

Unfortunately the backtrace you provided is useless, please either compile without --enable-shared or use --disable-stripping to get binaries that are useful for debugging.

Consider testing versions 01a14ce and 15f8941

comment:2 by Kyle, 12 years ago

I just compiled the latest git, it still segfaults. It's also not limited to ffmpeg, it occurs with ffplay as well as VLC when compiled with the latest git of ffmpeg.

Here's the backtrace with --disable-stripping.

#0  0x00007ffff68e48b0 in ff_h264_decode_nal (h=0x0, 
    src=0x659985 "B\300\f\226t\n\017\330\n\004", dst_length=0x7fffffffc4ec, 
    consumed=0x7fffffffc4e8, length=38) at libavcodec/h264.c:175
#1  0x00007ffff692a449 in decode_nal_units (h=0x7fffe8a22040, buf=0x659980 "", 
    buf_size=43) at libavcodec/h264.c:4266
#2  0x00007ffff692b245 in ff_h264_decode_extradata (h=0x7fffe8a22040, 
    buf=0x659980 "", size=43) at libavcodec/h264.c:1206
#3  0x00007ffff694edd7 in h264_parse (s=0x658a00, avctx=0x650920, 
    poutbuf=0x7fffffffc6d0, poutbuf_size=0x7fffffffc6d8, buf=0x658ba0 "", 
    buf_size=691) at libavcodec/h264_parser.c:311
#4  0x00007ffff6a9721d in av_parser_parse2 (s=0x658a00, avctx=0x650920, 
    poutbuf=0x7fffffffc6d0, poutbuf_size=0x7fffffffc6d8, buf=<optimized out>, 
    buf_size=<optimized out>, pts=0, dts=-9223372036854775808, pos=-1)
    at libavcodec/parser.c:149
#5  0x00007ffff76bc423 in parse_packet (s=0x64a0c0, pkt=0x7fffffffc7f0, 
    stream_index=<optimized out>) at libavformat/utils.c:1183
#6  0x00007ffff76bc8dc in read_frame_internal (s=0x64a0c0, pkt=0x7fffffffc9f0)
    at libavformat/utils.c:1352
#7  0x00007ffff76be311 in avformat_find_stream_info (ic=0x64a0c0, 
    options=0x661f00) at libavformat/utils.c:2582
#8  0x0000000000413156 in opt_input_file (o=0x7fffffffcf40, 
    opt=<optimized out>, filename=<optimized out>) at ffmpeg.c:4300
#9  0x0000000000419950 in parse_option (optctx=0x7fffffffcf40, 
    opt=0x7fffffffe3c3 "i", 
    arg=0x7fffffffe3c5 "rtsp://(hidden)", 
    options=0x6217a0) at cmdutils.c:311
#10 0x0000000000419af3 in parse_options (optctx=0x7fffffffcf40, argc=3, 
    argv=0x7fffffffe098, options=0x6217a0, 
    parse_arg_function=0x414780 <opt_output_file>) at cmdutils.c:344
#11 0x00000000004062d9 in main (argc=3, argv=0x7fffffffe098) at ffmpeg.c:5897

comment:3 by Kyle, 12 years ago

Also just incase you need it, the valgrind output

valgrind ffmpeg -i rtsp://(hidden)
==11942== Memcheck, a memory error detector
==11942== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==11942== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==11942== Command: ffmpeg -i rtsp://(hidden)
==11942== 
ffmpeg version N-41656-g1125606 Copyright (c) 2000-2012 the FFmpeg developers
  built on Jun 17 2012 15:39:21 with gcc 4.6.3
  configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-nonfree --enable-shared --enable-postproc --enable-libx264 --enable-frei0r --enable-librtmp --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libdc1394 --enable-libmp3lame --enable-libtheora --enable-libopenjpeg --enable-libvpx --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libvorbis --enable-libxvid --enable-libfaac --cpu=amdfam10 --arch=x86_64 --enable-x11grab --enable-libxavs --enable-libfreetype --e  libavutil      51. 58.100 / 51. 58.100
  libavcodec     54. 25.100 / 54. 25.100
  libavformat    54.  6.101 / 54.  6.101
  libavdevice    54.  0.100 / 54.  0.100
  libavfilter     2. 81.100 /  2. 81.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
==11942== Invalid write of size 4
==11942==    at 0x58858B0: ff_h264_decode_nal (h264.c:175)
==11942==    by 0x58CB448: decode_nal_units (h264.c:4266)
==11942==    by 0x58CC244: ff_h264_decode_extradata (h264.c:1206)
==11942==    by 0x58EFDD6: h264_parse (h264_parser.c:311)
==11942==    by 0x5A3821C: av_parser_parse2 (parser.c:149)
==11942==    by 0x53C1422: parse_packet (utils.c:1183)
==11942==    by 0x53C18DB: read_frame_internal (utils.c:1352)
==11942==    by 0x53C3310: avformat_find_stream_info (utils.c:2582)
==11942==    by 0x413155: opt_input_file (ffmpeg.c:4300)
==11942==    by 0x41994F: parse_option (cmdutils.c:311)
==11942==    by 0x419AF2: parse_options (cmdutils.c:344)
==11942==    by 0x4062D8: main (ffmpeg.c:5897)
==11942==  Address 0x4cb48 is not stack'd, malloc'd or (recently) free'd
==11942== 
==11942== 
==11942== Process terminating with default action of signal 11 (SIGSEGV)
==11942==  Access not within mapped region at address 0x4CB48
==11942==    at 0x58858B0: ff_h264_decode_nal (h264.c:175)
==11942==    by 0x58CB448: decode_nal_units (h264.c:4266)
==11942==    by 0x58CC244: ff_h264_decode_extradata (h264.c:1206)
==11942==    by 0x58EFDD6: h264_parse (h264_parser.c:311)
==11942==    by 0x5A3821C: av_parser_parse2 (parser.c:149)
==11942==    by 0x53C1422: parse_packet (utils.c:1183)
==11942==    by 0x53C18DB: read_frame_internal (utils.c:1352)
==11942==    by 0x53C3310: avformat_find_stream_info (utils.c:2582)
==11942==    by 0x413155: opt_input_file (ffmpeg.c:4300)
==11942==    by 0x41994F: parse_option (cmdutils.c:311)
==11942==    by 0x419AF2: parse_options (cmdutils.c:344)
==11942==    by 0x4062D8: main (ffmpeg.c:5897)
==11942==  If you believe this happened as a result of a stack
==11942==  overflow in your program's main thread (unlikely but
==11942==  possible), you can try to increase the size of the
==11942==  main thread stack using the --main-stacksize= flag.
==11942==  The main thread stack size used in this run was 8388608.
==11942== 
==11942== HEAP SUMMARY:
==11942==     in use at exit: 1,363,620 bytes in 2,868 blocks
==11942==   total heap usage: 4,466 allocs, 1,598 frees, 1,526,169 bytes allocated
==11942== 
==11942== LEAK SUMMARY:
==11942==    definitely lost: 61 bytes in 2 blocks
==11942==    indirectly lost: 336 bytes in 4 blocks
==11942==      possibly lost: 0 bytes in 0 blocks
==11942==    still reachable: 1,363,223 bytes in 2,862 blocks
==11942==         suppressed: 0 bytes in 0 blocks
==11942== Rerun with --leak-check=full to see details of leaked memory
==11942== 
==11942== For counts of detected and suppressed errors, rerun with: -v
==11942== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)
Segmentation fault (core dumped)

comment:4 by Kyle, 12 years ago

It also segfaults with 01a14ce and 15f8941

in reply to:  4 comment:5 by Carl Eugen Hoyos, 12 years ago

Replying to kyl416:

It also segfaults with 01a14ce

If it works with c7b9eab (as you wrote above), git bisect will help you to find the responsible change.

comment:6 by Carl Eugen Hoyos, 12 years ago

Component: undeterminedavcodec
Keywords: h264 regression added

comment:7 by Kyle, 12 years ago

ddece75 was the last commit that didn't segfault.

in reply to:  7 comment:8 by Carl Eugen Hoyos, 12 years ago

Replying to kyl416:

ddece75 was the last commit that didn't segfault.

The following commit - bb85048 - changes mjpeg encoding, so this seems a bit unlikely.
Perhaps you could try: make distclean && git checkout bb85048 && ./configure && make

comment:9 by Kyle, 12 years ago

Yeah it's no longer segfaulting.

in reply to:  9 comment:10 by Carl Eugen Hoyos, 12 years ago

Replying to kyl416:

Yeah it's no longer segfaulting.

Does it also work with current git head?

comment:11 by Kyle, 12 years ago

Yes, the latest git is working.

comment:12 by Carl Eugen Hoyos, 12 years ago

Resolution: invalid
Status: newclosed

Thank you for the clarification.

Note: See TracTickets for help on using tickets.