SEGFAULT in libavcodec in ff_emu_edge_vfix3_mmx.body_loop when playing video in vlc.

[Michal Srb]

Summary: Playing a video in vlc crashes with SEGFAULT in libavcodec. Crash happens at random time (each time different). System is opensuse 12.3.

Video information:

Input #0, avi, from 'Videos/Boy Meets World Season 1 - 7 DVDRip/Season 4/Boy.Meets.World.S04E06.DVDRip.XviD-NODLABS.avi':
  Metadata:
    encoder         : Nandub v1.0rc2
  Duration: 00:21:57.70, start: 0.000000, bitrate: 1108 kb/s
    Stream #0:0: Video: mpeg4 (Advanced Simple Profile) (XVID / 0x44495658), yuv420p, 512x384 [SAR 1:1 DAR 4:3], 29.97 tbr, 29.97 tbn, 29.97 tbc
    Stream #0:1: Audio: mp3 (U[0][0][0] / 0x0055), 48000 Hz, stereo, s16p, 135 kb/s

Backtrace:

#0  ff_emu_edge_vfix3_mmx.body_loop () at libavcodec/x86/videodsp.asm:333
#1  0x00007fffcadc4f5c in emulated_edge_mc (h_extend_var=<optimized out>, hfix_tbl=0x7fffcb1db220 <hfixtbl_sse2>, v_extend_var=<optimized out>, 
    vfix_tbl=0x7fffcb1db160 <vfixtbl_sse>, h=192, w=3, src_y=<optimized out>, src_x=<optimized out>, block_h=9, block_w=<optimized out>, src_stride=<optimized out>, 
    src=<optimized out>, dst_stride=256, dst=<optimized out>) at libavcodec/x86/videodsp_init.c:175
#2  emulated_edge_mc_sse2 (buf=0x7fffcc04b410 "\200\200\200\202\201\201\202\202\203", buf_stride=256, src=<optimized out>, src_stride=<optimized out>, 
    block_w=<optimized out>, block_h=9, src_x=253, src_y=184, w=256, h=192) at libavcodec/x86/videodsp_init.c:232
#3  0x00007fffcab8bf03 in mpeg_motion_internal (mb_y=23, is_mpeg12=0, h=16, motion_y=0, motion_x=117, pix_op=0x7fffb4c16ec8, ref_picture=<optimized out>, 
    field_select=0, bottom_field=0, field_based=0, dest_cr=0x7fffcc05a840 "\202\202\202\202\201\201\201\200", dest_cb=0x7fffcc058840 "vvwwwxxx", 
    dest_y=0x7fffcc056840 "\026\026\026\026\026\026\025\025\031\032\033\035\037\"\" ", s=0x7fffb4c148c0) at libavcodec/mpegvideo_motion.c:333
#4  mpeg_motion (s=0x7fffb4c148c0, dest_y=0x7fffcc056840 "\026\026\026\026\026\026\025\025\031\032\033\035\037\"\" ", dest_cb=0x7fffcc058840 "vvwwwxxx", 
    dest_cr=0x7fffcc05a840 "\202\202\202\202\201\201\201\200", field_select=0, ref_picture=<optimized out>, pix_op=0x7fffb4c16ec8, motion_x=117, motion_y=0, h=16, 
    mb_y=23) at libavcodec/mpegvideo_motion.c:384
#5  0x00007fffcab8c9e7 in MPV_motion_internal (is_mpeg12=<optimized out>, qpix_op=<optimized out>, pix_op=<optimized out>, ref_picture=<optimized out>, 
    dir=<optimized out>, dest_cr=<optimized out>, dest_cb=<optimized out>, dest_y=<optimized out>, s=<optimized out>) at libavcodec/mpegvideo_motion.c:958
#6  ff_MPV_motion (s=s@entry=0x7fffb4c148c0, dest_y=dest_y@entry=0x7fffcc056840 "\026\026\026\026\026\026\025\025\031\032\033\035\037\"\" ", 
    dest_cb=dest_cb@entry=0x7fffcc058840 "vvwwwxxx", dest_cr=dest_cr@entry=0x7fffcc05a840 "\202\202\202\202\201\201\201\200", dir=dir@entry=1, 
    ref_picture=ref_picture@entry=0x7fffb4c152d8, pix_op=0x7fffb4c16ec8, qpix_op=0x7fffb4c169d8) at libavcodec/mpegvideo_motion.c:992
#7  0x00007fffcab74973 in MPV_decode_mb_internal (is_mpeg12=0, lowres_flag=0, block=<optimized out>, s=0x7fffb4c148c0) at libavcodec/mpegvideo.c:2796
#8  ff_MPV_decode_mb (s=s@entry=0x7fffb4c148c0, block=<optimized out>) at libavcodec/mpegvideo.c:2928
#9  0x00007fffca905870 in decode_slice (s=s@entry=0x7fffb4c148c0) at libavcodec/h263dec.c:274
#10 0x00007fffca906893 in ff_h263_decode_frame (avctx=0x7fffb4c18500, data=0x7fffb4c18020, got_frame=0x7fffd01cacdc, avpkt=<optimized out>) at libavcodec/h263dec.c:701
#11 0x00007fffcac789a2 in avcodec_decode_video2 (avctx=0x7fffb4c18500, picture=0x7fffb4c18020, got_picture_ptr=0x7fffd01cacdc, avpkt=0x7fffd01cace0)
    at libavcodec/utils.c:2062
#12 0x00007fffcb7e705e in ?? () from /usr/lib64/vlc/plugins/codec/libavcodec_plugin.so
#13 0x00007ffff7141380 in ?? () from /usr/lib64/libvlccore.so.7
#14 0x00007ffff71427a0 in ?? () from /usr/lib64/libvlccore.so.7
#15 0x00007ffff79aae0f in start_thread () from /lib64/libpthread.so.0
#16 0x00007ffff74da44d in clone () from /lib64/libc.so.6

Sample file:
​http://gdmt.cz/sample-file.avi
(I was able to reproduce it on this sample, happened after ~5 tries, crashed in first few seconds.)

I am not sure how (if possible) to reproduce it with ffmpeg command only.

[Carl Eugen Hoyos]

Replying to michalsrb:

Summary: Playing a video in vlc crashes with SEGFAULT in libavcodec.

I played your sample on Opensuse 12.3 with vlc ~ten times and I do not experience a crash, What cpu does your system use?

Crash happens at random time (each time different).

This would normally indicate a hardware problem. (Did you run memtest86 recently?)
Please test the following command line:
$ vlc --ffmpeg-threads=1 sample-file.avi
If you can not reproduce the problem, try heating your cpu at the same time, for example with the following command:
$ ffmpeg -f lavfi -i smptehdbars -vcodec png -f null -

I am not sure how (if possible) to reproduce it with ffmpeg command only.

You can try the following:
$ ffmpeg -i sample-file.avi -f null -

[Michal Srb]

I played your sample on Opensuse 12.3 with vlc ~ten times and I do not experience a crash, What cpu does your system use?

cat /proc/cpuinfo

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 58
model name      : Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
stepping        : 9
microcode       : 0x17
cpu MHz         : 1200.000
cache size      : 4096 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
bogomips        : 5787.01
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

(repeated 4 times)

This would normally indicate a hardware problem. (Did you run memtest86 recently?)

I've let it run now and all tests passed ok.

Please test the following command line:
$ vlc --ffmpeg-threads=1 sample-file.avi

Still crashes - I was able to reproduce it once, after 4 tries. The backtrace looked the same as before. I tried to reproduce it again, and second time it took something like 30 tries, the backtrace was still the same.

You can try the following:
$ ffmpeg -i sample-file.avi -f null -

It was running in loop many times and didn't crash even once. It was in parallel with the vlc tests, so CPU was under the same stress.

[Carl Eugen Hoyos]

Does the following command line allow you to reproduce the crash?

$ for i in {1..30}; do vlc --avi-index 2 --play-and-exit --ffmpeg-threads=1 --no-audio sample-file.avi; done;

Which video output module are you using? Does it also crash with -V dummy?

[Michal Srb]

Replying to cehoyos:

Does the following command line allow you to reproduce the crash?

$ for i in {1..30}; do vlc --avi-index 2 --play-and-exit --ffmpeg-threads=1 --no-audio sample-file.avi; done;

Yes, it crashed. However I noticed vlc is complaining about the --ffmpeg-threads parameter: "Warning: option --ffmpeg-threads no longer exists.".

And I realized that the vlc and ffmpeg are not from opensuse 12.3 repositories, but from Packman. (​http://packman.inode.at/suse/openSUSE_12.3/Essentials):
vlc-2.1.2-186.2
ffmpeg-2.1.2-2.1
libacodec55-2.1.2-2.1

Sorry for not mentioning it earlier.

Which video output module are you using? Does it also crash with -V dummy?

If I am not mistaken, XVideo:

[0x7f70bc001248] xcb_xv vout display debug: connected to X11.0 server
[0x7f70bc001248] xcb_xv vout display debug:  vendor : The X.Org Foundation
[0x7f70bc001248] xcb_xv vout display debug:  version: 11302000
[0x7f70bc001248] xcb_xv vout display debug: using screen 0xdb
[0x7f70bc001248] xcb_xv vout display debug: using XVideo extension v2.2
[0x7f70bc001248] xcb_xv vout display debug: using adaptor Intel(R) Textured Video
[0x7f70bc001248] xcb_xv vout display debug: using port 81
[0x7f70bc001248] xcb_xv vout display debug: using image format 0x30323449
[0x7f70bc001248] xcb_xv vout display debug: using X11 visual ID 0x21 (depth: 24)
[0x7f70bc001248] xcb_xv vout display debug: using X11 window 0x05a00000
[0x7f70bc001248] xcb_xv vout display debug: using X11 graphic context 0x05a00002

I couldn't reproduce it with -V dummy. I have let it run many times, hopefully I wasn't just (un)lucky.

[Michal Srb]

Btw here is core dump of vlc:
​http://gdmt.cz/vlc.core

[Carl Eugen Hoyos]

Replying to michalsrb:

Replying to cehoyos:

Does the following command line allow you to reproduce the crash?

$ for i in {1..30}; do vlc --avi-index 2 --play-and-exit --ffmpeg-threads=1 --no-audio sample-file.avi; done;

Yes, it crashed. However I noticed vlc is complaining about the --ffmpeg-threads parameter: "Warning: option --ffmpeg-threads no longer exists.".

It was apparently renamed to --avcodec-threads, please test that.

And I realized that the vlc and ffmpeg are not from opensuse 12.3 repositories, but from Packman. (​http://packman.inode.at/suse/openSUSE_12.3/Essentials):

Of course.

[Michal Srb]

Replying to cehoyos:

It was apparently renamed to --avcodec-threads, please test that.

Tested vlc --avi-index 2 --play-and-exit --avcodec-threads=1 --no-audio sample-file.avi, crashed too.

[Carl Eugen Hoyos]

I tested the following with vlc 2.1.2-186.3 and libavcodec55 2.1.1-2.2 without experiencing a crash:

$ for i in {1..300}; do vlc -V xv --avi-index 2 --play-and-exit --avcodec-threads=1 --no-audio sample-file.avi; done

I wonder if this could be related to your video drivers, can you change them?
Please add the missing information to your backtrace, see ​http://ffmpeg.org/bugreports.html

[Carl Eugen Hoyos]

I was finally able to reproduce the crash, this is the same as this Debian report: ​https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=801745
A fix was discussed and will be pushed soon: ​http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/202091

[Carl Eugen Hoyos]

Fixed by Ronald in 52f84d82bdf1851ecfcc412c1719e5f6f3396209
Thank you for the important report and sorry that this wasn't fixed earlier!

[Carl Eugen Hoyos]

I suspect this was a regression since face578d56c2d1375e40d5e2a28acc122132bc55