Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#3722 closed defect (invalid)

Illegal instruction, SIGILL when encoding libvpx-vp9

Reported by: xxleite Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: libvpx vp9 sigill crash
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

I'm using the last version of libvpx (v1.3.0-3148-g6cfb854) and ffmpeg (N-64012-g61df081).

I tried to decode at least three diferent videos and process receive a SIGILL "Illegal instruction" and stops.

How to reproduce:

gdb --args /usr/local/bin/ffmpeg -i /home/xico/test/big_buck_bunny.mov -passlogfile /tmp/x0y0z0 -s hd480 -c:v libvpx-vp9 -threads 0 -b:v 1027k -quality good -pass 1 -y -f webm -an /dev/null
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/bin/ffmpeg...(no debugging symbols found)...done.
(gdb) r
Starting program: /usr/local/bin/ffmpeg -i /home/xico/test/big_buck_bunny.mov -passlogfile /tmp/x0y0z0 -s hd480 -c:v libvpx-vp9 -threads 0 -b:v 1027k -quality good -pass 1 -y -f webm -an /dev/null
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
ffmpeg version N-64012-g61df081-syslint Copyright (c) 2000-2014 the FFmpeg developers
  built on Jun 17 2014 06:13:41 with gcc 4.7 (Debian 4.7.2-5)
  configuration: --prefix=/usr/local/cpffmpeg --enable-shared --enable-nonfree --enable-iconv --enable-gpl --enable-pthreads --enable-libopencore-amrnb --enable-decoder=liba52 --enable-libopencore-amrwb --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-libvpx --extra-cflags=-I/usr/local/cpffmpeg/include/ --extra-ldflags=-L/usr/local/cpffmpeg/lib --enable-version3 --extra-version=syslint --enable-libass
  libavutil      52. 89.100 / 52. 89.100
  libavcodec     55. 67.100 / 55. 67.100
  libavformat    55. 43.100 / 55. 43.100
  libavdevice    55. 13.101 / 55. 13.101
  libavfilter     4.  8.100 /  4.  8.100
  libswscale      2.  6.100 /  2.  6.100
  libswresample   0. 19.100 /  0. 19.100
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/xico/test/big_buck_bunny.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  
    creation_time   : 2008-05-27 18:32:32
    timecode        : 00:00:00:00
  Duration: 00:09:56.46, start: 0.000000, bitrate: 3342 kb/s
    Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 854x480, 2899 kb/s, 24 fps, 24 tbr, 2400 tbn, 4800 tbc (default)
    Metadata:
      creation_time   : 2008-05-27 18:32:32
      handler_name    : Apple Alias Data Handler
      encoder         : H.264
    Stream #0:1(eng): Data: none (tmcd / 0x64636D74) (default)
    Metadata:
      creation_time   : 2008-05-27 18:32:32
      handler_name    : Apple Alias Data Handler
      timecode        : 00:00:00:00
    Stream #0:2(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 437 kb/s (default)
    Metadata:
      creation_time   : 2008-05-27 18:32:32
      handler_name    : Apple Alias Data Handler
[libvpx-vp9 @ 0x6518e0] v1.3.0-3148-g6cfb854
Output #0, webm, to '/dev/null':
  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  
    timecode        : 00:00:00:00
    encoder         : Lavf55.43.100
    Stream #0:0(eng): Video: vp9 (libvpx-vp9), yuv420p, 852x480, q=-1--1, pass 1, 1027 kb/s, 24 fps, 1k tbn, 24 tbc (default)
    Metadata:
      creation_time   : 2008-05-27 18:32:32
      handler_name    : Apple Alias Data Handler
      encoder         : Lavc55.67.100 libvpx-vp9
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> vp9 (libvpx-vp9))
Press [q] to stop, [?] for help

Program received signal SIGILL, Illegal instruction.
0x00007ffff3923c40 in vp9_fdct4x4_avx2 () from /usr/local/cpffmpeg/lib/libvpx.so.1
(gdb) bt
#0  0x00007ffff3923c40 in vp9_fdct4x4_avx2 () from /usr/local/cpffmpeg/lib/libvpx.so.1
#1  0x00007ffff38b5228 in encode_block_intra () from /usr/local/cpffmpeg/lib/libvpx.so.1
#2  0x00007ffff3852147 in vp9_foreach_transformed_block_in_plane () from /usr/local/cpffmpeg/lib/libvpx.so.1
#3  0x00007ffff38b75ea in vp9_encode_intra_block_plane () from /usr/local/cpffmpeg/lib/libvpx.so.1
#4  0x00007ffff38ba3fd in vp9_first_pass () from /usr/local/cpffmpeg/lib/libvpx.so.1
#5  0x00007ffff38d40ba in vp9_get_compressed_data () from /usr/local/cpffmpeg/lib/libvpx.so.1
#6  0x00007ffff388b621 in encoder_encode () from /usr/local/cpffmpeg/lib/libvpx.so.1
#7  0x00007ffff37dfc4c in vpx_codec_encode () from /usr/local/cpffmpeg/lib/libvpx.so.1
#8  0x00007ffff6559336 in ?? () from /usr/local/cpffmpeg/lib/libavcodec.so.55
#9  0x00007ffff6728774 in avcodec_encode_video2 () from /usr/local/cpffmpeg/lib/libavcodec.so.55
#10 0x000000000041ec67 in ?? ()
#11 0x000000000040898d in ?? ()
#12 0x00007ffff503fead in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#13 0x0000000000409149 in ?? ()
#14 0x00007fffffffe618 in ?? ()
#15 0x000000000000001c in ?? ()
#16 0x0000000000000016 in ?? ()
#17 0x00007fffffffe8d6 in ?? ()
#18 0x00007fffffffe8ec in ?? ()
#19 0x00007fffffffe8ef in ?? ()
#20 0x00007fffffffe912 in ?? ()
#21 0x00007fffffffe91f in ?? ()
#22 0x00007fffffffe92b in ?? ()
#23 0x00007fffffffe92e in ?? ()
#24 0x00007fffffffe934 in ?? ()
#25 0x00007fffffffe939 in ?? ()
#26 0x00007fffffffe944 in ?? ()
#27 0x00007fffffffe94d in ?? ()
#28 0x00007fffffffe94f in ?? ()
#29 0x00007fffffffe954 in ?? ()
#30 0x00007fffffffe95a in ?? ()
#31 0x00007fffffffe963 in ?? ()
#32 0x00007fffffffe968 in ?? ()
#33 0x00007fffffffe96e in ?? ()
#34 0x00007fffffffe970 in ?? ()
#35 0x00007fffffffe973 in ?? ()
#36 0x00007fffffffe976 in ?? ()
#37 0x00007fffffffe97b in ?? ()
#38 0x00007fffffffe97f in ?? ()
#39 0x0000000000000000 in ?? ()
(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0x7ffff3923c20 to 0x7ffff3923c60:
   0x00007ffff3923c20 <FDCT32x32_2D+13216>: or     %al,(%rcx)
   0x00007ffff3923c22 <FDCT32x32_2D+13218>: add    %al,(%rax)
   0x00007ffff3923c24 <FDCT32x32_2D+13220>: movdqa %xmm8,0x148(%rsp)
   0x00007ffff3923c2e <FDCT32x32_2D+13230>: movdqa %xmm1,0x68(%rsp)
   0x00007ffff3923c34 <FDCT32x32_2D+13236>: jmpq   0x7ffff3921191 <FDCT32x32_2D+2321>
   0x00007ffff3923c39:  nop
   0x00007ffff3923c3a:  nop
   0x00007ffff3923c3b:  nop
   0x00007ffff3923c3c:  nop
   0x00007ffff3923c3d:  nop
   0x00007ffff3923c3e:  nop
   0x00007ffff3923c3f:  nop
=> 0x00007ffff3923c40 <vp9_fdct4x4_avx2+0>: vpxor  %xmm10,%xmm10,%xmm10
   0x00007ffff3923c45 <vp9_fdct4x4_avx2+5>: movslq %edx,%rax
   0x00007ffff3923c48 <vp9_fdct4x4_avx2+8>: vpxor  %xmm1,%xmm1,%xmm1
   0x00007ffff3923c4c <vp9_fdct4x4_avx2+12>:  vpxor  %xmm3,%xmm3,%xmm3
   0x00007ffff3923c50 <vp9_fdct4x4_avx2+16>:  vmovdqa 0x6aab8(%rip),%xmm8        # 0x7ffff398e710
   0x00007ffff3923c58 <vp9_fdct4x4_avx2+24>:  vpinsrq $0x0,(%rdi,%rax,2),%xmm10,%xmm10
   0x00007ffff3923c5f <vp9_fdct4x4_avx2+31>:  lea    (%rdx,%rdx,1),%eax
End of assembler dump.
(gdb) info all-registers 
rax            0x7fffee45c740 140737190938432
rbx            0x7fffefc9d020 140737216368672
rcx            0x0  0
rdx            0x10 16
rsi            0xb999c0 12163520
rdi            0x7fffefc9d020 140737216368672
rbp            0x7fffefc9d020 0x7fffefc9d020
rsp            0x7fffffffd5c8 0x7fffffffd5c8
r8             0x7fffebb17740 140737147664192
r9             0x4a0  1184
r10            0x0  0
r11            0x0  0
r12            0x7fffefca3140 140737216393536
r13            0x7fffee45b4c0 140737190933696
r14            0x7fffebb164c0 140737147659456
r15            0x10 16
rip            0x7ffff3923c40 0x7ffff3923c40 <vp9_fdct4x4_avx2>
eflags         0x10246  [ PF ZF IF RF ]
cs             0x33 51
ss             0x2b 43
ds             0x0  0
es             0x0  0
fs             0x0  0
gs             0x0  0
st0            -nan(0xff90ff90ff90ff90) (raw 0xffffff90ff90ff90ff90)
st1            -nan(0x80008000800080) (raw 0xffff0080008000800080)
st2            -nan(0xff90ff90ff90ff90) (raw 0xffffff90ff90ff90ff90)
st3            -nan(0x80008000800080) (raw 0xffff0080008000800080)
st4            -nan(0x101010180808080)  (raw 0xffff0101010180808080)
st5            -nan(0x101010180808080)  (raw 0xffff0101010180808080)
st6            -nan(0x20002000200020) (raw 0xffff0020002000200020)
st7            -inf (raw 0xffff0000000000000000)
fctrl          0x27f  639
fstat          0x20 32
ftag           0xaaaa 43690
fiseg          0x7fff 32767
fioff          0xf55edb87 -178332793
foseg          0x7fff 32767
fooff          0xffffd718 -10472
fop            0x0  0
xmm0           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x40, 0xf, 0x56, 0x0, 0x40, 0xf, 0x56, 0x0, 0x40, 0xf, 0x56, 0x0, 0x40, 0xf, 0x56, 0x0}, v8_int16 = {
    0xf40, 0x56, 0xf40, 0x56, 0xf40, 0x56, 0xf40, 0x56}, v4_int32 = {0x560f40, 0x560f40, 0x560f40, 0x560f40}, v2_int64 = {0x560f4000560f40, 0x560f4000560f40}, 
  uint128 = 0x00560f4000560f4000560f4000560f40}
xmm1           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0xc4, 0x9, 0x0, 0x0, 0xc4, 0x9, 0x0, 0x0, 0xc4, 0x9, 0x0, 0x0, 0xc4, 0x9, 0x0, 0x0}, v8_int16 = {0x9c4, 
    0x0, 0x9c4, 0x0, 0x9c4, 0x0, 0x9c4, 0x0}, v4_int32 = {0x9c4, 0x9c4, 0x9c4, 0x9c4}, v2_int64 = {0x9c4000009c4, 0x9c4000009c4}, uint128 = 0x000009c4000009c4000009c4000009c4}
xmm2           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x40, 0xf, 0x56, 0x0, 0x40, 0xf, 0x56, 0x0, 0x40, 0xf, 0x56, 0x0, 0x40, 0xf, 0x56, 0x0}, v8_int16 = {
    0xf40, 0x56, 0xf40, 0x56, 0xf40, 0x56, 0xf40, 0x56}, v4_int32 = {0x560f40, 0x560f40, 0x560f40, 0x560f40}, v2_int64 = {0x560f4000560f40, 0x560f4000560f40}, 
  uint128 = 0x00560f4000560f4000560f4000560f40}
xmm3           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x8000000000000000, 0x8000000000000000}, v16_int8 = {0xff, 0xff, 0xff, 0x7f, 0xff, 0xff, 0xff, 0x7f, 0xff, 0xff, 0xff, 0x7f, 
    0xff, 0xff, 0xff, 0x7f}, v8_int16 = {0xffff, 0x7fff, 0xffff, 0x7fff, 0xffff, 0x7fff, 0xffff, 0x7fff}, v4_int32 = {0x7fffffff, 0x7fffffff, 0x7fffffff, 0x7fffffff}, v2_int64 = {
    0x7fffffff7fffffff, 0x7fffffff7fffffff}, uint128 = 0x7fffffff7fffffff7fffffff7fffffff}
xmm4           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x3, 0x0, 0x0, 0x0, 0x3, 0x0, 0x0, 0x0, 0x3, 0x0, 0x0, 0x0, 0x3, 0x0, 0x0, 0x0}, v8_int16 = {0x3, 0x0, 
    0x3, 0x0, 0x3, 0x0, 0x3, 0x0}, v4_int32 = {0x3, 0x3, 0x3, 0x3}, v2_int64 = {0x300000003, 0x300000003}, uint128 = 0x00000003000000030000000300000003}
xmm5           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 
    0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm6           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0, 0x3d, 0x58, 0x1, 0x0, 0x3d, 0x58, 0x1, 0x0, 0x3d, 0x58, 0x1, 0x0, 0x3d, 0x58, 0x1}, v8_int16 = {
    0x3d00, 0x158, 0x3d00, 0x158, 0x3d00, 0x158, 0x3d00, 0x158}, v4_int32 = {0x1583d00, 0x1583d00, 0x1583d00, 0x1583d00}, v2_int64 = {0x1583d0001583d00, 0x1583d0001583d00}, 
  uint128 = 0x01583d0001583d0001583d0001583d00}
xmm7           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 
    0x0}, v2_int64 = {0x0, 0x0}, uint128 = 0x00000000000000000000000000000000}
xmm8           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x40, 0x23, 0x0, 0x0, 0x40, 0x23, 0x0, 0x0, 0x40, 0x23, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {
    0x2340, 0x0, 0x2340, 0x0, 0x2340, 0x0, 0x0, 0x0}, v4_int32 = {0x2340, 0x2340, 0x2340, 0x0}, v2_int64 = {0x234000002340, 0x2340}, uint128 = 0x00000000000023400000234000002340}
xmm9           {v4_float = {0x0, 0x2, 0x0, 0x0}, v2_double = {0x18, 0x0}, v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0xc0, 0x38, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0, 
    0x0, 0xc000, 0x4038, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x4038c000, 0x0, 0x0}, v2_int64 = {0x4038c00000000000, 0x0}, uint128 = 0x00000000000000004038c00000000000}
xmm10          {v4_float = {0x0, 0x4b, 0x0, 0x0}, v2_double = {0x5ffffffffff, 0x0}, v16_int8 = {0x2c, 0xfd, 0xff, 0xff, 0xff, 0xff, 0x97, 0x42, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, 
  v8_int16 = {0xfd2c, 0xffff, 0xffff, 0x4297, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0xfffffd2c, 0x4297ffff, 0x0, 0x0}, v2_int64 = {0x4297fffffffffd2c, 0x0}, 
  uint128 = 0x00000000000000004297fffffffffd2c}
xmm11          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x1, 0x24, 0xc, 0xb3, 0x25, 0xd7, 0x5a, 0x3e, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {
    0x2401, 0xb30c, 0xd725, 0x3e5a, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0xb30c2401, 0x3e5ad725, 0x0, 0x0}, v2_int64 = {0x3e5ad725b30c2401, 0x0}, uint128 = 0x00000000000000003e5ad725b30c2401}
xmm12          {v4_float = {0x0, 0x1, 0x0, 0x0}, v2_double = {0x1, 0x0}, v16_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xf0, 0x3f, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {0x0, 0x0, 
    0x0, 0x3ff0, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x3ff00000, 0x0, 0x0}, v2_int64 = {0x3ff0000000000000, 0x0}, uint128 = 0x00000000000000003ff0000000000000}
xmm13          {v4_float = {0xf9507408, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0x7f, 0xf1, 0xd5, 0xcc, 0x24, 0x22, 0xb6, 0xbb, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, 
  v8_int16 = {0xf17f, 0xccd5, 0x2224, 0xbbb6, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0xccd5f17f, 0xbbb62224, 0x0, 0x0}, v2_int64 = {0xbbb62224ccd5f17f, 0x0}, 
  uint128 = 0x0000000000000000bbb62224ccd5f17f}
xmm14          {v4_float = {0x0, 0x2, 0x0, 0x0}, v2_double = {0xb, 0x0}, v16_int8 = {0x80, 0xdb, 0xe9, 0x2e, 0x27, 0x91, 0x27, 0x40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {
    0xdb80, 0x2ee9, 0x9127, 0x4027, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x2ee9db80, 0x40279127, 0x0, 0x0}, v2_int64 = {0x402791272ee9db80, 0x0}, uint128 = 0x0000000000000000402791272ee9db80}
xmm15          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0}, v16_int8 = {0xa0, 0x83, 0x47, 0x3, 0x1d, 0x3c, 0x8a, 0xb5, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, v8_int16 = {
    0x83a0, 0x347, 0x3c1d, 0xb58a, 0x0, 0x0, 0x0, 0x0}, v4_int32 = {0x34783a0, 0xb58a3c1d, 0x0, 0x0}, v2_int64 = {0xb58a3c1d034783a0, 0x0}, uint128 = 0x0000000000000000b58a3c1d034783a0}
mxcsr          0x1fa8 [ OE PE IM DM ZM OM UM PM ]

cpu:

less /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 2
model name      : QEMU Virtual CPU version 1.0
stepping        : 3
microcode       : 0x1
cpu MHz         : 1999.999
cache size      : 4096 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 4
wp              : yes
flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm up rep_good nopl pni vmx cx16 popcnt hypervisor lahf_lm
bogomips        : 3999.99
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Change History (2)

comment:1 Changed 4 years ago by jamal

  • Keywords sigill added; encoding crash removed
  • Resolution set to invalid
  • Status changed from new to closed

libvpx is trying to execute an avx2 function but your CPU evidently doesn't support that instruction set.

This is not a bug with ffmpeg but libvpx instead.

comment:2 Changed 4 years ago by cehoyos

  • Component changed from undetermined to avcodec
  • Keywords crash added
Note: See TracTickets for help on using tickets.