Opened 10 years ago

Closed 10 years ago

#3188 closed defect (fixed)

vp9 crash (fuzzed input, MT regression)

Reported by: Clément Bœsch Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: vp9 regression crash SIGSEGV
Cc: rsbultje@gmail.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

☭ ./ffmpeg -threads 1 -f ivf -c:v vp9 -i ~/samples/vp9/fuzzed0.ivf -f null -             
ffmpeg version N-58699-ge3d7a39 Copyright (c) 2000-2013 the FFmpeg developers
  built on Dec  2 2013 11:55:32 with gcc 4.8.2 (GCC)
  configuration: --enable-gpl --enable-libx264 --enable-libmp3lame --enable-x11grab --enable-libvorbis --samples=/home/ux/fate-samples --enable-libfreetype --enable-libvpx --cpu=native --cc='ccache cc'
  libavutil      52. 56.100 / 52. 56.100
  libavcodec     55. 44.100 / 55. 44.100
  libavformat    55. 22.100 / 55. 22.100
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     3. 91.100 /  3. 91.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Truncating packet of size 402024711 to 1663093
Input #0, ivf, from '/home/ux/samples/vp9/fuzzed0.ivf':
  Duration: N/A, start: 0.000001, bitrate: N/A
    Stream #0:0: Video: vp9 (vP[25]0 / 0x30195076), yuv420p, 256x244, 0k tbr, 0k tbn, 0k tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf55.22.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x244, q=2-31, 200 kb/s, 90k tbn, 0k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 -> rawvideo)
Press [q] to stop, [?] for help
zsh: segmentation fault (core dumped)  ./ffmpeg -threads 1 -f ivf -c:v vp9 -i ~/samples/vp9/fuzzed0.ivf -f null -
☭ gdb --args ./ffmpeg_g -threads 1 -f ivf -c:v vp9 -i ~/samples/vp9/fuzzed0.ivf -f null -            
GNU gdb (GDB) 7.6.1
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/ux/src/ffmpeg/ffmpeg_g...done.
(gdb) r
Starting program: /home/ux/src/ffmpeg/./ffmpeg_g -threads 1 -f ivf -c:v vp9 -i /home/ux/samples/vp9/fuzzed0.ivf -f null -
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
ffmpeg version N-58699-ge3d7a39 Copyright (c) 2000-2013 the FFmpeg developers
  built on Dec  2 2013 11:55:32 with gcc 4.8.2 (GCC)
  configuration: --enable-gpl --enable-libx264 --enable-libmp3lame --enable-x11grab --enable-libvorbis --samples=/home/ux/fate-samples --enable-libfreetype --enable-libvpx --cpu=native --cc='ccache cc'
  libavutil      52. 56.100 / 52. 56.100
  libavcodec     55. 44.100 / 55. 44.100
  libavformat    55. 22.100 / 55. 22.100
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     3. 91.100 /  3. 91.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Truncating packet of size 402024711 to 1663093
Input #0, ivf, from '/home/ux/samples/vp9/fuzzed0.ivf':
  Duration: N/A, start: 0.000001, bitrate: N/A
    Stream #0:0: Video: vp9 (vP[25]0 / 0x30195076), yuv420p, 256x244, 0k tbr, 0k tbn, 0k tbc
[New Thread 0x7ffff39a3700 (LWP 29856)]
[New Thread 0x7ffff31a2700 (LWP 29857)]
[New Thread 0x7ffff29a1700 (LWP 29858)]
[New Thread 0x7ffff21a0700 (LWP 29859)]
[New Thread 0x7ffff199f700 (LWP 29860)]
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf55.22.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 256x244, q=2-31, 200 kb/s, 90k tbn, 0k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 -> rawvideo)
Press [q] to stop, [?] for help

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff4a3aa20 in __memcpy_sse2_unaligned () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff4a3aa20 in __memcpy_sse2_unaligned () from /usr/lib/libc.so.6
#1  0x00000000009cd0b1 in vp9_alloc_frame (f=0x16c4b18, ctx=0x16c3540)
    at libavcodec/vp9.c:268
#2  vp9_decode_frame (ctx=0x16c3540, frame=0x16b6920, 
    got_frame=0x7fffffffe1fc, pkt=<optimized out>) at libavcodec/vp9.c:3512
#3  0x0000000000947e30 in avcodec_decode_video2 (avctx=0x16c3540, 
    picture=picture@entry=0x16b6920, 
    got_picture_ptr=got_picture_ptr@entry=0x7fffffffe1fc, 
    avpkt=avpkt@entry=0x7fffffffe480) at libavcodec/utils.c:2064
#4  0x00000000004787b3 in decode_video (ist=ist@entry=0x16c39a0, 
    pkt=pkt@entry=0x7fffffffe480, 
    got_output=got_output@entry=0x7fffffffe1fc) at ffmpeg.c:1695
#5  0x000000000046639a in output_packet (pkt=0x7fffffffe420, ist=0x16c39a0)
    at ffmpeg.c:1908
#6  process_input (file_index=<optimized out>) at ffmpeg.c:3216
#7  transcode_step () at ffmpeg.c:3312
#8  transcode () at ffmpeg.c:3364
#9  main (argc=<optimized out>, argv=<optimized out>) at ffmpeg.c:3544
(gdb) 
76bd878d959c79ef17ed90cc7d13dffea9327ee2 is the first bad commit
commit 76bd878d959c79ef17ed90cc7d13dffea9327ee2
Author: Ronald S. Bultje <rsbultje@gmail.com>
Date:   Sat Nov 30 09:08:54 2013 -0500

    vp9: add a 2-pass decoding mode, and add frame-mt support.
    
    For a random 1080p sample, decoding time went from 9.7sec (1 threads)
    to 6.0sec (2 threads) and 5.2sec (4 threads) in 2-pass decoding mode.
    I don't have any samples that use the parallelmode feature, but the
    gains should be higher.

Attachments (2)

fuzzed0.ivf (1.9 MB ) - added by Clément Bœsch 10 years ago.
fuzzed1.ivf (1.9 MB ) - added by Clément Bœsch 10 years ago.

Change History (7)

by Clément Bœsch, 10 years ago

Attachment: fuzzed0.ivf added

comment:1 by Carl Eugen Hoyos, 10 years ago

Keywords: crash SIGSEGV added

comment:2 by Carl Eugen Hoyos, 10 years ago

Reproduced by developer: set
Resolution: fixed
Status: newclosed

Fixed by Ronald in acafbb4d

comment:3 by Clément Bœsch, 10 years ago

Resolution: fixed
Status: closedreopened

Another crash, with same commit as regression.

☭ ./ffmpeg -threads auto -f ivf -c:v vp9 -i ~/samples/vp9/fuzzed1.ivf -f null -
ffmpeg version N-59315-gacafbb4 Copyright (c) 2000-2013 the FFmpeg developers
  built on Dec 24 2013 12:43:25 with gcc 4.8.2 (GCC)
  configuration: --enable-nonfree --enable-gpl --enable-libx264 --enable-libmp3lame --enable-x11grab --enable-libvorbis --samples=/home/ux/fate-samples --enable-libvpx --cpu=native --enable-libfaac --cc='ccache cc'
  libavutil      52. 59.100 / 52. 59.100
  libavcodec     55. 46.100 / 55. 46.100
  libavformat    55. 22.100 / 55. 22.100
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     4.  0.100 /  4.  0.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Input #0, ivf, from '/home/ux/samples/vp9/fuzzed1.ivf':
  Duration: 00:08:42.22, start: 342228469.800797, bitrate: 31 kb/s
    Stream #0:0: Video: vp9 (VP90 / 0x30395056), yuv420p, 320x180, 26.42 tbr, 1004 tbn, 1004 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf55.22.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x180, q=2-31, 200 kb/s, 90k tbn, 26.42 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 -> rawvideo)
Press [q] to stop, [?] for help
DTS -17592186044376, next:996 st:0 invalid dropping
PTS -17592186044376, next:996 invalid dropping st:0
DTS -17592186044336, next:1992 st:0 invalid dropping
PTS -17592186044336, next:1992 invalid dropping st:0
DTS -17592186044296, next:2988 st:0 invalid dropping
PTS -17592186044296, next:2988 invalid dropping st:0
DTS -17592186044256, next:3984 st:0 invalid dropping
PTS -17592186044256, next:3984 invalid dropping st:0
[null @ 0x1c65720] Encoder did not produce proper pts, making some up.
DTS -17592186043192, next:4980 st:0 invalid dropping
PTS -17592186043192, next:4980 invalid dropping st:0
DTS -17592186044176, next:5976 st:0 invalid dropping
PTS -17592186044176, next:5976 invalid dropping st:0
Input stream #0:0 frame changed from size:320x180 fmt:yuv420p to size:320x8372 fmt:yuv420p
[vp9 @ 0x1c69fa0] Invalid sync code
DTS -17592152489704, next:6972 st:0 invalid dropping
PTS -17592152489704, next:6972 invalid dropping st:0
zsh: segmentation fault (core dumped)  ./ffmpeg -threads auto -f ivf -c:v vp9 -i ~/samples/vp9/fuzzed1.ivf -f null -
☭ gdb --args ./ffmpeg_g -f ivf -c:v vp9 -i ~/samples/vp9/fuzzed1.ivf -f null -
GNU gdb (GDB) 7.6.2
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/ux/src/ffmpeg/ffmpeg_g...done.
(gdb) r
Starting program: /home/ux/src/ffmpeg/./ffmpeg_g -f ivf -c:v vp9 -i /home/ux/samples/vp9/fuzzed1.ivf -f null -
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
ffmpeg version N-59315-gacafbb4 Copyright (c) 2000-2013 the FFmpeg developers
  built on Dec 24 2013 12:43:25 with gcc 4.8.2 (GCC)
  configuration: --enable-nonfree --enable-gpl --enable-libx264 --enable-libmp3lame --enable-x11grab --enable-libvorbis --samples=/home/ux/fate-samples --enable-libvpx --cpu=native --enable-libfaac --cc='ccache cc'
  libavutil      52. 59.100 / 52. 59.100
  libavcodec     55. 46.100 / 55. 46.100
  libavformat    55. 22.100 / 55. 22.100
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     4.  0.100 /  4.  0.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Input #0, ivf, from '/home/ux/samples/vp9/fuzzed1.ivf':
  Duration: 00:08:42.22, start: 342228469.800797, bitrate: 31 kb/s
    Stream #0:0: Video: vp9 (VP90 / 0x30395056), yuv420p, 320x180, 26.42 tbr, 1004 tbn, 1004 tbc
[New Thread 0x7ffff3a99700 (LWP 16828)]
[New Thread 0x7ffff3298700 (LWP 16829)]
[New Thread 0x7ffff2a97700 (LWP 16830)]
[New Thread 0x7ffff2296700 (LWP 16831)]
[New Thread 0x7ffff1a95700 (LWP 16832)]
[New Thread 0x7ffff1294700 (LWP 16833)]
[New Thread 0x7ffff0a93700 (LWP 16834)]
[New Thread 0x7ffff0292700 (LWP 16835)]
[New Thread 0x7fffefa91700 (LWP 16836)]
[New Thread 0x7fffef290700 (LWP 16837)]
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf55.22.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x180, q=2-31, 200 kb/s, 90k tbn, 26.42 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 -> rawvideo)
Press [q] to stop, [?] for help
DTS -17592186044376, next:996 st:0 invalid dropping
PTS -17592186044376, next:996 invalid dropping st:0
DTS -17592186044336, next:1992 st:0 invalid dropping
PTS -17592186044336, next:1992 invalid dropping st:0
DTS -17592186044296, next:2988 st:0 invalid dropping
PTS -17592186044296, next:2988 invalid dropping st:0
DTS -17592186044256, next:3984 st:0 invalid dropping
PTS -17592186044256, next:3984 invalid dropping st:0
[null @ 0x1724500] Encoder did not produce proper pts, making some up.
DTS -17592186043192, next:4980 st:0 invalid dropping
PTS -17592186043192, next:4980 invalid dropping st:0
DTS -17592186044176, next:5976 st:0 invalid dropping
PTS -17592186044176, next:5976 invalid dropping st:0
Input stream #0:0 frame changed from size:320x180 fmt:yuv420p to size:320x8372 fmt:yuv420p
[vp9 @ 0x1728e00] Invalid sync code
[Thread 0x7ffff3a99700 (LWP 16828) exited]
[Thread 0x7ffff1a95700 (LWP 16832) exited]
[Thread 0x7ffff3298700 (LWP 16829) exited]
[Thread 0x7ffff2a97700 (LWP 16830) exited]
[Thread 0x7ffff2296700 (LWP 16831) exited]
[New Thread 0x7ffff1a95700 (LWP 16838)]
[New Thread 0x7ffff2296700 (LWP 16839)]
[New Thread 0x7ffff2a97700 (LWP 16840)]
[New Thread 0x7ffff3298700 (LWP 16841)]
[New Thread 0x7ffff3a99700 (LWP 16842)]
DTS -17592152489704, next:6972 st:0 invalid dropping
PTS -17592152489704, next:6972 invalid dropping st:0

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff0292700 (LWP 16835)]
0x00000000009d8945 in dc_top_4x4_c (dst=<optimized out>, stride=176, left=<optimized out>, 
    top=0x7fffe4023f20 '\200' <repeats 20 times>, "\201\201\201\201\201\201\201\201\202\202\202\202") at libavcodec/vp9dsp.c:380
380	    AV_WN32A(dst + stride * 1, dc);
(gdb) bt
#0  0x00000000009d8945 in dc_top_4x4_c (dst=<optimized out>, stride=176, left=<optimized out>, 
    top=0x7fffe4023f20 '\200' <repeats 20 times>, "\201\201\201\201\201\201\201\201\202\202\202\202") at libavcodec/vp9dsp.c:380
#1  0x00000000009c3408 in intra_recon (y_off=y_off@entry=67584, uv_off=uv_off@entry=16896, ctx=0x16aec20) at libavcodec/vp9.c:2288
#2  0x00000000009c7419 in decode_b (ctx=ctx@entry=0x16aec20, row=row@entry=24, col=col@entry=0, lflvl=lflvl@entry=0x7fffe4000d70, 
    yoff=yoff@entry=67584, uvoff=uvoff@entry=16896, bl=bl@entry=BL_64X64, bp=bp@entry=PARTITION_NONE) at libavcodec/vp9.c:2770
#3  0x00000000009d3354 in decode_sb (bl=BL_64X64, uvoff=16896, yoff=67584, lflvl=0x7fffe4000d70, col=0, row=24, ctx=0x16aec20)
    at libavcodec/vp9.c:2867
#4  vp9_decode_frame (ctx=<optimized out>, frame=<optimized out>, got_frame=<optimized out>, pkt=<optimized out>)
    at libavcodec/vp9.c:3637
#5  0x00000000008af8ea in frame_worker_thread (arg=0x17252f0) at libavcodec/pthread_frame.c:153
#6  0x00007ffff66aa0a2 in start_thread () from /usr/lib/libpthread.so.0
#7  0x00007ffff49d43dd in clone () from /usr/lib/libc.so.6
(gdb) 

by Clément Bœsch, 10 years ago

Attachment: fuzzed1.ivf added

in reply to:  3 comment:4 by Carl Eugen Hoyos, 10 years ago

Replying to ubitux:

Another crash, with same commit as regression.

Works fine here with the version you originally tested:

$ valgrind ffmpeg_g -f ivf -c:v vp9 -i fuzzed1.ivf -f null -
==29720== Memcheck, a memory error detector
==29720== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==29720== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==29720== Command: ffmpeg_g -f ivf -c:v vp9 -i fuzzed1.ivf -f null -
==29720==
ffmpeg version N-59315-gacafbb4 Copyright (c) 2000-2013 the FFmpeg developers
  built on Jan  8 2014 23:58:04 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      52. 59.100 / 52. 59.100
  libavcodec     55. 46.100 / 55. 46.100
  libavformat    55. 22.100 / 55. 22.100
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     4.  0.100 /  4.  0.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Input #0, ivf, from 'fuzzed1.ivf':
  Duration: 00:08:42.22, start: 342228469.800797, bitrate: 31 kb/s
    Stream #0:0: Video: vp9 (VP90 / 0x30395056), yuv420p, 320x180, 26.42 tbr, 1004 tbn, 1004 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf55.22.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x180, q=2-31, 200 kb/s, 90k tbn, 26.42 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 -> rawvideo)
Press [q] to stop, [?] for help
DTS -17592186044376, next:996 st:0 invalid dropping
PTS -17592186044376, next:996 invalid dropping st:0
DTS -17592186044336, next:1992 st:0 invalid dropping
PTS -17592186044336, next:1992 invalid dropping st:0
DTS -17592186044296, next:2988 st:0 invalid dropping
PTS -17592186044296, next:2988 invalid dropping st:0
DTS -17592186044256, next:3984 st:0 invalid dropping
PTS -17592186044256, next:3984 invalid dropping st:0
DTS -17592186043192, next:4980 st:0 invalid droppingbitrate=N/A
PTS -17592186043192, next:4980 invalid dropping st:0
DTS -17592186044176, next:5976 st:0 invalid droppingbitrate=N/A
PTS -17592186044176, next:5976 invalid dropping st:0
DTS -17592152489704, next:6972 st:0 invalid dropping
PTS -17592152489704, next:6972 invalid dropping st:0
[vp9 @ 0xc083b00] Invalid sync code
DTS -17592186044096, next:7968 st:0 invalid dropping
PTS -17592186044096, next:7968 invalid dropping st:0
[vp9 @ 0xc092700] Not all references are available
[null @ 0x773a660] Encoder did not produce proper pts, making some up.
Input stream #0:0 frame changed from size:320x180 fmt:yuv420p to size:320x8372 fmt:yuv420p
[vp9 @ 0xc0a12e0] Marker bit was set
Input stream #0:0 frame changed from size:320x8372 fmt:yuv420p to size:320x180 fmt:yuv420p
frame=    2 fps=0.7 q=0.0 Lsize=N/A time=00:00:00.07 bitrate=N/A    N/A
video:0kB audio:0kB subtitle:0 global headers:0kB muxing overhead -111.458333%
==29720==
==29720== HEAP SUMMARY:
==29720==     in use at exit: 80 bytes in 2 blocks
==29720==   total heap usage: 8,112 allocs, 8,110 frees, 19,936,539 bytes allocated
==29720==
==29720== LEAK SUMMARY:
==29720==    definitely lost: 0 bytes in 0 blocks
==29720==    indirectly lost: 0 bytes in 0 blocks
==29720==      possibly lost: 0 bytes in 0 blocks
==29720==    still reachable: 80 bytes in 2 blocks
==29720==         suppressed: 0 bytes in 0 blocks
==29720== Rerun with --leak-check=full to see details of leaked memory
==29720==
==29720== For counts of detected and suppressed errors, rerun with: -v
==29720== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)

comment:5 by Clément Bœsch, 10 years ago

Resolution: fixed
Status: reopenedclosed

All fuzzed samples where fixed by Ronald in various commits. (@cehoyos : probably reproducible with -threads 9 or something)

Note: See TracTickets for help on using tickets.