Opened 3 months ago

Closed 2 months ago

#7441 closed defect (fixed)

deshake filter crashes

Reported by: Chris Owned by:
Priority: important Component: avfilter
Version: git-master Keywords: deshake crash SIGSEGV regression
Cc: schmidt@wos.net Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:
ffmpeg deshake filter with blocksize > 20 crashes every time.

How to reproduce:

Starting program: ffmpeg.exe -i MVI_0288.MOV -filter:v deshake=blocksize=64 -loglevel debug MVI_0288_deshake.avi
[New Thread 12044.0x1640]
ffmpeg version N-91961-g5109c38162 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20180813
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 19.101 / 56. 19.101
  libavcodec     58. 30.100 / 58. 30.100
  libavformat    58. 18.101 / 58. 18.101
  libavdevice    58.  4.103 / 58.  4.103
  libavfilter     7. 32.100 /  7. 32.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
Splitting the commandline.
Reading option '-i' ... matched as input url with argument 'MVI_0288.MOV'.
Reading option '-filter:v' ... matched as option 'filter' (set stream filtergraph) with argument 'deshake=blocksize=64'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option 'MVI_0288_deshake.avi' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url MVI_0288.MOV.
Successfully parsed a group of options.
Opening an input file: MVI_0288.MOV.
[NULL @ 0000000001d17080] Opening 'MVI_0288.MOV' for reading
[file @ 0000000001d17a80] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000000001d17080] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000000001d17080] ISO: File Type Major Brand: qt
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000000001d17080] Unknown dref type 0x73696c61 size 12
    Last message repeated 1 times
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000000001d17080] Before avformat_find_stream_info() pos: 221312516 bytes read:69432 seeks:2 nb_streams:2
[h264 @ 0000000001d2a0c0] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0000000001d2a0c0] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0000000001d2a0c0] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0000000001d2a0c0] Format yuvj420p chosen by get_format().
[h264 @ 0000000001d2a0c0] Reinit context to 1920x1088, pix_fmt: yuvj420p
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000000001d17080] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000000001d17080] After avformat_find_stream_info() pos: 459232 bytes read:430352 seeks:3 frames:46
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'MVI_0288.MOV':
  Metadata:
    major_brand     : qt
    minor_version   : 537331968
    compatible_brands: qt  CAEP
    com.apple.quicktime.make: Canon
    com.apple.quicktime.model: Canon EOS 700D
    com.apple.quicktime.rating.user: 0.000000
    creation_time   : 2018-09-11T12:44:35.000000Z
  Duration: 00:00:38.80, start: 0.000000, bitrate: 45631 kb/s
    Stream #0:0(eng), 1, 1/25000: Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuvj420p(pc, bt709, left), 1920x1080 (1920x1088), 0/1, 44074 kb/s, 25 fps, 25 tbr, 25k tbn, 50k tbc (default)
    Metadata:
      creation_time   : 2018-09-11T12:44:35.000000Z
    Stream #0:1(eng), 45, 1/48000: Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)
    Metadata:
      creation_time   : 2018-09-11T12:44:35.000000Z
Successfully opened the file.
Parsing a group of options: output url MVI_0288_deshake.avi.
Applying option filter:v (set stream filtergraph) with argument deshake=blocksize=64.
Successfully parsed a group of options.
Opening an output file: MVI_0288_deshake.avi.
[file @ 0000000001db8300] Setting default whitelist 'file,crypto'
Successfully opened the file.
detected 4 logical cores
[h264 @ 0000000001dba940] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0000000001dba940] nal_unit_type: 8(PPS), nal_ref_idc: 3
[New Thread 12044.0x52e8]
[New Thread 12044.0x3a40]
[New Thread 12044.0x202c]
[New Thread 12044.0x2f14]
[New Thread 12044.0x5070]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
  Stream #0:1 -> #0:1 (pcm_s16le (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[New Thread 12044.0x50fc]
[New Thread 12044.0x4fb0]
[New Thread 12044.0x40d8]
[New Thread 12044.0x52cc]
[graph_1_in_0_1 @ 0000000002993f00] Setting 'time_base' to value '1/48000'
[graph_1_in_0_1 @ 0000000002993f00] Setting 'sample_rate' to value '48000'
[graph_1_in_0_1 @ 0000000002993f00] Setting 'sample_fmt' to value 's16'
[graph_1_in_0_1 @ 0000000002993f00] Setting 'channel_layout' to value '0x3'
[graph_1_in_0_1 @ 0000000002993f00] tb:1/48000 samplefmt:s16 samplerate:48000 chlayout:0x3
[format_out_0_1 @ 0000000002994a40] Setting 'sample_fmts' to value 's32p|fltp|s16p'
[format_out_0_1 @ 0000000002994a40] Setting 'sample_rates' to value '44100|48000|32000|22050|24000|16000|11025|12000|8000'
[format_out_0_1 @ 0000000002994a40] Setting 'channel_layouts' to value '0x4|0x3'
[format_out_0_1 @ 0000000002994a40] auto-inserting filter 'auto_resampler_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_1'
[AVFilterGraph @ 0000000002992980] query_formats: 4 queried, 6 merged, 3 already done, 0 delayed
[auto_resampler_0 @ 0000000002995840] picking s16p out of 3 ref:s16
[auto_resampler_0 @ 0000000002995840] [SWR @ 0000000002996840] Using s16p internally between filters
[auto_resampler_0 @ 0000000002995840] ch:2 chl:stereo fmt:s16 r:48000Hz -> ch:2 chl:stereo fmt:s16p r:48000Hz
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
    Last message repeated 45 times
[h264 @ 0000000001dba940] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0000000001dba940] Format yuvj420p chosen by get_format().
[h264 @ 0000000001dba940] Reinit context to 1920x1088, pix_fmt: yuvj420p
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0000000001dbf600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 000000000293a740] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0000000002957040] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
[New Thread 12044.0x2c64]
[h264 @ 0000000002973a80] [New Thread 12044.0xd48]
nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
[New Thread 12044.0x4ac4]
[New Thread 12044.0xf14]
[Parsed_deshake_0 @ 00000000055ca400] Setting 'blocksize' to value '64'
[Parsed_deshake_0 @ 00000000055ca400] cx: -1, cy: -1, cw: -1, ch: -1, rx: 16, ry: 16, edge: 3 blocksize: 64 contrast: 125 search: 0
[graph 0 input from stream 0:0 @ 00000000058bd580] Setting 'video_size' to value '1920x1080'
[graph 0 input from stream 0:0 @ 00000000058bd580] Setting 'pix_fmt' to value '12'
[graph 0 input from stream 0:0 @ 00000000058bd580] Setting 'time_base' to value '1/25000'
[graph 0 input from stream 0:0 @ 00000000058bd580] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 00000000058bd580] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 00000000058bd580] Setting 'frame_rate' to value '25/1'
[graph 0 input from stream 0:0 @ 00000000058bd580] w:1920 h:1080 pixfmt:yuvj420p tb:1/25000 fr:25/1 sar:0/1 sws_param:flags=2
[format @ 00000000058c12c0] Setting 'pix_fmts' to value 'yuv420p'
[auto_scaler_0 @ 00000000058c2a80] Setting 'flags' to value 'bicubic'
[auto_scaler_0 @ 00000000058c2a80] w:iw h:ih flags:'bicubic' interl:0
[format @ 00000000058c12c0] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_deshake_0' and the filter 'format'
[AVFilterGraph @ 00000000055cb300] query_formats: 4 queried, 2 merged, 1 already done, 0 delayed
[swscaler @ 00000000058c2fc0] deprecated pixel format used, make sure you did set range correctly
[auto_scaler_0 @ 00000000058c2a80] w:1920 h:1080 fmt:yuvj420p sar:0/1 -> w:1920 h:1080 fmt:yuv420p sar:0/1 flags:0x4
[New Thread 12044.0x4378]
[New Thread 12044.0x3f2c]
[New Thread 12044.0x3b94]
[New Thread 12044.0x1fd8]
[mpeg4 @ 0000000001d1ac80] intra_quant_bias = 0 inter_quant_bias = -64
[avi @ 0000000001d19100] reserve_index_space:0 master_index_max_size:256
[avi @ 0000000001d19100] duration_est:38.800, filesize_est:0.0GiB, master_index_max_size:256
Output #0, avi, to 'MVI_0288_deshake.avi':
  Metadata:
    major_brand     : qt
    minor_version   : 537331968
    compatible_brands: qt  CAEP
    com.apple.quicktime.make: Canon
    com.apple.quicktime.model: Canon EOS 700D
    com.apple.quicktime.rating.user: 0.000000
    ISFT            : Lavf58.18.101
    Stream #0:0(eng), 0, 1/25: Video: mpeg4, 1 reference frame (FMP4 / 0x34504D46), yuv420p(left), 1920x1080, 0/1, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      creation_time   : 2018-09-11T12:44:35.000000Z
      encoder         : Lavc58.30.100 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
    Stream #0:1(eng), 0, 3/125: Audio: mp3 (libmp3lame) (U[0][0][0] / 0x0055), 48000 Hz, stereo, s16p, delay 1105 (default)
    Metadata:
      creation_time   : 2018-09-11T12:44:35.000000Z
      encoder         : Lavc58.30.100 libmp3lame
Clipping frame in rate conversion by 0.000008
[h264 @ 0000000001dba940] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
[h264 @ 0000000001dbf600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
frame=    3 fps=0.0 q=2.0 size=     266kB time=00:00:00.86 bitrate=2519.1kbits/s speed= 1.4x
Program received signal SIGSEGV, Segmentation fault.
0x000007fee28adca0 in avfilter_transform () from avfilter-7.dll

Change History (3)

comment:1 Changed 3 months ago by Chris

  • Cc schmidt@wos.net added
  • Version changed from unspecified to git-master

comment:2 Changed 3 months ago by cehoyos

  • Component changed from ffmpeg to avfilter
  • Keywords deshake crash SIGSEGV regression added
  • Priority changed from normal to important
  • Reproduced by developer set
  • Status changed from new to open

Regression since 8f86e6623811f7713d5e72c13797e20fffb3df62

$ ffmpeg -f lavfi -i testsrc=hd1080 -t 10 -pix_fmt yuv420p out.mov
(gdb) r -i out.mov -filter:v deshake=blocksize=64 -f null -t 1 -
Starting program: ffmpeg_g -i out.mov -filter:v deshake=blocksize=64 -f null -t 1 -
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ffmpeg version N-91965-gb0cfb2c Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.4.0 (GCC)
  configuration: --enable-gpl --enable-gnutls --enable-libxml2
  libavutil      56. 19.101 / 56. 19.101
  libavcodec     58. 30.100 / 58. 30.100
  libavformat    58. 18.101 / 58. 18.101
  libavdevice    58.  4.103 / 58.  4.103
  libavfilter     7. 32.100 /  7. 32.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf55.48.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 166 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 163 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc55.69.100 libx264
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf58.18.101
    Stream #0:0(eng): Video: wrapped_avframe, yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.30.100 wrapped_avframe
frame=   10 fps=8.9 q=-0.0 size=N/A time=00:00:00.40 bitrate=N/A speed=0.355x
Program received signal SIGSEGV, Segmentation fault.
block_contrast (blocksize=32, stride=1920, y=16, x=16, src=0x7fffc5bfe040 '\020' <repeats 200 times>...)
    at libavfilter/vf_deshake.c:200
200                 if (src[pos] < lowest)
(gdb) bt
#0  block_contrast (blocksize=32, stride=1920, y=16, x=16, src=0x7fffc5bfe040 '\020' <repeats 200 times>...)
    at libavfilter/vf_deshake.c:200
#1  find_motion (deshake=deshake@entry=0x28a7c80, src1=0x7fffee20f040 '\020' <repeats 200 times>...,
    src2=0x7fffc5bfe040 '\020' <repeats 200 times>..., width=1920, height=1080, stride=1920, t=0x7fffffffcdc0)
    at libavfilter/vf_deshake.c:263
#2  0x000000000050d4b3 in filter_frame (link=link@entry=0x2121a80, in=<optimized out>) at libavfilter/vf_deshake.c:456
#3  0x00000000004c4b5a in ff_filter_frame_framed (frame=<optimized out>, link=0x2121a80) at libavfilter/avfilter.c:1071
#4  ff_filter_frame_to_filter (link=0x2121a80) at libavfilter/avfilter.c:1219
#5  ff_filter_activate_default (filter=<optimized out>) at libavfilter/avfilter.c:1268
#6  ff_filter_activate (filter=<optimized out>) at libavfilter/avfilter.c:1429
#7  0x00000000004c82fc in ff_filter_graph_run_once (graph=graph@entry=0x2122dc0) at libavfilter/avfiltergraph.c:1454
#8  0x00000000004c936c in push_frame (graph=0x2122dc0) at libavfilter/buffersrc.c:181
#9  av_buffersrc_add_frame_internal (ctx=ctx@entry=0x2122700, frame=frame@entry=0x22339c0, flags=flags@entry=4)
    at libavfilter/buffersrc.c:255
#10 0x00000000004c981d in av_buffersrc_add_frame_flags (ctx=0x2122700, frame=frame@entry=0x22339c0, flags=flags@entry=4)
    at libavfilter/buffersrc.c:164
#11 0x00000000004a0697 in ifilter_send_frame (frame=0x22339c0, ifilter=0x2100340) at fftools/ffmpeg.c:2196
#12 send_frame_to_filters (ist=ist@entry=0x20f5fc0, decoded_frame=decoded_frame@entry=0x22339c0) at fftools/ffmpeg.c:2270
#13 0x00000000004a0f88 in decode_video (ist=ist@entry=0x20f5fc0, pkt=pkt@entry=0x7fffffffd300, got_output=<optimized out>,
    duration_pts=<optimized out>, eof=<optimized out>, decode_failed=<optimized out>) at fftools/ffmpeg.c:2471
#14 0x00000000004a2cfb in process_input_packet (ist=0x20f5fc0, pkt=0x7fffffffd720, no_eof=0) at fftools/ffmpeg.c:2625
#15 0x00000000004a48d7 in process_input (file_index=<optimized out>) at fftools/ffmpeg.c:4507
#16 transcode_step () at fftools/ffmpeg.c:4627
#17 transcode () at fftools/ffmpeg.c:4681
#18 0x0000000000482c23 in main (argc=<optimized out>, argv=0x7fffffffdcf8) at fftools/ffmpeg.c:4888
(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0x50c8a0 to 0x50c8e0:
   0x000000000050c8a0 <find_motion+528>:        repz sub $0xa,%rbx
   0x000000000050c8a5 <find_motion+533>:        mov    %rdi,0x18(%rsp)
   0x000000000050c8aa <find_motion+538>:        mov    0x70(%rsp),%rdi
   0x000000000050c8af <find_motion+543>:        mov    %r13,0x70(%rsp)
   0x000000000050c8b4 <find_motion+548>:        mov    %rsi,%r13
   0x000000000050c8b7 <find_motion+551>:        nopw   0x0(%rax,%rax,1)
=> 0x000000000050c8c0 <find_motion+560>:        movzbl (%rdi),%ecx
   0x000000000050c8c3 <find_motion+563>:        cmp    %edx,%ecx
   0x000000000050c8c5 <find_motion+565>:        jl     0x50c8ce <find_motion+574>
   0x000000000050c8c7 <find_motion+567>:        cmp    %ecx,%eax
   0x000000000050c8c9 <find_motion+569>:        cmovl  %ecx,%eax
   0x000000000050c8cc <find_motion+572>:        mov    %edx,%ecx
   0x000000000050c8ce <find_motion+574>:        mov    %rdi,%rdx
   0x000000000050c8d1 <find_motion+577>:        sub    0x8(%rsp),%rdx
   0x000000000050c8d6 <find_motion+582>:        movzbl (%rdx,%r13,1),%esi
   0x000000000050c8db <find_motion+587>:        cmp    %esi,%ecx
   0x000000000050c8dd <find_motion+589>:        jg     0x50c8e6 <find_motion+598>
   0x000000000050c8df <find_motion+591>:        cmp    %esi,%eax
End of assembler dump.
(gdb) info register
rax            0x10     16
rbx            0x6      6
rcx            0x10     16
rdx            0x10     16
rsi            0x1      1
rdi            0x7fffc5bfd8d0   140736511072464
rbp            0xe      0xe
rsp            0x7fffffffcc80   0x7fffffffcc80
r8             0x11     17
r9             0x3      3
r10            0x4      4
r11            0x5      5
r12            0x8      8
r13            0xf      15
r14            0x9      9
r15            0xa      10
rip            0x50c8c0 0x50c8c0 <find_motion+560>
eflags         0x10283  [ CF SF IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0

comment:3 Changed 2 months ago by cehoyos

  • Resolution set to fixed
  • Status changed from open to closed
Note: See TracTickets for help on using tickets.