FFmpeg 3.3+3.4-dev crash when scaling to same dimensions with scale_npp

[Oleg Larionov]

Summary of the bug:
I am trying to transcode and scale video from mp4+h264 to mp4+h264 using h264_nvenc encoder, h264_cuvid decoder and scale_npp filter. FFmpeg crashes with segfault if source and target dimensions (from scale_npp) are same.

How to reproduce:

% ffmpeg -y -hwaccel cuvid -c:v h264_cuvid -i bug_input.mp4 -c:v h264_nvenc -vf "scale_npp=w=1280:h=720" -vb 4096k -an -sn -f mp4 out.mp4
ffmpeg version N-85548-g3390a2b Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-4)
  configuration: --prefix=/opt/ffmpeg_gpu --enable-static --disable-shared --disable-ffplay --disable-ffserver --disable-network --disable-doc --disable-indev=jack --enable-debug --enable-nonfree --enable-version3 --enable-gpl --enable-libfdk-aac --enable-gray --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-frei0r --enable-ladspa --enable-libass --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopenjpeg --enable-libopus --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-nvenc --enable-cuvid --enable-cuda --enable-libnpp --shlibdir=/usr/lib64 --extra-cflags=-I../nv_sdk --extra-ldflags=-L../nv_sdk --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 --extra-cflags=-I/usr/local/include/opus --extra-ldflags=-L/usr/local/lib/
  libavutil      55. 61.100 / 55. 61.100
  libavcodec     57. 92.100 / 57. 92.100
  libavformat    57. 72.101 / 57. 72.101
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 84.101 /  6. 84.101
  libswscale      4.  7.101 /  4.  7.101
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bug_input.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
  Duration: 00:00:05.01, start: 0.000000, bitrate: 17393 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 17390 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_cuvid) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
Segmentation fault

Input file is attached via videolan file uploader.

[Oleg Larionov]

Input file ffmpeg report

[Oleg Larionov]

GDB segfault output

[kikobyte]

Happens to me as well, when source and destination dimensions match for scale_npp.

Debug 3.3.5:

Program received signal SIGSEGV, Segmentation fault.
nppscale_filter_frame (link=link@entry=0x3b7f8a0, in=0x3b82620) at libavfilter/vf_scale_npp.c:496
496	    AVCUDADeviceContext *device_hwctx = frames_ctx->device_ctx->hwctx;
(gdb) where
#0  nppscale_filter_frame (link=link@entry=0x3b7f8a0, in=0x3b82620) at libavfilter/vf_scale_npp.c:496
#1  0x00000000004c0fb2 in ff_filter_frame_framed (frame=0x3b82620, link=0x3b7f8a0) at libavfilter/avfilter.c:1116
#2  ff_filter_frame_to_filter (link=0x3b7f8a0) at libavfilter/avfilter.c:1264
#3  ff_filter_activate_default (filter=<optimized out>) at libavfilter/avfilter.c:1315
#4  ff_filter_activate (filter=<optimized out>) at libavfilter/avfilter.c:1476
#5  0x00000000004c4604 in ff_filter_graph_run_once (graph=graph@entry=0x287b060) at libavfilter/avfiltergraph.c:1449
#6  0x00000000004c54d0 in push_frame (graph=0x287b060) at libavfilter/buffersrc.c:181
#7  av_buffersrc_add_frame_internal (ctx=ctx@entry=0x3b7fb80, frame=frame@entry=0x2797dc0, flags=flags@entry=4) at libavfilter/buffersrc.c:262
#8  0x00000000004c5b6d in av_buffersrc_add_frame_flags (ctx=0x3b7fb80, frame=frame@entry=0x2797dc0, flags=flags@entry=4) at libavfilter/buffersrc.c:164
#9  0x00000000004a07a5 in ifilter_send_frame (frame=0x2797dc0, ifilter=0x2012e20) at ffmpeg.c:2216
#10 send_frame_to_filters (ist=ist@entry=0x201c860, decoded_frame=decoded_frame@entry=0x2797dc0) at ffmpeg.c:2294
#11 0x00000000004a11da in decode_video (ist=0x201c860, pkt=<optimized out>, got_output=<optimized out>, eof=<optimized out>, decode_failed=<optimized out>) at ffmpeg.c:2494
#12 0x00000000004a3648 in process_input_packet (ist=0x201c860, pkt=0x7fffffffd9b0, no_eof=0) at ffmpeg.c:2643
#13 0x0000000000483e27 in process_input (file_index=<optimized out>) at ffmpeg.c:4417
#14 transcode_step () at ffmpeg.c:4528
#15 transcode () at ffmpeg.c:4582
#16 main (argc=2, argv=0x201c860) at ffmpeg.c:4787

Seems that the output's hw_frames_ctx was not properly initialized:

(gdb) info locals
ctx = 0x3b7e7a0
s = 0x3b7e8a0
outlink = 0x3b804e0
frames_ctx = <error reading variable frames_ctx (Cannot access memory at address 0x8)>

494     AVFilterLink             *outlink = ctx->outputs[0];
495     AVHWFramesContext     *frames_ctx = (AVHWFramesContext*)outlink->hw_frames_ctx->data;

(gdb) print *(AVFilterLink*)ctx->outputs[0]
$2 = {src = 0x3b7e7a0, srcpad = 0x3b7ebe0, dst = 0x3b80300, dstpad = 0x3b7e5c0, type = AVMEDIA_TYPE_VIDEO, w = 4096, h = 2048, sample_aspect_ratio = {num = 1, den = 1}, channel_layout = 0, sample_rate = 0, format = 129, time_base = {
    num = 1, den = 15360}, in_formats = 0x0, out_formats = 0x0, in_samplerates = 0x0, out_samplerates = 0x0, in_channel_layouts = 0x0, out_channel_layouts = 0x0, request_samples = 0, init_state = AVLINK_INIT, graph = 0x287b060, 
  current_pts = -9223372036854775808, current_pts_us = -9223372036854775808, age_index = -1, frame_rate = {num = 30, den = 1}, partial_buf = 0x0, partial_buf_size = 0, min_samples = 0, max_samples = 0, channels = 0, flags = 0, 
  frame_count_in = 0, frame_count_out = 0, frame_pool = 0x0, frame_wanted_out = 0, hw_frames_ctx = 0x0, 
  reserved = "\370\005\270\003\000\000\000\000\001",