Linux ffmpeg always segfaults using NVIDIA hw decoders/encoders

[bubbleguuum]

Summary: using NVIDIA hardware decoding features always crashes ffmpeg with a Segmentation fault.

How to reprod:

• Required: a NVIDIA card (starting with Geforce 600 series) and working NVIDIA binary drivers (reproduced with v375.66, v378.13 and the latest at this time: 381.22)
• ffmpeg compiled with NVIDIA support (at least with the h264_nvdec decoder)

This can be quickly reproduced using the static ffmpeg 3.3.1 (or git) build from ​https://johnvansickle.com/ffmpeg/, with either the 32 bit or 64 bit version.

Crashing output, trying to encode with h264_nvenc:

ffmpeg -v debug -i http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv -map 0 -c:0 h264_nvenc -f null /dev/null
ffmpeg version 3.3.1-static http://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.1 (Debian 5.4.1-8) 20170304
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-5 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gray --enable-libass --enable-libfreetype --enable-libfribidi --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libzimg
  libavutil      55. 58.100 / 55. 58.100
  libavcodec     57. 89.100 / 57. 89.100
  libavformat    57. 71.100 / 57. 71.100
  libavdevice    57.  6.100 / 57.  6.100
  libavfilter     6. 82.100 /  6. 82.100
  libswscale      4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
  libpostproc    54.  5.100 / 54.  5.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'debug'.
Reading option '-i' ... matched as input url with argument 'http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0'.
Reading option '-c:0' ... matched as option 'c' (codec name) with argument 'h264_nvenc'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'null'.
Reading option '/dev/null' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv.
Successfully parsed a group of options.
Opening an input file: http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv.
[http @ 0x3fde720] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[http @ 0x3fde720] request: GET /bubbleupnpserver/transcode_test/tos_sample_h264.mkv HTTP/1.1
User-Agent: Lavf/57.71.100
Accept: */*
Range: bytes=0-
Connection: close
Host: bubblesoftapps.com
Icy-MetaData: 1


[matroska,webm @ 0x3fddec0] Format matroska,webm probed with size=2048 and score=100
st:0 removing common factor 1000000 from timebase
[matroska,webm @ 0x3fddec0] Before avformat_find_stream_info() pos: 843 bytes read:3779 seeks:0 nb_streams:1
[h264 @ 0x3fe5300] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x3fe5300] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x3fe5300] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x3fe5300] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x3fe5300] user data:"x264 - core 148 r2762 90a61ec - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00"
[h264 @ 0x3fe5300] nal_unit_type: 6, nal_ref_idc: 0
[h264 @ 0x3fe5300] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0x3fe5300] user data:"x264 - core 148 r2762 90a61ec - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00"
[h264 @ 0x3fe5300] Reinit context to 1920x800, pix_fmt: yuv420p
[h264 @ 0x3fe5300] no picture 
[matroska,webm @ 0x3fddec0] All info found
[matroska,webm @ 0x3fddec0] After avformat_find_stream_info() pos: 89449 bytes read:91521 seeks:0 frames:4
Input #0, matroska,webm, from 'http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv':
  Metadata:
    COMPATIBLE_BRANDS: mp42isomavc1
    MAJOR_BRAND     : mp42
    MINOR_VERSION   : 0
    ENCODER         : Lavf57.72.101
  Duration: 00:00:02.00, start: 0.000000, bitrate: 1359 kb/s
    Stream #0:0, 4, 1/1000: Video: h264 (High), 1 reference frame, yuv420p(progressive, left), 1920x800 [SAR 1:1 DAR 12:5], 0/1, 24 fps, 24 tbr, 1k tbn, 48 tbc (default)
    Metadata:
      ENCODER         : Lavc57.93.100 libx264
      DURATION        : 00:00:02.000000000
Successfully opened the file.
Parsing a group of options: output url /dev/null.
Applying option map (set input stream mapping) with argument 0.
Applying option c:0 (codec name) with argument h264_nvenc.
Applying option f (force format) with argument null.
Successfully parsed a group of options.
Opening an output file: /dev/null.
Successfully opened the file.
detected 8 logical cores
[h264 @ 0x3fe9840] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x3fe9840] nal_unit_type: 8, nal_ref_idc: 3
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
    Last message repeated 1 times
[h264 @ 0x3fe9840] nal_unit_type: 6, nal_ref_idc: 0
[h264 @ 0x3fe9840] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0x3fe9840] user data:"x264 - core 148 r2762 90a61ec - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00"
[h264 @ 0x3fe9840] Reinit context to 1920x800, pix_fmt: yuv420p
[h264 @ 0x3fe9840] no picture 
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3ffe2e0] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x3ffe2e0] no picture 
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x4005a20] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x40933c0] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x40bfb80] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x40db720] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x40f72c0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x4113160] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x412f000] nal_unit_type: 1, nal_ref_idc: 0
[h264 @ 0x3fe9840] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3ffe2e0] nal_unit_type: 1, nal_ref_idc: 2
[graph 0 input from stream 0:0 @ 0x40b8520] Setting 'video_size' to value '1920x800'
[graph 0 input from stream 0:0 @ 0x40b8520] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x40b8520] Setting 'time_base' to value '1/1000'
[graph 0 input from stream 0:0 @ 0x40b8520] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x40b8520] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x40b8520] Setting 'frame_rate' to value '24/1'
[graph 0 input from stream 0:0 @ 0x40b8520] w:1920 h:800 pixfmt:yuv420p tb:1/1000 fr:24/1 sar:1/1 sws_param:flags=2
[format @ 0x40b8ca0] compat: called with args=[yuv420p|nv12|p010le|yuv444p|yuv444p16le|bgr0|rgb0|cuda]
[format @ 0x40b8ca0] Setting 'pix_fmts' to value 'yuv420p|nv12|p010le|yuv444p|yuv444p16le|bgr0|rgb0|cuda'
[AVFilterGraph @ 0x46254c0] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
Segmentation fault (core dumped)

strace (see attached file) shows that the crash happens shortly after /usr/lib/x86_64-linux-gnu/libcuda.so.1 and /usr/lib/x86_64-linux-gnu/libnvidia-fatbinaryloader.so.378.13 are dynamically loaded.

Using -hwaccel cuvid and/or *_cuvid encoders also crash identically.

This issue is not GPU specific as I know someone else got it with a different NVIDIA GPU than me.

This crash is also mentioned in one of the last comment of this gist page, using a custom build of ffmpeg:
​https://gist.github.com/Brainiarc7/988473b79fd5c8f0db54b92ebb47387a

Could this issue be related to issue #6405 ?

Could this issue be specific to static ffmpeg builds ?

In any case, this issue makes using NVIDIA acceleration impossible currently on Linux.

[Timo R.]

Your exact comandline works perfectly for me on both 3.3 and master.
Please create a backtrace with gdb, strace is not overly useful.

% ./ffmpeg -v debug -i http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv -map 0 -c:0 h264_nvenc -f null /dev/null
ffmpeg version n3.3.1-9-g1cbeb16187 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Gentoo 5.4.0-r3 p1.3, pie-0.6.5)
  configuration: --enable-nonfree --extra-cflags=-I/opt/cuda/include --extra-ldflags=-L/opt/cuda/lib64 --enable-libnpp
  libavutil      55. 58.100 / 55. 58.100
  libavcodec     57. 89.100 / 57. 89.100
  libavformat    57. 71.100 / 57. 71.100
  libavdevice    57.  6.100 / 57.  6.100
  libavfilter     6. 82.100 /  6. 82.100
  libswscale      4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'debug'.
Reading option '-i' ... matched as input url with argument 'http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0'.
Reading option '-c:0' ... matched as option 'c' (codec name) with argument 'h264_nvenc'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'null'.
Reading option '-' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv.
Successfully parsed a group of options.
Opening an input file: http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv.
[http @ 0x34437c0] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[http @ 0x34437c0] request: GET /bubbleupnpserver/transcode_test/tos_sample_h264.mkv HTTP/1.1
User-Agent: Lavf/57.71.100
Accept: */*
Range: bytes=0-
Connection: close
Host: bubblesoftapps.com
Icy-MetaData: 1


[matroska,webm @ 0x34430c0] Format matroska,webm probed with size=2048 and score=100
st:0 removing common factor 1000000 from timebase
[matroska,webm @ 0x34430c0] Before avformat_find_stream_info() pos: 843 bytes read:2579 seeks:0 nb_streams:1
[h264 @ 0x3449120] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x3449120] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x3449120] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x3449120] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x3449120] user data:"x264 - core 148 r2762 90a61ec - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00"
[h264 @ 0x3449120] nal_unit_type: 6, nal_ref_idc: 0
[h264 @ 0x3449120] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0x3449120] user data:"x264 - core 148 r2762 90a61ec - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00"
[h264 @ 0x3449120] Reinit context to 1920x800, pix_fmt: yuv420p
[h264 @ 0x3449120] no picture
[matroska,webm @ 0x34430c0] All info found
[matroska,webm @ 0x34430c0] After avformat_find_stream_info() pos: 89449 bytes read:92355 seeks:0 frames:4
Input #0, matroska,webm, from 'http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv':
  Metadata:
    COMPATIBLE_BRANDS: mp42isomavc1
    MAJOR_BRAND     : mp42
    MINOR_VERSION   : 0
    ENCODER         : Lavf57.72.101
  Duration: 00:00:02.00, start: 0.000000, bitrate: 1359 kb/s
    Stream #0:0, 4, 1/1000: Video: h264 (High), 1 reference frame, yuv420p(progressive, left), 1920x800 [SAR 1:1 DAR 12:5], 0/1, 24 fps, 24 tbr, 1k tbn, 48 tbc (default)
    Metadata:
      ENCODER         : Lavc57.93.100 libx264
      DURATION        : 00:00:02.000000000
Successfully opened the file.
Parsing a group of options: output url -.
Applying option map (set input stream mapping) with argument 0.
Applying option c:0 (codec name) with argument h264_nvenc.
Applying option f (force format) with argument null.
Successfully parsed a group of options.
Opening an output file: -.
Successfully opened the file.
detected 4 logical cores
[h264 @ 0x348df40] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x348df40] nal_unit_type: 8, nal_ref_idc: 3
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
    Last message repeated 1 times
[h264 @ 0x348df40] nal_unit_type: 6, nal_ref_idc: 0
[h264 @ 0x348df40] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0x348df40] user data:"x264 - core 148 r2762 90a61ec - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00"
[h264 @ 0x348df40] Reinit context to 1920x800, pix_fmt: yuv420p
[h264 @ 0x348df40] no picture
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3462320] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x3462320] no picture
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3469ac0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x34f73e0] nal_unit_type: 1, nal_ref_idc: 0
[h264 @ 0x3523bc0] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x348df40] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3462320] nal_unit_type: 1, nal_ref_idc: 2
[graph 0 input from stream 0:0 @ 0x37aaf00] Setting 'video_size' to value '1920x800'
[graph 0 input from stream 0:0 @ 0x37aaf00] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x37aaf00] Setting 'time_base' to value '1/1000'
[graph 0 input from stream 0:0 @ 0x37aaf00] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x37aaf00] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x37aaf00] Setting 'frame_rate' to value '24/1'
[graph 0 input from stream 0:0 @ 0x37aaf00] w:1920 h:800 pixfmt:yuv420p tb:1/1000 fr:24/1 sar:1/1 sws_param:flags=2
[format @ 0x37aace0] compat: called with args=[yuv420p|nv12|p010le|yuv444p|yuv444p16le|bgr0|rgb0|cuda]
[format @ 0x37aace0] Setting 'pix_fmts' to value 'yuv420p|nv12|p010le|yuv444p|yuv444p16le|bgr0|rgb0|cuda'
[AVFilterGraph @ 0x37a9fa0] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[h264_nvenc @ 0x348cce0] Loaded Nvenc version 8.0
[h264_nvenc @ 0x348cce0] Nvenc initialized successfully
[h264_nvenc @ 0x348cce0] 1 CUDA capable devices found
[h264_nvenc @ 0x348cce0] [ GPU #0 - < GeForce GTX 1050 > has Compute SM 6.1 ]
[h264_nvenc @ 0x348cce0] supports NVENC
Output #0, null, to 'pipe:':
  Metadata:
    COMPATIBLE_BRANDS: mp42isomavc1
    MAJOR_BRAND     : mp42
    MINOR_VERSION   : 0
    encoder         : Lavf57.71.100
    Stream #0:0, 0, 1/24: Video: h264 (h264_nvenc) (Main), 1 reference frame, yuv420p(left), 1920x800 [SAR 1:1 DAR 12:5], 0/1, q=-1--1, 2000 kb/s, 24 fps, 24 tbn, 24 tbc (default)
    Metadata:
      DURATION        : 00:00:02.000000000
      encoder         : Lavc57.89.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: -1
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3469ac0] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x34f73e0] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3523bc0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x348df40] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3462320] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
    Last message repeated 1 times
[h264 @ 0x3469ac0] nal_unit_type: 1, nal_ref_idc: 0
[h264 @ 0x34f73e0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3523bc0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x348df40] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3462320] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3469ac0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x34f73e0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3523bc0] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
    Last message repeated 1 times
[h264 @ 0x348df40] nal_unit_type: 1, nal_ref_idc: 0
[h264 @ 0x3462320] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
    Last message repeated 1 times
[h264 @ 0x3469ac0] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x34f73e0] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3523bc0] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x348df40] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3462320] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3469ac0] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
    Last message repeated 1 times
[h264 @ 0x34f73e0] nal_unit_type: 1, nal_ref_idc: 0
[h264 @ 0x3523bc0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x348df40] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3462320] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3469ac0] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x34f73e0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3523bc0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x348df40] nal_unit_type: 1, nal_ref_idc: 0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x3462320] nal_unit_type: 1, nal_ref_idc: 0
[h264 @ 0x3469ac0] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x34f73e0] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x3523bc0] nal_unit_type: 1, nal_ref_idc: 0
[h264 @ 0x348df40] nal_unit_type: 1, nal_ref_idc: 0
[h264 @ 0x3462320] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x3469ac0] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x34f73e0] nal_unit_type: 1, nal_ref_idc: 0
[h264 @ 0x3523bc0] nal_unit_type: 1, nal_ref_idc: 0
[h264 @ 0x348df40] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x3462320] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x3469ac0] nal_unit_type: 1, nal_ref_idc: 0
[out_0_0 @ 0x351c200] EOF on sink link out_0_0:default.
No more output streams to write to, finishing.
frame=   48 fps=0.0 q=20.0 Lsize=N/A time=00:00:02.00 bitrate=N/A speed=4.07x
video:371kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (http://bubblesoftapps.com/bubbleupnpserver/transcode_test/tos_sample_h264.mkv):
  Input stream #0:0 (video): 48 packets read (338524 bytes); 48 frames decoded;
  Total: 48 packets (338524 bytes) demuxed
Output file #0 (pipe:):
  Output stream #0:0 (video): 48 frames encoded; 48 packets muxed (379700 bytes);
  Total: 48 packets (379700 bytes) muxed
48 frames successfully decoded, 0 decoding errors
[h264_nvenc @ 0x348cce0] Nvenc unloaded
[AVIOContext @ 0x34466a0] Statistics: 339762 bytes read, 0 seeks

[Timo R.]

I can confirm that it crashes with the static build from ​https://johnvansickle.com/ffmpeg/
As it works with all local builds of the same commits I have done, this is an issue with that specific build, not ffmpeg itself.

[bubbleguuum]

Replying to oromit:

I can confirm that it crashes with the static build from ​https://johnvansickle.com/ffmpeg/
As it works with all local builds of the same commits I have done, this is an issue with that specific build, not ffmpeg itself.

Thank you for the test.

Is it possible this issue has been fixed recently, in the last 2 weeks ? John's static git build from 20170517 also crashes.
Also, I suppose you did not test your build as a static build ?

[bubbleguuum]

backtrace is not super explicit :(

[AVFilterGraph @ 0x3c9d060] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed

Thread 1 "ffmpeg" received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007fffc8e80fa9 in ?? ()
#2  0x00007fffffffcb50 in ?? ()
#3  0x0000000000000010 in ?? ()
#4  0x0000000000800000 in ?? ()
#5  0x8000000100000001 in ?? ()
#6  0x00000000043a5640 in ?? ()
#7  0x0000000000000005 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000001 in ?? ()
#10 0x0000000000000006 in ?? ()
#11 0x00007fff00000008 in ?? ()
#12 0x00007fffffffcc00 in ?? ()
#13 0x00007fffffffcdc0 in ?? ()
#14 0x0000000000000000 in ?? ()[AVFilterGraph @ 0x3c9d060] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed

[Timo R.]

Depends on what you mean by static build.
I used the default configuration, which links the libav* libraries statically, but I did not make special efforts to have no dynamic dependencies.
nvenc/cuvid are not any different in a static vs. a shared build, they always load their stuff at runtime via dlopen.

I tested current master and ffmpeg 3.3.1, so the exact same version you can download statically from that site, and it did not crash for me.

The build seems to be lacking debug symbols, and I can't see any on the site either, so I guess we're not getting a trace from that build. I'd contact the author of those builds about the issue.

[bubbleguuum]

I tried to contact John for something else and he's slow to respond, if he responds at all.

By curiosity, what is your output for 'ldd ffmpeg' ? Also can you tell what distro and NVIDIA driver version do you use ?

In any case, looks like the only solution left I have is to give a go at compiling ffmpeg myself...

[bubbleguuum]

I compiled ffmpeg myself and found the explanation for this crash.

long story short: nvdec, cuda and friends using dlopen() are not compatible with ffmpeg compiled with -static generally used to produce a fully static build.

The reason is that it segfaults dlopen'ing libcuda.so.1, due to this warning displayed at compilation:

warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

After much tinkering, I was able to produce a build with all libraries statically linked except the glibc (and thus libdl):

1. pass --extra-ldexeflags="-Wl,-Bstatic" to configure. This replaces -static and the libraries following this option will be linked statically *exept the glibc*. Thus this is not the same thing than -static that force static linking for all libraries including the glibc.

2. pass --extra-libs="-Wl,-Bdynamic -lm -ldl". This force dynamic linking for libm and libdl . This override is necessary because these libraries cannot be linked statically when the glibc is linked dynamically

3. You must edit the configure script (yuk), function probe_cc():

change:

_flags_filter=echo

to:

_flags_filter='filter_out -lm|-ldl'

This will filter out all -lm and -ldl occurences before -Wl,-Bdynamic. This is necessary for both these libs to not be linked statically. Finding this trick took me forever.

If all goes well, at the end you have a ffmpeg binary with everything statically linked except the glibc and dynamic linker, with working NVIDIA support:

ldd ffmpeg
	linux-vdso.so.1 =>  (0x00007ffece9e6000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f19931dc000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1992fd8000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1992dc1000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1992ba4000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f19927db000)
	/lib64/ld-linux-x86-64.so.2 (0x0000560591227000)

Suggestion: make the configure script issue a big fat warning (or fail entirely) if -static is specified and any library using dlopen() is to be included in the build.