h264_nvenc Cannot init CUDA

[Andrew Udvare]

Summary of the bug:

In the past a command utilising NVENC worked (as did OBS). Now I always get "Cannot init CUDA". I'm on a Gentoo machine and I've made sure to rebuild everything that uses ffmpeg. I have a GeForce 980 GTX card (Maxwell).

I have Nvidia drivers version 410.57.

I have minimally debugged this and found that it's from ​line ~558 in nvenc.c. The return value from cuInit() is CUDA_ERROR_UNKNOWN. Unfortunately I have not seen any more information on how to debug this.

$ qlist nvidia-drivers | fgrep encode
/usr/lib32/libnvidia-encode.so.410.57
/usr/lib64/libnvidia-encode.so.410.57
/usr/lib64/libnvidia-encode.so.1
/usr/lib64/libnvidia-encode.so
/usr/lib32/libnvidia-encode.so.1
/usr/lib32/libnvidia-encode.so

How to reproduce:

% ffmpeg -y -i yai.mp4 -c:v h264_nvenc -an out.mp4
ffmpeg version 3.4.4 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 8.2.0 (Gentoo 8.2.0-r2 p1.2)
configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-3.4.4/html --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags='-O2 -pipe -march=native -fomit-frame-pointer -ggdb' --disable-static --enable-avfilter --enable-avresample --disable-stripping --enable-nonfree --disable-indev=oss --disable-indev=jack --disable-outdev=oss --enable-version3 --enable-nonfree --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-gcrypt --disable-gnutls --disable-gmp --enable-gpl --enable-hardcoded-tables --enable-iconv --disable-lzma --enable-network --disable-opencl --enable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --enable-sdl2 --disable-vaapi --enable-vdpau --enable-xlib --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-zlib --enable-libcdio --enable-libiec61883
--disable-libdc1394 --enable-libcaca --disable-openal --enable-opengl --disable-libv4l2 --disable-libpulse --disable-libdrm --enable-libopencore-amrwb --enable-libopencore-amrnb --enable-libfdk-aac --enable-libopenjpeg --enable-libbluray --disable-libcelt --enable-libgme --enable-libgsm --disable-mmal --enable-libmodplug --enable-libopus --disable-libilbc --enable-librtmp --enable-libssh --enable-libspeex --enable-librsvg --enable-libvorbis --enable-libvpx --enable-libzvbi --disable-appkit --disable-libbs2b --enable-chromaprint --enable-libflite --disable-frei0r --enable-libfribidi --disable-fontconfig --disable-ladspa --enable-libass --enable-libfreetype --enable-librubberband --disable-libzmq --enable-libzimg --disable-libsoxr --enable-pthreads --disable-libvo-amrwbenc --enable-libmp3lame --disable-libkvazaar --enable-nvenc --disable-libopenh264 --disable-libsnappy --enable-libtheora --enable-libtwolame --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-vfpv3 --disable-armv8 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-altivec --disable-amd3dnow --disable-amd3dnowext --disable-fma4 --disable-xop--cpu=host --disable-doc --disable-htmlpages --enable-manpages
libavutil      55. 78.100 / 55. 78.100
libavcodec     57.107.100 / 57.107.100
libavformat    57. 83.100 / 57. 83.100
libavdevice    57. 10.100 / 57. 10.100
libavfilter     6.107.100 /  6.107.100
libavresample   3.  7.  0 /  3.  7.  0
libswscale      4.  8.100 /  4.  8.100
libswresample   2.  9.100 /  2.  9.100
libpostproc    54.  7.100 / 54.  7.100

...

Duration: 00:10:08.80, start: 0.000000, bitrate: 2741 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 2598 kb/s, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc (default)
Metadata:
handler_name    : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name    : SoundHandler
Stream #0:2: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], 90k tbr, 90k tbn, 90k tbc
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
[h264_nvenc @ 0x55bcb71bfb20] Cannot init CUDA
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

[Carl Eugen Hoyos]

Please test current FFmpeg git head.

[Timo R.]

See ​https://bugs.gentoo.org/663234
Not an ffmpeg issue.