H264_cuvid Decoding Causes NVRM XID 31

[AlexR]

Decoding h264 stream using h264_cuvid causes a NVRM XID 31 (NVidia driver error).
​http://docs.nvidia.com/deploy/xid-errors/index.html indicates XID 31 is a "GPU memory page fault", and can be either the Driver or the Application's issue.

Replicating is as simple as:
ffmpeg -c:v h264_cuvid -i ./bbb_sunflower_1080p_30fps_stereo_abl.mp4 ./test.mp4

After the error, ffmpeg hangs (waiting on IO), and seemingly anything else querying the NVidia driver also hangs (e.g. nvidia-smi run at the same time will hang). Killing ffmpeg (goes zombie still waiting on IO), and waiting a few minutes eventually unfreezes and allows normal use of the NVidia drivers again.

Not sure if this is ffmpeg or the NVidia drivers, but I figured I'd post here first ;).

I originally encountered this on a system with a Tesla P40 and a 1080TI, and have also replicated it on another system with only a 1080TI (same software stack on both).

This is using:
ffmpeg 3.3.5
linux 4.4.87-gentoo-r2-rev1.16
nvidia-drivers 384.90
nvidia-cuda-cudnn 9.0-r1
nvidia-cuda-sdk 9.0.176
nvidia-cuda-toolkit 9.0.176
nvidia-video-sdk 6.0.1

Full build options:

ffmpeg version 3.3.5 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Gentoo Hardened 5.4.0-r3 p1.4, pie-0.6.5)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-3.3.5/html --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags='-O2 -pipe' --disable-static --enable-avfilter --enable-avresample --disable-stripping --enable-version3 --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=alsa --disable-indev=oss --disable-indev=jack --disable-outdev=alsa --disable-outdev=oss --disable-outdev=sdl --enable-bzlib --disable-runtime-cpudetect --disable-debug --enable-gcrypt --enable-gnutls --disable-gmp --enable-gpl --enable-hardcoded-tables --enable-iconv --disable-lzma --enable-network --disable-openssl --enable-postproc --enable-libsmbclient --disable-ffplay --disable-sdl2 --enable-vaapi --enable-vdpau --disable-xlib --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --enable-opengl --disable-libv4l2 --disable-libpulse --disable-libopencore-amrwb --disable-libopencore-amrnb --disable-libfdk-aac --disable-libopenjpeg --disable-libbluray --enable-libcelt --disable-libgme --disable-libgsm --disable-mmal --disable-libmodplug --disable-libopus --disable-libilbc --disable-librtmp --disable-libssh --disable-libschroedinger --disable-libspeex --disable-libvorbis --disable-libvpx --disable-libzvbi --disable-libbs2b --disable-chromaprint --disable-libflite --disable-frei0r --disable-libfribidi --disable-fontconfig --disable-ladspa --disable-libass --disable-libfreetype --disable-librubberband --disable-netcdf --disable-libzmq --disable-libzimg --disable-libsoxr --enable-pthreads --disable-libvo-amrwbenc --disable-libmp3lame --disable-libkvazaar --enable-nvenc --disable-libopenh264 --disable-libsnappy --disable-libtheora --disable-libtwolame --disable-libwavpack --disable-libwebp --enable-libx264 --enable-libx265 --disable-libxvid --disable-amd3dnow --disable-amd3dnowext --disable-aesni --disable-avx --disable-avx2 --disable-fma3 --disable-fma4 --disable-xop --disable-doc --disable-htmlpages --enable-manpages

Any help would be appreciated.

• Alex.

[AlexR]

I've submitted a driver bug report to NVidia via their form (with a link here).

[Funky Penguin]

I have the same issue, have a thread about it here:
​https://devtalk.nvidia.com/default/topic/1042835/linux/nvidia-docker-based-host-hangs-when-gpu-memory-exceeded-with-ffmpeg-transcodes/post/5289940/?offset=9#5290132

Next step is to upgrade my nVidia drivers to v410

[Funky Penguin]

Sad to report that the driver update did not fix the issue:

root@docker1:# cat nvidia-bug-report.log  | grep Xid
Oct 16 17:14:02 docker1 kernel: NVRM: Xid (PCI:0000:03:00): 31, Ch 00000030, engmask 00008100, intr 10000000
Oct 16 17:14:02 docker1 kernel: NVRM: Xid (PCI:0000:03:00): 68, CCMDs 00000030 0000c2b0
[17264.379746] NVRM: Xid (PCI:0000:03:00): 31, Ch 00000030, engmask 00008100, intr 10000000
[17264.535490] NVRM: Xid (PCI:0000:03:00): 68, CCMDs 00000030 0000c2b0
root@docker1:#

I'm using ffmpeg version 4.0.2-emby_2018_09_13

[hamhands]

I'm experiencing this exact issue with a couple of our Quadro P5000s
Upgraded GPUs to v410 as well with no luck.

[kxuanobj]

Same issue on 1050Ti.

[Balling]

I think this is fixed, no?

[Balling]

I think this is fixed, no?