Compile NVDEC with Quadro RTX 6000 : "ERROR: failed checking for nvcc"

[Maximiliano Lira Del Canto]

Summary of the bug:

When you try to compile FFmpeg 4.3 - 4.3.1 with Nvidia options enabled, results in a Nvidia compiler error. during the configure phase.

The details from the error log (config.log) are:

check_nvcc cuda_nvcc
test_nvcc
BEGIN /tmp/ffconf.lcEnfW9f/test.cu
    1   extern "C" {
    2       __global__ void hello(unsigned char *data) {}
    3   }
END /tmp/ffconf.lcEnfW9f/test.cu
nvcc -gencode arch= ,code=sm_30 -O2 -m64 -ptx -c -o /tmp/ffconf.lcEnfW9f/test.o /tmp/ffconf.lcEnfW9f/test.cu
nvcc fatal   : Unsupported gpu architecture 'compute_30'
ERROR: failed checking for nvcc.

---

How to reproduce:

Try to build ffmpeg, in my case using a self made script that you can find here​https://github.com/markus-perl/ffmpeg-build-script/blob/master/build-ffmpeg.

---

Why happens:

This error happens because since the latest CUDA SDK (11.0) only accepts compute capability 5.2 - 8.0 (Maxwell, Pascal, Volta, Turing, Ampere). Compute_30

How to fix (Workaround):

The fix is to modify the compute version from 30 to 52 in the configure file:

if enabled cuda_nvcc; then
    nvcc_default="nvcc"
    nvccflags_default="-gencode arch=compute_52,code=sm_52 -O2"
else
    nvcc_default="clang"
    nvccflags_default="--cuda-gpu-arch=sm_52 -O2"
    NVCC_C=""
fi

System data

Quadro RTX6000 card with the latest cuda sdk (11.0) and the nvidia drivers 450.51.05.
OS: Centos 7

Nvidia SMI data:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.51.05    Driver Version: 450.51.05    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro RTX 6000     Off  | 00000000:3B:00.0 Off |                    0 |
| N/A   31C    P0    23W / 250W |      0MiB / 22698MiB |      5%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

[Maximiliano Lira Del Canto]

I was studying this and I found that CC 3.5 is enough.

Why happens:

This error happens because since the latest CUDA SDK (11.0) accepts compute capability 3.5 - 8.6

How to fix (Workaround):

The fix is to modify the compute version from 30 to 35 in the configure file:

if enabled cuda_nvcc; then
    nvcc_default="nvcc"
    nvccflags_default="-gencode arch=compute_35,code=sm_35 -O2"
else
    nvcc_default="clang"
    nvccflags_default="--cuda-gpu-arch=sm_35 -O2"
    NVCC_C=""
fi

[infinoid]

I just ran into this here, so I can confirm.

Debian bullseye recently updated its (nonfree) cuda packages to version 11.0, and this caused ffmpeg's configure script to stop detecting nvcc.

I changed compute_30 and sm_30 to compute_35 and sm_35 in the configure script, as @mliradelc suggests. This built successfully, and the resulting binary is able to encode video with h264_nvenc successfully.

Note, the configure script hides the following warning from nvcc:

nvcc warning : The 'compute_35', 'compute_37', 'compute_50', 'sm_35', 'sm_37' and 'sm_50' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).

So they work for now, but won't work for much longer.

My system info:
amd64
linux-image-5.8.18 5.8.18-1
nvidia-cuda-toolkit 11.0.3-2
nvidia-kernel-dkms 450.80.02-1
41:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU104 [GeForce RTX 2080 SUPER] [10de:1e81] (rev a1)

[Maximiliano Lira Del Canto]

I found another workaround, this time it doesn't involve change the config file
add this line when you compile ffmpeg:

'-gencode arch=compute_52,code=sm_52'

That will overwrite the default compute code version from 30 to 52, also can be changed to 35 to allow compatibility with older cards.