CUVID decoder doesn't perform a capability check on chroma subsampling

[HolyWu]

Summary of the bug:
When using external wrapper decoder such as hevc_cuvid on a video stream whose YUV subsampling is not supported by the device, for example my GTX 1060 doesn't support HEVC 4:4:4 hardware decoding, FFmpeg CLI keeps printing error messages endlessly before I press [q] to stop.

How to reproduce:

% ffmpeg -c:v hevc_cuvid -i test.265 -f null -
ffmpeg version N-94953-gef50cf7b32 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9.1.1 (GCC) 20190807
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 35.100 / 56. 35.100
  libavcodec     58. 57.100 / 58. 57.100
  libavformat    58. 32.104 / 58. 32.104
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 58.102 /  7. 58.102
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, hevc, from 'test.265':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: hevc (Rext), yuv444p(tv), 1920x1080, 25 fps, 25 tbr, 1200k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (hevc_cuvid) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[hevc_cuvid @ 000002a1631dc380] ctx->cvdl->cuvidCreateDecoder(&ctx->cudecoder, &cuinfo) failed -> CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
[hevc_cuvid @ 000002a1631dc380] cuvid decode callback error
Error while decoding stream #0:0: Generic error in an external library
[hevc_cuvid @ 000002a1631dc380] cuvid decode callback error
Error while decoding stream #0:0: Generic error in an external library
[hevc_cuvid @ 000002a1631dc380] cuvid decode callback error
Error while decoding stream #0:0: Generic error in an external library
.
.
.
[hevc_cuvid @ 000002a1631dc380] cuvid decode callback error
Error while decoding stream #0:0: Generic error in an external library
[hevc_cuvid @ 000002a1631dc380] cuvid decode callback error
Error while decoding stream #0:0: Generic error in an external library
[hevc_cuvid @ 000002a1631dc380] cuvid decode callback error
Error while decoding stream #0:0: Generic error in an external library
Finishing stream 0:0 without any data written to it.
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.32.104
    Stream #0:0: Video: wrapped_avframe, yuv444p, 1920x1080, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.57.100 wrapped_avframe
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed=   0x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Conversion failed!

[HolyWu]

I have sent a ​patch to fix this issue. Could any developer take a look?

[Balling]

​https://patchwork.ffmpeg.org/patch/15509/

[Balling]

.

[Balling]

This was done for nvdec here: 83c7ac2e47efd96927127c1c385cdbb5fb53cb02

Again, patch is here.

[Balling]

Did you see the patch?

[Balling]

So again, this does not affect 2080 Ti. But older that do not support 444 in HEVC. Please apply.