Intermittent Vulkan encoding failures with 4060TI(driver 550), Ubuntu 24.04, Vulkan SDK 1.4.304.0

[Marth64]

Given the following environment

• Install NVIDIA RTX 4060TI GPU
• Install Xubuntu 24.04
• Install nvidia-driver-550 via Ubuntu Additional Drivers widget
• Install Vulkan SDK 1.4.304.0 via apt (from LunarG)
• Provide an MPEG-2 source (720x480, 29.9fps with soft pull down so 23.98) or H264 source (1920x1080, 24fps) [this is only what I have tested with, but 3 diverse ones]
• FFmpeg head as of 957eb2323a924aa0b148927889eae581185c367b

Then run:

ffmpeg -init_hw_device vulkan=vk:0 -y -i "$IN" -filter:v "format=nv12,hwupload" -c:v h264_vulkan sample_h264_vulkan.mkv

Observe that the operation will sometimes work but then intermittently fail with various error codes.
I tried to av_err2str() the codes, but only their numeric value is still printed as a fallback since they are not known.

The error codes appear to be random, here are a few:
-1851303552
-1776594112
-1241304448
-1195352896
-1474912192
-1080228480

ffmpeg version N-118406-g957eb2323a Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.3.0-6ubuntu2~24.04)
  configuration: --enable-debug=3 --enable-ffplay --enable-gpl --enable-nonfree --enable-ffnvcodec --enable-vulkan --enable-libx264 --enable-libfdk_aac --enable-openssl --enable-version3 --enable-demuxer=dvdvideo --enable-libdvdread --enable-libdvdnav --samples=fate-suite/
  libavutil      59. 56.100 / 59. 56.100
  libavcodec     61. 32.100 / 61. 32.100
  libavformat    61.  9.106 / 61.  9.106
  libavdevice    61.  4.100 / 61.  4.100
  libavfilter    10.  9.100 / 10.  9.100
  libswscale      8. 13.100 /  8. 13.100
  libswresample   5.  4.100 /  5.  4.100
  libpostproc    58.  4.100 / 58.  4.100
Input #0, matroska,webm, from 'MyDVD.mkv':
  Metadata:
    ENCODER         : Lavf61.9.106
  Duration: 00:06:00.14, start: 0.000000, bitrate: 7790 kb/s
  Stream #0:0: Video: mpeg2video (Main), yuv420p(tv, smpte170m, progressive), 720x480 [SAR 32:27 DAR 16:9], 29.97 fps, 29.97 tbr, 1k tbn
    Metadata:
      DURATION        : 00:06:00.143000000
    Side data:
      cpb: bitrate max/min/avg: 9800000/0/0 buffer size: 1835008 vbv_delay: N/A
  Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 448 kb/s
    Metadata:
      DURATION        : 00:06:00.006000000
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> h264 (h264_vulkan))
  Stream #0:1 -> #0:1 (ac3 (native) -> ac3 (native))
Press [q] to stop, [?] for help
[h264_vulkan @ 0x64d194ac7540] No rate control settings specified, using fixed QP = 18
Output #0, matroska, to 'sample_h264_vulkan.mkv':
  Metadata:
    encoder         : Lavf61.9.106
  Stream #0:0: Video: h264 (High) (H264 / 0x34363248), vulkan(tv, smpte170m, progressive), 720x480 [SAR 32:27 DAR 16:9], q=2-31, 29.97 fps, 1k tbn
    Metadata:
      encoder         : Lavc61.32.100 h264_vulkan
      DURATION        : 00:06:00.143000000
  Stream #0:1(eng): Audio: ac3 ([0] [0][0] / 0x2000), 48000 Hz, 5.1(side), fltp, 448 kb/s
    Metadata:
      encoder         : Lavc61.32.100 ac3
      DURATION        : 00:06:00.006000000
[h264_vulkan @ 0x64d194ac7540] Encode failed: -1851303552.
[vost#0:0/h264_vulkan @ 0x64d191a6a380] [enc:h264_vulkan @ 0x64d1918e0e00] Error submitting video frame to the encoder
[vost#0:0/h264_vulkan @ 0x64d191a6a380] [enc:h264_vulkan @ 0x64d1918e0e00] Error encoding a frame: Error number -1851303552 occurred
[vost#0:0/h264_vulkan @ 0x64d191a6a380] Task finished with error code: -1851303552 (Error number -1851303552 occurred)
[vost#0:0/h264_vulkan @ 0x64d191a6a380] Terminating thread with return code -1851303552 (Error number -1851303552 occurred)
[out#0/matroska @ 0x64d191a68840] video:0KiB audio:51KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 1.939655%
frame=    0 fps=0.0 q=0.0 Lsize=      52KiB time=N/A bitrate=N/A speed=N/A    
Conversion failed!

[MasterQuestionable]

͏    I fear you'd have to ask Nvidia...
͏    Whatsoever, as general debug hint:

͏    Try sticking the input to a single, certain source. (e.g. `-f lavfi -i "testsrc2=1920x1080:24"`)
͏    And test if it would always fail on the same pattern.

͏    Refer also:
͏    ​https://trac.ffmpeg.org/ticket/11260#comment:1
͏    https://ffmpeg.org/ffmpeg-filters.html#testsrc

[Marth64]

Replying to MasterQuestionable:

͏    I fear you'd have to ask Nvidia...
͏    Whatsoever, as general debug hint:

͏    Try sticking the input to a single, certain source. (e.g. `-f lavfi -i "testsrc2=1920x1080:24"`)
͏    And test if it would always fail on the same pattern.

͏    Refer also:
͏    ​https://trac.ffmpeg.org/ticket/11260#comment:1
͏    https://ffmpeg.org/ffmpeg-filters.html#testsrc

This is a very good idea. I tried with testsrc2 and could reproduce the issue (on 24fps, 30fps, etc.). I fear you are correct but I will see if there is any way to get more useful logging.

[vytskalt]

I observe a very similar issue using the RADV driver on an AMD GPU.

Sometimes it works, sometimes it doesn't. When it doesn't work, it either gives a random error number such as -455748544 or just simply segfaults...

./ffmpeg_g -y -init_hw_device vulkan=vk:0 -hwaccel vulkan -hwaccel_output_format vulkan -filter_hw_device vk -i input.mp4 -c:v h264_vulkan output.mp4

[h264_vulkan @ 0x55b1e4d5d440] Encode failed: Error number -455748544 occurred.
[vost#0:0/h264_vulkan @ 0x55b1e4ceadc0] [enc:h264_vulkan @ 0x55b1e4d29600] Error submitting video frame to the encoder
[vost#0:0/h264_vulkan @ 0x55b1e4ceadc0] [enc:h264_vulkan @ 0x55b1e4d29600] Error encoding a frame: Error number -455748544 occurred
[vost#0:0/h264_vulkan @ 0x55b1e4ceadc0] Task finished with error code: -455748544 (Error number -455748544 occurred)
[vost#0:0/h264_vulkan @ 0x55b1e4ceadc0] Terminating thread with return code -455748544 (Error number -455748544 occurred)