Opened 4 days ago
#11219 new defect
On the state of Vulkan video encoding on Intel and NVIDIA Hardware
| Reported by: | Dennis E. Mungai | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | avcodec |
| Version: | git-master | Keywords: | hevc_vulkan, h264_vulkan, libplacebo, vulkan |
| Cc: | Blocked By: | ||
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description
Hello there,
I have had the chance to test the Vulkan video encoding stack on the current ffmpeg git tip, on both Intel and NVIDIA platforms, as shown below.
So far, basic pipelines work as expected. However, more common use cases that involve handling of Vulcan-based decoding, HDR signalling and filtering (with the likes of libplacebo, etc, (as advertised to be near-feature parity with VAAPI), often fail, as shown in the examples below.
Platform info:
(a). Intel:
ffmpeg -v verbose -init_hw_device vulkan ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04) configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable-libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable-zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable-liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable-pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir/Blackmagic-SDK-14/Linux/include/ libavutil 59. 40.100 / 59. 40.100 libavcodec 61. 20.100 / 61. 20.100 libavformat 61. 9.100 / 61. 9.100 libavdevice 61. 4.100 / 61. 4.100 libavfilter 10. 6.100 / 10. 6.100 libswscale 8. 4.100 / 8. 4.100 libswresample 5. 4.100 / 5. 4.100 libpostproc 58. 4.100 / 58. 4.100 [AVHWDeviceContext @ 0x589b038cdf00] Supported layers: [AVHWDeviceContext @ 0x589b038cdf00] VK_LAYER_MESA_device_select [AVHWDeviceContext @ 0x589b038cdf00] VK_LAYER_MESA_overlay [AVHWDeviceContext @ 0x589b038cdf00] Using instance extension VK_KHR_portability_enumeration [AVHWDeviceContext @ 0x589b038cdf00] GPU listing: [AVHWDeviceContext @ 0x589b038cdf00] 0: Intel(R) Graphics (RPL-P) (integrated) (0xa7a0) [AVHWDeviceContext @ 0x589b038cdf00] Device 0 selected: Intel(R) Graphics (RPL-P) (integrated) (0xa7a0) [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_push_descriptor [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_descriptor_buffer [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_physical_device_drm [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_shader_atomic_float [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_cooperative_matrix [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_maintenance1 [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_external_memory_fd [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_external_memory_dma_buf [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_image_drm_format_modifier [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_external_semaphore_fd [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_EXT_external_memory_host [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_queue [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_encode_queue [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_decode_queue [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_encode_h264 [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_decode_h264 [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_encode_h265 [AVHWDeviceContext @ 0x589b038cdf00] Using device extension VK_KHR_video_decode_h265 [AVHWDeviceContext @ 0x589b038cdf00] Queue families: [AVHWDeviceContext @ 0x589b038cdf00] 0: graphics compute transfer sparse protected (queues: 1) [AVHWDeviceContext @ 0x589b038cdf00] 1: encode decode (queues: 2) [AVHWDeviceContext @ 0x589b038cdf00] Using device: Intel(R) Graphics (RPL-P) [AVHWDeviceContext @ 0x589b038cdf00] Alignments: [AVHWDeviceContext @ 0x589b038cdf00] optimalBufferCopyRowPitchAlignment: 128 [AVHWDeviceContext @ 0x589b038cdf00] minMemoryMapAlignment: 4096 [AVHWDeviceContext @ 0x589b038cdf00] nonCoherentAtomSize: 64 [AVHWDeviceContext @ 0x589b038cdf00] minImportedHostPointerAlignment: 4096 [AVHWDeviceContext @ 0x589b038cdf00] Using queue family 0 (queues: 1) for graphics compute transfers [AVHWDeviceContext @ 0x589b038cdf00] Using queue family 1 (queues: 2) for encode decode
I've tested this implementation on Intel's ANV driver.
This was built against the mesa main branch, which now has the enablement bits for KHR_video_maintenance1 on anv.
See the related MR https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31436
Also, the environment variables:
ANV_VIDEO_ENCODE=1 ANV_VIDEO_DECODE=1
Are also set.
i. With hwaccel:
ffmpeg \ -init_hw_device vaapi=intel:/dev/dri/renderD128 \ -init_hw_device vulkan=vk@intel \ -hwaccel vulkan -hwaccel_output_format vulkan \ -hwaccel_device vk -filter_hw_device vk \ -loglevel info \ -fflags +genpts \ -i '4KIOS.mov' \ -vf "libplacebo=w=1920:h=1080" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v hevc_vulkan \ -r:v 60 -g:v 120 -tune:v hq -usage:v transcode \ -async_depth:v 4 -b_depth:v 1 -bf:v 0 \ -c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \ -map "0:v" -map "0:a" \ -max_delay 5000000 -max_muxing_queue_size 8192 -max_interleave_delta 0 \ -flags -global_header+cgop \ -y -f mpegts "ticket0.ts"
Output:
ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable-libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable-zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable-liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable-pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir/Blackmagic-SDK-14/Linux/include/
libavutil 59. 40.100 / 59. 40.100
libavcodec 61. 20.100 / 61. 20.100
libavformat 61. 9.100 / 61. 9.100
libavdevice 61. 4.100 / 61. 4.100
libavfilter 10. 6.100 / 10. 6.100
libswscale 8. 4.100 / 8. 4.100
libswresample 5. 4.100 / 5. 4.100
libpostproc 58. 4.100 / 58. 4.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '4KIOS.mov':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2016-01-30T17:46:24.000000Z
com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 6s
com.apple.quicktime.software: 9.2.1
com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500
Duration: 00:00:18.55, start: 0.000000, bitrate: 51218 kb/s
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 51119 kb/s, 29.98 fps, 29.97 tbr, 600 tbn (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
encoder : H.264
Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 86 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Metadata
Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Metadata
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vulkan))
Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
MESA-INTEL: warning: ../src/intel/vulkan/anv_formats.c:811: FINISHME: support more multi-planar formats with DRM modifiers
Output #0, mpegts, to 'ticket0.ts':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500
com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 6s
com.apple.quicktime.software: 9.2.1
encoder : Lavf61.9.100
Stream #0:0(und): Video: hevc (Main), vulkan(tv, bt709, progressive), 1920x1080, q=2-31, 2250 kb/s, 60 fps, 90k tbn (default)
Metadata:
encoder : Lavc61.20.100 hevc_vulkan
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
Stream #0:1(und): Audio: aac, 48000 Hz, stereo, s16, 128 kb/s (default)
Metadata:
encoder : Lavc61.20.100 libfdk_aac
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
frame= 1 fps=0.0 q=-0.0 size= 256KiB time=00:00:00.01 bitraframe= 1 fps=1.0 q=-0.0 size= 256KiB time=00:00:00.01 bitra[hevc_vulkan @ 0x5720d9a16280] Unable to perform query: VK_ERROR_DEVICE_LOST!
[hevc_vulkan @ 0x5720d9a16280] Output failed: -542398533.
[vost#0:0/hevc_vulkan @ 0x5720d9a190c0] [enc:hevc_vulkan @ 0x5720d9a12c80] Error submitting video frame to the encoder
[vost#0:0/hevc_vulkan @ 0x5720d9a190c0] [enc:hevc_vulkan @ 0x5720d9a12c80] Error encoding a frame: Generic error in an external library
[vost#0:0/hevc_vulkan @ 0x5720d9a190c0] Task finished with error code: -542398533 (Generic error in an external library)
[vost#0:0/hevc_vulkan @ 0x5720d9a190c0] Terminating thread with return code -542398533 (Generic error in an external library)
[libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504)
[libplacebo @ 0x796454004140] Attempting to release an unheld image?
Last message repeated 1 times
[libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504)
[libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105)
[libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504)
[libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105)
[libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504)
[libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105)
[libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504)
[libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105)
[h264 @ 0x5720d9a130c0] [vk @ 0x79643417da00] Unable to submit command buffer: VK_ERROR_DEVICE_LOST
[h264 @ 0x5720d9a130c0] hardware accelerator failed to decode picture
[libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504)
[libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105)
[libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504)
[libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105)
[libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504)
[libplacebo @ 0x796454004140] Retrieving query pool results: VK_ERROR_DEVICE_LOST (../src/vulkan/gpu.c:105)
[libplacebo @ 0x796454004140] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:504)
Last message repeated 1 times
[out#0/mpegts @ 0x5720d9a46dc0] video:260KiB audio:291KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 12.472680%
frame= 1 fps=0.8 q=-0.0 Lsize= 620KiB time=00:00:00.01 bitrate=304553.9kbits/s dup=8 drop=0 speed=0.0139x
ii. And without H/W accelerated decoding, ie -hwaccel vulkan is disabled:
ffmpeg -v debug \ -init_hw_device vaapi=intel:/dev/dri/renderD128 \ -init_hw_device vulkan=vk@intel \ -filter_hw_device vk \ -loglevel info \ -fflags +genpts \ -i '4KIOS.mov' \ -vf "hwupload,libplacebo=w=1920:h=1080:format=nv12" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v hevc_vulkan \ -r:v 60 -g:v 120 -tune:v hq -usage:v transcode \ -async_depth:v 4 -b_depth:v 1 -bf:v 0 \ -c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \ -map "0:v" -map "0:a" \ -noautoscale -max_delay 5000000 -max_muxing_queue_size 8192 -max_interleave_delta 0 \ -flags -global_header+cgop \ -y -f mpegts "ticket1.ts"
Output:
ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable-libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable-zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable-liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable-pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir/Blackmagic-SDK-14/Linux/include/
libavutil 59. 40.100 / 59. 40.100
libavcodec 61. 20.100 / 61. 20.100
libavformat 61. 9.100 / 61. 9.100
libavdevice 61. 4.100 / 61. 4.100
libavfilter 10. 6.100 / 10. 6.100
libswscale 8. 4.100 / 8. 4.100
libswresample 5. 4.100 / 5. 4.100
libpostproc 58. 4.100 / 58. 4.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '4KIOS.mov':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2016-01-30T17:46:24.000000Z
com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 6s
com.apple.quicktime.software: 9.2.1
com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500
Duration: 00:00:18.55, start: 0.000000, bitrate: 51218 kb/s
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 51119 kb/s, 29.98 fps, 29.97 tbr, 600 tbn (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
encoder : H.264
Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 86 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Metadata
Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Metadata
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vulkan))
Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
MESA-INTEL: warning: ../src/intel/vulkan/anv_formats.c:811: FINISHME: support more multi-planar formats with DRM modifiers
Output #0, mpegts, to 'ticket1.ts':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500
com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 6s
com.apple.quicktime.software: 9.2.1
encoder : Lavf61.9.100
Stream #0:0(und): Video: hevc (Main), vulkan(tv, bt709, progressive), 1920x1080, q=2-31, 2250 kb/s, 60 fps, 90k tbn (default)
Metadata:
encoder : Lavc61.20.100 hevc_vulkan
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
Stream #0:1(und): Audio: aac, 48000 Hz, stereo, s16, 128 kb/s (default)
Metadata:
encoder : Lavc61.20.100 libfdk_aac
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
[hevc_vulkan @ 0x56d91fd19480] Encode failed: -1073731200.
[vost#0:0/hevc_vulkan @ 0x56d91fd76cc0] [enc:hevc_vulkan @ 0x56d91fdaa140] Error submitting video frame to the encoder
[vost#0:0/hevc_vulkan @ 0x56d91fd76cc0] [enc:hevc_vulkan @ 0x56d91fdaa140] Error encoding a frame: Error number -1073731200 occurred
[vost#0:0/hevc_vulkan @ 0x56d91fd76cc0] Task finished with error code: -1073731200 (Error number -1073731200 occurred)
[vost#0:0/hevc_vulkan @ 0x56d91fd76cc0] Terminating thread with return code -1073731200 (Error number -1073731200 occurred)
[out#0/mpegts @ 0x56d91fda12c0] video:0KiB audio:291KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 21.645238%
frame= 0 fps=0.0 q=0.0 Lsize= 354KiB time=N/A bitrate=N/A dup=2 drop=0 speed=N/A
Conversion failed!
Both encoding scenarios fail. Now, on to testing with NVIDIA.
(b). NVIDIA:
Platform info:
ffmpeg -v verbose -init_hw_device vulkan ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04) configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable-libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable-zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable-liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable-pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir/Blackmagic-SDK-14/Linux/include/ libavutil 59. 40.100 / 59. 40.100 libavcodec 61. 20.100 / 61. 20.100 libavformat 61. 9.100 / 61. 9.100 libavdevice 61. 4.100 / 61. 4.100 libavfilter 10. 6.100 / 10. 6.100 libswscale 8. 4.100 / 8. 4.100 libswresample 5. 4.100 / 5. 4.100 libpostproc 58. 4.100 / 58. 4.100 [AVHWDeviceContext @ 0x5fd8a3820f00] Supported layers: [AVHWDeviceContext @ 0x5fd8a3820f00] VK_LAYER_NV_optimus [AVHWDeviceContext @ 0x5fd8a3820f00] VK_LAYER_MESA_device_select [AVHWDeviceContext @ 0x5fd8a3820f00] VK_LAYER_MESA_overlay [AVHWDeviceContext @ 0x5fd8a3820f00] Using instance extension VK_KHR_portability_enumeration [AVHWDeviceContext @ 0x5fd8a3820f00] GPU listing: [AVHWDeviceContext @ 0x5fd8a3820f00] 0: NVIDIA GeForce RTX 4060 Laptop GPU (discrete) (0x28a0) [AVHWDeviceContext @ 0x5fd8a3820f00] Device 0 selected: NVIDIA GeForce RTX 4060 Laptop GPU (discrete) (0x28a0) [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_push_descriptor [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_descriptor_buffer [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_physical_device_drm [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_shader_atomic_float [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_cooperative_matrix [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_NV_optical_flow [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_shader_object [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_maintenance1 [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_external_memory_fd [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_external_memory_dma_buf [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_image_drm_format_modifier [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_external_semaphore_fd [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_EXT_external_memory_host [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_queue [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_encode_queue [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_decode_queue [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_encode_h264 [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_decode_h264 [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_encode_h265 [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_decode_h265 [AVHWDeviceContext @ 0x5fd8a3820f00] Using device extension VK_KHR_video_decode_av1 [AVHWDeviceContext @ 0x5fd8a3820f00] Queue families: [AVHWDeviceContext @ 0x5fd8a3820f00] 0: graphics compute transfer sparse (queues: 16) [AVHWDeviceContext @ 0x5fd8a3820f00] 1: transfer sparse (queues: 2) [AVHWDeviceContext @ 0x5fd8a3820f00] 2: compute transfer sparse (queues: 8) [AVHWDeviceContext @ 0x5fd8a3820f00] 3: transfer decode sparse (queues: 1) [AVHWDeviceContext @ 0x5fd8a3820f00] 4: transfer encode sparse (queues: 1) [AVHWDeviceContext @ 0x5fd8a3820f00] 5: transfer sparse optical_flow (queues: 1) [AVHWDeviceContext @ 0x5fd8a3820f00] Using device: NVIDIA GeForce RTX 4060 Laptop GPU [AVHWDeviceContext @ 0x5fd8a3820f00] Alignments: [AVHWDeviceContext @ 0x5fd8a3820f00] optimalBufferCopyRowPitchAlignment: 1 [AVHWDeviceContext @ 0x5fd8a3820f00] minMemoryMapAlignment: 64 [AVHWDeviceContext @ 0x5fd8a3820f00] nonCoherentAtomSize: 64 [AVHWDeviceContext @ 0x5fd8a3820f00] minImportedHostPointerAlignment: 4096 [AVHWDeviceContext @ 0x5fd8a3820f00] Using queue family 0 (queues: 16) for graphics [AVHWDeviceContext @ 0x5fd8a3820f00] Using queue family 2 (queues: 8) for compute [AVHWDeviceContext @ 0x5fd8a3820f00] Using queue family 1 (queues: 2) for transfers [AVHWDeviceContext @ 0x5fd8a3820f00] Using queue family 3 (queues: 1) for decode [AVHWDeviceContext @ 0x5fd8a3820f00] Using queue family 4 (queues: 1) for encode
And with Vulkan hwaccel enabled:
ffmpeg \ -init_hw_device vulkan=vk:0 \ -hwaccel vulkan -hwaccel_output_format vulkan \ -hwaccel_device vk -filter_hw_device vk \ -loglevel info \ -fflags +genpts \ -i '4KIOS.mov' \ -vf "libplacebo=w=1920:h=1080" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v hevc_vulkan \ -r:v 60 -g:v 120 -tune:v hq -usage:v transcode \ -async_depth:v 4 -b_depth:v 1 -bf:v 0 \ -c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \ -map "0:v" -map "0:a" \ -max_delay 5000000 -max_muxing_queue_size 8192 -max_interleave_delta 0 \ -flags -global_header+cgop \ -y -f mpegts "ticket3.ts"
Output:
ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable-libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable-zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable-liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable-pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir/Blackmagic-SDK-14/Linux/include/
libavutil 59. 40.100 / 59. 40.100
libavcodec 61. 20.100 / 61. 20.100
libavformat 61. 9.100 / 61. 9.100
libavdevice 61. 4.100 / 61. 4.100
libavfilter 10. 6.100 / 10. 6.100
libswscale 8. 4.100 / 8. 4.100
libswresample 5. 4.100 / 5. 4.100
libpostproc 58. 4.100 / 58. 4.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '4KIOS.mov':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2016-01-30T17:46:24.000000Z
com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 6s
com.apple.quicktime.software: 9.2.1
com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500
Duration: 00:00:18.55, start: 0.000000, bitrate: 51218 kb/s
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 51119 kb/s, 29.98 fps, 29.97 tbr, 600 tbn (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
encoder : H.264
Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 86 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Metadata
Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Metadata
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vulkan))
Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
Output #0, mpegts, to 'ticket3.ts':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500
com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 6s
com.apple.quicktime.software: 9.2.1
encoder : Lavf61.9.100
Stream #0:0(und): Video: hevc (Main), vulkan(tv, bt709, progressive), 1920x1080, q=2-31, 2250 kb/s, 60 fps, 90k tbn (default)
Metadata:
encoder : Lavc61.20.100 hevc_vulkan
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
Stream #0:1(und): Audio: aac, 48000 Hz, stereo, s16, 128 kb/s (default)
Metadata:
encoder : Lavc61.20.100 libfdk_aac
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
[hevc_vulkan @ 0x61c807f06040] Unable to finish command buffer: VK_ERROR_INITIALIZATION_FAILED
[hevc_vulkan @ 0x61c804be8c40] Encode failed: -542398533.
[vost#0:0/hevc_vulkan @ 0x61c807d90340] [enc:hevc_vulkan @ 0x61c807db2540] Error submitting video frame to the encoder
[vost#0:0/hevc_vulkan @ 0x61c807d90340] [enc:hevc_vulkan @ 0x61c807db2540] Error encoding a frame: Generic error in an external library
[vost#0:0/hevc_vulkan @ 0x61c807d90340] Task finished with error code: -542398533 (Generic error in an external library)
[vost#0:0/hevc_vulkan @ 0x61c807d90340] Terminating thread with return code -542398533 (Generic error in an external library)
[out#0/mpegts @ 0x61c807db48c0] video:0KiB audio:291KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 21.645238%
frame= 0 fps=0.0 q=0.0 Lsize= 354KiB time=N/A bitrate=N/A dup=4 drop=0 speed=N/A
The ffmpeg process hangs indefinitely.
ii. With Vulkan H/W Accelerated decode disabled:
ffmpeg \ -init_hw_device vulkan=vk:0 \ -filter_hw_device vk \ -loglevel info \ -fflags +genpts \ -i '4KIOS.mov' \ -vf "hwupload,libplacebo=w=1920:h=1080:format=nv12" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v hevc_vulkan \ -r:v 60 -g:v 120 -tune:v hq -usage:v transcode \ -async_depth:v 4 -b_depth:v 1 -bf:v 0 \ -c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \ -map "0:v" -map "0:a" \ -noautoscale -max_delay 5000000 -max_muxing_queue_size 8192 -max_interleave_delta 0 \ -flags -global_header+cgop \ -y -f mpegts "ticket4.ts"
Output:
ffmpeg version N-117283-gffe419d3a3 Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
configuration: --libdir=/usr/local/lib --enable-shared --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-ffplay --enable-alsa --enable-cuvid --enable-libplacebo --enable-librav1e --enable-libdav1d --enable-libshaderc --enable-libv4l2 --enable-libvmaf --enable-libvpl --enable-libxml2 --enable-nvdec --enable-nvenc --enable-cuda-llvm --nvcc=clang --nvccflags='--cuda-gpu-arch=sm_60 -O2' --enable-opencl --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libharfbuzz --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libsvtav1 --enable-libx264 --enable-nonfree --enable-openssl --enable-lzma --enable-zlib --enable-gmp --enable-libfdk_aac --enable-postproc --enable-libzmq --enable-lv2 --enable-ladspa --enable-libxeve --enable-libxevd --enable-liblcevc-dec --enable-libzvbi --enable-decklink --enable-libklvanc --enable-libopenjpeg --enable-libjxl --enable-libzimg --enable-pocketsphinx --enable-libmysofa --enable-libquirc --enable-libqrencode --enable-libkvazaar --enable-libaom --enable-libsrt --enable-librist --extra-libs='-lpthread -lm -lz -ldl' --extra-cflags=-I/tmp/workdir/Blackmagic-SDK-14/Linux/include/
libavutil 59. 40.100 / 59. 40.100
libavcodec 61. 20.100 / 61. 20.100
libavformat 61. 9.100 / 61. 9.100
libavdevice 61. 4.100 / 61. 4.100
libavfilter 10. 6.100 / 10. 6.100
libswscale 8. 4.100 / 8. 4.100
libswresample 5. 4.100 / 5. 4.100
libpostproc 58. 4.100 / 58. 4.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '4KIOS.mov':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2016-01-30T17:46:24.000000Z
com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 6s
com.apple.quicktime.software: 9.2.1
com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500
Duration: 00:00:18.55, start: 0.000000, bitrate: 51218 kb/s
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 51119 kb/s, 29.98 fps, 29.97 tbr, 600 tbn (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
encoder : H.264
Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 86 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
Stream #0:2[0x3](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Metadata
Stream #0:3[0x4](und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Metadata
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vulkan))
Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
Output #0, mpegts, to 'ticket4.ts':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
com.apple.quicktime.creationdate: 2016-01-30T12:46:24-0500
com.apple.quicktime.location.ISO6709: +33.9173-084.0055+294.114/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 6s
com.apple.quicktime.software: 9.2.1
encoder : Lavf61.9.100
Stream #0:0(und): Video: hevc (Main), vulkan(tv, bt709, progressive), 1920x1080, q=2-31, 2250 kb/s, 60 fps, 90k tbn (default)
Metadata:
encoder : Lavc61.20.100 hevc_vulkan
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
Stream #0:1(und): Audio: aac, 48000 Hz, stereo, s16, 128 kb/s (default)
Metadata:
encoder : Lavc61.20.100 libfdk_aac
creation_time : 2016-01-30T17:46:24.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
[hevc_vulkan @ 0x5cdf853901c0] Encode failed: -939502976.
[vost#0:0/hevc_vulkan @ 0x5cdf821dd440] [enc:hevc_vulkan @ 0x5cdf821e1100] Error submitting video frame to the encoder
[vost#0:0/hevc_vulkan @ 0x5cdf821dd440] [enc:hevc_vulkan @ 0x5cdf821e1100] Error encoding a frame: Error number -939502976 occurred
[vost#0:0/hevc_vulkan @ 0x5cdf821dd440] Task finished with error code: -939502976 (Error number -939502976 occurred)
[vost#0:0/hevc_vulkan @ 0x5cdf821dd440] Terminating thread with return code -939502976 (Error number -939502976 occurred)
[out#0/mpegts @ 0x5cdf8539c500] video:0KiB audio:196KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 21.917399%
frame= 0 fps=0.0 q=0.0 Lsize= 239KiB time=N/A bitrate=N/A dup=2 drop=0 speed=N/A
Conversion failed!
Attempting to strip out HDR and a53cc with -units:v -hdr-a53_cc also results in the same failure states above.
H.264 encoding pipelines from the media samples I've tested with also fail/die the same way, with the exception of lavfi sources like this:
(a). HEVC:
ffmpeg -v verbose -init_hw_device vulkan=vk:0 \ -filter_hw_device vk \ -f lavfi -i testsrc=duration=3600:size=1920x1080:rate=60,format=nv12 -vf "hwupload" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v hevc_vulkan \ -profile:v main -r:v 60 -g:v 120 -tune:v hq -usage:v transcode -content:v rendered \ -async_depth:v 4 -refs:v 8 -b_depth:v 1 -bf:v 0 -strict -1 \ -f null /dev/null -y
That passes with NVIDIA and FAILS on Intel, and;
(b). H.264/AVC:
ffmpeg -v verbose -init_hw_device vulkan=vk:0 \ -filter_hw_device vk \ -f lavfi -i testsrc=duration=3600:size=1920x1080:rate=60,format=nv12 -vf "hwupload" \ -b:v 2250k -minrate:v 2250k -maxrate:v 2250k -bufsize:v 2250k -c:v h264_vulkan \ -profile:v high -r:v 60 -g:v 120 -tune:v hq -usage:v transcode -content:v rendered \ -async_depth:v 4 -refs:v 8 -b_depth:v 1 -bf:v 0 -strict -1 \ -f null /dev/null -y
Which passes on BOTH NVIDIA and Intel.
The flag -strict 1 has no impact on the results.
Note:
See TracTickets
for help on using tickets.
