libvulkan.so not linked when `--enable-vulkan`.

[Atemu]

Summary of the bug:

When building ffmpeg with --enable-vulkan with vulkan-loader and vulkan-headers present, the vulkan-loader's libvulkan.so is not linked into the ffmpeg executable and ffmpeg is unable to find it at runtime.

Adding --enable-libplacebo aswell as adding libplacebo to the dependencies makes the executable linked against the vulkan-loader again.

$ ldd ./result-bin/bin/ffmpeg | rg vulkan

vs.

$ ldd ./result-bin/bin/ffmpeg | rg vulkan

libvulkan.so.1 => /nix/store/pkrmapls8xvicblivs4mr8qp4xvcm5kh-vulkan-loader-1.3.268.0/lib/libvulkan.so.1 (0x00007f37c4efd000)

Trying to run something like RADV_PERFTEST=video_decode ./result-bin/bin/ffmpeg -init_hw_device "vulkan=vk:0" -hwaccel vulkan -hwaccel_output_format vulkan -i ... results in:

[AVHWDeviceContext @ 0x1db4280] Unable to open the libvulkan library!
Device creation failed: -1313558101.
Failed to set value 'vulkan=vk:0' for option 'init_hw_device': Unknown error occurred
Error parsing global options: Unknown error occurred

strace reveals that it doesn't even attempt to look in vulkan-loader's lib dir.

This bug does _not_ reproduce on ffmpeg 4.4.4, even without libplacebo. libvulkan is linked there. It does repro with 5.1.3.

How to reproduce:

1. Build with --enable-vulkan and vulkan-loader + vulkan-headers in the library search path
2. ldd the resulting executable

Discovered in: ​https://github.com/NixOS/nixpkgs/pull/281753

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[quinkblack]

libvulkan.so and libvulkan.so.1 is used via dlopen. Just make sure the lib can be find via dlopen , it's not a bug.

[Atemu]

Hi, that is indeed the issue; it is not found via dlopen. That is not the root cause however.

We ensure all libs required by the executable can be found via dlopen by explicitly setting the rpath to input-addressed paths of all libraries required by the executable.

Above version 5 however, the ffmpeg executables do not declare themselves to require libvulkan.so unless --enable-libplacebo is set (see output of ldd in the OP).

My understanding is that, because libvulkan.so is not required, the Nixpkgs stdenv does not inject the library into the executable's rpath. I don't know how that would work out on regular FHS distros because it's been years since I used one but I can't imagine it'd work.

The root cause here appears to be ffmpeg's build scripts producing an ELF that is missing the libvulkan.so dependency.

Are you able to reproduce the ldd output of --enable-vulkan --disable-libplacebo vs. --enable-vulkan --enable-libplacebo that I observed on your distro aswell?

[quinkblack]

libplacebo links to libvulkan.so if vulkan is enabled for it, and libavfilter links to libplacebo. FFmpeg don't link to vulkan directly. I don't understand what's the issue here.

[Atemu]

To re-iterate, the issue lies in trying to enable ffmpeg's own Vulkan support for i.e. hwaccel, not libplacebo's filters that use VK internally. Please take a closer look at the original report again.

Enabling (but not using) libplacebo and providing its libs make ffmpeg's Vulkan video hwaccel work on ffmpeg v5 and greater. In ffmpeg v4, Vulkan support works without libplacebo as expected.

(All using the same packaging modulo enabling things in ffmpeg>4 that only exist in ffmpeg>4.)

Are you able to run the reproducer on ffmpeg>4 when configured with --enable-vulkan --disable-libplacebo?

[Atemu]

Using patchelf to add libvulkan.so to the ffmpeg binary's NEEDED section works around the issue.

Why isn't the build system doing that on its own when --enable-vulkan is set?

[Atemu]

(Further investigation revealed that the actual issue is libavcodec.so not having libvulkan.so in NEEDED and therefore not put in rpath by Nix stdenv, so the "proper" workaround is to do that. ​https://github.com/NixOS/nixpkgs/pull/281753/commits/3f89316af000885845d34c9a6776d8344af6a4c9)