libplacebo segfault on invocation

[Dennis E. Mungai]

Summary of the bug:

The filter libplacebo will segfault on any invocation.

How to reproduce:

The segfault was caught by gdb:

gdb --args /home/brainiarc7/ffmpeg_sources/FFmpeg/ffmpeg_g -init_hw_device cuda=cuda:0 -filter_hw_device cuda -i feeds.mp4 -filter_complex "[0:v]hwupload=derive_device=vulkan,libplacebo=w=1920:h=1080,hwupload=derive_device=cuda[s0]" -map "[s0]" -c:v:0 h264_nvenc -map "0:a" -c:a copy -f mpegts cheek.ts

Output:

gdb /home/brainiarc7/ffmpeg_sources/FFmpeg/ffmpeg_g -init_hw_device cuda=cuda:0 -filter_hw_device cuda -i feeds.mp4 -filter_complex "[0:v]hwupload=derive_device=vulkan,libplacebo=w=1920:h=1080,hwupload=derive_device=cuda[s0]" -map "[s0]" -c:v:0 h264_nvenc -map "0:a" -c:a copy -f mpegts cheek.ts
gdb: unrecognized option '-init_hw_device'
Use `gdb --help' for a complete list of options.
brainiarc7@brainiarc7-P7xxTM1:~/Desktop/src$ gdb --args /home/brainiarc7/ffmpeg_sources/FFmpeg/ffmpeg_g -init_hw_device cuda=cuda:0 -filter_hw_device cuda -i feeds.mp4 -filter_complex "[0:v]hwupload=derive_device=vulkan,libplacebo=w=1920:h=1080,hwupload=derive_device=cuda[s0]" -map "[s0]" -c:v:0 h264_nvenc -map "0:a" -c:a copy -f mpegts cheek.ts
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/brainiarc7/ffmpeg_sources/FFmpeg/ffmpeg_g...
(gdb) run
Starting program: /home/brainiarc7/ffmpeg_sources/FFmpeg/ffmpeg_g -init_hw_device cuda=cuda:0 -filter_hw_device cuda -i feeds.mp4 -filter_complex \[0:v\]hwupload=derive_device=vulkan,libplacebo=w=1920:h=1080,hwupload=derive_device=cuda\[s0\] -map \[s0\] -c:v:0 h264_nvenc -map 0:a -c:a copy -f mpegts cheek.ts
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
ffmpeg version N-104582-g85a6b7f7b7 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --pkg-config-flags=--static --enable-static --disable-shared --prefix=/home/brainiarc7 --bindir=/home/brainiarc7/bin --enable-cuda-nvcc --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include/ --extra-ldflags=-L/usr/local/cuda/lib64/ --enable-nvenc --enable-libass --enable-debug --enable-gpl --cpu=native --enable-libfdk-aac --enable-libx264 --enable-libx265 --enable-openssl --enable-librtmp --enable-libzvbi --enable-version3 --enable-opencl --enable-pic --enable-librav1e --enable-libglslang --enable-vulkan --enable-libsrt --enable-avisynth --enable-libsvtav1 --enable-libvpx --enable-libplacebo --enable-librav1e --nvccflags='-gencode arch=compute_75,code=sm_75 -O2' --extra-libs='-lpthread -lm -lz -ldl' --enable-nonfree
  libavutil      57.  9.100 / 57.  9.100
  libavcodec     59. 13.100 / 59. 13.100
  libavformat    59.  9.101 / 59.  9.101
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 17.100 /  8. 17.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
[New Thread 0x7fffe573c000 (LWP 259774)]
[New Thread 0x7fffe4f3b000 (LWP 259775)]
[New Thread 0x7fffe1fff000 (LWP 259776)]
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'feeds.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.5.100
  Duration: 01:00:02.13, start: 0.000000, bitrate: 13533 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 13144 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 384 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
    Side data:
      audio service type: main
File 'cheek.ts' already exists. Overwrite? [y/N] y
[New Thread 0x7fffe17fe000 (LWP 259809)]
[New Thread 0x7fffe0ffd000 (LWP 259810)]
[New Thread 0x7fffbffff000 (LWP 259811)]
[New Thread 0x7fffbf7fe000 (LWP 259812)]
[New Thread 0x7fffbeffd000 (LWP 259813)]
[New Thread 0x7fffbe7fc000 (LWP 259814)]
[New Thread 0x7fffbdffb000 (LWP 259815)]
[New Thread 0x7fffbd7fa000 (LWP 259816)]
[New Thread 0x7fffbcff9000 (LWP 259817)]
[New Thread 0x7fffbc7f8000 (LWP 259818)]
[New Thread 0x7fffbbff7000 (LWP 259819)]
[New Thread 0x7fffbb7f6000 (LWP 259820)]
[New Thread 0x7fffbaff5000 (LWP 259821)]
[New Thread 0x7fffba7f4000 (LWP 259822)]
[New Thread 0x7fffb9ff3000 (LWP 259823)]
[New Thread 0x7fffb97f2000 (LWP 259824)]
Stream mapping:
  Stream #0:0 (h264) -> hwupload
  hwupload -> Stream #0:0 (h264_nvenc)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[New Thread 0x7fffb8ff1000 (LWP 259827)]
[New Thread 0x7fff7bfff000 (LWP 259828)]
[New Thread 0x7fff7b7fe000 (LWP 259829)]
[New Thread 0x7fff7affd000 (LWP 259830)]
[New Thread 0x7fff7a7fc000 (LWP 259831)]
[New Thread 0x7fff79ffb000 (LWP 259832)]
[New Thread 0x7fff797fa000 (LWP 259833)]
[New Thread 0x7fff78ff9000 (LWP 259834)]
[New Thread 0x7fff73fff000 (LWP 259835)]
[New Thread 0x7fff737fe000 (LWP 259836)]
[New Thread 0x7fff72ffd000 (LWP 259837)]
[New Thread 0x7fff727fc000 (LWP 259838)]
[New Thread 0x7fff71ffb000 (LWP 259839)]
[New Thread 0x7fff717fa000 (LWP 259840)]
[New Thread 0x7fff70ff9000 (LWP 259841)]
[New Thread 0x7fff707f8000 (LWP 259842)]
[New Thread 0x7fff6650a000 (LWP 259869)]
[New Thread 0x7fff65d09000 (LWP 259870)]
[New Thread 0x7fff65508000 (LWP 259871)]
[New Thread 0x7fff64d07000 (LWP 259872)]
[New Thread 0x7fff57fff000 (LWP 259873)]
[New Thread 0x7fff577fe000 (LWP 259874)]
[New Thread 0x7fff56ffd000 (LWP 259875)]
[New Thread 0x7fff567fc000 (LWP 259876)]
[New Thread 0x7fff55ffb000 (LWP 259877)]
[New Thread 0x7fff557fa000 (LWP 259878)]
[New Thread 0x7fff54ff9000 (LWP 259879)]
[New Thread 0x7fff33fff000 (LWP 259880)]
[New Thread 0x7fff2bfff000 (LWP 259881)]
[New Thread 0x7fff337fe000 (LWP 259882)]
[New Thread 0x7fff32ffd000 (LWP 259883)]
[New Thread 0x7fff327fc000 (LWP 259884)]
[New Thread 0x7fff31ffb000 (LWP 259885)]
[New Thread 0x7fff317fa000 (LWP 259886)]
[New Thread 0x7fff30ff9000 (LWP 259887)]
[New Thread 0x7fff2b7fe000 (LWP 259888)]
[New Thread 0x7fff2affd000 (LWP 259889)]
[New Thread 0x7fff2a7fc000 (LWP 259890)]
[New Thread 0x7fff29ffb000 (LWP 259891)]
[New Thread 0x7fff297fa000 (LWP 259892)]
[New Thread 0x7fff28ff9000 (LWP 259893)]
[New Thread 0x7ffef3fff000 (LWP 259894)]
[New Thread 0x7ffef37fe000 (LWP 259895)]
[New Thread 0x7ffef2ffd000 (LWP 259896)]
[New Thread 0x7ffef27fc000 (LWP 259897)]
[New Thread 0x7ffef1ffb000 (LWP 259898)]
[New Thread 0x7ffef17fa000 (LWP 259899)]
[New Thread 0x7ffef0ff9000 (LWP 259900)]
[New Thread 0x7ffed3fff000 (LWP 259901)]
[New Thread 0x7ffecbfff000 (LWP 259902)]
[New Thread 0x7ffed37fe000 (LWP 259903)]
[New Thread 0x7ffed2ffd000 (LWP 259904)]
[New Thread 0x7ffed27fc000 (LWP 259905)]
[New Thread 0x7ffed1ffb000 (LWP 259906)]
--Type <RET> for more, q to quit, c to continue without paging--

Thread 1 "ffmpeg_g" received signal SIGSEGV, Segmentation fault.
0x00005555568af99f in glslang::TScanContext::tokenizeIdentifier() ()

Warm regards,

Dennis.

Update:

This issue can be resolved by explicitly disabling glslang and enabling shaderc instead in libplacebo, via meson's configure stage as shown in the example below:

meson .. -Dprefix="$HOME/ffmpeg_build" -Dbuildtype=release -Dd3d11=disabled -Ddemos=false --libdir=lib -Dvulkan=enabled -Dglslang=disabled --default-library=static -Dvulkan-link=false -Dvulkan-registry="/usr/share/vulkan/registry/vk.xml"

I'll close this ticket (for now), marking it as invalid.

[Balling]

Here on windows it prints Pixel format 'yuv420p10le' is not supported on HDR and works on 8 bit.

[Dennis E. Mungai]

I'll re-open, courtesy of the report by @Bailing.

[Dennis E. Mungai]

@Bailing,

I can get it to work with HDR to SDR tone-mapping on NVENC.
The error you're getting comes from the encoder wrapper you're using, not the filter.
You can specify the correct supported format(s) in the filter directly as shown in the example(s) below (for the respective NVENC encoder wrappers):

HDR to SDR linear Tonemap, to 8-bit H.264

Sample file fetched from ​https://drive.google.com/file/d/1Ic9DZXMSo07EJMqCFaQRKSSrSw6y1mYv/view

Download renamed to ioshdr10.mov

ffmpeg -init_hw_device cuda=cuda:0 -filter_hw_device cuda \
-i ioshdr10.mov -filter_complex "[0:v]hwupload=derive_device=vulkan,libplacebo=w=1920:h=1080:tonemapping=5:format=nv12:colorspace=bt709:color_primaries=1:color_trc=1,hwupload=derive_device=cuda[s0]" \
-map "[s0]" -b:v 13800k -minrate:v 13800k -maxrate:v 13800k -bufsize:v 13800k -c:v h264_nvenc -r:v 59.94 \
-profile:v high -preset:v p7 -rc:v cbr -g:v 120 -gpu:v 0 -strict_gop:v 1 -bf:v 0 -tune:v ll -map "0:a" -c:a aac -f mp4 -y tonemapped_h264.mp4

HDR to SDR linear Tonemap, to 10-bit HEVC:

ffmpeg -init_hw_device cuda=cuda:0 -filter_hw_device cuda \
-i ioshdr10.mov -filter_complex "[0:v]hwupload=derive_device=vulkan,libplacebo=w=1920:h=1080:tonemapping=5:format=p010le:colorspace=bt709:color_primaries=1:color_trc=1,hwupload=derive_device=cuda[s0]" \
-map "[s0]" -b:v 13800k -minrate:v 13800k -maxrate:v 13800k -bufsize:v 13800k -c:v hevc_nvenc -r:v 59.94 \
-profile:v main10 -preset:v p7 -rc:v cbr -g:v 120 -gpu:v 0 -strict_gop:v 1 -bf:v 0 -tune:v ll -map "0:a" -c:a aac -f mp4 -y tonemapped_hevc_10bit.mp4

HDR to SDR linear Tonemap to 8-bit HEVC:

ffmpeg -init_hw_device cuda=cuda:0 -filter_hw_device cuda \
-i ioshdr10.mov -filter_complex "[0:v]hwupload=derive_device=vulkan,libplacebo=w=1920:h=1080:tonemapping=5:format=nv12:colorspace=bt709:color_primaries=1:color_trc=1,hwupload=derive_device=cuda[s0]" \
-map "[s0]" -b:v 13800k -minrate:v 13800k -maxrate:v 13800k -bufsize:v 13800k -c:v hevc_nvenc -r:v 59.94 \
-profile:v main -preset:v p7 -rc:v cbr -g:v 120 -gpu:v 0 -strict_gop:v 1 -bf:v 0 -tune:v ll -map "0:a" -c:a aac -f mp4 -y tonemapped_hevc.mp4

I'll be testing this later on with QuickSync and VAAPI.

[Balling]

What about without tonemap? My main two display are both HDR. :)