libx265 segfault error while encoding

[Arno BERND]

Summary of the bug: Unpredictable segault errors when trying to encode multiple videos with libx265.
How to reproduce:

Just use about 6 parallel running commands
% ffmpeg -loglevel quiet -y -i <file_mp4> -vcodec libx265 -x265-params log-level=-1 <file2_mp4> 
ffmpeg version 4.3.1
built on SUSE Linux with gcc 10

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Arno BERND]

output from dmesg when error occurs

[Carl Eugen Hoyos]

Replying to babu59:

Unpredictable segault errors

This usually does not indicate a bug that can be fixed in software.

To make this a valid ticket, please test current FFmpeg git head and provide backtrace, disassembly and register dump as explained on https://ffmpeg.org/bugreports.html

[Arno BERND]

I discovered, that this error seems to be a run into architecture limitations, when I use libx265 without pool and frame limitations. I cannot really ensure this because I am still not able to reproduce the error consistently.

Since I limited "pools" and "frames" I am able to run multiple conversions in parallel to be able to use 100% CPU on a modern processor with 16/32 cores/threads.

I assume it's a stack pointer overflow.

Problem is now solved for me!

[Arno BERND]

To complete my infomation:

Command which works without errors is now

ffmpeg -loglevel quiet -y -i <file_mp4> -vcodec libx265 -x265-params log-level=-1:frame-threads=1:pools=6 <file2>

Running 10 ffmpeg-tasks in parallel with full-HD (1920x1080) uses about 99% CPU on AMD Ryzen 9 5950x.