FFmpeg QSV initialization fails on KBL HadesCanyon (= both Intel & AMD GPUs) when DISPLAY is set

[eero-t]

Setup:

• Ubuntu 18.04
• KBL i7-8809G: ​https://ark.intel.com/products/130409/Intel-Core-i7-8809G-Processor-with-Radeon-RX-Vega-M-GH-graphics-8M-Cache-up-to-4-20-GHz-
• Latest FFmpeg, MediaSDK, libva, intel-driver and gmmlib built from sources at GitHub
• drm-git kernel (e.g. v4.19 or v4.20)

To reproduce, use FFmpeg QSV backend. For example:

ffmpeg -hwaccel qsv -c:v mpeg2_qsv -i 1920x1080i_29.97_20mb_mpeg2_high.mpv -c:v h264_qsv -b:v 6M 0024_HD17i7_1.0.h264
...
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (mpeg2_qsv) -> h264 (h264_qsv))
Press [q] to stop, [?] for help
DRM_IOCTL_I915_GEM_APERTURE failed: Invalid argument
Assuming 131072kB available aperture size.
May lead to reduced performance or incorrect rendering.
get chip id failed: -1 [2]
param: 4, val: 0
[AVHWDeviceContext @ 0x55a0ad0ee600] Failed to initialise VAAPI connection: 1 (operation failed).
Error creating a QSV device
qsv hwaccel requested for input stream #0:0, but cannot be initialized.
Error while decoding stream #0:0: Operation not permitted
[mpeg2_qsv @ 0x55a0ad0f8e80] video_get_buffer: image parameters invalid
[mpeg2_qsv @ 0x55a0ad0f8e80] get_buffer() failed
Error while decoding stream #0:0: Invalid argument
[mpeg2_qsv @ 0x55a0ad0f8e80] video_get_buffer: image parameters invalid
[mpeg2_qsv @ 0x55a0ad0f8e80] get_buffer() failed
...
[mpeg2_qsv @ 0x55a0ad0f8e80] Too many errors when draining, this is a bug. Stop draining and force EOF.
Error while decoding stream #0:0: Internal bug, should not have happened
[h264_qsv @ 0x55a0ad0fa2c0] Selected ratecontrol mode is unsupported
[h264_qsv @ 0x55a0ad0fa2c0] Current frame rate is unsupported
[h264_qsv @ 0x55a0ad0fa2c0] Current picture structure is unsupported
[h264_qsv @ 0x55a0ad0fa2c0] Current resolution is unsupported
[h264_qsv @ 0x55a0ad0fa2c0] Current pixel format is unsupported
[h264_qsv @ 0x55a0ad0fa2c0] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

It's expected to work, because it works:

• on other KBL devices (GT2, GT3e)
• using FFmpeg with VA-API acceleration instead of QSV with same drivers
• using MediaSDK sample transcode application to do the same thing

QSV backend works if DISPLAY environment variable is unset.

From "strace -f -e openat" output I can see following files to have been opened, when DISPLAY is not set:

...
<input video>
<output video>
/dev/dri/renderD128
/path/to/iHD_drv_video.so

Whereas with DISPLAY set, following are opened instead:

...
<input video>
<output video>
/home/user/.Xauthority
/dev/dri/card1
/path/to/iHD_drv_video.so

When using "latrace" to trace library calls, without DISPLAY I see following:

...
XOpenDisplay(nil)
XDisplayName(0, , , 4720)
vaGetDisplayDRM(5, , 2, 0)
vaSetErrorCallback(, , , )
vaSetInfoCallback(, , , )
vaInitialize(, , , ) = 0

Whereas with DISPLAY set, it goes:

...
XOpenDisplay(nil)
vaGetDisplay(, 85, , 0)
XDisplayName(0, 0, , )
vaSetErrorCallback(, , , )
vaSetInfoCallback(, , , )
vaInitialize(, , , ) = 1

Above differences seem to come from "vaapi_device_create":
​https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/hwcontext_vaapi.c#L1482

Even if user specifies correct device for FFmpeg with -hwaccel_device /dev/dri/renderD128, same thing is done when DISPLAY is set, although X doesn't have anything to do with video transcoding.

[Zhong,Li]

hwaccel_device is ignored to create a vaapi device: ​https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/hwcontext_qsv.c#L1232, vaapi_device_create() always receives a NULL device. Thus is a problem for multi-gpu case.
Actually hwaccel_device for qsv just means "device selects a value in ‘MFX_IMPL_*’".

[Zhong,Li]

Please use "qsv_device" to specify the child_device, and try again:
fmpeg -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v mpeg2_qsv -i 1920x1080i_29.97_20mb_mpeg2_high.mpv -c:v h264_qsv -b:v 6M 0024_HD17i7_1.0.h264

[eero-t]

Specifying qsv_device makes it work, as does unsetting $DISPLAY.

As I need to specify device already for VA-API to work, it's fine for me to do that also for QSV.

Before fixing the render node selection in code, please document "qsv_device" usage (for multi-card use-cases) in FFmpeg QSV examples: ​https://trac.ffmpeg.org/wiki/Hardware/QuickSync

[Zhong,Li]

Replying to eero-t:

Specifying qsv_device makes it work, as does unsetting $DISPLAY.

Unsetting $DISPALY is just a workaround as you meantioned. But specifying qsv child device is necessary when you have multiple GPU devices, and this is a real solution.

As I need to specify device already for VA-API to work, it's fine for me to do that also for QSV.

So, is it ok to close this ticket now?

Before fixing the render node selection in code, please document "qsv_device" usage (for multi-card use-cases) in FFmpeg QSV examples: ​https://trac.ffmpeg.org/wiki/Hardware/QuickSync

Good suggestion, will add it later. Thanks for remind.

[eero-t]

On devices which have only single Intel GPU, it should be possible for a driver of that HW to be able to find the correct one without user needing to specify it.

For example, if /sys/class/drm/renderD128/device/vendor isn't Intel, /dev/dri/renderD128 is unlikely to be a suitable device for Intel video driver.

And if on some machine multiple GPUs identify themselves with Intel vendor, driver could default to first which /sys/class/drm/<rendernode>/device/device ID matches a supported HW ID, unless user specifies something else.

I guess correct place for these checks would be in the libva backend, as it's the one that knows what HW it supports. If there isn't any support for that, could you file a bug about that to appropriate place, and link it here as a blocker?

[Zhong,Li]

Replying to eero-t:

And if on some machine multiple GPUs identify themselves with Intel vendor, driver could default to first which /sys/class/drm/<rendernode>/device/device ID matches a supported HW ID, unless user specifies something else.

I guess correct place for these checks would be in the libva backend, as it's the one that knows what HW it supports. If there isn't any support for that, could you file a bug about that to appropriate place, and link it here as a blocker?

​https://github.com/intel/media-driver is the driver used by MSDK and FFmpeg-qsv. You can report an issue there.
However, I don't think it is helpful When you set a AMD GPU as display device but the libva driver is to get a device from a X11 display. The only way in my mind is to make FFmpeg connect to a drm device directly, this is not a default path and should be set manually (you suggestion is just to chose an appropriate default drm device automatically)

[Carl Eugen Hoyos]

Which commit fixed the issue?

[Zhong,Li]

Replying to cehoyos:

Which commit fixed the issue?

No commit, but using "qsv_device" to specify the child_device can fix the issue.
Maybe we should not mark it as "fixed", just something else just like "invalid" or "worksforme"?

[eero-t]

Well, I wouldn't say that it's invalid, more of a issue in the driver itself that the poor thing cannot find the only device which it works with, without user explicitly pointing out the device through FFmpeg command line option.

E.g. in Mesa bug tracker, resolution for that would be NOTOURBUG.

[eero-t]

Replying to cehoyos:

There is actually something that SHOULD to be fixed. None of the QSV examples in the QSV wiki page mention "-qsv_device" option, or it being needed on multi-GPU systems:

​https://trac.ffmpeg.org/wiki/Hardware/QuickSync

[Zhong,Li]

Replying to eero-t:

Replying to cehoyos:

There is actually something that SHOULD to be fixed. None of the QSV examples in the QSV wiki page mention "-qsv_device" option, or it being needed on multi-GPU systems:

​https://trac.ffmpeg.org/wiki/Hardware/QuickSync

General speaking, only some common/typical usage examples are needed to be introduced.
However, I believe this ticket is an typical case, so it has been documented now.

[eero-t]

Please add a subheading for that single example, something like "Setups with multiple GPUs". Otherwise it's way too each to miss.

(I haven't found/noticed "-qsv_device" being mentioned anywhere else in the FFmpeg documentation.)