FFmpeg Windows version with QSV hwaccel fails over TERMINAL

[Mark Sanders]

Summary of the bug:
How to reproduce:

 ffmpeg.exe -hwaccel qsv -c:v h264_qsv -i file.ts -filter_complex "[i:256]deinterlace_qsv,scale_qsv=720:576[out]" -map [out] -map i:257 -c:0 h264_qsv -preset medium -c:1 copy -f mpegts output.ts

The privious command executed in Windows, reads one MPEG-TS file with one video stream (pid 256) and one audio stream (pid 257). The filter graph deinterlaces the input video (H.264 1080i content) and downscale to PAL size. Then it compress it using default settings in H.264 with the hardware encoder. All is done in the GPU.

OK, this works... when executed inside a COMMAND TERMINAL in a Windows session (even over RDP remote connections).

However, when executing identical command inside a OpenSSH session this error appears:

[AVHWDeviceContext @ 000002914a559000] Failed to create Direct3D device
Error creating a QSV device
qsv hwaccel requested for input stream #0:0, but cannot be initialized.
Error while decoding stream #0:0: Operation not permitted

The OpenSSH session used is the STANDARD from Microsoft for Windows 10. So no 3rd party or experimental code is used.

I'm not sure if this can be fixed. But I report it here as using the CUVID hwaccel for NVIDIA it works as expected inside the same TERMINAL.

I hope this will be fixed sometime.
Regards.

[Hendrik]

QSV currently uses D3D9. For full headless operation, D3D11 would be required. I do not know if its fully compatible with that yet.

CUVID on the other hand can natively work headless (ie. through CUDA), and NVENC encoding also with D3D11, so its more flexible in that regard.

[Mark Sanders]

Hi heleppkes,

Thank you for the response! ;)

So the problem is the D3D9 use instead of D3D11 for the QSV accelerator?
If yes, then I hope someone can upgrade the code. Or if this is impossible, then that Intel will upgrade his library for DX11 support.

Any idea about it?

[Mark Sanders]

Hi,

Regarding the use of D3D11 or D3D9 with the QSV accelator I noted this:

• The current QSV filters, have support for both types of handles. See as example: ​https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/qsvvpp.c#L74

• But the hardware device initialization for QSV only has support for D3D9. See the code: ​https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/hwcontext_qsv.c#L93

So I figure that the hwcontext for QSV needs to add support for handles with D3D11. Then the offscreen processing will be available.

I'm right?

[Mark Sanders]

Hi,

I found specific information in offical documentation from Intel:
​https://jp.xlsoft.com/documents/intel/media/2014/Intel_Media_Developers_Guide_0.pdf

In section "4.18.1 Muti-GPU and “headless” configurations" indicates:

Applications wishing to leverage the Intel Media SDK’s hardware acceleration library
when a discrete card is the primary device, or on devices without a monitor attached –
such as “Session 0” modes, are required to initialize the Intel Media SDK to utilize the
DirectX11 infrastructure, as well as provide its own memory allocation routines that
manage DirectX 11 surfaces.

So, the solution seems to include a new parameter when initializing the QSV context to select D3D11 when calling to MFXInit().

What you think?

[Mark Sanders]

Hi,

I need help to overcome this problem. I really need to use the *_QSV filters offscreen.
So, any idea to move from D3D9 to D3D11 with the QSV context?

[compn]

do you need strictly headless? or will psexec work ?

I see others using ffmpeg with psexec ...

Until Windows 10 allows export DISPLAY.. everything ran from ssh in windows 10 is "session 0" , and windows has locked it down so that you cannot get to other sessions.

[Anton Torp]

Hi,

I found a solution to this problem: Use initialization of D3D11 in the command line.

• Example (based on the initial command described above):

ffmpeg.exe -init_hw_device d3d11va=qsv:MFX_IMPL_hw_any -hwaccel qsv -filter_hw_device qsv -c:v h264_qsv -i file.ts -filter_complex "[i:256]deinterlace_qsv,scale_qsv=720:576[out]" -map [out] -map i:257 -c:0 h264_qsv -preset medium -c:1 copy -f mpegts output.ts

So the trick is to use "-init_hw_device d3d11va=qsv:MFX_IMPL_hw_any" before the "-hwaccel qsv", and "-filter_hw_device qsv" after it. Using this, instead of initialize the MFX context using a D3D9 surface, it's enforced to use a D3D11 surface.

I hope it helps!

Note: I suggest to add some example of D3D11va initialization in the documentation: ​https://trac.ffmpeg.org/wiki/Hardware/QuickSync

[Anton Torp]

Hi,

I found the root cause of this problem:

• As described by @heleppkes the difference is the type of the surfaces used. When using D3D11 surfaces the processing runs wihtout troubles even offscreen (from the Terminal). But the error appears when using D3D9 surfaces.

However, the problem can be fixed with a very small change: initializing the QSV surfaces as D3D11. When using one nVIDIA card in the same machine with one INTEL CPU with iGPU is possible to initialize using D3D11 using the current ffmpeg binary:

ffmpeg -hwaccel cuvid -c:v h264_cuvid -i input.ts -c:v h264_qsv -f mpegts output.ts

When executing this command the ffmpeg tool uses the QSV hardware accelerator to encode using the H264 codec. However, first it uses the NVIDIA card to decode the input. In this case, as the DECODER initializes D3D11 surfaces, the ENCODER receives D3D11 surfaces and it works!

So the QSV encoder source code doesn't need to be changed. Only the initialization part requires a small change to move from D3D9 surfaces to D3D11 surfaces. The rest of the code already works.

I hope this helps to fix this problem.
Regards.

[ajax16384]

possible solution ​https://trac.ffmpeg.org/ticket/7933#comment:3

[MrGrymReaper]

This really could do with being fixed, as it is holding up other projects. Possibly by seeing if the fix referenced in comment 10 is viable.