Feature Request: Add ability to use Quick Sync to transcode video files

[zjacobs]

Hello,

Intel recently open sourced the necessary tools to be able to use hardware accelerated video encoding and decoding. This would be a valuable feature for FFmpeg to utilize this technology for transcoding workflows.

​https://01.org/linuxgraphics/downloads/2013/2013q1-intel-graphics-stack-release

[Carl Eugen Hoyos]

Isn't it both slower and worse quality than libx264?

[zjacobs]

According to my research and actual use, it may be "worse quality" than libx264 but it's faster. Additionally, it's hard to notice much of a difference when transcoding for mobile devices anyway.

​http://www.anandtech.com/show/5771/the-intel-ivy-bridge-core-i7-3770k-review/21
​http://forum.doom9.org/showthread.php?t=166958

[Carl Eugen Hoyos]

Replying to zjacobs:

According to my research and actual use, it may be "worse quality" than libx264 but it's faster. Additionally, it's hard to notice much of a difference when transcoding for mobile devices anyway.

​http://www.anandtech.com/show/5771/the-intel-ivy-bridge-core-i7-3770k-review/21

I did not find any comparison between x264 and QuickSync in this article. Did I miss it?

​http://forum.doom9.org/showthread.php?t=166958

It is claimed here that x264 Ultrafast is faster than QuickSync.

​http://www.compression.ru/video/codec_comparison/h264_2012/mpeg4_avc_h264_video_codecs_comparison.pdf seems to indicate to me that QuickSync has no advantages over x264 but maybe your results are different?

[zjacobs]

Replying to cehoyos:

Replying to zjacobs:

According to my research and actual use, it may be "worse quality" than libx264 but it's faster. Additionally, it's hard to notice much of a difference when transcoding for mobile devices anyway.

​http://www.anandtech.com/show/5771/the-intel-ivy-bridge-core-i7-3770k-review/21

I did not find any comparison between x264 and QuickSync in this article. Did I miss it?

They used x264 in their handbrake test: "We took the profile and performed the same transcode, the result is listed above as the Core i7 3770K (Handbrake). You will notice that the Handbrake x86/x264 path is definitely faster than Cyberlink's software path, by over 50% to be exact. However even using Handbrake as a reference, Quick Sync transcodes over 2x faster."

​http://forum.doom9.org/showthread.php?t=166958

It is claimed here that x264 Ultrafast is faster than QuickSync.

One poster states ultrafast is "probably quite a bit faster than quicksync but provides no report of his actual experiences". Towards the bottom of the thread 2 users share their experiences that Quick Sync is faster. In my experiences, Ultrafast is pretty fast but still not as fast as QuickSync. It also maxes out all 4 of my CPU cores rendering my machine pretty useless.

​http://www.compression.ru/video/codec_comparison/h264_2012/mpeg4_avc_h264_video_codecs_comparison.pdf seems to indicate to me that QuickSync has no advantages over x264 but maybe your results are different?

Thanks for the link. The conclusions behind appendix 6.6 might point to reasons that my platform doesn't perform as well with x264. From the comparison: "This test shows that using Laptop hardware with weaker CPU with basic integrated GPU hardware encoder QuickSync is better in terms speed/quality trade-off than best pure software encoder x264 at very high-speed encoding."

[Carl Eugen Hoyos]

Replying to zjacobs:

​http://www.anandtech.com/show/5771/the-intel-ivy-bridge-core-i7-3770k-review/21

I did not find any comparison between x264 and QuickSync in this article. Did I miss it?

They used x264 in their handbrake test: "We took the profile and performed the same transcode, the result is listed above as the Core i7 3770K (Handbrake). You will notice that the Handbrake x86/x264 path is definitely faster than Cyberlink's software path, by over 50% to be exact. However even using Handbrake as a reference, Quick Sync transcodes over 2x faster."

This is missing the clarification that they actually tested with ultrafast and superfast (and possibly some finer presets in-between to test how quality compares at the exact same speed), but please understand that I did not want to move the discussion here (I very rarely encode to h264 and have honestly no opinion on this matter), I just wanted to warn you that more educated people on this matter than I am think that once this is implemented you will be very disappointed with the results.
Otoh, if you need to use your CPU while re-encoding, hardware encoding may have its values (if you don't care about the encoding quality).

[Tim]

In my experience, Quick Sync is most useful when it handles both video decoding and encoding (much lower CPU usage, fan noise and power consumption) - and that's where Quick Sync will outperform x264 ultrafast by a large margin on consumer-grade hardware. Still, Quick Sync encode-only can also outperform ultrafast on just about any ultrabook, AFAIK.

FWIW, there is ongoing to work to integrate QSV as a hwaccel in Libav:

​https://github.com/maximd33/libav/tree/qsv
​https://github.com/DonDiego/libav/tree/qsv

Only H.264 decode is implemented so far, IIRC - haven't been following this very closely though.

[dreamcat4]

A fork of ffmpeg has implemented quick sync support.

​https://github.com/drocon11/ffmpeg-qsv

HOWEVER, like all before it - depends on the Intel Media SDK as a build requirement. Which Intel only provides Windows and Linux. (no OS X, no FreeBSD).

Unlikely to not need some pre-compiled assembly, DLL or ELF / .so lib files to link from the Media SDK. Even Videolan VLC source code must also use Intel Media SDK.

​https://github.com/videolan/vlc/blob/master/modules/codec/qsv.c

Best we can hope for is to make some new optional build choice on Linux and Windows only. To include / exclude optional Intel Media SDK dependancy.

[Carl Eugen Hoyos]

If libmfx.lib is available only as a binary, configure has to be changed so it requires --enable-nonfree for qsv: You cannot fulfill the requirements of the GPL for libmfx.

[Em Ka]

Now every System is supported. So please add this.
And especially Apple seems to really like Quick-Sync (used Airplay, Quicktime, FaceTime, iMovie).

Wikipedia:

Linux
Quick Sync support by Intel Media SDK on Linux is available,[9] although as of March 2013 no application has been reported to integrate it.

Microsoft Windows
Microsoft offers a wide support for Quick Sync in Windows based on supporting driver software from Intel and good support through both DirectShow/DirectX as well as WMF (Windows Media Foundation). A wide range of applications are based upon this base support for the technology in Windows. Windows support is available from Windows Vista onward.

OS X
Apple added Quick Sync support in OS X Mountain Lion for AirPlay Mirroring, FaceTime and QuickTime X.[10] iMovie 10 uses Quick Sync when exporting videos.

[dreamcat4]

Replying to cehoyos:

If libmfx.lib is available only as a binary, configure has to be changed so it requires --enable-nonfree for qsv: You cannot fulfill the requirements of the GPL for libmfx.

So if I understand what you are saying… there is not even a non-default option to compile with quick sync support. Because that would violate ffmpeg's licensing terms? Some BSD License (for ffmpeg) might have really helped here. In this instance I really don't understand how the GPL?|LGPL? is any less draconian than Intel's terms on their Media SDK.

[Hendrik]

There is ​https://git.videolan.org/?p=mfx_dispatch.git which is supposed to be a "open" dispatcher library to replace the binary libmfx.lib.

I have no idea if its fully functional yet.

[Carl Eugen Hoyos]

Replying to dreamcat4:

Replying to cehoyos:

If libmfx.lib is available only as a binary, configure has to be changed so it requires --enable-nonfree for qsv: You cannot fulfill the requirements of the GPL for libmfx.

So if I understand what you are saying… there is not even a non-default option to compile with quick sync support. Because that would violate ffmpeg's licensing terms? Some BSD License (for ffmpeg) might have really helped here. In this instance I really don't understand how the GPL?|LGPL? is any less draconian than Intel's terms on their Media SDK.

Before the patch in comment:8 can be reviewed (and applied), it has to be changed so that --enable-nonfree is necessary to allow compilation with libmfx.

[dreamcat4]

Replying to cehoyos:

Before the patch in comment:8 can be reviewed (and applied), it has to be changed so that --enable-nonfree is necessary to allow compilation with libmfx.

Ah okay (sorry for going off on one). I have contacted the persons involved and ask them here. For to make the necessary amendment. Many thanks.

[Matt]

Apologies for posting a question here, but you guys seem pretty smart so you'll probably know the answer straight away.

I'm trying to compile a version of ffmpeg with quicksync support for use in a Windows environment. I've used the pre-compiled binary from the drocon11 fork, but unfortunately I need a few other libraries included in the binary (so this is why I figured I need to build this myself?).

I'm just a little confused with the exact environment and tools I should be building with. I've tried to follow the various instructions within some of the dependencies but for instance I couldn't even work out if I should be doing this in a "msys" on Windows, or within Linux.

If somebody could outline with a bit more detail how I should go about building the drocon11/ffmpeg-qsv project I'd really appreciate it. Particularly if you could mention the OS, tools and any installation guidelines I should follow that would be great.

If it helps, the background to this is that I have a low power j1900 cpu running media browser 3 as a media server, but the cpu struggles why real-time transcoding, however the j1900 supports quicksync and I can get some pretty good performance using quicksync. Unfortunately, the way in which media browser uses ffmpeg seems to require some additional libraries which are bit included in the drocon11 build.

Thanks again, Matt

[Matt]

OK, a few weeks later and I've made a little progress now, having figured out I should be compiling in mysys on Windows using a MinGW-W64 32bit toolchain.

I seem to be able to compile ​https://github.com/lu-zero/mfx_dispatch, but when it comes to compiling ​https://github.com/drocon11/ffmpeg-qsv I get the following error when running configure:

check_pkg_config libmfx mfx/mfxvideo.h MFXInit
pkg-config --exists --print-errors libmfx
check_func_headers mfx/mfxvideo.h MFXInit -IC:/MinGW/msys/1.0/local/include -LC:/MinGW/msys/1.0/local/lib -lmfx -lstdc++
check_ld cc -IC:/MinGW/msys/1.0/local/include -LC:/MinGW/msys/1.0/local/lib -lmfx -lstdc++
check_cc -IC:/MinGW/msys/1.0/local/include -LC:/MinGW/msys/1.0/local/lib
BEGIN /tmp/ffconf..VisualStudio-PC.500.10576.c
    1   #include <mfx/mfxvideo.h>
    2   long check_MFXInit(void) { return (long) MFXInit; }
    3   int main(void) { return 0; }
END /tmp/ffconf..VisualStudio-PC.500.10576.c
gcc -D_ISOC99_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -U__STRICT_ANSI__ -D__USE_MINGW_ANSI_STDIO=1 -D__printf__=__gnu_printf__ -std=c99 -fomit-frame-pointer -IC:/MinGW/msys/1.0/local/include -LC:/MinGW/msys/1.0/local/lib -c -o /tmp/ffconf..VisualStudio-PC.500.10576.o /tmp/ffconf..VisualStudio-PC.500.10576.c
gcc -Wl,--nxcompat -Wl,--dynamicbase -Wl,--as-needed -IC:/MinGW/msys/1.0/local/include -LC:/MinGW/msys/1.0/local/lib -o /tmp/ffconf..VisualStudio-PC.500.10576.exe /tmp/ffconf..VisualStudio-PC.500.10576.o -lmfx -lstdc++ -lm -lz -lpsapi -ladvapi32 -lshell32 -lsupc++ -lstdc++
C:/MinGW/msys/1.0/local/lib/libmfx.a(main.o): In function `MFXInit':
C:\MinGW\msys\1.0\home\JohnSmith\mfx_dispatch-master/src/main.cpp:454: undefined reference to `MFX::MFXDefaultPlugins::MFXDefaultPlugins(mfxVersion, MFX_DISP_HANDLE*, int)'
C:\MinGW\msys\1.0\home\JohnSmith\mfx_dispatch-master/src/main.cpp:460: undefined reference to `MFX::MFXPluginsInHive::MFXPluginsInHive(int, wchar_t const*, mfxVersion)'
C:\MinGW\msys\1.0\home\JohnSmith\mfx_dispatch-master/src/main.cpp:468: undefined reference to `MFX::MFXPluginsInHive::MFXPluginsInHive(int, wchar_t const*, mfxVersion)'
C:\MinGW\msys\1.0\home\JohnSmith\mfx_dispatch-master/src/main.cpp:472: undefined reference to `MFX::MFXPluginsInFS::MFXPluginsInFS(mfxVersion)'
collect2.exe: error: ld returned 1 exit status
ERROR: libmfx not found

I have detailed the steps I've taken to set-up my build environment and compile lib_mfx here:
​https://github.com/lu-zero/mfx_dispatch/issues/4#issuecomment-61520137 I don't suppose anyone has seen this error before, or know what could be the problem?

Matt

[Mista_D]

Is it not included in 2.7 already? There's a qsv module now with libmfx dependency.

[TheTroll]

Yes but it is widely incomplete, decoding does not work at all (under linux at least), and VPP is not supported...

[TheTroll]

Yes but it is widely incomplete, decoding does not work at all (under linux at least), and VPP is not supported...

[Mista_D]

$500 CAD bounty (Paypal or WesternUnion) for a complete integration of Intel QuickSync's encoding, decoding and VPP filtering features. Bounty expires on November 1st 2015.

[iaz-art]

Replying to Mista_D:

$500 CAD bounty (Paypal or WesternUnion) for a complete integration of Intel QuickSync's encoding, decoding and VPP filtering features. Bounty expires on November 1st 2015.

How can I contact you regarding complete integration of Intel QuickSync's encoding, decoding and VPP filtering features. with ffmpeg

[Mista_D]

Replying to iaz-art:

Replying to Mista_D:

$500 CAD bounty (Paypal or WesternUnion) for a complete integration of Intel QuickSync's encoding, decoding and VPP filtering features. Bounty expires on November 1st 2015.

How can I contact you regarding complete integration of Intel QuickSync's encoding, decoding and VPP filtering features. with ffmpeg

Please email me at "toronto.ffmpeg _AT_ gmail.com"

[bordon]

Hi,iaz-art:
Did you have integrate Intel QuickSync's decoding and VPP filtering features into ffmpeg ? I want using it too, and I can pay for you if your integration can work correctly.
My Email: ​lucky.bordon@gmail.com

[bordon]

Replying to iaz-art:

I also want to use it if you can integrate Intel QSV decoding and VPP filtering features into ffmpeg,and I can pay for you if your integration can work correctly.
My Email: ​​lucky.bordon@gmail.com

[Carl Eugen Hoyos]

I believe this was implemented some time ago. If necessary features are missing that are not part of a newer ticket, please open a new ticket with clear explanations.