How to choose the best hwaccel?

[Mads Johansen]

I have been using auto and consistently gotten dxva2 as the result.
For example ffmpeg -hwaccel auto -i input output will write [vist#0:0/h264 @ 0000021c3915e700] [dec:h264 @ 0000021c39fa4740] Using auto hwaccel type dxva2 with new default device.

With ffmpeg -hwaccels I get

ffmpeg version N-117642-g08b1bffa49 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 14.2.0 (Rev1, Built by MSYS2 project)
  configuration:  --pkg-config=pkgconf --cc='ccache gcc' --cxx='ccache g++' --ld='ccache g++' --extra-cxxflags=-fpermissive --extra-cflags=-Wno-int-conversion --disable-autodetect --enable-cuda --enable-cuvid --enable-d3d12va --enable-d3d11va --enable-dxva2 --enable-nvenc --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-libvmaf --enable-libaom --enable-libvpx --enable-libx264 --enable-libx265 --disable-debug --enable-gpl --enable-vulkan --enable-libplacebo --enable-opencl --enable-schannel --disable-w32threads
  libavutil      59. 45.100 / 59. 45.100
  libavcodec     61. 22.100 / 61. 22.100
  libavformat    61.  9.100 / 61.  9.100
  libavdevice    61.  4.100 / 61.  4.100
  libavfilter    10.  6.100 / 10.  6.100
  libswscale      8.  9.100 /  8.  9.100
  libswresample   5.  4.100 /  5.  4.100
  libpostproc    58.  4.100 / 58.  4.100
Hardware acceleration methods:
cuda
dxva2
d3d11va
opencl
vulkan
d3d12va

My assumption is that the hwaccel option talks with the hardware, so by changing the option the only thing that will be different (for supported codecs) is the speed of the method.

So my question is: How do I chose which (if any, other than auto) hwaccel to choose use?
As methodology for the an informed choice, I would use standard H264 (for example h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160, 25000 kb/s, 25 fps, 25 tbr, 25k tbn (default)) and use any of the above hwaccels, read the file with hwaccel number -> output to NULL and benchmark the wall time each hwaccel takes and chose the fastest.

[Balling]

So, the way I do it, if d3d11 fails I use nvdec, if even it fails I use -c:v hevc_cuvid as input option.

Nvdec is probably the best.

[Mads Johansen]

As stated above, I created a 3 minute Mandelbrot video in yuv420p, yuv422p, yuv444p, yuv420p10le, yuv422p10le and yuv444p10le in 1080p and 2160p.
3 minutes was chosen to allow the decoding to flatten out (as I noticed massive differences in the beginning vs end).
I did 6 decodes of each file to remove jitter.

On windows I used the following command, changing cuda with the other options written above.

ffmpeg -benchmark -hwaccel cuda -i mandelbrot-yuv420p-2160p.mov -f null - > mandelbrot-yuv420p-2160p1cuda.txt 2>&1

The results are as follows:

	utime	stime	rtime
CUDA	393.46	6.14	33.79
d3d11va	396.74	9.35	34.60
d3d12va	393.98	6.44	33.87
dxva2	392.62	6.17	33.79
vulkan	392.14	7.45	33.80
opencl  446.94	6.30	33.69

The outlyer is opencl, which exclusively used CPU to decode the videos.

The only conclusion I can come up with is that it doesn't matter what hwaccel you use.

[Balling]

On windows I used the following command, changing cuda with the other options written above

You are supposed to use -f null NUL instead of -f null - on windows. - (pipe) is not fast on Windows, I believe.

The only conclusion I can come up with is that it doesn't matter what hwaccel you use.

No true, I think nvdec (-hwaccel cuda) takes less energy and less memory than -c:v hevc_cuvid -i mandelbrot-yuv420p-2160p.mov

you can see that in nvidia-smi

[Mads Johansen]

Replying to Balling:

On windows I used the following command, changing cuda with the other options written above

You are supposed to use -f null NUL instead of -f null - on windows. - (pipe) is not fast on Windows, I believe.

No, I'm not :)

ffmpeg -benchmark -hwaccel cuda -i mandelbrot-yuv420p-2160p.mov -f null NUL -  1>mandelbrot-yuv420p-2160p1cuda.txt 2>&1

[AVFormatContext @ 000001a4062b92c0] Unable to choose an output format for 'pipe:'; use a standard extension for the filename or specify the format manually.
[out#1 @ 000001a46f7a1940] Error initializing the muxer for pipe:: Invalid argument
Error opening output file -.
Error opening output files: Invalid argument

The only conclusion I can come up with is that it doesn't matter what hwaccel you use.

No true, I think nvdec (-hwaccel cuda) takes less energy and less memory than -c:v hevc_cuvid -i mandelbrot-yuv420p-2160p.mov

you can see that in nvidia-smi

I don't care about memory, but have the bench: maxrss lines if absolutely required.

About the power consumption:

ffmpeg -benchmark -hwaccel cuda -i mandelbrot-yuv420p-2160p.mov -f null - > mandelbrot-yuv420p-2160p1cuda.txt 2>&1

68W

ffmpeg -benchmark -hwaccel d3d12va -i mandelbrot-yuv420p-2160p.mov -f null - > mandelbrot-yuv420p-2160p1d3d12va.txt 2>&1

68W

ffmpeg -benchmark -hwaccel vulkan -i mandelbrot-yuv420p-2160p.mov -f null - > mandelbrot-yuv420p-2160p1vulkan.txt 2>&1

68W

ffmpeg -benchmark -hwaccel dxva2 -i mandelbrot-yuv420p-2160p.mov -f null - > mandelbrot-yuv420p-2160p1dxva2.txt 2>&1

68W

Conclusion about power consumption: It's identical. That means, again, that my original conclusion was right.
It does not matter which hwaccel you use. EOD.

[Balling]

-f null NUL -

makes no sense. - is pipe. NUL is a file (well, virtual file) on Windows. You cannot output to NUL and to - at the same time.

​https://www.reddit.com/r/ffmpeg/comments/101cgdu/twopass_f_null_nul_still_generate_output_windows/

it needs to be -f null NUL

About the power consumption:

Where is -c:v hevc_cuvid -i mandelbrot-yuv420p-2160p.mov

dxva2 I believe is too ancient and has bugs (Will fail with older libplacebo, but there is no reason to use dxva2
anyway. d3d11va is available on all supported platforms and is bit-exact
unlike dxva2 which always does conversion to 8-bit BGR, see ​https://github.com/mpv-player/mpv/commit/bd3ff140fa8c02253fa14939cba43da074329120). So not realistic to use it nowadays.

[Mads Johansen]

ffmpeg -benchmark -c:v hevc_cuvid -i mandelbrot-yuv420p-2160p.mov -f null - > "cv hevc_cuvid.txt" 2>&1

[Mads Johansen]

Replying to Balling:

-f null NUL -

makes no sense. - is pipe. NUL is a file (well, virtual file) on Windows. You cannot output to NUL and to - at the same time.

​https://www.reddit.com/r/ffmpeg/comments/101cgdu/twopass_f_null_nul_still_generate_output_windows/

it needs to be -f null NUL

About the power consumption:

Where is -c:v hevc_cuvid -i mandelbrot-yuv420p-2160p.mov

dxva2 I believe is too ancient and has bugs (Will fail with older libplacebo, but there is no reason to use dxva2
anyway. d3d11va is available on all supported platforms and is bit-exact
unlike dxva2 which always does conversion to 8-bit BGR, see ​https://github.com/mpv-player/mpv/commit/bd3ff140fa8c02253fa14939cba43da074329120). So not realistic to use it nowadays.

1) The command I posted works, so I'm not going to mess with it.

2) The log file is attached. As I suspected -c:v is an output option, so it made no sense to have it in the beginning.

3) If dxva2 is too ancient, why is it supported?

[Balling]

command I posted works, so I'm not going to mess with it.

It errors out, you said so yourself.

If dxva2 is too ancient, why is it supported

hevc_cuvid and h264_cuvid decoder is literally EXECUTED (or can be if you do not complie h26x decoder) on a GPU. Without any CPU use. That is impossible with dxva2 and d3d11va and d3d11vpp. No one will remove working APIs... We are key library that people rely on.

As I suspected -c:v is an output option, so it made no sense to have it in the beginning.

No, you are just wrong. h264_cuvid, what codec you encoded the file with?

[Balling]

Oh, and also there is a bug on Linux vs windows for real time decoding, not like in your case for as fast as possible, Nvidia-smi takes more energy on Linux, I hope they fix it soon. (GPU is stuck at P2 state, while on Windows it goes to P5, and P3.)

​https://github.com/NVIDIA/open-gpu-kernel-modules/issues/333#issuecomment-2646549118

It does go to P0 when you decode as fast as possible, i.e. what you proposed (slightly wrong on windows, pipe "-" is not supposed to be used, instead use NT kernel's abstraction NUL (this is like /dev/null or like .. and . in linux folders)): ffmpeg -hwaccel cuda -i video.mp4 -an -c:v rawvideo -f null NUL

In fact you can get perfect 100% utilisation if you decode with -c:v h264_cuvid on a file with too much slices (usually Blu-ray has 4 slices, while that file I tested has 41). Samples in my issue here: ​https://code.videolan.org/videolan/vlc/-/issues/26918

[MasterQuestionable]

͏    “Slightly wrong on Windows: pipe "-" is not supposed to be used.
͏    Instead use NT kernel's abstraction "NUL" (this is like "/dev/null").”
͏    .
͏    Probably not much real difference.
͏    Piping nothing or piping to that consumes nothing:
͏    Fundamentally couldn't differ.
͏    ("-f null" of FFmpeg already gives nothing, I believe)

[Balling]

("-f null" of FFmpeg already gives nothing, I believe)

It does not even work, it says

At least one output file must be specified

Did you even try it? Repeatedly trying
ffmpeg -hwaccel cuda -i example.h264 -an -c:v rawvideo -f null -

gives speed=37.7x maximium
ffmpeg -hwaccel cuda -i example.h264 -an -c:v rawvideo -f null NUL
gives speed= 38x

example.h264 file is here ​https://github.com/wang-bin/QtAV/files/1132086/example.zip

it also switches to P0, BTW, as I said: 41% 37C P0 84W / 260W

in nvidia-smi

[MasterQuestionable]

͏    Interesting finding. But I don't use Windows. (though test at times)
͏    Which Windows version did you test on?

͏    What about writing the "null" output to certain actual file?
͏    (would it differ?)

͏    Also maybe it's better to time the whole transcoding task:
͏    Rather than reading speed stats. (lest potential wrong display)


͏    On Nvidia GPU P (Performance/Power) states:
͏    ​https://docs.nvidia.com/gameworks/content/gameworkslibrary/coresdk/nvapi/group__gpupstate.html#details
͏    .
͏    The 2 commands:
͏    ffmpeg -hwaccel cuda -i "example.h264" -an -c:v rawvideo -f null -
͏    ffmpeg -hwaccel cuda -i "example.h264" -an -c:v rawvideo -f null NUL
͏    ; are basically identical.
͏    So probably not the real cause for the P-state switch.

[Balling]

Did you even try it? Repeatedly trying

Frankly speaking unless Anton says I am correct this may just be a cache issue. I tried both ways and waiting. IMHO, NUL file is faster. And I mean it is also recommended to use /dev/null on Linux...

So probably not the real cause for the P-state switch.

It does switch only after 3 or 4 tries and thus a lot of work done...

[MasterQuestionable]

͏    “IMHO, "NUL" file is faster.
͏    And I mean it is also recommended to use "/dev/null" on Linux...”
.
͏    The exact behavior would be mostly implementation dependent.
͏    As the actual effective operation has described:
͏    Hardly real difference under the hood.
͏    (that significantly differs would typically imply deficiency)