NVENC: "InitializeEncoder failed" error occurs when level 5.2 is specified

[alexpigment]

Summary of the bug:

When using h264_nvenc, if -level 5.2 is specified in the command line parameters, the following error is given:

[h264_nvenc @ 0546f2c0] InitializeEncoder failed: invalid param (8)
Error initializing output stream 0:0 -- Error while opening encoder for output s
tream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

While NVENC via FFMPEG will similarly fail if trying to specify a level that is not valid for the current set of parameters, this case is different because it will fail even when the parameters are valid. Leaving off -level 5.2 in this scenario will produce a file that is Level 5.2.

How to reproduce:

% ffmpeg -f lavfi -i smptebars -c:v h264_nvenc -s 3840x2160 -aspect 16:9 -profile:v high -r 59.94 -level 5.2 -t 5 output.mp4

Confirm that the above code fails. Now do the same encode without specifying -level 5.2:

% ffmpeg -f lavfi -i smptebars -c:v h264_nvenc -s 3840x2160 -aspect 16:9 -profile:v high -r 59.94 -t 5 output.mp4

Notice that this works, and produces a video that is Level 5.2 according to MediaInfo.

ffmpeg version N-86330-gbd1179e
built on 2017-06-01

[hydra3333]

according to ffmpeg help, the levels only go up to 5.1 :(

and that's with the git as of two days ago (2020.06.12)

[hydra3333]

I needed to used level 5.2 or 6 for 4k, and ffmpeg bailed on me.

I had to leave -level off the commandline and hope for the best.

[hydra3333]

thank you

[Carl Eugen Hoyos]

Are you sure that the user option level (as far as it currently works) does what you want it to do?

[hydra3333]

Sadly, no I am not sure. I do want to encode 4k using an nvidia 2060 Super which I gather goes beyond level 5.1 and I have to leave off -level.

The h.264 spec ​https://en.wikipedia.org/wiki/Advanced_Video_Coding#Levels says it goes to level 6.2,

and the constants in ​https://github.com/FFmpeg/nv-codec-headers/blob/master/include/ffnvcodec/nvEncodeAPI.h specify up to 5.2 (it may be greater now with newer cards/drivers),

however here ​https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/nvenc_h264.c (at least) it specifies only to level 5.1,

Thus there seems to be some inconsistency at present.

I'm unsure if the code otherwise checks and constrains it to what a driver may report (not sure where to look for that).

[Hendrik]

specifying level only adds constraints, it does not lift them. So if you don't specify a level, it'll use the highest required for your content.

That means that if you want to encode 5.2, there is no need to specify a level, since it'll automatically pick one.

The only reason to specify a level is if you want to constraint yourself to a specific level and not accidentally go over it.

The lack of th 5.2 level in nvenc_h264 should still be fixed, but its not necessarily required to specify it.

[hydra3333]

Yes. OK.

As you mention, it would be good to fix, as the omissions remove capacity to specify a valid level lower than the prevailing "upper limit" level.

I wonder if the code checks a specified/calculated level against what the installed hardware/driver allow as an upper limit ?

[hydra3333]

I see changes also aligning with the sdk10 changes :)
Thank you !!

​https://git.videolan.org/?p=ffmpeg.git;a=blobdiff;f=libavcodec/nvenc_h264.c;h=de8f4bd180550301f2a5dfefd131f8affb16ad5a;hp=0c1a32178c71aff4c46f1e014218cfa757be5bbe;hb=3223f6b32d5d60c73b1e14475d4751770ae9344d;hpb=9115d7722205590b8601d87146087f3559d29d54

[hydra3333]

I tried it and am happily transcoding videos with this combination of BRAND NEW ffmpeg options

-preset p7 -multipass fullres -rc:v vbr -cq:v 24 -qmin 16 -qmax 48 
-b:v 3746899 -minrate:v 749379 -maxrate:v 7493798 -bufsize 7493798 
-profile:v high -level 5.2 
-movflags +faststart+write_colr

which seems to work (the bitrate selection is auto per-video via a script).

No idea if it's a valid combination of options, but something comes out the other end :)

So, -level 5.2 works now.

[Balling]

-level 5.2 works now.

Even the original command works now, LOL and is bitperfect without -level 5.2. Also please note that your command is wrong:

-r 60/1.001 should have been used.

[Balling]

This is VERY wrong that when you do ffmpeg -f lavfi -i smptebars -c:v h264_nvenc -s 3840x2160 -aspect 16:9 -profile:v high -r 60/1.001 -level 6.2 -t 5 output.mp4

only 2 bytes are changed that say what level it is!! WTF. It should write correct profile which is 5.2 in this case.