NVENC: -coder 0 parameter produces no error, but does not disable CABAC

[alexpigment]

Summary of the bug:

When using h264_nvenc, if -coder 0 is specified in command line, no errors or warnings are given, yet the resulting video still uses CABAC. If this is not the valid parameter to use, it should give a warning. Otherwise, it should be hooked up to the appropriate parameter from the API.

How to reproduce:

% ffmpeg -f lavfi -i smptebars -c:v h264_nvenc -s 1920x1080 -aspect 16:9 -r 29.97 -t 30 -coder 0 output.mp4

View result in MediaInfo to confirm that CABAC is used.

ffmpeg version N-86330-gbd1179e
built on 2017-06-01

[Carl Eugen Hoyos]

No warning can be shown for unused output options.

[jkqxz]

Replying to cehoyos:

No warning can be shown for unused output options.

The option isn't unused, but the behaviour is exactly as expected.

"-coder 0" indicates that the encoder should choose the mode automatically:
<​http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/nvenc_h264.c;h=c3b4bac7494dc24faf250423f2539830358b4c63;hb=HEAD#l122>
<​http://git.videolan.org/?p=ffmpeg.git;a=blob;f=compat/nvenc/nvEncodeAPI.h;h=e662880f4d94e880004f536dfbaa485bce9daf9a;hb=HEAD#l592>

The encoder then chooses to use CABAC, since it is more efficient and you haven't indicated any other constraint (e.g. profile) which would force it to use CAVLC instead.

(It is generally recommended that you use named options rather than integer values where they are available. See ffmpeg -h encoder=nvenc_h264.)

[Carl Eugen Hoyos]

Yes, this is indeed a bug.
How on earth can such things happen?

[Timo R.]

I don't see how this is a bug.
The option is called "coder", coder 0 has no specific meaning. You are supposed to select one by name.

[Carl Eugen Hoyos]

Replying to oromit:

I don't see how this is a bug.
The option is called "coder", coder 0 has no specific meaning.

I don't know how you come to this conclusion (and it contradicts the first comment): 0 is defined as auto for nvenc and as cavlc for everything (?) else, at least for x264, where the option-name originated. Please use the unstable-api period to fix this.

[Timo R.]

The value 0 comes directly from the nvidia header.
It's not part of the documented ffmpeg API or ABI. Only the names are.

The only direct value ffmpeg makes up here is -1, which translates to "do not set this setting at all and leave it on whatever is in there by default".
This option is similar do what x264 has, but not the same. I picked the const names for it to be compatible in most cases of normal usage.

[Carl Eugen Hoyos]

Simply translate the values or rename the option.

[Timo R.]

Sorry, but nvenc is not x264, and the option works slightly differently, but similar enough to name it the same, as it does the same thing.
If you use the names, like you should, the options are already compatible. Even on the x264 side of things the numerical values are not documented, only the values "ac" and "vlc" are, which are supported by nvenc to achieve the same thing.

[Carl Eugen Hoyos]

Replying to oromit:

Sorry, but nvenc is not x264, and the option works slightly differently, but similar enough to name it the same, as it does the same thing.

And this is exactly the issue here.

[Balling]

So was this solved? I found that one commit.. ​https://github.com/FFmpeg/FFmpeg/commit/a0b69e2b0a7ba994b722393a0273bb3464fd5efe

[Balling]

Maybe just document it? I mean what is the point of solving it.

[Balling]

Will this ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/1585926932-23744-1-git-send-email-linjie.fu@intel.com/ fix it?

See e4d37abcc13b236a544d37b9a8a4aa16bebe1fda

[Balling]

Okay, maybe just allow for -coder 2 to be cavlc for h264_nvenc? What do you think? Of course now even e4d37abcc13b236a544d37b9a8a4aa16bebe1fda is coder 0 of CABAC, but...