nvenc with interlace video never create packets with AV_PKT_FLAG_KEY

[Marek Fort]

Summary of the bug:
How to reproduce:

• Create codec with flag AV_CODEC_FLAG_INTERLACED_DCT
• Encode video
• look for encoded packet AV_PKT_FLAG_KEY flag.

Result:

• Key frame never produced.

When I disable the interlace flag then key frames are created properly.

[Carl Eugen Hoyos]

How would you expect FFmpeg to behave? Aren't you describing a feature of H.264?

[Timo R.]

FFMpeg is passing those flags correctly.
Both the INTERLACED_DCT one to NVENC, and the FLAG_KEY one back from NVENC.

If NVENC itself messes up with interlaced encoding, there isn't realy anything FFMpeg can do about that.
So Nvidia would be the right one to report a bug to.

I wouldn't be surprised if nobody ever tested interlaced encoding, nobody does that anymore.

[Marek Fort]

When I encode the same video stream with Intel QSV then keyframes are generated properly.
Could you point me to H.264 spec which says that interlace video frames cannot have key frames? (resp. IDR or I frames).
Thanks.

[Marek Fort]

The consequence is that the resulting MOV file looks like that every packet has KEY_FLAG set. This can be checked by ffprobe --show_packets. And certainly not all packets could be considered as key packets.

[Carl Eugen Hoyos]

Replying to maf:

The consequence is that the resulting MOV file looks like that every packet has KEY_FLAG set.

I don't understand: The summary indicates that the nv encoder never sets the flag?

I any case: You have to report this to the nvenc developers unless you can show that libavcodec incorrectly passes the flag from nvenv to libavformat.

[Marek Fort]

Replying to cehoyos:

Replying to maf:

The consequence is that the resulting MOV file looks like that every packet has KEY_FLAG set.

I don't understand: The summary indicates that the nv encoder never sets the flag?

Right. Key flag is not stored.

The full story is this. Files created by nvenc+ffmpeg were not correctly played. So I found out that nvenc+ffmpeg did not produce key frames. I thought this was an error assuming that some code in ffmpeg detected this and rather made all packets KEY. If you say that it is perfectly fine to have no KEY frames then I am fine with that.

But when you open it later by ffmpeg then all packets are marked as KEY. I do not know which code is responsible for this. Probably there are more slices in interlaced video and this is not handled correctly. I do not have the detailed knowledge about H264.

[Hendrik]

Replying to cehoyos:

Replying to maf:

The consequence is that the resulting MOV file looks like that every packet has KEY_FLAG set.

I don't understand: The summary indicates that the nv encoder never sets the flag?

I any case: You have to report this to the nvenc developers unless you can show that libavcodec incorrectly passes the flag from nvenv to libavformat.

Unless you can prove the opposite, maybe you shouldn't close the issue on a whim and let a developer look at the problem.

[Timo R.]

I'm the maintainer and author of the FFMpeg NVENC encoder, I looked at it and the flags are propperly passed and mapped on the FFMpeg/lavc side, just as I commented before.

If you think I'm wrong on that, please point out what you think FFMpeg is doing wrong.

[Hendrik]

FWIW I just tested, and adding -flags ildct does not change the number of keyframes in the output packets.

Tested using this command:

ffmpeg -i tos3k-vp9-b5000.webm -flags:v ildct -c:v nvenc_h264 -profile:v high -b:v 1000k interlaced-out.mkv

With and Without -flags ildct.

Maybe your GPU is not interlaced capable? It might make sense to test the capability in the NVENC encoder before trying to use it (through NvEncGetEncodeCaps with NV_ENC_CAPS_SUPPORT_FIELD_ENCODING)

[Marek Fort]

Replying to heleppkes:

FWIW I just tested, and adding -flags ildct does not change the number of keyframes in the output packets.

I run the same command. My video is 1920x1080 25fps.
In output MKV only the first packet has flags=K (checked by ffprobe).
When I do the same with MOV output then all packets have KEY flag (again, ffprobe's report).
This inconsistency is strange.

Maybe your GPU is not interlaced capable? It might make sense to test the capability in the NVENC encoder before trying to use it (through NvEncGetEncodeCaps with NV_ENC_CAPS_SUPPORT_FIELD_ENCODING)

Added this check. The returned caps value is 1. Documented as "Interlaced field mode encoding is supported". Documentation also mentions value 2 = "Interlaced frame encoding and field mode encoding are both supported."

I'll try another PC.

[malakudi]

FFMPEG version used

ffmpeg 
ffmpeg version 3.4.1 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18) 20170516
  configuration: --disable-decoder=amrnb --disable-decoder=libopenjpeg --disable-mips32r2 --disable-mips32r6 --disable-mips64r6 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-msa --disable-libopencv --disable-podpages --disable-stripping --enable-avfilter --enable-avresample --enable-gcrypt --enable-gnutls --enable-gpl --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx265 --enable-libxvid --enable-libzvbi --enable-nonfree --enable-opengl --enable-openssl --enable-postproc --enable-pthreads --enable-shared --enable-version3 --enable-libwebp --incdir=/usr/include/x86_64-linux-gnu --libdir=/usr/lib/x86_64-linux-gnu --prefix=/usr --toolchain=hardened --enable-frei0r --enable-chromaprint --enable-libx264 --enable-libiec61883 --enable-libdc1394 --enable-vaapi --disable-opencl --enable-libmfx --disable-altivec --shlibdir=/usr/lib/x86_64-linux-gnu
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

I have to reopen this ticket because there is still some underlying problem with NVENC interlaced encoding. While outputing files (mkv, ts, flv) has no issue and ffprobe correctly displays the keyframes in output files, when outputing to flv with rtmp output or to hls output, keyframes are not detected. Since testing flc with rtmp output is complicated, I will only discuss the hls issue since if this is resolved, I guess it will be resolved with flv rtmp as well.

Sample commands that show the issue:

ffmpeg -f mpegts -i sample.ts -flags +ildct+ilme -vcodec h264_nvenc -preset slow -b:v 2200k -profile:v high -level 4.1 -c:a aac -ac 2 -b:a 128k -f mkv out.mkv
ffprobe -show_frames out.mkv 2> /dev/null | grep pict_type=I | wc -l

result: 6 key frames, all ok

ffmpeg -f mpegts -i sample.ts -flags +ildct+ilme -vcodec h264_nvenc -preset slow -b:v 2200k -profile:v high -level 4.1 -c:a aac -ac 2 -b:a 128k -f mpegts out.ts
ffprobe -show_frames out.ts 2> /dev/null | grep pict_type=I | wc -l

result: 6 key frames, all ok

ffmpeg -f mpegts -i sample.ts -flags +ildct+ilme -vcodec h264_nvenc -preset slow -b:v 2200k -profile:v high -level 4.1 -c:a aac -ac 2 -b:a 128k -f hls -hls_time 8 -hls_playlist_type vod playlist.m3u8
ffprobe -show_frames playlist0.ts 2> /dev/null | grep pict_type=I | wc -l

results: 0 (no key frames)
Command fails to split segments at first keyframe after 8 seconds (hls_time 8) and creates only one ts file, called playlist0.ts, which as we can see in ffprobe command has no keyframes.

ffmpeg -f mpegts -i sample.ts -flags +ildct+ilme -vcodec libx264 -preset slow -b:v 2200k -profile:v high -level 4.1 -c:a aac -ac 2 -b:a 128k -f hls -hls_time 8 hls_playlist_type vod playlistnew.m3u8

Same hls output but with libx264 as encoder results in correct splitting on keyframes and a playlist with seven files (input is 62 seconds and keyframe every 10 seconds, so 7 files). Every file has 1 keyframe.

for f in `ls playlistnew*ts` ; do ffprobe -show_frames $f 2> /dev/null | grep pict_type=I | wc -l ; done
1
1
1
1
1
1
1

And finally, NVENC encoding to HLS without interlacing:

ffmpeg -f mpegts -i sample.ts -vcodec h264_nvenc -preset slow -b:v 2200k -profile:v high -level 4.1 -c:a aac -ac 2 -b:a 128k -f hls -hls_time 8 -hls_playlist_type vod playlistprogressive.m3u8

Again, ffmpeg correctly creates 7 segments with one keyframe each.

So, the issue affects only NVENC interlaced encoding but not on standard file output.

[Carl Eugen Hoyos]

Please open a new ticket: Test current FFmpeg git head (nothing else is supported here), provide the command line you tested together with the complete, uncut console output and explain what is wrong with the output.