ffmpeg issue with NVENC and keyframes when encoding interlaced video

[malakudi]

Summary of the bug: When encoding interlaced H264 video with NVENC codec, there is an issue with keyframes which affects hls output or flv rtmp output. It doesn't affect file output. Issue is similar to the one originally reported in ticket 5440.

How to reproduce:

ffmpeg -f mpegts -i sample.ts -flags +ildct+ilme -vcodec h264_nvenc -preset slow -b:v 2200k -profile:v high -level 4.1 -c:a aac -ac 2 -b:a 128k -f hls -hls_time 8 playlist.m3u8
ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18) 20170516
  configuration: --disable-decoder=amrnb --disable-decoder=libopenjpeg --disable-mips32r2 --disable-mips32r6 --disable-mips64r6 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-msa --disable-libopencv --disable-podpages --disable-stripping --enable-avfilter --enable-avresample --enable-gcrypt --enable-gnutls --enable-gpl --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx265 --enable-libxvid --enable-libzvbi --enable-nonfree --enable-opengl --enable-openssl --enable-postproc --enable-pthreads --enable-shared --enable-version3 --enable-libwebp --incdir=/usr/include/x86_64-linux-gnu --libdir=/usr/lib/x86_64-linux-gnu --prefix=/usr --toolchain=hardened --enable-frei0r --enable-chromaprint --enable-libx264 --enable-libiec61883 --enable-libdc1394 --enable-vaapi --disable-opencl --enable-libmfx --disable-altivec --shlibdir=/usr/lib/x86_64-linux-gnu
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, mpegts, from 'sample.ts':
  Duration: 00:01:02.56, start: 1.400000, bitrate: 13795 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 384 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[hls @ 0x5563975f08c0] Opening 'playlist0.ts' for writing
Output #0, hls, to 'playlist.m3u8':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0: Video: h264 (h264_nvenc) (High), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2200 kb/s, 25 fps, 90k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.107.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/2200000 buffer size: 4400000 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.107.100 aac
[hls @ 0x5563975f08c0] Opening 'playlist.m3u8.tmp' for writing/A dup=36 drop=0 speed=3.07x    
frame= 1564 fps= 76 q=36.0 Lsize=N/A time=00:01:02.52 bitrate=N/A dup=36 drop=0 speed=3.06x    
video:16977kB audio:945kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[aac @ 0x556397828420] Qavg: 1400.956

ffmpeg creates hls with only one segment although it should split to keyframe and create 7 segments. The single file created does not have keyframes as can be seen from command:

ffprobe -show_frames playlist0.ts 2> /dev/null | grep pict_type=I | wc -l
0

If run same command without +ildct+ilme, then all is ok, as can be seen in following command:

ffmpeg -f mpegts -i sample.ts -vcodec h264_nvenc -preset slow -b:v 2200k -profile:v high -level 4.1 -c:a aac -ac 2 -b:a 128k -f hls -hls_time 8 playlist.m3u8
ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18) 20170516
  configuration: --disable-decoder=amrnb --disable-decoder=libopenjpeg --disable-mips32r2 --disable-mips32r6 --disable-mips64r6 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-msa --disable-libopencv --disable-podpages --disable-stripping --enable-avfilter --enable-avresample --enable-gcrypt --enable-gnutls --enable-gpl --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx265 --enable-libxvid --enable-libzvbi --enable-nonfree --enable-opengl --enable-openssl --enable-postproc --enable-pthreads --enable-shared --enable-version3 --enable-libwebp --incdir=/usr/include/x86_64-linux-gnu --libdir=/usr/lib/x86_64-linux-gnu --prefix=/usr --toolchain=hardened --enable-frei0r --enable-chromaprint --enable-libx264 --enable-libiec61883 --enable-libdc1394 --enable-vaapi --disable-opencl --enable-libmfx --disable-altivec --shlibdir=/usr/lib/x86_64-linux-gnu
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, mpegts, from 'sample.ts':
  Duration: 00:01:02.56, start: 1.400000, bitrate: 13795 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 384 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[hls @ 0x55f2a7a25c20] Opening 'playlist0.ts' for writing
Output #0, hls, to 'playlist.m3u8':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0: Video: h264 (h264_nvenc) (High), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2200 kb/s, 25 fps, 90k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.107.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/2200000 buffer size: 4400000 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.107.100 aac
[hls @ 0x55f2a7a25c20] Opening 'playlist1.ts' for writingate=N/A dup=36 drop=0 speed= 5.2x    
[hls @ 0x55f2a7a25c20] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x55f2a7a25c20] Opening 'playlist2.ts' for writingate=N/A dup=36 drop=0 speed=5.55x    
[hls @ 0x55f2a7a25c20] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x55f2a7a25c20] Opening 'playlist3.ts' for writingate=N/A dup=36 drop=0 speed= 5.6x    
[hls @ 0x55f2a7a25c20] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x55f2a7a25c20] Opening 'playlist4.ts' for writingate=N/A dup=36 drop=0 speed=5.67x    
[hls @ 0x55f2a7a25c20] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x55f2a7a25c20] Opening 'playlist5.ts' for writingate=N/A dup=36 drop=0 speed=5.67x    
[hls @ 0x55f2a7a25c20] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x55f2a7a25c20] Opening 'playlist6.ts' for writingate=N/A dup=36 drop=0 speed=5.69x    
[hls @ 0x55f2a7a25c20] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x55f2a7a25c20] Opening 'playlist.m3u8.tmp' for writing
frame= 1564 fps=141 q=33.0 Lsize=N/A time=00:01:02.52 bitrate=N/A dup=36 drop=0 speed=5.64x    
video:16605kB audio:945kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[aac @ 0x55f2a7ac9e40] Qavg: 1400.956

As can be seen, it creates 7 segments, all having one keyframe, as can be seen from next command:

for f in `ls playlist*ts` ; do ffprobe -show_frames $f 2> /dev/null | grep pict_type=I | wc -l ; done1
1
1
1
1
1
1

Creating interlaced output with libx264 works as expected, in next command:

ffmpeg -f mpegts -i sample.ts -flags +ildct+ilme -vcodec libx264 -preset slow -b:v 2200k -profile:v high -level 4.1 -c:a aac -ac 2 -b:a 128k -f hls -hls_time 8 -hls_playlist_type vod playlist.m3u8
ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18) 20170516
  configuration: --disable-decoder=amrnb --disable-decoder=libopenjpeg --disable-mips32r2 --disable-mips32r6 --disable-mips64r6 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-msa --disable-libopencv --disable-podpages --disable-stripping --enable-avfilter --enable-avresample --enable-gcrypt --enable-gnutls --enable-gpl --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx265 --enable-libxvid --enable-libzvbi --enable-nonfree --enable-opengl --enable-openssl --enable-postproc --enable-pthreads --enable-shared --enable-version3 --enable-libwebp --incdir=/usr/include/x86_64-linux-gnu --libdir=/usr/lib/x86_64-linux-gnu --prefix=/usr --toolchain=hardened --enable-frei0r --enable-chromaprint --enable-libx264 --enable-libiec61883 --enable-libdc1394 --enable-vaapi --disable-opencl --enable-libmfx --disable-altivec --shlibdir=/usr/lib/x86_64-linux-gnu
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, mpegts, from 'sample.ts':
  Duration: 00:01:02.56, start: 1.400000, bitrate: 13795 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 384 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x564f648148c0] interlace + weightp is not implemented
[libx264 @ 0x564f648148c0] using SAR=1/1
[libx264 @ 0x564f648148c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x564f648148c0] profile High, level 4.1
[libx264 @ 0x564f648148c0] 264 - core 155 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=4 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=8 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=tff bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=0 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=50 rc=abr mbtree=1 bitrate=2200 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[hls @ 0x564f64772d00] Opening 'playlist0.ts' for writing
Output #0, hls, to 'playlist.m3u8':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0: Video: h264 (libx264), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2200 kb/s, 25 fps, 90k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.107.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/2200000 buffer size: 0 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.107.100 aac
[hls @ 0x564f64772d00] Opening 'playlist1.ts' for writingate=N/A dup=36 drop=0 speed=1.39x    
[hls @ 0x564f64772d00] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x564f64772d00] Opening 'playlist2.ts' for writingate=N/A dup=36 drop=0 speed=1.23x    
[hls @ 0x564f64772d00] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x564f64772d00] Opening 'playlist3.ts' for writingate=N/A dup=36 drop=0 speed=1.18x    
[hls @ 0x564f64772d00] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x564f64772d00] Opening 'playlist4.ts' for writingate=N/A dup=36 drop=0 speed=1.16x    
[hls @ 0x564f64772d00] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x564f64772d00] Opening 'playlist5.ts' for writingate=N/A dup=36 drop=0 speed=1.14x    
[hls @ 0x564f64772d00] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x564f64772d00] Opening 'playlist6.ts' for writingate=N/A dup=36 drop=0 speed=1.11x    
[hls @ 0x564f64772d00] Opening 'playlist.m3u8.tmp' for writing
[hls @ 0x564f64772d00] Opening 'playlist.m3u8.tmp' for writing
frame= 1564 fps= 27 q=-1.0 Lsize=N/A time=00:01:02.44 bitrate=N/A dup=36 drop=0 speed=1.09x    
video:16944kB audio:945kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x564f648148c0] frame I:7     Avg QP:23.51  size:176528
[libx264 @ 0x564f648148c0] frame P:403   Avg QP:26.40  size: 26468
[libx264 @ 0x564f648148c0] frame B:1154  Avg QP:32.62  size:  4721
[libx264 @ 0x564f648148c0] consecutive B-frames:  0.9%  1.5%  1.9% 95.7%
[libx264 @ 0x564f648148c0] mb I  I16..4: 22.8% 41.5% 35.7%
[libx264 @ 0x564f648148c0] mb P  I16..4:  3.9%  2.6%  0.7%  P16..4: 35.7%  8.6%  5.5%  0.0%  0.0%    skip:43.0%
[libx264 @ 0x564f648148c0] mb B  I16..4:  0.3%  0.1%  0.0%  B16..8: 34.2%  1.8%  0.2%  direct: 0.9%  skip:62.6%  L0:43.9% L1:53.1% BI: 3.1%
[libx264 @ 0x564f648148c0] final ratefactor: 27.00
[libx264 @ 0x564f648148c0] field mbs: intra: 13.5% inter:21.8% skip:6.7%
[libx264 @ 0x564f648148c0] 8x8 transform intra:35.6% inter:65.5%
[libx264 @ 0x564f648148c0] direct mvs  spatial:99.4% temporal:0.6%
[libx264 @ 0x564f648148c0] coded y,uvDC,uvAC intra: 25.4% 42.4% 19.0% inter: 4.3% 5.5% 0.8%
[libx264 @ 0x564f648148c0] i16 v,h,dc,p: 26% 57%  7% 10%
[libx264 @ 0x564f648148c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 12% 22% 38%  4%  5%  5%  6%  4%  6%
[libx264 @ 0x564f648148c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 32% 13%  4%  6%  5%  6%  5%  7%
[libx264 @ 0x564f648148c0] i8c dc,h,v,p: 34% 47% 16%  3%
[libx264 @ 0x564f648148c0] ref P L0: 62.9% 21.3% 13.6%  0.9%  0.9%  0.5%  0.0%  0.0%
[libx264 @ 0x564f648148c0] ref B L0: 82.5% 14.2%  2.8%  0.3%  0.1%  0.1%
[libx264 @ 0x564f648148c0] ref B L1: 88.3% 11.2%  0.3%  0.2%
[libx264 @ 0x564f648148c0] kb/s:2218.72
[aac @ 0x564f64819020] Qavg: 1400.956

Creating file output, for example mkv (and not hls) also works as expected:

ffmpeg -f mpegts -i sample.ts -flags +ildct+ilme -vcodec h264_nvenc -preset slow -b:v 2200k -profile:v high -level 4.1 -c:a aac -ac 2 -b:a 128k -f matroska out.mkv
ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18) 20170516
  configuration: --disable-decoder=amrnb --disable-decoder=libopenjpeg --disable-mips32r2 --disable-mips32r6 --disable-mips64r6 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-msa --disable-libopencv --disable-podpages --disable-stripping --enable-avfilter --enable-avresample --enable-gcrypt --enable-gnutls --enable-gpl --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx265 --enable-libxvid --enable-libzvbi --enable-nonfree --enable-opengl --enable-openssl --enable-postproc --enable-pthreads --enable-shared --enable-version3 --enable-libwebp --incdir=/usr/include/x86_64-linux-gnu --libdir=/usr/lib/x86_64-linux-gnu --prefix=/usr --toolchain=hardened --enable-frei0r --enable-chromaprint --enable-libx264 --enable-libiec61883 --enable-libdc1394 --enable-vaapi --disable-opencl --enable-libmfx --disable-altivec --shlibdir=/usr/lib/x86_64-linux-gnu
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, mpegts, from 'sample.ts':
  Duration: 00:01:02.56, start: 1.400000, bitrate: 13795 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 384 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
Output #0, matroska, to 'out.mkv':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0: Video: h264 (h264_nvenc) (High) (H264 / 0x34363248), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2200 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.107.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/2200000 buffer size: 4400000 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.107.100 aac
frame= 1528 fps= 73 q=36.0 Lsize=   17439kB time=00:01:02.52 bitrate=2285.0kbits/s speed=2.97x    
video:16463kB audio:945kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.179919%
[aac @ 0x564c18bf2460] Qavg: 1400.956
ffprobe -show_frames out.mkv 2> /dev/null | grep pict_type=I | wc -l
7

file has 7 keyframes as it should.

Report is with version 3.4.2 from deb-multimedia. If this is not acceptable, I will try to build myself from git but building with NVENC is a bit difficult.

[Timo R.]

nvenc does not support encoding interlaced fields as one interleaved frame.
Instead, it emits two independent fields per frame. (no support for MBAFF)

Due to each field needing its own timestamp, but being in the same packet with only one shared timestamp, FFmpeg does not properly support handling those coming out of an encoder, and how well it works is pretty much up to luck depending on the container and transport method in use.

I looked into it a while ago, but concluded that fixing that would be a major non-trivial change to pretty much the entire FFmpeg codebase, which I'm not going to be able to make on my own. I'd even say it's unlikely that this will ever be fixed, unless nvidia implements mbaff support.

[Carl Eugen Hoyos]

Please confirm that this issue is not specific to version 3.4.

Why can't we just combine the field into one packet?

[malakudi]

Yes issue happens since forever. It used to affect file output also but it got fixed sometime between 2.5.4 and 3.1.x . I can try to find when file output was fixed if that would help.

[Carl Eugen Hoyos]

The question is if the issue is reproducible with current FFmpeg.

[malakudi]

Just tried with almost current git, nightly build for windows. Problem also there.

ffmpeg -f mpegts -i sample.ts -flags +ildct+ilme -vcodec h264_nvenc -preset slow -b:v 2200k -profile:v high -level 4.1 -c:a aac -ac 2 -b:a 128k -f hls -hls_time 8 playlist.m3u8
ffmpeg version N-90288-g2536bd8632 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.3.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56.  9.100 / 56.  9.100
  libavcodec     58. 14.100 / 58. 14.100
  libavformat    58. 10.100 / 58. 10.100
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 12.100 /  7. 12.100
  libswscale      5.  0.102 /  5.  0.102
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
Input #0, mpegts, from 'sample.ts':
  Duration: 00:01:02.56, start: 1.400000, bitrate: 13795 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 384 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[hls @ 000001230d7f0500] Opening 'playlist0.ts' for writing
Output #0, hls, to 'playlist.m3u8':
  Metadata:
    encoder         : Lavf58.10.100
    Stream #0:0: Video: h264 (h264_nvenc) (High), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2200 kb/s, 25 fps, 90k tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.14.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/2200000 buffer size: 4400000 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc58.14.100 aac
[hls @ 000001230d7f0500] Opening 'playlist.m3u8.tmp' for writing dup=36 drop=0 speed=8.65x
frame= 1564 fps=216 q=36.0 Lsize=N/A time=00:01:02.56 bitrate=N/A dup=36 drop=0 speed=8.65x
video:16968kB audio:953kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[aac @ 000001230b89e840] Qavg: 1101.940

[malakudi]

@oromit: How the issue you mention affects the keyframes? And why output to file doesn't have this issue? Can you point to look somewhere in the code about this?

Same issue used to occur on file output as well, as mentioned in ticket 5440, but got fixed in the process.

[Timo R.]

Because the two fields have different parameters. Like, one of them can be a keyframe, but the other isn't, and whichever one comes first gets its parameter set on the packet, but the ones for the other field are lost. That includes its timestamp, and flags like being keyframe.

If it works depends on how much the output format depends on the information in the packet. Some don't use it at all, others rely heavily on it.
The two fields are combined in one packet, which is exactly the problem. If I split them, I end up with the rest of ffmpeg exploding because an encoder is not expected to return two packets for one input frame, even though the new API technically supports it.

[Carl Eugen Hoyos]

If the first field can be decoded independently (“I”) I believe you should set the keyframe flag.

[malakudi]

@oromit: I've seen you are the code maintainer of nvenc in ffmpeg so definitely you know better from anyone. Can the issue be fixed with a workaround? A code that checks the output from the nvidia encoder and "fix" the keyframe issue by inspecting both fields? Or, if this is not possible, can we add a workaround to muxers affected? I have reproduced the problem in rtmp output and hls output.

[malakudi]

With my little understanding of the ffmpeg code, I think that mpegts encoder does not suffer from the issue of nvenc interlaced encoding because it uses the flags from payload and not packet (ts_st->payload_flags).
I couldn't understand why matroska encoder also doesn't suffer from the issue.

I will try to modify hls encoder (hlsenc.c) to also use the flags from payload.

[Timo R.]

The most reliable "workaround" I found was writing the video to a raw .h264 file and then using that as input.

I have some code somewhere that makes nvenc do the right thing, but when I tested it it made the whole rest of ffmpeg and libav* blow up.

I'm thinking to put the whole interlaced encoding with nvenc behind -strict experimental for now.

[Balling]

Replying to oromit:

I have some code somewhere that makes nvenc do the right thing,

Do you still have it? We need it for interlaced hevc.