ffmpeg h264_qsv force_key_frames incorret

[mikejson]

Summary of the bug:
How to reproduce:
when i user ffmpeg command line, like this:
./ffmpeg.exe -c:v mjpeg_qsv -rtbufsize 2000M -s 2560*1440 -f dshow -i video="Smart Camera" -vcodec h264_qsv -b:v 6000k -r 30 -preset veryfast -tune:v zerolatency -force_key_frames expr:gte\(t,n_forced*5\) -g 30*10 iii.h264

The key frame is generated at the fifth second; but the later is not incorret; every frame is key frame.

[Carl Eugen Hoyos]

Please provide the command line you tested together with the complete, uncut console output to make this a valid ticket.

[mikejson]

$ ./ffmpeg.exe -c:v mjpeg_qsv -rtbufsize 2000M -s 2560*1440 -f dshow -i video="Smart Camera" -vcodec h264_qsv -b:v 6000k -r 30 -preset veryfast -tune:v zerolatency -force_key_frames expr:gte\(t,n_forced*5\) -g 30*10 iii.h264
ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers

built with gcc 10.2.1 (GCC) 20200726
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libgsm --disable-w32threads --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100

[mjpeg @ 00a44500] EOI missing, emulating
Input #0, dshow, from 'video=Smart Camera':

Duration: N/A, start: 22972.029000, bitrate: N/A

Stream #0:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), yuvj420p(pc, bt470bg/unknown/unknown), 2560x1440 [SAR 96:96 DAR 16:9], 30 fps, 30 tbr, 10000k tbn, 10000k tbc

Codec AVOption tune (The metric that the encoder tunes for. Automatically chosen by the encoder by default) specified for output file #0 (iii.h264) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Stream mapping:

Stream #0:0 -> #0:0 (mjpeg (mjpeg_qsv) -> h264 (h264_qsv))

Press [q] to stop, ? for help
Output #0, h264, to 'iii.h264':

Metadata:

encoder : Lavf58.45.100
Stream #0:0: Video: h264 (h264_qsv), nv12, 2560x1440 [SAR 96:96 DAR 16:9], q=-1--1, 6000 kb/s, 30 fps, 30 tbn, 30 tbc
Metadata:

encoder : Lavc58.91.100 h264_qsv

Side data:

cpb: bitrate max/min/avg: 0/0/6000000 buffer size: 0 vbv_delay: N/A

frame= 1060 fps= 30 q=51.0 size= 26112kB time=00:00:35.03 bitrate=6105.9kbits/s dup=22 drop=0 speed=0.99x

[Sergey Tikhonov]

Same problem for me, only with force_key_frames parameter present.

$ ffmpeg  -y -i timecode.mp4 -init_hw_device qsv=foo:/dev/dri/renderD128 -filter_hw_device foo -filter_complex "[0:v]format=nv12,hwupload=extra_hw_frames=64[v]" -map '[v]' -c:v h264_qsv -g 50 -flags +cgop -r 25 -keyint_min 49 -forced_idr 1 -force_key_frames "expr:gte(t,(n_forced*2)+ 2)" out_cgop.mp4
ffmpeg version N-98692-ga5ac81952e Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --enable-libx264 --enable-libfdk-aac --enable-gpl --enable-nonfree --enable-ffplay --enable-vaapi --enable-libmfx --enable-libdrm --enable-indev=kmsgrab --enable-libpulse --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-filter=drawtext
  libavutil      56. 58.100 / 56. 58.100
  libavcodec     58.100.100 / 58.100.100
  libavformat    58. 50.100 / 58. 50.100
  libavdevice    58. 11.101 / 58. 11.101
  libavfilter     7. 87.100 /  7. 87.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Guessed Channel Layout for Input Stream #0.1 : 5.1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'timecode.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Westworld S03E01 - EniaHD
    encoder         : Lavf58.50.100
  Duration: 00:02:00.02, start: 0.000000, bitrate: 6818 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 6420 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(rus): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 393 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 (h264) -> format
  hwupload -> Stream #0:0 (h264_qsv)
Press [q] to stop, [?] for help
Output #0, mp4, to 'out_cgop.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Westworld S03E01 - EniaHD
    encoder         : Lavf58.50.100
    Stream #0:0: Video: h264 (h264_qsv) (avc1 / 0x31637661), qsv, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 1000 kb/s, 25 fps, 12800 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc58.100.100 h264_qsv
    Side data:
      cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 0 vbv_delay: N/A
frame= 3000 fps=274 q=51.0 Lsize=   18147kB time=00:01:59.88 bitrate=1240.0kbits/s speed=10.9x    
video:18117kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.161976%

ffprobe -show_frames -select_streams v:0 out_cgop.mp4 | egrep '(pict_type|key_frame|pkt_pts_time)' | xargs -n 3 echo | grep I

• without force_key_frames a key frame is set to every 50 frames.
• with force_key_frames, first key frame is at 0:00:00.0, second at 0:02:00.0 and then randomly with average distance 2-5 frames.
• two consequent launches produces different key frames, so there may be a race condition
• output with or without force_key_frames is same and does not contain any warning or error.

[Xu Guangxin]

Hi mikejson, zimbler:
Thanks for reporting the issue.
It fixed at ​https://github.com/intel-media-ci/ffmpeg/pull/330
I will send the patch if it's passed our internal review.

thanks

[Xu Guangxin]

Hi mikejson, zimbler
patch sent to maillist ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210106031234.13435-1-guangxin.xu@intel.com/

You can watch the discussion and merging process.

thanks

[Balling]

every frame is key frame.

Wow. Just like with VC1 decoder (in TS, not in mp4). Patch is still not applied.

[haihao]

Fixed in 3bbe0c210b05fc6fbd7b1d4bbd8479db7f2cf957 (and commit 1cdbccaa163eb3bd50403ecc75fc3da9d5d75c02 fixed the memory leak introduced by 3bbe0c210b05fc6fbd7b1d4bbd8479db7f2cf957).