[FFmpeg-vaapi][encode] caanot setup number of refference frames through `-refs` option

[Andrey]

Summary of the bug:
Number of reference frames on output for ffmpeg-qsv is 3. But for ffmpeg-vaapi by default - 2 and I cannot set it through -refs 3 option. May be I'm wrong, but I cannot find any other cmd-line options for reference frame number. Please see pictures in attach.

How to reproduce:

INPUT STREAM: http://fate-suite.ffmpeg.org/h264-conformance/AUD_MW_E.264

commit:
    Author:			Guo, Yejun <yejun.guo@intel.com>
    Author date:	1 week ago (02-Sep-19 07:35:58)
    Committer:		Pedro Arthur <bygrandao@gmail.com>
    Commit date:	1 week ago (04-Sep-19 17:13:21)
    Commit hash:	022f50d3febe5232e67d38b0292d86f7aa8a360c
    Parent:			3b3150c45f
    libavfilter/dnn: add header into native model file

    Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
    Signed-off-by: Pedro Arthur <bygrandao@gmail.com>

### build/lin_x64/bin/ffmpeg -y -hwaccel qsv -c:v h264_qsv -i ./AUD_MW_E.264 -c:v h264_qsv -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -preset medium -g 256 -bf 3 -async_depth 1 out.h264

ffmpeg version 4.2.git Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
  configuration: --prefix=/msdk/ffmpeg_visualGDB/ffmpeg_build --enable-libmfx --disable-stripping --enable-debug=3 --extra-cflags=-gstabs+ --disable-optimizations
  libavutil      56. 34.100 / 56. 34.100
  libavcodec     58. 56.101 / 58. 56.101
  libavformat    58. 32.104 / 58. 32.104
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 58.102 /  7. 58.102
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
Input #0, h264, from './AUD_MW_E.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 176x144, 25 fps, 25 tbr, 1200k tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_qsv) -> h264 (h264_qsv))
Press [q] to stop, [?] for help
m_param.numberOfThreads = 8
m_param.numberOfThreads = 8
m_param.numberOfThreads = 8
m_param.numberOfThreads = 8
par->mfx.NumRefFrame = 3
par->mfx.GopPicSize = 256
par->mfx.GopRefDist = 4
par->mfx.CodecLevel = 21
Output #0, h264, to 'out.h264':
  Metadata:
    encoder         : Lavf58.32.104
    Stream #0:0: Video: h264 (h264_qsv), qsv, 176x144, q=-1--1, 6000 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.56.101 h264_qsv
    Side data:
      cpb: bitrate max/min/avg: 6000000/6000000/6000000 buffer size: 0 vbv_delay: N/A
[h264_qsv @ 0x55fa599189c0] A decode call did not consume any data: expect more data at input (-10)
    Last message repeated 2 times
frame=  100 fps=0.0 q=44.0 Lsize=    2564kB time=00:00:04.20 bitrate=5000.1kbits/s speed=12.2x
video:2564kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

### build/lin_x64/bin/ffmpeg -y -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i ./AUD_MW_E.264 -c:v h264_vaapi -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -compression_level 4 -g 256 -bf 3 -b_depth 2 out_vaapi.h264

ffmpeg version 4.2.git Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
  configuration: --prefix=/msdk/ffmpeg_visualGDB/ffmpeg_build --enable-libmfx --disable-stripping --enable-debug=3 --extra-cflags=-gstabs+ --disable-optimizations
  libavutil      56. 34.100 / 56. 34.100
  libavcodec     58. 56.101 / 58. 56.101
  libavformat    58. 32.104 / 58. 32.104
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 58.102 /  7. 58.102
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
Input #0, h264, from './AUD_MW_E.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 176x144, 25 fps, 25 tbr, 1200k tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi))
Press [q] to stop, [?] for help
Output #0, h264, to 'out_vaapi.h264':
  Metadata:
    encoder         : Lavf58.32.104
    Stream #0:0: Video: h264 (h264_vaapi) (High), vaapi_vld, 176x144, q=-1--1, 6000 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.56.101 h264_vaapi
frame=  100 fps=0.0 q=-0.0 Lsize=    2747kB time=00:00:03.92 bitrate=5740.3kbits/s speed=13.4x
video:2747kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

### build/lin_x64/bin/ffmpeg -y -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i ./AUD_MW_E.264 -c:v h264_vaapi -b:v 6000k -minrate:v 6000k -maxrate:v 6000k -compression_level 4 -g 256 -bf 3 -b_depth 2 -refs 3 out_vaapi_refs3.h264

ffmpeg version 4.2.git Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
  configuration: --prefix=/msdk/ffmpeg_visualGDB/ffmpeg_build --enable-libmfx --disable-stripping --enable-debug=3 --extra-cflags=-gstabs+ --disable-optimizations
  libavutil      56. 34.100 / 56. 34.100
  libavcodec     58. 56.101 / 58. 56.101
  libavformat    58. 32.104 / 58. 32.104
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 58.102 /  7. 58.102
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
Input #0, h264, from './AUD_MW_E.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 176x144, 25 fps, 25 tbr, 1200k tbn, 50 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi))
Press [q] to stop, [?] for help
Output #0, h264, to 'out_vaapi_refs3.h264':
  Metadata:
    encoder         : Lavf58.32.104
    Stream #0:0: Video: h264 (h264_vaapi) (High), vaapi_vld, 176x144, q=-1--1, 6000 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.56.101 h264_vaapi
frame=  100 fps=0.0 q=-0.0 Lsize=    2747kB time=00:00:03.92 bitrate=5740.3kbits/s speed=13.9x
video:2747kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

[Andrey]

qsv_output_structure

[Andrey]

vaapi_output_structure

[Andrey]

vaapi_ref3_output_structure

[Zhong,Li]

From the code:https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/vaapi_encode_h264.c#L1043

    av_assert0(pic->nb_refs <= 2);
    if (pic->nb_refs >= 1) {
        // Backward reference for P- or B-frame.
        av_assert0(pic->type == PICTURE_TYPE_P ||
                   pic->type == PICTURE_TYPE_B);
        vslice->RefPicList0[0] = vpic->ReferenceFrames[0];
    }
    if (pic->nb_refs >= 2) {
        // Forward reference for B-frame.
        av_assert0(pic->type == PICTURE_TYPE_B);
        vslice->RefPicList1[0] = vpic->ReferenceFrames[1];
    }

I doubt multiple referneces is supported.

[Andrey]

Would it be possible to add multiple references support ?

[wenbin,chen]

​https://github.com/FFmpeg/FFmpeg/blob/2367affc2cea32f18a4d309835bed57adb1baa94/libavcodec/vaapi_encode.h#L43

It seems that there is a MAX_PICTURE_REFERENCES that limit the refs.