VAAPI: HEVC (h265) VBR encode bitrate way too low

[U. Artie Eoff]

Summary of the bug:

Encoding HEVC 8bit and 10bit with vaapi VBR mode produces output files much lower than requested minimum bitrate when using i965 driver. This happens for any resolution, bitrate and input file. Also, the output bitrate always appears to be the same (+/-) for same input file regardless of requested bitrate setting.

This does not occur when using iHD driver on vaapi or qsv plugin. This also does not occur with gstreamer-vaapi on i965 or iHD drivers. That is, these combinations produce output bitrates within reason of requested min/max bitrates.

How to reproduce:

% ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -v verbose \
  -f rawvideo -pix_fmt yuv420p -s:v 1920x1080 -r:v 30 \
  -i input.yuv -vf 'format=nv12,hwupload' -c:v hevc_vaapi \
  -b:v 4000k -maxrate 8000k -vframes 150 -y test.h265

ffmpeg version N-92732-g568e1b229b85 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 8 (GCC)
  configuration: --prefix=/home/uaeoff/Work/workspace/media/install --disable-static --enable-shared --enable-libdrm --enable-vaapi --enable-libmfx --disable-amf --disable-audiotoolbox --disable-cuda --disable-cuda-sdk --disable-cuvid --disable-d3d11va --disable-dxva2 --disable-libnpp --disable-mmal --disable-nvdec --disable-nvenc --disable-omx --disable-omx-rpi --disable-rkmpp --disable-v4l2-m2m --disable-vdpau --disable-videotoolbox --enable-nonfree --enable-gpl
  libavutil      56. 24.101 / 56. 24.101
  libavcodec     58. 42.103 / 58. 42.103
  libavformat    58. 24.101 / 58. 24.101
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 46.101 /  7. 46.101
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
[AVHWDeviceContext @ 0x24835c0] Opened VA display via DRM device /dev/dri/renderD128.
[AVHWDeviceContext @ 0x24835c0] libva: VA-API version 1.4.0
[AVHWDeviceContext @ 0x24835c0] libva: va_getDriverName() returns 0
[AVHWDeviceContext @ 0x24835c0] libva: Trying to open /home/uaeoff/Work/workspace/media/install/lib/dri/i965_drv_video.so
[AVHWDeviceContext @ 0x24835c0] libva: Found init function __vaDriverInit_1_4
[AVHWDeviceContext @ 0x24835c0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x24835c0] Initialised VAAPI connection: version 1.4
[AVHWDeviceContext @ 0x24835c0] VAAPI driver: Intel i965 driver for Intel(R) Kaby Lake - 2.4.0.pre1 (2.4.0.pre1).
[AVHWDeviceContext @ 0x24835c0] Driver not found in known nonstandard list, using standard behaviour.
[rawvideo @ 0x2494380] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from 'input.yuv':
  Duration: 00:00:07.23, start: 0.000000, bitrate: 746496 kb/s
    Stream #0:0: Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p, 1920x1080, 746496 kb/s, 30 tbr, 30 tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (hevc_vaapi))
Press [q] to stop, [?] for help
[graph 0 input from stream 0:0 @ 0x24a63c0] w:1920 h:1080 pixfmt:yuv420p tb:1/30 fr:30/1 sar:0/1 sws_param:flags=2
[auto_scaler_0 @ 0x24a9440] w:iw h:ih flags:'bicubic' interl:0
[Parsed_format_0 @ 0x24a4900] auto-inserting filter 'auto_scaler_0' between the filter 'graph 0 input from stream 0:0' and the filter 'Parsed_format_0'
[auto_scaler_0 @ 0x24a9440] w:1920 h:1080 fmt:yuv420p sar:0/1 -> w:1920 h:1080 fmt:nv12 sar:0/1 flags:0x4
[hevc_vaapi @ 0x24a1d00] Input surface format is nv12.
[hevc_vaapi @ 0x24a1d00] Using VAAPI profile VAProfileHEVCMain (17).
[hevc_vaapi @ 0x24a1d00] Using VAAPI entrypoint VAEntrypointEncSlice (6).
[hevc_vaapi @ 0x24a1d00] Using VAAPI render target format YUV420 (0x1).
[hevc_vaapi @ 0x24a1d00] RC mode: VBR, 50% of 8000000 bps over 1000 ms.
[hevc_vaapi @ 0x24a1d00] RC buffer: 8000000 bits, initial fullness 6000000 bits.
[hevc_vaapi @ 0x24a1d00] Using intra, P- and B-frames (supported references: 3 / 1).
[hevc_vaapi @ 0x24a1d00] All wanted packed headers available (wanted 0xd, found 0x1f).
[hevc_vaapi @ 0x24a1d00] Using level 4.
Output #0, hevc, to 'test.h265':
  Metadata:
    encoder         : Lavf58.24.101
    Stream #0:0: Video: hevc (hevc_vaapi) (Main), 1 reference frame, vaapi_vld, 1920x1080, q=-1--1, 4000 kb/s, 30 fps, 30 tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.42.103 hevc_vaapi
No more output streams to write to, finishing.e=00:00:04.66 bitrate= 449.4kbits/s speed=2.24x    
frame=  150 fps= 68 q=-0.0 Lsize=     385kB time=00:00:04.96 bitrate= 634.4kbits/s speed=2.25x    
video:385kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Input file #0 (input.yuv):
  Input stream #0:0 (video): 150 packets read (466560000 bytes); 150 frames decoded; 
  Total: 150 packets (466560000 bytes) demuxed
Output file #0 (test.h265):
  Output stream #0:0 (video): 150 frames encoded; 150 packets muxed (393885 bytes); 
  Total: 150 packets (393885 bytes) muxed
[AVIOContext @ 0x24a2980] Statistics: 0 seeks, 2 writeouts
[AVIOContext @ 0x24954c0] Statistics: 466560000 bytes read, 0 seeks

% ls -al test.h265
-rw-rw-r--. 1 uaeoff uaeoff 393885 Dec 18 09:25 test.h265

% echo "393885*8*30/1024.0/150" | bc
615

Software stack:

libva (master) heads/master-0-g566a1388b4e5  https://github.com/01org/libva
gmmlib (master) tags/intel-gmmlib-18.4.1-0-g413896ed8e7e  https://github.com/intel/gmmlib
intel-vaapi-driver (master) heads/master-0-g72f10f16f7e5  https://github.com/01org/intel-vaapi-driver
intel-media-driver (master) heads/master-0-g273ef13413a1  https://github.com/intel/media-driver
msdk (master) heads/master-0-gdca3b5aec66c  https://github.com/Intel-Media-SDK/MediaSDK
ffmpeg (master) heads/master-0-g568e1b229b85  https://git.ffmpeg.org/ffmpeg
gstreamer (master) heads/master-0-g23c1a81503f4  https://gitlab.freedesktop.org/gstreamer/gstreamer
gst-plugins-base (master) heads/master-0-g362f771fe871  https://gitlab.freedesktop.org/gstreamer/gst-plugins-base.git
gst-plugins-good (master) heads/master-0-g7aebe608099b  https://gitlab.freedesktop.org/gstreamer/gst-plugins-good.git
gst-plugins-bad (master) heads/master-0-g5b372a97077f  https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad.git
gstreamer-vaapi (master) heads/master-0-g70d6b4002ce9  https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi.git

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Carl Eugen Hoyos]

Why do you think that this issue can be fixed in FFmpeg?

[U. Artie Eoff]

Replying to cehoyos:

Why do you think that this issue can be fixed in FFmpeg?

Because it does not appear to be a driver issue based on the fact that HEVC VBR + i965 works fine in gst-vaapi but not ffmpeg-vaapi.

[Zhong,Li]

Took a quick look at this issue. I believe this is a i965 driver issue instead of ffmpeg issue.

Here are the file size of encoded bitstreams via FFmpeg with same bitrate setting as bug description but just changing encoder type and BRC mode.(prefix "i965" means i965 driver, "ihd" means iHD driver)

ls -ls test_*
2564 -rw-r--r-- 1 root root 2624921 12月 19 15:19 test_i965_cbr.h265
2560 -rw-r--r-- 1 root root 2621123 12月 19 15:17 test_i965.h264
384 -rw-r--r-- 1 root root 391474 12月 19 15:19 test_i965_vbr.h265
2556 -rw-r--r-- 1 root root 2613360 12月 19 15:19 test_ihd_cbr.h265
2536 -rw-r--r-- 1 root root 2596236 12月 19 15:17 test_ihd.h264
2556 -rw-r--r-- 1 root root 2617061 12月 19 15:18 test_ihd_vbr.h264
2544 -rw-r--r-- 1 root root 2604431 12月 19 15:20 test_ihd_vbr.h265

You can see the file size is not normal for "i965 driver + VBR mode + HEVC" case, any other cases are normal.

Checked i965 driver: ​https://github.com/intel/intel-vaapi-driver/blob/master/src/gen9_hevc_encoder.c#L1770
You can find the case "target_percentage <= HEVC_BRC_MIN_TARGET_PERCENTAGE" hasn't been handled.
HEVC_BRC_MIN_TARGET_PERCENTAGE is 50, but gstreamer-vaapi always set target_percentage to be 70 for VBR (see: ​https://github.com/GStreamer/gstreamer-vaapi/blob/master/gst-libs/gst/vaapi/gstvaapiencoder.c#L939), that's why gstreamer-vaapi won't trigger it.

Current setting will make target_percentage=50 in FFmpeg (see: ​https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/vaapi_encode.c#L1344) and then trigger the driver issue.

To further make sure, I have tried to set the maxrate to be 7M (then target_percentage is larger then 50):
LIBVA_DRIVER_NAME=i965 ./ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -v verbose -f rawvideo -pix_fmt yuv420p -s:v 1920x1080 -r:v 30 -i /root/Videos/1080p_blue_sky.yuv -vf 'format=nv12,hwupload' -c:v hevc_vaapi -b:v 4000k -maxrate 7000k -vframes 150 -y test_i965_maxrate7M_vbr.h265

ls -ls test_i965_maxrate7M_vbr.h265
2626930 12月 19 15:51 test_i965_maxrate7M_vbr.h265
echo "2626930*8*30/1024.0/150" |bc
4104
You can see the encoded bit-rate is close to target bit-rate in this case.

[U. Artie Eoff]

@lizhong1008 thanks for the root-cause analysis. Driver issue filed here: ​https://github.com/intel/intel-vaapi-driver/issues/430