Opened 7 weeks ago

Last modified 4 weeks ago

#7650 new defect

FFmpeg MPEG-2 encode fails to "invalid RC mode" when iHD driver is used through VA-API instead of QSV

Reported by: eero-t Owned by:
Priority: normal Component: avcodec
Version: unspecified Keywords: vaapi
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Setup:

  • Ubuntu 18.04
  • Any HW with MPEG-2 HW acceleration support enabled by intel-driver (Intel Skylake, Kabylake, Coffeelake...)
  • Latest FFmpeg, MediaSDK, libva, intel-driver and gmmlib built from sources at Github
  • drm-git kernel (e.g. v4.19 or v4.20)

Bug can be reproduced by any FFmpeg MPEG-2 encoding with VA-API. For example:

ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 720x480p_30.00_4mb_h264_cabac.264 -c:v mpeg2_vaapi -b:v 2M -compression_level 4 -y output.mpg
...
Input #0, h264, from '720x480p_30.00_4mb_h264_cabac.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(tv, smpte170m, progressive), 720x480 [SAR 10:11 DAR 15:11], 30 fps, 30 tbr, 1200k tbn, 60 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg2video (mpeg2_vaapi))
Press [q] to stop, [?] for help
[mpeg2_vaapi @ 0x564a286e6100] Driver does not support some wanted packed headers (wanted 0x3, found 0x10).
Assertion 0 && "Invalid RC mode." failed at src/libavcodec/vaapi_encode_mpeg2.c:545

Exactly same use-case works HW accelerated when done:

  • through FFmpeg QSV API, or
  • using MediaSDK sample transcode application

Looking at the FFmpeg source code, it seems to accept only VA_RC_CQP mode: https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/vaapi_encode_mpeg2.c#L523

And mode is something set before FFmpeg vaapi_encode_mpeg2_configure() is called, as it's not set in ff_cbs_init(): https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/cbs.c#L74

Change History (7)

comment:1 Changed 6 weeks ago by fulinjie

VA_RC_VBR mode is set in vaapi_encode_init_rate_control:
https://github.com/FFmpeg/FFmpeg/blob/eb81fd792fa88f9015b5e99e4940a464de9182a5/libavcodec/vaapi_encode.c#L1362

And calls ctx->codec->configure() in ff_vaapi_encode_init to set vaapi_encode_mpeg2_configure.

You may try to use default CQP mode by remove -b:v 2M, or add support for VBR and CBR:

--- a/libavcodec/vaapi_encode_mpeg2.c
+++ b/libavcodec/vaapi_encode_mpeg2.c
@@ -541,6 +541,13 @@ static av_cold int vaapi_encode_mpeg2_configure(AVCodecContext *avctx)
                "%d / %d / %d for I- / P- / B-frames.\n",
                priv->quant_i, priv->quant_p, priv->quant_b);

+    } else if (ctx->va_rc_mode == VA_RC_CBR ||
+               ctx->va_rc_mode == VA_RC_VBR) {
+        // These still need to be set for quantiser_scale_code.
+        priv->quant_i = 10;
+        priv->quant_p = 10;
+        priv->quant_b = 10;
+
     } else {
         av_assert0(0 && "Invalid RC mode.");
     }


comment:2 Changed 6 weeks ago by eero-t

I can verify that either of these will allow FFmpeg to work.

However, with the patch, I get much worse PSNR / SSIM with VA-API:

ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 720x480p_30.00_4mb_cabac.264 -c:v mpeg2_vaapi -b:v 2M -compression_level 4 -y output.mpg
...
ffmpeg -i 720x480p_30.00_4mb_cabac.264 -i output.mpg -filter_complex "ssim;[0:v][1:v]psnr" -frames 2400 -f null -
...
SSIM Y:0.592489 (3.898609) U:0.864354 (8.675915) V:0.905782 (10.258669) All:0.690015 (5.086599)
PSNR y:16.888131 u:27.188619 v:30.218137 average:18.499885 min:13.885003 max:36.662048

than with QSV:

ffmpeg -hwaccel qsv -c:v h264_qsv -i 720x480p_30.00_4mb_h264_cabac.264 -c:v mpeg2_qsv -b:v 2M -compression_level 4 -y output.mpg
...
ffmpeg -i 720x480p_30.00_4mb_cabac.264 -i output.mpg -filter_complex "ssim;[0:v][1:v]psnr" -frames 2400 -f null -
...
SSIM Y:0.963254 (14.347895) U:0.976697 (16.325860) V:0.976484 (16.286427) All:0.967700 (14.907913)
PSNR y:37.747504 u:44.358034 v:44.510943 average:39.066017 min:31.195431 max:52.336077

I.e. average PSNR drops to half, from 39 with QSV, to 18.5 with VA-API.

In case it matters, there are following additional warnings with VA-API:

[mpeg2_vaapi @ 0x56228b35f100] Driver does not support some wanted packed headers (wanted 0x3, found 0x10).
[mpeg2_vaapi @ 0x56228b35f100] Sample aspect ratio 10:11 is not representable, signalling square pixels instead.

However, if I drop the "-b:v 2M" option, I still see these extra warnings, but PSNR with VA-API is close to one with QSV.

In few other tests I did, where output is H264 encoded, there was no PSNR/SSIM metrics difference between using VA-API and QSV.

comment:3 Changed 6 weeks ago by cehoyos

  • Keywords vaapi added

Please provide your (simplified) command line together with the complete, uncut console output to make this a valid ticket.

comment:4 Changed 5 weeks ago by fulinjie

I think the psnr issue was caused by the mismatch between FFmpeg and iHD driver.

In FFmpeg VAAPI level, hrd_buffer_size and hrd_initial_buffer_fullness will be set in vaapi_encode_init_rate_control:
https://github.com/FFmpeg/FFmpeg/blob/282a4718576d6928b6c5900db89b45d83556407a/libavcodec/vaapi_encode.c#L1305
with a higher priority (vbvBuffer_size set in vaapi_encode_mpeg2_init_sequence_params will be covered in driver).

The mismatch exists in the unit of the m_vbvBufferSize:
ffmpeg vaapi: in bits;
mpeg2 in driver: in 16kbits;

Another problem is the m_rateControlMethod was set to RATECONTROL_CBR if calling DdiEncodeMpeg2::ParseMiscParamVBV in driver.

One way to solve this PSNR issue in FFmpeg level is set the bufsize specially for mpeg2 as the unit of 16kb, and discard the default CBR mode in driver.

Another way is to fix this all in driver level:
https://github.com/intel/media-driver/pull/495

comment:5 Changed 5 weeks ago by fulinjie

Try this to fix in FFmpeg level, and discard the default CBR mode in driver as well:
https://patchwork.ffmpeg.org/patch/11770/

comment:6 follow-up: Changed 5 weeks ago by eero-t

After building latest FFmpeg from git with the above bitrate patch, and this:

https://patchwork.ffmpeg.org/patch/11770/

I get similar SSIM & PSNR values as with QSV, so this looks good.

Still get the same warnings, but I guess they can be ignored:

[mpeg2_vaapi @ 0x56458668d9c0] Driver does not support some wanted packed headers (wanted 0x3, found 0x10).
[mpeg2_vaapi @ 0x56458668d9c0] Sample aspect ratio 10:11 is not representable, signalling square pixels instead.
[mpeg @ 0x564586689e80] VBV buffer size not set, using default size of 230KB

comment:7 in reply to: ↑ 6 Changed 4 weeks ago by fulinjie

Replying to eero-t:

After building latest FFmpeg from git with the above bitrate patch, and this:

https://patchwork.ffmpeg.org/patch/11770/

I get similar SSIM & PSNR values as with QSV, so this looks good.

Still get the same warnings, but I guess they can be ignored:

[mpeg2_vaapi @ 0x56458668d9c0] Driver does not support some wanted packed headers (wanted 0x3, found 0x10).

Driver only supports VA_ENC_PACKED_HEADER_RAW_DATA for mpeg2 profile
https://github.com/intel/media-driver/blob/2d4ae991d359389b594f6e67080909029640b690/media_driver/linux/common/ddi/media_libva_caps.cpp#L540
But desired was set to VA_ENC_PACKED_HEADER_SEQUENCE | VA_ENC_PACKED_HEADER_SEQUENCE
I think a patch can be sent to avoid this warning.

[mpeg2_vaapi @ 0x56458668d9c0] Sample aspect ratio 10:11 is not representable, signalling square pixels instead.

This is due to unprovided aspected ratio. Maybe could define a new ration if not provided like h264 does.

[mpeg @ 0x564586689e80] VBV buffer size not set, using default size of 230KB
}}}

Note: See TracTickets for help on using tickets.