Support for HW accelerated 10-bit -> 8-bit conversion support

[eero-t]

Summary of the bug:

MediaSDK sample application can transcode 10-bit (HEVC) content to 8-bit (AVC), so same should(?) be possible in HW accelerated manner also with VAAPI and/or QSV backend.

However, it fails, and one needs to do the conversion with round trip through CPU side.

How to reproduce:

ffmpeg -y -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i SES.Astra.UHD.Test.1.2160p.UHDTV.AAC.HEVC.x265-LiebeIst.mkv -c:v h264_vaapi -vf format=nv12 output.h264
ffmpeg version N-93264-g85051febc6 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
...
Input #0, matroska,webm, from 'SES.Astra.UHD.Test.1.2160p.UHDTV.AAC.HEVC.x265-LiebeIst.mkv':
  Metadata:
    encoder         : libebml v1.3.1 + libmatroska v1.4.2
    creation_time   : 2015-10-03T13:49:42.000000Z
  Duration: 00:01:49.29, start: 0.816000, bitrate: 18484 kb/s
    Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv), 3840x2160 [SAR 1:1 DAR 16:9], 60 fps, 60 tbr, 1k tbn, 60 tbc (default)
...
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_vaapi))
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
Conversion failed!

FFmpeg gives exactly the same error with i965 driver (in Ubuntu 18.04) as with above git version of iHD media-driver.

QSV with git version of MediaSDK gives also exactly the same error:

ffmpeg -y -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v hevc_qsv -i SES.Astra.UHD.Test.1.2160p.UHDTV.AAC.HEVC.x265-LiebeIst.mkv -c:v h264_qsv -vf format=nv12  output.h264

With MediaSDK sample application, it works fine (when one tells with -ec option the format to which the 10-bit data needs to be converted):

sample_multi_transcode -i::h265 SES.Astra.UHD.Test.1.2160p.UHDTV.AAC.HEVC.x265-LiebeIst.h265 -ec::nv12 -o::h264 output.h264

With VAAPI, one can do the conversion with roundtrip through CPU side, but that's several times slower:

ffmpeg -y -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i SES.Astra.UHD.Test.1.2160p.UHDTV.AAC.HEVC.x265-LiebeIst.mkv -c:v h264_vaapi -vf hwdownload,format=nv12,hwupload output.h264

FYI: Test video is from here: ​http://4ksamples.com/ses-astra-uhd-test-1-2160p-uhdtv/

[eero-t]

There was earlier bug in MSDK or media-driver, but nowadays format conversion works also with MSDK sample_decode application which with this is easier to check against:
sample_decode -hw h265 -w 300 -h 300 -nv12 -i 1920x1080_10bit_50fps.h265 -o /dev/null

[eero-t]

VAAPI & QSV command lines work (at least with latest Git versions of everything), if one does scale_vaapi/scale_qsv=<size> in addition to format conversion. Just doing format conversion fails (which is nearly reverse of issue #7691).

Hoewever, even that fails, if input is 4K instead of FullHD (tested on KBL).

[tvsht]

Just got hit by this on AMD RAVEN APU with the same error, while trying to transcode 10bit hevc to h264 via vaapi

[eero-t]

Just got hit by this on AMD RAVEN APU with the same error, while trying to transcode 10bit hevc to h264 via vaapi

• Does that HW and its driver support 10bit HEVC decode + 10-bit -> 8-bit conversion i.e. are you sure it's FFmpeg VAAPI backend and not driver issue?
• If yes, does it work if you add scaling, like happens with iHD?
• If yes, have you still a problem in 4K like I have with iHD?

[Linjie.Fu]

Color space convert should be inserted into the pipeline.

For QSV:
ffmpeg -y -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v hevc_qsv -i SES.Astra.UHD.Test.1.2160p.UHDTV.AAC.HEVC.x265-LiebeIst.mkv -vf scale_qsv=format=nv12 -c:v h264_qsv output.h264

[Linjie.Fu]

For VAAPI:
ffmpeg -y -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i ./10_sunflower.mp4 -vf scale_vaapi=format=nv12 -c:v h264_vaapi output.mp4

This pipeline works.

However, it failed with the provided clips and needs to be investigated more.

[Linjie.Fu]

10bit hevc input clips

[Linjie.Fu]

This issue happens when resolution is large(4K).

ffmpeg -y -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 4K_10_sunflower.mp4 -vf scale_vaapi=w=1920:h=1080 -c:v hevc_vaapi output.mp4

or

ffmpeg -y -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 4K_10_sunflower.mp4 -vf scale_vaapi=format=nv12 -c:v hevc_vaapi output.mp4

Both got errors:

[VP]:   CRITICAL - RenderPhase:5936: Failed to create filter description.
[VP]:   CRITICAL - RenderMultiPhase:2008: MOS returned error, eStatus = 0x19
[VP]:   CRITICAL - Render:2340: MOS returned error, eStatus = 0x19
[VP]:   CRITICAL - RenderComposite:953: MOS returned error, eStatus = 0x19
[VP]:   CRITICAL - RenderPass:776: MOS returned error, eStatus = 0x19
[VP]:   CRITICAL - Render:1205: MOS returned error, eStatus = 0x19
[VP]:   CRITICAL - DdiVp_EndPicture:3524: Failed to call render function.
[Parsed_scale_vaapi_0 @ 0x5615ed191600] Failed to start picture processing: 1 (operation failed).
Error while filtering: Input/output error
Failed to inject frame into filter network: Input/output error

Related issue in media driver:
​https://github.com/intel/media-driver/issues/499

[Sun, Xinpeng]

This is a driver issue and the related issue in media driver is:
​https://github.com/intel/media-driver/issues/760
More details are in this issue description.

[U. Artie Eoff]

This should be fixed according to ​https://github.com/intel/media-driver/issues/760#issuecomment-590476083

[eero-t]

This should be fixed

Thanks, so it seems!

With the fixed media-driver, one just needs to use "scale_(vaapi|qsv)=format=..." instead of "format=...", and it works, also with 4K input.

This is a driver issue

As this wasn't FFmpeg issue, marking this as invalid.