Opened 3 months ago

Last modified 7 weeks ago

#7691 new defect

GPU side format conversions working with MediaSDK and old i965 driver, fail with new iHD driver

Reported by: eero-t Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug: HW upload with format conversion for the decoded data doesn't work for iHD QSV & VA-API backends, although that same conversion works fine with the MediaSDK sample application, and with older i965 driver VA-API backend.

Setup:

  • Distro: Ubuntu 18.04
  • FFmpeg: latest compiled from Git
  • MediaSDK & its deps: latest compiled from Git
  • HW: tested on KBL GT2, KBL GT3e and CFL GT2

How to reproduce:

  1. Use iHD driver:
    $ export LIBVA_DRIVER_NAME=iHD
    
  2. Decode + format conversion with MediaSDK sample app:
    $ sample_decode -hw h265 -w 1920 -h 1080 -i420 -i 1920x540_8bit_60fps.h265 -o /dev/null
    
  3. Decode + format conversion with QSV:
    $ ffmpeg -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v hevc_qsv -i 1920x540_8bit_60fps.h265 -vf scale_qsv=w=1920:h=1080,hwdownload,format=yuv420p -f null -
    
  4. Decode + format conversion with VA-API:
    $ ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 1920x540_8bit_60fps.h265 -vf scale_vaapi=w=1920:h=1080,hwdownload,format=yuv420p -f null -
    
  5. Switch to i965 driver:
    $ export LIBVA_DRIVER_NAME=i965
    
  6. Decode + format conversion with VA-API:
    $ ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 1920x540_8bit_60fps.h265 -vf scale_vaapi=w=1920:h=1080,hwdownload,format=yuv420p -f null -
    

Expected output:

  • Decoded video format conversion works fine with all the drivers (it's just somewhat slower than hwdownloading the Media driver native format)

Actual outcome:

  • Format conversion works with MSDK decode application and i965 VA-API, but fails with iHD VA-API and QSV. Those seem to support only "nv12" for 8-bit data and "p010" for 10-bit data

Note:

  • With i965 driver, following formats work without errors: "yuv420p", "yuyv422", "uyvy422", "nv12", "rgb0", "bgr0", and "p010". With i965, "p010" works only with 10-bit input, whereas MediaSDK sample application seems to support also 8-bit <=> 10-bit format conversions

(Besides transcode, doing something for the decoded data is the other important use-case for video driver. For that, the video format needs to be something that decoded data consumer can do use. If that conversion can't also be done on the GPU, especially for larger video sizes, there's much less point performance-wise in doing the decode there either.)

Change History (4)

comment:1 Changed 3 months ago by cehoyos

Please provide a (actually one) command line that you tested together with the complete, uncut console output to make this a valid ticket.

comment:2 Changed 3 months ago by eero-t

iHD driver "yuv420p" format failure with QSV:

$ LIBVA_DRIVER_NAME=iHD ffmpeg -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v hevc_qsv -i 1920x540_60_yuv420p_4800.h265 -vf scale_qsv=w=1920:h=1080,hwdownload,format=yuv420p -f null -
ffmpeg version N-93005-gd92f06eb66 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
  configuration: --prefix=/opt/install/Nightly_1626 --enable-libmfx --enable-vaapi --enable-sdl2 --disable-libx265 --disable-libx264 --enable-libvpx --enable-libvorbis --enable-libopus --disable-libmp3lame --disable-libass --disable-sndio --enable-libfreetype --enable-gpl --disable-doc
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 44.100 / 58. 44.100
  libavformat    58. 26.100 / 58. 26.100
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 48.100 /  7. 48.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, hevc, from '1920x540_60_yuv420p_4800.h265':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x540 [SAR 1:2 DAR 16:9], 60 fps, 60 tbr, 1200k tbn, 60 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (hevc_qsv) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[NULL @ 0x55e66ece3380] missing picture in access unit with size 1
    Last message repeated 1 times
[hevc_qsv @ 0x55e66ecf6700] A decode call did not consume any data: expect more data at input (-10)
[NULL @ 0x55e66ece3380] missing picture in access unit with size 1
[hevc_qsv @ 0x55e66ecf6700] A decode call did not consume any data: expect more data at input (-10)
[NULL @ 0x55e66ece3380] missing picture in access unit with size 1
[hevc_qsv @ 0x55e66ecf6700] A decode call did not consume any data: expect more data at input (-10)
[NULL @ 0x55e66ece3380] missing picture in access unit with size 1
    Last message repeated 2 times
[hwdownload @ 0x55e66ed60a80] Invalid output format yuv420p for hwframe download.
[Parsed_hwdownload_1 @ 0x55e66f104e40] Failed to configure output pad on Parsed_hwdownload_1
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:0
Conversion failed!

iHD VA-API fail (everything build from git yesterday like in above case):

$ LIBVA_DRIVER_NAME=iHD ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i
 1920x540_60_yuv420p_4800.h265 -vf scale_vaapi=w=1920:h=1080,hwdownload,format=yuv420p -f null -
ffmpeg version N-93005-gd92f06eb66 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
  configuration: --prefix=/opt/install/Nightly_1626 --enable-libmfx --enable-vaapi --enable-sdl2 --disable-libx265 --disable-libx264 --enable-libvpx --enable-libvorbis --enable-libopus --disable-libmp3lame --disable-libass --disable-sndio --enable-libfreetype --enable-gpl --disable-doc
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 44.100 / 58. 44.100
  libavformat    58. 26.100 / 58. 26.100
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 48.100 /  7. 48.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, hevc, from '1920x540_60_yuv420p_4800.h265':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x540 [SAR 1:2 DAR 16:9], 60 fps, 60 tbr, 1200k tbn, 60 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.26.100
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc58.44.100 wrapped_avframe
[AVHWFramesContext @ 0x558ee4ad3440] Failed to sync surface 0x22: 1 (operation failed).
[hwdownload @ 0x558ee48ef180] Failed to download frame: -5.
Error while filtering: Input/output error
Failed to inject frame into filter network: Input/output error
Error while processing the decoded data for stream #0:0
Conversion failed!

i965 VA-API works fine ("i965_drv_video.so" is from Ubuntu 18.04, rest is like above):

$ LIBVA_DRIVER_NAME=i965 ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 1920x540_60_yuv420p_4800.h265 -vf scale_vaapi=w=1920:h=1080,hwdownload,format=yuv420p -f null -
ffmpeg version N-93005-gd92f06eb66 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
  configuration: --prefix=/opt/install/Nightly_1626 --enable-libmfx --enable-vaapi --enable-sdl2 --disable-libx265 --disable-libx264 --enable-libvpx --enable-libvorbis --enable-libopus --disable-libmp3lame --disable-libass --disable-sndio --enable-libfreetype --enable-gpl --disable-doc
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 44.100 / 58. 44.100
  libavformat    58. 26.100 / 58. 26.100
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 48.100 /  7. 48.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, hevc, from '1920x540_60_yuv420p_4800.h265':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x540 [SAR 1:2 DAR 16:9], 60 fps, 60 tbr, 1200k tbn, 60 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.26.100
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc58.44.100 wrapped_avframe
frame= 4800 fps=376 q=-0.0 Lsize=N/A time=00:01:20.00 bitrate=N/A speed=6.27x    
video:2512kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

As does the iHD driver if output format is changed from yuv420p to nv12 (for 8bit input):

$ LIBVA_DRIVER_NAME=iHD ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 1920x540_60_yuv420p_4800.h265 -vf scale_vaapi=w=1920:h=1080,hwdownload,format=nv12 -f null -
ffmpeg version N-93005-gd92f06eb66 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
  configuration: --prefix=/opt/install/Nightly_1626 --enable-libmfx --enable-vaapi --enable-sdl2 --disable-libx265 --disable-libx264 --enable-libvpx --enable-libvorbis --enable-libopus --disable-libmp3lame --disable-libass --disable-sndio --enable-libfreetype --enable-gpl --disable-doc
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 44.100 / 58. 44.100
  libavformat    58. 26.100 / 58. 26.100
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 48.100 /  7. 48.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, hevc, from '1920x540_60_yuv420p_4800.h265':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x540 [SAR 1:2 DAR 16:9], 60 fps, 60 tbr, 1200k tbn, 60 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.26.100
    Stream #0:0: Video: wrapped_avframe, nv12, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc58.44.100 wrapped_avframe
frame= 4800 fps=408 q=-0.0 Lsize=N/A time=00:01:20.00 bitrate=N/A speed= 6.8x    
video:2512kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

PS. test Video is converted from http://4ksamples.com/ses-astra-uhd-test-1-2160p-uhdtv/ with FFmpeg / libx265.

comment:3 Changed 3 months ago by eero-t

As a reference, at the bottom of this page is a list of platform / format combinations supported by libva iHD backend:
https://github.com/intel/media-driver/blob/master/docs/media_features.md

comment:4 Changed 7 weeks ago by eero-t

I just noticed that in the vaapi backend iHD driver case, instead of hwdownloading nv12 format, one could also remove the scale_vaapi stuff and it works.

I.e. either format conversion, or scaling, but not both, are supported by FFmpeg with iHD vaapi driver, although doing both works with i965 vaapi driver (and when using iHD driver with MediaSDK sample_decode application).

Removing scale_qsv doesn't help with FFmpeg QSV backend though, only nv12 hwdownload works with that.

Note: See TracTickets for help on using tickets.