Opened 6 years ago
Last modified 6 years ago
#7523 new defect
VAAPI: Fail to Decode an H264 LP Multi-Slice Encoded Video
Reported by: | U. Artie Eoff | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | undetermined |
Version: | git-master | Keywords: | vaapi |
Cc: | sw@jkqxz.net, zhong.li@intel.com, eero.t.tamminen@intel.com | Blocked By: | |
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Summary of the bug:
With ffmpeg-vaapi on iHD driver, unable to decode video that was encoded with AVC low-power multi-slice.
On the flip-side, the i965 driver refuses even to encode with low-power multi-slice (i.e. VAConfigAttribEncSliceStructure returns not supported). Not sure if that is a driver bug or not.
Multi-slice support enabled in:
commit a769e72c750e113c75b1f106296ade94caa28748 Author: Mark Thompson <sw@jkqxz.net> Date: Sun Sep 23 22:52:58 2018 +0100 vaapi_encode_h264: Enable multiple-slice support
How to reproduce:
Encode:
% LIBVA_DRIVER_NAME=iHD ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 \ -v verbose -f rawvideo -pix_fmt yuv420p -s:v 1280x720 -i ./input.yuv \ -vf 'format=nv12,hwupload' -c:v h264_vaapi -low_power 1 -profile:v high \ -g 30 -qp 14 -bf 0 -quality 4 -slices 4 -vframes 10 -y test.h264
Encode Output:
ffmpeg version N-92277-ga769e72c750e Copyright (c) 2000-2018 the FFmpeg developers built with gcc 8 (GCC) configuration: --prefix=/home/uaeoff/Work/workspace/media/install --disable-static --enable-shared --enable-libdrm --enable-vaapi --enable-libmfx --disable-amf --disable-audiotoolbox --disable-cuda --disable-cuda-sdk --disable-cuvid --disable-d3d11va --disable-dxva2 --disable-libnpp --disable-mmal --disable-nvdec --disable-nvenc --disable-omx --disable-omx-rpi --disable-rkmpp --disable-v4l2-m2m --disable-vdpau --disable-videotoolbox --enable-nonfree --enable-gpl libavutil 56. 20.100 / 56. 20.100 libavcodec 58. 34.100 / 58. 34.100 libavformat 58. 19.102 / 58. 19.102 libavdevice 58. 4.106 / 58. 4.106 libavfilter 7. 38.100 / 7. 38.100 libswscale 5. 2.100 / 5. 2.100 libswresample 3. 2.100 / 3. 2.100 libpostproc 55. 2.100 / 55. 2.100 [AVHWDeviceContext @ 0x1879880] Opened VA display via DRM device /dev/dri/renderD128. [AVHWDeviceContext @ 0x1879880] libva: VA-API version 1.4.0 [AVHWDeviceContext @ 0x1879880] libva: va_getDriverName() returns 0 [AVHWDeviceContext @ 0x1879880] libva: User requested driver 'iHD' [AVHWDeviceContext @ 0x1879880] libva: Trying to open /home/uaeoff/Work/workspace/media/install/lib/dri/iHD_drv_video.so [AVHWDeviceContext @ 0x1879880] libva: Found init function __vaDriverInit_1_4 [AVHWDeviceContext @ 0x1879880] libva: va_openDriver() returns 0 [AVHWDeviceContext @ 0x1879880] Initialised VAAPI connection: version 1.4 [AVHWDeviceContext @ 0x1879880] VAAPI driver: Intel iHD driver - 1.0.0. [AVHWDeviceContext @ 0x1879880] Driver not found in known nonstandard list, using standard behaviour. [rawvideo @ 0x18b4340] Estimating duration from bitrate, this may be inaccurate Input #0, rawvideo, from '/home/uaeoff/Work/workspace/media/input.yuv': Duration: 00:00:24.16, start: 0.000000, bitrate: 276480 kb/s Stream #0:0: Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p, 1280x720, 276480 kb/s, 25 tbr, 25 tbn, 25 tbc Stream mapping: Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_vaapi)) Press [q] to stop, [?] for help [graph 0 input from stream 0:0 @ 0x18cff00] w:1280 h:720 pixfmt:yuv420p tb:1/25 fr:25/1 sar:0/1 sws_param:flags=2 [auto_scaler_0 @ 0x18d33c0] w:iw h:ih flags:'bicubic' interl:0 [Parsed_format_0 @ 0x18ce540] auto-inserting filter 'auto_scaler_0' between the filter 'graph 0 input from stream 0:0' and the filter 'Parsed_format_0' [auto_scaler_0 @ 0x18d33c0] w:1280 h:720 fmt:yuv420p sar:0/1 -> w:1280 h:720 fmt:nv12 sar:0/1 flags:0x4 [h264_vaapi @ 0x18c1240] Input surface format is nv12. [h264_vaapi @ 0x18c1240] Using VAAPI profile VAProfileH264High (7). [h264_vaapi @ 0x18c1240] Using VAAPI entrypoint VAEntrypointEncSliceLP (8). [h264_vaapi @ 0x18c1240] Using VAAPI render target format YUV420 (0x1). [h264_vaapi @ 0x18c1240] Using constant-quality mode. [h264_vaapi @ 0x18c1240] Using intra and P-frames (supported references: 3 / 0). [h264_vaapi @ 0x18c1240] Slice count rounded up to 45 (from 4) due to driver constraints on slice structure. [h264_vaapi @ 0x18c1240] Encoding pictures with 45 slices (default size 1 block rows). [h264_vaapi @ 0x18c1240] All wanted packed headers available (wanted 0xd, found 0x1f). [h264_vaapi @ 0x18c1240] Using level 3.1. Output #0, h264, to 'test.h264': Metadata: encoder : Lavf58.19.102 Stream #0:0: Video: h264 (h264_vaapi) (High), 1 reference frame, vaapi_vld, 1280x720, q=-1--1, 25 fps, 25 tbn, 25 tbc Metadata: encoder : Lavc58.34.100 h264_vaapi No more output streams to write to, finishing. frame= 10 fps=0.0 q=-0.0 Lsize= 3444kB time=00:00:00.40 bitrate=70526.0kbits/s speed=3.19x video:3444kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000% Input file #0 (/home/uaeoff/Work/workspace/media/src/otc-media/assets/yuv/720p5994_parkrun_ter_1280x720_I420.yuv): Input stream #0:0 (video): 10 packets read (13824000 bytes); 10 frames decoded; Total: 10 packets (13824000 bytes) demuxed Output file #0 (test.h264): Output stream #0:0 (video): 10 frames encoded; 10 packets muxed (3526298 bytes); Total: 10 packets (3526298 bytes) muxed [AVIOContext @ 0x18cbc80] Statistics: 0 seeks, 14 writeouts [AVIOContext @ 0x18bcac0] Statistics: 13824000 bytes read, 0 seeks
Decode:
% LIBVA_DRIVER_NAME=iHD ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 \ -v verbose -i ./test.h264 -pix_fmt yuv420p -f rawvideo \ -vsync passthrough -vframes 10 -y test.yuv
Decode Output:
ffmpeg version N-92277-ga769e72c750e Copyright (c) 2000-2018 the FFmpeg developers built with gcc 8 (GCC) configuration: --prefix=/home/uaeoff/Work/workspace/media/install --disable-static --enable-shared --enable-libdrm --enable-vaapi --enable-libmfx --disable-amf --disable-audiotoolbox --disable-cuda --disable-cuda-sdk --disable-cuvid --disable-d3d11va --disable-dxva2 --disable-libnpp --disable-mmal --disable-nvdec --disable-nvenc --disable-omx --disable-omx-rpi --disable-rkmpp --disable-v4l2-m2m --disable-vdpau --disable-videotoolbox --enable-nonfree --enable-gpl libavutil 56. 20.100 / 56. 20.100 libavcodec 58. 34.100 / 58. 34.100 libavformat 58. 19.102 / 58. 19.102 libavdevice 58. 4.106 / 58. 4.106 libavfilter 7. 38.100 / 7. 38.100 libswscale 5. 2.100 / 5. 2.100 libswresample 3. 2.100 / 3. 2.100 libpostproc 55. 2.100 / 55. 2.100 [AVHWDeviceContext @ 0x885500] Opened VA display via DRM device /dev/dri/renderD128. [AVHWDeviceContext @ 0x885500] libva: VA-API version 1.4.0 [AVHWDeviceContext @ 0x885500] libva: va_getDriverName() returns 0 [AVHWDeviceContext @ 0x885500] libva: User requested driver 'iHD' [AVHWDeviceContext @ 0x885500] libva: Trying to open /home/uaeoff/Work/workspace/media/install/lib/dri/iHD_drv_video.so [AVHWDeviceContext @ 0x885500] libva: Found init function __vaDriverInit_1_4 [AVHWDeviceContext @ 0x885500] libva: va_openDriver() returns 0 [AVHWDeviceContext @ 0x885500] Initialised VAAPI connection: version 1.4 [AVHWDeviceContext @ 0x885500] VAAPI driver: Intel iHD driver - 1.0.0. [AVHWDeviceContext @ 0x885500] Driver not found in known nonstandard list, using standard behaviour. [h264 @ 0x8c1380] Reinit context to 1280x720, pix_fmt: yuv420p [h264 @ 0x8c1380] slice type 32 too large at 80 [h264 @ 0x8c1380] decode_slice_header error [h264 @ 0x8c1380] slice type 32 too large at 160 <snip> [h264 @ 0x8c1380] concealing 3520 DC, 3520 AC, 3520 MV errors in I frame [h264 @ 0x8c1380] slice type 32 too large at 80 [h264 @ 0x8c1380] decode_slice_header error [h264 @ 0x8c1380] slice type 32 too large at 160 [h264 @ 0x8c1380] decode_slice_header error <snip> [h264 @ 0x8c1380] concealing 3520 DC, 3520 AC, 3520 MV errors in P frame <snip> [AVHWFramesContext @ 0x7f44780a1e00] Failed to sync surface 0x1c: 23 (internal decoding error). [h264 @ 0x94a1c0] Failed to transfer data to output frame: -5. Error while processing the decoded data for stream #0:0 [AVIOContext @ 0x8c2e80] Statistics: 0 seeks, 0 writeouts [AVIOContext @ 0x8c8740] Statistics: 3526298 bytes read, 0 seeks <snip> Conversion failed!
Software Stack:
libva (master) heads/master-0-g285267586a3d https://github.com/01org/libva gmmlib (master) heads/master-0-gaedd82e2da01 https://github.com/intel/gmmlib intel-media-driver (master) heads/master-0-g9d258e13eba1 https://github.com/intel/media-driver intel-vaapi-driver (master) heads/master-0-gdc20b6405a53 https://github.com/01org/intel-vaapi-driver msdk (master) heads/master-0-gc6b3d1cd4569 https://github.com/Intel-Media-SDK/MediaSDK ffmpeg (master) heads/master-0-ga769e72c750e https://git.ffmpeg.org/ffmpeg
Change History (10)
comment:1 by , 6 years ago
Priority: | critical → normal |
---|
comment:2 by , 6 years ago
comment:3 by , 6 years ago
Yes, this is definitely a iHD driver bug. Filed issue at https://github.com/intel/media-driver/issues/389
comment:4 by , 6 years ago
VAConfigAttribEncSliceStructure is a hint only, so it is not mandatory to support VAConfigAttribEncSliceStructure in a backend driver. I think a patch is required for FFmpeg too.
comment:5 by , 6 years ago
This bug is not really about i965 backend driver. It was only mentioned because it was unclear whether i965 should actually support it or if it was because of a HW limitation. In case of HW limitation, the conclusion would be that iHD is reporting it wrong. But, HW limitation or not, it seemed odd that i965 has logic to support VAConfigAttribEncMaxSlices > 1 for LP but not support for VAConfigAttribEncSliceStructure.
Nonetheless, on i965, VAConfigAttribEncSliceStructure returns NOT SUPPORTED, and ffmpeg doesn't get choked up about VAConfigAttribEncMaxSlices > 1... and thus, refuses to proceed with encoding. This seems like reasonable behavior for ffmpeg in that case, right?
Finally (the original reason for this bug), since iHD VAConfigAttribEncSliceStructure returns EQUAL_ROWS | MAX_SLICE_SIZE
, ffmpeg proceeds to encode it with this setting since iHD said it's supported. Unfortunately, iHD seems to have trouble encoding with this setting.
So, I'm not sure what you would expect needs patched in ffmpeg.
comment:6 by , 6 years ago
I mentioned VAConfigAttribEncSliceStructure is a hint only, driver may or may not support this hint. My understanding is that any structure is supported as soon as the structure meet the spec when the driver returns NOT SUPPORTED for this hint attribute.
comment:7 by , 6 years ago
Ok, I guess I am a little confused about the meaning. Maybe @jkqxz understands better.
comment:8 by , 6 years ago
What I mean is that returning NOT SUPPORTED for VAConfigAttribEncSliceStructure doesn't indicate the backend driver doesn't support multiple slice encoding.
comment:9 by , 6 years ago
That isn't how I would read the documentation, though I admit it's a bit unclear - given that all the options are constraining the user to particular slice layouts, the fallback position being the most lenient seems inappropriate. Perhaps it would be best if you send a patch to libva clarifying the documentation, and any discussion can happen there?
In any case, this is very easy to fix on the i965 driver side to enable it with the stricter interpretation - add the missing EncSliceStructure
attribute for the relevant LP encode entrypoints.
comment:10 by , 6 years ago
Cc: | added |
---|
The driver seems to have only used part of (or overwritten some of?) the slice header when run in low-power mode.
Given a 64x64 input, with the normal encoder:
With the low-power encoder, the second slice header (which is written identically to the above case) has been messed up somehow: