Opened 5 years ago
Last modified 21 months ago
#7830 new defect
Allocation size of input surface for qsv not takes into account alignment required by qsv
| Reported by: | alexandr-konovalov | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | avutil |
| Version: | git-master | Keywords: | qsv |
| Cc: | Blocked By: | ||
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description
I'm encoding P010 stream to h265 using qsv. Command line is below. I see that output streams are not bit-to-bit equal for several runs. In valgrind it looks like Intel MSDK reads after allocated
space.
==11312== Thread 10: ==11312== Invalid read of size 8 ==11312== at 0x4C367EE: memmove (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==11312== by 0xB2B4B7D: unsigned char* std::__copy_move<false, true, std::random_access_iterator_tag>::__copy_m<unsigned char>(unsigned char const*, unsigned char const*, unsigned char*) (stl_algobase.h:372) ==11312== by 0xB2B4B34: unsigned char* std::__copy_move_a<false, unsigned char const*, unsigned char*>(unsigned char const*, unsigned char const*, unsigned char*) (stl_algobase.h:390) ==11312== by 0xB2B4A7B: unsigned char* std::__copy_move_a2<false, unsigned char const*, unsigned char*>(unsigned char const*, unsigned char const*, unsigned char*) (stl_algobase.h:428) ==11312== by 0xB2B4999: unsigned char* std::copy<unsigned char const*, unsigned char*>(unsigned char const*, unsigned char const*, unsigned char*) (stl_algobase.h:460) ==11312== by 0xB2F1825: int mfxCopyRect<unsigned char>(unsigned char const*, int, unsigned char*, int, mfxSize, int) (fast_copy.h:221) ==11312== by 0xB2F0D67: FastCopy::Copy(unsigned char*, unsigned int, unsigned char*, unsigned int, mfxSize, int) (fast_copy.h:250) ==11312== by 0xB2EE2FA: CoreDoSWFastCopy(mfxFrameSurface1*, mfxFrameSurface1*, int) (libmfx_core.cpp:1309) ==11312== by 0xB2FC905: VAAPIVideoCORE::DoFastCopyExtended(mfxFrameSurface1*, mfxFrameSurface1*) (libmfx_core_vaapi.cpp:1242) ==11312== by 0xB2FC029: VAAPIVideoCORE::DoFastCopyWrapper(mfxFrameSurface1*, unsigned short, mfxFrameSurface1*, unsigned short) (libmfx_core_vaapi.cpp:1033) ==11312== by 0xB5E453B: MfxHwVideoProcessing::VideoVPPHW::CopyPassThrough(mfxFrameSurface1*, mfxFrameSurface1*) (mfx_vpp_hw.cpp:1822) ==11312== by 0xB5EAC3D: MfxHwVideoProcessing::VideoVPPHW::SyncTaskSubmission(MfxHwVideoProcessing::DdiTask*) (mfx_vpp_hw.cpp:3791) ==11312== Address 0x8dc8540 is 0 bytes after a block of size 115,264 alloc'd ==11312== at 0x4C31E76: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==11312== by 0x4C31F91: posix_memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==11312== by 0xFB9BC9: av_malloc (mem.c:87) ==11312== by 0xF98468: av_buffer_alloc (buffer.c:72) ==11312== by 0x7B7EF4: av_grow_packet (avpacket.c:139) ==11312== by 0x74806A: append_packet_chunked (utils.c:289) ==11312== by 0x7056AA: rawvideo_read_packet (rawvideodec.c:78) ==11312== by 0x7493CD: ff_read_packet (utils.c:856) ==11312== by 0x74C833: read_frame_internal (utils.c:1582) ==11312== by 0x750BFB: avformat_find_stream_info (utils.c:3772) ==11312== by 0x47FF3F: open_input_file (ffmpeg_opt.c:1126) ==11312== by 0x48228E: open_files (ffmpeg_opt.c:3273) ==11312== by 0x48228E: ffmpeg_parse_options (ffmpeg_opt.c:3313)
The problem seems that calculation of an input surface buffer size in av_image_fill_pointers() doesn't take into account an alignment that later done in qsv_init_surface()
surf->Info.Height = FFALIGN(ctx->height, 16);
while preparation to MFXVideoVPP_RunFrameVPPAsync() call.
ffmpeg_g -v 9 -loglevel 99 -init_hw_device qsv=qsv:hw -hwaccel qsv -filter_hw_device qsv -v verbose -f rawvideo -pix_fmt p010le -s:v 192x200 -r:v 25 -i IMG-192x200_1.p010.yuv -vf 'format=p010le,hwupload=extra_hw_frames=64' -an -c:v hevc_qsv -profile:v main10 -g 30 -slices 1 -bf 0 -b:v 500k -maxrate 500k -vframes 1 -y curr1.h265
ffmpeg version N-93532-g772c73e Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 6.3.1 (GCC) 20170216 (Red Hat 6.3.1-3)
configuration: --arch=x86_64 --disable-shared --enable-static --extra-libs=-ldl --disable-yasm --enable-libmfx --extra-ldflags=-L/tmp/akonovx/msdk/__bin/debug
libavutil 56. 26.100 / 56. 26.100
libavcodec 58. 48.101 / 58. 48.101
libavformat 58. 26.101 / 58. 26.101
libavdevice 58. 7.100 / 58. 7.100
libavfilter 7. 48.100 / 7. 48.100
libswscale 5. 4.100 / 5. 4.100
libswresample 3. 4.100 / 3. 4.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-init_hw_device' ... matched as option 'init_hw_device' (initialise hardware device) with argument 'qsv=qsv:hw'.
Reading option '-hwaccel' ... matched as option 'hwaccel' (use HW accelerated decoding) with argument 'qsv'.
Reading option '-filter_hw_device' ... matched as option 'filter_hw_device' (set hardware device used when filtering) with argument 'qsv'.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'verbose'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'p010le'.
Reading option '-s:v' ... matched as option 's' (set frame size (WxH or abbreviation)) with argument '192x200'.
Reading option '-r:v' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '25'.
Reading option '-i' ... matched as input url with argument 'IMG-192x200_1.p010.yuv'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'format=p010le,hwupload=extra_hw_frames=64'.
Reading option '-an' ... matched as option 'an' (disable audio) with argument '1'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'hevc_qsv'.
Reading option '-profile:v' ... matched as option 'profile' (set profile) with argument 'main10'.
Reading option '-g' ... matched as AVOption 'g' with argument '30'.
Reading option '-slices' ... matched as AVOption 'slices' with argument '1'.
Reading option '-bf' ... matched as AVOption 'bf' with argument '0'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '500k'.
Reading option '-maxrate' ... matched as AVOption 'maxrate' with argument '500k'.
Reading option '-vframes' ... matched as option 'vframes' (set the number of video frames to output) with argument '1'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option 'curr1.h265' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Applying option init_hw_device (initialise hardware device) with argument qsv=qsv:hw.
[AVHWDeviceContext @ 0x3e38c40] Cannot open X11 display .
[AVHWDeviceContext @ 0x3e38c40] Opened VA display via DRM device /dev/dri/renderD128.
[AVHWDeviceContext @ 0x3e38c40] libva: VA-API version 1.4.0
[AVHWDeviceContext @ 0x3e38c40] libva: va_getDriverName() returns 0
[AVHWDeviceContext @ 0x3e38c40] libva: User requested driver 'iHD'
[AVHWDeviceContext @ 0x3e38c40] libva: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
[AVHWDeviceContext @ 0x3e38c40] libva: Found init function __vaDriverInit_1_4
[AVHWDeviceContext @ 0x3e38c40] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x3e38c40] Initialised VAAPI connection: version 1.4
[AVHWDeviceContext @ 0x3e38c40] Format 0x41524742 -> bgra.
[AVHWDeviceContext @ 0x3e38c40] Format 0x42475241 -> argb.
[AVHWDeviceContext @ 0x3e38c40] Format 0x41424752 -> rgba.
[AVHWDeviceContext @ 0x3e38c40] Format 0x52474241 -> abgr.
[AVHWDeviceContext @ 0x3e38c40] Format 0x58524742 -> bgr0.
[AVHWDeviceContext @ 0x3e38c40] Format 0x42475258 -> 0rgb.
[AVHWDeviceContext @ 0x3e38c40] Format 0x58424752 -> rgb0.
[AVHWDeviceContext @ 0x3e38c40] Format 0x52474258 -> 0bgr.
Last message repeated 1 times
[AVHWDeviceContext @ 0x3e38c40] Format 0x3231564e -> nv12.
[AVHWDeviceContext @ 0x3e38c40] Format 0x3132564e -> unknown.
[AVHWDeviceContext @ 0x3e38c40] Format 0x32595559 -> yuyv422.
[AVHWDeviceContext @ 0x3e38c40] Format 0x59565955 -> uyvy422.
[AVHWDeviceContext @ 0x3e38c40] Format 0x32315659 -> yuv420p.
[AVHWDeviceContext @ 0x3e38c40] Format 0x56555949 -> yuv420p.
[AVHWDeviceContext @ 0x3e38c40] Format 0x30313259 -> unknown.
[AVHWDeviceContext @ 0x3e38c40] Format 0x36313259 -> unknown.
[AVHWDeviceContext @ 0x3e38c40] Format 0x48323234 -> yuv422p.
[AVHWDeviceContext @ 0x3e38c40] Format 0x56323234 -> yuv440p.
[AVHWDeviceContext @ 0x3e38c40] Format 0x30303859 -> gray.
[AVHWDeviceContext @ 0x3e38c40] Format 0x50313134 -> yuv411p.
[AVHWDeviceContext @ 0x3e38c40] Format 0x33434d49 -> unknown.
[AVHWDeviceContext @ 0x3e38c40] Format 0x50343434 -> yuv444p.
[AVHWDeviceContext @ 0x3e38c40] Format 0x50424752 -> unknown.
[AVHWDeviceContext @ 0x3e38c40] Format 0x50524742 -> unknown.
[AVHWDeviceContext @ 0x3e38c40] Format 0x38303250 -> unknown.
[AVHWDeviceContext @ 0x3e38c40] Format 0x36313050 -> unknown.
[AVHWDeviceContext @ 0x3e38c40] Format 0x30313050 -> p010le.
[AVHWDeviceContext @ 0x3e38c40] VAAPI driver: Intel iHD driver - 18.4.git_e61f5c_2019-01-29.
[AVHWDeviceContext @ 0x3e38c40] Driver not found in known nonstandard list, using standard behaviour.
[AVHWDeviceContext @ 0x3e38840] Initialize MFX session: API version is 1.28, implementation version is 1.29
[AVHWDeviceContext @ 0x3e38840] MFX compile/runtime API: 1.28/1.29
Applying option filter_hw_device (set hardware device used when filtering) with argument qsv.
Applying option v (set logging level) with argument verbose.
[rawvideo @ 0x3e834c0] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from 'IMG-192x200_1.p010.yuv':
Duration: 00:00:00.04, start: 0.000000, bitrate: 23040 kb/s
Stream #0:0: Video: rawvideo, 1 reference frame, p010le, 192x200, 23040 kb/s, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> hevc (hevc_qsv))
Press [q] to stop, [?] for help
[graph 0 input from stream 0:0 @ 0x3ebee00] w:192 h:200 pixfmt:p010le tb:1/25 fr:25/1 sar:0/1 sws_param:flags=2
[AVHWDeviceContext @ 0x3ebde00] VAAPI driver: Intel iHD driver - 18.4.git_e61f5c_2019-01-29.
[AVHWDeviceContext @ 0x3ebde00] Driver not found in known nonstandard list, using standard behaviour.
VPP PIPELINE:
MFX_EXTBUFF_VPP_RSHIFT_IN
RESIZE
MFX_EXTBUFF_VPP_LSHIFT_OUT
VPP PIPELINE:
MFX_EXTBUFF_VPP_RSHIFT_IN
RESIZE
MFX_EXTBUFF_VPP_LSHIFT_OUT
VPP PIPELINE:
MFX_EXTBUFF_VPP_RSHIFT_IN
RESIZE
MFX_EXTBUFF_VPP_LSHIFT_OUT
VPP PIPELINE:
MFX_EXTBUFF_VPP_RSHIFT_IN
RESIZE
MFX_EXTBUFF_VPP_LSHIFT_OUT
[hevc_qsv @ 0x3eb9740] Using the constant bitrate (CBR) ratecontrol method
[AVHWDeviceContext @ 0x3f02d40] VAAPI driver: Intel iHD driver - 18.4.git_e61f5c_2019-01-29.
[AVHWDeviceContext @ 0x3f02d40] Driver not found in known nonstandard list, using standard behaviour.
[hevc_qsv @ 0x3eb9740] profile: main10; level: 20
[hevc_qsv @ 0x3eb9740] GopPicSize: 30; GopRefDist: 1; GopOptFlag: closed ; IdrInterval: 1
[hevc_qsv @ 0x3eb9740] TargetUsage: 4; RateControlMethod: CBR
[hevc_qsv @ 0x3eb9740] BufferSizeInKB: 125; InitialDelayInKB: 62; TargetKbps: 500; MaxKbps: 500; BRCParamMultiplier: 1
[hevc_qsv @ 0x3eb9740] NumSlice: 1; NumRefFrame: 3
[hevc_qsv @ 0x3eb9740] RateDistortionOpt: unknown
[hevc_qsv @ 0x3eb9740] RecoveryPointSEI: unknown IntRefType: 0; IntRefCycleSize: 0; IntRefQPDelta: 0
[hevc_qsv @ 0x3eb9740] MaxFrameSize: 0; MaxSliceSize: 0;
[hevc_qsv @ 0x3eb9740] BitrateLimit: unknown; MBBRC: unknown; ExtBRC: OFF
[hevc_qsv @ 0x3eb9740] Trellis: auto
[hevc_qsv @ 0x3eb9740] VDENC: OFF
[hevc_qsv @ 0x3eb9740] RepeatPPS: OFF; NumMbPerSlice: 0; LookAheadDS: unknown
[hevc_qsv @ 0x3eb9740] AdaptiveI: unknown; AdaptiveB: unknown; BRefType: off
[hevc_qsv @ 0x3eb9740] MinQPI: 0; MaxQPI: 0; MinQPP: 0; MaxQPP: 0; MinQPB: 0; MaxQPB: 0
[hevc_qsv @ 0x3eb9740] GPB: ON
[hevc_qsv @ 0x3eb9740] FrameRateExtD: 1; FrameRateExtN: 25
Output #0, hevc, to 'curr1.h265':
Metadata:
encoder : Lavf58.26.101
Stream #0:0: Video: hevc (hevc_qsv), 1 reference frame, qsv, 192x200, q=2-31, 500 kb/s, 25 fps, 25 tbn, 25 tbc
Metadata:
encoder : Lavc58.48.101 hevc_qsv
Side data:
cpb: bitrate max/min/avg: 500000/0/500000 buffer size: 0 vbv_delay: -1
No more output streams to write to, finishing.
frame= 1 fps=0.0 q=-0.0 Lsize= 4kB time=00:00:00.04 bitrate= 819.2kbits/s speed=5.43x
video:4kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
Input file #0 (IMG-192x200_1.p010.yuv):
Input stream #0:0 (video): 1 packets read (115200 bytes); 1 frames decoded;
Total: 1 packets (115200 bytes) demuxed
Output file #0 (curr1.h265):
Output stream #0:0 (video): 1 frames encoded; 1 packets muxed (4096 bytes);
Total: 1 packets (4096 bytes) muxed
[AVIOContext @ 0x3e8eb40] Statistics: 0 seeks, 1 writeouts
[AVIOContext @ 0x3e8c780] Statistics: 115200 bytes read, 0 seeks
Attachments (1)
Change History (10)
by , 5 years ago
| Attachment: | IMG-192x200_1.p010.yuv added |
|---|
comment:1 by , 5 years ago
comment:2 by , 5 years ago
System memory is allocated by av_buffer_alloc() (inside libavutil), size is 115264 Bytes. Then this memory is send to MSDK to do copy from system to video memory. This copy is done by CPU, i.e. it’s invisible to iHD driver. And this is the place where access to unallocated memory occurred, because height (and width) is the aligned value, as you correctly mentioned, but allocation in av_buffer_alloc() was done without taking this into account.
So, yes, the allocated surface is enough, but this is too late.
comment:3 by , 5 years ago
qsvenc.c submit_frame specifically has some logic to make a copy of the input frame if the incoming alignment is not adequate. This checks for a height alignment of at least 32, among other things.
I'm not sure how an unaligned frame would enter there, unless one of the checks is incomplete somehow.
comment:4 by , 5 years ago
This is because input frames are submitted via another code path, where submit_frame() is absent.
#0 qsv_transfer_data_to (ctx=0x1f5a000, dst=0x1fb3140, src=<optimized out>) at libavutil/hwcontext_qsv.c:926 #1 0x0000000000fb042f in av_hwframe_transfer_data (dst=0x1fb3140, src=0x1fb2700, flags=flags@entry=0) at libavutil/hwcontext.c:456 #2 0x0000000000530538 in hwupload_filter_frame (link=link@entry=0x1f558c0, input=<optimized out>) at libavfilter/vf_hwupload.c:175 #3 0x00000000004be93a in ff_filter_frame_framed (frame=<optimized out>, link=0x1f558c0) at libavfilter/avfilter.c:1071 #4 ff_filter_frame_to_filter (link=0x1f558c0) at libavfilter/avfilter.c:1219 #5 ff_filter_activate_default (filter=<optimized out>) at libavfilter/avfilter.c:1268 #6 ff_filter_activate (filter=<optimized out>) at libavfilter/avfilter.c:1429 #7 0x00000000004c20dc in ff_filter_graph_run_once (graph=graph@entry=0x1f53b80) at libavfilter/avfiltergraph.c:1454 #8 0x00000000004c314c in push_frame (graph=0x1f53b80) at libavfilter/buffersrc.c:187 #9 av_buffersrc_add_frame_internal (ctx=ctx@entry=0x1f55d00, frame=frame@entry=0x1f53540, flags=flags@entry=4) at libavfilter/buffersrc.c:261 #10 0x00000000004c37fd in av_buffersrc_add_frame_flags (ctx=0x1f55d00, frame=frame@entry=0x1f53540, flags=flags@entry=4) at libavfilter/buffersrc.c:170 #11 0x0000000000498451 in ifilter_send_frame (frame=0x1f53540, ifilter=0x1f52100) at fftools/ffmpeg.c:2196 #12 send_frame_to_filters (ist=ist@entry=0x1f4eac0, decoded_frame=decoded_frame@entry=0x1f53540) at fftools/ffmpeg.c:2270 #13 0x0000000000498bfe in decode_video (ist=ist@entry=0x1f4eac0, pkt=pkt@entry=0x7fffffffdb90, got_output=<optimized out>, duration_pts=<optimized out>, eof=<optimized out>, decode_failed=<optimized out>) at fftools/ffmpeg.c:2469 #14 0x000000000049a97b in process_input_packet (ist=0x1f4eac0, pkt=0x7fffffffdfb0, no_eof=0) at fftools/ffmpeg.c:2623 #15 0x000000000049c567 in process_input (file_index=<optimized out>) at fftools/ffmpeg.c:4516 #16 transcode_step () at fftools/ffmpeg.c:4636 #17 transcode () at fftools/ffmpeg.c:4690 #18 0x000000000047a7b3 in main (argc=<optimized out>, argv=0x7fffffffe588) at fftools/ffmpeg.c:4892
comment:5 by , 5 years ago
Thanks Alexandr.
FFmpeg allocates system memory as the original size(width, height) in try_decode_frame, and calls
if (avci->buffer_frame->buf[0]) {
av_frame_move_ref(frame, avci->buffer_frame);
}
in avcodec_receive_frame() to move the buffer_frame to use in hardware decode.
I think this was caused by the improper alignment check in qsv_transfer_data_to().
This patch is trying to fix this:
https://patchwork.ffmpeg.org/patch/12684/
But the psnr issue still exists after applying the patch according to CI report.
https://github.com/intel-media-ci/ffmpeg/pull/35
comment:6 by , 5 years ago
The patch looks right to me, and the issue with bit-to-bit mismatch that I see locally is resolved.
As for the remaining test failures, it might be something different.
comment:8 by , 2 years ago
Note:
See TracTickets
for help on using tickets.
I'm not sure I fullly understood the root cause you've filed, but I tried the provided command line, and dumped the allocated memory in iHD driver:
The result shows that the allocated surface is enough(192 x 208, which means height is the aligned value).