mplayer crashed with vda h264 decoder

[niltsh]

Hi although this issue is reproduced with mplayer, but in the log I think it should be related with ffmpeg too.
Please help to take a look.

HW late 2011Macbook Pro intel core 7
Sw 10.10 latest ffmpeg and latest mplayer

When compling mplayer , i used ffmpeg src dirctly and used --enable-decoder=vda for mplayer's configure.

Compling is fine, but when try to opeb any h264 content with ffh264vda decoder, the following error came out.

[h264_vda @ 0x10136b590]Could not find an AVHWAccel for the pixel format: vda_vldAssertion choices[n] != AV_PIX_FMT_NONE failed at libavcodec/utils.c:1232

MPlayer interrupted by signal 6 in module: decode video
ID_SIGNAL=6

• MPlayer crashed.

I am not sure whether it is mplayer's issue or ffmpeg', but from the log it looks like inside of ffmpeg, or it is related with mplayer's configure.

Would you please help to take a look on this? Thank you.

[gjdfgh]

Works fine in mpv.

[reimar]

I think this is clearly a FFmpeg issue, in so far as applications should not be able to trigger an assert.
However most likely this is caused by MPlayer picking a pixel format VDA does not like.
Please provide a full stack trace from a debug build (since I think the developers with access to a Mac with VDA support are a bit rare).

[reimar]

Oh, sorry. I think the error is actually before the assert (I misread the output a bit):
Could not find an AVHWAccel for the pixel format: vda_vld
That is a mistake in MPlayer's configure, however I think there is a mistake in FFmpeg's as well which would cause the same behaviour:
h264_vda_decoder_deps="vda"
h264_vda_decoder_select="h264_decoder"
This is not correct, the h264_vda_decoder does not only require VDA, but it also requires h264_vda_hwaccel to be enabled, otherwise this seems to happen.
I.e. I expect this to be reproducible in FFmpeg/ffplay if --disable-hwaccel=vda is used.

[niltsh]

Hi Rimar

Thank you for the reply.

Let me share the call stack, I also tried as far as I could, but clearly this part is beyond my knowledge.

1. the original crash callstack is following,

============================================

frame #0: 0x00007fff92da2282 libsystem_kernel.dylib`pthread_kill + 10
frame #1: 0x00007fff92d874c3 libsystem_pthread.dylib`pthread_kill + 90
frame #2: 0x00007fff8a870b73 libsystem_c.dylib`abort + 129
frame #3: 0x00000001006ac519 mplayer`ff_get_format(avctx=<unavailable>, fmt=<unavailable>) + 601 at utils.c:1232
frame #4: 0x00000001003b18c9 mplayer`get_pixel_format(h=<unavailable>, force_callback=<unavailable>) + 761 at h264_slice.c:1112
frame #5: 0x00000001003af679 mplayer`ff_h264_decode_slice_header(h=0x0000000103201000, h0=0x0000000103201000) + 1801 at h264_slice.c:1415
frame #6: 0x000000010036cf24 mplayer`decode_nal_units(h=0x0000000103201000, buf=0x000000010281d000, buf_size=53775, parse_extradata=0) + 3076 at h264.c:1592
frame #7: 0x000000010036eb3c mplayer`h264_decode_frame(avctx=0x00000001028a4400, data=0x0000000101d3f740, got_frame=0x00007fff5fbfe6cc, avpkt=<unavailable>) + 540 at h264.c:1908
frame #8: 0x00000001006e6f58 mplayer`vdadec_decode(avctx=0x00000001028a4400, data=0x0000000101d3f740, got_frame=0x00007fff5fbfe6cc, avpkt=<unavailable>) + 152 at vda_h264_dec.c:136
frame #9: 0x00000001006af3d8 mplayer`avcodec_decode_video2(avctx=0x00000001028a4400, picture=0x0000000101d3f740, got_picture_ptr=0x00007fff5fbfe6cc, avpkt=0x00007fff5fbfe668) + 392 at utils.c:2349
frame #10: 0x0000000100128759 mplayer`decode(sh=0x0000000101c4d6c0, data=<unavailable>, len=53775, flags=<unavailable>) + 665 at vd_ffmpeg.c:944
frame #11: 0x000000010004aa04 mplayer`decode_video(sh_video=0x0000000101c4d6c0, start=<unavailable>, in_size=<unavailable>, drop_frame=0, pts=0, full_frame=0x0000000000000000) + 68 at dec_video.c:398
frame #12: 0x0000000100011f88 mplayer`update_video [inlined] generate_video_frame(sh_video=0x0000000101c4d6c0, d_video=0x0000000101d00160) + 431 at mplayer.c:1815
frame #13: 0x0000000100011dd9 mplayer`update_video(blit_frame=0x00007fff5fbfe944) + 57 at mplayer.c:2527

============================================

And about the assertion, I think it is because that via_h264_dec.c, LINE 73, the decoder returns AV_PIX_FMT_VDA_VLD in get_format.
Then I tried to change the return value to AV_PIX_FMT_VDA, the assertion passed, but the crash call stack changed to the following

=======================================
thread #1: tid = 0xddd45, 0x00000001006e6fc7 mplayer`vdadec_decode(avctx=0x0000000103017800, data=0x0000000101e162e0, got_frame=<unavailable>, avpkt=<unavailable>) + 263 at vda_h264_dec.c:145, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)

frame #0: 0x00000001006e6fc7 mplayer`vdadec_decode(avctx=0x0000000103017800, data=0x0000000101e162e0, got_frame=<unavailable>, avpkt=<unavailable>) + 263 at vda_h264_dec.c:145

142
143 CVPixelBufferRetain(cv_buffer);
144 CVPixelBufferLockBaseAddress(cv_buffer, 0);

-> 145 context->cv_buffer = cv_buffer;

146 pic->format = ctx->pix_fmt;
147 if (CVPixelBufferIsPlanar(cv_buffer)) {
148 int i, count = CVPixelBufferGetPlaneCount(cv_buffer);

(lldb) bt
thread #1: tid = 0xddd45, 0x00000001006e6fc7 mplayer`vdadec_decode(avctx=0x0000000103017800, data=0x0000000101e162e0, got_frame=<unavailable>, avpkt=<unavailable>) + 263 at vda_h264_dec.c:145, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)

frame #0: 0x00000001006e6fc7 mplayer`vdadec_decode(avctx=0x0000000103017800, data=0x0000000101e162e0, got_frame=<unavailable>, avpkt=<unavailable>) + 263 at vda_h264_dec.c:145
frame #1: 0x00000001006af3d8 mplayer`avcodec_decode_video2(avctx=0x0000000103017800, picture=0x0000000101e162e0, got_picture_ptr=0x00007fff5fbfe6cc, avpkt=0x00007fff5fbfe668) + 392 at utils.c:2349
frame #2: 0x0000000100128759 mplayer`decode(sh=0x0000000101c2a820, data=<unavailable>, len=144, flags=<unavailable>) + 665 at vd_ffmpeg.c:944
frame #3: 0x000000010004aa04 mplayer`decode_video(sh_video=0x0000000101c2a820, start=<unavailable>, in_size=<unavailable>, drop_frame=0, pts=0.056677777777777778, full_frame=0x0000000000000000) + 68 at dec_video.c:398
frame #4: 0x0000000100011f88 mplayer`update_video [inlined] generate_video_frame(sh_video=0x0000000101c2a820, d_video=0x0000000101e00160) + 431 at mplayer.c:1815
frame #5: 0x0000000100011dd9 mplayer`update_video(blit_frame=0x00007fff5fbfe944) + 57 at mplayer.c:2527

=======================================

[Carl Eugen Hoyos]

Is this still reproducible with current MPlayer svn and current FFmpeg git head.

[niltsh]

Today I checked the latest ffmpeg and mplayer.

Although there are other issues, this crash has been fixed, please close.