H264 10 bit 422 .MOV hardware decode not working on Apple M1 Pro

[andrewke]

Summary of the bug:
I am trying to play a H264 10 bit 422 .MOV from the Lumix S5ii camera using hardware decoding on Apple M1 Pro chip.
ffmpeg hardware decode does not work and uses software decode fallback.
How to reproduce:

%ffmpeg -hwaccel videotoolbox -i P1109555.MOV -f null -

The terminal output repeatedly shows

[h264 @ 0x7fa37400cb40] vt decoder cb: output image buffer is null: -12909
[h264 @ 0x7fa374048c80] hardware accelerator failed to decode picture
[vist#0:0/h264 @ 0x7fa374007e00] [dec:h264 @ 0x7fa37400b0c0] Error submitting packet to decoder: Unknown error occurred

When I played the file directly in Quicktime, hardware decoding seems to be working (7% cpu usage). When using mpv with ffmpeg, hardware decode doesnt work and cpu usage is around 40%.

However, using Movist Pro 2.11.4 (211) based on ffmpeg, hardware decode seems to work as cpu usage is 7%. Movist Pro's website ​https://movistprime.com/faqs.html says that their "Hardware (Force)" option uses special handling to get hardware decode working in special cases.

The sample video is available here ​https://drive.google.com/file/d/1L8mlk10chMrq7TSvxVoAT048TTHqvF8x/view

ffmpeg version N-117234-g8a951ef5e1-tessus
built on 2024-09-28