videotoolbox fails to decode certain h264 videos

[der_richter]

some h264 videos fail with videotoolbox on certain hardware. the error code returned by ​VTDecompressionSessionCreate is -12913 (kVTVideoDecoderNotAvailableNowErr). it works on my Mid2010 MacBook Pro but not on my Hackintosh with an i7-6700k. this was first reported in the mpv issue tracker ​here. also tested the same file with VLC on my hackintosh and there the hw accelerated decoding works.

a sample file can be found ​here.

How to reproduce:

akemi$ ffmpeg -hwaccel videotoolbox -i /Volumes/AkemiData/Downloads/sSKl.mp4 -f null -
ffmpeg version git-2018-11-12-ae4323548a Copyright (c) 2000-2018 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-ae43235 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --enable-avresample --cc=/usr/bin/clang --host-cflags='-Os -w -pipe -march=native -mmacosx-version-min=10.14 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I/System/Library/Frameworks/JavaVM.framework/Versions/Current/Headers/' --host-ldflags='-L/usr/local/opt/openssl/lib -L/usr/local/opt/libxml2/lib -L/usr/local/lib -F/usr/local/Frameworks -Wl,-headerpad_max_install_names -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk' --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libgsm --enable-librsvg --enable-libsoxr --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libxml2 --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --disable-decoder=jpeg2000 --extra-cflags=-I/usr/local/Cellar/openjpeg/2.3.0/include/openjpeg-2.3 --enable-openssl --enable-libopus --enable-librtmp --enable-librubberband --enable-ffplay --enable-libsnappy --enable-libspeex --enable-libtesseract --enable-libtheora --enable-libwavpack --enable-libwebp --enable-libx265 --enable-lzma --enable-libzmq --enable-libzimg --enable-videotoolbox --enable-opencl
  libavutil      56. 23.101 / 56. 23.101
  libavcodec     58. 39.100 / 58. 39.100
  libavformat    58. 22.100 / 58. 22.100
  libavdevice    58.  6.100 / 58.  6.100
  libavfilter     7. 43.100 /  7. 43.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Volumes/AkemiData/Downloads/sSKl.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.71.100
  Duration: 00:00:16.72, start: 0.013000, bitrate: 3455 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 3258 kb/s, 29.97 fps, 29.97 tbr, 29971 tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(unk): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 188 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
  Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help

Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[h264 @ 0x7fb24b001200] decode_slice_header error
[h264 @ 0x7fb24b001200] no frame!

...

Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[h264 @ 0x7fb24883ae00] decode_slice_header error
[h264 @ 0x7fb24883ae00] no frame!
Error while decoding stream #0:0: Invalid data found when processing input

Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[h264 @ 0x7fb24b001200] decode_slice_header error
[h264 @ 0x7fb24b001200] no frame!
Error while decoding stream #0:0: Invalid data found when processing input

Too many packets buffered for output stream 0:1.

Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[h264 @ 0x7fb248822600] decode_slice_header error
[h264 @ 0x7fb248822600] no frame!
Conversion failed!