Opened 15 months ago

Last modified 14 months ago

#5272 open defect

videotoolbox fails to initialize for mpeg2, mpeg4 codec

Reported by: lucaswang Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: videotoolbox regression
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
ffmpeg 2.8 branch can decode mpeg2video, mpeg4 and h264, but 3.0 and master can only decode h264. videotoolbox_default_init returns AVERROR_UNKNOWN for mpeg2video and mpeg4.
How to reproduce:

% ffmpeg version N-76033-g1a29804 Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 7.0.2 (clang-700.1.81)
  configuration: --extra-version=QtAV --disable-doc --disable-debug --disable-shared --enable-pic --enable-runtime-cpudetect --enable-hwaccels --enable-avresample --disable-postproc --enable-videotoolbox --cc=clang --extra-cflags='-mmacosx-version-min=10.8' --extra-ldflags='-mmacosx-version-min=10.8'
  libavutil      55. 19.100 / 55. 19.100
  libavcodec     57. 25.101 / 57. 25.101
  libavformat    57. 26.100 / 57. 26.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 36.100 /  6. 36.100
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
Input #0, mpeg, from '/Volumes/Share/samples/big_buck_bunny_480p_MPEG2_MP2_25fps_1800K.MPG':
  Duration: 00:01:14.68, start: 0.240000, bitrate: 2123 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 854x480 [SAR 1:1 DAR 427:240], max. 2496 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x1c0]: Audio: mp2, 44100 Hz, stereo, s16p, 160 kb/s
File '/tmp/t.ts' already exists. Overwrite ? [y/N] Output #0, mpegts, to '/tmp/t.ts':
  Metadata:
    encoder         : Lavf57.26.100
    Stream #0:0: Video: mpeg2video (Main), yuv420p, 854x480 [SAR 1:1 DAR 427:240], q=2-31, 200 kb/s, 25 fps, 90k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.25.101 mpeg2video
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: mp2, 44100 Hz, stereo, s16, 384 kb/s
    Metadata:
      encoder         : Lavc57.25.101 mp2
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> mpeg2video (native))
  Stream #0:1 -> #0:1 (mp2 (native) -> mp2 (native))
Press [q] to stop, [?] for help
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fa632804000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fa632804000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fa632804000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fa632804000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.

Attachments (1)

test.mpg (2.3 MB) - added by ponpon 14 months ago.

Change History (11)

comment:1 Changed 15 months ago by cehoyos

  • Keywords regression added
  • Priority changed from normal to important

Please run git bisect to find the change that introduced the regression.

comment:2 Changed 14 months ago by ponpon

i searched but didn't find the problem commit because i don't know how to use git bisect.
i examined each commit by getting zip from https://github.com/FFmpeg/FFmpeg.
but the commits of 30/9/2015 and 1/10/2015 are not ffmpeg but avconv.

found result
good
FFmpeg-fc97b1f091ff38210699b2e5819c94e4f4ebd1fa in 29/9/2015
bad
FFmpeg-fc97b1f091ff38210699b2e5819c94e4f4ebd1fa in 2/10/2015

i have been tired.
i leave the job up to ffmpeg team.

the result of FFmpeg-fc97b1f091ff38210699b2e5819c94e4f4ebd1fa in 29/9/2015

ffmpeg -hwaccel videotoolbox -i "test.mpg" -f null - 
ffmpeg version 2.8.git Copyright (c) 2000-2015 the FFmpeg developers
  built with clang version 3.7.0 (tags/RELEASE_370/final)
  configuration: --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 --disable-ffplay --disable-ffprobe --disable-ffserver --disable-doc
  libavutil      55.  2.100 / 55.  2.100
  libavcodec     57.  3.100 / 57.  3.100
  libavformat    57.  2.100 / 57.  2.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6.  9.100 /  6.  9.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.100 /  2.  0.100
Input #0, mpeg, from '/Users/m/Downloads/2016-02-26/test.mpg':
  Duration: 00:00:10.00, start: 0.516667, bitrate: 1923 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 704x576 [SAR 12:11 DAR 4:3], max. 104857 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.2.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc57.3.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Input stream #0:0 frame changed from size:704x576 fmt:yuv420p to size:704x576 fmt:nv12
[null @ 0x7fd07980f000] Encoder did not produce proper pts, making some up.
frame=  600 fps=206 q=-0.0 Lsize=N/A time=00:00:10.00 bitrate=N/A    
video:56kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

the result of FFmpeg-fc97b1f091ff38210699b2e5819c94e4f4ebd1fa in 2/10/2015

ffmpeg -hwaccel videotoolbox -i "test.mpg" -f null - 
ffmpeg version 2.8.git Copyright (c) 2000-2015 the FFmpeg developers
  built with clang version 3.7.0 (tags/RELEASE_370/final)
  configuration: --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 --disable-ffplay --disable-ffprobe --disable-ffserver --disable-doc
  libavutil      55.  2.100 / 55.  2.100
  libavcodec     57.  3.100 / 57.  3.100
  libavformat    57.  2.100 / 57.  2.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6.  9.100 /  6.  9.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.100 /  2.  0.100
Input #0, mpeg, from '/Users/m/Downloads/2016-02-26/test.mpg':
  Duration: 00:00:10.00, start: 0.516667, bitrate: 1923 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 704x576 [SAR 12:11 DAR 4:3], max. 104857 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.2.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc57.3.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7f874b80c000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7f874b80c000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7f874b80c000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7f874b80c000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted

...

Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7f874b80c000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7f874b80c000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7f874b80c000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7f874b80c000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Conversion failed!
Last edited 14 months ago by cehoyos (previous) (diff)

Changed 14 months ago by ponpon

comment:3 follow-up: Changed 14 months ago by ponpon

i have found the commit.
there is a change from kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder to kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder to use not the decoder of videotoolbox but that of ffmpeg in ID: 78cc19f15ead41909ddaf85cd5bad10036a93883.
in the last-minute commit ID: 98ed0716fb79004f97a8348b29557c25c21afa32, the above problem don't arise.

i request revert back to previous state because videotoolbox decode has the option of quality of service, as shown by

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/VideoToolbox.framework/Versions/A/Headers/VTDecompressionProperties.h

e.g. deinterlace mode, threads, resize, etc. deinterlace of videotoolbox is excellent from playback with quicktime player or

ffmpeg  -hwaccel videotoolbox -i input

note it seem deinterlace of videotoolbox with the present fmpeg doesn't function when scan type is mbaff.
it's a shame to discard these options and potential. i think those who want or need to use the decoder of ffmpeg don't use -hwaccel videotoolbox.

i request ffmpeg can access these options when decoding with videotoolbox.
should i make a new ticket?

mpeg2 decode result

ffmpeg -hwaccel videotoolbox -i "test.mpg" -f null - 
ffmpeg version N-75644-g98ed071 Copyright (c) 2000-2015 the FFmpeg developers
  built with clang version 3.7.0 (tags/RELEASE_370/final)
  configuration: --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 --disable-ffplay --disable-ffprobe --disable-ffserver --disable-doc
  libavutil      55.  2.100 / 55.  2.100
  libavcodec     57.  3.100 / 57.  3.100
  libavformat    57.  2.100 / 57.  2.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6.  9.100 /  6.  9.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.100 /  2.  0.100
Input #0, mpeg, from '/Users/m/Downloads/2016-02-26/test.mpg':
  Duration: 00:00:10.00, start: 0.516667, bitrate: 1923 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 704x576 [SAR 12:11 DAR 4:3], max. 104857 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.2.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc57.3.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Input stream #0:0 frame changed from size:704x576 fmt:yuv420p to size:704x576 fmt:nv12
[null @ 0x7faf2080e600] Encoder did not produce proper pts, making some up.
frame=  600 fps=234 q=-0.0 Lsize=N/A time=00:00:10.00 bitrate=N/A    
video:56kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

ffmpeg -hwaccel videotoolbox -i "test.mpg" -f null - 
ffmpeg version N-75645-g78cc19f Copyright (c) 2000-2015 the FFmpeg developers
  built with clang version 3.7.0 (tags/RELEASE_370/final)
  configuration: --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 --disable-ffplay --disable-ffprobe --disable-ffserver --disable-doc
  libavutil      55.  2.100 / 55.  2.100
  libavcodec     57.  3.100 / 57.  3.100
  libavformat    57.  2.100 / 57.  2.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6.  9.100 /  6.  9.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.100 /  2.  0.100
Input #0, mpeg, from '/Users/m/Downloads/2016-02-26/test.mpg':
  Duration: 00:00:10.00, start: 0.516667, bitrate: 1923 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 704x576 [SAR 12:11 DAR 4:3], max. 104857 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.2.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc57.3.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fdbaa800a00] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fdbaa800a00] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fdbaa800a00] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fdbaa800a00] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.

...

videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fdbaa800a00] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fdbaa800a00] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fdbaa800a00] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fdbaa800a00] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Conversion failed!
Last edited 14 months ago by cehoyos (previous) (diff)

comment:4 in reply to: ↑ 3 ; follow-up: Changed 14 months ago by cehoyos

  • Status changed from new to open

Replying to ponpon:

i have found the commit.
there is a change from kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder to kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder to use not the decoder of videotoolbox but that of ffmpeg in ID: 78cc19f15ead41909ddaf85cd5bad10036a93883.

This seems like a (very) useful commit to me: Why do you believe it should be reverted?

comment:5 Changed 14 months ago by gjdfgh

Note: the videotoolbox decoder should not be a hwaccel. It should be made its own separate decoder, and then it would make sense to make hardware decoding optional too.

comment:6 in reply to: ↑ 4 ; follow-up: Changed 14 months ago by ponpon

Replying to cehoyos:
moreover, videotoolbox decode has the option of quality of service as the above.
there is a commit of such a function in vlc player.
[vlc-commits] videotoolbox: add temporal deinterlacer option
https://mailman.videolan.org/pipermail/vlc-commits/2015-August/031929.html
as above, it's a shame to discard these options and potential.

comment:7 in reply to: ↑ 6 Changed 14 months ago by cehoyos

Replying to ponpon:

Replying to cehoyos:
moreover, videotoolbox decode has the option of quality of service as the above.

This sounds completely unrelated, or what do I miss?

comment:8 Changed 14 months ago by ponpon

if not reverted, those functions is waste its abilities in other codecs, mpeg2, mpeg4, h263.
videotoolbox switchs between software and hardware decoding automatically if kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder is used.
i think a new implement of videotoolbox software decoding for those functions is a waste.
do i misunderstand videotoolbox decoding?

should i make a new ticket for those functions?

comment:9 Changed 14 months ago by ponpon

i don't understand the below. please tell me.

question1
though videotoolbox_mpeg1, videotoolbox_mpeg2, videotoolbox_mpeg4, videotoolbox_h263 are displayed in the section of hwaccel when configuring ffmpeg, does videotoolbox hardware decoding support for mpeg1, mpeg2, mpeg4, h263.at all?

as long as reading
https://software.intel.com/en-us/articles/2nd-gen-hd-graphics-quick-reference-guide/
https://software.intel.com/en-us/articles/3rd-gen-hd-graphics-quick-reference-guide
https://software.intel.com/en-us/articles/quick-reference-guide-to-intel-processor-graphics/
mpeg1, mpeg4, h263 aren't suppoted.

question2
is the auto switch introduced into the initial implement of videotoolbox on ffmpeg for the further extensions?

question3
what is the advantage of commit ID: 78cc19f15ead41909ddaf85cd5bad10036a93883 discarding the possibility of the further extensions?
as mentioned above, i don't think those who want or need to use the decoder of ffmpeg use -hwaccel videotoolbox.

comment:10 Changed 14 months ago by ponpon

measure to avoid this error

this error arises from two changes.
first: from EnableHardwareAcceleratedVideoDecoder? to RequireHardwareAcceleratedVideoDecoder? of videotoolbox.c in ID: 78cc19f15ead41909ddaf85cd5bad10036a93883.
second: from videotoolbox_extralibs="-framework CoreFoundation? -framework VideoToolbox? -framework CoreMedia? -framework QuartzCore? -framework CoreVideo?" to videotoolbox_hwaccel_extralibs="-framework QuartzCore?" of configure in ID: 3af71ac3f9f635856145503f2ace829aad813636

execute the following. but probably, videotoolbox software decoding is used.

sed -i -e 's/RequireHardwareAcceleratedVideoDecoder/EnableHardwareAcceleratedVideoDecoder/g' "./libavcodec/videotoolbox.c"
sed -i -e 's/videotoolbox_hwaccel_extralibs=\"-framework QuartzCore\"/videotoolbox_extralibs=\"-framework CoreFoundation -framework VideoToolbox -framework CoreMedia -framework QuartzCore -framework CoreVideo\"/g' "./configure"
Last edited 14 months ago by ponpon (previous) (diff)
Note: See TracTickets for help on using tickets.