Opened 2 years ago

Last modified 2 years ago

#5272 open defect

videotoolbox fails to initialize for mpeg2, mpeg4 codec

Reported by: lucaswang Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: videotoolbox regression
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
ffmpeg 2.8 branch can decode mpeg2video, mpeg4 and h264, but 3.0 and master can only decode h264. videotoolbox_default_init returns AVERROR_UNKNOWN for mpeg2video and mpeg4.
How to reproduce:

% ffmpeg version N-76033-g1a29804 Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 7.0.2 (clang-700.1.81)
  configuration: --extra-version=QtAV --disable-doc --disable-debug --disable-shared --enable-pic --enable-runtime-cpudetect --enable-hwaccels --enable-avresample --disable-postproc --enable-videotoolbox --cc=clang --extra-cflags='-mmacosx-version-min=10.8' --extra-ldflags='-mmacosx-version-min=10.8'
  libavutil      55. 19.100 / 55. 19.100
  libavcodec     57. 25.101 / 57. 25.101
  libavformat    57. 26.100 / 57. 26.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 36.100 /  6. 36.100
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
Input #0, mpeg, from '/Volumes/Share/samples/big_buck_bunny_480p_MPEG2_MP2_25fps_1800K.MPG':
  Duration: 00:01:14.68, start: 0.240000, bitrate: 2123 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 854x480 [SAR 1:1 DAR 427:240], max. 2496 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x1c0]: Audio: mp2, 44100 Hz, stereo, s16p, 160 kb/s
File '/tmp/t.ts' already exists. Overwrite ? [y/N] Output #0, mpegts, to '/tmp/t.ts':
  Metadata:
    encoder         : Lavf57.26.100
    Stream #0:0: Video: mpeg2video (Main), yuv420p, 854x480 [SAR 1:1 DAR 427:240], q=2-31, 200 kb/s, 25 fps, 90k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.25.101 mpeg2video
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: mp2, 44100 Hz, stereo, s16, 384 kb/s
    Metadata:
      encoder         : Lavc57.25.101 mp2
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> mpeg2video (native))
  Stream #0:1 -> #0:1 (mp2 (native) -> mp2 (native))
Press [q] to stop, [?] for help
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fa632804000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fa632804000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fa632804000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fa632804000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.

Attachments (1)

test.mpg (2.3 MB) - added by ponpon 2 years ago.

Change History (11)

comment:1 Changed 2 years ago by cehoyos

  • Keywords regression added
  • Priority changed from normal to important

Please run git bisect to find the change that introduced the regression.

comment:2 Changed 2 years ago by ponpon

i searched but didn't find the problem commit because i don't know how to use git bisect.
i examined each commit by getting zip from https://github.com/FFmpeg/FFmpeg.
but the commits of 30/9/2015 and 1/10/2015 are not ffmpeg but avconv.

found result
good
FFmpeg-fc97b1f091ff38210699b2e5819c94e4f4ebd1fa in 29/9/2015
bad
FFmpeg-fc97b1f091ff38210699b2e5819c94e4f4ebd1fa in 2/10/2015

i have been tired.
i leave the job up to ffmpeg team.

the result of FFmpeg-fc97b1f091ff38210699b2e5819c94e4f4ebd1fa in 29/9/2015

ffmpeg -hwaccel videotoolbox -i "test.mpg" -f null - 
ffmpeg version 2.8.git Copyright (c) 2000-2015 the FFmpeg developers
  built with clang version 3.7.0 (tags/RELEASE_370/final)
  configuration: --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 --disable-ffplay --disable-ffprobe --disable-ffserver --disable-doc
  libavutil      55.  2.100 / 55.  2.100
  libavcodec     57.  3.100 / 57.  3.100
  libavformat    57.  2.100 / 57.  2.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6.  9.100 /  6.  9.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.100 /  2.  0.100
Input #0, mpeg, from '/Users/m/Downloads/2016-02-26/test.mpg':
  Duration: 00:00:10.00, start: 0.516667, bitrate: 1923 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 704x576 [SAR 12:11 DAR 4:3], max. 104857 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.2.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc57.3.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Input stream #0:0 frame changed from size:704x576 fmt:yuv420p to size:704x576 fmt:nv12
[null @ 0x7fd07980f000] Encoder did not produce proper pts, making some up.
frame=  600 fps=206 q=-0.0 Lsize=N/A time=00:00:10.00 bitrate=N/A    
video:56kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

the result of FFmpeg-fc97b1f091ff38210699b2e5819c94e4f4ebd1fa in 2/10/2015

ffmpeg -hwaccel videotoolbox -i "test.mpg" -f null - 
ffmpeg version 2.8.git Copyright (c) 2000-2015 the FFmpeg developers
  built with clang version 3.7.0 (tags/RELEASE_370/final)
  configuration: --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 --disable-ffplay --disable-ffprobe --disable-ffserver --disable-doc
  libavutil      55.  2.100 / 55.  2.100
  libavcodec     57.  3.100 / 57.  3.100
  libavformat    57.  2.100 / 57.  2.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6.  9.100 /  6.  9.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.100 /  2.  0.100
Input #0, mpeg, from '/Users/m/Downloads/2016-02-26/test.mpg':
  Duration: 00:00:10.00, start: 0.516667, bitrate: 1923 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 704x576 [SAR 12:11 DAR 4:3], max. 104857 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.2.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc57.3.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7f874b80c000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7f874b80c000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7f874b80c000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7f874b80c000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted

...

Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7f874b80c000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7f874b80c000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7f874b80c000] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7f874b80c000] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Conversion failed!
Last edited 2 years ago by cehoyos (previous) (diff)

Changed 2 years ago by ponpon

comment:3 follow-up: Changed 2 years ago by ponpon

i have found the commit.
there is a change from kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder to kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder to use not the decoder of videotoolbox but that of ffmpeg in ID: 78cc19f15ead41909ddaf85cd5bad10036a93883.
in the last-minute commit ID: 98ed0716fb79004f97a8348b29557c25c21afa32, the above problem don't arise.

i request revert back to previous state because videotoolbox decode has the option of quality of service, as shown by

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/System/Library/Frameworks/VideoToolbox.framework/Versions/A/Headers/VTDecompressionProperties.h

e.g. deinterlace mode, threads, resize, etc. deinterlace of videotoolbox is excellent from playback with quicktime player or

ffmpeg  -hwaccel videotoolbox -i input

note it seem deinterlace of videotoolbox with the present fmpeg doesn't function when scan type is mbaff.
it's a shame to discard these options and potential. i think those who want or need to use the decoder of ffmpeg don't use -hwaccel videotoolbox.

i request ffmpeg can access these options when decoding with videotoolbox.
should i make a new ticket?

mpeg2 decode result

ffmpeg -hwaccel videotoolbox -i "test.mpg" -f null - 
ffmpeg version N-75644-g98ed071 Copyright (c) 2000-2015 the FFmpeg developers
  built with clang version 3.7.0 (tags/RELEASE_370/final)
  configuration: --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 --disable-ffplay --disable-ffprobe --disable-ffserver --disable-doc
  libavutil      55.  2.100 / 55.  2.100
  libavcodec     57.  3.100 / 57.  3.100
  libavformat    57.  2.100 / 57.  2.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6.  9.100 /  6.  9.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.100 /  2.  0.100
Input #0, mpeg, from '/Users/m/Downloads/2016-02-26/test.mpg':
  Duration: 00:00:10.00, start: 0.516667, bitrate: 1923 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 704x576 [SAR 12:11 DAR 4:3], max. 104857 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.2.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc57.3.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Input stream #0:0 frame changed from size:704x576 fmt:yuv420p to size:704x576 fmt:nv12
[null @ 0x7faf2080e600] Encoder did not produce proper pts, making some up.
frame=  600 fps=234 q=-0.0 Lsize=N/A time=00:00:10.00 bitrate=N/A    
video:56kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

ffmpeg -hwaccel videotoolbox -i "test.mpg" -f null - 
ffmpeg version N-75645-g78cc19f Copyright (c) 2000-2015 the FFmpeg developers
  built with clang version 3.7.0 (tags/RELEASE_370/final)
  configuration: --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 --disable-ffplay --disable-ffprobe --disable-ffserver --disable-doc
  libavutil      55.  2.100 / 55.  2.100
  libavcodec     57.  3.100 / 57.  3.100
  libavformat    57.  2.100 / 57.  2.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6.  9.100 /  6.  9.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.100 /  2.  0.100
Input #0, mpeg, from '/Users/m/Downloads/2016-02-26/test.mpg':
  Duration: 00:00:10.00, start: 0.516667, bitrate: 1923 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 704x576 [SAR 12:11 DAR 4:3], max. 104857 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf57.2.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc57.3.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fdbaa800a00] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fdbaa800a00] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fdbaa800a00] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fdbaa800a00] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.

...

videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fdbaa800a00] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fdbaa800a00] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
Error creating Videotoolbox decoder.
videotoolbox hwaccel requested for input stream #0:0, but cannot be initialized.
[mpeg2video @ 0x7fdbaa800a00] decoding to AV_PIX_FMT_NONE is not supported.
[mpeg2video @ 0x7fdbaa800a00] mpeg_decode_postinit() failure
Error while decoding stream #0:0: Operation not permitted
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Conversion failed!
Last edited 2 years ago by cehoyos (previous) (diff)

comment:4 in reply to: ↑ 3 ; follow-up: Changed 2 years ago by cehoyos

  • Status changed from new to open

Replying to ponpon:

i have found the commit.
there is a change from kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder to kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder to use not the decoder of videotoolbox but that of ffmpeg in ID: 78cc19f15ead41909ddaf85cd5bad10036a93883.

This seems like a (very) useful commit to me: Why do you believe it should be reverted?

comment:5 Changed 2 years ago by gjdfgh

Note: the videotoolbox decoder should not be a hwaccel. It should be made its own separate decoder, and then it would make sense to make hardware decoding optional too.

comment:6 in reply to: ↑ 4 ; follow-up: Changed 2 years ago by ponpon

Replying to cehoyos:
moreover, videotoolbox decode has the option of quality of service as the above.
there is a commit of such a function in vlc player.
[vlc-commits] videotoolbox: add temporal deinterlacer option
https://mailman.videolan.org/pipermail/vlc-commits/2015-August/031929.html
as above, it's a shame to discard these options and potential.

comment:7 in reply to: ↑ 6 Changed 2 years ago by cehoyos

Replying to ponpon:

Replying to cehoyos:
moreover, videotoolbox decode has the option of quality of service as the above.

This sounds completely unrelated, or what do I miss?

comment:8 Changed 2 years ago by ponpon

if not reverted, those functions is waste its abilities in other codecs, mpeg2, mpeg4, h263.
videotoolbox switchs between software and hardware decoding automatically if kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder is used.
i think a new implement of videotoolbox software decoding for those functions is a waste.
do i misunderstand videotoolbox decoding?

should i make a new ticket for those functions?

comment:9 Changed 2 years ago by ponpon

i don't understand the below. please tell me.

question1
though videotoolbox_mpeg1, videotoolbox_mpeg2, videotoolbox_mpeg4, videotoolbox_h263 are displayed in the section of hwaccel when configuring ffmpeg, does videotoolbox hardware decoding support for mpeg1, mpeg2, mpeg4, h263.at all?

as long as reading
https://software.intel.com/en-us/articles/2nd-gen-hd-graphics-quick-reference-guide/
https://software.intel.com/en-us/articles/3rd-gen-hd-graphics-quick-reference-guide
https://software.intel.com/en-us/articles/quick-reference-guide-to-intel-processor-graphics/
mpeg1, mpeg4, h263 aren't suppoted.

question2
is the auto switch introduced into the initial implement of videotoolbox on ffmpeg for the further extensions?

question3
what is the advantage of commit ID: 78cc19f15ead41909ddaf85cd5bad10036a93883 discarding the possibility of the further extensions?
as mentioned above, i don't think those who want or need to use the decoder of ffmpeg use -hwaccel videotoolbox.

comment:10 Changed 2 years ago by ponpon

measure to avoid this error

this error arises from two changes.
first: from EnableHardwareAcceleratedVideoDecoder? to RequireHardwareAcceleratedVideoDecoder? of videotoolbox.c in ID: 78cc19f15ead41909ddaf85cd5bad10036a93883.
second: from videotoolbox_extralibs="-framework CoreFoundation? -framework VideoToolbox? -framework CoreMedia? -framework QuartzCore? -framework CoreVideo?" to videotoolbox_hwaccel_extralibs="-framework QuartzCore?" of configure in ID: 3af71ac3f9f635856145503f2ace829aad813636

execute the following. but probably, videotoolbox software decoding is used.

sed -i -e 's/RequireHardwareAcceleratedVideoDecoder/EnableHardwareAcceleratedVideoDecoder/g' "./libavcodec/videotoolbox.c"
sed -i -e 's/videotoolbox_hwaccel_extralibs=\"-framework QuartzCore\"/videotoolbox_extralibs=\"-framework CoreFoundation -framework VideoToolbox -framework CoreMedia -framework QuartzCore -framework CoreVideo\"/g' "./configure"
Last edited 2 years ago by ponpon (previous) (diff)
Note: See TracTickets for help on using tickets.