Opened 10 years ago
Closed 9 years ago
#4143 closed enhancement (fixed)
Support Core Image of MacOSX
Reported by: | rusghon | Owned by: | Thilo Borgmann |
---|---|---|---|
Priority: | wish | Component: | avfilter |
Version: | git-master | Keywords: | osx |
Cc: | thilo.borgmann@mail.de | Blocked By: | |
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Hello,
I would like to thank the whole FFMPEG team.
I would like to support Core Image of MacOSX.
Core Image allow us to use GPU acceleration video filtering.
This can reduce burden of CPU and realize high speed encode.
In specific, CICategorySharpen, CICategoryBlur, CICategoryColorAdjustment is useful.
One can know this effect with Core Image Fun House of developer tool of MacOSX.
Sincerely thankful for your attention
Core Image Reference Collection
https://developer.apple.com/library/mac/documentation/graphicsimaging/Reference/CoreImagingRef/_index.html#//apple_ref/doc/uid/TP40001171
Core Image Programming Guide
https://developer.apple.com/library/mac/documentation/graphicsimaging/Conceptual/CoreImaging/ci_intro/ci_intro.html#//apple_ref/doc/uid/TP30001185
Core Image Filter Reference
https://developer.apple.com/library/mac/documentation/graphicsimaging/Reference/CoreImageFilterReference/index.html
Attachments (2)
Change History (20)
comment:1 by , 10 years ago
Priority: | wish → normal |
---|
follow-up: 3 comment:2 by , 10 years ago
Keywords: | osx added |
---|---|
Priority: | normal → wish |
Status: | new → open |
Version: | unspecified → git-master |
comment:3 by , 10 years ago
Also, CICategoryGeometryAdjustmen is useful.
It has Crop and LanczosScale, etc.
comment:4 by , 9 years ago
i have found Core Image Filter Plug-In for Quick Time.
the source of it is is included in CoreVF.dmg of x264Encoder.dmg.
though i dont know if it is useful for ffmoeg, i attach its url.
http://download.cnet.com/x264Encoder/3000-2170_4-185068.html
by , 9 years ago
Attachment: | x264Encoder.c added |
---|
comment:5 by , 9 years ago
Cc: | added |
---|
comment:6 by , 9 years ago
+1 for this enhancement
i am rooting for you and looking forward to your work
comment:7 by , 9 years ago
Core Image filtering patch sent to ffmpeg-devel:
http://article.gmane.org/gmane.comp.video.ffmpeg.devel/211493
MACs are rare around FFmpeg devs. So I would appreciate more testing on this patch, especially from people who want to use it once it hits FFmpeg/HEAD. Please stay with the features already provided for testing. Additional feature requests should be addressed in seperate tickets after the patch hits FFmpeg/HEAD. Bugs regarding this patch should be posted directly to ffmpeg-devel, please (I might miss them here).
Thanks!
follow-ups: 9 11 comment:8 by , 9 years ago
Owner: | set to |
---|
comment:9 by , 9 years ago
Replying to thilo.borgmann:
i report the result of build with the recent patch under osx1095 and clang37 because i am not a member of devel mailing list.
failure
./configure --prefix=${TARGET} --enable-pthreads --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 --optflags="-Ofast" --target-os=darwin --arch=x86_64 --cpu=corei7-avx && make -j 8 doc/filters.texi:4958: misplaced { doc/filters.texi:4958: misplaced } doc/filters.texi:4969: @option expected braces doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:5021: unknown command `inputCorrectionLevel' make: *** [doc/ffmpeg-filters.html] Error 1 make: *** Waiting for unfinished jobs.... doc/filters.texi:4958: misplaced { doc/filters.texi:4958: misplaced } doc/filters.texi:4969: @option expected braces doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:5021: unknown command `inputCorrectionLevel' make: *** [doc/ffplay-all.html] Error 1 doc/filters.texi:4958: misplaced { doc/filters.texi:4958: misplaced } doc/filters.texi:4969: @option expected braces doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:5021: unknown command `inputCorrectionLevel' make: *** [doc/ffprobe-all.html] Error 1 doc/filters.texi:4958: misplaced { doc/filters.texi:4958: misplaced } doc/filters.texi:4969: @option expected braces doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:5021: unknown command `inputCorrectionLevel' make: *** [doc/ffmpeg-all.html] Error 1 STRIP libavcodec/x86/vp9itxfm.o doc/filters.texi:4958: misplaced { doc/filters.texi:4958: misplaced } doc/filters.texi:4969: @option expected braces doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:5021: unknown command `inputCorrectionLevel' make: *** [doc/ffserver-all.html] Error 1
success with --disable-htmlpages
./configure --prefix=${TARGET} --enable-pthreads --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 --optflags="-Ofast" --target-os=darwin --arch=x86_64 --cpu=corei7-avx --disable-htmlpages && make -j 8
comment:10 by , 9 years ago
From a clang manpage:
-Ofast Enables all the optimizations from -O3 along with other aggressive optimizations that may violate strict compliance with language standards.
If this is true, it sounds like a very bad idea. Did you benchmark it?
--enable-pthreads --target-os=darwin --arch=x86_64
These have no effect (not even the effect you believe they have), please remove them.
follow-up: 12 comment:11 by , 9 years ago
thank you for your pointing out.
the result is the same. the filter isn't tried still.
./configure --prefix=${TARGET} --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 && make -j 8 CC libavutil/xtea.o HTML doc/ffmpeg.html HTML doc/ffplay.html HTML doc/ffprobe.html HTML doc/ffserver.html HTML doc/ffmpeg-all.html HTML doc/ffplay-all.html HTML doc/ffprobe-all.html HTML doc/ffserver-all.html HTML doc/ffmpeg-filters.html HTML doc/libswscale.html HTML doc/libswresample.html HTML doc/libavcodec.html HTML doc/libavformat.html HTML doc/libavdevice.html HTML doc/libavfilter.html HTML doc/developer.html HTML doc/faq.html HTML doc/fate.html doc/filters.texi:4958: misplaced { doc/filters.texi:4958: misplaced } doc/filters.texi:4969: @option expected braces doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:5021: unknown command `inputCorrectionLevel' make: *** [doc/ffmpeg-filters.html] Error 1 make: *** Waiting for unfinished jobs.... HTML doc/general.html doc/filters.texi:4958: misplaced { doc/filters.texi:4958: misplaced } doc/filters.texi:4969: @option expected braces doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:5021: unknown command `inputCorrectionLevel' make: *** [doc/ffplay-all.html] Error 1 doc/filters.texi:4958: misplaced { doc/filters.texi:4958: misplaced } doc/filters.texi:4969: @option expected braces doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:5021: unknown command `inputCorrectionLevel' make: *** [doc/ffprobe-all.html] Error 1 doc/filters.texi:4958: misplaced { doc/filters.texi:4958: misplaced } doc/filters.texi:4969: @option expected braces doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:5021: unknown command `inputCorrectionLevel' make: *** [doc/ffmpeg-all.html] Error 1 doc/filters.texi:4958: misplaced { doc/filters.texi:4958: misplaced } doc/filters.texi:4969: @option expected braces doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:4985: unexpected @ doc/filters.texi:5021: unknown command `inputCorrectionLevel' make: *** [doc/ffserver-all.html] Error 1
man page
man ffmpeg-filters copy Copy the input source unchanged to the output. This is mainly useful for testing purposes. at section coreimage Video filtering on GPU using Apple's CoreImage API on OSX. Hardware acceleration is based on an OpenGL context. Usually, this means it is processed by video hardware. However, software-based OpenGL implementations exist which means there is no guarantee for hardware processing. It depends on the respective OSX. There are many filters and image generators provided by Apple that come with a large variety of options. The filter has to be referenced by its name along with its options. The coreimage filter accepts the following options: at table @option at item list_filters List all available filters along with all their respective options as well as possible minimum and maximum values along with the default values. at example coreimage=list_filters=true at end example at item filter Specifiy all filters by their respective name and options. Use B<list_filters> to determine all valid filter names and options. Numerical options are specified by a float value and are automatically clamped to their respective value range. Vector and color options have to be specified by a list of space separated float values. Character escaping has to be done. A special option name C<default> is available to use default options for a filter. It is required to specify either C<default> or at least one of the filter options. All omitted options are used with their default values. The syntax of the filter string is as follows: at example filter=E<lt>NAMEE<gt>@E<lt>OPTIONE<gt>=E<lt>VALUEE<gt>[@E<lt>OPTIONE<gt>=E<lt>VALUEE<gt>][...][#E<lt>NAMEE<gt>@E<lt>OPTIONE<gt>=E<lt>VALUEE<gt>[@E<lt>OPTIONE<gt>=E<lt>VALUEE<gt>][...]] at end example at end table Several filters can be chained for successive processing without GPU-HOST transfers allowing for fast processing of complex filter chains. Currently, only filters with zero (generators) or exactly one (filters) input image and one output image are supported. Also, transition filters are not yet usable as intended. Some filters generate output images with additional padding depending on the respective filter kernel. The padding is automatically removed to ensure the filter output has the same size as the input image. For image generators, the size of the output image is determined by the given input image. The generators do not use the pixel information of the input image to generate their output. However, the generated output is blended onto the input image, resulting in partial or complete coverage of the output image. at subsection Examples at itemize at item List all filters available: at example coreimage=list_filters=true at end example at item Use the CIBoxBlur filter with default options to blur an image: at example coreimage=filter=CIBoxBlur at default at end example at item Use a filter chain with CISepiaTone at default values and CIVignetteEffect with its center at 100x100 and a radius of 50 pixels: at example coreimage=filter=CIBoxBlur at default#CIVignetteEffect at inputCenter=100\ 100 at inputRadius=50 at end example at item Use nullsrc and CIQRCodeGenerator to create a QR code for the FFmpeg homepage, given as complete and escaped command-line for Apple's standard bash shell: at example ./ffmpeg -f lavfi -i nullsrc=s=100x100,coreimage=filter=CIQRCodeGenerator at inputMessage=https\\\\\://FFmpeg.org/@inputCorrectionLevel=H -frames:v 1 QRCode.png at end example at end itemize crop Crop the input video to given dimensions.
comment:12 by , 9 years ago
Replying to ponpon:
the result is the same. the filter isn't tried still.
You can use the configure option --disable-doc
or make ffmpeg
instead of make
as workarounds.
./configure --prefix=${TARGET} --cc=clang-mp-3.7 --cxx=clang++-mp-3.7 && make -j 8
If forgot: I don't think --cxx=clang++-mp-3.7
has any effect in your configure line (but I may miss something).
follow-up: 14 comment:13 by , 9 years ago
Thanks for pointing this out! However, if you had posted a reply on ffmpeg-devel, I would have found this earlier.
I think I've just missed to properly escape @ and {} in the texinfo file.
See ffmpeg-devel for an updated patch.
comment:14 by , 9 years ago
Replying to thilo.borgmann:
i tried the new patch (Date: 2016-03-13 20:09:39 GMT).
but
patch -p1 < lavf_Add_coreimage_filter_for_GPU_based_image_filtering_on_OSX.patch patch: **** Only garbage was found in the patch input.
patch version
patch -v patch 2.5.8 Copyright (C) 1988 Larry Wall Copyright (C) 2002 Free Software Foundation, Inc. This program comes with NO WARRANTY, to the extent permitted by law. You may redistribute copies of this program under the terms of the GNU General Public License. For more information about these matters, see the file named COPYING. written by Larry Wall and Paul Eggert
in the patch, there is no @. only <at>. i understand differences are marked with @@. original diff? replace error?
follow-up: 16 comment:15 by , 9 years ago
What I see in my inbox from ffmpeg-devel it is correct. If you're copy-pasting from gmane.org or similar, you'll have @ replaced by <at>.
I'll attach it here again, you should consider subscribing to ffmpeg-devel...
by , 9 years ago
Attachment: | 0002-lavf-Add-coreimage-filter-for-GPU-based-image-filter.patch added |
---|
Preliminary patch from ffmpeg-devel
comment:16 by , 9 years ago
Replying to thilo.borgmann:
thank you for the attach.
i can build and the filter functions.
but slow and it seems the filter uses dGPU from gfxcardstatu.
is there a penalty in deta transher by using dGPU?
though graphic conveter use coreimage, it seems graphic conveter uses iGPU.
sorry. i hesitated. this is because junk mails are too many since my mail addresss is open to public inspection by work-related matters.
i am preparing to subscribe to ffmpeg-devel temporarily. i can test and root for you.
comment:17 by , 9 years ago
So I assume you use a MAC with two or more GPUs. Unfortunately I don't have such a MAC available. Without access to such a machine I can't really tell what happens in your case nor can I find out how to select a specific GPU.
If you mean that you can grant me access to such a machine by saying "I can test and root for you", then I would appreciate that very much and would try to make the GPU selctable and/or autoselected. If that's the case you can mail me directly at thilo.borgmann@mail.de.
If subscribing to ffmpeg-devel is a problem for you, you could clone my github at:
https://github.com/thiloborgmann/FFmpeg/
I try to keep the coreimage branch of my repo up-to-date with what I post on the mailing list.
p.s. just found this on developer.apple.com:
"By default, once your application creates an OpenGL context (by either calling OpenGL directly or an API that relies on OpenGL such as Core Animation, Core Image, etc), the MacBook Pro automatically switches to the higher-end discrete GPU for performance concerns and won't switch back until the application quits."
This means using the dGPU (discrete GPU) is what you would want to do anyway. However, making the GPU selectable is still desirable!
When you say slow, what exactly have you measured and compared to what exactly?
Of course, uploading before processing and downloading afterwards is a speed penalty. The bigger the filter chain on GPU the less important this penalty becomes. For very simple filtering the overall processing time on the CPU can be better.
comment:18 by , 9 years ago
Resolution: | → fixed |
---|---|
Status: | open → closed |
Supported with commit 4d251723.
Patch welcome.