Opened 5 years ago
Last modified 21 months ago
#7965 open enhancement
Support for HEVC with Alpha
Reported by: | leemartin | Owned by: | |
---|---|---|---|
Priority: | wish | Component: | avcodec |
Version: | git-master | Keywords: | hevc |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
At WWDC this year, Apple introduced support for HEVC with Alpha to both their app and browser platforms. You can find their talk on the subject here:
https://developer.apple.com/videos/play/wwdc2019/506
I'm an amateur at FFmpeg but I went down a rabbit hole today, trying to create a video in this new format using the latest version of FFmpeg. I took a rather simple approach of using ffprobe on one of their example video files (found in the resources on the link above) to get a profile and attempting to tune an ffmpeg encoding command to fit the same profile.
While the following command does recreate the profile, it does not retain the transparency of the source prores .mov file.
% ffmpeg -i puppets_with_alpha_prores.mov -c:v libx265 -tag:v hvc1 -pix_fmt yuv420p -colorspace bt709 -color_primaries bt709 -color_trc bt709 puppets_with_alpha_hevc.mov
Apple is pretty clear about how they are achieving this encoding in their talk and examples. Naturally, I didn't follow anything they said and simply tried to hack a solution.
So I'm wondering if this is something FFmpeg can do or would be able to support in the future?
Thanks for the help!
Attachments (2)
Change History (15)
comment:1 by , 5 years ago
Priority: | normal → wish |
---|
follow-up: 3 comment:2 by , 5 years ago
Component: | ffmpeg → avcodec |
---|---|
Keywords: | hevc added |
Version: | unspecified → git-master |
by , 5 years ago
Attachment: | puppets_with_alpha_hevc.mov added |
---|
follow-up: 4 comment:3 by , 5 years ago
Replying to cehoyos:
A small sample file will be useful.
Attached Apple's sample file. You can also see it in action on iOS 13 Safari with the following code block
<video src="puppets_with_alpha_hevc.mov" autoplay muted playsinline loop></video>
As seen in this tweet:
follow-up: 5 comment:4 by , 5 years ago
comment:5 by , 5 years ago
Sorry about that. Here's the right link:
https://twitter.com/leemartin/status/1141336580797476865
Replying to cehoyos:
Replying to leemartin:
Replying to cehoyos:
A small sample file will be useful.
As seen in this tweet:
404
comment:6 by , 5 years ago
You would need to use a pix_fmt that has an alpha channel as well. Any of the yuv(a) formats thats also supported by h.265.
comment:7 by , 5 years ago
Using my command above, I try swapping the pix_fmt for 'yuva420p' I receive the warning:
Incompatible pixel format 'yuva420p' for codec 'libx265', auto-selecting format 'yuv420p'
Is there an alpha supported pixel format you can suggest?
comment:8 by , 5 years ago
I just mean if the feature is added you would need an alpha channel pix_fmt. The one you end up using will depend on whats supported as far as color sub sampling and bit depth.
comment:9 by , 5 years ago
Note that this ticket - as shown by the sample attached - is about supporting decoding of transparency-containing (Apple) files and providing the transparency.
Encoding requests cannot be made here as FFmpeg does not contain a native hevc encoder.
by , 4 years ago
Attachment: | HEVC-Video-with-Alpha-Interoperability-Profile.pdf added |
---|
Apple interoperability profile proposal
comment:10 by , 4 years ago
In case it might help, Apple had released a technical document that I did not see referenced here, so I uploaded it.
I really hope ffmpeg will provide support for decoding and encoding of transparency/alpha supporting video (I know that encoding depends on libx265 providing it)
comment:11 by , 4 years ago
Status: | new → open |
---|
Support in hevc_videotoolbox https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210223050227.65698-1-bouno@rouge.plala.or.jp/
Also nvidia started to add this thingy. See: #9088
As I understand x265 does not support alpha https://bitbucket.org/multicoreware/x265_git/issues/577/support-for-alpha-transparency-per-apple
comment:12 by , 3 years ago
It would be great to see HEVC alpha transparency supported in FFMpeg. It might look like a minor issue but it’s really not. In WebGL toolkits like Verge3D, transparent textures are used to create various cool effects. In Chrome/Firefox/Edge we have WebM, but for Safari (for both desktop and mobile), HEVC is the only option. Right now we recommend our users to use FFMpeg with VideoToolbox encoder, but it is limited to Macs only. Thank you for consideration!
comment:13 by , 21 months ago
Some support https://patchwork.ffmpeg.org/project/ffmpeg/patch/tencent_5F1F4DA9CFDF5521A752B8C9161A4D23A007@qq.com/
There is also support added in one of Nvidia's projects.
A small sample file will be useful.