Opened 5 years ago

Last modified 15 months ago

#7965 open enhancement

Support for HEVC with Alpha

Reported by: leemartin Owned by:
Priority: wish Component: avcodec
Version: git-master Keywords: hevc
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

At WWDC this year, Apple introduced support for HEVC with Alpha to both their app and browser platforms. You can find their talk on the subject here:

https://developer.apple.com/videos/play/wwdc2019/506

I'm an amateur at FFmpeg but I went down a rabbit hole today, trying to create a video in this new format using the latest version of FFmpeg. I took a rather simple approach of using ffprobe on one of their example video files (found in the resources on the link above) to get a profile and attempting to tune an ffmpeg encoding command to fit the same profile.

While the following command does recreate the profile, it does not retain the transparency of the source prores .mov file.

% ffmpeg -i puppets_with_alpha_prores.mov -c:v libx265 -tag:v hvc1 -pix_fmt yuv420p -colorspace bt709 -color_primaries bt709 -color_trc bt709 puppets_with_alpha_hevc.mov

Apple is pretty clear about how they are achieving this encoding in their talk and examples. Naturally, I didn't follow anything they said and simply tried to hack a solution.

So I'm wondering if this is something FFmpeg can do or would be able to support in the future?

Thanks for the help!

Attachments (2)

puppets_with_alpha_hevc.mov (2.3 MB ) - added by leemartin 5 years ago.
HEVC-Video-with-Alpha-Interoperability-Profile.pdf (137.9 KB ) - added by pecus 3 years ago.
Apple interoperability profile proposal

Change History (15)

comment:1 by leemartin, 5 years ago

Priority: normalwish

comment:2 by Carl Eugen Hoyos, 5 years ago

Component: ffmpegavcodec
Keywords: hevc added
Version: unspecifiedgit-master

A small sample file will be useful.

by leemartin, 5 years ago

Attachment: puppets_with_alpha_hevc.mov added

in reply to:  2 ; comment:3 by leemartin, 5 years ago

Replying to cehoyos:

A small sample file will be useful.

Attached Apple's sample file. You can also see it in action on iOS 13 Safari with the following code block

<video src="puppets_with_alpha_hevc.mov" autoplay muted playsinline loop></video>

As seen in this tweet:

https://twitter.com/leemartin/status/114133658079747686

in reply to:  3 ; comment:4 by Carl Eugen Hoyos, 5 years ago

Replying to leemartin:

Replying to cehoyos:

A small sample file will be useful.

As seen in this tweet:

https://twitter.com/leemartin/status/114133658079747686

404

in reply to:  4 comment:5 by leemartin, 5 years ago

Sorry about that. Here's the right link:

https://twitter.com/leemartin/status/1141336580797476865

Replying to cehoyos:

Replying to leemartin:

Replying to cehoyos:

A small sample file will be useful.

As seen in this tweet:

https://twitter.com/leemartin/status/114133658079747686

404

comment:6 by shaocaholica, 5 years ago

You would need to use a pix_fmt that has an alpha channel as well. Any of the yuv(a) formats thats also supported by h.265.

comment:7 by leemartin, 5 years ago

Using my command above, I try swapping the pix_fmt for 'yuva420p' I receive the warning:

Incompatible pixel format 'yuva420p' for codec 'libx265', auto-selecting format 'yuv420p'

Is there an alpha supported pixel format you can suggest?

Last edited 5 years ago by leemartin (previous) (diff)

comment:8 by shaocaholica, 5 years ago

I just mean if the feature is added you would need an alpha channel pix_fmt. The one you end up using will depend on whats supported as far as color sub sampling and bit depth.

Last edited 5 years ago by shaocaholica (previous) (diff)

comment:9 by Carl Eugen Hoyos, 5 years ago

Note that this ticket - as shown by the sample attached - is about supporting decoding of transparency-containing (Apple) files and providing the transparency.
Encoding requests cannot be made here as FFmpeg does not contain a native hevc encoder.

by pecus, 3 years ago

Apple interoperability profile proposal

comment:10 by pecus, 3 years ago

In case it might help, Apple had released a technical document that I did not see referenced here, so I uploaded it.

I really hope ffmpeg will provide support for decoding and encoding of transparency/alpha supporting video (I know that encoding depends on libx265 providing it)

comment:11 by Balling, 3 years ago

Status: newopen

comment:12 by Alexander, 2 years ago

It would be great to see HEVC alpha transparency supported in FFMpeg. It might look like a minor issue but it’s really not. In WebGL toolkits like Verge3D, transparent textures are used to create various cool effects. In Chrome/Firefox/Edge we have WebM, but for Safari (for both desktop and mobile), HEVC is the only option. Right now we recommend our users to use FFMpeg with VideoToolbox encoder, but it is limited to Macs only. Thank you for consideration!

Last edited 2 years ago by Alexander (previous) (diff)
Note: See TracTickets for help on using tickets.