Consider NV12 / P010 output pixel format support

[Robert Mader]

The currently used pixel formats like yuv420 and especially yuv420_10 are unfortunately not well suited for existing hardware (like display engines) or APIs (like Vulkan, (Linux) KMS, GL), forcing clients to do extra copies either on the CPU or GPU, impacting real-world playback performance.

If libav-decoders would optionally support formats that are typically used by hardware decoders, such as P010 and NV12, a big part of existing consumer devices that don't have a HW-AV1 decoder could archive the optimal number of copies - up to zero-copy in many cases. This would allow bringing overall playback performance close to it's theoretical optimum - across platforms / OSs.

From my (limited) understanding of decoders like libav, using p010 instead of yuv420_10 would be mostly free from a computational side, only changing the place where data is saved and doing a bit-shift - the main burden would likely come from increased code complexity. Therefor I'd like to ask if the libav project would accept patches for new formats - and if there maybe already are plans for that among contributors?

---

Context:

Over the last years a number of developments changed the landscape for what is possible with software video decoding:

1. Consumer devices are increasingly converging to SoCs with display engines with IOMMUs, making it easy for clients/OSs to allocate buffers that can be used with fixed-function hardware to display videos. This includes pretty much all laptops and phones, but also small/embedded devices like the Raspberry Pi 5.
2. Similarly an increasing number of hardware supports displaying HDR10 content properly.
3. Across platforms clients increasingly support zero-copy video playback paths for hardware decoded video - examples here include e.g. Firefox and Chromium.
4. The Wayland/Android/Linux ecosystem in particular sees a lot of development in this area, coupled with evolving HDR APIs, and toolkits like GTK, coupled with e.g. Gstreamer, allow apps to have zero-copy playback with little effort.

I'm part of a group of Gstreamer developers investigating possibilities to allow video players when using sw-decoding in the Wayland ecosystem. In particular we try to allow clients to reuse hardware decoding paths as much as possible. We already are looking into steps like:

1. Using "dmabuf" allocators like "udmabuf", so buffers can be directly be imported/used by GPUs and display engines (will add link once ​https://gitlab.freedesktop.org/ is back).
2. Using options to pre-allocate buffers for decoders to avoid copies - in case of dav1d: ​https://github.com/rust-av/dav1d-rs/pull/107
3. Plumbing formats/shaders in GPU drivers (Mesa), compositors etc. to allow usage of formats like yuv420_10 as directly as possibly (will add link once ​https://gitlab.freedesktop.org/ is back).

Experiments with the steps from above already allow us heavily improved performance compared to what we had previously (or what players like mpv currently allow), however in particular the yuv420_10 format causes some unfortunate issues (apart from missing plumbing in places like Mesa, DRM etc.):

1. It's not compatible with existing Vulkan formats. VK_FORMAT_G16_B16_R16_3PLANE_420_UNORM would be strong contender, but the 6-bit padding is on the "wrong" side (please correct me if there actually is a matching VK format already). Formats like P010, P012 and P016 have the padding on the other side, allowing GPUs to just treat the same and using a single shader for all of them.
2. AFAICS there's not a single display engine supporting this format. yuv420 also is uncommen compared to NV12, but at least some hardware like the RPi4/5 and some qcom devices support it. Support in the display engine is needed for zero-copy playback and allows to power down the GPU.
3. Just doing a CPU conversion adds a significant burden in cases where the CPU or bandwidth is already the limiting factor.

Adding P010 (and potentially NV12 and 422/444 variants) support to libav thus looks like a very promising improvement to me. Thanks for considering :)

---

Related dav1d issue: ​https://code.videolan.org/videolan/dav1d/-/issues/454

[James]

I'm not sure i understand. Semi planar formats like NV12, P010 and such are already supported. What is missing according to you?

[Robert Mader]

Replying to James:

I'm not sure i understand. Semi planar formats like NV12, P010 and such are already supported. What is missing according to you?

If I'm not mistaken a *conversion* to NV12 or P010 is supported - but the decoding always first creates yuv420/yuv420_10. The conversion, however, requires a full copy. What I'm asking / suggesting is supporting direct decoding to NV12 or P010.

I understand that this may require significant changes to performance-critical code. The reason I think it's still worth investigating is that those formats are supported by the vast majority of display controllers on consumer devices and avoiding a full size copy - either on the CPU or GPU - can thus increase playback performance, especially in scenarios where memory bandwidth is the limiting factor.

[Kieran Kunhya]

With the notable exception of AV1, what 10-bit codecs can be decoded on a weak CPU in software that the embedded device doesn't already have a hardware decoder for?

In the end, the "embedded" industry needs to do this work to workaround their hardware limitations, not FFmpeg.

[Balling]

Software decoders still struggle with 4K VP9 sdr 8 bit on my Intel Ultra 7 and you want 10 bit VP9? LOL

[Robert Mader]

FTR. in the related discussion at ​https://code.videolan.org/videolan/dav1d/-/issues/454 we already concluded that supporting these formats would likely require significant changes and is thus probably not worth the effort.

For now we'll instead look into plumbing everything necessary in Vulkan/DRM/Wayland/Mesa to accelerate the currently used formats.

[Balling]

BTW, NV61 is still missing format. We only have NV16. Even though it exists ​https://github.com/FFmpeg/FFmpeg/blob/d435b641a2663c81e4b528cff945543f9bd1d220/libavcodec/v4l2_buffers.c#L386

[Balling]

I still do not get this issue. Imagine you have 120 fps stream. The inverse of that is how much you have time to spend on convert, and it is so much more than conversion from anything to semi-planar NV12 or weird packed chroma, planar luma format like P010. Whatever you are doing here it is clearly wrong, maybe you use cuda_scale, when CPU scales faster. AVX512 and other can be used to accelerate this further. Also note that there are 4 types of P010, Intel and Microsoft define it differently, and BE/LE formats exist.

GPUs adding more garbage formats is also stupid since it adds even more transistors in silicon budget. Which will have to be converted to YCbCr or RGB in the HDMI anyway...

Changing h264 decoder or x264 encoder or whatever to a different pixel format is even more insane, since the question is what and how pixels are moved internally (many, many times) and it is 100x more complex than whatever you want to output. Remember, h264 is mostly bitexact, okay? It is complex...

Moroever nvenc supports p010 input and it makes sense to use HW encoder more less energy and less problems.

Finally, as I said it is impossible to decode 4k60 VP9 8 bit already in software (this was a bug in my Intel Arrow Lake CPU, sorry ​https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/14100), so it does not even matter, because with Hardware overlays (color management of which Intel will fix soon, I hope) output to display directly, CPU just draws the overlay and menus.

[Robert Mader]

Replying to Balling:

I still do not get this issue.

The central points are:

1. The pixel-format / pixel-representation is not dictated by the codec and decoders are free to choose whatever format they see fit.
2. Unfortunately, for whatever reason, there is a mismatch between the formats chosen by most SW decoders like libav, dav1d, vpx etc. and what most HW decoders chose. Display engines in turn usually follow the HW decoders.
3. Software can be changed while existing hardware can't. There are billions of devices from various vendors out there with display engines that support P010 (as defined in Linux DRM, which is compatible Vulkans VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16 and, to my knowledge, with what Microsoft recommends for Windows, see ​https://learn.microsoft.com/en-us/windows/win32/medfound/10-bit-and-16-bit-yuv-video-formats).
4. The SW decoder >= 10bit formats in turn don't seem to be supported by any display engines and, from what I can see, haven't even been defined in graphics APIs (the later I'm hoping to change for VK and Linux DRM).
5. That's why I'm asking to consider supporting the most common format natively - if that is considered feasible. If the conclusion is that it would require too much work, that's totally fine.

GPUs adding more garbage formats is also stupid since it adds even more transistors in silicon budget. Which will have to be converted to YCbCr or RGB in the HDMI anyway...

Again, existing display engines (which do this job) were for some reason build for formats like P010 and not what ffmpeg decided to use.

Moroever nvenc supports p010 input and it makes sense to use HW encoder more less energy and less problems.

I don't see what this has to do with the topic. This is about SW decoding, not HW encoding on Nvidia hardware.

Finally, as I said it is impossible to decode 4k60 VP9 8 bit already in software

I have various older / low power test devices here that can decode 4k60 VP9 8 bit videos as shipped by youtube fine - AFAICS it mainly depends on bitrate and features. In various cases they struggle to get the pixels on screen though because doing several buffer-size copies exhausts their memory bandwidth. This issue about helping with that in common scenarios where buffers can be passed through.

so it does not even matter, because with Hardware overlays (color management of which Intel will fix soon, I hope) output to display directly, CPU just draws the overlay and menus.

Err, I don't know where this is coming from but it sounds pretty out-dated. Proper clients - on Linux/Wayland this includes GTK4-based players (see ​https://blog.gtk.org/2023/11/15/introducing-graphics-offload/), Chromium and Firefox (in development / not enabled by default yet) - draw overlays on the GPU and arrange things so that the system compositor can pass through video content to the display engines.

[Balling]

The pixel-format / pixel-representation is not dictated by the codec and decoders are free to choose whatever format they see fit.

H.264 spec describes a decoder only, not encoder and does in fact mandate pixel formats in which the overlaying of vector/raster primitives must happen. Also they may be 9 bits per pixel, you know? Whether as output or internally.

and what most HW decoders chose

Okay at this point why not go to 10 years ago when this happened 81147b5596ea19f7c5c153f4a534e9314d291fd3

Chromium and Firefox (in development / not enabled by default yet

It is enabled by default including yuv overlays on Firefox and without yuv overlays on Chrome every since I was crying about how their software decoder was elevating black level, and it is enabled since (not including) Haswell.

[Robert Mader]

Semi-related to this topic: I submitted a patch to Linux for the 10 and 12 bit formats used by ffmpeg: ​https://lore.kernel.org/all/20250407191314.500601-1-robert.mader@collabora.com/. Having DRM FOURCCs upstream will e.g. allow buffers to be directly imported by GPU drivers (if allocated as DMABuf), avoiding unnecessary copies.

Comments / suggestions (especially regarding naming) very welcome :)

[compn]

GPUbuf ? /s

patches are welcome for this change in ffmpeg.

to clear up things in this trac issue, would you want ffmpeg to pad everything to a certain colorspace internally? or would this be on a per-codec basis?

or do you want an output device for nv12? like the xv shared memory buffer outdev ​https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavdevice/xv.c

said another way, which specific files in ffmpeg would you want to be changed ?

maybe i'll just run linux on my gpu. ​https://dmitryduka.github.io/linux-on-gpu-presentation/

[Robert Mader]

Replying to compn:

GPUbuf ? /s

patches are welcome for this change in ffmpeg.

to clear up things in this trac issue, would you want ffmpeg to pad everything to a certain colorspace internally? or would this be on a per-codec basis?

This ticket is purely about the pixel-formats (as in memory representations). The core point is that GPUs, or more specifically display engines, often have fixed function hardware for colorspace etc. conversions. Right now those can typically only be used with hw-decoders. Those produce the same values as sw-decoders, just with different formats.

or do you want an output device for nv12? like the xv shared memory buffer outdev ​https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavdevice/xv.c

I don't think there's a need for something like that. Clients like Gstreamer can allocate DMABufs that are compatible with both the GPU/display engine and the decoder (not trivial but possible), and pass references to them around through various APIs (including Wayland and KMS) - that already works.

said another way, which specific files in ffmpeg would you want to be changed ?

From what I understand this would require (substantial) changes to the individual codec implementations - which is why I can fully understand reluctance from codec devs.

[Balling]

Finally, as I said it is impossible to decode 4k60 VP9 8 bit already in software

Sorry this was a bug in my Intel Arrow Lake CPU firmware. ​https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/14100

[Robert Mader]

Just FTR / in case somebody stumbles over this in the future: In Gstreamer 1.28 the libav elements (and other like dav1d etc.) now support decoding into a dmabufs, allowing zero-copy import by GPUs. The relevant formats are supported by most Mesa drivers now (the 10/12/16 formats ended up being called DRM_FORMAT_S010 etc.) and some display engines (like the one on the RPi5) support some of them as well.

Video players like Showtime (the Gnome Video Player) now try to pass through the buffers as far as possible, to the point of skipping the GPU altogether if possible.

Thus closing the ticket.