Opened 10 years ago

Closed 3 years ago

#3657 closed enhancement (fixed)

float pixel format

Reported by: Björn Sonnenschein Owned by:
Priority: wish Component: swscale
Version: git-master Keywords:
Cc: nfxjfg@googlemail.com Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Hello,
As there are more and more Cameras able to record with more than 8 bit per pixel and there already are programs like Shotcut and Blender which support high-precision internal processing, e.g. blender is unable to handle 10-bit videos properly, because converting a 16 bit buffer from the sws yuv to rgb conversion to its internal 32bit buffer would be too slow and therefore it has to use an 8 bit buffer.
If ffmpeg could convert to 32bit directly, there wouldn't be the need to do two conversion steps, 10 ->16->32 and native high precision video support would be much easier to implement!

Change History (17)

comment:1 by gjdfgh, 10 years ago

Cc: nfxjfg@googlemail.com added

What kind of 32 bit? Integer? Float?

comment:2 by Björn Sonnenschein, 10 years ago

Oh, sorry that I forgot to mention that blender uses float type buffer.

Last edited 10 years ago by Björn Sonnenschein (previous) (diff)

comment:3 by Björn Sonnenschein, 10 years ago

(Double post)

Last edited 10 years ago by Björn Sonnenschein (previous) (diff)

comment:4 by gjdfgh, 10 years ago

Float images would be useful for some other things too, but I bet nobody wants to fight libswscale to add conversion for this (which would probably be very non-trivial).

comment:5 by Cigaes, 10 years ago

When people request float values for pixels (or samples), I always wonder if they really need the features of floats, i.e. more absolute accuracy for numbers near 0. It may make sense for samples, to keep the same relative accuracy regardless of the overall volume. For pixels, maybe issues with gamma will need more accuracy near 0, but I am not really convinced.

Apart from that, I wonder if people realize that 32-bits fixed point is more accurate than floats on 99.21875% of the range.

comment:6 by Carl Eugen Hoyos, 10 years ago

Status: newopen
Summary: 32 Bit per Channel sws conversionfloat pixel format
Version: unspecifiedgit-master

comment:7 by Umair, 8 years ago

Hey,
Sorry for commenting on an almost 2 years old ticket.
But, is this still required ?
I'd love to start with this.

in reply to:  7 ; comment:8 by Carl Eugen Hoyos, 8 years ago

Replying to omerjerk:

I'd love to start with this.

How would you use the pix_fmt after adding it to libavutil?

in reply to:  8 ; comment:9 by Umair, 8 years ago

Replying to cehoyos:

Replying to omerjerk:

I'd love to start with this.

How would you use the pix_fmt after adding it to libavutil?

I went through some of the code present in utils.c file in libswscale.
I'm not really sure how difficult this feature is.

in reply to:  9 comment:10 by Carl Eugen Hoyos, 8 years ago

Replying to omerjerk:

Replying to cehoyos:

Replying to omerjerk:

I'd love to start with this.

How would you use the pix_fmt after adding it to libavutil?

I went through some of the code present in utils.c file in libswscale.
I'm not really sure how difficult this feature is.

Allow me to repeat my question:
Once you have mastered all difficulties and added the new pixel format to libavutil and libswscale, what usage would the new pix_fmt have within FFmpeg?

comment:11 by Elon Musk, 8 years ago

To amuse people. like you.

comment:12 by gjdfgh, 8 years ago

Certainly more usage than "important" formats like MONOWHITE.

comment:13 by Carl Eugen Hoyos, 8 years ago

I created tickets #5302 and #5303 with floating point samples.

comment:14 by emilyb, 7 years ago

We would like to have this too.

The use case is high bit depth processing in HDR mode where we want to be able to have super white/black (i.e. < 0.0 and > 1.0 values) at certain stages in the processing. Using floats is simpler to work with than 32bit integers with headroom and still has enough precision. We also do some advanced image processing which requires conversion to float at some point so having the conversion early is beneficial for performance for us.

As one person noted earlier floats have higher precision around 0.0 than 1.0,but in the range [0.5, 1.0] you still have 23 bits per component in precision which is enough.

comment:15 by mtc, 6 years ago

i guess this means a component color space (RGB/XYZ based, not luminance chroma (YUV) )
RGB16float (16*3 48-bit) or
RGB32float (16*3 48-bit)

also using linear transfer characteristics
(not sRGB-gamma or HLG or Perceptual-Quantizer ...

floating with support for negative-values
to prevent data-loss (clamping-artifacts) from super-whites and outside of color-primaries'gamut colors
what can occur from conversion from YUV(luminance-chroma) color spaces

importing from exif2.0 sYCC (most modern camera jpegs) can clip overwhites esp blues

also if then thus also support outputting into float-supporting formats e.g. OpenEXR for export into supporting rendering programs

support for linearRGB heps make it easier to support linear-gamma(radiometric) image resizing

Last edited 6 years ago by mtc (previous) (diff)

comment:16 by mtc, 6 years ago

gimp 2.9 now supports floating-point RGB formats

GIMP 2.9/2.10 Color Management and Color Mixing

Last edited 6 years ago by mtc (previous) (diff)

comment:17 by Elon Musk, 3 years ago

Resolution: fixed
Status: openclosed

There are gray and gbr(a)p float formats now.

Note: See TracTickets for help on using tickets.