Opened 4 years ago

Last modified 4 weeks ago

#3657 open enhancement

float pixel format

Reported by: sun Owned by:
Priority: wish Component: swscale
Version: git-master Keywords:
Cc: nfxjfg@googlemail.com Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Hello,
As there are more and more Cameras able to record with more than 8 bit per pixel and there already are programs like Shotcut and Blender which support high-precision internal processing, e.g. blender is unable to handle 10-bit videos properly, because converting a 16 bit buffer from the sws yuv to rgb conversion to its internal 32bit buffer would be too slow and therefore it has to use an 8 bit buffer.
If ffmpeg could convert to 32bit directly, there wouldn't be the need to do two conversion steps, 10 ->16->32 and native high precision video support would be much easier to implement!

Change History (16)

comment:1 Changed 4 years ago by gjdfgh

  • Cc nfxjfg@googlemail.com added

What kind of 32 bit? Integer? Float?

comment:2 Changed 4 years ago by sun

Oh, sorry that I forgot to mention that blender uses float type buffer.

Last edited 4 years ago by sun (previous) (diff)

comment:3 Changed 4 years ago by sun

(Double post)

Last edited 4 years ago by sun (previous) (diff)

comment:4 Changed 3 years ago by gjdfgh

Float images would be useful for some other things too, but I bet nobody wants to fight libswscale to add conversion for this (which would probably be very non-trivial).

comment:5 Changed 3 years ago by Cigaes

When people request float values for pixels (or samples), I always wonder if they really need the features of floats, i.e. more absolute accuracy for numbers near 0. It may make sense for samples, to keep the same relative accuracy regardless of the overall volume. For pixels, maybe issues with gamma will need more accuracy near 0, but I am not really convinced.

Apart from that, I wonder if people realize that 32-bits fixed point is more accurate than floats on 99.21875% of the range.

comment:6 Changed 3 years ago by cehoyos

  • Status changed from new to open
  • Summary changed from 32 Bit per Channel sws conversion to float pixel format
  • Version changed from unspecified to git-master

comment:7 follow-up: Changed 22 months ago by omerjerk

Hey,
Sorry for commenting on an almost 2 years old ticket.
But, is this still required ?
I'd love to start with this.

comment:8 in reply to: ↑ 7 ; follow-up: Changed 22 months ago by cehoyos

Replying to omerjerk:

I'd love to start with this.

How would you use the pix_fmt after adding it to libavutil?

comment:9 in reply to: ↑ 8 ; follow-up: Changed 22 months ago by omerjerk

Replying to cehoyos:

Replying to omerjerk:

I'd love to start with this.

How would you use the pix_fmt after adding it to libavutil?

I went through some of the code present in utils.c file in libswscale.
I'm not really sure how difficult this feature is.

comment:10 in reply to: ↑ 9 Changed 22 months ago by cehoyos

Replying to omerjerk:

Replying to cehoyos:

Replying to omerjerk:

I'd love to start with this.

How would you use the pix_fmt after adding it to libavutil?

I went through some of the code present in utils.c file in libswscale.
I'm not really sure how difficult this feature is.

Allow me to repeat my question:
Once you have mastered all difficulties and added the new pixel format to libavutil and libswscale, what usage would the new pix_fmt have within FFmpeg?

comment:11 Changed 22 months ago by richardpl

To amuse people. like you.

comment:12 Changed 22 months ago by gjdfgh

Certainly more usage than "important" formats like MONOWHITE.

comment:13 Changed 21 months ago by cehoyos

I created tickets #5302 and #5303 with floating point samples.

comment:14 Changed 14 months ago by emilyb

We would like to have this too.

The use case is high bit depth processing in HDR mode where we want to be able to have super white/black (i.e. < 0.0 and > 1.0 values) at certain stages in the processing. Using floats is simpler to work with than 32bit integers with headroom and still has enough precision. We also do some advanced image processing which requires conversion to float at some point so having the conversion early is beneficial for performance for us.

As one person noted earlier floats have higher precision around 0.0 than 1.0,but in the range [0.5, 1.0] you still have 23 bits per component in precision which is enough.

comment:15 Changed 4 weeks ago by mtc

i guess this means a component color space (RGB/XYZ based, not luminance chroma (YUV) )
RGB16float (16*3 48-bit) or
RGB32float (16*3 48-bit)

also using linear transfer characteristics
(not sRGB-gamma or HLG or Perceptual-Quantizer ...

floating with support for negative-values
to prevent data-loss (clamping-artifacts) from super-whites and outside of color-primaries'gamut colors
what can occur from conversion from YUV(luminance-chroma) color spaces

importing from exif2.0 sYCC (most modern camera jpegs) can clip overwhites esp blues

also if then thus also support outputting into float-supporting formats e.g. OpenEXR for export into supporting rendering programs

support for linearRGB heps make it easier to support linear-gamma(radiometric) image resizing

Last edited 4 weeks ago by mtc (previous) (diff)

comment:16 Changed 4 weeks ago by mtc

gimp 2.9 now supports floating-point RGB formats

GIMP 2.9/2.10 Color Management and Color Mixing

Last edited 4 weeks ago by mtc (previous) (diff)
Note: See TracTickets for help on using tickets.