Context Navigation

Colorspace support in FFmpeg

What is colorspace? Why should we care?

Colorspace describes how an array of pixel values should be displayed on the screen. It provides information like how pixel values are stored within a file, and what the range and meaning of those values are.

Basically, it tells the format of the array of pixels (e.g. RGB or YUV), and how the values of each color component should be translated, to be properly displayed by the photons of the screen. (i.e. picking a colorspace randomly seems unlikely to be a good choice...)

The differences between RGB and YUV:

RGB distinguishes color pixel values into 3 components: Red, Green, Blue. (hence the name)
YUV uses a different representation schema that represents color pixel values in: Luma (Y, or brightness), Chroma (U and V, two color differences). This format exploits the fact that humans are able to better notice details in luminance (approximated by luma) than in chroma. ^{(Note: YUV represents color in 3 components)}
Note: The term "YUV" is ambiguous and often used wrongly, including the definition of pixel formats in FFmpeg. A more accurate term for how color is stored in digital video would be YCbCr. Y'UV on other other hand specifies a colorspace consisting of luma (Y') and chrominance (UV) components. For more info read the respective Wikipedia article. In the following, the term YUV is used as in the FFmpeg pixel formats, referring to YCbCr in digital video.

The conversion between YUV pixel buffer representation and its visual representation depends on the type of the YUV represented in the pixel buffers, which are essentially device-dependent. The accuracy of grey vs color separation depends on how well the transformation matrix matches the underlying RGB space.

Examples are:

BT.601 ("Standard-Definition" or SD)
BT.709 ("High-Definition" or HD)
BT.2020 ("Ultra-High-Definition" or UHD)

These standards describe not just things like how to convert a YUV signal to RGB, but also how a RGB signal should be represented in terms of photon emission, in a device-independent way.

How does FFmpeg identify colorspaces?

In FFmpeg, colorspaces are represented in the form of pixel format.

In practical terms, the properties you care about are:

The signal format in the pixel buffer:
1. the signal type: RGB, YUV, or other type, and
2. the signal bit-depth.
For YUV signals, the color range: full/pc/jpeg or limited/restricted/tv/mpeg. For 8bpc, full means 0-255 and limited means 16-235.
The transformation matrix between YUV and RGB.
The linearization function from RGB to a linear RGB signal.
The conversion matrix between the linearized RGB and the device-independent XYZ colorspace.

FFmpeg stores these properties in the AVFrame struct:

The format (type and bit-depth)->format
The signal range->color_range
The YUV/RGB transformation matrix->colorspace
The linearization function (a.k.a. transformation characteristics)->color_trc
The RGB/XYZ matrix->color_primaries

How to use FFmpeg to convert the colorspace

Pixel format conversion is often performed by libswscale, which you will most likely access through the FFmpeg Scaler or the scale video filter.

Comparing and contrasting `colorspace`, `colormatrix`, libswscale, and `zscale`

colorspace and colormatrix can only convert YUV to YUV
libswscale can convert between YUV and RGB
colormatrix only supports 8bpc (8-bit per component) pixel formats
colorspace supports 8bpc, 10bpc, and 12bpc
colormatrix does not apply gamma and primaries correction (i.e. which exact color "red", "green", and "blue" each is)
colorspace applies gamma and primaries correction by default. (Use fast=1 to disable this for faster conversion or to make output compatible with output produced by colormatrix.)
colormatrix is C only, whereas colorspace uses x86 SIMD so it's faster
colormatrix produces horrible quality for 10bpc and 12bpc
colorspace produces something decent for 10bpc
For 8bpc, colorspace seems to produce slightly better quality than colormatrix
zscale produces better results at all bit depths

`colorspace` and friends

The easiest way to use these filters is to ensure that the input AVFrames have all relevant struct members set to the appropriate values. On ffmpeg command-line this is what options such as colorspace and color_trc do; you can also put them on the output side to modify what space the output is in.

If everything goes well, libswscale figures out how to do the conversion and you can avoid calling the filter manually. However, AVFrames does not cover every option needed, so advanced use will go back to colorspace or zscale.

See also sws_setColorspaceDetails() and F.X.'s colorspace answer.

sRGB

There is no colorspace default called sRGB in FFmpeg, because it's identical to BT.709 except for a different transfer function. Just use one of the following:

sRGB input, fast: -vf "colorspace=all=bt709:fast=1", aka -colorspace bt709 (on the input side!)
sRGB input, accurate: -vf "colorspace=all=bt709:itrc=srgb:fast=0", aka -colorspace bt709 -color_trc srgb (on the input side!)

See also F.X.'s HDR answer and Gyan's sRGB output answer. If srgb does not work, try the more academic name iec61966-2-1.

"color_primaries"

͏ Quick reference for somewhat cryptic "color_primaries" names:
͏ 1: "bt709" (BT.709)
͏ 5: "bt470bg" / "gamma28" (BT.470 BG)
͏ 6: "smpte170m" (SMPTE 170 M)
͏ (alike for "colorspace" "color_trc")
͏ .
͏ See also:
(`ffmpeg -h full` derived)
[[

-color_range <int>  color range (from 0 to INT_MAX) (default unknown)
  0 unknown          Unspecified
    unspecified
  1 tv               MPEG (( 219 * 2^( n - 8 ) ))
    mpeg
    limited
  2 pc               JPEG (( 2^( n - 1 ) ))
    jpeg
    full

-colorspace <int>      color space (from 0 to INT_MAX) (default unknown)
   0 rgb                RGB
   1 bt709              BT.709
   2 unknown            Unspecified
     unspecified
   4 fcc                FCC
   5 bt470bg            BT.470 BG
   6 smpte170m          SMPTE 170 M
   7 smpte240m          SMPTE 240 M
   8 ycgco              YCGCO
     ycocg
   9 bt2020nc           BT.2020 NCL
     bt2020_ncl
  10 bt2020c            BT.2020 CL
     bt2020_cl
  11 smpte2085          SMPTE 2085
  12 chroma-derived-nc  Chroma-derived NCL
  13 chroma-derived-c   Chroma-derived CL
  14 ictcp              ICtCp
  15 ipt-c2             IPT-C2
  16 ycgco-re           YCGCO-R (even add)
  17 ycgco-ro           YCGCO-R (odd add)

-color_primaries <int>  color primaries (from 1 to INT_MAX) (default unknown)
   1 bt709               BT.709
   2 unknown             Unspecified
     unspecified
   4 bt470m              BT.470 M
   5 bt470bg             BT.470 BG
   6 smpte170m           SMPTE 170 M
   7 smpte240m           SMPTE 240 M
   8 film                Film
   9 bt2020              BT.2020
  10 smpte428            SMPTE 428-1
     smpte428_1
  11 smpte431            SMPTE 431-2
  12 smpte432            SMPTE 422-1
  22 jedec-p22           JEDEC P22
     ebu3213             EBU 3213-E

-color_trc <int>  color transfer characteristics (from 1 to INT_MAX) (default unknown)
   1 bt709         BT.709
   2 unknown       Unspecified
     unspecified
   4 gamma22       BT.470 M
   5 gamma28       BT.470 BG
   6 smpte170m     SMPTE 170 M
   7 smpte240m     SMPTE 240 M
   8 linear        Linear
   9 log100        Log
     log
  10 log316        Log square root
     log_sqrt
  11 iec61966-2-4  IEC 61966-2-4
     iec61966_2_4
  12 bt1361e       BT.1361
     bt1361
  13 iec61966-2-1  IEC 61966-2-1
     iec61966_2_1
  14 bt2020-10     BT.2020 10-bit
     bt2020_10bit
  15 bt2020-12     BT.2020 12-bit
     bt2020_12bit
  16 smpte2084     SMPTE 2084
  17 smpte428      SMPTE 428-1
     smpte428_1
  18 arib-std-b67  ARIB STD-B67

]]
͏ [ Still cryptic whatsoever... ]

Examples (with Zeranoe 20190416)

͏ Note:
͏ Use [ https://github.com/MasterInQuestion/Markup/blob/main/picdif.htm ] for testing.

The input source (rgb24):

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/origin(rgb24).png

`colormatrix` (yuv444p10le):

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colormatrix=bt470bg:bt709" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colormatrix_yuv444p10le.avi"
ffmpeg -i "colormatrix_yuv444p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colormatrix_yuv444p10le.png"

Output bit-identical as `colormatrix` (yuv444p).

`colorspace` (yuv444p10le):

Note: SSIM > 99.98% with The input source (rgb24). Should be 100% if the conversion algorithms are improved, though.

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p10le.avi"
ffmpeg -i "colorspace_yuv444p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p10le.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p10le.png

Reference (yuv444p):

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "yuv444p.avi"
ffmpeg -i "yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "yuv444p.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/yuv444p.png

`colormatrix` (yuv444p):

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -vf "colormatrix=bt470bg:bt709" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colormatrix_yuv444p.avi"
ffmpeg -i "colormatrix_yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colormatrix_yuv444p.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colormatrix_yuv444p.png

`colorspace` (yuv444p):

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p.avi"
ffmpeg -i "colorspace_yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p.png

`colorspace` (yuv444p10le -> yuv444p):

ffmpeg -i "colorspace_yuv444p10le.avi" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p10le-yuv444p.avi"
ffmpeg -i "colorspace_yuv444p10le-yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p10le-yuv444p.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p10le-yuv444p.png

`colorspace` with `fast=0` `iall=bt601-6-625`:

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_fast=0_iall=bt601-6-625.avi"
ffmpeg -i "colorspace_fast=0_iall=bt601-6-625.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_fast=0_iall=bt601-6-625.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_fast=0_iall=bt601-6-625.png

`colorspace` with `fast=0` `iall=bt601-6-525`:

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-525" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_fast=0_iall=bt601-6-525.avi"
ffmpeg -i "colorspace_fast=0_iall=bt601-6-525.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_fast=0_iall=bt601-6-525.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_fast=0_iall=bt601-6-525.png

`colorspace` with `fast=1` `iall=bt601-6-525`:

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-525:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_fast=1_iall=bt601-6-525.avi"
ffmpeg -i "colorspace_fast=1_iall=bt601-6-525.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_fast=1_iall=bt601-6-525.png"

Output bit-identical as `colorspace` (yuv444p10le).

Reference (yuv420p10le):

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p10le -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "yuv420p10le.avi"
ffmpeg -i "yuv420p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "yuv420p10le.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/yuv420p10le.png

`colorspace` (yuv420p10le):

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv420p10le.avi"
ffmpeg -i "colorspace_yuv420p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv420p10le.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv420p10le.png

Reference (yuv420p):

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "yuv420p.avi"
ffmpeg -i "yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "yuv420p.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/yuv420p.png

`colorspace` (yuv420p):

ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv420p.avi"
ffmpeg -i "colorspace_yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv420p.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv420p.png

`colorspace` (yuv420p10le -> yuv420p):

ffmpeg -i "colorspace_yuv420p10le.avi" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv420p10le-yuv420p.avi"
ffmpeg -i "colorspace_yuv420p10le-yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv420p10le-yuv420p.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv420p10le-yuv420p.png

`colorspace` (yuv444p10le -> yuv420p):

ffmpeg -i "colorspace_yuv444p10le.avi" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p10le-yuv420p.avi"
ffmpeg -i "colorspace_yuv444p10le-yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p10le-yuv420p.png"

https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p10le-yuv420p.png

Last modified 37 minutes ago Last modified on May 2, 2024, 11:07:35 PM

Attachments (14)

origin(rgb24).png (1.2 MB ) - added by gdgsdg123 5 years ago.
colorspace_yuv444p10le.png (1.2 MB ) - added by gdgsdg123 5 years ago.
yuv444p.png (1.3 MB ) - added by gdgsdg123 5 years ago.
colormatrix_yuv444p.png (1.3 MB ) - added by gdgsdg123 5 years ago.
colorspace_yuv444p.png (1.3 MB ) - added by gdgsdg123 5 years ago.
colorspace_yuv444p10le-yuv444p.png (1.5 MB ) - added by gdgsdg123 5 years ago.
colorspace_fast=0_iall=bt601-6-625.png (1.3 MB ) - added by gdgsdg123 5 years ago.
colorspace_fast=0_iall=bt601-6-525.png (1.3 MB ) - added by gdgsdg123 5 years ago.
yuv420p10le.png (1.2 MB ) - added by gdgsdg123 5 years ago.
colorspace_yuv420p10le.png (1.2 MB ) - added by gdgsdg123 5 years ago.
yuv420p.png (1.2 MB ) - added by gdgsdg123 5 years ago.
colorspace_yuv420p.png (1.3 MB ) - added by gdgsdg123 5 years ago.
colorspace_yuv420p10le-yuv420p.png (1.4 MB ) - added by gdgsdg123 5 years ago.
colorspace_yuv444p10le-yuv420p.png (1.4 MB ) - added by gdgsdg123 5 years ago.

Note: See TracWiki for help on using the wiki.

Download in other formats:

Plain Text

Context Navigation

Colorspace support in FFmpeg

What is colorspace? Why should we care?

How does FFmpeg identify colorspaces?

How to use FFmpeg to convert the colorspace

Comparing and contrasting colorspace, colormatrix, libswscale, and zscale

colorspace and friends

sRGB

"color_primaries"

Examples (with ​Zeranoe 20190416)

The input source (rgb24):

colormatrix (yuv444p10le):

colorspace (yuv444p10le):

Reference (yuv444p):

colormatrix (yuv444p):

colorspace (yuv444p):

colorspace (yuv444p10le -> yuv444p):

colorspace with fast=0 iall=bt601-6-625:

colorspace with fast=0 iall=bt601-6-525:

colorspace with fast=1 iall=bt601-6-525: