Changes between Version 5 and Version 6 of colorspace


Ignore:
Timestamp:
Jun 7, 2019, 9:36:45 PM (22 months ago)
Author:
gdgsdg123
Comment:

clarification and notes, style

Legend:

Unmodified
Added
Removed
Modified

  • colorspace

    v5 v6  
    11= Colorspace support in FFmpeg =
    22
    3 == What is colorspace? ==
    4 
    5 Colorspace describes how pixel values are stored within a file or displayed on the screen, and what the range and meaning of those values are.
    6 
    7 For example, it describes how the values of pixels store color information (e.g. RGB or YUV), and how the values of each color component (e.g., R, G, or B) should be translated in order to be properly displayed by the screen.
    8 
    9 The difference between RGB and YUV is, in simple terms:
    10 
    11 - RGB distinguishes color in three components: Red, Green, Blue
    12 - YUV represents color in brightness (Y) and color differences (UV).
    13 
     3== What is colorspace? Why should we care? ==
     4Colorspace describes how an array of pixel values should be displayed on the screen. It provides information like how pixel values are stored within a file, and what the range and meaning of those values are.
     5
     6Basically, it tells the format of the array of pixels (e.g. RGB or YUV), and how the values of each color component should be translated, to be properly displayed by the photons of the screen. (i.e. picking a colorspace randomly seems unlikely a good choice...)
     7[[BR]]
     8[[BR]]
     9[[BR]]
     10The difference between RGB and YUV should be rather obvious:
     11
     12- RGB distinguishes color pixel values into 3 components: Red, Green, Blue. (hence the name)
     13- YUV uses a different representation schema that represents color pixel values in: Luminance (Y, or brightness), Chroma (UV, or color differences). ^(Note: YUV represents color in 3 components)^
    1414{{{
    1515#!div style="border: 1px solid #e5e5c7; margin: 1em; background-color: #ffd;"
    1616'''Note:''' The term "YUV" is ambiguous and often used wrongly, including the definition of pixel formats in FFmpeg. A more accurate term for how color is stored in digital video would be [https://en.wikipedia.org/wiki/YCbCr YCbCr]. Y'UV on other other hand specifies a colorspace consisting of luma (Y') and chrominance (UV) components. For more info read [https://en.wikipedia.org/wiki/YUV the respective Wikipedia article]. In the following, the term YUV is used as in the FFmpeg pixel formats, referring to YCbCr in digital video.
    1717}}}
    18 
     18[[BR]]
     19[[BR]]
    1920The conversion between YUV pixel buffer representation and its visual representation depends on the type of the YUV represented in the pixel buffers, which are essentially device-dependent.
    2021
    … …  
    2425- [https://en.wikipedia.org/wiki/Rec._2020 BT.2020] ("Ultra-High-Definition" or UHD)
    2526
    26 These standards describe not just converions from a YUV signal to RGB, but also how a RGB signal should be represented in terms of photon emission, in a device-independent way.
     27These standards describe not just things like how to convert a YUV signal to RGB, but also how a RGB signal should be represented in terms of photon emission, in a device-independent way.
    2728
    2829== How does FFmpeg identify colorspaces? ==
    2930
    30 In FFmpeg, colorspace is represented in [http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavutil/pixfmt.h;hb=HEAD pixel formats].
    31 
    32 In practical terms, the things you care about are:
    33 
    34 1. Whether the pixel buffer contains RGB, YUV or some other type of pixel format, and the bit-depth.
    35 2. Whether the signals are full range or restricted range. (YUV only, unlikely a problem for other type of signals...)
     31In FFmpeg, colorspaces are represented in the form of [http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavutil/pixfmt.h;hb=HEAD pixel format].
     32[[BR]]
     33[[BR]]
     34In practical terms, the things you care are:
     35
     361. Whether the pixel buffer contains RGB, YUV or some other type of signals, and the bit-depth.
     372. Whether the signals are full range or restricted range. (YUV only, unlikely a problem for other type of signals...) ^(Note: "Full range" refers to the YUV component values being in the range 0–255, whereas "restricted range" has values between 16–235)^
    36383. The transformation matrix between YUV and RGB.
    37394. The linearization function from RGB to a linear RGB signal.
    38405. The conversion matrix between the linearized RGB and the device-independent XYZ colorspace.
    39 
    40 "Full range" refers to the YUV component values being in the range 0–255, whereas "restricted range" has values between 16–235.
    41 
     41[[BR]]
    4242FFmpeg stores all these properties in the [https://ffmpeg.org/doxygen/trunk/structAVFrame.html AVFrame] struct:
    4343
    … …  
    4545- The signal range, in AVFrame->[https://ffmpeg.org/doxygen/trunk/structAVFrame.html#a853afbad220bbc58549b4860732a3aa5 color_range]
    4646- The YUV/RGB transformation matrix, in AVFrame->[https://ffmpeg.org/doxygen/trunk/structAVFrame.html#a9262c231f1f64869439b4fe587fe1710 colorspace]
    47 - The linearization function (aka. transformation characteristics), in AVFrame->[https://ffmpeg.org/doxygen/trunk/structAVFrame.html#ab09abb126e3922bc1d010cf044087939 color_trc]
     47- The linearization function (a.k.a. transformation characteristics), in AVFrame->[https://ffmpeg.org/doxygen/trunk/structAVFrame.html#ab09abb126e3922bc1d010cf044087939 color_trc]
    4848- The RGB/XYZ matrix, in AVFrame->[https://ffmpeg.org/doxygen/trunk/structAVFrame.html#a59a3f830494f2ed1133103a1bc9481e7 color_primaries]
    4949
    … …  
    5151
    5252Conversion between RGB/YUV is typically done using swscale. Conversion between different color properties (bit-depth, range, matrix, transfer characteristics, primaries) can be done using the [https://ffmpeg.org/ffmpeg-filters.html#colorspace colorspace] or [https://ffmpeg.org/ffmpeg-filters.html#colormatrix colormatrix] video filter. There's also a filter using the external library [https://ffmpeg.org/ffmpeg-filters.html#zscale zscale]. (for both aforementioned purposes) ^(...and seems to be a more reliable choice for all these swscale hazards)^
    53 
     53[[BR]]
     54[[BR]]
     55[[BR]]
    5456Video filter `colorspace`, `colormatrix` have the following relationship:
    5557
    … …  
    5759- `colormatrix` supports only 8bpc (8-bit per component) pixel formats, whereas `colorspace` supports 10bpc, 12bpc also.
    5860- `colormatrix` does not apply gamma (primaries) correction, whereas `colorspace` does (it has an option `fast=1` to disable this if you want faster conversion, or compatible output with that produced by `colormatrix`). ^(Note: With `fast=0` (default) it seems to produce significantly worse quality (discoloration)... gamma miscorrection?..)^
    59 - `colormatrix` is C only, whereas `colorspace` uses x86 SIMD (ie. it's faster).
    60 
    61 The major difference between them is `colormatrix` produces horrible quality for anything > 8bpc (8-bit per component)... while `colorspace` produces something decent, at least for 10bpc (for 8bpc they both produce similar bad quality... probably due to improper design in the algorithms). ^(floor instead of round on color approximation?..)^[[BR]]
    62 
    63 For 8bpc, `colorspace` still seems to produce slightly better quality than `colormatrix` (while it's pointless... as doing things in 10bpc first, then 10bpc -> 8bpc seems to be a better approach... if you don't mind dithering). ^([https://trac.ffmpeg.org/ticket/4614 dithering is enforced in swscale YUV 10bpc -> 8bpc])^
    64 
     61- `colormatrix` is C only, whereas `colorspace` uses x86 SIMD (i.e. it's faster).
     62[[BR]]
     63Anyway the major difference between them is `colormatrix` produces horrible quality for anything > 8bpc (8-bit per component)... while `colorspace` produces something decent, at least for 10bpc (for 8bpc they both produce similar bad quality... probably due to improper design in the algorithms). ^(floor instead of round on color approximation?..)^[[BR]]
     64Anyway for 8bpc... `colorspace` still seems to produce slightly better quality than `colormatrix` (while it's pointless... as doing things in 10bpc first, then 10bpc -> 8bpc seems to be a better approach... if you don't mind dithering). ^([https://trac.ffmpeg.org/ticket/4614 dithering is enforced in swscale YUV 10bpc -> 8bpc])^
     65[[BR]]
     66[[BR]]
     67[[BR]]
     68[[BR]]
     69[[BR]]
    6570Read the filters' respective documentation to read up exactly on how to use them.
    6671
    6772The easiest way to use these filters is to ensure that the input AVFrames have all relevant struct members set to the appropriate value. (or got to specify them manually as the filter arguments...)[[BR]]
    6873Then, set the target color properties on the video filter, and it will output the converted frames.
    69 
     74[[BR]]
     75[[BR]]
     76[[BR]]
    7077To convert RGB/YUV or scale using swscale, use swscale and set the appropriate color properties using [https://ffmpeg.org/doxygen/trunk/group__lsws.html#ga541bdffa8149f5f9203664f955faa040 sws_setColorspaceDetails]().
    7178
    72 == Examples ==
    73 
    74 These use a [https://zeranoe.com/builds/win64/static/ffmpeg-20190416-e2f766e-win64-static.zip build from Zeranoe].
    75 
    76 Caveat: For all these images to be properly displayed, a decent browser is a requisite.
    77 
     79== Examples ^(Build from: https://zeranoe.com/builds/win64/static/ffmpeg-20190416-e2f766e-win64-static.zip)^ ==
     80
     81Caveat: For all these images to be properly displayed... some decent browser shall be a requisite.
     82[[BR]]
     83[[BR]]
     84[[BR]]
    7885==== The input source (rgb24): ====
    7986
    8087[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/origin(rgb24).png, 720px)]]
    81 
     88[[BR]]
     89[[BR]]
     90[[BR]]
     91[[BR]]
     92[[BR]]
     93[[BR]]
     94[[BR]]
    8295==== `colormatrix` (yuv444p10le): ====
    83 
    8496{{{
    8597ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colormatrix=bt470bg:bt709" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colormatrix_yuv444p10le.avi"
    8698ffmpeg -i "colormatrix_yuv444p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colormatrix_yuv444p10le.png"
    8799}}}
    88 
    89100Output bit-identical as [#colormatrixyuv444p: `colormatrix` (yuv444p)].
    90 
     101[[BR]]
     102[[BR]]
    91103==== `colorspace` (yuv444p10le): ^(Note: SSIM > 99.98% with [#Theinputsourcergb24: The input source (rgb24)]. Should be 100% if the conversion algorithms are improved though...)^ ====
    92 
    93104{{{
    94105ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p10le.avi"
    95106ffmpeg -i "colorspace_yuv444p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p10le.png"
    96107}}}
    97 
    98108[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p10le.png, 720px)]]
    99 
     109[[BR]]
     110[[BR]]
     111[[BR]]
     112[[BR]]
     113[[BR]]
    100114==== Reference (yuv444p): ====
    101 
    102115{{{
    103116ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "yuv444p.avi"
    104117ffmpeg -i "yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "yuv444p.png"
    105118}}}
    106 
    107119[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/yuv444p.png, 720px)]]
    108 
     120[[BR]]
     121[[BR]]
    109122==== `colormatrix` (yuv444p): ====
    110 
    111123{{{
    112124ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -vf "colormatrix=bt470bg:bt709" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colormatrix_yuv444p.avi"
    113125ffmpeg -i "colormatrix_yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colormatrix_yuv444p.png"
    114126}}}
    115 
    116127[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colormatrix_yuv444p.png, 720px)]]
    117 
     128[[BR]]
     129[[BR]]
    118130==== `colorspace` (yuv444p): ====
    119 
    120131{{{
    121132ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p.avi"
    122133ffmpeg -i "colorspace_yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p.png"
    123134}}}
    124 
    125135[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p.png, 720px)]]
    126 
     136[[BR]]
     137[[BR]]
    127138==== `colorspace` (yuv444p10le -> yuv444p): ====
    128 
    129139{{{
    130140ffmpeg -i "colorspace_yuv444p10le.avi" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p10le-yuv444p.avi"
    131141ffmpeg -i "colorspace_yuv444p10le-yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p10le-yuv444p.png"
    132142}}}
    133 
    134143[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p10le-yuv444p.png, 720px)]]
    135 
     144[[BR]]
     145[[BR]]
     146[[BR]]
     147[[BR]]
     148[[BR]]
     149[[BR]]
     150[[BR]]
    136151==== `colorspace` with `fast=0` `iall=bt601-6-625`: ====
    137 
    138152{{{
    139153ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_fast=0_iall=bt601-6-625.avi"
    140154ffmpeg -i "colorspace_fast=0_iall=bt601-6-625.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_fast=0_iall=bt601-6-625.png"
    141155}}}
    142 
    143156[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_fast%3D0_iall%3Dbt601-6-625.png, 720px)]]
    144 
     157[[BR]]
     158[[BR]]
    145159==== `colorspace` with `fast=0` `iall=bt601-6-525`: ====
    146 
    147160{{{
    148161ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-525" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_fast=0_iall=bt601-6-525.avi"
    149162ffmpeg -i "colorspace_fast=0_iall=bt601-6-525.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_fast=0_iall=bt601-6-525.png"
    150163}}}
    151 
    152164[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_fast%3D0_iall%3Dbt601-6-525.png, 720px)]]
    153 
     165[[BR]]
     166[[BR]]
    154167==== `colorspace` with `fast=1` `iall=bt601-6-525`: ====
    155 
    156168{{{
    157169ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-525:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_fast=1_iall=bt601-6-525.avi"
    158170ffmpeg -i "colorspace_fast=1_iall=bt601-6-525.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_fast=1_iall=bt601-6-525.png"
    159171}}}
    160 
    161172Output bit-identical as [#colorspaceyuv444p10le:Note:SSIM99.98withTheinputsourcergb24.Shouldbe100iftheconversionalgorithmsareimprovedthough... `colorspace` (yuv444p10le)].
    162 
     173[[BR]]
     174[[BR]]
     175[[BR]]
     176[[BR]]
     177[[BR]]
     178[[BR]]
     179[[BR]]
    163180==== Reference (yuv420p10le): ====
    164 
    165181{{{
    166182ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p10le -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "yuv420p10le.avi"
    167183ffmpeg -i "yuv420p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "yuv420p10le.png"
    168184}}}
    169 
    170185[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/yuv420p10le.png, 720px)]]
    171 
     186[[BR]]
     187[[BR]]
    172188==== `colorspace` (yuv420p10le): ====
    173 
    174189{{{
    175190ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv420p10le.avi"
    176191ffmpeg -i "colorspace_yuv420p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv420p10le.png"
    177192}}}
    178 
    179193[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv420p10le.png, 720px)]]
    180 
     194[[BR]]
     195[[BR]]
     196[[BR]]
     197[[BR]]
     198[[BR]]
    181199==== Reference (yuv420p): ====
    182 
    183200{{{
    184201ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "yuv420p.avi"
    185202ffmpeg -i "yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "yuv420p.png"
    186203}}}
    187 
    188204[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/yuv420p.png, 720px)]]
    189 
    190 
     205[[BR]]
     206[[BR]]
    191207==== `colorspace` (yuv420p): ====
    192 
    193208{{{
    194209ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv420p.avi"
    195210ffmpeg -i "colorspace_yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv420p.png"
    196211}}}
    197 
    198212[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv420p.png, 720px)]]
    199 
     213[[BR]]
     214[[BR]]
    200215==== `colorspace` (yuv420p10le -> yuv420p): ====
    201 
    202216{{{
    203217ffmpeg -i "colorspace_yuv420p10le.avi" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv420p10le-yuv420p.avi"
    204218ffmpeg -i "colorspace_yuv420p10le-yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv420p10le-yuv420p.png"
    205219}}}
    206 
    207220[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv420p10le-yuv420p.png, 720px)]]
    208 
     221[[BR]]
     222[[BR]]
    209223==== `colorspace` (yuv444p10le -> yuv420p): ====
    210 
    211224{{{
    212225ffmpeg -i "colorspace_yuv444p10le.avi" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p10le-yuv420p.avi"
    213226ffmpeg -i "colorspace_yuv444p10le-yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p10le-yuv420p.png"
    214227}}}
    215 
    216228[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p10le-yuv420p.png, 720px)]]