Changes between Version 4 and Version 5 of colorspace


Ignore:
Timestamp:
May 12, 2019, 4:19:24 PM (7 years ago)
Author:
slhck
Comment:

clarification and notes, style

Legend:

Unmodified
Added
Removed
Modified

  • colorspace

    v4 v5  
    11= Colorspace support in FFmpeg =
    22
    3 == What is colorspace? Why should we care? ==
    4 Colorspace describes how an array of pixel values should be displayed on the screen.
    5 
    6 For example, it tells the format of the array of pixels (eg. RGB or YUV), and how the values of each color component should be translated, to be properly displayed by the photons of the screen. (ie. picking a colorspace randomly seems unlikely a good choice...)
    7 [[BR]]
    8 [[BR]]
    9 [[BR]]
    10 The difference between RGB and YUV should be rather obvious:
    11 
    12 - RGB distinguishes color pixel values into 3 components: Red, Green, Blue. (hence the name)
    13 - YUV is an orthogonal representation that represents color pixel values in: Luminance (Y, or brightness), Chroma (UV, or color differences). ^(Note: YUV represents color in 3 components)^
    14 [[BR]]
    15 [[BR]]
     3== What is colorspace? ==
     4
     5Colorspace describes how pixel values are stored within a file or displayed on the screen, and what the range and meaning of those values are.
     6
     7For example, it describes how the values of pixels store color information (e.g. RGB or YUV), and how the values of each color component (e.g., R, G, or B) should be translated in order to be properly displayed by the screen.
     8
     9The difference between RGB and YUV is, in simple terms:
     10
     11- RGB distinguishes color in three components: Red, Green, Blue
     12- YUV represents color in brightness (Y) and color differences (UV).
     13
     14{{{
     15#!div style="border: 1px solid #e5e5c7; margin: 1em; background-color: #ffd;"
     16'''Note:''' The term "YUV" is ambiguous and often used wrongly, including the definition of pixel formats in FFmpeg. A more accurate term for how color is stored in digital video would be [https://en.wikipedia.org/wiki/YCbCr YCbCr]. Y'UV on other other hand specifies a colorspace consisting of luma (Y') and chrominance (UV) components. For more info read [https://en.wikipedia.org/wiki/YUV the respective Wikipedia article]. In the following, the term YUV is used as in the FFmpeg pixel formats, referring to YCbCr in digital video.
     17}}}
     18
    1619The conversion between YUV pixel buffer representation and its visual representation depends on the type of the YUV represented in the pixel buffers, which are essentially device-dependent.
    1720
    … …  
    2124- [https://en.wikipedia.org/wiki/Rec._2020 BT.2020] ("Ultra-High-Definition" or UHD)
    2225
    23 These standards describe not just things like how to convert a YUV signal to RGB, but also how a RGB signal should be represented in terms of photon emission, in a device-independent way.
     26These standards describe not just converions from a YUV signal to RGB, but also how a RGB signal should be represented in terms of photon emission, in a device-independent way.
    2427
    2528== How does FFmpeg identify colorspaces? ==
    2629
    27 In practical terms, the things you care are:
    28 
    29 1. Whether the pixel buffer contains RGB, YUV or some other type of signals, and the bit-depth.
     30In FFmpeg, colorspace is represented in [http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavutil/pixfmt.h;hb=HEAD pixel formats].
     31
     32In practical terms, the things you care about are:
     33
     341. Whether the pixel buffer contains RGB, YUV or some other type of pixel format, and the bit-depth.
    30352. Whether the signals are full range or restricted range. (YUV only, unlikely a problem for other type of signals...)
    31363. The transformation matrix between YUV and RGB.
    32374. The linearization function from RGB to a linear RGB signal.
    33385. The conversion matrix between the linearized RGB and the device-independent XYZ colorspace.
    34 [[BR]]
     39
     40"Full range" refers to the YUV component values being in the range 0–255, whereas "restricted range" has values between 16–235.
     41
    3542FFmpeg stores all these properties in the [https://ffmpeg.org/doxygen/trunk/structAVFrame.html AVFrame] struct:
    3643
    … …  
    4451
    4552Conversion between RGB/YUV is typically done using swscale. Conversion between different color properties (bit-depth, range, matrix, transfer characteristics, primaries) can be done using the [https://ffmpeg.org/ffmpeg-filters.html#colorspace colorspace] or [https://ffmpeg.org/ffmpeg-filters.html#colormatrix colormatrix] video filter. There's also a filter using the external library [https://ffmpeg.org/ffmpeg-filters.html#zscale zscale]. (for both aforementioned purposes) ^(...and seems to be a more reliable choice for all these swscale hazards)^
    46 [[BR]]
    47 [[BR]]
    48 [[BR]]
     53
    4954Video filter `colorspace`, `colormatrix` have the following relationship:
    5055
    … …  
    5358- `colormatrix` does not apply gamma (primaries) correction, whereas `colorspace` does (it has an option `fast=1` to disable this if you want faster conversion, or compatible output with that produced by `colormatrix`). ^(Note: With `fast=0` (default) it seems to produce significantly worse quality (discoloration)... gamma miscorrection?..)^
    5459- `colormatrix` is C only, whereas `colorspace` uses x86 SIMD (ie. it's faster).
    55 [[BR]]
    56 Anyway the major difference between them is `colormatrix` produces horrible quality for anything > 8bpc (8-bit per component)... while `colorspace` produces something decent, at least for 10bpc (for 8bpc they both produce similar bad quality... probably due to improper design in the algorithms). ^(floor instead of round on color approximation?..)^[[BR]]
    57 Anyway for 8bpc... `colorspace` still seems to produce slightly better quality than `colormatrix` (while it's pointless... as doing things in 10bpc first, then 10bpc -> 8bpc seems to be a better approach... if you don't mind dithering). ^([https://trac.ffmpeg.org/ticket/4614 dithering is enforced in swscale YUV 10bpc -> 8bpc])^
    58 [[BR]]
    59 [[BR]]
    60 [[BR]]
    61 [[BR]]
    62 [[BR]]
     60
     61The major difference between them is `colormatrix` produces horrible quality for anything > 8bpc (8-bit per component)... while `colorspace` produces something decent, at least for 10bpc (for 8bpc they both produce similar bad quality... probably due to improper design in the algorithms). ^(floor instead of round on color approximation?..)^[[BR]]
     62
     63For 8bpc, `colorspace` still seems to produce slightly better quality than `colormatrix` (while it's pointless... as doing things in 10bpc first, then 10bpc -> 8bpc seems to be a better approach... if you don't mind dithering). ^([https://trac.ffmpeg.org/ticket/4614 dithering is enforced in swscale YUV 10bpc -> 8bpc])^
     64
    6365Read the filters' respective documentation to read up exactly on how to use them.
    6466
    6567The easiest way to use these filters is to ensure that the input AVFrames have all relevant struct members set to the appropriate value. (or got to specify them manually as the filter arguments...)[[BR]]
    6668Then, set the target color properties on the video filter, and it will output the converted frames.
    67 [[BR]]
    68 [[BR]]
    69 [[BR]]
     69
    7070To convert RGB/YUV or scale using swscale, use swscale and set the appropriate color properties using [https://ffmpeg.org/doxygen/trunk/group__lsws.html#ga541bdffa8149f5f9203664f955faa040 sws_setColorspaceDetails]().
    7171
    72 == Examples ^(Build from: https://zeranoe.com/builds/win64/static/ffmpeg-20190416-e2f766e-win64-static.zip)^ ==
    73 
    74 Caveat: For all these images to be properly displayed... some decent browser shall be a requisite.
    75 [[BR]]
    76 [[BR]]
    77 [[BR]]
     72== Examples ==
     73
     74These use a [https://zeranoe.com/builds/win64/static/ffmpeg-20190416-e2f766e-win64-static.zip build from Zeranoe].
     75
     76Caveat: For all these images to be properly displayed, a decent browser is a requisite.
     77
    7878==== The input source (rgb24): ====
    7979
    8080[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/origin(rgb24).png, 720px)]]
    81 [[BR]]
    82 [[BR]]
    83 [[BR]]
    84 [[BR]]
    85 [[BR]]
    86 [[BR]]
    87 [[BR]]
     81
    8882==== `colormatrix` (yuv444p10le): ====
     83
    8984{{{
    9085ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colormatrix=bt470bg:bt709" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colormatrix_yuv444p10le.avi"
    9186ffmpeg -i "colormatrix_yuv444p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colormatrix_yuv444p10le.png"
    9287}}}
     88
    9389Output bit-identical as [#colormatrixyuv444p: `colormatrix` (yuv444p)].
    94 [[BR]]
    95 [[BR]]
     90
    9691==== `colorspace` (yuv444p10le): ^(Note: SSIM > 99.98% with [#Theinputsourcergb24: The input source (rgb24)]. Should be 100% if the conversion algorithms are improved though...)^ ====
     92
    9793{{{
    9894ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p10le.avi"
    9995ffmpeg -i "colorspace_yuv444p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p10le.png"
    10096}}}
     97
    10198[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p10le.png, 720px)]]
    102 [[BR]]
    103 [[BR]]
    104 [[BR]]
    105 [[BR]]
    106 [[BR]]
     99
    107100==== Reference (yuv444p): ====
     101
    108102{{{
    109103ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "yuv444p.avi"
    110104ffmpeg -i "yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "yuv444p.png"
    111105}}}
     106
    112107[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/yuv444p.png, 720px)]]
    113 [[BR]]
    114 [[BR]]
     108
    115109==== `colormatrix` (yuv444p): ====
     110
    116111{{{
    117112ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -vf "colormatrix=bt470bg:bt709" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colormatrix_yuv444p.avi"
    118113ffmpeg -i "colormatrix_yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colormatrix_yuv444p.png"
    119114}}}
     115
    120116[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colormatrix_yuv444p.png, 720px)]]
    121 [[BR]]
    122 [[BR]]
     117
    123118==== `colorspace` (yuv444p): ====
     119
    124120{{{
    125121ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p.avi"
    126122ffmpeg -i "colorspace_yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p.png"
    127123}}}
     124
    128125[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p.png, 720px)]]
    129 [[BR]]
    130 [[BR]]
     126
    131127==== `colorspace` (yuv444p10le -> yuv444p): ====
     128
    132129{{{
    133130ffmpeg -i "colorspace_yuv444p10le.avi" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p10le-yuv444p.avi"
    134131ffmpeg -i "colorspace_yuv444p10le-yuv444p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p10le-yuv444p.png"
    135132}}}
     133
    136134[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p10le-yuv444p.png, 720px)]]
    137 [[BR]]
    138 [[BR]]
    139 [[BR]]
    140 [[BR]]
    141 [[BR]]
    142 [[BR]]
    143 [[BR]]
     135
    144136==== `colorspace` with `fast=0` `iall=bt601-6-625`: ====
     137
    145138{{{
    146139ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_fast=0_iall=bt601-6-625.avi"
    147140ffmpeg -i "colorspace_fast=0_iall=bt601-6-625.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_fast=0_iall=bt601-6-625.png"
    148141}}}
     142
    149143[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_fast%3D0_iall%3Dbt601-6-625.png, 720px)]]
    150 [[BR]]
    151 [[BR]]
     144
    152145==== `colorspace` with `fast=0` `iall=bt601-6-525`: ====
     146
    153147{{{
    154148ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-525" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_fast=0_iall=bt601-6-525.avi"
    155149ffmpeg -i "colorspace_fast=0_iall=bt601-6-525.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_fast=0_iall=bt601-6-525.png"
    156150}}}
     151
    157152[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_fast%3D0_iall%3Dbt601-6-525.png, 720px)]]
    158 [[BR]]
    159 [[BR]]
     153
    160154==== `colorspace` with `fast=1` `iall=bt601-6-525`: ====
     155
    161156{{{
    162157ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-525:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_fast=1_iall=bt601-6-525.avi"
    163158ffmpeg -i "colorspace_fast=1_iall=bt601-6-525.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_fast=1_iall=bt601-6-525.png"
    164159}}}
     160
    165161Output bit-identical as [#colorspaceyuv444p10le:Note:SSIM99.98withTheinputsourcergb24.Shouldbe100iftheconversionalgorithmsareimprovedthough... `colorspace` (yuv444p10le)].
    166 [[BR]]
    167 [[BR]]
    168 [[BR]]
    169 [[BR]]
    170 [[BR]]
    171 [[BR]]
    172 [[BR]]
     162
    173163==== Reference (yuv420p10le): ====
     164
    174165{{{
    175166ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p10le -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "yuv420p10le.avi"
    176167ffmpeg -i "yuv420p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "yuv420p10le.png"
    177168}}}
     169
    178170[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/yuv420p10le.png, 720px)]]
    179 [[BR]]
    180 [[BR]]
     171
    181172==== `colorspace` (yuv420p10le): ====
     173
    182174{{{
    183175ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p10le -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv420p10le.avi"
    184176ffmpeg -i "colorspace_yuv420p10le.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv420p10le.png"
    185177}}}
     178
    186179[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv420p10le.png, 720px)]]
    187 [[BR]]
    188 [[BR]]
    189 [[BR]]
    190 [[BR]]
    191 [[BR]]
     180
    192181==== Reference (yuv420p): ====
     182
    193183{{{
    194184ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 5 -color_primaries 5 -color_trc 6 "yuv420p.avi"
    195185ffmpeg -i "yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "yuv420p.png"
    196186}}}
     187
    197188[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/yuv420p.png, 720px)]]
    198 [[BR]]
    199 [[BR]]
     189
     190
    200191==== `colorspace` (yuv420p): ====
     192
    201193{{{
    202194ffmpeg -i "origin(rgb24).png" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-625:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv420p.avi"
    203195ffmpeg -i "colorspace_yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv420p.png"
    204196}}}
     197
    205198[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv420p.png, 720px)]]
    206 [[BR]]
    207 [[BR]]
     199
    208200==== `colorspace` (yuv420p10le -> yuv420p): ====
     201
    209202{{{
    210203ffmpeg -i "colorspace_yuv420p10le.avi" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv420p10le-yuv420p.avi"
    211204ffmpeg -i "colorspace_yuv420p10le-yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv420p10le-yuv420p.png"
    212205}}}
     206
    213207[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv420p10le-yuv420p.png, 720px)]]
    214 [[BR]]
    215 [[BR]]
     208
    216209==== `colorspace` (yuv444p10le -> yuv420p): ====
     210
    217211{{{
    218212ffmpeg -i "colorspace_yuv444p10le.avi" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv420p -sws_flags spline+accurate_rnd+full_chroma_int -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "colorspace_yuv444p10le-yuv420p.avi"
    219213ffmpeg -i "colorspace_yuv444p10le-yuv420p.avi" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +accurate_rnd+full_chroma_int "colorspace_yuv444p10le-yuv420p.png"
    220214}}}
     215
    221216[[Image(https://trac.ffmpeg.org/raw-attachment/wiki/colorspace/colorspace_yuv444p10le-yuv420p.png, 720px)]]