| 8 | | The conversion between YUV pixel buffer representation and visual representation depends on the type of YUV represented in the pixel buffers, which are essentially device-dependent. Examples are [https://en.wikipedia.org/wiki/Rec._601 Bt-601] (“Standard-definition” or SD), [https://en.wikipedia.org/wiki/Rec._709 Bt-709] (“High-definition” or HD) or [https://en.wikipedia.org/wiki/Rec._2020 Bt-2020] (“Ultra-high-definition” or UHD). These standards describe not just things like how to convert the YUV signals to RGB, but also how the RGB signal should be represented in terms of photon emission in a device-independent way. |
| | 12 | - RGB distinguishes color pixel values into 3 components: Red, Green, Blue. (hence the name) |
| | 13 | - YUV is an orthogonal representation that represents color pixel values in: Luminance (Y, or brightness), Chroma (UV, or color differences). ^(Note: YUV represents color in 3 components)^ |
| | 14 | [[BR]] |
| | 15 | [[BR]] |
| | 16 | The conversion between YUV pixel buffer representation and its visual representation depends on the type of the YUV represented in the pixel buffers, which are essentially device-dependent. |
| | 29 | 1. Whether the pixel buffer contains RGB, YUV or some other type of signals, and the bit-depth. |
| | 30 | 2. Whether the signals are full range or restricted range. (YUV only, unlikely a problem for other type of signals...) |
| | 31 | 3. The transformation matrix between YUV and RGB. |
| | 32 | 4. The linearization function from RGB to a linear RGB signal. |
| | 33 | 5. The conversion matrix between the linearized RGB and the device-independent XYZ colorspace. |
| | 34 | [[BR]] |
| | 35 | FFmpeg stores all these properties in the [https://ffmpeg.org/doxygen/trunk/structAVFrame.html AVFrame] struct: |
| 18 | | == How do you convert between colorspaces in FFmpeg == |
| | 37 | - The format (type and bit-depth), in AVFrame->[https://ffmpeg.org/doxygen/trunk/structAVFrame.html#aed14fa772ce46881020fd1545c86432c format] |
| | 38 | - The signal range, in AVFrame->[https://ffmpeg.org/doxygen/trunk/structAVFrame.html#a853afbad220bbc58549b4860732a3aa5 color_range] |
| | 39 | - The YUV/RGB transformation matrix, in AVFrame->[https://ffmpeg.org/doxygen/trunk/structAVFrame.html#a9262c231f1f64869439b4fe587fe1710 colorspace] |
| | 40 | - The linearization function (aka. transformation characteristics), in AVFrame->[https://ffmpeg.org/doxygen/trunk/structAVFrame.html#ab09abb126e3922bc1d010cf044087939 color_trc] |
| | 41 | - The RGB/XYZ matrix, in AVFrame->[https://ffmpeg.org/doxygen/trunk/structAVFrame.html#a59a3f830494f2ed1133103a1bc9481e7 color_primaries] |
| 22 | | - they both do only YUV-to-YUV colorspace conversion; YUV-to-RGB or scaling requires swscale. |
| 23 | | - colormatrix supports only 8bit pixel formats; colorspace supports 10/12bit content also. |
| 24 | | - colormatrix does not do gamma/primary correction, whereas colorspace does (it has an option to disable this if you want a faster conversion). |
| 25 | | - colormatrix is C only, whereas colorspace has x86 SIMD (i.e. it’s faster). |
| | 45 | Conversion between RGB/YUV is typically done using swscale. Conversion between different color properties (bit-depth, range, matrix, transfer characteristics, primaries) can be done using the [https://ffmpeg.org/ffmpeg-filters.html#colorspace colorspace] or [https://ffmpeg.org/ffmpeg-filters.html#colormatrix colormatrix] video filter. There's also a filter using the external library [https://ffmpeg.org/ffmpeg-filters.html#zscale zscale]. (for both aforementioned purposes) ^(...and seems to be a more reliable choice for all these swscale hazards)^ |
| | 46 | [[BR]] |
| | 47 | [[BR]] |
| | 48 | [[BR]] |
| | 49 | Video filter `colorspace`, `colormatrix` have the following relationship: |
| 27 | | Read the filters’ respective documentation to read up exactly on how to use them. The easiest way to use these filters is to ensure that the input AVFrame has all relevant struct members set to the correct value. Then, set the target colorspace property on the video filter, and it will output converted frames. |
| | 51 | - They both do only YUV to YUV colorspace conversion; YUV to RGB, and scaling requires swscale. |
| | 52 | - `colormatrix` supports only 8bpc (8-bit per component) pixel formats, whereas `colorspace` supports 10bpc, 12bpc also. |
| | 53 | - `colormatrix` does not apply gamma (primaries) correction, whereas `colorspace` does (it has an option `fast=1` to disable this if you want faster conversion, or compatible output with that produced by `colormatrix`). ^(Note: With `fast=0` (default) it seems to produce significantly worse quality... gamma miscorrection?..)^ |
| | 54 | - `colormatrix` is C only, whereas `colorspace` uses x86 SIMD (ie. it's faster). |
| | 55 | [[BR]] |
| | 56 | Anyway the major difference between them is `colormatrix` produces horrible quality for anything > 8bpc (8-bit per component)... while `colorspace` produces something decent, at least for 10bpc (for 8bpc they both produce similar bad quality... probably due to improper design in the algorithms). ^(floor instead of round on color approximation?..)^[[BR]] |
| | 57 | Anyway for 8bpc... `colorspace` still seems to produce slightly better quality than `colormatrix` (while it's pointless... as doing things in 10bpc first, then 10bpc -> 8bpc seems to be a better approach... if you don't mind dithering). ^([https://trac.ffmpeg.org/ticket/4614 dithering is enforced in swscale YUV 10bpc -> 8bpc])^ |
| | 58 | [[BR]] |
| | 59 | [[BR]] |
| | 60 | [[BR]] |
| | 61 | [[BR]] |
| | 62 | [[BR]] |
| | 63 | Read the filters' respective documentation to read up exactly on how to use them. |
