Opened 17 months ago
Last modified 9 months ago
#9491 open defect
ass/subtitle filter incorectly handles colorspaces
|Reported by:||alwinramirus||Owned by:|
|Blocking:||Reproduced by developer:||no|
|Analyzed by developer:||no|
Summary of the bug:
I am trying to burn in ass subtitles on to a video. The resulting video differs in color to what media players and subtitle editors display.
The video is in BT.709 limited and the ass subtitles file contains the header line YCbCr Matrix: TV.709.
What I assume happens:
The ass videofilter incorrectly converts the RGB values to BT.601 instead of BT.709 resulting in shifted colors.
How to reproduce:
% ffmpeg -i video.mp4 -vf ass=subtitles.ass out.mp4 ffmpeg version N-104454-gd92fdc7144-20211031 Copyright (c) 2000-2021 the FFmpeg developers
I have attached the files used for testing.
Change History (12)
by , 17 months ago
by , 17 months ago
by , 17 months ago
comment:1 by , 17 months ago
comment:2 by , 17 months ago
The ass files just contains a draw command to draw a green rectangle on the left side with the color (0, 245, 0). This is the same color as the video. In every media player this results in just green being displayed. Using ffmpeg (see out.mp4) there are two shades of green. Probably due to using the incorect matrix.
comment:3 by , 17 months ago
0, 245, 0 is R'G'B' already, not YCbCr, so no YCBCr matrix is needed. I suppose that is c&H00F500. Well, it is not that simple then.
comment:4 by , 17 months ago
Yes I know that those are RGB values.
This was my guess as to what happens since the resulting rectangle is drawn with the color (0, 206, 0) instead of (0, 245, 0) which is what happens if you go RGB --BT.601--> YCbCr --BT.709--> RGB.
Looking at the code it seems like drawutils.c#L163-L165 converts the RGB values to YCbCr with BT.601.
comment:5 by , 17 months ago
|Status:||new → open|
the RGB values to YCbCr with BT.601.
But then it just should do composition in matrix of the video. It has nothing to do with TV.709 line in ass file.
The bug is there though.
comment:6 by , 17 months ago
I am way out of my area of experties here, but as I understand it the YCbCr Matrix line is there for historical reasons to maintain backwards compatibility with old subtitles.
What is worse, VSFilter originally always used BT.601, even for BT.709 videos, which means there are many subtitles out there which have RGB colors that match the BT.709 video only if you use BT.601 for the video's YCbCr -> RGB conversion! Very ugly, indeed. Because of this problem the "YCbCr matrix" field was added to the ASS header, which allows interested parties (e.g. the video renderer) to understand which matrix the subtitle was created for, and if that happens to mismatch the video's actual encoded matrix, extra measures could be applied (e.g. by the video renderer) to correct for the mismatch.
As I understand it this is what is supposed to happen depending on the YCbCr Matrix value:
TV.601 = Output BT.601 with 16-235 YCbCr level range
TV.709 = Output BT.709 with 16-235 YCbCr level range
PC.601 = Output BT.601 with 0-255 YCbCr level range
PC.709 = Output BT.709 with 0-255 YCbCr level range
None = [Decide] based on video
If 'YCbCr Matrix' is missing from a script,[...] use TV.601 for legacy script compatibility
There are also some notes in the libass code: https://github.com/libass/libass/blob/49f116ab1fa4386f6a5191f322ac29872279516e/libass/ass_types.h#L105-L156
comment:7 by , 17 months ago
BTW, there is no way to encode any 8 bit R'G'B' triplet losslessly in YCbCr 8 bit. YCbCr is much bigger than R'G'B', even with full YCbCr and full R'G'B' YCbCr is bigger. So I do not see how it should work.
comment:8 by , 13 months ago
I just closed https://trac.ffmpeg.org/ticket/9682 (duplicate of this). In principle this problem affects all the users of the drawutils function that do conversion from RGB. So there are several other filters where the colors are wrong as well. So all of these filters that do color conversion from RGB have the same bug: https://github.com/FFmpeg/FFmpeg/search?q=ff_draw_color
comment:9 by , 9 months ago
Well, there is now some interesting stuff in 6c3a82f0433de8ff9c35def971a736056cc8ff38.
using BT.601 matrix, not to BT.601. Anyway, I do not get it, there is nothing in ass file??