Suboptimal RGB <-> YUV conversion accuracy in default configuration

[y3kcjd5]

Summary: I've discovered that when I'm converting videos to the rgb24 format (in attempting to use the libx264rgb codec) I get banding and discoloration in darker areas with (color) gradients. To eliminate the possibility that it was the codec (as opposed to the format filter) causing the problem, I did some tests converting the video to .png images. Specifically, I found that:
yuv420p10le(source video) -- looks nice
yuv420p10le -> rgb24 -- banding/discoloration
yuv420p10le -> rgb48be -- looks nice
yuv420p10le -> rgb48be -> rgb24 -- banding/discoloration
yuv420p10le -> yuv420p -> rgb24 -- banding but no discoloration

I assume that the banding in the yuv420p example is simply a result of the reduced colorspace, but I don't see why any of the other conversions should result in any visible changes.
I'm using one of Zeranoe's windows builds (win7 64bit) so I don't know if this happens on Linux. Log file output for the yuv420p10le->rgb24 example follows:

ffmpeg started on 2012-07-28 at 10:56:16
Report written to "ffmpeg-20120728-105616.log"
Command line:
ffmpeg -i vtst0.mp4 -map 0:0 -vsync 0 -report -filter:v "format=rgb24" "tst%03d.png"
ffmpeg version N-42704-g85761ef Copyright (c) 2000-2012 the FFmpeg developers
  built on Jul 20 2012 20:39:19 with gcc 4.7.1 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-runtime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass --enable-libcelt --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      51. 65.100 / 51. 65.100
  libavcodec     54. 42.100 / 54. 42.100
  libavformat    54. 18.100 / 54. 18.100
  libavdevice    54.  1.100 / 54.  1.100
  libavfilter     3.  2.100 /  3.  2.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000002e2ec0] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000002e2ec0] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000002e2ec0] File position before avformat_find_stream_info() is 7074306
[h264 @ 00000000002f40a0] no picture 
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000002e2ec0] All info found
rfps: 23.750000 0.011875
rfps: 23.833333 0.004757
rfps: 23.916667 0.000859
rfps: 24.000000 0.000180
rfps: 24.083333 0.002721
rfps: 24.166667 0.008482
rfps: 24.250000 0.017461
rfps: 47.666667 0.019029
rfps: 47.750000 0.009623
rfps: 47.833333 0.003436
rfps: 47.916667 0.000469
rfps: 48.000000 0.000722
rfps: 48.083333 0.004194
rfps: 48.166667 0.010885
rfps: 23.976024 0.000046
rfps: 47.952048 0.000183
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000002e2ec0] File position after avformat_find_stream_info() is 1243950
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'vtst0.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf54.18.100
  Duration: 00:00:10.09, start: 0.000000, bitrate: 5607 kb/s
    Stream #0:0(und), 41, 1/1000: Video: h264 (High 10) (avc1 / 0x31637661), yuv420p10le, 1920x1080, 5604 kb/s, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc
    Metadata:
      handler_name    : VideoHandler
[buffer @ 0000000004ba0f20] Setting entry with key 'video_size' to value '1920x1080'
[buffer @ 0000000004ba0f20] Setting entry with key 'pix_fmt' to value '72'
[buffer @ 0000000004ba0f20] Setting entry with key 'time_base' to value '1/1000'
[buffer @ 0000000004ba0f20] Setting entry with key 'pixel_aspect' to value '0/1'
[buffer @ 0000000004ba0f20] Setting entry with key 'sws_param' to value 'flags=2'
[buffer @ 0000000004ba0f20] Setting entry with key 'frame_rate' to value '24000/1001'
[graph 0 input from stream 0:0 @ 0000000004d73980] w:1920 h:1080 pixfmt:yuv420p10le tb:1/1000 fr:24000/1001 sar:0/1 sws_param:flags=2
[Parsed_format_0 @ 0000000004d738e0] auto-inserting filter 'auto-inserted scaler 0' between the filter 'graph 0 input from stream 0:0' and the filter 'Parsed_format_0'
[auto-inserted scaler 0 @ 0000000004d73b20] w:1920 h:1080 fmt:yuv420p10le sar:0/1 -> w:1920 h:1080 fmt:rgb24 sar:0/1 flags:0x4
[h264 @ 00000000002f40a0] detected 2 logical cores
Output #0, image2, to 'tst%03d.png':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf54.18.100
    Stream #0:0(und), 0, 1/90000: Video: png, rgb24, 1920x1080, q=2-31, 200 kb/s, 90k tbn, 23.98 tbc
    Metadata:
      handler_name    : VideoHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> png)
Press [q] to stop, [?] for help
[h264 @ 00000000003aa100] no picture 
DTS -42, next:-62146 st:0 invalid dropping
[h264 @ 0000000004cdbf00] no picture 
frame=    3 fps=0.0 q=0.0 size=       0kB time=00:00:00.12 bitrate=   0.0kbits/s    
frame=    5 fps=3.1 q=0.0 size=       0kB time=00:00:00.20 bitrate=   0.0kbits/s    
frame=    7 fps=3.0 q=0.0 size=       0kB time=00:00:00.29 bitrate=   0.0kbits/s    

...
 
frame=  237 fps=2.6 q=0.0 size=       0kB time=00:00:09.88 bitrate=   0.0kbits/s    
No more inputs to read from, finishing.
[output stream 0:0 @ 0000000004d73a00] EOF on sink link output stream 0:0:default.
frame=  242 fps=2.6 q=0.0 Lsize=       0kB time=00:00:10.26 bitrate=   0.0kbits/s    

video:276218kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.000000%
[AVIOContext @ 000000000216fd80] Statistics: 4336059 bytes read, 2 seeks

[Carl Eugen Hoyos]

Please provide a sample.

[y3kcjd5]

yuv420p10le->rgb24 frame #150 (banding/discoloration)

[y3kcjd5]

OK, seeing as how I can't get the other samples to fit:
Source video clip vtst0.mp4: ​http://www.datafilehost.com/download-fb229dc6.html
yuv420p10le->rgb48be frame 150 (looks like source): ​http://www.datafilehost.com/download-bc7b949d.html
yuv420p10le->yuv420p->rgb24 frame 150 (banding no discoloration): ​http://www.datafilehost.com/download-8a131612.html

[Carl Eugen Hoyos]

Just to make sure:
Is the image I attached, rgb48.png ok, but the result of ffmpeg -i rgb48.png -pix_fmt rgb24 out24.png shows the banding?

[y3kcjd5]

That is correct.

[ff2012]

I have the same problem!
More info: https://ffmpeg.org/trac/ffmpeg/ticket/1611

[Carl Eugen Hoyos]

Replying to fftest800:

I have the same problem!

More info: https://ffmpeg.org/trac/ffmpeg/ticket/1611

I don't see a conversion from higher bpp to lower bpp of the same type of colourspace in that ticket, why do you think it is a duplicate?

[Ridley Combs]

Found this issue while working on some screenshot comparisons; figured I'd post 'em as another reproduction:
rgb48 = frame extracted from H.264 High 10 video with -ss and -vframes 1
rgb24 = same, but with -filter:v "format=rgb24"
VLCSnap = snapshot of same frame using VLC (probably irrelevant)
Preview = rgb48 frame downsampled to rgb24 using OSX's Preview.app (to show that it's possible to downsample this image with significantly less banding)
rgb48 vs rgb24: ​http://screenshotcomparison.com/comparison/20164
VLCSnap vs rgb24: ​http://screenshotcomparison.com/comparison/20165
rgb24 vs Preview: ​http://screenshotcomparison.com/comparison/20166
rgb48 vs Preview: ​http://screenshotcomparison.com/comparison/20167

[EDIT: 2013-10-02]
More snapshots from the same file? 0.o
New images from a new ffmpeg build:
built on Oct 1 2013 19:49:14 with Apple LLVM version 5.0 (clang-500.1.58) (based on LLVM 3.3svn)
rgb48 output (perfect): ​https://dl.dropboxusercontent.com/u/46348978/Madoka_Snapshot_02.png
rgb48 output, downsampled to rgb24 using convert (very nice): ​https://dl.dropboxusercontent.com/u/46348978/Madoka_Snapshot_02_24_cheat.png
rgb24 output (banding, discoloration): ​https://dl.dropboxusercontent.com/u/46348978/Madoka_Snapshot_02_24.png

[Ridley Combs]

$100 to whoever fixes, where "fixed" is defined as "ffmpeg -ss <time> -i <10bit_h264_file> -vframes 1 -pix_fmt rgb24 out.png" produces output with no discernible image-wide discoloration as compared to the same cmdline without the -pix_fmt, and the amount of banding in the image is comparable to that in the same frame downsampled using ImageMagick.

[Michael Niedermayer]

add bounty keyword so this can be found.
See: ​https://trac.ffmpeg.org/ticket/1582#comment:9

[Carl Eugen Hoyos]

Replying to 11rcombs:

"ffmpeg -ss <time> -i <10bit_h264_file> -vframes 1 -pix_fmt rgb24 out.png" produces output with no discernible image-wide discoloration as compared to the same cmdline without the -pix_fmt, and the amount of banding in the image is comparable to that in the same frame downsampled using ImageMagick.

Could you test the following?

$ ffmpeg -ss <time> -i <10bit_h264_file> -vframes 1 -sws_flags +full_chroma_int -pix_fmt rgb24 out.png

[Ridley Combs]

Replying to cehoyos:

Could you test the following?
… -sws_flags +full_chroma_int …

Looks outstanding. Can this be made the default behavior, or are there issues with the implementation as swscale.h suggests?

[Ridley Combs]

Reducing bounty offer to $10, because it looks like this isn't actually going to require any new dithering code; just a change to default swscale flags in ffmpeg, and that's definitely not worth $100.

[y3kcjd5]

New findings:
With "+full_chroma_int" I was able to fix the discoloration, but I was still getting some mild luma banding until I finally figured out to use "-sws_flags full_chroma_int+accurate_rnd", at which point output was finally for all intents and purposes visually indiscernible from input. +1 lobby for this being default behavior.

[gdgsdg123]

Sitrep from recent version

Still occurs.


Suggested workaround:

Always add +accurate_rnd+full_chroma_int to sws_flags (by -sws_flags +accurate_rnd+full_chroma_int or sws_flags=+accurate_rnd+full_chroma_int in filtergraphs, or both), unless speed is prefered over quality.





Debug info:


Build from: ​https://zeranoe.com/builds/win64/static/ffmpeg-20190402-6aeaac3-win64-static.zip

Command:

C:\>ffmpeg -report -i "yuv420p10le.avi" -vframes 1 -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags spline+print_info+accurate_rnd ".\-.png"
C:\>ffmpeg -report -i "yuv420p10le.avi" -vframes 1 -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags spline+print_info+accurate_rnd+full_chroma_int ".\+.png"
C:\>ffmpeg -report -i "yuv420p10le.avi" -vframes 1 -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags spline+print_info ".\--.png"
C:\>ffmpeg -report -i "yuv420p10le.avi" -vframes 1 -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags spline+print_info+full_chroma_int ".\-+.png"

Relevant files attached.



Difference highlight:
(line 93 of the log files)

"+.log"/"-+.log":

[swscaler] chr srcW=720 srcH=418 dstW=1440 dstH=836 xInc=32768 yInc=32768

"-.log"/"--.log":

[swscaler] chr srcW=720 srcH=418 dstW=720 dstH=836 xInc=65536 yInc=32768

Note:
"-.png" and "--.png", "+.png" and "-+.png", are bitexact respectively.

[gdgsdg123]

Extended tests have been done regarding this issue.



Command:

C:\>FOR /R "INPUT" %X IN (*) DO ffmpeg -report -i "%X" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +print_info+accurate_rnd+full_chroma_int "%~nX.png"
C:\>FOR /R "INPUT" %X IN (*) DO ffmpeg -report -i "%X" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +print_info+accurate_rnd ".\-full_chroma_int_%~nX.png"
C:\>FOR /R "INPUT" %X IN (*) DO ffmpeg -report -i "%X" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +print_info+full_chroma_int ".\-accurate_rnd_%~nX.png"

(INPUT: yuv420p, yuv420p10le, yuv422p, yuv422p10le, yuv444p, yuv444p10le)


Files attached.

Influenced files are marked with ! in filename.





Highlight:

line 91 in "!-full_chroma_int_yuv420p.log":

[swscaler] chr srcW=720 srcH=418 dstW=720 dstH=836 xInc=65536 yInc=32768

line 91 in "yuv420p.log":

[swscaler] chr srcW=720 srcH=418 dstW=1440 dstH=836 xInc=32768 yInc=32768

line 92 in "!-accurate_rnd_yuv420p.log":

[swscaler] using unscaled yuv420p -> rgb24 special converter

[Carl Eugen Hoyos]

Would you mind stopping to attach compressed files? I believe this is mentioned clearly on https://ffmpeg.org/bugreports.html

[gdgsdg123]

Replying to cehoyos:

Would you mind stopping to attach compressed files? I believe this is mentioned clearly on https://ffmpeg.org/bugreports.html

Sorry but... any good choice? (and these don't quite fit the definition of "compressed movie files"...)

BTW does that 2.5 MB size limit really exist?.. If not things can be a lot less messy.

[MasterQuestionable]

͏    See also: ​https://trac.ffmpeg.org/wiki/colorspace#yuv420p10le
͏    Much origin of gdgsdg123.

͏    Changing the default to prefer accuracy over speed, supposedly: shouldn't cause undesirable consequences for general use cases?