Opened 12 years ago

Last modified 5 years ago

#1582 open defect

Banding/discoloration in darker gradients when converting to rgb24

Reported by: y3kcjd5 Owned by:
Priority: normal Component: swscale
Version: git-master Keywords: swscale, bounty
Cc: rodger.combs@gmail.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary: I've discovered that when I'm converting videos to the rgb24 format (in attempting to use the libx264rgb codec) I get banding and discoloration in darker areas with (color) gradients. To eliminate the possibility that it was the codec (as opposed to the format filter) causing the problem, I did some tests converting the video to .png images. Specifically, I found that:
yuv420p10le(source video) -- looks nice
yuv420p10le -> rgb24 -- banding/discoloration
yuv420p10le -> rgb48be -- looks nice
yuv420p10le -> rgb48be -> rgb24 -- banding/discoloration
yuv420p10le -> yuv420p -> rgb24 -- banding but no discoloration

I assume that the banding in the yuv420p example is simply a result of the reduced colorspace, but I don't see why any of the other conversions should result in any visible changes.
I'm using one of Zeranoe's windows builds (win7 64bit) so I don't know if this happens on Linux. Log file output for the yuv420p10le->rgb24 example follows:

ffmpeg started on 2012-07-28 at 10:56:16
Report written to "ffmpeg-20120728-105616.log"
Command line:
ffmpeg -i vtst0.mp4 -map 0:0 -vsync 0 -report -filter:v "format=rgb24" "tst%03d.png"
ffmpeg version N-42704-g85761ef Copyright (c) 2000-2012 the FFmpeg developers
  built on Jul 20 2012 20:39:19 with gcc 4.7.1 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-runtime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libass --enable-libcelt --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libnut --enable-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libutvideo --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      51. 65.100 / 51. 65.100
  libavcodec     54. 42.100 / 54. 42.100
  libavformat    54. 18.100 / 54. 18.100
  libavdevice    54.  1.100 / 54.  1.100
  libavfilter     3.  2.100 /  3.  2.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000002e2ec0] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000002e2ec0] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000002e2ec0] File position before avformat_find_stream_info() is 7074306
[h264 @ 00000000002f40a0] no picture 
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000002e2ec0] All info found
rfps: 23.750000 0.011875
rfps: 23.833333 0.004757
rfps: 23.916667 0.000859
rfps: 24.000000 0.000180
rfps: 24.083333 0.002721
rfps: 24.166667 0.008482
rfps: 24.250000 0.017461
rfps: 47.666667 0.019029
rfps: 47.750000 0.009623
rfps: 47.833333 0.003436
rfps: 47.916667 0.000469
rfps: 48.000000 0.000722
rfps: 48.083333 0.004194
rfps: 48.166667 0.010885
rfps: 23.976024 0.000046
rfps: 47.952048 0.000183
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000002e2ec0] File position after avformat_find_stream_info() is 1243950
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'vtst0.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf54.18.100
  Duration: 00:00:10.09, start: 0.000000, bitrate: 5607 kb/s
    Stream #0:0(und), 41, 1/1000: Video: h264 (High 10) (avc1 / 0x31637661), yuv420p10le, 1920x1080, 5604 kb/s, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc
    Metadata:
      handler_name    : VideoHandler
[buffer @ 0000000004ba0f20] Setting entry with key 'video_size' to value '1920x1080'
[buffer @ 0000000004ba0f20] Setting entry with key 'pix_fmt' to value '72'
[buffer @ 0000000004ba0f20] Setting entry with key 'time_base' to value '1/1000'
[buffer @ 0000000004ba0f20] Setting entry with key 'pixel_aspect' to value '0/1'
[buffer @ 0000000004ba0f20] Setting entry with key 'sws_param' to value 'flags=2'
[buffer @ 0000000004ba0f20] Setting entry with key 'frame_rate' to value '24000/1001'
[graph 0 input from stream 0:0 @ 0000000004d73980] w:1920 h:1080 pixfmt:yuv420p10le tb:1/1000 fr:24000/1001 sar:0/1 sws_param:flags=2
[Parsed_format_0 @ 0000000004d738e0] auto-inserting filter 'auto-inserted scaler 0' between the filter 'graph 0 input from stream 0:0' and the filter 'Parsed_format_0'
[auto-inserted scaler 0 @ 0000000004d73b20] w:1920 h:1080 fmt:yuv420p10le sar:0/1 -> w:1920 h:1080 fmt:rgb24 sar:0/1 flags:0x4
[h264 @ 00000000002f40a0] detected 2 logical cores
Output #0, image2, to 'tst%03d.png':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf54.18.100
    Stream #0:0(und), 0, 1/90000: Video: png, rgb24, 1920x1080, q=2-31, 200 kb/s, 90k tbn, 23.98 tbc
    Metadata:
      handler_name    : VideoHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> png)
Press [q] to stop, [?] for help
[h264 @ 00000000003aa100] no picture 
DTS -42, next:-62146 st:0 invalid dropping
[h264 @ 0000000004cdbf00] no picture 
frame=    3 fps=0.0 q=0.0 size=       0kB time=00:00:00.12 bitrate=   0.0kbits/s    
frame=    5 fps=3.1 q=0.0 size=       0kB time=00:00:00.20 bitrate=   0.0kbits/s    
frame=    7 fps=3.0 q=0.0 size=       0kB time=00:00:00.29 bitrate=   0.0kbits/s    

...
 
frame=  237 fps=2.6 q=0.0 size=       0kB time=00:00:09.88 bitrate=   0.0kbits/s    
No more inputs to read from, finishing.
[output stream 0:0 @ 0000000004d73a00] EOF on sink link output stream 0:0:default.
frame=  242 fps=2.6 q=0.0 Lsize=       0kB time=00:00:10.26 bitrate=   0.0kbits/s    

video:276218kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.000000%
[AVIOContext @ 000000000216fd80] Statistics: 4336059 bytes read, 2 seeks

Attachments (12)

tst150rgb24.png (1.1 MB ) - added by y3kcjd5 12 years ago.
yuv420p10le->rgb24 frame #150 (banding/discoloration)
rgb48.png (382.1 KB ) - added by Carl Eugen Hoyos 12 years ago.
yuv420p10le.avi (842.2 KB ) - added by gdgsdg123 5 years ago.
output+log.7z (2.4 MB ) - added by gdgsdg123 5 years ago.
INPUT.7z.001 (2.5 MB ) - added by gdgsdg123 5 years ago.
INPUT.7z.002 (2.3 MB ) - added by gdgsdg123 5 years ago.
OUTPUT.7z.001 (2.5 MB ) - added by gdgsdg123 5 years ago.
OUTPUT.7z.002 (2.5 MB ) - added by gdgsdg123 5 years ago.
OUTPUT.7z.003 (2.5 MB ) - added by gdgsdg123 5 years ago.
OUTPUT.7z.004 (2.5 MB ) - added by gdgsdg123 5 years ago.
OUTPUT.7z.005 (2.5 MB ) - added by gdgsdg123 5 years ago.
OUTPUT.7z.006 (2.0 MB ) - added by gdgsdg123 5 years ago.

Change History (30)

comment:1 by Carl Eugen Hoyos, 12 years ago

Please provide a sample.

by y3kcjd5, 12 years ago

Attachment: tst150rgb24.png added

yuv420p10le->rgb24 frame #150 (banding/discoloration)

comment:2 by y3kcjd5, 12 years ago

OK, seeing as how I can't get the other samples to fit:
Source video clip vtst0.mp4: http://www.datafilehost.com/download-fb229dc6.html
yuv420p10le->rgb48be frame 150 (looks like source): http://www.datafilehost.com/download-bc7b949d.html
yuv420p10le->yuv420p->rgb24 frame 150 (banding no discoloration): http://www.datafilehost.com/download-8a131612.html

by Carl Eugen Hoyos, 12 years ago

Attachment: rgb48.png added

comment:3 by Carl Eugen Hoyos, 12 years ago

Keywords: win64 removed

Just to make sure:
Is the image I attached, rgb48.png ok, but the result of ffmpeg -i rgb48.png -pix_fmt rgb24 out24.png shows the banding?

comment:4 by y3kcjd5, 12 years ago

That is correct.

comment:5 by Carl Eugen Hoyos, 12 years ago

Component: avfilterswscale
Reproduced by developer: set
Status: newopen
Version: unspecifiedgit-master

comment:6 by ff2012, 12 years ago

I have the same problem!
More info: https://ffmpeg.org/trac/ffmpeg/ticket/1611

in reply to:  6 comment:7 by Carl Eugen Hoyos, 12 years ago

Replying to fftest800:

I have the same problem!

More info: https://ffmpeg.org/trac/ffmpeg/ticket/1611

I don't see a conversion from higher bpp to lower bpp of the same type of colourspace in that ticket, why do you think it is a duplicate?

comment:8 by Ridley Combs, 11 years ago

Cc: rodger.combs@gmail.com added

Found this issue while working on some screenshot comparisons; figured I'd post 'em as another reproduction:
rgb48 = frame extracted from H.264 High 10 video with -ss and -vframes 1
rgb24 = same, but with -filter:v "format=rgb24"
VLCSnap = snapshot of same frame using VLC (probably irrelevant)
Preview = rgb48 frame downsampled to rgb24 using OSX's Preview.app (to show that it's possible to downsample this image with significantly less banding)
rgb48 vs rgb24: http://screenshotcomparison.com/comparison/20164
VLCSnap vs rgb24: http://screenshotcomparison.com/comparison/20165
rgb24 vs Preview: http://screenshotcomparison.com/comparison/20166
rgb48 vs Preview: http://screenshotcomparison.com/comparison/20167

[EDIT: 2013-10-02]
More snapshots from the same file? 0.o
New images from a new ffmpeg build:
built on Oct 1 2013 19:49:14 with Apple LLVM version 5.0 (clang-500.1.58) (based on LLVM 3.3svn)
rgb48 output (perfect): https://dl.dropboxusercontent.com/u/46348978/Madoka_Snapshot_02.png
rgb48 output, downsampled to rgb24 using convert (very nice): https://dl.dropboxusercontent.com/u/46348978/Madoka_Snapshot_02_24_cheat.png
rgb24 output (banding, discoloration): https://dl.dropboxusercontent.com/u/46348978/Madoka_Snapshot_02_24.png

Last edited 10 years ago by Ridley Combs (previous) (diff)

comment:9 by Ridley Combs, 10 years ago

$100 to whoever fixes, where "fixed" is defined as "ffmpeg -ss <time> -i <10bit_h264_file> -vframes 1 -pix_fmt rgb24 out.png" produces output with no discernible image-wide discoloration as compared to the same cmdline without the -pix_fmt, and the amount of banding in the image is comparable to that in the same frame downsampled using ImageMagick.

Last edited 10 years ago by Ridley Combs (previous) (diff)

comment:10 by Michael Niedermayer, 10 years ago

Keywords: swscale bounty added

add bounty keyword so this can be found.
See: https://trac.ffmpeg.org/ticket/1582#comment:9

in reply to:  9 ; comment:11 by Carl Eugen Hoyos, 10 years ago

Replying to 11rcombs:

"ffmpeg -ss <time> -i <10bit_h264_file> -vframes 1 -pix_fmt rgb24 out.png" produces output with no discernible image-wide discoloration as compared to the same cmdline without the -pix_fmt, and the amount of banding in the image is comparable to that in the same frame downsampled using ImageMagick.

Could you test the following?

$ ffmpeg -ss <time> -i <10bit_h264_file> -vframes 1 -sws_flags +full_chroma_int -pix_fmt rgb24 out.png

in reply to:  11 comment:12 by Ridley Combs, 10 years ago

Replying to cehoyos:

Could you test the following?
… -sws_flags +full_chroma_int …

Looks outstanding. Can this be made the default behavior, or are there issues with the implementation as swscale.h suggests?

comment:13 by Ridley Combs, 10 years ago

Reducing bounty offer to $10, because it looks like this isn't actually going to require any new dithering code; just a change to default swscale flags in ffmpeg, and that's definitely not worth $100.

comment:14 by y3kcjd5, 10 years ago

New findings:
With "+full_chroma_int" I was able to fix the discoloration, but I was still getting some mild luma banding until I finally figured out to use "-sws_flags full_chroma_int+accurate_rnd", at which point output was finally for all intents and purposes visually indiscernible from input. +1 lobby for this being default behavior.

comment:15 by gdgsdg123, 5 years ago

Sitrep from recent version



Still occurs.


Suggested workaround:

Always add +accurate_rnd+full_chroma_int to sws_flags (by -sws_flags +accurate_rnd+full_chroma_int or sws_flags=+accurate_rnd+full_chroma_int in filtergraphs, or both), unless speed is prefered over quality.





Debug info:


Build from: https://zeranoe.com/builds/win64/static/ffmpeg-20190402-6aeaac3-win64-static.zip

Command:

C:\>ffmpeg -report -i "yuv420p10le.avi" -vframes 1 -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags spline+print_info+accurate_rnd ".\-.png"
C:\>ffmpeg -report -i "yuv420p10le.avi" -vframes 1 -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags spline+print_info+accurate_rnd+full_chroma_int ".\+.png"
C:\>ffmpeg -report -i "yuv420p10le.avi" -vframes 1 -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags spline+print_info ".\--.png"
C:\>ffmpeg -report -i "yuv420p10le.avi" -vframes 1 -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags spline+print_info+full_chroma_int ".\-+.png"

Relevant files attached.



Difference highlight:
(line 93 of the log files)

"+.log"/"-+.log":

[swscaler] chr srcW=720 srcH=418 dstW=1440 dstH=836 xInc=32768 yInc=32768

"-.log"/"--.log":

[swscaler] chr srcW=720 srcH=418 dstW=720 dstH=836 xInc=65536 yInc=32768



Note:
"-.png" and "--.png", "+.png" and "-+.png", are bitexact respectively.

Last edited 5 years ago by gdgsdg123 (previous) (diff)

by gdgsdg123, 5 years ago

Attachment: yuv420p10le.avi added

by gdgsdg123, 5 years ago

Attachment: output+log.7z added

comment:16 by gdgsdg123, 5 years ago

Extended tests have been done regarding this issue.



Command:

C:\>FOR /R "INPUT" %X IN (*) DO ffmpeg -report -i "%X" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +print_info+accurate_rnd+full_chroma_int "%~nX.png"
C:\>FOR /R "INPUT" %X IN (*) DO ffmpeg -report -i "%X" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +print_info+accurate_rnd ".\-full_chroma_int_%~nX.png"
C:\>FOR /R "INPUT" %X IN (*) DO ffmpeg -report -i "%X" -compression_level 10 -pred mixed -pix_fmt rgb24 -sws_flags +print_info+full_chroma_int ".\-accurate_rnd_%~nX.png"

(INPUT: yuv420p, yuv420p10le, yuv422p, yuv422p10le, yuv444p, yuv444p10le)


Files attached.

Influenced files are marked with ! in filename.





Highlight:

line 91 in "!-full_chroma_int_yuv420p.log":

[swscaler] chr srcW=720 srcH=418 dstW=720 dstH=836 xInc=65536 yInc=32768

line 91 in "yuv420p.log":

[swscaler] chr srcW=720 srcH=418 dstW=1440 dstH=836 xInc=32768 yInc=32768

line 92 in "!-accurate_rnd_yuv420p.log":

[swscaler] using unscaled yuv420p -> rgb24 special converter

by gdgsdg123, 5 years ago

Attachment: INPUT.7z.001 added

by gdgsdg123, 5 years ago

Attachment: INPUT.7z.002 added

by gdgsdg123, 5 years ago

Attachment: OUTPUT.7z.001 added

by gdgsdg123, 5 years ago

Attachment: OUTPUT.7z.002 added

by gdgsdg123, 5 years ago

Attachment: OUTPUT.7z.003 added

by gdgsdg123, 5 years ago

Attachment: OUTPUT.7z.004 added

by gdgsdg123, 5 years ago

Attachment: OUTPUT.7z.005 added

by gdgsdg123, 5 years ago

Attachment: OUTPUT.7z.006 added

comment:17 by Carl Eugen Hoyos, 5 years ago

Would you mind stopping to attach compressed files? I believe this is mentioned clearly on https://ffmpeg.org/bugreports.html

in reply to:  17 comment:18 by gdgsdg123, 5 years ago

Replying to cehoyos:

Would you mind stopping to attach compressed files? I believe this is mentioned clearly on https://ffmpeg.org/bugreports.html

Sorry but... any good choice? (and these don't quite fit the definition of "compressed movie files"...)

BTW does that 2.5 MB size limit really exist?.. If not things can be a lot less messy.

Note: See TracTickets for help on using tickets.