Opened 4 years ago

Closed 2 years ago

#8463 closed defect (wontfix)

Screen blending regression

Reported by: antmarot Owned by:
Priority: normal Component: avfilter
Version: git-master Keywords: blend
Cc: Marton Balint Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

Incorrect resulting colors when using screen blending.
Looks like a regression since it works well in ffmpeg version 3.2.10 but does not work in ffmpeg version 4.2.1.

How to reproduce:

% ffmpeg -filter_complex "color=green:size=hd720,format=rgba[a]; color=black:size=hd720,format=rgba[b]; [a][b] blend=all_mode=screen,format=yuv422p10le" -pix_fmt yuv420p -t 1 /tmp/test.mp4

The previous command should produce a green video but the result is purple (v4.2.1).

v4.2.1 output:

ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.8)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.1_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-13.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-13.jdk/Contents/Home/include/darwin -fno-stack-check' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Stream mapping:
  format -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[libx264 @ 0x7ff359001200] using SAR=1/1
[libx264 @ 0x7ff359001200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7ff359001200] profile High, level 3.1
[libx264 @ 0x7ff359001200] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=22 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/tmp/test.mp4':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame=   25 fps=0.0 q=-1.0 Lsize=       5kB time=00:00:00.88 bitrate=  41.9kbits/s speed=2.52x    
video:3kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 33.352619%
[libx264 @ 0x7ff359001200] frame I:1     Avg QP:16.00  size:   672
[libx264 @ 0x7ff359001200] frame P:6     Avg QP:17.00  size:   237
[libx264 @ 0x7ff359001200] frame B:18    Avg QP:19.67  size:    37
[libx264 @ 0x7ff359001200] consecutive B-frames:  4.0%  0.0%  0.0% 96.0%
[libx264 @ 0x7ff359001200] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x7ff359001200] mb P  I16..4:  0.0%  0.0%  0.0%  P16..4: 16.3%  0.0%  0.0%  0.0%  0.0%    skip:83.7%
[libx264 @ 0x7ff359001200] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.0%  0.0%  0.0%  direct: 0.0%  skip:100.0%
[libx264 @ 0x7ff359001200] 8x8 transform intra:0.0%
[libx264 @ 0x7ff359001200] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 4.1% 0.0%
[libx264 @ 0x7ff359001200] i16 v,h,dc,p: 98%  0%  2%  0%
[libx264 @ 0x7ff359001200] i8c dc,h,v,p: 100%  0%  0%  0%
[libx264 @ 0x7ff359001200] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7ff359001200] kb/s:22.14

v3.2.10 output:

ffmpeg version 3.2.10-1~deb9u1~bpo8+1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 4.9.2 (Debian 4.9.2-10)
  configuration: --prefix=/usr --extra-version='1~deb9u1~bpo8+1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --disable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
File '/tmp/test.mp4' already exists. Overwrite ? [y/N] y
[libx264 @ 0x55cab3d60cc0] using SAR=1/1
[libx264 @ 0x55cab3d60cc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
[libx264 @ 0x55cab3d60cc0] profile High, level 3.1
[libx264 @ 0x55cab3d60cc0] 264 - core 142 r2431 a5831aa - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/tmp/test.mp4':
  Metadata:
    encoder         : Lavf57.56.101
    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.64.101 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
  format -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
frame=   25 fps= 16 q=-1.0 Lsize=       3kB time=00:00:00.88 bitrate=  31.3kbits/s speed=0.572x
video:2kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 50.349346%
[libx264 @ 0x55cab3d60cc0] frame I:1     Avg QP:15.00  size:   672
[libx264 @ 0x55cab3d60cc0] frame P:6     Avg QP:16.83  size:    43
[libx264 @ 0x55cab3d60cc0] frame B:18    Avg QP:19.67  size:    37
[libx264 @ 0x55cab3d60cc0] consecutive B-frames:  4.0%  0.0%  0.0% 96.0%
[libx264 @ 0x55cab3d60cc0] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x55cab3d60cc0] mb P  I16..4:  0.0%  0.0%  0.0%  P16..4:  0.0%  0.0%  0.0%  0.0%  0.0%    skip:100.0%
[libx264 @ 0x55cab3d60cc0] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.0%  0.0%  0.0%  direct: 0.0%  skip:100.0%
[libx264 @ 0x55cab3d60cc0] 8x8 transform intra:0.0%
[libx264 @ 0x55cab3d60cc0] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0%
[libx264 @ 0x55cab3d60cc0] i16 v,h,dc,p: 98%  0%  2%  0%
[libx264 @ 0x55cab3d60cc0] i8c dc,h,v,p: 100%  0%  0%  0%
[libx264 @ 0x55cab3d60cc0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x55cab3d60cc0] kb/s:12.81

Change History (7)

comment:1 by Carl Eugen Hoyos, 4 years ago

Keywords: regression added
Version: 4.2unspecified

Is the issue reproducible with current FFmpeg git head, the only version supported here?

comment:2 by antmarot, 4 years ago

Version: unspecifiedgit-master

Just checked and yes, it's reproducible with current FFmpeg git head.

comment:3 by pdr0, 4 years ago

The blend operation in RGB is ok. The problem is inaccurate RGB => YUV conversion

If you test output RGB24, the output is "green" or the expected 0,128,0

ffmpeg -filter_complex "color=green:size=hd720,format=rgba[a]; color=black:size=hd720,format=rgba[b]; [a][b] blend=all_mode=screen,format=rgb24" -vframes 1 rgb24.png -y

To increase accuracy of RGB<=>YUV conversions, either use swsflags with -vf scale, or zscale, with appropriate matrix (709 for HD is convention)

eg. swsflags

ffmpeg -filter_complex "color=green:size=hd720,format=rgba[a]; color=black:size=hd720,format=rgba[b]; [a][b] blend=all_mode=screen,scale=out_color_matrix=bt709:flags=bicubic+accurate_rnd+full_chroma_int+full_chroma_inp,format=yuv420p" -t 1 test_swsflags.mp4 -y

eg. zscale

ffmpeg -filter_complex "color=green:size=hd720,format=rgba[a]; color=black:size=hd720,format=rgba[b]; [a][b] blend=all_mode=screen,zscale=matrix=709,format=yuv420p" -t 1 test_zscale.mp4 -y

comment:4 by antmarot, 4 years ago

Thank you pdr0. I confirm it now produces the expected outputs.
Out of curiosity, does it mean that it is the RGB => YUV conversion that has regressed or it's the expected behavior?

comment:5 by Marton Balint, 4 years ago

Cc: Marton Balint added

It is not a precision issue. The result depends on which pixel format the blend filter use, RGB or YUV. Blending in YUV gets you purple, blending in RGB gets you green. The automagically selected format can change (depending on what formats the filters support) between versions.

So this is not necessarily a bug. Admittedly some blending modes makes little sense for YUV, and the filter gives you too much freedom here. Maybe an option should be added to the filter so you can limit the formats to RGB only to always get the expected results?

comment:6 by Elon Musk, 4 years ago

Keywords: regression removed

comment:7 by Elon Musk, 2 years ago

Resolution: wontfix
Status: newclosed
Note: See TracTickets for help on using tickets.