Opened 2 years ago

Closed 2 years ago

#9750 closed defect (fixed)

overlay filter alpha premultiplied performs brighter on transparent images

Reported by: windowsair Owned by:
Priority: normal Component: avfilter
Version: git-master Keywords:
Cc: windowsair Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by windowsair)

Summary of the bug:

I'm using ffmpeg's overlay filter with alpha premultiplied parameter. I noticed that when a completely transparent image was overlay to the background video, the output video was much brighter.

In the following demo, I will use RGBA8888 format images for overlay. Its use of RGBA (255,255,255,0) indicates full transparency (aka completely transparent white), and the result demonstrates a very noticeable brightening (the expected result should be unchanged, since the image is completely transparent).

ps: Using RGBA (0,0,0,0) (aka completely transparent black) also results in a brightened image.

How to reproduce:

% ffmpeg.exe  -loglevel debug -i a1.mp4 -i a1.png -filter_complex "[0:v][1:v]overlay=x=0:y=0:alpha=premultiplied[v]" -map "[v]"  -c:v:0 libx264 -f mp4  output.mp4

ffmpeg output

ffmpeg version n5.0.1-3-gb655beb025-20220425 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (crosstool-NG 1.24.0.533_681aaef)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --enable-shared --disable-static --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20220425
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100

The complete output is given in the file ffmpeg.log

How to reproduce (Using transparent black)

This command uses lavfi to simulate the transparent black output of RGBA. For demonstration purposes only, you may need ctrl+c to prevent infinite output.

% ffmpeg.exe -y -i a1.mp4 -f lavfi -i "color=color=black@0.0:size=1920x540,format=rgba" -filter_complex "[0:v][1:v]overlay=x=0:y=0:alpha=premultiplied[v]" -map "[v]"  -c:v:0 libx264 -f mp4  output.mp4

The results show that the upper part is more whitish (brighter).

Attachments (8)

a1.mp4 (1.4 MB ) - added by windowsair 2 years ago.
a1.png (8.4 KB ) - added by windowsair 2 years ago.
output.mp4 (210.0 KB ) - added by windowsair 2 years ago.
ffmpeg.log (345.1 KB ) - added by windowsair 2 years ago.
compare.png (1.2 MB ) - added by windowsair 2 years ago.
a.hex.zip (13.6 KB ) - added by windowsair 2 years ago.
a.hex RGBA input
wrong_output.png (355.7 KB ) - added by windowsair 2 years ago.
expected_output.png (342.4 KB ) - added by windowsair 2 years ago.

Change History (20)

by windowsair, 2 years ago

Attachment: a1.mp4 added

by windowsair, 2 years ago

Attachment: a1.png added

by windowsair, 2 years ago

Attachment: output.mp4 added

by windowsair, 2 years ago

Attachment: ffmpeg.log added

comment:1 by windowsair, 2 years ago

Description: modified (diff)

comment:2 by windowsair, 2 years ago

Description: modified (diff)
Priority: normalimportant

comment:3 by Elon Musk, 2 years ago

Priority: importantnormal

Are you sure this is not your wrong use of filters.

For example what happens if you change overlay parameter to not use default yuv colorspace.

comment:4 by windowsair, 2 years ago

Hi, thanks for your quick reply!

As per your instructions, I tested different color spaces and some of the results are as follows:

yuv420, yuv422, yuv444: Consistent with the original results.

yuv420p10, yuv422p10: Not the result of alpha premultiplied, the result is the same as using alpha straight.

It's a little weird.

by windowsair, 2 years ago

Attachment: compare.png added

by windowsair, 2 years ago

Attachment: a.hex.zip added

a.hex RGBA input

in reply to:  3 comment:5 by windowsair, 2 years ago

Replying to Elon Musk:

Are you sure this is not your wrong use of filters.

For example what happens if you change overlay parameter to not use default yuv colorspace.

Here I want to provide you with more specific test steps and results comparison charts to explain the problem. My original input is a picture in the RGBA8888 format, which is represented by "a.hex". Among them, 4byte's 0x00, 0x00, 0x00, 0x00 represents completely transparent black.

I use the following command:

ffmpeg.exe  -i a1.mp4 -f rawvideo -s 1920x540 -pix_fmt rgba -i a.hex -filter_complex "[0:v][1:v]overlay=x=0:y=0:alpha=premultiplied[v]" -map "[v]"  -c:v:0 libx264 -f mp4  output.mp4

Similarly, I test the following three cases in total.

  1. alpha=premultiplied
  2. alpha=premultiplied:format=yuv420p10
  3. alpha=straight

Test Result:
https://trac.ffmpeg.org/raw-attachment/ticket/9750/compare.png

comment:6 by Elon Musk, 2 years ago

Why you havent tried rgb or gbrp formats for overlay?

in reply to:  6 comment:7 by windowsair, 2 years ago

Replying to Elon Musk:

Why you havent tried rgb or gbrp formats for overlay?

ok, here are the test results.

gbrp, png input: completely white
gbrp, lavfi/raw input : ok

rgb, png input : transparency seems incorrect
rgb, lavfi/raw input: ok

Oh, I'm not sure if this is a problem.

comment:8 by Elon Musk, 2 years ago

Please provide original input png files, and expected ooutput. Do not post ffmpeg output.

comment:9 by windowsair, 2 years ago

Let's first focus on the RGBA file "a.hex" as rawvideo input. (I have uploaded that before)

  1. get the expected output:
ffmpeg.exe  -i a1.mp4 -f rawvideo -s 1920x540 -pix_fmt rgba -i a.hex -filter_complex "[0:v][1:v]overlay=x=0:y=0:alpha=premultiplied:format=rgb[v]" -map "[v]"  -c:v:0 libx264 -f mp4  output.mp4
  1. get the wrong output (black becomes bright)
ffmpeg.exe  -i a1.mp4 -f rawvideo -s 1920x540 -pix_fmt rgba -i a.hex -filter_complex "[0:v][1:v]overlay=x=0:y=0:alpha=premultiplied:format=yuv420[v]" -map "[v]"  -c:v:0 libx264 -f mp4  output.mp4

Maybe, as you said, there is a loss of accuracy in the different pixel format conversions.

But when I first set the output format of overlay to rgb and then convert it back to yuv420p, I get the correct output as expected. I use the following command:

ffmpeg.exe  -i a1.mp4 -i imagename.png -filter_complex "[0:v][1:v]overlay=x=0:y=0:format=rgb:alpha=premultiplied[v0];[v0]format=yuv420p[v]" -map "[v]"  -c:v:0 libx264 -f mp4  output.mp4

Sorry, I don't know anything about FFmpeg. Hope this helps!

by windowsair, 2 years ago

Attachment: wrong_output.png added

by windowsair, 2 years ago

Attachment: expected_output.png added

comment:10 by Elon Musk, 2 years ago

a.hex is nothing like the output you post. Its mainly black/transparent.

comment:11 by windowsair, 2 years ago

a.hex is 1920x540 transparent background + semi-transparent text input image.

The top half of the screen is overlayed in the output, and the bottom half of the screen is the original input that is not overlayed.

comment:12 by Elon Musk, 2 years ago

Resolution: fixed
Status: newclosed

Fixed in master.

Note: See TracTickets for help on using tickets.