WebP vs PNG alpha

[Jozef Chutka]

using ffmpeg version 2022-07-31-git-1368b5a725-full_build-www.gyan.dev

Attached x.png and x.webp (webp created from png) have transparent background covered by white X shape.

For some reason FFmpeg scale filter treats the x.png and x.webp differently.

Applying scale filter on the x.webp, causes the output video alpha around the X shape to be premultiplied and so renders with ugly edges.

Following commands, esp. out-webp-scale.png output demonstrates the issue:

ffmpeg -filter_complex "color=white:200x200[b];movie=x.png,[b]overlay" -y out-png.png

ffmpeg -filter_complex "color=white:200x200[b];movie=x.png,scale=200:200,[b]overlay" -y out-png-scale.png

ffmpeg -filter_complex "color=white:200x200[b];movie=x.webp,[b]overlay" -y out-webp.png

ffmpeg -filter_complex "color=white:200x200[b];movie=x.webp,scale=200:200,[b]overlay" -y out-webp-scale.png

I tried all possible scale filter flags, and also format changing (before and after scale) but no luck to achieve the same result. I am able to un-premultipy the problematic one with unpremultiply filter but I would like to avoid it.

I wonder if this is a bug or an expected behaviour.

[Carl Eugen Hoyos]

Are you reporting that different input files lead to different output files?

Or that the scale filter handles different input pixel formats differently? If this is the case, using rawvideo input probably makes the issue easier to understand.

In any case, please understand that valid tickets on this bug tracker show a command line including the complete, uncut console output.

[Jozef Chutka]

Hi Carl,

files are different in format png vs webp, however the content/pixels it contains is the same. I am reporting that the scale filter produces different output for such setup (same pixels, different input format).

Can you please elaborate how can I use rawvideo in my case?

Attaching console output of the problematic command:

ffmpeg -filter_complex "color=white:200x200[b];movie=x.webp,scale=200:200,[b]overlay" -y out-webp-scale.png
ffmpeg version 2022-07-31-git-1368b5a725-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57. 30.100 / 57. 30.100
  libavcodec     59. 40.100 / 59. 40.100
  libavformat    59. 29.100 / 59. 29.100
  libavdevice    59.  8.101 / 59.  8.101
  libavfilter     8. 46.101 /  8. 46.101
  libswscale      6.  8.101 /  6.  8.101
  libswresample   4.  8.100 /  4.  8.100
  libpostproc    56.  7.100 / 56.  7.100
Stream mapping:
  overlay:default -> Stream #0:0 (png)
Press [q] to stop, [?] for help
Output #0, image2, to 'out-webp-scale.png':
  Metadata:
    encoder         : Lavf59.29.100
  Stream #0:0: Video: png, rgba(pc, gbr/unknown/unknown, progressive), 200x200 [SAR 1:1 DAR 1:1], q=2-31, 200 kb/s, 25 fps, 25 tbn
    Metadata:
      encoder         : Lavc59.40.100 png
[Parsed_movie_1 @ 0000023dfd060640] EOF timestamp not reliable7 bitrate=  -0.0kbits/s speed=N/A
[image2 @ 0000023dfd056680] The specified filename 'out-webp-scale.png' does not contain an image sequence pattern or a pattern is invalid.
[image2 @ 0000023dfd056680] Use a pattern such as %03d for an image sequence or use the -update option (with -frames:v 1 if needed) to write a single image.
[image2 @ 0000023dfd056680] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
av_interleaved_write_frame(): Invalid argument
Error muxing a packet for output file #0
frame=    2 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.60 bitrate=N/A speed=21.4x
video:7kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

[Jozef Chutka]

The workaround is to create yet another color=#ffffff00 and overlay with the webp before the scale and the following filter graph. (movie replaced by color+movie+overlay):

ALPHA PROBLEM:
ffmpeg -filter_complex "color=white:200x200[b];movie=x.webp,scale=200:200,[b]overlay" -y out-webp-scale.png

ALPHA FIXED:
ffmpeg -filter_complex "color=white:200x200[b];color=color=#ffffff00:size=100x100[c];movie=x.webp,[c]overlay,scale=200:200,[b]overlay" -y out-webp-scale-fix.png
scale-fix.png

This makes me think that there might some strange format used for webp movie, which can not be enforced/changed by format filter, but can be changed/fixed by overlay-ing it to artificial color

[Jozef Chutka]

I have attached new x.png and x.webp which is better for demonstration - not completely white, yet still demonstrates the alpha issue around the white edges

[Jozef Chutka]

After some more testing (and updated .webp and .png exports from my graphic program) it appears the issue not related to webp or pixel format itself, but rather related to the RGB values of the semi-transparent pixel and scale filter. While the shape edge is rendered correctly without scale filter used, once scale is used, the alpha of the edge pixels gets "premultiplied".

Could it be that scale filter handles rgb separately from alpha channel, running effectively the scaling algorithm twice and then merging rgb+alpha resulting in alpha artifacts?