Cannot draw opaque text on transparent frame

[Andrey Utkin]

Summary of the bug:
Below testcase may seem somewhat artifical and impractical.
But actually with ffmpeg we can save transparent video (e.g. -vcodec png -pix_fmt rgba out.mov )
So let's say we want to render a video with opaque text over transparent background, for later overlaying. It ends out with the fact that we cannot: with background alpha = 0 and text alpha = 1, nothing is seen on image. We must increase background alpha, which not acceptable for desired result.

How to reproduce:

ffmpeg -f lavfi -i "testsrc[testsrc];color=color=white@0.0,drawtext=text=TEST:fontsize=50:fontfile=/usr/share/fonts/corefonts/verdana.ttf:fontcolor=red@1.0[text];[testsrc][text]overlay" /tmp/tmp.ts
ffmpeg version N-59815-gb79bccb Copyright (c) 2000-2014 the FFmpeg developers
  built on Jan 14 2014 03:29:50 with gcc 4.6.3 (Gentoo 4.6.3 p1.13, pie-0.5.2)
  configuration: --enable-gpl --enable-libx264 --enable-encoder=libx264 --disable-stripping --enable-debug --extra-cflags='-O0 -g -ggdb' --enable-libopus --enable-libvpx --enable-x11grab --enable-libfreetype --enable-filter=drawtext --enable-libzvbi
  libavutil      52. 62.100 / 52. 62.100
  libavcodec     55. 48.101 / 55. 48.101
  libavformat    55. 23.103 / 55. 23.103
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     4.  1.100 /  4.  1.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Input #0, lavfi, from 'testsrc[testsrc];color=color=white@0.0,drawtext=text=TEST:fontsize=50:fontfile=/usr/share/fonts/corefonts/verdana.ttf:fontcolor=red@1.0[text];[testsrc][text]overlay':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (Y4[11][8] / 0x80B3459), yuva420p, 320x240 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc
File '/tmp/tmp.ts' already exists. Overwrite ? [y/N] y
Output #0, mpegts, to '/tmp/tmp.ts':
  Metadata:
    encoder         : Lavf55.23.103
    Stream #0:0: Video: mpeg2video, yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo -> mpeg2video)
Press [q] to stop, [?] for help
frame= 2313 fps=750 q=9.4 Lsize=    3047kB time=00:01:32.48 bitrate= 269.9kbits/s    
video:2538kB audio:0kB subtitle:0 global headers:0kB muxing overhead 20.054615%
Received signal 2: terminating.

[Carl Eugen Hoyos]

Replying to Krieger:

Below testcase may seem somewhat artifical and impractical.

I consider it useful.

[Andrey Utkin]

This issue is still actual. Drawing a text over transparent background doesn't work, although should.

This bug may be related to ​http://trac.ffmpeg.org/ticket/3317

Seems the problem is that ff_blend_mask() from drawutils.c treats alpha, the 4th component on the 1st plane of dst, just like all other components (R, G, B). And, for example, in case of

ffmpeg -f lavfi -i "color=color=white@0.0:size=2x2,format=pix_fmts=rgba,drawtext=text=TEST:fontsize=50:fontfile=/usr/share/fonts/corefonts/verdana.ttf:fontcolor=red@1.0,format=pix_fmts=rgba" -vframes 1 -vcodec png /tmp/tmp.png

in the end of this routine i have such contents:

Breakpoint 3, ff_blend_mask (draw=0x19e91d0, color=0x19e9200, dst=0x19f6580, dst_linesize=0x19f65c0, dst_w=2, dst_h=2, mask=0x19f6c90 '\377' <repeats 124 times>, mask_linesize=31, 
    mask_w=2, mask_h=1, l2depth=3, endianness=0, x0=0, y0=1) at libavfilter/drawutils.c:504
504     }
(gdb) x/8xb dst[0]
0x19f6840:      0xff    0xff    0xff    0x00    0xff    0xff    0xff    0x00
(gdb) x/8xb (dst[0] + dst_linesize[0])
0x19f6860:      0xff    0x00    0x00    0x00    0xff    0x00    0x00    0x00
(gdb) 

It shows that topmost line is 2 white pixels (background color), and the bottom line is 2 red pixels (color of text). However, red-colored pixels have alpha = 0 and the whole picture is represented as fully transparent.

It is possible to parametrize the algorithm for resulting alpha. So that old behaviour can be preserved, but also we get a new mode to preserve alpha value of mask (i.e. ignore background alpha), which can be desired in some cases; more strategies can be added.

[Andrey Utkin]

It turned out to be not easy.
After fighting with drawutils, which doesn't work with dst transparency by default, i'm getting inconsistent results, will show in attachments.

Seems at the moment there's no suitable mechanism in FFmpeg for subject feature, and modifying drawutils code is quite tricky. Any comments and propositions are appreciated.

[Andre Wachsmuth]

For everybody coming across this issue as well as I did now, here is a workaround for now.

ffmpeg -f lavfi -i color=color=0x000000@1.0,format=yuva444p -vf "drawtext=shadowx=10:shadowy=20:shadowcolor=white:x=100:y=100:borderw=3:bordercolor=white:fontsize=40:text=Hello:fontcolor=0xFFFFFF@1.0,colorchannelmixer=rr=0:rg=0:rb=0:ra=0:gr=0:gg=0:gb=0:ga=0:br=0:bg=0:bb=0:ba=0:ar=2.0:ag=2.0:ab=2.0:aa=0,drawtext=text=Hello:fontcolor=yellow:fontsize=40:x=100:y=100:borderw=3:bordercolor=black:shadowx=10:shadowy=20:shadowcolor=green" -vframes 1 -f image2 /tmp/out.png

This produces a transparent image with a yellow text, a black border and a green shadow and saves it to /tmp/out.png

Draws the text once in white on a black background, uses the colorchannelmixer filter to extract an alpha mask, and then draws the text again.

[Andre Wachsmuth]

workaround for this issue by drawing the text twice

[Andrey Utkin]

Thanks for this hardcore workaround, but it is not flawless because when you overlay the resulting text over light color, you have "dirt" at the edge of letters because of hinting. See

ffmpeg -f lavfi -i color=color=0x000000@1.0,format=yuva444p -vf "drawtext=fontfile=/usr/share/fonts/corefonts/cour.ttf:x=100:y=100:fontsize=40:text=Hello:fontcolor=0xFFFFFF@1.0,colorchannelmixer=rr=0:rg=0:rb=0:ra=0:gr=0:gg=0:gb=0:ga=0:br=0:bg=0:bb=0:ba=0:ar=2.0:ag=2.0:ab=2.0:aa=0,drawtext=fontfile=/usr/share/fonts/corefonts/cour.ttf:text=Hello:fontcolor=yellow:fontsize=40:x=100:y=100" -vframes 1 -f image2 /tmp/out.png

[Andrey Utkin]

Replying to andrey.utkin:

Thanks for this hardcore workaround, but it is not flawless because when you overlay the resulting text over light color, you have "dirt" at the edge of letters because of hinting. See

ffmpeg -f lavfi -i color=color=0x000000@1.0,format=yuva444p -vf "drawtext=fontfile=/usr/share/fonts/corefonts/cour.ttf:x=100:y=100:fontsize=40:text=Hello:fontcolor=0xFFFFFF@1.0,colorchannelmixer=rr=0:rg=0:rb=0:ra=0:gr=0:gg=0:gb=0:ga=0:br=0:bg=0:bb=0:ba=0:ar=2.0:ag=2.0:ab=2.0:aa=0,drawtext=fontfile=/usr/share/fonts/corefonts/cour.ttf:text=Hello:fontcolor=yellow:fontsize=40:x=100:y=100" -vframes 1 -f image2 /tmp/out.png

ft_load_flags=monochrome+no_hinting+render seems to fix this.

[Elon Musk]

Fixed in 9d16e46d9eb4643ce119e26ed5c3975cb6cb7aa1.

[Carl Eugen Hoyos]

No text visible here with the original command line:

$ ffmpeg -f lavfi -i "testsrc[testsrc];color=color=white@0.0,drawtext=text=TEST:fontsize=50:fontfile=/usr/share/fonts/truetype/DejaVuSans.ttf:fontcolor=red@1.0[text];[testsrc][text]overlay" -vframes 1 out.ts
ffmpeg version N-81709-g492259c Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --enable-libfreetype
  libavutil      55. 30.100 / 55. 30.100
  libavcodec     57. 57.101 / 57. 57.101
  libavformat    57. 50.100 / 57. 50.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 62.100 /  6. 62.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Input #0, lavfi, from 'testsrc[testsrc];color=color=white@0.0,drawtext=text=TEST:fontsize=50:fontfile=/usr/share/fonts/truetype/DejaVuSans.ttf:fontcolor=red@1.0[text];[testsrc][text]overlay':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (Y4[11][8] / 0x80B3459), yuva420p, 320x240 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc
[mpegts @ 0x39e7c80] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf57.50.100
    Stream #0:0: Video: mpeg2video (Main), yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 90k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.57.101 mpeg2video
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> mpeg2video (native))
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=4.7 Lsize=       9kB time=00:00:00.00 bitrate=6699636.4kbits/s speed=0.00697x
video:8kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 16.504364%

[Carl Eugen Hoyos]

Sorry for the incorrect test, it works fine now!