Crash in palettegen filter

[byteslice]

Summary of the bug:
FFmpeg, when filtering a video using palettegen, crashes on exit when linked against the latest musl libc due to an out-of-bounds heap write.

Thread 1 "ffmpeg" received signal SIGSEGV, Segmentation fault.
get_nominal_size (end=0x7f738bd41ffc, p=0x7f738b72780) at src/malloc/mallocng/free.c:110
169     src/malloc/mallocng/meta.h: No such file or directory.
(gdb) bt
#0  get_nominal_size (end=0x7f738bd41ffc, p=0x7f738b72780) at src/malloc/mallocng/meta.h:169
#1  __libc_free (p=0x7f738b727280) at src/malloc/mallocng/free.c:110
#2  0x00007f73921a79df in av_buffer_pool_uninit () from /usr/lib/libavutil.so.56
#3  0x00007f7393bff6f8 in avfilter_link_free () from /usr/lib/libavfilter.so.7
#4  0x00007f7393c006f1 in avfilter_free () from /usr/lib/libavfilter.so.7
#5  0x00007f7393c027cc in avfilter_graph_free () from /usr/lib/libavfilter.so.7

How to reproduce:

% ffmpeg -y -i test.webm -vf palettegen test.png

The out-of-bounds writes can be observed when linked against glibc as well when running with valgrind. Example traceback:

==11185== Invalid write of size 8
==11185==    at 0x62C8118: ff_yuv_420_rgb32_ssse3 (yuv_2_rgb.asm:378)
==11185==    by 0x90DF07F: ???
==11185==    by 0x91431BF: ???
==11185==  Address 0xb359898 is 6,400,024 bytes inside a block of size 6,400,031 alloc'd
==11185==    at 0x48A709E: memalign (vg_replace_malloc.c:1267)
==11185==    by 0x48A7195: posix_memalign (vg_replace_malloc.c:1432)
==11185==    by 0x6317B14: av_malloc (mem.c:86)
==11185==    by 0x6304F08: av_buffer_alloc (buffer.c:72)
==11185==    by 0x6304F7D: av_buffer_allocz (buffer.c:85)
==11185==    by 0x6305724: pool_alloc_buffer (buffer.c:352)
==11185==    by 0x6305724: av_buffer_pool_get (buffer.c:388)
==11185==    by 0x49EBB28: ff_frame_pool_get (framepool.c:222)
==11185==    by 0x4B6DA5F: ff_default_get_video_buffer (video.c:90)
==11185==    by 0x4AF9201: scale_frame (vf_scale.c:731)
==11185==    by 0x4AF98D5: filter_frame (vf_scale.c:820)
==11185==    by 0x49C8A88: ff_filter_frame_framed (avfilter.c:1085)
==11185==    by 0x49C8A88: ff_filter_frame_to_filter (avfilter.c:1233)
==11185==    by 0x49C8A88: ff_filter_activate_default (avfilter.c:1282)
==11185==    by 0x49C8A88: ff_filter_activate (avfilter.c:1441)
==11185==    by 0x49CD1DF: push_frame (buffersrc.c:157)
==11185==    by 0x49CD1DF: av_buffersrc_add_frame_flags (buffersrc.c:225)

This crash may be fixed by removing an off-by-one adjustment in libavfilter/framepool.c:

-        pool->pools[i] = av_buffer_pool_init(pool->linesize[i] * h + 16 + 16 - 1,
+        pool->pools[i] = av_buffer_pool_init(pool->linesize[i] * h + 16 + 16,

[Elon Musk]

Fixed in 17a59a634c39b00a680c6ebbaea58db95594d13d