heap-buffer-overflow at libavfilter/vf_bitplanenoise.c

[Suhwan]

Summary of the bug:
There is a heap-buffer-overflow at libavfilter/vf_bitplanenoise.c:174 in filter_frame.
I compiled ffmpeg with "--toolchain=clang-asan" to check the heap buffer overflow and attached log file.

How to reproduce:

% ffmpeg_g -y -r 109 -i $PoC -filter_complex bitplanenoise -target svcd -loglevel 99 -map 0 -c:s:9 jpeg2000 -aframes 55 -ab 878k -ar 48000 -b:v 762k tmp.s16be

ffmpeg version N-95314-g1331e00179 Copyright (c) 2000-2019 the FFmpeg developers
built with clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
configuration: --cc=clang --cxx=clang++ --ld=clang --enable-debug --toolchain=clang-asan

Here's ASAN log

==47006==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x619000008262 at pc 0x0000009b6138 bp 0x7fffffffa5b0 sp 0x7fffffffa5a8
READ of size 2 at 0x619000008262 thread T0
    #0 0x9b6137 in filter_frame ffmpeg/libavfilter/vf_bitplanenoise.c:174:13
    #1 0x827289 in ff_filter_activate_default ffmpeg/libavfilter/avfilter.c:1071:11
    #2 0x827289 in ff_filter_activate ffmpeg/libavfilter/avfilter.c:1430
    #3 0x870182 in push_frame ffmpeg/libavfilter/buffersrc.c:187:15
    #4 0x870182 in av_buffersrc_add_frame_internal ffmpeg/libavfilter/buffersrc.c:261
    #5 0x86ebc2 in av_buffersrc_add_frame_flags ffmpeg/libavfilter/buffersrc.c:170:16
    #6 0x666867 in ifilter_send_frame ffmpeg/fftools/ffmpeg.c:2196:11
    #7 0x666867 in send_frame_to_filters ffmpeg/fftools/ffmpeg.c:2270
    #8 0x6075f7 in decode_video ffmpeg/fftools/ffmpeg.c:2469:11
    #9 0x6075f7 in process_input_packet ffmpeg/fftools/ffmpeg.c:2623
    #10 0x6450b8 in process_input ffmpeg/fftools/ffmpeg.c:4313:23
    #11 0x5e7157 in transcode_step ffmpeg/fftools/ffmpeg.c:4638:11
    #12 0x5e7157 in transcode ffmpeg/fftools/ffmpeg.c:4692
    #13 0x5db65b in main ffmpeg/fftools/ffmpeg.c:4894:9
    #14 0x7ffff5c93b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
    #15 0x41def9 in _start (ffmpeg/ffmpeg_g+0x41def9)

0x619000008262 is located 30 bytes to the left of 1055-byte region [0x619000008280,0x61900000869f)
allocated by thread T0 here:
    #0 0x4de9e8 in posix_memalign (ffmpeg/ffmpeg_g+0x4de9e8)
    #1 0x8564fb1 in av_malloc ffmpeg/libavutil/mem.c:87:9
    #2 0x84cc231 in av_buffer_alloc ffmpeg/libavutil/buffer.c:72:12
    #3 0x84cc231 in av_buffer_allocz ffmpeg/libavutil/buffer.c:85
    #4 0x84d0a56 in pool_alloc_buffer ffmpeg/libavutil/buffer.c:313:26
    #5 0x84d0a56 in av_buffer_pool_get ffmpeg/libavutil/buffer.c:349
    #6 0x91af8d in ff_frame_pool_get ffmpeg/libavfilter/framepool.c:222:29
    #7 0x15a660c in ff_default_get_video_buffer ffmpeg/libavfilter/video.c:90:13
    #8 0x124c7f9 in scale_frame ffmpeg/libavfilter/vf_scale.c:460:11
    #9 0x124a8ec in filter_frame ffmpeg/libavfilter/vf_scale.c:549:11
    #10 0x827289 in ff_filter_activate_default ffmpeg/libavfilter/avfilter.c:1071:11
    #11 0x827289 in ff_filter_activate ffmpeg/libavfilter/avfilter.c:1430
    #12 0x870135 in push_frame ffmpeg/libavfilter/buffersrc.c:187:15
    #13 0x870135 in av_buffersrc_add_frame_internal ffmpeg/libavfilter/buffersrc.c:261
    #14 0x86ebc2 in av_buffersrc_add_frame_flags ffmpeg/libavfilter/buffersrc.c:170:16
    #15 0x666867 in ifilter_send_frame ffmpeg/fftools/ffmpeg.c:2196:11
    #16 0x666867 in send_frame_to_filters ffmpeg/fftools/ffmpeg.c:2270
    #17 0x6075f7 in decode_video ffmpeg/fftools/ffmpeg.c:2469:11
    #18 0x6075f7 in process_input_packet ffmpeg/fftools/ffmpeg.c:2623
    #19 0x6450b8 in process_input ffmpeg/fftools/ffmpeg.c:4313:23
    #20 0x5e7157 in transcode_step ffmpeg/fftools/ffmpeg.c:4638:11
    #21 0x5e7157 in transcode ffmpeg/fftools/ffmpeg.c:4692
    #22 0x5db65b in main ffmpeg/fftools/ffmpeg.c:4894:9
    #23 0x7ffff5c93b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310

SUMMARY: AddressSanitizer: heap-buffer-overflow ffmpeg/libavfilter/vf_bitplanenoise.c:174:13 in filter_frame

Please confirm.
Thanks

[Suhwan]

poc