signed integer overflow in decode_coeffs_b_generic()

[tsmith]

Summary of the bug:
UBSan: libavcodec/vp9.c:2255:13: runtime error: signed integer overflow: -262210 * 18766 cannot be represented in type 'int'

How to reproduce:
Built with Undefined behavior sanitizer (-fsanitize=undefined)

% ffmpeg -f ivf -i <test_case> -f null -
ffmpeg version N-83756-g8aa4f3b Copyright (c) 2000-2017 the FFmpeg developers
  built with clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
  configuration: --cc=afl-clang-fast --cxx=afl-clang-fast++ --disable-libxcb --disable-xlib --disable-logging --disable-ffprobe --disable-ffplay --disable-sdl2 --disable-ffserver --disable-doc --disable-pthreads --disable-network --disable-d3d11va --disable-dxva2 --disable-vaapi --disable-vda --disable-vdpau --disable-stripping --disable-runtime-cpudetect --disable-postproc --disable-securetransport --disable-iconv --disable-swscale --disable-avdevice --disable-videotoolbox --disable-everything --enable-encoder=wrapped_avframe --disable-lzma --enable-protocol='file,pipe' --enable-muxer=null --enable-demuxer=ivf --enable-parser=vp9 --enable-decoder=vp9 --disable-pic

libavcodec/vp9.c:2255:13: runtime error: signed integer overflow: -262210 * 18766 cannot be represented in type 'int'
    #0 0x9610af in decode_coeffs_b_generic libavcodec/vp9.c:2255:13
    #1 0x9610af in decode_coeffs_b_16bpp libavcodec/vp9.c:2289
    #2 0x8fb97a in decode_coeffs libavcodec/vp9.c:2408:9
    #3 0x8fb97a in decode_coeffs_16bpp libavcodec/vp9.c:2477
    #4 0x80377d in decode_b libavcodec/vp9.c:3240:30
    #5 0x7e632f in decode_sb libavcodec/vp9.c:3476:13
    #6 0x7e6b50 in decode_sb libavcodec/vp9.c:3444:17
    #7 0x7b1de2 in vp9_decode_frame libavcodec/vp9.c:4204:29
    #8 0x78369a in avcodec_decode_video2 libavcodec/utils.c:2263:19
    #9 0x791e73 in do_decode libavcodec/utils.c:2796:15
    #10 0x791354 in avcodec_send_packet libavcodec/utils.c:2885:12
    #11 0x7038f0 in try_decode_frame libavformat/utils.c:3000:19
    #12 0x6ee52b in avformat_find_stream_info libavformat/utils.c:3701:9
    #13 0x5094c8 in open_input_file ffmpeg_opt.c:1012:11
    #14 0x5059dc in open_files ffmpeg_opt.c:3197:15
    #15 0x505071 in ffmpeg_parse_options ffmpeg_opt.c:3237:11
    #16 0x5743d0 in main ffmpeg.c:4679:11
    #17 0x7fd38f21082f in __libc_start_main /build/glibc-t3gR2i/glibc-2.23/csu/../csu/libc-start.c:291
    #18 0x4198b8 in _start (ffmpeg+0x4198b8)

[Michael Niedermayer]

Does not seem to reproduce here with git master

[Michael Niedermayer]

Closing as it does not reproduce