Heap-buffer-overflow in ff_rate_control_init (libavcodec/ratecontrol.c:560)

[Bachir Bendrissou]

Summary of the bug:

While attempting to apply a two-pass video encoding to a small mp4 video, we encountered a heap-buffer-overflow bug in file: libavcodec/ratecontrol.c, line: 560.

How to reproduce:

The bug is reproducible on the latest commit 34a47b97de.

The following builds ffmpeg:

git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
cd ffmpeg
./configure --disable-asm --toolchain=gcc-asan --disable-ffplay --disable-ffprobe
make

Next, prepare a log file, which is required by the ffmpeg two-pass video encoding command:

touch ffmpeg2pass-0.log

Run the ffmpeg command:

./ffmpeg_g -y -v 9 -loglevel 99 -i small_movie.mp4 -pass 2 small_movie_out.mp4

Options -v 9 -loglevel 99 are optional and were only added to produce verbose logs.

Console output:

ffmpeg version N-113328-g34a47b97de Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --disable-asm --toolchain=gcc-asan --disable-ffplay --disable-ffprobe
  libavutil      58. 36.101 / 58. 36.101
  libavcodec     60. 37.100 / 60. 37.100
  libavformat    60. 20.100 / 60. 20.100
  libavdevice    60.  4.100 / 60.  4.100
  libavfilter     9. 17.100 /  9. 17.100
  libswscale      7.  6.100 /  7.  6.100
  libswresample   4. 13.100 /  4. 13.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-i' ... matched as output url with argument 'small_movie.mp4'.
Reading option '-pass' ... matched as option 'pass' (select the pass number (1 to 3)) with argument '2'.
Reading option 'small_movie_out.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option v (set logging level) with argument 9.
Successfully parsed a group of options.
Parsing a group of options: input url small_movie.mp4.
Successfully parsed a group of options.
Opening an input file: small_movie.mp4.
[AVFormatContext @ 0x617000000080] Opening 'small_movie.mp4' for reading
[file @ 0x60c000000280] Setting default whitelist 'file,crypto,data'
Probing mov,mp4,m4a,3gp,3g2,mj2 score:100 size:1267
Probing mp3 score:1 size:1267
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'ftyp' parent:'root' sz: 24 8 1267
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] ISO: File Type Major Brand: mp42
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'free' parent:'root' sz: 8 32 1267
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'mdat' parent:'root' sz: 405 40 1267
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'moov' parent:'root' sz: 830 445 1267
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'mvhd' parent:'moov' sz: 108 8 822
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] time scale = 90000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'trak' parent:'moov' sz: 714 116 822
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'tkhd' parent:'trak' sz: 92 8 706
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'mdia' parent:'trak' sz: 614 100 706
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'mdhd' parent:'mdia' sz: 32 8 606
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'hdlr' parent:'mdia' sz: 52 40 606
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] ctype=[0][0][0][0]
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stype=vide
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'minf' parent:'mdia' sz: 522 92 606
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'vmhd' parent:'minf' sz: 20 8 514
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'dinf' parent:'minf' sz: 36 28 514
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'dref' parent:'dinf' sz: 28 8 28
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'stbl' parent:'minf' sz: 458 64 514
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'stsd' parent:'stbl' sz: 162 8 450
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] size=146 4CC=avc1 codec_type=0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'avcC' parent:'stsd' sz: 60 8 60
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'stts' parent:'stbl' sz: 24 170 450
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] track[0].stts.entries = 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] sample_count=12, sample_duration=1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'ctts' parent:'stbl' sz: 96 194 450
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] track[0].ctts.entries = 10
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] count=1, duration=1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] count=1, duration=3
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] count=2, duration=0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] count=1, duration=4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] count=1, duration=1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] count=1, duration=-1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] count=1, duration=0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] count=1, duration=3
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] count=2, duration=0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] count=1, duration=-1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] dts shift 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'stsc' parent:'stbl' sz: 52 290 450
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] track[0].stsc.entries = 3
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'stsz' parent:'stbl' sz: 68 342 450
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] sample_size = 0 sample_count = 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'stco' parent:'stbl' sz: 28 410 450
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] type:'stss' parent:'stbl' sz: 20 438 450
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] keyframe_count = 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 0, offset 28, dts -1, size 124, distance 0, keyframe 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 1, offset a4, dts 0, size 64, distance 1, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 2, offset e4, dts 1, size 25, distance 2, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 3, offset fd, dts 2, size 25, distance 3, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 4, offset 116, dts 3, size 24, distance 4, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 5, offset 12e, dts 4, size 21, distance 5, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 6, offset 143, dts 5, size 21, distance 6, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 7, offset 158, dts 6, size 21, distance 7, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 8, offset 16d, dts 7, size 25, distance 8, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 9, offset 186, dts 8, size 21, distance 9, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 10, offset 19b, dts 9, size 21, distance 10, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] AVIndex stream 0, sample 11, offset 1b0, dts 10, size 5, distance 11, keyframe 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] Setting codecpar->delay to 2 for stream st: 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] on_parse_exit_offset=1267
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] Before avformat_find_stream_info() pos: 1267 bytes read:1267 seeks:0 nb_streams:1
[h264 @ 0x619000000080] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x619000000080] Decoding VUI
[h264 @ 0x619000000080] nal_unit_type: 8(PPS), nal_ref_idc: 3
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 0, dts -83333
[h264 @ 0x619000000080] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x619000000080] Decoding VUI
[h264 @ 0x619000000080] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x619000000080] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x619000000080] Format yuv420p chosen by get_format().
[h264 @ 0x619000000080] Reinit context to 48x144, pix_fmt: yuv420p
[h264 @ 0x619000000080] no picture 
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0: start_time: 0.0833333 duration: 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] format: start_time: 0.083333 duration: 1 (estimate from stream) bitrate=10 kb/s
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] After avformat_find_stream_info() pos: 164 bytes read:1267 seeks:0 frames:1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'small_movie.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 19529854
    compatible_brands: mp42isom
    creation_time   : 2014-11-14T07:34:24.000000Z
  Duration: 00:00:01.00, start: 0.083333, bitrate: 10 kb/s
  Stream #0:0[0x1](eng), 1, 1/12: Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuv420p(tv, smpte170m, progressive, left), 48x144 [SAR 1:1 DAR 1:3], 0/1, 3 kb/s, 12 fps, 12 tbr, 12 tbn (default)
      Metadata:
        creation_time   : 2014-11-14T07:34:24.000000Z
        handler_name    : Video Media Handler
        vendor_id       : [0][0][0][0]
        encoder         : AVC Coding
Successfully opened the file.
Parsing a group of options: output url small_movie_out.mp4.
Applying option pass (select the pass number (1 to 3)) with argument 2.
Successfully parsed a group of options.
Opening an output file: small_movie_out.mp4.
[out#0/mp4 @ 0x611000000b80] No explicit maps, mapping streams automatically...
[vost#0:0/mpeg4 @ 0x617000000780] Created video stream from input stream 0:0
[file @ 0x60d000001080] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0x612000000c40] Statistics: 0 bytes read, 0 seeks
detected 4 logical cores
[h264 @ 0x619000001980] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x619000001980] Decoding VUI
[h264 @ 0x619000001980] nal_unit_type: 8(PPS), nal_ref_idc: 3
[file @ 0x60d000001700] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
[vost#0:0/mpeg4 @ 0x617000000780] Starting thread...
[vf#0:0 @ 0x60e0000002e0] Starting thread...
[vist#0:0/h264 @ 0x613000000740] Starting thread...
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x612000000040] Starting thread...
Press [q] to stop, [?] for help
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 1, dts 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 2, dts 83333
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 3, dts 166667
[h264 @ 0x619000001980] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x619000001980] Format yuv420p chosen by get_format().
[h264 @ 0x619000001980] Reinit context to 48x144, pix_fmt: yuv420p
[h264 @ 0x619000001980] no picture 
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 4, dts 250000
[h264 @ 0x619000001e80] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 0x619000001e80] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 0x619000001e80] no picture 
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 5, dts 333333
[h264 @ 0x619000002380] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 0x619000002380] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 6, dts 416667
[h264 @ 0x619000002880] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 0x619000002880] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 7, dts 500000
[h264 @ 0x619000002d80] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 0x619000002d80] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[h264 @ 0x619000001980] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 0x619000001980] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 8, dts 583333
[h264 @ 0x619000001e80] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 0x619000001e80] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
[h264 @ 0x619000002380] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 0x619000002380] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
[h264 @ 0x619000002880] nal_unit_type: 9(AUD), nal_ref_idc: 0
[h264 @ 0x619000002880] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 9, dts 666667
[graph 0 input from stream 0:0 @ 0x610000038140] Setting 'video_size' to value '48x144'
[graph 0 input from stream 0:0 @ 0x610000038140] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x610000038140] Setting 'time_base' to value '1/12'
[graph 0 input from stream 0:0 @ 0x610000038140] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x610000038140] Setting 'colorspace' to value 'smpte170m'
[graph 0 input from stream 0:0 @ 0x610000038140] Setting 'range' to value 'tv'
[graph 0 input from stream 0:0 @ 0x610000038140] Setting 'frame_rate' to value '12/1'
[graph 0 input from stream 0:0 @ 0x610000038140] w:48 h:144 pixfmt:yuv420p tb:1/12 fr:12/1 sar:1/1 csp:smpte170m range:tv
[format @ 0x610000038340] Setting 'pix_fmts' to value 'yuv420p'
[AVFilterGraph @ 0x609000024800] query_formats: 4 queried, 9 merged, 0 already done, 0 delayed
[mpeg4 @ 0x619000001480] intra_quant_bias = 0 inter_quant_bias = -64
=================================================================
==4058163==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000000be1 at pc 0x7f9d69dc8f65 bp 0x7f9d631f5a00 sp 0x7f9d631f51a8
READ of size 1 at 0x603000000be1 thread T7 (vf#0:0)
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x617000000080] stream 0, sample 10, dts 750000
    #0 0x7f9d69dc8f64 in __interceptor_strchr ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:708
    #1 0x563ffaa3e31c in ff_rate_control_init libavcodec/ratecontrol.c:560
    #2 0x563ffaa29812 in ff_mpv_encode_init libavcodec/mpegvideo_enc.c:956
    #3 0x563ffaa20466 in encode_init libavcodec/mpeg4videoenc.c:1291
    #4 0x563ffb6ea8ab in avcodec_open2 libavcodec/avcodec.c:337
    #5 0x563ffab14142 in enc_open fftools/ffmpeg_enc.c:319
    #6 0x563ffab6dfa0 in enc_open fftools/ffmpeg_sched.c:1462
    #7 0x563ffab6dfa0 in send_to_enc fftools/ffmpeg_sched.c:1571
    #8 0x563ffab6dfa0 in sch_filter_send fftools/ffmpeg_sched.c:2154
    #9 0x563ffab1e98c in fg_output_frame fftools/ffmpeg_filter.c:2303
    #10 0x563ffab2177c in fg_output_step fftools/ffmpeg_filter.c:2406
    #11 0x563ffab2177c in read_frames fftools/ffmpeg_filter.c:2467
    #12 0x563ffab2a2ee in read_frames fftools/ffmpeg_filter.c:2423
    #13 0x563ffab2a2ee in filter_thread fftools/ffmpeg_filter.c:2879
    #14 0x563ffab63dc8 in task_wrapper fftools/ffmpeg_sched.c:2200
    #15 0x7f9d69af3ac2  (/lib/x86_64-linux-gnu/libc.so.6+0x94ac2)
    #16 0x7f9d69b8584f  (/lib/x86_64-linux-gnu/libc.so.6+0x12684f)

0x603000000be1 is located 0 bytes to the right of 1-byte region [0x603000000be0,0x603000000be1)
allocated by thread T0 here:
    #0 0x7f9d69e4057c in __interceptor_posix_memalign ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:226
    #1 0x563ffd2020ac in av_malloc libavutil/mem.c:105

Thread T7 (vf#0:0) created by T0 here:
    #0 0x7f9d69de3685 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
    #1 0x563ffab61ae9 in task_start fftools/ffmpeg_sched.c:403

SUMMARY: AddressSanitizer: heap-buffer-overflow ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:708 in __interceptor_strchr
Shadow bytes around the buggy address:
  0x0c067fff8120: fa fa 04 fa fa fa fa fa fd fd fa fa fa fa fd fd
  0x0c067fff8130: fd fa fa fa fd fd fa fa fa fa fd fd fd fa fa fa
  0x0c067fff8140: fd fd fa fa fa fa fd fd fd fa fa fa fd fd fd fa
  0x0c067fff8150: fa fa fd fd fd fa fa fa fd fd fa fa fa fa fd fd
  0x0c067fff8160: fd fa fa fa fd fd fd fa fa fa fd fd fd fa fa fa
=>0x0c067fff8170: fd fd fa fa fa fa fd fd fd fa fa fa[01]fa fa fa
  0x0c067fff8180: fa fa fd fd fa fa fa fa fd fa fa fa fa fa fd fd
  0x0c067fff8190: fa fa fa fa fd fa fa fa fa fa fd fa fa fa fa fa
  0x0c067fff81a0: 00 05 fa fa fa fa 00 05 fa fa fa fa 00 04 fa fa
  0x0c067fff81b0: fa fa 00 fa fa fa fa fa 00 00 00 fa fa fa 04 fa
  0x0c067fff81c0: fa fa fa fa fd fd fa fa fa fa fd fd fd fd fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==4058163==ABORTING

Environment:

ffmpeg version N-113328-g34a47b97de
gcc 11.4.0 
Ubuntu 22.04.2

[Bachir Bendrissou]

This is the input file to ffmpeg.