segmentation violation in ffmpeg (libavcodec/mpegvideo_enc.c:2204)

[Youngseok Choi]

Hello, we are developing a new fuzzing technique, and it found a SEGV bug in ffmpeg.

How to reproduce:

% ./ffmpeg -i <input_file> -f mp4 -ildctcmp 1 -flags ildct e

<input_file> is available at ​https://github.com/3-24/oss-fuzz-reports/raw/master/ffmpeg/poc_5/poc_file.

Command output:

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
ffmpeg version N-109968-gcc76e8340d Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --extra-cflags='-fsanitize=address -g -O0' --extra-cxxflags='-fsanitize=address -g -O0' --extra-ldflags='-fsanitize=address -g -O0' --disable-optimizations --disable-stripping
  libavutil      58.  3.100 / 58.  3.100
  libavcodec     60.  6.100 / 60.  6.100
  libavformat    60.  4.100 / 60.  4.100
  libavdevice    60.  2.100 / 60.  2.100
  libavfilter     9.  4.100 /  9.  4.100
  libswscale      7.  2.100 /  7.  2.100
  libswresample   4. 11.100 /  4. 11.100
[h261 @ 0x617000000080] Format h261 detected only with low score of 25, misdetection possible!
[h261 @ 0x619000000580] warning: first frame is no keyframe
[h261 @ 0x619000000580] illegal ac vlc code at 6x0
[h261 @ 0x619000000580] Error at MB: 6
Input #0, h261, from 'poc_file':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h261, yuv420p, 176x144, 29.97 tbr, 1200k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h261 (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
[New Thread 0x7ffff1eff700 (LWP 22015)]
[Thread 0x7ffff1eff700 (LWP 22015) exited]
[h261 @ 0x619000001980] warning: first frame is no keyframe
[h261 @ 0x619000001980] illegal ac vlc code at 6x0
[h261 @ 0x619000001980] Error at MB: 6
[New Thread 0x7ffff16fe700 (LWP 22016)]
[New Thread 0x7ffff0efd700 (LWP 22017)]
[New Thread 0x7ffff06fc700 (LWP 22018)]
[New Thread 0x7fffefefb700 (LWP 22019)]
[New Thread 0x7fffef6fa700 (LWP 22020)]
[New Thread 0x7fffeeef9700 (LWP 22021)]
[New Thread 0x7fffee6f8700 (LWP 22022)]
[New Thread 0x7fffedef7700 (LWP 22023)]
[New Thread 0x7fffed6f6700 (LWP 22024)]
[New Thread 0x7fffecef5700 (LWP 22025)]
[New Thread 0x7fffec6f4700 (LWP 22026)]
[New Thread 0x7fffebef3700 (LWP 22027)]
[New Thread 0x7fffeb6f2700 (LWP 22028)]
[New Thread 0x7fffeaef1700 (LWP 22029)]
[New Thread 0x7fffea6f0700 (LWP 22030)]
[New Thread 0x7fffe9eef700 (LWP 22031)]
[New Thread 0x7fffe96ee700 (LWP 22032)]
[New Thread 0x7fffe8eed700 (LWP 22033)]
[New Thread 0x7fffe86ec700 (LWP 22034)]
[New Thread 0x7fffe7eeb700 (LWP 22035)]
[New Thread 0x7fffe76ea700 (LWP 22036)]
[New Thread 0x7fffe6ee9700 (LWP 22037)]
[New Thread 0x7fffe66e8700 (LWP 22038)]
[New Thread 0x7fffe5ee7700 (LWP 22039)]
[mpeg4 @ 0x619000002d80] too many threads/slices (10), reducing to 9
Output #0, mp4, to 'e':
  Metadata:
    encoder         : Lavf60.4.100
  Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p(progressive), 176x144, q=2-31, 200 kb/s, 29.97 fps, 30k tbn
    Metadata:
      encoder         : Lavc60.6.100 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
[New Thread 0x7fffe56e6700 (LWP 22040)]

Thread 1 "ffmpeg_g" received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()

Backtrace:

#0  0x0000000000000000 in ?? ()
#1  0x00005555574a9622 in encode_mb_internal (chroma_format=1, chroma_y_shift=1, chroma_x_shift=1, mb_block_count=6, mb_block_width=8,
    mb_block_height=8, motion_y=0, motion_x=0, s=0x625000014100) at libavcodec/mpegvideo_enc.c:2204
#2  encode_mb (motion_y=0, motion_x=0, s=0x625000014100) at libavcodec/mpegvideo_enc.c:2504
#3  encode_thread (c=0x619000002d80, arg=0x625000005408) at libavcodec/mpegvideo_enc.c:3431
#4  0x000055555761fadf in worker_func (priv=0x619000002d80, jobnr=6, threadnr=6, nb_jobs=9, nb_threads=9) at libavcodec/pthread_slice.c:77
#5  0x0000555558d8a45e in run_jobs (ctx=0x611000001a80) at libavutil/slicethread.c:65
#6  0x0000555558d8b54e in avpriv_slicethread_execute (ctx=0x611000001a80, nb_jobs=9, execute_main=0) at libavutil/slicethread.c:192
#7  0x000055555761ffe2 in thread_execute (avctx=0x619000002d80, func=0x55555749e4c1 <encode_thread>, arg=0x6250000053d8, ret=0x0, job_count=9,
    job_size=8) at libavcodec/pthread_slice.c:115
#8  0x00005555574bc3d8 in encode_picture (s=0x625000005100) at libavcodec/mpegvideo_enc.c:3837
#9  0x00005555574872cd in ff_mpv_encode_picture (avctx=0x619000002d80, pkt=0x610000002640, pic_arg=0x616000011d80, got_packet=0x7fffffffd390)
    at libavcodec/mpegvideo_enc.c:1801
#10 0x0000555556e486a3 in ff_encode_encode_cb (avctx=0x619000002d80, avpkt=0x610000002640, frame=0x616000011d80, got_packet=0x7fffffffd390)
    at libavcodec/encode.c:223
#11 0x0000555556e49220 in encode_simple_internal (avctx=0x619000002d80, avpkt=0x610000002640) at libavcodec/encode.c:309
#12 0x0000555556e49369 in encode_simple_receive_packet (avctx=0x619000002d80, avpkt=0x610000002640) at libavcodec/encode.c:323
#13 0x0000555556e498a6 in encode_receive_packet_internal (avctx=0x619000002d80, avpkt=0x610000002640) at libavcodec/encode.c:357
#14 0x0000555556e4a41d in avcodec_send_frame (avctx=0x619000002d80, frame=0x616000009080) at libavcodec/encode.c:506
#15 0x0000555555af6272 in encode_frame (of=0x611000000900, ost=0x618000000080, frame=0x616000009080) at fftools/ffmpeg.c:904
#16 0x0000555555af772f in submit_encode_frame (of=0x611000000900, ost=0x618000000080, frame=0x616000009080) at fftools/ffmpeg.c:985
#17 0x0000555555afa8a1 in do_video_out (of=0x611000000900, ost=0x618000000080, next_picture=0x616000009080) at fftools/ffmpeg.c:1340
#18 0x0000555555afb4fc in reap_filters (flush=0) at fftools/ffmpeg.c:1426
#19 0x0000555555b173a2 in transcode_step () at fftools/ffmpeg.c:4002
#20 0x0000555555b175c4 in transcode () at fftools/ffmpeg.c:4039
#21 0x0000555555b1821e in main (argc=10, argv=0x7fffffffe0b8) at fftools/ffmpeg.c:4177

Environment:

• OS: Ubuntu 18.04
• gcc: 7.5.0
• ffmpeg: version N-109968-gcc76e8340d (git-master)

Note that I built ffmpeg with address sanitizer.

./configure --extra-cflags="-fsanitize=address -g -O0" \
--extra-cxxflags="-fsanitize=address -g -O0" --extra-ldflags="-fsanitize=address -g -O0" \
--disable-optimizations --disable-stripping

Many thanks.