signed integer overflow at libavformat/mpegenc.c:919

[Suhwan]

Summary of the bug:
There is a signed integer overflow at libavformat/mpegenc.c:919

I compiled ffmpeg with "--toolchain=clang-usan" to check the undefined-behaviours and attached log file.

I also ran this for 3mins to trigger this bug.

How to reproduce:

% ffmpeg_g -y -i $PoC -filter_complex rgbtestsrc -target svcd -loglevel 0 -map 0 -psnr -c dirac -c:s:11 mpeg2video tmp.tedcaptions

ffmpeg version N-95517-g0b8956b25c Copyright (c) 2000-2019 the FFmpeg developers
built with clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
configuration: --cc=clang --cxx=clang++ --ld=clang --enable-debug --toolchain=clang-usan

Here's UBSAN log

libavformat/mpegenc.c:919:33: runtime error: signed integer overflow: -2147482382 - 2300 cannot be represented in type 'int'

Thread 1 "ffmpeg_g" hit Breakpoint 1, 0x00000000004288b0 in __ubsan::ScopedReport::~ScopedReport() ()
(gdb) bt
#0  0x00000000004288b0 in __ubsan::ScopedReport::~ScopedReport() ()
#1  0x000000000042b0eb in void handleIntegerOverflowImpl<__ubsan::Value>(__ubsan::OverflowData*, unsigned long, char const*, __ubsan::Value, __ubsan::ReportOptions) ()
#2  0x000000000042c94f in __ubsan_handle_sub_overflow ()
#3  0x00000000017936bc in flush_packet (ctx=<optimized out>, stream_index=<optimized out>, 
    pts=<optimized out>, dts=<optimized out>, scr=<optimized out>, trailer_size=4554)
    at libavformat/mpegenc.c:919
#4  0x0000000001790c84 in output_packet (ctx=0x93cbd40, flush=0) at libavformat/mpegenc.c:1105
#5  0x000000000178d87a in mpeg_mux_write_packet (ctx=0x93cbd40, pkt=<optimized out>)
    at libavformat/mpegenc.c:1224
#6  0x00000000017f5905 in write_packet (s=0x93cbd40, pkt=<optimized out>) at libavformat/mux.c:754
#7  0x00000000017fc2c6 in av_interleaved_write_frame (s=<optimized out>, pkt=0x7fffffffc418)
    at libavformat/mux.c:1245
#8  0x00000000004b5e0b in write_packet (of=<optimized out>, pkt=0x7fffffffc418, ost=<optimized out>, 
    unqueue=<optimized out>) at fftools/ffmpeg.c:815
#9  0x00000000004c58a6 in do_video_out (of=0x93ca0c0, ost=<optimized out>, next_picture=<optimized out>, 
    sync_ipts=36766.000007629402) at fftools/ffmpeg.c:1321
#10 0x00000000004c0f2b in reap_filters (flush=0) at fftools/ffmpeg.c:1504
#11 0x000000000048c3d5 in transcode_from_filter (graph=0x93c7800, best_ist=<optimized out>)
    at fftools/ffmpeg.c:4533
#12 transcode_step () at fftools/ffmpeg.c:4606
#13 transcode () at fftools/ffmpeg.c:4682
#14 0x0000000000487dc4 in main (argc=18, argv=<optimized out>) at fftools/ffmpeg.c:4884
(gdb) c

Please confirm.
Thanks

[Suhwan]

poc

[Michael Niedermayer]

I left the testcase running alot longer than 3 minutes but it doesnt reproduce, there are also plenty of changes and fixes in mpegenc since this was reported so i suspect this was fixed. Please reopen if it still reproduces for you