signed integer overflow in libavformat/flvenc.c

[Suhwan]

Summary of the bug:
There're a signed integer overflow in libavformat/flvenc.c

libavformat/flvenc.c:1043:36: runtime error: signed integer overflow: -9223372036854775808 - 130 cannot be represented in type 'long'

How to reproduce:

% ./ffmpeg_g -loglevel 99 -y -r 11 -i CAFI1_SVA_C.264 -map 0 -c copy -r 74 -ab 123k -ar 48000 -ac 12 -b:v 433k -strict 1 output/tmp.flv

ffmpeg version N-94887-ge55018ee11 (git master)
built on ubuntu 18.04 with clang-6 and UBSAN option.

[Suhwan]

poc

[Steven Liu]

Can you try the master branch?

StevenLiu:dash StevenLiu$ ./ffmpeg_g -loglevel 99 -y -r 11 -i ~/Movies/Test/CAFI1_SVA_C.264 -map 0 -c copy -r 74 -ab 123k -ar 48000 -ac 12 -b:v 433k -strict 1 output/tmp.flv
ffmpeg version N-94819-ga04f507323 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --enable-libass --enable-opengl --enable-libx264 --enable-libmp3lame --enable-gpl --enable-nonfree --prefix=/usr/local --enable-libtesseract --enable-libspeex --enable-libfreetype --enable-libfontconfig --enable-libfdk-aac --enable-videotoolbox --enable-libxml2 --enable-librsvg --enable-libvmaf --enable-version3 --disable-stripping --disable-optimizations --enable-libvmaf --disable-memory-poisoning --target-exec='valgrind --error-exitcode=1 --malloc-fill=0xa2 --track-origins=yes --leak-check=full --gen-suppressions=all --suppressions=./tests/fate-valgrind.supp'
  libavutil      56. 35.100 / 56. 35.100
  libavcodec     58. 56.101 / 58. 56.101
  libavformat    58. 32.104 / 58. 32.104
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 58.102 /  7. 58.102
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '11'.
Reading option '-i' ... matched as input url with argument '/Users/StevenLiu/Movies/Test/CAFI1_SVA_C.264'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0'.
Reading option '-c' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '74'.
Reading option '-ab' ... matched as option 'ab' (audio bitrate (please use -b:a)) with argument '123k'.
Reading option '-ar' ... matched as option 'ar' (set audio sampling rate (in Hz)) with argument '48000'.
Reading option '-ac' ... matched as option 'ac' (set number of audio channels) with argument '12'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '433k'.
Reading option '-strict' ...Routing option strict to both codec and muxer layer
 matched as AVOption 'strict' with argument '1'.
Reading option 'output/tmp.flv' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument 99.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url /Users/StevenLiu/Movies/Test/CAFI1_SVA_C.264.
Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 11.
Successfully parsed a group of options.
Opening an input file: /Users/StevenLiu/Movies/Test/CAFI1_SVA_C.264.
[NULL @ 0x7ff74d80b800] Opening '/Users/StevenLiu/Movies/Test/CAFI1_SVA_C.264' for reading
[file @ 0x7ff74cd47b40] Setting default whitelist 'file,crypto'
Probing h264 score:51 size:2048
[h264 @ 0x7ff74d80b800] Format h264 probed with size=2048 and score=51
[h264 @ 0x7ff74d80b800] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[AVBSFContext @ 0x7ff74cf073c0] nal_unit_type: 7(SPS), nal_ref_idc: 3
[AVBSFContext @ 0x7ff74cf073c0] nal_unit_type: 8(PPS), nal_ref_idc: 3
[AVBSFContext @ 0x7ff74cf073c0] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x7ff74e808600] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x7ff74e808600] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x7ff74e808600] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x7ff74e808600] Format yuv420p chosen by get_format().
[h264 @ 0x7ff74e808600] Reinit context to 720x480, pix_fmt: yuv420p
[h264 @ 0x7ff74e808600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
    Last message repeated 2 times
[h264 @ 0x7ff74e808600] Increasing reorder buffer to 1
[h264 @ 0x7ff74e808600] no picture
[h264 @ 0x7ff74e808600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
    Last message repeated 1 times
[h264 @ 0x7ff74e808600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
    Last message repeated 1 times
[h264 @ 0x7ff74e808600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
    Last message repeated 1 times
[h264 @ 0x7ff74e808600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
    Last message repeated 1 times
[h264 @ 0x7ff74e808600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
    Last message repeated 1 times
[h264 @ 0x7ff74e808600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
    Last message repeated 1 times
[h264 @ 0x7ff74d80b800] stream 0: start_time: -7686143364045.646 duration: -7686143364045.646
[h264 @ 0x7ff74d80b800] format: start_time: -9223372036854.775 duration: -9223372036854.775 bitrate=0 kb/s
[h264 @ 0x7ff74d80b800] After avformat_find_stream_info() pos: 257764 bytes read:257764 seeks:0 frames:66
Input #0, h264, from '/Users/StevenLiu/Movies/Test/CAFI1_SVA_C.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0, 66, 1/1200000: Video: h264 (Main), 1 reference frame, yuv420p(top first, left), 720x480, 0/1, 25.42 fps, 25 tbr, 1200k tbn, 50 tbc
Successfully opened the file.
Parsing a group of options: output url output/tmp.flv.
Applying option map (set input stream mapping) with argument 0.
Applying option c (codec name) with argument copy.
Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 74.
Applying option ab (audio bitrate (please use -b:a)) with argument 123k.
Applying option ar (set audio sampling rate (in Hz)) with argument 48000.
Applying option ac (set number of audio channels) with argument 12.
Applying option b:v (video bitrate (please use -b:v)) with argument 433k.
Successfully parsed a group of options.
Opening an output file: output/tmp.flv.
[file @ 0x7ff74cd48200] Setting default whitelist 'file,crypto'
Successfully opened the file.
Output #0, flv, to 'output/tmp.flv':
  Metadata:
    encoder         : Lavf58.32.104
    Stream #0:0, 0, 1/1000: Video: h264 (Main), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p(top first, left), 720x480 (0x0), 0/1, q=2-31, 433 kb/s, 25.42 fps, 25 tbr, 1k tbn, 74 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[flv @ 0x7ff74d805e00] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
No more output streams to write to, finishing.
frame=   66 fps=0.0 q=-1.0 Lsize=     253kB time=00:00:05.91 bitrate= 351.1kbits/s speed=4.62e+03x
video:252kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.619947%
Input file #0 (/Users/StevenLiu/Movies/Test/CAFI1_SVA_C.264):
  Input stream #0:0 (video): 66 packets read (257764 bytes);
  Total: 66 packets (257764 bytes) demuxed
Output file #0 (output/tmp.flv):
  Output stream #0:0 (video): 66 packets muxed (257764 bytes);
  Total: 66 packets (257764 bytes) muxed
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x7ff74cd48300] Statistics: 1 seeks, 1 writeouts
[AVIOContext @ 0x7ff74cd47c80] Statistics: 257764 bytes read, 0 seeks
StevenLiu:dash StevenLiu$
StevenLiu:dash StevenLiu$
StevenLiu:dash StevenLiu$
StevenLiu:dash StevenLiu$

[Suhwan]

I tried git master branch and it is still triggered. I think it can be reproduced when the ffmpeg is compiled with "--toolchain=clang-usan"

ffmpeg version N-94931-g8e8fd25272 Copyright (c) 2000-2019 the FFmpeg developers
  built with clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
  configuration: --cc=clang --cxx=clang++ --ld=clang --enable-debug --toolchain=clang-usan
  libavutil      56. 35.100 / 56. 35.100
  libavcodec     58. 56.102 / 58. 56.102
  libavformat    58. 32.104 / 58. 32.104
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 58.102 /  7. 58.102
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '11'.
Reading option '-i' ... matched as input url with argument 'samples/h264/CAFI1_SVA_C.264'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0'.
Reading option '-c' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '74'.
Reading option '-ab' ... matched as option 'ab' (audio bitrate (please use -b:a)) with argument '123k'.
Reading option '-ar' ... matched as option 'ar' (set audio sampling rate (in Hz)) with argument '48000'.
Reading option '-ac' ... matched as option 'ac' (set number of audio channels) with argument '12'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '433k'.
Reading option '-strict' ...Routing option strict to both codec and muxer layer
 matched as AVOption 'strict' with argument '1'.
Reading option 'output/tmp.flv' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument 99.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url samples/h264/CAFI1_SVA_C.264.
Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 11.
Successfully parsed a group of options.
Opening an input file: samples/h264/CAFI1_SVA_C.264.
[NULL @ 0x61b000000080] Opening 'samples/h264/CAFI1_SVA_C.264' for reading
[file @ 0x610000000040] Setting default whitelist 'file,crypto'
Probing h264 score:51 size:2048
[h264 @ 0x61b000000080] Format h264 probed with size=2048 and score=51
[h264 @ 0x61b000000080] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
libavcodec/startcode.c:41:17: runtime error: load of misaligned address 0x619000000a85 for type 'const uint64_t' (aka 'const unsigned long'), which requires 8 byte alignment
0x619000000a85: note: pointer points here
 00 00 01 67 4d 40 1e  8d 94 c0 5a 3c 90 00 00  00 01 68 fe 38 80 00 00  00 01 65 88 80 00 50 00  67
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior libavcodec/startcode.c:41:17 in 
libavcodec/startcode.c:42:22: runtime error: load of misaligned address 0x619000000a85 for type 'const uint64_t' (aka 'const unsigned long'), which requires 8 byte alignment
0x619000000a85: note: pointer points here
 00 00 01 67 4d 40 1e  8d 94 c0 5a 3c 90 00 00  00 01 68 fe 38 80 00 00  00 01 65 88 80 00 50 00  67
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior libavcodec/startcode.c:42:22 in 
[AVBSFContext @ 0x60a000000200] nal_unit_type: 7(SPS), nal_ref_idc: 3
[AVBSFContext @ 0x60a000000200] nal_unit_type: 8(PPS), nal_ref_idc: 3
[AVBSFContext @ 0x60a000000200] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x619000000580] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x619000000580] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x619000000580] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x619000000580] Format yuv420p chosen by get_format().
[h264 @ 0x619000000580] Reinit context to 720x480, pix_fmt: yuv420p
[h264 @ 0x619000000580] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
    Last message repeated 2 times
[h264 @ 0x619000000580] Increasing reorder buffer to 1
[h264 @ 0x619000000580] no picture 
[h264 @ 0x619000000580] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
    Last message repeated 1 times
[h264 @ 0x619000000580] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
    Last message repeated 1 times
[h264 @ 0x619000000580] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
    Last message repeated 1 times
[h264 @ 0x619000000580] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
    Last message repeated 1 times
[h264 @ 0x619000000580] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0
    Last message repeated 1 times
[h264 @ 0x619000000580] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
    Last message repeated 1 times
[h264 @ 0x61b000000080] stream 0: start_time: -7686143364045.646 duration: -7686143364045.646
[h264 @ 0x61b000000080] format: start_time: -9223372036854.775 duration: -9223372036854.775 bitrate=0 kb/s
[h264 @ 0x61b000000080] After avformat_find_stream_info() pos: 257764 bytes read:257764 seeks:0 frames:66
Input #0, h264, from 'samples/h264/CAFI1_SVA_C.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0, 66, 1/1200000: Video: h264 (Main), 1 reference frame, yuv420p(top first, left), 720x480, 0/1, 25.42 fps, 25 tbr, 1200k tbn, 50 tbc
Successfully opened the file.
Parsing a group of options: output url output/tmp.flv.
Applying option map (set input stream mapping) with argument 0.
Applying option c (codec name) with argument copy.
Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 74.
Applying option ab (audio bitrate (please use -b:a)) with argument 123k.
Applying option ar (set audio sampling rate (in Hz)) with argument 48000.
Applying option ac (set number of audio channels) with argument 12.
Applying option b:v (video bitrate (please use -b:v)) with argument 433k.
Successfully parsed a group of options.
Opening an output file: output/tmp.flv.
[file @ 0x610000000440] Setting default whitelist 'file,crypto'
Successfully opened the file.
Output #0, flv, to 'output/tmp.flv':
  Metadata:
    encoder         : Lavf58.32.104
    Stream #0:0, 0, 1/1000: Video: h264 (Main), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p(top first, left), 720x480 (0x0), 0/1, q=2-31, 433 kb/s, 25.42 fps, 25 tbr, 1k tbn, 74 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[flv @ 0x61b000005480] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
libavformat/flvenc.c:1043:36: runtime error: signed integer overflow: -9223372036854775808 - 130 cannot be represented in type 'long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior libavformat/flvenc.c:1043:36 in 
No more output streams to write to, finishing.
frame=   66 fps=0.0 q=-1.0 Lsize=     253kB time=00:00:05.91 bitrate= 351.1kbits/s speed= 521x    
video:252kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.619947%
Input file #0 (samples/h264/CAFI1_SVA_C.264):
  Input stream #0:0 (video): 66 packets read (257764 bytes); 
  Total: 66 packets (257764 bytes) demuxed
Output file #0 (output/tmp.flv):
  Output stream #0:0 (video): 66 packets muxed (257764 bytes); 
  Total: 66 packets (257764 bytes) muxed
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x6130000003c0] Statistics: 1 seeks, 1 writeouts
[AVIOContext @ 0x613000000040] Statistics: 257764 bytes read, 0 seeks

[Suhwan]

It can be reproduced if "output" directory exists or following cmd.

./ffmpeg_g -loglevel 99 -y -r 11 -i CAFI1_SVA_C.264 -map 0 -c copy -r 74 -ab 123k -ar 48000 -ac 12 -b:v 433k -strict 1 tmp.flv

[Michael Niedermayer]

This was fixed in 14d3384cf31a620ff451062f7263942f7fe3a972