Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#2645 closed defect (fixed)

mpeg4 encoder potential buffer overflow with certain video dimensions

Reported by: Darrell Walisser Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: regression
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

mpeg4 encoder has a buffer overflow in ff_draw_edges_mmx() when input dimension is 856x480. No error for 848x480 (nearest multiple of 16 width).

On Linux x64/gcc4.6.3 does not crash, but valgrind shows an invalid write.

I have also tried on win32/gcc4.7.1, it does crash occasionally and never with gdb running.

Temporary solution is to force a width that is multiple of 16.

How to reproduce:

dwalisser@loki:~/devel/kiosk/src$ valgrind ffmpeg -f lavfi -i "testsrc=duration=10:size=856x480:rate=30" -loglevel 99 -vcodec mpeg4 out.mp4
==24178== Memcheck, a memory error detector
==24178== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==24178== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==24178== Command: ffmpeg -f lavfi -i testsrc=duration=10:size=856x480:rate=30 -loglevel 99 -vcodec mpeg4 out.mp4
==24178== 
ffmpeg version git-2013-06-05-31983ae Copyright (c) 2000-2013 the FFmpeg developers
  built on Jun  5 2013 15:28:46 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
  configuration: --enable-debug --disable-stripping --disable-optimizations --enable-runtime-cpudetect --disable-static --enable-shared --disable-ffserver --extra-ldflags=-L/usr/local/lib
  libavutil      52. 35.100 / 52. 35.100
  libavcodec     55. 15.100 / 55. 15.100
  libavformat    55.  8.102 / 55.  8.102
  libavdevice    55.  2.100 / 55.  2.100
  libavfilter     3. 75.101 /  3. 75.101
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
Splitting the commandline.
Reading option '-f' ... matched as option 'f' (force format) with argument 'lavfi'.
Reading option '-i' ... matched as input file with argument 'testsrc=duration=10:size=856x480:rate=30'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'mpeg4'.
Reading option 'out.mp4' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument 99.
Successfully parsed a group of options.
Parsing a group of options: input file testsrc=duration=10:size=856x480:rate=30.
Applying option f (force format) with argument lavfi.
Successfully parsed a group of options.
Opening an input file: testsrc=duration=10:size=856x480:rate=30.
detected 2 logical cores
[Parsed_testsrc_0 @ 0xbf5f3c0] Setting 'duration' to value '10'
[Parsed_testsrc_0 @ 0xbf5f3c0] Setting 'size' to value '856x480'
[Parsed_testsrc_0 @ 0xbf5f3c0] Setting 'rate' to value '30'
[Parsed_testsrc_0 @ 0xbf5f3c0] size:856x480 rate:30/1 duration:10.000000 sar:1/1
[AVFilterGraph @ 0xbf5ea60] query_formats: 2 queried, 1 merged, 0 already done, 0 delayed
[lavfi @ 0xbf5dd40] All info found
Input #0, lavfi, from 'testsrc=duration=10:size=856x480:rate=30':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0, 1, 1/30: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 856x480 [SAR 1:1 DAR 107:60], 1/30, 30 tbr, 30 tbn, 30 tbc
Successfully opened the file.
Parsing a group of options: output file out.mp4.
Applying option vcodec (force video codec ('copy' to copy stream)) with argument mpeg4.
Successfully parsed a group of options.
Opening an output file: out.mp4.
File 'out.mp4' already exists. Overwrite ? [y/N] y
Successfully opened the file.
[graph 0 input from stream 0:0 @ 0xc26af60] Setting 'video_size' to value '856x480'
[graph 0 input from stream 0:0 @ 0xc26af60] Setting 'pix_fmt' to value '2'
[graph 0 input from stream 0:0 @ 0xc26af60] Setting 'time_base' to value '1/30'
[graph 0 input from stream 0:0 @ 0xc26af60] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0xc26af60] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0xc26af60] Setting 'frame_rate' to value '30/1'
[graph 0 input from stream 0:0 @ 0xc26af60] w:856 h:480 pixfmt:rgb24 tb:1/30 fr:30/1 sar:1/1 sws_param:flags=2
[format @ 0xc270760] compat: called with args=[yuv420p]
[format @ 0xc270760] Setting 'pix_fmts' to value 'yuv420p'
[auto-inserted scaler 0 @ 0xf3791e0] Setting 'flags' to value '0x4'
[auto-inserted scaler 0 @ 0xf3791e0] w:iw h:ih flags:'0x4' interl:0
[format @ 0xc270760] auto-inserting filter 'auto-inserted scaler 0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 0xc2696c0] query_formats: 4 queried, 3 merged, 1 already done, 0 delayed
==24178== Invalid read of size 8
==24178==    at 0x74911CF: __GI___strncasecmp_l (strcmp.S:213)
==24178==    by 0x744373F: ____strtod_l_internal (strtod_l.c:574)
==24178==    by 0x6CB51CB: av_strtod (eval.c:96)
==24178==    by 0x6CB6965: parse_primary (eval.c:322)
==24178==    by 0x6CB771B: parse_pow (eval.c:481)
==24178==    by 0x6CB77BD: parse_dB (eval.c:496)
==24178==    by 0x6CB77E6: parse_factor (eval.c:503)
==24178==    by 0x6CB7940: parse_term (eval.c:530)
==24178==    by 0x6CB7A58: parse_subexpr (eval.c:554)
==24178==    by 0x6CB7B70: parse_expr (eval.c:582)
==24178==    by 0x6CB7F97: av_expr_parse (eval.c:671)
==24178==    by 0x6CB811E: av_expr_parse_and_eval (eval.c:708)
==24178==  Address 0xf3c0220 is 0 bytes inside a block of size 3 alloc'd
==24178==    at 0x4C29BE2: memalign (vg_replace_malloc.c:694)
==24178==    by 0x4C29C3B: posix_memalign (vg_replace_malloc.c:835)
==24178==    by 0x6CC0381: av_malloc (mem.c:93)
==24178==    by 0x6CB7E9A: av_expr_parse (eval.c:648)
==24178==    by 0x6CB811E: av_expr_parse_and_eval (eval.c:708)
==24178==    by 0x50B784F: config_props (vf_scale.c:209)
==24178==    by 0x507DACE: avfilter_config_links (avfilter.c:242)
==24178==    by 0x507DA60: avfilter_config_links (avfilter.c:231)
==24178==    by 0x50808D4: graph_config_links (avfiltergraph.c:267)
==24178==    by 0x508311D: avfilter_graph_config (avfiltergraph.c:1112)
==24178==    by 0x41585C: configure_filtergraph (ffmpeg_filter.c:862)
==24178==    by 0x41F0FC: transcode_init (ffmpeg.c:2283)
==24178== 
==24178== Invalid read of size 8
==24178==    at 0x74911D7: __GI___strncasecmp_l (strcmp.S:215)
==24178==    by 0x744373F: ____strtod_l_internal (strtod_l.c:574)
==24178==    by 0x6CB51CB: av_strtod (eval.c:96)
==24178==    by 0x6CB6965: parse_primary (eval.c:322)
==24178==    by 0x6CB771B: parse_pow (eval.c:481)
==24178==    by 0x6CB77BD: parse_dB (eval.c:496)
==24178==    by 0x6CB77E6: parse_factor (eval.c:503)
==24178==    by 0x6CB7940: parse_term (eval.c:530)
==24178==    by 0x6CB7A58: parse_subexpr (eval.c:554)
==24178==    by 0x6CB7B70: parse_expr (eval.c:582)
==24178==    by 0x6CB7F97: av_expr_parse (eval.c:671)
==24178==    by 0x6CB811E: av_expr_parse_and_eval (eval.c:708)
==24178==  Address 0xf3c0228 is 5 bytes after a block of size 3 alloc'd
==24178==    at 0x4C29BE2: memalign (vg_replace_malloc.c:694)
==24178==    by 0x4C29C3B: posix_memalign (vg_replace_malloc.c:835)
==24178==    by 0x6CC0381: av_malloc (mem.c:93)
==24178==    by 0x6CB7E9A: av_expr_parse (eval.c:648)
==24178==    by 0x6CB811E: av_expr_parse_and_eval (eval.c:708)
==24178==    by 0x50B784F: config_props (vf_scale.c:209)
==24178==    by 0x507DACE: avfilter_config_links (avfilter.c:242)
==24178==    by 0x507DA60: avfilter_config_links (avfilter.c:231)
==24178==    by 0x50808D4: graph_config_links (avfiltergraph.c:267)
==24178==    by 0x508311D: avfilter_graph_config (avfiltergraph.c:1112)
==24178==    by 0x41585C: configure_filtergraph (ffmpeg_filter.c:862)
==24178==    by 0x41F0FC: transcode_init (ffmpeg.c:2283)
==24178== 
[auto-inserted scaler 0 @ 0xf3791e0] w:856 h:480 fmt:rgb24 sar:1/1 -> w:856 h:480 fmt:yuv420p sar:1/1 flags:0x4
[mpeg4 @ 0xc25cec0] intra_quant_bias = 0 inter_quant_bias = -64
Output #0, mp4, to 'out.mp4':
  Metadata:
    encoder         : Lavf55.8.102
    Stream #0:0, 0, 1/15360: Video: mpeg4 ( [0][0][0] / 0x0020), yuv420p, 856x480 [SAR 1:1 DAR 107:60], 1/30, q=2-31, 200 kb/s, 15360 tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo -> mpeg4)
Press [q] to stop, [?] for help
[swscaler @ 0xf3c09a0] Warning: data is not aligned! This can lead to a speedloss
==24178== Invalid write of size 8
==24178==    at 0x5D5B8B9: ff_draw_edges_mmx (dsputil_mmx.c:389)
==24178==    by 0x5B25AD1: load_input_picture (mpegvideo_enc.c:1063)
==24178==    by 0x5B271E2: ff_MPV_encode_picture (mpegvideo_enc.c:1448)
==24178==    by 0x5C56E80: avcodec_encode_video2 (utils.c:1762)
==24178==    by 0x41894E: do_video_out (ffmpeg.c:947)
==24178==    by 0x4192C4: reap_filters (ffmpeg.c:1091)
==24178==    by 0x42321D: transcode_step (ffmpeg.c:3162)
==24178==    by 0x4232D6: transcode (ffmpeg.c:3205)
==24178==    by 0x423846: main (ffmpeg.c:3383)
==24178==  Address 0xf6d0830 is 0 bytes after a block of size 114,704 alloc'd
==24178==    at 0x4C29BE2: memalign (vg_replace_malloc.c:694)
==24178==    by 0x4C29C3B: posix_memalign (vg_replace_malloc.c:835)
==24178==    by 0x6CC0381: av_malloc (mem.c:93)
==24178==    by 0x6CB2AD6: av_buffer_alloc (buffer.c:70)
==24178==    by 0x6CB2B3D: av_buffer_allocz (buffer.c:83)
==24178==    by 0x6CB31DA: pool_alloc_buffer (buffer.c:305)
==24178==    by 0x6CB3353: av_buffer_pool_get (buffer.c:343)
==24178==    by 0x5C530F7: video_get_buffer (utils.c:550)
==24178==    by 0x5C53502: avcodec_default_get_buffer2 (utils.c:615)
==24178==    by 0x5C53EA5: get_buffer_internal (utils.c:830)
==24178==    by 0x5C53F08: ff_get_buffer (utils.c:842)
==24178==    by 0x5B8BEA9: thread_get_buffer_internal (pthread.c:923)
==24178== 
frame=   48 fps=2.7 q=4.7 Lsize=     169kB time=00:00:01.60 bitrate= 864.6kbits/s    
video:168kB audio:0kB subtitle:0 global headers:0kB muxing overhead 0.591599%
48 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0xc269240] Statistics: 34 seeks, 76 writeouts
Received signal 2: terminating.
==24178== 
==24178== HEAP SUMMARY:
==24178==     in use at exit: 0 bytes in 0 blocks
==24178==   total heap usage: 7,826 allocs, 7,826 frees, 161,765,989 bytes allocated
==24178== 
==24178== All heap blocks were freed -- no leaks are possible
==24178== 
==24178== For counts of detected and suppressed errors, rerun with: -v
==24178== ERROR SUMMARY: 102 errors from 3 contexts (suppressed: 0 from 0)

Change History (3)

comment:1 by Carl Eugen Hoyos, 11 years ago

Keywords: regression added
Priority: normalimportant
Reproduced by developer: set
Status: newopen

Regression since 1e78679

$ valgrind ./ffmpeg_g -f rawvideo -s 856x480 -i /dev/zero -vcodec mpeg1video -f null -vframes 2 -
==5806== Memcheck, a memory error detector
==5806== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==5806== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==5806== Command: ./ffmpeg_g -f rawvideo -s 856x480 -i /dev/zero -vcodec mpeg1video -f null -vframes 2 -
==5806==
ffmpeg version N-53859-g258a05b Copyright (c) 2000-2013 the FFmpeg developers
  built on Jun  6 2013 00:35:52 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 35.100 / 52. 35.100
  libavcodec     55. 15.100 / 55. 15.100
  libavformat    55.  8.102 / 55.  8.102
  libavdevice    55.  2.100 / 55.  2.100
  libavfilter     3. 75.101 /  3. 75.101
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
Input #0, rawvideo, from '/dev/zero':
  Duration: N/A, start: 0.000000, bitrate: 123264 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 856x480, 123264 kb/s, 25 tbr, 25 tbn, 25 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf55.8.102
    Stream #0:0: Video: mpeg1video, yuv420p, 856x480, q=2-31, 200 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo -> mpeg1video)
Press [q] to stop, [?] for help
==5806== Invalid write of size 8
==5806==    at 0xA72853: ff_draw_edges_mmx (dsputil_mmx.c:389)
==5806==    by 0x8E77EB: ff_MPV_encode_picture (mpegvideo_enc.c:1063)
==5806==    by 0x9C84C6: avcodec_encode_video2 (utils.c:1762)
==5806==    by 0x46B675: reap_filters (ffmpeg.c:947)
==5806==    by 0x45BF14: main (ffmpeg.c:3162)
==5806==  Address 0x74c9eb0 is 0 bytes after a block of size 114,704 alloc'd
==5806==    at 0x4C290FE: memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==5806==    by 0x4C291A7: posix_memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==5806==    by 0xBF6F39: av_malloc (mem.c:93)
==5806==    by 0xBEAFED: av_buffer_allocz (buffer.c:70)
==5806==    by 0xBEB5CB: av_buffer_pool_get (buffer.c:305)
==5806==    by 0x9C5546: video_get_buffer (utils.c:550)
==5806==    by 0x9C6B4E: get_buffer_internal (utils.c:830)
==5806==    by 0x9C7075: ff_get_buffer (utils.c:842)
==5806==    by 0x921EDD: ff_thread_get_buffer (pthread.c:923)
==5806==    by 0x8CCAA2: ff_alloc_picture (mpegvideo.c:234)
==5806==    by 0x8E7651: ff_MPV_encode_picture (mpegvideo_enc.c:1020)
==5806==    by 0x9C84C6: avcodec_encode_video2 (utils.c:1762)
==5806==
frame=    2 fps=0.0 q=2.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:6kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.347496%
==5806==
==5806== HEAP SUMMARY:
==5806==     in use at exit: 0 bytes in 0 blocks
==5806==   total heap usage: 2,161 allocs, 2,161 frees, 11,511,876 bytes allocated
==5806==
==5806== All heap blocks were freed -- no leaks are possible
==5806==
==5806== For counts of detected and suppressed errors, rerun with: -v
==5806== ERROR SUMMARY: 4 errors from 1 contexts (suppressed: 2 from 2)

comment:2 by Michael Niedermayer, 11 years ago

Resolution: fixed
Status: openclosed

comment:3 by Carl Eugen Hoyos, 11 years ago

Ticket #2569 is probably the same issue.

Note: See TracTickets for help on using tickets.