rmdec: crash in low mem situation

[ami_stuff]

I will send a possible patch soon.

​http://www1.datafilehost.com/d/f36c9362

knoppix@Microknoppix:/media/sdb1$ ulimit -Sv 20500 -c unlimited
knoppix@Microknoppix:/media/sdb1$ ./ffmpeg_g -i vsamples/rv8.rm -f null -
ffmpeg version 1.1.git Copyright (c) 2000-2013 the FFmpeg developers
  built on Jul  9 2013 10:16:01 with gcc 4.7 (Debian 4.7.2-4)
  configuration: --disable-asm --enable-gpl --disable-ffprobe --disable-ffserver
  libavutil      52. 38.100 / 52. 38.100
  libavcodec     55. 18.102 / 55. 18.102
  libavformat    55. 11.101 / 55. 11.101
  libavdevice    55.  2.100 / 55.  2.100
  libavfilter     3. 79.100 /  3. 79.100
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
[rm @ 0x9077c50] Invalid stream index 2 for index at pos 858064
Truncating packet of size 268437708 to 704179
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 268507400 to 702844
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 903434109 to 702835
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 268679955 to 701516
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 268884243 to 700186
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 269064968 to 698926
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 386564182 to 698858
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 269261028 to 697579
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 1039270462 to 697562
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 631372420 to 697553
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 145223533 to 697537
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 269474627 to 696236
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 58203055 to 696227
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 41141093 to 691342
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 965264131 to 681519
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 741170995 to 681510
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 944849978 to 681492
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 922749185 to 680239
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 91585198 to 680228
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 922819411 to 678920
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 922982672 to 677596
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 923171604 to 676248
[rm @ 0x9077c50] Impossibly sized packet
Truncating packet of size 923339540 to 674927
[rm @ 0x9077c50] Impossibly sized packet
Segmentation fault (core dumped)
knoppix@Microknoppix:/media/sdb1$ ulimit -Sv 20500000 -c unlimited
knoppix@Microknoppix:/media/sdb1$ gdb -c core ffmpeg_g
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /media/sdb1/ffmpeg_g...done.
[New LWP 22528]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Failed to read a valid object file image from memory.
Core was generated by `./ffmpeg_g -i vsamples/rv8.rm -f null -'.
Program terminated with signal 11, Segmentation fault.
#0  0x081f0e84 in rm_assemble_video_frame (timestamp=<synthetic pointer>, 
    pseq=0xbff3b748, len=1320, pkt=0xbff3b8f8, vst=0x9078aa0, pb=0x9080210, 
    s=0x9077c50, rm=<optimized out>) at libavformat/rmdec.c:725

warning: Source file is more recent than executable.
725	    AV_WL32(vst->pkt.data - 7 + 8*vst->cur_slice, 1);
(gdb) bt
#0  0x081f0e84 in rm_assemble_video_frame (timestamp=<synthetic pointer>, 
    pseq=0xbff3b748, len=1320, pkt=0xbff3b8f8, vst=0x9078aa0, pb=0x9080210, 
    s=0x9077c50, rm=<optimized out>) at libavformat/rmdec.c:725
#1  ff_rm_parse_packet (s=s@entry=0x9077c50, pb=0x9080210, 
    st=st@entry=0x9078450, ast=0x9078aa0, len=len@entry=1329, 
    pkt=pkt@entry=0xbff3b8f8, seq=seq@entry=0xbff3b748, flags=0, 
    timestamp=-9223372036854775808) at libavformat/rmdec.c:780
#2  0x081f1458 in rm_read_packet (s=0x9077c50, pkt=0xbff3b8f8)
    at libavformat/rmdec.c:926
#3  0x0822e097 in ff_read_packet (s=s@entry=0x9077c50, 
    pkt=pkt@entry=0xbff3b8f8) at libavformat/utils.c:642
#4  0x082309f9 in read_frame_internal (s=s@entry=0x9077c50, 
    pkt=pkt@entry=0xbff3bae8) at libavformat/utils.c:1294
#5  0x08234738 in avformat_find_stream_info (ic=0x9077c50, options=0x90791b0)
    at libavformat/utils.c:2772
#6  0x080a3b85 in open_input_file (o=o@entry=0xbff3bdc0, 
    filename=<optimized out>) at ffmpeg_opt.c:815
#7  0x080a23e9 in open_files (inout=<optimized out>, 
    inout@entry=0x887d25b "input", 
    open_file=open_file@entry=0x80a37a0 <open_input_file>, 
    l=<error reading variable: Unhandled dwarf expression opcode 0xfa>, 
    l=<error reading variable: Unhandled dwarf expression opcode 0xfa>)
    at ffmpeg_opt.c:2495
---Type <return> to continue, or q <return> to quit---
#8  0x080aa5f9 in ffmpeg_parse_options (argc=argc@entry=6, 
    argv=argv@entry=0xbff3c244) at ffmpeg_opt.c:2532
#9  0x0809faca in main (argc=6, argv=0xbff3c244) at ffmpeg.c:3367
(gdb) 

[Carl Eugen Hoyos]

Isn't this a duplicate of ticket #2724?

[ami_stuff]

Replying to cehoyos:

Isn't this a duplicate of ticket #2724?

You're right, I forgot about this ticket.