jpeg2000: infinite loop with rpcl and 30 levels

[ami_stuff]

I can't get backtrace under mingw (CTRL+C quits gdb)

C:\>kdu_compress.exe -i test.bmp -o levels30.jp2 Clevels=30 Corder=RPCL
Note:
    The default rate control policy for colour images employs visual (CSF)
    weighting factors.  To minimize MSE instead, specify `-no_weights'.
Note:
    If you want quality scalability, you should generate multiple layers with
    `-rate' or by using the "Clayers" option.

Generated 1 tile-part(s) for a total of 1 tile(s).
Code-stream bytes (excluding any file format) = 388,919 = 4.158166 bits/pel.
Layer bit-rates (possibly inexact if tiles are divided across tile-parts):
        4.158166
Layer thresholds:
        0
Processed using the multi-threaded environment, with
    8 parallel threads of execution

C:\>ffmpeg -i levels30.jp2 out.bmp
ffmpeg version N-73069-g6504047 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.5.0 (GCC) 20100414 (Fedora MinGW 4.5.0-1.fc14)
  configuration: --prefix=/var/www/users/research/ffmpeg/snapshots/build --arch=
x86 --target-os=mingw32 --cross-prefix=i686-pc-mingw32- --cc='ccache i686-pc-min
gw32-gcc' --enable-pthreads --enable-memalign-hack --enable-runtime-cpudetect --
enable-cross-compile --enable-static --disable-shared --extra-libs='-lws2_32 -lw
inmm -lpthread' --extra-cflags='--static -I/var/www/users/research/ffmpeg/snapsh
ots/build/include' --extra-ldflags='-static -L/var/www/users/research/ffmpeg/sna
pshots/build/lib' --enable-bzlib --enable-zlib --enable-gpl --enable-version3 --
enable-nonfree --enable-libx264 --enable-libspeex --enable-libtheora --enable-li
bvorbis --enable-libfaac --enable-libxvid --enable-libopencore-amrnb --enable-li
bopencore-amrwb --enable-libmp3lame --enable-libfreetype --enable-libvpx --disab
le-decoder=libvpx
  libavutil      54. 27.100 / 54. 27.100
  libavcodec     56. 42.101 / 56. 42.101
  libavformat    56. 37.100 / 56. 37.100
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 17.100 /  5. 17.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.100 /  1.  2.100
  libpostproc    53.  3.100 / 53.  3.100
[jpeg2000 @ 0x298f820] Progression order RPCL
[jpeg2000 @ 0x298f820] prc -1 0 outside limits 1 1
    Last message ^Cpeated 808794 times

[Carl Eugen Hoyos]

(gdb) r -i levels30.jp2
Starting program: ffmpeg_g -i levels30.jp2
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ffmpeg version N-73119-g041aa80 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      54. 27.100 / 54. 27.100
  libavcodec     56. 44.101 / 56. 44.101
  libavformat    56. 38.101 / 56. 38.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 17.100 /  5. 17.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.100 /  1.  2.100
  libpostproc    53.  3.100 / 53.  3.100
[jpeg2000 @ 0x1bb12c0] Progression order RPCL
[jpeg2000 @ 0x1bb12c0] prc -1 0 outside limits 1 1
    Last message repeated 185858 times
Program received signal SIGINT, Interrupt.
0x00007ffff524d230 in __write_nocancel () from /lib64/libc.so.6
(gdb) bt
#0  0x00007ffff524d230 in __write_nocancel () from /lib64/libc.so.6
#1  0x00007ffff51e9123 in _IO_new_file_write () from /lib64/libc.so.6
#2  0x00007ffff51e9002 in new_do_write () from /lib64/libc.so.6
#3  0x00007ffff51e9c1e in _IO_new_file_xsputn () from /lib64/libc.so.6
#4  0x00007ffff51bf034 in buffered_vfprintf () from /lib64/libc.so.6
#5  0x00007ffff51b98ae in vfprintf () from /lib64/libc.so.6
#6  0x00007ffff51c3ec7 in fprintf () from /lib64/libc.so.6
#7  0x0000000000f3906e in av_log_default_callback (ptr=<optimized out>, level=24, fmt=<optimized out>, vl=<optimized out>) at libavutil/log.c:327
#8  0x0000000000f39371 in av_vlog (vl=0x7fffffffced8, fmt=0x1084370 "prc %d %d outside limits %d %d\n", level=<optimized out>, avcl=<optimized out>) at libavutil/log.c:374
#9  av_log (avcl=<optimized out>, level=<optimized out>, level@entry=24, fmt=fmt@entry=0x1084370 "prc %d %d outside limits %d %d\n") at libavutil/log.c:366
#10 0x0000000000944fe6 in jpeg2000_decode_packets (tile=0x1bb2840, s=0x1bb1a60) at libavcodec/jpeg2000dec.c:1121
#11 jpeg2000_read_bitstream_packets (s=0xffffffffffffffc6) at libavcodec/jpeg2000dec.c:1800
#12 jpeg2000_decode_frame (avctx=<optimized out>, data=0x1bb2560, got_frame=0x7fffffffd28c, avpkt=<optimized out>) at libavcodec/jpeg2000dec.c:1966
#13 0x0000000000b07938 in avcodec_decode_video2 (avctx=0x1bb12c0, picture=0x1bb2560, got_picture_ptr=got_picture_ptr@entry=0x7fffffffd28c, avpkt=avpkt@entry=0x7fffffffd2c0) at libavcodec/utils.c:2388
#14 0x0000000000656fd1 in try_decode_frame (s=s@entry=0x1bb03e0, st=st@entry=0x1bb0ec0, avpkt=avpkt@entry=0x1baf640, options=0x1bafa60) at libavformat/utils.c:2672
#15 0x000000000065fc59 in avformat_find_stream_info (ic=0x1bb03e0, options=0x1bafa60) at libavformat/utils.c:3315
#16 0x0000000000480a11 in open_input_file (o=o@entry=0x7fffffffd700, filename=<optimized out>) at ffmpeg_opt.c:917
#17 0x000000000047a924 in open_files (inout=inout@entry=0xfe0dbf "input", open_file=open_file@entry=0x480460 <open_input_file>, l=<optimized out>, l=<optimized out>) at ffmpeg_opt.c:2816
#18 0x0000000000482289 in ffmpeg_parse_options (argc=argc@entry=3, argv=argv@entry=0x7fffffffdd48) at ffmpeg_opt.c:2853
#19 0x0000000000473123 in main (argc=3, argv=0x7fffffffdd48) at ffmpeg.c:4071

[Michael Niedermayer]

Fixed in 9ff5e7974a8fe32c3b595a481c06a41fa316a9ac