jpegls: deadlock with fuzzed file

[ami_stuff]

​http://www1.datafilehost.com/d/60cdea49

(gdb) r -threads 1 -i ./fjpegls.avi -f null -
Starting program: /media/sdb1/ffmpeg-HEAD-da30d0c/ffmpeg_g -threads 1 -i ./fjpegls.avi -f null -
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
ffmpeg version 2.0-da30d0c Copyright (c) 2000-2013 the FFmpeg developers
  built on Oct 22 2013 14:57:21 with gcc 4.7 (Debian 4.7.2-5)
  configuration: --disable-yasm --disable-ffprobe --disable-ffserver --enable-gpl
  libavutil      52. 47.101 / 52. 47.101
  libavcodec     55. 37.102 / 55. 37.102
  libavformat    55. 19.103 / 55. 19.103
  libavdevice    55.  4.100 / 55.  4.100
  libavfilter     3. 89.100 /  3. 89.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
[avi @ 0x91b4dc0] Something went wrong during header parsing, I will ignore it and try to continue anyway.
Input #0, avi, from './fjpegls.avi':
  Duration: 00:00:12.64, start: 0.000000, bitrate: 3717 kb/s
    Stream #0:0: Video: jpegls (MJLS / 0x534C4A4D), rgb24, 111x111, SAR 148:109 DAR 148:109, 23.97 tbr, 23.97 tbn, 23.97 tbc
[New Thread 0xb7df8b70 (LWP 19800)]
[New Thread 0xb75f8b70 (LWP 19801)]
[New Thread 0xb6df8b70 (LWP 19802)]
[New Thread 0xb65f8b70 (LWP 19803)]
[New Thread 0xb5df8b70 (LWP 19804)]
[New Thread 0xb55f8b70 (LWP 19805)]
[New Thread 0xb4df8b70 (LWP 19806)]
[New Thread 0xb45f8b70 (LWP 19807)]
[New Thread 0xb3df8b70 (LWP 19808)]
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf55.19.103
    Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 111x111 [SAR 148:109 DAR 148:109], q=2-31, 200 kb/s, 90k tbn, 23.97 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (jpegls -> rawvideo)
Press [q] to stop, [?] for help
[null @ 0x91b63c0] Encoder did not produce proper pts, making some up.
[jpegls @ 0x91b56c0] decode_sos: index(3) out of components
    Last message repeated 1 times
[jpegls @ 0x91b56c0] Unhandled pixel format 0x11110000
Error while decoding stream #0:0: Not yet implemented in FFmpeg, patches welcome
[jpegls @ 0x91b56c0] Subsampling in JPEG-LS is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
Error while decoding stream #0:0: Not yet implemented in FFmpeg, patches welcome
[jpegls @ 0x91b56c0] decode_sos: ac/dc index out of range
Input stream #0:0 frame changed from size:111x111 fmt:rgb24 to size:127x111 fmt:rgb24
[Thread 0xb45f8b70 (LWP 19807) exited]
[Thread 0xb65f8b70 (LWP 19803) exited]
[Thread 0xb55f8b70 (LWP 19805) exited]
[Thread 0xb5df8b70 (LWP 19804) exited]
[Thread 0xb6df8b70 (LWP 19802) exited]
[Thread 0xb7df8b70 (LWP 19800) exited]
[Thread 0xb3df8b70 (LWP 19808) exited]
[Thread 0xb4df8b70 (LWP 19806) exited]
[Thread 0xb75f8b70 (LWP 19801) exited]
[New Thread 0xb3df8b70 (LWP 19818)]
[New Thread 0xb45f8b70 (LWP 19819)]
[New Thread 0xb4df8b70 (LWP 19820)]
[New Thread 0xb55f8b70 (LWP 19821)]
[New Thread 0xb5df8b70 (LWP 19822)]
[New Thread 0xb7df8b70 (LWP 19823)]
[New Thread 0xb75f8b70 (LWP 19824)]
[New Thread 0xb6df8b70 (LWP 19834)]
[New Thread 0xb65f8b70 (LWP 19835)]
[jpegls @ 0x91b56c0] JPEG-LS that is not <= 8 bits/component or 16-bit gray is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
Error while decoding stream #0:0: Not yet implemented in FFmpeg, patches welcome
Input stream #0:0 frame changed from size:127x111 fmt:rgb24 to size:111x111 fmt:rgb24
[Thread 0xb55f8b70 (LWP 19821) exited]
[Thread 0xb3df8b70 (LWP 19818) exited]
[Thread 0xb6df8b70 (LWP 19834) exited]
[Thread 0xb7df8b70 (LWP 19823) exited]
[Thread 0xb4df8b70 (LWP 19820) exited]
[Thread 0xb45f8b70 (LWP 19819) exited]
[Thread 0xb75f8b70 (LWP 19824) exited]
[Thread 0xb5df8b70 (LWP 19822) exited]
[Thread 0xb65f8b70 (LWP 19835) exited]
[New Thread 0xb65f8b70 (LWP 19847)]
[New Thread 0xb6df8b70 (LWP 19848)]
[New Thread 0xb75f8b70 (LWP 19849)]
[New Thread 0xb7df8b70 (LWP 19851)]
[New Thread 0xb5df8b70 (LWP 19852)]
[New Thread 0xb55f8b70 (LWP 19853)]
[New Thread 0xb4df8b70 (LWP 19854)]
[New Thread 0xb45f8b70 (LWP 19855)]
[New Thread 0xb3df8b70 (LWP 19856)]
[jpegls @ 0x91b56c0] decode_sos: invalid len (44)
[jpegls @ 0x91b56c0] decode_sos: ac/dc index out of range
[jpegls @ 0x91b56c0] Subsampling in JPEG-LS is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
Error while decoding stream #0:0: Not yet implemented in FFmpeg, patches welcome
[jpegls @ 0x91b56c0] decode_sos: invalid len (28)
[jpegls @ 0x91b56c0] Invalid sampling factor in component 0 1:0
Error while decoding stream #0:0: Invalid data found when processing input
[jpegls @ 0x91b56c0] Can not process SOS before SOF, skipping
[jpegls @ 0x91b56c0] Found EOI before any SOF, ignoring
[jpegls @ 0x91b56c0] No JPEG data found in image
Error while decoding stream #0:0: Invalid data found when processing input
[jpegls @ 0x91b56c0] Can not process SOS before SOF, skipping
[jpegls @ 0x91b56c0] Found EOI before any SOF, ignoring
[jpegls @ 0x91b56c0] No JPEG data found in image
Error while decoding stream #0:0: Invalid data found when processing input
[jpegls @ 0x91b56c0] Found EOI before any SOF, ignoring
[jpegls @ 0x91b56c0] decode_sos: invalid len (140)
[jpegls @ 0x91b56c0] invalid id 137
Error while decoding stream #0:0: Invalid data found when processing input
[jpegls @ 0x91b56c0] invalid id 0
Error while decoding stream #0:0: Invalid data found when processing input
[jpegls @ 0x91b56c0] Subsampling in JPEG-LS is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
Error while decoding stream #0:0: Not yet implemented in FFmpeg, patches welcome
[jpegls @ 0x91b56c0] decode_sos: index(3) out of components
[jpegls @ 0x91b56c0] Can not process SOS before SOF, skipping
[jpegls @ 0x91b56c0] Found EOI before any SOF, ignoring
[jpegls @ 0x91b56c0] No JPEG data found in image
Error while decoding stream #0:0: Invalid data found when processing input
[jpegls @ 0x91b56c0] Subsampling in JPEG-LS is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
Error while decoding stream #0:0: Not yet implemented in FFmpeg, patches welcome
[jpegls @ 0x91b56c0] decode_sos: invalid len (76)

Program received signal SIGINT, Interrupt.
0x0855f168 in ls_get_code_runterm (limit_add=0, RItype=1, state=0x91a72a0, 
    gb=0x91aaa88) at libavcodec/jpeglsdec.c:128
128	    for (k = 0; (state->N[Q] << k) < temp; k++)
(gdb) bt
#0  0x0855f168 in ls_get_code_runterm (limit_add=0, RItype=1, state=0x91a72a0, 
    gb=0x91aaa88) at libavcodec/jpeglsdec.c:128
#1  ls_decode_line (state=state@entry=0x91a72a0, s=s@entry=0x91aaa80, 
    last=last@entry=0xb32041d2, dst=dst@entry=0xb320a382, last2=255, 
    w=w@entry=24909, stride=stride@entry=3, comp=comp@entry=2, bits=8)
    at libavcodec/jpeglsdec.c:221
#2  0x08561456 in ff_jpegls_decode_picture (s=s@entry=0x91aaa80, 
    near=near@entry=128, point_transform=point_transform@entry=0, 
    ilv=ilv@entry=1) at libavcodec/jpeglsdec.c:346
#3  0x0857bbd3 in ff_mjpeg_decode_sos (s=s@entry=0x91aaa80, 
    mb_bitmask=mb_bitmask@entry=0x0, reference=reference@entry=0x0)
    at libavcodec/mjpegdec.c:1376
#4  0x0857dabd in ff_mjpeg_decode_frame (avctx=0x91b56c0, data=0x91aa3c0, 
    got_frame=0xbffff4e4, avpkt=0xbffff288) at libavcodec/mjpegdec.c:1926
#5  0x086c8026 in avcodec_decode_video2 (avctx=0x91b56c0, 
    picture=picture@entry=0x91aa3c0, 
    got_picture_ptr=got_picture_ptr@entry=0xbffff4e4, 
    avpkt=avpkt@entry=0xbffff730) at libavcodec/utils.c:2007
#6  0x080b64fd in decode_video (ist=ist@entry=0x91b6040, 
    pkt=pkt@entry=0xbffff730, got_output=got_output@entry=0xbffff4e4)
    at ffmpeg.c:1668
#7  0x080ba41a in output_packet (pkt=0xbffff6c8, ist=0x91b6040)
    at ffmpeg.c:1866
---Type <return> to continue, or q <return> to quit---
#8  process_input (file_index=1) at ffmpeg.c:3104
#9  0x080a5b83 in transcode_step () at ffmpeg.c:3200
#10 transcode () at ffmpeg.c:3252
#11 main (argc=<optimized out>, argv=<optimized out>) at ffmpeg.c:3430
(gdb) 

[Michael Niedermayer]

Fixed in cc0e47b55096361723b364afa43b79a3f5619cdc