Opened 10 years ago
Closed 10 years ago
#3143 closed defect (fixed)
H.261 encoding crashes with trellis
Reported by: | Maik Merten | Owned by: | |
---|---|---|---|
Priority: | important | Component: | avcodec |
Version: | git-master | Keywords: | h261 crash SIGSEGV |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | yes | |
Analyzed by developer: | no |
Description (last modified by )
Summary of the bug:
How to reproduce:
% ffmpeg -i /tmp/test.y4m -f h261 -vb 256k -trellis 2 /tmp/test.h261 ffmpeg version N-58112-g5592d1b Copyright (c) 2000-2013 the FFmpeg developers built on Nov 15 2013 18:16:18 with gcc 4.8 (Ubuntu/Linaro 4.8.1-10ubuntu8)
There's some revived interest in H.261 as possible guaranteed-patent-free baseline codec for videocommunication. While ffmpeg's H.261 usually works fine, it does not work with advanced techniques such as trellis quantization (or most rate/distortion options, for that matter). This is about trellis.
It would be awesome if some of the cool advanced encoder features that are available for MPEG-1 (e.g., trellis) would also work for H.261. Think "pig with rockets attached". If this is not possible the encoder at least should not crash.
Program received signal SIGSEGV, Segmentation fault. 0x000000000083536a in dct_quantize_trellis_c (s=0x16342a0, block=0x1626d60, n=1, qscale=4, overflow=<optimized out>) at libavcodec/mpegvideo_enc.c:3619 3619 int score= distortion + length[UNI_AC_ENC_INDEX(run, level)]*lambda; (gdb) bt #0 0x000000000083536a in dct_quantize_trellis_c (s=0x16342a0, block=0x1626d60, n=1, qscale=4, overflow=<optimized out>) at libavcodec/mpegvideo_enc.c:3619 #1 0x0000000000842d13 in encode_mb_internal (mb_block_count=6, mb_block_width=8, mb_block_height=8, motion_y=0, motion_x=0, s=0x16342a0) at libavcodec/mpegvideo_enc.c:2060 #2 encode_mb (motion_y=0, motion_x=0, s=0x16342a0) at libavcodec/mpegvideo_enc.c:2168 #3 encode_thread (c=<optimized out>, arg=<optimized out>) at libavcodec/mpegvideo_enc.c:3042 #4 0x000000000090d8bf in avcodec_default_execute (c=0x1633b20, func=0x83f8b0 <encode_thread>, arg=<optimized out>, ret=<optimized out>, count=1, size=8) at libavcodec/utils.c:1016 #5 0x000000000083a2c7 in encode_picture (picture_number=0, s=0x16342a0) at libavcodec/mpegvideo_enc.c:3435 #6 ff_MPV_encode_picture (avctx=0x1633b20, pkt=0x7fffffffd910, pic_arg=<optimized out>, got_packet=0x7fffffffd90c) at libavcodec/mpegvideo_enc.c:1494 #7 0x000000000090e6a5 in avcodec_encode_video2 (avctx=avctx@entry=0x1633b20, avpkt=avpkt@entry=0x7fffffffd910, frame=frame@entry=0x1628700, got_packet_ptr=got_packet_ptr@entry=0x7fffffffd90c) at libavcodec/utils.c:1861 #8 0x000000000046da85 in do_video_out (in_picture=0x1628700, ost=0x1633f80, ---Type <return> to continue, or q <return> to quit--- s=0x1633200) at ffmpeg.c:953 #9 reap_filters () at ffmpeg.c:1098 #10 0x000000000045d1df in transcode_step () at ffmpeg.c:3223 #11 transcode () at ffmpeg.c:3266 #12 main (argc=<optimized out>, argv=<optimized out>) at ffmpeg.c:3444
Change History (6)
comment:1 by , 10 years ago
comment:2 by , 10 years ago
Component: | undetermined → avcodec |
---|---|
Description: | modified (diff) |
Keywords: | crash SIGSEGV added; crasher removed |
comment:3 by , 10 years ago
I'm testing with uncompressed test sequences stored in YUV4MPEG format. Suitable input is e.g.
http://media.xiph.org/video/derf/y4m/mad900_cif.y4m (this sequence is so totally 90ies).
Will try to test an old version.
comment:4 by , 10 years ago
Version 0.5.13 behaves in a similar way, crashes at the same statement in the corresponding file.
Starting program: /home/maik/builds/ffmpeg-0.5.13/ffmpeg_g -i /tmp/test.y4m -f h261 -vb 256k -trellis 2 /tmp/test.h261 FFmpeg version 0.5.13, Copyright (c) 2000-2013 Fabrice Bellard, et al.
Program received signal SIGSEGV, Segmentation fault. 0x0000000000521add in dct_quantize_trellis_c (s=0xe9d6e0, block=0xeadeb0, n=1, qscale=4, overflow=<optimized out>) at libavcodec/mpegvideo_enc.c:3100 3100 int score= distortion + length[UNI_AC_ENC_INDEX(run, level)]*lambda; (gdb) bt #0 0x0000000000521add in dct_quantize_trellis_c (s=0xe9d6e0, block=0xeadeb0, n=1, qscale=4, overflow=<optimized out>) at libavcodec/mpegvideo_enc.c:3100 #1 0x0000000000535f6f in encode_mb_internal (mb_block_count=6, mb_block_height=8, motion_y=0, motion_x=0, s=0xe9d6e0) at libavcodec/mpegvideo_enc.c:1635 #2 encode_mb (motion_y=0, motion_x=0, s=0xe9d6e0) at libavcodec/mpegvideo_enc.c:1731 #3 encode_thread (c=<optimized out>, arg=<optimized out>) at libavcodec/mpegvideo_enc.c:2554 #4 0x00000000004d623f in avcodec_default_execute (c=0xe94f70, func=0x532ad0 <encode_thread>, arg=<optimized out>, ret=<optimized out>, count=1, size=8) at libavcodec/utils.c:399 #5 0x0000000000524e51 in encode_picture (picture_number=0, s=0xe9d6e0) at libavcodec/mpegvideo_enc.c:2912 #6 MPV_encode_picture (avctx=<optimized out>, buf=<optimized out>, buf_size=<optimized out>, data=<optimized out>) at libavcodec/mpegvideo_enc.c:1214 #7 0x00000000004d66f6 in avcodec_encode_video (avctx=avctx@entry=0xe94f70, buf=<optimized out>, buf_size=<optimized out>, pict=pict@entry=0x7fffffffd340) at libavcodec/utils.c:515 #8 0x00000000004378c5 in do_video_out (ist=0x0, frame_size=<synthetischer Zeiger>, in_picture=0x7fffffffcfe0, ost=0xe9d510, s=<optimized out>) at ffmpeg.c:974 #9 output_packet (ist=ist@entry=0xe9d4a0, ist_index=ist_index@entry=0, ost_table=ost_table@entry=0xe9d4f0, nb_ostreams=nb_ostreams@entry=1, pkt=pkt@entry=0x7fffffffd560) at ffmpeg.c:1358 #10 0x000000000043a5fc in av_encode (nb_output_files=nb_output_files@entry=1, nb_input_files=nb_input_files@entry=1, nb_stream_maps=<optimized out>, stream_maps=0xb23420 <stream_maps>, input_files=0xb27420 <input_files>, output_files=0xb259c0 <output_files>) at ffmpeg.c:2153 #11 0x0000000000435018 in main (argc=<optimized out>, argv=<optimized out>) at ffmpeg.c:3928
comment:5 by , 10 years ago
Keywords: | h261 added |
---|---|
Priority: | normal → important |
Reproduced by developer: | set |
Status: | new → open |
(gdb) r -i tests/lena.pnm -s 176x144 -trellis 2 out.h261 Starting program: ffmpeg_g -i tests/lena.pnm -s 176x144 -trellis 2 out.h261 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". ffmpeg version N-58200-g92cbd77 Copyright (c) 2000-2013 the FFmpeg developers built on Nov 17 2013 04:21:45 with gcc 4.7 (SUSE Linux) configuration: --enable-gpl libavutil 52. 53.100 / 52. 53.100 libavcodec 55. 43.100 / 55. 43.100 libavformat 55. 21.100 / 55. 21.100 libavdevice 55. 5.100 / 55. 5.100 libavfilter 3. 91.100 / 3. 91.100 libswscale 2. 5.101 / 2. 5.101 libswresample 0. 17.104 / 0. 17.104 libpostproc 52. 3.100 / 52. 3.100 Input #0, image2, from 'tests/lena.pnm': Duration: 00:00:00.04, start: 0.000000, bitrate: N/A Stream #0:0: Video: ppm, rgb24, 256x256, 25 tbr, 25 tbn, 25 tbc [New Thread 0x7ffff59eb700 (LWP 23636)] [New Thread 0x7ffff51ea700 (LWP 23637)] [New Thread 0x7ffff49e9700 (LWP 23638)] [New Thread 0x7ffff41e8700 (LWP 23639)] [New Thread 0x7ffff39e7700 (LWP 23640)] [New Thread 0x7ffff31e6700 (LWP 23641)] [New Thread 0x7ffff29e5700 (LWP 23642)] [New Thread 0x7ffff21e4700 (LWP 23643)] [New Thread 0x7ffff19e3700 (LWP 23644)] Output #0, h261, to 'out.h261': Metadata: encoder : Lavf55.21.100 Stream #0:0: Video: h261, yuv420p, 176x144, q=2-31, 200 kb/s, 90k tbn, 25 tbc Stream mapping: Stream #0:0 -> #0:0 (ppm -> h261) Press [q] to stop, [?] for help Program received signal SIGSEGV, Segmentation fault. 0x000000000093d2f9 in dct_quantize_trellis_c (s=0x17cb5e0, block=0x17ef2a0, n=0, qscale=3, overflow=0x7fff0000001a) at libavcodec/mpegvideo_enc.c:3619 3619 int score= distortion + length[UNI_AC_ENC_INDEX(run, level)]*lambda; (gdb) bt #0 0x000000000093d2f9 in dct_quantize_trellis_c (s=0x17cb5e0, block=0x17ef2a0, n=0, qscale=3, overflow=0x7fff0000001a) at libavcodec/mpegvideo_enc.c:3619 #1 0x00000000009488c1 in encode_mb_internal (mb_block_count=6, mb_block_width=8, mb_block_height=8, motion_y=0, motion_x=0, s=0x17cb5e0) at libavcodec/mpegvideo_enc.c:2060 #2 encode_mb (motion_y=0, motion_x=0, s=0x17cb5e0) at libavcodec/mpegvideo_enc.c:2168 #3 encode_thread (c=<optimized out>, arg=<optimized out>) at libavcodec/mpegvideo_enc.c:3042 #4 0x0000000000a2fda7 in avcodec_default_execute (c=0x17cafa0, func=0x945310 <encode_thread>, arg=<optimized out>, ret=<optimized out>, count=1, size=8) at libavcodec/utils.c:1016 #5 0x000000000094fc21 in encode_picture (picture_number=0, s=0x17cb5e0) at libavcodec/mpegvideo_enc.c:3435 #6 ff_MPV_encode_picture (avctx=0x17cafa0, pkt=0x7fffffffda60, pic_arg=<optimized out>, got_packet=0x7fffffffd91c) at libavcodec/mpegvideo_enc.c:1494 #7 0x0000000000a30ff7 in avcodec_encode_video2 (avctx=avctx@entry=0x17cafa0, avpkt=avpkt@entry=0x7fffffffda60, frame=frame@entry=0x182d3a0, got_packet_ptr=got_packet_ptr@entry=0x7fffffffd91c) at libavcodec/utils.c:1863 #8 0x0000000000471552 in do_video_out (in_picture=0x182d3a0, ost=0x17cb400, s=0x17caa00) at ffmpeg.c:965 #9 reap_filters () at ffmpeg.c:1110 #10 0x0000000000461548 in transcode_step () at ffmpeg.c:3235 #11 transcode () at ffmpeg.c:3278 #12 main (argc=<optimized out>, argv=<optimized out>) at ffmpeg.c:3456 (gdb) disass $pc-32,$pc+32 Dump of assembler code from 0x93d2d9 to 0x93d319: 0x000000000093d2d9 <dct_quantize_trellis_c+1177>: jmp 0x93d2eb <dct_quantize_trellis_c+1195> 0x000000000093d2db <dct_quantize_trellis_c+1179>: nopl 0x0(%rax,%rax,1) 0x000000000093d2e0 <dct_quantize_trellis_c+1184>: movslq %edx,%rax 0x000000000093d2e3 <dct_quantize_trellis_c+1187>: movslq 0x4e0(%rsp,%rax,4),%rsi 0x000000000093d2eb <dct_quantize_trellis_c+1195>: mov %r9d,%edi 0x000000000093d2ee <dct_quantize_trellis_c+1198>: sub %esi,%edi 0x000000000093d2f0 <dct_quantize_trellis_c+1200>: mov %edi,%eax 0x000000000093d2f2 <dct_quantize_trellis_c+1202>: shl $0x7,%eax 0x000000000093d2f5 <dct_quantize_trellis_c+1205>: add %ebx,%eax 0x000000000093d2f7 <dct_quantize_trellis_c+1207>: cltq => 0x000000000093d2f9 <dct_quantize_trellis_c+1209>: movzbl 0x0(%r13,%rax,1),%eax 0x000000000093d2ff <dct_quantize_trellis_c+1215>: imul %r15d,%eax 0x000000000093d303 <dct_quantize_trellis_c+1219>: add %r10d,%eax 0x000000000093d306 <dct_quantize_trellis_c+1222>: add 0x3d0(%rsp,%rsi,4),%eax 0x000000000093d30d <dct_quantize_trellis_c+1229>: cmp %ecx,%eax 0x000000000093d30f <dct_quantize_trellis_c+1231>: jge 0x93d323 <dct_quantize_trellis_c+1251> 0x000000000093d311 <dct_quantize_trellis_c+1233>: mov %edi,0x1b0(%rsp,%r8,4) End of assembler dump. (gdb) info register rax 0x41 65 rbx 0x41 65 rcx 0x78000000 2013265920 rdx 0x0 0 rsi 0x1 1 rdi 0x0 0 rbp 0x0 0x0 rsp 0x7fffffff0bc0 0x7fffffff0bc0 r8 0x2 2 r9 0x1 1 r10 0xfffffe20 4294966816 r11 0x1 1 r12 0x0 0 r13 0x0 0 r14 0x0 0 r15 0x24f 591 rip 0x93d2f9 0x93d2f9 <dct_quantize_trellis_c+1209> eflags 0x10206 [ PF IF RF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0
comment:6 by , 10 years ago
Resolution: | → fixed |
---|---|
Status: | open → closed |
Implemented in 331a90cec42f600c8b63231bb0d5be6df6bf1717
Is this reproducible with
-f lavfi -i testsrc
(or tests/lena.pnm) as input or does it need a specific file to reproduce the crash?Could you test a very old version (0.5 or 0.7)?