Opened 7 years ago

Closed 6 years ago

#1097 closed defect (fixed)

prores_kostya: invalid read

Reported by: ami_stuff Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: prores
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

-vcodec prores works ok

(gdb) r -i 600.png -vcodec prores_kostya out.mov
Starting program: d:\mingw\msys\1.0\ffmpeg\ffmpeg_g.exe -i 600.png -vcodec prore
s_kostya out.mov
[New Thread 448.0xbe4]
ffmpeg version 0.9.1.git Copyright (c) 2000-2012 the FFmpeg developers
  built on Mar 10 2012 16:15:15 with gcc 4.6.1
  configuration: --disable-yasm --disable-ffprobe
  libavutil      51. 42.100 / 51. 42.100
  libavcodec     54. 10.100 / 54. 10.100
  libavformat    54.  2.100 / 54.  2.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 63.100 /  2. 63.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  7.100 /  0.  7.100
Input #0, image2, from '600.png':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: png, rgb24, 600x412, 25 tbr, 25 tbn, 25 tbc
File 'out.mov' already exists. Overwrite ? [y/N] y
Incompatible pixel format 'rgb24' for codec 'prores_kostya', auto-selecting form
at 'yuv444p10le'
[buffer @ 03871cc0] w:600 h:412 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 03871f40] auto-inserting filter 'auto-inserted scale 0' between th
e filter 'src' and the filter 'out'
[scale @ 038714c0] w:600 h:412 fmt:rgb24 -> w:600 h:412 fmt:yuv444p10le flags:0x
4
Output #0, mov, to 'out.mov':
  Metadata:
    encoder         : Lavf54.2.100
    Stream #0:0: Video: prores (apcn / 0x6E637061), yuv444p10le, 600x412, q=2-31
, 200 kb/s, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (png -> prores_kostya)
Press [q] to stop, [?] for help

Program received signal SIGSEGV, Segmentation fault.
prores_fdct_c (src=0x411dc40, linesize=1216, block=0x3865650)
    at libavcodec/proresdsp.c:64
64                  block[y * 8 + x] = tsrc[x];
(gdb) bt
#0  prores_fdct_c (src=0x411dc40, linesize=1216, block=0x3865650)
    at libavcodec/proresdsp.c:64
#1  0x00805fc0 in get_slice_data (ctx=0x38635c0, src=0x411b640,
    linesize=1216, x=<optimized out>, y=400, w=600, h=412, blocks=0x3865650,
    mbs_per_slice=8, blocks_per_mb=4, is_chroma=1)
    at libavcodec/proresenc_kostya.c:257
#2  0x0080683f in find_slice_quant (mbs_per_slice=8, y=25, x=<optimized out>,
    trellis_node=16, pic=0x22d9d8, avctx=0x386ab20)
    at libavcodec/proresenc_kostya.c:582
#3  encode_frame (avctx=0x386ab20, pkt=0x22db40, pic=0x22d9d8,
    got_packet=0x22dc0c) at libavcodec/proresenc_kostya.c:756
#4  0x004f945d in avcodec_encode_video2 (avctx=0x386ab20, avpkt=0x22db40,
    frame=0x22d9d8, got_packet_ptr=0x22dc0c) at libavcodec/utils.c:1219
#5  0x00405de0 in do_video_out (s=0x386a620, ost=0x3863320,
    in_picture=0x3873800, ist=<optimized out>) at ffmpeg.c:1619
#6  0x00407d6c in transcode_video (pkt_pts=<optimized out>,
    got_output=<optimized out>, pkt=<optimized out>, ist=<optimized out>)
    at ffmpeg.c:2178
#7  output_packet (ist=0x3871640, ost_table=0x3863320, nb_ostreams=1,
    pkt=0x22fb28) at ffmpeg.c:2270
#8  0x0040bf3b in transcode (output_files=0x3871a80, nb_output_files=1,
    input_files=0x3871700, nb_input_files=1) at ffmpeg.c:3082
#9  0x0022ff48 in ?? ()
Backtrace stopped: Not enough registers or memory available to unwind further

Attachments (1)

600.png (1.7 KB) - added by ami_stuff 7 years ago.

Download all attachments as: .zip

Change History (9)

Changed 7 years ago by ami_stuff

comment:1 Changed 7 years ago by dbuitenh

I cannot reproduce this with either ffmpeg or libav. :|

comment:2 Changed 7 years ago by ami_stuff

That's strange, crashes here as well with two builds not compiled be me, including this one:

http://ffmpeg.zeranoe.com/builds/

$ gdb ffmpeg1.exe
GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from d:\mingw\msys\1.0\ffmpeg\ffmpeg1.exe...(no debugging symbol
s found)...done.
(gdb) r -i 600.png -vcodec prores_kostya out.mov
Starting program: d:\mingw\msys\1.0\ffmpeg\ffmpeg1.exe -i 600.png -vcodec prores
_kostya out.mov
[New Thread 3600.0x858]
ffmpeg version N-38622-g1eabd71 Copyright (c) 2000-2012 the FFmpeg developers
  built on Mar  7 2012 00:18:03 with gcc 4.6.2
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-ru
ntime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libope
ncore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --en
able-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger -
-enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwben
c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-
libxvid --enable-zlib
  libavutil      51. 42.100 / 51. 42.100
  libavcodec     54. 10.100 / 54. 10.100
  libavformat    54.  2.100 / 54.  2.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 63.100 /  2. 63.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  7.100 /  0.  7.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, image2, from '600.png':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: png, rgb24, 600x412, 25 tbr, 25 tbn, 25 tbc
File 'out.mov' already exists. Overwrite ? [y/N] y
Incompatible pixel format 'rgb24' for codec 'prores_kostya', auto-selecting form
at 'yuv444p10le'
[buffer @ 02161F20] w:600 h:412 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 021614E0] auto-inserting filter 'auto-inserted scale 0' between th
e filter 'src' and the filter 'out'
[scale @ 02161FC0] w:600 h:412 fmt:rgb24 -> w:600 h:412 fmt:yuv444p10le flags:0x
4
Output #0, mov, to 'out.mov':
  Metadata:
    encoder         : Lavf54.2.100
    Stream #0:0: Video: prores (apcn / 0x6E637061), yuv444p10le, 600x412, q=2-31
, 200 kb/s, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (png -> prores_kostya)
Press [q] to stop, [?] for help

Program received signal SIGSEGV, Segmentation fault.
0x009c30f4 in ?? ()
(gdb)
$ gdb ffmpeg2.exe
GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from d:\mingw\msys\1.0\ffmpeg\ffmpeg2.exe...(no debugging symbol
s found)...done.
(gdb) r -i 600.png -vcodec prores_kostya out.mov
Starting program: d:\mingw\msys\1.0\ffmpeg\ffmpeg2.exe -i 600.png -vcodec prores
_kostya out.mov
[New Thread 2364.0x974]
ffmpeg version N-38862-g967bdb8 Copyright (c) 2000-2012 the FFmpeg developers
  built on Mar 18 2012 02:23:57 with gcc 4.5.0 20100414 (Fedora MinGW 4.5.0-1.fc
14)
  configuration: --prefix=/var/www/users/research/ffmpeg/snapshots/build --arch=
x86 --target-os=mingw32 --cross-prefix=i686-pc-mingw32- --cc='ccache i686-pc-min
gw32-gcc' --enable-w32threads --enable-memalign-hack --enable-runtime-cpudetect
--enable-cross-compile --enable-static --disable-shared --extra-libs='-lws2_32 -
lwinmm' --extra-cflags='--static -I/var/www/users/research/ffmpeg/snapshots/buil
d/include' --extra-ldflags='-static -L/var/www/users/research/ffmpeg/snapshots/b
uild/lib' --enable-bzlib --enable-zlib --enable-gpl --enable-version3 --enable-n
onfree --enable-libx264 --enable-libspeex --enable-libtheora --enable-libvorbis
--enable-libfaac --enable-libxvid --enable-libopencore-amrnb --enable-libopencor
e-amrwb --enable-libmp3lame --enable-libvpx --disable-decoder=libvpx
  libavutil      51. 42.100 / 51. 42.100
  libavcodec     54. 10.100 / 54. 10.100
  libavformat    54.  2.100 / 54.  2.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 65.100 /  2. 65.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  7.100 /  0.  7.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, image2, from '600.png':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: png, rgb24, 600x412, 25 tbr, 25 tbn, 25 tbc
File 'out.mov' already exists. Overwrite ? [y/N] y
Incompatible pixel format 'rgb24' for codec 'prores_kostya', auto-selecting form
at 'yuv444p10le'
[buffer @ 0x1d91cc0] w:600 h:412 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 0x1d91f40] auto-inserting filter 'auto-inserted scale 0' between t
he filter 'src' and the filter 'out'
[scale @ 0x1d914c0] w:600 h:412 fmt:rgb24 -> w:600 h:412 fmt:yuv444p10le flags:0
x4
Output #0, mov, to 'out.mov':
  Metadata:
    encoder         : Lavf54.2.100
    Stream #0:0: Video: prores (apcn / 0x6E637061), yuv444p10le, 600x412, q=2-31
, 200 kb/s, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (png -> prores_kostya)
Press [q] to stop, [?] for help

Program received signal SIGSEGV, Segmentation fault.
0x00985717 in ?? ()
(gdb)

comment:3 Changed 7 years ago by cbsrobot

Crashes for me too:

gdb ~/devel/ffmpeg/ffmpeg_g 
GNU gdb 6.3.50-20050815 (Apple version gdb-1515) (Sat Jan 15 08:33:48 UTC 2011)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared libraries .
.... done
ffmpeg version N-38872-g106ea6a Copyright (c) 2000-2012 the FFmpeg developers
  built on Mar 18 2012 21:12:20 with gcc 4.2.1 (Apple Inc. build 5666) (dot 3)
  configuration: --enable-gpl --enable-version3 --enable-nonfree --enable-postproc --enable-libfaac --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-libvpx --enable-libmp3lame --enable-libfreetype --enable-libopenjpeg --enable-libass --prefix=/usr/local
  libavutil      51. 42.100 / 51. 42.100
  libavcodec     54. 10.100 / 54. 10.100
  libavformat    54.  2.100 / 54.  2.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 65.100 /  2. 65.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  7.100 /  0.  7.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, image2, from '/Users/cbsrobot/Downloads/600.png':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: png, rgb24, 600x412, 25 tbr, 25 tbn, 25 tbc
File '/Users/cbsrobot/Desktop/test.mov' already exists. Overwrite ? [y/N] y
Incompatible pixel format 'rgb24' for codec 'prores_kostya', auto-selecting format 'yuv444p10le'
[buffer @ 0x101c122e0] w:600 h:412 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 0x101c12680] auto-inserting filter 'auto-inserted scale 0' between the filter 'src' and the filter 'out'
[scale @ 0x101c12aa0] w:600 h:412 fmt:rgb24 sar:0/1 -> w:600 h:412 fmt:yuv444p10le sar:0/1 flags:0x4
Output #0, mov, to '/Users/cbsrobot/Desktop/test.mov':
  Metadata:
    encoder         : Lavf54.2.100
    Stream #0:0: Video: prores (apcn / 0x6E637061), yuv444p10le, 600x412, q=2-31, 200 kb/s, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (png -> prores_kostya)
Press [q] to stop, [?] for help

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x000000010296f3c0
prores_fdct_c (src=0x10296f3c0, linesize=<value temporarily unavailable, due to optimizations>, block=0x10205f890) at libavcodec/proresdsp.c:64
64	            block[y * 8 + x] = tsrc[x];
Error while running hook_stop:
Invalid type combination in ordering comparison.
gdb$ bt
#0  prores_fdct_c (src=0x10296f3c0, linesize=<value temporarily unavailable, due to optimizations>, block=0x10205f890) at libavcodec/proresdsp.c:64
#1  0x000000010048f7a0 in get_slice_data (ctx=0x10205d800, src=0x10296b600, linesize=0x4c0, x=<value temporarily unavailable, due to optimizations>, y=<value temporarily unavailable, due to optimizations>, w=0x258, h=0x19c, blocks=<value temporarily unavailable, due to optimizations>, mbs_per_slice=0x8, blocks_per_mb=0x4, is_chroma=0x1) at libavcodec/proresenc_kostya.c:257
gdb$

comment:4 Changed 7 years ago by cehoyos

  • Component changed from undetermined to avcodec
  • Keywords prores added
  • Reproduced by developer set
  • Status changed from new to open
  • Summary changed from prores_kostya: crash to prores_kostya: invalid read
  • Version changed from unspecified to git-master
$ valgrind ffmpeg_g -i 600.png -vcodec prores_kostya out.mov
ffmpeg version N-38873-gd19d52d Copyright (c) 2000-2012 the FFmpeg developers
  built on Mar 18 2012 21:52:27 with gcc 4.3.2
  configuration: --cc=/usr/local/gcc-4.3.2/bin/gcc --enable-gpl --enable-libspee
x
  libavutil      51. 42.100 / 51. 42.100
  libavcodec     54. 10.100 / 54. 10.100
  libavformat    54.  2.100 / 54.  2.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 65.101 /  2. 65.101
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  7.100 /  0.  7.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, image2, from '600.png':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: png, rgb24, 600x412, 25 tbr, 25 tbn, 25 tbc
Incompatible pixel format 'rgb24' for codec 'prores_kostya', auto-selecting form
at 'yuv444p10le'
[buffer @ 0x452c9c0] w:600 h:412 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 0x451a6c0] auto-inserting filter 'auto-inserted scale 0' between t
he filter 'src' and the filter 'out'
[scale @ 0x451ab60] w:600 h:412 fmt:rgb24 sar:0/1 -> w:600 h:412 fmt:yuv444p10le
 sar:0/1 flags:0x4
Output #0, mov, to 'out.mov':
  Metadata:
    encoder         : Lavf54.2.100
    Stream #0:0: Video: prores (apcn / 0x6E637061), yuv444p10le, 600x412, q=2-31
, 200 kb/s, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (png -> prores_kostya)
Press [q] to stop, [?] for help
==17000== Invalid read of size 2
==17000==    at 0x84D6A5A: prores_fdct_c (proresdsp.c:64)
==17000==  Address 0x488A420 is not stack'd, malloc'd or (recently) free'd
==17000==
==17000== Process terminating with default action of signal 11 (SIGSEGV)
==17000==  Access not within mapped region at address 0x488A420
==17000==    at 0x84D6A5A: prores_fdct_c (proresdsp.c:64)
==17000==
==17000== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 3 from 1)
==17000== malloc/free: in use at exit: 3,060,894 bytes in 115 blocks.
==17000== malloc/free: 401 allocs, 286 frees, 4,374,580 bytes allocated.
==17000== For counts of detected errors, rerun with: -v
==17000== searching for pointers to 115 not-freed blocks.
==17000== checked 8,974,180 bytes.
==17000==
==17000== LEAK SUMMARY:
==17000==    definitely lost: 0 bytes in 0 blocks.
==17000==      possibly lost: 0 bytes in 0 blocks.
==17000==    still reachable: 3,060,894 bytes in 115 blocks.
==17000==         suppressed: 0 bytes in 0 blocks.
==17000== Rerun with --leak-check=full to see details of leaked memory.

comment:5 follow-up: Changed 6 years ago by Opie

Thought I'd give this issue a poke as it still occurring in the current release. The only thing I can add to the debugging reports above is that this only seems to occur when using interlaced video as an input to the encoder. When using 1080i50 the encoder crashes in the prores_fdct_c() function. Using PAL, 720p50 and 1080p25 the encoder seems to work fine.

comment:6 in reply to: ↑ 5 ; follow-up: Changed 6 years ago by cehoyos

Replying to Opie:

Thought I'd give this issue a poke as it still occurring in the current release.

Is the crash also reproducible with the other prores encoder?

comment:7 in reply to: ↑ 6 Changed 6 years ago by Opie

Replying to cehoyos:

Replying to Opie:

Thought I'd give this issue a poke as it still occurring in the current release.

Is the crash also reproducible with the other prores encoder?

No with the other ProRes? encoder recording 1080i50 works OK.

comment:8 Changed 6 years ago by cehoyos

  • Resolution set to fixed
  • Status changed from open to closed

Fixed by Boris Maksalov.

Note: See TracTickets for help on using tickets.