Invalid reads when using -flags +qpel+ildct+ilme

[Carl Eugen Hoyos]

​http://thread.gmane.org/gmane.comp.video.ffmpeg.user/54671/focus=54687
A user reported a crash on Windows when encoding interlaced asp. I cannot reproduce a crash on Linux but I see many invalid reads, may not be a regression.

$ valgrind ./ffmpeg_g -cpuflags 0 -threads 1 -i 2014_10_12\ 17_42_02_cut.avi -qscale 4 -flags +qpel+ildct+ilme -threads 1 out.avi
==510== Memcheck, a memory error detector
==510== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==510== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==510== Command: ./ffmpeg_g -cpuflags 0 -threads 1 -i 2014_10_12\ 17_42_02_cut.avi -qscale 4 -flags +qpel+ildct+ilme -threads 1 out.avi
==510==
ffmpeg version N-67837-g0dba982 Copyright (c) 2000-2014 the FFmpeg developers
  built on Nov 20 2014 01:09:25 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      54. 14.100 / 54. 14.100
  libavcodec     56. 12.101 / 56. 12.101
  libavformat    56. 14.100 / 56. 14.100
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  2.103 /  5.  2.103
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, avi, from '2014_10_12 17_42_02_cut.avi':
  Metadata:
    encoder         : Lavf56.14.100
  Duration: 00:00:00.08, start: 0.000000, bitrate: 207821 kb/s
    Stream #0:0: Video: ffvhuff (FFVH / 0x48564646), yuv420p, 1440x1080, SAR 4:3 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, avi, to 'out.avi':
  Metadata:
    ISFT            : Lavf56.14.100
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1440x1080 [SAR 4:3 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.12.101 mpeg4
Stream mapping:
  Stream #0:0 -> #0:0 (ffvhuff (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
==510== Invalid read of size 1
==510==    at 0x9EE538: put_no_rnd_mpeg4_qpel16_h_lowpass (qpeldsp.c:696)
==510==    by 0x9FBD52: put_no_rnd_qpel16_mc21_c (qpeldsp.c:696)
==510==    by 0xD1B020: qpel_motion_search (motion_est.c:196)
==510==    by 0xD2572E: interlaced_search.constprop.7 (motion_est.c:797)
==510==    by 0xD2CF4B: ff_estimate_p_frame_motion (motion_est.c:1023)
==510==    by 0x98E0D4: estimate_motion_thread (mpegvideo_enc.c:2628)
==510==    by 0xABF0A6: avcodec_default_execute (utils.c:1098)
==510==    by 0x99C326: ff_mpv_encode_picture (mpegvideo_enc.c:3545)
==510==    by 0xABFF43: avcodec_encode_video2 (utils.c:2091)
==510==    by 0x4849B7: reap_filters (ffmpeg.c:1093)
==510==    by 0x46E124: main (ffmpeg.c:3705)
==510==  Address 0xc2f5a1f is not stack'd, malloc'd or (recently) free'd
==510==

...

...

==510== Invalid read of size 1
==510==    at 0x9FD4D3: put_no_rnd_qpel16_mc11_c (copy_block.h:83)
==510==    by 0xD1B020: qpel_motion_search (motion_est.c:196)
==510==    by 0xD2572E: interlaced_search.constprop.7 (motion_est.c:797)
==510==    by 0xD2CF4B: ff_estimate_p_frame_motion (motion_est.c:1023)
==510==    by 0x98E0D4: estimate_motion_thread (mpegvideo_enc.c:2628)
==510==    by 0xABF0A6: avcodec_default_execute (utils.c:1098)
==510==    by 0x99C326: ff_mpv_encode_picture (mpegvideo_enc.c:3545)
==510==    by 0xABFF43: avcodec_encode_video2 (utils.c:2091)
==510==    by 0x4849B7: reap_filters (ffmpeg.c:1093)
==510==    by 0x46E124: main (ffmpeg.c:3705)
==510==  Address 0xc2f5ede is not stack'd, malloc'd or (recently) free'd
==510==
frame=    2 fps=0.0 q=4.0 size=     283kB time=00:00:00.08 bitrate=28954.0kbits/s    ^Mframe=    2 fps=0.2 q=4.0 size=     283kB time=00:00:00.08 bitrate=28954.0kbits/s    ^Mframe=    2 fps=0.2 q=4.0 Lsize=     283kB time=00:00:00.08 bitrate=28958.0kbits/s
video:277kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.046354%
==510==
==510== HEAP SUMMARY:
==510==     in use at exit: 128 bytes in 3 blocks
==510==   total heap usage: 1,599 allocs, 1,596 frees, 19,877,473 bytes allocated
==510==
==510== LEAK SUMMARY:
==510==    definitely lost: 0 bytes in 0 blocks
==510==    indirectly lost: 0 bytes in 0 blocks
==510==      possibly lost: 0 bytes in 0 blocks
==510==    still reachable: 128 bytes in 3 blocks
==510==         suppressed: 0 bytes in 0 blocks
==510== Rerun with --leak-check=full to see details of leaked memory
==510==
==510== For counts of detected and suppressed errors, rerun with: -v
==510== ERROR SUMMARY: 13120 errors from 819 contexts (suppressed: 2 from 2)

$ valgrind ./ffmpeg_g -threads 1 -i 2014_10_12\ 17_42_02_cut.avi -qscale 4 -flags +qpel+ildct+ilme -threads 1 out.avi
==580== Memcheck, a memory error detector
==580== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==580== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==580== Command: ./ffmpeg_g -threads 1 -i 2014_10_12\ 17_42_02_cut.avi -qscale 4 -flags +qpel+ildct+ilme -threads 1 out.avi
==580==
ffmpeg version N-67837-g0dba982 Copyright (c) 2000-2014 the FFmpeg developers
  built on Nov 20 2014 01:09:25 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl
  libavutil      54. 14.100 / 54. 14.100
  libavcodec     56. 12.101 / 56. 12.101
  libavformat    56. 14.100 / 56. 14.100
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  2.103 /  5.  2.103
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, avi, from '2014_10_12 17_42_02_cut.avi':
  Metadata:
    encoder         : Lavf56.14.100
  Duration: 00:00:00.08, start: 0.000000, bitrate: 207821 kb/s
    Stream #0:0: Video: ffvhuff (FFVH / 0x48564646), yuv420p, 1440x1080, SAR 4:3 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc
Please use -q:a or -q:v, -qscale is ambiguous
Output #0, avi, to 'out.avi':
  Metadata:
    ISFT            : Lavf56.14.100
    Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1440x1080 [SAR 4:3 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.12.101 mpeg4
Stream mapping:
  Stream #0:0 -> #0:0 (ffvhuff (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
==580== Invalid read of size 8
==580==    at 0xDA18CD: ??? (qpeldsp.asm:301)
==580==    by 0x6300650064006C: ???
==580==    by 0x6500640066006D: ???
==580==    by 0x6300650064006B: ???
==580==    by 0xC31278: put_no_rnd_qpel16_mc21_mmxext (qpeldsp_init.c:505)
==580==    by 0xD1B020: qpel_motion_search (motion_est.c:196)
==580==    by 0xD2572E: interlaced_search.constprop.7 (motion_est.c:797)
==580==    by 0xD2CF4B: ff_estimate_p_frame_motion (motion_est.c:1023)
==580==    by 0x98E0D4: estimate_motion_thread (mpegvideo_enc.c:2628)
==580==    by 0xABF0A6: avcodec_default_execute (utils.c:1098)
==580==    by 0x99C326: ff_mpv_encode_picture (mpegvideo_enc.c:3545)
==580==    by 0xABFF43: avcodec_encode_video2 (utils.c:2091)
==580==  Address 0xc2f5a1f is not stack'd, malloc'd or (recently) free'd

...

...

==580== Invalid read of size 8
==580==    at 0xDA19A0: ??? (qpeldsp.asm:301)
==580==    by 0x6665646463615D58: ???
==580==  Address 0xc2f5f91 is 9 bytes after a block of size 40 alloc'd
==580==    at 0x4C290FE: memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==580==    by 0x4C291A7: posix_memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==580==    by 0xE5BB41: av_mallocz (mem.c:95)
==580==    by 0xE4E350: av_buffer_allocz (buffer.c:34)
==580==    by 0xE4E8FB: av_buffer_pool_get (buffer.c:305)
==580==    by 0xABC778: video_get_buffer (utils.c:667)
==580==    by 0xABEA99: get_buffer_internal (utils.c:1012)
==580==    by 0xABEDB5: ff_get_buffer (utils.c:1025)
==580==    by 0x9E13FD: ff_thread_get_buffer (pthread_frame.c:763)
==580==    by 0x97F330: ff_alloc_picture (mpegvideo.c:496)
==580==    by 0x99D1AF: ff_mpv_encode_picture (mpegvideo_enc.c:1147)
==580==    by 0xABFF43: avcodec_encode_video2 (utils.c:2091)
==580==
frame=    2 fps=1.9 q=4.0 size=     283kB time=00:00:00.08 bitrate=28940.8kbits/s    video:277kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.047666%
==580==
==580== HEAP SUMMARY:
==580==     in use at exit: 128 bytes in 3 blocks
==580==   total heap usage: 1,595 allocs, 1,592 frees, 19,877,178 bytes allocated
==580==
==580== LEAK SUMMARY:
==580==    definitely lost: 0 bytes in 0 blocks
==580==    indirectly lost: 0 bytes in 0 blocks
==580==      possibly lost: 0 bytes in 0 blocks
==580==    still reachable: 128 bytes in 3 blocks
==580==         suppressed: 0 bytes in 0 blocks
==580== Rerun with --leak-check=full to see details of leaked memory
==580==
==580== For counts of detected and suppressed errors, rerun with: -v
==580== ERROR SUMMARY: 1357 errors from 739 contexts (suppressed: 2 from 2)

[Michael Niedermayer]

Fixed in b50e003e1cb6a215df44ffa3354603bf600b4aa3