heavy memory leak when reading video from a DV/firewire camera (via iec61883)

[bugtrack]

Summary of the bug:
When reading from a DV/firewire camera (via iec61883),
ffmpeg leaks memory equivalent to the total amount of data acquired from the input.
This causes ffmpeg's memory usage to diverge at a rate of approx. 28 Mbit/s.

How to reproduce:

% ffmpeg -y -f iec61883 -i 0 -f null /dev/null

The memory allocated by ffmpeg steadily increases at the rate of the DV data coming in.

For other inputs, like a USB webcam,

% ffmpeg -y -f video4linux2 -i /dev/video0 -f null /dev/null

memory usage does not diverge.

Valgrind shows, that the memory actually leaks:

valgrind --leak-check=full ffmpeg -y -v 9 -loglevel 99 -f iec61883 -i 0 -t 30 -f null /dev/null > verbose_valgrind_output 2>&1

[bugtrack]

Valgrind_leak-check_full_Output

[bugtrack]

Older ffmpeg (tested on debian 8.8 aka jessie) does not have this problem.

ffmpeg version 2.6.9 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.9.2 (Debian 4.9.2-10)
  configuration: --prefix=/usr --extra-cflags='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security ' --extra-ldflags='-Wl,-z,relro' --cc='ccache cc' --enable-shared --enable-libmp3lame --enable-gpl --enable-nonfree --enable-libvorbis --enable-pthreads --enable-libfaac --enable-libxvid --enable-postproc --enable-x11grab --enable-libgsm --enable-libtheora --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libx264 --enable-libspeex --enable-nonfree --disable-stripping --enable-libvpx --enable-libschroedinger --disable-encoder=libschroedinger --enable-version3 --enable-libopenjpeg --enable-librtmp --enable-avfilter --enable-libfreetype --enable-libvo-aacenc --disable-decoder=amrnb --enable-libvo-amrwbenc --enable-libaacplus --libdir=/usr/lib/x86_64-linux-gnu --disable-vda --enable-libbluray --enable-libcdio --enable-gnutls --enable-frei0r --enable-openssl --enable-libass --enable-libopus --enable-fontconfig --enable-libpulse --disable-mips32r2 --disable-mipsdspr1 --disable-mipsdspr2 --enable-libvidstab --enable-libzvbi --enable-avresample --disable-htmlpages --disable-podpages --enable-libutvideo --enable-libfdk-aac --enable-libx265 --enable-libiec61883 --enable-vaapi --enable-libdc1394 --disable-altivec --shlibdir=/usr/lib/x86_64-linux-gnu
  libavutil      54. 20.100 / 54. 20.100
  libavcodec     56. 26.100 / 56. 26.100
  libavformat    56. 25.101 / 56. 25.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 11.102 /  5. 11.102
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100

[bugtrack]

ffmpeg Version 2.9-dev (resp. release 2.8.14) is the last good version,
ffmpeg Version 3.0 leaks memory.

The "culprit" is commit 01bcc2d5c23fa757d163530abb396fd02f1be7c8:

Using git bisect one finds that
commit b27ddffbfb29f456d43c2f155eef5230b940a48b (Merge commit 'dc70c19476e76f1118df73b5d97cc76f0e5f6f6c') is still good
while commit 87c88122703f2befcf96383d05bdf14373c22df9 (Merge commit '01bcc2d5c23fa757d163530abb396fd02f1be7c8') is already bad and leaks memory.

[Carl Eugen Hoyos]

Please confirm that this issue is reproducible with current FFmpeg git head, nothing else is supported here.

[bugtrack]

The bug was reproduced with current git-master as well ( 396c019795677659613900dd04a7708428756906 ).
Version 3.0 was the first release that leaked memory here equivalent to the total processed video data.

[James]

The two Valgrind dumps above are with an unfinished fix of mine applied. Just for the record in case others reading them get confused as to why it's only leaking 33k.

[James]

Patches in the ml.

[bugtrack]

As shown by the valgrind output above, the two patches

[FFmpeg-devel] [PATCH 1/2] avdevice/iec61883: return reference counted packets   James Almer
[FFmpeg-devel] [PATCH 2/2] avdevice/iec61883: free the private context at the end   James Almer 

from jamrial on the ML reduce the memory leaks to a constant value of just 72 bytes (independent of the amount or duration of the processed data).

These remaining 72 bytes were leaked before the regression discussed above, too.
So far, I haven't found a revision of ffmpeg leaking less when reading from firewire
and the remaining leak does not prevent long captures or livestreaming anymore.

Thus I consider this bug resolved. Please close it if anyone agrees with me.

Thank you very much for this racy bug-hunting!

[James]

Fixed in b8629654c6460a28c507f816a977914e3a6f2520 and 5079e96bcc7aaa9cae82a58397ce986e124028e4.