memory leak when encoding to native mpeg video encoders

[Vitalie Ceban]

Summary of the bug:
when transcode from udp multicast to rtmp, RAM usage increasing even when there is no free RAM, goes to swap, in 12 hours can use approximative 9GB of RAM
How to reproduce:
ffmpeg command with output

root@video3:/home/ffmpeg-2.8# /home/ffmpeg-2.8/ffmpeg -loglevel verbose -re -i "udp://238.0.0.72:32000?timeout=90000000&fifo_size=10000000&buffer_size=128000&reuse=1&overrun_nonfatal=1" -progress "http://192.168.1.50:8024/live/on_tc_progress?live_id=25eafb96-d2bb-4ce6-9bd2-307b28f869a0&app=admin-vip" -flags +global_header -vcodec flv -qscale 1 -acodec aac -strict -2 -f flv "rtmp://server.com/vip/5eced911-c1fa-41ce-b7cd-afbaa60a29e3?token=3eaf27f04fe508253a7cfee71d45f3fbe8e795e1bb3386773f95b5744ce59ea93c658038d95fcb8490f2b7496b40494946229185fe59b3e8ae9c202e9b268b615f3067385aef11f3502f26bf8690aed2355fc8636800b962ff79be735a7d9726d71bcbcad9ee6c92586c9c98a1522b4240de23531d3bba2768a9d8cd56cdb66f8a9d8cb914ddc7f1cfaf9e89b507591524d3933d68ccc60159f93dd13864c575ab40a0d9c3f7fbc30f3d128d2b621e73&title=tests&user_id=4d1c98dd-6e8e-4653-9141-266d1d4f5c8c&channel_id=5ed8db40-df69-4a9b-80e1-94b676c4d6fe"
ffmpeg version 2.8 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.7 (Debian 4.7.2-5)
  configuration: --enable-nonfree --enable-gpl --enable-libmp3lame --enable-libopenjpeg --enable-libx264 --enable-librtmp --disable-yasm
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Routing option strict to both codec and muxer layer
[mpeg2video @ 0x2030be0] Invalid frame dimensions 0x0.
    Last message repeated 7 times
[mpegts @ 0x200c300] max_analyze_duration 5000000 reached at 5016000 microseconds st:1
Input #0, mpegts, from 'udp://238.0.0.72:32000?timeout=90000000&fifo_size=10000000&buffer_size=128000&reuse=1&overrun_nonfatal=1':
  Duration: N/A, start: 36124.738333, bitrate: N/A
  Program 3220 
    Metadata:
      service_name    : Jurnal Tv
      service_provider: J 
                      : Sun Tv 
    Stream #0:0[0xc95]: Video: mpeg2video (Main), 1 reference frame ([2][0][0][0] / 0x0002), yuv420p(tv), 720x576 [SAR 16:15 DAR 4:3], max. 5467 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0xc98]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
Please use -q:a or -q:v, -qscale is ambiguous
Parsing...
Parsed protocol: 0
Parsed host    : server.com
Parsed app     : vip
RTMP_Connect1, ... connected, handshaking
HandShake: Type Answer   : 03
HandShake: Server Uptime : 653115290
HandShake: FMS Version   : 0.0.0.0
HandShake: Handshaking finished....
RTMP_Connect1, handshaked
Invoking connect
HandleServerBW: server BW = 5000000
HandleClientBW: client BW = 5000000 2
HandleChangeChunkSize, received: chunk size change to 4096
RTMP_ClientPacket, received: invoke 190 bytes
(object begin)
Property: <Name:           no-name., STRING:    _result>
Property: <Name:           no-name., NUMBER:    1.00>
Property: <Name:           no-name., OBJECT>
(object begin)
Property: <Name:             fmsVer, STRING:    FMS/3,0,1,123>
Property: <Name:       capabilities, NUMBER:    31.00>
(object end)
Property: <Name:           no-name., OBJECT>
(object begin)
Property: <Name:              level, STRING:    status>
Property: <Name:               code, STRING:    NetConnection.Connect.Success>
Property: <Name:        description, STRING:    Connection succeeded.>
Property: <Name:     objectEncoding, NUMBER:    0.00>
(object end)
(object end)
HandleInvoke, server invoking <_result>
HandleInvoke, received result for method call <connect>
Invoking releaseStream
Invoking FCPublish
Invoking createStream
RTMP_ClientPacket, received: invoke 29 bytes
(object begin)
Property: <Name:           no-name., STRING:    _result>
Property: <Name:           no-name., NUMBER:    4.00>
Property: NULL
Property: <Name:           no-name., NUMBER:    1.00>
(object end)
HandleInvoke, server invoking <_result>
HandleInvoke, received result for method call <createStream>
Invoking publish
RTMP_ClientPacket, received: invoke 105 bytes
(object begin)
Property: <Name:           no-name., STRING:    onStatus>
Property: <Name:           no-name., NUMBER:    0.00>
Property: NULL
Property: <Name:           no-name., OBJECT>
(object begin)
Property: <Name:              level, STRING:    status>
Property: <Name:               code, STRING:    NetStream.Publish.Start>
Property: <Name:        description, STRING:    Start publishing>
(object end)
(object end)
HandleInvoke, server invoking <onStatus>
HandleInvoke, onStatus: NetStream.Publish.Start
[graph 0 input from stream 0:0 @ 0x20022e0] w:720 h:576 pixfmt:yuv420p tb:1/90000 fr:25/1 sar:16/15 sws_param:flags=2
[graph 1 input from stream 0:1 @ 0x2087060] tb:1/48000 samplefmt:s16p samplerate:48000 chlayout:0x3
[audio format for output stream 0:1 @ 0x20872a0] auto-inserting filter 'auto-inserted resampler 0' between the filter 'Parsed_anull_0' and the filter 'audio format for output stream 0:1'
[auto-inserted resampler 0 @ 0x22ed160] ch:2 chl:stereo fmt:s16p r:48000Hz -> ch:2 chl:stereo fmt:fltp r:48000Hz
Output #0, flv, to 'rtmp://server.com/vip/5eced911-c1fa-41ce-b7cd-afbaa60a29e3?token=3eaf27f04fe508253a7cfee71d45f3fbe8e795e1bb3386773f95b5744ce59ea93c658038d95fcb8490f2b7496b40494946229185fe59b3e8ae9c202e9b268b615f3067385aef11f3502f26bf8690aed2355fc8636800b962ff79be735a7d9726d71bcbcad9ee6c92586c9c98a1522b4240de23531d3bba2768a9d8cd56cdb66f8a9d8cb914ddc7f1cfaf9e89b507591524d3933d68ccc60159f93dd13864c575ab40a0d9c3f7fbc30f3d128d2b621e73&title=tests&user_id=4d1c98dd-6e8e-4653-9141-266d1d4f5c8c&channel_id=5ed8db40-df69-4a9b-80e1-94b676c4d6fe':
  Metadata:
    encoder         : Lavf56.40.101
    Stream #0:0: Video: flv1 (flv), 1 reference frame ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 16:15 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.60.100 flv
    Stream #0:1: Audio: aac ([10][0][0][0] / 0x000A), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc56.60.100 aac
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> flv1 (flv))
  Stream #0:1 -> #0:1 (mp2 (native) -> aac (native))
Press [q] to stop, [?] for help
[mpeg2video @ 0x2035360] Warning MVs not available28:20.48 bitrate=3740.3kbits/s    
[mpeg2video @ 0x2035360] concealing 132 DC, 132 AC, 132 MV errors in P frame
[mpeg2video @ 0x2035360] Warning MVs not available29:50.84 bitrate=3718.5kbits/s    
[mpeg2video @ 0x2035360] concealing 87 DC, 87 AC, 87 MV errors in B frame
[mpeg2video @ 0x2035360] ac-tex damaged at 39 210:34:05.52 bitrate=3682.8kbits/s    
[mpeg2video @ 0x2035360] Warning MVs not available
[mpeg2video @ 0x2035360] concealing 180 DC, 180 AC, 180 MV errors in P frame

input ffprobe:

Input #0, mpegts, from 'udp://238.0.0.72:32000':
  Duration: N/A, start: 35794.690333, bitrate: N/A
  Program 3220 
    Metadata:
      service_name    : Jurnal Tv
      service_provider: J 
                      : Sun Tv 
    Stream #0:0[0xc95]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 720x576 [SAR 16:15 DAR 4:3], max. 5467 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0xc98]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s

CPU:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 46
model name      : Intel(R) Xeon(R) CPU           E7520  @ 1.87GHz
stepping        : 6
microcode       : 0x5
cpu MHz         : 1867.000
cache size      : 18432 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 x2apic popcnt lahf_lm dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips        : 3733.33
clflush size    : 64
cache_alignment : 64
address sizes   : 44 bits physical, 48 bits virtual
power management:
 ---------- 16 cores

RAM: 32GB

[Carl Eugen Hoyos]

Is the issue only reproducible with udp input or also with file (or testsrc) input?
Is the issue only reproducible with rtmp output or also with file output?
Please test current FFmpeg git head to make this a valid ticket and please provide valgrind output, this is required for reports concerning memory leaks.

[Vitalie Ceban]

Replying to cehoyos:

Is the issue only reproducible with udp input or also with file (or testsrc) input?
Is the issue only reproducible with rtmp output or also with file output?
Please test current FFmpeg git head to make this a valid ticket and please provide valgrind output, this is required for reports concerning memory leaks.

ok, in the next days I'll provide all information

[Vitalie Ceban]

valgrind and other outputs

[Vitalie Ceban]

Replying to cehoyos:
I attached a file with valgrind output and some ffmpeg tests with different params

[Carl Eugen Hoyos]

Is the issue only reproducible with udp input or also with file (or testsrc) input?
Is the issue only reproducible with rtmp output or also with file output?
Is the issue only reproducible with --enable-librtmp or also with ./configure && make?

Unrelated: You should not use --enable-nonfree --disable-yasm in your configure line.

[Vitalie Ceban]

Replying to cehoyos:

• Is the issue only reproducible with udp input or also with file (or testsrc) input?
• Only with udp input.
• Is the issue only reproducible with rtmp output or also with file output?
• Also with file output.
• Is the issue only reproducible with --enable-librtmp or also with ./configure && make?
• Also with ./configure && make.

[Carl Eugen Hoyos]

Regression since b1fad7ac200d0779a44fc267085d7d46ce2391b9

[Vitalie Ceban]

when applying video filter like scale leak not persist, you can see this in attached file

[Michael Niedermayer]

how can this be reproduced ?

[Carl Eugen Hoyos]

I use ffmpeg -re -f lavfi -i testsrc=r=5 -f mpegts udp://127.0.0.1:12345 and valgrind ./ffmpeg_g -i udp://127.0.0.1:12345 -r 20 -vcodec mpeg1video -qscale 2 -mbd 2 -f null -t 5 -

[Carl Eugen Hoyos]

Also reproducible with the following command line, see ticket #5004:

$ ffmpeg -hide_banner -loglevel quiet -f lavfi -i testsrc -t 2 -f mpeg2video - | valgrind --leak-check=yes ./ffmpeg_g -i - -vcodec mpeg1video -f null -

[Roger Pack]

also seems reproducible with the following, from #5118:

./ffmpeg_g  -f lavfi -i color=c=red:size=10x10 -f mpegts - | ./ffmpeg_g -i - -vcodec mpeg4 -f null out

Also seems to not be tied to mpeg2video input video codec, as the following also repro's it:

./ffmpeg_g  -f lavfi -i color=c=red:size=10x10 -vcodec mpeg4 -f mpegts - | ./ffmpeg_g -i - -vcodec mpeg4 -f null out

(however, if the input is libx264 then it doesn't leak, odd)

The output codec also doesn't seem to matter (seems that any native encoder shows the problem, mpeg2video, mpeg4, jpeg):

 ./ffmpeg_g  -f lavfi -i color=c=red:size=10x10 -f mpegts - | ./ffmpeg_g -i -  -f image2 -y -updatefirst 1  out.jpg

however, if I output to libx264, then the problem also doesn't seem to occur.

[Roger Pack]

Here is a valgrind output (not very helpful I'm afraid [I'm on OS X]): ​https://gist.github.com/rdp/42c8e315cc6438f96249 (the biggest leak is reported at the very bottom).

Also it doesn't matter if input is mpegts or avi, still leaks.
Also note that with testsrc input there's no leak (possibly because its rawvideo).
A possible offending aspect of the commit may be this:

diff --git a/libavcodec/mpegvideo_enc.c b/libavcodec/mpegvideo_enc.c
index 35c7839..84e9dc8 100644
--- a/libavcodec/mpegvideo_enc.c
+++ b/libavcodec/mpegvideo_enc.c
@@ -1475,8 +1475,7 @@ static void frame_end(MpegEncContext *s)
         }
     }
 
-    s->avctx->coded_frame = s->current_picture_ptr->f;
-
+    av_frame_copy_props(s->avctx->coded_frame, s->current_picture.f);
 }

reverting it in my own hackish manner and the leaks at least seem less now, in case useful to anybody:

diff --git a/libavcodec/mpegvideo_enc.c b/libavcodec/mpegvideo_enc.c
index 01604e5..783bcc9 100644
--- a/libavcodec/mpegvideo_enc.c
+++ b/libavcodec/mpegvideo_enc.c
@@ -1673,7 +1673,8 @@ static void frame_end(MpegEncContext *s)
 
 #if FF_API_CODED_FRAME
 FF_DISABLE_DEPRECATION_WARNINGS
-    av_frame_copy_props(s->avctx->coded_frame, s->current_picture.f);
+    // disabled for now #4899 av_frame_copy_props(s->avctx->coded_frame, s->current_picture.f);
+    s->avctx->coded_frame = s->current_picture_ptr->f;
 FF_ENABLE_DEPRECATION_WARNINGS
 #endif
 #if FF_API_ERROR_FRAME
diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index 19f3f0a..20b7db6 100644
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -2549,7 +2549,7 @@ av_cold int avcodec_close(AVCodecContext *avctx)
         av_freep(&avctx->extradata);
 #if FF_API_CODED_FRAME
 FF_DISABLE_DEPRECATION_WARNINGS
-        av_frame_free(&avctx->coded_frame);
+        // disabled for now #4899 av_frame_free(&avctx->coded_frame);
 FF_ENABLE_DEPRECATION_WARNINGS
 #endif
     }

[Michael Niedermayer]

Fixed in 4099e4a77d2d49e2308415d92766ad1511f62f9a