Opened 9 years ago

Closed 6 years ago

Last modified 5 years ago

#1475 closed defect (needs_more_info)

Memory leak in ff_read_packet when decoding udp mpegts multicast stream

Reported by: Thomas Hutschenreuther Owned by:
Priority: normal Component: avformat
Version: unspecified Keywords: udp leak
Cc: cus@passwd.hu, jsantiago@fastmail.us Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

I am receiving 4 udp multicast streams on different ports, which each contain 4 programs.
I transcode one program out of each of those using one ffmpeg instance.
Everything works fine until at some random point in time, the memory usage of ffmpeg starts to grow constantly.

It seams as if at this point in time, the mpegts demuxer reports a bogus new stream and sets its request_probe property to 1.

Now since this stream is bogus there won't be any subsequent packets for this stream.

The problem is, that this packet will be put to the raw_packet_buffer in ff_read_packet.

Now the loop in ff_read_packet has no chance of ever terminating because each new packet will be put on the packet buffer but the first packet will never be popped since probing will never be done because no new packet for the stream to be probed will arrive.

Thus I suggest the following patch

diff --git a/libavformat/utils.c b/libavformat/utils.c
index 284cb9f..f998b0e 100644
--- a/libavformat/utils.c
+++ b/libavformat/utils.c
@@ -720,10 +720,12 @@ int ff_read_packet(AVFormatContext *s, AVPacket *pkt)
         if (pktl) {
             *pkt = pktl->pkt;
             st = s->streams[pkt->stream_index];
-            if(st->request_probe <= 0){
+            if(st->request_probe <= 0 || s->raw_packet_buffer_remaining_size <= 0){
                 s->raw_packet_buffer = pktl->next;
                 s->raw_packet_buffer_remaining_size += pkt->size;
                 av_free(pktl);
+                if(s->raw_packet_buffer_remaining_size <= 0)
+                    av_log(s, AV_LOG_WARNING, "probing stream %d failed", st->index);
                 return 0;
             }
         }

How to reproduce:
The reproduction is a bit tricky since I only encountered this problem with multicast udp mpegts streams

I have attached a complete log file.
The memory growth starts after the entry in line 28650.

When running ffmpeg in gdb and interrupting after the memory growth has started, extremely large negative values for s->raw_buffer_remaining_size can be observed.

Since this is my first bug report here, I want to apologize for any formal mistakes.

Attachments (1)

ffmpeg-20120621-152411.log.tgz (220.8 KB ) - added by Thomas Hutschenreuther 9 years ago.
log-file generated while observing the bug

Download all attachments as: .zip

Change History (11)

by Thomas Hutschenreuther, 9 years ago

log-file generated while observing the bug

comment:1 by Carl Eugen Hoyos, 9 years ago

Keywords: udp added

Please post patches to ffmpeg-devel, they receive more attention there.

comment:2 by Michael Niedermayer, 9 years ago

The check for printing the warning is after the raw_packet_buffer_remaining_size is changed

in reply to:  2 comment:3 by Thomas Hutschenreuther, 9 years ago

Replying to michael:

You are right...better check on the other condition (st->request_probe > 0)
But that also leaves some problems open.
Should the request_probe property of the corresponding stream be negated?
Should the stream be closed / is it possible to terminate a stream once opened?
From what I see in the corresponding sources (libavformat/utils.c and libavformat/mpegts.c), there only seems to be functionality to spawn new streams and not to close them.
Maybe one should also free the packet on reporting probing failure.

comment:4 by Dave, 9 years ago

Any word on this issue? This memory leak is pretty severe and basically makes UDP input useless for long data reception. I would guess it's probably fine for short files where you restart ffmpeg before the memory leak becomes severe enough to take the process down.

Any workarounds? It would be difficult for me to roll back to an earlier ffmpeg version since I use Xuggler and bundled ffmpeg libraries. In my case the memory leak just runs the jvm process up to machine installed memory and then crashes the jvm.

in reply to:  4 comment:5 by Thomas Hutschenreuther, 9 years ago

For me this only occurs in correlation with #1510.
Setting ts->auto_guess to 0 in mpegts.c:1954 (the name of the function is mpegts_read_header) worked in my case and I did not see any bad side effects after several weeks of permanently decoding udp streams I generate from dvb signals with another tool using the libraries compiled this way.

Last edited 9 years ago by Thomas Hutschenreuther (previous) (diff)

comment:6 by Carl Eugen Hoyos, 9 years ago

Keywords: leak added

comment:7 by MB, 7 years ago

I've experienced an apparently identical issue with recent ffmpeg builds (Zeranoe, 2015-02-03).
Would be useful to know at least a workaround for the issue, of course a fix would be great.

comment:8 by Marton Balint, 6 years ago

Cc: cus@passwd.hu added

Is this still reproducable?

comment:9 by Carl Eugen Hoyos, 6 years ago

Resolution: needs_more_info
Status: newclosed

This would need valgrind output to be a valid ticket.

comment:10 by Jose Santiago, 5 years ago

Cc: jsantiago@fastmail.us added
Note: See TracTickets for help on using tickets.