Opened 4 years ago

Closed 7 months ago

Last modified 4 months ago

#1475 closed defect (needs_more_info)

Memory leak in ff_read_packet when decoding udp mpegts multicast stream

Reported by: thutschen Owned by:
Priority: normal Component: avformat
Version: unspecified Keywords: udp leak
Cc: cus@passwd.hu, jsantiago@fastmail.us Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

I am receiving 4 udp multicast streams on different ports, which each contain 4 programs.
I transcode one program out of each of those using one ffmpeg instance.
Everything works fine until at some random point in time, the memory usage of ffmpeg starts to grow constantly.

It seams as if at this point in time, the mpegts demuxer reports a bogus new stream and sets its request_probe property to 1.

Now since this stream is bogus there won't be any subsequent packets for this stream.

The problem is, that this packet will be put to the raw_packet_buffer in ff_read_packet.

Now the loop in ff_read_packet has no chance of ever terminating because each new packet will be put on the packet buffer but the first packet will never be popped since probing will never be done because no new packet for the stream to be probed will arrive.

Thus I suggest the following patch

diff --git a/libavformat/utils.c b/libavformat/utils.c
index 284cb9f..f998b0e 100644
--- a/libavformat/utils.c
+++ b/libavformat/utils.c
@@ -720,10 +720,12 @@ int ff_read_packet(AVFormatContext *s, AVPacket *pkt)
         if (pktl) {
             *pkt = pktl->pkt;
             st = s->streams[pkt->stream_index];
-            if(st->request_probe <= 0){
+            if(st->request_probe <= 0 || s->raw_packet_buffer_remaining_size <= 0){
                 s->raw_packet_buffer = pktl->next;
                 s->raw_packet_buffer_remaining_size += pkt->size;
                 av_free(pktl);
+                if(s->raw_packet_buffer_remaining_size <= 0)
+                    av_log(s, AV_LOG_WARNING, "probing stream %d failed", st->index);
                 return 0;
             }
         }

How to reproduce:
The reproduction is a bit tricky since I only encountered this problem with multicast udp mpegts streams

I have attached a complete log file.
The memory growth starts after the entry in line 28650.

When running ffmpeg in gdb and interrupting after the memory growth has started, extremely large negative values for s->raw_buffer_remaining_size can be observed.

Since this is my first bug report here, I want to apologize for any formal mistakes.

Attachments (1)

ffmpeg-20120621-152411.log.tgz (220.8 KB) - added by thutschen 4 years ago.
log-file generated while observing the bug

Download all attachments as: .zip

Change History (11)

Changed 4 years ago by thutschen

log-file generated while observing the bug

comment:1 Changed 4 years ago by cehoyos

  • Keywords udp added

Please post patches to ffmpeg-devel, they receive more attention there.

comment:2 follow-up: Changed 4 years ago by michael

The check for printing the warning is after the raw_packet_buffer_remaining_size is changed

comment:3 in reply to: ↑ 2 Changed 4 years ago by mufin

Replying to michael:

You are right...better check on the other condition (st->request_probe > 0)
But that also leaves some problems open.
Should the request_probe property of the corresponding stream be negated?
Should the stream be closed / is it possible to terminate a stream once opened?
From what I see in the corresponding sources (libavformat/utils.c and libavformat/mpegts.c), there only seems to be functionality to spawn new streams and not to close them.
Maybe one should also free the packet on reporting probing failure.

comment:4 follow-up: Changed 4 years ago by hockey_dave

Any word on this issue? This memory leak is pretty severe and basically makes UDP input useless for long data reception. I would guess it's probably fine for short files where you restart ffmpeg before the memory leak becomes severe enough to take the process down.

Any workarounds? It would be difficult for me to roll back to an earlier ffmpeg version since I use Xuggler and bundled ffmpeg libraries. In my case the memory leak just runs the jvm process up to machine installed memory and then crashes the jvm.

comment:5 in reply to: ↑ 4 Changed 4 years ago by mufin

For me this only occurs in correlation with #1510.
Setting ts->auto_guess to 0 in mpegts.c:1954 (the name of the function is mpegts_read_header) worked in my case and I did not see any bad side effects after several weeks of permanently decoding udp streams I generate from dvb signals with another tool using the libraries compiled this way.

Last edited 4 years ago by mufin (previous) (diff)

comment:6 Changed 4 years ago by cehoyos

  • Keywords leak added

comment:7 Changed 18 months ago by mbat

I've experienced an apparently identical issue with recent ffmpeg builds (Zeranoe, 2015-02-03).
Would be useful to know at least a workaround for the issue, of course a fix would be great.

comment:8 Changed 7 months ago by cus

  • Cc cus@passwd.hu added

Is this still reproducable?

comment:9 Changed 7 months ago by cehoyos

  • Resolution set to needs_more_info
  • Status changed from new to closed

This would need valgrind output to be a valid ticket.

comment:10 Changed 4 months ago by jlsantiago0

  • Cc jsantiago@fastmail.us added
Note: See TracTickets for help on using tickets.