Opened 3 years ago

Last modified 3 years ago

#4937 new defect

memory leak in muxing.c

Reported by: blinky0815 Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords: leak
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

This is a tiny but very nasty memory leak and is only noticeable when encoding/muxing a large video.
I first noticed it when encoding/muxing a webcam video and sound for more than two hours, hence the 7200 seconds. My program eventually ran out of memory because of this issue.

Since this is an important example other programmers rely on in order to write their own code, it's critical. At least very important.

At first, I assumed a bug in my own code, but the issue can be easily reproduced by using the muxing.c example.

How to reproduce:
1: Edit the source of muxing.c and set the duration to 7200 seconds. Really, any long duration will do. The longer, the better.
2: Set the FPS to 30. The more frames per second, the faster the memory gets eaten away.
3: Optional: comment the code in log_packet or disable logging.
4: compile using make or PKG_CONFIG_PATH=pc-uninstalled make
4: start with ./muxing test.mkv
5: open top and watch the bytes disappear.

My System:
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty

Compiler:
gcc (Ubuntu 4.9.2-0ubuntu1~14.04) 4.9.2

Kernel:
3.13.0-65-generic

I know someone already posted a memory leak issue regarding muxing.c back in 2013, but the information given was brief and a bit inconclusive to say the least.

Attachments (4)

massif.out.23892 (27.6 KB) - added by blinky0815 3 years ago.
Massif output 7200 seconds
muxing.c (20.6 KB) - added by blinky0815 3 years ago.
muxing containing changes as described
muxing_memory.csv (263.7 KB) - added by blinky0815 3 years ago.
out put of top logged into a file. The format is total, used, free, buffers
top_muxing_memory.png (30.8 KB) - added by blinky0815 3 years ago.
muxing_memory.csv plotted

Download all attachments as: .zip

Change History (22)

comment:1 Changed 3 years ago by cehoyos

  • Keywords memory in muxing.c removed
  • Priority changed from critical to normal

Please explain how to reproduce the issue and please provide valgrind output for memory leaks.

comment:2 follow-up: Changed 3 years ago by blinky0815

Hi,
thanks for looking into the issue so quickly.
As stated in the original message. To quickly reproduce, just follow the steps provided under How to reproduce

Anyway, I am currently running valgrind leak check, and heap and stack usage over time and post the results.

comment:3 in reply to: ↑ 2 Changed 3 years ago by cehoyos

Replying to blinky0815:

Hi,
thanks for looking into the issue so quickly.
As stated in the original message. To quickly reproduce, just follow the steps provided under How to reproduce

It says Edit the source of muxing.c but no patch is provided.

Anyway, I am currently running valgrind leak check, and heap and stack usage over time and post the results.

Please do, valgrind output is necessary for reports about memory leaks.

comment:4 Changed 3 years ago by blinky0815

Last edited 3 years ago by blinky0815 (previous) (diff)

comment:5 follow-up: Changed 3 years ago by blinky0815

I let it run on a really long video to hopefully get really useful data.
I also set the video size to 640x480.
I am really sorry, I just assumed that in this case editing the source of muxing.c -- just changing the value of some constants -- would be O.K.

Anyway, the moment I have the results, I am gonna post them.

comment:6 in reply to: ↑ 5 Changed 3 years ago by cehoyos

Replying to blinky0815:

I let it run on a really long video to hopefully get really useful data.

That is generally not necessary (and not useful): Even if valgrind reports a (very) small memleak that you cannot see with other tools, it is still sufficient to qualify as bug, if the the leak is increasing (for example if you encode ten instead of five frames), it is likely an important issue.

comment:7 Changed 3 years ago by blinky0815

All of my 16 GB of RAM eventually "disappeared" causing the program to crash. So, I assume somewhere memory is allocated and not freed or reused. With 30 FPS, 640 x 480, the leak or whatever it is must be quite small, because it takes hours worth of video to make the 16 GB to "disappear". I am currently running the test on my computer at work and will report when I am back. I might also run further tests on my computer at home.
I now also using valgrind not just because you asked for it, but to really further investigate the issue.

Thanks!

comment:8 Changed 3 years ago by cehoyos

Please do not wait until oom (I guess no report will be possible afterwards), just press "q" or start FFmpeg with the "t" parameter and provide the console output.

If FFmpeg really crashes on oom, this may be another (important) bug worth to investigate!

comment:9 Changed 3 years ago by blinky0815

This is not about the ffmpeg console program but about the muxing.c example.

Changed 3 years ago by blinky0815

Massif output 7200 seconds

Changed 3 years ago by blinky0815

muxing containing changes as described

Changed 3 years ago by blinky0815

out put of top logged into a file. The format is total, used, free, buffers

Changed 3 years ago by blinky0815

muxing_memory.csv plotted

comment:10 follow-up: Changed 3 years ago by blinky0815

So, the good news is that Valgrind and top do not indicate a memory leak. Top shows constant memory usage for the muxing process.
And as promised I ran Massif for the full 7200 seconds, which took almost forever to finish.
The output is attached to this ticket.
The attached muxing.c contains the changes described in the initial post.
However, memory does disappear while running the muxing example.
To further illustrate this, I ran muxing and logged top's output into a file and plotted the result.
Note that this graph really reflects the memory consumption while running muxing. I wasn't even in the office while it was running.

The result so far is really strange.

comment:11 in reply to: ↑ 10 Changed 3 years ago by cehoyos

Replying to blinky0815:

So, the good news is that Valgrind and top do not indicate a memory leak.

(top can neither indicate memory leaks nor can it indicate that there is no memory leak.)
So there is no issue that can be fixed in FFmpeg?

comment:12 Changed 3 years ago by blinky0815

Sure, but why does the memory "disappear"? What top can certainly do is showing me the mount of overall available memory. And that goes to zero the longer you run muxing.c, which uses FFMPEG.
Just try running the muxing.c yourself. Maybe even longer than 7200 seconds.

comment:13 Changed 3 years ago by Cigaes

What fields in top did you observe and plot? Odds are this is just another instance of not knowing what the page cache is for.

comment:14 Changed 3 years ago by blinky0815

As I already mentioned, the line showing total, used, free, buffers
and not the line showing
KiB Swap: total, used, free. cached Mem.

comment:15 Changed 3 years ago by Cigaes

Seems to confirm what I suspected. I just found this page:
http://www.linuxatemyram.com/
I suggest you read it and close this bug if applicable.

comment:16 Changed 3 years ago by blinky0815

Fair enough, but then why will muxing.c eventually run out of memory?
Have you ever compiled muxing.c and actually observed what's happening while trying to produce a 2 or more hours long video?

Last edited 3 years ago by blinky0815 (previous) (diff)

comment:17 Changed 3 years ago by Cigaes

No idea. I have not found any evidence that it does.

By the way, if you suspect a program leaks memory and want to test it without waiting forever and without trashing your system, you should run it with limited allowed memory. I do this with:

With zsh, that is done with the limit builtin; with other shell it is probably within ulimit. The only limit I have found to work for that is addressspace. For example:

limit addressspace 2048M
ffmpeg -lavfi 'testsrc, split[a][b]; [b]fifo[c]; [a][c] concat' -f null -

(this command will eat memory by queuing frames in the fifo before concat)

You must adjust the number so that the command does not crash immediately. My main build of ffmpeg requires 460M to start running due to the large number of shared libraries.

comment:18 Changed 3 years ago by blinky0815

Ok, I will do further tests and report the results running muxing.c

Note: See TracTickets for help on using tickets.