Opened 6 years ago

Closed 7 weeks ago

#4937 closed defect (needs_more_info)

memory leak in muxing.c

Reported by: blinky0815 Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords: leak
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

This is a tiny but very nasty memory leak and is only noticeable when encoding/muxing a large video.
I first noticed it when encoding/muxing a webcam video and sound for more than two hours, hence the 7200 seconds. My program eventually ran out of memory because of this issue.

Since this is an important example other programmers rely on in order to write their own code, it's critical. At least very important.

At first, I assumed a bug in my own code, but the issue can be easily reproduced by using the muxing.c example.

How to reproduce:
1: Edit the source of muxing.c and set the duration to 7200 seconds. Really, any long duration will do. The longer, the better.
2: Set the FPS to 30. The more frames per second, the faster the memory gets eaten away.
3: Optional: comment the code in log_packet or disable logging.
4: compile using make or PKG_CONFIG_PATH=pc-uninstalled make
4: start with ./muxing test.mkv
5: open top and watch the bytes disappear.

My System:
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty

Compiler:
gcc (Ubuntu 4.9.2-0ubuntu1~14.04) 4.9.2

Kernel:
3.13.0-65-generic

I know someone already posted a memory leak issue regarding muxing.c back in 2013, but the information given was brief and a bit inconclusive to say the least.

Attachments (4)

massif.out.23892 (27.6 KB ) - added by blinky0815 6 years ago.
Massif output 7200 seconds
muxing.c (20.6 KB ) - added by blinky0815 6 years ago.
muxing containing changes as described
muxing_memory.csv (263.7 KB ) - added by blinky0815 6 years ago.
out put of top logged into a file. The format is total, used, free, buffers
top_muxing_memory.png (30.8 KB ) - added by blinky0815 6 years ago.
muxing_memory.csv plotted

Download all attachments as: .zip

Change History (23)

comment:1 by Carl Eugen Hoyos, 6 years ago

Keywords: memory in muxing.c removed
Priority: criticalnormal

Please explain how to reproduce the issue and please provide valgrind output for memory leaks.

comment:2 by blinky0815, 6 years ago

Hi,
thanks for looking into the issue so quickly.
As stated in the original message. To quickly reproduce, just follow the steps provided under How to reproduce

Anyway, I am currently running valgrind leak check, and heap and stack usage over time and post the results.

in reply to:  2 comment:3 by Carl Eugen Hoyos, 6 years ago

Replying to blinky0815:

Hi,
thanks for looking into the issue so quickly.
As stated in the original message. To quickly reproduce, just follow the steps provided under How to reproduce

It says Edit the source of muxing.c but no patch is provided.

Anyway, I am currently running valgrind leak check, and heap and stack usage over time and post the results.

Please do, valgrind output is necessary for reports about memory leaks.

comment:4 by blinky0815, 6 years ago

Last edited 6 years ago by blinky0815 (previous) (diff)

comment:5 by blinky0815, 6 years ago

I let it run on a really long video to hopefully get really useful data.
I also set the video size to 640x480.
I am really sorry, I just assumed that in this case editing the source of muxing.c -- just changing the value of some constants -- would be O.K.

Anyway, the moment I have the results, I am gonna post them.

in reply to:  5 comment:6 by Carl Eugen Hoyos, 6 years ago

Replying to blinky0815:

I let it run on a really long video to hopefully get really useful data.

That is generally not necessary (and not useful): Even if valgrind reports a (very) small memleak that you cannot see with other tools, it is still sufficient to qualify as bug, if the the leak is increasing (for example if you encode ten instead of five frames), it is likely an important issue.

comment:7 by blinky0815, 6 years ago

All of my 16 GB of RAM eventually "disappeared" causing the program to crash. So, I assume somewhere memory is allocated and not freed or reused. With 30 FPS, 640 x 480, the leak or whatever it is must be quite small, because it takes hours worth of video to make the 16 GB to "disappear". I am currently running the test on my computer at work and will report when I am back. I might also run further tests on my computer at home.
I now also using valgrind not just because you asked for it, but to really further investigate the issue.

Thanks!

comment:8 by Carl Eugen Hoyos, 6 years ago

Please do not wait until oom (I guess no report will be possible afterwards), just press "q" or start FFmpeg with the "t" parameter and provide the console output.

If FFmpeg really crashes on oom, this may be another (important) bug worth to investigate!

comment:9 by blinky0815, 6 years ago

This is not about the ffmpeg console program but about the muxing.c example.

by blinky0815, 6 years ago

Attachment: massif.out.23892 added

Massif output 7200 seconds

by blinky0815, 6 years ago

Attachment: muxing.c added

muxing containing changes as described

by blinky0815, 6 years ago

Attachment: muxing_memory.csv added

out put of top logged into a file. The format is total, used, free, buffers

by blinky0815, 6 years ago

Attachment: top_muxing_memory.png added

muxing_memory.csv plotted

comment:10 by blinky0815, 6 years ago

So, the good news is that Valgrind and top do not indicate a memory leak. Top shows constant memory usage for the muxing process.
And as promised I ran Massif for the full 7200 seconds, which took almost forever to finish.
The output is attached to this ticket.
The attached muxing.c contains the changes described in the initial post.
However, memory does disappear while running the muxing example.
To further illustrate this, I ran muxing and logged top's output into a file and plotted the result.
Note that this graph really reflects the memory consumption while running muxing. I wasn't even in the office while it was running.

The result so far is really strange.

in reply to:  10 comment:11 by Carl Eugen Hoyos, 6 years ago

Replying to blinky0815:

So, the good news is that Valgrind and top do not indicate a memory leak.

(top can neither indicate memory leaks nor can it indicate that there is no memory leak.)
So there is no issue that can be fixed in FFmpeg?

comment:12 by blinky0815, 6 years ago

Sure, but why does the memory "disappear"? What top can certainly do is showing me the mount of overall available memory. And that goes to zero the longer you run muxing.c, which uses FFMPEG.
Just try running the muxing.c yourself. Maybe even longer than 7200 seconds.

comment:13 by Nicolas George, 6 years ago

What fields in top did you observe and plot? Odds are this is just another instance of not knowing what the page cache is for.

comment:14 by blinky0815, 6 years ago

As I already mentioned, the line showing total, used, free, buffers
and not the line showing
KiB Swap: total, used, free. cached Mem.

comment:15 by Nicolas George, 6 years ago

Seems to confirm what I suspected. I just found this page:
http://www.linuxatemyram.com/
I suggest you read it and close this bug if applicable.

comment:16 by blinky0815, 6 years ago

Fair enough, but then why will muxing.c eventually run out of memory?
Have you ever compiled muxing.c and actually observed what's happening while trying to produce a 2 or more hours long video?

Last edited 6 years ago by blinky0815 (previous) (diff)

comment:17 by Nicolas George, 6 years ago

No idea. I have not found any evidence that it does.

By the way, if you suspect a program leaks memory and want to test it without waiting forever and without trashing your system, you should run it with limited allowed memory. I do this with:

With zsh, that is done with the limit builtin; with other shell it is probably within ulimit. The only limit I have found to work for that is addressspace. For example:

limit addressspace 2048M
ffmpeg -lavfi 'testsrc, split[a][b]; [b]fifo[c]; [a][c] concat' -f null -

(this command will eat memory by queuing frames in the fifo before concat)

You must adjust the number so that the command does not crash immediately. My main build of ffmpeg requires 460M to start running due to the large number of shared libraries.

comment:18 by blinky0815, 6 years ago

Ok, I will do further tests and report the results running muxing.c

comment:19 by Elon Musk, 7 weeks ago

Resolution: needs_more_info
Status: newclosed
Note: See TracTickets for help on using tickets.