Opened 12 months ago

Closed 8 months ago

#8778 closed defect (fixed)

Ogg Vorbis HTTP Stream Excess Memory/CPU Usage

Reported by: Kyra Owned by:
Priority: important Component: avformat
Version: git-master Keywords: ogg regression
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:

HTTP streams featuring Ogg Vorbis content appear to leak memory with the resident size increasing in pace with the input data. Eventually constant mmap and munmap calls in the main ffmpeg thread cause an increase in CPU usage well beyond reason.

This does not occur with ffmpeg 4.2.3, nor does it seem to occur with other codecs (MP3 and AAC at least).

How to reproduce:

ffmpeg -i http://sharprouter.sharp-stream.com:8000/totallyradio.ogg -f null -                                                                          
ffmpeg version 4.3 Copyright (c) 2000-2020 the FFmpeg developers                                                                                                       
  built with gcc 9.3.0 (Gentoo 9.3.0-r1 p3)

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

Change History (14)

comment:1 by Carl Eugen Hoyos, 12 months ago

Resolution: needs_more_info
Status: newclosed

Please test current FFmpeg git head and please provide valgrind output for memory leaks.

comment:2 by Kyra, 12 months ago

Component: undeterminedavformat
Resolution: needs_more_info
Status: closedreopened
Version: unspecifiedgit-master

Latest HEAD tested and a run through Valgrind yielded little benefit.

Bisecting commits points to e983197cbc93420b67aa7e811be47d7278c2c8a2 being the apparent cause.

Last edited 11 months ago by Carl Eugen Hoyos (previous) (diff)

comment:3 by Kyra, 12 months ago

Specifically the call to ffio_ensure_seekback seems to be key.

comment:4 by richardpl, 12 months ago

How much more memory is used?

comment:5 by Kyra, 12 months ago

Progressively more as the stream continues (i.e. as much as the input data), along with CPU usage as this is constantly reallocated.

comment:6 by Kyra, 12 months ago

Summary: Memory Leak With Ogg Vorbis HTTP StreamsOgg Vorbis HTTP Stream Excess Memory/CPU Usage

comment:7 by richardpl, 12 months ago

When usage of memory/cpu becomes noticeable? After how many minutes of input stream?

comment:8 by Kyra, 12 months ago

Increased memory usage should be apparent right away (watch RES in htop or such), increased CPU should be apparent after about an hour. Running strace on the main process should show the reason for the increased CPU.

in reply to:  8 comment:9 by Carl Eugen Hoyos, 12 months ago

Keywords: ogg regression added

Replying to Kyra:

Increased memory usage should be apparent right away (watch RES in htop or such)

Of course.

increased CPU should be apparent after about an hour.

(As also asked on irc)
How can I reproduce this: Which systems shows CPU usage when allocating memory?

comment:10 by Kyra, 11 months ago

It's more the constant reallocation of memory which greatly increases CPU usage. For example on a Rock Pi S (with only 512MiB of RAM) it jumps to 46MiB resident using ~10% of the CPU (RK3308 at 1.3GHz) after five minutes.

getsockopt(3, SOL_SOCKET, SO_RCVBUF, [367360], [4]) = 0
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=0, tv_nsec=100000000}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=99992127})
recvfrom(3, "n\335;\311\345\333I\362\301\200;z\262\333+w\326\24STV\225\7]\220d-8\362\250\374\323"..., 32768, 0, NULL, NULL) = 4194
pselect6(1, [0], NULL, NULL, {tv_sec=0, tv_nsec=0}, NULL) = 0 (Timeout)
mmap(NULL, 7061504, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f80d04000
munmap(0x7f813c0000, 7057408)           = 0
getsockopt(3, SOL_SOCKET, SO_RCVBUF, [367360], [4]) = 0
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=0, tv_nsec=100000000}, NULL, 0) = 0 (Timeout)
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=0, tv_nsec=100000000}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=64956041})
recvfrom(3, " hS\254\351\204\371#\361\33\315\362Q\22\366s\246\216\263\332\320R\234C\352\326\7=\313\375\t\257"..., 32768, 0, NULL, NULL) = 236
pselect6(1, [0], NULL, NULL, {tv_sec=0, tv_nsec=0}, NULL) = 0 (Timeout)
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=0, tv_nsec=100000000}, NULL, 0) = 0 (Timeout)
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=0, tv_nsec=100000000}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=57274531})
recvfrom(3, "OggS\0\0\0\241N\0\0\0\0\0\356\4@Kr\1\0\0L\304\372\200\34\3772\3776\377"..., 32768, 0, NULL, NULL) = 2896
mmap(NULL, 7065600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f81a78000
munmap(0x7f80d04000, 7061504)           = 0
Last edited 11 months ago by Kyra (previous) (diff)

comment:11 by Carl Eugen Hoyos, 11 months ago

Priority: normalimportant
Reproduced by developer: set
Status: reopenedopen

Not sure why but the issue is reproducible.

comment:12 by richardpl, 8 months ago

Resolution: fixed
Status: openclosed

comment:13 by Carl Eugen Hoyos, 8 months ago

Resolution: fixed
Status: closedreopened

comment:14 by richardpl, 8 months ago

Resolution: fixed
Status: reopenedclosed

Fixed my Marton Balint.

Note: See TracTickets for help on using tickets.