Opened 8 years ago

Closed 7 years ago

Last modified 7 years ago

#5735 closed defect (fixed)

CPU usage increases over time (possibly caused by loudnorm)

Reported by: Ryan Anstey Owned by:
Priority: normal Component: avfilter
Version: git-master Keywords: loudnorm
Cc: cus@passwd.hu Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

I am taking an RTP stream and adding loudnorm and rebroadcasting it. ffmpeg will use the cpu more and more over time. See this graph: http://puu.sh/qeg9W/8529b8a5b5.png (over a time span of 3 hours).

When I took out the loudnorm filter, it seems to not be increasing slowly over time, or just a lot less. I need more time to verify it's not going up anymore. If it is, it's a lot slower and a lot less dramatic.

My command:

ExecStart=/root/bin/ffmpeg -loglevel panic -i rtp://127.0.0.1:5001 -c:a libmp3lame -af loudnorm=I=-15 -f tee -map 0:a "[f=rtp]rtp://224.3.3.3:1235|[f=mpegts]udp://192.168.0.113:1235/"

More Info:

# ffmpeg -v 9 -loglevel 99 -i rtp://127.0.0.1:5001
ffmpeg version 3.1.1-1~xenial Copyright (c) 2000-2016 the FFmpeg developers

built with gcc 5.3.1 (Ubuntu 5.3.1-14ubuntu2.1) 20160413
configuration: --prefix=/usr --extra-version='1~xenial' --libdir=/usr/lib/ffmpeg --shlibdir=/usr/lib/ffmpeg --disable-static --disable-debug --toolchain=hardened --enable-pthreads --enable-runtime-cpudetect --enable-gpl --enable-shared --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv --enable-libkvazaar --enable-libopenh264 --enable-nonfree --enable-libfdk-aac --enable-libfaac
libavutil 55. 28.100 / 55. 28.100
libavcodec 57. 48.101 / 57. 48.101
libavformat 57. 41.100 / 57. 41.100
libavdevice 57. 0.101 / 57. 0.101
libavfilter 6. 47.100 / 6. 47.100
libavresample 3. 0. 0 / 3. 0. 0
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 1.100 / 2. 1.100
libpostproc 54. 0.100 / 54. 0.100

Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-i' ... matched as input file with argument 'rtp://127.0.0.1:5001'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Successfully parsed a group of options.
Parsing a group of options: input file rtp://127.0.0.1:5001.
Successfully parsed a group of options.
Opening an input file: rtp://127.0.0.1:5001.
Probing rtp score:100 size:0
[rtp @ 0x25c89a0] No default whitelist set
[udp @ 0x25c8e60] No default whitelist set
[udp @ 0x25c8e60] end receive buffer size reported is 131072
[udp @ 0x25d9140] No default whitelist set
[udp @ 0x25d9140] end receive buffer size reported is 131072
[rtp @ 0x25c69e0] SDP:
v=0
c=IN IP4 127.0.0.1
m=application 5001 RTP/AVP 33

[rtp @ 0x25c69e0] sdp: v='0'
[rtp @ 0x25c69e0] sdp: c='IN IP4 127.0.0.1'
[rtp @ 0x25c69e0] sdp: m='application 5001 RTP/AVP 33'
[rtp @ 0x25c69e0] Filter: pid=0x11 type=1
[rtp @ 0x25c69e0] Filter: pid=0x0 type=1
[rtp @ 0x25c89c0] No default whitelist set
[udp @ 0x25c90e0] No default whitelist set
[udp @ 0x25c90e0] end receive buffer size reported is 131072
[udp @ 0x25c8e80] No default whitelist set
[udp @ 0x25c8e80] end receive buffer size reported is 131072
[rtp @ 0x25c69e0] setting jitter buffer size to 500
[rtp @ 0x25c69e0] PAT:
[rtp @ 0x25c69e0] sid=0x1 pid=0x42
[rtp @ 0x25c69e0] new_program: id=0x0001
[rtp @ 0x25c69e0] Filter: pid=0x42 type=1
[rtp @ 0x25c69e0] PMT: len 27
[rtp @ 0x25c69e0] sid=0x1 sec_num=0/0 version=20 tid=2
[rtp @ 0x25c69e0] pcr_pid=0x47
[rtp @ 0x25c69e0] Filter: pid=0x47 type=0
[rtp @ 0x25c69e0] stream=0 stream_type=3 pid=47 prog_reg_desc=
[rtp @ 0x25c69e0] tag: 0x0a len=4
[rtp @ 0x25c69e0] pid=47 pes_code=0x1
(repeats, works fine except for my cpu usage problem)

Change History (13)

comment:1 by Carl Eugen Hoyos, 8 years ago

Is network input required to reproduce the issue or is it also reproducible with file or lavfi input?
Is network output required to reproduce?
In any case please remove the loglevel option and provide the complete, uncut console output to make this a valid ticket.

comment:2 by Ryan Anstey, 8 years ago

I'm in an office situation, so I need some time to test with a non-network stream. I could try non-rtp easily, but that's not what you asked for.

Here is the full command output without the log level option:

# /root/bin/ffmpeg -i rtp://127.0.0.1:5001 -c:a libmp3lame -af loudnorm=I=-15 -f tee -map 0:a "[f=rtp]rtp://224.3.3.3:1235|[f=mpegts]udp://192.168.0.113:1235/"
ffmpeg version N-81026-gb701ec4 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.1) 20160609
  configuration: --prefix=/root/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-nonfree --enable-libebur128
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 50.100 / 57. 50.100
  libavformat    57. 41.100 / 57. 41.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 47.100 /  6. 47.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Input #0, rtp, from 'rtp://127.0.0.1:5001':
  Duration: N/A, start: 86568.446233, bitrate: 192 kb/s
  Program 1
    Stream #0:0: Audio: mp2 ([3][0][0][0] / 0x0003), 44100 Hz, stereo, s16p, 192 kb/s
[tee @ 0x2286cc0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[mpegts @ 0x231f600] frame size not set
Output #0, tee, to '[f=rtp]rtp://224.3.3.3:1235|[f=mpegts]udp://192.168.0.113:1235/':
  Metadata:
    encoder         : Lavf57.41.100
    Stream #0:0: Audio: mp3 (libmp3lame), 48000 Hz, stereo, fltp
    Metadata:
      encoder         : Lavc57.50.100 libmp3lame
Stream mapping:
  Stream #0:0 -> #0:0 (mp2 (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
size=N/A time=00:00:22.89 bitrate=N/A speed=1.09x

comment:3 by Ryan Anstey, 8 years ago

  • Disabling the loudnorm filter fixes the CPU issue, so I believe that is the component at fault.
  • CPU usage seems to raise by 1% every 10 minutes (or maybe a bit less)
  • I have a graph of before/after loudnorm:

http://puu.sh/qfdHr/2009c29e8e.png

  • I will attempt with a non-network input next.

comment:4 by Ryan Anstey, 8 years ago

I'm an ffmpeg noob and have no idea how to stream out a local mp3 file. It plays back at high speed and uses 100% CPU. Can you/someone help me get the command right?

My command is:

# /root/bin/ffmpeg -i /root/music2.mp3 -c:a libmp3lame -af loudnorm=I=-15 -f tee -map 0:a "[f=rtp]rtp://224.3.3.3:1235|[f=mpegts]udp://192.168.0.113:1235/"
ffmpeg version N-81026-gb701ec4 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.1) 20160609
  configuration: --prefix=/root/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-nonfree --enable-libebur128
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 50.100 / 57. 50.100
  libavformat    57. 41.100 / 57. 41.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 47.100 /  6. 47.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Input #0, mp3, from '/root/music2.mp3':
  Metadata:
    title           : Maschine Sounds (Proton Radio)
    artist          : Oliver Lieb
    album           : Maschine Sounds (Proton Radio)
    date            : 2007
    comment         : TALiON
    genre           : Electronic
  Duration: 02:27:13.18, start: 0.025057, bitrate: 191 kb/s
    Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16p, 192 kb/s
    Metadata:
      encoder         : LAME3.97
    Side data:
      replaygain: track gain - -2.500000, track peak - unknown, album gain - unknown, album peak - unknown,
[tee @ 0x23811e0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[mpegts @ 0x2422800] frame size not set
Output #0, tee, to '[f=rtp]rtp://224.3.3.3:1235|[f=mpegts]udp://192.168.0.113:1235/':
  Metadata:
    title           : Maschine Sounds (Proton Radio)
    artist          : Oliver Lieb
    album           : Maschine Sounds (Proton Radio)
    date            : 2007
    comment         : TALiON
    genre           : Electronic
    encoder         : Lavf57.41.100
    Stream #0:0: Audio: mp3 (libmp3lame), 48000 Hz, stereo, fltp
    Metadata:
      encoder         : Lavc57.50.100 libmp3lame
Stream mapping:
  Stream #0:0 -> #0:0 (mp3 (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
size=N/A time=00:07:52.89 bitrate=N/A speed=5.72x

comment:5 by Carl Eugen Hoyos, 8 years ago

Are you searching for the -re option?

comment:6 by Ryan Anstey, 8 years ago

Yes thank you. Testing with that now.

Update on current situation:

  • Running ffmpeg without loudnorm for 1.5 hours: CPU usage started at 16% and finished off at 28%... so it is not loudnorm.
  • Running VLC for the same time period: Started at 3% and ended up at 159%. Wow.

I have no idea why both ffmpeg and vlc would have the same symptoms. They both started rising up at the same time together, which is strange. Something fishy is going on...

http://puu.sh/qfmUR/b276738a25.png

comment:7 by Ryan Anstey, 8 years ago

I'm seeing the gradual increase of CPU usage while streaming a local file too.

My command is

root/bin/ffmpeg -re -i /root/music2.mp3 -c:a libmp3lame -af loudnorm=I=-15 -f tee -map 0:a "[f=rtp]rtp://224.3.3.3:1235|[f=mpegts]udp://192.168.0.113:1235/"

comment:8 by Carl Eugen Hoyos, 8 years ago

Is an mp3 file required as input or is it also reproducible with -f lavfi -i sine?
Is the issue also reproducible with only rtp output? Or only mpegts output? What about mpegts file output (if this is about mpegts)?

comment:9 by LeoN Nortje, 7 years ago

Can confirm that this behaviour is present in ffmpeg 3.1.2 and is definitely related to the loudnorm filter.

CPU escalates by ±5% every hour until 100% CPU is consumed. (On a 1 CPU Digital Ocean droplet)

Reproducing is as as simple as:

ffmpeg -i -ANYLIVESTREAM- -af loudnorm -f null -

Input format does not seem to make a difference, tried reading from a local Icecast AAC stream, a remote MP3 stream and even a local named pipe receiving WAV, all exhibiting the same behaviour.

Nor does output format seem to make a difference. Pushed to AAC, WAV, even null output causes issue to arise.

Specific loudnorm settings does not seem to make a difference, even using the default values causes issue.

Used as -af switch and in a -filter_complex both with same results.

Full output:

/usr/local/bin/ffmpeg -i http://amber.hydrosila.com:8000/record.mp3 -af loudnorm -f null -
ffmpeg version 3.1.2 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  configuration: --pkg-config-flags=--static --extra-cflags=-I/home/master/ffmpeg_build/include --extra-ldflags=-L/home/master/ffmpeg_build/lib --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libebur128 --enable-nonfree
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 48.101 / 57. 48.101
  libavformat    57. 41.100 / 57. 41.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 47.100 /  6. 47.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
[mp3 @ 0x3ac63a0] Skipping 218 bytes of junk at 0.
Input #0, mp3, from 'http://amber.hydrosila.com:8000/record.mp3':
  Metadata:
    icy-br          : 128
    icy-description : Hot Dance Music
    icy-genre       : Misc
    icy-name        : Radio Record
    icy-pub         : 1
    icy-url         : http://www.radiorecord.ru
    StreamTitle     : FEDER/ALEX AIONO - Lordly (Record Mix)
  Duration: N/A, start: 0.000000, bitrate: 128 kb/s
    Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16p, 128 kb/s
[null @ 0x3acdaa0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, null, to 'pipe:':
  Metadata:
    icy-br          : 128
    icy-description : Hot Dance Music
    icy-genre       : Misc
    icy-name        : Radio Record
    icy-pub         : 1
    icy-url         : http://www.radiorecord.ru
    StreamTitle     : FEDER/ALEX AIONO - Lordly (Record Mix)
    encoder         : Lavf57.41.100
    Stream #0:0: Audio: pcm_s16le, 192000 Hz, stereo, s16, 6144 kb/s
    Metadata:
      encoder         : Lavc57.48.101 pcm_s16le
Stream mapping:
  Stream #0:0 -> #0:0 (mp3 (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help

in reply to:  9 comment:10 by Carl Eugen Hoyos, 7 years ago

Replying to leonnortje:

Can confirm that this behaviour is present in ffmpeg 3.1.2

Please test current FFmpeg git head.

Do I understand correctly that the issue is only reproducible with network input?

comment:11 by Marton Balint, 7 years ago

Cc: cus@passwd.hu added

This should be fixed in 005d058f4230f3207ebcf1131df7426d4f57392f, but please check.

comment:12 by Marton Balint, 7 years ago

Resolution: fixed
Status: newclosed

Reopen if this is still an issue.

comment:13 by Carl Eugen Hoyos, 7 years ago

Component: undeterminedavfilter
Reproduced by developer: set
Version: unspecifiedgit-master

I can confirm the issue and the fix for the following command line:

$ ffmpeg -i http://amber.hydrosila.com:8000/record.mp3 -af loudnorm -f null -

The change reduced the (initial) cpu load by 75% here.

Note: See TracTickets for help on using tickets.