Memory Leak with named pipe input

[Xenoxis]

Summary of the bug: ffmpeg grows up in memory and in CPU usage without any controls ...

I'm using a script (which call ffmpeg) in combination of icecast2, the script choose musics randomly, it use a first instance of ffmpeg to re-encode them, then it pass it to a second ffmpeg instance by a named pipe, and send the data to icecast2 server

This is the entire script that i use (everything works unless the memory leak which occur, so the stream begin to struggle after 20mn ...)

#!/bin/bash

ReadFilesToServer()
{
	FILES_LIST=()
	for ITEM in `find "$PATH_TO_MUSIC" -name '*.flac' -exec echo "{}|" \; 2> /dev/null`
	do
		FILES_LIST+=(`echo $ITEM | tr -d '\n'`)
	done 
}

PATH_TO_MUSIC='./source'

mkfifo DATA_FILE

{
< DATA_FILE ffmpeg -re -f ogg -i pipe: -c:a copy -metadata:s:a:0 TITLE="WebRadio" -metadata:s:a:0 ARTIST="Mix" -metadata:s:a:0 ALBUM="Mix" -content_type application/ogg -f ogg icecast://user:password@localhost:8000/stream
} &
 
IFS='|'
(
while true
do
	ReadFilesToServer
	
	RANDOM_NBR=$(( $RANDOM % ${#FILES_LIST[*]} ))

	echo "${FILES_LIST[$RANDOM_NBR]}"
	ffmpeg -re -i "${FILES_LIST[$RANDOM_NBR]}" -map 0:a -c:a flac -sample_fmt s16 -ar 44100 -compression_level 12 -f ogg -
done
)> DATA_FILE

The command that make the Memory Leak is

< DATA_FILE ffmpeg -re -f ogg -i pipe: -c:a copy -metadata:s:a:0 TITLE="WebRadio" -metadata:s:a:0 ARTIST="Mix" -metadata:s:a:0 ALBUM="Mix" -content_type application/ogg -f ogg icecast://user:passwordlocalhost:8000/stream

I use -re on this command cause it seems to reduce the CPU usage and delay the time when the stream begin to struggle.

Apparently this is a well know bug caused by avcodec_decode.

Information about my ffmpeg configuration, i've compiled ffmpeg myself and added somes CPU optimizations for my old CPU.

ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-11ubuntu0~18.04.1)
  configuration: --cpu=athlon-xp --prefix=/usr --disable-ffplay --disable-ffprobe --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libopus --enable-libopenjpeg --enable-libmp3lame --enable-libfdk-aac --enable-libass --enable-gpl --enable-version3 --enable-nonfree --extra-cflags='-march=athlon-xp -mtune=athlon-xp -ffast-math -mfpmath=both -O3' --extra-cxxflags='-march=athlon-xp -mtune=athlon-xp -ffast-math -mfpmath=both -O3'
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100

[Xenoxis]

A found a way to avoid this problem : i convert the flac stream to PCM_s16le, and push it into the pipe, and then i re-encode the pcm stream to flac and send it finally to the icecast2 server.

This does not produce the sort of memory leak (i say "sort of" cause i've tried to use valorant on the command that cause problems, but it report no memory leaks after running for 10 minutes)

[Marton Balint]

Please try git master and see if you can reproduce.

[Carl Eugen Hoyos]

And please provide valgrind output whenever reporting a memory leak.

[Xenoxis]

I don't have send the report of valgrind cause it report no memory leaks.

I have downloaded and compiled the version b0a8b40294ea212c1938348ff112ef1b9bf16bb3 (with same parameters as my 4.3.1 version), and it seems to be fixed.

[Carl Eugen Hoyos]

Thank you for testing again and reporting back.