wiki:

StreamingGuide


Version 45 (modified by rogerdpack, 4 years ago) (diff)

--

Streaming

FFmpeg can basically stream through one of two ways: It either streams to a some "other server", which restreams for it, or it can stream via UDP/TCP directly to some destination receiver, or alternatively directly to a multicast destination.

Servers which can receive from FFmpeg (to restream) include ffserver (linux only, though with cygwin it might work), or Wowza Media Server, or Flash Media Server. Even VLC can pick up the stream from ffmpeg, then redistribute it, acting as a server. Since FFmpeg is at times more efficient than VLC at doing the raw encoding, this can be a useful option compared to doing both transcoding and streaming in VLC. Nginx also has an rtmp redistribution plugin, as does apache etc. and there is probably more out there for apache, etc.. You can also live stream to online redistribution servers like own3d.tv or justin.tv (for instance streaming your desktop). Also any rtmp server will most likely work to receive streams from FFmpeg (these typically require you to setup a running instance on a server).

NB that when you are testing your streams, you may want to test them with both VLC and FFplay, as FFplay sometimes introduces its own artifacts when it is scaled (it uses poor quality scaling, which can be inaccurate). Don't use ffplay as your baseline for determining quality.

Also note that encoding it to the x264 "baseline" is basically for older iOS devices or the like, see here. Some people argue that just using mpeg4video codec is better than x264 baseline (where possible) since it is a simpler codec.

The -re flag

The FFmpeg's "-re" flag means to "Read input at native frame rate. Mainly used to simulate a grab device." i.e. if you want to play a video file, but at realtime, then use this. My guess is you typically don't want to use this flag when streaming from a live device.

Also avoid the -sameq flag, which means "same quantizer" not same quality, and probably should be avoided.

Setting

Here's how one guy broadcast a live stream:

$ ffmpeg -y -loglevel warning -f dshow -i video="screen-capture-recorder" -vf crop=690:388:136:0 -r 30 -s 962x388 -threads 2 -vcodec libx264 -vpre baseline -vpre my_ffpreset -f flv rtmp:///live/myStream.sdp

Here is my FFmpeg preset (libx264-my_ffpreset.ffpreset):
coder=1
flags2=+wpred+dct8x8
level=31
maxrate=1200000
bufsize=200000
wpredp=0
g=60
refs=1
subq=3
trellis=0
bf=0
rc_lookahead=0

Here is what another person did:

ffmpeg -f dshow -I video="Virtual-Camera" -vcodec libx264 -tune zerolatency
-b 900k -f mpegts udp://10.1.0.102:1234

And here is what another person did:

ffmpeg -f dshow -i video="screen-capture-recorder":audio="Stereo Mix (IDT High
Definition" -vcodec  libx264 -preset ultrafast -tune zerolatency -r 10
-async 1 -acodec libmp3lame -ab 24k -ar 22050 -bsf:v h264_mp4toannexb
-maxrate 750k -bufsize 3000k   -f mpegts udp://192.168.5.215:48550

NB that they also (for directshow devices) had to adjust the rtbufsize in that example. Also note that newer version of FFmpeg may need a different syntax for specifying preset/tune.

You can see a description of what some of these means, (for example bufsize, bitrate settings) in the [x264EncodingGuide].

Latency

You may be able to decrease latency by specifing that I-frames come "more frequently" (or basically always, in the case of x264's zerolatency setting), though this can increase frame size and decrease quality, see here for some more background. Basically for typical x264 streams, it inserts an I-frame every 250 frames. This means that new clients that connect to the stream may have to wait up to 250 frames before they can start receiving the stream (or start with old data). So increasing I-frame frequency (makes the stream larger, but might decrease latency). For real time captures you can also decrease latency of audio in windows dshow by using the dshow audio_buffer_size setting. You can also decrease latency by tuning any broadcast server you are using to minimize latency, and finally by tuning the client that receives the stream to not "cache" any incoming data, which, if it does, increases latency.

Sometimes audio codecs also introduce some latency of their own. You may be able to get less latency by using speex, for example, or opus, in place of libmp3lame.

You will also want to try and decrease latency at the server side, for instance wowza hints.

Also setting -probesize and -analyzeduration to low values may help your stream start up more quickly (it uses these to scan for "streams" in certain muxers, like ts, where some can appears "later", and also to estimate the duration, which, for live streams, the latter you don't need anyway).

Reducing cacheing at the client side can help, too, for instance mplayer has a "-nocache" option, other players may similarly has some type of pre-playback buffering that is occurring.

Using an encoder that encodes more quickly (or possibly even raw format?) might reduce latency.

You might get less latency by using one of the "point to point" protocols described in this document, at well. You'd lose the benefit of having a server, of course.

Cpu usage/File size

In general, the more cpu you use to compress, the better the output image will be, or the smaller of a file the output will be.

Basically, the easiest way to save cpu is to decrease the input frame rate/size, or decrease the output frame rate/size.

Also you could (if capturing from live source), instruct the live source to feed a "smaller stream" (ex: webcam stream 640x480 instead of 1024x1280), or you could set a lower output "output quality" setting (q level), or specify a lower output desired bitrate (see x264EncodingGuide for a background). Or try a different output codec, or specify new parameters to your codec (for instance, a different profile or preset for libx264). Specifying $ -threads 0 instructs the encoder to use all available cpu cores, which is the default. You could also resize the input first, before transcoding it, so it's not as large. Applying a smoothing filter like hqdn3d before encoding might help it compress better, yielding smaller files.

You can also set a lower output frame rate to of course decrease cpu usage.

If you're able to live capture in a pixel format that matches your output format (ex: yuv420p output from a webcam, instead of mjpeg), that might help with cpu usage, since it avoids an extra conversion. Using 64-bit instead of 32-bit executables (for those that have that choice) can result in a slight speedup. If you're able to use -vcodec copy that, of course, uses the least cpu of all options since it just sends the frames verbatim to the output.

Sometimes you can change the "pixel formats" somehow, like using rgb16 instead of rgb24, to save time/space (or yuv420 instead of yuv444 or the like, since 420 stores less information it may compress better).

Streaming a simple RTP audio stream from FFmpeg

FFmpeg can stream a single stream using the RTP protocol. In order to avoid buffering problems on the other hand, the streaming should be done through the -re option, which means that the stream will be streamed in real-time (i.e. it slows it down to simulate a live streaming source.

For example the following command will generate a signal, and will stream it to the port 1234 on localhost:

ffmpeg -re -f lavfi -i aevalsrc="sin(400*2*PI*t)" -ar 8000 -f mulaw -f rtp rtp://127.0.0.1:1234

To play the stream with ffplay, run the command:

ffplay rtp://127.0.0.1:1234

Note that rtp by default uses UDP, which, for large streams, can cause packet loss. See the "point to point" section in this document for hints if this ever happens to you.

Codecs

The most popular streaming codec is probably libx264, though if you're streaming to a device which requires a "crippled" baseline h264 implementation, some have argued that the mp4 video codec is better. You can also use mpeg2video, or really any other video codec you want, typically, as long as your receiver can decode it, and it suits your needs.

mpeg4 (the video codec) sometimes comes "within a few percentage" of the compression of x264, but using much less cpu to do the encoding.

HTTP Live Streaming and Streaming with multiple bitrates

FFmpeg supports splitting files (using "-f segment" for the output, see segment muxer) into time based chunks, useful for HTTP live streaming style file output.

How to stream with several different simultaneous bitrates is described here.

See also http://sonnati.wordpress.com/2012/07/02/ffmpeg-the-swiss-army-knife-of-internet-streaming-part-v

Saving a file and Streaming at the same time

See Creating multiple outputs. Basically, you may only be able to accept from a webcam or some other source from at most one process, in this case you'll need to "split" your output if you want to save it and stream it simultaneously. Streaming and saving simultaneously (and only encoding once) can also save cpu.

Transcoding/repeating

Ffmpeg can also receive from "a source" (for instance live or udp) and then transcode and re-broadcast the stream.

One mailing list user wrote this, quote:

In my application, I have a server running vlc sucking streams from some cameras, encoding them as MPEG2 streams and sending them to ports 5000 through 5003 on my intermediary server (I'll have to let someone else explain how to set that up, as that was someone else's part of the project). I have another server running Wowza, with an instance named "live". And I've got an intermediate server that sucks in MPEG2 streams coming in on ports 5000 to 5003, transcoding them into mp4 streams with H.264 and AAC codecs, and pushing the transcoded streams on to Wowza.

The command line I use to pull the stream from port 5000, transcode it, and push it is: ffmpeg -i 'udp://localhost:5000?fifo_size=1000000&overrun_nonfatal=1' -crf 30 -preset ultrafast -acodec aac -strict experimental -ar 44100 -ac 2 -b:a 96k -vcodec libx264 -r 25 -b:v 500k -f flv 'rtmp://<wowza server IP>/live/cam0'

-i 'udp://localhost:5000?fifo_size=1000000&overrun_nonfatal=1' tells ffmpeg where to pull the input stream from. The parts after the ? are probably not needed most of the time, but I did need it after all.

-crf 30 sets the Content Rate Factor. That's an x264 argument that tries to keep reasonably consistent video quality, while varying bitrate during more 'complicated' scenes, etc. A value of 30 allows somewhat lower quality and bit rate. See [x264EncodingGuide].

-preset ultrafast as the name implies provides for the fastest possible encoding. If some tradeoff between quality and encode speed, go for the speed. This might be needed if you are going to be transcoding multiple streams on one machine.

-acodec aac sets the audio codec (internal AAC encoder)

-strict experimental allows use of some experimental codecs (the internal AAC encoder is experimental)

-ar 44100 set the audio sample rate

-ac 2 specifies two channels of audio

-b:a 96k sets the audio bit rate

-vcodec libx264 sets the video codec

-r 25 set the frame rate

-b:v 500k set the video bit rate

-f flv says to deliver the output stream in an flv wrapper

'rtmp://<wowza server IP>/live/cam0' is where the transcoded video stream gets pushed to

Adjusting bitrate based on line conditions

FFmpeg doesn't (today) support varying the encoding bitrate based on fluctuating network conditions. It does support outputting in several "different" fixed bitrates, at the same time, however, see "Streaming with multiple bitrates" on this page, which is vaguely related. Also if you are during direct capture from directshow, the input device starts dropping frames when there is congestion, which somewhat simulates a variable outgoing bitrate.

Troubleshooting Streaming

If you get a "black/blank" screen from your server, try sending it yuv422p or yuv420p type input. Some servers get confused if you send them yuv444 input (which is the default for libx264).

Point to point streaming

If you want to stream "from one computer to another", you could start up a server on one, and then stream from FFmpeg to that server, then have the client connect to that server (server could either be on client or server side computers). Or you could do a point to point type stream, like:

ffmpeg -i INPUT -acodec libmp3lame -ar 11025 --f rtp rtp://host:port

where host is the receiving IP. Then receive the stream using VLC or ffmpeg from that port (since rtp uses UDP, the receiver can start up any time).

or

ffmpeg -i INPUT -f mpegts udp://host:port

If you run into packet loss (since UDP is not guaranteed delivery, this might occur) first make sure your FFmpeg is compiled with pthreads support enabled (if it is, then it uses a separate thread to receive from the UDP port, which can cause less packet loss).

Another option is to use some transmission type that uses TCP for your transport. (The rtmp protocol, popular in streaming to servers, uses TCP probably for this reason--you just can't use that for point to point streaming).

One option to use tcp is like this:

ffmpeg -i INPUT -f mpegts tcp://host:port

which I would guess will try and (as a client) establish a connection do that host on that port (assuming it has a server waiting for the incoming connection). You could receive it like this:

ffmpeg -i tcp://local_hostname:port?listen

Another option is to use rtp (which by default uses udp) but by specifying it use tcp:

ffmpeg -i input -f rtsp -rtsp_transport tcp rtsp://localhost:8888/live.sdp lists the option

Then you may receive it like this (ffplay or ffmpeg):

ffplay -rtsp_flags listen rtsp://localhost:8888/live.sdp?tcp # ending ?tcp may not be needed -- you will need to start the server up first, before the sending client

ffmpeg also has a "listen" option for rtmp so it may be able to receive a "straight" rtmp streams from a single client that way.

With tcp based streams you can probably use any formatting/muxer, but with udp you need to be careful and use a muxer that supports 'connecting anytime' like mpegts.

This tutorial.