Changes between Initial Version and Version 1 of Create a mosaic out of several input videos


Ignore:
Timestamp:
Jun 9, 2013, 3:45:43 PM (7 years ago)
Author:
burek
Comment:

initial content

Legend:

Unmodified
Added
Removed
Modified
  • Create a mosaic out of several input videos

    v1 v1  
     1[[PageOutline(1-100,Contents)]]
     2
     3= Overview =
     4
     5One of the great features of ffmpeg filtering library is the ability to create overlays, allowing users to put one video over another. We can also use this feature to implement the mosaic video output, usually used in security surveillance systems. An example of what we are going to achieve in this tutorial is displayed in the following screenshot:
     6
     7[[Image(http://s15.postimg.org/jf26vul97/vlcsnap_2013_06_09_13h37m05s228.png)]]
     8
     9We can see one video, displaying 4 different inputs at the same time. This can be done using the following ffmpeg command line (which we'll explain in detail):
     10{{{
     11ffmpeg
     12        -i 1.avi -i 2.avi -i 3.avi -i 4.avi
     13        -filter_complex "
     14                nullsrc=size=640x480 [base];
     15                [0:v] setpts=PTS-STARTPTS, scale=320x240 [upperleft];
     16                [1:v] setpts=PTS-STARTPTS, scale=320x240 [upperright];
     17                [2:v] setpts=PTS-STARTPTS, scale=320x240 [lowerleft];
     18                [3:v] setpts=PTS-STARTPTS, scale=320x240 [lowerright];
     19                [base][upperleft] overlay=shortest=1 [tmp1];
     20                [tmp1][upperright] overlay=shortest=1:x=320 [tmp2];
     21                [tmp2][lowerleft] overlay=shortest=1:y=240 [tmp3];
     22                [tmp3][lowerright] overlay=shortest=1:x=320:y=240
     23        "
     24        -c:v libx264 output.mkv
     25}}}
     26The command line above, if written in the shell directly, should look like this of course:
     27{{{
     28ffmpeg -i 1.avi -i 2.avi -i 3.avi -i 4.avi -filter_complex "nullsrc=size=640x480 [base]; [0:v] setpts=PTS-STARTPTS, scale=320x240 [upperleft]; [1:v] setpts=PTS-STARTPTS, scale=320x240 [upperright]; [2:v] setpts=PTS-STARTPTS, scale=320x240 [lowerleft]; [3:v] setpts=PTS-STARTPTS, scale=320x240 [lowerright]; [base][upperleft] overlay=shortest=1 [tmp1]; [tmp1][upperright] overlay=shortest=1:x=320 [tmp2]; [tmp2][lowerleft] overlay=shortest=1:y=240 [tmp3]; [tmp3][lowerright] overlay=shortest=1:x=320:y=240" -c:v libx264 output.mkv
     29}}}
     30
     31= Detailed explanation =
     32
     33Let's explain how exactly does this work, so you can create your own variants of overlay filter usage. First of all get yourself familiar with [http://www.ffmpeg.org/ffmpeg-all.html#overlay-1 the overlay filter] and all of its options. Also, pay a very close attention to the provided examples section.
     34
     35What we need to do, in this tutorial, is to overlay 4 input videos on top of the blank background video. The end result should look like this:
     36
     37[[Image(http://s16.postimg.org/c034hnwqd/mosaic2.png)]]
     38
     39The filter graph, for this particular case, looks something like this:
     40
     41[[Image(http://s2.postimg.org/f110cjus9/mosaic3.png)]]
     42
     43Now, let's get back to the command line we've used and lets explain it line by line:
     44{{{
     45ffmpeg
     46        -i 1.avi -i 2.avi -i 3.avi -i 4.avi
     47        -filter_complex "
     48                nullsrc=size=640x480 [base];
     49                [0:v] setpts=PTS-STARTPTS, scale=320x240 [upperleft];
     50                [1:v] setpts=PTS-STARTPTS, scale=320x240 [upperright];
     51                [2:v] setpts=PTS-STARTPTS, scale=320x240 [lowerleft];
     52                [3:v] setpts=PTS-STARTPTS, scale=320x240 [lowerright];
     53                [base][upperleft] overlay=shortest=1 [tmp1];
     54                [tmp1][upperright] overlay=shortest=1:x=320 [tmp2];
     55                [tmp2][lowerleft] overlay=shortest=1:y=240 [tmp3];
     56                [tmp3][lowerright] overlay=shortest=1:x=320:y=240
     57        "
     58        -c:v libx264 output.mkv
     59}}}
     60
     61First, we created a background for our output video, using [http://www.ffmpeg.org/ffmpeg-all.html#color_002c-haldclutsrc_002c-nullsrc_002c-rgbtestsrc_002c-smptebars_002c-smptehdbars_002c-testsrc nullsrc filter], which has the dimension of 640x480 pixels (4 videos of 320x240 pixels) and we tagged it with the name "base". Then, we also tagged each ffmpeg input to be able to reference them later in the filter graph. The way we did it was like this:
     62{{{
     63[0:v] setpts=PTS-STARTPTS, scale=320x240 [upperleft]
     64}}}
     65The [http://www.ffmpeg.org/ffmpeg-all.html#Stream-selection stream specifier] "{{{[0:v]}}}" is telling ffmpeg to use the '''video stream''' from the '''first input'''. After we specified the input we want, we also made sure that PTS of that video starts from zero, using [http://www.ffmpeg.org/ffmpeg-all.html#setpts_002c-asetpts setpts filter] with "setpts=PTS-STARTPTS". We also did the same for all the other video inputs to make sure everything will be in sync. After that, we scaled our input video to the appropriate size and finally tagged it with a descriptive name like "upperleft".
     66
     67After tagging, we started to overlay videos, one by one. First, we took the "base" video (our empty background video) and then overlaid the "upperleft" video on top of it:
     68{{{
     69[base][upperleft] overlay=shortest=1 [tmp1];
     70}}}
     71We used "{{{shortest=1}}}" here to specify that we want the output video to stop when the shortest input video stops. Note that we didn't specify any coordinates (x, y) so the upperleft video would be located at (0, 0) i.e. in the upper-left area of the output video. After all that, we tagged that temporary step with the name "tmp1", because we will need to overlay the next input video on that temporary result:
     72{{{
     73[tmp1][upperright] overlay=shortest=1:x=320 [tmp2];
     74}}}
     75That line specifies that the "upperright" video should be overlaid on top of the "tmp1" video, which we produced in the previous step. Here we specified the output x position to 320 pixels, in order to move the video to the right. Also, we did not specify any y position so it defaults to y=0.
     76
     77Same thing was used in the subsequent lines, but there is one important thing to notice for the last line:
     78{{{
     79[tmp3][lowerright] overlay=shortest=1:x=320:y=240
     80}}}
     81Here, we didn't specify the tag name for our result, because we want our result to be actual output of the whole filter graph that we created so far. Also note there is no semi-column at the end of that line.
     82
     83= Final word =
     84
     85That's pretty much it. If it sounds easy, that's because [http://git.videolan.org/?p=ffmpeg.git;a=blob_plain;f=MAINTAINERS;hb=HEAD great people behind FFmpeg project] have worked hard to provide you with such an amazing tool :) One way to say "Thank You" to them is by considering a [http://www.ffmpeg.org/donations.html donation to FFmpeg project] ;)