Multiple named pipes don't work

[burek]

Summary of the bug: When 1 ffmpeg is used to produce multiple outputs to named pipes and another ffmpeg is used to read those named pipes as inputs, everything just stucks and doesn't work.

How to reproduce:

• Run 2 shells.

• Create 2 test named pipes (1 for audio and 1 for video):

  mkfifo /tmp/aaa /tmp/vvv
  

• In first shell type the following, which would basically split the input to raw audio and raw video, sending the results to audio/video named pipes:

  ffmpeg -y -i 1.flv -map 0:a -f u16le -acodec pcm_s16le -ac 2 -ar 44100 /tmp/aaa -map 0:v -f yuv4mpegpipe -vcodec rawvideo /tmp/vvv
  

• In second shell type the following, which should join inputs and create the output:

  ffmpeg -y -i /tmp/mcs_aaa -i /tmp/mcs_vvv out.avi
  

The result: It doesn't work.

But, if you use 2 ffmpeg processes in the second shell instead, to separately grab the audio named pipe input and video named pipe input, then everything works as expected:

ffmpeg -y -i /tmp/mcs_aaa bla1.avi &
ffmpeg -y -i /tmp/mcs_vvv bla2.avi

Conclusion: Something is implemented wrong in the ffmpeg's part that reads inputs in such a way that it prevents ffmpeg from reading both named pipes as inputs at the same time (second shell).

[burek]

Btw, by "it doesn't work" I mean everything hangs, just like ffmpeg in second shell is waiting for something, and it doesn't read both inputs in parallel.

The first ffmpeg shows that it read the input properly and is just about to create outputs:

ffmpeg version N-43235-g2db097c Copyright (c) 2000-2012 the FFmpeg developers
  built on Aug  5 2012 11:33:55 with gcc 4.6 (Debian 4.6.3-1)
  configuration: --enable-static --enable-shared --enable-gpl --enable-nonfree --enable-postproc --enable-libx264 --enable-libaacplus
  libavutil      51. 66.101 / 51. 66.101
  libavcodec     54. 49.100 / 54. 49.100
  libavformat    54. 22.100 / 54. 22.100
  libavdevice    54.  2.100 / 54.  2.100
  libavfilter     3.  5.102 /  3.  5.102
  libswscale      2.  1.101 /  2.  1.101
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[h264 @ 0xb3e640] reference picture missing during reorder
    Last message repeated 3 times
[h264 @ 0xb3e640] Missing reference picture
[h264 @ 0xb3e640] decode_slice_header error
[h264 @ 0xb3e640] concealing 920 DC, 920 AC, 920 MV errors in P frame
Input #0, flv, from '1.flv':
  Metadata:
    encoder         : Lavf54.9.100
  Duration: 00:15:02.96, start: 0.000000, bitrate: 88 kb/s
    Stream #0:0: Video: h264 (High), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 16.67 tbr, 1k tbn, 2000k tbc
    Stream #0:1: Audio: aac, 44100 Hz, stereo, s16

The second ffmpeg just hangs after the intro text:

ffmpeg version N-43235-g2db097c Copyright (c) 2000-2012 the FFmpeg developers
  built on Aug  5 2012 11:33:55 with gcc 4.6 (Debian 4.6.3-1)
  configuration: --enable-static --enable-shared --enable-gpl --enable-nonfree --enable-postproc --enable-libx264 --enable-libaacplus
  libavutil      51. 66.101 / 51. 66.101
  libavcodec     54. 49.100 / 54. 49.100
  libavformat    54. 22.100 / 54. 22.100
  libavdevice    54.  2.100 / 54.  2.100
  libavfilter     3.  5.102 /  3.  5.102
  libswscale      2.  1.101 /  2.  1.101
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100

[Cigaes]

Having two pipes between processes without very careful synchronization has never worked, for ffmpeg or anything else.

What is your real-world use for that kind of construct?

[burek]

Well, to be honest, I don't see why should it hang and not work, i.e. I don't see why do those need to be synchronized. The first process outputs 2 different independent outputs, which another process collects and processes. No big deal there.

The real-world example is the splitting of the video and audio to batch script the concatenation of audio/video streams, like described in this faq item: ​http://ffmpeg.org/faq.html#Concatenating-using-raw-audio-and-video

The idea is to split all the audio/video streams, do the cat magic and join it again.

[Cigaes]

Without synchronization, you will quite rapidly get one process trying to write on one pipe while the other is trying to read on the other. Since pipes have a small buffer, it will quite rapidly result in a deadlock.

Since you are considering working with decoded raw video and audio, you should probably use the concat filter instead.

[burek]

Well if I'm wrong (and I've used named pipes a lot in the past and never had such issues like this one mentioned in this ticket) then the faq item might be wrong as well. Since it does exactly what I described here.

Btw, such an example of a dead lock, that you offered, simply shouldn't happen in this scenario, since 2 outputs are completely independent (raw audio vs raw video) and if I'm not wrong, ffmpeg does that in separate threads, so that situation should never happen.

Also, 1 process is using both pipes for write only and 2nd process uses it as read only. So that's also the indicator that such deadlock should never occur in theory, for this scenario.

[reimar]

I don't think FFmpeg really uses to different threads.
But regardless of that, both FFmpeg instances will try to keep A-V sync and as soon as one makes a different decision on whether audio or video must come next to keep sync it will hang.
It can at best work when at least one has A-V sync handling disabled (which is basically what the second approach does).
It would still be very brittle, the only approach I can recommend is to just mux the two streams together if you want to process them together.

[burek]

I'm trying to understand how does deadlock happen in this case, where 1 process is write-only and another is read-only. Where exactly does the deadlock occur?

[Cigaes]

The deadlock happens when the first one is trying to write video while the second one is trying to read audio. (Or the opposite, but audio may fit into a pipe buffer.)

[burek]

I see, if I understand correctly, ffmpeg is doing 2 outputs in 1 thread and not in separate threads? Because of that it stucks when it can't write any of video/audio pipe, because the other process is not reading that one at the moment?

Can this be resolved by making ffmpeg do each output in a separate thread? If it's a command line option, do you happen to know which one is it?

If there is no command line option for that, would it be more natural to implement the creation of multiple outputs in multiple threads? That way, IMHO, it would work more natural than with a single thread. Also, it would solve this deadlock problem too, right?

[Nick]

This still seems to be an issue when using the latest code.

Is piping into ffmpeg from various sources a supported way of using ffmpeg in the general case? With rawvideo input?

If it should work, is there a reasonable path forward for fixing this bug? Is it likely to be a priority?

[teslan]

I think it should work. At least for raw stream inputs. Or is there any better way to convert raw video frames and raw audio from external application?

[Ryan Williams]

burek,

Consider reworking your command to use non-named pipes and introduce the 'pv' command into the pipeline to create a fixed-size buffer. Checkout this article which may be of assistance ​http://unix.stackexchange.com/questions/23488/non-blocking-buffered-named-pipe

Alternatively experiment with the fifo and afifo filters, but in my experience they consume large amounts of memory.