ffspeak

[burek]

I've been thinking a lot about the ways how to use ff* tools to achieve a simple audio/video conference scenario and I've come to a conclusion that ffmpeg has already got most of the things needed so why not try to implement such a feature which might boost the popularity of (and hopefully donations/contributions to) the ffmpeg project.

Regarding the basic scenario for such a feature, consider the attached diagram. Client side is imho already present in the ff* code base, through ffmpeg (capturing) and ffplay (playback), so the client-side part of this system might simply be just a wrapper/script around these 2 tools, with proper command line parameters or something similar.

The server part could be an ffserver-based thing or even more simplified new tool, which would only use predefined vcodec/acodec (which developers find the most suitable for this scenario), without too much complications. The server's purpose would be to only buffer/rebroadcast the received packets.

It would be (imho) a very popular innovation in ffmpeg family of tools, which would bring a lot of attention to ffmpeg project, since there is no company today, which doesn't use some form of conference communication.

I know this is not a 2-day work, but I think that the cause might be worth the effort.

[Cigaes]

With a server and several clients like that, you have to handle authentication, user interface, incoming messages notifications, possibly chat rooms and private messages, etc. That is way beyond the scope of FFmpeg. Just install a Jabber client with support for video.

What could be in the scope of FFmpeg: a shell script (or simple program) that can establish a low-latency stream between two users using any possible means to establish their rendez-vous. Something like that maybe:

fftalk --call 10.0.0.2:1234

fftalk --accept 10.0.0.3:4567

[burek]

I guess what you suggested might be put into a todo for some future versions, but for a start, it would be enough to enable just audio/video, without auth, chat support, etc. just a basic audio/video conference. I believe the client part is already implemented in ff* libraries/tools, and the server part could just do simple queuing/buffering and rebroadcasting to all but the source of the audio and that's it. As simple as possible and working, to keep it in the spirit of ffmpeg.

Of course, later on there could be added some additional features, like you mentioned, but I think right now, it would probably be the best to just wrap all that is already existing in ffmpeg code base and make use of it in this way, to provide a new feature to the users. After that, we can see if the feature will be as popular as expected and if it is, we can start planning some kind of an api or a todo for future versions of the feature.

Also, I like the name fftalk (it sounds more convenient), but I don't know how to change the topic of the ticket.

[Stefano Sabatini]

Replying to burek:

I guess what you suggested might be put into a todo for some future versions, but for a start, it would be enough to enable just audio/video, without auth, chat support, etc. just a basic audio/video conference. I believe the client part is already implemented in ff* libraries/tools, and the server part could just do simple queuing/buffering and rebroadcasting to all but the source of the audio and that's it. As simple as possible and working, to keep it in the spirit of ffmpeg.

How is this different from what ffserver does (not a rethoric question)?

And I agree, proper conferencing is way beyond the scope of ffmpeg, there already exist standards and dedicated tools for that (SIP to name just one, which is indeed *the* standard in enterprises and in VOIP), *or* should be implemented on top of other libraries/standards/protocols.

Of course, later on there could be added some additional features, like you mentioned, but I think right now, it would probably be the best to just wrap all that is already existing in ffmpeg code base and make use of it in this way, to provide a new feature to the users. After that, we can see if the feature will be as popular as expected and if it is, we can start planning some kind of an api or a todo for future versions of the feature.

Also, I like the name fftalk (it sounds more convenient), but I don't know how to change the topic of the ticket.

I don't like to name things we don't even know what they are (a program, an API, a framework?).

[burek]

Replying to saste:

Replying to burek:

I guess what you suggested might be put into a todo for some future versions...

How is this different from what ffserver does (not a rethoric question)?

If ffserver can already do all this, then great! We could only create some wrapper scripts for users to be able to easily start everything and it's already done. But somehow I doubt that ffserver can combine or switch to different audio inputs, to be able to broadcast it to other listeners... ffserver can easily do one-to-many scenario, but it is not always the case that one speaker is actively talking, which requires that ffserver either combines all audio inputs and broadcasts it or to switch the currently active input (active speaker) for rebroadcast.

Regarding the video, this should be an easier part since the video inputs would all be collected at all time and joined into some kind of a matrix output (tile filter or some other, more suitable), without the need for switching or excluding the sender in the rebroadcast.

Also, I like the name fftalk (it sounds more convenient), but I don't know how to change the topic of the ticket.

I don't like to name things we don't even know what they are (a program, an API, a framework?).

In the best case scenario, if everything is already implemented in the source code parts and we only need to put all the things together, it would be some kind of a wrapper. However, I believe it's the developers who should decide which way to go on this topic, but I just stated that it might be named something like that, because it sounds nice.

[burek]

btw, I just figured, there is no need to use tile filter, instead a multiple smaller playback windows for each participant in the conference might be used.. that way things could be even simpler..?

[burek]

Just to gently bump this question. It would be very popular to provide at least a basic functionality on this topic and, through that, bring FFmpeg to attention of many people, hopefully some of them who are developers, willing to help or maybe even sponsors/donators, which could make things go faster, too.

Videolan guys also did something similar: ​https://wiki.videolan.org/VideoLan_VideoConference