Flash-based Audio and Video in Openfire

As the lead developer of the web-based version of a voice trading system used world-wide by some of the largest financial organizations, I have learned one or two things about web-based VoIP applications. When we designed our trading application, web-based VoIP was not ready for the 99% reliability and quality demanded for financial transactions. We therefore decided to keep the voice on a regular telephone (POTS) and used the web browser to handle the real-time signaling and associated data using a push method called pushlet.

My work on the Red5 plugin for Openfire is a personal attempt to investigate alternative communication technologies and get involved in the open source community. I could not have picked a better place than the Igniterealtime community. It will be interesting to see what happens when Adobe release their own VoIP enabled Flash client, but until then Flash and Red5 in my opinion is still the best way to do web-based VoIP applications.

The telephone landscape is changing. It’s less about exclusive voice communication and more about real-time collaboration where voice is just one of the many communication channels used to do effective collaboration. I believe presence is a very important ingredient that brings intelligence to finding the right channel to communicate with one or many contacts in real time. This is a vision that I share with Jive Software and that why I endorse and work with Openfire on a personal basis. The plugin approach on both server and client is a very smart idea and has enabled me to integrate Flash and Red5 with Spark and Openfire with little or no effort. Implementing HTTP-binding in Openfire was icing on the cake and enabled me to add audio and video to a web-based client like JWChat with relative ease.

This brings me to the purpose of this blog.

At the moment, Spark users get a choice of SIP, Jingle and Red5 for voice. These features enable Spark to make calls to SIP phones, XMPP users and web-based XMPP users. SparkWeb does not do VOIP yet, but that will change when Jingle supports a web-compatible transport.

Web-based XMPP users could not call SIP phones until now.

I have designed a gateway (audio bridge) which converts red5 calls to and from SIP. I have now implemented a version for Windows XP/2k3 using Asteriskwin32. The basic engine is an ActiveX component written in Visual Basic and the source code is open. To solve the problem of transcoding the proprietary Nellymoser codec to SIP, I have used 8 pairs of virtual soundcards provided by a commercial product called VAC (Virtual Audio cable). It is very popular and has been used on other projects to bridge Skype for example.

I now have tight integration between Openfire, Red5 and Asterisk and this has lead to a new version of the Red5 plugin which can control the red5Gateway and implement some interesting ideas. For example, the gateway makes it possible to make every user and group JID become a public SIP address. The gateway will route and convert all SIP calls for user@domain.com to a red5 call. In reverse, if you make a Red5 call to an extension nnnnnn, the gateway will convert it to SIP and pass it on to your configured SIP service provider to route. In the case of group@example, Asterisk is used to hold the caller in a queue while every group member is called until someone answers.

Red5 gateway support is disabled by default on the Red5 plugin, as it is an optional component and is not part of the red5.war file. I am in the process of creating a hosted server to demo it over the next few weeks. If your Openfire runs on Windows and you are interested in trying it before then, send me an email and I will send you the link and documentation as soon as I finish writing it.

In the meantime, enjoy the latest version of the red5 plugin 0.0.7 which is now compatible with Openfire 3.3.0 and Spark 2.5.1. It has the latest Red5 server code: 0.6RC3. I tried to downgrade it to java5, but it was incompatible with Openfire, so I gave up trying :(. I have added support for viewing vcard avatars. I will add the ability to upload images in the next release so you will need Spark to upload the photographs/images for now. As I use jwchat5 myself, I have made the chat UI much like Pandion style, which I like. I only got one bug report from the previous release and have fixed it.

A big thanks to everyone at Igniterealtime, especially the folks at Jive who have added Red5 support duties to their already busy day jobs.

As always, all feedback positive or negative helps to motivate an open-source development and influence product design. The current version 0.0.7 is still far from 1.0.0, so please keep the comments coming.

-dele

The red5gateway demo service is up and running on Openfire at at red5.4ng.net

I have configured two groups

groupa with 9 users test1a, test2a,…test9a

groupb with 9 users test1b, test2b,…test9b

user passwords are the same as the usernames

Outgoing

hi,dele

I had try your App at red5.4ng.net,it’s great.And then I get a VAC with 8 devices.Now there are problems when I try to run your release red5GateWay of version 0.0.7:

1.the plugin can’t run,with a lot of errors in the console

2.where is the activeX “red5Gateway.gateway”? I can’t find it in the plugin

3.how did you get the voice data whith multi-line SIP calls? setting default playback device before a new line created?

Thanks for the feedback 1900.

Red5 gateway support is disabled by default on the Red5 plugin, as it is an optional component and is not part of the red5.war file. Please be patient and wait for me to upload and post the link as soon as I can write up a basic readme file.

Yes and No. I change the default playback/record device when each gateway channel is initialised from the plugin. Each gateway channel has a flash activex object which ties itself to the set default playback/recording VAC.

Thanks for your reply

Keep on working,I’m waitting for your release.

Hi dele,

I have installed the VAC.

How did you acomplish to specify Flash Player sound output to Virtual Cable 1 (or another)?

I can only specify sound output by PC Sound System settings in Control Panel, but this will allow me to use only 1 audio stream.

Model situation:

You have .swf - this .swf is playing incoming NetStream to system default sound device (but I want this default setting change - how?). Then you use VAC and transfer the sound from system default device to Virtual Cable 1 - then I use Skype (or another software) and set Virtual Cable 1 as Sound Input (immitate microphone). Then I start call via skype but sound is comming from Flash Player, which is playing incoming NetStream.

This model situation can work only with one Flash Player, because I can’t setup different output device of Flash Player - how to do this?

Thanks

Tom

Here is a tool that can convert NellyMoser audo encoded FLV files to wav or other format.

http://blog.andrewpaulsimmons.com/2007/05/recording-audio-with-flash-player-to.h tml#links

This works on the server with live streams! The link above includes directions on how to do this.

Hi,

We are developing a community portal for a specific demographic with highly controlled access. Only known people can be part of the messages, viewing photos, music, etc. We wanted to have a voice - intercom and video features as well (on a Windows XP or MAC) machines. Users address book, etc. will all reside in a hosted data content server and very little to no content will be on the PC. Would your Red5 Plugin work or this?

Thanks.

Yogendra Jain

Great Job ! where is the source code ?

Hi Dele, great job !!!

Is possible to know the code of ActiveX component written in Visual Basic to conect to the Asterisk by SIP ??

Thanks a lot.

Good job, but where is the source code of your Visual Basic component?

Dele,

great job, is there a document to detail how to set it up, is it open source?

Thanks

Hi dele, we are building a large-scale web-based Vo-IP service and we could really use your great expertise on a couple of issues we’re facing…Please be kind to contact me when you have some spare time.

This could be very interesting for you.

Thanks, Emil

Hi,

We are doing a project about VoIP and we’re interested in Red5 to SIP convertion (Red5 to Asterisk).

I would like to test your solution as well. Could you please give me some detail how to set it up?

Thank you.

Hello dele,

I currently run red5 under windows enviroment under Tomcat.

I have both rtmp and rtmpt working with Apache 2.2.4 / Tomcat 6.0.13.

I run the latest trunk from red5.

Any chance of your offering for RED5 / Tomcat?

Regards,

Lenny

The functionality of this plugin sounds great, but I am having trouble setting it up. Since I am also using Asterisk (Trixbox) I am very interested in the SIP functionality.

Where can I find the red5 gateway?

So far I can only use the video/audio test page… Is there some documentation available?

Thanks,

Jens

Hi.Where is the link and documentation of red5 plugin? Thanks.

private:

Hi Dele,

I am interested in licensing your technology,

please contact me, regarding this

Ashod