Topic: HTTP Streaming

To react about BigBrownChunx feature proposal (http://community.xajaxproject.org/viewtopic.php?id=459) , I open a topic here about HTTP streaming.

This feature is really interesting for applis which uses polling requests.
With a such feature it would be possible to handle several request during a single HTTP request!
This is very interesting in order to reduce server overload (due to repeating http request) and improve the server to client communication reactivity.

A more detailed description can be found here :
http://ajaxpatterns.org/HTTP_Streaming

What do you think about it ? Is it workable in xajax ?

Re: HTTP Streaming

It's very workable: I'm working on it.

Not sure about the point you mentioned above: reducing server overload.
Since each connection takes up a seperate thread on the server, and some servers limit the number of threads that can be open at one time.

But I'll keep plodding away and working on it smile  xajax streaming is a much better name than anything i could come up with smile

Re: HTTP Streaming

Thank you! I will be a beta tester if you need it smile

Re: HTTP Streaming

Cheers for this feature!

Re: HTTP Streaming

Almost there... I'm just debugging large streams from the server  and disconnects from timeouts.

I'll put up a demo page too. smile

Re: HTTP Streaming

It'll be a big step forwards for xajax.

Waiting for beta testing.

Re: HTTP Streaming

So I've been working on it, and got it going perfectly in Opera.

Then I fire up Firefox and it complains about multiple xml responses from the server, and IE doesn't even work at all.

Might need a rethink.  I'm still working on it though.  Silly browsers eh?

8 (edited by freejack 2006-03-15 11:45:55 PM)

Re: HTTP Streaming

A little fyi for those who will consider using http streaming in the future.  We have an APR module that streams exchange data to the browser.  From our observations, we've noticed that apache will continue to hold onto the memory used by the data sent out until the process ends.  If you have several processes streaming constantly all day, your server will run out of memory very quickly.  The workaround is to have your app (in this case your xajax server) reconnect to the server after a set number of messages or after a certain number of bytes has been transmitted so the apache child process can die and free up more ram.

Re: HTTP Streaming

yep, working on that problem too smile

Doing it automatically so that there's no problem of memory hogs on the server end with a predefined, adjustable timeout.
i'm trying to make it so that the server drops the connection when the memory usage gets to be too much and then the client reconnects.

thanks for the tip though

Re: HTTP Streaming

Hey BigBrownChunx  i was very excited to hear you working on HTTP streaming through xajax. I was jsut wondering the status of your endeavour, and how your making out...   

This feature is probally the holy grail of AJAX integration. Once this is completed, it will open a whole new world of Application Web Programming.

Simply Amazing cool

Re: HTTP Streaming

This is my first post to this forum and I'd like to say thanks to the devs for a great utility.  I've found it immensely useful in my project and really appreciate your efforts.

I'd like to add my voice to those who are already requesting the "streaming" feature in xajax. I'm using xajax in a chat type application and everything works very smoothly except that I have to poll the server for updates. I anticipate that eventually 50 to 100 people (possibly more) will be connected to the server at once and accessing 30 to 50 seperate databases. 

I'm concerned about the load this is going to create on the server.  Some type of streaming or server push technology would be immensely useful to me.

Are there any recent developments on this front?  Anything that's useful yet?

ayla

Re: HTTP Streaming

I kinda stopped working on this for a while to focus on File Uploads.  This was project number two for me.

I've been playing around with code occasionally (nothing that works properly, of course) to try out some ideas particularly in the area of memory consumption and load in client and server (we can't have a really useful tool thats too bloated to use)

ayla wrote:

I'm concerned about the load this is going to create on the server.

me too!  For a start, server pushing/streaming puts far less load on the server than server polling: much less HTTP overhead and much less connections since updates are only made when there's new data.
Secondly, I've tried timeouts on both client and server with great success, which will probably just end up being only on the server to simplify things
Third, i'm looking at checking memory usage, so that if it gets too much it'll tell the client to reconnect.
Fourth, i'm trying out some cyclic connection dropping, so that everyone isn't dumped at the same time, but one by one (but this requires some fun, cross-process stuff)

So the idea is still alive, but it's on hold til I play around with file uploads and plugins and such.  but i've been on holiday too, so another hold up there.

very good to see interest in this smile please keep comments and questions coming big_smile

Re: HTTP Streaming

I am still very excited about this feature.  Im especially interested to see it in action along side the new features in the 0.2.4 release.  Being able to stream arrays of PHP database information back through to javascript is a dream come true for a lot of programmers out there.

Keep up the good work BigBrownChunx!

Re: HTTP Streaming

Yes thats an dream coming true.

I'm very interested in the results. For both file transfer and http streaming. You are doing an great job. I will search for an old script library. I still have. Maybe its an bit usefull for you guys.

Re: HTTP Streaming

Couple of good links in this area. This technology is being termed Comet.:

http://alex.dojotoolkit.org/wp-content/ … cyData.pdf

http://alex.dojotoolkit.org/?p=545

BBC any progress on this??

-V

Re: HTTP Streaming

Yep.  A Comet/HTTP Streaming plugin will be featuring with the new xajax 0.5 smile
My focus at the moment is on File Uploads, but this may, optionally, use Comet/HTTP Streaming to send back progress on the upload (when I get that sorted out)

Re: HTTP Streaming

Is xajax 0.5 in the SVN? What's the planned release date for 0.5?

-V

Re: HTTP Streaming

No planned release date yet, but we're probably looking at a 2-3 month timeframe until final release -- however, due to the nature of the upgrade, it's very possible we'll have a beta test version out sooner than that.

Project manager for xajax

Re: HTTP Streaming

Well I took the day off work today so that I could work on xajax Comet/HTTP Streaming. smile
Have a peek at http://www.xajaxproject.org/BigBrownChunx/comet.php and tell me what you think.

Re: HTTP Streaming

BigBrownChunx wrote:

Well I took the day off work today so that I could work on xajax Comet/HTTP Streaming. smile
Have a peek at http://www.xajaxproject.org/BigBrownChunx/comet.php and tell me what you think.

Works great. Is this for v 0.5? Will need more docs to build on it

-V

Re: HTTP Streaming

Yeah.... this is using v0.5... but there's no reason it can't be ported to 0.2.4 smile

Documentation is in the works.  Right now I just wanted to put it out there and see what people thought.

Re: HTTP Streaming

Very nice.

Any chance of a alpha release of 0.5? Wanted to take a peek at SVN a few times, but the daim webinterface thing kept giving me errors.

Re: HTTP Streaming

Ported to 0.2.4?? Maybe an alpha release of 0.5?

And is this really streaming or just an loop, cause I couldnt figure that out from the source which is in the document. Since it just looks like an loop only its missing the delay part in the function.

Re: HTTP Streaming

I can't wait for HTTP Streaming wink Hope 0.5 is coming very very soon wink

Re: HTTP Streaming

Maybe a v0.5alpha isn't such a bad idea.  We've just got to fix up the code a little so that it works again, but I wonder what the others think of that.  Not up to me to make those decisions.

statiic wrote:

is this really streaming or just an loop

Its both.  If your client browser is Firefox or Opera then it uses streaming, if your client browser is Safari/Konqueror or Internet Explorer then it degrades back to a continual loop.  There is no browser detection however, so if IE or Safari get upgraded in the future, you dont have to change the code

The only think you as a developer has to worry about is the new function addCometFunction, and the xajaxComet php and javascript does the rest.

statiic wrote:

it just looks like an loop only its missing the delay part in the function.

The delay is put in on the server side so as to not flood the server nor client with requests.

Argentus wrote:

Very nice.

Thanks big_smile