More on Ajax and server push
So far I've looked at two ways of doing server push. The first used a hidden IFRAME which had the SRC attribute set to a persistent connection on the server. The server would push Javascript code along this connection whenever it wanted the client to do something. This had the downside of the browser always displaying a 'loading' indicator as the page never completed loading. From a UI perspective this makes it unusable except as a fallback method of last resort.
The second was outlined in my previous post and used XMLHttpRequest. Like the IFRAME, the request was for a persistent connection. This time the browser wouldn't display a 'loading' indicator and was pretty much exactly what I wanted. I'd prefer it if I didn't have to close the connection after each event to prevent the need to re-establish a connection as this limits scalability a little.
The problem with not closing the connection is you don't know when an event is completely sent back to the client so there is no real guarantee that the client has received the entire event data at one time.
A way of getting around this is to use a mime type of multipart/x-mixed-replace. This allows you to define the 'chunks' in your page. So the browser knows when to process each chunk. Not all browsers support this for XMLHttpRequest initiated transactions though. Mozilla appears to have recently got support for it but IE does not. Alex Russell points to a Microsoft Support article that provides another means to do the same sort of thing but then you have to have two data formats depending on which method you are using.
Even if multipart/x-mixed-replace was the tool of choice there is one further problem with persistent connections. Most (if not all) web browsers only have a limited number of connections it uses for HTTP requests. Internet Explorer only uses two for example. This means if you use one for a persistent connection then all other requests only go through the single connection. If you load another page that uses a persistent connection then you're stuck. No more HTTP requests can be made. This pretty much kills the idea of keeping the connection persistent for general use, and is another reason why my previous post closed the connection between events.
In fact, I've since changed the example (not yet available for download) so that it closes the connection after a set timeout as well, to allow other HTTP requests to be processed. I was having problems with using Backbase whereby some of its server requests were blocking due to the persistent connection being around.
I've done some research over the last couple of days and found a third possible option. Flash allows persistent connections to the server via its XMLSocket class. It is also possible to communicate between the browser and an embedded flash application via Javascript. This would allow writing a small flash application that did nothing but make the connection to the server and then Javascript is used for sending and receiving events across that connection.
This has the advantage of using a different connection pool that the browsers, and you can use any protocol. The disadvantage is that XMLSocket can only connect to ports numbered greater than 1024 and it is not tunneled over HTTP so could be blocked by a firewall. But if the firewall issue is not a problem then it would be possible to write a small Erlang tcp server that communicated across that socket and to the browser.
Backbase actually does something very similar for its charting controls. These controls are embedded flash with the data sent to the control using Javascript from the browser. It does not use a socket connection to the server however.
Another library that does similar is the Canvas object in Nevow. That library embeds a flash object in the page which is used for displaying vector graphics. It makes a connection to the server and allows the server to update the canvas as desired via communication over that socket.
The best general library for doing this sort of thing would seem to be the AFLAX library. This amazing library allows calling any Flash library call from Javascript. Unfortunately it's Flash 8 only and that version of Flash is not yet available for Linux as far as I'm aware.
What I plan to do is write a small flash application to embed to do the server connection and explore using that for the server push events. As a fallback I can have the library use XMLHttpRequest if there is a problem due to firewall restrictions. The final fallback could be an IFRAME if the browser doesn't support XMLHttpRequest.
Categories: erlang, backbase, ajax
The second was outlined in my previous post and used XMLHttpRequest. Like the IFRAME, the request was for a persistent connection. This time the browser wouldn't display a 'loading' indicator and was pretty much exactly what I wanted. I'd prefer it if I didn't have to close the connection after each event to prevent the need to re-establish a connection as this limits scalability a little.
The problem with not closing the connection is you don't know when an event is completely sent back to the client so there is no real guarantee that the client has received the entire event data at one time.
A way of getting around this is to use a mime type of multipart/x-mixed-replace. This allows you to define the 'chunks' in your page. So the browser knows when to process each chunk. Not all browsers support this for XMLHttpRequest initiated transactions though. Mozilla appears to have recently got support for it but IE does not. Alex Russell points to a Microsoft Support article that provides another means to do the same sort of thing but then you have to have two data formats depending on which method you are using.
Even if multipart/x-mixed-replace was the tool of choice there is one further problem with persistent connections. Most (if not all) web browsers only have a limited number of connections it uses for HTTP requests. Internet Explorer only uses two for example. This means if you use one for a persistent connection then all other requests only go through the single connection. If you load another page that uses a persistent connection then you're stuck. No more HTTP requests can be made. This pretty much kills the idea of keeping the connection persistent for general use, and is another reason why my previous post closed the connection between events.
In fact, I've since changed the example (not yet available for download) so that it closes the connection after a set timeout as well, to allow other HTTP requests to be processed. I was having problems with using Backbase whereby some of its server requests were blocking due to the persistent connection being around.
I've done some research over the last couple of days and found a third possible option. Flash allows persistent connections to the server via its XMLSocket class. It is also possible to communicate between the browser and an embedded flash application via Javascript. This would allow writing a small flash application that did nothing but make the connection to the server and then Javascript is used for sending and receiving events across that connection.
This has the advantage of using a different connection pool that the browsers, and you can use any protocol. The disadvantage is that XMLSocket can only connect to ports numbered greater than 1024 and it is not tunneled over HTTP so could be blocked by a firewall. But if the firewall issue is not a problem then it would be possible to write a small Erlang tcp server that communicated across that socket and to the browser.
Backbase actually does something very similar for its charting controls. These controls are embedded flash with the data sent to the control using Javascript from the browser. It does not use a socket connection to the server however.
Another library that does similar is the Canvas object in Nevow. That library embeds a flash object in the page which is used for displaying vector graphics. It makes a connection to the server and allows the server to update the canvas as desired via communication over that socket.
The best general library for doing this sort of thing would seem to be the AFLAX library. This amazing library allows calling any Flash library call from Javascript. Unfortunately it's Flash 8 only and that version of Flash is not yet available for Linux as far as I'm aware.
What I plan to do is write a small flash application to embed to do the server connection and explore using that for the server push events. As a fallback I can have the library use XMLHttpRequest if there is a problem due to firewall restrictions. The final fallback could be an IFRAME if the browser doesn't support XMLHttpRequest.
Categories: erlang, backbase, ajax
posted by Chris Double at 9:33 PM
17 Comments:
I added a socket example to AFLAX to do what you wanted. I will place it online shortly.
That's fantastic Paul, thanks for that. I like the potential of AFLAX a lot and will be using it on my Windows machine. I hope a Linux version of Flash 8 comes out in the near future so I can use it there as well.
Ok, the demo is online. Check it out...
http://www.aflax.org/examples/sockets/sockets.html
Hello Chris...I was reading the slashdot article that you have linked to your site. Are you familiar with the JSR-168 specification (portal/portlet) and if you are, could you give me your opinion on AJAX's role in a portal environment. What I'm thinking/wondering, if this is an updating technology that could work well within a portal paradigm.
(I'm a former programmer/architect that has moved to the CTO ranks and I haven't programmed in ~3+ years...still trying to stay up on latest developments)
I think AJAX would just be another tool used to build portlets. The idea being that an individual portlet can use AJAX to make requests to the server and update parts of the portlet without the entire portlet having to be updated.
I'm not sure how portlet containers behave in the way they update the page though. When a portlet needs to be updated does the whole page update? Or do JSR-168 portals use AJAX behind the scenes to only update the specific portlets? From the looks of the spec it seems that AJAX could be a useful tool for portlets. At the very least, they'd be useful for writing the portal, which aggregates the portlets.
The restriction on the XMLSocket ports was dropped a while ago (since the version 7.0.19 of the Flash Player).
(see http://livedocs.macromedia.com/flash/mx2004/main_7_2/wwhelp/wwhimpl/common/html/wwhelp.htm?context=Flash_MX_2004&file=00001099.html)
You could use the port 443 (assuming you don't need SSL) which is usually open on firewalls.
Thank you, I'm solving almost the same problem now, your post proved very useful :)
Has anyone looked further into the MS PUSH option - using ondatavailable (possibly along with ondatasetchange and ondatasetcomplete)?
The format difference could be easily handled, if the architecture accounted for the difference in transport method used.
What about using a good old Java applet in a hidden frame having a persistent connection to the server and use the netscape.javascript.JSObject to call the javascript part when an event is received on the socket?
Is netscape.javascript.JSObject available on browsers other than Netscape? If so, sounds like a good option.
netscape.javascript.JSObject works fine in Netscape, IE and Firefox anyway, saw some guys reporting problems with Opera, have not tested this myself.
This provides an easy API for using flash's XMLSockets from Javascript. It also works with flash version 6 and also on Linux.
http://www.devpro.it/xmlsocket/
Thanks, that looks great!
seems to me that it kinda defies the point to use Flash for AJAX / javascripty stuff, of course, we could ALWAYS just use flash for ajax stuff
Hi,paul,i have a question,i'm coding a comet program in order to cross the proxy,i want to know,can the request and response message using one connection?
I can only using two connection now,one request to server,and the other long-lived connection waiting the response for the server.
I want to use one long-lived http connection to received chunked data from sever and this use this connection to send request to server too.How can i do this?
Have you looked at the StreamHub comet server? Seems to work in all browsers except Opera. Looks like they are using an iframe. Didn't think that worked in all browsers?
i disagree with these tricks, iframe is messy and any "keeping connection alive" method is expensive in my opinion, also for the server (imagine having few tens of thousands of them), I have simply used PIPE model and what happens server can control the pipe (and retrieval interval) and cumulate all bits of information, then on client side scripts/processes grab from there by ID all necessary content :), also callback pattern is good in this case :-)
Post a Comment
<< Home