Web Applications Blog

Event Streaming to Web Browsers

, , , , , , , , , , , ,

One cool feature we added to Opera 9 is Server-Sent Events from the WHATWG Web Applications 1.0 specification. Using SSE you can push DOM events continously from your web server to the visitor's browser. This creates a lot of exciting opportunities for web application authors.

Traditionally, when building an Ajax application, the browser continually polls the server, sending requests to the server, asking to get data back, making new HTTP requests for every single poll, putting more strain on the server than needed.

The event streaming approach instead opens a persistent connection to the server, sending data to the client when new information is available, eliminating the need for continuous polling. This method for doing remoting offers a tremendous advantage, since the server no longer has to handle the overhead associated with clients asking for new data. Instead, the server simply sends back data every connected client when appropriate, thus reducing the load on the server, with the added advantage of offering instant feedback to the user.

Opera Web Chat

Opera Web Chat Screenshot To provide you with a starting point on how to build your own event streaming application, we have built Opera Web Chat. This is a web based chatroom offering some of the features from the built in to the Opera IRC client. Currently the chat only offers one single chatroom. (A screenshot is available here)

Keep in mind that it is an experimental service, which means it may not always be available for use.

How to use Server-Sent Events

To use Server-Sent Events in a web application, add an <event-source> element to the document, with a src attribute pointing to a event source URL. This URL should provide a persistent HTTP connection that sends a data stream containing the events. The connection must use the content type application/x-dom-event-stream.

It is possible to send events with any name, and specify the properties of the event object. Opera 9.01 only supports the data property of the event, so this is what we are going to use in these examples.

The server side event source writes the events whenever they occur, and sends them over HTTP to the client. This is a basic example of event data. This is more thoroughly explained in the specification.

Event: server-time
data: [time on the server]

Event: the-answer
data: 42

This will send two events to the browser, and it's possible to catch them as DOM events. The following JavaScript example listens for the "server-time" event, and alerts the content.

document.getElementsByTagName("event-source")[0]
        .addEventListener("server-time", eventHandler, false);

function eventHandler(event)
{
    // Alert time sent by the server
    alert(event.data); 
}

This is a very simple Python CGI example which sends a new event every 3 seconds. Every event is named "server-time", and sends an event with the data property set to the current time of the server in seconds.

Keep in mind that when a CGI script outputs data, there is no guarantee that it is sent immediately. There are often caching mechanisms and so on in place. For this reason it may be necessary to explicitly flush the output.

Here is the example code written in python.

#!/usr/bin/python
import sys
import time
print "Content-Type: application/x-dom-event-stream\n\n"

while True:
    print "Event: server-time"
    print "data: %f\n" % (time.time(),)
    sys.stdout.flush()
    time.sleep(3)

The same example written in PHP:

<?php
header("Content-Type: application/x-dom-event-stream");
while(true) {
    echo "Event: server-time\n";
    $time = time();
    echo "data: $time\n";
    echo "\n";
    flush();
    sleep(3);
}
?>

Opportunities

In addition to the chat application we made, there are lots of different applications that can be made with Server-Sent Events. For instance games or instant messaging clients, such as MSN Messenger, Jabber or AIM. You could also build stock and news tickers, status and log file monitors, or anyhing you can come up with.

What will you build?

Moving and resizing widgetsWidget Web Chats

Comments

GuillermoGuille Friday, September 1, 2006 4:12:51 PM

This is so great, I was looking for this for a widget that I'm working on.

I still can't make the chat sample to work.

Anonymous Friday, September 1, 2006 4:23:25 PM

graste writes: Awesome. How about an IMAP widget? Perhaps I should write to my email hosters, that they should think about that for the future. ;)

Brian HuismanGreyWyvern Friday, September 1, 2006 4:26:29 PM

This opens up a whole new level of web interaction! But since it's Opera only, I guess it is mostly confined to widgets ATM smile

Let's see when the first two player action widget debuts!

Anonymous Friday, September 1, 2006 5:19:00 PM

Milo writes: How does this reduce server load? Now the server has to manage many more open connections at once. I think any server that uses this will have a much harder time scaling once demand increases.

porneL Friday, September 1, 2006 6:28:10 PM

@Milo: Ofcourse it's crazy to have thousands of PHP processes on Apache server, but you can handle large number of connections with a dedicated server and save a lot of bandwidth (TCP/IP alone doesn't generate any traffic when there aren't any changes, unlike regular AJAX requests which pump packets all the time).

Anonymous Friday, September 1, 2006 7:40:57 PM

Jay writes: Is there support for the x-dom-event-stream type in Opera Mobile or Mini?

velmu Friday, September 1, 2006 8:42:02 PM

WebApplications speaks seldom, but to the point smile Tried the demo, quite impressive

Remco Lantingremcolanting Friday, September 1, 2006 10:32:30 PM

The OperaWatch article was also on the frontpage of digg.com. I've heard a lot of positive things about this. But I also have to say I've never seen so much crashes of Opera on one day.

This feature is just new to the general public, so there could still be bugs, but other than that, it's just a great new feature bigsmile

grafio Friday, September 1, 2006 10:36:10 PM

It's cool, but the chat demo crashes my Opera (9.01) every time I write something.

piper-noiter Friday, September 1, 2006 10:50:23 PM

I had no crashes or problems... I'm still using 8515 though.

Anonymous Saturday, September 2, 2006 3:52:49 AM

Eric Blade writes: I would guess that this would be close to a final step of eliminating the operating system as a platform for many types of applications, and making them web-based. Might even be cooler than Java should have been. I could really use some more specific documentation and examples, though. How about source for the chat program?

Anonymous Saturday, September 2, 2006 5:59:10 AM

Anonymous writes: What's new? If you keep the connection open, you can do all that with plain javascript.

Evgeny Stepanischevbolk Saturday, September 2, 2006 7:29:26 AM

Your PHP code is very ugly.

<?php
header("Content-Type: application/x-dom-event-stream");

for (;wink
{
echo "Event: server-time\n",
'data: ', time(), "\n\n";

flush();
sleep(3);
}
?>

"ob_flush" should be use only if "ob_start" was used.

JasmoJazmo Saturday, September 2, 2006 8:15:19 AM

What about Firefox, does it come to support this? How about IE7?

chesss Saturday, September 2, 2006 8:20:46 AM

THis is awesome, even on My PIII , 64kbs connection the chat app opened in seconds , and is running fast and smooth!!
very very cool!!

Anonymous Saturday, September 2, 2006 9:21:10 AM

Anonymous writes: For support of WHATWG in Firefox, check http://wiki.mozilla.org/XUL:Priority_List#Priority_6

Anonymous Saturday, September 2, 2006 9:23:11 AM

fforw writes: @jazmo: Firefox has been supporting something similar using XMLHttpRequest and multipart/x-mixed-replace, but that is far from being as clean as this. (see e.g. http://www.subbu.org/weblogs/main/2006/04/dissecting_ajax_1.html#pushDemo )

Anonymous Saturday, September 2, 2006 11:04:57 AM

Anonymous writes: HTML client example please

Haavardhaavard Saturday, September 2, 2006 12:35:08 PM

If anyone is getting crashes when trying the chat demo, please report them:

http://www.opera.com/support/search/supsearch.dml?index=790

Anonymous Saturday, September 2, 2006 2:54:10 PM

Ned Baldessin writes: So I gather this is a proper standardized implementation of what is currently being called "commet", am I correct? http://en.wikipedia.org/wiki/Comet_%28programming%29

Anonymous Sunday, September 3, 2006 2:55:30 AM

Kimberley Burchett writes: Why does Opera only support the "data" property of events? I work on a product designed to work with generic event streams, and this limitation seems quite unfortunate. I think the SSE spec should restrict itself to generic event streams, and be agnostic about how the receiver of an event should interpret it. The fact that the spec has special support for things like "Bubbles" seems like a mistake to me. It's like making the HTML spec have special support for Flash. I have the feeling that it's this over-reaching of the spec that makes the Opera folks think it's reasonable to only support "data" properties. Supporting things like Bubbles would make the implementation more difficult. I'd rather see Opera deal with this by just ignoring the special semantics than by refusing to support event streams with more information.

Arve Bersvendsenvirtuelvis Sunday, September 3, 2006 6:00:21 PM

@Kimberley: The implementation should still be considered "experimental", and part of an emerging specification, which is why we do not have a full implementation yet.

If you have comments in specific about the specification, they are best addressed on the WHATWG mailing list.

Edgar P. NashNetegrof Monday, September 4, 2006 3:47:40 AM

I think that idea is very good and necessary in this moment for better the Opera browser, fix some detailes.

Janizomg Monday, September 4, 2006 4:57:46 PM

"Let's see when the first two player action widget debuts!"

Had source code for a two player widget for a while... Haven't written the networking part for it though but I have something to base that stuff on. Not very "action" though... a board game.


I'm having trouble getting this stuff running.. might be a problem in my PHP or Apache configuration.

Anonymous Thursday, September 7, 2006 7:42:13 AM

Svenner80 writes: Server-Sent Events are a great feature indeed! We've been playing around with this and even though it's still a bit Beta (Opera crashes from time to time when multiple browser instances get opened using server streaming) this will probably get stable over time. What are the policies maintaining connections? E.g. when I change pages (including complete page refresh and reload) and have the same element pointing to the same URI, will the browser open a new connection to the server for this. Or would one have to use something like frames (quite ugly) to prevent the reload of the html parts where is included?

Anonymous Thursday, September 7, 2006 5:22:40 PM

sindhu writes: all this sounds great (although ive no idea how you guys do it) but as a consumer :) of opera. think it should be good if this server side events gives it an edge over other exisiting browsers.

Anonymous Monday, September 11, 2006 8:11:27 AM

Anonymous writes: Something that would definitely be addressed would be how to deal with this on a server implementation. I would definitely love to take advantage of this for a web game that I am developing, so we can easily support real-time operations, and I can see it gracefully failing for non-Opera browsers (though I bet it'd be 5 years before IE8 comes up with half-arsed support for it, maybe Mozilla might have it in the next year or so) ... but it's going to just choke a server if you're dealing with it with a web server. Having a seperate open persistent apache thread running perl or php for every indivdual user connected could bring even high end computers to their knees. You'd be better off writing some sort of application that ran in a single thread, and used select() to deal with incoming connections, and I wonder if PHP could actually even deal with this. (Does PHP have the ability to be used for more robust socket applications? Does PHP have the ability to open a port, listen for many connections, and serve them?)

Phil Endecottendecotp Tuesday, September 12, 2006 8:14:03 PM

In Anyterm (http://anyterm.org/ - site has demos, not sure if they work with Opera!) I have achieved something much like this with plain old HTTP. Basically, the client makes an HTTP request, and the server doesn't send a response until it has some data to send. When it gets the response the client makes another request, which should be over the same persistent HTTP connection. The server has to send an empty response eventually if there is no data so that the client doesn't time out, but it's only necessary to do this perhaps every 30 seconds; if it was polling it would need to be 100 times as fast to get the same level of responsiveness.

This new stuff is certainly interesting, and it would have been great if it had existed from day 1. But it will be a long long time before it is widespread in browsers and for now we can get the same effect using regular HTTP.

Elainemezzomama Tuesday, September 19, 2006 7:10:18 AM

I like being able to chat. It's much more personal than "forums". I managed to log in, but there's no button to log out. I presume that it didn't ask me for a password because I was already logged in to Opera Community. If that is not the case, how secure is it?

Arve Bersvendsenvirtuelvis Thursday, September 21, 2006 4:59:25 PM

mezzomama: The SSE chat is a proof-of-concept demo chat that doesn't use/require authentication[1], so your only identification is the nickname you choose.

However: It is entirely possible to build secure applications with authentication using this technology.

Janizomg Sunday, September 24, 2006 10:30:01 PM

@anonymous user, # 11. September 2006, 10:11:27

I've written a stand-alone server for SSE with Python. So far it's been performing quite nicely on my 333mhz/64mb ram server box

You can find it from this page with some details on how to use it
http://zeeohemgee.blogspot.com/2006/09/writing-sse-backends-with-python_09.html

Elessar Friday, October 20, 2006 10:46:32 PM

@ #11: have a look at lighttpd. Single threaded event based http server that can use kqueue() and other efficient methods (depending on the platform) instead of the old select() and poll() stuff. Linear searches don't scale that well afterall.

PHP is handled via a pool conntected over a fastcgi interface. In terms of ressources this is much more server friendly that apache with mpm_prefork/mod_php or similar.

_Grey_ Wednesday, October 25, 2006 11:27:36 PM

Spelling error: Connect"ng" instead of Connect"ing".

Haavardhaavard Sunday, October 29, 2006 12:13:03 PM

That could be on purpose smile

wykis Saturday, November 4, 2006 3:21:38 PM

This could be used for creating live gaming and chat, how awsome smile Opera browser just rules!!!

stephenvs Thursday, November 30, 2006 6:05:58 AM

My PHP: Version 5.1.2 My Server: Windows 2000 My Internet connection: 1MB SDSL BroadBand: My Testing Opera Simulator: Opera Mini Server Push Data Test Following is the sameple code from Opera tech forum, I have already all the neccessary changes in my server php.ini i change the server time out=0, CGI time out also i put to 60min. why the OM-simulator still return a warning message? Error Message: Warning!The Browser may need to exit to download this file continue? IE also return the same message. <?php header("Content-Type: application/x-dom-event-stream"); while(true) { echo "Event: server-time\n"; $time = time(); echo "data: $time\n"; echo "\n"; flush(); sleep(3); } ?> ****************************************************************** My second example (Client Pulling data from server) ----------------- In this example when client calling server time through button on click we can get the data from server. the same function when we call under javascript timer server cant return anything to client. why?. please share you ideas Total two file Page1.Html, Page1.Php Page1.Html ----------- <html> <head> Demo 1 - The Basic's <script type="text/javascript"> var x = 20; var y = 1; // Get base url url = document.location.href; xend = url.lastIndexOf("/") + 1; var base_url = url.substring(0, xend); var ajax_get_error = false; function startClock(){ x = x-y; setTimeout("startClock()", 1000); if(x==0){ ajax_do('page1.php'); x = 20; } } function ajax_do (url) { // Does URL begin with http? if (url.substring(0, 4) != 'http') { url = base_url + url; } // Create new JS element var jsel = document.createElement('SCRIPT'); jsel.type = 'text/javascript'; jsel.src = url; // Append JS element (therefore executing the 'AJAX' call) document.body.appendChild (jsel); return true; } </script> </head> <body onload="startClock()">
<input type="button" onclick="ajax_do ('page1.php');" value="Get content" / id=button1 name=button1> </body> </html> **************************************************************** Page1.php --------- <?php header("Pragma: public"); header("Expires: Mon, 26 Jul 1997 05:00:00 GMT"); header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT"); header("Cache-Control: must-revalidate"); header("Content-type: text"); $oTime = "GMT = ".date("Ymdhis")."
"; putenv("TZ=Asia/Kuala_Lumpur"); $NTime = "KLTime = ".date("Ymdhis")."
"; $html = $oTime.$NTime; ?> div = document.getElementById('contentdiv'); div.innerHTML = '<?php echo $html; ?>'; ******************

M Amroabadiamroabadi Tuesday, December 26, 2006 2:22:57 PM

This is really interesting,
but the question is:

How to open a PERSISTENT connection to the server ?

I'd be thankful if someone kindly clarify this for me

amirsaied Tuesday, January 9, 2007 7:06:20 PM

Not sure yet, but my examples are not working with 9.10 anymore, is SSE support dropped in 9.10?

Smir Thursday, September 13, 2007 9:50:53 AM

My comment is a bit late here, but except the simple example I haven't played around with this one much.

Now I was writing a small http server, which should send updates to all connected clients, whenever one client connects or disconnects to the event-source. And that was, when I noticed, that server-sent events only work with \n as line ending and not \r\n.

But as far as I understand the HTTP RFC, line endings should be \r\n:

CR = <US-ASCII CR, carriage return (13)>
LF = <US-ASCII LF, linefeed (10)>
CRLF = CR LF

CRLF is used as line-endings.

jacobolus Friday, November 9, 2007 12:14:58 PM

Smir: you're right, it only accepts \n. I've filed it as an Opera bug, #290806. -Jacob

eddie Friday, December 14, 2007 9:56:05 AM

I am a bit interested on de server-side of this technique. Can you post an more detailed example (preferrable of the chat application)?

Does the server page actually needs an infinite loop? How does the script know there is a new event that needs to be sent to the connected clients?

rio258k Wednesday, July 30, 2008 4:42:49 PM

I too am interested in this.

Does anyone else have a more detailed example? I've tried to construct this simple demo, but it isn't working!

grafio Wednesday, October 22, 2008 6:46:21 AM

If the PHP/Python example doesn't work for you it might be because of antivirus caching incoming data. I had this problem wih avast! - you can add application/x-dom-event-stream to MIME exception list in avast! WWW Shield to fix this. But there must be some better way, because Opera Web Chat example works for me without any changes in avast! config. Maybe it depends on server configuration, I don't know...

MyOpera team, please fix this!fearphage Friday, December 19, 2008 4:28:47 PM

I found this post which lists some issues (bugs?) with Opera's event streaming implementation.

jax510 Friday, February 6, 2009 1:07:08 PM

Moderator edit: This comment has been removed for breaching our terms of use.

MauriceNull Wednesday, June 24, 2009 4:31:36 PM

Hi guys,

I need some help, copied the php example:
<?php header("Content-Type: application/x-dom-event-stream"); while(true) { echo "Event: server-time\n"; $time = time(); echo "data: $time\n"; echo "\n"; flush(); sleep(3); } ?>

But if I put this in a file (clock.php), upload it and surf to it in Opera, Opera asks me to download the file. So it doesn't open the page and show the time. Any ideas? THx

kesor Wednesday, July 1, 2009 2:31:27 PM

http://dev.w3.org/html5/eventsource/ states that you should have an object called EventSource.

I want to do capability sniffing in my javascript, something like this:

var capable = {
eventsource: (typeof(EventSource) == "function"),
activex_htmlfile: (typeof(ActiveXObject) == "function"),
plainxhr: (typeof(XMLHttpRequest) == "function")
}

How can I know that a browser supports the <event-source> tag and event-source events? How can I check for it in my javascript, without checking for browsers and browser versions?

Mauricio Silveiramsilveira Sunday, July 12, 2009 2:57:55 PM

As miles stated first: This means "1 connection per user".

Drawbacks:
1.) What if someone is accessing 2 pages on the same server: will it keep only 1 connection open or will there be 2 connections active?
2.) What if dozens of thousands of users keep a page open concurrently on non-clustered servers? Scale it to to clustered servers * other dozens!? - By keeping open I mean forgetting it open.

Maybe this feature will find it's place on controlled environments for a better "real-time" application. Using timeouts for sessions

pepijndevos Monday, October 26, 2009 2:25:31 PM

Does anyone have more information about the differences and workarounds between Webkit and Opera? It seems Webkit uses another conent-type and uses EventSource instead of event-source and has a different event system.

ESP SolutionsESPWebmaster Tuesday, January 26, 2010 7:31:10 AM

Opera rocks! after this feature opera is my favorite browser bigsmile

Write a comment

You must be logged in to write a comment. If you're not a registered member, please sign up.