Web Applications Blog

Poking through my newsreader I realised that it's been a long time since the Web Applications Team actually shared any real code samples with you guys, and I think it's about high time we fixed that.

Back in September last year Arve Bersvendsen (virtuelvis) introduced you to Event Streaming to Web Browsers along with a few primitive code samples to get you started.

Some of you, quite rightly, pointed out that with a regular apache configuration the concept of keeping open many long lived requests doesn't hold. Also many of you requested that we provide more detailed examples of code working with this. I hope that today's blog post will go some way to addressing both issues.

The Theory

First for the boring part, a quick discussion of how to tackle the issue of
scalability.

In a normal web application when a server receives a HTTP request, an operating system thread is allocated from a pool and it is not released until the whole request is complete. When using SSE the request does not complete until the user closes the html document that has requested the event stream, this means that while a user is using your application the thread that is dealing with the SSE request can not be returned to the pool to serve requests for other users even if the SSE channel is currently idle.

The solution to this problem is to, while there is no data to be written to the outbound socket, release the thread so that it can process another client's request. Most languages have a socket APIs that can be exploited to implement this kind of behaviour however it is only recently that this technology is being exploited fully by web server developers in response to the AJAX wave of technologies.

The Solution

As far as hosting sites on apache is concerned I have not yet found any solution for doing this in a scalable manner in "traditional" languages such as PHP.

In python is is reasonably easy to hack together your own HTTP server using asyncore or socketserver however I don't feel that everyone should be writing their own HTTP 1.1 implementation just to achieve a scalable solution. This is where I think twisted.web has a good shot at being an elegant solution. The python twisted framework is an event based networking layer that allows reasonably easy access to the networking techniques we need and twisted.web is a HTTP layer built on top of that. In fact the existing SSE based web chat is a sort of compromise here, the twisted framework was used for low level connectivity but we did write our own, very primitive HTTP layer on top of that. One thing I'm interested in doing this year is having a go at this myself but using the full twisted.web stack rather than writing my own HTTP implementation.

In the Java world (where my heart lies) I'm ashamed to say that each vendor appears to have gone their own way regarding how to implement threadless waiting. Tomcat 6 features the newCometProcessor interface (why oh why did they choose a name that tied it to Comet:doh:) whereas Jetty have implemented continuations in the server. For a more complete assessment of the options in the Java world, see Greg's blog article on the subject (he also has GWT related code samples if you look around.

After looking at the above options I find that I am in general agreement with Greg, and have decided to proceed with the rest of this blog article to show
how to implement an SSE service in Java running on Jetty using their continuation support to achieve scalability.

The Code

The first part of the process was to model the Events themselves. Using the DOM Level 3 Events Specification and the Web Applications 1.0 Draft I came up with the following two classes and also decided to make them responsible for serialising themselves to a format that can be pushed out onto the output stream.

public class ServerSentEvent implements Serializable
{
    /** DOM 3 Event namespace */
    private static final String DOM3_EVENTS_NAMESPACE =
            "http://www.w3.org/2001/xml-events";    
    private String type;    
    private String target;
    private Boolean bubbles;    
    private Boolean cancelable;    
    private String namespace;
    // *snip * The usual getters and setters
    
    protected void generateEventContent(StringBuffer sb) {}
    
    public final String toString()
    {
        // Provide default event properties
        StringBuffer sb = new StringBuffer("");
        sb.append("Event: ").append(type).append("\n");
        sb.append("Class: ").append(this.getClass().getSimpleName()).append(
                "\n");
        if (target != null)
        {
            sb.append("Target: ").append(target).append("\n");
        }
        if (bubbles != null)
        {
            sb.append("Bubbles: ")
            .append(bubbles.booleanValue() ? "Yes" : "No").append("\n");
        }
        if (cancelable != null)
        {
            sb.append("Cancelable: ").append(
                    cancelable.booleanValue() ? "Yes" : "No").append("\n");
        }
        if (namespace != null)
        {
            sb.append("Namespace: ").append(namespace).append("\n");
        }
        
        // Provide method for subclasses to provide more event info
        this.generateEventContent(sb);
        
        // Terminate the event with a blank line
        sb.append("\n");
        return sb.toString();
    }
}

public class RemoteEvent extends ServerSentEvent
{
    /** Namespace for remote events */
    private static final String REMOTE_EVENT_NAMESPACE =
            "uuid:755e2d2d-a836-4539-83f4-16b51156341f";
    
    private List<String> data = new ArrayList<String>();
    
    public RemoteEvent(String type)
    {
        super(type, REMOTE_EVENT_NAMESPACE);
    }
    
    public void addData(String data)
    {
        this.data.add(data);
    }
    
    protected void generateEventContent(StringBuffer sb)
    {
        super.generateEventContent(sb);
        for (String currentData : data)
        {
            sb.append("data: ").append(currentData).append("\n");
        }
    }
}

There are also a couple of interfaces defined for listening to and generatingserver sent events in a similar manner to how such event listeners are handled in swing.

Next we need to implement a class to represent a client connection to the system, this is where the real meat lies.

public class SSEClient implements ServerSentEventListener
{
    public SSEClient(HttpServletRequest request, HttpServletResponse response)
    throws IOException
    {
        continuation = ContinuationSupport.getContinuation(request, this);
        this.response = response;
        this.request = request;
        
        writeHeaders(this.response);
    }
    
    private void writeHeaders(HttpServletResponse response) throws IOException
    {
        // Set the necessary headers
        response.setContentType("application/x-dom-event-stream");
        response.setCharacterEncoding("utf-8");
        response.setHeader("Cache-Control", "no-cache");
        response.setHeader("Transfer-Encoding", "Chunked");
        response.flushBuffer();
    }
    
    private void writeEvent(ServerSentEvent event) throws IOException
    {
        byte[] eventBytes = event.toString().getBytes();
        ServletOutputStream os = response.getOutputStream();
        os.println(Integer.toHexString(eventBytes.length));
        os.println(event.toString());
        os.println();
        os.flush();
        response.flushBuffer();
    }
    
    public void run()
    {
        try
        {
            // This outer while loop ensures correct operation when emulating
            // continuations i.e. not running in Jetty
            while (true)
            {
                synchronized (this)
                {
                    while (events.peek() != null)
                    {
                        log.debug("Found an event, writing it to the stream");
                        writeEvent(events.remove());
                    }
                }
                continuation.suspend(WAIT_TIME);
            }
        }
        catch (IOException e)
        {
            this.disconnect();
        }
    }
}

Finally we need a service class to manage the currently connected clients that can be called on by servlet code. This is so simple I'll let you poke at that in the download.

And that's it! My entire JavaSSE framework in ~400 lines of code.

Using this framework I have created a couple of very simple demo services that rely on no major libraries such as spring in order to keep them short and sweet. All of this is covered by a BSD style license.

They can be found in the zip file below along with a maven pom that will allow you to run the demo with the command

$ mvn jetty:run

JavaSSE.zip

The Results

The Hardware

In order to make this a sensible test I comandeered a couple of spare blade servers that I had left over from testing a previous project. Both have the following specification.

* 2 dual core 3.20GHz Xeon processors
* 3Gb RAM
* Debian Etch amd64 kernel

The Proceedure

In order to test out the scalability of this implementation I thought it would be sufficient to just throw apache bench at the SSE stream for the server time example in the package above. This gives very rough indications.

On the Jetty machine I placed a war file of the project in the jetty webapps
directory and fired up the server process with

$ java -XX:MaxPermSize=128m -Xmx3072M -jar start.jar

Then on the Apache bench machine I set up a bunch of requests with a command such as

$ ab -n 50000 -c 2000 -t 18000  http://jettyserver:8080/JavaSSE/ServerTime

The Results

Using apache bench I was able to step up the number of simultaneous connections to 4000 before I experienced any problems with the service. This is a huge improvement on the 200 you would get out of a standard servlet container on default configuration but still nothing stellar performance wise.

Note that these tests were done on a default configuration of Jetty and so by tweaking the config perhaps I can squeeze even more out of the container.

One final point, after running apache bench for about an hour the jetty process froze, my best explanation for this is that there is a memory leak somewhere in the application. I don't have enough time to investigate this now but I'm sure we could find a t-shirt for the first person that does.

Wrap Up

I'm hoping that I get to maintain my current workload and therefore can continue to post more technical articles. I have plans to make a beefed up version of the chat that was made before and perhaps I will get through this exercise with twisted.web. If anyone has any requests let me know in the comments.