Transparent Client-Server PubSub implementation details

For those of you interested in source code, here are the links to the implementation of my prototype of a transparent publish-subscribe system between client and server:

Server

• three new URLs are "exposed":
  
• /event/forward accepts events published on the client and publishes them on the server
  
• /event/bind send the information about which events the server would like to subscribe to to the client
• /event/listen is a comet connection which is used to transfer events published by the server to the client
• lib/bespinserv/pubsub.js implements the actual pubsub mechanism in the server.
• a java.util.concurrent.locks.ReentrantLock() is used to implement the lock for the comet implementation. This was taken and slightly modified from the ServerJS comet example.
• Whenever a subscription is fired on the server, the event queue (and thus the associated client) is saved in a global variable (remember JS is single threaded) so that when the server does a publish we know where to send the event.
• Multicast publishes are not supported.
  

Client

• Upon "/event/bind" the client creates stub event subscriptions which forward the actual event to the server using /event/forward
• The client connects to /event/listen. Once it returns the client publishes the returned event and starts listening again immeditately.

Please excuse the messed up indentation, I was using a new editor (not bespin :).

I'm personally not convinced that this architecture should be used because the beauty of REST goes away. On the other hand, I feel that the artificial border between client and server does not make sense for some applications and thus it is great if it goes away. Opinions?

One event loop to rule them all and in the darkness bind them

Based on my earlier work to bring transparent pub-sub semantics to web workers I implemented a pub-sub framework that does the same for server sided JavaScript.
The implementation is built on the excellent ServerJS platform and Kevin Dangoor's early prototype ServerJS bespin backend.

This means that you can now do something like this on the server side:

bespin.subscribe("test", function (event) {
bespin.publish("test:receive", event)
})

and then do

bespin.publish("test", { some: "data" })

on the client. Because the server listens for that event, the event defined above will now fire. In the code snippet above the server immediately fires "test:receive". If now the client is subscriped to that event, this subscription will fire.

Events are transferred to the server via simple HTTP requests. Also the client maintains a comet connection to the server which is used to transfer events which are published on the server to the client.

Based on my twitter stream the implementation of this protoype took under 2 hours. Considering it was my first ever ServerJS application, I can only applaud the people behind ServerJS to the awesomeness of their work. While the current state is still somewhat experimental, it is amazing how much was achieved in so little time.

PS: It is only bespin.subscribe above because I did: var bespin = require("./pubsub").pubsub;

Video of my talk on postmodern web applications

Here is a video of my talk on postmodern web application at next conference:



...and the slides to follow along:


The target audience of the presentation was marketing guys with a tech affinity :) I'm going to enhance this with some more in depth tech content and a good part about Joose, of course, for my talk at SWDC2009 next Monday (By the way, there are ticket for 150 Euros for people attending from outside of sweden).

One event loop to rule them all

While experimenting with web workers in bespin I made a small change that makes working with them a lot nicer. In bespin we make heavy use of custom events to loosely bind things together in the application. So you see a lot of

bespin.publish("something:happened", { ... })

and

bespin.subscribe("something:happened", function () { ... })

The change I made is that you can now do the same thing in workers and it will Do The Right Thing (tm). Meaning you can subscribe to events that might be triggered by the UI or anywhere else from within the worker and also publish events that might be observed by handlers which live in the "main page". Overall this makes working with workers much more seamless and first class from a bespin perspective because it means that as long as one is not doing any direct UI work (as opposed to sending events to UI components) one can do everything from the worker without building custom interfaces.

For me working on bespin is really an experiment on how to design an event driven client side web applications (or postmodern web application). One of the things that felt kinda awkward up until now was how to know the right point in time to initialize a particular component. You might want to wait until the settings have been loaded (asynchronously via Ajax) and another component has been initialized. All these events are, of course, signaled by custom events but the order might be totally random and they might have already happened when we start looking for them. The solution I came up with (whether it is a good one remains to be seen) is to have a function that checks a list of events against all events that have fired already and then waits for the rest of the events to fire and finally calls a callback when this is done:

    // ** {{{ assertEvents }}} **
   //
   // Given an array of topics, fires given callback as soon as all of the topics have
   // fired at least once
   assertEvents: function (topics, callback) {
       var count = topics.length;
       var done  = function () {
           if(count == 0) {
               callback();
           }
       };
       for(var i = 0; i < topics.length; ++i) {
           var topic = topics[i];
           if (this.eventLog[topic]) {
               --count;
           } else {
               bespin.subscribe(topic, function () {
                   --count;
                   done()
               })
           }
           done();
       }
   },

Postmodern Web Applications

I had a great time today talking about what I called "Postmodern Web Applications" at next conference in Hamburg, Germany.

For everyone interested in what this is about, here are the slides:

Postmodern Web Apps

View more presentations from malteubl.