Wednesday, December 31, 2008

Multicast feature added to UniversalComet / Chat sample app

One of the goals of my project UniversalComet is to enable client-only P2P applications. I thus added the feature to subscribe to scopes and do multicast posts to all subscribers. This enables "discovering" peers and talking to all of them at once.

The feature could be especially useful for things like widgets or facebook apps.

Using this feature I created a sample chat application.

The chat example is hosted on the Google Code SVN repository. There is no server side magic involved. The whole chat is implemented in a couple lines of JavaScript code (using jQuery for DOM access):

$(document).ready(function () {

// Output a message
var log$ = $("#log");
function log(msg) {
log$.text(""+msg + "\n" + log$.text())
}

// Make me a server with Google Gears support enabled
var server = new Addressable.Server({ useGears: true });

// When a message arrives, just "log" it
server.onmessage = function (message) {
log(message)
}

// Out subscription scope is the current URL
var scope = location.href;

// Connect to the UniversalComet server
server.connect(function (id, url) {
log("Connected.")
// When connected subcribe to the scope
server.subscribe(scope, function () {
log("Joined channnel")
// When subscribed, post "Hi All"
server.post(scope, "Hi All")
})

// Subscription expire after 5 Minutes. Renew ours every 4 Minutes.
setInterval(function () { // renew subscription
server.subscribe(scope, function () {})
}, 1000 * 4 * 60)
})

// When the "chatForm" is submitted post the message to the scope on UniversalComet
var message = $("#message").focus();
$("#chatForm").submit(function (e) {
server.post(scope, message.val())
message.val("")
e.preventDefault() // Dont really submit the form
})

})

Security
Anyone can subscribe and post to any scope. You can work around this by using "secret" scope names and doing the subscription and posts on your server to work around this. I might add Authentication to the process later. However, UniversalComet is meant as a low-level API for doing Comet. There is nothing stopping you to layer any authentication layer on top of it that you like.

Compatibility
UniversalComet has now been tested on the following platforms
  • FF3 on Mac (With and without Google Gears)
  • FF2 on Windows (With and without Google Gears)
  • Safari 3 on Mac
  • IE7
  • iPhone
  • Android

Monday, December 29, 2008

UniversalComet - Making every client addressable through URIs

Based on my earlier experiments, I have developed a JavaScript library that makes it extremely easy to incorporate "comet" or server-push technology into any web application without need for special programming or any special support on the server. Ressources needed on the server are also practically zero.

When you use the JavaScript library, you will receive an URI that identifies the client. Now you can make simple HTTP-GET requests to that URI with a message parameter and the client will receive the message immediately. Because only your application will know the un-guessable URI, the client should be save from unwanted messages. The use of HTTP-GET messages also allows sending messages in a "P2P"-fashion from client to client using JSONP cross-domain-messages.

To use the JavaScript library put this piece of code on your website:
<script type="text/javascript" src="http://universal-comet.appspot.com/static/client-server-mini.js"></script>
<script type="text/javascript">
var server = new Addressable.Server();

server.onmessage = function (message) {
alert(message)
}

server.connect(function (id, url) {
// send url to you server using AJAX
})
</script>
A fully working HTML-page can be copied from here.

The function that is passed to the connect-method receives the URI of the client and as soon as messages arrive they will be send to the onmessage-callback.

A Google App Engine application is used as a kind of reverse-proxy for the messages to the clients. Thus there will be no extra load on your server. The JavaScript application always maintains only one connection to the comet-proxy per domain. If a clients has Google Gears installed (Google Gears is not needed for operation) there will only be one connection per web browser. This minimizes the load on the comet-proxy.

The JavaScript application uses "channels" to differentiate between different windows that the same client has open at any given time. These are cached in window.name. If you set window.name yourself this value is used. You can also set server.setChannel("myChannel") before you connect to name the channel yourself.

You can try out the library in the iframe below. This is extremely alpha, practially untested code. If you have a problem, please post a comment (Rudimentary testing has been done in FF3 and Safari on the Mac, FF2 and IE7 on Windows. All with and without Gears.).


Update: There is now a chat sample app.

Thursday, December 25, 2008

Bridge between Ext and Joose

Nickolay Platonov of Symbie has developed an extremely interesting bridge between Joose and Ext. The cool thing is that he has overriden Ext.extend to bootstrap all Ext classes to be full blown Joose classes. Meaning you can easily inherit from them using Joose facilities and especially apply roles to ext classes and object.

More info can be found in this post to the Ext forum and check out the joosified Ext samples.

Nickolay's approach can, of course, be applied to any JS framework that provides a pseudo-class layer so I'm looking forward to joosified YUI, Prototype, etc.

Merry Christmas! Long live the Joosification.

Monday, December 8, 2008

The first days in the clouds

Thanks to Ajaxian and many others there was a lot of traffic on my post about the JavaScript-in-the-cloud-system.
The first effect was that the mighty cloud went down all the time. This was largely due to the fact, that the system requires a somewhat long running (100-600 milliseconds) process on the App Engine servers. Google placed very strict quotas on these requests in order to keep their system stable und thus they frequently disabled the application. Marzia suggested on the App Engine group to ask for an increased quota and it looks like Google followed suit. At least the problems seems to have disappeared.

Since this was all just an experiment, I don't really know whether I'll do anything with it. It really seemed to drive people's imaginations, though. Some of this ideas where:
  • Create SETI for JavaScript
  • Start a botnet
  • Calculate PI
  • Use it to build a very parallel (as in every user of the internet) web crawler
  • Build this into Flash banners to put them to good use
In any case the system shows that Joose can really increase JavaScript's productivity. The whole system was created between 01:30 am and 05:30 am which I think is not too much time for something which could also have just not worked.

If anybody is interested in using the system, just drop me a mail malte.ubl (at) gmail.com or write to the Joose mailing list.

Saturday, December 6, 2008

Massively parallel crowd-sourced JavaScript app server cloud

Reporting live from the "Why? Because you can!"-department:
Probably due to the fact that I watched way too many episodes of Terminator - The Sarah Connor Chronicles, seeing people talk about Skynet, or because I really wanted to build Google App Engine applications with JavaScript or because I have way too much time on my hand, I decided to build a JavaScript application server that runs within the browsers of the people that visit the site that the app server is delivering. Are you following me? :)

The system works as follows:
  • When a user makes a request to the site
  • ...the request is handled by a Google App Engine application
  • ...that puts the request into a request queue
  • Meanwhile on another computer that runs the JavaScript app server within a Google Gears worker within the browser
  • ...a comet-client maintains a pseudo permanent connection to the App Engine app
  • ...looks into the request queue and starts to process the request
  • ...when it is finished it sends the response to the App Engine app
  • ...which immediately send the request back to the client that originated the request in the first place.
Crowd-sourced cloud computing
  • Ressouce requirements on the server side are reduced because every client can share some of its own ressources with every other user of the system
  • Works with my favorite language JavaScript
  • Enables absolutely seamless integration of client- and server software
Limitations
  • Every app server is totally untrusted.
  • For security reasons the response of the JavaScript server application must be JSON. It cannot be plain HTML, because every untrusted client could inject evil scripts into the stream
  • Google App Engine unfortunately does provide a good way to implemented comet-style applications with long-polling because they enforce very strict rules with respect to request timeouts
  • The App Engine SDK builtin-web server is not multi-threaded. Thus the application can not be tested (easily) within the SDK but has to be deployd to the production system.
  • My implementation is very-very proof-of-concept do-the-easiest-thing-that-could-work style with no error checking and various evil race conditions with respect to the request queue.
  • There is currently no builtin way to persist data. However, applications may of course use any kind of web-service that is accessible via JavaScript.
  • There is currently no way to read-or-write HTTP headers from the applications
Demo Client
You can try the the demo application (a simple counter here). It will only work if you either started the server yourself (below) or if somebody else is currently running the server application.


Demo Server
The server is running in this iframe:


The app engine application might be disabled due to overuse of quotas. If that is the case, I'm sorry.

If you are interested in come code, check out the implementation of the server.