Saturday, December 19, 2009

Summary of JSConf.eu and Video of my talk about JavaScript meta programming with Joose

I haven't really written about the blast I had organizing JSConf.eu together with @hblank and @janl. We really didn't know what we were doing and it all worked out well, anyway :)

Here is what Steve Souders said in a comment to a blog post on the Yahoo Developer Network.
... I was so impressed with the quality of this conference. The speakers were great, but even more important the *audience* was hugely technical and informed about the issues of JS development. Being a conference co-chair, I know what it takes to put on an event like this. Hats off to Jan, Malte, and Holger for organizing this first time event and making it such a success.
We are really proud of this and we are going to do it again next year!

Of course, I totally underestimated the effort to orchestrate such a conference and agreed to do a talk myself. Honestly I was quite nervous but it wasn't too bad. Here is the video (And yes, the title of the presentation is supposed to be ironic).

Tuesday, October 13, 2009

Joose moves to GitHub

The official Joose repo has been moved to GitHub (Thanks to Nickolay for the work). Looking forward to your code!

Wednesday, September 30, 2009

A DSL for generating HTML with JS

Maybe some people find this interesting. I created a little DSL to generate HTML with JavaScript. Here is an example:
var things = [{text: "Hello"}, {text: "World"}];

with(HTML) {

var html = div({
id: "foo",
content: a({
href: "#",
content: ["test", span("bar")]
})
}) +

div({
className: "baz",
content: map(things, function (thing) {
return p(thing.text)
})
});

alert(html)

document.write(html)
}

Yes, I know, with is very evil but it is cool in this situation.

Here is the full source. I hacked it up in 30 minutes and it is in bad need for a cleanup, but otherwise it seems to work alright.

var HTML = (function () {
function encode(text) {
text = text.replace(/&/g, "&", "g");
text = text.replace(/</g, "&lt;", "g");
text = text.replace(/>/g, "&gt;", "g");
text = text.replace(/"/g, "&quot;", "g");
text = text.replace(/'/g, "&apos;", "g");
return text;
}

function renderAttr(attr) {
if(!attr) return '';
var html = '';
for(var name in attr) {
var value = attr[name];
if(name === "className") name == "class";
if(name === "content") continue;
html += ' '+name+'="'+encode(value)+'"'
}
html.encoded = true;
return html
}

function makeTag (name) {
return function (paras, content) {
if(arguments.length < 2) {
content = [];
}
if(typeof content === "string") content = [content]
var attr = {};
if(typeof paras === "string") {
content = [encode(paras)];
} else if(paras == null) {
content = [];
} else if(paras.content) {
var c = paras.content;
delete paras.content;
return arguments.callee(paras, c);
} else if(paras.length > 0) {
content = paras;
} else {
attr = paras;
}
var empty = content.length === 0 ? true : false;

var html = '<'+name+renderAttr(attr)+(empty ? '/' : '')+'>';
if(!empty) {
html += content.join("");

html += '';
}
return html;
}
}

function map(enumerable, fn) {
var result = [];
for(var i = 0, len = enumerable.length; i < len; ++i) {
result.push(fn(enumerable[i]))
}
return result;
}

var tags = ["div", "a", "p", "span"]

var HTML = {
map: map,
encode: encode
};

map(tags, function (name) {
HTML[name] = makeTag(name)
});

return HTML;
})();

Joose IRC Channel

We'd like to announce that Joose now has a dedicated IRC channel on irc.freenode.org:

#joose

I'm currently working on a large online store project that is using a platform based on server side JavaScript. We use Joose on both the server side and the client side. Very good stuff. More on that later.

Wednesday, August 26, 2009

Running the Bespin command line from the, well, the command line

I felt a little masochistic today which left two possibilities open:
  1. fix IE6 bugs in (pick any project)
  2. program in AppleScript
I chose #2.

The following script will execute it's command line arguments as commands inside bespin (if it is running in the current tab of Safari).
on run argv
tell application "Safari"

set docs to every document

set cmd to ""
repeat with c in argv
set cmd to cmd & c & " "
end repeat

set js to "(function () {if(typeof bespin != 'undefined') { return bespin.get('commandLine').executeCommand('" & cmd & "').output.replace(/]*>/g, '\\n').replace(/<[^>]*>/g, '') } else { return '' }})()"

set out to ""

repeat with doc in docs

set output to do JavaScript js in doc

set out to out & "
" & output

end repeat

out

end tell
end run


Wrap this in a little shell script:
#!/bin/bash
cp bespin.scpt bespin.work.scpt
osascript bespin.work.scpt "$@"

(Don't ask about line two (it is part of the masochism experience :))

... and now stuff like this works:
-bash-3.2$ ./bespin set | grep collab
collaborate = off

Thursday, August 13, 2009

Bringing the awesome JSConf to Europe

@janl, @hblank and @me are proud to announce that we will bring JSConf to Europe!

From the announcement:

JSConf in Washington, DC in April this year proved to be the homecoming of a JavaScript community that didn't know it existed. The conference got top marks all around for content, community and fun. With JSConf.eu, we let that spirit live on and give JavaScript a home in Europe. (why let the americans have all the fun? :)

Mark November 7th/8th in your calendar, we'll be presenting top speakers in the field and a relaxing atmosphere in Berlin, Germany's pulsing capital. Join us for a two-day, community-focussed indie conference with top technical content around your favourite language and great opportunities to meet like-minded people.

Our speakers so far include:
Amy Hoy,
Thomas Fuchs,
John Resig,
Dion Almaer &
Ben Galbraith

I'm really looking forward to the event. See you there!

Sunday, August 2, 2009

Joose 2.1 released

We are proud to announce the release of Joose 2.1

This release is a pure maintenance release with minor fixes and enhancements:
  • Fix a rare issue that occurs with Firefox 3.5 native JSON supports when using Joose.Storage
  • Fix an issue with the Google Gears detection code in Joose.Gears when Gears is installed but disabled in Firefox 3.5
  • Various performance enhancements in heavy used methods
  • Joose.Singleton's singletonInitialize methods now receives the parameters which are passed to the first call of getInstance().
If you are not using either Joose.Storage or Joose.Gears there is no immediate need to upgrade.

Wednesday, July 22, 2009

Google Chrome's Very Incomplete Web Worker Support

In bespin we use a facade that allows us running JavaScript code in Web Workers, Gears Workers and if those are not available in the main thread.

Google Chrome does not yet support Web Workers but since it has Google Gears built in it should use the appropriate fallback. Turns out it didn't with our original code.

I naively implemented object detection to check for the availability of the Worker-object like this:
if (typeof Worker == "undefined") {
For some reason the statement above is actually true in Chrome 2, even though as stated above support for the Worker API has not been implemented.

I then tried to instantiate the Worker object. All this does is to throw an exception with the message "Worker is not enabled". This looks like an unfinished implementation that was only partially removed or something in that direction.

This code handles the special case:
var noWorker = typeof Worker == "undefined" ? true : false;
if(!noWorker) {
try { new Worker("test.js") }
catch(e) {
e = e + ""
if(e.indexOf("Worker is not enabled") != -1) {
noWorker = true;
}
}
}
I'd be very interested why this fragment of the worker implementation was left in the code? Most likely it is a bug but it is questionable whether it will be fixed since the next version of Chrome will actually support the Worker API.

Tuesday, June 23, 2009

The Internet is broken

For a couple years now we are breaking the internet. When popular short URL providers like tinyurl, bit.ly or tr.im have a downtime, loose a database or even go out of business, we all are using massive amounts of connections through the World Wide Web.

The way to fix this would be to do away with the need for short URL providers. Until then I think we should start archiving all short URLs and their mappings to long URLs so that we can still resolve them when the original provider can no longer do it.

For this purpose I created a little Google App Engine app permanize.org.
It resolves short URLs and stores the mapping. Upon the next resolve it will use its own database to do the resolving. You can also download the database to distribute the network (a real P2P architecture would be cool here) and make the archiving more save.
I also created a very experimental Firefox plugin with the ultra-cool Mozilla Jetpack that replaces short-urls with long urls (only the href values of links for now) and automatically archives all short URLs that you visit with the browser. To install it, download Jetpack and then go to the permanize-site.

Monday, June 15, 2009

Scalable PubSub with Node.js

I made some extentions to my server sided pub sub implementation based on Node.js.

When a client establishes a comet connections to listen for events published by the server it in turn receives the internal URL of the Node that it it attached to.
Now when the client publishes an event to any server (or actually a different node process running on a particular machine) and the server in turn publishes an event to client, the node that received the event relays the response to the correct Node via that URL which then uses the open comet connection to send the message to the client.

Overall this implements a solution to do PubSub between client and server without the need for stateful load balancing.

Here is the changeset for the interested.

Sunday, June 7, 2009

Server Sided PubSub via Node.js

A couple days ago ry released the very exciting JavaScript web server Node.js which is purely built on event-based IO using v8 (Btw: Did anyone notice that this blog is now hosted under the domain nonblocking.io :).
From a JS perspective this means building scalable comet application is as easy as not sending a response immediately but rather doing that in a callback function whenever an event might happen. As long as there is only a single node, writing an event queue is also easy. Just use an object that maps queue-ids to listeners.
To give Node.js a try I ported the work I did to allow servers to subscribe to custom events on the client. It turned out to be really straight forward. Here is the result.

Saturday, June 6, 2009

Saving the environment and fixing IE6 issues with Omniture tracking

One of the bugs in internet explorer JavaScript engine that can really ruin your day if it bites you is the fact that string performance slows down exponentially with respect to the number of operations.
One of the evil parts of this bug is that because it only starts really being bad once you go above a certain threshold of operations. This means that the part that is being slowed down might not be the part to fix, but rather just happens to be just after your application passed the threshold. This is why the the behavior of Omniture's client side tracking code that I will describe might never happen to your site. But it can happen at any time when you add more JavaScript code and if it happens we are seeing that loading the Omniture tracking starts taking up to 25 seconds (yes 25 seconds, aka an eternity) in Internet Explorer 6.

Saving the environment
Now you might not care about performance issues in Internet Explorer 6 because Internet Explorer 6 should die, but there are other issues to consider, too:
Every time a visitor does a page view on a site that is using Omniture for tracking, the Omniture code runs through the following steps:
  1. Eval the script
  2. Deobfuscation Part 1 (The deobfuscation operations include multiple substitutions and shuffling of parts of the string using a "key")
  3. Deobfuscation Part 2
  4. Eval of the result of the deobfuscation.
Steps 1-3 are totally unneccessary, all they do is slow down every page load and waste energy doing operations that add absolutely no value to your site. Now the good thing is, that there is an easy fix:
Obfuscating JavaScript code is obvisously a very futile endeavour because your script will eventually have to create regular, executable JavaScript. This is good because all we have to do is to take the JavaScript just before it is being evaled and use that as the Omniture tracking code. Thus eliminating steps 1-3 and saving a little bit of energy upon every page load :)

Now it is not just that easy, there is one more catch: The guys that created the omniture obfuscation code tried to be smart and to make our lifes harder. Step 3 of the deobfuscations add bullshit characters like \0 to the script which can be passed to string-eval but which cannot exist in regular JavaScript files. There might be other ways around this, but I took the easy path: Instead of directly pasting the output of step 3 into my page I used the standard escape() function to escape the string. The resulting string can then be passed to unescape() to create the real thing which can then be evaled.

This is how you can fix your tracking code until Omniture releases a fix for the issue (Works with H.19.3 but should work in later versions):
  1. Somewhere in your tracking code there is a part saying: c = s_d(c)
  2. The function might be called something else but the name should end in "_d"
  3. Add a JavaScript statement here that says something like console.log(escape(s_d(c)));
  4. Copy the result of the console.log to your clip board
  5. Substitude the statement c = s_d(c) with c = unescape("PASTE_FROM_CLIPBOARD")
We tried to contact Omniture about this issue but they haven't responded to our paid support inquiry in weeks. Meanwhile their twitter account is much more responsive, but couldn't help us either.

The fix is running without issues on on of our customer's sites thus saving 30 million deobfuscations per month already. Writing this blog article took about 4 times as long as figuring out the deobfuscation itself. The competitive advantage that might be gained with the extra "security" mechanism is thus only worth a couple of minutes.

Thursday, May 28, 2009

Transparent Client-Server PubSub implementation details

For those of you interested in source code, here are the links to the implementation of my prototype of a transparent publish-subscribe system between client and server:

Server
  • three new URLs are "exposed":
  • /event/forward accepts events published on the client and publishes them on the server
  • /event/bind send the information about which events the server would like to subscribe to to the client
  • /event/listen is a comet connection which is used to transfer events published by the server to the client
  • lib/bespinserv/pubsub.js implements the actual pubsub mechanism in the server.
  • a java.util.concurrent.locks.ReentrantLock() is used to implement the lock for the comet implementation. This was taken and slightly modified from the ServerJS comet example.
  • Whenever a subscription is fired on the server, the event queue (and thus the associated client) is saved in a global variable (remember JS is single threaded) so that when the server does a publish we know where to send the event.
  • Multicast publishes are not supported.
Client
  • Upon "/event/bind" the client creates stub event subscriptions which forward the actual event to the server using /event/forward
  • The client connects to /event/listen. Once it returns the client publishes the returned event and starts listening again immeditately.
Please excuse the messed up indentation, I was using a new editor (not bespin :).

I'm personally not convinced that this architecture should be used because the beauty of REST goes away. On the other hand, I feel that the artificial border between client and server does not make sense for some applications and thus it is great if it goes away. Opinions?

Wednesday, May 27, 2009

One event loop to rule them all and in the darkness bind them

Based on my earlier work to bring transparent pub-sub semantics to web workers I implemented a pub-sub framework that does the same for server sided JavaScript.
The implementation is built on the excellent ServerJS platform and Kevin Dangoor's early prototype ServerJS bespin backend.

This means that you can now do something like this on the server side:
bespin.subscribe("test", function (event) {
bespin.publish("test:receive", event)
})
and then do
bespin.publish("test", { some: "data" })
on the client. Because the server listens for that event, the event defined above will now fire. In the code snippet above the server immediately fires "test:receive". If now the client is subscriped to that event, this subscription will fire.

Events are transferred to the server via simple HTTP requests. Also the client maintains a comet connection to the server which is used to transfer events which are published on the server to the client.

Based on my twitter stream the implementation of this protoype took under 2 hours. Considering it was my first ever ServerJS application, I can only applaud the people behind ServerJS to the awesomeness of their work. While the current state is still somewhat experimental, it is amazing how much was achieved in so little time.

PS: It is only bespin.subscribe above because I did: var bespin = require("./pubsub").pubsub;

Sunday, May 17, 2009

Video of my talk on postmodern web applications

Here is a video of my talk on postmodern web application at next conference:



...and the slides to follow along:


The target audience of the presentation was marketing guys with a tech affinity :) I'm going to enhance this with some more in depth tech content and a good part about Joose, of course, for my talk at SWDC2009 next Monday (By the way, there are ticket for 150 Euros for people attending from outside of sweden).

Monday, May 11, 2009

One event loop to rule them all

While experimenting with web workers in bespin I made a small change that makes working with them a lot nicer. In bespin we make heavy use of custom events to loosely bind things together in the application. So you see a lot of
bespin.publish("something:happened", { ... })
and
bespin.subscribe("something:happened", function () { ... })
The change I made is that you can now do the same thing in workers and it will Do The Right Thing (tm). Meaning you can subscribe to events that might be triggered by the UI or anywhere else from within the worker and also publish events that might be observed by handlers which live in the "main page". Overall this makes working with workers much more seamless and first class from a bespin perspective because it means that as long as one is not doing any direct UI work (as opposed to sending events to UI components) one can do everything from the worker without building custom interfaces.

For me working on bespin is really an experiment on how to design an event driven client side web applications (or postmodern web application). One of the things that felt kinda awkward up until now was how to know the right point in time to initialize a particular component. You might want to wait until the settings have been loaded (asynchronously via Ajax) and another component has been initialized. All these events are, of course, signaled by custom events but the order might be totally random and they might have already happened when we start looking for them. The solution I came up with (whether it is a good one remains to be seen) is to have a function that checks a list of events against all events that have fired already and then waits for the rest of the events to fire and finally calls a callback when this is done:
    // ** {{{ assertEvents }}} **
//
// Given an array of topics, fires given callback as soon as all of the topics have
// fired at least once
assertEvents: function (topics, callback) {
var count = topics.length;
var done = function () {
if(count == 0) {
callback();
}
};
for(var i = 0; i < topics.length; ++i) {
var topic = topics[i];
if (this.eventLog[topic]) {
--count;
} else {
bespin.subscribe(topic, function () {
--count;
done()
})
}
done();
}
},

Wednesday, May 6, 2009

Postmodern Web Applications

I had a great time today talking about what I called "Postmodern Web Applications" at next conference in Hamburg, Germany.

For everyone interested in what this is about, here are the slides:

Wednesday, April 15, 2009

Conference Tour: JSConf, Next, SWDC

Making final arrangements for my (first) "conference tour" this summer:

I'll be at JSConf at 24th and 25th of April in Washington, DC, probably speaking about Joose on Track B.

Next up is my employer's own Next Conference here in Hamburg. I'm still actively working on the presentation. The topic will be "Building Postmodern Web Applications". I'll look into the building blocks of the post Web 2.0 era web applications and I'll be talking about applying techniques from 90s era desktop apps to the kind of web apps we'll be building. My general example will be the bespin browser-based source code editor which is my current favorite open-source project.

Last but not least, I'll be at the Swedish Web Developer Conference in Stockholm, once again talking about Joose and maybe some other stuff. Here is the summary of my talk:

Joose - JavaScript 2 Enterprise Edition
Come see how we turn JavaScript from a beautiful expressive light weight programming language into a slow, committee driven, 500 pound beast. Oops.^M
Joose is a self-hosting meta object system for JavaScript with support for classes, inheritance, mixins, traits, method modifiers and more that aims to make object-oriented programming with JavaScript expressive, declarative and very productive. The talk will introduce the core features of Joose, show how the object system can help you WriteBetterCode (TM) and walk though examples and the code that drives them.

Tuesday, April 14, 2009

The Joose Mutability branch

Nickolay has been hard at work pushing new features into Joose and refactoring the core to make it more accessible and extendible. Here is a feature of his work taken from the mailing list (be aware that this is core stuff. None of it should shine through to user level, although there are some nice new things comming out of it):

The sources are now divided into 3 "meta-levels".

Level 1 - Joose.Proto

This level uses the same inheritance principle, as ExtJS framework. Later I found out, that its
called "Crockford object model" (http://javascript.crockford.com/prototypal.html)
As an addition to this model, it possible to use this.SUPER() calls

Class definition at this level looks like:
TestClass = new Joose.Proto.Class('TestClass', {
isa : SuperClass,

attr1 : initValue1,
attr2 : initValue2,

method1 : function () { ... },
method2 : function () { ... }
}).c;
By default every class inherit from Joose.Proto.Object, which contains 'initialize', 'toString', 'SUPER', etc

At this level, attributes and methods are installed directly in class prototype, thus no Roles exists.

Level 2 - Joose.Managed

This level implements a set of classes, which represents "managed" properties.

Class definition at this level looks almost like Joose class:
TestClass = new Joose.Managed.Class('TestClass', {
isa : SuperClass,
does : [ Role ],

have : {
attr1 : initValue1,
attr2 : initValue2,
},

methods : {
method1 : function () { ... },
method2 : function () { ... }
},

after : {
method1 : function () { ... }
}
}).c;
At this level, attributes and methods are represented with instances of special classes. Also appears method modifiers.
Methods are installed not directly but with wrappers, which allow to use the same method in different classes - and implement Roles.

Also - there is no complex attributes on this level.

Level 3 - Joose.MetaClass, MetaRole

This level allows to use complex attributes and class definition looks like normal Joose class:
Class('TestClass', {
isa : SuperClass,
does : [ Role ]
...
});

Mutability

Any class or role, can be extended with following construction:
TestClass.meta.extend({
doesnt : [ Role ],
does : [ Role1],

havenot : [ attr1 ],

...
})
Attributes, methods, roles - can be removed or added. The change will reflect in whole inheritance hierarchy (for classes) or through all composition relations (for roles)

Inheritance

Mutability implied a switch toward single inheritance scheme. I think Roles fully compensate this.

Its safe to inherit from lower "meta-level" - calls to SUPER are transparent.
Inheritance is now cheap operation (due to "crockford model")

Roles


Roles are implemented on Traits spec (http://www.iam.unibe.ch/%7Escg/Archive/Papers/Scha03aTraits.pdf)
They support aliasing and excluding:
    Class('Creature', {
does : [{
role : Walk,
alias : {
stop : 'stopWalk'
},
exclude : [ 'stop' ]
}, {
role : Eat,
alias : {
stop : 'stopEat'
},
exclude : [ 'stop' ]
}]
});
Conflicting properties are marked with conflict markers.

Roles can be applied also to instances (with some limitations)


Class methods

Instead of classical class methods, built-in singletons (symbiont class) are implemented. They are defined via builder 'my' and can contain any usual class builders:
Class('TestClass', {
isa : SuperClass,
does : [ Role ]
...,


my : {
have : { attr1 : initValue1, attr2 : initValue2 },
methods : { method1 : function () { ... } },
does : [ Role ]
...
}
});
they can be accessed as:

if (TestClass.my.attr1 == 'foo') TestClass.my.method1();


Startup time

As inheritance operation is now cheap, startup time for now mostly depends from how many additional roles are applied to metaclasses.

Without additional attributes roles and "depended" roles its the same as for 2.0.
With all those features though, its about twice longer (all measurements were done in FF on Ubuntu)

This can be improved if roles will be applied grouped (not in sequential order), or if the roles will be included to the sources of appropriate classes from start.

SUPER calls (and serialization possibilty)

From start, SUPER calls on "managed" level were implemented with attached properties to wrappers.
Though this reflects on performance (each call to SUPER used up to 2 calls to arguments.callee.caller), so I switched them to use closures, like in current version. So now the SUPER calls are at the same performance, but seriazliation is not possible.

Though, this can be configurable, and if such performance hit is acceptable, serialization can be done (with some additional code)

Btw - about installing JS modules. Yes, Ingy already uses the approach when the JS modules are published on CPAN. Their's sources in his approach lies near the corresponding *.pm files.

And if we'll change the "target installation" directory to something different from standard perl library, we'll get a nice, separate JavaScript lilbrary. Thats the goal, I hope he'll do it.

Otherwise, I'm going to tweak the Module::Build::JSAN for this, it looks not complex.

Wednesday, April 1, 2009

A structured outline view for bespin

After some major parse tree wrestling I finished an experimental structured outline view for JavaScript files in bespin. It's primary feature is that it can be modified at runtime to adapt to your favorite dialect of JavaScript.


While most programming languages have fix constructs like class, namespace or even event declarations, with JavaScript you have to build all these things from simple building blocks. On the other hand, real-world JS code usually follows quite strict rules to build these things. E.g. bespin itself uses dojo to declare "classes" (or constructors, whatever you want to call them) like this:
dojo.declare("bespin.parser.CodeInfo", null, {})
and this pattern to subscribe to and publish events:
bespin.subscribe("parser:error", function(error) {
bespin.publish("message", {
msg: 'Syntax error: ' + error.message + ' on line ' + error.row,
tag: 'autohide'
})
})
With my personal favorite object framework Joose you will find code like this:
Module("Bank", function (m) {
Class("Account", {})
})
While a programmer, who knows nothing about JavaScript will easily recognize the meaning of these statements, JavaScript has to actually execute them to derive the meaning.

My new outline generator works around this problem by being configurable through a kind of DSL which tells the parser more about the specific dialect of JavaScript that you are speaking. Here is the code to tell the parser about the specific pattern that bespin uses to publish and subscribe to events:

bespinEventPublish: {
declaration: "bespin.publish",
description: "Publish"
},
bespinEventSubscription: {
declaration: "bespin.subscribe",
description: "Subscribe to"
}

How it works:
At the heart of the outline generator is a visitor function that gets invoked for every node. It receives a stack of nodes that are lexically before and above it. It also receives a stack of indexes that tell us where a node lies within the child nodes of its parents. This allows finding predessors and successors of nodes above us. Long story short: Once we find a string like "subscribe", we can check our lexical predecessors for "bespin" and assume the next string node is the name of the event. Voila.

Other improvements:
I added an emulation for importScripts for Safari 4 workers that allows loading external source code from the worker. While this is specced, it is missing from Safari. Because Safari workers are also missing XMLHttpRequest I simply abuse the hosting page to do the requests and post the source back to the worker.
This allows removing the narcissus library from the worker bootstrap script and might soon enable us to remove packaging narcissus with bespin altogether.

And an improvement for people using my worker facade: You can now use console.log to write to the console from inside the worker.

Saturday, March 14, 2009

On-the-fly Syntax Checker and Code Outline View for Bespin

I continued to work on bespin and submitted my first complex patch for review. The patch builds on my work about moving JS objects into web workers and implements syntax checking as well as a simple code outline view for JavaScript. All these features are automatically disabled when you do not have access to web workers because syntax checking is too complex to be performed inside the UI thread.

The outline view is activated by typing the outline command:

The functions names are clickable and move the cursor to the line where the function is defined.

Syntax errors are displayed as short info on the bottom of the screen:

It might be possible to underline the relevant code, but I need to dive deeper into bespin to do this (plus get more info out of the JS parser).

While working on the code I made some discoveries about Gears and Web Workers:
  • Apparently you are not allowed to define a constant called Block inside a Gears worker
  • Safari 4 currently does not support any way to load source code into a worker (both importScripts and XMLHttpRequest are not implemented). For now I pasted all source into a bootstrap-script. Another possible solution would be to send a message to the main page asking it to make the http request and send back the result.
  • I removed the ugly hack to send source to the worker inside the hash part of the url and instead send the source to the worker via postMessage immediately after loading.
I tested the system in Safari 4, FF 3.1 beta and FF 3.0.x with Google Gears. The current patch only supports syntax checking for JavaScript. Other languages could be added in a similar fashion to language-dependend syntax highlighters, though.

Tuesday, March 10, 2009

bespin learns JavaScript

Today I hooked up bespin to Brendan Eich's JavaScript parser written in JavaScript.
Some evil recursive tree wrestling later, we can now find out the names, line and column numbers of all functions in a JS document. I also added some logic, to look up the tree for anonymous functions to see whether they were declared in an object literal, so we can use the key as the inferred name.

The parse tree could be used for all kinds of interesting things, like outlines and even IntelliSense(TM)(R). JS being a very dynamic language this, however, only goes so far. In particular systems like Joose or even dojo's simplistic object system do not go so well with static analysis.

As a next step, I'll move the parser to a web worker (I designed the API to be async, so this should be easy) and implement a simple outline view.

Monday, March 9, 2009

bespin extensibility / auto-indenting

bespin allows you to extend itself with JavaScript. Obviously this JavaScript is edited with bespin itself (I love StrangeLoops(TM)).
The easiest way to do this is to edit the config.js file within the standard BespinSettings project. Here is a sample that enables indenting after line breaks. When you hit enter the next line will have the same leading white space as the current line and when you hit enter after an opening { it will add an extra two spaces (Note that because this is my personal config.js I don't have to worry about configurable tab width :)
bespin.publish("bespin:editor:bindkey", {
key: "ENTER",
action: function (args) {
console.log("Pressed enter")
var editor = bespin.get("editor");
editor.ui.actions.newline(args);
var line = editor.model.getRowArray(args.modelPos.row).join("");
var match = line.match(/^(\s*)/)
var leadingWs = match[1];
var chunk = leadingWs;
var newBlock = line.match(/{\s*$/) ? true : false;

if(newBlock) chunk += " "

args.chunk = chunk;
editor.ui.actions.insertChunk(args)
}
})
Note1: More Info on the bespin custom events
Note2: For this to work you may need to enable the setting 'autoconfig': 'on' first
Note3: Seems like there currently is no easy way to say "In this case ignore me" for an event key event handler which would make it easy to do the right thing in case of an active selection.

Saturday, March 7, 2009

Offloading "arbitrary" JS Objects to Workers / Async drawing for bespin

While doing the experiment to marry Joose and bespin I got sucked directly into bespin development. I started digging into ways to offload part of bespin's painting work into web workers. Web workers are independent JavaScript processes which communicate with the primary page using message events. Because they run in different threads they don't block the UI of the web page and can thus safely do complex calculations without hampering it's responsiveness.

Offloading arbitrary objects into workers
I started with a prototype to offload the work bespin is doing to do syntax highlighting into workers. bespin currently cheats by only looking at the lines which are actually visible (many editors do this). This strategy is fast but might lead to errors when there is not enough information within the currently visible lines.
I'm a lazy person and I had some free time so I decided to build a more general solution which could be used to put all kinds of things into workers. Thus I build a framework that creates a facade for any given JS object that acts like the original object but delegates all work to another objects which actually has all the functions and state of the original object but lives inside a worker. Clear? :)


Turning the whole syntax highlighting framework of bespin into a system that runs in the background is now as easy as this:
this.syntaxModel = new bespin.worker.WorkerFacade(new bespin.syntax.Model());

Of course, there is a little more to it:
There are some important limitations for the objects which can be passed to workers:
  • The objects may not reference DOM nodes (because workers have no access to them)
  • The objects may not have circular references (this restriction could be lifted)
  • And most importantly: None of the functions may be closures. This might seem harsher than it is. It basically means that you need to maintain all state inside the object.
Also, all method calls on the facade will be async. I chose to do a jQuery-style fluid-interface for registering callbacks.

Aside: The Web Worker API
Creating a web worker is as easy as calling new Worker("myWorker.js"). The WorkerPool implementation is Google Gears also included an API to create Workers from a string of JavaScript source. Unfortunately this part did not make it into the official spec. My system needs this feature, so I tried a couple of work arounds. First I tried to put the source into a data URI and use that. I found out that this was even suggested when discussing the spec. Unfortunately at least Safari 4 does not seem to support data URIs for workers. (Should this be reported as a bug to the WebKit team?) The next solution I came up with was to use a small bootstrapping worker that loads the source from the hash-part of its URI. This works well but might have security implications.

Building the Facade
The actual facade object is basically a copy of the original object with all methods exchanged with methods to call into the background worker. The source is quite scary. If you, like me, like source code, please enjoy, otherwise no need to read it:
createFacade: function (obj) {

var facade = this;

for(var prop in obj) {
if(prop.charAt(0) != "_") { // supposedly we dont need "private" methods. Delete if assumption is wrong
(function () { // make a lexical scope
var val = obj[prop];
var method = prop
if(typeof val == "function") { // functions are replaced with code to call the worker
facade[prop] = function () {
var self = this;
var index = CALL_INDEX++ // each call gets a globally unique index
var paras = Array.prototype.slice.call(arguments);
if(this.__hasWorkers__) {
var data = {
callIndex: index,
method: method,
paras: paras
}
if(!USE_GEARS) data = dojo.toJson(data) // here we should really test whether our postMessage supports structured data. Safari 4 does not
// send the method to a worker
this.__getWorker__().postMessage(data)
} else {
// No worker implementation available. Use an async call using
// setTimeout instead
var self = this;
window.setTimeout(function () {
var retVal = self.__obj__[method].apply(self.__obj__, paras);
var callback = self.__callbacks__[index];
delete self.__callbacks__[index]
if(callback) {
callback(retVal)
}
}, 0)
}
// Return an object to create a "fluid-interface" style callback generator
// callback will be applied against context
// callback will be part of the mutex
// paras is an array of extra paras for the callback
return {
and: function (context, mutex, paras, callback) {
var func = callback
if(mutex instanceof bespin.worker.Mutex) {
mutex.start()
func = function () {
callback.apply(this, arguments)
mutex.stop()
}
}

self.__callbacks__[index] = function () {
paras = Array.prototype.slice.call(arguments).concat(paras)
func.apply(context, paras)
}
}
}
}
}
else {
// put instance vars here, too?
}
})()
}
}
},
Inside the source you see this:
this.__getWorker__()
By encapsulating access to the actual worker behind the facade it is possible to put multiple workers behind one object. I currently implemented a simple round robin scheduling, but this could of course be extended to use a more sophisticated mechanism (e.g. picking any currently idle worker) (Beware that multiple workers only work reliably for "stateless" objects).

Gears WorkerPool VS. Web Workers
The Gears WorkerPool API predates the Web Workers API. It is very likely that the Gears API will eventually change to more closely follow the W3C API. For now I implemented a simple facade for the Gears API to make it look like the Web Worker API.
This works quite well with some limitations:
  • DOM Message Events are currently not supported (Support could be added)
  • The current postMessage interface which is used to communicate with workers does not support structured data. My implementation uses Gear's native support for structured data and uses JSON for standard workers
Again here is the source for the facade if your care:
// If there is no Worker API (http://www.whatwg.org/specs/web-workers/current-work/) yet,
// try to build one using Google Gears API
if(typeof Worker == "undefined") {
BespinGearsInitializeGears() // this functions initializes Gears only if we need it
if(window.google && google.gears) {
USE_GEARS = true; // OK, gears is here

var wp = google.gears.factory.create('beta.workerpool');
var workers = {};
Worker = function (uri) { // The worker class
this.isGears = true;

// We can pass the source directly. So we decode the source ourselves.
// To make this more general purpose we could of course also load the
// actual JS file.
var source = uriDecodeSource(uri)

this.id = wp.createWorker(source)
workers[this.id] = this;
}

Worker.prototype = { // we can post messages to the worker
postMessage: function (data) {
wp.sendMessage(data, this.id)
}
}

// upon receiving a message we call our onmessage callback
// DOM-Message-Events are not supported
wp.onmessage = function (a, b, message) {
var worker = workers[message.sender];
var cb = worker.onmessage;
if(cb) {
cb.call(worker, {
data: message.body
})
}
}
}
}

Mutexes / Semaphores / etc.
Once you do real multi-process programming with JavaScript you need to start to worry about some of the issues that come with programming parallel systems (not all of them, because workers are more like processes so there are no issues related to pure multi-threading).
Bespin has a quite complicated paint() method which does the actual drawing of the editing area using a canvas-tag. This paint method calls the syntax highlighter for every line that is draws upon every paint. Because all those calls happen asynchronously, potentially in parallel and in undefined order, I needed a mechanism which basically says: Execute this code right after all those async calls finished.
My current solutions is to have an object which maintains a counter of method calls belonging to the same group. Starting an async method increments the counter, calling the method's callback decrements it. You can schedule yet more callbacks which execute once the counter reaches zero again (when all async method calls are finished). I called the class Mutex which is probably wrong but at least on topic.
Even more source for the interested:
//** {{{ bespin.worker.Mutex }}} **
//
// Object that maintains a counter of running workers/async processes.
// Calling after(callback) schedules a function to be called until all
// async processes are finished
//
// Is Mutex the correct term?
dojo.declare("bespin.worker.Mutex", null, {
constructor: function(name, options) {
this.name = name;
this.count = 0;
this.afterJobs = [];
this.options = options || {}
},
start: function () {
this.count = this.count + 1
},
stop: function () {
this.count = this.count - 1
if(this.count == 0) {
if(this.options.onlyLast) {
var last = this.afterJobs[this.afterJobs.length-1];
if(last) {
last()
}
} else {
for(var i = 0; i < this.afterJobs.length; ++i) {
var job = this.afterJobs[i];
job()
}
}
this.afterJobs = []
}
},
after: function (context, func) {
this.afterJobs.push(function () {
func.call(context)
})
}
})

Conclusion
While my current patch will not make it into bespin (because the current syntax highlighter is so fast that the overhead of the Worker is larger than the win) I still think that the approach above has a lot of potential (e.g. analyzing more lines than the currently visible ones). "Transparently" moving objects into workers while for the most part maintaining the original interface at least gives JavaScript a nice Erlang touch (as @psvensson put it).

Update:
I should clarify that it is no problem for functions to pass the worker-bounday which generate closures. Non of the functions which are already present inside the object which is sent to a worker may include a closure, though.

Wednesday, March 4, 2009

Closures VS. Properties / arguments.callee is expensive

I did some quick benchmarking to test whether it makes a difference to substitute a closure with a function that has a property (sometimes called inside-out-objects). Both are basically the same thing, only that the function property is mutable from the outside while access to the bound lexical variables of the closure is private to the closure.

Here is the source for two simple "getter-makers" that use the different strategies:
var makeGetterProperty = function (name) {
var func = function () {
return this[arguments.callee.__prop_name__]
}
func.__prop_name__ = name;
return func
}

var makeGetterClosure = function (name) {
return function () {
return this[name]
}
}
I timed both execution time of the "make" functions and of the actual getters. Making the getters is almost equally fast in both version; however executing the actual getters differs by a rate of about 150. The closure variant is significantly faster. (All these numbers are only worth their money in FF3.0.x on OSX). Interestingly turning the property variant into a closure that references itself to access the name of the property makes them both almost equally fast.

Conclusion: If you need to build inside-out-objects without closures (e.g. because you need to serialize the function state) that use self-referencing functions instead of accessing arguments.callee.

Monday, March 2, 2009

VisualWorks for JavaScript: Using Bespin to edit Joose components

When I first saw Bespin, the new "code-in-the-cloud" editor that was created by Dion and Ben at Mozilla, and it's multi-pane editor, ideas kept popping up in my head that this might be the ideal platform to complete my ultimate secret masterplan: Recreate VisualWorks for Smalltalk for JavaScript.

In VisualWorks one navigated (some people actually still do this in present tense) the loaded classes and the components of classes like methods and instance variables by progressively clicking through multiple panes. As soon as one reached an item that could be edited in source the edit view switched to that item. Saving after editing an item would directly compile the new source and put it into the running system.

Don't get me started on the debugger, which was simply awesome, and in my opinion no IDE has ever reached its elegance.

Anyway, back to the secret masterplan. Bespin also has a multi-pane view to navigate files which makes it the perfect candidate to act as a platform for VisualWorks for Joose :)

The goals for this platform would be:
  • Allow editing of loaded Joose components
  • Allow creating classes, methods, instance vars, etc. using some kind of UI
  • Saving a change should update the loaded class. One could, thus, edit the loaded application while it is running (This is the most important feature for VisualWorks-feeling)
  • Persisting changes to the server would be nice, too :)
I started work on a prototype. After the first 4 hours of hacking the system allows navigating loaded Joose classes using the multi-pane view and loading method source code into the edit-view:




To make this actually work, there are, however, still many challenges to overcome:
  • The whole persistence part is practially unsolvable with the current state of Joose (However, Nickolay is working on this problem)
  • Bespin is switching to a different "physical" URL to edit a document. This isnt really cool, if you want to be able to edit stuff that is loaded on your current page.
The architecture would look as follows:
  • Joose objects created using the Bespin editor would have an extra property to store the actual source that was used for their creation.
  • One could also create extra properties to store stuff like method level documentation, etc.
  • When "saving" we actually serialize all loaded Joose classes to a text representation. The text representation will probably not look like a classic Joose class but rather we a sequence of statements like MyClass.meta.addMethod("test", function () { return "test" })
  • Loading loads that representation and then uses reflection to show the actual elements.

Friday, February 20, 2009

New Mailing List Joose Commits

To decrease the traffic on our main mailing list, we created a new list only for commits and issue changes: http://groups.google.com/group/joose-commits

Tuesday, February 17, 2009

A CPAN for JavaScript

While there have been attempts to create a CPAN (comprehensive module directory) for JavaScript it has failed just as this has failed for pretty much every language but Perl.

In the last days I've been thinking about a new approach to create such a repository while also solving the problem of high performance JavaScript loading at the same time:
  • Create a central site (such as an application hosted on Google AppEngine) that does the following things:
  • Host any JavaScript file published to it
  • Serve as a reverse proxy server for JS files hosted on other web servers
  • Provide a simple mechanism to associate dependency and versioning meta data with JS files
  • Make any combination of JS files downloadable through a simple URL that serve all data in a single request
This is, of course, only a very rough sketch. The central idea is that sources are never actually installed anywhere but always downloaded on demand when an application requests them. While creating its own set of trust-issues and security holes this would easily solve the installation problem by eliminating it :)

What do you think?

Next Steps for Joose

While Joose 2.0 is happily living in the wild here is a look out to what is up in the days ahead:
  • Nickolay has in a tremendous effort split up the inner workings of Joose into more accesible parts. The basic strategy was to identify those parts which are absolutely neccessary for bootstrapping Joose (make it self-hosting) and then adding features step-by-step in further bootstrapping steps. This will make it easier for other contributors to understand the Joose guts and make their own contribution. The Joose test suite is still passing all tests so their should be no user visible changes.
  • Nickolay then continued to improve the module system making is reentrant and stackable. Modules can now be put into one another, can also be classes and can be loaded asynchronously
  • Based on this work Nickolay then proceeded to create an asynchronous dependency loader for Joose components and now works on making something that might become a central repository for Joose modules.
As a summary: Big thanks to Nickolay :)

Meanwhile me and my team are happily hacking on a large public website that is using Joose to represent models, views and controllers on the client side and my colleagues are using Joose on the server side inside Demandware (which is in turn using Rhino) to build web shops.

Sunday, February 1, 2009

Joose 2.0 "death of parseInt" released

The Joose team is very proud to announce the release of Joose 2.0.

Joose is a self-hosting meta object system for JavaScript with support for classes, inheritance, mixins, traits, method modifiers and more.

Joose makes object-oriented programming with JavaScript easy, declarative and very productive. The Joose meta-object system is multi-paradigm. It supports class-based and prototype-based programming styles as well as class-based inheritance and role-based extention.

The Joose framework has been successfully used in multiple production systems for twelve months now and has been proven to be very stable. Joose is being tested using an automated unit-test suite that is being run in all major browsers (Firefox, IE, Safari, Opera and Chrome).

To get started with Joose we recommend to read the getting started guide.

Major News

Support for types and type coercions in attributes and method signatures
JavaScript purists might find this a little disturbing but Joose now supports type declaration. Joose types, however, are quite the opposite of static types, because they try very hard to convert everything you throw at them to the type you want. parseInt is dead!
Here is the documentation.

News

  • Builtin support for turning a class into a singleton
  • Support for Rhino without need for special configuration
  • The default namespace for types (which used to be experimental) is now Joose.Type
  • Experimental support for multi method dispatch. Examples.
  • Made integrating Joose with other class builders easier.
  • Fixed type checks for arrays (thanks Silvan).
  • Fixed an issue when overriding getters and setters defined in a super class.
  • Improved error messages when defining properties of classes, etc. but passing null to the property (happens easily when you have a typo in the last part of a name spaced identifier).
Other News
Download
Compatability
Joose has been carefully tested to play well with other JavaScript frameworks like jQuery, Prototype and YUI and with both browser based and stand-alone/server-sided JavaScript engines. See this page or a full list.

Have fun playing with Joose.

Monday, January 26, 2009

Joose 2.0 Call for Testers

Hey,

we are in the final phases of prerating the next release of Joose. Please click on this link to the test suite and report any cases of green or especially non-green output.

Notice that the test suite will prompt you for Google Gears authorization if you have Gears installed. It is, however, not neccessary to run the tests.

Thank You!

Positive Results so far:
  • FF 3.05 OSX
  • Safari 3 OSX
  • IE8 beta Windows XP
  • FF 2 Windows XP
  • Chrome Windows XP
  • Opera 9.62

Sunday, January 18, 2009

Multi-method dispatch for JavaScript

Once again, reporting live from the because-you-can department:
Jeremy has created an initial implementation of multi-method dispatch in Joose. It allows you to define a set of methods in a class that have the same name but differ with respect to the signature of the argument types. Once you call such a method-name, the dispatcher will look for the most appropriate method (based on the runtime types of the parameters (Are they still parameters or are they already arguments at this stage? (I think still parameters))).
Here is a link to the raw test-suite, that is currently passing.

The implementation hasn't been hooked up to a class-builder syntax and probably needs a lot of edge case work and performance tuning. However, I think it is a great example of what can be done with JS and I'm eager to try it out in practice to check whether it gives value to every-day JavaScript programming.

Typed Method Signatures in Joose/JavaScript (parseInt never again)

Typed method signatures have landed in Joose. This is what you get from the new feature:
  • You can define type constraints, classes and roles that a method parameter must match
  • If the type checks fail there is a runtime exception
  • If you enable coercions, Joose tries it's best to convert the parameter types to the one you want!
That is why I call type coercions the death of parseInt (and friends). JavaScript client side programming deals with a lot of user provided string data that needs to be validated and converted to domain values. With type coercions, once you have defined your types (or the use the built-in types), input validation and conversion is completely transparent to your code.

Lets consider this example: If you want to write code that adds the number 3 to the value of this input field: (Just an input-field with a 2 inside (for the feed readers))

To make this reliable, you need to
  • check whether the value of the input field is actually something that can be added to (in a numerical sense)
  • convert the value from a string to a number
This is where Joose type coercions can help you. Here is the code doing all that using type coercions:
    // Make a class thats adds to its instance variable "amount"
Class("Adder", {
has: {
amount: { is: "rw" }
},
methods: {
// add is a method with a signature
add: {
signature: [Joose.Type.Int], // the first parameter must be an integer
method: function add (num) {
return num + this.amount // add the argument num to our instance var amount
},
coerce: true // enable coercions for arguments
}
}
})

// Make an adder that always adds 3
var addTo3 = new Adder({ amount: 3 })

// add 3 to the value of the input field
alert(addTo3.add(form.addMe.value));
If you would run the above code without type constraints and coercions the result of the method would actually be "23". The code above, however, does the right thing and alerts 5.
This is obviously a lot of code for such a trivial use case, but once things start to become more complicated you save a lot of typing, produce code that might have less bugs and document your parameters a little more thoroughly. The extra benefit of the Joose built in types is that we already wrote unit tests for them (allthough there is always room for more) so you don't have to.
I'm aware that a lot of JavaScript purist will find this totally over the top, but as we go into writing ever more complex JS applications things like this have their place. I, personally, find it great that I get extra security while being lazy at the same time.

Thanks again for the great input from the discussion on the best builder syntax. The current syntax is not final, however. I chose it, because
  • it fits well into Joose
  • the types are close to the actual signatures
It is not ideal, because
  • unless, you repeat (which is bad) the method name in the function declaration, your methods will be called "method" in firebug and similar tools.
  • is is pretty verbose
New ideas that would eliminate these short comings are very welcome.
You might have noticed, that coercions needs to be enabled by opt-in. This is due to the fact, that coercions come with a pretty heavy performance penalty. It is absolutely no problem in a use case where you have a "front-end" function to parse user input (DOM access is magnitudes slower) but might be too big for your canvas ray tracer.

Tuesday, January 13, 2009

ToDo List for Joose 2.0

  • Finish Types implementation (stricter RegExes, more tests)
  • Implement experimental support for typed method parameters
  • Finish Dojo bridge in addition to the Ext and PureMVC helpers (not really neccessary but would be nice)
  • Make the traits implementation more spec compliant (Based on Joseph Hurst's work on JSTraits)
Meanwhile Nikolay is working in a branch to refactor Joose's internals to make them easier comprehensible. This work, however, won't be included into the next release.

Any wishes for the next release?

Sunday, January 11, 2009

A PureMVC/Joose example

I changed the strategy a little with the Joose2PureMVC bridge. It no longer "joosifies" all classes of the framework but rather allows subclassing classes of the PureMVC framework using Joose. The reason is that I had to use too much dark magic to make the it work and it proved not be very stable.

If you care: The dark magic involved getting the meta data of the super class in a setTimeout call because the Objs.js system used in the PureMVC code changes prototypes after class declaration.

To try it out you can download I converted some code of an example app from the PureMVC website to use the Joose bridge. You can download it here. I joosified a single class that you can see at the very end of the file bin/employee_admin_demo.js.

The object system used by the PureMVC JS port requires to write a lot of boilerplate code for inheritance and calling the constructors of the super classes. This is all eliminated with Joose.

If you don't want to download the example app, here is the source code of the Joose class:
Module("org.puremvc.js.demos.js.employeeadmin.view", function () {

var EventS = Objs.load("net.tekool.events.EventS");
var Relegate = Objs.load("net.tekool.utils.Relegate");
var IMediator = Objs.load("org.puremvc.js.interfaces.IMediator");
var INotification = Objs.load("org.puremvc.js.interfaces.INotification");
var Mediator = Objs.load("org.puremvc.js.patterns.mediator.Mediator");
var Notification = Objs.load("org.puremvc.js.patterns.observer.Notification");
var ApplicationFacade = Objs.load("org.puremvc.js.demos.js.employeeadmin.ApplicationFacade");
var UserVO = Objs.load("org.puremvc.js.demos.js.employeeadmin.model.vo.UserVO");
var UserProxy = Objs.load("org.puremvc.js.demos.js.employeeadmin.model.UserProxy");
var UserList = Objs.load("org.puremvc.js.demos.js.employeeadmin.view.components.UserList");

Class("UserListMediator", {
meta: PureMVC,
isa: Mediator,

after: {
initialize: function () {
var userList = this._getUserList();
userList.addEventListener( UserList.NEW, Relegate.create(this,this._onNew) );
userList.addEventListener( UserList.DELETE, Relegate.create(this,this._onDelete) );
userList.addEventListener( UserList.SELECT, Relegate.create(this,this._onSelect) );
var userProxy = this._facade.retrieveProxy( UserProxy.NAME );
userList.setUsers(userProxy.getUsers());
}
},

methods: {
_getUserList: function() {
return this._viewComponent;
},
_onNew: function( event ) {
var user = new UserVO();
this.sendNotification( ApplicationFacade.NEW_USER, user );
},
_onDelete: function( event ) {
var selectedUser = this._getUserList().selectedUser ;
if(selectedUser == null)
return;
this.sendNotification
(
ApplicationFacade.DELETE_USER,
this._getUserList().selectedUser
);
},
_onSelect: function( event ) {
this.sendNotification
(
ApplicationFacade.USER_SELECTED,
this._getUserList().selectedUser
)
},
listNotificationInterests: function() {
return [
ApplicationFacade.CANCEL_SELECTED,
ApplicationFacade.USER_UPDATED,
ApplicationFacade.USER_ADDED,
ApplicationFacade.USER_DELETED
];
},
handleNotification: function( note ) {
var userList = this._getUserList();
var userProxy = this._facade.retrieveProxy( UserProxy.NAME );
switch( note.getName() )
{
case ApplicationFacade.CANCEL_SELECTED:
userList.deSelect();
break;
case ApplicationFacade.USER_UPDATED:
userList.setUsers( userProxy.getUsers() );
userList.deSelect();
break;
case ApplicationFacade.USER_ADDED:
userList.setUsers( userProxy.getUsers() );
userList.deSelect();
break;
case ApplicationFacade.USER_DELETED:
userList.setUsers( userProxy.getUsers() );
userList.deSelect();
break;
}
}
}
})
});

Thursday, January 8, 2009

Joosifying PureMVC

Based on Nickolay's work for Ext I implemented a Joose to PureMVC bridge. PureMVC is a MVC framework that was originally implemented for Flash/ActionScript3 but which has implementations for JavaScript, Java, C# and other languages.

Using the new Joose bridge you can use Joose to build PureMVC applications. You can use Joose's facilities to inherit from the PureMVC base classes and apply Joose-roles to all instances and classes included in PureMVC. The code works on the sample application. There seems to be no test suite in the main repository, so it cant be fully verified.

The bridge code is actually very simple. PureMVC uses a simple JavaScript class framework called objs.js. This framework provides a method called "register" to register classes and a method called "extend" to inherit from a super class. The bridge code overrides these methods. Thus automatically all classes from PureMVC can be turned into Joose classes without any further changes to the actual code of the framework.

You can download the code from the Joose repository. Just replace the file objs.js from the PureMVC download with the file from Joose and you are good to go.
The code was only minimally tested but seems to work. If there is further interest or as soon as I have a work project that uses the framework I'll invest further time in it.

Tuesday, January 6, 2009

Suggestions for method signature syntax

There have been some great suggestions for Joose's syntax for method signatures. More comments would be great.

Work to release Joose 2.0 is under way. Might happen next weekend.

Sunday, January 4, 2009

Type constraints for method parameters (and return values)

As an add on to type constraints for instance variables (see last post) we think about adding them to method parameters and maybe even method return values. The hard part here is not the implementation but rather to find a nice syntax to declare these constraints. Here are a couple examples of possible definitions for typed method signatures:
    Class("TypedMethodParas", {
methods: {
typed1: [
[Joose.Type.Int, Joose.Type.Date],
function (int, date) {
// ...
}
],

typed2: {
takes: [Joose.Type.Int, Joose.Type.Date],
does: function (int, date) {
// ...
},
returns: Joose.Type.Int
},

typed3: Method(Joose.Type.Int, Joose.Type.Date, function (int, date) {
//
},
Joose.Type.Int)
}
})

Which one do you like best? Do you have better ideas? Is this feature even worthwhile?

Go to Comments

Type Contraints and Type Coercions in Joose

There is a new feature in the upcoming Joose 2.0 that is called type constraints that has great potential of changing the nature of many JavaScript applications:

Joose type constraints are only somewhat related to types in the traditional sense of computer science. They allow the programmer to define a set of contraints which a value that is stored in a variable will have to fulfil. As they apply to values rather than containers these checks are performed at runtime rather than compile time.

In this class definition the attribute "anAttribute" is constrained to the type Joose.Type.MySmallNumber:
Class('MyClass', {
has: {
anAttribute: {
is: 'rw', // TypeConstraints only work for rw attributes
isa: Joose.Type.MySmallNumber, // use the MySmallNumber constraint
coerce: true // turn on coercions for this attribute
}
}
})
The definition of the type itself looks like this:
Type('MySmallNumber', {
uses: Joose.Type.Int, // this is a specilization of the Joose.Type.Int Class
where: function (value) {
if ( value > 1 && value < 5 ) { // we only accept Ints between 1 and 5
return true;
}
return false;
}
});
This type specializes the built-in type Joose.Type.Int which only accepts integers. Only values greater than 1 and smaller than 5 are accepted. If you assign values to instance variables which do not match this constraint, the program will throw an exception (As an alternative you may also add an onError callback as a second parameter to a setter-Method to handle the exception).

Type Coercions
The real magic starts when we add in so called type coercions. Type coercions are rules how values can be tranformed from one type to another (kind of like casting). Here is an example of the above type with one coercion rule:
Type('MySmallNumber', {
uses: Joose.Type.Int, // this is a specilization of the Joose.Type.Int Class
where: function (value) {
if ( value > 1 && value < 5 ) { // we only accept Ints between 1 and 5
return true;
}
return false;
},
coerce: [{ //optional coercion definition list
from: Joose.Type.Str, //coercion from string to number
via: function(str) {
return new Number(Str); //perform our coercion
}
}]

});
This coercion transforms values that have the built-in type Joose.Type.Str to integers.

Joose provides a set of built-in types for your convenience. These are organized in a hierarchy. Things that are further down the hierarchy need to match all the contraints of parent-types.

Joose type constraint hierarchy
This diagram has been created with the Joose sample app blok.


Conclusion
Type coercions have been proven to be extremely useful in JavaScript code because many tasks in JavaScript deal with user provided data that is in string form. Coercions provide a very convenient way to transform this string data into meaningful things. After the definition of the types your application code can basically "ignore" things like data validation and conversion. This makes the code more expressive and should reduce errors because it is harder to forget to validate data.

Further topics
Defining inline types
Localization
Type constraint API

This post serves as a draft for the future documentation for Joose's type contraint system and will then move to the Joose wiki page on type constraints.
If you want to play with the features, you will have to download the Joose 2.0 release candidate.

Friday, January 2, 2009

Aspect Oriented Programming with Roles/Traits and Joose

Joose is different from other JavaScript frameworks because it focuses on only providing tools for making the programming with JavaScript for robust and productive. There are no facilities for DOM access or AJAX. Joose is thus complementary to using a more traditional Ajax library and also works nicely in non-browser environments like Rhino.

Many JavaScript libraries with the notable Exception of jQuery provide some helper methods to make things like class or prototype inheritance a more pleasant experience. Joose's capabilities, however, go way beyond these basic features. My personal favorite feature in Joose is the combination of Roles (sometimes also called Traits) and method modifiers.

Here is the definition from the Traits research page:
Traits are a simple composition mechanism for structuring object-oriented programs. A Trait is essentially a parameterized set of methods; it serves as a behavioral building block for classes and is the primitive unit of code reuse. With Traits, classes are still organized in a single inheritance hierarchy, but they can make use of Traits to specify the incremental difference in behavior with respect to their superclasses.
In Joose creating a role (aka trait) is as easy as writing
Role("Eq", {
requires: "equalTo",

methods: {
notEqualTo: function (other) {
return !this.equalTo(other)
}
}
})
This role can be applied to things that implement the method equalTo and gives them the method notEqualTo. Things here means classes and (because this is JavaScript!) objects.
Applying to a class looks like this:
Class("TestClass", {
does: Eq
}
Applying to an object is a one liner:
Eq.meta.apply(myObject)
So much for simple traits. Where things get really interesting is when the concept of roles is combined with the concept of method modifiers (I call them M&Ms). M&Ms allow augmenting methods with extra functionality by doing stuff before or after a method is executed (The full list of modifiers is before, after, around, override and augment).
In this example Test derives from SuperClass and modifies SuperClass's add method with a before-modifier:
Class("Test") {
isa: SuperClass,
before: {
add: function () { this.result += "3" }
}
}
Now it is also possible to define method modifiers in roles. When the role is applied the method modifiers are applied to the methods of the target class or object.
From here on I'll use real world examples from the Joose example app blok. Things placed on blok's viewport are instances of the class Shape. Shapes can be resized, dragged, styled, edited, etc. All of these "behaviors" are implemented as roles that are applied to shapes.
A shape keeps a jQuery object in it's instance variable called $ that represents the DOM node of the actual shape-html on the viewport. Whenever the DOM node receives or looses focus the respective methods "focus" and "blur" are called on the shape object. The method "place" is called to place a shape on the view port. One of the roles that can be applied to shapes is the Focusable role. This role gives a shape the ability to actually receive a visual recognizable focus (aka an outline). Here is the implementation of the Focusable role:
Module("block.ui.role", function () {
Role("Focusable", {
after: {
place: function () {
var me = this;
this.$.mousedown(function focusClick (e) {
e.preventDefault()
document.manager.switchFocus(me, e.shiftKey)
return false;
})
},

focus: function () {
this.$.append('<div class="focusDiv"></div>')
},

blur: function () {
this.$.find(".focusDiv").remove()
}
}
})
})
This particular role augments the methods "place", "focus", and "blur". When the shape that uses this role is placed on the view port a mousedown event is added to the jQuery object. This event is than delegated to the global document.manager singleton-object that manages focus on the page (This object later calls the focus and blur methods on the shape objects. This is different from the browser's understanding of focus, because shapes may keep focus while a control elsewhere is used). When the shape receives focus an extra div is added to the jQuery object (this is not done using only css classes for stupid technical reasons). When the focus is lost, the div is removed again.

The examples should demonstrate that roles are great ways to assemble "reusable units of behavior" (as the orginal authors put it). Using method modifiers further reduces conflict potential between multiple roles because an unlimited number of M&Ms can be applied to the same method. Thus other roles like Resizable, Draggable, Connectable or Editable can be safely added to the same shape.

I particularly like the pattern outlined above because it feels really JavaScript-y. The use of roles makes the pattern superior to simple aspect oriented techniques using only method modifiers because multiple related modifiers can be applied in a single stepped.

I hope this post was interesting, now go play with Joose :)

PS: Thanks to Moose for comming up with all these concepts so that I could copy them for JavaScript.

Earlier post on the same subject.

Thursday, January 1, 2009

Comet or no comet?

People seem to be quite emotional whether UniversalComet deserves the word comet in the name, so here is some background on the technical details.
The UniversalComet API described in the last two posts uses a very simple/naive technique called polling to achieve the effect of near instant delivery of messages. The reasons for this kind of implementation are very simple and related to the fact that the application is implemented on top of Google App Engine (GAE):
  • GAE only allows a single unified response for HTTP requests. This means that techniques involving multi-part responses with multiple script-tags do not work.
  • The GAE quota system puts severe penalties on calls to sleep. Thus it is not really possible to implement a system where you would minimize the number of HTTP requests by waiting on the server until a reponse is available (even very short delays within 1 second are not accepted). I'm actually in touch with Google about this topic and there is hope that this limitation might be removed.
  • There is also a related issue on the GAE issue tracker.
Thus it made sense to use the simple polling implementation. On the other hand, it is true, that Google App Engine is currently not the right platform for implementing scalable comet solution (Although it deals quite well with the current traffic amount). One would have to look for custom hosting or a more liberal services like EC2 instead. However, App Engine is great to deploy just-for-fun proof-of-concept applications without having to worry all the hard stuff behind the scenes.

Anyway, the motivation behind UniversalComet was not really to explore implementation techniques for comet but rather an experiment to provide a very convenient way to give an URL to visitors of a website through which they can be contacted in near realtime by means of simple HTTP requests which I still think is a very interesting idea.