Wednesday, April 1, 2009

A structured outline view for bespin

After some major parse tree wrestling I finished an experimental structured outline view for JavaScript files in bespin. It's primary feature is that it can be modified at runtime to adapt to your favorite dialect of JavaScript.

While most programming languages have fix constructs like class, namespace or even event declarations, with JavaScript you have to build all these things from simple building blocks. On the other hand, real-world JS code usually follows quite strict rules to build these things. E.g. bespin itself uses dojo to declare "classes" (or constructors, whatever you want to call them) like this:
dojo.declare("bespin.parser.CodeInfo", null, {})
and this pattern to subscribe to and publish events:
bespin.subscribe("parser:error", function(error) {
bespin.publish("message", {
msg: 'Syntax error: ' + error.message + ' on line ' + error.row,
tag: 'autohide'
With my personal favorite object framework Joose you will find code like this:
Module("Bank", function (m) {
Class("Account", {})
While a programmer, who knows nothing about JavaScript will easily recognize the meaning of these statements, JavaScript has to actually execute them to derive the meaning.

My new outline generator works around this problem by being configurable through a kind of DSL which tells the parser more about the specific dialect of JavaScript that you are speaking. Here is the code to tell the parser about the specific pattern that bespin uses to publish and subscribe to events:

bespinEventPublish: {
declaration: "bespin.publish",
description: "Publish"
bespinEventSubscription: {
declaration: "bespin.subscribe",
description: "Subscribe to"

How it works:
At the heart of the outline generator is a visitor function that gets invoked for every node. It receives a stack of nodes that are lexically before and above it. It also receives a stack of indexes that tell us where a node lies within the child nodes of its parents. This allows finding predessors and successors of nodes above us. Long story short: Once we find a string like "subscribe", we can check our lexical predecessors for "bespin" and assume the next string node is the name of the event. Voila.

Other improvements:
I added an emulation for importScripts for Safari 4 workers that allows loading external source code from the worker. While this is specced, it is missing from Safari. Because Safari workers are also missing XMLHttpRequest I simply abuse the hosting page to do the requests and post the source back to the worker.
This allows removing the narcissus library from the worker bootstrap script and might soon enable us to remove packaging narcissus with bespin altogether.

And an improvement for people using my worker facade: You can now use console.log to write to the console from inside the worker.

No comments: