Ocsigen: OCaml based web framework

Ocsigen is a web framework written in OCaml.

In particular, some recent research papers have shown that functional programming works very well for this domain (use of continuations or closures, continuation passing style ...). Accordingly, Ocsigen considers each URL as a function taking arguments and producing a web page. Clicking on a link or a form triggers execution of this function.

Another significant issue addressed by Ocsigen is the generation of valid xhtml pages. The web relies on norms edicted by the W3 consortium, and the only means to ensure universal availability of information is to respect these standards. Today, very few web sites are fully standard compliant. Ocsigen uses an advanced type system that guarantees that your web site will be valid (or very close).

Link from Phil Wadlers blog.

Categories: ocaml, web

Little Smalltalk and Javascript for Minix

I've been working on porting some small programs to Minix to get familiar with what's required for ports. From there I hope to get some bigger stuff working that I really want to use like Factor and Erlang.

So far I have Little Smalltalk working (including the web based class browser and editor) and SpiderMonkey Javascript.

Porting is a little bit painful as there's quite a few things different under Minix. Eventually I want to try and develop some simple device drivers using these languages to see if the idea of user space device drivers in alternate languages is workable.

Categories: factor, smalltalk, javascript, erlang, minix

HTML Template Languages

There's a discussion going on in the Erlang mailing list about HTML template engines. This is JSP-style definition of HTML pages with portions replaced dynamically with data from an application.

I've leaned towards building HTML pages in code using Lisp style S-expressions for HTML, or Seaside style building of pages. This is also the approach I took in building the Factor continuation based web server and examples.

For the recent Factor program I built I wanted to use a template engine though. It was a bigger program and it was easier to modify templates and see the change automatically. For that I used an engine that another contributer had provided which allowed embedded Factor code.

The mailing list post pointed to a paper called 'Enforcing Strict Model-View Separation in Template Engines' which describes limiting the power of a template engine to prevent developers from embedding to much application logic in the template page. I was dubious but the paper is quite convincing.

It mentions 'StringTemplate', an engine built on these ideas. I'm still looking for a good way of doing templates in Factor, Erlang, and other languages, so this might be a good source of ideas.

If anyone has pointers to other articles or ideas on the topics please feel free to leave a comment or email me.

Categories: factor, erlang

Real-time Wiki

From the Erlang mailing list comes a pointer to 'Real-time Wiki'.

It's a Google Summer of Code project that allows using XMPP (ie. Jabber) from a browser based client. Values can be changed from the server and the change is broadcast to the clients and instantly updated. It seems that the application for this is a Wiki that updates automatically when others change it.

The implementation on the server side uses ejabberd, a Jabber server written in Erlang.

Categories: javascript, erlang

Playing around with Minix

I like the idea of writing device drivers and operating system components in other languages, as mentioned in a previous post about Haskell and Minix.

I installed Minix 3 and ran it under qemu to try it out. It's very functional but doesn't have much in the way of software or 'nice to have' features yet. But the source is all there and its very easy to build and follow.

I went looking for the book describing the internals of Minix, Operating Systems, Design and Implementation, and it's about $105 US online. Ouch. I tried a few Wellington bookstores and Capital Books had a copy for $110 NZ. They're probably the only bookstore in Wellington that was even likely to have a copy as they are specialists in technical books.

Now the task is to build/port some of the language implementations under Minix and try a simple driver. As I know very little about Minix I'm documenting the user aspects (setting it up, porting software, etc) in a weblog I setup for that purpose.

The interesting thing about that weblog is I'm hosting it on a Minix 3 system, running 'httpd' which comes with Minix. It's also running under 'qemu' on the same user mode linux machine as this weblog. There are some issues with doing that but it'll be interesting to see how it works - and it gives me a network facing Minix machine to experiment with. I suspect, due to the multiple levels of indirection, it's unlikely to handle load too well but we'll see.

Categories: minix

Unenterprisey languages going strong

The Unenterprisey Languages Meeting held in Wellington yesterday went very well. The turnout was good with about 16 people there and the talks by Jonathan, Robert and Geoff were great. It's good to see and hear about languages like Io, Common Lisp and Erlang being used.

Jonathan talked about Io's prototype based OO system and how inheritance works in it. He went through an overview of Io itself, how the inheritance works, and how he used ideas gained from Io in a project he worked on recently. His project used a domain specific language he developed that used the inheritance ideas of Io to help manage the project. It looked pretty cool.

Geoff talked about Erlang, going through its history, syntax, reliability, and concurrency system. It was good to see him talk about the OTP system as I wasn't very familiar with that and his demo showed sending messages across processes with dynamic failover of nodes. When the primary node went down the secondary node took over. When it came back up again it again took over. Very neat!

Robert Strandh talked about GSharp, his musical score editor written in Common Lisp. It uses CLIM for the GUI and looks very nice. The focus of the talk was how he embedded a domain specific language into GSharp to handle fonts better. Specifically the language was modelled after MetaFont, but embedded as a DSL in Common Lisp as a macro. This would then get compiled as PostScript bezier paths. The macro itself was a nice example of intentional variable capture (ie. It was intentionally a non-hygienic macro) which was something I'd not seen in a 'real world' usage before.

Robert also showed another CLIM application he wrote to preview the fonts from the DSL. In 172 lines of Common Lisp code it had an impressive amount of functionality. CLIM looks to be a very powerful framework.

My talk was about Javascript. But Javascript used slightly differently than normal. I gave a brief overview of Javascript and its prototype based OO system. I then talked about using Javascript on the server as well as the client, and about the possibilities of having the same code run on both the server and client for things like field validation. This way one copy of the code can validate input fields in client side Javascript and when posted to the server the same code can also check that the server side POST was valid.

To demonstrate running the same code on the client and server I used Wiky, a nifty Javascript bidirectional markup converter. I showed the Wikybox running on the client, and demonstrated loading the exact same Javascript in Rhino and doing the same conversions on the server. The idea being you can have the markup converted to HTML for a preview on the client, send the markup to the server when saving, and use the same code on the server to store the HTML.

From there I demonstrated using Server Side Javascript to dynamically modify a running web server, adding servlets on the fly in Javascript.

I went on to talking about continuations and Javascript. For client side continuations I showed some Narrative Javascript examples and went through the lightweight threading and process communication examples I posted about previously.

Finally I finished with server side continuations in Javascript showing the thread migration example.

After the talks we all went out for a great meal at a local restaurant and ate, drank, and talked for most of the night.

Overall I'd say the alternative language interest and usage in New Zealand is definitely looking good. Thanks to everyone who attended and those that organised it - very worthwhile!

Categories: misc

Migrating Javascript Threads

A while ago in my old weblog I posted some Sisc Scheme code to demonstrate a thread migrating from one Sisc session to another. Since Rhino, the Java based Javascript implementation, has serializable continuations I decided to try porting it to Javascript.

The code demonstrating it is in migrate.js. Loading this into a Rhino session provides a function called 'messageListener' that starts up a socket server that listens for incoming objects on a port. The object should be a function or continuation. When received it's called on a seperate thread.

The 'send' function will send a function to a running message listener. This can be on the same machine or across the network. 'migrateThread' will migrate the currently running thread to another message listener, again this could be on another machine across the network. For example:

Machine A:
js> load("migrate.js")
js> messageListener(10000)

Machine B:
js> load("migrate.js")
js> messageListener(20000)
js> send("host.running.A", 10000, function() { print("Hello A!"); });

This will result in the message 'Hello A!" appearing on machine A. The function is serialised, sent across the network and deserialized and called on A. An example function that does thread migration:

function pingPong(host1, port1, host2, port2) {
  while(1) {
    print("ping\n");
    java.lang.Thread.sleep(1000);
    migrateThread(host2, port2);
    print("pong\n");
    java.lang.Thread.sleep(1000);
    migrateThread(host1, port1);
  }
}

If run on Machine A, specifying the host and port for machines A and B it will first print 'ping', then sleep for 1,000 milliseconds. Then the thread will migrate to machine B, print 'pong', sleep and migrate back to A, etc.

The code has a workaround for the fact that Rhino doesn't deserialize and serialize to a stream. Instead the functions only work on files. So I serialize to a file first, then read that file and write it to the stream. For deserialisation it reads from the stream, writes it to a file then deserialises from that file. For real world code this should be changed to actually operate on the stream itself, or use random temporary files and clean them up afterwards.

To run the code you'll need to use both js.jar (The Rhino implementation) and xbean.jar (for serializing some of the Javascript objects). An easy way of just getting these to try it out is to use the jars from the Server Side Javascript example and run it with:

java -cp js.jar:xbean.jar org.mozilla.javascript.tools.shell.Main

For Windows, replace the ':' with a ';'.

This was one of the things I talked about and demonstrated at the Unenterprisey Languages Meeting in Wellington on Saturday.

Categories: javascript

Termite Distributed Scheme Released

Termite, the Scheme-based system with Erlang style concurrency, has been released.

The Termite paper (pdf) outlines the features of Termite. I've been looking forward to trying this out!

Categories: scheme

Minix and Haskell

There's an interesting post on the Haskell group about Minix and the possibility of writing OS core components in Haskell for it.

Minix is an open source operating system that's been around for quite some time. I first tried it out sometime in the early 90's after downloading over some very slow dialup connection connected to a BBS.

From the post:

Andrew Tanenbaum said that it is important for a language like OCaml or Haskell to get more visibility among the OS developers; and Minix offers a very good way to do that. Minix3 is based on micro-kernel; all of the OS services (memory manager, file system, all the drivers, etc.) run as regular processes communicating through a well-defined protocol. It is irreleveant what language these services are written in, so long as they obey the protocol.

The poster goes on to suggest a way of slowly migrating the system routines to Haskell:

With Minix, we can replace one kernel service of the full-fledged, working OS with the one written in Haskell -- and immediately see how it all works. If it doesn't, the rest of OS still works and so we can unload the faulty service and load another implementation. Minix3 does indeed offer an easy road to the OS kernel for a language other than C. I said that there may be quite a bit of interest in Haskell community in such a project. I wonder if I'm right...

Categories: haskell

Mt Vic Dojo Open Day - Kyokushin Karate, BJJ, Judo and Yoga

The Mt Vic Dojo is having an open day on Sunday 11th June. The day includes demonstrations, free classes, t-shirts and discounts to be won. All the styles that hve their home at the Mt Vic Dojo will be there. These being:

• Kyokushin Karate
• Brazilian Jiu Jitsu
• Judo
• Yoga and Yoga Bugs

A poster with details is here (very high quality pdf).

If you're at all curious about any of these great activities go along and have a look. It'll be well worth your while. The dojo is at Level 2, 25 Home St, Wellington, behind City Nissan, next door to Spec-R .

Categories: martialarts

Unenterprisey Languages meeting

There's an 'Unenterprisey Languages meeting' at 4pm this Saturday (the 10th of June) in Wellington, New Zealand. It's being held in the L2 boardroom at Catalyst, Level 2, 150 Willis St.

Speakers so far are:

• Robert Strandh, who will probably be talking about Common Lisp and the G# score editor
• Jonathan Wright who will be talking about the Io programming language.
• Geoff Cant, who will be talking about Erlang and maybe Sisc scheme.
• And me, but I have no idea what I'm going to talk about just yet.

Last time there was a gathering like this (A Lisp users gathering) I talked about Aviarc, the web development product I work on for my day job, and its underlying continuation based web server written in Scheme. I haven't decided what to talk about this time...and I'm running out of time!

Categories: commonlisp, io, scheme, erlang

Phobos available for download

Phobos appears to be available for download now.

Phobos is the server side javascript system that Sun is making available. Robert Chinnici has a blog outlining features of Phobos and answering the question about why you'd want to use Javascript on the server.

Categories: javascript, web

Javascript 1.7

It looks like Javascript 1.7 is getting some interesting new features. Brendan Eich did a presentation about what is being added and changed.

One that looks useful for the concurrency things I'm playing with is destructuring assignment:

var [a, b, c] = [ 'one', 'two', 'three' ];

This would make breaking apart messages received from processes a bit easier.

Interestingly Javascript 1.7 has Python style generators. These have been used in Python to build lightweight threads so it looks like it'll be possible to do the threading described in my previous posts natively one day.

This document describes how to build pre-release versions of the Javascript shell to play with this.

Categories: javascript

More concurrency in Narrative Javascript

[ 2006-06-05: Updated the code to use 'concede' instead of 'yield' to prevent clashes with the new keyword in Javascript 1.7 ]

I've added to my lightweight threading example in Narrative Javascript to include support for sending and receiving messages between processes. The style of communication is based on the Erlang style of concurrency whereby processes communicate by sending messages. The threads2 example is here.

This page has two processes running on it. They start when the page loads. Both processes block immediately waiting for a message. When the message is received the append the text of the message to a textarea on the page:

function t1() {
  while(true) {
    var msg = receive->();
    var dest = document.getElementById("p2");
    dest.value += (msg +"\n");
  }
}

The 'receive' call blocks immediately if there are no messages in the processes mailbox. If a message arrives it unblocks and continues. It must be called as an asynchronous operation using the '->' operator for this to work. Processes are started using 'spawn':

var process1 = spawn(t1);
var process2 = spawn(t2);

Messages can be sent from one process to another using 'send'. This is also an asynchroous operation and must use the '->' operator. Here's the code that is run when a button is pressed:

function send1() {
  spawn(function() {
    var msg = document.getElementById('msg1').value;
    send->(process1, msg);
  });
}

It spawns a process which obtains the text from the text input area and sends it to one of the processes. This will cause that process to unblock and continue from the 'receive' call.

The updated code is:

• schedule.njs
• thread2.njs
• thread2.html

Some ideas for things that would be nice to add or build upon:

• Provide the equivalent of Termite's '!?' operator to allow sending a message and waiting for a reply to that message.
• Use a persistent connection (ie. Comet) to an Erlang server and allow messages to be sent back and forth from Erlang processes and Narrative Javascript processes.
• Add futures and promises allowing dataflow style programming.

Note that this code is just 'playing around' code and is not intended to be robust or production ready. I'm just exploring how these ideas that have traditionally been used on the server work in a client side browser based language.

Categories: javascript

Lightweight threads with Narrative Javascript

[ 2006-06-05: Updated the code to use 'concede' instead of 'yield' to prevent clashes with the new keyword in Javascript 1.7 ]

I've put together a very simple lightweight threads example with Narrative Javascript. The demo creates two threads:

function t1() {
  var element = document.getElementById("one");
  var count = 0;
  while(true) {
    sleep->(100);
    element.innerHTML=count++;    
    concede->();
  }
}

function t2() {
  var element = document.getElementById("two");
  var count = 100;
  while(true) {
    sleep->(100);
    element.innerHTML=count--;    
    concede->();
  }
}

They set the text of an element on the page to the result of an incrementing or decrementing counter. It then sleeps for approximately 100 milliseconds. These are wrapped inside a process object and a scheduler started:

function start_demo() {
  var t = [];
  t[0] = new Process(t1);
  t[1] = new Process(t2);
  start(t);
  start_scheduler();
}

This 'start_demo' is called when a button is pressed on the page and the threads start running. 'setTimeout' is used for the threads to give up timeslices back to the browser and to prevent the 'too much recursion' error that can otherwise occur.

The demo page is here. Source code to the scheduler is in schedule.njs and for the thread1 demo in thread1.njs.

Categories: javascript, continuations

Partial Continuations and Narrative Javascript

I've been playing with Narrative Javascript and decided to have a go at porting the bshift and breset Scheme example to it. I'd previously had a go at this with Rhino.

Trying some simple examples out it turns out that the continuations provided by Narrative Javascript are really partial continuations. Here's an example demonstrating this:

var ex4cc ;

function ex4a(cont) {
  ex4cc = cont;
  cont();

}

function ex4() {
  var x = 0;
  ex4a->();
  x = x +1;
  print("here:" + x);
  return x;

}

function callex4() {
  var x = ex4cc();
  print("Returned: "+ x);

}

js> ex4()
here:1
js> ex4cc()
here:2
js> ex4cc()
here:3
js> callex4()
here:4
Returned: undefined

Full continuations don't return to their caller since they effectively replace the entire call stack with the original call stack of the continuation. This example shows 'callex4' printing 'Returned...' after the original continuation was called. The partial continuation appears to be delimited by the nearest function in the call stack which was not called with the '->' operator. This makes a simple shift/reset something like this:

 function reset(f) {
  return f();

}

function shift(f, cont) {
  f(cont);
}

The 'range' function and examples:

 function range(from, to) {
   return  shift->(function(pcc) {
               while(from < to) {
                 pcc(from++);
               }
               return to;
             });
} 

function test1() {
  reset(function() {
    print(range->(1, 5));
  });

}

function test2() {
  var sum = 0;
  reset(function() {
          sum += range->(1, 10);
        });
  print(sum);

}

js> test1()
1
2
3
4
5

js> test2()
55 

Once compiled with Narrative Javascript's compiler the resulting code runs fine in a standard web browser.

Neil Mix, the author of Narrative Javascript, has set up a google group for discussion about it.

Categories: javascript, continuations

Hop: Web Application System

Hop is an interesting web application system that appears to be similar in terms of goals to Links. From the website:

Hop is a new higher-order language designed for programming interactive web applications such as web agendas, web galleries, music players, etc. It exposes a programming model based on two computation levels. The first one is in charge of executing the logic of an application while the second one is in charge of executing the graphical user interface. Hop separates the logic and the graphical user interface but it packages them together and it supports strong collaborations between the two engines. The two execution flows communicate through function calls and event loops. Both ends can initiate communications.

It seems to be implemented in Bigloo Scheme and the demo's look very nice.

Categories: web, ajax, scheme

E4X with Server Side Javascript

I've updated the Server Side Javascript distribution to include an example which uses E4X.

E4X is described in Ecma-357 and adds a Native XML syntax to Javascript. The XML is converted to a Javascript object which can be used to query and manipulate the XML.

The E4X example servlet generates the XHTML to be displayed using:

var result = 
      <html>
        <head><title>HTML Using E4X!</title></head>
        <body>
          <p>Hello from E4X on: {new Date()}.</p>
        </body>
      </html>;

It embeds the current date and time by calling out to Javascript's Date object. This is done by wrapping the Javascript expression inside curly bracket: {new Date()}.

Given the XML object now stored in the 'result' variable it can be sent over the wire in the same manner as the other servlet examples:

    var text = doctype + result.toString();
    resp.setContentType("text/html")
    resp.setContentLength(text.length)
    resp.getOutputStream().print(text)
    resp.flushBuffer()

I prepend an XHTML doctype to the output as the XML generated is actually XML not HTML. This means things like <br> need to be <br/>, etc. I've not found a way to get E4X to generate HTML which is a pity - if anyone knows if it is possible I'd appreciate you dropping a comment here.

The updated code is available as javascript-server-0.2.tar.gz and in the darcs repository:

darcs get http://www.bluishcoder.co.nz/repos/javascript-server

Categories: javascript