Wellington for the Weekend

I'm back in Wellington for the weekend. I mentioned before I was looking at taking the Overlander to get back from Auckland.

The Overlander is a train that goes from Auckland to Wellington daily along a very scenic route. The service was nearly removed due to lack of customers and it not being cost effective as a result. That doesn't seem to be a problem now though, I tried to book and they were full for the next week or so. I ended up flying down putting some of my air points (or air dollars as they are now called) to good use.

I bumped into a couple of people I worked for at State Insurance about 17 years ago on the plane. It was quite strange seeing them after all those years. It was good to find out what's been happening with them and amazing that we recognised each other and remembered names.

I'm looking forward to a weekend of catching up with friends and family, and then head back to Auckland on Monday morning.

Why have I moved to Auckland? Find out in an upcoming blog post before the weekends over - No fair those that know spoiling the fun :-)

Categories: misc

Article Management System

I usually have a collection of notes and howto's that I've gathered as I worked out how to do things and file them away.

Sometimes they're not really relevant for blog postings or I prefer to be able to edit them over time to keep them up to date - which is not really useful for weblog postings without spamming people's RSS aggregators with old posts.

So around new years eve last year I wrote a very simple article management system in Factor. It allows me to write articles online using Wiky syntax, with an online preview. I can add tags and it automatically links tag pages to articles, lists of articles, etc. It basically does the bare minimum to do what I need. I'm hosting a couple of sites using it but the one for my technical articles is here: http://articles.bluishcoder.co.nz. Hopefully it stays up long enough after this weblog posting! It has been running another of my sites for the past month with reasonable traffic so it seems to be pretty stable.

The article manager can run multiple websites with different articles for each site and they're stored in a sqlite database.

I'll be tidying up the source code and making it available soon. It's written using the Factor 'furnace' web framework, and is the reason for the basic authentication code I wrote. It uses it to control site setup, content posting, etc.

Categories: factor

Zimbra Development with Eclipse

Following on from my earlier post about installing Zimbra, I wanted to setup a development environment so I could work on the source code. My development system is currently a Windows XP machine and I documented the steps I went through in this article: Zimbra Development with Eclipse.

Now I can use Eclipse to edit the code and deploy it on Tomcat for testing, using the Zimbra subversion repository code.

Categories: zimbra

Installing Zimbra on Ubuntu 6.10

Zimbra is an open source email server. It does email, calendar, contacts, and various other useful things that software like Microsoft Exchange does.

Zimbra has some very nice web based tools that make it operate very much like a desktop application. Some screenshots from Wild Bill's blogdom are here.

Zimbra is open source. They have a commercial version with extra features and support. And an open source version with limited support through the forums. You can apparently build from source but the process is a little painful and not well documented. The problem is that Zimbra has a number of dependencies on other open source projects and they don't provide direct downloads for those dependencies.

This means you spend a lot of time tracking down what software is needed (OpenLDAP, MySQL, etc) and need to get the specific versions required by Zimbra. These versions are listed in a 'howto build from cvs' document but that is out of date. So you need to trawl through the Makefiles for each of the Zimbra third party requirements to find the version numbers and track down where these can be found. Even after all that I was unable to get things to build so for now I'm sticking to their binary release until I can find out what the problem is.

The binary release for Ubuntu was also a bit tricky to install. After installation it wouldn't start OpenLDAP, giving this error:

Initializing ldap...TLS: error:02001002:
system library:fopen:No such file or directory bss_file.c:352

It turns out that this is the result of Ubuntu 6.10 not using 'bash' as its primary shell. '/bin/sh' is symbolically linked to '/bin/dash', a lightweight bash-alike. It doesn't have 'source' which is required by most of the Zimbra scripts ('source' is the same as '.' which Zimbra could also have used). Changing Ubuntu to use '/bin/bash' for 'bin/sh' fixes the problem and Zimbra installed fine.

Here are the basic steps I did to get Zimbra installed on Ubuntu 6.10 under VMWare Server:

1. Install Ubuntu 6.10
2. relink '/bin/sh' to point to '/bin/bash'
3. sudo apt-get install curl openssl libxml2 libexpat1 libgmp3c2 libpcre3 fetchmail libidn11 openssh-server
4. Setup DNS A and MX records for the server
5. Download and unpack the Zimbra Ubuntu open source binary
6. Run the installer

The key part is changing the 'bin/sh' link.

Categories: zimbra

My Tent My Castle

I moved from Wellington to Auckland last weekend but hadn't found a place to live there yet. So for the first few days I lived in a tent staying at the Avondale Motor Park.

The nice thing about living in a tent is it's cheap and doesn't require posting any form of bond or deposit. It also forces you to find non-computer related things to do as there's no power or internet.

The motor park advertises that they have internet access but that's limited to a single computer which does web browsing only for $2 per 15 minutes. Ouch! It was nice waking up in the morning to the sound of birdsong.

The downside is it's difficult to get anything computer related done. After a bit of hunting around I've found a more permanent place to stay. I'm now at a boarding lodge called Econ Lodge in Mt Albert, Auckland. They provide a private room with shared kitchen, toilet and shower facilities. I'm not sure how many others are here, probably about 20 I guess. The internet access is free as long as you don't do large downloads.

The first night at the Econ Lodge was interesting. They have a 'quiet time' rule after 10pm but one of the residents was playing music till about midnight. The landlord knocked on their door and asked them to turn it off and there was some shouting and swearing from the resident. The landlord used his key to get into their room which resulted in more and much louder shouting and swearing. Probably quite justified given that the landlord entered uninvited I have to admit. Threats to get the police, to kick the resident out followed.

Things quietened down after that but they still played music loud till about 3am - probably in protest - with the occasional thumping from upstairs to try and get them to be quiet, followed by them shouting out 'Come down here and do that you !#@!@*!'. Ahh, the joys of cheap accomodation. Things improved on the second night as that resident moved out.

I'll probably stay here for a while and see how it goes. I'll be keeping my eye out for more permanent accommodation, preferably my own place, I think. Worst case I can always go back to the tent :-)

I don't have transport at the moment so I'm walking and using public transport. This makes it a bit hard to get to places in Auckland which is quite spread out compared to Wellington, so I need to make sure anywhere I stay is on the public transport route. Hopefully I can find a way to get some training in while I'm up here. I should get a car but one of my goals in moving up here was to live as cheaply as possible.

I'll be down in Wellington again next weekend (the 27th/28th) to get some stuff, visit friends and attend Jesse's 2nd birthday. Hopefully I can find a cheap way to get there and back. The Overlander looks to be the cheapest option at the moment although I'd like to get down there for cheaper than that if possible.

Categories: misc

Parser Combinators

Gilad Bracha has a post on parser combinators in Smalltalk. It's a good introduction to parser combinators and how to implement them in an OO language.

Christian Plesner follows up with more details on writing and using parser combinators in Smalltalk.

Christian mentions the left recursion problem for parsing things like lists. For example this grammar:

<expr-list> -> <expr-list> "," <expr> | <expr>

With a direct translation to parser combinators you get an infinite recursion due to expr-list calling itself immediately on entry.

The solution Christian presents is a good one and we have it in the Factor parser combinator library as well. The 'list-of' combinator takes two parsers and returns a parser that parses a list of items. The first parser that 'list-of' expects is the parser for the items, and the second is the parser for the separators. For example, a parser for a comma seperated list of numbers:

'number' "," token list-of

Another combinator I recently added was 'pack'. It is for parsing items enclosed in a begin and end token. It takes three parsers. Two of them are the parsers for the begin and end tokens and the other is the parser for the body inside them. So a expression surrounded in parenthesis would be:

 "(" token 'expr' ")" token pack

. Or a Dylan style block:

"begin" token 'expr' <*> "end" token pack

Categories: factor

JSON Web Services with Factor

There's a post on Matt Croydon's weblog about using Lua to process JSON Web Services. The resulting Lua code is very easy to read. Here's my attempt at the same code in Factor:

REQUIRES: libs/http-client libs/json ;
USING: kernel io http-client json hashtables 
       prettyprint sequences math namespaces ;

[ 
  "http://local.yahooapis.com/MapsService/V1/" %
  "trafficData?appid=YahooDemo&street=" %
  "701+First+Street&city=Sunnyvale&state=CA" %
  "&output=json" %
] "" make http-get rot 200 = [
  nip json> 
  "Result" "ResultSet" rot hash hash
  1 swap [ 
    "Result: " write 
    >r dup . 1+ r> [
      swap write ": " write .
    ] hash-each
  ] each drop
] [
  2drop
] if

I split the URL up into a 'make' block so it wouldn't wrap in this weblog post. I could have used the long string itself. It looks ok and is readable if you know Factor but not as readable as the Lua example.

In Factor the tendancy is to factor things out into small words to make things more readable and to reduce the use of stack manipulation words. Here's a refactored version:

REQUIRES: libs/http-client libs/json ;
USING: kernel io http-client json hashtables 
       prettyprint sequences math namespaces 
       errors ;

: json-get ( url -- object )
  http-get rot 200 = [ 
    nip json> 
  ] [  
    "Could not process result" throw
  ] if ;

: hash-path ( hash seq -- object )
  swap [ swap hash ] reduce ;

: counted-each ( seq quot -- )
  #! Call the quotation for each element in
  #! the sequence. The quotation receives a count
  #! of the number of times the quotation has been called
  #! and should have stack effect ( element count -- ).
  0 -rot swap [ >r >r 1+ dup r> r> -rot call ] each-with drop ; 

: print-hashtable ( hash -- )
  [ swap write ": " write . ] hash-each ;

[ 
  "http://local.yahooapis.com/MapsService/V1/" %
  "trafficData?appid=YahooDemo&street=" %
  "701+First+Street&city=Sunnyvale&state=CA" %
  "&output=json" %
] "" make json-get { "ResultSet" "Result" } hash-path [
  "Result: " write . print-hashtable
] counted-each

While longer the result is some words that are immediately reusable in other contexts. Especially 'json-get', 'hash-path' and 'counted-each'. Anyone want to contribute a different Factor version in the comments below?

Please note that I didn't do this exercise to prove that language A is better than language B, it just gave me a chance to use the JSON library in Factor and see how usable it was. In fact I found a bug in my JSON implementation while doing it so you'll need the fixes from my repository:

darcs get http://www.bluishcoder.co.nz/repos/factor

Categories: factor

ODBC Interface for Factor

I've written a simple wrapper around the ODBC API for Factor. I based it on some Forth code I wrote a few years ago.

It has only been tested in Windows but it works quite well and I plan to do some more work on it as I need features. In the meantime it's in my repository:

darcs get http://www.bluishcoder.co.nz/repos/factor

It will likely be in the standard Factor repository soon.

Currently it supports opening and closing databases given a DSN string, running queries, getting column information, and retrieving fields and rows.

Categories: factor

Erlang and Mobile Phones

Luke Gorrie has started a new weblog and he's posting about his experiences learning Squeak Smalltalk. In his first post to the weblog he mentions what he's been working on the past couple of years:

The past couple of years I've instead worked really hard with some friends to build up a happy little Erlang company with an office where we drink our coffee. We've developed and marketed a system for mobile phone companies to keep track of all the mobile phones their customers are using and to automatically send configurations for picture messaging, web browsing, and so on. We've installed these systems in over 60 countries and have more than 200 million of you puny humans represented in our Mnesia databases. We're the market leaders and have only a fraction of our competitors' staff size, but of course you guessed this when I said "Erlang". :-)

Categories: erlang

Implementing concatenative words with Pattern Matching

There's a dicussion on the concatenative mailing list about concatenative languages and macros. The discussion veered into pattern matching and Manfred Von Thun wrote about using pattern matching to implement common concatenative operations.

I wrote previously about using pattern matching in Factor to do similar, calling the word 'shuffle'. For example, 2dup, could be implemented as:

{ ?a ?b } { ?a ?b ?a ?b } shuffle

I didn't think of it at the time but Manfred's article prompted me to try doing 'cons' and similar operations that actually have to look inside data structures. Since pattern matching does this already, 'shuffle' does it as well. For example, curry could be:

: my-curry { ?a [ ?b ] } { [ ?a ?b ] } shuffle ;
"hello" [ print ] my-curry .
! => [ "hello" print ]

Categories: factor

Basic Authentication added to Factor web server

I've added a simple basic authentication mechanism to the Factor web server (called 'httpd'). Basic Authentication is built into all browsers so it makes for a simple
authentication method.

It has one major downside and that is the password and username are sent from the browser to the client in clear text. You can work around this by using SSL for your web app.

Basic authentication uses usernames and passwords. These are defined to exist within a named realm. I'ved added a 'realms' symbol to httpd that holds a mapping (ie. a hashtable) from the realm name to the realm data.

The realm data can be a hashtable of usernames to passwords or a quotation with stack effect ( username password -- bool ). If it is a quotation it will be called to see if the user is valid in the given realm. If it is a hashtable the information is looked up directly. If it is anything else then the user is denied. An 'add-realm' word is provided to add (or replace) realm information:

H{ { "test1" "password1" } { "test2" "password2" } } "my-realm" add-realm
! or
[ "password" = swap "chris" = and ] "my-realm" add-realm

Realms can be set globally, per vhost or per responder in the same way as other httpd variables work.

The 'with-basic-authentication' word takes a realm name and a quotation. Before the quotation is run the http headers are checked to see if the user has been authenticated. If not a '401 Access Denied' reply is sent back to the browser with a request for basic authentication under the given realm.

The browser will then prompt the user for the username and password details and resend the request with the correct authorization headers.

If these headers exist then with-basic-authentication runs the quotation. So a simple responder is:

: my-responder ( -- )
 "my-realm" [
   "<html><body>Hello</body></html>" write
 ] with-basic-authentication ;

It can also be used with furnace, cont-responder, etc.

Another quick change I added was a 'responder-url' variable. This provides the responder portion of the URL requested. So for the default responder you'll get "/" and for other responders it will be "/responder/foo/". It always ends with a trailing slash.

This is needed for generating links and 301/307 location forwarding which require absolute url's. By using 'responder-url' you can make your responder not depend on the particular path it is installed under.

This in my repository and hopefully soon in the main factor respository. My repository is available with:

darcs get http://www.bluishcoder.co.nz/repos/factor

Oh, and Happy New Year!

Categories: factor