File Uploads with HAppS

I've updated the Not a HAppS Tutorial cookbook with an example of how to handle file uploads using file input types in forms.

Next step is to take a look at how sessions and session data is handled.

Categories: haskell, happs

Not a Tutorial on HAppS

A long time ago I took a look at HAppS, a Haskell web application framework.

I recently got the excellent book Real World Haskell and have been working through the book. I wanted something to explore while doing this so I revisited HAppS to play around with web programming in Haskell.

There's some documentation out there on HAppS, compared to last time I looked at it. Some pointers:

• HAppS Google Group
• HAppS Tutorial 2
• The Cabalized, Self Demoing, HAppS Tutorial
• Software Simply's introduction to HAppS

I found these very useful resources to learn about HAppS.

I'm documenting what I learn as I go, writing Not a HAppS Tutorial, but a cookbook of examples doing small things as I learnt them.

Not a HAppS Tutorial is hosted on a wiki written in HAppS called Gitit. It's a nice piece of work and it also proved very useful in learning HAppS, looking at the code for a working piece of software.

Categories: haskell, happs

Random and Binary IO using Iteratees

I wrote previously about interesting developments related to left folds and i/o. Oleg Kiselyov has come up with more Iteratee based goodness involving binary i/o and random access.

Iteratees presuppose sequential processing. A general-purpose input method must also support random IO: processing a seek-able input stream from an arbitrary position, jumping back and forth through the stream. We demonstrate random IO with iteratees, as well as reading non-textual files and converting raw bytes into multi-byte quantities such as integers, rationals, and TIFF dictionaries. Positioning of the input stream is evocative of delimited continuations.
...
We show a representative application of the library: reading a sample TIFF file, printing selected values from the TIFF dictionary, verifying the values of selected pixels and computing the histogram of pixel values. The pixel verification procedure stops reading the pixel matrix as soon as all specified pixel values are verified. The histogram accumulation does read the entire matrix, but incrementally. Neither pixel matrix processing procedure loads the whole matrix in memory. In fact, we never read and retain more than the IO-buffer-full of raw data.

Categories: haskell

Dynamic Compilation and Loading of Modules in Haskell

The Haskell system GHC has libraries that provide the ability to compile Haskell code and dynamically load it into a running Haskell program. A library that provides this functionality is hs-plugins. Unfortunately hs-plugins doesn't work with the latest GHC release, 6.10.1.

There is an API to the internals of GHC that allows implementing runtime compilation and loading. The api is documented but there aren't very many usage examples that work with the latest GHC release. I stumbled across a blog post which no longer exists describing an older version of the API, but managed to retrieve it from archive.org which provided a lot of help since the API's are similar.

As a means to learn how to use the API I decided on a very simple use case of compiling and loading a very simple Haskell module containing a single function 'print' that outputs a string. The module looks like this:

module Test (Test.print) where

print :: String -> IO ()
print x = 
  putStrLn x

To compile and load this code dynamically we need a compilation target. This is created using the function guessTarget. It's passed a string that refers to the name of a module, or the name of a source file. By using a source file we can tell the Haskell system to compile and load it. The phase argument I set to Nothing as I have no idea what that is.

Once we have the target it needs to be added to the current compilation session. This is done via the addTarget function. When all the targets are added, a call to load will do the equivalent of a ghc --make to build the module if needed. So all the relevant target code looks like:

do
  target <- guessTarget "Test.hs" Nothing
  addTarget target
  load LoadAllTargets

Notice this is run within the 'do' syntax. This is because all these compiler functions must be called with a special Monad, called GhcMonad. The way this works is described later but just imagine that the GhcMonad is implicitly passed to all the Ghc functions we are calling automatically.

'load' returns a success flag that can be pattern matched on to determine if it succeeds or failed. If it succeeds we can start using the new module. To be able to access exported functions from the module we need to get a reference to the module using findModule and use setContext on it. 'setContext' takes two arrays containing modules. All modules in the first array will have their top level scope available. All modules listed in the second array will have only their exports available. In this example we want to access the 'print' function of 'Test', which is exported.

The function compileExpr is used to compile a string containing a Haskell expression. The result of this is an HValue which we can do things with. The expression that is compiled can access any of the bindings available in the context we set up via 'setContext'. So passing "Test.print" to 'compileExpr' will return the function 'print' from the 'Test' module that we dynamically built and loaded. Code similar to this:

do
  r <- load LoadAllTargets
  case r of
    Failed -> error "Compilation Failed"
    Succeeded -> do
      m <- findModule (mkModuleName "Test") Nothing
      setContext [] [m]
      value <- compileExpr "Test.print"

Unfortunately the result of 'compileExpr' is of no use to us as an HValue if we want the actual function type so we can call it. Haskell being strongly typed we have to 'cheat' by telling Haskell 'yes, I know you think this is an HValue, but I can assure you, it's really a function'. This is Haskell's super dangerous Unsafe.Coerce. It allows us to coerce from one type to another and is, as the name implies, completely unsafe. If the type you are coercing isn't really of the type you are coercing too then you'll very likely crash your program and/or corrupt data. It's the reinterpret_cast<> of the Haskell world. But for handling dynamic loading of code it seems exactly what we want. I'm not sure of any other way to convert the HValue type.

The type of 'print' in the 'Test' module is 'String -> IO ()'. 'unsafeCoerce' can be called to convert the 'HValue' to this:

value <- compileExpr "Test.print"
  do let value' =  (unsafeCoerce value) :: String -> IO ()
     return value'

The receiver of the result, value', can now call the function as it knows its type. The complete code for this example is:

mport GHC
import GHC.Paths
import DynFlags
import Unsafe.Coerce

main :: IO ()
main =
    defaultErrorHandler defaultDynFlags $ do
      func <- runGhc (Just libdir) $ do
        dflags <- getSessionDynFlags
        setSessionDynFlags dflags
        target <- guessTarget "Test.hs" Nothing
        addTarget target
        r <- load LoadAllTargets
        case r of
          Failed -> error "Compilation failed"
          Succeeded -> do
            m <- findModule (mkModuleName "Test") Nothing
            setContext [] [m]
            value <- compileExpr ("Test.print")
            do let value' = (unsafeCoerce value) :: String -> IO ()
               return value'
      func "Hello"
      return ()

Build using:

$ ghc -package ghc --make Api.hs
[1 of 1] Compiling Main             ( Api.hs, Api.o )
Linking Api ...
$ ./Api
Hello
$

You can test that it is dynamically compiling and loading by modifying 'Test.hs' so the output is different. Change the 'putStrLn' to putStrLn $ x ++ " World!" and rerun 'Api' without recompiling it:

$ ./Api
Hello World!
$

I mentioned previously the implicit 'GhcMonad' that is passed to each GHC API function. This is provided by the runGhc word. It takes a pointer to the directory where GHC's library files reside. In this example I use the GHC.Paths (available from Hackage package to deal with this using 'libdir'. The second argument is the action to perform, the GhcMonad, and is built using the 'do' syntax. Haskell's 'do' syntax is syntactic sugar that does the magic of threading the GhcMonad state holding the targets, modules, contexts, etc through the various function calls. Find a favourite Monad tutorial/explanation if you want to know more.

'defaultErrorHandler' sets up the environment to handle compilation errors in a standard way. The 'DynFlags' related calls get and set standard flags that can be used to configure the compilation. You can set the equivalent of GHC arguments to control extra features that GHC can use, or use the interpreter (as used by GHCi) rather than the compiler.

Hopefully that helps explain some of what is going on and how to use the current GHC api. I'd appreciate comments on better ways of doing things, and other examples of the API.

Categories: haskell