Bluish Coder

Moving away from blogger

noreply@blogger.com (Chris Double) — Sat, 13 Feb 2010 06:10:00 +0000

Blogger are deprecating their ftp publishing support and are recommending users to switch to their custom domain feature where they host the blog files for you. I prefer to keep my posts on my own server so will be switching weblog systems in the near future.

This will break existing links to Atom feeds for specific labels (eg. the feed for Mozilla specific posts) as they are on a blogger controlled domain. So if you're reading this through one of those Atom feeds and want to continue with it then you might want to switch to:

http://bluishcoder.co.nz/tags/[tagname]/atom.xml

For example, the Mozilla specific posts will be in http://bluishcoder.co.nz/tags/mozilla/atom.xml.

My existing posts will remain at their current URL's.

Using the Wasp Lisp Secure Remote Injection Framework (MOSREF)

noreply@blogger.com (Chris Double) — Fri, 27 Nov 2009 22:19:00 +0000

MOSREF is a secure remote injection framework written in Wasp Lisp (which I previously wrote about here). With MOSREF you have a 'console' program running which can create dones applications that are run on a target system.

The console and drones can communicate with each other, executing shell commands or Wasp Lisp code. Drones can create other drones to 'bridge' bridge communications so the console can send commands to drones it is not directly connected too.

Lisp code can be compiled and sent to drones to execute. One of the 'built in' capabilities is a Socks proxy server. This can be run on a drone and data is tunneled to the console which the controller can use as a socks proxy.

MOSREF isn't built by default with Wasp. To build it you need to run 'waspc' to compile and create an executable (example assumes you are in the root of the Wasp source directory):

$ cd mod
$ waspc -exe ../mosref bin/mosref
BUILD: bin/mosref
BUILD: core/macro
BUILD: core/config
BUILD: site/config
...
BUILD: mosref/cmd/with
BUILD: mosref/cmds
BUILD: bin/mosref
$ chmod +x ../mosref
$ rlwrap ../moseref
console> help
Commands: clear <key> ...
          cp <src-file> <dst-file>
          do <lisp-expr>
          drone <file> <id> <platform>
          exit
          help [<command>]
          load <path>
          nodes
          on <node-id> [<command>]
          proxy [<portno> [<secret>]]
          recover <id>
          set [<key>[=<value>] [<command>]]
          sh <cmd>
          with <key>[=<value>] [<command>]

The first thing you need to do on the running console is set the IP address and the port it will listen on (Replacing xx.xx.xx.xx with the IP address):

console> set addr=xx.xx.xx.xx
Set.
console> set port=10000
Set.

Now you need to create a drone executable to be run on the target system:

console> drone drone1 foo linux-x86
Drone executable created.
Listening for drone on 10000...

This creates an executable for Linux systems called 'drone1'. It is given the name node name 'foo'. It will show in the list of nodes available from the console:

console> nodes
NODES: console online address: xx.xx.xx.xx port: 10000
       foo offline

When the 'drone1' executable is run on a target system it will connect to the console using an encrypted connection. You will need to find a way to copy and run the executable onto the target. The following shows it being run on the target:

target$ ./drone1
  DRONE: Preparing keys...
  DRONE: Sending Drone Public Key...
  DRONE: Sending Drone IV...
  DRONE: Reading Console IV...
  DRONE: CONSOLE IV CT is ...
  DRONE: CONSOLE IV PT is ...
  DRONE: Confirming Console IV...
  DRONE: Waiting for Console to confirm Drone IV...
  DRONE: Affiliation complete....

The connection will show on the console:

console> nodes
NODES: console online address: xx.xx.xx.xx port: 10000
       foo online

Notice it shows 'foo' is now online. It's now possible to execute commands on the target machine by sending them to the drone. Using 'sh' you can execute shell commands. From the console:

console> on foo sh ifconfig
eth0      Link encap:Ethernet  HWaddr ...
          inet addr:yy.yy.yy.yy  
...
          RX bytes:2458924520 (2.2 GB)  TX bytes:2458924520 (2.2 GB)
console> on foo set addr=yy.yy.yy.yy
Set.
console> on foo set port=10000
Set.

Here we execute 'ifconfig' on the target so we can find out the IP address. The output of 'ifconfig' is sent back to the console. Using this information the addr and port is set on the drone. This can be used to have the target create new drones on internal machines that only it can access. It will then act as a bridge between the console and the internal machine.

As well as running shell commands you can run Wasp Lisp code:

console> on foo do (+ 1 2)
:: 3
console> on foo load lib/http-file-server.ms
:: spawn-http-file-server
console> on foo do (offer-http-file 2080 "/test" "text/plain" "Hello world!")
:: [queue 824A098]

Using 'on foo do' we compile a Lisp expression and send it to 'foo' to be run. In this case a simple addition. 'on foo load' will load a Lisp file located on the console, compile it, and send the byte code to be run on 'foo'. Then we 'on foo do' to run a function contained in that file on the 'foo' machine. In this case, it runs a simple webserver. Any Lisp code can be sent and run.

It's possible to copy files between nodes too. In this next example we create another drone that will be run on a machine internal to the network that the target is on. Note that the build is performed on the console so we need to copy the executable from there to the target.

console> on foo drone drone2 bar linux-x86
Drone executable created.
Listening for drone on 10000...
console> cp console:/home/console/waspvm/mod/drone2 foo:/tmp/drone2
Copy from console:/home/console/waspvm/mod/drone2 to foo:/tmp/drone2

Using some magic means (an exploit, copying file using shell commands on the target drone, etc) we get 'drone2' running on another machine available internally on the target network:

target2$ ./drone2
  DRONE: Preparing keys...
  DRONE: Sending Drone Public Key...
  DRONE: Sending Drone IV...
  DRONE: Reading Console IV...
  DRONE: CONSOLE IV CT is ...
  DRONE: CONSOLE IV PT is ...
  DRONE: Confirming Console IV...
  DRONE: Waiting for Console to confirm Drone IV...
  DRONE: Affiliation complete....

From the console we now see the 'bar' node:

console> nodes
NODES: bar online
       console online address: xx.xx.xx.xx port: 10000
foo online address: yy.yy.yy.yy port: 10000

MOSREF has a 'proxy' command that lets you set up a Socks 4 proxy on the console that tunnels traffic to and from a target node. Here we set up a proxy to the 'bar' node:

console> on bar proxy 5000
SOCKS Proxy created, listening on port 5000.

This will result in the 'console' machine having a Socks 4 proxy running on port 5000. If you configure Firefox to use this proxy you can access local webservers available from the internal network that 'bar' can see. Note that 'console' can't see 'bar' directly. It is tunnelling traffic to 'foo', from there to 'bar', then back from 'bar' to 'foo' and to 'console.

The source for MOSREF is in the Wasp Lisp distribution and makes for a good body of Wasp Lisp code for learning.

Categories: waspvm

Wasp Lisp - a Small Scheme-like Lisp

noreply@blogger.com (Chris Double) — Fri, 27 Nov 2009 08:28:00 +0000

Wasp Lisp is a small Scheme-like Lisp implementation developed by Scott Dunlop. It features a lightweight concurrency model (with similarities to Erlang and Termite).

Wasp Lisp was originally derived from MOSREF - the Mosquito Secure Remote Execution Framework. It includes an implementation of MOSREF so can do similar things that the original was built for.

Wasp feels a lot like Scheme. It has a REPL which you can use to try Lisp interactively. You can spawn lightweight threads and use channels to communicate between threads. Here's a simple example with a thread that loops forever, waits for data to be sent to a channel and then prints that out. From the REPL strings are interactively sent to the channel:

$ rlwrap ../wasp
>> (define channel (make-queue))
:: [queue 98154C0]
>> (define (looper channel)
..   (forever
..     (define data (wait channel))
..     (print data)))
:: looper
>> (spawn looper channel)
:: [process looper]
>> (send "hello\n" channel)
hello
:: [queue-output 9815500]
>> (send "world\n" channel)
world
:: [queue-output 9815500]

Threads are co-operative in Wasp. You need to manually yield to switch from a thread. The following example doesn't manually yield and will constantly print 'a' to the terminal:

(begin 
  (spawn 
    (lambda () 
      (forever (print "a\n")))) 
  (spawn 
    (lambda () 
      (forever (print "b\n")))))

To yield you use the 'pause' function. Adding this to the 'forever' loop in the example above will switch between the two threads:

(begin 
  (spawn 
    (lambda () 
      (forever 
        (pause)
        (print "a\n")))) 
  (spawn 
    (lambda () 
      (forever 
        (pause)
        (print "b\n")))))

Wasp Lisp code can get compiled to a compact bytecode format using the 'waspc' command. This can produce a binary executable for the platform:

$ cat >test.ms
(define (main)
  (print "hello world!\n"))
$ waspc -exe hello test.ms
BUILD: test
BUILD: core/macro
BUILD: core/config
BUILD: site/config
BUILD: core/file
BUILD: core/module
BUILD: core/io
BUILD: core/macro
BUILD: core/config
BUILD: site/config
BUILD: core/file
BUILD: core/module
BUILD: core/io
BUILD: test
$ chmod +x hello
$ ./hello
hello world!

There is quite a bit of example code, including a simple HTTP server:

$ rlwrap ../wasp
>> (import "lib/http-file-server")
:: #t
>> (offer-http-file 2080 "/test" "text/plain" "Hello world!")
:: [queue 985A1F0]

This serves the text 'Hello world!' when http://localhost:2080/test is requested. The Wasp VM site has an informal speed test of serving data comparing against THTTPD.

Wasp builds and runs on Linux, Mac OS X and Windows. Instructions for building and links to other information are here. The source is available on launchpad. libevent is needed to build.

Categories: waspvm

Displaying images with Self

noreply@blogger.com (Chris Double) — Mon, 27 Jul 2009 11:30:00 +0000

The Self source code has some example programs, one of which is a web browser. This was developed quite a long time ago and is unable to nicely display most pages on the modern web but it does a reasonable job of basic pages. On the Linux port there were some problems with displaying images however. I tracked this down to two issues:

It only natively decoded image formats that aren't really used on the web (Sun Raster format for example). The GIF and JPG handling code spawned external programs, which I didn't have on my Linux install, to convert to a format it could handle before displaying.
There was a bug in the color management code on X11 for displays greater than 8 bit.

The first was a quick fix by changing the programs that it used to do the conversion to one readily available on Linux. I changed it to use the Image Magick 'convert' program. This enabled displaying code in an imageMorph by setting the image for the morph:

setImage: ui2Image copyFromGIFFile: 'test.gif'

I tracked the second issue down to code that was sending a message to an object that didn't understand that message (Static typing fans can say 'I told you so' here). Self has a 'paintManager' which handles color mapping for displays that can't display a large number of colors. On 8 bit displays this is a 'paintManager'. On displays with more colors it's a 'unmappedPaintManager'. The latter doesn't have some of the slots of the former and there was code trying to do this. I changed this to check if the paintManager instance can do color mapping before sending the messages.

With that change the web browser could display images. An example page that works with the browser (one of the few...), and code to display it:

'http://www.bluishcoder.co.nz/self/imagetest.html' asURL getPageForUser

The patches with these fixes is available from the webbrowser branch of my fork of the Self git repostory on github. The fixes were only a few lines of code.

I did a short screencast showing loading image morph's and displaying the web page in the browser. And before you say, "wow, that GUI looks so 90's", don't forget - it was.

Categories: self

Building and using Self on Linux

noreply@blogger.com (Chris Double) — Sun, 26 Jul 2009 05:51:00 +0000

The original implementation of the Self programming language is easy to build from source. Russell Allen maintains a git repository on github containing the source.

I use Arch Linux with 'flex' and 'tcsh' packages installed for various parts of the build process. To build a Self binary:

$ git clone git://github.com/russellallen/self.git
$ cd self/release
$ sh buildLinuxVM
$ cd ..
$ ./Self

The build takes about 30 minutes on my laptop. There will be many compilation warnings but these can be safely ignored. The result of the build is a 'Self' executable in the root directory of the cloned source. The executable can be run but most of the Self library is not loaded. You need to create an image to run or use an existing one. About the only thing you can do with the Self VM without an image is use primitives.

The 'objects' directory contains the source for the core Self library, the User Inteface, as well as other applications and interesting stuff. These can be used to build an image to run Self. The file 'all2.self' can be used to create an image including the graphical user interface. 'small.self' can be used to build an image without the user interface. The primitive '_RunScript' is used to load these files. Once loaded you can save a 'snapshot' (this is the term Self uses for images) which can be resumed in later sessions.

$ cd objects
$ ../Self
Self Virtual Machine Version 4.1.13, Sun 26 Jul 09 16:59:28 Linux
Copyright 1989-2003: The Self Group (type _Credits for credits)

for I386:  LogVMMessages = true
for I386:  PrintScriptName  = true
for I386:  Inline = true
for I386:  SICDeferUncommonBranches = false (not implemented)
for I386:  SICReplaceOnStack = false (not implemented)
for I386:  SaveOutgoingArgumentsOfPatchedFrames = true
VM # 'all2.self' _RunScript
reading all2.self...
reading ./core/init.self...
reading ./core/allCore.self...
reading ./core/systemStructure.self...
reading ./core/defaultBehavior.self...
...

reading ./ui2/outliner/profileSliceGrpMod.self...
reading ./ui2/outliner/profileSelfSlotMdl.self...
reading ./ui2/outliner/powerOperations.self...
verifying newgen: eden from to oldgen: old0 old1 z p r S v O m N M i  done
Starting: Collecting Garbage...
Finished: Collecting Garbage
Starting: Refilling module cache...
Finished: Refilling module cache
"Self 0" saveAs: 'ui.snap'
Starting: Writing snapshot to ui.snap...
Finished: Writing snapshot to ui.snap
shell
"Self 1" quit
Save to ui.snap before quitting? 
  y => save, then quit
  n => quit without saving
  RET => cancel
Response: n

Notice the call to 'saveAs:' to save the image snapshot. 'quit' is used to exit the running Self session. This image can now be resumed using the '-s' argument to 'Self' whenever you want to run Self:

$ cd objects
$ ../Self -s ui.snap
for I386:  LogVMMessages = true
for I386:  PrintScriptName  = true
for I386:  Inline = true
for I386:  SICDeferUncommonBranches = false (not implemented)
for I386:  SICReplaceOnStack = false (not implemented)
for I386:  SaveOutgoingArgumentsOfPatchedFrames = true

 Welcome to the Self system!  (Version 4.4)


Copyright 1992-2009 AUTHORS, Sun Microsystems, Inc. and Stanford University.
See the LICENSE file for license information.

Type _Credits for full credits.

VM version: 4.1.13

  "Self 1"

At this point you can now use any Self expressions. All code is run in the contents of an object called 'shell'. You can start the graphical user interface by passing the 'open' message to the 'desktop' object:

"Self 1" desktop open

The desktop will appear with a 'trash can' object and an outliner for the 'shell' object. Left clicking on the middle button labelled 'E' in the group of three buttons on the right of the shell will open an expression evaluator allowing you to enter Self expressions. Middle click on objects to get an object specific menu. On the desktop this provides a menu item to quit when you want to leave the session.

There are some interesting applications in the 'objects' directory. For example, there is a Smalltalk implementation written in Self. This can be loaded at the shell with the command:

"Self 1" bootstrap read: 'smalltalk' From: 'applications/smalltalk'

When it completes loading you can find a 'smalltalk' item on the desktop's right click menu. Choosing that provides a menu to load a standard Smalltalk browser, workspace and inspector, allowing you to write Smalltalk code. There's a document that describes the Smalltalk emulator for more information about it. There are other interesting applications in the objects directory including a web browser, Java emulator, a C preprocessor, Cecil implementation and a parser/lexer generator. Most of this code is old and in various states of usability but it provides some interesting examples of Self code to look at and play with.

The Self tutorial provides a good introduction to using the Self user interface.

Categories: self

Prototype Based Programming Languages

noreply@blogger.com (Chris Double) — Thu, 16 Jul 2009 08:08:00 +0000

I've been reading up on protoype based programming languages recently. Mainly using the Io Programming Language and Self but also looking at LambdaMOO and similar languages. A good overview of using the prototype based approach to building programs is Organizing Programs without Classes. This post is based on examples from that paper and from Attack of the Clones which covers design patterns using Self.

Self

In the Self programming language objects are created using a literal object syntax. This syntax defines code and slots within the (| and |) delimiters. An example object looks like:

point = (|
  parent* = traits point.
  x.
  y.
|)

This creates an object with 3 slots. The slots are 'parent', 'x' and 'y'. The 'parent' slot is assigned an inital value of 'traits point' which is the traits object for points (more on what this is later). The '*' that is suffixed to the 'parent' slot means it is a prototype slot and used in the prototype slot lookup chain.

This means when a message is sent to the 'point' object the lookup starts with the 'point' object. If a slot with the messages name is not found in that object then each prototype slot (those suffixed with '*') are searched looking for a slot with that name.

So in the 'point' case, looking for 'x' will find it immediately in the point object. Looking for 'print' will not so it will look for it in the 'traits point' object. If it's not there it will look in that objects prototype slots and so on until it is found or the search is exhausted.

The idiom in Self (and other prototype based languages) is to create global objects like these and use 'clone' to create copies of it. So creating two different points would look like:

a = point clone.
b = point clone.

Notice that 'point' has no method slots. Only the 'x' and 'y' which contain data. The methods are defined in the 'traits point' object. The definition of that could look something like:

traits point = (|
  parent* = traits clonable.
  print = x println y println.
|)

This provides a method to print the 'x' and 'y' values of the object and another parent that provides other basic functionality lie the ability to clone. The 'traits point' object doesn't define any data slots. It defines only methods. However it uses 'x' and 'y' messages that aren't defined. It expects to be used in a prototype slot of another object that defines the 'x' and 'y' slots (like ur 'point' example earlier.

Separating the code out into data objects and trait objects allows the trait object to be reused in other objects. For example, an object that computes the 'x' and 'y' values rather than storing them can re-use the traits object:

computed_point = (|
  parent* = traits point.
  x = ..compute x..
  y = ..compute y..
|)

This 'computed_point' can be used anywhere a point is expected. The traits object sends the 'x' and 'y' messages when it needs their value and it doesn't matter if there stored as data as in the 'point' object, or as methods that calculate the value as in the 'computed_point' object. Each 'point' and 'computed_point' object shares a single trait object instance. This avoids the need to have multiple copies of the methods in each point object instance.

The prototype slots of an object effectively form an inheritance relationship. The way to subclass objects in Self is to create a new object and set a prototype slot to an instance of the parent object. Usually it's the trait objects that map the subclass relationship since it is those objects that contain the object behaviour (ie. the methods and no data). An example follows in Self of how a 'polygon' and 'filled_polygon' object can be modelled (this is from the 'Organizing Programs without Classes' paper):

polygon_traits = (|
  draw = ...draw using 'vertices' slot to get points...
|)

polygon = (|
  parent* = polygon_traits.
  vertices.
|)

filled_polygon_traits = (|
  parent* = polygon_traits.
  draw = ...draw using 'vertices' and 'fillPattern'...
|)

filled_polygon = (|
  parent* = filled_polygon_traits.
  vertices.
  fillPattern;
|)

Cloning the 'polygon' object and sending the 'draw' message will draw the polygon using the list of points held in the 'vertices' slot. Cloning the 'filled_polygon' object and sending the 'draw' message will draw using the specialized 'draw' method that also uses the 'fillPattern' slot to fill the polygon when drawing. This can re-use the 'draw' method in the 'polygon_traits' object if needed.

The new 'filled_polygon' object did require defining a new 'vertices' slot. Self allows multiple prototype slots, each of which is involved in the lookup for slot names. We can share the 'vertices' from the 'polygon' object by making that an additional prototype slot in 'filled_polygon'. This is often termed a 'data parent':

filled_polygon = (|
  parent* = filled_polygon_traits.
  dataParent* = polygon clone.
  fillPattern;
|)

Notice the 'dataParent' slot is a prototype slot (suffixed with a '*'). This means it participates in the slot lookup process. The data parent approach has an advantage over the previous example in that if we change the representation of the 'polygon' object then all new 'filled_polygon' instances will get this new representation. We don't need to edit the 'filled_polygon' definition for the modified or additional slots.

In the 'filled_polygon' example we re-used the 'vertices' slot from 'polygon'. We can also define subclasses that implement 'vertices' differently than 'polygon'. For example, a rectangle that stores the four corners and computes the vertices from this. Due to the seperation of state and behaviour this can be modelled easily:

rectangle_traits = (|
  parent* = polygon_traits.
  draw = ...draw rectangle using left, right, top, bottom...
  vertices = ...compute vertices list using left, right, etc...
|)

rectangle = (|
  parent* = rectangle_traits.
  left.
  right.
  top.
  bottom.
|)

Inheritance in Self can be dynamic if the prototype slots are made assignable. This means, at runtime, we can change the value of the slot used during message lookup, resulting in different behaviour. This can be used for objects that can be in different states.

An example is a 'file' object. It can be opened or closed. Some methods in 'file' can only be used when the file is open, and some only when it is closed. This could be managed by conditional checks in each method. Or the parent of the object could be changed to a different traits object depending on the state - this avoids the need for each method to check if the file is in the open or closed state:

open_file_traits = (|
  read = ...
  close = setParent: close_file_traits.
|)

closed_file_traits = (|
  open = setParent: open_file_traits.
|)

file = (|
  parent* = closed_file_traits.
|)

Methods like 'open' are only available on closed files. 'read' can only be called on opened files. This is basically the Strategy pattern made easy using Self's dynamic inheritance.

Io

Whereas Self defines the prototype lookup chain to be that of the prototype slots in an object, Io instead has a slot called 'protos' which is a list of all objects in the prototype chain. Instead of creating slots with a name suffixed with '*' you append to the existing 'protos' list.

The 'protos' list is initially populated when you clone an object with the object that you cloned. This is unlike Self where copying an object does a shallow copy of all the slots of that object. In Io you get a 'differential inheritance' model where your newly created object has no slots, just a 'protos' field that contains the original object that was cloned. The Self 'point' example I used earlier looks like:

Point := Object clone do(
  x := 0
  y := 0
)

Calling 'clone' on this new 'Point' object results in a new object that does not contain it's own 'x' and 'y' values. Instead its 'protos' field points to the 'Point' object which contains the values. When you set the 'x' value on the clone it will then create its own 'x' slot rather than changing the prototypes. In this way clones of big objects where relatively few slots are changed will save some memory:

a := Point clone
b := Point clone
a x = 5
a x println
 => 5
b x println
 => 0
a protos first == Point
 => true
b protos first == Point
 => true

This does have an interesting side effect in that if you clone a clone then you can end up with a longish prototype chain for the method lookup:

a := Point clone
b := a clone
c := b clone
c protos first == b
 => true
c protos first protos first == a
 => true
c protos first protos first protos first == Point
 => true

Inheritance is handled in much the same manner as Self but you need to manipulate the 'protos' slot instead of having multiple prototype slots. The filled polygon example looks like:

PolygonTraits = Object clone do(
  draw := method(vertices foreach(v, v println))
)

Polygon := PolygonTraits clone do(
  vertices := list(1,2,3,4)
)

FilledPolygonTraits := PolygonTraits clone do(
  draw := method(resend; fillPattern println)
)

FilledPolygon := FilledPolygonTraits clone do(
  appendProto(Polygon clone)
  fillPattern := "solid"
)

Polygon clone draw
  => 1
     2
     3
FilledPolygon clone draw
  => 1
     2
     3
     "solid"

'appendProto' appends an object to the prototype chain which is initially 'FilledPolygonTraits' in this examples as that was the initial object we cloned. The dynamic inheritance example can also be done in Io:

OpenFileTraits := Object clone do(
  read := method(n, "Reading #{n} bytes" interpolate println)
  close := method(
    self removeProto(OpenFileTraits)
    self appendProto(ClosedFileTraits)
  )
)

ClosedFileTraits := Object clone do(
  open := method(
    self removeProto(ClosedFileTraits)
    self appendProto(OpenFileTraits)
  )
)

File := Object clone do(
  init := method(self appendProto(ClosedFileTraits))
)

f := File clone
f read(255)
  => Exception: File does not respond to 'read'
f open
f read(255)
  => reading 255 bytes
f open
  => Exception: File does not respond to 'open'
f close

It's a bit more work than in Self to manually manage the prototype chain but does work.

JavaScript

JavaScript is also a prototype based programming language. Unlike Self or Io it only allows one object to be used as the prototype in any given object. This is stored in a hidden '__proto__' member and cannot be updated once set on construction (some implementations allow changing it however). Objects are created by using the 'new' keyword on a constructor function that initializes the object. For now I'll leave it as an exercise for the reader to implement the examples above in JavaScript. I'd be interested in the approaches people take.

Categories: self, io, javascript

Playing Ogg files with audio and video in sync

noreply@blogger.com (Chris Double) — Sat, 27 Jun 2009 10:06:00 +0000

My last post in this series had Vorbis audio playing but with Theora video out of sync. This post will go through an approach to keeping the video in sync with the audio.

To get video in sync with the audio we need a timer incrementing from when we start playback. We can't use the system clock for this as it is not necessarily keeping the same time as the audio or video being played. The system clock can drift slightly and over time this audio and video to get out of sync.

The audio library I'm using, libsydneyaudio, has an API call that allows getting the playback position of the sound sample being played by the audio system. This is a value in bytes. Since we know the sample rate and number of channels of the audio stream we can compute a time value from this. Synchronisation becomes a matter of continuously feeding the audio to libsydneybackend, querying the current position, converting it to a time value, and displaying the frame for that time.

The time for a particular frame is returned by the call to th_decode_packetin. The last parameter is a pointer to hold the 'granulepos' of the decoded frame. The Theora spec explains that the granulepos can be used to compute the time that this frame should be displayed up to. That is, when this time is exceeded this frame should no longer be displayed. It also enables computing the location of the keyframe that this frame depends on - I'll cover what this means when I write about how to do seeking.

The libtheora API th_granule_time converts a 'granulepos' to an absolute time in seconds. So decoding a frame gives us 'granulepos'. We store this so we know when to stop displaying the frame. We track the audio position, convert it to a time. If it exceeds this value we decode the next frame and display that. Here's a breakdown of the steps:

Read the headers from the Ogg file. Stop when we hit the first data packet.
Read packets from the audio stream in the Ogg file. For each audio packet:
1. Decode the audio data and write it to the audio hardware.
2. Get the current playback position of the audio and convert it to an absolute time value.
3. Convert the last granulepos read (defaulting to zero if none have been read) to an absolute time value using the libtheora API.
4. If the audio time is greater than the video time:
  1. Read a packet from the Theora stream.
  2. Decode that packet and display it
  3. Store the granulepos from that decoded frame so we know when to display the next frame.

Notice that the structure of the program is different to the last few articles. We no longer read all packets from the stream, processing them as we get them. Instead we specifically process the audio packets and only handle the video when it's time to display them. Since we are driving our a/v sync off the audio clock we must continously feed the audio data. I think it tends to be a better user experience to have flawless audio with video frame skipping rather than skipping audio but smooth video. Worse is to have both skipping of course.

The example code for this article is in the 'part4_avsync' branch on github.

This example takes a slightly different approach to reading headers. I use ogg_stream_packetpeek to peek ahead in the bitstream for a packet and do the header processing on the peeked packet. If it is a header I then consume the packet. This is done so I don't consume the first data packet when reading the headers. I want the data packets to be consumed in a particular order (audio, followed by video when needed).

// Process all available header packets in the stream. When we hit
// the first data stream we don't decode it, instead we
// return. The caller can then choose to process whatever data
// streams it wants to deal with.
ogg_packet packet;
while (!headersDone &&
       (ret = ogg_stream_packetpeek(&stream->mState, &packet)) != 0) {
assert(ret == 1);

// A packet is available. If it is not a header packet we exit.
// If it is a header packet, process it as normal.
headersDone = headersDone || handle_theora_header(stream, &packet);
headersDone = headersDone || handle_vorbis_header(stream, &packet);
if (!headersDone) {
  // Consume the packet
  ret = ogg_stream_packetout(&stream->mState, &packet);
  assert(ret == 1);
}

To read packets for a particular stream I use a 'read_packet' function that operates on a stream passed as a parameter:

bool OggDecoder::read_packet(istream& is, 
                             ogg_sync_state* state, 
                             OggStream* stream, 
                             ogg_packet* packet) {
  int ret = 0;

  while ((ret = ogg_stream_packetout(&stream->mState, packet)) != 1) {
    ogg_page page;
    if (!read_page(is, state, &page))
      return false;

    int serial = ogg_page_serialno(&page);
    assert(mStreams.find(serial) != mStreams.end());
    OggStream* pageStream = mStreams[serial];
    
    // Drop data for streams we're not interested in.
    if (stream->mActive) {
      ret = ogg_stream_pagein(&pageStream->mState, &page);
      assert(ret == 0);
    }
  }
  return true;
}

If we need to read a new page (to be able to get more packets) we check the stream for the read page and if it is not for the stream we want we store the packet in the bitstream for that page so it can be retrieved later. I've added an 'active' flag to the streams so we can ignore streams that we aren't intersted in. We don't want to continuously buffer data for alternative audio tracks we aren't playing for example. The streams are marked inactive when the headers are finished reading.

The code that does the checking to see if it's time to display a frame is:


// At this point we've written some audio data to the sound
// system. Now we check to see if it's time to display a video
// frame.
//
// The granule position of a video frame represents the time
// that that frame should be displayed up to. So we get the
// current time, compare it to the last granule position read.
// If the time is greater than that it's time to display a new
// video frame.
//
// The time is obtained from the audio system - this represents
// the time of the audio data that the user is currently
// listening to. In this way the video frame should be synced up
// to the audio the user is hearing.
//
ogg_int64_t position = 0;
int ret = sa_stream_get_position(mAudio, SA_POSITION_WRITE_SOFTWARE, &position);
assert(ret == SA_SUCCESS);
float audio_time = 
  float(position) /
  float(audio->mVorbis.mInfo.rate) /
  float(audio->mVorbis.mInfo.channels) /
  sizeof(short);

float video_time = th_granule_time(video->mTheora.mCtx, mGranulepos);
if (audio_time > video_time) {
  // Decode one frame and display it. If no frame is available we
  // don't do anything.
  ogg_packet packet;
  if (read_packet(is, &state, video, &packet)) {
    handle_theora_data(video, &packet); 
    video_time = th_granule_time(video->mTheora.mCtx, mGranulepos);
  }
}

The code for decoding and display the Theora video is similar to the Theora decoding article. The main difference is we store the granulepos in mGranulepos so we know when to stop displaying the frame.

This version of 'plogg' should play Ogg files with a Theora and Vorbis track in sync. You can test it on the transformers trailer for example. It does not play Theora files with no audio track - we can't synchronise to the audio clock if there is no audio. This can be worked around by falling back to delaying for the required framerate as the previous Theora example did.

The a/v sync is not perfect however. If the video is large and decoding keyframes takes a while then we can fall behind in displaying the video and go out of sync. This is because we only play one frame when we check the time. One approach to fixing this is to decode, but not display, all frames up until the audio time rather than just the next time.

The other issue is that the API call we are using to write to the audio hardware is blocking. This is using up valuable time that we could be using to decode a frame. When the write to the sound hardware returns we have very little time to decode a frame before glitches start appearing in the audio due to buffer underruns. Try playing a larger video (like the Ghostbusters HD Trailer and the audio and video will skip (depending on the speed of your hardware). This isn't a pleasant experience. Because of the blocking audio writes we can't skip more than one frame due to the frame decoding time taking too long causing audio skip.

The fixes for these aren't too complex and I'll go through it in my next article. The basic approach is to move to an asynchronous method of writing the audio, skip displaying frames when needed (to reduce the cost of the YUV decoding), skip decoding frames if possible (depending on location of keyframes we can do this), and to check how much audio data we have queued before decoding to always ensure we won't drop audio while decoding.

With these fixes in place I can play the 1080p Ogg version of Big Buck Bunny on a Macbook laptop (running Arch Linux) with no audio interruption and with a/v syncing correctly. There is a fair amount of frame skipping however but it's a lot more watchable than if you try playing it without these modifications in place. And better than watching with the video lagging further and further behind the longer you watch it. Further improvements can be made to reduce the frame skipping by utilising threads to take advantage of extra core's on the PC.

After the followup article on improving the a/v sync I'll look at covering seeking.

Categories: ogg, theora, vorbis, firefox

Decoding Vorbis files with libvorbis

noreply@blogger.com (Chris Double) — Thu, 25 Jun 2009 03:52:00 +0000

Decoding Vorbis streams require a very similar approach to that used when decoding Theora streams. The public interface to the libvorbis library is very similar to that used by libtheora. Unfortunately the libvorbis documentation doesn't contain an API reference that I could find so I'm following the approached used by the example programs.

Assuming we have already obtained an ogg_packet, the general steps to follow to decode and play Vorbis streams are:

Call vorbis_synthesis_headerin to see if the packet is a Vorbis header packet. This is passed a vorbis_info and vorbis_comment object to hold the information read from those header packets. The return value of this function is zero if the packet is a Vorbis header packet. Unfortunately it doesn't return a value to see that it's a Vorbis data packet. To check for this you need to check if the stream is a Vorbis stream (by knowing you've previously read Vorbis headers from it) and the return value is OV_ENOTVORBIS.
Once all the header packets are read create a vorbis_dsp_state and vorbis_block object. Initialize these with vorbis_synthesis_init and vorbis_block_init respectively. These objects hold the state of the Vorbis decoding. vorbis_synthesis_init is passed the vorbis_info object that was filled in during the reading of the header packets.
For each data packet:
1. Call vorbis_synthesis passing it the vorbis_block created above and the ogg_packet containing the packet data. If this succeeds (by returning zero), call vorbis_synthesis_blockin passing it the vorbis_dsp_state and vorbis_block objects. This call copies the data from the packet into the Vorbis objects ready for decoding.
2. Call vorbis_synthesis_pcmout to get an pointer to an array of floating point values for the sound samples. This will return the number of samples in the array. The array is indexed by channel number, followed by sample number. Once obtained this sound data can be sent to the sound hardware to play the audio.
3. Call vorbis_synthesis_read, passing it the vorbis_dsp_state object and the number of sound samples consumed. This allows you to consume less data than vorbis_synthesis_pcmout returned. This is useful if you can't write all the data to the sound hardware without blocking.

In the example code in the github repository I create a VorbisDecode object that holds the objects needed for decoding. This is similar to the TheoraDecode object mentioned in my Theora post:

class VorbisDecode {
  ...
  vorbis_info mInfo;
  vorbis_comment mComment;
  vorbis_dsp_state mDsp;
  vorbis_block mBlock;
  ...

  VorbisDecode()
  {
    vorbis_info_init(&mInfo);
    vorbis_comment_init(&mComment);    
  }
};

I added a TYPE_VORBIS value to the StreamType enum and the stream is set to this type when a Vorbis header is successfully decoded:

  int ret = vorbis_synthesis_headerin(&stream->mVorbis.mInfo,
                      &stream->mVorbis.mComment,
                      packet);
  if (stream->mType == TYPE_VORBIS && ret == OV_ENOTVORBIS) {
    // First data packet
    ret = vorbis_synthesis_init(&stream->mVorbis.mDsp, &stream->mVorbis.mInfo);
    assert(ret == 0);
    ret = vorbis_block_init(&stream->mVorbis.mDsp, &stream->mVorbis.mBlock);
    assert(ret == 0);
    stream->mHeadersRead = true;
    handle_vorbis_data(stream, packet);
  }
  else if (ret == 0) {
    stream->mType = TYPE_VORBIS;
  }
}

The example program uses libsydneyaudio for audio output. This requires sound samples to be written as signed short values. When I get the floating point data from Vorbis I convert this to signed short and send it to libsydneyaudio:

  int ret = 0;
    
  if (vorbis_synthesis(&stream->mVorbis.mBlock, packet) == 0) {
    ret = vorbis_synthesis_blockin(&stream->mVorbis.mDsp, &stream->mVorbis.mBlock);
    assert(ret == 0);
  }

  float** pcm = 0;
  int samples = 0;
  while ((samples = vorbis_synthesis_pcmout(&stream->mVorbis.mDsp, &pcm)) > 0) {
    if (!mAudio) {
      ret = sa_stream_create_pcm(&mAudio,
                 NULL,
                 SA_MODE_WRONLY,
                 SA_PCM_FORMAT_S16_NE,
                 stream->mVorbis.mInfo.rate,
                 stream->mVorbis.mInfo.channels);
      assert(ret == SA_SUCCESS);

      ret = sa_stream_open(mAudio);
      assert(ret == SA_SUCCESS);
    }

    if (mAudio) {
      short buffer[samples * stream->mVorbis.mInfo.channels];
      short* p = buffer;
      for (int i=0;i < samples; ++i) {
    for(int j=0; j < stream->mVorbis.mInfo.channels; ++j) {
      int v = static_cast<int<(floorf(0.5 + pcm[j][i]*32767.0));
      if (v > 32767) v = 32767;
      if (v <-32768) v = -32768;
      *p++ = v;
    }
      }

      ret = sa_stream_write(mAudio, buffer, sizeof(buffer));
      assert(ret == SA_SUCCESS);
    }
    
    ret = vorbis_synthesis_read(&stream->mVorbis.mDsp, samples);
    assert(ret == 0);
  }
}

A couple of minor changes were also made to the example program:

Continue processing pages and packets when the 'end of file' is reached. Otherwise a few packets that are buffered after we've reached the end of the file will be missed.
After reading a page don't just try to read one packet, call ogg_stream_packetout until it returns a result saying there are no more packets. This means we process all the packets from the page immediately and prevents a build up of buffered data.

The code for this example is in the 'part3_vorbis' branch of the github repository. This also includes the Theora code but does not do any a/v synchronisation. Files containing Theora streams will show the video data but it will not play smoothly and will not be synchronised with the audio. Fixing that is the topic of the next post in this series.

Categories: firefox, ogg, theora, vorbis

Decoding Theora files using libtheora

noreply@blogger.com (Chris Double) — Wed, 24 Jun 2009 13:14:00 +0000

My last post covered read Ogg files using libogg. The resulting program didn't do much but it covered the basic steps needed to get an ogg_packet which we need to decode the data in the stream. The thing step I want to cover is decoding Theora streams using libtheora.

In the previous post I stored a count of the number of packets in the OggStream object. For theora decoding we need a number of different objects to be stored. I encapsulate this in a TheoraDecode structure:

class TheoraDecode { 
  ...
  th_info mInfo;
  th_comment mComment;
  th_setup_info *mSetup;
  th_dec_ctx* mCtx;
  ...
};

th_info, th_comment and th_setup_info contain data read from the Theora headers. The Theora stream contains three headers packets. These are the info, comment and setup headers. There is one object for holding each of these as we read the headers. The th_dec_ctx object holds information that the decoder requires to keep track of the decoding process.

th_info and th_comment need to be initialized using th_info_init and th_comment_init. Notice that th_setup_info is a pointer. This needs to be free'd when we're finished with it using th_setup_free. The decoder context object also needs to be free'd. Use th_decode_free. A convenient place to do this is in the TheoraDecode constructor and destructor:

class TheoraDecode {
  ...
  TheoraDecode() :
    mSetup(0),
    mCtx(0)
  {
    th_info_init(&mInfo);
    th_comment_init(&mComment);
  }

  ~TheoraDecode() {
    th_setup_free(mSetup);
    th_decode_free(mCtx);
  }   
  ...

The TheoraDecode object is stored in the OggStream structure. The OggStream stucture also gets a field holding the type of the stream (Theora, Vorbis, Unknown, etc) and a boolean indicating whether the headers have been read:

class OggStream
{
  ...
  int mSerial;
  ogg_stream_state mState;
  StreamType mType;
  bool mHeadersRead;
  TheoraDecode mTheora;
  ...
};

Once we get the ogg_packet from an Ogg stream we need to find out if it is a Theora stream. The approach I'm using to do this is to attempt to extract a Theora header from it. If this succeeds, it's a Theora stream. th_decode_headerin will attempt to decode a header packet. A return value of '0' indicates that we got a Theora data packet (presumably the headers have been read already). This function gets passed the info, comment, and setup objects and it will populate them with data as it reads the headers:

ogg_packet* packet = ...got this previously...;
int ret = th_decode_headerin(&stream->mTheora.mInfo,
                             &stream->mTheora.mComment,
                             &stream->mTheora.mSetup,
                             packet);
if (ret == TH_ENOTFORMAT)
  return; // Not a theora header

if (ret > 0) {
  // This is a theora header packet
  stream->mType = TYPE_THEORA;
  return;
}

assert(ret == 0);
// This is not a header packet. It is the first 
// video data packet.
stream->mTheora.mCtx = 
  th_decode_alloc(&stream->mTheora.mInfo, 
                  stream->mTheora.mSetup);
assert(stream->mTheora.mCtx != NULL);
stream->mHeadersRead = true;
...decode data packet...

In this example code we attempt to decode the header. If it fails it bails out, possibly to try decoding the packet using libvorbis or some other means. If it succeeds the stream is marked as type TYPE_THEORA so we can handle it specially later.

If all headers packets are read and we got the first data packet then we call th_decode_alloc to get a decode context to decode the data.

Once the headers are all read, the next step is to decode each Theora data packet. To do this we first call th_decode_packetin. This adds the packet to the decoder. A return value of '0' means we can get a decoded frame as a result of adding the packet. A call to th_decode_ycbcr_out gets the decoded YUV data, stored in a th_ycbcr_buffer object. This is basically an array of the YUV data.

ogg_int64_t granulepos = -1;
int ret = th_decode_packetin(stream->mTheora.mCtx,
                             packet,
                             &granulepos);
assert(ret == 0);

th_ycbcr_buffer buffer;
ret = th_decode_ycbcr_out(stream->mTheora.mCtx, buffer);
assert(ret == 0);
...copy yuv data to SDL YUV overlay...
...display overlay...
...sleep for 1 frame...

The 'granulepos' returned by the th_decode_packetin call holds information regarding the presentation time of this frame, and what frame contains the keyframe that is needed for this frame if it is not a keyframe. I'll write more about this in a future post when I cover synchronising the audio and video. For now it's going to be ignored.

Once we have the YUV data I use SDL to create a surface, and a YUV overlay. This allows SDL to do the YUV to RGB conversion for me. I won't copy the code for this since it's not particularly relevant to using the libtheora API - you can see it in the github repository.

Once the YUV data is blit to the screen the final step is to sleep for the period of one frame so the video can playback at approximately the right framerate. The framerate of the video is stored in the th_info object that we got from the headers. It is represented as the fraction of two numbers:

float framerate = 
  float(stream->mTheora.mInfo.fps_numerator) / 
  float(stream->mTheora.mInfo.fps_denominator);
SDL_Delay((1.0/framerate)*1000);

With all that in place, running the program with an Ogg file containing a Theora stream should play the video at the right framerate. Adding Vorbis playback is almost as easy - the main difficulty is synchronising the audio and video. I'll cover these topics in a later post.

Categories: firefox, theora, ogg

Reading Ogg files using libogg

noreply@blogger.com (Chris Double) — Wed, 24 Jun 2009 11:37:00 +0000

Reading data from an Ogg file is relatively simple. The file format is well documented in RFC 3533. I showed how to read the format using JavaScript in a previous post.

For C and C++ programs it's easier to use the xiph.org libraries. There are libraries for decoding specific formats (libvorbis, libtheora) and there is a library for reading data from Ogg files (libogg).

I'm prototyping some approaches to improve the performance of the Firefox Ogg video playback and while I'm at it I'll write some posts on using these libraries to decode/play Ogg files. Hopefully it'll prove useful to others using them and I can get some feedback on usage.

All the code for this is in the plogg git repository on github. The 'master' branch contains the work in progress player that I'll describe in a series of posts, and there are branches specific to the examples in each post.

The libogg documentation describes the API that I'll be using in this post. All that this example will do is read an Ogg file, read each stream in the file and count the number of packets for that stream. It prints the number of packets. It doesn't decode the data or do anything really useful. That'll come later.

You can think of an Ogg file as containing logical streams of data. Each stream has a serial number that is unique within the file to identify it. A file containing Vorbis and Theora data will have two streams. A Vorbis stream and a Theora stream.

Each stream is split up into packets. The packets contain the raw data for the stream. The process of decoding a stream involves getting a packet from it, decoding that data, doing something with it, and repeating.

That describes the logical format. The physical format of the Ogg file is split into pages of data. Each physical page contains some part of the data for one stream.

The process of reading and decoding an Ogg file is to read pages from the file, associating them with the streams they belong to. At some point we then go through the pages held in the stream and obtain the packets from it. This is the process the code in this example follows.

The first thing we need to do when reading an Ogg file is find the first page of data. We use a ogg_sync_state structure to keep track of search for the page data. This needs to be initialized with ogg_sync_init and later cleaned up with ogg_sync_clear:

ifstream file("foo.ogg", ios::in | ios::binary);
ogg_sync_state state;
int ret = ogg_sync_init(&state);
assert(ret==0);
...look for page...

Note that the libogg functions return an error code which should be checked, A result of '0' generally indicates success. We want to obtain a complete page of Ogg data. This is held in an ogg_page structure. The process of obtaining this structure is to do the following steps:

Call ogg_sync_pageout. This will take any data current stored in the ogg_sync_state object and store it in the ogg_page. It will return a result indicating when the entire pages data has been read and the ogg_page can be used. It needs to be called first to initialize buffers. It gets called repeatedly as we read data from the file.
Call ogg_sync_buffer to obtain an area of memory we can reading data from the file into. We pass the size of the buffer. This buffer is reused on each call and will be resized if needed if a larger buffer size is asked for later.
Read data from the file into the buffer obtained above.
Call ogg_sync_wrote to tell libogg how much data we copied into the buffer.
Resume from the first step, calling ogg_sync_buffer. This will copy the data from the buffer into the page, and return '1' if a full page of data is available.

Here's the code to do this:

ogg_page page;
while(ogg_sync_pageout(&state, &page) != 1) {
  char* buffer = ogg_sync_buffer(oy, 4096);
  assert(buffer);

  file.read(buffer, 4096);
  int bytes = stream.gcount();
  if (bytes == 0) {
    // End of file
    break;
  }

  int ret = ogg_sync_wrote(&state, bytes);
  assert(ret == 0);
}

We need to keep track of the logical streams within the file. These are identified by serial number and this number is obtained from the page. I create a C++ map to associate the serial number with an OggStream object which holds information I want associated with the stream. In later examples this will hold data needed for the Theora and Vorbis decoding process.

class OggStream
{
  ...
  int mSerial;
  ogg_stream_state mState;
  int mPacketCount;
  ...
};

typedef map StreamMap;

Each stream has an ogg_stream_state object that is used to keep track of the data read that belongs to the stream. We're storing this in the OggStream object that we associated with the stream serial number. Once we've read a page as described above we need to tell libogg to add this page of data to the stream.

StreamMap streams;
ogg_page page = ...obtained previously...;
int serial = ogg_page_serialno(&page);
OggStream* stream = 0;
if (ogg_page_bos(&page) {
  stream = new OggStream(serial);
  int ret = ogg_stream_init(&stream->mState, serial);
  assert(ret == 0);
  streams[serial] = stream;
}
else
  stream = streams[serial];

int ret = ogg_stream_pagein(&stream->mState, &page);
assert(ret == 0);

This code uses ogg_page_serialnoto get the serial number of the page we just read. If it is the beginning of the stream (ogg_page_bos) then we create a new OggStream object, initialize the stream's state with ogg_stream_init, and store it in out streams map. If it's not the beginning of the stream we just get our existing entry in the map. The final call to ogg_stream_pagein inserts the page of data into the streams state object. Once this is done we can start looking for completed packets of data and decode them.

To decode the data from a stream we need to retrieve a packet from it. The steps for doing this are:

Call ogg_stream_packetout. This will return a value indicating if a packet of data is available in the stream. If it is not then we need to read another page (following the same steps previously) and add it to the stream, calling ogg_stream_packetout again until it tells us a packet is available. The packet's data is stored in an ogg_packet object.
Do something with the packet data. This usually involves calling libvorbis or libtheora routines to decode the data. In this example we're just counting the packets.
Repeat until all packets in all streams are consumed.

while (..read a page...) {
  ...put page in stream...  
  ogg_packet packet;
  int ret = ogg_stream_packetout(&stream->mState, &packet);    
  if (ret == 0) {
    // Need more data to be able to complete the packet
    continue;
  }
  else if (ret == -1) {
    // We are out of sync and there is a gap in the data.
    // We lost a page somewhere.
    break;
  }

  // A packet is available, this is what we pass to the vorbis or
  // theora libraries to decode.
  stream->mPacketCount++;
}

That's all there is to reading an Ogg file. There are more libogg functions to get data out of the stream, identify end of stream, and various other useful functions but this covers the basics. Try out the example program in the github repository for more information.

Note that the libogg functions don't require reading from a file. You can use these routines with any data you've obtained. From a socket, from memory, etc.

In the next post about reading Ogg files I'll go through using libtheora to decode the video data and display it.

Categories: theora, ogg, firefox

Reading Ogg files with JavaScript

noreply@blogger.com (Chris Double) — Fri, 05 Jun 2009 00:51:00 +0000

On tinyvid.tv I do quite a bit of server side reading of Ogg files to get things like duration and bitrate information when serving information about the media. I wondered if it would be possible to do this sort of thing using JavaScript running in the browser.

The format of the Ogg container is defined in RFC 3533. The difficulty comes in reading binary data from JavaScript. The XMLHttpRequest object can be used to retrieve data via a URL from JavaScript in a page but processing the binary data in the Ogg file is problematic. The response returned by XMLHttpRequest assumes text or XML (in Firefox at least).

One way of handling binary data is described in this Mozilla Developer article. Trying this method out works in Firefox and I can download and read the data in the Ogg file.

Ideally I don't want to download the entire file. It might be a large video. I thought by handling the 'progress' event or ready state 3 (data received) I'd be able to look at the data currently retrieved. This does work but on each call to the 'responseText' attribute in these events Firefox copies its internal copy of the downloaded data into a JavaScript array. Doing this every time a portion of the file is downloaded results in major memory use and slow downs proving impractical for even small files.

I think the only reliable way to process the file in chunks is to use byte range requests and do multiple requests. Is there a more reliable way to do binary file reading via JavaScript using XMLHttpRequest? I'd like to be able to process the file in chunks using an Iteratee style approach.

I put up a rough quick demo of loading the first 100Kb of a video and displaying information from each Ogg packet. This probably works in Firefox only due to the workaround needed to read binary data. Click on the 'Go' button in the demo page. This will load transformers320.ogg and display the contents of the first Ogg physical page.

I decode the header packets for Theora and Vorbis. So the first page shown will show it is for a Theora stream with a given size and framerate. Clicking 'Next' will move on to the Next page. This is a Vorbis header with the rate and channel information. Clicking 'Next' again gets the comment header for the Theora stream. The demo reads the comments and displays them. The same for the Vorbis comment records. As you 'Next' through the file it displays the meaning of the granulepos for each page. It shows whether the Theora data is for a keyframe, what time position it is, etc.

Something like this could be used to read metadata from Ogg files, read subtitle information, show duration, etc. More interesting would be to implement a Theora and/or Vorbis decoder in JavaScript and see how it performs.

The main issues with doing this from JavaScript seem to be:

Handling binary data using XMLHttpRequest in a cross browser manner
Processing the file in chunks so the entire file does not need to be kept in memory
Files need to be hosted on the same domain as the page. tinyvid.tv adds the W3C Access Control headers so they can be accessed cross domain but it also hosts some files on Amazon S3 where these headers can't be added. As a result even tinyvid itself can't use XMLHttpRequest to read these files.

Categories: tinyvid, javascript, theora, firefox

Video for Everybody - HTML 5 video fallback

noreply@blogger.com (Chris Double) — Tue, 02 Jun 2009 00:46:00 +0000

Kroc Camen has made available Video for Everybody, an HTML snippet that uses HTML 5 video if it's available in the browser, otherwise falling back to different video playback options.

What's interesting about 'Video for Everybody' is it doesn't use scripting at all. It uses the fallback mechanism built into HTML. The video playback mechanism used, in order of availability in the browser, is:

HTML 5 <video>
Adobe Flash
Quicktime
Windows Media Player
Text explaining how to get video support if none of the above is available

The fallback options work across multiple browsers and even works on the iPhone. The 'Video For Everybody' page also goes through how to encode videos in Ogg and MP4 format.

Categories: video, firefox

Third Party Comment Engines

noreply@blogger.com (Chris Double) — Sat, 16 May 2009 06:53:00 +0000

I've been playing around with different commenting engines on tinyvid.

Currently I use Intense Debate. An example of a video using this comment system is here. I like this service a lot. You can sign on with OpenId, the moderation tools are good and it seems to be generally reliable.

Intense Debate has support for plugins that anyone can write. Potentially I could write a video plugin that allowed embedding <video> into comments for example.

But it does have some issues. Sometimes comments don't seem to appear in the right order. I'll get an email that a comment was left, visit the page, and it's nowhere. Somehow it gets buried amongst other older comments. This started happening when I switched the ordering of the comments to have the most recent comment at the top.

Sometimes, depending on the content of your comment, the engine gets confused and the comment is scrambled. This seems to happen if I edit a comment that has a URL in it. It converts the URL to an HTML <a href/> and when editing and saving seems to reconvert, or do some weirdness with it. I've lost comments through this. Long comments can get silently truncated or trashed.

I set up tinyvid so I can easily add different comment engines and switch them on and off for individual videos for testing. I recently tried js-kit. An example of video with js-kit comments for testing is here.

I like js-kit too. I'm using their 'rating' widget and that seems to work nicely. The comments are ok too. You can sign in with OpenId. But there is some weirdness there. If I sign in using OpenId, leave a comment, then revisit the page later, then the comments don't appear. I have to delete the js-kit cookie to have them appear again.

There is a limit to the size of the comments, similar in size to Intense Debate's, of 3,000 characters. I suspect this limit isn't a problem in practice however.

There's a lot to like about js-kit, apart from the OpenId issue, but I need to try it out a bit more to see what other issues it has.

The last one I tried was Disqus. Example here. The Disqus site lets you register using OpenId. However you can't login to leave a comment using OpenId. You must login using a Disqus Id that gets assigned to you when you've registered.

When I registered using OpenId, then tried to login to leave a comment it requested a password. I had to go back to the Disqus site, change my profile to add a password. This seems sub-optimal since the point of me using OpenId is to avoid multiple passwords and accounts.

Disqus requires anonymous posters to leave a valid email address. I couldn't find a way to turn this option off. I'm sure this will just result in people leaving bogus email addresses.

One step of the Disqus setup requires adding a script block to the page that does document.write. This won't work on XHTML sites. And I dislike document.write usage in general. Thankfully this step appears optional. It's for converting marked up items in the page to a comment count.

After leaving a comment Disqus does a full page refresh. This unfortunately results in the video on the page restarting and reloading.

Apart from those issues Disqus seemed quite nice. Simple to set up and configurable.

So the end result is I'm not sure which is best. I'll play around with them for a bit more and get feedback. Are there any other's worth trying?

Categories: tinyvid

Simple Nanojit Example

noreply@blogger.com (Chris Double) — Mon, 11 May 2009 11:45:00 +0000

I've been playing around with Nanojit, thinking about using it in a toy project of mine.

Nanojit is a library for generating machine code with backends for x86 and ARM. It's used by Tamarin and TraceMonkey for their just in time compilers.

The Mozilla documentation contains some documentation which gives an overview of what's there:

Currently Nanojit seems to be directly included in both the Tamarin and Mozilla repositories. To make it easy to reuse in my various test projects I split out the nanojit code and put it in a github repository. The build is a simple makefile and is only setup for x86 since that's that platform I'm trying it on for now.

In the 'example' directory of the nanojit github repository is the example from the Mozilla documentation. This is what I'm basing my usage of it from.

As a simple test of using it I wrote an expression evaluator. You enter simple mathematical expressions using addition, subtraction, multiplication and division at the command line and it compiles the expression into machine code using nanojit and calls it to get the result.

I started with an interpreter first and added the compiling once that was working. To parse the expressions I used Ian Piumarta's Parsing Expression Grammar library. I tweaked it a bit to be able to use it from C++. The grammar is simple and produces a basic Abstract Syntax tree:

Expr      = lhs:Product (  "+" rhs:Product  { lhs = new Add(lhs, rhs); }
                         | "-" rhs:Product  { lhs = new Subtract(lhs, rhs); }
                        )*                  { $$ = lhs }
Product   = lhs:Value   (  "*" rhs:Value    { lhs = new Multiply(lhs, rhs); }
                         | "/" rhs:Value    { lhs = new Divide(lhs, rhs); }
                        )* _                { $$ =lhs }
Value     = Number _
Number    = < [0-9]+ ('.' [0-9]+)?>     { $$ = new Number(atof(yytext)); }

Running this through the 'leg' program produces C code that can be included in the project to compile the expression entered by the user. The interpreter just walks over the AST, calling an 'eval' method, to produce the result. Code looks like:

float Number::eval()  { return value; }
float Add:eval()      { return lhs->eval() + rhs->eval(); }
float Subtract:eval() { return lhs->eval() - rhs->eval(); }
float Multiply:eval() { return lhs->eval() * rhs->eval(); }
float Divide:eval()   { return lhs->eval() / rhs->eval(); }
int main() {
  Object* result = parse();
  cout << "Result: " << result->eval() << endl;  
  return 0;
}

Converting this to generate machine code and call it is pretty easy. Instead of an 'eval' method I created a 'compile' method. 'compile' takes an object as an argument that I call methods on to generate Nanojit intermediate representation (LIR). It returns the resulting generated LIR instruction. The code for the Number AST object looks like:

LIns* Number::compile(LirWriter* writer) {
  return writer->insImmf(value);
}

The LirWriter object contains methods to generate LIR code. For Number we generate an immediate float value and return a pointer to the generated instruction.

The operators are implemented in a similar manner. I'll just show the code for Add:

LIns* Add::compile(LirWriter* writer)
{
  LIns* a = lhs->compile(writer);
  LIns* b = rhs->compile(writer);
  return writer->ins2(LIR_fadd, a, b);
}

Here we compile the left and right hand sides of the expression, saving pointers to the generated instructions. Then we generate a 'float add' LIR instruction. This is passed the instructions that were generated for the left and right parts of the expression. By calling compile on the topmost AST object we end up generating the code for the entire expression.

The code required to set up Nanojit and create the LirWriter is a bit more complicated. Most of what I did was copy and pasted from the example code. Basically I create a Fragmento object and a LirBuffer. Then each time an expression is entered I create a Fragment object which will contain the final generated code and a LirBufferWriter which is the base LirWriter object used by the 'compile' methods.

LirWriter's can be added to a LirBufferWriter to form a pipeline that instructions go through to do different things like optimisation or output debugging code before ending up in the LirBuffer. After this I generate a 'start' instruction, call the 'compile' method to do the rest, and generate a required epilogue instruction:

Fragmento *fragmento = new (gc) Fragmento(&core, 20);
LirBuffer *buf = new (gc) LirBuffer(fragmento, NULL);
...
Object* result = parse();
...
Fragment *f = new (gc) Fragment(NULL);
LirBufWriter writer(buf);
...
writer.ins0(LIR_start);
writer.ins1(LIR_fret, result->compile(&writer));
...
f->lastIns = writer.insGuard(LIR_loop, writer.insImm(1), rec_ins);

// Compile the fragment.
compile(fragmento->assm(), f);

// Call the compiled function.
typedef JS_FASTCALL float (*ExprFunction)();
ExprFunction fn = reinterpret_cast(f->code());
cout << "Result: " << fn() << endl;

The expression evaluator interpreter and compiler example can be downloaded and played with by cloning my 'play' repository on github. Once cloned you'll need to initialize and update the submodules to pull in the nanojit and peg third party libraries that it uses:

$ git clone git://github.com/doublec/play.git
$ cd play
$ git submodule init
$ git submodule clone

Run 'make' in the 'expr-interpreter' and 'expr-compiler' directories and run the 'expr' program that results. If you compile both nanojit and expr-compiler with DEBUG defined you get some debug output. It show's the LIR and the assembler code generated. For example, the LIR for: 2+3+4*2 is:

start
#40000000:0 /* 2 */
#40080000:0 /* 3 */
fadd1 = fadd #0x40000000:0, #0x40080000:0
#40100000:0 /* 4 */
#40000000:0 /* 2 */
fmul1 = fmul #0x40100000:0, #0x40000000:0
fadd2 = fadd fadd1, fmul1
ret fadd2

The assembly code generated for this is:

mov -8(ebp),0                  
mov -4(ebp),1073741824         
mov -16(ebp),0                 
mov -12(ebp),1074266112        
fldq -16(ebp)                  
fadd -8(ebp)                    f0(#0x40080000:0)
fstpq -8(ebp)                   f0(fadd1)
mov -16(ebp),0                 
mov -12(ebp),1074790400        
mov -24(ebp),0                 
mov -20(ebp),1073741824        
fldq -24(ebp)                  
fmul -16(ebp)                   f0(#0x40000000:0)
fadd -8(ebp)                    f0(fmul1)

It's easy to change the code to use integer math instead of floating point. Use LIR_ret instead of LIR_fret for the return, and use LIR_add, LIR_sub, etc. This is the assembly code generated for '2+3+4*2' using integer math:

mov eax,2                      
add eax,3                       eax(2)
mov ebx,4                       eax(add1)
mov ecx,2                       eax(add1) ebx(4)
mul ebx,ecx                     eax(add1) ecx(2) ebx(4)
add eax,ebx                     eax(add1) ebx(mul1)

Categories: nanojit

Theora encoder improvements

noreply@blogger.com (Chris Double) — Fri, 08 May 2009 00:17:00 +0000

A lot of work has been going on improving the Theora video encoder. Given a better encoder we end up with better quality Theora videos without the need to change the decoder in existing players. The project name for the Theora encoder improvements is 'Thusnelda'.

Periodic updates have been released by Xiph showing the progress of the new encoder. The latest update makes interesting reading and show's graphs comparing it against other Theora builds, and against x264 (an H.264 encoder).

The update show's that some great improvements have been made, and in some cases is comparible to the results from the x264 encoder. Greg Maxwell gives some more information in this reddit comment about the results.

Remember that the intent of this is not to compare Theora vs x264, or to say one is better than the other in the general case. It's to show the improvement of Theora vs older Theora versions. The fact that it compares favourably against x264 in this one case, using one particular test of quality, is a good result though.

Categories: video, firefox

Another tinyvid.tv update

noreply@blogger.com (Chris Double) — Thu, 09 Apr 2009 13:30:00 +0000

I've made some more changes and fixes to tinyvid.tv since my last update.

I've updated to the latest Factor version, picking up some bug fixes and improvements to the libraries that I use.

The look and feel of the site has a new logo and banner thanks to Hans Schmucker. There have been some great contributions and bug reports in the feedback part of the site from Hans and others.

Support for converting Daily Motion videos is now in place. This gives three sites that can be converted from now. The other two being YouTube and Vimeo. In all cases it tries to convert the highest quality video available from the site.

A number of bugs in the file uploads have been fixed. There still are a few annoyances in this area, mainly to do with not very good error reporting when an upload fails. I'm still playing around with this area.

Previously logging in was done with Facebook Connect. Now I've added support for Open ID as well as Facebook and others.

This support was easily added using the great service from RPX Now. I implemented server support in Factor very easily and it gives a nice authentication login interface. If you don't want to associate an existing account with your tinyvid.tv videos you can use a service like myopenid.com to get an Open ID login and use that to authenticate with tinyvid.

I'm trialing the bookmarklet written by Hans Schmucker as the main player on the site. It provides visibility of the controls outside the video area so you can always see them without obscuring the video, has a volume control, and allows resizing the video. If you want to try the browser's standard controls you can click on the 'download' link on the video page to do that. I may switch between this player and the native controls based on feedback or if problems are found.

I added an 'embed snippet' on the video page so you can copy/paste it into your own pages to have the video embedded.

If you have any ideas for things to add or improve let me know.

Categories: video, tinyvid

The Linux Link Tech Show on HTML 5 Audio and Video

noreply@blogger.com (Chris Double) — Thu, 02 Apr 2009 20:43:00 +0000

Yesterday I was a guest on The Linux Link Tech Show talking about Firefox and HTML 5 audio/video support. It was a great show with lots of interesting questions and discussions about video in the browser, the choice of Theora for the video codec, and different ways video can be used on the web.

The show is archived and you can download and listen to episode 292 recorded on the 1st April 2009. Vorbis and MP3 versions are available.

Categories: firefox, video

oggplayer: A simple Ogg player using liboggplay

noreply@blogger.com (Chris Double) — Wed, 25 Mar 2009 11:21:00 +0000

To help track down performance issues, and fix some a/v synchronisation bugs, I wrote a simple Ogg player in C++ using the same libraries and a similar approach to what I use in the Firefox video implementation.

The source to 'oggplayer' is available on github.

I've setup the repository to facilitate my experimenting with different approaches and different library versions. What this means is I've taken the, probably unusual, approach of having a 'thirdparty' directory containing git submodules for each of the Ogg based libraries I'm using. I configure and build these and link statically to them. This lets me use experimental versions of the Ogg libraries without dealing with 'shared library hell'.

This is all explained in the README, but just quickly these are the steps to clone oggplayer, and get the third party libraries from git:

$ git clone git://github.com/doublec/oggplayer.git
$ cd oggplayer
$ git submodules init
$ git submodules update

Once that's done you need to build the third party libraries:

$ cd thirdparty
$ sh build.sh

Run 'make' to build oggplayer itself. The third party libraries are built and installed in a 'local' subdirectory.

You'll notice the unix-centric leaning of these commands. I've only built on Linux. I'm using cross platform libraries so it should build with minor changes on Mac OS X. Mainly changing the audio library to link against in the Makefile. For Windows you'll probably need to use Cygwin or MSYS.

To run, just pass the filename on the command line:

$ ./oggplayer foo.ogg

Press ESC to exit the playback, and space bar to toggle full screen on supported platforms (those that SDL supports full screen).

These are the third party libraries I use, and include as submodules:

libogg
libvorbis
libtheora
liboggz
libfishsound
liboggplay
libsydneyaudio

You'll need the following system libraries to build:

Boost C++ libraries
SDL

There's not much error checking, I mostly use 'assert' to bail on an error. I don't want to spend time creating an awesome player, rather something I can use to try test out the way I'm using the third party libraries.

Interestingly with this example player I can play back the problematic HD video I mentioned previously. Part of this is the newer YUV conversion routines in liboggplay and the more lower level optimized SDL libraries for blitting the video data. I'm also not seeing audio buffer underruns which I see with some videos in Firefox. Time to do some investigating!

Categories: firefox, video

Very nice web technology demo

noreply@blogger.com (Chris Double) — Mon, 23 Mar 2009 01:57:00 +0000

This 'glimpse of future web technologies', all in one document is pretty nifty. Lots of different web stuff in there, including rotated videos, fancy CSS, and DFXP captions for videos.

Thanks to Sam Ruby for pointing to Philippe Le Hegaret's work.

Categories: firefox

Replacing

noreply@blogger.com (Chris Double) — Fri, 13 Mar 2009 01:39:00 +0000

[Update: Chris Blizzard has a neat screencast showing the bookmarklet and player replacement in action.]

There's been some interesting demo's of <video> usage and manipulation discussed in the TinyVid Feedback comments section.

One user, hansschmucker, has done some nice demonstrations of what <video> can offer.

For example, this bookmarklet can take a TinyVid video and replace the player using the built in controls with one written in JavaScript provided by the bookmarklet. The new interface includes seeking, play, pause and volume control. It looks very nice.

To use the bookmarket, visit the bookmarklet page, click on the button that appears there and bookmark the resulting page. Next visit a TinyVid video and visit the bookmarklet. The built in player will be replaced by the new one. This will also work on many pages that use video using the built in controls, for example this transparency camp video.

This example show's it's possible for users to create and customize their own video players and use it on sites of their choice which is pretty nice.

hansschmucker also provided some demo's of using <video> with <canvas> that allow you to view a video in 3D using red/cyan glasses. I've copied his comment from the feedback page below:

=== Using VIDEO as data source for 3D ===
I just wrote a little demo to demonstrate how powerful VIDEO elements can be as data source: This little script (it's 20 lines for god's sake) uses a video (which was also produced via JS by the way, thanks to Radiohead's choice of using simple CSV for the data, the power of Canvas and a bit of base64 decoding) to offset pixels in a randomly generated noise map, saving the original map as red and the altered image as red/blue.

You just have to get out your red/cyan glasses and you'll be able to enjoy the video in 3D. Lots more could be done with proper optimization (in fact, the whole thing may be done as a set of SVG filters) and just a little more attention to detail (random noise is possibly the worst thing you can use for a map), but it's still amazing how easily you can create something amazong with VIDEO.
...
Two more variants: uses a prerendered noise (looks much better) and warps the source image itself which probably makes most sense here.

Categories: tinyvid, video, firefox

tinyvid.tv updates

noreply@blogger.com (Chris Double) — Sat, 07 Mar 2009 02:16:00 +0000

I did some updates to tinyvid.tv, my test site for using the HTML 5 video element, over the last week or so.

I mentioned the first in a previous post, using Facebook Connect for authentication. All user accounts have an internal identifier which is associated with the Facebook account when the user chooses to authorize TinyVid. This means all activities that require user authentication require a Facebook account. Eventually I'll probably add other methods (OpenId, etc).

The main feature addition was uploads. If you are authenticated you can upload files. They are initially set to private visibility. You can edit that to make them public.

There are two ways of doing uploads. The first is via the standard HTML file input element. This requires the uploaded file to be in Ogg format already. I check this when the upload is complete and fail the upload if it is not valid.

The second is using the firefogg Firefox extension. This lets you upload a file in almost any video format. It transcodes the video to Ogg format before uploading. It provides a progress bar for the transcode, and the upload progress, providing a slightly better user experience than the first method.

File uploads are still in testing and I'll be enabling/disabling it as I find problems and fix them.

I changed the fallback mechanism so that browsers that don't have native video support will immediately use the Cortado Java applet instead of requiring a click through.

I added a page showing popular videos. It shows videos with the most views over the last 24 hours, and for all time (since I enabled this feature which is only a few days ago).

There is an account management feature that lists all the videos you've uploaded or converted. You can edit the metadata for these videos (title, description, private/public visibility) or delete them. These requires being authenticated.

Search was added. Currently it searches through the titles. I'll add searching through the description field later.

I fixed an issue that prevented Safari from playing some of the videos due to a bug in the Factor webserver. Safari users will need to install the XiphQT plugin to get Ogg support however.

Categories: tinyvid, video

User Authentication - OpenID and Facebook

noreply@blogger.com (Chris Double) — Fri, 27 Feb 2009 00:20:00 +0000

I'm still looking at the different options for tinyid to handle registration and authentication of users. My recent change added logging in using your Facebook account using Facebook Connect. Another option I'm considering is OpenID.

There's a good blog post comparing the Facebook vs the OpenID stack. The same weblog has a post on using OpenID in a popup without leaving the page, similar to how Facebook Connect does its magic. You can see a demo of this concept in action.

Categories: facebook, openid, tinyvid

Fallback options for HTML5 video

noreply@blogger.com (Chris Double) — Wed, 25 Feb 2009 22:24:00 +0000

The HTML5 <video> element only works on a few, often bleeding edge, browsers. For tinyvid I've been only supporting this subset of browsers. Recently I put in support for a simple fallback option for other browsers.

In browsers that don't support <video> the element is ignored. The contents of the element are still processed however. This means any HTML elements within <video>...</video> will display to the user on those browsers. The most basic fallback option is to tell the user that their browser doesn't support video:

<video>
  <p>pSorry, your browser doesn't support the video element<p>
</video>

In my blog in the past I've used the video element to display the Theora video, and fallback to a YouTube hosted video if the browser doesn't support it. This was done with code like:

<video src="/video_bling.ogg" controls>
  <object width="425" height="350">
    <param name="movie" 
              value="http://www.youtube.com/v/vvtdkxCIKC8"></param>
    <embed src="http://www.youtube.com/v/vvtdkxCIKC8" 
              type="application/x-shockwave-flash" 
              width="425"
              height="350"> </embed> 
  </object>>/video>

On <video> enabled browsers this uses built in support. On others it uses the Flash player to play the YouTube video.

It's possible to play Theora videos using a Java Applet called Cortado. This can be used as a fallback for users that have Java support but no HTML5 video support:

<video src="test.ogg" controls>
  <applet code="com.fluendo.player.Cortado.class" 
             archive="cortado.jar" 
             width="320"
             height="240">
    <param name="url" value="test.ogg"></param>
    <param name="duration" value="229.015"></param>            
  </applet>
</video>

The Cortado applet cannot be used for video's hosted on another domain however. They must be on the same domain as that serving the cortado applet.

These fallback options don't allow creating your own controls with JavaScript and using the nice HTML5 media API. They work best when you only want a single fallback option and use the built-in controls.

There are JavaScript libraries that can be used to emulate the HTML5 media API. Or you can write a simple wrapper to do this yourself. The basic approach is to use <video> in your page and insert a JavaScript file at the top. When loaded this fill searches the document for usages of <video>. It detects what capabilities the browser has (native video support, Flash, Cortado, mplayer/vlc plugin, etc) and replaces the <video> element with the correct HTML code to instantiate the fallback. The JavaScript can then create a wrapper object that emulates the HTML5 API and calls the underlying plugin methods, etc if possible.

Examples of this type of library can be seen in use:

I've added very simple fallback to tinyvid using the Cortado applet. If the browser doesn't support <video> it displays a link the user can click to go to a page that uses the applet. The server decides whether to use native video or the applet depending on a URL parameter. Here's a link to a native video player, and the applet version:

For the applet version the server queries the Ogg file for the width, height and duration of the video so it can embed that data in the applet parameters. In some cases I can't get the width and height. This is due to some of the videos being stored on backup storage (via Amazon S3) and I can't locally analyze it to get the data. In that case the applet dimensions are set to 320x240.

Categories: video

Testing Facebook Connect Locally

noreply@blogger.com (Chris Double) — Wed, 25 Feb 2009 22:14:00 +0000

I've been experimenting with integrating features from the social networking site Facebook into tinyvid. To do this I'm using the 'Facebook Connect' API.

When you register a Facebook application you need to provide URL's to your application's site so Facebook knows where to send the user on redirects from logging in, and handle messaging between Facebook and your application.

To handle the case of being able to develop and test on a local server, and deploying to the live server. This is the 'Callback URL' setting in the application setup. This can contain an IP address and port number so for testing locally you can set it to a local IP address. For example, http://127.0.0.1:9000.

Facebook doesn't try to contact this directly, it redirects the browser to it, so this works fine when testing the application locally. You can create two applications, one with the testing Callback URL, and one with the production one, and make this a parameter in your application.

Another option that worked for me was to set the Callback URL to the production domain (in this case, tinyvid.tv) and locally set my hosts file so that tinyvid.tv resolved to localhost. The web server needs to run in port 80 locally for this to work.

Categories: facebook

Daily Motion, OLPC and Theora

noreply@blogger.com (Chris Double) — Wed, 28 Jan 2009 01:06:00 +0000

A while back it was announced that Daily Motion, an online video site, had opened an OLPC channel for sharing videos encoded using Theora for playback on OLPC's.

The channel, http://olpc.dailymotion.com, contains theora videos aimed at the OLPC audience. What's nice is that the videos playback in Firefox 3.1 using the native Theora support and don't require a plugin, for example this video.

Looking at the Daily Motion page it seems they use the <object> element to playback the Ogg Theora file, which uses the internal decoder and player user interface. This is a nice result from the adding of support for direct loading of Ogg files that Robert worked on.

Categories: firefox, theora, video