Bluish Coder

Programming Languages, Martials Arts and Computers. The Weblog of Chris Double.


2011-08-08

Temporal Media Fragment Support in Firefox

The W3C has a Media Fragments Working Group whose mission is to specify temporal and media fragments in the Web using URI’s. The draft specification goes through in detail how these fragments work. I recently became a member of the working group and I’ve been working on adding support for the temporal dimension portion of the specification to Firefox.

In the most basic form you can specify a start time and an end time in the fragment part of a URI in an HTML video or audio element. For example:

<video src='/mediafrag/AudioAPI.webm#t=50,100'></video>

The ‘#t’ portion of the URI identifies the fragment as being a temporal media fragment. In this example ‘50,100’ means start the video at a current time of 50 seconds, and stop playing at 100 seconds. There are various other formats for the temporal media fragment defined in the specification. Examples can be seen in the UA Test Cases.

Development of this feature is being done in bug 648595. I’ve done test builds of the first iteration of the patch and they are available at my Firefox media fragment test builds page. The page has builds, an example, and a list of limitations which are currently:

  • Temporal syntax only. This means no spatial or track dimensions.
  • NPT time support only. No SMPTE time codes or Wall-clock time codes.
  • When changing the media fragment portion of a URL the media is not immediately updated. You need to refresh the page to see the change. This is most noticeable when directly navigating to the video and adding or changing a fragment.
  • The user interface for identifying the fragment in the standard controls is ugly and needs polish.
  • The HTML standard includes an ‘initialTime’ attribute for obtaining the start time. There is no way to obtain an end time so I’ve exposed a ‘mozFragmentEnd’ attribute on the video DOM object.
Tags: mozilla  video 

2011-08-08

Safe destruction in the presence of sharing in ATS

While using the libevent API from ATS I came across a scenario where it was important to call a function to release objects in a particular order. I wanted to have ATS enforce at compile time that the destruction occurs safely in the right order. The following example uses the built in libevent HTTP API for creating simple web servers. It uses the ATS libevent wrapper I wrote about previously.

fn cb {l1,l2:agz} (request: !evhttp_request l1, arg: !event_base l2): void = let
  val () = printf("here\n", @())
in
 ()
end

fn server () = let
  val _ = signal (SIGPIPE, SIG_IGN)
  val base = event_base_new ()
  val () = assert_errmsg (~base, "event_base_new failed")

  val http = evhttp_new (base)
  val () = assert_errmsg (~http, "evhttp_new failed")

  val r = evhttp_set_cb_with_base (http, "/", cb, base)
  val () = assert_errmsg (r = 0, "evhttp_set_cb_with_base failed")

  val r = evhttp_bind_socket (http, "0.0.0.0", uint16_of_int(8080))
  val () = assert_errmsg (r = 0, "evhttp_bind_socket failed")

  val r = event_base_dispatch (base)
  val () = assert_errmsg (r >= 0, "event_base_dispatch failed")

  val () = evhttp_free (http)
  val () = event_base_free (base)
in
  ()
end

implement main () = server ()

The call to http_new requires an event_base object. Internally, inside the C API, the returned evhttp object holds a pointer to this event_base. This results in the pointer being shared in two places and requires careful destruction to prevent using a destroyed object.

Later I call evhttp_free to destroy and release resources associated with the evhttp object, followed by event_base_free to do the same with the event_base object. I do it in this order to prevent a dangling reference to the event_base inside the evhttp object. The evhttp object is associated with that particular event_base and it’s important not to pass an incorrect base to functions that use the http object. Ideally code like the following shouldn’t compile:

val base = event_base_new ()
val base2 = event_base_new ()
val http = evhttp_new (base)

// Wrong event_base passed, fail compilation
val r = evhttp_set_cb_with_base (http, "/", cb, base2)

// Destruction out of order, fail compilation
val () = event_base_free (base)
val () = evhttp_free (http)

To model this in ATS I changed the ATS definition of evhttp to have an event_base associated with it at the type level and modified evhttp_new to return this relationship (agz is an address that cannot be NULL, agez is an address that can be NULL):

absviewtype evhttp (l:addr, l2:addr)
fun http_new {l1:agz} (base: !event_base l1): [l2:agez] evhttp (l2,l1) = "mac#evhttp_new"

With this change the type for evhttp depends on the value of the pointer to the event_base given as an argument. Now we can use this in the definition of evhttp_set_cb_with_base to ensure that the correct event_base object is used:

fun evhttp_set_cb_with_base {l1,l2:agz}
       (http: !evhttp (l1, l2),
        path: string,
        callback: {l3:agz} (!evhttp_request l3, !event_base l2) -> void,
        arg: !event_base l2): int = "mac#evhttp_set_cb"

This definition states that the event_base given as the arg paramater, and as the second argument to the callback, must be the same as that associated with the http object. This is done by using the same l2 dependent type argument with these types. This gives us the desired error checking in the first ‘fail compilation’ test above.

The next step is to prevent out of order destruction. This is done by passing the event_base object as a proof argument to the evhttp_free function:

fun evhttp_free {l1,l2:agz} (base: !event_base l1 | http: evhttp (l2, l1)): void = "mac#evhttp_free"

By adding this as proof argument I’m stating that the caller of evhttp_free must have a usable event_base object that was used to create this evhttp object to prove we can safely destroy it. The following is now a compile time error:

val () = event_base_free (base)
val () = evhttp_free (base | http)

This is due to event_base_free consuming the base linear object. The type of base following this call is no longer defined. The evhttp_free call is now an error due to using an undefined base. The following will work:

val () = evhttp_free (base | http)
val () = event_base_free (base)

This method of ensuring correct order of destruction and resource usage requires no changes to the libevent C code. Everything occurs during type checking. The generated C code looks exactly like normal libevent C usage with no runtime overhead for tracking the association between evhttp and event_base objects.

Tags: ats 

2011-08-08

WDCNZ HTML Media Presentation

Last month I went down to Wellington to give a joint talk at the WDCNZ conference. The topic was “HTML Media: Where we are and where we need to go”. The talk was shared between Nigel Parker, Mobile and Developer Evangelist for Microsoft, and myself. The conference was excellent with some great speakers and talks.

Nigel and I covered using HTML video and audio elements and how they can be used today across multiple browsers and mobile devices. We also covered upcoming API’s and directions in the web media area. Demo’s were shown on Microsoft Internet Explorer 9, Windows Phone 7 (running the Mango update which has a browser that supports HTML media) and Firefox. Nigel has slides and a summary in his blog post about the talk. I think it took a bit for attendees to get over the shock over a Mozilla and Microsoft representative sharing the stage and working together! Nigel’s post explains how this came about.

I spoke to a few Kiwi developers afterwards about using HTML video and there was a fair bit of interest in using it. The main obstacles seemed to be people unsure what codec to use and wanting support for adaptive streaming. The first is an education issue, getting people aware of what codecs to support for maximum coverage across browsers and how to encode to those formats. With regards to adaptive streaming there has been discussion between interested parties in various mailing lists and groups - it’s definitely something that is wanted.

At the time of the talk Nigel and I weren’t able to find any existing New Zealand based sites that use HTML video. Hopefully this will change in the future and Microsoft New Zealand are leading by example by using HTML video on their own site.

Tags: mozilla  video 

2011-05-12

Namecoin - A DNS alternative based on Bitcoin

Namecoin is a domain name system based on Bitcoin. It extends Bitcoin to add transactions for registering, updating and transferring names. The idea behind this is to provide an alternative to the existing DNS system where names can be taken from their owners by groups that control the DNS servers.

The project was originally announced in the bitcoin forums and has seen some uptake. The namecoin author, vinced, states in the post:

  • This is a new blockchain, separate from the main Bitcoin chain
  • Name/value pairs are stored in the blockchain attached to coins
  • Names are acquired through new transaction types - new, first-update and update
  • Names expire after 12000 blocks unless renewed with an update
  • No two unexpired names can be identical
  • Block validation is extended to reject transactions that do not follow the above rules
  • The code is here: https://github.com/vinced/namecoin

A number of projects have been created around this to provide a mapping from namecoin names to standard DNS. This allows resolving namecoin names to a ‘.bit’ suffixed domain. I’ll go through building the namecoin software, registering and updating names, then the software to use these names.

Building Namecoin

Namecoin needs to be built from source. The following steps on a Linux based system will build without UPNP support:

$ git clone git://github.com/vinced/namecoin.git
$ cd namecoin
namecoin $ make -f makefile.unix USE_UPNP=

Once built you’ll need to create a ~/.namecoin/bitcoin.conf file that contains entries for a username and password used for the JSON-RPC server that namecoind runs. Notice the name of the .conf file is bitcoin.conf even though this is namecoind. It won’t clash with an existing bitcoin installation as it is in a ~/.namecoin directory. To prevent conflict with an existing bitcoin install I suggest running namecoind on a different port. An example ~/.namecoin/bitcoin.conf is:

rpcuser=me
rpcpassword=password
rpcport=9332

Running namecoind will start the daemon and you can then use namecoind to execute commands:

$ ./namecoind
bitcoin server starting
$ ./namecoind getblockcount
2167

Yes, it prints out ‘bitcoin server starting’. There are still bitcoin references in the code that need to be changed apparently.

Getting Namecoins

To register a name you need to have some namecoins. These can be obtained via mining, just like bitcoins. Or you can buy them. To mine namecoins you can run any of the standard bitcoin miners and point them to the server and port that is running namecoind. The difficulty level for namecoin mining is currently very low (about 290 at the time of writing) so even CPU miners have a chance. Generating a block gets you 50 namecoins.

You can also buy namecoins as described here. The going rate seems to be about 1BTC for 50 namecoins.

Registering a name

The name_new command will register a name. An example invocation is:

$ ./namecoind name_new d/myname
[
    "1234567890123456789012345678901234567890",
    "0987654321"
]   

This will start the registration process for the name myname. Note the two hash values returned. Once this is done you need to wait for 12 blocks to be generated by the namecoin network. You then need to run a name_firstupdate command:

$ ./namecoind name_firstupdate d/myname 0987654321 '{"map":{"":"1.2.3.4"}}'

We pass to name_firstupdate the domain name we are updating, the shorter hash that we got from name_new and a JSON value that defines how that name is mapped to an IP address.

In this case the name is mapped to the IP address 1.2.3.4. Using the existing systems for mapping names this would make myname.bit resolve to 1.2.3.4. You can also do subdomains (See the update example later).

The cost to do a name_new, followed by a name_firstupdate, varies depending on how many blocks there are in the namecoin block chain. It started at 50 namecoins and slowly reduces. The formula is defined in the namecoin design document as:

  • Network fees start out at 50 NC per operation at the genesis block

  • Every block, the network fees decreases based on this algorithm, in 1e-8 NC:

      res = 500000000 >> floor(nBlock / 8192)
      res = res - (res >> 14)*(nBlock % 8192)
  • nBlock is zero at the genesis block

  • This is a decrease of 50% every 8192 blocks (about two months)

  • As 50 NC are generated per block, the maximum number of registrations in the first 8192 blocks is therefore 2/3 of 8192, which is 5461

  • Difficulty starts at 512

Updating a name

To update the domain mapping you use name_update:

$ ./namecoind name_update d/myname '{"map":{"":"1.2.3.4","www":"5.6.7.8"}}'

This example updates the value of myname so it includes a www subdomain. The name www.myname.bit will now map to 5.6.7.8.

There are other possibilities for the JSON mapping. See the namecoin README for details. Note that the JSON code must be valid JSON (ie. use double quotes, unlike the examples currently shown in the README unfortunately).

Transferring a name

To transfer a name to another person you need to get their namecoin address and do an update passing that address:

$ ./namecoind name_update d/myname '{"map":{"":"1.2.3.4"}}' NGZs7UndoWgpfTstoxryYEW8b1GtDLPwMa

Addresses can be generated with:

$ ./namecoind getnewaddress
N9jzzaptnQ28uiLgWm19WZAqrGqRVVGkFX

Transferring namecoins

You can transfer namecoins to other people by sending coins to their address just like bitcoin:

$ ./namecoind sendtoaddress N9jzzaptnQ28uiLgWm19WZAqrGqRVVGkFX 50

This will send 50 namecoins to N9jzzaptnQ28uiLgWm19WZAqrGqRVVGkFX.

Listing registered names

You can list all registered namecoin names using name_scan:

$ ./namecoind name_scan
{
    "name" : "d/bluishcoder",
    "value" : "{\"map\":{\"\":\"69.164.206.88\"}}",
    "txid" : "....",
    "expires_in" : 10874
},

You can also list only the names you’ve registered using name_list:

$ ./namecoind name_list
{
    "name" : "d/bluishcoder",
    "value" : "{\"map\":{\"\":\"69.164.206.88\"}}",
    "expires_in" : 10874
}

Using namecoin names

Software needs to be modified to use namecoind to lookup the name, or you can run DNS software that connects to namecoin to do lookups. To be able to try out namecoin I modified an HTTP proxy and later tried using DNS software.

HTTP Proxy

I modified the Polipo web proxy to use namecoin for lookups. The modified source is available at https://github.com/doublec/namecoin-polipo. This can be built and run with:

$ git clone https://github.com/doublec/namecoin-polipo
$ cd namecoin-polipo
$ make
$ ./polipo namecoindServer="127.0.0.1:9332" namecoindUsername=rpcuser namecoindPassword=rpcpassword

Changing your browser to point to the proxy on localhost, port 8123, will allow .bit domains to be used. See my forum post about it for more details.

dnsmasq

Another approach I tried was to write a program that generates a ‘host file’ from namecoind and uses dnsmasq to run a local DNS server that serves domains from this host file, falling back to the standard DNS server. The ‘quick and dirty’ code to generate the hosts file is in namecoin-hosts.c and uses libcurl and libjansson to build:

$ gcc -o namecoin-hosts namecoin-hosts.c -lcurl -ljansson

I added the following to my dnsmasq.conf:

local=/.bit/
local-ttl=300
addn-hosts=/tmp/hosts.txt

And created a shell script to update /tmp/hosts.txt with the namecoin related data:

while true; do
  ./namecoin-hosts 127.0.0.1:9332 rpcuser rpcpassword >/tmp/hosts.txt
  kill -HUP `cat /var/run/dnsmasq/dnsmasq.pid`
  echo `date`
  sleep 300
done

Pointing my OS DNS resolver to the dnsmasq IP address and port allowed .bit names to resolve.

Public .bit DNS servers

Details of a public .bit DNS server that doesn’t require you to run namecoin are available at namecoin.bitcoin-contact.org. That site also provides details on using namecoin.

More Information

Namecoin seems to be very much an experiment in having an alternative DNS like system. The developer has taken the approach of ‘release early’ and iterate towards a solution. As such it may fizzle out and go nowhere. Or it may prove a useful test-bed for ideas that make it into a successful DNS alternative.

More details about Namecoin can be obtained from:

Are there any other alternatives to DNS around with similar ideas?

Tags: namecoin  bitcoin 

2011-05-11

Building Mozart/Oz on Windows

The Mozart Programming System is an implementation of the Oz programming language. It’s the language used in the book Concepts, Techniques, and Models of Computer Programming by Peter Van Roy and Seif Haridi. From the Mozart website:

Mozart is based on the Oz language, which supports declarative programming, object-oriented programming, constraint programming, and concurrency as part of a coherent whole. For distribution, Mozart provides a true network transparent implementation with support for network awareness, openness, and fault tolerance. Mozart supports multi-core programming with its network transparent distribution and is an ideal platform for both general-purpose distributed applications as well as for hard problems requiring sophisticated optimization and inferencing abilities.

The last release of Mozart was a couple of years ago and the steps to build on Windows no longer seem to work. It required Cygwin to build but used the MingW compiler to get a native Windows build.

Mozart/Oz has recently seen a bit of a life with activity in the mailing lists and a move to github for source control and issue tracking. I was working on a project that needed Windows and Linux support so thought I’d have a try at converting Mozart to compile using MingW on Windows.

I’ve got a basic build working and have patches that I hope will eventually be merged. In the meantime I thought I’d post about how to build using MingW with my patches. The following steps will build Mozart, the standard library, Emacs (used as the IDE) and Tcl/Tk (for GUI).

  1. Download and install mingw-get-inst. Install the ‘MSYS Basic System’ and the ‘C++ Compiler’.

  2. From the ‘MingW Shell’, run the following commands to install the required support packages:

     $ mingw-get install mingw32-libgmp
     $ mingw-get install mingw32-gmp
     $ mingw-get install mingw32-libz
     $ mingw-get install msys-libgdbm
     $ mingw-get install msys-libregex
     $ mingw-get install mingw32-autoconf2.1
     $ mingw-get install mingw32-autoconf
     $ mingw-get install msys-flex
     $ mingw-get install msys-bison
     $ mingw-get install mingw32-gcc-g++
     $ mingw-get install msys-wget
  3. Download and install the msysgit package. I installed it in C:/git. It’s best to install it in a directory that doesn’t have spaces in the name.

  4. Make a directory to store the resulting ‘Mozart/Oz’ binaries. I used /p to keep my command lines short but you could also use /mingw to install alongside the existing MingW programs:

     $ mkdir /p
  5. Download and build Tcl/Tk 8.5 from source, configured to install in the directory above:

     $ wget http://prdownloads.sourceforge.net/tcl/tcl8.5.9-src.tar.gz
     $ wget http://prdownloads.sourceforge.net/tcl/tk8.5.9-src.tar.gz
     $ tar xvf tcl8.5.9-src.tar.gz
     $ tar xvf tk8.5.9-src.tar.gz
     $ cd tcl8.5.9
     tcl8.5.9 $ ./win/configure --prefix=/p
     tcl8.5.9 $ make && make install
     tcl8.5.9 $ cd ../tk8.5.9
     tk8.5.9 $ ./win/configure --prefix=/p --with-tcl=`pwd`/../tcl8.5.9
     tk8.5.8 $ make && make install
  6. Download and build emacs from source:

     $ wget http://ftp.gnu.org/gnu/emacs/emacs-23.3.tar.gz
     $ tar xvf emacs-23.3.tar.gz
     $ cd emacs-23.3
     emacs-23.3 $ cd nt
     emacs-23.3/nt $ cmd /c "configure --prefix=/p --without-xpm \
                      --without-png --without-jpeg --without-tiff --without-gif"
     emacs-23.3/nt $ make && make install
  7. Build my win32_build branch of Mozart. Note that I add the directory where I installed Git into the path. I also add the ‘bin’ directory of where I set Mozart to be installed.

     $ export PATH=$PATH:/c/git/bin:/p/bin
     $ git clone git://github.com/doublec/mozart.git
     $ cd mozart
     mozart $ git checkout -b win32_build origin/win32_build
     mozart $ cd ..
     $ mkdir build
     $ cd build
     build $ windlldir=/p ../mozart/configure --prefix=/p \
              --with-inc-dir=/p/include --with-lib-dir=/p/lib \
              --with-tcl=/p --with-tk=/p --disable-contrib-compat \
              --disable-contrib --enable-modules-static \
              --disable-doc --disable-chm
     build $ make && make install
  8. Build my win32_build branch of the Mozart standard library:

     $ git clone git://github.com/doublec/mozart-stdlib.git
     $ cd mozart-stdlib
     mozart-stdlib $ git checkout -b win32_build origin/win32_build
     mozart-stdlib $ cd ..
     $ mkdir build-stdlib
     $ cd build-stdlib
     build-stdlib $ ../mozart-stdlib/configure --prefix=/p
     build-stdlib $ make && make install
  9. You can now test Mozart by running the comand oz. This should start emacs with a Mozart/Oz system running. You can evaluate an example by entering {Browse 1+1} into the topmost emacs pane and evaluating with Ctrl+. Ctrl+l. You’ll need to set the OZEMACS environment variable to point to the location of emacs:

     $ export OZEMACS=/p/bin/emacs.exe
     $ oz

That’s a large number of steps but it gives a complete Mozart/Oz environment. From there you can work through the tutorial. There’s work to be done to make it easier and get more testing. One contributor is looking at creating a Visual Studio project to do the builds as well as to improve on the basic MingW support I’ve got working, so there’s hope for less steps in the future.

The Mozart/Oz system is interesting and there’s some neat projects written in it. A short list of some of them:

  • BeerNet, a P2P network.
  • TransDraw, a distributed shared drawing program. I run a live transdraw instance which you can connect too. Instructions here.
  • Roads, a web application framework.
  • EBL/Tk, an UI library that can do migration of user interface elements across the network.
Tags: mozartoz 


Tags

Archives
Links