Bluish Coder: mozilla

Travelling to Pitcairn Island

2012-04-11T12:00:00+12:00

From Sunday 15th April through to 29th April I’ll be mostly offline as I take some leave to visit Pitcairn Island, one of the remotest inhabited islands with a population of about 50 people.

My first stop is flying from New Zealand to Tahiti where I spend a couple of days, then I fly to Mangareva on the 17th to board the Xplore sailing yacht for the approximately two day trip to Pitcairn. I spend a couple of days on the island itself, then return on the Xplore back to Mangareva, followed by flying back to Tahiti for a few more days.

The trip was easy to organise through Pitcairn Travel. Longer trips are available than the one I’m taking but none are scheduled at this time of year. Assuming this trip goes well I hope to go for longer, and maybe to the other Islands in the Pitcairn group, in the future.

Why Pitcairn? Pitcairn is the island that was settled by the Bounty mutineers. My grandmother was born on the island and through her I’m a descendant of three mutineers (Fletcher Christian, John Mills, Ned Young and their Tahitian wives are my sixth great grandparents). I’m looking forward to visiting the Bounty monument in Tahiti and the Bounty Plaque on Pitcairn.

Electricity is available on Pitcairn for about 10 hours per day which limits laptop/gadget usage time. Luckily I plan to spend as much time as possible exploring the island, weather permitting.

I’ll have internet access while in Tahiti but I suspect access to be a bit hit or miss on Pitcairn. In the past internet access was available by sharing satellite internet that was provided by a United States Geologic Survey station on the island. A description of the setup is available here.

Later the Pitcairn Island Government arranged their own satellite internet capability. Recently speeds have been improved to 512 kilobits per second - shared amongst the approximately 50 people on the island. Costs for residents of the island are around $40 per 400MB of usage from what I hear. I would imagine that if someone wanted to regularly access the internet there for work they’d require a dedicated satellite internet connection just for that (Something like Pactel’s VSAT internet maybe). I’ll be sure to do a later post on what using the modern web is like in this part of the world.

Anyone in the area of Tahiti, Mangareva or Pitcairn, let me know, I’d be keen to meet.

Building and Running Boot To Gecko on the Nexus S

2012-03-20T12:00:00+13:00

Update 2012-03-29 - The Nexus S port has moved to an ICS base system and the existing Gingerbread base no longer works correctly. I’ve adjusted the instructions below to build the ICS based system. It just involves using ‘config-nexuss-ics’ instead of ‘config-nexus’.

Last year Mozilla announced the Boot to Gecko project - a mobile OS based on web technologies. Recently it was demoed at MWC 2012. Work is being done to improve video playback on B2G using hardware codecs in bug 714408.

I’ve built and run B2G on the emulator before but I wanted to try it out on real hardware to test the video support and play around with the OS. I upgraded my main phone to a Galaxy Note recently leaving a my Nexus S spare for trying different ROMS on it. Support for the Nexus S has started becoming available for B2G (previously the main consumer phone for testing was the Galaxy S II) so I gave it a try. The Nexus S I have is the GSM (non-4G) version. The steps to get the source code:

$ git clone git://github.com/andreasgal/B2G
$ cd B2G
$ make sync

This takes a long time (on New Zealand networks anyway…). Multiple gigabytes of git submodules are cloned.

You’ll want to make sure you have a build environment set up, as per this MDN article so that ‘adb’ and other android tools work. Once done, configure for a Nexus S build:

$ make config-nexuss-ics

This will download binaries for the phone and get you to confirm a bunch of licenses. To build:

$ make gonk
...
$ make
...

The ‘gonk’ make invocation builds the underlying android layer. The following ‘make’ builds gecko and related parts of B2G. Once those are completed you can flash the phone with the result. Note, you do the following at your own risk! You’re flashing your phone, overwriting everything, with experimental, possibly buggy software.

To flash a Nexus S you need to have unlocked the bootloader. If you haven’t done this yet, boot into the bootloader (hold the up volume key down while pressing the power button) and run:

$ make unlock-bootloader

This runs “fastboot oem unlock” which is the command to unlock the Nexus S bootloader. You’ll need to agree to it on the phone. I also installed the CyanogenMod recovery firmware but this is optional. Instructions to do that are here.

To flash, boot the phone into recovery mode (Hold down the up volume key while pressing power, from the menu that appears choose ‘Recovery’), plugin in the USB cable to your PC. Run:

$ make flash-only

‘flash-only’ will flash your B2G build onto the phone. You’ll lose everything on the phone, sorry.

If your Nexus S was running a 2.3 based Android this should just work. If it was running ICS (as mine was) then you might get an error about unsupported baseband and/or bootloader versions. If you get this, I fixed it by editing ‘glue/gonk/device/samsung/crespo/board-info.txt’ to add the versions that my bootloader and baseband had. My file looked like:

require board=herring
require version-bootloader=I9020XXJK1|I9020XXKA3|I9020XXKL1
require version-baseband=I9020XXJK8|I9020XXKB1|...|I9020UCKE1|I9020XXKI1

Note the addition of ‘I9020XXKL1’ and ‘I9020XXKI1’ to the bootloader and baseband lines respectively. After editing I redid “make gonk”, “make” and “make flash-only”.

After ‘flash-only’ your phone will reboot. If it boots into an error box saying “no homescreen found” then do the following while the USB cable is connected and that error is showing:

$ make install-gaia
$ adb reboot

This will install the user files and reboot the phone. B2G should now be running on the device. Enjoy! I’ve found calls, text messages, Wifi and Web Browsing works.

Performing Compatible Updates of Mozilla Central Git Forks

2011-10-28T16:00:00+13:00

Earlier this year I posted about how I created my git mirror of the mozilla-central mercurial repository. I’ve been keeping a fork on github updated regularly since then that a number of people have started using.

Users of my mirror have wanted to be able to keep up to date with the mercurial mozilla-central repository themselves, or add updates from other branches of the Mozilla source. It takes a large amount of time to start a conversion from scratch but it’s possible to start from the existing git mirror, perform the incremental updates from mercurial yourself, and still stay compatible with the incremental updates I push to my github mirror.

hg-git uses a git-mapfile in the .hg directory of the mercurial clone to keep track of the mapping between mercurial and git commits. By using the same git-mapfile that I use for my mirror you can start your own incremental update from the latest data in the git-mapfile instead of from the beginning. I keep an up-to-date git-mapfile in git-mapfile.bz2. It’s compressed with bzip2 as the uncompressed version is quite large.

The steps to start your own update process are:

Clone the mercurial mozilla-central repository
Clone my git mirror as a bare repository in .hg/git
Place the git-mapfile in .hg
Do an hg bookmark -f -r default master to mark the commit to convert to
Perform hg gexport to update .hg/git with recent mercurial commits

The .hg/git directory should now be up-to-date with respect to the mercurial repository. And the additional git commits will have the same SHA id as any commits I push into my mirror when I perform my own update.

The steps to do an incremental update are the normal:

Pull from the mercurial mozilla-central repository
Run hg bookmark -f -r default master
Run hg gexport
Push or pull to/from .hg/git as needed

As a working example, the following shell commands on a Linux system should set up your own repository ready for incremental updating:

$ hg clone https://hg.mozilla.org/mozilla-central
$ cd mozilla-central/.hg
$ git clone --bare git://github.com/doublec/mozilla-central.git git
$ wget http://www.bluishcoder.co.nz/git-mapfile.bz2
$ bunzip2 git-mapfile.bz2
$ cd ..
$ hg bookmark -f -r default master
$ hg gexport
$ cd .hg/git
$ git push --all ~/some/other/repo.git

Incremental updates just repeat the last few commands above:

$ hg pull -u
$ hg bookmark -f -r default master
$ hg gexport
$ cd .hg/git
$ git push --all ~/some/other/repo.git

Note that I use hg gexport and push from the .hg/git repository using the git command instead of doing an hg push and relying on hg-git to do the conversion and push. In my original article I did the latter. The hg-git push method uses slower python based routines to do the push which can take a long time and large amounts of memory on big repositories like mozilla-central. By splitting this up into gexport and using the native git command I save a lot of time and memory.

Temporal Media Fragment Support in Firefox

2011-08-08T12:00:00+12:00

The W3C has a Media Fragments Working Group whose mission is to specify temporal and media fragments in the Web using URI’s. The draft specification goes through in detail how these fragments work. I recently became a member of the working group and I’ve been working on adding support for the temporal dimension portion of the specification to Firefox.

In the most basic form you can specify a start time and an end time in the fragment part of a URI in an HTML video or audio element. For example:

<video src='/mediafrag/AudioAPI.webm#t=50,100'></video>

The ‘#t’ portion of the URI identifies the fragment as being a temporal media fragment. In this example ‘50,100’ means start the video at a current time of 50 seconds, and stop playing at 100 seconds. There are various other formats for the temporal media fragment defined in the specification. Examples can be seen in the UA Test Cases.

Development of this feature is being done in bug 648595. I’ve done test builds of the first iteration of the patch and they are available at my Firefox media fragment test builds page. The page has builds, an example, and a list of limitations which are currently:

Temporal syntax only. This means no spatial or track dimensions.
NPT time support only. No SMPTE time codes or Wall-clock time codes.
When changing the media fragment portion of a URL the media is not immediately updated. You need to refresh the page to see the change. This is most noticeable when directly navigating to the video and adding or changing a fragment.
The user interface for identifying the fragment in the standard controls is ugly and needs polish.
The HTML standard includes an ‘initialTime’ attribute for obtaining the start time. There is no way to obtain an end time so I’ve exposed a ‘mozFragmentEnd’ attribute on the video DOM object.

WDCNZ HTML Media Presentation

2011-08-08T09:30:00+12:00

Last month I went down to Wellington to give a joint talk at the WDCNZ conference. The topic was “HTML Media: Where we are and where we need to go”. The talk was shared between Nigel Parker, Mobile and Developer Evangelist for Microsoft, and myself. The conference was excellent with some great speakers and talks.

Nigel and I covered using HTML video and audio elements and how they can be used today across multiple browsers and mobile devices. We also covered upcoming API’s and directions in the web media area. Demo’s were shown on Microsoft Internet Explorer 9, Windows Phone 7 (running the Mango update which has a browser that supports HTML media) and Firefox. Nigel has slides and a summary in his blog post about the talk. I think it took a bit for attendees to get over the shock over a Mozilla and Microsoft representative sharing the stage and working together! Nigel’s post explains how this came about.

I spoke to a few Kiwi developers afterwards about using HTML video and there was a fair bit of interest in using it. The main obstacles seemed to be people unsure what codec to use and wanting support for adaptive streaming. The first is an education issue, getting people aware of what codecs to support for maximum coverage across browsers and how to encode to those formats. With regards to adaptive streaming there has been discussion between interested parties in various mailing lists and groups - it’s definitely something that is wanted.

At the time of the talk Nigel and I weren’t able to find any existing New Zealand based sites that use HTML video. Hopefully this will change in the future and Microsoft New Zealand are leading by example by using HTML video on their own site.

My Git Workflow for Mozilla Development

2011-04-16T17:00:00+12:00

Since my post on converting mozilla-central to git I’ve had a few requests about what my git workflow is for Mozilla Development.

When working on bugs I’ll create a git branch for development. The workflow in this branch looks like:

git fetch origin
git checkout -b bug/123456 origin/master
...make changes...
git commit
...make more changes...
git commit

Occasionally I’ll want to merge with the latest code from mozilla-central. I do this by rebasing my changes on top of the latest trunk code:

git fetch origin
git rebase origin/master

I’ll often have multiple temporary branches as I work on different ideas during the fix. I find being able to diff between the branches, cherry pick patches, etc useful.

When I’ve finished with the fix and want to generate a patch for review I rebase on top of trunk and squash all my commits down into one patch. I do this use git’s interactive rebase:

git fetch origin
git rebase -i origin/master
...squash commits into one commit and set commit message to my checkin commit message...
git hgp >~/mypatch.patch

The last command, git hgp, uses an alias I got from Rafael Espindola’s Blog. You can install this alias by adding the following to your ~/.gitconfig file:

[alias]
hgp = show --format=\"From: %an <%ae>%n%s%n%b\" -U8

The adds an ‘hgp’ git command that does the same as git show but includes a header for the committers name. This allows an hg import of the patch to include the correct patch authors details which is useful if you use the checkin-needed keyword for others to commit the patch.

I attach the patch file to the bug for review. If I get review comments I then go back to the git branch and make the necessary changes. Regenerating the patch involves repeating the steps above.

When the patch needs to be committed to mozilla-central it can be imported and pushed directly using mercurial:

hg clone https://hg.mozilla.org/mozilla-central
cd mozilla-central
hg import ~/mypatch.patch
hg push

A Quick Look at the Rust Programming Language

2011-03-31T20:00:00+13:00

The Rust Programming Language is a systems programming language being developed by Mozilla. It was announced last year and has seen quite a bit of development since then.

I’ve only been lightly following the development over the past year but recently decided to spend a bit more time looking at it. The following is a look at the language and implementation from someone who isn’t heavily involved in the development so don’t take anything I write as gospel. Hopefully I don’t get too many things wrong.

The Rust Language FAQ lists the following summary of features:

Memory safe. No null pointers, wild pointers, etc. Automatic storage management.
Mutability control. Immutable by default. No shared mutable state across tasks.
Dynamic execution safety: task failure / unwinding, trapping, logging. RAII / dtors.
Typestate system: ability to define complex invariants that hold over data structures.
Explicit memory control. Layout and allocation control. Interior / value types.
Very lightweight tasks (coroutines). Cheap to spawn thousands-to-millions.
Stack iterators (effectively lambda-blocks w/o heap allocation).
Static, native compilation. Emits ELF / PE / Mach-o files.
Direct and simple interface to C code (switch stacks and call, ~8 insns).
Multi-paradigm. pure-functional, concurrent-actor, imperative-procedural, OO.
First class functions with bindings.
Structurally-typed objects (no nominal types or type hierarchy).
Multi-platform. Developed on Windows, Linux, OSX.
UTF8 strings, assortment of machine-level types.
Works with existing native toolchains. GDB / Valgrind / Shark / etc.
Practical rule-breaking: can break safety rules, if explicit about where and how.

The Rust implementation is still in the ‘only useful for Rust developers’ state since the implementation and libraries are still being developed. There are two Rust compilers in various states of development. They are:

rustboot - written in O’Caml with it’s own x86 code generation backend. This is being used to bootstrap the ‘rustc’ implementation.
rustc - self hosting compiler written in Rust and bootstrapped using ‘rustbost’. Code generation is done using LLVM.

rustboot can be used to write Rust programs and try out the language. rustc is still in heavy development and only very basic stuff works from what I can see.

Building

Please note, as of 2011-10-24, the instructions for building Rust below are out of date. For updated instructions read my recent post on building Rust.

The instructions to build Rust are in the wiki. One important point is you need an SVN version of LLVM. I built this from source using a git mirror of LLVM (on 64 bit x86 Linux):

$ git clone git://github.com/earl/llvm-mirror.git
$ cd llvm-miror
~/llvm-mirror $ CXX='g++ -m32' CC='gcc -m32' CFLAGS=-m32 CXXFLAGS=-m32 \
                LDFLAGS=-m32 ./configure --enable-shared --disable-bindings \
                 --{build,host,target}=i686-unknown-linux-gnu \
                 --enable-targets=x86,x86_64,cbe
~/llvm-mirror $ make && make install

Once you have LLVM and the other pre-requisites installed:

$ git clone git://github.com/graydon/rust.git
$ cd rust
~/rust $ mkdir build
~/rust $ cd build
~/rust/build $ ../configure
~/rust/build $ make check

This builds the rustboot compiler, uses that to compile rustc, then runs a series of tests using both builds. If you have valgrind installed this will be slow as the tests are run under valgrind.

Bootstrap Compiler

The bootstrap compiler executable is boot/rustboot. The command to execute it, assuming the build occurred in the directory ~/rust/build is:

~/rust/build $ OCAMLRUNPARAM="b1" boot/rustboot -L boot -o hello hello.rs

This will compile a Rust program in hello.rs and leave an executable hello. The -L argument references the boot subdirectory containing the Rust standard library in libstd.so. A ‘hello world’ program for testing is:

use std;

fn main() {
  log "Hello World";
}

To run the executable you’ll need to adjust the LD_LIBRARY_PATH to include the rt subdirectory to pick up the Rust runtime shared library:

~/rust/build $ OCAMLRUNPARAM="b1" boot/rustboot -L boot -o test test.rs
~/rust/build $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/rust/build/rt
~/rust/build $ ./hello
rt: ---
rt: ca09:main:main: rust: Hello World

rustc Compiler

The rustc compiler lives in stage0/rustc. The output of this compiler is LLVM bytecode which must then be compiled using LLVM tools. To compile the hello.rs program mentioned in the previous section:

~/rust/build $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/rust/build/rt:~/rust/build/rustllvm
~/rust/build $ stage0/rustc -nowarn -L stage0 -o hello.bc hello.rs
~/rust/build $ llc -march=x86 -relocation-model=pic -o hello.s hello.bc
~/rust/build $ gcc -fPIC -march=i686 -m32 -fno-rtti -fno-exceptions -g \
               -o hello.o -c hello.s
~/rust/build $ gcc -fPIC -march=i686 -m32 -fno-rtti -fno-exceptions -g \
               stage0/glue.o -o hello hello.o -Lstage0 -Lrt -lrustrt

Note the need to add the rustllvm directory to LD_LIBRARY_PATH to pick up a shared library. That sequence of commands will compile the hello.rs file to LLVM bytecode. llc compiles the bytecode to x86 assembly. gcc compiles this to an object file, followed by a final gcc invocation to link it. And to run:

~/rust/build $ ./hello
rt: ---
rt: ee0b:main:main: rust: Hello World

Rust Language Details

For examples of the Rust language there are multiple tests and the source code to rustc itself in the github repository. The wiki has a link to the PDF documentation, currently a snapshot from 2011-02-25.

The following are some quick examples of Rust features that work with the bootstrap compiler.

Foreign Function Interface

This program uses the C FFI to call the ‘puts’ function from the C shared library:

use std;
import std._str.rustrt.sbuf;
import std._str;

native mod libc = "libc.so.6" {
    fn puts(sbuf s) -> int;
}

unsafe fn main() {
  libc.puts(_str.buf("hello from C\n"));
}

Compile and run with:

~/rust/build $ OCAMLRUNPARAM="b1" boot/rustboot -L boot -o cffi cffi.rs
~/rust/build $ ./cffi
hello from C

Command Line Arguments

use std;

fn main(vec[str] args) {
  for(str s in args) {
    log s;
  }
}

The main function takes a vector of strings as the argument. This holds the command line arguments passed to the program. The example iterates over the vector printing out each element.

~/rust/build $ OCAMLRUNPARAM="b1" boot/rustboot -L boot -o args args.rs
~/rust/build $ ./args a b c
rt: ---
rt: ccca:main:main: rust: ./args
rt: ccca:main:main: rust: a
rt: ccca:main:main: rust: b
rt: ccca:main:main: rust: c

Factorial

use std;

fn fac(uint x) -> uint {
  if (x <= 1u) {
    ret 1u;
  }
  else {
    ret x * fac(x-1u);
  }
}

fn main() {
  log fac(5u);
}

No language is complete without showing how factorial can be computed.

~/rust/build $ ./fac
rt: ---
rt: d158:main:main: rust: 120 (0x78)

Spawning Tasks

use std;

impure fn logger(port[str] logs) {
  let int i = 0;
  while (i < 2) {
    auto msg <- logs;
    log msg;
    i = i + 1;
  }
  log "logger exited";
}

impure fn main() {
  let port[str] logs = port();
  let task p = spawn logger(logs);
  auto out = chan(logs);
  out <| "Hello";
  out <| "World";
  join p;
}

A port is the receiving end of a typed inter task communication mechanism. A channel is the sending end of the communication mechanism. <| sends to the port and <- receives from the channel.

~/rust/build $ OCAMLRUNPARAM="b1" boot/rustboot -L boot -o spawn spawn.rs
~/rust/build $ ./spawn
rt: ---
rt: 9a73:main:spawn.rs:1: rust: Hello
rt: 9a73:main:spawn.rs:1: rust: World
rt: 9a73:main:spawn.rs:1: rust: logger exited

Types

use std;

tag list {
  nil;
  cons(int, @list);
}

fn list_len(@list l) -> uint {
   fn len(@list l, &uint n) -> uint {
    alt (*l) {
      case (nil) {
        ret n;
      }
      case (cons(_, ?xs)) {
        ret len(xs, n+1u);
      }
    }
  }
  ret len(l, 0u);
}

fn main() {
  let @list l = @cons(1, @cons(2, @cons(3, @nil)));
  log list_len(l);
}

This example creates a list type. It has constructors for an empty list, nil, and a cons containing an integer and the rest of the list. The ’@’ prefixed to the list type in places means that that variable holds a boxed object. That is, it’s a reference-counted heap allocation.

The list_len function shows the use of a local function which pattern matches over the list (using the alt keyword) and keeps a running total of the list length. The main function creates a list and prints the length.

~/rust/build $ OCAMLRUNPARAM="b1" boot/rustboot -L boot -o types types.rs
~/rust/build $ ./types
rt: ---
rt: 8126:main:main: rust: 3 (0x3)

Typestate

The typestate system is one of the things that most interests me about Rust. If you’ve been reading my ATS posts, in particular the ones involving type safe checking of array bounds, you’ll know I’m interested in languages that can help with detecting programming problems at compile time. It seems to me that the typesafe system can help here.

Here’s a contrived example demonstrating one use of typestate. main takes a vector of strings containing the command line arguments. I want to call a function, dosomething, that will use these arguments but for some reason there must be never more than 2 arguments. Imagine I’m calling some C routine that’ll die if I do.

I could check at runtime that the number of arguments is less than three. Here’s an example that does this:

use std;
import std._vec;

fn dosomething(&vec[str] args) {
  log "vector length is less than 3";
}

fn main(vec[str] args) {
  log _vec.len[str](args);
  check (_vec.len[str](args) < 3u);
  dosomething(args);
}

Some example runs:

~/rust/build $ OCAMLRUNPARAM="b1" boot/rustboot -L boot -o ts1 ts1.rs
~/rust/build $ ./ts1 a
rt: ---
rt: b3b1:main:main: rust: 2 (0x2)
rt: b3b1:main:main: rust: vector length is less than 3
~/rust/build $ ./ts1 a b
rt: ---
rt: d884:main:main: rust: 3 (0x3)
rt: d884:main:main: upcall fail '.t0 < 3u', ts2.rs:5

Notice the last invocation has failed since three arguments are used (the program name is counted as an argument).

It would be good to be able to check at compile time that somewhere the assertion holds that the number of arguments is less than three. In our example if we left the check call out, the program would still compile, but dosomething might do something disasterous. We can tell the typestate system that the precondition must hold for callers of dosomething and to fail at compile time by adding a ‘prove statement’ to the function:

use std;
import std._vec;

fn prove_length(&vec[str] args, uint n) -> bool {
  ret _vec.len[str](args) < n;
}

fn dosomething(&vec[str] args) : prove_length(args, 3u) {
  log "vector length is less than 3";
}

fn main(vec[str] args) {
  log _vec.len[str](args);
  dosomething(args);
}

The addition of : prove_length(args, 3u) to the function tells the typestate system that this boolean function must evaluate to true. It examines what it knows of the constraints made via check statements and the like to see if it can prove that this is the case. If it is not then a compile error occurs. The above program will fail to compile:

~/rust/build $ OCAMLRUNPARAM="b1" boot/rustboot -L boot -o ts2 ts2.rs
ts2.rs:14:13:14:19: error: Unsatisfied precondition constraint prove_length(args, 3u)

If we add a check statement we are adding an assertion check that this precondition holds. Typestate makes note of this and will now allow that call to dosomething to compile:

use std;
import std._vec;

fn prove_length(&vec[str] args, uint n) -> bool {
  ret _vec.len[str](args) < n;
}

fn dosomething(&vec[str] args) : prove_length(args, 3u) {
  log "vector length is less than 3";
}

fn main(vec[str] args) {
  log _vec.len[str](args);
  check prove_length(arg, 3u);
  dosomething(args);
}

~/rust/build $ OCAMLRUNPARAM="b1" boot/rustboot -L boot -o ts3 ts3.rs
~/rust/build $ ./ts3
rt: ---
rt: d9e6:main:main:                       rust: 1 (0x1)
rt: d9e6:main:main:                       rust: vector length is less than 3

It’ll be interesting to see how typestate is used in the standard library and third party libraries to help with compile time checking of code.

Conclusion

Rust is an interesting language and there’s quite a bit more to it than I’ve covered here. I just picked random features to try. I’m looking forward to rustc being more complete and trying out some of the language features in more ‘real world’ examples to get a feel for it.

New Wave Audio Backend Landed

2011-03-31T16:00:00+13:00

Yesterday I landed bug 635649 on mozilla-central. This is a refactoring of the wave backend for the HTML audio element. The initial work on this bug was done by Matthew Gregan with me completing the patch.

Prior to this landing the wave backend implemented the HTML audio API itself, sharing very little implementation details with the other audio backends. This meant the backend had to implement the intricate details of what events get raised and when they get raised. When we wrote the WebM backend we refactored the code so we could share this sort of thing amongst backends. This bug refactored the wave backend to share this code.

The advantages of this are:

Less likelihood of ‘wave backend only’ bugs.
Less code to maintain.
The Audio Data API can now use wave files as this was implemented in the shared backend code.

Git Conversion of Mozilla Central

2011-02-10T14:00:00+13:00

The Mozilla repository for Firefox development, mozilla-central, is a mercurial repository. I prefer using git for version control so I usually do my Firefox development using a git mirror of the mozilla-central repository.

To convert the repository into git I used hg-fast-export. This has worked well over the last couple of years. Unfortunately I’ve noticed that my git mirror and the official mercurial repository differ slightly. The git mirror contains files that have been deleted or moved in the official repository and lately the mirror has stopped building due to some commits not being correctly converted. As a result I stopped updating my mirror and used mercurial directly.

I still prefer developing using git though so I tried the hg-git mercurial plugin. This took about six days on my laptop to do the conversion and peaked at 14GB of memory. The result was a git repository with a working tree that, according to diff, is an exact copy of the original mercurial repositories working tree. Another nice feature of the hg-git is that commit id’s in the resulting git repository are the same across multiple runs of hg-git. This would allow other users to push and pull from a git repository that they converted independently.

To perform the conversion I installed hg-git and ran the following series of commands:

$ mkdir git
$ cd git
$ git init --bare
$ cd ..
$ hg clone https://hg.mozilla.org/mozilla-central hg
$ cd hg
$ hg bookmark -r default master
$ hg push ../git

Six days and 14GB later, the resulting git repository ended up being about 7GB in size. I did a git gc --aggressive on it to get this down to 200MB. Incremental updates of new mercurial commits are done with:

$ cd hg
$ hg pull -u
$ hg bookmark -f -r default master
$ hg push ../git

I’ve pushed a mirror of this to github at http://github.com/doublec/mozilla-central. I’m using this for development and will see if it is able to continue importing without diverging from the original mozilla-central source.

More information about the use of git for Mozilla development can be found in the followup posts I did on the subject:

WebM Decoder in Flash using Alchemy

2011-01-15T02:10:00+13:00

Ralph Hauwert has been posting on twitter about work he’s done on getting WebM decoding to work in Flash by compiling the libvpx source code using Adobe’s Alchemy technology.

Alchemy is a research project that allows compilation of C and C++ libraries into code that runs on the ActionScript virtual machine used by Flash. Ralph originally tweeted:

Had a quickshot porting #WebM codec to Flash 10 using libvpx and alchemy. Surprisingly, got a preliminary version to work.

He followed up with a post saying the intial performance of a 1920x1080 VP8 video with no audio was decoding to YUV at about 1.5 frames per second. He provides an image of a decoded frame.

This is good stuff and it’ll be worth following his twitter feed if you’re interested in a pure flash implementation of a WebM decoder.

I’ve been interested in having decoders written in JavaScript, which I wrote about in my reading ogg files with JavaScript post. Maybe something similar to the Flash/Alchemy approach could be done with emscripten, an LLVM to JavaScript compiler, to get a WebM decoder in JavaScript.