Video Element and Ogg Theora

There has been a lot of attention lately around the WHATWG <video> specification recommending Theora and Vorbis as baseline codecs.

The issue seems to have gained some attention since the position paper Nokia submitted to the W3C Video on the Web workshop which made it clear they didn't want Ogg included.

The reference to Theora and Vorbis has since been removed from the WHATWG specification and replaced with the wording:

It would be helpful for interoperability if all browsers could support the same codecs. However, there are no known codecs that satisfy all the current players: we need a codec that is known to not require per-unit or per-distributor licensing, that is compatible with the open source development model, that is of sufficient quality as to be usable, and that is not an additional submarine patent risk for large companies. This is an ongoing issue and this section will be updated once more information is available.

This has also caused quite a bit of discussion around the web.

From what I can see the main objection to Theora is the submarine patent issue. Theora is not 'patent-free'. Rather all known patents relating to it have been released to the public. Submarine patents are those which are unknown. They refer to the practice of keeping quiet about a patent on a technology until some company with a large amount of money implements it. Then the patent holder surfaces and attempts to get large amounts of money for it.

The problem of submarine patents is not specific to Theora. All forms of software technology face the risk. H.264 seems to be the current popular technology for those that oppose the use of Theora in the specification. H.264 also has a risk of submarine patents (like any software) but companies have already exposed themselves to this risk because they ship H.264 based systems. By having to implement Theora they then open themselves up to a second avenue of risk, one which they didn't face before.

One could argue that implementing anything new contains this element of risk, so following that logic these companies won't be doing any new implementations of anything. That's not the case obviously, or nothing would be done. They weigh up the risk of patent issues vs the reward of implementing things. With the current perceived low usage of Theora they probably don't see any advantage to them for implementing it vs the risk. So that appears to be why Theora is not a desired choice but some groups.

I've seen questions asked in the discussions asking why don't we settle on H.264. The big companies are already using it. There is the open source x264 encoder and ffmpeg decoder. While these projects are open source my understanding is that any user of the product must pay the required license fees to the patent holders on the techology they use. In the case of H.264, this is the MPEG-LA. The summary of the terms lists the fees. There is a cap on the total amount a company should pay so why not just pay this amount and ship H.264 support?

For an open source project this creates some difficulty. If the source for an H.264 decoder is included then anyone who downloads and builds from source, forks the project, or otherwise distributes a build would seem to have to deal with the licensing issues seperately. That is, the cap wouldn't cover all usage of the source.

This would immediately limit the distribution of the browser and the ability to embed the engine in other products without removing the H.264 capability. With the capability removed you're back to the problem of what codec should be supported with <video>. To effectively implement the HTML <video> specification in a way that can play compatible video streams with other (closed source) browsers, you'd have to front up with that fee.

It might seem that the best approach is to not specify a baseline codec for <video>. This also has problems. The big advantage of a baseline codec is a content producer can provide video in a format they they know all conforming browsers on any platform can display. Without a baseline codec, content producers will upload video in formats specific to particular platforms and we have little advantage over the existing object/embed elements.

The issue of support for DRM as outlined in Nokia's position paper isn't that big of an issue. Even with a royalty-free baseline codec, implementors are still able to have other codecs supported. They can support whatever DRM specific codec is required by 'big media'. But it is important that a free baseline is available for those that just want to supply video in a convenient manner without having to pay any money. Robert O'Callahan mentions this in his recent blog posting.

The W3C Video on the Web workshop starts tomorrow (Wednesday). The subject of HTML 5 and codecs is going to be discussed there. I'll be there talking about Mozilla's position and taking part in the discussion of the codec issue.

For more good commentary on the issue you might like to read The HTML 5 Wars (and why you should avoid them). If you are passionate about the use of Ogg Theora and <video> one of the best things you can do is start using it.

Do compelling demos. Release video in Theora format. It may be easy to use a service that provides video for you in exchange for giving them certain rights but if you want your format to succeed, then increased usage is the way.

Categories: firefox

New <video> enabled experimental Firefox builds

I've uploaded new builds of Firefox with experimental support for the WHATWG <video> element. They are (compiled from this git commit):

• Intel Mac OS X 10.4.x
• Windows XP
• Ubuntu Linux x86

The git repository has been updated with the code for this build.

Note that the code in these builds, and the git repository, is advanced from that in the patch attached to the video element bugzilla entry. That code can be accessed from git via the patch11 tag.

The usual disclaimer with the builds I provide applies.

Be aware that these are builds from a random point in the Mozilla CVS tree, with the Video patch applied. I don't guarantee they'll work for much more than demonstrating video support and it's very likely to contain bugs. That said, I run these builds often.

Two main additions with this build. The first is fixing the annoying bug whereby leaving a page that is playing video would leave the sound running in the background. This was due to the 'bfcache' keeping the page around in case the user hits the back or forward buttons. The video is effectively paused now when the page is in the bfcache.

The second is preliminary support for the <source> element, which can be used to provide a list of media resources, with mime types, and the browser selects out of the list the media resource to play based on what it supports. This initial implementation only supports a 'type' attribute set to 'video/ogg'. What this will let you do though, is add a source element for an MP4 encoding, and one for an Ogg encoding. When played on Firefox the Ogg version will play, and a browser that currently doesn't support Ogg but supports MP4 can choose to play that instead.

Issues I have partial fixes for but haven't yet made it into the main branch of the git repository, and not in these builds are:

1. Sound support on Linux is patchy. I'm using sydney audio's OSS support which has issues when multiple devices open /dev/dsp. This can occasionally affect synchronisation of the video.
2. There is no buffering for network loads. The video plays immediately so on a slow pipe or large video the playback will stutter. You can pause the video to let the download catch up but there is no progress indication yet. This can make performance look worse than it is in bandwidth constrained environments.
3. Seeking forward and back in the stream. There is some support, but it's a work in progress and quite broken.

You might notice that full page zoom works in this build. And it works with videos. You can zoom a page and any videos on the page are zoomed. You can test this with Ctrl+ (Apple-+ on the Mac).

Test videos are available at my test page, or Opera's page.

There are sites using the <video> right now.

The Metavid project has support for the <video> element. If you look at the videos in a <video> enabled browser, the player is the interface provided with the 'controls' attribute set. For example, Senate Proceeding 11-16-07. Nice job, Metavid, and thanks for using <video>!

The Wikimedia video's seems to have experimental support for it. Unfortunately the test to see if the video element is supported appears to be done last, so usually the Java applet, etc is found first. You can get around this by setting a cookie. When the Wikimedia video page open, enter this in the URL bar:

javascript:wgOggPlayer.setCookie('ogg_player', 'videoElement', 7*86400*1000, false, false, false, false)

Remove it by deleting the 'commons.wikimedia.org' cookie if you want to go back to the default ordering of selections.

You'll notice some issues, which is probably why it's currently right at the end of the chain. Clicking on the video picture and pressing play works fine. But clicking the play button on the main page listing all the videos does not correctly display the video. I'll look into what's causing this.

If you know of any other <video> supported sites, please let me know in the comments.

I'll be attending the W3C Video on the Web Workshop in San Jose on the 12th and 13th of December. I fly out from Auckland on the 9th and leave San Francisco on the 16th. If you're in the area and want to catch up, let me know.

Categories: firefox, ogg