Google reaffirms intent to derail HTML5 H.264 video with WebM browser plugins

article thumbnail

After igniting a hailstorm of controversy over its intent to drop HTML5's H.264 support from its Chrome browser, Google has reaffirmed its intent to push its own open WebM video codec via Flash-like plugins for Internet Explorer and Safari users. The reason: Google wants to ship free platforms without incurring external licensing fees.

Google's Mike Jazayeri detailed the company's new push behind WebM by writing in a detailed blog posting that the groups involved in developing the HTML5 video distribution standard "are at an impasse. There is no agreement on which video codec should be the baseline standard."

The impasse that wasn't there

That "impasse" actually occurred years ago, when Opera and Firefox developer Mozilla began pushing to define Ogg Theora as the baseline for HTML5 video, a position that was staunchly opposed by Apple and Nokia back in 2009, in part because their mobile devices already relied on the hardware optimized H.264 video codec. Conversely, Mozilla rejected H.264 because it involved paying royalty fees.

The HTML5 working group members finally agreed to disagree; rather than defining Ogg Theora or H.264 or anything else as the "baseline" codec for video served via the HTML5 video tag, they left the decision up to the market and to the votes of web users and Internet broadcasters. This decision parallels how HTML works with every other type of media file; there is no baseline graphic or audio format, for example, which allows web publishers to decide for themselves whether to use GIF, JPEG, or PNG graphics formats, or whether to use MP3, AAC, or raw WAV audio files. Modern browsers support them all.

Ever since the debate about HTML5 video was settled in the specification by simply skipping over a defined "baseline" video format, web video producers have overwhelmingly chose to move to H.264. There's a few good reasons for this. First, H.264 is very efficient at compressing data, and lots of tools exist that can encode data at high quality and high speed.

Secondly, H.264 was already widely supported by consumer devices and applications, in particular Apple's iPod and iPhone and iTunes, which essentially only play H.264. Apple's strong push behind H.264 and its openness as a standard that anyone can implement (although there are royalties involved) has encouraged hardware makers to similarly offer mobile devices that very efficiently play H.264 video using hardware acceleration.

Baseline video among browsers: H.264

On the web, all of the major browsers added the ability to play H.264 video apart from Firefox, which held onto the ideal of only supporting Ogg Theora because handling H.264 would involve licensing fees that Mozilla would have to cover for a potentially unlimited number of users that might reuse its code under the GPL license that Firefox is offered under. That makes licensing H.264 prohibitively expensive for Firefox.

That situation created two camps of browsers: one, the commercial browsers from Google (Chrome), Apple (Safari), and Microsoft (Internet Explorer), all of whom have no problem paying for the H.264 license, and the open source browsers from Google (Chromium) and Mozilla (Firefox), neither of which could properly license H.264 for an unlimited global audience under the GPL/LGPL. Google's new policy shifts Chrome into the second group.

However, the vast majority of Firefox users are running on Windows or Mac OS X, both of which already supply licensed code supporting H.264 video playback. Thus, the only users affected by the "impasse" described by Google are Linux users. This is currently a commercially insignificant demographic, but Google has aspirations of shifting the world to Linux, in mobile devices with its Android OS and among netbooks and low end PCs with its upcoming Chrome OS. If this happens, the tiny market share of Linux among consumers will suddenly matter.

On page 2 of 3: The Open Source ideal, Why Google isn't afraid of patents

Freedom's just another word for nothing else to pay

However Google doesn't sell either Android or Chrome OS; it gives them away. Microsoft sells Windows, while Apple bundles Mac OS X and the iOS as part of its hardware sales. That leaves Google as the only significant platform vendor that won't be selling its platforms, but will still need to be licensing commercial technologies, and in particular H.264 video playback, if the world continues to standardize on H.264 for video distribution.

So in this dispute, Google isn't standing up for open standards, it's standing up for the right to push ads through platforms based on free operating system software, relieving it from having to deal with Microsoft and Apple and other platform vendors. The only way it can afford to do that is if video is delivered without technologies that cost money.

Google has little stake in delivering high quality video; it serves moderate quality web videos through YouTube that could be (and have been) delivered with less than start-of-the-art codec technologies. Apple and Microsoft both deliver HD video products, sell commercial movies, and have plans to deliver the future of HD television. Google's TV strategy primarily revolves around injecting its ads into other broadcasters' content, a concept that so far has been an unmitigated disaster.

Google therefore sees HTML5 video as desperately needing a free video codec to prevent it from having to sell its platform, which could otherwise be free. In order to make that happen, Google purchased On2 and hastily released that company's commercially failing VP8 video codec as an open standard under the new brand WebM. The codec isn't as sophisticated as the open H.264, in part because it was created internally by On2 as a proprietary product.

Why H.264 is better

H.264, on the other hand, is a pool of the intellectual property and engineering talents of every significant organization currently involved in developing video technology, ranging from Sony and Philips to Apple and Microsoft. All those companies participating in the H.264 standard have agreed to pool their patent claims and charge all users a fair, nominal royalty fee to benefit those that collectively created the technologies that make H.264 so efficient and sophisticated. They're also collectively working on future enhancements to H.264 and its eventual replacement, H.265.

H.264 is also much more mature, having been adopted as the preferred codec by Apple's QuickTime 7 back in April of 2005. Since then, Apple has incorporated H.264 support throughout its product line, ranging from desktop systems to mobile devices with hardware-based codec acceleration.

The rest of the industry has also adopted H.264, ranging from mobile devices and cell phones to HD disc formats (such as Blu-ray) to cable and satellite TV transmissions to web video, from Google's own YouTube to Vimeo to Brightcove to Adobe's preferred Flash Player codec. The actual H.264 specification is continually evolving, with new and expanded profiles and amendments.

Microsoft's competing Windows Media codec (aka VC-1) is used by its Xbox 360 and within Silverlight, and was the primary codec in the now discontinued HD-DVD format. It's included in the Blu-ray specification, but most Blu-ray titles are encoded using H.264. Windows Media codecs are based on the MPEG-2 H.263 specification.

Sorenson's proprietary video codecs were once the primary codecs used by Apple's QuickTime, until sophisticated MPEG specifications began to emerge as industry standards. Sorenson 3 was actually based on the MPEG-4 H.264 draft specification. Apple moved to its own internal implementation of H.264 in 2005. The MPEG 4 container format is based on QuickTime, which Apple contributed to MPEG in 1998.

Timeline of video codecs

The Open Source ideal

In the minds of free software advocates, the notion that anyone should benefit from the licensing of their own technology is ideologically abhorrent. Thus, they've been seeking to develop a video codec technology assembled by volunteers that can be used without anyone paying for it. The problem is that you get what you pay for.

Until Google swept in to buy On2, open source advocates had standardized on Ogg Theora, a nearly decade-old specification based on an earlier proprietary codec that had been abandoned by On2. It was neither efficient nor commercially viable nor capable of being decoded by existing hardware, a feature that is essential to efficient video playback by mobile devices.

These show-stopper problems did not thwart open source advocates, because there were no other credible free alternatives. With Google's purchase of On2 and the open sourcing of On2's more modern VP8 as WebM, it appeared that Ogg Theora could be replaced with a more modern version of a codec that the very wealthy Google would be supporting, one day perhaps even eclipsing the sophistication of H.264.

A key problem is that all of the catchup progress in developing WebM's video codec must pass through a number of areas that are patent minefields. WebM simply can't be improved without running afoul of the vast patent pool of all the companies collectively working to define MPEG standards like today's H.264 and the upcoming H.265 that will one day replace it. Everyone knows this, and many observers, including the MPEG Licensing Authority that manages H.264, also believe that WebM already infringes upon its members' patented technologies.

Why Google isn't afraid of patents

Google must also know this, but it isn't afraid of running into patent issues because it isn't directly earning any money selling anything that might infringe upon those patents. If Google's Android oversteps patents held by Nokia or Apple or Microsoft (and it allegedly has), those companies won't sue Google, they'll sue HTC and Motorola and other Google licensees that are using Android to sell their hardware (as they already have).

Unlike hardware makers such as Apple, Microsoft, Motorola and Nokia, Google hasn't been the target of nearly as many patent lawsuits. Google's first major brush with patent lawsuits, when Overture sued the company for infringing upon its paid search business model, was settled by Yahoo buying Overture, and then Google giving Yahoo billions of its stock to prop it up as a phony competitor.

That cost Google nearly nothing, while all the talent from Overture subsequently flowed to Microsoft and Google as Yahoo crushed the acquired Overture team under its own incompetent leadership, killing more birds for Google: a major competitor, the potential threat of a monopoly investigation, and the need for hire additional talented employees.

Google's act, arguably pulling off the world's most blatant patent infringement and then building a company worth $200 billion on top of it, was simply an overture to its next: ripping off the world's biggest consumer technology product using a clone of the world's largest write-once, run anywhere platform. While Apple didn't sue Google over the iPhone (something that would be hard to seek damages over, given that Google gives away its Android software), Java's new owner Oracle decided it would.

Oracle expects to demand licensing revenues from Google's distribution of Android. That would force Google to either create a new, non-infringing mobile platform from scratch, or begin paying licensing fees to Oracle that might cause the Android project to be worth less to Google than it earns from Android's mobile ad revenue. However, that case hasn't been settled yet, so Google is currently emboldened to replicate its success in stomping on others' patents and simply paying them off afterward, much as Microsoft did in the 90s.

This confidence is also behind Google's rapid open-sourcing of On2's VP8. Unlike Netscape's methodical open sourcing of its browser to create Mozilla (which took years) or the very slow process Sun took in opening up Solaris and portions of Java, or similarly cautious open source projects created by other companies (including Apple's efforts to open up its Darwin OS), Google simply threw the WebM specification on the web almost immediately after acquiring On2.

Like a drunk teen barreling down the freeway in the wrong direction, Google seems confident that nothing bad will happen because the company hasn't yet experienced any significant problems doing that sort of thing in the past. At the same time, if Google were truly confident that its WebM is free of patent hazards, it would indemnify third parties who implement it from patent threats. It hasn't. Google is essentially gambling with the resources of its hardware partners. Even if they lose, Google still wins.

On page 3 of 3: Open source vs. opening other's source, Outlook for WebM not so good

Not motivated by open innovation

Google is insisting that its move to thwart H.264 adoption by removing support from Chrome is simply an effort to push the web to technologies that any company can afford to use, while noting that "it is clear that there will not be agreement to specify H.264 as the baseline codec in the HTML video standard due to its licensing requirements."

However, while this may be true, the only hold outs in supporting H.264 are Opera, which has negligible share among web browsers, and Mozilla, which is almost entirely financially supported by Google. This is a problem Google could solve by covering Firefox's licensing fees, or by simply pulling the plug on Firefox and allowing it to go out of business just as Netscape itself did before its software product was propped up as free open source project living off Google's life support.

If Google were truly interested in open ideals, it would also abandon Adobe Flash, which also delivers its video via H.264, but does so as a proprietary plugin that pays its own way onto Google's platforms.

Partnered with Adobe's Flash Player, Google can roll out commercial video on Android and Chrome OS and simply have Adobe pay for it as the middleware vendor of a proprietary plugin platform running on top of Google's own. This makes it clear that Google's WebM strategy has little to do with openness, and is really just intended to save Google money.

If the planets align for WebM: a best case scenario

Google has actually created a new impasse of its own: HTML5 video, if forced in principle by Chrome to use WebM as its baseline (and exclusive) codec, will require all web users to either select Google's Chrome, the Firefox or Opera browsers, or install plugins to view WebM content; Google announced it will be supplying plugins for Safari and Internet Explorer. This holds out the potential for an ideal future where every browser plays video via WebM's free codec as a baseline, while Safari and IE could also offer H.264 playback as an option.

However, WebM is worthless on existing mobile devices, most of which can't even install a plugin to play WebM content. This could be addressed in the future by simply upgrading all new mobile devices with new WebM hardware acceleration (which doesn't yet exist), but there are other issues involved, including the lack of commercial viable WebM encoding tools. However, the biggest problem by far is that WebM will eventually run into patent issues from everyone else in the video industry that has already worked to hammer out the vast H.264 specification, which address far more than just the web video role WebM hopes to address.

If WebM isn't already in trouble on the patent front, it certainly will be every time it attempts to improve along the same patented footsteps H.264 developers have already worked their way though at considerable expense. Google may be professing support for open ideals, but its real motivation is to conduct its advertising business without paying to use the technology other companies have already invented. In doing so, it may cause more problems that it attempts to solve.

Open source vs. opening other's source

Google has taken WebKit's browser code and appropriated it into the successful Chrome browser, but Apple, KHTML, and other contributors that helped build the WebKit codebase willingly offered their efforts up as a free software project under a permissive reuse license to encourage its use, specifically in order to enable products like Chrome to exist.

Google's attempt to take commercial technologies and turn them into open source projects under its own control and at the objection of their rightful owners is a very different situation. Apple benefits from sharing WebKit; it does not benefit from Google simply taking the iPhone and replicating its patented features. Oracle similarly benefits from some aspects of open development, but is not interested in giving away its proprietary Java technology to Google for free distribution as part of Android.

Similarly, all of the contributors of the H.264 specification are not going to be content with Google duplicating their patented efforts and offering it for free to enable more efficient distribution of Google's ad platforms. That's why the MPEG LA has already promised to take legal action over WebM patent infringement, and will undoubtedly demand licensing fees from anyone who attempts to use WebM in place of H.264 or borrow further improvements from the H.264 patent pool in order to embellish WebM as a technology in the future.

Apple's "open" is collaborative

In contrast, Apple's decision to promote H.264 in place of proprietary video codecs such as Sorenson or On2 or Microsoft's Windows Media was done with the intent of having world class video technology to use in building its hardware products. Apple solely supported H.264 and the related AAC in the iPod, which helped the new standards take off and, through economies of scale, resulted in widespread, low cost availability of hardware optimized silicon that can play H.264 efficiently on mobile devices from any vendor.

Additionally, Apple's attempts to replace dependance upon the Adobe Flash plugin with ubiquitous web standards including HTML5, H.264, CSS and JavaScript are not based upon duplicating Adobe's technology and offering it up as open source, but rather rely upon existing open standards that can be enhanced and extended without running into Adobe's patent portfolio. And if they do, Apple can negotiate to open those patents by including them into the joint development of those web standards, in the same way Apple has contributed its own patented technologies, royalty free, into HTML5 (such as was the case with the Canvas element Apple invented).

Apple's promotion of open web standards, in particular the open HTML5 specification and the open source WebKit code implementation, demonstrate that the company is aware of the benefits that open software and open standards can provide in creating interoperability and a level playing field. If Apple thought that the same could be done in the area of video codecs, it would have pursued doing just that nearly a decade ago when it began work on implementing H.264.

While Apple had the ability to borrow KHTML's browser code under its open source license to build what would become WebKit and the Safari browser, it could not do the same to build an open video codec, because video technology is progressing too rapidly under the intense competitive pressure of a number of technology companies. That calls for an approach that respects the existing work already created and patented by stake holders.

Incidentally, the open, collaborative development of MPEG standards was what killed the proprietary efforts to develop video codec technologies at Microsoft, Sorenson, and On2 (before Google acquired it). While Sorenson and On2 were already failing five years ago, Microsoft was working hard to displace MPEG standards with its own Windows Media codec. Without Apple's heavy consumer-oriented push behind H.264, the world's video likely would have fallen under the control of Microsoft, as was being confidently predicted to occur by most pundits around 2005.

Apple pays for pre-existing IP, Google not so much

Apple is often cited as a royalty recipient of H.264 by its open source critics, but the company is by far a minority stakeholder in the MPEG patent pool. If Apple actually owned H.264, it wouldn't be licensing it from the MPEG LA; it would be distributing it as closed source technology just like its Cocoa frameworks. Alternatively, if Apple thought it could build a quality implementation of an open video codec based on an existing open source project, it would do so and distribute it for free just like its WebKit software or CUPS printing software or Darwin core OS.

As it is, Apple recognizes that other entities already own those video technologies, so it's paying the MPEG LA to license them, just as it pays to license Microsoft's Exchange ActiveSync, or Amazon's One Click retail patent, or the DVD specification (which includes MPEG-2), or the trademarked name "iOS" from Cisco, among many other things.

Google is trying to pull off a hybrid of those two options, taking proprietary software that is likely to infringe patented technology and "opening" it as a free solution, apparently oblivious of how it will evolve given that all avenues for progress have already been patented. While it has yet to be stopped by patent action from Oracle, Google's premise of a free web codec will definitely be assailed by the collective action of all of the MPEG patent holders, making Google's WebM effort a dead end pursuit that can only help to hold back the commercial progress of HTML5 and H.264 outside of the ideological bubble of the open source movement.

Outlook for WebM not so good

Rather than replacing H.264 with a nearly as good but free alternative, WebM can really only possibly thwart the progress of web publishers who want to serve video that their viewers can see, doing so using open, interoperable web standards. Rather than using HTML5 and H.264, Google hopes to promote HTML5 via WebM, but it is actually just incentivising a return to delivering video via Adobe Flash, which can already reach Opera, Firefox and Chrome users without reencoding existing content from H.264 into WebM (something that is slow and inefficient with the existing tools available).

At the same time, while Google can certainly throw chaos into play by removing H.264 playback from its own free platforms and browsers, that move will likely hurt Google more than it will adoption of H.264. Neither WebM nor Flash are supported on iOS, which is not only a significant percentage of the mobile market, but also represents a far more valuable demographic than Google's Android, which according to Gartner is primarily being deployed in China and emerging markets by no-name "other" manufacturers. If Android can't play H.264, it won't push vendors to jump through hoops to encode for WebM or the poorly performing mobile version of Flash. It will push smartphone users to abandon Android.

Similarly, if the Chrome browser and forthcoming Chrome OS can't play H.264 without involving Flash, it will only make those products less attractive to would-be buyers. Moving from Chrome back to Internet Explorer or Safari is not difficult (and will be even easier if Apple ever fixes the memory leaks and poor performance dogging its own browser), while selling Chrome OS and its inherent limitations as a web-only platform will be very difficult if it can't play industry standard video.

This gives Google very little leverage in trying to force WebM adoption the same way Apple has pushed the adoption of H.264, HTML5, ACC, and other open standards, whether they involve third party licensing or not. Additionally, even if Google removes H.264 support from Chrome and or Android, users (or hardware makers or carriers) can add it back, greatly diluting Google's leverage against H.264 among mobile devices.

Ultimately, Google's efforts to derail H.264 are unlikely to have a significant impact on H.264, but may cause uncertainty that could slow the adoption of HTML5, resulting in H.264 video being delivered instead via Flash on the web, integrated within iOS apps, or served as straight HTML5 H.264 videos that won't play on Chrome, Chrome OS and Android without an H.264 plugin. The primary casualty of those outcomes would be Google itself.

 

Latest News