WebKit releases new SunSpider 1.0 benchmark to chart the future of JavaScript performance

Apple's WebKit team has released a revamped new SunSpider 1.0 benchmark suite for profiling the performance of web browser JavaScript execution, replacing the 0.9 version initially released at the end of 2007.

SunSpider was initially released with the goal of measuring how quickly a JavaScript engine can perform a balanced variety of tasks, with the goal of profiling how effective new advances will be.

JavaScript is the programing language of the web, handling tasks ranging from numerical calculations to date and regular expression manipulation to object oriented programming tasks that have, collectively, turned the originally simple web browser into a platform capable of hosting feature-rich web applications.

JavaScript is also used to power interactivity outside of the web browser, such as in iOS' iAd, the interactive widgets used in iBooks and in developing dynamic content in iTunes including iTunes LP albums and iTunes Extras for DVD-like, interactive menus for movies and related bonus material.

Apple's new iAd Producer 4.0 (pictured below) provides tools to develop all three types of non-browser, JavaScript powered content.

In the five and a half years since SunSpider 0.9 was released, it has documented a more than 30x improvement in JavaScript performance from a typical 2.33 GHz computer of 2007 running Safari 3 to a modern 2.2GHz i7 MacBook Pro running Safari 6.

The new 1.0 version just released "fixes a number of bugs and aims to further increase test accuracy and repeatability," taking into consideration the much more sophisticated design of modern JavaScript engines.

Tracking the progress of JavaScript engines to power HTML5

Back in 2007, when the SunSpider benchmark first appeared, the execution of JavaScript hadn't changed dramatically since its introduction by Netscape in 1995 as a way to perform basic calculations within a web browser on the local client side (rather than remotely by the web server).

After SunSpider appeared, an intense new push to drive more programatic sophistication into dynamic web pages resulted in dramatic new leaps in JavaScript technology, first under the term "Ajax" (relating to web pages that can update themselves within the browser without needing to be redrawn by the server) and increasingly related to the promise of "rich web apps" related to HTML5 (see also: "Why Apple is betting on HTML 5: a web history").

In 2008, the WebKit team announced a rewriting of the original JavaScriptCore as a direct-dispatch register based, high-level bytecode virtual machine named "SquirrelFish." Mozilla subsequently announced its TraceMonkey engine for FireFox and Google acquired the V8 engine to accelerate JavaScript performance in Chrome.

Before the year was out, and before SquirrelFish even made it into a shipping version of Safari, the project was enhanced further under the name "SquirrelFish Extreme," which accelerated JavaScript execution using a Just In Time compiler to turn JavaScript into native machine code as an alternative to generating bytecode.

In 2009, Apple applied the enhancements to Safari 4 under the brand "Nitro," noting that the new implementation could run JavaScript up to 4.5 times faster.

The next year, it subsequently released new SquirrelFish Extreme enhancements in Safari 5, boosting JavaScript performance on the Mac another 30 percent over the previous Safari 4. In 2011, the company added Nitro to mobile Safari in iOS 4.3, boosting JavaScript performance by 200 percent on Apple's mobile devices.

A new benchmark to chart the future progress of JavaScript

A year ago, WebKit announced further enhancements to JavaScriptCore that rendered it a "triple tier virtual machine," incorporating an efficient new LLInt (Low Level Interpreter) used by JavaScriptCore to attempt the execution of code before passing it to the standard interpreters, either the bytecode virtual machine or the JIT (Just In Time complier) which builds native machine code on the fly.

The revamped new SunSpider takes into consideration these increasing levels of complexity in modern JavaScript engines. As the WebKit announcement explains, due to this complexity "it makes sense to make our benchmarks perform some validation of correctness. Benchmarks are uniquely capable of stressing the optimizing JIT in a way that conformance tests cannot. This observation comes from our own experience: as we added validation to our benchmarks, we were able to catch bugs sooner."

In addition to new validation checks, the revised benchmark eliminates delays in the original version that were added to allow the browser to complete each portion of the tests before moving to the next. However, these delays started distorting the results because on fast hardware, the test suite's built in delays were allowing hardware power management features to kick in, sporadically creating the false appearance of poor performance.

Competing for the future of HTML5

While Apple and Google have worked together closely for over half a decade to improve WebKit, particularly with the goal of making HTML5 rich apps possible across both desktop browsers and on mobile devices, the two companies are now separating their efforts after Google decided to fork the project and take its own Chromium code solo (under a project now named "Blink").

This allows Google to purge any links to Apple's Nitro engine (which it doesn't use in Chrome), necessitating a similar purge of Chromium code from the main WebKit project. This may likely impact third parties such as Samsung, which uses Apple's WebKit web engine but pairs it with Chromium's V8 JavaScript engine, now part of Chromium.

Somewhat ironically, Google's fragmentation of Apple's WebKit platform (as the platform's largest independent user) is similar to what many expect Samsung to do to Google's Android project, ultimately leaving Google to maintain its platform while Samsung runs with its own copy, following the footsteps of Amazon and a variety of Chinese forks.

At the same time, Google also invested in a project to replace JavaScript with its own new web programming language named Dart, a move that was opposed by Apple, Mozilla and Microsoft.

At the end of 2011, Apple's Oliver Hunt warned that "adding direct and exposed support for a non-standard language [like Dart] is hostile to the open-web by skipping any form 'consensus' driven language development that might happen, and foisting whatever language we want on the web instead. This implicitly puts any browser that supports additional proprietary extensions in the same position as a browser supporting something like [Microsoft's proprietary] vbscript, and has the same effect: breaking the open web by making content that only works effectively in a single product."

Alongside its flirtation with Dash in 2011, Google also advocated Adobe Flash (hardwiring it into its Chrome browser) and worked to derail the H.264 video standard in favor of its own WebM video codec, two other efforts that were clobbered by the reality of mobile devices dominated by iOS. Google has since stated that it does not plan to push adoption of Dart within its own "Blink" Chromium fork.

Apple is expected to introduce new versions of Safari incorporating the latest advances in WebKit for iOS 7 and OS X 10.9 at next month's Worldwide Developer Conference.