In a collaborative effort between Google, Mozilla, and the WebKit team, JetStream 3.0 was introduced as a major overhaul of the cross-browser benchmark suite. This update addresses the ever-changing landscape of web performance, particularly focusing on how WebAssembly (Wasm) is measured and the scale of modern web applications. Below, we explore key questions about this new benchmark through a detailed Q&A.
What is JetStream 3 and why was it created?
JetStream 3 is the latest version of a benchmark suite designed to measure browser performance in a comprehensive and realistic way. It was developed jointly by the teams behind WebKit, Google Chrome, and Mozilla Firefox to reflect the current state of web technologies. The primary motivation was to replace JetStream 2, which had become outdated due to rapid advancements in browser engines, especially regarding WebAssembly (Wasm). As web applications grew more complex and Wasm usage expanded beyond initial expectations, the old benchmark no longer captured real-world performance accurately. JetStream 3 introduces new workloads, revised scoring mechanisms, and a broader range of tests to ensure that browsers are challenged in ways that matter for modern users.
How did WebAssembly benchmarking evolve from JetStream 2 to 3?
When JetStream 2 was released, WebAssembly was still in its early stages, primarily used for large C/C++ projects migrating from asm.js. The old suite measured Wasm in two separate phases: Startup and Runtime. This approach assumed that users would tolerate long startup times for high-throughput gains. Over time, browser engines dramatically improved Wasm instantiation speeds. JetStream 3 fundamentally changes this by integrating Wasm workloads into a seamless single score, reflecting how Wasm is now used in libraries, image decoders, and UI frameworks where startup is part of the critical path. The new suite also includes more diverse Wasm tests to represent its modern role on the web.
What was the 'infinity problem' in JetStream 2?
The 'infinity problem' arose from the scoring formula used in JetStream 2: Score = 5000 / Time. As browser engines optimized Wasm startup to near-zero milliseconds, the time measured with Date.now() sometimes became 0 ms due to rounding. Dividing by zero resulted in an infinite score, which effectively broke the benchmark and made it useless for comparing browser performance. The WebKit team had to patch the harness in JetStream 2.2 by clamping the score to 5000 to prevent this absurdity. This issue was a clear indicator that JetStream 2 had been outgrown; the subtests were too easy for modern engines, and a new approach was necessary.
How did browser engines outgrow JetStream 2's Wasm subtests?
Browser engines, particularly JavaScriptCore in WebKit, optimized the Wasm startup path so aggressively that for smaller workloads, instantiation times effectively hit zero. While this sounds like a success, it meant that the benchmark no longer differentiated performance. The micro-optimizations that once yielded significant gains became irrelevant as the baseline improved. Additionally, the original subtests assumed a pattern of large, one-time startups, but modern Wasm usage is more granular—many small modules loaded dynamically. JetStream 2's binary startup/runtime split didn't reflect this reality. Consequently, the benchmark needed a fundamental rethink to stay relevant, leading to the redesigned Wasm tests in JetStream 3 that emphasize integrated performance measurement.
What changes in modern web usage prompted the update?
The web has evolved significantly since JetStream 2's release. WebAssembly is no longer limited to gaming or computationally heavy tasks; it now powers critical libraries for image decoding, UI frameworks, and even small helper functions. These use cases demand fast startup times because Wasm is loaded as part of the initial page load. The old benchmark's assumption that users would accept a long one-time compile cost no longer holds. Moreover, web applications have grown in scale, with more code and larger datasets. JetStream 3 incorporates these shifts by including tests that reflect real-world patterns, such as multiple small Wasm modules, asynchronous loading, and interleaved JavaScript/Wasm execution. The update ensures that benchmark results correlate with actual user experience on today's web.
How does JetStream 3 differ in measuring performance?
JetStream 3 introduces several key changes to performance measurement. First, it replaces the separate Startup and Runtime scores for Wasm with a unified score that combines both phases seamlessly. Second, it uses a more precise timing mechanism than Date.now() to avoid the infinity problem, typically employing high-resolution timers. Third, the suite includes a more diverse set of workloads: larger JavaScript benchmarks, more realistic WebAssembly tasks, and tests that simulate modern web applications like rich media editing, data visualization, and collaborative tools. Finally, the scoring formula has been revised to give a balanced weight to different components, ensuring that no single test can dominate the overall result. These changes make JetStream 3 a more accurate and fair tool for benchmarking browser engines.
What role did WebKit, Google, and Mozilla play?
The development of JetStream 3 was a collaborative effort among the three major browser engine teams: WebKit (Apple), Google Chrome, and Mozilla Firefox. Each team contributed expertise and testing to ensure the benchmark suite was fair and representative. WebKit, for instance, focused on improving the Wasm measurement methodology based on their own optimization experience. Google provided insights from Chromium's performance work, while Mozilla contributed their deep knowledge of compiler optimization and runtime design. The joint announcement underscores the shared goal of driving web performance forward in a transparent and vendor-neutral manner. By working together, they created a benchmark that no single engine can easily game, promoting genuine improvements that benefit all users.