JetStream 3: A New Era for Browser Performance Benchmarking

Benchmarks are the compass for browser engine developers, guiding performance improvements. But the web doesn't stand still, and neither can the tools we use to measure it. JetStream 3 emerges as a critical update—not just a refresh but a fundamental rethink of how we gauge speed, especially for WebAssembly and large-scale web apps. Developed alongside Google and Mozilla, this suite tackles challenges like the infamous 'infinity problem' and reflects how browser engines have evolved. Dive into the key questions below to understand the shift.

What is JetStream 3 and why was it created?

JetStream 3 is the latest version of a cross-browser benchmark suite, co-announced with Google and Mozilla. It replaces JetStream 2, which had become outdated as web technologies and browser engines advanced. The primary goal is to provide a more accurate and challenging performance test for modern web applications. The creation was driven by the realization that old benchmarks no longer reflected real-world workloads—especially for WebAssembly, which is now used in critical paths like libraries, image decoders, and UI frameworks. JetStream 3 introduces new workloads and scoring methods to ensure optimizations translate to genuine user benefits rather than narrow benchmark-specific gains.

How was WebAssembly benchmarking changed in JetStream 3?

JetStream 2 measured WebAssembly (Wasm) in two distinct phases: Startup and Runtime. This split was designed for large C/C++ applications (like games) where users accepted a long one-time startup cost for high throughput. However, over time, browser engines drastically reduced startup times. For smaller workloads, startup became nearly instantaneous. JetStream 3 rethinks this by moving away from a separate startup metric, integrating Wasm workloads that reflect its modern use—quick instantiations for libraries and UI components. The new suite emphasizes overall throughput and responsiveness across varied Wasm tasks, ensuring benchmarks match contemporary usage patterns.

What was the 'infinity problem' in JetStream 2?

The infinity problem emerged because JetStream 2 computed scores using the formula Score = 5000 / Time. When browser engines optimized WebAssembly startup so aggressively that some workloads finished in less than 1 millisecond, the Date.now() timing method rounded down to 0 ms. This caused a division by zero, yielding an infinite score. A patch in JetStream 2.2 clamped the score to 5000 as a workaround. While an infinite score sounds like a win, it revealed that the benchmark had become too easy—browser engines had outgrown the subtests. JetStream 3 eliminates this issue by using more precise timing and workloads that require meaningful computation, avoiding zero-time scenarios.

Why did browser engines outgrow JetStream 2?

Browser engines like WebKit's JavaScriptCore invested heavily in WebAssembly optimization. They made startup times so fast that micro-optimizations once deemed noise became significant—shaving 0.1 ms off a 2 ms instantiation yields a 5% gain. This led to startup times effectively reaching zero seconds for smaller Wasm modules in JetStream 2's tests. Meanwhile, real-world web applications grew more complex, using Wasm in performance-critical roles that demand both fast startup and sustained throughput. JetStream 2's heaviest workloads no longer stressed engines sufficiently, and its scoring inflated results. JetStream 3 introduces larger, more diverse workloads that better reflect today's web, ensuring the benchmark keeps pace with engine improvements.

What does JetStream 3 reveal about modern web applications?

The update signals that WebAssembly is no longer an exotic feature for porting desktop games; it's a core technology in libraries, image decoders, UI frameworks, and other critical path components. Modern web apps are larger and more interactive, requiring balanced performance across parsing, compilation, and execution. JetStream 3's design acknowledges that startup time, while important, must be measured alongside sustained performance in a integrated manner. The suite also includes more JavaScript and DOM-heavy tests to reflect real-world usage. This shift helps developers focus on optimizations that matter for users, such as smoother page loads and faster interactions, rather than chasing artificially low benchmark scores.

How do benchmarks like JetStream 3 drive real performance improvements?

Benchmarks serve as a training ground for browser engine developers. When a suite like JetStream 3 accurately represents web workloads, optimizations made to improve scores also enhance real user experiences. For instance, by eliminating the infinity problem and updating Wasm tests, JetStream 3 encourages engines to optimize for efficient instantiation of small modules without neglecting large-scale processing. The collaborative nature of JetStream—between Apple, Google, and Mozilla—ensures no single browser gains an unfair advantage, fostering healthy competition. Ultimately, these benchmarks help push the entire web platform forward, making sites faster and more responsive for everyone.