WebGPU testing finds browser bugs using smart test changes

Dynamic Conformance Testing of WebGPU Through Specification-Driven Mutation

Software EngineeringCryptography and Security

Summary

WebGPU allows web apps to use advanced graphics features from your computer’s graphics card. The current official tests check for normal, correct usage but may miss tricky or rare bugs. The authors created a tool named LANTERN that changes these tests based on the official WebGPU rules and description to try new, edge-case scenarios, some intentionally incorrect. Running these changed tests on a popular browser found real bugs, showing this approach can improve browser reliability for WebGPU.

What this means in practice

  • For web browser developers: Detect hard-to-find GPU-related bugs by automatically generating varied WebGPU test cases from the specification.
  • For graphics driver engineers: Validate GPU driver handling of edge cases and invalid inputs by running specification-guided mutated WebGPU tests.

Authors

Mahya Samdaliri, Zhihao Yao, Kasthuri Jayarajah

Abstract

WebGPU is a low-level graphics and compute API that exposes modern GPU functionality to web applications. While the official WebGPU Conformance Test Suite (CTS) focuses on well-formed usage under the WebGPU specification, it is not designed to stress implementations with semantic edge cases or adversarial inputs. General-purpose fuzzers, in contrast, struggle with WebGPU because of its complex graphics stack and multi-process architecture. We introduce LANTERN, a specification-guided dynamic conformance testing framework that mutates CTS tests using constraints extracted from the WebGPU specification. LANTERN extracts explicit syntactic API rules from WebIDL definitions and recovers semantic constraints, such as command ordering and object lifetimes, from natural-language specification text. Selected rules guide AST-located textual transformations that generate both valid and intentionally invalid CTS variants. We execute the resulting tests at scale on AddressSanitizer-instrumented Chromium to discover bugs. Our evaluation discovers three reproducible bugs, including a heap corruption, in Chromium versions current at the time of study. These results demonstrate that syntactically seeded, semantics-aware mutation of conformance tests provides a way to uncover browser bugs during WebGPU testing.