Evolving web apps benchmark reveals key challenges for coding AI
EvoWeb: A Benchmark for Coding Agents on Evolving Web Applications
Artificial IntelligenceSoftware Engineering
Summary
Building web applications usually involves many back-and-forth steps to add new features or fix issues. The authors created EvoWeb, a test that mimics this process with real web apps and many small requests, helping see how coding AIs handle ongoing changes. They find that AI models struggle more as features depend on earlier additions, especially when combining new and old code. This benchmark lets developers measure and improve AI systems over long, complex coding tasks.
What this means in practice
- •For web developers: Evaluate and improve AI tools that assist with incremental web app development involving multiple feature additions over time.
- •For software testing teams: Use EvoWeb to benchmark automated testing agents on their ability to verify evolving web applications with dependent features.
Authors
David Nukrai, Bar Alon, Nativ Levy, Ido Andrew Atad, Ido Cohen, Itamar Zimerman, Lior Wolf, Eli David, Ilan Naiman
Abstract
Web application development often proceeds through successive requests that extend an existing implementation. Yet web-generation benchmarks primarily evaluate development toward fixed specifications. We introduce EvoWeb, a novel benchmark comprising 320 cumulative requests over 32 web applications, grounded in real-world product features and structured by functional dependency graphs. Agents iteratively modify a persistent codebase, while a multimodal web agent tests each intermediate application in a live browser, assessing newly requested functionality, cross-request integration, and previously requested functionality. To characterize reliability throughout development, we integrate trajectory performance across user-tolerance thresholds rather than impose a fixed stopping criterion. Experiments across open- and closed-source frontier models show that EvoWeb reveals clear performance differences between models, with cross-request integration emerging as a key challenge. Further analysis shows that later requests with deeper or more numerous dependencies are solved less accurately, while runtime errors become more frequent over successive requests. EvoWeb provides a testbed for evaluating and improving coding agents in long-horizon, multi-request development.