Update from Hell: Can Coding Agents Survive Hidden Breakage in Dependency Upgrades?
2026-08-31 • Software Engineering
Software Engineering
AI summaryⓘ
The authors explain that updating software libraries that a program depends on can be tricky because changes in the libraries might require changes in the program itself. They created a test called DEPBENCH with 203 real-world examples where such updates need hidden code fixes. When they tested popular automated coding tools on these examples, the best tools only solved about half of the tasks. This shows there is still a big gap between current coding assistants and the needs of real software upgrading.
dependency upgradethird-party librariessource code adaptationsoftware maintenancecoding agentsbenchmarkAPIspackage ecosystemsruntime semanticssoftware reliability
Authors
Zijian Luo, Runzhi He, Pengfei Gao, Yu Kang, Zeqi Lin, Minghua Ma, Qingwei Lin, Saravan Rajmohan, Yongqiang Tian
Abstract
Modern software systems rely heavily on third-party dependencies, but upgrading those dependencies remains a costly maintenance activity. Dependency upgrades do not always preserve the function signatures, type systems, APIs, or runtime semantics assumed by existing code. Consequently, developers often need to perform source code adaptations to accommodate dependency-induced changes. However, such code-level changes are often not explicitly communicated to project maintainers, posing a significant challenge to software reliability. Meanwhile, coding agents have emerged as a new form of software development tool and are increasingly adopted by developers due to their automation capabilities. In this paper, we introduce DEPBENCH, a benchmark consisting of 203 real-world dependency-upgrade tasks across five package ecosystems spanning five language communities, each involving hidden code-level changes that require source code adaptation. We evaluate mainstream coding agents on DEPBENCH. The best completed configuration solves only 104/203 tasks (51.2%), with substantial variation across agent harnesses, models, and ecosystems, highlighting an important gap between current agent capabilities and real-world software maintenance needs.