Computer agents struggle to detect and recover from errors in long tasks
CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use Agents
Artificial IntelligenceComputation and LanguageMachine Learning
Summary
When software agents use computers over long periods and across many programs, small errors can sneak in and cause bigger problems later. The authors created a new test called CUA-Recovery to see how well these agents notice and fix errors during their work. They found that current agents often miss these mistakes and struggle to finish tasks correctly. Training agents on specially made examples improved their error awareness and task success rates, helping them recover better and do tasks from the beginning more successfully. This new benchmark helps evaluate and improve the reliability of computer-use agents over long workflows.
What this means in practice
- •For software developers: Evaluate and improve the reliability of agents that automate complex desktop workflows involving multiple applications by testing their error recovery abilities.
- •For enterprise automation teams: Enhance the robustness of automated computer-use systems by training agents to detect and fix mistakes in long task sequences across desktop environments.
Authors
Xu Hu, Haoming Li, Yuxiang Wang, Jinxin Yang, Kangshuo Li, Licheng Wang, Dingyi Kang, Jiayu Song, Tianhao Wang, Zhanhua Hu, Xu Yang, Qiannan Li, Bingzhe Li, Feng Chen
Abstract
As computer-use agents (CUAs) move to long-horizon workflows spanning multiple applications, undetected errors can silently corrupt persistent state and cause cascading failures in subsequent actions. Reliable error detection and recovery have become critical bottlenecks for building robust CUAs. However, existing benchmarks focus primarily on end-to-end task performance, providing limited insight into whether agents can detect and recover from errors during execution. The few benchmarks that evaluate recovery rely largely on manually injected errors, which may not reflect the types or horizons of failures induced by real agent policies. We introduce CUA-Recovery, a benchmark comprising test and training suites across multiple desktop applications. We construct it using ReRail, a framework for synthesizing long-horizon tasks and trajectories in a persistent desktop environment. ReRail composes MyPCBench tasks into new long-horizon workflows with executable ground truth, collects verifier-confirmed failures from multiple policies, and turns them into verified recovery trajectories. CUA-Recovery comprises a human-verified test suite of 2,250 erroneous states over 100 tasks and a training suite of 8.6K trajectories over 377 tasks. Evaluation on the test suite shows that agents struggle both to recognize inherited errors and to complete the task. Across six agents and six takeover depths, Error Awareness Rate (EAR) reaches at most 44.0% and Pass@3 at most 25.5%. After fine-tuning Qwen3.5-35B-A3B on the training suite, its average EAR rises from 19.9% to 35.2% on held-out recovery states, and its Rubric Score goes from 5.0% to 16.1%. It also gains 15.3 points on MyPCBench and 10.8 points on Odysseys for Pass@3, showing that the training trajectories help both with recovery and with completing tasks from a clean start. Our code is available at https://github.com/xu-hu-2002/CUA-Recovery.