Robot policy recovery benchmark tests responses to execution errors
RoboRecover: Benchmarking Robot Policy Recovery under Execution Deviations
Robotics
Summary
Robots often face unexpected situations while working, such as objects moving differently than planned. The authors created RoboRecover, a tool that tests how well robot control programs can recover if they encounter such unexpected changes during a task. Instead of only judging success by whether a robot finishes a task from the start, RoboRecover looks at how the robot handles mid-task surprises. The authors show that performing well from the beginning doesn’t guarantee good recovery abilities, making recovery an important new way to evaluate robot intelligence.
What this means in practice
- •For robotics development teams: Evaluate robot control programs for their ability to recover from unexpected execution errors during tasks using the RoboRecover benchmark.
- •For industrial automation engineers: Test recovery capabilities of control policies handling off-nominal object interactions to improve reliability in manufacturing robots.
Authors
Yang Li, Chen Zhao, Zhuoran Wang, Jiankang Wang, Chao Shao, Yihan Lin, Haitao Shen, Jing Zhang
Abstract
Robot-policy benchmarks increasingly cover diverse tasks and preset out-of-distribution conditions, but typically evaluate complete trajectories from predefined initial states. These evaluations often focus on the initialized scene and the final outcome, while paying less attention to the dynamic interaction process. During closed-loop execution, actions and contacts can alter object relations and task progress, producing off-nominal intermediate states that need recovery. Recovery requires a policy to infer how task progress has changed, correct the relevant relations, and continue the original goal. We introduce RoboRecover, a benchmark for robot policy recovery under execution deviations. RoboRecover selects deviation states from trajectories, reconstructs them by replaying action prefixes, and evaluates policies on the original task. RoboRecover contains 2,000 scenarios across RoboTwin and LIBERO, with 1,000 scenarios and a fixed 800/200 train/test split on each platform. Results show that initial-state performance does not determine recovery performance and policies exhibit different recovery strengths across scenarios. Using its training split, RoboRecover further supports study on recovery interventions. RoboRecover establishes recovery from execution-induced intermediate states as a distinct dimension of robot policy evaluation.