System improves robots by learning from repeated task failures
SEES: A Self-Evolving Embodied System via Failure-Guided VLA Policy Adaptation
Robotics
Summary
Robots that follow vision, language, and action instructions often struggle with long and complex tasks. The authors introduce SEES, a system that helps robots learn from their mistakes without needing experts to intervene. SEES breaks big tasks into smaller ones and figures out which small parts fail the most. It then uses simulations and smart language models to fine-tune those parts, helping the robot get better over time. This approach works with different robot designs and even improves performance on new tasks.
What this means in practice
- •For robotics engineers: Enable robots to autonomously improve task performance by learning from failures without new expert data, especially for complex multi-stage tasks.
- •For automation system integrators: Upgrade embodied AI solutions to adapt online and reduce downtime by focusing training on failure-prone task segments in simulated environments.
Authors
Ziwen Li, Hanlue Zhang, Zhenyang Ren, Tianyu Huang, Runqi Lin, Haoyu Wang, Zhengqing Gao, Yandong Guo, Fakhri Karray, Tongliang Liu, Chris Russell, Mingming Gong
Abstract
Recent vision-language-action (VLA) policies demonstrate promising generalization across diverse short-horizon tasks. However, they remain unreliable on long-horizon tasks, partly because the large-scale training data is biased toward single-stage manipulation tasks that are cheaper to demonstrate. A single weak atomic skill can cause failures across multiple multi-stage tasks. To address such failures, existing methods often require experts to identify the bottleneck and provide additional demonstrations, making the improvement costly and potentially impractical after deployment. To this end, we present a Self-Evolving Embodied System (SEES) that learns from failures and improves the VLA policy without additional expert demonstrations. SEES decomposes long-horizon tasks into atomic tasks and routes them to corresponding family policies. Each family consists of related atomic skills that share one VLA adapter. During execution, the system automatically monitors atomic-task outcomes to identify the most frequently failing atomic skills as the current bottlenecks. To overcome these bottlenecks, SEES constructs tailored RL tasks in simulation by restoring previously encountered states and generating task-specific success criteria with an LLM. Online RL updates the shared family adapters to promote positive transfer among related atomic skills and cumulative improvement across evolution rounds. Extensive experiments show that SEES can be integrated with different VLA backbones to progressively improve their long-horizon performance. We also observe continued improvement on unseen tasks, providing evidence of transfer beyond the evolution settings.