Robot adapts efficiently using self-improving simulated training loops
EmbodiRSI: Recursive Self-Improvement for Data-Efficient Robot Adaptation
Robotics
Summary
Robots usually need lots of real-world practice to get better at tasks, which can be slow and costly. The authors created EmbodiRSI, a method that lets robots learn in a simulated environment tailored to their future tasks, then improve themselves by identifying and fixing their mistakes. This repeated cycle of practice and correction in simulation, with occasional real-world fine-tuning, helps the robot get much better using far less real-world data. Their tests showed EmbodiRSI helped robots succeed much more often across different manipulation tasks while using fewer real trials.
What this means in practice
- •For roboticists: Improve robot policy training by iteratively refining behaviors in task-specific simulations before real-world deployment
- •For automation engineers: Reduce costly physical robot demonstration requirements by using simulated feedback loops for error analysis and correction
Authors
Haoran Lang, Haotao Lu, Shiyu Sang, Haoyang Luo, Guo Chen, Qun Li, Jingyi Yu, Ye Shi, Jingya Wang
Abstract
Adapting robot manipulation policies to new tasks and environments remains highly data-intensive, while the data needed for further improvement depends on the policy's current capabilities and failure modes. We introduce EmbodiRSI, an agentic system for recursive self-improvement (RSI) in a real-to-sim-to-real setting, where task-specific simulations are constructed from target deployment scenarios and used as low-cost environments for iterative policy improvement before transfer back to the physical world. EmbodiRSI uses policy execution feedback to guide subsequent experience acquisition and policy updates. Two complementary mechanisms close this loop: Collaborative Error Correction generates agent-assisted corrective trajectories from policy-reached states, while Adaptive Data Collection directs expert demonstration generation toward the current policy's weaknesses. The task-specific simulation serves as a reusable workspace for policy warm-up, repeatable evaluation, failure diagnosis, and targeted data generation across successive RSI rounds. Across three tabletop environments and 14 subtasks, EmbodiRSI increases scene-balanced autonomous simulation success from 50.4% to 83.5% over two RSI updates. With 400 adaptive simulated trajectories and only ten real-world refinement trajectories per subtask, EmbodiRSI achieves 83.1% scene-balanced autonomous real-world success, compared with 75.0% for adaptation using 200 real-world demonstrations per subtask. These results demonstrate that feedback-driven recursive improvement in deployment-specific simulations can enable data-efficient adaptation of embodied policies to physical environments.