Digital twin boosts robot learning with visual rewards in lab tasks
TwinBridge: Digital Twin-Driven Policy and Reward Transfer for Real-World Reinforcement Learning in Laboratory Robotics
Robotics
Summary
Teaching robots to perform complex tasks in labs is hard and slow because real-world practice is costly and feedback is limited. The authors created TwinBridge, which uses a computer-generated model (a digital twin) to train robots first in simulation and to create detailed visual hints that help robots learn faster in real life. They tested TwinBridge on five lab tasks and found it helped robots succeed almost all the time after just a short time learning in the real world. The digital twin helps by providing rich information on how well the robot is doing, making training more efficient and effective.
What this means in practice
- •For laboratory automation teams: Improve robot training speed and success in handling precise lab manipulations by using digital twins to provide richer feedback and pretraining.
- •For industrial robot developers: Use digital twin models to jumpstart reinforcement learning policies and generate detailed visual rewards that reduce real-world trial time.
Authors
Shu'ang Sun, Yibo Qiu, Haoliang Ye, Zan Huang, Ao Tian, Kan Ding, Ronald Xu, Mingzhai Sun
Abstract
Online reinforcement learning offers an effective approach to precise laboratory manipulation, yet costly real-world demonstrations and limited access to dense process rewards hinder its practical application. We present TwinBridge, a digital twin-driven framework that supplies policy initialization and dense visual rewards for real-world online learning. We construct digital twins by combining 3D Gaussian splatting with physics simulation. Here, simulator states provide task events and geometric progress under physical prerequisites and ordering constraints, yielding rewards for policy pretraining with simulated demonstrations and reinforcement learning. We distill this supervision into a visual reward model. During real-world interaction, its predictions are combined with execution history to compute dense rewards for human-in-the-loop policy refinement. Across five real-world laboratory tasks, TwinBridge achieves a final mean autonomous success rate of 98% with at most 25 minutes of real-world interaction per task. Experiments show that dense rewards improve interaction efficiency over sparse rewards, and simulation-based policy initialization further accelerates learning.