Summary
Robots trained in simulation often learn using detailed information about their environment that isn’t available from camera images when they operate in the real world. Instead of training a new vision-based robot controller to copy the actions of this simulation-trained expert, the authors propose keeping the expert and teaching a visual system only to estimate its needed state information. This approach helps the robot perform tasks more successfully since it focuses on making state estimates that matter for actions, not just matching their values. They demonstrate better results in various simulated tasks and on a real robot without retraining the expert controller.
What this means in practice
- •For robotics engineers: Build more robust robot manipulation systems that use simulated control policies combined with learned visual inputs to improve real-world task success without retraining.
- •For manufacturing automation teams: Deploy robots capable of complex, contact-rich tasks using camera observations by leveraging pretrained control modules with a learned vision interface, reducing development time.
Authors
Denis Shcherba, Adrian Abel, Eckart Cobo-Briesewitz, Wojciech Samek, Marc Toussaint
Abstract
Simulation-trained manipulation policies can exploit privileged state information to learn effective contact-rich behaviours, but deployment requires acting from partial observations such as noisy camera images. A common solution is teacher-student distillation, in which a visuomotor policy is trained to reproduce the actions of the privileged expert. This requires the student to jointly infer the task-relevant state and relearn the expert's action mapping that is already available. An alternative is to reuse the state-based expert and learn only a perceptual interface that reconstructs its missing state inputs. However, minimising the state estimate error alone does not necessarily minimise the downstream control error induced by these estimates. To bridge this gap, we train a visual state estimator using both direct state supervision and an action-consistency loss backpropagated through the frozen, differentiable expert. A scheduled objective first establishes a physically meaningful state estimate and progressively emphasises errors that affect the expert's actions. Across five goal-conditioned manipulation tasks, retaining the expert consistently outperforms direct pixel-to-action imitation from the same expert demonstration corpus. We further demonstrate sim-to-real transfer on a physical Panda robot, achieving 76% success without retraining the underlying expert.