Vision based robotic surgery control learns better from state policies
Revisiting Distribution Matching for State-to-Vision Policy Distillation in Laparoscopic Robotic Manipulation
RoboticsComputer Vision and Pattern Recognition
Summary
Controlling surgical robots using video alone is very hard and needs lots of expert examples. The authors show that teaching vision-based robot controllers by mimicking controllers that use detailed robot info (states) helps overcome these challenges. This method uses fewer expert examples and produces stable, consistent robot performance in simulated laparoscopic surgery tasks. They found that training vision-based controllers directly with reinforcement learning is difficult, but their distillation approach improves learning.
What this means in practice
- •For medical robotics engineers: Develop robotic surgical control systems that use camera inputs effectively while relying on learned models from precise robot state policies.
- •For robotics simulation developers: Create simulated environments and tools that help train vision-based robotic manipulation policies through transfer from state-based teacher policies.
Authors
Mingyu Sheng, Jianan Fan, Dongnan Liu, Guoyan Zheng, Ron Kikinis, Weidong Cai
Abstract
End-to-end control policies trained via Reinforcement Learning (RL) have emerged as an alternative to conventional surgical robotic control approaches, with recent studies exploring state-based policies trained on privileged state observations. However, such approaches rely on precise physical states that are difficult to obtain reliably in surgical environments. Vision-based RL provides an alternative by learning manipulation policies directly from RGB observations, avoiding explicit state estimation and manually engineered perception pipelines while enabling policies to acquire task-relevant visual representations. Nevertheless, obtaining vision-based policies via pure RL methods for surgical manipulation remains challenging. Thus, imitation learning (IL) becomes another solution, yet it requires a large number of expert demonstrations, which are difficult to obtain in minimally invasive surgery. In this work, we propose a framework for state-to-vision policy distillation in laparoscopic robotic manipulation. Specifically, a state-based policy with privileged observations is employed as the teacher, and a vision-based policy learns from RGB observations and robot proprioceptive information, which requires less expert demonstrations than IL and introduces superior performance than RL methods. We conduct extensive experiments on various surgical manipulation tasks in the simulated environment. Experimental results show that, in the context of laparoscopic robotic manipulation, training vision-based policies with general-purpose RL methods is challenging, whereas our proposed state-to-vision policy distillation leads to more consistent and stable performance across tasks.