Visual feedback improves surgical robot learning from partial successes
Phase-and-First-Arrival VLM Feedback for Sparse-Reward Reinforcement Learning in Surgical Manipulation
Robotics
Summary
Robots learning to perform complex surgical tasks usually get little helpful feedback when they fail, making it hard to improve. The authors propose a method where a visual model identifies how far a robot got in a task and when it reached that point, so it can learn from partial progress instead of treating all failures as the same. They tested this approach in simulated and real surgical scenarios and showed it helps robots complete tasks more successfully. This improves how robots can learn from example attempts without constant detailed rewards.
What this means in practice
- •For robotics engineers: Improve robot training by integrating phase-aware visual feedback to reuse partial task successes in complex manipulation scenarios.
- •For medical device developers: Develop surgical robots capable of learning from incomplete attempts by using visual phase verification for better task success.
Authors
Wanli Liuchen, Fangyuan Wang, Bin Li, Anqing Duan, Yunhui Liu, Peng Zhou, David Navarro-Alarcon
Abstract
Sparse outcome feedback limits what robots can learn from unsuccessful attempts at complex manipulation. Failed multi-stage surgical attempts can contain grasps, lifts, or transfers worth reusing. In sparse-reward reinforcement learning, terminal rewards collapse such attempts to the same outcome, while scalar vision-language model (VLM) ratings reveal neither what progress merits credit nor when it occurred. We introduce phase-and-first-arrival feedback: one VLM query per recorded episode identifies the furthest visually verified task phase and when that phase is first reached, allowing the learner to reuse partial behavior and localize credit. We instantiate it in SurgPhaseBench, a phase-structured suite spanning rigid and deformable tasks, and evaluate it in simulation and hardware. Across five simulated tasks, our method reaches 75.2% mean success, compared with 52.1% for a reward based on Contrastive Language-Image Pre-training (CLIP) using the same visual input; the advantage persists when only the feedback representation changes. On hardware, the same record supports autonomous block picking and slip recovery. Together, these results show that trajectory-level visual supervision can preserve partial progress while providing the temporal credit needed for sparse-reward control.