Reinforcement learning improves safety and control in surgical robots
A Reinforcement Learning Framework for Safety-Aware Haptic Shared Control in Surgical Robotics
Robotics
Summary
Controlling surgical robots safely while letting humans stay involved is tricky. The authors created a way to model the robot and human as two players learning to cooperate using reinforcement learning, which means they learn by trying and adjusting without needing exact rules. They found one learning method made the robot follow tasks well but sometimes risked safety, while another was safer but more cautious. Using real human inputs showed some methods worked faster while others kept control commands more aligned. This research helps understand the trade-offs between safety and smooth control when robots assist humans in surgery.
What this means in practice
- •For surgical robot developers: Design new control algorithms that balance safety and human input using multi-agent reinforcement learning in robotic surgery tasks.
- •For medical device safety engineers: Create safety frameworks for shared-control devices by evaluating geometric constraints and operator-robot coordination through reinforcement learning methods.
Authors
Zihao Zhang, Karl Handwerker, Balint Varga
Abstract
Haptic shared control in surgical robotics combines human-side and robot-side forces at a shared end effector while preserving operator involvement. However, the effects of geometric safety choices on task execution and force coordination remain unclear. We model the interaction as a two-agent cooperative Markov game under centralized training and decentralized execution, comparing learned human control with a task-tuned fixed-impedance baseline. This model-free multi-agent reinforcement learning framework supports joint policy adaptation without requiring a complete analytical interaction model. It compares MADDPG and rMAPPO with an epigraph-constrained approach under common geometric task definitions. In simulation, rMAPPO generally achieved the highest return and tracking performance, whereas the epigraph approach produced more conservative motions and lower geometric exposure, including no observed collision or dangerous-region exposure after geometry transfer. With real operator input, fixed-impedance-trained policies were faster and successful in all recorded trials. Policies trained with a learned partner exhibited greater command opposition, while the fixed epigraph condition produced the most aligned commands. This work provides a benchmark for studying human-control representation and constraint-handling trade-offs in haptic shared control.