Voice controlled multi-agent system improves smart operating rooms
A Voice-Interactive Multi-Agent System for Smart Operating Rooms: Architecture Design and Key Technologies
Artificial IntelligenceComputation and LanguageHuman-Computer Interaction
Summary
Operating rooms have many devices and need accurate records, but controlling everything by hand can be hard. This paper presents SurgicalRoomAgent, a system that understands spoken commands to control devices, record operations, and make surgical notes automatically. The authors developed techniques to make the system respond quickly and focus on the right information depending on the surgery stage. Their tests show it works efficiently within the limits of current AI models and meets the timing needs of real surgeries.
What this means in practice
- •For operating room teams: Automate voice control and data recording of multiple devices during surgeries to improve workflow and documentation accuracy.
- •For medical device integrators: Integrate diverse surgical devices into a unified voice-controlled system that adapts prompt instructions by surgical phase and user role.
Authors
Tianxiang Zhou
Abstract
This paper presents SurgicalRoomAgent, a voice-interactive multi-agent system for smart operating rooms based on large language models (LLMs). The system achieves natural language understanding, device control, intraoperative recording, and surgical report generation through a layered architecture comprising a voice interaction pipeline (wake, ASR, turn detection, agent reasoning, TTS) and an agent core (skill registry, task planner, device manager). Three key technologies are investigated: (1) KV Cache prefix warming for low-latency inference, reducing recomputation overhead from approximately 500 ms to tens of milliseconds via byte-level Longest Common Prefix reuse; (2) streaming partial JSON parsing with early parallel task execution, reducing end-to-end latency by approximately 30%; and (3) progressive skill prompt disclosure, which dynamically filters system prompts based on user role, connected devices, and surgical phase to maximize information density within limited context windows. The system is implemented using the Qwen3-27B model with llama.cpp/sglang inference engines. Experimental analysis demonstrates effective operation within a 16,384-token context limit and multi-device parallel control response times meeting OR real-time requirements.