Sterilizable Scene Graph Generation for Operating Rooms
2026-08-17 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created a new, much smaller model called SG-NCA to understand surgical videos by identifying objects and how they relate to each other. Unlike existing big models, SG-NCA uses Neural Cellular Automata, which is less demanding and can run on simple, fanless devices that are safer and more practical in operating rooms. They tested SG-NCA on videos of eye and gallbladder surgeries and found it performed similarly to larger models but was 55 times smaller. This shows it could help with real-time video analysis and privacy in hospitals without needing bulky hardware.
Scene Graph GenerationSurgical Video AnalysisNeural Cellular Automata (NCA)Multiclass SegmentationEdge ComputingFanless DevicesOperating Room HygieneRelation PredictionModel Parameter EfficiencyIntraoperative Scene Understanding
Authors
Nick Lemke, Ssharvien Kumar Sivakumar, Antoine P. Sanner, John Kalkhof, Henry John Krumb, Ghazal Ghazaei, Anirban Mukhopadhyay
Abstract
Scene graph generation from surgical video enables a holistic and structured understanding of surgical scenes by modeling objects and their semantic relationships. Despite recent advances, state-of-the-art approaches rely on large, parameter-heavy deep learning models that are impractical for deployment in the operating room (OR) due to hardware footprint, hygiene constraints, latency, and data privacy concerns. To the best of our knowledge, this is the first scene graph generation method built on NCAs and the first NCA framework capable of learning structured representations. We introduce SG-NCA, a lightweight scene graph generation framework based on Neural Cellular Automata (NCA), designed for inference in fanless devices critical for OR hygiene protocols. SG-NCA is the first scene graph generation combining NCA-based multiclass segmentation for efficient object detection and feature extraction with a lightweight relation predictor. We evaluate SG-NCA on videos of cataract surgery and cholecystectomy, demonstrating performance comparable to established baselines while requiring 55x fewer parameters. We showcase deployment on fanless edge devices better suited for the OR and demonstrate downstream applications such as surgical video captioning, highlighting SG-NCA's potential for affordable, privacy-preserving, and OR-ready intraoperative scene understanding.