HTT-Net: Hierarchical Text-guided Transition Modeling for Surgical Video Phase Recognition
Computer Vision and Pattern Recognition
Summary
The authors developed HTT-Net, a system to better understand different stages in surgical videos by using detailed surgical knowledge. Their method groups video frames into meaningful segments based on surgical phases and their transitions, helping to reduce confusion caused by unclear visuals. They tested their approach on two surgical video datasets and showed it improves the accuracy and consistency of recognizing surgical phases. This work helps computers follow the surgery workflow more precisely, which could assist medical professionals during operations.
Authors
Kunjie Deng, Jinghui Zhang, Weidong Chen, Ganbin Li, Xiangjun Lyu, Zhendong Mao, Yingchi Yang
Abstract
Surgical video phase recognition is a fundamental task in computer-assisted intervention, supporting workflow understanding, intraoperative guidance, and surgical quality assessment. Although recent visual-temporal models have achieved promising progress, accurate and temporally coherent phase recognition remains challenging due to local visual ambiguity, transient prediction noise, and insufficient use of procedural semantics. To address these challenges, we propose HTT-Net, a Hierarchical Text-guided Transition modeling Network for surgical video phase recognition. The key idea is to introduce structured surgical semantic knowledge into phase-aware segment construction and semantic refinement. Specifically, we construct a hierarchical surgical semantic memory with intra-phase descriptions, inter-phase transition descriptions, and fine-grained semantic units. Based on this memory, the proposed Transition-Aware Segment Construction (TAS-Con) organizes frame-level evidence into coherent segment representations and handles boundary clips with inter-phase transition descriptions. Furthermore, we introduce Transition-Aware Segment Calibration (TAS-Calib), which calibrates phase-aware segment representations through hierarchical surgical semantics and improves discrimination under visual ambiguity without dense frame-level vision-language fusion. Experiments on Cholec80 and LCRS-100 demonstrate the effectiveness of HTT-Net for robust surgical video phase recognition.