Surgical instrument segmentation improves with balanced data approach
Rethinking Data Imbalance in Class Incremental Surgical Instrument Segmentation
Computer Vision and Pattern Recognition
Summary
As more types of surgical tools are introduced, computer programs that identify these tools in images often forget earlier ones or get confused because some tools appear more than others. The authors propose a new method called SurgCSS that balances the training data by mixing tool images with different backgrounds and uses special learning techniques to reduce bias. They also combine old and new model knowledge to improve performance. Their approach was tested on a new combined dataset and showed better results than previous methods.
What this means in practice
- •For medical imaging teams: Enhance surgical tool detection systems to maintain accuracy as new instruments are introduced over time.
- •For computer vision engineers: Develop improved continual learning models that handle class imbalance in incremental segmentation tasks for complex environments.
Authors
Shifang Zhao, Long Bai, Kun Yuan, Feng Li, Jieming Yu, Wenzhen Dong, Guankun Wang, Mobarakol Islam, Nicolas Padoy, Nassir Navab, Hongliang Ren
Abstract
In surgical instrument segmentation, the increasing variety of instruments over time poses a significant challenge for existing neural networks, as they are unable to effectively learn such incremental tasks and suffer from catastrophic forgetting. When learning new data, the model experiences a sharp performance drop on previously learned data. Although several continual learning methods have been proposed for incremental understanding tasks in surgical scenarios, the issue of data imbalance often leads to a strong bias in the segmentation head, resulting in poor performance. Data imbalance can occur in two forms: (i) class imbalance between new and old data, and (ii) class imbalance within the same time point of data. Such imbalances often cause the dominant classes to take over the training process of continual semantic segmentation (CSS). To address this issue, we propose SurgCSS, a novel plug-and-play CSS framework for surgical instrument segmentation under data imbalance. Specifically, we generate realistic surgical backgrounds through inpainting and blend instrument foregrounds with the generated backgrounds in a class-aware manner to balance the data distribution in various scenarios. We further propose the Class Desensitization Loss by employing contrastive learning to correct edge biases caused by data imbalance. Moreover, we dynamically fuse the weight parameters of the old and new models to achieve a better trade-off between the biased and unbiased model weights. To investigate the data imbalance problem in surgical scenarios, we construct a new benchmark for surgical instrument CSS by integrating datasets. Extensive experiments demonstrate the effectiveness of the proposed framework, achieving significant performance improvement against existing baselines.