Spatial Normalization for Cross-Domain Retinal Layer Segmentation in Optical Coherence Tomography
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
The authors studied how to improve the accuracy of detecting layers in eye scans called Optical Coherence Tomography (OCT), which is important for understanding diseases affecting the brain. They introduced a way to align the scans to a common eye feature (the fovea) before analysis to handle differences between scans from various sources. They tested this method on advanced deep learning models using multiple ways to check segmentation quality, even when exact answers were not available. Their findings show that this normalization step helps make the layer detection more consistent and reliable, which is useful for studying neurodegenerative diseases.
Authors
Iker Moran-Cavero, Monica Hernandez, Elvira Mayordomo, Naiara Artiaga, Beatriz Pardiñas, Beatriz Cordon, Elena Garcia-Martin
Abstract
Retinal layer segmentation in Optical Coherence Tomography (OCT) is a fundamental step for extracting quantitative biomarkers of retinal structure. Indeed, there is a growing interest in the analysis of OCTs in the context of neurodegenerative diseases. However, segmentation remains challenging due to speckle noise, shadowing artifacts, low contrast between adjacent layers, anatomical variability across subjects, and domain shifts arising from different acquisition protocols and clinical populations. While deep learning methods have achieved remarkable performance, their robustness and generalization across heterogeneous datasets remain limited. In this work, we investigate the role of spatial normalization as a preprocessing strategy to mitigate geometric domain shifts and improve the consistency of retinal layer segmentation. Inspired by standard practices in neuroimaging, we introduce a fovea-centered normalization framework that aligns OCT volumes into a common anatomical reference. We perform a comprehensive evaluation of state-of-the-art deep learning architectures. To provide a comprehensive assessment of segmentation quality, we combine conventional overlap-based metrics at B-scan level with topology-aware metrics at A-scan level and thickness-based measures at the en-face level. In cases where a ground truth is not available, we propose topology violation quantitative metrics that do not require ground truth annotations and a thickness-based qualitative assessment that captures structural consistency and clinically relevant patterns at the en-face level. The results demonstrate the importance of spatial normalization in OCT segmentation pipelines toward the development of robust and clinically meaningful retinal analysis tools, enabling reliable biomarker extraction and downstream computational analysis in neurodegenerative research.