Image segmentation improved by combining frequency and spatial insights
FSANet: Frequency-Spatial Aware Network for Image Segmentation
Computer Vision and Pattern Recognition
Summary
Image segmentation is difficult when things block each other, lighting is bad, or objects have irregular shapes. The authors designed FSANet, a new system that looks at images in two ways—frequency and spatial details—to better identify objects and edges. They included special parts to remember missing details, separate important features from noise, and make edges clearer. They also created a new dataset called SceneX to test segmentation in tough real-world conditions. Together, these contributions help make image segmentation more accurate and reliable.
What this means in practice
- •For autonomous vehicle teams: Improve scene understanding in challenging conditions by integrating frequency and spatial information for better obstacle detection.
- •For security system developers: Enhance surveillance video analysis by refining object boundaries and reducing noise effects in low-light or obstructed scenes.
Authors
Ruibo Wang, Ziyi Shen, Huaming Wu, Dong Liang, Kun Shang
Abstract
Image segmentation remains challenging due to occlusions, poor lighting, and irregular structures. Although transformer-based methods achieve high accuracy, they rely heavily on long-range spatial features, leading to high computational costs and neglecting prior knowledge or noise patterns, resulting in missing details and unclear boundaries. To address these issues, we propose Frequency Spatial Aware Network (FSANet), which integrates prior knowledge with a dual-domain solver to sequentially adapt to diverse segmentation tasks. Specifically, we design three key modules: (1) Structure Prior Module, which recovers overlooked details; (2) Dual-Domain Awareness Module, which captures salient features while disentangling noise; and (3) Edge Estimation Module, which enhances edge awareness for more precise segmentation. In addition, the limited availability of comprehensive segmentation datasets covering various real-world scenarios hinders the performance of existing methods. To address this, we introduce SceneX, a novel open-source dataset featuring 10 challenging non-ideal scenarios, establishing a new benchmark for evaluating and improving the robustness and real-world applicability of the segmentation models. Extensive experiments demonstrate the efficiency and effectiveness of FSANet.