Frequency guided fusion improves semantic image segmentation accuracy
FGD2HF: Frequency-guided Dual-domain Hypergraph Fusion for Semantic Segmentation
Computer Vision and Pattern Recognition
Summary
Semantic segmentation is the task of labeling every pixel in an image with a category like road or person. The authors found that using information from the frequency domain—how image features behave at different scales—can help this task. They created a new method called FGD2HF that combines frequency features with spatial features more effectively and efficiently than before. Their approach improves the precision of segmenting boundaries and overall accuracy, while needing less computation than comparable methods.
What this means in practice
- •For autonomous vehicle engineers: Enhance scene understanding by more accurately segmenting roads, pedestrians, and obstacles with efficient frequency guided fusion.
- •For digital mapping teams: Improve urban scene mapping by generating clearer segmented images with better boundary accuracy using low-computation semantic segmentation.
Authors
Xiaofan Yang, Wei Pan, Yubin Liu, Junyu Wu, Junming Zhang, Jie Zhao
Abstract
Frequency-domain information is beneficial for semantic segmentation. However, existing frequency-based methods often suffer from insufficient frequency utilization, inefficient fusion, and high computational complexity. To address these issues, we propose FGD2HF, a dual-domain fusion network that leverages frequency features to guide spatial-domain fusion and enhancement. Specifically, we introduce a novel Hybrid-Attention Encoder (NAHA) that jointly models local and global relationships through neighborhood and hypergraph attention, enhancing the network's ability to extract effective features. We further introduce a Wavelet Transform and Enhancement (WTE) module to extract and refine low- and high-frequency cues from preceding-stage features, followed by Frequency-guided Dual-domain Fusion (FGDF) modules that use frequency cues to condition current-stage spatial grouping and feature propagation. This design incorporates both global and local frequency guidance into spatial feature modulation, enhancement and fusion with low computational cost. Extensive comparisons with numerous state-of-the-art semantic segmentation methods demonstrate that our method is highly competitive. Our full FGD2HF-B model achieves 48.7% mIoU on ADE20K (+0.3% over SegMAN-B with 18% fewer GFLOPs), 83.2% mIoU on Cityscapes (comparable to SegMAN-B with 21% fewer GFLOPs), and 1.1% higher mIoU than VWFormer-T on SBD with 21% fewer GFLOPs, while also achieving significant improvements in boundary segmentation accuracy.