CoordFormer enables accurate segmentation on huge images with less computing
CoordFormer: Give Me Any Coordinates and I Will Give You Labels
Computer Vision and Pattern Recognition
Summary
Looking at very large, detailed images and labeling every part is hard because it needs a lot of computer power and can miss tiny details. The authors designed CoordFormer, a method that can label any exact spot in these images by combining detailed local features with a broad view of the whole image. This approach keeps the image sharp and detailed while using less memory and computing power. It also smartly focuses effort on edges where details matter most, speeding up the process without losing accuracy.
What this means in practice
- •For remote sensing analysts: Process large satellite images to label land cover accurately and efficiently without needing massive computing resources.
- •For medical imaging teams: Segment very detailed medical scans by focusing computation on critical boundaries to reduce cost and improve precision.
Authors
Iacopo Curti, Pierluigi Zama Ramirez, Alioscia Petrelli, Luigi Di Stefano
Abstract
Semantic segmentation on very-high-resolution images remains challenging due to the high computational cost and the difficulty of capturing fine-grained details. We propose CoordFormer, a novel coordinate-based architecture for semantic segmentation that predicts labels at arbitrary spatial locations through a Coordinate Decoder equipped with a Localized Cross-Attention mechanism. The decoder combines coordinate embeddings with high-resolution local patch features and interacts with global tokens extracted from a downsampled image processed by a ViT foundation encoder, enabling rich semantic context while preserving pixel-level precision. This design enables flexible inference at arbitrary resolutions while keeping memory low on very-high-resolution inputs, and supports an efficient semantic-edge-focused strategy that concentrates computation along boundaries, maintaining fine-grained accuracy while reducing latency and computational cost. CoordFormer achieves state-of-the-art performance on MaSS13K and outperforms comparably sized and higher-parameter methods on DIS5K and KPIs, demonstrating its effectiveness for high-quality, very-high-resolution semantic segmentation.