CoordFormer improves labeling on ultra-high-resolution images efficiently
CoordFormer: Give Me Any Coordinates and I Will Give You Labels
Computer Vision and Pattern Recognition
Summary
Labeling parts of very large images is hard because it needs a lot of computer power and must keep tiny details clear. The authors created CoordFormer, a new system that can label any spot in the image by combining detailed local patches with a bigger, low-resolution view to understand context. It uses a smart attention method to focus on important edges, which saves time and memory. CoordFormer works well on several tests, beating other systems while using fewer resources.
semantic segmentationvery-high-resolution imagescoordinate embeddingscross-attentionVision Transformer (ViT)local patch featuresglobal contextefficient inferencesemantic edgescomputational cost
Authors
Iacopo Curti, Pierluigi Zama Ramirez, Alioscia Petrelli, Luigi Di Stefano
Abstract
Semantic segmentation on very-high-resolution images remains challenging due to the high computational cost and the difficulty of capturing fine-grained details. We propose CoordFormer, a novel coordinate-based architecture for semantic segmentation that predicts labels at arbitrary spatial locations through a Coordinate Decoder equipped with a Localized Cross-Attention mechanism. The decoder combines coordinate embeddings with high-resolution local patch features and interacts with global tokens extracted from a downsampled image processed by a ViT foundation encoder, enabling rich semantic context while preserving pixel-level precision. This design enables flexible inference at arbitrary resolutions while keeping memory low on very-high-resolution inputs, and supports an efficient semantic-edge-focused strategy that concentrates computation along boundaries, maintaining fine-grained accuracy while reducing latency and computational cost. CoordFormer achieves state-of-the-art performance on MaSS13K and outperforms comparably sized and higher-parameter methods on DIS5K and KPIs, demonstrating its effectiveness for high-quality, very-high-resolution semantic segmentation.