SAT: Selective Aggregation Transformer for Image Super-Resolution
2026-04-09 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed a new transformer model called Selective Aggregation Transformer (SAT) for improving image super-resolution, which means making low-resolution images clearer. Their method reduces the number of tokens (data points) processed by 97% using a Density-driven Token Aggregation technique, making it much more efficient while keeping important image details. This lets their model look at the whole image without heavy computing costs and still improve image quality better than previous methods. They showed that SAT performs better and uses less computational power than a leading existing approach.
TransformerImage Super-ResolutionSelf-AttentionToken AggregationReceptive FieldComputational ComplexityDensity-driven AggregationKey-Value MatricesHigh-Frequency DetailsWindow-based Attention
Authors
Dinh Phu Tran, Thao Do, Saad Wazir, Seongah Kim, Seon Kwon Kim, Daeyoung Kim
Abstract
Transformer-based approaches have revolutionized image super-resolution by modeling long-range dependencies. However, the quadratic computational complexity of vanilla self-attention mechanisms poses significant challenges, often leading to compromises between efficiency and global context exploitation. Recent window-based attention methods mitigate this by localizing computations, but they often yield restricted receptive fields. To mitigate these limitations, we propose Selective Aggregation Transformer (SAT). This novel transformer efficiently captures long-range dependencies, leading to an enlarged model receptive field by selectively aggregating key-value matrices (reducing the number of tokens by 97\%) via our Density-driven Token Aggregation algorithm while maintaining the full resolution of the query matrix. This design significantly reduces computational costs, resulting in lower complexity and enabling scalable global interactions without compromising reconstruction fidelity. SAT identifies and represents each cluster with a single aggregation token, utilizing density and isolation metrics to ensure that critical high-frequency details are preserved. Experimental results demonstrate that SAT outperforms the state-of-the-art method PFT by up to 0.22dB, while the total number of FLOPs can be reduced by up to 27\%.