TailProp adapts vision models with two flexible spatial mixing styles
TailProp: content-adaptive light- and heavy-tailed propagation for vision
Computer Vision and Pattern RecognitionMachine Learning
Summary
Understanding images in computers often involves mixing information across pixels in ways that change for different pictures, parts of the image, or processing steps. The authors propose TailProp, a new approach that combines two different ways of mixing information—one that focuses on nearby pixels and one that reaches far away—to better adapt to different needs. This method mixes these influences efficiently and consistently improves performance on many vision tasks like classification and detection. Tests show using both types of mixing together works better than just one type alone.
What this means in practice
- •For computer vision engineers: Implement new adaptive mixing layers to improve image recognition and segmentation accuracy in vision systems.
- •For autonomous vehicle developers: Enhance robustness and precision of object detection in driving scenes by using TailProp's content-adaptive propagation.
Authors
Jiahao Kong, Zihan Li
Abstract
Science-inspired vision models show that explicit propagation dynamics can provide structured and interpretable alternatives to conventional token mixing. Existing formulations, however, typically construct and adapt visual propagation within a particular dynamical family, while visual representations can require substantially different spatial interactions across samples, channels, and network stages. We explore cross-regime adaptive propagation and introduce TailProp, a hierarchical vision backbone built upon the Tail Propagation Operator (TPO). TPO uses Gaussian and Cauchy stable-process propagators as complementary bases with rapidly decaying and heavy-tailed spatial influence, and predicts a content-conditioned channel-wise coefficient to adaptively combine them. Because this coefficient is spatially shared, the two responses are fused directly in the DCT domain with a single DCT/IDCT pair, yielding $O(N^{1.5})$ spatial mixing for square feature maps with $N=HW$ and fixed channel width. Across image classification, object detection, semantic segmentation, robustness, and cross-backbone restoration, TailProp consistently outperforms matched propagation baselines; TailProp-B reaches 84.4% Top-1 accuracy on ImageNet-1K, 50.3/44.8 box/mask AP under the 3x Mask R-CNN schedule, and 50.8% mIoU on ADE20K. Controlled ablations further show that these gains are not explained by single-basis propagation, an additional same-family branch, or within-family adaptive order alone, supporting complementary two-basis propagation as an effective design principle for visual representation learning.