Function-Space transformer improves prediction and recognition with adaptive anchors
Function-Space Transformer with Adaptive Anchors
Machine Learning
Summary
Many types of data, like images or physical fields, come as smooth functions but are only observed at scattered points. The authors developed a method called the Function-Space Transformer that learns to represent these functions using flexible, changing reference points called anchors that adjust based on the input. This approach helps the model understand and predict complex patterns better than some previous methods, including for scientific simulations and image recognition. Their experiments showed better accuracy with fewer parameters compared to other strong baselines.
What this means in practice
- •For scientific computing teams: Enhance accuracy in predicting solutions to physical simulation problems like fluid flow by using adaptive function representations.
- •For computer vision engineers: Improve image classification accuracy while lowering model size by using function-space transformers with adaptive feature points.
Authors
Guorui Sang, Pedram Rooshenas
Abstract
Many forms of data, including physical fields, geometric shapes, and visual signals, are naturally described by functions over continuous domains but are observed through discrete samples. Representing these functions on fixed uniform grids imposes a trade-off between resolving localized variation and increasing computation across the domain. Neural operators address this mismatch by learning mappings between functions, while latent-attention architectures provide flexible processing of sampled observations. We introduce the Function-Space Transformer (FST), a framework for learning from functions through a spatially adaptive continuous latent representation. FST stores features at anchors whose locations are predicted from the input observations and recursively refines these anchor features through function-space interactions. This allows the representation to adapt its spatial organization to each input rather than inherit that of the observation grid, while supporting both spatially resolved and finite-dimensional outputs. On PDE solution prediction using PDEBench Burgers and Darcy flow, FST substantially outperforms the Perceiver IO baseline, whose latent representation lacks explicit spatial organization, and is highly competitive with the Fourier Neural Operator. On ImageNet-1K, FST achieves higher classification accuracy than the Vision Transformer baseline, with fewer parameters across these comparisons. Ablations further support the benefits of function-space updates and recursive refinement. Together, these results highlight the potential of adaptive continuous representations for both scientific prediction and visual recognition.