Fast plug-in module improves AI model adaptation for unknown data
Tail Cues, Principal Corrections: Plug-and-Play Rectification for Open-Set Test-Time Adaptation
Machine LearningComputer Vision and Pattern Recognition
Summary
AI models trained on one type of data often struggle when they see new or different data. This paper presents TALOR, a simple add-on that quickly adjusts features inside the model to handle new or unusual inputs without changing the model itself. The authors show TALOR works well on image datasets with shifts and unknown categories, improving accuracy while using less memory and running faster than previous methods. It can also enhance other adaptation tools when added on afterward.
What this means in practice
- •For machine learning engineers: Improve deployment of models that must adapt to changing data without retraining or heavy memory use by applying a quick, plug-in correction to feature representations.
- •For computer vision teams: Enhance image classification under real-world conditions where new object classes or corrupted images appear, increasing robustness with minimal overhead.
Authors
Yingkai Yang, Chaoqi Chen, Hui Huang
Abstract
Open-Set Test-Time Adaptation (OSTTA) operates on non-stationary target streams where covariate and semantic shifts coexist. Existing TTA methods often obtain adaptation signals by updating model parameters or maintaining mutable memories, which can be costly and vulnerable to unknown samples. We present TALOR, a lightweight, backward-free rectification module that uses the source linear head as a geometric anchor. The head-induced basis decomposes each normalized feature into high-gain principal coordinates and low-gain tail coordinates. TALOR estimates a soft-routed affine correction in the principal subspace, combining a regime-level principal bias with a tail-conditioned slope that captures sample-level tail-to-principal coupling. Using weighted ridge regression, it subtracts this drift estimate from the principal coordinates, reconstructs the feature and feeds it into the source head. On the ImageNet-C benchmark with six csOOD datasets, TALOR outperforms the next-best UniEnt by 2.2% H-score while running 2.5x faster and using only 12% of its GPU memory; as a post-correction plug-in, it further boosts COME to 68.4% H-score with only 2% additional memory overhead.