Active test-time adaptation saves labels by choosing when to ask
WISE-ATTA: When to Ask for Labels in Budgeted Active Test-Time Adaptation
Artificial Intelligence
Summary
Models that adjust to new data while running usually ask for help on every batch, which can be costly if experts must label the data. The authors study how to save labeling effort by deciding when to ask for labels, not just what to label. They propose a method called WISE-ATTA that smartly picks times to ask for labels and which samples need updating most. Their tests show this approach keeps models accurate while using fewer labels.
What this means in practice
- •For machine learning engineers: Save labeling costs by selectively updating deployed models only at important times during operation under changing data conditions.
- •For computer vision teams: Improve image recognition robustness under domain shifts while minimizing human labeling efforts during continuous deployment.
Authors
Muhammad Huzaifa, Lea Schönherr, Thorsten Eisenhofer
Abstract
Active test-time adaptation (ATTA) improves robustness under distribution shift by updating a deployed model during inference while selectively querying supervision. However, most existing ATTA methods implicitly assume that supervision can be requested for every incoming test batch, which can incur substantial annotation cost over long test streams. In this work, we introduce \emph{budgeted ATTA} in which labels are available for only a fraction of test batches. This formulation shifts the central challenge from deciding \emph{what} to label within a batch to deciding \emph{when} supervision should be applied over time. To address this challenge, we propose a budget-aware approach \emph{WISE-ATTA} that allocates supervision over the test stream based on lightweight signals computed online, prioritizing periods where supervision is likely to be most useful. When a batch is selected for supervision, we further employ a drift-based sample selection criterion that targets samples exhibiting ongoing, unconverged adaptation dynamics, enabling effective updates from a single labeled example. We evaluate this approach on synthetic corruptions (ImageNet-C) and natural distribution shifts (ImageNet-R/K/A). Across settings, WISE-ATTA achieves competitive or improved performance compared to recent ATTA methods while requiring substantially fewer labels. Overall, we find that the timing of supervision is a key, yet underexplored, aspect of active test-time adaptation. Code: https://github.com/Muhammad-Huzaifaa/WISE-ATTA