Image prototypes help models adapt to new data without forgetting
Distilling Image Prototypes for Guided Test-Time Adaptation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Machine learning models often struggle when used in new situations different from what they were trained on. This paper presents a method where a small set of synthetic images acts like a memory anchor, helping the model remember old knowledge while learning from new data. This approach also improves the way the model judges its own uncertainty, reducing mistakes from wrong guesses. The authors’ method shows better results than other techniques, especially when conditions change a lot.
What this means in practice
- •For computer vision engineers: Improve model reliability in real-time systems by maintaining learned knowledge while adapting to new visual conditions.
- •For autonomous vehicle developers: Enhance perception models to adapt safely to new environments or weather by reducing forgetting and error accumulation.
Authors
Liwen Wang, Xingbo Dong, Iman Yi Liao, Deyin Liu, Massimo Tistarelli, Lin Yuanbo Wu, Zhe Jin
Abstract
Test-Time Adaptation (TTA) enhances the robustness of models against distribution shifts but faces two critical challenges: error accumulation from noisy pseudo-labels and catastrophic forgetting of source knowledge. Uncertainty-based approaches designed to mitigate error accumulation often yield overconfident or computationally expensive estimates, while strategies intended to prevent forgetting via prototype replay rely on static representations that easily become misaligned as the model adapts. To address these issues, this paper proposes a novel framework, Distilling Image Prototype for Guided Test-Time Adaptation (DIPTTA). The core of the proposed approach is the introduction of a Distill Image Prototype (DIP), a compact set of synthetic images that serves as a dynamic and regenerative anchor of source knowledge. This prototype enables a dynamic feature replay mechanism that continuously generates feature prototypes aligned with the current state of the model, thus effectively preventing catastrophic forgetting. Furthermore, the DIP anchors a source-calibrated uncertainty estimation method, which provides a less biased measure of sample reliability by leveraging stable source knowledge, thereby robustly suppressing error accumulation. Extensive experiments on multiple benchmarks demonstrate that DIPTTA significantly outperforms state-of-the-art methods, particularly under severe domain shifts. The source code is available at https://github.com/LiwenWang919/DIPTTA.