Continual Model Merging with Test-Time Adaptation for Whole-Slide Image Analysis
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors studied how to combine models trained on different tasks without keeping old training data, using a method called model merging with test-time adaptation. They tested this approach on cancer image classification, comparing it to traditional continual learning methods. Their findings show that adapting the merged models during testing helps remember past knowledge and perform well, but results vary depending on the order of tasks and how well the model balances new and old information. This work suggests model merging with test-time adaptation is a useful approach for continual learning in medical image analysis.
model mergingcontinual learningtest-time adaptationwhole slide image classificationdomain shiftrehearsal-free learningcancer subtypingTCGA datasettask incremental learningclass incremental learning
Authors
Duc-Thanh Le, Doanh C. Bui, Maï K. Nguyen, Khang Nguyen
Abstract
Model merging offers a practical alternative to conventional continual learning by integrating independently fine-tuned models without retaining previous training data. Recent state-of-the-art model merging methods employ test-time adaptation (TTA-guided merging) to address distribution shifts by adjusting merging-related variables using unlabeled target data. However, these methods have primarily been studied in multi-task or single-target settings, and their behavior under sequential continual learning remains insufficiently understood. We present a benchmark study that maps this family of methods to rehearsal-free continual Whole Slide Image classification and evaluates them against traditional continual-learning approaches. Experiments on six TCGA cancer-subtyping cohorts cover CLASS-IL and TASK-IL scenarios, in-domain and out-of-domain evaluation, and different task orders. The results show that adapting model merging at test time can provide strong task-specific performance and improve retention of previously acquired knowledge without storing historical WSIs. Nevertheless, performance remains sensitive to task order and to the interaction between adaptation on the current distribution and accumulated knowledge. This benchmark identifies model merging with test-time adaptation as a promising direction for continual computational pathology and motivates future methods that balance adaptation to domain shift with explicit preservation of historical knowledge.