Multi group temperature scaling improves confidence in continual learning
MT-CC: Multi-Group Temperature Scaling for Asymmetric Calibration Behavior in Class-Incremental Learning
Machine LearningArtificial Intelligence
Summary
When AI models learn new things over time, they sometimes get overconfident or underconfident in their predictions, which can be misleading. The authors found that using a single adjustment for confidence doesn’t work well as tasks change. They created a method called Multi-Group Temperature Scaling (MT-CC) that adjusts confidence differently for groups of examples based on how the model feels about them and which task they come from. This makes the model’s confidence more trustworthy without hurting its accuracy.
What this means in practice
- •For machine learning engineers: Improve model trustworthiness by better calibrating confidence over sequential learning tasks.
- •For autonomous system developers: Ensure reliable confidence estimates in continual learning models used for evolving decision-making environments.
Authors
Sungjun Yun, Yonghee Choi, Dong-Jun Han
Abstract
As deep neural networks are required to continuously adapt to evolving data, class-incremental learning (CIL) has become an active research area. However, most existing works primarily focus on improving accuracy while overlooking confidence calibration, which measures the trustworthiness of the model's predictions. Although temperature scaling (TS) is effective for static calibration, it has rarely been explored in continual settings where data distributions evolve over time. In this paper, we introduce Multi-Group Temperature Scaling for Continual Calibration (MT-CC), a novel post-hoc calibration framework that addresses the asymmetric calibration behavior in CIL, where the rate of confidence changes does not align with the rate of accuracy degradation across tasks, leading to different levels of miscalibration. Our key contribution is to design multiple temperature parameters through a lightweight grouping network that separates samples by confidence statistics and task-aware signals in CIL. Furthermore, a Groupwise Difference-between-Confidence-and-Accuracy (GDCA) regularization is incorporated to promote intra-group consistency. Experiments demonstrate that MT-CC significantly improves calibration performance while maintaining accuracy, offering a principled framework for reliable continual learning.