Learning Probabilistic Prompt for Continual Learning
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors study a way for computers to keep learning new groups of images without forgetting what they learned before. They focus on a technique called prompt-based learning, where small sets of parameters (prompts) guide the learning process. They noticed prompts often become too similar, which makes it hard to learn diverse images. To fix this, the authors treat prompts as distributions and create a mixture to generate more varied prompts, also adding a way to keep these distributions stable during training. They tested their method on popular image datasets and found it works better for continual learning.
continual learningprompt-based learningprompt collapseprobabilistic distributionmixture of distributionsdistribution regularizationImageNet-RCIFAR-100CUB-200parameter optimization
Authors
Hyekang Park, Sanghoon Lee, Geon Lee, Jongyoun Noh, Bumsub Ham
Abstract
Continual learning aims to progressively learn from a sequence of tasks, each containing a disjoint subset of classes, while preserving previously learned knowledge. Prompt-based continual learning methods propose to learn a small set of parameters, i.e., prompts, by associating them with a query feature of an input image. These methods optimize the prompts, attempting to represent diverse patterns of images. However, we have observed that existing prompt-based methods suffer from a prompt collapse problem, that is, the prompts tend to be highly similar to each other, thereby failing to capture the diverse data distributions in continual learning scenarios. To address this issue, we propose in this paper a novel prompt-based continual learning framework that captures diverse patterns of images across a sequence of tasks. To this end, we model each prompt as a probabilistic distribution and construct a mixture of these distributions, from which we sample diverse prompts. This enables our model to effectively capture highly diverse image distributions in the continual learning process. We also present a distribution regularization loss to prevent abrupt changes in the prompt distributions throughout the training process. We show extensive experimental results for continual learning on standard benchmarks, including ImageNet-R, CIFAR-100, and CUB-200, demonstrating the effectiveness of our framework.