Dynamic Distribution-Aware Uncertainty Tracking in Vision-Language Representation Learning
2026-08-10 • Machine Learning
Machine Learning
AI summaryⓘ
The authors focus on measuring how confident Vision-Language Models (VLMs) are in their predictions, which is important for using these models safely. They point out that current methods usually assume the test data stays the same as the training data, which isn't always true. To fix this, they created a new method called DDA-UQ that adapts to changing data by modeling the distribution of the model's internal data during training and updates uncertainty estimates when new test data appears. Their tests show this approach works better than previous methods.
Uncertainty QuantificationVision-Language ModelsPost-hoc MethodsGaussian Mixture ModelEmbedding SpaceData DistributionDynamic AdaptationModel Confidence
Authors
Ao Zhou, Zhiwei Jiang, Zifeng Cheng, Cong Wang, Shufan Yang, Haoru Chen, Qing Gu
Abstract
Uncertainty Quantification (UQ) aims to measure the reliability of model predictions, serving as a critical safeguard for deploying Vision-Language Models (VLMs) in safety-critical scenarios. Post-hoc approaches are widely adopted due to their lightweight nature, mapping the outputs of VLMs to uncertainty measures through learnable modules or inductive summarization. However, Post-hoc approaches remain inherently confined to fitting the failure patterns of the source domain, ignoring the dynamic nature of test distributions. To address this challenge, we propose a Dynamic Distribution-Aware Uncertainty Quantification framework (DDA-UQ) that shifts the paradigm from static mapping to a dynamic distribution-aware process. During training, we leverage a Gaussian Mixture Model to model the VVLMs'embedding space and extract distributional evidence, thereby dynamically deriving uncertainty estimates. During inference, the design dynamically responds to changes in the data distribution. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods.