Stochastic nonlinearities enhance uncertainty estimation and preserve accuracy
Stochastic Nonlinearities Improve Uncertainty Estimation
Computer Vision and Pattern Recognition
Summary
Knowing how sure a computer is about its picture recognition helps make better decisions. The authors show that by letting parts of a neural network randomly choose different activation steps, the model can better guess when it’s uncertain. This method doesn’t need extra complex training or big systems and keeps the model working well on its main task. They tested it on several tasks like classifying images and spotting unusual data, showing it improves uncertainty estimates.
What this means in practice
- •For machine learning engineers: Improve confidence measures in image classification models without adding complex architectures or ensemble training.
- •For computer vision teams: Enhance detection of unusual or unseen visual inputs by integrating stochastic nonlinearities into existing recognition systems.
Authors
Mohammad Sarhangzadeh, Amirreza M. Shebly, Doruk Oner
Abstract
Reliable uncertainty estimation is important across visual recognition tasks, providing information about the confidence and reliability of model predictions. Many uncertainty estimation methods obtain predictive diversity through mechanisms that can suppress intermediate features, perturb the input, or require substantial additional computation. Stochastic activation functions, where network units randomly select between multiple nonlinearities during a forward pass, have previously been used to improve optimization behavior, mitigate limitations of fixed ReLU activations, and increase output diversity. In this work, we reframe stochastic activation switching as a lightweight mechanism for uncertainty estimation. The proposed method samples between nonlinear activation functions, inducing predictive diversity while keeping intermediate feature pathways active. This enables uncertainty estimation from a single trained model without the training cost of ensembles, task-specific uncertainty heads, or architectural redesign. We evaluate the proposed approach across classification, semantic segmentation, and out-of-distribution detection using convolutional and transformer-based architectures on diverse datasets. Across these settings, stochastic activation switching improves uncertainty quality while preserving competitive primary-task performance, showing that stochastic nonlinearities provide an effective and information-preserving source of predictive variation for visual recognition.