Bayesian neural networks learn better online with full covariance tracking

Full-Covariance Smoothing of Bayesian Neural Networks for Online Adaptation

Machine Learning

Summary

Neural networks can learn from data arriving over time, but usually, they treat each part of the network independently, missing how parts relate to each other. This paper presents a method that keeps track of how different neurons depend on one another during learning, improving the network’s ability to adapt quickly and accurately. The authors show this approach works well on tasks like changing classifications, learning dynamics, and adapting policies for vision-language models. Their method updates knowledge efficiently without many repeated training steps.

What this means in practice

  • For online machine learning engineers: Improve real-time adaptation of models to changing data distributions by tracking correlations between neurons for better prediction accuracy.
  • For robotics developers: Enhance robot policy adaptation by efficiently updating vision-language-action models to respond to dynamic environments during operation.

Authors

Oren Wright, Haoming Jing, Qiaoan Shen, Koichiro Niinuma, Yorie Nakahira, José M. F. Moura

Abstract

A neural network's layers can be treated as time steps of a state-space model, turning Bayesian training into a smoothing problem: a forward pass propagates Gaussian moments through the network, and a backward Rauch--Tung--Striebel pass updates the weight posteriors in closed form. Such methods learn from each observation in a single pass, in an uncertainty-aware manner, and without gradient-based iterations or replay, which makes them well suited for online adaptation and data-efficient learning. Existing smoothing-based methods, however, are restricted to diagonal covariances across activations, discarding correlations between neurons. We overcome this limitation via a cross-covariance identity that enables full-covariance propagation through a network's nonlinear activations. We derive a one-step-per-layer smoother that approximates as Gaussian only each layer's affine output, and that applies both to deterministic systems with noisy observations and to stochastic systems described by output statistics. We demonstrate this method in non-stationary classification, online dynamics learning, and policy adaptation of a vision-language-action model, and find that it is generally more accurate than other smoothing-based methods.