Visual bayesian regression improves forecasting of physical and visual data
ViBR-WM: Visual Bayesian Regression for World Modeling
Computer Vision and Pattern RecognitionMachine Learning
Summary
Predicting how things change over time, like the movement of objects or sunlight levels, can be tricky when the data is both visual and physical. The authors combine images with physical measurements using a special kind of statistical method that can understand trends and patterns like seasons or cycles. Their approach also carefully selects which information is important, making predictions more accurate and reliable. Tests on tasks including object motion, plant greenness, and solar power showed their method beats several existing techniques.
What this means in practice
- •For climate modeling teams: Forecast vegetation health and solar power output by integrating images and physical data more accurately over time.
- •For autonomous vehicle developers: Improve prediction of object motion by combining visual and physical sensor data with interpretable regression.
Authors
Jifan Li, Ning Ning
Abstract
Modeling temporal dependence and uncertainty is central to forecasting with world models. The Visual Bayesian Regression World Model combines visual features, physical histories and known covariates through interpretable regression, within a modular architecture supporting trend, seasonal and cycle dynamics. Visual compression reduces representation dimension, while Bayesian variable selection reduces active regression dimension. Posterior prediction combines forecasts across predictor subsets using their posterior probabilities as weights and accounts for parameter uncertainty and future disturbances. The model forecasts joint visual--physical states recursively and physical targets directly. Across four forecasting tasks spanning object motion, vegetation greenness and solar power, ViBR-WM achieves lower mean overall physical-target error than Temporal Straightening, ConvLSTM, PredRNN and SimVP on every task. Repeated fitting and resampling support these overall gains.