Adaptive data collection improves learning of controlled nonlinear systems
Information-theoretic receding-horizon active learning of nonlinear dynamical systems
Machine Learning
Summary
Learning how complex systems change over time can be tricky because you only get limited chances to observe them. The authors focus on systems where the exact rules are unknown but can be influenced by inputs, and data is collected along one path. They create a way to update their understanding as they go, using math that estimates uncertainty and chooses experiments to gather the most useful information next. Tests show their approach learns the system more accurately and efficiently than common methods that just try random inputs.
What this means in practice
- •For control system engineers: Design experiments that quickly identify dynamics of systems controlled under uncertain conditions using adaptive, uncertainty-aware input sequences.
- •For robotics developers: Improve model learning for robots operating in noisy environments by actively selecting control actions that maximize knowledge gained during limited trials.
Authors
Juncal Arbelaiz, Anushri Arora, Jonathan W. Pillow
Abstract
Accurately learning nonlinear dynamics from a finite-duration experiment requires the efficient collection of informative data. We address this challenge for stochastic controlled nonlinear dynamical systems whose state is observed along a single trajectory. Our goal is to reconstruct the unknown controlled state-increment map over a prescribed compact subset of state-input space. We construct a parametric estimator of the map using fixed nonlinear features, so that the model is nonlinear in the state and input, but linear in the unknown parameters. A Gaussian prior over the parameters yields recursive Bayesian posterior updates as data stream in, enabling online quantification of predictive uncertainty in the reconstructed dynamics over the target set. We formulate an optimal adaptive-design problem over an information state, using a prediction-oriented acquisition criterion based on the mean marginal mutual information between candidate future trajectories and the reconstructed dynamics over the target set. We then approximate the resulting adaptive-design problem by a non-myopic receding-horizon formulation, evaluate its remaining expectation using a scenario-based sample average, and solve the resulting deterministic program with the cross-entropy method, leveraging parallel candidate-scenario evaluations. Numerical experiments on a noisy multistable system demonstrate that the proposed adaptive information-seeking strategy reduces predictive uncertainty and reconstruction error more efficiently than common excitation baselines under comparable experimental constraints.