PAC Bayesian meta learning speeds up identifying limited data systems
PAC-Bayesian Meta-Learning for Few-Shot Identification of Linear Dynamical Systems
Machine Learning
Summary
Identifying how systems change over time can be tough when data is limited, noisy, or complicated. The authors created a method that learns from many similar systems to quickly understand a new one using only a small amount of data. Their approach also gives a way to measure how confident we are in these understandings. They developed new math tools to handle data that changes over time rather than being random and independent, which helps in making better predictions about system behavior.
What this means in practice
- •For control engineers: Improve tuning of controllers for new machines by quickly identifying system dynamics from minimal operational data.
- •For predictive maintenance teams: Generate reliable models of machine behavior in early operation stages to anticipate failures using few observations.
Authors
Chenfeng Huang, George Michailidis
Abstract
Identifying linear time-invariant (LTI) dynamical systems is challenging when trajectories are short, noisy, or high-dimensional. Traditional system identification typically treats each system independently and cannot exploit shared structure across related systems. We propose PBML-LTI, a PAC-Bayesian meta-learning framework for few-shot LTI system identification that learns a transferable prior over task-specific dynamics while preserving task heterogeneity. Each task corresponds to an unknown LTI system, and the meta-learner uses training trajectories to learn a data-dependent prior over transition matrices. For a new system with limited data, PBML-LTI performs Bayesian adaptation under this prior to obtain a task-specific posterior, providing accurate estimates and principled uncertainty quantification. A key challenge is temporal dependence, since LTI trajectories violate the i.i.d. assumptions underlying most PAC-Bayes meta-learning analyses. We address this with a martingale PAC-Bayes analysis for dependent trajectory losses and derive a support-query predictive-risk bound that motivates a fit-KL meta-training objective. The bound clarifies the roles of empirical fit, posterior complexity, and prior quality in few-shot adaptation under sequential dependence. We further derive corollaries for transition-matrix recovery and multi-step trajectory prediction, connecting uncertainty-aware meta-identification with finite-sample guarantees for dependent dynamical data.