Learning linear systems with missing data through structured constraints
Learning structured linear dynamical systems from missing observations
Machine Learning
Summary
Sometimes, systems that change over time have missing pieces of information, making them hard to understand. This paper looks at how to learn about such systems even when only some parts are seen at each time step. The authors propose a method that corrects for missing data and limits the search to certain structured patterns, which helps make better predictions with less data. They also show that their method works well when following certain rules and conditions.
What this means in practice
- •For signal processing engineers: Improve system identification from partial and irregular sensor data using structured constraints to reduce required observation time.
- •For financial data analysts: Estimate dynamic models from incomplete time series data by exploiting known structure, enabling better forecasting with limited observations.
Authors
Aravinda Kanchana Ruwanpathirana, Hemant Tyagi, Sunny G. W. Wang
Abstract
We consider the problem of learning structured linear dynamical systems over convex sets $\mathcal{K}$, where only a small subset of the observations are available at each time point. An estimator which minimizes a bias-corrected, potentially non-convex objective function is proposed. Non-asymptotic bounds are obtained for the statistical error, which depend on the local complexity of $\mathcal{K}$, the trajectory length $T$, and the sub-sampling probability $p$. Convergence of the projected gradient descent algorithm is also established. The general theory is applied to settings where (i) $\mathcal{K}$ is a subspace, (ii) $\mathcal{K}$ is the set of bi-isotonic matrices, and (iii) $\mathcal{K}$ is the set of matrices whose rows are formed by sampling Lipschitz functions. We show meaningful recovery of the transition matrix is possible for values of $T$ much smaller than what is required in the unconstrained case, and for $p = o(1)$.