RTS Smoother-Guided Learning of Physics-Based Neural Differential Models
Machine Learning
Summary
The authors developed a method to help figure out unknown parts of systems described by ordinary differential equations (ODEs), especially when only some variables can be measured. They combine known physics with neural networks to fill in missing details, using a two-step process that switches between guessing hidden states and updating the neural network. Their approach works on various test problems and keeps the known physical parts clear while improving predictions about how the system behaves over time. This method helps understand complex systems even when some information is missing.
Authors
Ahmet Demirkaya, Georgios Stratis, Tales Imbiriba, Zachary D. Danziger, Deniz Erdogmus
Abstract
Ordinary differential equations (ODEs) are widely used to model dynamical systems in physics, biology, neuroscience, and physiology, but in many applications some equations of the dynamics are unknown and only a subset of the state variables are measured. We propose a hybrid neural--physics framework in which the known components of the ODE are kept explicit and the missing components are represented by a neural network. The proposed method consists of two stages where we alternate between state and parameter estimation and iterate until a predetermined criterion is met. Specifically, in the first step, we treat the model parameters as being known and we infer the latent states from the available measurements using a Rauch--Tung--Striebel (RTS) smoother. In the second stage, we treat the smoothed trajectories as being known and use them to estimate the neural networks' parameters through backpropagation. We evaluate the method on benchmark systems spanning linear, nonlinear, and stiff dynamics under partial state observation. Across these settings, the proposed method learns missing ODE components from incomplete measurements while exploiting and retaining interpretable mechanistic structure and improving latent-state reconstruction and long-horizon prediction.