A Differentiable Covariance Calculus for Linear Gaussian Bayesian Networks
2026-07-06 • Information Theory
Information Theory
AI summaryⓘ
The authors study linear Gaussian Bayesian networks where variables are vectors and connections are matrices. They show that many inference and estimation problems can be handled in one unified way using a differentiable map from parameters to covariance matrices. Their approach allows efficient calculation of many statistical tasks like conditioning and maximum-likelihood estimation using automatic differentiation. They test their method on standard models and confirm it matches classical results like the Kalman filter and Cramér–Rao bound. This unifies and simplifies working with complex vector-valued networks.
Bayesian networkslinear Gaussian modelscovariance matrixautomatic differentiationmaximum-likelihood estimationconditional independencemutual informationCramér–Rao boundKalman filterstructural equation models
Authors
Tadashi Wadayama
Abstract
Linear Gaussian Bayesian networks, equivalently linear Gaussian structural equation models, recur across statistics, control, and communications; in the vector-valued setting that motivates this work, their nodes are vectors and their edges are matrices. Every quantity of interest is a function of sub-blocks of the joint covariance, which is itself a classical, differentiable map (the K-recursion) from the local edge and innovation parameters. Yet the resulting inference and estimation tasks are usually derived and implemented separately, per task and per topology. Taking this covariance chart as a single backend, we build on it a unified, differentiable covariance calculus in which each task reduces to a few linear-algebra primitives on the one covariance, and automatic differentiation returns every gradient in a single backward sweep, over arbitrary vector-valued directed acyclic graphs and parametrizations, including tied and structured ones. The calculus covers conditioning, conditional-independence testing through mutual information, maximum-likelihood estimation with hidden nodes, and the Slepian--Bangs Fisher information with the local identifiability and Cramér--Rao reliability it induces. It is validated on a linear Gaussian state-space model and a skip-connected (non-chain) extension against the Kalman recursions, d-separation, and the Cramér--Rao bound.