Hessian rank helps find hidden causes in complex nonlinear data
Hessian Rank Constraint for Learning Structure of Nonlinear Latent Variable Models
Machine LearningArtificial Intelligence
Summary
It is difficult to discover hidden factors and their relationships from complicated observations. The authors propose using a mathematical property called the cross-Hessian Rank Constraint to reveal these hidden variables even when the data involves nonlinear interactions. Their method builds on classical techniques that worked only for simpler linear cases and can identify the number and connections of the hidden causes. They test their idea on simulated and real data, confirming their approach works under certain mathematical assumptions.
What this means in practice
- •For data scientists: Identify hidden factors and their causal links in datasets with nonlinear relationships using rank properties of the observed data.
- •For signal processing engineers: Locate latent signals and recover their causal structure from nonlinear mixtures in measurement systems with low noise.
Authors
Zijian Li, Ruichu Cai, Feng Xie, Xinshuai Dong, Haoyue Dai, Yuewen Sun, Yujia Zheng, Guangyi Chen, Yingyao Hu, Kun Zhang
Abstract
Uncovering latent variables and their causal relations from observed data is a fundamental yet challenging problem. Existing methods often rely on restrictive assumptions, such as linear relations or invertible mixing functions. To better address this problem under general nonlinear mixing procedures, we propose a condition called the cross-Hessian Rank Constraint (HRC), which serves as a primitive rank-based tool for nonlinear latent causal discovery. In particular, we show that a rank-based property arises from the cross-Hessian of the observed-data log-density in the nonlinear case, revealing information about the latent variables, and reduces to the Tetrad constraints in the linear Gaussian case. More specifically, when two groups of observed variables are d-separated by a set of lower-dimensional latent variables, the rank of this cross-Hessian is equal to the dimension of the latent variables, under a mild affine derivative assumption on the conditional log-density derivatives. This assumption can be naturally satisfied when the noise level is low or the relevant nonlinearity is moderate. As a downstream application, we instantiate HRC in the pure one-factor measurement setting for locating latent variables and recovering their causal structure up to Markov equivalence. Experimental results on synthetic and real-world datasets support the theoretical claims.