Noise-aware methods improve learning causal models from data
Learning Non-Gaussian SEMs with Conditional Risk, and Why the Direction of Recursion Matters
Machine Learning
Summary
Causal models help explain how different things affect each other, and they usually assume errors follow a bell-curve shape (Gaussian noise). This paper shows that by using a more general way to measure prediction errors, it's possible to learn better causal models when the errors don't follow this typical shape. The researchers found that two common approaches, which worked equally well for normal noise, behave differently with other types of errors. They also show that one method is more reliable across many kinds of error patterns, and simulations confirm their findings.
What this means in practice
- •For data science teams: More accurately recover causal relationships in data with non-normal noise using noise-adapted learning methods.
- •For financial risk modelers: Improve models that predict causal links in financial data where error patterns are heavy-tailed or asymmetrical.
Tested on simulated data.
Authors
Ujas Shah, Panagiota Birmpa, Rebecca Morrison
Abstract
Variance-based methods recover a Gaussian linear structural equation model (SEM) with equal or suitably constrained error variances, either by recursively identifying the source nodes with smallest conditional variance (top-down method) or sink nodes with highest conditional variance (bottom-up method). We show that the use of conditional variance is incidental: replacing it with the conditional risk, i.e., the minimal expected loss of linear prediction under the negative log-likelihood of the noise distribution, extends both methods to non-Gaussian noise. Existing methods are the least-squares special case. This yields two main results. First, exploiting the noise distribution improves structure recovery: the matched-loss procedures recover the SEM more accurately than their least-squares counterparts and the LiNGAM methods, which are designed for non-Gaussianity. Second, the two methods, interchangeable in the Gaussian setting, separate under non-Gaussian noise: the top-down method is correct for every noise density with finite differential entropy, whereas the bottom-up method requires a regularity condition and can fail without it, for instance under mean-zero exponential or Cauchy noise. Simulations confirm both results.