New method finds causes behind data without knowing structure
Structure-agnostic Causal Representation Learning
Machine LearningArtificial Intelligence
Summary
It is hard to understand the hidden causes in data if you don’t already know how those causes relate to each other. The authors propose SaCRL, a method that learns both the causal relationships and the important features from data at the same time, without needing prior structural knowledge. This approach helps models make better predictions even when the testing data looks different from the training data. The authors show that their method works well on several tests, including synthetic data and real-world image datasets.
What this means in practice
- •For machine learning engineers: Improve model robustness by jointly learning causal features and structures when prior cause relationships are unknown across shifting environments.
- •For computer vision teams: Enhance image classification accuracy in varied domains by automatically identifying relevant causal features without predefined causal assumptions.
Authors
Arman Behnam, Binghui Wang
Abstract
Causal representation learning aims to discover robust features by exploiting the causal structure underlying data generation. Existing methods require specifying the causal structure a priori, yet different structures demand fundamentally incompatible invariance constraints, and misspecification leads to representations that discard predictive information. We introduce SaCRL, a framework that jointly identifies the causal structure and learns the corresponding invariant representation without prior structural knowledge. Our approach formulates structure selection as a soft optimization over candidate invariances using HSIC-based violation metrics, with adaptive weights that automatically concentrate on the achievable structure. We provide theoretical guarantees for structure identification, including under random-feature approximation, invariance satisfaction, and out-of-distribution generalization. Empirically, SaCRL recovers the true structure on synthetic and semi-synthetic Bayesian-network benchmarks, outperforms fixed-invariance baselines on Colored MNIST, achieves state-of-the-art accuracy on three DomainBed benchmarks (PACS, VLCS, OfficeHome), and degrades gracefully under structural misspecification and limited environment diversity. Code is available at: https://github.com/ArmanBehnam/sacrl.