Domain-adapted diffusion models improve conditional independence testing
Domain-Adapted Diffusion Models for Conditional Independence Testing
Machine Learning
Summary
Conditional independence testing helps find out if one thing affects another, once you know a third thing. The authors noticed current methods can make mistakes because they don't estimate needed probabilities well enough. They treated this challenge like learning from different but related sources, improving estimates by borrowing knowledge from similar data. This approach leads to better control of errors and stronger test results compared to older methods.
What this means in practice
- •For machine learning engineers: Enhance conditional independence tests in models trained on limited target data by incorporating related source-domain datasets with domain-adapted diffusion models.
- •For data scientists in finance: Improve hypothesis testing about variable relationships under changing market conditions by leveraging multi-source domain adaptation techniques for more reliable inferences.
Authors
Yanfeng Yang, Junda Zhao, Yijie Gao, Jiaqi Yang, Xinyu Shi, Ziqi Chen, Shunyu Zhao, Shuai Li, Wei Huang, Eshant English, Kenji Fukumizu
Abstract
Conditional independence (CI) is a fundamental concept in statistics and machine learning. Recent advances in conditional generative modeling provide flexible tools for generative-model-based CI tests, which rely on an estimated conditional distribution to generate randomized samples. However, errors in estimating this distribution accumulate in existing Type I error bounds, and consistency of the generative estimator alone does not guarantee asymptotic Type I error control. To address this limitation, we formulate conditional generative modeling as a domain adaptation problem and leverage auxiliary data from multiple source domains to improve estimation in the target CI testing domain. We propose Domain-Adapted Diffusion (DA-Diff), a multi-source domain adaptation framework for conditional diffusion models based on weighted empirical risk minimization over both target and source domains. We establish the convergence rate of DA-Diff and show how transferable source data can improve target-domain estimation through an increased effective sample size while controlling transfer bias. Building on DA-Diff, we further propose Domain-Adapted Conditional Independence Testing (DA-CIT) and show that its Type I error satisfies $P(p \leq α) \leq α+ o(1)$. Experiments demonstrate that DA-Diff improved conditional generation quality compared with transfer-learning diffusion baselines, while DA-CIT provides strong Type I error control and competitive power.