LLM agents improve discovering cause and effect in anonymous datasets
Benchmarking and Enhancing Agentic Causal Discovery
Machine Learning
Summary
Figuring out what causes what in data can be tricky, especially when the data doesn't have helpful labels. The authors created a set of anonymous test datasets to see how well smart language AI models can find cause-and-effect relationships. They found that these models can do better than existing algorithms by actually looking more closely at the data and experimenting further. Some cause-and-effect connections the models found had never been discovered by any algorithm before. This shows that AI agents can enhance causal discovery by combining reasoning with data analysis.
What this means in practice
- •For data scientists: Improve identification of causal relationships in complex and unlabeled datasets by combining AI reasoning with algorithmic methods.
- •For business analysts: Enhance decision-making by using advanced AI agents to uncover hidden cause-effect connections in customer or operational data.
Authors
Pingchuan Ma, Rui Ding, Shuai Wang
Abstract
LLM agents can now run causal discovery algorithms on a dataset and report what comes out. Whether they do more than that, such as noticing where an algorithm went wrong and fixing it with further tests, is hard to tell from current benchmarks. Most of them give the variables meaningful names, and an LLM shown only the names of the Tübingen cause-effect pairs already gets 99% of the directions right. With the names removed, the data are usually clean enough that picking a better algorithm than PC closes most of the gap. We introduce Heuresis, a benchmark of anonymous simulated datasets whose mechanisms range from linear Gaussian to heavy-tailed. For each family of datasets it separates what a better existing algorithm can gain from what only a closer look at the data can gain, and it traces every edge an agent changes. Every family leaves a gap that none of the algorithms, the router and the two causal-discovery foundation models we ran can close, so a score above the best chosen algorithm has to come from using the data. We instantiate the benchmark with an agent harness and three LLMs, GPT-6.1 Sol, Claude Opus 5.5 and Qwen 3.8 Flash. GPT-6.1 Sol surpasses the best chosen algorithm on all six families, by 14% adjacency F1 on the most difficult one, and Claude Opus 5.5 does so by 7.4% there. A third of the true edges the two agents restore on the two hardest families appear in no algorithm's output. The code and data will be publicly released to facilitate future work.