Language models improve temporal causal reasoning with new architectures
From Document Layout to Causal Topology: An End-to-End Architecture for Temporal Causal Reasoning
Artificial Intelligence
Summary
Understanding cause and effect over time is hard for current language models. The authors propose two new models that think about events as connected causes instead of just sequences. One model uses a mix of rules and learning to explain causes, while the other learns these ideas directly inside itself. Their new approach works better than existing models, including some very large ones, especially at making deep cause-and-effect decisions over time.
What this means in practice
- •For software engineers: Improve AI systems that interpret timelines by integrating better temporal cause-effect analysis for more reliable event understanding.
- •For automation system designers: Design systems that make decisions based on cause-effect chains over time using efficient end-to-end models instead of manual symbolic rules.
Authors
Zhifeng Yang, Wei Luo, Yongjian Chen, Shiqi Ou
Abstract
Large language models (LLMs) generally lack deep causal reasoning capabilities in temporal reasoning. Inspired by document layout reordering, this paper proposes a paradigm shift from discrete narratives to causal topologies. It constructs two architectures: a training-free neuro-symbolic framework, temporal causal reasoning agent (TCR-Agent), which achieves interpretable causal deduction and counterfactual truncation through external causal graph instantiation and a symbolic intervention engine; and an end-to-end model, temporal causal reasoning former (TCR-Former), which internalizes these mechanisms into the Transformer attention space via a Causal-Topological Attention Mask and temporal span biases. Experiments on the TempoCausal benchmark demonstrate that TCR-Agent comprehensively surpasses existing training-free baselines. At the same time, TCR-Former achieves an accuracy 16.5% higher than that of state-of-the-art (SOTA) methods, outperforming large-scale models such as GPT-5.4 and DeepSeek-V3.2 with only approximately 8.6B parameters. Further analysis reveals a structure-reasoning disconnection phenomenon in existing methods: external symbolic constraints and chain-of-thought reasoning can improve reasoning steps, yet fail to enhance deep causal deductive capability. By internalizing temporal causal reasoning capabilities into the model, TCR-Former effectively bridges this gap, providing an efficient and reliable end-to-end foundation for complex reasoning scenarios.