Chain improves event forecasting calibration with causal temporal modeling
CHAIN: Calibrated LLM Forecasting via Causal-Temporal Hypergraph Inference
Machine LearningComputation and Language
Summary
Predicting future events with large language models is helpful but often not very accurate in showing how confident the models are. The authors identify that this issue comes from hidden patterns in how the predictions are made over time and cause-effect relationships. They create a new method called CHAIN that breaks down the problem into steps and improves confidence scores by carefully weighting, combining, and balancing information based on cause and time. Tests show CHAIN makes better and more trustworthy forecasts across different topics.
What this means in practice
- •For forecasting engineers: Improve trustworthiness of probabilistic event forecasts by integrating causal and temporal information during prediction.
- •For risk management teams: Use calibrated probability outputs from causal-temporal models to make better decisions under uncertainty across varied domains.
Authors
Wenjin Liu, Chenxi Wang, Yue Lu, Zhe Cui, Haoran Luo
Abstract
Large language models have achieved significant progress in event forecasting, yet their probability outputs exhibit systematic calibration bias that varies heterogeneously across different domains and question types, undermining the trustworthiness of probabilistic outputs for decision-making under uncertainty. However, existing calibration methods typically correct probability outputs after prediction is complete, without modeling the structural sources of bias within the prediction process itself. To address this challenge, we decompose probabilistic prediction over causal-temporal hypergraphs into three stages, evidence weighting, evidence aggregation, and source fusion, and propose CHAIN, which designs stage-specific mechanisms to mitigate bias at each stage: (i) modulating the temporal decay function by causal topological distance, (ii) aggregating approximately independent causal chains via Noisy-OR after direction-aware deduplication, and (iii) driving adaptive fusion by causal coverage and directional balance. Experimental results on cross-domain forecasting benchmarks show CHAIN outperforms existing methods in expected calibration error, Brier score, and accuracy. Our project is available at https://github.com/QwenQKing/Chain.