FORESIGHT-9: Prospective and Process-Aware Evaluation of Adaptive Trading Agents
Artificial Intelligence
Summary
The authors point out problems with testing trading agents only by looking back at past data, since that can miss important issues and bias results. They created FORESIGHT-9, a new testing setup that uses nine different possible future market scenarios starting from July 2026, each with preset economic events and asset behaviors. This system lets them see how trading agents adapt over time in different situations, not just one historical path. Their tests showed agents performed very differently depending on the scenario and model used, and a simple fixed strategy often did better than most adaptive methods. Their approach also helped detect hidden failures in how agents adapt internally, not just in final profits.
Authors
Xiangxin Luo, Chengtian Hong, Haohua Li, Yongyi Xie
Abstract
Retrospective backtests provide a limited test of adaptive trading agents: they cannot rule out historical contamination, expose sensitivity to a single realized market path, or reveal internal degeneration during long-horizon adaptation. We introduce FORESIGHT-9, a prospective and process-aware benchmark built from nine auditable counterfactual stress worldlines branching from a common July 2026 information boundary. Each worldline specifies staged macro-financial events and joint multi-asset anchors; a deterministic generator realizes the trajectories, while observations are disclosed according to in-world time. A common contract standardizes observations and execution while preserving each agent's native adaptation loop. We evaluate two adaptive trading-agent frameworks with two foundation-model backbones across 36 long-horizon runs. Agent rankings vary substantially across worldlines and backbones, and a fixed equal-weight policy outperforms 31 of 36 runs. Process telemetry exposes failures that terminal returns conceal: in one high-return run, the live factor library collapsed while executed holdings converged to the equal-weight fallback, even though decision records continued to report an active factor ensemble. FORESIGHT-9 therefore evaluates not only portfolio outcomes, but whether adaptive agent state and execution remain coherent across alternative futures. We release the worldlines, trajectories, audit traces, and regeneration scripts.