FIDES: A Concordance Protocol for LLM-Generated Trading Strategies
2026-08-24 • Cryptography and Security
Cryptography and SecurityArtificial Intelligence
AI summaryⓘ
The authors introduce FIDES, a method to check if a trading strategy's explanation, code, and actual results all match up. They tested it on 40 strategies across several models and found that just because a strategy sounds good or claims to work, it usually doesn’t beat a simple buy-and-hold approach. Also, the models often overestimate their success, and changing how the strategies are judged can flip what the model says it will do. The authors emphasize that FIDES is about measuring consistency rather than proving a strategy's effectiveness.
LLM (Large Language Model)trading strategybacktestingSharpe ratiobuy-and-holdsandbox executionconcordanceout-of-sampleself-assessmentfinancial modeling
Authors
Arther Tian, Alex Ding, Simon Wu, Aaron Chan
Abstract
An LLM asked for a trading strategy returns three artifacts at once: a natural-language rationale, an executable implementation, and once run, a track record. Whether these are the same object is rarely checked. We present FIDES, a measurement protocol that treats them as three views to be reconciled rather than one deliverable to be graded. Through dual delivery, a single model call returns both a natural-language strategy with an explicit claimed edge and a self-contained strategy(df) function. FIDES executes the code in a sandbox against a lag-one out-of-sample backtest and scores three concordance gaps: say to do, do to real, and say to result. On 8 liquid US ETFs across four models plus a two-stage elicitation arm, 40 strategies, 2023 to 2024 out-of-sample, three findings stand out. First, concordance does not predict profit: only 2 of 40 strategies beat buy-and-hold, and a plain sma(50,200) rule outperforms every model's mean Sharpe. Second, self-assessment is badly calibrated: 32 of 40 strategies claim to beat buy-and-hold and exactly one does. Third, swapping the language-code judge for a second model flips say to do on more than half of items. Injecting Close.shift(-1) drops do to real by 0.33 on average, while our runtime future-information probe fired on neither clean nor injected code. We frame FIDES as a protocol for measurement fidelity, not a claim about market performance.