AStockArena audits real-time financial trading decisions beyond returns
AStockArena: Beyond Returns, Can We Entrust Real Capital to the Decisions of Financial Agents?
Computation and LanguageArtificial Intelligence
Summary
Financial trading AIs are often judged only by how much money they make or how risky their trades are. The authors point out that these measures miss whether the trades were actually done as reported and if there is enough evidence supporting them. They built AStockArena, a tool that checks over 3,000 trading decisions from AI models on China’s stock market to see if trades match records and rules. They found many issues with the reported trades, showing that just looking at profits isn’t enough to trust these AI traders. This auditing helps verify if AI financial agents make reliable decisions.
What this means in practice
- •For financial technology developers: Integrate comprehensive auditing checks into AI trading platforms to verify trade authenticity and improve decision transparency.
- •For financial compliance teams: Use AStockArena’s auditing framework to detect inconsistencies and evidence gaps in automated trade records for regulatory compliance.
Authors
Zeyu Liu, Sing Kwong Lai, Xuanzhi Feng, HaoYang Shang, Yang Li, Zihao Fang, Xuan Liu
Abstract
Large language models (LLMs) are increasingly used in dynamic financial trading, yet real-time trading benchmarks mainly evaluate agents by cumulative returns and risk-adjusted performance. Such outcome metrics cannot show whether trades are faithfully reported, supported by available evidence, and consistent with account states. We introduce AStockArena, a real-time decision-auditing benchmark for continuous trading in China's A-share market. Under shared market conditions and execution constraints, we evaluate 13 open- and closed-source models over 3,198 sequential decisions. AStockArena reconstructsthe actions actually posted to accounts by aligning reports, trade requests, execution responses, and account states, and audits them with eight checks spanning execution traceability (O), evidence sufficiency (RK), and financial meta-awareness (B), combining record verification with consensus expert annotation. Of 1,070 trading points, 72.99% exhibit at least one audit gap; model-level rates range from52.2% to 91.2%, with evidence gaps most prevalent. Historical trade-filtering replay further shows thatportfolio outcomes are sensitive to these flagged trades. Decision-process auditing thus offers a verifiable complement to return-based evaluation of financial agents.