Limits found in evaluating AI financial agents under stress and defects
Measurement Boundaries in LLM Financial Agent Evaluation: Fixed-Tape Execution and Multi-Defect Auditing
Machine LearningArtificial IntelligenceComputational Engineering, Finance, and ScienceSoftware Engineering
Summary
Measuring how well AI financial agents perform can be tricky because different testing conditions can change their results. The authors found that running the same AI agent under ideal and stressed conditions often gives very different results, even when replaying the same recorded decisions. They also showed that using recall to check if the AI catches errors isn’t enough because the auditor often falsely flags tasks with no errors. These findings highlight the need for careful controls in testing and auditing AI agents in finance.
What this means in practice
- •For financial software engineers: Improve testing setups for AI financial agents by incorporating fixed-response replay to better isolate execution effects from model variability.
- •For compliance auditing teams: Develop audit procedures that account for multi-defect detection limitations and reduce false positive findings in AI agent evaluations.
Authors
Weicheng Xue
Abstract
What controls are needed to interpret execution performance and audit scores in LLM agent evaluations? We study two limits on these interpretations in a financial agent harness. In Study~A, comparing independent runs under idealized and stressed execution on three synthetic settings that share one 24-day upward phase mixes the execution rule with fresh model responses and portfolio feedback: the parsed decision paths agree in only $19.8\%$ of $450$ pairs. Replaying each stored response tape through both execution destinations gives a narrower result. Conditional on those responses, stressed execution changes total return by $-0.0170$ (95\% interval $[-0.0230,-0.0117]$), or $10.4\%$ of the idealized baseline, and ten seed clusters do not resolve the model ranking. Study~B corrects an incomplete answer key and replaces legacy tasks with matched zero-, one-, and two-defect tasks under an explicit multi-label prompt. The drop in target violation recall from one to two defects is positive in five of six combinations of auditor and source (median $0.267$), with three surviving Holm correction. Yet the auditor that includes both target labels most often has micro-precision $0.149$, emits findings on $98/100$ zero-defect tasks, and returns the exact dual-defect set in only $21/100$ cases. Target recall by itself therefore gives a poor account of audit quality on this construction. The studies address different limits: what an execution comparison estimates, and what target recall captures. Together, they show how fixed conditions and diagnostic controls bound the claims a score can support.