Financial vision language models struggle with reliable chart action advice
E2A-Bench: Benchmarking Evidence-to-Action Reliability in Financial Chart Reasoning
Computation and Language
Summary
Financial models that look at charts and give advice often say things that don't fully match the data they see. The authors created a new test called E2A-Bench to check if these models truly base their advice on chart evidence and if their reasoning is consistent. When testing 20 popular models, they found many give unreliable or biased action suggestions, especially without tracing the whole reasoning from evidence to action. This work shows it’s important to check the entire chain of evidence supporting financial advice from AI models, not just whether their claims sound correct.
What this means in practice
- •For financial technology teams: Test and improve reliability of AI models that analyze financial charts and suggest trading actions.
- •For business intelligence developers: Build tools that ensure AI-driven financial insights are properly traced from data evidence to final recommendations.
Authors
Xiaoya Wang, Yutong Xu, Junjie Wang
Abstract
Can financial vision-language models (VLMs) turn chart evidence into reliable action recommendations? Existing hallucination evaluations are mostly claim-centric; they assess whether generated statements are supported, but not whether evidence remains traceable through rationale, confidence, and final action. We introduce E2A-Bench, a 969-query benchmark for financial chart reasoning, constructed from 323 HS300 constituents under three input modalities with deterministic OHLCV-derived evidence anchors. E2A-Bench evaluates grounding, reasoning-action consistency, evidence-confidence calibration, and directional coverage through UCR, RCI, ECI, and NDR, where NDR measures coverage-aware evidence-to-action reliability rather than realized trading performance. Evaluating 20 VLMs reveals three failures hidden by scalar hallucination scores: the lowest-UCR model ranks near the bottom by NDR due to only 6.4% directional coverage; oracle-aided verification reduces unsupported claims but can collapse coverage; and financial fine-tuning amplifies the BUY:SELL ratio by factors of 4.21 to 4.68 across strict base-fine-tuned pairs. These results show that financial VLM evaluation should trace the full evidence-to-action chain rather than rely on a single hallucination score. Code and data: https://github.com/wanng-ide/E2A-Bench