Multilingual models struggle to find all financial facts despite high concept recognition
Recognition Is Not Grounding: Benchmarking Multilingual Financial Evidence Extraction
Artificial IntelligenceComputational Engineering, Finance, and Science
Summary
Comparing financial information across different languages is tough because it requires not only recognizing what the financial concepts are but also extracting all the related numbers from texts. The authors created a new benchmark called MULTIFINGROUND with financial reports in English, French, and Greek to test how well models do both tasks together. They found that while many models are very good at recognizing concepts, they often miss some important numbers tied to those concepts. This shows that high concept recognition doesn’t guarantee complete extraction of financial facts, emphasizing the need to check both together.
What this means in practice
- •For financial data engineers: Improve cross-language financial data processing pipelines by evaluating and selecting models that better balance concept recognition and complete fact extraction.
- •For multilingual ai developers: Use MULTIFINGROUND benchmark to test models’ ability to both identify and extract numerical financial facts in multiple languages, guiding model improvement.
Authors
Lingfei Qian, Yan Wang, Xueqing Peng, Polydoros Giannouris, Nadhem Zmandar, Yi Han, Yanguang Liu, Víctor Gutiérrez-Basulto
Abstract
Comparing financial disclosures across languages requires identifying shared accounting concepts and recovering all numerical facts that express them. Existing benchmarks evaluate aspects of these capabilities, but provide limited insight into whether both succeed on the same multilingual evidence. In particular, it remains unclear whether language models can recognize these concepts across languages and exhaustively recover their corresponding numerical evidence from the same contexts. We introduce MULTIFINGROUND, a benchmark designed to investigate this recognition-grounding gap in multilingual financial reports. Constructed from FY2025 disclosures of 90 issuers in English, French, and Greek, MULTIFINGROUND aligns independently authored reports through a shared accounting taxonomy and provides traceable fact-level annotations. The benchmark evaluates financial fact identification, concept recognition, exhaustive fact selection, and concept discrimination without candidate labels. Across 15 evaluated models, eight API and open-weight reference models achieve 97.3-99.4\% concept recognition accuracy, yet miss at least one target fact in 26.6-41.9\% of correctly recognized groups. Further analyses reveal substantial variation in concept discrimination and limited gains from the evaluated finance-specialized models. Our findings demonstrate that strong concept recognition can coexist with incomplete evidence recovery, highlighting the importance of evaluating semantic recognition and evidence completeness together.