Summary
Multimodal AI models that read financial cheques sometimes incorrectly copy the same amount into both the written and numeric fields, even when those fields are different. This confusion, called CARLAR influence, happens especially when parts of the cheque are unclear or redacted. The authors tested two AI models using various visual prompts and found the problem occurs frequently with high confidence in wrong answers. They tried a new approach using compressed images as examples to reduce mistakes, which helped one model but had mixed results for another. Their work suggests the need for better checks that consider how related information on documents affects extraction accuracy.
What this means in practice
- •For financial software developers: Improve cheque processing systems by addressing hallucination errors when AI models extract amounts from ambiguous or redacted cheque images.
- •For document digitization teams: Enhance audit and validation of multimodal extraction tools by incorporating tests for visual context and cross-field consistency in financial documents.
Authors
Joong Ho, Choi, Aditya Paul, Nidhi Singh
Abstract
Multimodal large language models (LLMs) are increasingly used for structured extraction from financial documents, but their behavior under visual ambiguity is poorly understood. We study CARLAR (Courtesy Amount Recognition and Legal Amount Recognition) influence: a model returns the same amount for both cheque fields even though their annotations differ, including cases with a blank annotation encoded as zero. Across ten visual prompting conditions, we evaluate GPT-4.1 and Claude 4.5 Sonnet on 243 primary cheques (124 with two observed unequal amounts) and 598 strict mismatches from two additional datasets. Under courtesy redaction, GPT-4.1 CARLAR reaches 73.7 percent (95 percent bootstrap interval: 68.3 to 79.0; p below 0.001 versus baseline). Model-reported confidence can remain high when predictions agree; expected calibration error reaches 0.93 among CARLAR cases. We also test a compressed-image few-shot intervention that presents examples as one visual table without model fine-tuning. On the 243-case cohort, Claude's Control condition increases paired joint extraction accuracy by 11.5 percentage points (95 percent interval: 6.2 to 16.9; adjusted p = 0.00054), but its strict-124 gain is smaller and not significant after correction. The intervention's effect on predicted-field agreement is model-dependent: it decreases for Claude and increases for GPT-4.1. These results motivate validation methods that account for visual context and dependencies between extracted fields.