Papers for

mobile device makers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vision language models reuse reading parts to specialize in OCR tasks

From Retrieval to Recognition:How Vision--Language Models Become OCR Specialists

Abstract: Does a general vision--language model acquire specialized OCR ability by developing a new reading circuit or by reusing an existing mechanism? We address this question in the setting of full-sequence OCR, rather than local-answer retrieval. Using an evidence-grounded protocol with held-out causal interventions, we identify sparse and stable OCR-head sets in GLM-OCR, MinerU2.5, and PaddleOCR-VL-1.6. We then investigate the mechanistic origin of these OCR heads by comparing them with independently identified textual retrieval/copy heads in general VLMs. Across two general VLMs, visual OCR heads strongly overlap independently identified textual retrieval/copy heads, yielding untuned top-20 intersections of 73.3% and all-head Spearman correlations of 0.677-0.886. The overlap and causal interventions suggest that full-sequence OCR operates as dense sequential multimodal copy-and-paste, repeatedly retrieving visual evidence and routing it to the current output position. Finally, we examine how this shared circuit changes as a general VLM becomes an OCR specialist. Matched base-to-specialized comparisons show that OCR specialization largely preserves head identity, retaining 17-20 of the top 20 heads per task with all-head rank correlations of 0.874-0.942, while redistributing their functional and causal strengths.

Fri 18 SeptComputer Vision and Pattern Recognition
The gist
Reading text in images is a tricky task for computers, and it wasn’t clear if general vision language models had to build new special parts to get good at it. The authors found that these models actually reuse certain components made for copying and retrieving text to become experts at OCR, which is reading out full text sequences. They also saw that as the models specialize, they keep the same parts but change how strong or important each part is for reading. This means OCR in such models works like repeatedly copying visual clues and pasting them in the right order.
Open → 2609.21543v1