Scaling State-Space Models from Lines to Paragraphs: An Ablation of Mamba-based OCR
2026-06-22 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors studied a new type of model (called Mamba, based on State-Space Models) for reading text, especially long paragraphs, which can work faster than the common Transformer models. They found that Mamba works well and fast on clean, synthetic printed text, but struggles more with real handwritten text, making more mistakes. They showed that the problem with handwriting is mostly because Mamba needs a lot of data to learn well, not because the model is inherently worse. Their work helps understand when these new models are useful for reading lots of text and when they still need improvement.
OCRTransformerState-Space ModelsMambaautoregressive modelsCharacter Error Ratehandwriting recognitionsequence modelingdecoder hyperparameterssynthetic paragraphs
Authors
Merveilles Agbeti-Messan, Pierrick Tranouez, Stéphane Nicolas, Clément Chatelain, Thierry Paquet
Abstract
End-to-end OCR increasingly relies on autoregressive sequence models, where the quadratic cost of Transformer attention limits efficient transcription of long, paragraph-level text. State-Space Models (SSMs) such as Mamba offer linear-time decoding and have recently been shown to match Transformer accuracy on printed historical lines, but their behavior as sequences grow from short lines to full paragraphs, and their generalization to handwriting, remain poorly understood. We study how a Mamba-based OCR recognizer scales from lines to paragraphs. We first conduct a systematic exploration of its four core hyperparameters (decoder depth, state dimension, expansion factor, and connector depth) on synthetic paragraphs from 100 to 1,000 characters, identifying the recurrent state dimension and the expansion factor as the dominant levers for long-sequence accuracy. We then compare the recognizer against a Transformer baseline trained under an identical protocol. On clean synthetic paragraphs, both models stay below 1% CER at every length while the SSM runs 1.4 to 4.5 times faster, the speedup growing with sequence length. On real handwriting, however, the SSM lags clearly behind: it reaches 8.2% CER on IAM lines and 10.0% on IAM paragraphs, against 4.2% and 3.5% for the Transformer baseline. Through controlled experiments we show that a substantial part of this gap stems from data scarcity rather than from an intrinsic architectural limit: the autoregressive SSM decoder is markedly data-hungry on long sequences. Our study clarifies when SSMs are a practical choice for large-scale document transcription and when they are not.