Accelerating document OCR by verifying parallel drafts together
Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decoding
Computation and LanguageComputer Vision and Pattern Recognition
Summary
Reading text from images of documents is usually slow because the computer reads one character at a time. The authors created a new way called GravityOCR that makes guesses about many characters at once and then double-checks these guesses before finalizing them. This method lets the computer read documents faster without making more mistakes. They tested GravityOCR and found it speeds up reading without losing accuracy.
What this means in practice
- •For document processing teams: Speed up the text extraction from scanned documents by committing multiple tokens per decoding step with verification to reduce errors.
- •For mobile app developers: Build faster mobile OCR apps that read document sections in parallel while maintaining accuracy using the joint drafting and verification model.
Authors
Dohyun Kim, Sungjun Han, Hyungguk Kim, Yusik Kim, Jamin Shin, Paul Hongsuck Seo, Hongjoon Ahn
Abstract
Autoregressive OCR vision-language models accurately convert document images into text and structured markup, but require one sequential decoding step per output token, limiting inference speed. Unlike open-ended text generation, OCR outputs are strongly grounded in the input image, making diffusion-based parallel generation promising. However, when several tokens are predicted in one diffusion step, each is predicted before the others are known. Committing them directly can therefore introduce errors. We therefore introduce GravityOCR, a parameter-shared AR-block-diffusion model jointly trained for parallel drafting and causal AR verification. Verifying drafts before commitment lets the model commit multiple output tokens per round without a separate drafting network. The causal AR path also enables GRPO with sequence- and structure-level OCR rewards, avoiding diffusion-trajectory likelihood estimation while updating the shared drafter parameters. On OmniDocBench v1.6, AR-path GRPO improves the Overall score from 94.92 to 95.16 without reducing diffusion drafting efficiency, while the final model remains close to the original GLM-OCR score of 95.48. In an SGLang serving deployment, GravityOCR commits an average of 9.7 output tokens per forward pass and achieves a $3.94\times$ decode-only speedup on region crops and a $1.32\times$ end-to-end page-processing speedup over AR decoding.