Self-play method improves accuracy in document optical character recognition
SP-DocReader: Difference-Aware Self-Play for Precise Document OCR
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Reading text from images of documents is still tricky even for advanced AI models because of limited training resources. The authors created a new method called SP-DocReader that focuses on fixing mistakes that remain after normal training. Their method compares the AI’s output with the correct text to find differences and gives extra attention to these errors during training. This approach improves how accurately the AI reads characters and answers questions about documents.
What this means in practice
- •For document processing teams: Improve accuracy of document transcription systems by focusing training on leftover recognition errors after initial fine-tuning.
- •For automated customer support developers: Enhance text extraction quality from documents to enable better automated text-based customer query answering.
Authors
Wenjie Liao, Xiaohui Song, Liangjie Zhao, Haonan Lu
Abstract
Accurate page transcription remains difficult for vision language models under limited input and training budgets. We present SP-DocReader, a self-play framework for optical character recognition (OCR) that targets residual errors after supervised fine-tuning. Reading Discrepancy Masking aligns reference and generated model tokens through a longest common subsequence, then scores unmatched positions with their full conditioning prefixes. Focused Fidelity Loss adds direct negative log-likelihood supervision at unmatched ground-truth positions. Only the OCR module is trained, while the backbone remains frozen. We derive the combined gradient to distinguish relative score optimization from direct supervision. Compared with SFT-2, SP-DR-3 reduces Vary-600K character error rate on both backbones. On Qwen3-VL-4B, it reduces character error rate by approximately 54 percent and improves DocVQA Average Normalized Levenshtein Similarity (ANLS) by 3.7 points. These results show the value of focusing self-play training on the discrepancies that remain after supervised fine-tuning.