Deep network improves smartphone document image correction
Rethinking Supervision in Document Unwarping: A Self-consistent Flow-free Approach
Computer Vision and Pattern Recognition
Summary
Taking pictures of documents with smartphones often causes distortion, making the text hard to read or use. The authors developed DRNet, a new deep learning method that fixes these distorted images by understanding the natural shapes and features in documents. Instead of relying on complicated maps of deformations, their method learns by comparing the corrected images directly to clean, undistorted examples. Tests show DRNet works better than previous methods on a standard dataset.
What this means in practice
- •For mobile app developers: Develop camera apps that automatically correct distorted document images for improved readability and digital archiving.
- •For digital archiving teams: Enhance automatic cleanup of digitized documents to maintain more accurate records from smartphone captures.
Authors
Shaokai Liu, Hao Feng, Wengang Zhou
Abstract
In recent years, the proliferation of smartphones has led to an upsurge in the digitization of document files via these portable devices. However, images captured by smartphones often suffer from distortions, thereby negatively affecting digital preservation and downstream applications. To address this issue, we introduce DRNet, a novel deep network for document image rectification. Our approach is based on three key designs. Firstly, we exploit the intrinsic geometric consistency inherent in document images to guide the learning process of distortion rectification. Secondly, we design a coarse-to-fine rectification network to leverage the representations derived from the distorted document image, thereby enhancing the rectification result. Thirdly, we propose a unique perspective for supervising the learning of rectification networks, where undistorted document images are employed for supervision, which is free of warping mesh as ground truth in existing methods. Technically, both low-level pixel alignment and high-level semantic alignment jointly contribute to the learning of the mapping relationship between deformed document images and distortion-free ones. We evaluate our method on the challenging DocUNet Benchmark dataset, where it sets a series of state-of-the-art records, demonstrating the superiority of our approach compared to existing learning-based solutions. Additionally, we conduct a comprehensive series of ablation experiments to further validate the effectiveness and merits of our method.