Multispectral document binarization improved with adaptive feature fusion
Cross-Spectral Feature Fusion for Multispectral Document Binarization
Computer Vision and Pattern Recognition
Summary
Fading ink and damaged backgrounds make old documents hard to read, especially when seen in different types of light. The researchers found a better way to combine information from visible and near-infrared light to distinguish text from background in these documents. They used a special neural network that adapts how it mixes features from multiple light wavelengths and guided the process with edge information. Their method improved accuracy in identifying text compared to older fixed ways of combining these images.
What this means in practice
- •For digital archivists: Enhance readability of degraded historical documents by producing cleaner binary images that separate text from background more accurately using multispectral data fusion.
- •For cultural heritage conservators: Support restoration decisions by providing high-quality binarized images from multispectral scans that reveal faint or damaged text areas.
Authors
Yasas Savinda, Maheshi Buddhinee Dissanayake
Abstract
Historical document binarization remains challenging when faded ink, bleed-through, and background degradation exhibit wavelength-dependent behavior across visible and near-infrared observations. This study investigates whether adaptive cross-spectral feature fusion and targeted structural guidance can improve multispectral document binarization beyond fixed fusion and generic attention mechanisms. Four registered SMADI bands at 500, 600, 700, and 1000 nm are processed using a multi-branch conditional generative adversarial network (cGAN). Visible, NIR, and structural feature streams are fused at encoder depth L = 3 using spatial- and channel-wise softmax gating, while auxiliary edge maps derived from selected VIS-NIR band pairs provide targeted structural guidance. Nine controlled fusion and structural configurations are evaluated using five-fold document-level cross-validation, DIBCO-standard metrics, and Holm-corrected statistical testing. The F3 + NIR edge-guided configuration provides the strongest overall performance among the evaluated models, achieving mean F1 = 0.7916, p-FM = 0.8044, and IoU = 0.6654. Relative to the fixed Mid AVG baseline, it improves F1 by 0.0269, p-FM by 0.0360, and IoU by 0.0321, with all three gains remaining statistically significant after Holm correction. Mean NRM and DRD are also reduced, although these reductions are not statistically significant after correction. Targeted cross-spectral structural guidance provides an effective mechanism for regulating intermediate multispectral fusion, improving primary foreground-overlap performance while retaining the complete four-band observation.