Document segmentation improves explanations for image classifiers
When Superpixels Fail on Documents: A Study of Segmentation for LIME Explanations
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Explanation methods help us understand how image classifiers make decisions, but these methods depend on how the images are divided into parts. The authors show that common ways to split images into pieces work poorly for documents because they don't match text or layout areas. They found that using document-specific divisions based on text detection leads to clearer and more accurate explanations. This approach also helps uncover when classifiers cheat by relying on hidden codes in the documents.
LIMEimage classificationsegmentationsuperpixelsOCRpost-hoc explanationlocal fidelityRVL-CDIPQuickshiftSLIC
Authors
Quentin Telnoff, Emanuela Boros, Mickaël Coustaty, Robin Jarry, Fabrice Crohas, Antoine Doucet
Abstract
Post-hoc explanation methods are widely used to inspect image classifiers, but their reliability depends on design choices that are often treated as implementation details. We study this issue for LIME on document image classification, focusing on the segmentation step that defines the interpretable units being perturbed. Standard image-based LIME typically relies on natural-image superpixels, which are poorly aligned with document structure such as text regions, layout blocks, and identification codes. Using RVL-CDIP, we compare Quickshift and SLIC with document-aware segmentations based on OCR bounding boxes and regular grids. Our results show that segmentation strongly affects explanation consistency, correctness, and local fidelity. Document-aware segmentations produce more stable and faithful explanations, require fewer perturbations to converge, and expose shortcut behaviour based on document identification codes, a known RVL-CDIP bias that superpixel-based LIME often obscures. These findings show that reliable post-hoc explanation requires domain-aware interpretable representations, and that segmentation should be treated as part of the explanation method rather than as neutral preprocessing.