Document version linking detects duplicates and conflicts accurately
GVD: Governed Versioning and Deduplication for Document Repositories
Artificial Intelligence
Summary
Document collections change over time, with many versions that sometimes have repeated or contradictory information. Existing methods look only at pairs of documents and don’t handle the whole collection well. The authors present GVD, a system that groups related document versions, finds duplicate and conflicting rules, and checks for version conflicts with a new technique they call counterfactual span probing. Their method keeps a clear record of versions and runs locally without big AI models, achieving high accuracy on enterprise documents.
What this means in practice
- •For enterprise document managers: Automatically organize and update evolving corporate policies by linking versions and highlighting only important changes for review.
- •For legal compliance teams: Maintain a clear, auditable history of policy changes and contradictions without relying on cloud AI services.
Authors
Mohammadreza Sediqin, Shivali Dalmia, Sumukha Thoppanahalli, Abhishek Mukherji
Abstract
Document repositories evolve continuously. Guidelines and policies are revised, superseded, and re-uploaded, so the same content recurs in different wording and newer versions refine or contradict earlier ones. These inconsistencies belong to the growing collection rather than to any single document, yet existing work treats versioning, duplicate detection, and contradiction detection as isolated pairwise tasks and stops once a pair is labeled. We present GVD (Governed Versioning and Deduplication), a framework that unifies cross-document version linking with rule-level conflict resolution under an auditable update policy. Incoming documents are assigned to version families through bidirectional rule alignment, and their rules are compared against the family memory to identify duplicates, contradictions, asymmetric refinements, and new knowledge, with Counterfactual Span Probing (CSP) resolving related pairs that inference misclassifies as neutral. Relation-specific policies suppress duplicates and escalate only consequential changes for review, retaining version lineage as an audit trail. The pipeline runs fully locally, with no large language model. On 120 enterprise documents processed as 140 ingestions across 59 version families, GVD reaches an F1 of 0.97 for version-family construction and 0.94 for rule-level consistency, with CSP raising rule consistency from 0.90 to 0.94.