Cost Scales with Change, Not Corpus Size: Incrementally Maintaining an Evolving Semantic Substrate
2026-08-17 • Artificial Intelligence
Artificial IntelligenceDatabasesInformation Retrieval
AI summaryⓘ
The authors explore how to keep a summary of a text collection's meaning updated efficiently as new documents are added. Instead of re-analyzing the entire collection for every new question, they suggest building and maintaining a compact mathematical summary once and updating it incrementally. Their tests show that updating this summary is much faster and uses less work than redoing everything from scratch, while still being very accurate. They also demonstrate a way to update parts of the summary after changing how texts are represented without reprocessing the whole collection. This supports the idea of maintaining a ready-to-query summary rather than re-deriving meaning each time.
retrieval-augmented question-answeringsemantic substratesingular value decomposition (SVD)incremental updatesembedding modellow-rank approximationorthogonal Procrustescosine similarityprincipal-anglevector re-embedding
Authors
Yusuke Takahashi, Kyle Wild, Asako Uraki
Abstract
Retrieval-augmented and agentic question-answering systems increasingly re-derive the meaning of a corpus at query time. Put plainly, instead of re-deriving what a corpus means on every question, the work is done once when a document arrives and is thereafter merely consulted -- a compiler, not an interpreter, of meaning. An alternative is to compile that meaning once, at ingest time, into a compact, queryable semantic substrate and maintain it as the corpus evolves. The central objection is maintenance cost: rebuilding a truncated singular value decomposition (SVD) on every change appears prohibitive, and a change of embedding model seems to force a full re-embedding. We argue and show empirically that maintenance cost scales with the amount of change, not corpus size. On a controlled synthetic pilot (dimension 256, rank 32, a corpus grown from 3,000 to 9,000 documents over 50 update events), incremental low-rank updates were 33.7 times cheaper per update than full re-SVD and 23.8 times cheaper cumulatively, while the incremental subspace tracked the full recomputation to within floating-point precision (maximum principal-angle drift below 1e-11 degrees; recall@10 = 1.0). An orthogonal Procrustes virtual axis update recovered 0.95 mean cosine to truly re-embedded vectors by re-embedding only about 10 percent of the corpus. The results support maintaining, rather than repeatedly reconstructing, a semantic substrate.