TopoCompress: Long Context Compression via Graph-Wired Semantic Trajectories
2026-08-31 • Computation and Language
Computation and LanguageMachine Learning
AI summaryⓘ
The authors present TopoCompress, a method that helps large language models handle very long texts more efficiently by picking out important pieces of information without needing extra training or relying on a specific model. Their approach scores parts of the text based on how relevant they are to a question, then links related parts together in a graph to spread and improve these relevance scores. They tested TopoCompress on multiple reading tasks and found it works better than other methods while using less space and running faster. This means it can help models understand long documents quicker and cheaper.
long-context compressionlarge language modelssemantic spansquery relevancehybrid graphsemantic similaritysequential adjacencyHotpotQAcompression budgetinference latency
Authors
Daniel Agyei Asante, Yang Li
Abstract
Long-context compression is essential for reducing the cost and latency of large language model inference. However, existing methods can fragment important evidence, require additional training or alignment, and often depend on the target model for effective compression. We introduce TopoCompress, a training-free and model-agnostic framework that compresses long contexts by selecting coherent semantic spans. TopoCompress first scores each span using dense and lexical query relevance together with semantic acceleration. It then constructs a hybrid graph that connects spans based on semantic similarity and sequential adjacency, and propagates the query-guided relevance scores over the graph. Across five long-context tasks-HotpotQA, 2WikiMQA, MuSiQue, Qasper, and MultiFieldQA-en-TopoCompress consistently outperforms strong compression baselines. Notably, TopoCompress achieves performance comparable to the strongest baseline while using a 4x smaller compression budget, and provides a 1.41x smaller compression time over the fastest baseline.