Hybrid memory routing improves knowledge updates in large language models
ALLOT: Budgeted Hybrid-Memory Routing for Knowledge Updates in LLMs
Artificial Intelligence
Summary
Updating facts inside large language models can be expensive because changing the model’s parameters is costly. The authors present ALLOT, a method that saves on these costly updates by using a smart memory routing system that decides what facts to store externally versus update in the model. This approach lets the system keep important facts up to date while using fewer parameter changes, making the process more efficient. They tested this on specific benchmarks and showed it retains most of the benefits of a full update with much less effort.
What this means in practice
- •For ai product teams: Improve language model fact updates with fewer costly parameter changes by prioritizing what information is stored externally versus inside the model.
- •For cloud service engineers: Reduce resource use in maintaining updated knowledge for large-scale language inference services by combining retrieval confidence and relation metadata for memory writes.
Authors
Shanfeng Huang, Zhou Fang, Song Xiao, Hai Du
Abstract
For large language models (LLMs), parametric adaptation is costly when retrieval already suffices. We introduce ALLOT, a hybrid-memory routing framework that separates learned write priority from a hard parametric budget. A memory-aware router combines frozen text representations, retrieval confidence, and relation metadata; a single ranking supports multiple write budgets while preserving all facts in external memory. On CounterFact with Qwen3-4B, ALLOT reaches 0.760 accuracy at a 20% parametric-write budget and recovers 78.4% of the budget-matched oracle gain, with 80% fewer parametric writes than dual-writing every fact. At this budget, jointly adding retrieval and relation features to text improves normalized oracle gain by 6.2 percentage points. Complementary Qwen3-0.6B shared-store results achieve dual-write-level accuracy with 6-14.5% parametric writes, and cross-benchmark transfer retains approximately 88% of in-domain gain. These results support allocating adaptation capacity according to its incremental value rather than treating every factual update as an equally valuable training target.