Latent communication boosts language model teamwork speed and accuracy
Think Before You Speak: Enabling Training-Free Latent Communication Between Language Models
Computation and LanguageMachine Learning
Summary
Language models often talk by exchanging text, which can slow things down. The paper shows that sharing internal memory directly between models helps them work together faster and better without any extra training. The authors created a simple way to mix these memories from different models that makes teamwork easier. This approach resulted in higher accuracy and much faster communication compared to normal text chat methods.
What this means in practice
- •For ai system engineers: Enable multi-model AI systems to share internal states for faster and more accurate joint reasoning without retraining communication modules.
- •For cloud ai platform operators: Reduce latency in multi-model AI services by using latent cache sharing instead of text messages for model coordination.
- •For enterprise software developers: Implement multi-agent workflows that leverage cross-model memory blending to improve complex task performance in business applications.$Commercial implications: This method enables software products integrating multiple AI models to deliver faster, more accurate results, offering a competitive edge.
Authors
Rishi Khare
Abstract
Large language models (LLMs) have demonstrated remarkable capabilities in solving complex tasks in coding, mathematical logic, and iterative reasoning, especially by coordinating effectively within multi-agent systems (MAS). However, language agents traditionally communicate via text exchange, which introduces a bottleneck in information transfer and adds prefill latency for downstream agents. Recent work proposes that directly sharing the key-value (KV) cache between language models can enable efficient latent collaboration, entirely bypassing standard text-based communication. To enable latent communication between heterogeneous models of different sizes, prior work introduces trained fusion adapters for each model pair, which limits practical deployment. We introduce CacheFuse, a training-free method which demonstrates that augmenting a downstream model with an upstream model's KV cache using linear interpolation consistently enables superior performance than the standalone downstream model can achieve. We systematically analyze CacheFuse across four reasoning and knowledge benchmarks and demonstrate up to 8.6 percentage points higher accuracy compared to single-model baselines and up to 12.2x TTFT speedup compared to text-based communication in multi-model inference.