LaCache: Robust Semantic Caching for LLM Serving
2026-08-03 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors study how systems that speed up large language models by remembering similar past requests can be tricked by attackers who insert harmful queries, messing up future answers. They propose LaCache, which not only checks if a new query matches stored ones but also verifies early parts of its answer to ensure it's safe. This approach makes it very hard for bad queries to interfere with real users and also helps find better matches for faster responses. Tests show that LaCache effectively blocks attacks and improves efficiency.
Semantic cachingLarge Language ModelsCache-collision attacksEmbeddingsCache hitAdversarial queriesToken decodingCache retrievalSecurity in cachingEfficiency in LLM serving
Authors
Jiacheng Liang, Yuhui Wang, Tanqiu Jiang, Ting Wang
Abstract
Semantic caching, which reuses responses to semantically similar requests via their embeddings, has seen growing adoption in LLM serving, offering faster responses and reduced costs. Yet existing schemes are fundamentally vulnerable to cache-collision attacks, wherein an adversary pollutes the cache by injecting crafted queries, corrupting responses to subsequent legitimate requests. We present LaCache, a novel semantic caching scheme that addresses this vulnerability through a conceptually simple yet principled redesign. The key insight is that while the adversary has full control over the adversarial query, it has far less control over its response, which must simultaneously satisfy multiple semantic constraints. Rather than checking only the cache hit of a query, LaCache additionally checks the cache hit of its first k (speculatively) decoded tokens. This design yields two concrete benefits. First, it provides formally guaranteed resilience against cache-collision attacks: we prove that it is impossible to craft adversarial queries that simultaneously elicit malicious responses and collide with benign queries. Second, the enriched index supplies additional semantic context for cache retrieval, improving response relevance. Empirical evaluation across diverse LLMs and benchmarks validates both LaCache's security guarantees and efficiency gains, pointing to a promising direction for robust semantic caching.