Safety and performance in cross model cache reuse for key value pairs
To Cache or Not to Cache: Understanding and Mitigating Safety Incompatibility in Cross-Model KV Cache Reuse
Machine LearningCryptography and SecuritySoftware Engineering
Summary
Some AI models speed up their work by reusing parts of previous computations from related models, but this can cause problems with how safely they behave. The authors studied how different ways of reusing these parts can change safety and found that reusing certain pieces causes bigger safety issues. They created a method called VReparo that fixes these safety problems by adjusting reused data before the model uses it. This method keeps the speed benefits while making the output safer according to their tests.
What this means in practice
- •For ai developers: Safely reuse computation caches between related language models to reduce latency without increasing harmful outputs.
- •For cloud service engineers: Integrate safety-aware cache reuse techniques to optimize resource use and response times in multi-model AI services.
Authors
Zhiwei Lin, Jiahao Liu, Yan Liu, Chengpeng Wang
Abstract
Cross-model KV cache sharing reduces prefill computation by allowing a receiver model to reuse prefix KV states generated by a related sender model. Existing methods optimize such reuse for latency and generation quality, but its safety implications remain unclear. In this work, we systematically evaluate this issue across diverse sender-receiver pairs and reuse configurations. We refer to deviations from the receiver's native safety behavior as safety incompatibility. Our study shows that both the direction and magnitude of safety incompatibility depend on the model pair and reuse configuration, with value reuse causing substantially larger deviations than key reuse across all evaluated pairs. Furthermore, we propose VReparo, a technique that learns per-layer and per-KV-head affine maps offline to align reused sender value tensors with their receiver-native counterparts before decoding. Across two model pairs under full KV reuse and the state-of-the-art selective reuse setting, VReparo reduces the average HarmBench attack success rate (ASR) from 57.28% to 39.21% and the average StrongREJECT score from 0.4215 to 0.2163 upon our benchmarks, while retaining an average Time to First Token (TTFT) reduction of 85.62%.