AI summaryⓘ
The authors studied how storing insecure coding habits in the long-term memory of large language models (LLMs) affects the security of the code these models generate later. They tested four LLMs with five programming languages and found that insecure memories made the models produce more vulnerable code, and warnings about these vulnerabilities did not keep up with the increase. They also found it is hard to remove these insecure memories through normal use, and these memories affect outputs even with different prompts. Finally, the authors tested ways to reduce vulnerabilities, showing some methods lower security risks but can hurt code correctness, while filtering risky memories can fully stop insecure influences.
LLMlong-term memorycode generationvulnerable codesecurity riskprompt engineeringmemory filteringfunctional correctnessmitigation strategiesprogramming languages
Authors
Yuchen Chen, Wei Cheng, Yuan Xiao, Zhou Yang, Weifeng Sun, Chunrong Fang, Xiang Chen, Baowen Xu, David Lo, Zhenyu Chen
Abstract
LLM-based systems increasingly incorporate long-term memory to improve cross-session continuity. However, once insecure coding preferences are stored, they may silently influence security-critical decisions in subsequent generations. In this study, we conduct the first systematic empirical study on the impact of insecure coding preferences stored in long-term memory on the security of LLM-based code generation. We evaluate four LLMs (ChatGPT, Gemini, Qwen, and Grok) across five programming languages (Python, C, C++, Go, and JavaScript). Our results show that insecure memories significantly increase the risk of generating vulnerable code by 2.7-50.3 percentage points (pp). Moreover, they create a 5.4-14.0 percentage-point risk-warning gap, where warning-rate increases lag behind vulnerability-rate increases. Further analysis reveals that insecure memories are difficult to overwrite through normal interactions and can broadly influence model outputs even when prompts are phrased differently. Finally, we evaluate three mitigation strategies: security-requirement appending and memory storage reduce vulnerability rates by 19.7-33.6 pp but may degrade functional correctness by up to 15.9 pp; memory-level safety filtering achieves a 100\% detection rate on our evaluated risky memory entries and restores generation behavior to the without-memory baseline. Based on these findings, we provide actionable suggestions to improve the security of long-term memory in LLM-based code generation.