FlashRT: Towards Computationally and Memory Efficient Red-Teaming for Prompt Injection and Knowledge Corruption
2026-04-30 • Cryptography and Security
Cryptography and Security
AI summaryⓘ
The authors developed FlashRT, a tool that helps test the security of large language models (LLMs) that handle very long inputs. These models can be attacked through prompt injection and knowledge corruption, but current testing methods are slow and use a lot of computer memory. FlashRT makes these security tests much faster and more efficient, using less GPU memory. This lets researchers better understand and improve the safety of LLMs. The tool works with existing optimization methods and is publicly available.
large language modelslong contextprompt injectionknowledge corruptionoptimization-based attacksred-teamingGPU memoryblack-box optimizationFlashRTsecurity evaluation
Authors
Yanting Wang, Chenlong Yin, Ying Chen, Jinyuan Jia
Abstract
Long-context large language models (LLMs)-for example, Gemini-3.1-Pro and Qwen-3.5-are widely used to empower many real-world applications, such as retrieval-augmented generation, autonomous agents, and AI assistants. However, security remains a major concern for their widespread deployment, with threats such as prompt injection and knowledge corruption. To quantify the security risks faced by LLMs under these threats, the research community has developed heuristic-based and optimization-based red-teaming methods. Optimization-based methods generally produce stronger attacks than heuristic attacks and thus provide a more rigorous assessment of LLM security risks. However, they are often resource-intensive, requiring significant computation and GPU memory, especially for long context scenarios. The resource-intensive nature poses a major obstacle for the community (especially academic researchers) to systematically evaluate the security risks of long-context LLMs and assess the effectiveness of defense strategies at scale. In this work, we propose FlashRT, the first framework to improve the efficiency (in terms of both computation and memory) for optimization-based prompt injection and knowledge corruption attacks under long-context LLMs. Through extensive evaluations, we find that FlashRT consistently delivers a 2x-7x speedup (e.g., reducing runtime from one hour to less than ten minutes) and a 2x-4x reduction in GPU memory consumption (e.g., reducing from 264.1 GB to 65.7 GB GPU memory for a 32K token context) compared to state-of-the-art baseline nanoGCG. FlashRT can be broadly applied to black-box optimization methods, such as TAP and AutoDAN. We hope FlashRT can serve as a red-teaming tool to enable systematic evaluation of long-context LLM security. The code is available at: https://github.com/Wang-Yanting/FlashRT