Endoprompt creates prefixes that weaken AI task performance
ENDOPROMPT: Victim-Side Pseudo-References for Utility Degradation
Artificial IntelligenceCryptography and Security
Summary
Sometimes, AI systems can be tricked so they do worse on normal tasks without producing harmful content. The authors present Endoprompt, a method that learns short text additions (prefixes) to a request that make AI helpers less effective. Their approach does this by comparing the AI's usual responses and finding changes that reduce the quality of the answers, without needing labeled examples. This method works on different AI models and a variety of tasks, showing a clear drop in how well the AI performs.
What this means in practice
- •For security teams: Identify how AI systems can be subtly manipulated to reduce their effectiveness without triggering harmful outputs.
- •For ai system testers: Test robustness of instruction-tuned language models by generating subtle input prefixes that degrade their task performance.
Authors
Qingyu Wu, Zeyu Feng, Yongda Yu, Yuzhe Luo, Hua Cheng
Abstract
Prompt injection can degrade benign task performance without eliciting harmful content. Yet many attack objectives depend on task labels or predefined target responses. We present ENDOPROMPT, a white-box method that learns utility-degrading prefixes from unlabeled instructions. Its generator takes the request text as input. Clean victim continuations serve as pseudo-references: local search identifies prefixes that reduce continuation likelihood, and preference fitting on comparisons within the same instruction, followed by reward refinement, distills this signal into a generator. At deployment, the generator produces one prefix per request without further victim-side search. Across four instruction-tuned models and the complete splits of seven benign benchmarks, ENDOPROMPT yields a mean utility change of -26.8 percentage points; 27 of 28 cells are negative. Failure analysis reveals output expansion and prefix reuse; the controls do not establish a degradation advantage from request matching. Victim-derived supervision can reveal utility weaknesses without benchmark feedback or prescribed failure responses. The code will be released upon acceptance.