Jailbreak detection improves with prototype-based contrastive learning
Jailbreak Detection on Large Language Models via Contrastive Prototype Structures
Computation and Language
Summary
Jailbreaks and harmful requests make it risky to use large language models safely. The authors propose a new method called JDPI that detects unsafe prompts by learning typical examples of harmful and safe inputs and seeing how close a new input is to these. JDPI requires fewer labeled examples than traditional classifiers and works well across several tests. It also improves accuracy by using previous conversation history and shows better robustness when applied to different types of data.
What this means in practice
- •For ai safety teams: Detect harmful or jailbreak prompts more efficiently using a method that needs fewer labeled examples and adapts well across datasets.
- •For chatbot developers: Improve conversation-level safety detection by incorporating user history into prompt evaluation for more accurate multi-turn analysis.
Authors
Yan Qiu, Yuanyuan Li
Abstract
Jailbreak attempts and harmful prompts remain a challenge for the safe deployment of large language models. We study whether contrastively learned representations can support safety detection without training an additional downstream classifier. We propose Jailbreak Detection with Prototype Inference (JDPI), which uses binary safety labels to construct harmful and benign prototypes and makes predictions from relative prototype distances. JDPI uses only binary safety labels for encoder adaptation, prototype construction, and validation. In our setup, it uses fewer labeled examples than the classifier baseline (Method A) by avoiding the separate split used to train the downstream classifier. Across four benchmarks covering different input formats, contrastive fine-tuning improves JDPI's detection accuracy by 9.1-18.0 percentage points. Using the same encoder, JDPI achieves slightly higher ROC-AUC on one benchmark and stays within 0.017 of the baseline across all four. With dataset-specific adaptation, JDPI exceeds the released Llama Guard 3-8B baseline by 0.173-0.310 in ROC-AUC across the four benchmarks. In multi-turn replay on TomGibbs, JDPI uses accumulated user history as the conversation unfolds, achieving higher conversation-level accuracy than when it uses the current turn alone. Across 12 directed transfers, JDPI outperforms the classifier by an average of 1.59 percentage points in accuracy, demonstrating stronger cross-dataset robustness.