Best of N jailbreaking success explained by a new physical barrier model

Escaping Alignment: A Physical Trap Model of Best-of-N Jailbreaking

Artificial Intelligence

Summary

Some AI language models have safeguards to avoid unsafe or harmful responses, but attackers try to trick them using many slightly changed versions of a risky question to get around these protections. Previous studies suggested that the chance of success follows a simple math rule, but this paper shows the pattern is more complex and affected by the data used. The authors propose a simple physical model like barriers that must be overcome to get unsafe answers, which helps explain how increasing the number of tries and different conditions impact the chance of success. This model matches what happens across different AI systems and lets people predict outcomes beyond the tested cases.

What this means in practice

  • For ai safety engineers: Assess and predict how best-of-N jailbreaking attempts succeed across various parameter settings and temperatures to improve guardrail designs.
  • For security analysts: Evaluate the vulnerability of deployed AI chatbots by understanding how prompt variations and sampling affect unsafe response generation.

Authors

Marco Biroli

Abstract

Best-of-$N$ jailbreaking (BoN) bypasses safeguards of aligned models by drawing $N$ independent augmentations of an unsafe prompt and sampling $M$ completions of each. Previous works have shown that the attack success rate (ASR) seems to follow a power-law in $N$, which we challenge. The exponent drifts with $N$, with an exponential crossover which is a finite-size artifact of the adversarial dataset. Little work has been done to explore the entire two-budget ($N, M$) attack surface as well as its dependence on the generation temperature $T$. We introduce a simple barrier model where each prompt has a baseline safety level and each augmentation a random thermally activated barrier. Then four numbers, each backed by an interpretable safety mechanism, determine the entire ($N, M$) attack surface. They extrapolate predictions from $N \leq 100$ to $N = 10^4$, collapse five distinct models on the same scaling function and predict ASR at different temperatures from the one they were fitted at.
View PDFOpen arXiv