Efficient methods to measure rare failure rates in fine-tuned language models
Amplify, Locate, Estimate (ALE) on Rare-Failure Rates of Fine-Tuned LLMs
Machine Learning
Summary
Measuring how often advanced language models make rare mistakes can be very hard because those mistakes happen so infrequently. The authors developed a new method called ALE that finds ways to increase the chance of these rare mistakes temporarily so they can be measured more easily. By carefully analyzing model behavior when mistakes are more common, ALE estimates how often these mistakes happen normally without needing huge amounts of data. This makes checking model safety faster and cheaper.
What this means in practice
- •For ai safety engineers: Estimate the frequency of rare undesirable behaviors in fine-tuned language models efficiently to improve model safety assessments.
- •For machine learning platform teams: Reduce the computational cost of evaluating fine-tuned language models for rare failure events using statistical amplification techniques.
Authors
Patrick Reichherzer, Jonathan Evertz, Thorsten Holz
Abstract
Safety fine-tuning can make undesirable model behaviors sufficiently rare that measuring their frequency by direct sampling becomes prohibitively expensive. We introduce Amplify, Locate, Estimate (ALE), a method for estimating such rare failure probabilities efficiently. ALE uses the change in token log-probabilities induced by the safety fine-tuning to define an amplification direction, along which we increase the failure rate for cheap measurement. We develop a score-function estimator that uses the selected amplified batch to recover the local slope of the log failure rate from the token probabilities of the sampled failures, enabling extrapolation back to rare target rates. We also use the amplified batch to obtain an importance-sampling estimate. Across 41 evaluation targets, ALE's estimate-to-reference ratios have a median close to one, with 37 ALE estimates within a factor of three of reference values obtained from independent direct measurements. Ablation tests show that with a budget of up to $\sim$1000 labeled rollouts, in addition to a cheap unlabeled scan, ALE estimates failure probabilities for Qwen3-8B fine-tunes down to about $10^{-5}$, where direct measurement can require hundreds of thousands of rollouts, as the required sampling budget scales inversely with failure probability at fixed relative precision.