Framework finds gaps in large language model safety tests
Specification-Based Testing of Large Language Models
Software EngineeringArtificial Intelligence
Summary
Large language models (LLMs) like GPT are tested for safety using specific benchmark prompts, but it's unclear if these tests actually cover the safety claims made by the models’ documentation. The authors created LEST, a method that uses safety claims as test guides and automatically changes prompts to find cases where the models' responses don't match their safety promises. They found many safety claims aren’t fully tested by existing benchmarks and that automated checks often mistake safe responses for failures, so humans need to verify results. This method links natural language safety promises directly to tests and verified counterexamples, helping improve how LLM safety is assessed.
What this means in practice
- •For ai safety teams: Identify which documented safety claims of large language models are actually tested by current benchmarks and discover gaps to improve testing coverage.
- •For ai system auditors: Use specification-driven prompt mutation and human validation to find real cases where model safety claims fail beyond automated detection.
Authors
Kausar Moshood, Jace Bolante, Roshan Aditya Murali, Manish Motwani
Abstract
Model cards, system cards, and safety reports increasingly document LLM safety behaviors, yet safety evaluations typically start from predefined benchmarks rather than asking which documented claims those benchmarks actually test. We introduce LEST (LLM Evaluation through Specification-driven Testing), a framework that treats documented safety claims as specifications for black-box testing. In LEST, each claim determines which benchmark prompt tests it and whether a Refuse-to-Comply transition, where a model refuses a prompt but complies with its mutated version, counts as evidence against it. LEST mutates these prompts to search for counterexamples, manually validates each candidate, and traces validated counterexamples back to the claims they contradict. We evaluate LEST using the system cards of Claude Opus 4.8 and GPT-5.5 with 97 seed prompts from SORRY-Bench across Mental Health and Self-Harm, Harmful and Violent Content, and Fairness and Bias. Of 56 identified safety claims, 29 (51.8%) are prompt-testable, but only 17 (58.6%) are exercised by an applicable benchmark prompt. Both models refuse >95% of benchmark prompts. Evaluating 1,836 mutated prompts on both models reveals 312 candidate Refuse-to-Comply transitions flagged by automated response classifiers. However, human validation confirms only 19 (6.1%) as both intent-preserving and genuinely compliant, showing automated evaluation substantially overestimates behavioral failures. Tracing these 19 transitions back to the claims reveals counterexamples for 6 of 9 covered claims for which refusal is an applicable oracle. Our results show that benchmark prompts may not be sufficient to validate all documented LLM safety behavior, and automated failure detection requires manual validation. LEST provides end-to-end traceability from natural-language safety documentation to executable tests and human-validated counterexamples.