Benchmark tests language model safety under missing facts
EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence
Artificial IntelligenceMachine Learning
Summary
Sometimes AI models use tools to get facts before acting. If some important facts are missing, the AI might do something unsafe even if the user’s request is harmless. The authors created a test called EpiSEM-Bench that checks how well different AI models handle missing facts and avoid unsafe actions. Their tests show that some models are better at finding missing information and acting safely, but there is still lots of variation.
What this means in practice
- •For ai developers: Evaluate and compare language model agents’ safety when they encounter missing critical facts in tool responses using this benchmark.
- •For software testers: Verify that AI agents do not perform unsafe actions when key facts are withheld by using the benchmark’s task pairs and audit tool.
Authors
Qijin Zhang, Hanzhen Wang, Yuehan Zhao, Haoyu Dai, Yao Shu, Zhongxiang Dai, Shuang Qiu
Abstract
Tool-using language model agents rely on environmental facts to decide whether an action is permitted. When a tool response omits a safety-relevant fact, an agent may carry out a prohibited action even though the user request is benign and every returned statement is true. We introduce \bench{}, an executable benchmark for evaluating unsafe execution, safety boundaries, and evidence recovery in LLM agents under missing evidence. It contains 300 matched unsafe/safe task pairs across six domains. By systematically suppressing critical facts, the benchmark measures unsafe execution under critical evidence omission and determines the minimum observed failure cardinality, including failures with full evidence and cases with no observed failure after exhaustive testing. An independent audit tool tests whether agents seek missing facts and then withhold prohibited actions while completing permissible tasks. The benchmark reveals substantial variation across the ten models in omission robustness, evidence acquisition, and appropriate action. Resistance to missing evidence does not necessarily translate into useful recovery. Recovery access reduces unsafe execution for all ten models, but agents differ in whether they actively obtain the missing facts and use them to act appropriately in both worlds.