Agents struggle to safely handle unclear task instructions
BARC-Bench: Benign Asks, Risky Consequences
Artificial Intelligence
Summary
Sometimes when people ask AI agents to do tasks, the instructions don’t clearly say what is safe or unsafe to do. The authors created BARC-Bench, a set of 110 example tasks where simple requests can lead to risky or unintended actions. They tested advanced AI models on these tasks and found that even the best models often make mistakes that break safety rules. This shows that AI agents still have trouble consistently following hidden safety boundaries while carrying out complex instructions.
What this means in practice
- •For autonomous system developers: Test and improve AI agents to handle underspecified tasks without causing risky outcomes in simulated workspaces.
- •For enterprise software teams: Assess AI models’ ability to respect implicit safety rules in workplace task automation to reduce unintended consequences.
Authors
Kangkang Li, Kelvin Luu, Lily Ma, Tu Trinh, Ying Liu, Mohamed Elfeki
Abstract
Task instructions from users for agents are inherently underspecified, especially regarding safety boundaries--creating behavioral risks. To study these risks, we introduce BARC-Bench, a collection of 110 tasks across nine simulated workspace environments where benign requests leave openings for unsafe actions. In this benchmark, we evaluate agent behavior against safety boundaries established by workspace records, implicit environmental context, and unstated social norms. For example, an agent asked to schedule off-boarding discussions must arrange the meeting without prematurely revealing an employee's resignation through a shared calendar invitation. Evaluating four frontier model configurations across five attempts per task reveals that models are not consistently safe: the share of tasks completed safely on all five attempts ranges from 9.1% to 72.7%, with even the strongest configuration producing a safety violation on 27.3% of tasks. These results highlight an asymmetric operational risk: while models frequently complete complex workflows, their inability to consistently respect unstated boundaries presents a substantial barrier to safe autonomous delegation.