Timely stepwise safety checks improve agent risk intervention decisions
StepSafe: Learning Timely Intervention with Context-Aware Safety Reasoning
Artificial Intelligence
Summary
Some smart systems take many actions over time, and whether each action is safe depends on what happened before. The authors introduce a method called ASPO that helps these systems decide the right moment to step in and stop risky actions by learning from each step separately. Their approach compares possible next actions at each stage and rewards timely decisions to continue, ask questions, or stop before harm happens. They also teach the model to learn from its own mistakes and successes to improve over time. Their system, StepSafe, shows better safety and usefulness compared to other open systems on several tests.
What this means in practice
- •For ai safety engineers: Introduce stepwise intervention training to improve timing and accuracy of AI system safety checks during ongoing actions.
- •For multi-agent system developers: Use adaptive feedback and red-teaming in training to enhance timely detection of unsafe behaviors between interacting agents.
Authors
Siyi Gu, Kunhan Gao, Chao Peter Yang, Sukanta Ganguly, Rex Ying, Arman Cohan
Abstract
Safety in agentic systems is inherently sequential: whether an action is safe depends on prior actions, tool outputs, information disclosures, and interactions among agents. Recent work has shifted agent safety guardrails from post-hoc trajectory-level assessment toward step-level action diagnosis and proactive risk prediction, yet existing training objectives provide limited supervision for how step evidence accumulates over time and when it becomes sufficient to justify intervention. We propose \methodlong~(ASPO): a training framework for step-level timely intervention. Pre-fix level stage alignment allows crediting each decision at its own step: it compares each response with the other responses sampled at the same prefix and rewards the action that fits the stage, continuing before the risk is visible, asking while clarification can still avert harm, and stopping before harm is irreversible. Adaptive feedback self-distillation further credits the reasoning behind each decision, with the model itself as a teacher that sees the step annotations on intervention steps and rules mined from its own false alarms on safe steps. Its training data carry a violation taxonomy and an intervention window at every step and are extended through self-evolving multi-agent red-teaming. Finally we present \textsc{StepSafe}, a step-level reasoning guardrail with timely intervention, that achieves the state-of-the-art peformance among open-source guardrails across five agent-safety benchmarks on safety-utility pareto curve with Youden's $J$ of 53.0 (4B) against 46.2.