Confidence improves safety filters for robot control policies
Confidence-Modulated Safety Filtering for End-to-End Policies
Robotics
Summary
Robots often rely on safety filters that decide when their actions are safe, but these filters don’t adapt to how confident the robot is in its own behavior. The authors suggest adjusting these safety checks based on how sure the robot’s controller feels about its actions. When the robot is confident, the filter allows more freedom for better performance; when it's unsure, the filter restricts actions more to keep safety. In simulations, this approach combines the best of strict and flexible filters, making robot actions safer and more efficient.
What this means in practice
- •For autonomous vehicle developers: Adapt safety filters in vehicle control software dynamically according to the confidence of driving models, enhancing performance without sacrificing safety.
- •For industrial robot programmers: Implement confidence-aware safety filtering in robot control to optimize operation when the robot’s behavior is reliable, and increase caution when uncertain.
Tested on simulated data.
Authors
Javier Borquez, Jaime F. Fisac
Abstract
End-to-end policies, from task-specific controllers to robot foundation models, learn complex behavior from raw observations but are not guaranteed to remain safe, so they are often wrapped in a safety filter. The resulting system operates under two domains: the filter's operational design domain (ODD), where its assumptions hold and safety is certified, and the policy's competence domain, where it is reliable. Filters typically fix their conservativeness regardless of the policy's competence, trading performance inside the ODD against margin beyond it. We ask whether the policy's confidence should set that level, and propose a filter-agnostic pipeline that modulates it at runtime: inside the ODD, where every setting is safe, confidence buys performance; beyond it, low confidence adds margin, which helps when it flags where the filter's model fails. Preliminary F1TENTH simulations, where this holds, show the modulated filter combining the strengths of permissive and conservative filters.