SafeHarness improves robot safety by avoiding obstacles during tasks
Coding Agents with an Obstacle-Aware Harness for Safe Robot Manipulation
RoboticsArtificial IntelligenceComputation and LanguageComputer Vision and Pattern Recognition
Summary
Robots sometimes bump into things when trying to complete tasks because they only focus on finishing the task, not avoiding accidents. The authors found that existing robot controllers don’t make safety a priority in planning how to move and touch objects. They created SafeHarness, a system that helps robots plan safe paths around obstacles and pick safe spots to touch objects. This approach helps robots complete tasks better and avoid collisions much more often.
What this means in practice
- •For robotics engineers: Build safer robot controllers that plan routes and contacts to avoid collisions with obstacles during manipulation tasks.
- •For industrial automation teams: Deploy robots in cluttered environments with improved obstacle-aware planning to reduce accidents while performing manipulation jobs.
Authors
Bingxin Xu, Yuzhang Shang, Zhen Dong, Emilio Ferrara
Abstract
Coding agents have emerged as a promising paradigm for robot manipulation: a language model writes the robot controller as a program, and agents built in this way now operate robots without robot-specific training.Whether this paradigm is also safe, however, has not been asked. We evaluate coding agent under a safety constraint, where each task pairs a manipulation goal with an obstacle the robot must not touch. The agent pursues the goal but collides with the obstacle in most cases, treating task completion as its sole objective while neglecting safety. The agent reasons about the obstacle in its traces, and the prompt already forbids touching it, so neither perception nor instruction is at fault; the fault lies in the planning, where the stated constraint never becomes a priority. By decomposing manipulation into a route phase and a contact-rich moment, we locate the source of the failure. Along the route, the model cannot prioritize the safety constraint, having no notion of a clearing route and none of replanning once a chosen route becomes infeasible. At the contact, it is unaware that contact execution is bounded by the same constraint. To close this gap, we present SafeHarness, which equips the model with two obstacle-aware harnesses that enable it to prioritize the safety constraint. Obstacle-aware route planning grounds the objects as bounding boxes and draws candidate routes over them as sequences of waypoints. The agent then plans a route in advance, verifies it, replans when necessary, and only then executes it. Obstacle-aware contact execution instead selects the contact position so that the contact itself avoids the obstacle. SafeHarness attains 71.9% task success and 87.5% collision avoidance, surpassing the previous SOTA by 6.5% and 27.0%, respectively. These results are $2.3\times$ and $1.5\times$ those of the same agent without harnesses.