Better robot policies can make failure checks more difficult
When Better Policies Leave Harder Failures: Coverage Shift in VLM Success Monitoring
Robotics
Summary
Sometimes when robots get better at their tasks, it actually becomes harder to spot their mistakes. The authors studied a computer system that judges if a robot successfully places an object. As the robot improved, the system missed more subtle errors, especially when failures occurred in tricky spots hidden from view. This happens because the types of failures change as the robot learns, making them harder for a fixed judge to detect. The authors suggest tracking different kinds of misses to better understand monitoring performance.
What this means in practice
- •For robotics engineers: Improve robot task monitoring by accounting for changes in failure types as policies get better, enhancing detection of subtle errors.
- •For quality assurance teams: Design validation tests that separately report absolute and conditional error rates to better audit performance of automated systems over time.
Authors
Isabel Guan, Dingyuan Zhang, Shipeng Lyu, I-Ming Chen
Abstract
Can a better robot policy become harder to monitor? We study a fixed vision-language model (VLM) that judges success while a policy improves. In a controlled 20-seed tray-placement experiment, training selects only simulator-verified successes. Success rises from 66.8% to 97.0%, and accepted hover failures fall from 5.0% to 0.9% of executions. Yet the judge accepts 50.0% of remaining hovers, versus 31.1% under untrained sampling. These residual failures more often lie below the tray rim, which can conceal the gap beneath a grasped cube. A separate boundary probe accepts 148/160 below-rim hovers and 0/160 above-rim hovers. We call this change in conditional monitoring sensitivity coverage shift: learning changes which failures reach a fixed judge. A broader diagnosis covers nine judges and 1,392 stimuli. Judge-filtered experiments across eight settings further show heterogeneous effects of state-dependent acceptance. The results distinguish task improvement from failure detectability and motivate reporting absolute misses, conditional misses, and audit coverage together.