Summary
Some AI systems that understand images and language to choose actions can be tricked by secret triggers to do bad things while looking normal. The authors introduce a method called TMT that watches the AI's behavior during use to catch these hidden bad behaviors, even when the AI faces new tasks it has never seen before. TMT checks for unusual patterns in how the AI processes information and spots errors when the AI's internal steps don't match expected behavior. They also suggest a way to clean the AI of these backdoors without needing a totally safe version beforehand. TMT outperforms other methods in tests against sneaky attacks on vision-language-action AI.
What this means in practice
- •For autonomous vehicle engineers: Detect hidden backdoors in AI decision-making systems to ensure safer vehicle responses during unexpected conditions.
- •For robotics control teams: Monitor and identify stealth manipulation of robot policies in real time to prevent harmful actions triggered by subtle inputs.
Authors
Zirun Zhou, Jingfeng Zhang, HaoChuan Xu, Xizhe Zhang, Elliott Wen, Jing Sun, Hong Jia
Abstract
Backdoored vision-language-action (VLA) policies can preserve benign task performance while producing malicious actions when a trigger appears. Detecting such activation is difficult because malicious behavior can comprise individually plausible actions, while unfamiliar tasks introduce legitimate changes in observations and behavior. We introduce TMT, a runtime backdoor detector based on Token Manifold and latent Transition modeling. Trained on benign rollouts, its two branches assess input-token structure and prediction errors in adjacent-layer latent dynamics. A suspicious rollout identified by the token manifold branch, once confirmed through latent deviations, guides transition selection for subsequent monitoring. We further explore policy purification through self-distillation: a frozen copy of the backdoored policy provides benign-input actions to supervise a student on paired benign and triggered observations, without requiring a separate clean reference policy. For evaluation, we adapt traditional backdoor detectors and repurpose anomaly and failure detection methods as VLA backdoor detectors. In a post-hoc comparison with ten baselines, TMT achieves state-of-the-art backdoor detection performance on unseen tasks across three VLA backdoor attacks. Our project page is available at https://zzr42.github.io/tmt/.