Reinforcement learning makes drone control safer against action attacks
ASGARD: Action-Space Guard for UAV Resilience via Reinforcement Learning
Machine LearningCryptography and SecurityRobotics
Summary
UAVs (drones) guided by reinforcement learning can have their commands tampered with before the drone acts on them, causing problems or crashes. The authors created ASGARD, a method that detects and corrects these tampered commands using a two-step training process with a teacher and a student model. ASGARD learns from extra information about attacks during training but can work alone during flight using only the drone's usual sensor data. Tests show that ASGARD helps drones finish missions safely even when facing unexpected or hidden attacks.
What this means in practice
- •For drone operators: Protect UAV control systems from tampered commands to ensure successful navigation despite signal interference or attacks.
- •For autonomous vehicle engineers: Incorporate action-command correction layers into RL-based controllers to increase safety of autonomous robots facing command corruption.
Authors
Mohsen Salehi, Karthik Pattabiraman
Abstract
Reinforcement learning (RL) controllers have been recently adopted for Unmanned Aerial Vehicles (UAV) navigation and control. However, they are susceptible to action-space attacks that overwrite the action commands after the policy generates them and before the actuators execute them. While most existing defenses target attacks on the policy's inputs, those addressing action-space attacks retrain the policy at training time and are not resilient to corrupted actions at runtime. We propose ASGARD, a two-phase teacher-student pipeline for making RL-based UAV control resilient to action-space attacks. In the teacher phase, an encoder combines the UAV's physical state with action-attack-related privileged information to produce an action-attack-aware latent that trains the RL control policy and a monitor that outputs corrected action commands to the actuators. In the student phase, both the encoder and the monitor are trained via supervised learning from their teacher counterparts to run on-board using only the UAV's physical state history. We evaluate ASGARD across attack scenarios targeting different action commands on UAV. We find that ASGARD is resilient to action-space attacks and completes the missions despite the attack. We further find that ASGARD generalizes to unseen attacks and remains resilient against stealthy attacks.