UAV mission intent recognized using synchronized multimodal data
AeroIntent: A Synchronized Multimodal Benchmark for UAV Mission Intent Recognition
Robotics
Summary
Understanding what a drone plans to do just by watching it is very important for keeping low flying airspace safe. The authors created a new test set called AeroIntent that combines different types of data like radar and video to better figure out drone missions. They also developed AeroMIR, a method that uses this data to recognize these missions even when some information is missing or incomplete. Their method works better than earlier approaches, especially when trying to guess the drone's intent early on or with limited radar data.
What this means in practice
- •For airspace surveillance teams: Improve detection and anticipation of low-altitude drone missions using combined radar and visual data for enhanced security measures.
- •For autonomous vehicle developers: Enable integration of multimodal intent recognition for better decision-making in vehicle navigation systems exposed to drones.
Authors
Yutong Chen, Jingxin Mao, Zhiwei Wei, Bing Li, Jianzhong Ni, Rongqing Zhang
Abstract
Reliable recognition of mission-level UAV intent from multimodal observations is essential for low-altitude airspace surveillance and security. Existing multimodal UAV datasets primarily support detection, tracking, and localization, leaving limited support for evaluating mission-intent recognition across held-out trajectory programs and incomplete radar histories. We introduce AeroIntent, a simulation-based benchmark comprising 990 synchronized multimodal sequences generated by 22 trajectory programs across five mission intents. The benchmark provides radar, RGB, infrared, and depth observations, together with program-disjoint, radar-gap, and observation-prefix evaluation protocols. Building on this benchmark, we propose AeroMIR, a structured Multimodal Intent Reasoner that integrates radar trajectories, derived behavioral features, selected visual evidence, and task knowledge for candidate-based UAV intent recognition. Under a strict program-disjoint protocol evaluating unseen tactical realizations, AeroMIR achieves a mean accuracy of 71.56% and a mean Macro-F1 of 69.84% across three seeds, outperforming the strongest trajectory-only baseline by 7.79 Macro-F1 points and a conventional radar-RGB late-fusion baseline by 19.32 points. AeroMIR also demonstrates strong performance under controlled radar gaps and early intent recognition settings. AeroIntent and AeroMIR provide a reproducible foundation for multimodal UAV intent recognition. The benchmark is available at: https://anonymous.4open.science/r/AeroIntent