RT-SHCUA: Real-Time Self-Hosted Computer-Use Agent for UAV Control
2026-07-20 • Cryptography and Security
Cryptography and SecurityArtificial IntelligenceRobotics
AI summaryⓘ
The authors explain that controlling drones using natural language is tricky because drones need quick and safe decisions, while typical computer agents work slower and aren’t designed for real-time control. They propose a new method where commands from the language agent are turned into clear, timed actions with safety checks before the drone executes them. This way, complex thinking happens in slower cloud systems, but the drone itself ensures commands are safe and authorized. Their prototype shows that this approach keeps drone control responsive and secure without risking unsafe behavior.
unmanned aerial vehiclesnatural-language controlself-hosted computer-use agentsreal-time controlsecurity enforcementtrusted execution environmentcloud computingedge computingdrone safetyagent-based systems
Authors
Di Lu, Bo Zhang, Xiyuan Li, Yongzhi Liao, Xuewen Dong, Yulong Shen, Zhiquan Liu, Jianfeng Ma
Abstract
Natural-language control offers a promising interface for unmanned aerial vehicles (UAVs), but directly applying self-hosted computer-use agents (SHCUAs) to UAV control introduces a structural mismatch. SHCUAs are designed for interactive host-side tool use, where delayed agent iterations are often acceptable. UAV control, however, is coupled with continuously changing physical states, strict timing constraints, safety risks, and security accountability. A stale, unauthorized, or tampered agent decision may therefore lead to unsafe or untraceable vehicle behavior. This paper proposes a real-time and security-oriented restructuring of SHCUA-based UAV control. Instead of allowing an SHCUA to directly issue flight commands, we transform its outputs into contract-bound UAV skill invocations with explicit timing, state, authority, fallback, and evidence semantics. Based on this abstraction, we design an architecture that separates semantic reasoning from onboard execution and security/safety enforcement. Slow cloud or edge reasoning is used for mission understanding, while onboard components validate and dispatch only timely, authorized, and state-consistent skills. Security-critical enforcement points can be protected by TEE-style or microcontroller isolation mechanisms without moving the full language agent or high-frequency flight-control loop into trusted components. Prototype evaluation shows that RT-SHCUA maintains bounded task-level responsiveness while supporting degraded handling, trusted admission, and auditable evidence preservation for SHCUA-mediated UAV actions.