Lightweight channel-aware adapter improves UAV vision perception under poor wireless conditions
FRESHLATENT: Channel-Aware Latent Adaptation for Resource-Constrained Embodied VLM Perception
Computer Vision and Pattern RecognitionDistributed, Parallel, and Cluster ComputingMachine LearningRobotics
Summary
Unmanned aerial vehicles (UAVs) often send visual data to computers for understanding their environment, but wireless interference can corrupt this data. The authors introduce FreshLatent, a small and efficient tool that adapts the data to handle these wireless problems without needing expensive hardware changes. FreshLatent helps UAVs keep seeing well even when the wireless connection is weak or noisy, using much less power and processing than bigger solutions. This approach lets UAVs operate more reliably on limited onboard resources and poor communication links.
What this means in practice
- •For drone system engineers: Implement more reliable onboard vision processing for UAVs operating under noisy wireless links with low power and latency overhead.
- •For mobile edge computing teams: Deploy efficient channel-aware adaptation techniques to maintain visual model interface quality in resource-limited edge devices with constrained communication.
Authors
Rajat Bhattacharjya, Minwoo Kim, Arnab Sarkar, Tamoghno Das, Sing-Yao Wu, Eli Bozorgzadeh, Marco Levorato, Nikil Dutt
Abstract
Mission-critical UAVs increasingly rely on split vision-language model (VLM) perception under tight onboard-resource and wireless-communication constraints. However, corruption of transmitted intermediate features creates a deployment mismatch for clean-trained split interfaces, while stronger channel-aware codecs can impose substantial onboard cost. We present FreshLatent, a lightweight channel-aware latent adapter that trains a power-normalized encoder-decoder through wireless corruption while keeping the surrounding VLM frozen. We formulate deployment around a mission-conditioned perception requirement and embedded interface cost, linking channel quality and communication budget to the operating conditions under which perception remains usable. At 0 dB and the tightest communication budget, FreshLatent improves gIoU and cIoU over clean split compression by 20.79 and 20.87 points, respectively. At the most adverse evaluated SNR (0 dB), across all three communication budgets, FreshLatent recovers 63.5-69.1% of the gIoU improvement achieved by a much heavier, range-trained feature-JSCC codec. On an NVIDIA Jetson AGX Xavier in 10-W mode, FreshLatent uses 37-40x fewer encoder parameters, 7.7-9.9x lower edge-interface latency, and 8.8-10.0x lower edge-interface energy than the heavier codec. Together, these results show that lightweight channel-aware adaptation can recover a substantial fraction of the robustness of a much larger communication interface while broadening quality-valid operation under constrained wireless conditions.