PerSeM improves long-term semantic mapping for UAVs with persistent memory
PerSeM: Persistent Semantic Memory for Long-Horizon Open-Vocabulary UAV Mapping
Computer Vision and Pattern RecognitionRobotics
Summary
UAVs can see and label things around them but often get inconsistent results when looking repeatedly from different angles. The authors developed PerSeM, which keeps a memory of what the UAV has seen over time, combining many observations to get more accurate and stable labels. This method doesn’t need to be trained again and improves the UAV’s understanding especially in tricky places. Tests showed PerSeM makes the UAV’s semantic maps both more accurate and reliable over long periods.
What this means in practice
- •For drone navigation teams: Generate more accurate and stable semantic maps over time, improving UAV sensing in complex environments without retraining perception models.
- •For environmental monitoring teams: Create consistent long-term semantic maps from UAV data to better track changes and features in natural areas like forests.
Authors
Saurbh Singh Jamwal, Ganesh Ramakrishnan
Abstract
Open-vocabulary segmentation enables rich semantic perception for UAVs, but frame-wise predictions can remain temporally inconsistent across repeated observations and changing viewpoints. We present PerSeM, a training-free persistent semantic memory framework for long-horizon open-vocabulary UAV mapping. PerSeM associates frame-wise semantic observations with persistent world-space voxels and constructs a majority-based semantic memory, which is conservatively refined through history-preserving spatial refinement, trust-aware replay, and context-guided verification. Experiments on the Forest and UAVScenes benchmarks show that persistent 3D memory provides substantial gains in semantic correctness and temporal stability over frame-wise predictions. Beyond this strong persistent-memory baseline, PerSeM provides consistent additional improvements, improving both semantic accuracy and temporal stability across all five evaluated UAVScenes sequences. Analysis using regions identified independently of the final PerSeM predictions further shows that these gains are concentrated in semantically difficult and temporally unstable regions, where majority-based memory is most likely to remain uncertain. These results demonstrate that persistent 3D aggregation provides a strong foundation for long-horizon semantic mapping, while conservative refinement of uncertain memory states can provide additional improvements without retraining or additional neural-network inference.