Robot memory stays accurate after map updates in navigation tasks
P-POSEMEM: Projective Semantic Memory for Consistent Language Grounding under Pose-Graph Rewrites
Robotics
Summary
Robots that follow spoken instructions need to remember which objects they are talking about, even when their internal maps of the environment change. The authors show that common map updates cause robots to confuse objects they previously identified. They developed a new memory method called P-POSEMEM that keeps observations linked to their original map points, allowing consistent object recognition despite map changes. Their tests prove this method almost completely stops confusing objects in various indoor scenes.
What this means in practice
- •For robot developers: Improve robot instruction following by preserving consistent object references after map corrections in navigation.
- •For autonomous vehicle teams: Maintain stable environmental understanding when vehicle maps are compressed or updated through pose-graph optimizations.
Authors
Ha Sier, Ali Salmasi, Mengya Xu, Haizhou Zhang, Jie Lu, Zhuo Zou, Xianjia Yu, Tomi Westerlund
Abstract
A robot following language instructions needs its semantic memory to keep naming the same physical object while the SLAM pose graph underneath is optimized, loop-closed and compressed. Maps committing each detection to a world coordinate cannot: a closure moves the anchor it was measured from, or the solver marginalizes that anchor, and the query then selects a different object although both graphs represent the same posterior. P-POSEMEM stores each observation as an immutable event at its birth keyframe, retains the Bayes-tree elimination conditional of every marginalized keyframe, and integrates the semantic likelihood over the reconstructed joint posterior of poses, anchors and identities. Dproj, the total-variation defect between the language-goal distributions of inference-equivalent full and marginalized graphs, measures this directly. Over 40 HM3DSem scenes and 112,000 queries, P-POSEMEM reproduces the full-graph oracle (Dproj = 0) and reduces goal flips against every memory-reducing baseline. On an eight-run campaign whose 761 closures rewrote the map by up to 47 m, Dproj stays below 10^-13 with 0/288 goal flips when elimination follows the closures, where every ablation and a coordinate committed at insertion flip goals it does not; under a live bounded solver the same memory flips 23/288 against 53 for that frozen coordinate. A pre-registered negative control is detected by Dproj while leaving calibration error and navigation success unchanged, indicating that these measures capture distinct failure modes. Retrieval is held fixed by a shared frozen detector, isolating the gain to memory consistency. Code and data: https://anonymous.4open.science/r/posemem-2328/.