VideoReloc improves indoor video localization using tiny semantic maps
VideoReloc: Long-Term Indoor Video Relocalization against a Kilobyte-Scale Semantic Scene Graph
Computer Vision and Pattern Recognition
Summary
Indoor video relocalization means figuring out where you are inside a building using video footage, even if the room looks different over time. The authors created VideoReloc, a method that looks at short video clips instead of single pictures to gather more clues from objects and motion, which helps it find the right location more reliably. It uses a very small map made of simple labels for objects, making it much smaller than typical maps. Tests showed VideoReloc can find locations much better over time despite changes like lighting or furniture being moved.
What this means in practice
- •For indoor navigation developers: Build indoor positioning systems that remain accurate despite furniture moves and lighting changes using very compact semantic object maps.
- •For robotics engineers: Improve robot localization indoors by verifying locations over video clips with spatial and motion context, reducing reliance on detailed visual appearance maps.
Authors
Qianru Li, Xuyang Chen, Xuqin Wang, Zhenghao Zhang, Hongyi Luo, Tao Wu, Daniel Cremers, Lu Liu, Yanfeng Zhang
Abstract
Given a compact semantic scene graph, long-term indoor video relocalization estimates a map-frame trajectory after lighting and furniture changes. Visual methods rely on appearance and become unreliable under these changes; localizing one frame at a time from object classes and geometry instead leaves sparse, ambiguous evidence. We introduce VideoReloc, whose adaptive clips use odometry to gather spatial evidence until object and motion criteria are met, adapting query length to the observed scene. Its run-level decision rechecks conflicting placements using evidence accumulated across connected clips, stabilizing the trajectory beyond adjacent-clip tracking. Hypothesis-first registration proposes poses from object triplets and verifies each using clip-wide object centers and box surfaces. Orientation-aware refinement uses box faces, gravity and wall directions to resolve ambiguity in camera orientation and refine the full pose. This reframes sparse-map relocalization as verification of spatially extended video queries, moving discriminative support from stored appearance to temporal context and permitting a 100 kB map of class-labelled boxes. On RIO10 and ReplicaCAD, the all-frame localization success rate at 1 m/10$^\circ$ is 73.5% and 61.1% under causal evaluation, rising to 90.6% and 74.8% with clip closure. The evaluated per-frame scene coordinate regressors reach up to 47.6% and 49.8%, respectively, with maps of 12.6-42 MB. Project page: https://videoreloc.github.io