Industrial system scales robot fleet memory by encoding only on demand
Industrial ReMEmbR: Scaling Spatiotemporal Memory to Robot Fleets
RoboticsArtificial IntelligenceSoftware Engineering
Summary
Warehouse robot fleets record lots of video to understand their environment, but saving all this data is costly. The authors designed IndustrialReMEmbR, a system that lets robots keep useful memory of their surroundings without sending raw video to central servers. Each robot creates smaller summaries and hides personal information to comply with privacy laws. The system delays creating detailed scene descriptions until needed, reducing computing work on the servers. This approach helps explain events like blocked aisles by searching through useful robot memories efficiently.
What this means in practice
- •For warehouse operations teams: Provide on-demand detailed reports of robot observations to quickly diagnose problems like aisle blockages without storing all raw video.
- •For security monitoring operators: Enable privacy-sensitive video monitoring by abstracting people from video data at the edge before storage and analysis.
Authors
Jad Yammine, Malek Wahidi, Abrar Anwar, Yan Chang, Mihir Acharya, Anthony Rizk
Abstract
Warehouse robot fleets continuously observe operational conditions, but preserving these observations as reusable semantic memory incurs substantial computation, communication, and storage costs. For example, explaining recurring aisle blockages can require reviewing video from multiple robots across several shifts. We present IndustrialRemembr, a shared spatiotemporal memory system that extends NVIDIA ReMEmbR to robot fleets and reduces server-side computation by generating scene descriptions only on demand. On each robot, an NVIDIA Jetson encodes video clips into retrieval embeddings with Cosmos-Embed1, vision tokens with Cosmos3-Edge, and reconstruction latents with the Wan2.2 VAE, without uploading raw camera frames. The encoding models are re-used to detect people in image patches, zeroing them out to support PII abstraction for GDPR-compliant deployments. We split Cosmos3-Edge at its vision-language boundary, placing the vision encoder on the robot and the language decoder on an on-premises server. A Nemotron agent retrieves memories by content, position, and time and queries the decoder using cached vision tokens. This design eliminates caption generation at write time, while cached-token decoding reproduces the unsplit model's output in all 450 tested decodes.