GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors created GROVE, a system that helps wearable assistants remember and use what they see over time without needing extra training. It organizes memories from videos into moments, episodes, and repeating patterns across days to better answer questions or proactively help users. Both answering questions and offering help use the same memory, just accessed differently. Their tests showed GROVE works better than similar methods, especially when looking at patterns over multiple days.
wearable assistantvideo-memory systemcausal memory growthtime-stamped momentsepisodescross-day patternsreactive question answeringproactive assistancememory retrieval
Authors
Sitong Gong, Caixin Kang, Tianyu Yan, Guo Chen, Bo Zheng, Kaipeng Zhang, Yunzhi Zhuge, Xiang Ruan, Huchuan Lu, Yifei Huang
Abstract
A wearable assistant should both answer questions about its visual history and recognize when that history is useful to the present situation. Existing video-memory systems primarily support question-conditioned recall, whereas proactive assistants typically use separate memory and control mechanisms. We introduce GROVE, a training-free framework that supports both behaviors with one memory grown causally from a continuous video stream. GROVE retains fine-grained perceptual evidence and incrementally consolidates it into time-stamped moments, coherent episodes, and recurring cross-day patterns. Each stratum is paired with a scale-native retrieval skill for locating an observation, replaying an activity, or traversing long-range regularities. Reactive QA and proactive assistance share this memory and access interface, differing in whether retrieval is initiated by a user query or the current situation. Across multiple benchmarks including the challenging MM-lifelong and EgoServe, GROVE achieves the best results among the compared methods. Controlled ablations show that the temporal strata and their access skills are complementary, with patterns providing the largest benefit when evidence spans multiple days. Code will be available at https://github.com/SitongGong/GROVE.