Model combines memory and flexible use for multiple visual tasks
Déjà View: A Shared Working-Memory Model for Visual Tasks
Computer Vision and Pattern RecognitionMachine Learning
Summary
Remembering pictures while doing different visual tasks is hard because the brain needs to keep track of what it saw and adjust what it focuses on. The authors created a computer model that can remember images and switch between tasks like recognizing a picture or deciding if it's new. The model uses a shared memory system that stores features temporarily and can handle distractions or delays. It shows that mistakes depend on what has been seen before and how many tasks are done, helping us understand flexible memory use better.
What this means in practice
- •For computer vision engineers: Design image recognition systems that maintain flexible memory under distraction and varying tasks.
- •For cognitive computing developers: Implement shared memory modules for AI models that perform multiple visual decision tasks using one system.
Authors
Mranmay Shetty, Gabriel Kreiman
Abstract
A working-memory model must remember stimuli and use its memory flexibly to perform different tasks. Here we introduce a model of working memory for images that combines dynamic activity with temporary associations between image features. We train the model on five tasks, including remembering a sample while other images are presented, categorizing distractor images while remembering the sample, and determining whether an image is new or has been presented before. The same trained model performs all tasks, guided by explicit cues about which response is required. We use this flexible memory model to ask how delay, distraction, and image history affect performance. Delays have little effect on sample-memory performance for the delays we tested, while seeing more images leads to more false alarms even when the target image appeared equally recently in all cases. On the sample-memory-plus-categorization task, the probability of correct responses on all questions decreases as more questions are asked, despite robust sample-memory performance. Finally, the model maintains stable performance across 40 brief trials of each task under the tested conditions. Together, these results demonstrate how a single working-memory system can be flexibly used to perform different tasks, and how its errors depend on both previous images and task demands.