Driving on Memory
2026-08-31 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionMachine LearningRobotics
AI summaryⓘ
The authors studied how well autonomous driving models perform when they use stored memories of a location instead of live camera input. They found that on the NAVSIM benchmark, models using only memories did almost as well as models seeing the current scene, meaning these models might not be reacting to real-time traffic as much as expected. However, on other benchmarks like Bench2Drive and RealEngine, using memories led to worse performance. This suggests that some benchmarks might not fully test a model's ability to respond to changing traffic conditions.
end-to-end autonomous drivingfuture trajectory planningbenchmark evaluationNAVSIMBench2Drivesimulation-based metricsscene understandingcamera inputmemory retrievaltraffic state
Authors
Christian Löwens, Thorben Funke, Alexandru Paul Condurache
Abstract
End-to-end autonomous driving models plan future trajectories from raw sensor input. While earlier driving benchmarks often measured deviation from the human trajectory, current benchmarks such as NAVSIM and Bench2Drive evaluate models with richer simulation-based metrics intended to capture safe and compliant driving. A high benchmark score should reflect that a model can understand the scene in front of it and act accordingly. But how much of that score specifically comes from reacting to the dynamic part of that scene? To probe this, we remove a model's camera input and replace it with memories from prior drives at the same location. The retrieved memories can provide persistent scene information, including road layout and location-conditioned regularities, but not the current traffic state. Surprisingly, memory is nearly sufficient on NAVSIM, reaching or even exceeding the performance of leading end-to-end methods without actually observing the evaluated scene. Our results suggest that a high NAVSIM score does not require a planner to react to the current traffic scene and should be treated with caution. This effect is benchmark-dependent: driving from memory causes substantially larger performance drops on Bench2Drive and RealEngine. We provide our code at https://github.com/boschresearch/MemoryDrivoR .