D-NOVA: In-Storage Retrieval Accelerator via Dual-Bound 3D NAND-Optimized Similarity Search with Vector Adaptation
2026-07-20 • Hardware Architecture
Hardware ArchitectureComputation and LanguageDatabasesEmerging TechnologiesInformation Retrieval
AI summaryⓘ
The authors designed D-NOVA, a system that speeds up and saves energy when computers look up facts from big databases while answering questions. Instead of processing data outside the memory, D-NOVA searches directly inside the special NAND memory using a new method called DTS to measure similarity. They also created a small tool that adjusts data to work well with this new method, keeping the accuracy high. Compared to regular CPUs and current accelerators, D-NOVA is much faster and more power-efficient, showing a new way to improve fact retrieval for language models.
Retrieval-Augmented Generation (RAG)Large Language Models (LLM)Dense Vector RetrievalIn-storage AcceleratorNAND MemoryInverted File (IVF) RetrievalDual-Bound Tight Similarity Sensing (DTS)Contrastive AdapterThroughputEnergy Efficiency
Authors
Chang Eun Song, Sumukh Pinge, Tianqi Zhang, Sung Eun Kim, Tajana S. Rosing, Mingu Kang
Abstract
Retrieval-Augmented Generation (RAG) enhances the factual grounding of large language model (LLM) inference by retrieving relevant information from external knowledge bases. However, its dense vector retrieval introduces significant latency and energy overhead, becoming the primary performance bottleneck. Although recent in-storage accelerators aim to reduce data movement, they still rely on host or embedded processors outside the memory, where nearly 70% of the total retrieval time is spent. As a result, they cannot fully overcome the bandwidth limitations, leading to yet another memory bottleneck. To tackle these limitations, we present D-NOVA, a hardware-software co-designed in-storage retrieval accelerator. D-NOVA executes an inverted file (IVF)-based hierarchical retrieval pipeline by deeply embedding the search functionality directly into the NAND memory array. This is achieved by incorporating a new distance metric, Dual-Bound Tight Similarity Sensing (DTS), which is specifically tailored for searching within the NAND string. In addition, we introduce a lightweight contrastive adapter that maps embedding vectors into a DTS-friendly domain, recovering near-software recall while improving performance and energy efficiency. D-NOVA is up to 41.7x faster and 71x more energy-efficient than a CPU baseline, and achieves 12.13x higher throughput while being up to 1.26x more energy-efficient than state-of-the-art in-storage RAG accelerators, demonstrating the potential of fully in-storage vector search for scalable RAG acceleration.