Local models answer scientific code questions using retrieval store
Retrieval-Augmented Generation for Scientific Code Understanding
Software EngineeringArtificial Intelligence
Summary
Large and powerful language models used for coding help are very costly and raise privacy concerns since they run on the cloud. The authors explore building a small, fully local assistant for understanding scientific code by doing heavy processing upfront and storing detailed information for quick answers later. They tested their approach on code in the C++ language and found that smaller models with good retrieval data performed better than bigger ones. This method allows local models to give precise answers related to a specific codebase while keeping the code private.
What this means in practice
- •For in-house software developers: Provide a private coding assistant that answers questions about proprietary scientific code without sending data to the cloud.
- •For software maintenance teams: Use small local models with pretrained code knowledge stores to quickly resolve questions about legacy scientific C++ code.
Authors
Aaron Nobile, Andreas Adelmann, Mohsen Sadr
Abstract
Large language models have become central to modern coding assistants, but state-of-the-art systems such as Claude Code or Codex rely on very large, cloud-hosted models with significant computational cost and data-privacy implications. This work investigates whether a useful, fully local coding agent can be built around small open-source models by shifting the computational burden away from inference. We develop a Retrieval-Augmented Generation (RAG) system for scientific code understanding that strictly separates an expensive offline ingestion stage parsing, structural graph construction, LLM-generated entity explanations, and embedding from a lightweight online answering stage. The system is evaluated on a 100-question benchmark spanning eleven categories over the IPPL scientific codebase written in C++, with answers scored by an independent frontier model as the judge. Across seven answering models, we find that model family and retrieval quality matter more than parameter count, i.e. a 9B model achieves the highest average score (0.795), outperforming both larger models within our pipeline and the same models embedded in the Claude Code retrieval architecture. The results indicate that front-loading code understanding into a reusable, codebase-specialised vector store enables small local models to deliver grounded and repository-specific answers, making the agent well suited as a privacy-preserving development tool for in-house scientific codebases.