Self-hosted ai framework boosts research literature understanding accuracy
MuninAI: A Self-Hosted Agentic Framework for Independent Research Groups
Computation and Language
Summary
Research groups often struggle to use language models that understand their unpublished papers without sharing these documents with external servers. The authors created Munin, a tool that runs locally and helps language models read, search, and analyze a group's own research papers and citations. Their experiments show that adding Munin's smart tools more than doubles the accuracy of language models answering questions about the group’s literature. The system can avoid guessing answers when relevant documents are missing and produces no fake references. This makes it reliable for research groups wanting privacy and accuracy.
What this means in practice
- •For academic research groups: Set up a local AI assistant that accurately answers questions about in-house unpublished research without exposing data online.
- •For enterprise research teams: Deploy private AI tools to analyze proprietary documents with improved accuracy and reliable citation tracking on consumer-grade hardware.
Authors
Markus Fischer, Jens Meiler, Matthias Elgeti
Abstract
Research groups that want a language model grounded in their own literature face a choice between hosted services that receive their unpublished work and self-hosted deployments that keep the data local but give the model no access to the corpus. Munin is an open-source framework for a self-hosted scientific assistant that indexes a group's corpus and its citation graph, and wraps whichever open-weight model the group can serve in an agent harness with specialised reading, search, and computation tools. We claim that the harness, rather than the specific model, is what makes the system useful on domain literature, and we ship the evaluation suite that tests this claim with the software. On 199 paired literature questions, the harness more than doubles the accuracy of the base model, from 0.387 to 0.874 (+0.487, 95% CI [0.407, 0.568]), while naive retrieve-then-answer scores below the unaided model on every backbone tested. Under the harness, roughly twice as many answer claims trace to a passage the model read itself as with fixed top-5 retrieval. When the source paper for a question is removed from the corpus, the system abstains on about nine in ten of the questions it had answered correctly on the Qwen backbones and eight in ten on gpt-oss-20b, and it produces no fabricated local citation across 300 fabricated references. The pattern holds on three backbones from two developers, including a 20-billion-parameter model that runs on a single consumer-grade GPU. With all outbound network access disabled, both Qwen backbones still exceed the published PaperQA2 accuracy. The evaluation suite runs as a single command, so an adopting group can re-run it on its own corpus and model.