Research agents improve retrieval augmented generation with open-weight models
RMIT-ADM+S at TREC RAG 2026: Research Agents, Retriever Ablations, and Open-Weight Replication
Information RetrievalArtificial IntelligenceComputation and Language
Summary
Finding the best way to get answers from lots of information is tricky. The RMIT-ADM+S team tested different tools to see what helps make answers better. They found that choosing the right writing model and how they group information chunks are key. Their mix of open and proprietary tools often did better than average on accuracy tests. Interestingly, what machines liked and what people preferred sometimes didn’t match, showing that cheaper models can still work well.
What this means in practice
- •For chatbot developers: Improve chatbot answer quality by selecting effective writer models and chunking strategies based on retrieval methods.
- •For enterprise search teams: Deploy lower-cost open-weight models combined with retrieval workflows to enhance document-answering systems without relying solely on proprietary APIs.
Authors
Oleg Zendel, Kun Ran, Chenglong Ma, Nguyen Khoi Vo, Shuai Wang
Abstract
We describe the RMIT-ADM+S team's participation in the TREC 2026 Retrieval-Augmented Generation (RAG) track using the ClimbMix corpus. We submitted seven research-agent runs using proprietary OpenAI and open-weight models, exploring different retrieval strategies, writer models, and agent workflows. Pre-submission ablations identified writer selection and adjacent-chunk expansion as the main factors affecting answer quality. In the official NIST evaluation, six of seven runs exceeded the track median across all three rubric/nugget collections. Our best-performing run, brief-hybrid, achieved the highest all-item coverage among our submissions and tied the track maximum (0.892) for Batch-1 vital coverage. Human side-by-side preferences differed across evaluation batches: the open-weight owa-glm5 led Batch 1 despite lower automated coverage, while sol-research-first led Batch 2. These findings highlight differences between automated coverage measures and human preferences, as well as the potential of lower-cost open-weight models for research-agent systems.