Librarian agents organize documents for better long context understanding
Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries
Machine LearningArtificial IntelligenceComputation and Language
Summary
Large language models have trouble working with very long documents because it’s hard to find and connect relevant information quickly. The authors introduce librarian agents that act like smart helpers who organize documents into easy-to-use folders and summaries before questions are asked. These agents learn how to create workspaces that future question-answering agents can use more efficiently, improving accuracy. Tests show their method beats traditional retrieval approaches and adapts well to new topics without extra training.
What this means in practice
- •For customer support teams: Provide quick and accurate answers by preorganizing large product document sets into tailored files before customer queries arrive.
- •For legal teams: Enable efficient review and cross-referencing of extensive legal documents by building curated, hierarchical document workspaces ahead of case analysis.
- •For documentation platform developers: Integrate proactive document organization tools into platforms to improve response speed and accuracy for complex user questions on large manuals and guides.$Commercial implications: Allows building commercial AI-enhanced document search products that outperform standard retrieval by anticipating user needs.
Authors
Anikait Singh, Teresa Zhang, Yoonho Lee, Roshen Sanjay Nair, Aviral Kumar, Chelsea Finn
Abstract
Reasoning over long document contexts remains a challenge for large language models (LLMs). Standard retrieval methods are inherently reactive, bottlenecking agents with complex, cross-document reasoning at inference time, while precomputed structures like knowledge graphs often fall short because they rely on rigid heuristics. Instead, can we allow agents to perform anticipatory directed curation: learning to proactively reorganize unstructured corpora prior to inference to efficiently serve diverse future queries. We build on a key asymmetry in modern agentic harnesses (e.g., Claude Code): while LLM agents are adept at navigating existing filesystems to solve an immediate task, they are rarely trained to construct reusable workspaces for future use. To instantiate this capability, we introduce Librarian Agents, a framework in which a builder agent transforms raw documents into a hierarchical filesystem of synthetic files and reference links, which forms a workspace tailored for a downstream answering agent. We formulate workspace construction as a Markov decision process and optimize the builder policy via reinforcement learning to directly maximize downstream answering accuracy. Empirically, Librarian Agents outperforms the strongest retrieval and precomputation baselines by 33 percentage points on BrowseComp-Plus, amortizes upfront construction costs across repeated queries, and generalizes zero-shot to unseen domains on BRIGHT with a 31-point average accuracy improvement.