Source learning improves language model understanding of persistent knowledge
From Knowledge Access to Source Learning: Developing Source-Specific Competence
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Many AI systems use big language models that look up information from trusted sources to answer questions or complete tasks. Usually, when these systems revisit the same source, they just treat it as repeated looking up instead of learning more about it over time. The paper introduces a way for AI to build up a deeper, reusable understanding of a specific source by adapting what it knows and how it applies that knowledge. This method, called SourceLearn, uses two ways to learn—by exploring what it doesn’t fully understand and by using task results to fix gaps. This approach leads to much better performance compared to other methods.
What this means in practice
- •For enterprise chatbot developers: Enhance chatbot accuracy by enabling it to build lasting understanding from company documents it repeatedly accesses.
- •For digital library managers: Improve AI-assisted content discovery tools by allowing progressive learning from the same authoritative digital sources over time.
Authors
Lucheng Fu, Kejing Xia, Yiyang Wang, Yiqiao Jin, Jinjin He, Xiyuan Yang, Haoxin Liu, Ye Yu, Haibo Jin, Yijia Xiao, Wenke Lee, B. Aditya Prakash, Haohan Wang
Abstract
Large language model (LLM) agents increasingly rely on persistent external sources to solve sequences of knowledge-intensive tasks. Existing methods improve how source content is accessed and organized, while agent-memory systems preserve reusable knowledge from prior interactions, but repeated use of the same source is still largely treated as repeated access rather than an opportunity to progressively improve understanding of that source. We study source learning: developing reusable source-specific competence over a persistent authoritative source. We represent this competence with a persistent source model that captures reusable understanding of the source, including how its knowledge is structured, interpreted, and applied. To construct and progressively refine such models, we propose SourceLearn, which combines two complementary learning mechanisms. Self-Directed Source Learning identifies what remains incompletely understood and adaptively revisits the source, while Task-Guided Source Learning uses downstream experience to reveal local representational gaps and recurring needs in how source knowledge should be organized. In both cases, learning signals determine what should be reconsidered, while persistent updates are reconstructed from the authoritative source. Across five benchmarks and three LLM backends, SourceLearn achieves the best performance in 13 of 15 settings, with gains of up to 22.6 points over Hybrid RAG and substantial overall improvements over static source representations and experience-based memory baselines.