Papers for

ai-powered product developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

How retrieved literature shapes ideas generated by large language models

Tracing the Influence of Retrieved Literature on Scientific Ideation in LLMs

Abstract: Large language models (LLMs) are increasingly used to generate research ideas from scientific literature. Existing systems combine literature retrieval with search, generation, and refinement, making it difficult to determine how the retrieval strategy itself shapes the ideas generated. We isolate this contribution by comparing five retrieval strategies and a No Literature baseline under a shared generation protocol, using 45,000 ideas across 15 machine learning topics and ten open-weight models. Relative to no literature basline, all five retrieval strategies reduce diversity among ideas generated from the same set of papers by $5.6\%$--$17.9\%$, while most increase diversity by $10.2\%$--$18.1\%$ when ideas from different sets are pooled. Effects on idea quality and novelty are modest on average but vary across topics. We further apply a JSD-based attribution method to trace the influence of individual input papers on each idea's motivation, core idea, approach, and validation. In $79.3\%$ of ideas, only one or two distinct papers receive the highest attribution scores across these four parts. These leading sources often recur across ideas generated from the same set of papers, and groups with more varied source attribution contain more diverse ideas ($r=0.70$). These findings motivate various sets of literature and the use of attribution to guide the choice of sources for subsequent ideas.

Wed 28 OctComputation and Language
The gist
Generating new research ideas using large language models often involves using retrieved scientific papers. The authors studied how different ways of choosing these papers affect the variety and quality of ideas the models come up with. They found that using retrieved papers generally changes the diversity of ideas compared to using no papers, and that a small number of key papers tend to influence most parts of each idea. This helps understand how the choice of literature impacts the model’s creativity and could guide better strategies for finding inspiring sources.
Open → 2610.39060v1