Language models learn to generate scientific ideas with greater impact
Learning to Ideate for Scientific Impact
Artificial IntelligenceComputation and Language
Summary
Scientific ideas are hard to judge right away using simple qualities like newness or clarity. The authors studied whether the long-term impact of ideas, as measured by future citations of scientific papers, can guide models to generate better ideas. They trained a system to predict a paper’s future impact from its goals and idea description, then fine-tuned the model to create ideas expected to have higher impact. Their experiments show that this approach helps produce more influential scientific suggestions compared to previous methods.
What this means in practice
- •For ai research teams: Develop ideation tools that generate research ideas expected to yield higher scholarly impact using long-term citation feedback.
- •For innovation strategy teams: Use language models aligned on scientific impact to explore promising research directions and prioritize project ideas.
Authors
Shubham Kale, Aniketh Garikaparthi, Manasi Patwardhan
Abstract
Scientific ideation is increasingly mediated by large language models, but current ideation systems are usually trained and evaluated on immediately judgeable proxies such as novelty, clarity, and feasibility. This leaves open whether delayed signals of scientific uptake can be used as feedback for steering models toward research directions with higher expected \emph{impact}. We study this question using citation-normalized impact as a noisy but scalable proxy for scholarly uptake. We construct a large-scale dataset from over 100K computer science papers by extracting goal-conditioned idea descriptions and assigning each paper an ordinal, year-normalized citation label. We then train a goal-conditioned reward model to predict citation-impact labels from research goal and idea pairs, and use this reward to align an idea generator through supervised fine-tuning followed by reinforcement learning. To reduce circularity, we evaluate generated ideas with a held-out, reference-grounded protocol that compares model outputs against historical ideas under the same research goal and weights judgments by the reference idea's citation-impact label. Experiments show that our RL-tuned model consistently produces ideas with higher estimated impact than both the base model and supervised fine-tuning baselines. Our findings position scientific impact as a practical, outcome-grounded feedback signal for aligning LLMs in open-ended scientific discovery.