Modeling how research ideas grow through detailed step tracking
Modeling Authentic Research Trajectories: A Graph View of Research Ideation
Artificial Intelligence
Summary
Research papers often present ideas that have changed a lot from when they first began. The authors show that the usual way of describing research misses all the small steps and reasons behind those changes. They suggest a new model called the Research Progress Graph that records each step in how an idea evolves, including the evidence and thinking behind it. This helps us see research as a process, not just a final product, which could improve how AI research assistants work.
What this means in practice
- •For research software developers: Design research assistants that model step-by-step idea changes instead of generating single final ideas.
- •For science policy analysts: Evaluate and train research tools based on how authentically they capture real research progress processes.
A position paper. It proposes an approach and reports no results.
Authors
Ruochen Li, Zhiyu Chen
Abstract
The research idea in a published paper is rarely the project's initial idea. Between the two lies a closed-loop process in which researchers run experiments, derive insights, identify challenges, explore the literature, and revise the idea, with each step motivated by evidence and reasoning they can articulate. We call the path one idea takes through this process, with the evidence and reasoning behind each step, its authentic research trajectory. Although it underlies every scientific contribution, it is rarely recorded. Existing LLM-based research agents generate an idea once, refine it against reviews, literature, or their own experimental results, or search over candidate ideas and experiments. In all of them the text of the idea evolves, while the evidence and reasoning that connected one version to the next are discarded. We propose the Research Progress Graph (RPG), a representation of authentic and complete research trajectories. An RPG records the evolution of an idea as typed research states connected by cognitive transitions, each annotated with a temporal index and the reasoning behind it, which makes this evolution an explicit object of study. We ground the representation in authentic trajectories whose steps their authors justify in annotations and papers. Comparing the representations of thirty-five research agents with the RPG, we find that each is an incomplete RPG; none keeps the state that changed, the evidence that changed it, and the reasoning connecting them as one evolving record. Modeling ideation with the RPG changes how research agents should be designed, evaluated, and trained. Agents predict the next research state instead of a complete idea, ideas are evaluated as trajectories instead of final texts, and authentic trajectories serve as training data. These directions shift research agents from generating static ideas to modeling how ideas evolve.