Model predicts best fixes for improving scientific ideas with ai
PRIME: Predictive Routing of Interventions by Modeling Effects for Scientific Ideation
Artificial Intelligence
Summary
Improving ideas generated by AI can be tricky because fixing one part might require changes in another, related part. The authors found this problem, which they call diagnosis-intervention mismatch, and created a system called PRIME to solve it. PRIME looks at feedback on scientific ideas, imagines possible ways to fix the issues, and chooses the best fix before applying it. This approach improves the quality of AI-generated scientific ideas and makes the fixing process more reliable.
What this means in practice
- •For ai product teams: Enhance scientific idea generation tools by guiding revisions through feedback-aware predictive modeling to produce better improvements.
- •For technical writers: Use predictive routing to select targeted revisions that improve drafts with complex interdependent parts based on reviewer feedback.
Authors
Fengxian Ji, Xie Yu, Zhexuan Cui, Yuke Li, Zirui Song, Fan Zhang, Qian Jiang, Jingpu Yang, Xiuying Chen, Preslav Nakov, Zhuohan Xie
Abstract
Feedback-driven iteration is widely used to improve LLM-generated scientific ideas, but existing systems either leave revision decisions implicit in generation, align revisions with feedback-localized weaknesses, or restrict action selection to system-specific operations. This is problematic because problems, hypotheses, and solutions are interdependent: the location where a weakness manifests does not itself determine the revision scope that will produce a better successor. We term this phenomenon diagnosis-intervention mismatch. We introduce Predictive Routing of Interventions by Modeling Effects (PRIME), a plug-and-play feedback-to-revision decision layer for existing scientific ideators. PRIME operates in three stages: (1) Structural Attribution interprets feedback over the problem-hypothesis-solution structure without fixing a revision target; (2) Counterfactual Routing constructs alternative scope-conditioned interventions and compares their projected successor utility and risks through lightweight rollouts; and (3) Intervention Execution compiles the selected intervention into an ordered generator-facing plan with semantic actions, preservation constraints, and dependency requirements for successor generation. We validate diagnosis intervention mismatch on 112 human-annotated idea-feedback pairs. Across four benchmarks, two ideators, and fixed- and open-problem settings, PRIME improves successor quality and routing reliability over strong baselines; removing counterfactual routing reduces quality from 3.463 to 1.960 and increases route regret from 0.426 to 1.929, confirming cross-scope effect modeling as the primary source of its gains.