COGTRL: Training LLMs for Scientific Discovery Assistance using Cognitive Traces via Reinforcement Learning
2026-08-31 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how large language models (LLMs) help with scientific discovery and found that teaching these models to show their detailed thinking steps—like considering mistakes and constraints—makes them better assistants. They introduced COGTRL, a training method that encourages LLMs to think more like scientists by combining reasoning steps with actual scientific work. Their approach improved performance on two scientific fields using smaller models and even competed with much bigger ones. Experts also preferred the results from their method over standard approaches.
Large Language ModelsScientific DiscoveryCognitive TracesReinforcement LearningTrajectory-level OptimizationAI ResearchMaterials ScienceModel TrainingInterleaved ReasoningModel Evaluation
Authors
Shrinidhi Kumbhar Santosh Mashetty Divij Handa Kevin Coutinho, Siddharth Sambhaji Ghule, Chitta Baral
Abstract
Large Language Models (LLMs) trained on extensive scientific research are increasingly integrated as assistants for scientific discovery. However, most research papers omit the fine-grained cognitive process of examining constraints, failed alternatives, and iterative decisions required to achieve the desired goal. Such cognitive processes are vital for real-world scientists working toward specific goals under constraints. In this paper, we show that LLMs, when trained to produce such cognitive traces, perform better as scientific discovery assistants than when trained solely on scientific literature. We propose COGTRL, a trajectory-level reinforcement learning framework that trains LLMs to emulate cognitively grounded reasoning by jointly optimizing cognitive traces and the scientific steps produced in an interleaved manner. Across two 3B-parameter models and two scientific domains (AI and Materials Science), COGTRL improves method quality by an average of 7.85 points over comparable 3B model baselines and achieves competitive performance relative to 70B parameter models. Moreover, analysis by domain experts shows a preference for methods generated by COGTRL over the baselines.