SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification
2026-06-03 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors explore using Process Reward Models (PRMs) to improve scientific problem solving in areas like biology and chemistry, where accuracy and proper tool use are crucial. They created a big dataset called SCIPRM70K that shows step-by-step reasoning combined with tool usage. Then, they developed Sci-PRM, a model that helps guide tool choice and checks the accuracy of each step. Their experiments show this model helps improve results during testing and also boosts learning when used with Reinforcement Learning methods.
Process Reward ModelsScientific reasoningChain-of-ToolReinforcement LearningBest-of-N selectionReward signalSci-PRMModel hallucinationDataset SCIPRM70KTool execution accuracy
Authors
Xiangyu Zhao, Hengyuan Zhao, Yiheng Wang, Wanghan Xu, Yuhao Zhou, Qinglong Cao, Zhiwang Zhou, Lei Bai, Wenlong Zhang, Xiao-Ming Wu
Abstract
While Process Reward Models (PRMs) have achieved remarkable success in mathematical reasoning, their application in complex scientific domains-such as biology, chemistry, and physics remains largely unexplored. Scientific problems demand not only logical rigor but also factual consistency and the precise usage of domain-specific tools, areas where current models often suffer from hallucinations and lack of verification. In this paper, we first construct SCIPRM70K, a large-scale dataset featuring Chain-of-Tool trajectories that explicitly interleave reasoning with the execution of scientific tools. Building upon this, we train an efficient reward model called Sci-PRM to provide fine-grained supervision on tool selection, execution accuracy, and result interpretation at each step in one inference. Experiments demonstrate that Sci-PRM significantly enhances foundation models in two key aspects: (1) it enables effective test-time scaling via Best-of-N selection; and (2) when integrated into Reinforcement Learning, it serves as a dense reward signal that mitigates the critical issue of advantage disappearance, allowing the model to break through existing performance ceilings.