Bidirectional reasoning model improves step-level logic checks in math problems
LLaDA-PRM: A Bidirectional Step-Level Reasoning Evaluator
Computation and LanguageArtificial Intelligence
Summary
Checking each step in a math solution is easier when the checking model can look forward and backward through all the steps, not just from the start up to the current step. The authors tested this idea and showed that models that use this bidirectional approach find more errors than traditional models looking only one way. They created a new evaluator called LLaDA-PRM that scores higher in spotting mistakes on math problem steps than bigger models built only for one-way checking. This new model can also help select better training examples for other AI models solving math problems.
What this means in practice
- •For ai model developers: Improve math solving models by using LLaDA-PRM to select better training data based on detailed reasoning evaluations.
- •For automated reasoning testers: Use bidirectional evaluators like LLaDA-PRM to more accurately detect errors in step-by-step reasoning for math problem solvers.
Authors
Yiming Feng, Naihao Deng, Yulong Chen, Rada Mihalcea
Abstract
Step-level reasoning evaluators are commonly based on autoregressive language models, whose causal attention restricts each step representation to the problem, previous steps, and the current step. Yet, when the complete solution is available, the validity of an earlier step may become clearer only through its downstream consequences. We validate this hypothesis through a controlled 54-run comparison of causal and bidirectional LLaDA evaluators at 1B--3B scale, changing only the self-attention mask, and find bidirectional attention yields consistent improvements. Building on this finding, we introduce \prm{}, an 8B bidirectional evaluator that reaches 88.8 step-level F1 on MR-MATH-invalid and 83.8 on the out-of-distribution MR-GSM8K original-question subset, outperforming ReasonEval-Llemma-34B by 11.3 and 10.3 F1 points, respectively. \prm{} also remains effective when evaluating incomplete reasoning traces in online settings, outperforming the strongest baselines on both benchmarks by a large margin. We further show that \prm{} provides an effective training-data selection signal, improving Mistral-7B performance on MATH-500.