Diffusion-GR2: Diffusion Generative Reasoning Re-ranker
2026-07-01 • Information Retrieval
Information RetrievalArtificial Intelligence
AI summaryⓘ
The authors study how to make recommendation models that explain their reasoning faster. Traditional autoregressive (AR) models produce reasoning step-by-step, which is accurate but slow. They propose Diffusion-GR2, which changes the model to do many steps in parallel (using block-diffusion) and applies new training methods to keep accuracy high. Experiments show their approach keeps almost the same accuracy as AR while being 2.4 to 3.5 times faster. Their methods fix problems caused by parallel decoding that would normally reduce model quality.
Generative reasoningAutoregressive decoderBlock-diffusion modelsRe-rankingPermutation decodingConversion fine-tuningOn-policy distillationReinforcement learningParallel decodingRecommendation systems
Authors
Zhuoxuan Zhang, Kangqi Ni, Yuhang Chen, Mingfu Liang, Xiaohan Wei, Yunchen Pu, Fei Tian, Chonglin Sun, Frank Shyu, Adam, Song, Sandeep Pandey, Luke Simon, Tianlong Chen, Xi Liu
Abstract
Generative reasoning re-rankers achieve strong recommendation accuracy by emitting a chain-of-thought before re-ordering a candidate list, but they are slow at inference: an autoregressive (AR) decoder spends one sequential forward pass per reasoning token, and the reasoning trace far exceeds the ranking it produces. To reduce this cost, block-diffusion language models decode many positions in parallel over a few denoising steps and are substantially faster, yet naively converting an AR re-ranker into one opens two accuracy gaps: (1) a structural gap: answer positions are denoised in parallel and scored independently, so the decoder emits invalid rankings (duplicated, dropped, or out-of-set identifiers) that AR avoids through left-to-right masking; and (2) a distributional gap: fine-tuning the converted model on fixed teacher trajectories is off-policy relative to its own decoding at inference, leaving a residual accuracy gap. To close both gaps while keeping the speedup, we propose \textbf{Diffusion-GR2}, a recipe that converts our AR reasoning re-ranker (GR2) into a block-diffusion re-ranker. First, conversion fine-tuning (CFT) adapts the AR-initialized diffusion model to denoise the answer into a valid permutation on its own, without an external constrained decoder. Next, on-policy distillation (OPD) then supervises the model on its own decoded trajectories with dense per-token targets from the AR teacher. Finally, we apply a reinforcement-learning (RL) stage against a re-ranking reward on top of OPD's on-policy policy. Experiments on Amazon Beauty demonstrate that Diffusion-GR2 recovers to near-parity with the AR re-ranker, while block-parallel decoding raises decode throughput by $2.4$--$3.5\times$ at the model's reasoning output length. Ablations show that CFT recovers most of the conversion gap, and that on-policy distillation further closes it to the AR reference.