Self repulsion sampling improves language model answer diversity and accuracy
Self-Repulsive Sampling for Diffusion Language Models
Machine Learning
Summary
When asking language models to answer questions multiple times, repeated answers reduce the benefit of getting more samples. The authors introduce a method called Self-Repulsion (SR) that makes multiple answer attempts more different from one another by discouraging repeated tokens during generation. This approach improves the chance of finding the correct answer by having more diverse possibilities without needing extra training or computation. In tests on math and truthfulness benchmarks, SR increased the accuracy of combined answers noticeably.
What this means in practice
- •For natural language developers: Improve the accuracy of AI assistants by generating more diverse answers that capture a wider range of correct responses.
- •For automated tutoring platforms: Use diverse answer generation to provide better solution coverage and explanations for complex problem-solving tasks in educational AI tools.
Authors
Michael Helcig, Martin Jaggi
Abstract
Sampling several responses and voting over their answers can improve a language model's accuracy, but repeated answers limit the benefit of additional samples. Raising temperature increases diversity at a potential cost to per-sample accuracy. We introduce Self-Repulsion (SR), a sampler for masked diffusion language models that uses peer commitments to diversify the pool. At each penalized denoising step, each path lowers a token's logit according to how many peers have committed that token at the same position. Paths share a batched forward pass and then commit in sequence, so later paths observe choices made earlier in the same step. This coupling requires no training or additional forward or backward pass and can produce distinct paths even at temperature zero. When all paths commit a position together from identical logits, the update exactly maximizes total logit minus a convex duplication cost. On LLaDA-8B-Instruct with ten paths and 128 denoising steps, deterministic SR reaches 80.38% plurality accuracy on GSM8K, compared with 70.17% for the unpenalized greedy decoder. At temperature 0.6 and matched model-evaluation budgets, the count penalty improves over self-consistency by 2.06 percentage points in blocks of 32 and 14.50 under pure diffusion. Experiments on GSM8K, MATH and TruthfulQA show that voting gains arise mainly from higher coverage of correct answers, with gains that vary by benchmark and decoding regime.