AI summaryⓘ
The authors studied very tiny (2-bit) versions of large reasoning models and found that while these versions save some computing power per step, they often take a lot more steps to give an answer, sometimes repeating themselves or getting stuck. This makes the overall process slower and less accurate. To fix this, the authors introduced two simple tricks: one where the model first plans in higher precision (FP16) before using the tiny 2-bit mode, and another that catches repeated loops and either finalizes an answer early or switches back to higher precision. These methods brought big improvements in accuracy and kept the speed benefits. Their work shows that very low-bit reasoning models can work well if these common problems are caught and controlled.
Large Reasoning Models2-bit QuantizationInference EfficiencyReasoning TracesFP16 PrecisionLoop DetectionModel AccuracyToken GenerationQwen3 ModelLow-bit Inference
Authors
Ekaterina Alimaskina, Darya Rudas, Denis Shveykin, Gleb Molodtsov, Pavel Vasiliev, Aleksandr Beznosikov
Abstract
Large Reasoning Models (LRMs) rely on long reasoning traces, making inference expensive. While low-bit quantization reduces per-token decoding cost, we show that aggressive 2-bit inference can fail to deliver end-to-end speedup because instability in the generation process inflates total token count. Instead of merely lowering answer accuracy, 2-bit quantization often produces much longer traces with repetitive loops, budget exhaustion, delayed commitment, and unclosed reasoning segments. We analyze full reasoning traces of Qwen3 reasoning models across mathematical and commonsense benchmarks and show that accuracy degradation is tightly linked to these process-level failures. To address them, we introduce two lightweight controls: FP16 planning, which gives the 2-bit model a short high-precision outline, and loop rescue, which detects repetitive traces and either commits to an earlier answer or falls back to FP16. On MATH-500, loop rescue improves Qwen3-8B accuracy from 17.2% to 74.2%, while planning plus loop rescue improves Qwen3-32B from 65.0% to 87.2%. Overall, our results show that extreme low-bit reasoning becomes practical when its failures are treated as controllable generation pathologies: with lightweight detection and selective FP16 support, 2-bit inference can recover accuracy while preserving real end-to-end speed. Our code is available at: https://github.com/brain-lab-research/quantized-reasoning.