VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models
2026-06-15 • Artificial Intelligence
Artificial IntelligenceComputation and Language
AI summaryⓘ
The authors created VibeThinker-3B, a small but powerful AI model with 3 billion parameters, to see how well complex reasoning can be done with fewer resources. They improved it using special training steps like fine-tuning, reinforcement learning, and self-teaching. Their tests show it performs as well as much larger models on tough reasoning tasks and maintains clear control over instructions. They suggest that tricky reasoning can be packed into smaller models, while bigger models are still needed for broad knowledge. This work shows that small models can offer a different way to reach top performance.
Dense modelParametersPost-trainingFine-tuningReinforcement learningSelf-distillationVerifiable reasoningOut-of-distribution generalizationInstruction controllabilityParametric Compression-Coverage Hypothesis
Authors
Sen Xu, Shixi Liu, Wei Wang, Jixin Min, Yingwei Dai, Zhibin Yin, Yirong Chen, Xin Zhou, Junlin Zhang
Abstract
This technical report introduces VibeThinker-3B, a compact dense model with 3B parameters developed to investigate how far verifiable reasoning can be pushed within a strictly small-model regime. Building upon the Spectrum-to-Signal post-training paradigm, we systematically enhance the model through an optimized pipeline that includes curriculum-based supervised fine-tuning, multi-domain reinforcement learning, and offline self-distillation. Experimental evaluations demonstrate that VibeThinker-3B achieves frontier-level performance on highly demanding verifiable tasks. Specifically, it attains a score of 94.3 on AIME26 (improving to 97.1 with claim-level test-time scaling), an 80.2 Pass@1 on LiveCodeBench v6, and exhibits strong out-of-distribution generalization with a 96.1\% acceptance rate on recent unseen LeetCode contests. This effectively places it in the performance band of first-tier reasoning systems, matching or exceeding flagship models that are orders of magnitude larger, such as DeepSeek V3.2, GLM-5, and Gemini 3 Pro. Furthermore, a score of 93.4 on IFEval confirms that this extreme reasoning enhancement does not compromise strict instruction controllability. Extending our previous 1.5B work, these findings motivate the Parametric Compression-Coverage Hypothesis, which views verifiable reasoning as compressible into compact reasoning cores, while open-domain knowledge and general-purpose competence require broad parameter coverage over facts, concepts, and long-tail scenarios. This perspective suggests that compact models are not merely deployment-efficient substitutes, but a complementary path toward frontier-level performance in parameter-dense capability regimes.