Compact controllers learn to replace expert AI helpers in reasoning tasks
From Collaboration to Capability: Internalizing Routed LLM Experts into Compact Reasoners
Artificial Intelligence
Summary
Sometimes, small AI programs control bigger expert systems by choosing who to ask for help and combining their answers. The authors studied if these small controllers could learn to do the experts' work themselves after watching how the experts reason and code. They built a system called RIVET that learns from recorded successful expert collaborations to improve its own reasoning and coding abilities. This lets the small controller operate on its own without needing the bigger expert AIs. Their experiments show better performance on math and science reasoning tasks after removing the experts.
What this means in practice
- •For ai system developers: Build smaller AI agents that can internalize and replicate the reasoning of larger expert models for more efficient deployment.
- •For automated math software teams: Improve the accuracy of automated reasoning in competition-level mathematics by using controllers trained on verified expert collaborations.
Authors
Frank Nie, Shuyao Wang, Ethan B. Liu
Abstract
A compact controller can coordinate stronger experts by selecting whom to consult, formulating requests, and integrating their responses. We study whether learning from both the controller's decisions and the experts' reasoning and code improves its generation after expert removal. We introduce \textsc{Rivet} for \emph{collaboration internalization}: expert-augmented reinforcement learning applies a shared outcome signal to controller decisions and returned expert spans, and verified trajectory internalization consolidates complete successful interactions through format-aware supervised training. The deployed controller generates reasoning, code, and interaction structure with local Python execution and no external LLM. Across seven competition-mathematics benchmarks, RIVET-1.7B and RIVET-4B achieve average accuracies of $28.25\%$ and $44.16\%$; Stage~II improves RIVET-4B's accuracy after expert removal by $6.49$ points, and GPQA-Diamond results provide evidence of generalization to scientific reasoning. Ablations show gains from ordinary trajectory supervision and additional format weighting, supporting the effectiveness of training on the content and structure of verified collaborations.