Small language models team with large models for long tasks
Sibyl: An Efficient Small-large Model Collaboration Framework for Long-horizon Tasks
Artificial Intelligence
Summary
Small language models are fast and use little power but struggle with complicated tasks that need many steps. The authors developed Sibyl, a system that lets small models ask bigger, smarter models for help only when really needed. This method teaches the small model when to reach out and how to remember the advice, making it very efficient. Tests showed Sibyl works better than previous methods while using the big models much less often.
What this means in practice
- •For mobile app developers: Build AI assistants that run efficiently on devices but consult powerful cloud models sparingly for complex tasks.
- •For cloud service engineers: Optimize cloud compute use by enabling small models to decide precisely when to request larger model help.
Authors
Zhewei Fang, Yuxin Zhang, Zhenwei Shao, Mengze Li, Zheng Lin, Long Chen, Zhou Yu, Zhe Chen, Zhiwen Chen, Zhaode Wang, chengfei lv
Abstract
Small language models (SLMs) offer a promising foundation for on-device agents through low-latency, resource-efficient inference, yet limited reasoning and planning capabilities constrain their performance on long-horizon tasks requiring multi-step interaction with the environment. Step-level collaboration between SLMs and larger cloud-hosted models can bridge this gap, but identifying states that warrant cloud assistance remains challenging: the contribution of each cloud call is entangled with subsequent actions and can be assessed only from the final task outcome. Compounding this challenge, the SLM must balance two competing objectives: maximizing task success and minimizing cloud calls. To address this, we propose Sibyl, an algorithm that trains SLM agents to selectively consult cloud models at the step level and internalize their guidance for subsequent decisions, achieving strong task performance with minimal cloud reliance. Sibyl follows a three-stage training pipeline that (1) builds a robust base policy through consultation-free self-evolving reinforcement learning (RL); (2) cold-starts consultation behavior via decisive-disagreement state mining; and (3) jointly optimizes consultation decisions and guidance internalization through consultation-aware RL. Experiments on ALFWorld and WebShop demonstrate that Sibyl, using only a 0.6B-parameter model, outperforms state-of-the-art baselines, including agent training and routing methods, by 95.2% and 80.4% in success rate while averaging only 0.8 and 3.9 cloud calls per trajectory, respectively.