Skill driven small language models improve robot task handling
Skill-SLM: Agent Skill-driven Small Language Models for Reliable Robot Operation
Robotics
Summary
Small language models can help robots make smart decisions, but current methods need lots of example tasks and don’t adapt well to new jobs. The authors created Skill-SLM, which breaks down robot instructions into smaller tasks and picks the right skills to complete them. This approach helps robots handle a wider variety of tasks more reliably. Tests showed it works well with drones and ground vehicles, especially for new tasks the robots hadn’t seen before.
What this means in practice
- •For robotics engineers: Enable robots to understand diverse complex instructions by breaking tasks into known skills and reliably sequencing them.
- •For drone operators: Improve drones’ ability to perform new missions by using small models that select and combine necessary skills effectively.
Authors
Wenhao Wang, Yanyan Li, Jiawei Yuan
Abstract
Small language models (SLMs) have been increasingly adopted for onboard robot operation because they enable intelligent decision-making. However, existing approaches are mainly distillation-oriented and rely on enumerating representative task-solution pairs. This makes dataset construction difficult and limits generalization to diverse robot tasks whose possible forms grow rapidly. This paper proposes Skill-SLM, a framework that reformulates SLM-driven robot operation as a task-decomposition and skill-composition problem. Given a natural language task instruction, Skill-SLM decomposes the task into subtasks, selects appropriate skills from the skill library, and orchestrates the selected skills into executable robot operations. First, to support the skill-driven workflow, we propose a novel robot operational skill aware context-free grammar (CFG) to extract the skills required to accomplish tasks and build the skill library accordingly. Then, we configure LLM teachers to induce and synthesize training datasets for the SLMs, enabling SLMs to decompose tasks and orchestrate skills reliably. Additionally, we employ a progressive skill orchestration strategy to improve the reliability of skill implementation and overall robot operation. Experiments on UAV operation tasks indicate that Skill-SLM substantially outperforms distillation-oriented baselines, especially on unseen tasks that require generalization of capabilities. Additional experiments on ground vehicle tasks further demonstrate that Skill-SLM can be applied to different robot platforms.