Small models improve AI learning with selective large model help

SCALE: Small-Model-First Concept Abstraction with LLM-Guided Experience Internalization

Artificial IntelligenceRobotics

Summary

AI programs often rely on large language models (LLMs) that are powerful but slow and expensive for everyday tasks. This paper presents SCALE, a method that lets smaller, faster models handle simple decisions while using the larger models only when mistakes happen or tricky situations arise. The large models help by looking back at past experiences and suggesting new ideas, which are then tested and taught to the smaller models. This approach makes AI systems more efficient by storing important knowledge in a compact form and reducing unnecessary thinking.

What this means in practice

  • For robotics engineers: Improve robot decision-making by enabling small onboard models to learn abstract concepts, reducing reliance on constant cloud-based reasoning.
  • For customer service platform developers: Create chatbots that use small models for common queries and escalate only complex issues to large language models, optimizing response speed and cost.$Commercial implications: Allows development of scalable, cost-efficient chatbot services using selective large-model invocation, enhancing commercial customer support platforms.

Authors

Hong Su

Abstract

Autonomous agents must continually acquire reusable knowledge from experience without relying on costly large language model (LLM) reasoning for every decision. However, existing experience-based approaches often struggle to transform accumulated observations and reasoning into compact knowledge that lightweight models can independently apply. This paper proposes SCALE, a small-model-first framework that integrates selective LLM reasoning, experience-driven concept abstraction, and knowledge internalization. A lightweight model handles routine predictions, while prediction errors and accumulated cross-episode evidence selectively trigger LLM reflection. The LLM analyzes relevant historical experiences, generates positive and negative counterfactual examples, and proposes reusable concept hypotheses. Independently validated concepts are progressively incorporated into hierarchical neural representations, enabling lightweight models to exploit increasingly abstract knowledge. Meanwhile, mastered experiences are excluded from active memory while their original records remain available in a complete archive. Experiments across ten random seeds show that historical recall improves held-out accuracy from 87.11% to 98.28%, hierarchical concept knowledge increases difficult-case accuracy from 55.33% to 66.47%, and abstract archiving reduces active experience records from 6,000 to 1,528.