ClusterFewshot improves language model task learning efficiency
ClusterFewshot: Improving Few-shot Optimization for LLMs workflow
Computation and Language
Summary
Large language models need good examples to learn new tasks quickly, but picking these examples randomly or by simple rules misses deeper connections. The authors propose ClusterFewshot, which groups examples by meaning and then picks the most useful ones, making the learning process cheaper and more accurate. They tested this approach and found it works better than older methods in tasks where models adjust or tune their prompts and settings.
What this means in practice
- •For ai engineers: Reduce computation cost and improve accuracy when fine-tuning language models using example selection in prompt-based workflows.
- •For software developers: Create more efficient AI assistants by structuring and selecting training examples to guide model responses better.
Authors
Omri Bar Haim, Shahar Katz, Lior Wolf
Abstract
The performance of large language model (LLM) workflows often depends on selecting a small set of in-context demonstrations to guide model behavior on new tasks. Recent methods improve this process by augmenting prompts with successful reasoning paths. However, their demonstration selection relies on random sampling or metric-based rankings, overlooking the semantic structure of the task. We propose ClusterFewshot, a strategy that combines semantic structuring with utility-aware scoring to construct representative and effective few-shot demonstration sets. Evaluated within DSPy-based pipelines, ClusterFewshot substantially reduces optimization cost across multiple benchmarks, while consistently improving accuracy relative to prior bootstrap-based methods in both standalone prompt tuning and hybrid prompt-weight optimization.