Prompt echoing reveals induction heads cause in small language models
Mirror, Mirror on the Wall: Prompt Echoing in Small Instruct Language Models
Computation and LanguageArtificial Intelligence
Summary
Sometimes small language models just repeat the prompt they are given instead of answering it. The researchers studied this behavior in several models and found it is mostly caused by parts inside the model called induction heads, not mainly because the prompt matches data it saw when training. They showed that prompt echoing overlaps a bit with the training data, but the key reason is how the model processes and copies text internally.
What this means in practice
- •For software developers: Improve small language model prompts to reduce unwanted prompt echoing in chatbots or assistants.
- •For ai infrastructure teams: Diagnose and fix internal copying behaviors in language models to enhance response quality and reduce data leakage risks.
Authors
Inez Okulska, Bartosz Naskręcki, Jan Piotrowski, Tomasz Steifer
Abstract
Prompt echoing is a recognized failure mode of instruct language models, in which a model instead of generating a response, mirrors the provided prompt, even though it did not receive a specific instruction to do so. Is this phenomenon a sign of the model leaking the content of its training dataset, or is it rather caused by a misaligned behavior of the internal induction/copying mechanisms? We investigate prompt echoing small language models from different families (Gemma, Llama, Qwen, SmolLM and OLMo) and show that echoing prompts are likely to have partial overlap with the training dataset but the phenomenon is primarily driven by the model's induction heads.