Generalization in large language models depends on output stability not accuracy
Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMs
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Large language models are often judged by how accurately they answer questions based on a single prompt style. This paper finds that true generalization means the model should give similar answers when the same question is asked in different ways. The authors created a new method called SAGO that measures how stable a model's answers and internal behaviors are across different prompts and tasks. They show that many current models are inconsistent in their answers, meaning their generalization is unstable and depends on how the question is asked.
What this means in practice
- •For llm developers: Evaluate large language models for consistent output across varied input formats using SAGO to improve stability-focused training.
- •For ai safety teams: Detect inconsistent behavior in language models by measuring variability in outputs and internal states to reduce unpredictable AI responses.
Authors
Nagham Omar, Mahmoud Jabarin, Maya Rozenshtein, Rom Himelstein, Avi Mendelson, Amit LeVi
Abstract
Generalization in large language models (LLMs) is the ability to produce consistent and semantically stable outputs when the same input is expressed in different ways. Existing work typically evaluates generalization through aggregate accuracy on a single prompt format, task, or set of variations, which conflates robustness with overall benchmark performance. In this work, we show generalization evaluation at the level of individual examples, across multiple input variants, and across different aspects of model behavior, focusing on variability rather than reducing performance to a score that can be improved through narrow training or other ways that obfuscate generalization evaluation. Following this view, we introduce the Stability-Aware Generalization Objective (SAGO), a framework that measures how much model behavior changes for the same input under different variations and benchmarks, capturing variability across several dimensions including generation consistency, internal activations, confidence, and response mirroring. We show that many commonly used models exhibit statistically significant and consistent generalization instability: no model generalizes uniformly, behavioral axes capture independent failure modes, and cross-dataset variation can reverse model rankings.