Emotion patterns in AI language models stay consistent across sizes and tuning
The Geometry of Emotion Representations Across Scale, Depth, and Instruction Tuning
Computation and LanguageMachine Learning
Summary
This paper looks at how six versions of Google's language models understand and represent emotions. The researchers found that the way these models map emotions is very similar, even between smaller and larger models and those adjusted with instructions. Bigger models spread emotional meaning across more components, but keep a strong link to how humans rate emotions as positive or negative. The study also explores at what depth in the model's layers certain emotion-related words appear, showing differences depending on model size. Finally, making the models follow instructions doesn't change these emotional patterns much.
What this means in practice
- •For natural language processing engineers: Optimize emotion-related features in language models by leveraging consistent emotion geometry across sizes and tuning approaches.
- •For sentiment analysis developers: Improve emotion detection tools with insights about how emotion words emerge at different network depths in language models.
Authors
Adam Hollowell
Abstract
We measure the emotion geometry of six LLMs spanning Google's Gemma 2 family. Using an extraction pipeline established by Sofroniew et al. (2026) on Claude Sonnet 4.5 and replicated by Hollowell (2026) on Gemma 2 27B base, we compare emotion geometry across base and instruction-tuned variants of Gemma 2 2B, 9B, and 27B. First, all six model cells exhibit the core emotion geometry. A linear map fit between two cells places held-out emotions nearest their own vectors in the target model. Across all fifteen model pairs, fit in one direction each, this failed exactly once, and the median rank was 1 in every pair. Linear maps refit on cleaned embedding-table vectors recover every held-out emotion at rank 1 in all thirty ordered pairs. Second, scale spreads the geometry's variance across more directions without weakening its tie to valence. The leading component's share of variance falls from 31.6 percent (2B) to 29.8 (9B) to 26.7 (27B) in the base models and from 31.9 (2B) to 29.2 (9B) to 27.3 (27B) in the tuned ones, while its correlation with human valence ratings stays between .71 and .78 in all six models. Third, we project each emotion vector onto the output vocabulary and ask at what depth the vector's own emotion word enters the top 10,000 of the 256,000 tokens. In the 2B and 9B models the median emotion word is there from layer 0, while in the 27B models it begins near chance, stays outside the top 10,000 for half the network, and enters at layer 23 in one step. Fourth, tuning leaves every registered analysis-layer measure within its floor. At every size, valence correlation rises and arousal correlation falls by amounts within their floors, the latter against prediction. PC2's variance share also falls, with a pooled decrease significant after correction. Mid-late layer stability exceeds its floor and traces to base 27B's final layer. Code, data, and audit reports are released.