Activation steering stores some memory but rarely recalls facts correctly
What Survives When Activation Steering Is Used as Memory
Artificial IntelligenceComputation and LanguageMachine Learning
Summary
Sometimes, AI systems try to remember past experiences to improve later answers by changing their internal signals. The authors tested a method that stores changes to an AI's internal state as a kind of memory. They found that this memory mostly stores general feelings or preferences, but facts are almost never recalled properly. Surprisingly, if the memory stores a dislike, the AI might actually recall it as a like unless carefully processed. This means that only very certain types of information survive this kind of memory.
What this means in practice
- •For ai system builders: Use activation steering memory to selectively retain consistent user preferences without disrupting unrelated responses in dialogue systems.
- •For chatbot developers: Design chatbots that incorporate leanings via activation shifts, avoiding reliance on text-based memory for factual knowledge.
Authors
Vedant Jumle
Abstract
Activation steering has recently been used as a memory for skills: past experiences of an agent are stored as steering vectors and retrieved later. Facts and preferences about the user are assumed to need text instead. We test that assumption with a training-free activation memory beside a frozen LLM. The change an experience causes in late-layer states is stored as a shift, filed under a whitened key for the conversation's gist, and added back in a later session only when the gist matches. We argue that such a shift is a fixed average of an effect that attention, in context, recomputes for every question, so only question-independent content can survive. In a pre-registered study on Qwen3.5-2B and 9B, with judges from four model families, the memory is selective (about 98% of unrelated replies are unchanged), but facts are stored as the word to say, in the last layers only, and are almost never used (at most 15% clean use, against 96-98% in context). Leanings transfer only in a narrow strength window. Most strikingly, the reference subtracted when writing a preference decides whether its concept or its direction survives: unless the opposite is subtracted, a stored dislike is recalled as a like.