Strength controls affect interventions on sparse autoencoder codes
Strength Controls Change How Interventions Compare Across Sparse Autoencoder Codes
Machine LearningComputation and Language
Summary
When changing parts of a machine learning system called a sparse autoencoder, the way you measure the size of changes affects how those changes compare. The authors studied different ways to match the strength of interventions, like norm matching and output calibration, to see which keeps comparisons fair. They found that some methods better preserve important differences when testing on language models named Gemma and Llama. Their work helps understand how to fairly compare changes in complex AI models.
What this means in practice
- •For machine learning engineers: Use strength control methods to better compare intervention effects when tuning sparse autoencoder features in models.
- •For natural language processing developers: Improve evaluation consistency of interventions on language model feature representations with output calibration techniques.
Authors
Jinchen Zhang, Zheyuan Liu, Yihan Zhu, Meng Jiang
Abstract
Changing sparse autoencoder (SAE) codes can alter both intervention direction and magnitude, so comparing intervention effects requires a strength control specifying what is held equal. Norm matching and output calibration provide different controls, but whether interventions yield different effects across codes after strength matching remains unclear. We compare intervention effects under these controls for fixed feature groups in Gemma and Llama and observe: (1) Whole-patch scaling. With one coefficient per input, full-output Kullback-Leibler (KL) calibration leaves smaller mean effect gaps from the encoder-based reference intervention than total-norm matching for nonnegative least-squares (NNLS) refits. On new Llama inputs, total-norm matching loses 11 reference transfers, whereas full-output calibration preserves every transfer decision. (2) Positionwise scaling. On a small matched gender-agreement subset, positionwise norm matching leaves smaller observed mean gaps than full-output calibration. (3) Answer exclusion. Compared with unscaled NNLS, answer-excluded calibration reduces gaps in the measured subject-verb agreement comparisons but increases them for gender refits. Full-output KL calibration also reduces larger gaps for norm-matched constructed candidates, although matching coverage is incomplete in several settings. Since calibration uses the evaluated input's outputs, it serves as a diagnostic of residual disagreement.