Hessian Null Space Continuation reveals diverse neural network solutions
Traversing the solution space of neural networks with Hessian Null Space Continuation
Machine Learning
Summary
Neural networks trained on the same task can learn many different ways to solve it, but people didn’t fully understand how these different solutions relate or change inside the network. The authors developed a new method, Hessian Null Space Continuation (HNC), to explore these hidden pathways in the network’s parameters without changing its behavior. Using this method, they showed that even close to one trained solution, there are many very different internal workings and strategies. This approach helps see how networks can behave the same yet think differently and opens doors to editing or combining models in new ways.
What this means in practice
- •For machine learning engineers: Locate and modify neural network parameters to achieve targeted internal behaviors while maintaining performance, improving model customization.
- •For reinforcement learning developers: Detect alternative agent strategies and uncover reward hacking without changing overall task success, increasing reliability of RL agents.
Authors
Ann Huang, Mitchell Ostrow, Zhouyang Lu, William T. Redman, Leo Kozachkov, Kanaka Rajan
Abstract
On a single task, deep networks can learn many solutions, depending on their optimizer, training data, architecture, and hyperparameters. Many of these solutions are mode-connected: rather than isolated points in weight space, they are connected by low-loss regions. Yet how their internal computation varies within these regions is unknown. A parallel line of work has identified the degeneracy of neural representations: many networks reach similar training loss with distinct internal structures. However, it is unclear how these solutions are related in weight space. We unify these subfields and show for the first time that many different internal mechanisms exist within a local mode-connected region in weight space. To do so, we introduce Hessian Null Space Continuation (HNC), a scalable method that uses local curvature to traverse regions of weight space that preserve network function, and can be steered toward solutions with specified properties. In RNNs trained on a memory task, HNC reaches drastically different representations and dynamics with maintained behavior. In ImageNet-trained Vision Transformers, HNC finds representations that differ more from the original network than any independently trained model with a different architecture or objective. In reinforcement-learning agents, HNC uncovers a distinct navigation strategy at comparable return and exposes reward hacking in an AI Safety Gridworld. Finally, HNC measures the local geometry of the solution set, showing how model size and task complexity shape its dimension and functional sensitivity. Our results show that a surprisingly large amount of representational diversity exists near a single trained solution, unseen by standard gradient-based optimization. HNC identifies and quantifies this diversity, opening new possibilities for mechanistic understanding of solution spaces and for model merging, editing, and fine-tuning.