Constrained Hebbian Learning Supports Efficient Representational Allocation under Structural Constraints
Machine LearningNeural and Evolutionary Computing
Summary
The authors studied how the brain might efficiently use limited resources when learning from audiovisual information. They tested a learning method called excitatory competitive Hebbian learning to see if it can better manage synaptic costs compared to other methods like backpropagation. Their results show that this Hebbian learning doesn't always improve accuracy but achieves a good balance between how much information is stored and how much is needed for the task. This suggests that Hebbian learning helps allocate neural resources efficiently, rather than just maximizing classification accuracy.
Authors
Patrick Inoue, Florian Röhrbein, Andreas Knoblauch
Abstract
Introduction: Biological systems face anatomical and metabolic constraints, including costly synaptic maintenance and limited connectivity. These constraints favor neural codes that compress behaviorally relevant information into low-redundancy patterns. We test whether an excitatory competitive Hebbian rule can support synaptic resource allocation under such constraints and whether the resulting representations occupy a more favorable cost-performance regime than reference learning rules. Methods: Representational cost is quantified using mutual-information-based measures derived from the Variational Information Bottleneck. Experiments use fixed audiovisual embeddings from three audiovisual benchmarks (AVE, Kinetics-Sounds, VGGSound100) to isolate downstream associative plasticity. Hebbian learning is compared with Dense Difference Target Propagation (DDTP) and backpropagation (BP) under matched sparsity and architectural constraints. Results: Hebbian learning achieves lower task-information cost (CTI) than sparse BP and DDTP in the main compressed comparisons, while reaching CTI values comparable to shallow BP with nonnegative weights. Rather than uniformly improving classification performance, Hebbian learning shifts the trade-off between task-relevant information and representational cost, yielding lower CTI at comparable functional performance in several settings. Discussion: The results indicate a cost-performance trade-off rather than uniform accuracy gains. For a given level of task-relevant information, Hebbian representations retain less input information while preserving functional performance, although accuracy is slightly reduced on some datasets. These findings support interpreting Hebbian learning as a mechanism for synaptic resource allocation rather than as a general strategy for maximizing audiovisual classification accuracy.