Compact encodings improve similarity mapping in spiking neural networks
Beyond the Hamming Cliff: Compact Spatial Encodings Preserving Semantic Distance in Spiking Neural Networks
Neural and Evolutionary Computing
Summary
Standard binary codes used for input in spiking neural networks can cause a problem called the "Hamming cliff," where similar values appear very different. Davies et al. show that by using other encoding methods like Gray, Thermometer, Johnson, or new hybrid codes, they can better preserve the true relationships between similar values. This helps the network recognize which inputs are close to each other more accurately, which is important for fast and effective processing. However, some encodings preserve the similarities better than others depending on their complexity and size.
What this means in practice
- •For neuromorphic hardware engineers: Improve input encoding schemes to preserve value similarity and reduce processing errors on spiking neural network chips like SpiNNaker.
- •For robotics system developers: Use hybrid encoding methods to provide continuous sensory inputs that maintain semantic relationships for faster, low-latency neural control.
Authors
Sergio Davies, Muhammad Aitsam, Morgan-Tyler Stone, Alessandro Di Nuovo, Liangxiu Han
Abstract
By pre-processing input signals with Gray, Thermometer, Johnson and hybrid encodings before passing them to spiking neural networks running on SpiNNaker hardware, Davies et al.(2025) address the semantic-spatial disparity that causes the ``Hamming cliff'' in standard binary representations. Continuous semantic patterns are presented as a set of simultaneous spikes at the input layer, preserving ultra-low latency processing. Training the network on multiple continuous values under each scheme, and measuring accuracy, required homeostatic scaling and false-positive rates, shows that removing the cliff is necessary but not sufficient: Gray encoding removes it yet retains little global distance structure, whereas Thermometer and Johnson preserve semantic distance at a code width that grows with the value range. The proposed hybrid encodings contain all three as limiting cases of a single parameter, recovering most of that structure at a fraction of the width. The homeostatic factor is shown to set a radius of tolerance around each trained pattern, so that the choice of encoding alone decides which values the network treats as similar.