Dense modular networks disentangle patterns for noisy data decoding
Dense auto-hetero associative memories applied to noisy communication channels
Cryptography and SecurityInformation Theory
Summary
Communicating messages over noisy channels is hard because signals get mixed up or corrupted. The authors explore a special kind of neural network that can separate out mixed messages into their individual parts even when the signals are noisy. This network uses complex connections called 'higher-order Hebbian interactions' to store many patterns and retrieve them correctly. They show mathematically and through simulations that their approach works well and can be used to decode messages even when traditional methods fail under strong noise.
What this means in practice
- •For communication engineers: Decode mixed and noisy transmitted signals by separating combined message patterns using attractor-based neural network dynamics.
- •For signal processing teams: Reconstruct individual signal components from unlabeled mixtures using dense auto-hetero associative networks for robust pattern retrieval.
Authors
Elena Agliari, Andrea Alessandrelli, Adriano Barra, Alberto Fachechi
Abstract
Networks of interacting Hebbian networks have recently been shown to perform a task beyond associative memory, namely \emph{pattern disentanglement}: when fed with a spurious mixture of stored patterns, the different modules spontaneously specialize on, and retrieve, the different constituents of the mixture. So far, this capability has only been established for pairwise interactions, which limits the number of patterns that can be handled. Here we introduce a dense extension of these modular networks, in which both the auto-associative couplings within each module and the hetero-associative couplings among modules are promoted to higher-order Hebbian interactions. We show that, with a suitable choice of the interaction orders, the network disentangles mixtures while storing a number of patterns that scales linearly with the module size, a regime where its pairwise counterpart fails. Through a statistical-mechanical analysis based on Guerra's interpolation, we derive the self-consistency equations for the order parameters and draw the phase diagrams identifying the region where disentanglement is achieved; these predictions are confirmed by Monte Carlo simulations. Finally, we show that disentanglement provides a natural decoding primitive, and we illustrate it with two applications: the explicit reconstruction of all the hidden patterns from the Hebbian tensors and a stream of unlabeled mixtures, and a proof-of-concept communication protocol in which each message token is transmitted as a masked mixture of hidden patterns and decoded by the network dynamics. Owing to its attractor-based decoding, the protocol degrades gracefully under strong channel corruption, where conventional secure-transmission pipelines fail abruptly.