Purin improves neural networks by mimicking brain synapses
Purin: A Biology-inspired Mechanism for Artificial Neural Networks
Artificial IntelligenceNeural and Evolutionary Computing
Summary
Artificial neural networks often use fixed connections that don’t change quickly, unlike real brains where connections can vary moment to moment. The authors created Purin, a new method that lets these networks adjust connection strengths temporarily as the brain does, without needing complex timing steps. They tested Purin on popular networks like AlexNet and GoogLeNet, and it improved their ability to classify images. This approach blends biology-inspired ideas with conventional artificial intelligence models.
What this means in practice
- •For machine learning engineers: Enhance common image classification models by incorporating temporary synaptic changes to improve accuracy.
- •For biomedical data scientists: Model neural signal variability more realistically in computational neuroscience tasks using Purin’s synapse modulation approach.
Authors
Zishu Liu, Chunbo Luo, Christos Grecos
Abstract
Artificial neural networks (ANNs) usually represent neural transmission with fixed trainable weights during a training batch, which omits short-term changes in synaptic efficacy. In addition, the discrete time-step simulation requires additional temporal processing that many conventional ANN architectures do not use. To overcome these challenges, we propose Purin, a biology-inspired and ANN-compatible mechanism, that introduces synaptic efficacy modulation into conventional convolutional neural networks. Purin uses a time-interval-based abstraction for neural activities, which allows Purin to introduce short- and long-term synaptic efficacy changes without using discrete time-steps. Purin introduces a bounded factor to represent temporary synaptic efficacy changes, together with two weight matrices that represent input-side and output-side efficacy. The weight matrices are updated by backpropagation and interpreted as the long-term synaptic efficacy changes. Experimental results show that after removing the confounding factors in the AlexNet, VGG11, and GoogLeNet architectures, Purin improves the classification accuracies in all three models across the evaluated datasets.