New neuron model improves activation flexibility in brainlike ai conversion
Activation-Flexible ANN-to-SNN Conversion with Finite-State Markov Neurons
Neural and Evolutionary Computing
Summary
Turning artificial neural networks (ANNs) into spiking neural networks (SNNs) usually depends on matching specific activation functions to particular spiking behaviors. This paper introduces a new kind of neuron based on a finite-state Markov model, which can mimic a wide range of activation functions accurately. The authors show that their method works well for common activations like ReLU and sigmoid, and it can reduce the computational cost in some cases. Their approach offers a more flexible way to convert ANNs to SNNs without needing fixed activation-spike correspondences.
What this means in practice
- •For embedded system engineers: Design energy-efficient neural processors that adapt neuron models to different activation functions for improved hardware performance.
- •For neuromorphic hardware developers: Implement flexible neuron models capable of approximating various activations to decrease spiking operations and improve accuracy in spiking networks.
Authors
Ruiyu Jia, Zhuo-Cheng Xiao
Abstract
Most ANN-to-SNN conversion methods rely on a specific correspondence between the source activation and the spiking neuron dynamics. We propose a finite-state continuous-time Markov chain (CTMC) neuron framework whose stationary spike flux can approximate every continuous nonnegative monotone activation function on a compact interval. For a generalized CTMC family with affine input-dependent transitions, we prove uniform approximation to arbitrary accuracy over this function class and derive an explicit approximation error bound. In practice, two- and three-state CTMCs fit ReLU, sigmoid, softplus, and clipped ReLU on the evaluated input ranges, and we evaluate corresponding MLP conversions for each activation with layerwise rate scaling. Moderate clipping improves the conversion cost-accuracy tradeoff on the MNIST MLP and reduces SynOps by 27% on VGG-11/MNIST at matched ANN-SNN accuracy gap criteria, whereas the trend reverses on VGG-11/CIFAR-10. Mean-field and layerwise diagnostics indicate that finite-window sampling and terminal-layer mismatch are the main residual errors. Overall, our results establish finite-state CTMC neurons as a theoretically grounded framework for activation-flexible ANN-to-SNN conversion beyond fixed activation-neuron correspondences.