BRIDGING THE GAP BETWEEN ANNS AND SNNS BY CALIBRATING OFFSET SPIKES

Abstract

Spiking Neural Networks (SNNs) have attracted great attention due to their distinctive characteristics of low power consumption and temporal information processing. ANN-SNN conversion, as the most commonly used training method for applying SNNs, can ensure that converted SNNs achieve comparable performance to ANNs on large-scale datasets. However, the performance degrades severely under low quantities of time-steps, which hampers the practical applications of SNNs to neuromorphic chips. In this paper, instead of evaluating different conversion errors and then eliminating these errors, we define an offset spike to measure the degree of deviation between actual and desired SNN firing rates. We perform a detailed analysis of offset spike and note that the firing of one additional (or one less) spike is the main cause of conversion errors. Based on this, we propose an optimization strategy based on shifting the initial membrane potential and we theoretically prove the corresponding optimal shifting distance for calibrating the spike. In addition, we also note that our method has a unique iterative property that enables further reduction of conversion errors. The experimental results show that our proposed method achieves state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet datasets. For example, we reach a top-1 accuracy of 67.12% on ImageNet when using 6 time-steps. To the best of our knowledge, this is the first time an ANN-SNN conversion has been shown to simultaneously achieve high accuracy and ultralow latency on complex datasets.

1. INTRODUCTION

Acclaimed as the third generation of Artificial Neural Networks (Maass, 1997) , Spiking Neural Networks (SNNs) have brought brand-new inspiration to computational neuroscience. As the corresponding neuron fires spikes only when the current membrane potential exceeds the firing threshold, SNNs have the distinctive characteristics of binary output, high sparsity, and biological plausibility. Therefore, compared with traditional ANN models, SNNs can further improve computational efficiency and reduce power consumption, which facilitates their remarkable superiority in the application of neuromorphic chips (Merolla et al., 2014; Davies et al., 2018; DeBole et al., 2019) . Considering that an effective learning algorithm has not yet been found for SNNs, ANN-SNN conversion and backpropagation through time (BPTT) are still the two most commonly applied training methods. Compared with BPTT, ANN-SNN conversion provides a way around the nondifferentiable problem in the direct training procedure for SNNs and thus reduces the overall training complexity. The aim in ANN-SNN conversion is to establish the mapping relationship between the activation output and the average firing rate. Traditional conversion methods exploit larger time-steps to overcome conversion errors and thus achieve high performance (Diehl et al., 2015) . Many of the following works have attempted to optimize the performance from multiple perspectives, including using the soft-reset mechanism (Han et al., 2020) , proposing more adaptive activation functions (Ho & Chang, 2021; Bu et al., 2022b) , adopting a trainable threshold (Sengupta et al., 2019; Ding et al., 2021; Bu et al., 2022a) , etc. However, these strategies cannot effectively eliminate the errors caused by the deviation between the actual and desired firing rates, especially when the number of timesteps is small. Some recent works explore compensating for the errors by introducing burst spikes (Li & Zeng, 2022) and signed spiking neurons (Li et al., 2022) . Unlike these works, our paper attempts to eliminate the errors with vanilla spiking neurons and answer the question of how to improve the performance of a converted SNN and possibly approach the upper bound performance. In this paper, we observe and identify the source of conversion errors and propose an iterative optimization method based on shifting the initial membrane potential, which can fulfil accurate mapping between ANNs and SNNs under ideal situations. Our main contributions are summarized as follows: 1 We introduce the concept of offset spike to infer the deviation between the actual SNN firing rate and the desired SNN firing rate. We note that cases of firing one additional (or one less) spike are the main reason cause of conversion errors. 2 We propose a method to judge offset spike based on the residual membrane potential and an optimization method to eliminate conversion errors by shifting the initial membrane potential up or down. We derive the optimal shifting distance and prove that one spike can be increased or decreased under this condition. 3 We evaluate our methods on CIFAR-10/100 and ImageNet datasets. The proposed method outperforms the existing state-of-the-art ANN-SNN conversion methods using fewer timesteps. For example, we achieve 67.12% top-1 accuracy on ImageNet with only 6 timesteps (4 time-steps for calibration and 2 time-steps for inference). Moreover, it is worth noting that we have reached the same level of performance as BPTT under the condition of significantly reduced memory and computing resources requirements. 4 We discover that our proposed method has an iterative property. Under ideal circumstances, the deviation within the range of k spikes will be eliminated entirely after adopting our approach k times. After 4 iterations, the mean-square error between the actual and desired firing rates of the output layer can reach 0.001 for the VGG-16 model on CIFAR-100.

2. RELATED WORKS

The 2021) fine-tuned the bias in each layer under the uniform current assumption. Nevertheless, the actual current would never be distributed uniformly. In terms of expectation, Bu et al. (2022b) proved that one-half of the threshold is the optimal value for the initial membrane potential, and that charging at this value can prompt neurons to spike more uniformly. However, as the authors pointed out: there is still a mismatch between ANN and SNN due to the so called "unevenness error". In addition, other methods like burst spikes (Li & Zeng, 2022) and signed spiking neurons (Wang et al., 2022a; Li et al., 2022) , have also been introduced to further improve performance. These efforts have aimed to alleviate the conversion loss. However, they undermined the biological plausibility and binary property of spiking neurons. In addition to ANN-SNN conversion, backpropagation with exact spike time is another common way to train SNNs. The surrogate gradient (O'Connor et al., 2018; Zenke & Ganguli, 2018; Bellec et al., 2018; Wu et al., 2018; 2019; Kim & Panda, 2020; Zenke & Vogels, 2021) has been widely used to tackle the nondifferentiable problem in the training process, which substitutes the Heaviside function with a derivable function. With the help of the surrogate gradient, backpropagation



principle of ANN-SNN conversion is to map the parameters from pretrained ANN models to SNNs, which avoids training SNNs directly and reduces energy consumption significantly. The primary goal is to match the ANN activation value and the average SNN firing rate. Cao et al. (2015) were the research pioneers in this field, and they replaced ReLU activation layers in ANNs with spiking neurons to fulfil the conversion procedure. Ho & Chang (2021); Bu et al. (2022b) proposed new activation functions, which better fit the finiteness and discreteness of the spike firing rate. Rueckauer et al. (2017); Han et al. (2020) adopted "reset-by-subtraction" mechanism, which alleviated the problem of information loss and effectively improved the precision of the conversion process. For the setting of firing threshold, various strategies have been proposed, includingRo- bustNorm (Rueckauer et al., 2017), SpikeNorm (Sengupta et al., 2019), and adjustable threshold(Han et al., 2020; Ding et al., 2021; Ho & Chang, 2021; Bu et al., 2022a;b), etc. Recently, spiking neural networks with high accuracy and low latency have become the focus and target of academic research. To reduce the time latency of the network, one must carefully address the exact spiking time of neurons. Deng & Gu (2021); Li et al. (

availability

Code is available at https://github.com/hzc1208/ANN2SNN_COS.

