An Online Supervised Learning Algorithm Based on Nonlinear Spike Train Kernels Xianghong Lin(&), Ning Zhang, and Xiangwen Wang School of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China
[email protected]
Abstract. The online learning algorithm is shown to be more appropriate and effective for the processing of spatiotemporal information, but very little researches have been achieved in developing online learning approaches for spikingneural networks. This paper presents an online supervised learning algorithm based on nonlinear spike train kernels to process the spatiotemporal information, which is more biological interpretability. The main idea adopts online learning algorithm and selects a suitable kernel function. At first, the Laplacian kernel function is selected, however, in some ways, the spike trains expressed by the simple kernel function are linear in the postsynaptic neuron. Then this paper uses nonlinear functions to transform the spike train model and presents the detail experimental analysis. The proposed learning algorithm is evaluated by the learning of spike trains, and the experimental results show that the online nonlinear spike train kernels own a super-duper learning effect. Keywords: Spiking neural networks kernels Online learning
Supervised learning
Spike train
1 Introduction Artificial neural networks (ANNs) have got great progress and successfully applied in many fields [1]. In recent years, the focus on ANNs is gradually turning to the spiking neural networks (SNNs) which are more biological plasticity, especially the learning methods and theoretical researches of the SNNs [2–4]. According to the learning rule, supervised learning methods based on temporal coding of SNNs mainly contain the supervised learning algorithm based on the gradient descent rule and the supervised learning algorithm based on the Hebb rule. The supervised learning algorithms based on the gradient descent rule are mainly divided into single spike learning and multi-spike learning (spike trains learning). The most typical single spike learning is the SpikeProp [5] and its various modified forms such as increasing the momentum item [6, 7], QuickProp and RProp [8] etc. However these methods operate pure single spike, which requires all of neurons in the input layer, hidden layer and output layer only fire one spike. Booij and Nguyen [9] continued to improve the SpikeProp, their algorithm was not restricted to the number of firing spikes for the neurons in the input layer and hidden layer, which achieved the spike trains learning in the input layer and hidden layer. Similarly, Ghosh-Dastidar et al. [10] derived a gradient descent spike © Springer International Publishing Switzerland 2015 D.-S. Huang et al. (Eds.): ICIC 2015, Part I, LNCS 9225, pp. 106–115, 2015. DOI: 10.1007/978-3-319-22180-9_11
An Online Supervised Learning Algorithm
107
trains learning rule for the synaptic weights of the output layer and hidden layer using Chain rules, named the Multi-SpikeProp algorithm. But the neurons in the output layer can only fire one spike. Recently Xu et al. [11] proposed a new supervised multiple-spike learning algorithm based on the gradient descent rule for SNNs. The algorithm does not restrict the number of firing spikes for neurons in all layers of the network, which realizes the spike trains spatial-temporal mode learning for multilayer feedforward spiking neural networks. However, the learning efficiency and the accuracy of supervised learning algorithms based on gradient descent rule are not high. These algorithms adopt pure mathematical methods, completely ignore the concept for neurons running and lack of biological plasticity. Then in order to simulate real biological neurons well, researchers have proposed Hebb supervised learning algorithms which own more biological plasticity. Hebb [12] firstly put forward a synaptic plasticity hypothesis: “If two neurons are excited at the same time, the synapses between them can be strengthened ”. The hypothesis emphasizes the importance of the synergistic activity and synaptic strengthening between the pre/postsynaptic neurons. In fact, spike trains can not only cause the continuous change of synapses, but also satisfy Spike Timing-Dependent Plasticity (STDP) [13] mechanism. Ponulak and Kasiński [14] represented a supervised learning method, named Remote Supervised Method (ReSuMe), which the basic idea of the method came from Widrow-Hoff rules. The method is composed of the Widrow-Hoff rule expression which uses the STDP rules to deduce the actually learning method. And the adjustment of the synaptic weights relies on the combination of the STDP and anti-STDP. The results of the study show that the ReSuMe owns a very good learning performance and widely applicable areas. Florian [15] proposed Chrontron learning methods, which included two supervised learning algorithms: the E-Learning with higher learning ability and I-Learning with more biological plasticity. Recently, with the continuous deepening of researches, the supervised learning algorithm based on kernel function gradually has formed a new system, SPAN and PSD are the representative. Mohemmed and Schliebs [16, 17] proposed SPAN (Spike Pattern Association Neuron) algorithm based on kernel function. Its main features are to use kernel function convert spike trains to the convolution signals, and then the transformed input spike trains, desired spike trains and actual output spike trains apply Widrow-Hoff rules to regulate the synaptic weights. Inspired by SPAN algorithm, Yu et al. [18] had given a different explanation which put the traditional Widrow-Hoff rules to SNNs and put forward the PSD (Precise-Spike-Driven) supervised learning algorithm. Adjustment of the synaptic weights is driven by the error between the desired output spike trains and the actual output spike trains, positive error will lead to enhanced long-term potentiation, negative error will lead to long-term depression. In this paper, we propose an online supervised learning algorithm based on the nonlinear spike train kernels (STK). Online learning is different from the offline learning or the batch learning, which is a supervised learning method with dynamic learning performance [19]. In the real world, we obtain the data with time and space characteristics, and the spatiotemporal data is generally represented as a continuous spike train current [20]. We require the learning algorithm to process real-time learning for neural networks, and synaptic weights are dynamic change with the spike input process. Therefore, online learning algorithm will be more appropriate and effective for
108
X. Lin et al.
the processing of this kind of real-time tasks [21]. To implement the learning of the complex spatiotemporal model on the spike trains, the key problem is to construct a suitable kernel function to express the spike trains. In the STK learning algorithm, the Laplacian kernel function is used to the input spike trains which are transformed into analog convolution signals. The spike trains expressed by these kernel functions acting on postsynaptic neuron are linear, therefore, this paper adopt the nonlinear model with more biological interpretability.
2 Online Supervised Learning Rules for Spiking Neuron In this section, we present a new online learning algorithm based on STK. The learning methods of traditional SNNs mostly adopt the way of offline learning, its synaptic weights will be adjusted after a large number of spike trains submit training. The offline learning can make neural networks form a network structure before the systems work, but the parameters and temporal of the actual systems are often unpredictable and varying, which causes the offline learning not to be spatiotemporal pattern learning. Nevertheless, the online learning rule of the spike neural networks is to train each spike train after submitting, and only learns one spike train at one moment, then in real time adjusts the network parameter values according to the training results. We main process spatiotemporal data in the real world, it demands the real-time network learning, synaptic weights process dynamic change with the input of the spike trains. Hence, online learning algorithm will be more appropriate and effective for this kind of real-time tasks.
2.1
Learning Rule Based on Linear Spike Train Kernels
In the learning process of spike trains, after the presynaptic membrane released neurotransmitter into the synaptic cleft, neurotransmitter diffuses into the postsynaptic membrane, which causes the change of postsynaptic membrane for some ion permeability and produces electric potential difference. Then it makes some charged ions in and out of the postsynaptic membrane, which forms a postsynaptic potential. Therefore we must simulate the process of spike trains occurrence in the synapse, and convert spike trains into a functional form model. For a spike train, the spike train s = {tf: f = 1, 2 … F} stands for the orderly sequence of neurons firing spike time, the spike train model can be shown as follows: sðtÞ ¼
F X d t tf
ð1Þ
f ¼1
where F is the number of spiking for a spike train, δ(t) is the Dirac delta function, if t = 0, δ(t) = 1, else δ(t) = 0, tf is the fth spiking firing time. So we represent the formalized spike trains as follows:
An Online Supervised Learning Algorithm
vi ðtÞ ¼
F X k t tf
109
ð2Þ
f
where k(t) is the kernel function which includes the Gaussian, Laplacian and α-function and so on, we choose the Laplacian kernel in our paper. Firstly, we employ the Widrow-Hoff rules for the ith synapse in the input neurons O, as the follow function: Dxoi ¼ qxi ðsd so Þ
ð3Þ
where ρ is a positive constant on behalf of the learning rate, xi is an input spike train of the input neurons O, sd and so refer to the desired and actual output neurons spike trains. In order to reduce the complexity of the calculation and improve learning efficiency, in this paper, we process convolution kernels computation just for the input spike trains. Then we can get the learning rule based on the linear STK like that: Dxi ðtÞ ¼ qvi ðtÞðsd ðtÞ so ðtÞÞ X k t t f ðsd ðtÞ so ðtÞÞ ¼q
ð4Þ
f
2.2
Learning Rule Based on Nonlinear Spike Train Kernels
However, the above description is flawed from the biological perspective. One of the basic defects is that it ignores the capability limitation of the dendrite receptor. When two spikes arrive to the postsynaptic membrane within a short period of time or at the same time, the first spike makes the dendrite receptor receive stimulation and lead to increase dendritic conductance. Because dendrite receptor is being influenced by the neurotransmitters which are released by the previous spike, the second spike will only induce a little influence. Therefore, the spike trains expressed by the Eq. (2) are linear to the postsynaptic neuron from another aspect. We use a nonlinear function f(x) to transform the spike train model. Here the nonlinear functions which we choose are f1 ðxÞ ¼ tanhðx=rÞ and f2 ðxÞ ¼ 1 ex=r . We apply it to the nonlinear STK, the equation can be expressed as: y
vi ðtÞ ¼ f
F X
! kðt t Þ f
ð5Þ
f
where the k(t) is still the kernel function like the Eq. (2), and we adopt the Laplacian kernel, kðtÞ ¼ ejtj=s HðtÞ, H(t) is the Heaviside function, if t < 0, H(t) = 0, else H(t) = 1. In Fig. 1, we give the different STK simulation processes and the STK synaptic weights adjustment process. In Fig. 1(a), the nonlinear STK1 and the nonlinear STK2 adopt the nonlinear functions f1(x) and f2(x) respectively. The synaptic weights adjustment depends on three aspects: When the desired spikes are the same with the
110
X. Lin et al.
output neuron spikes, we will get a zero error; there is not an output spike in times of the desired spike, we can get a positive error; a negative error is received when an output spike is not supposed by the desired spike. The adaptation process of the STK synaptic weights is shown in Fig. 1(b).
(a) i
0 2 1 0 2 1 0 2 1 0
(t)
20 40 60 80 100 120 140 160 180 200
0
50
100
150
200
spikes [ms]
Linear STK
Nonlinear STK1
d
(t)
o
(t)
i
(t)
Nonlinear STK2
Fig. 1. The STK simulation and weight adaptation process. (a) The kernel function transformation form of the input spike trains linear, the nonlinear STK1 and the nonlinear STK2, and the y-axis is the same scale. (b) The weight adaptation process. Si(t) is the input spike train, Sd(t) and So(t) are the desired and actual output neurons spike trains respectively.
Hence we obtain the nonlinear STK function, and then we use it to the renewal of the synaptic weight. We get a new online learning rule based on nonlinear STK, which can solve spatiotemporal data problems and possess more biological interpretability like that: y Dxi ðtÞ ¼ qvi ðtÞðsd ðtÞ so ðtÞÞ ! X f kðt t Þ ðsd ðtÞ so ðtÞÞ ¼ qf
ð6Þ
f
2.3
The Measure Criterion of the Spike Train Learning
In the spike trains learning, the measure criterion of learning result is the close extent between the desired output spike trains and the actually output spike trains at the end of the learning, which is the distance between the desired and the actually output spike trains. In this paper we adopt the method C based on the correlation measurement to describe the distance between the desired output spike trains and the actually output 2 spike trains. According to . the Cauchy-Schwarz inequality jvd vo j jvd vd j jvo vo j, we get that 0 jvd vo j2 jvd vd j jvo vo j 1. Where vd and vo are the vectors which represent a convolution of the desired output spike trains and the actually output spike
An Online Supervised Learning Algorithm
111
trains with a Gaussian low-pass filter respectively. Then we make the expression of measurement C to define as: vd vo C ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi j vd vd j j vo vo j
ð7Þ
The numerator is the inner product of two vectors in the equation, and the denominator is the product of two the Euclidean norms vectors. When two spike trains are completely same, the value of C is 1, and it gradually tends to zero with the decrease of their correlation.
3 Experimental Results In this section, we are by the learning problems of the spike trains to verify the performance of the linear STK and the two nonlinear STKs. There are 400 synaptic input neurons in our simulation, the length of the input and desired output spike trains are defined as 200 ms. The input and desired output spike trains are randomly generated by the frequency of 30 Hz Poisson process respectively. Neural model adopt the simple model SRM, the time delay constant of postsynaptic neuron is 5 ms, and the delay constant of the refractory period function selects 50 ms. The firing threshold of the Neurons is 1, the length of absolute refractory period after the spike fires is 1 ms. At first the max fired time of the every neuron is 200 ms, the time step is 0.1 ms. All of the learning rates set 0.05. We use the Laplacian kernel as the kernel function in our simulation, and its parameter τ uses 5. The value of σ is 0.2 in the nonlinear function. In each simulation, we process 100 times iterations, and each experimental result is the average result which runs 100 times experiments. First of all, as shown in Fig. 2, we analyze the learning process of spike neuron. Figure 2(a) − (c) shows the learning process of spike neuron, Fig. 2(a) represents the learning process of nonlinear STK1, which adopt the nonlinear function f1(x); Fig. 2(b) represents the learning process of nonlinear STK2, which adopt the nonlinear function f2(x); Fig. 2(c) represents the learning process of linear STK. In Fig. 2, ∇ represents the desired output spike train, Δ represents the output spike train before the learning of spike neuron, • represents the actual output spike train of some learning cycles in the learning process. From the learning process we can see that Fig. 2(a) and Fig. 2(b) neuron takes about 3 steps from the learning of initial random output spike train to the desired output spike train, while Fig. 2(c) takes about 12 steps. Figure 2(d) represents the change of learning precision curve in the learning process, as shown in Fig. 2(d), the learning precision of nonlinear STK1 which just takes about 2 learning cycles reaches 1, the nonlinear STK2 takes about 3 learning cycles, while linear STK takes about 12 learning cycles. Figure 3 analyses the learning performance of the algorithm in different length of spike train. In the experiment, the length of input and desired output spike train increases gradually from 100 to 1000 by the interval of 100 ms, other settings remain unchanged. The nonlinear STK1 and the nonlinear STK2 adopt the nonlinear functions f1(x) and f2(x) respectively. As we can see from Fig. 3(a), the highest learning accuracy
112
X. Lin et al.
epoch
(a) 10
5
0
0
20
40
60
80
100 120 time [ms]
140
160
180
200
0
20
40
60
80
100 120 time [ms]
140
160
180
200
20
40
60
80
100 120 time [ms]
140
160
180
200
epoch
(b) 10 5
0
epoch
(c) 20
10
0
(d)
0
1
C
0.8 Nonlinear STK1
0.6
Nonlinear STK2
0.4 0.2
Linear STK
2
4
6
8
10 12 learning epoch
14
16
18
20
Fig. 2. The learning process of spike train. (a) The learning process of the nonlinear STK1 spike neuron, which adopts the nonlinear function f1(x). (b) The learning process of the nonlinear STK2 spike neuron, which adopts the nonlinear function f2(x). (c) The learning process of the linear STK spike neuron. (d) The change curve of the learning precision.
of the two algorithms are decreasing along with the length of the spike train increased, the learning accuracy of the two nonlinear STKs is almost at the same, and the learning precision of the two nonlinear STKs online learning is higher than the linear STK online learning apparently. When the length of spike train reaches 1000 ms, both of the two nonlinear STKs arrive 0.85, and the linear STK is 0.8, then the curve almost remains unchanged. Therefore, online learning is more suitable for dealing with large-scale data problems. Figure 3(b) is the minimum iterations to the highest accuracy. From Fig. 3(b), we know that the iterations of the two nonlinear STKs are smaller than the linear STK and are faster get the highest learning accuracy. For example, when the length of spike train is 400 ms, the minimum iterations of the linear STK, the nonlinear STK1 and the nonlinear STK2 are 35, 26 and 24 respectively.
An Online Supervised Learning Algorithm
0.92
Nonlinear STK1
0.9
Nonlinear STK2 Linear STK
C
0.88 0.86 0.84
(b) 60
Nonlinear STK1
50
Nonlinear STK2
learning epoch
(a)
Linear STK
40 30 20 10
0.82 0.8
113
0
100 200 300 400 500 600 700 800 9001000 length of spike train [ms]
100 200 300 400 500 600 700 800 9001000 length of spike train [ms]
Fig. 3. The learning effects of spike train in different length. (a) The learning accuracy of spike train in different length. (b) The minimum iterations to the highest accuracy.
In Fig. 4, it analyzes the learning performance of input and desired output spike train when they are in different spike frequency. Setting the input and desired output spike trains are generated by the process of Poisson where they are in different firing frequencies. In the experiment, the firing frequency of input and desired output spike train increases gradually from 10 Hz to 100 Hz in 10 Hz intervals, also, the firing frequency between input spike train and the desired output spike train is equal, other settings remain unchanged. The nonlinear STK1 and the nonlinear STK2 adopt the nonlinear functions f1(x) and f2(x) respectively. As we can see from Fig. 4(a), the learning accuracy of the three algorithms are first increased and then tend to be gradual declined along with the increase of the firing frequency of the spike train, and the learning accuracy of the two nonlinear STK online learning is always higher than the linear STK online learning. When the firing rate of the spike train reaches 20 Hz, both of the nonlinear STK1 and STK2 arrive 0.88, the linear STK arrives the highest learning precision 0.85, but it is still below the nonlinear STK, and then tends to decline gradually. Figure 4(b) is the minimum iterations to the highest accuracy in different firing rate. From Fig. 4(b), we can get that the nonlinear STKs are faster to achieve the highest accuracy in different firing rate. Such as, when the firing rate of the
(b) 50
0.85
40
C
0.8 0.75 Nonlinear STK1
0.7 0.65
Nonlinear STK2 Linear STK
10 20 30 40 50 60 70 80 90 100 firing rate of spike trains [Hz]
learning epoch
(a) 0.9
Nonlinear STK1 Nonlinear STK2 Linear STK
30 20 10 0
10 20 30 40 50 60 70 80 90 100 firing rate of spike trains [Hz]
Fig. 4. The learning effects of spike train in different firing rate. (a) The learning accuracy in different firing rate. (b) The minimum iterations to the highest accuracy.
114
X. Lin et al.
spike train reaches 70 Hz, the nonlinear STK1 achieves the highest accuracy after 17 iterations; the nonlinear STK2 is through 22 iterations; while the linear STK is 35 iterations.
4 Conclusions This paper comes up with an online supervised learning algorithm based on nonlinear STK. To achieve the learning of complex spatiotemporal pattern on spike trains, the key issue is that we need to construct a proper kernel function to represent the spike train. In our paper, we choose the Laplacian kernel function. However, in some ways, the spike train which is expressed by the simple kernel function has the linear effect on the postsynaptic neurons, therefore, we propose nonlinear STK which has more biological interpretability. As distinguished from offline learning and batch learning, the learning speed of the online learning is superior to offline learning, because online learning trains one by one according to the order of the spike train, rather than each of iteration would train the entire spike train like the offline learning. So the online learning is more suitable for dealing with problems of large-scale data. This makes it possible that the online learning of spike neural networks can applied to real life. Based on the analysis and summary of advantages and disadvantages of the current spike neural networks online learning algorithms, the main problems we have to solve in the future are as follows: (1) In this paper, the network structure of nonlinear STK only have one layer of network, that is, only the input layer and the output layer, so it can be considered to extend to a multi-layer network. (2) We are only giving a learning of simple spike train, and then we should apply it to solve practical problems later. (3) The research of scholars on the online learning of spike neural networks is very few, so we could consider online learning to be applied to the rules of gradient descent learning. Acknowledgement. The work is supported by the National Natural Science Foundation of China under Grants No. 61165002 and No. 61363059.
References 1. Yegnanarayana, B.: Artificial Neural Networks. PHI Learning Pvt Ltd., New Delhi (2009) 2. Ahmed, F.Y.H., Yusob, B., Hamed, H.N.A.: Computing with spiking neuron networks a review. Int. J. Adv. Soft Comput. Its Appl.s 6(1), 1–21 (2014) 3. Yu, Q., Goh, S.K., Tang, H., Tan, K.C.: Application of precise-spike-driven rule in spiking neural networks for optical character recognition. In: Handa, H., Ishibuchi, K.-C., Ong, H., Tan, K.-C. (eds.) Proceedings of the 18th Asia Pacific Symposium on Intelligent and Evolutionary Systems, vol. 2, pp. 65–75. Springer International Publishing, NewYork (2015) 4. Carlson, K.D., Nageswaran, J.M., Dutt, N., Krichmar, J.L.: An efficient automated parameter tuning framework for spiking neural networks. Front. Neuroscience 8, 10 (2014)
An Online Supervised Learning Algorithm
115
5. Bohte, S.M., Kok, J.N., La Poutre, H.: Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1), 17–37 (2002) 6. Xin, J., Embrechts, M.J.: Supervised learning with spiking neural networks. In: Proceedings of International Joint Conference on Neural Networks, IJCNN 2001, vol. 3, pp. 1772–1777. IEEE (2001) 7. Schrauwen, B., Van Campenhout, J.: Extending spikeprop. In: Proceedings of 2004 IEEE International Joint Conference on Neural Networks, vol. 1. IEEE (2004) 8. McKennoch, S., Liu, D., Bushnell, L.G.: Fast modifications of the spikeprop algorithm. In: International Joint Conference on Neural Networks, IJCNN 2006, pp. 3970–3977. IEEE (2006) 9. Booij, O., Nguyen, T.H.: A gradient descent rule for spiking neurons emitting multiple spikes. Inf. Process. Lett. 95(6), 552–558 (2005) 10. Ghosh-Dastidar, S., Adeli, H.: A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection. Neural Netw. 22(10), 1419–1431 (2009) 11. Xu, Y., Zeng, X., Han, L., Yang, J.: A supervised multi-spike learning algorithm based on gradient descent for spiking neural networks. Neural Networks 43, 99–113 (2013) 12. Hebb, D.O.: The Organization of Behavior: a Neuropsychological Theory. Psychology Press, New York (2005) 13. Caporale, N., Dan, Y.: Spike timing-dependent plasticity: a Hebbian learning rule. Annu. Rev. Neurosci. 31, 25–46 (2008) 14. Ponulak, F., Kasinski, A.: Supervised learning in spiking neural networks with ReSuMe: sequence learning, classification, and spike shifting. Neural Comput. 22(2), 467–510 (2010) 15. Florian, R.V.: The chronotron: a neuron that learns to fire temporally precise spike patterns. PLoS ONE 7(8), e40233 (2012) 16. Mohemmed, A., Schliebs, S., Matsuda, S., Kasabov, N.: Span: spike pattern association neuron for learning spatio-temporal spike patterns. Int. J. Neural Syst. 22(04), 1250012 (2012) 17. Mohemmed, A., Schliebs, S., Matsuda, S., Kasabov, N.: Training spiking neural networks to associate spatio-temporal input–output spike patterns. Neurocomputing 107, 3–10 (2013) 18. Yu, Q., Tang, H., Tan, K.C., Li, H.: Precise-spike-driven synaptic plasticity: learning hetero-association of spatiotemporal spike patterns. PLoS ONE 8(11), e78318 (2013) 19. Wang, J., Belatreche, A., Maguire, L., McGinnity, M.: Online versus offline learning for spiking neural networks: a review and new strategies. In: 2010 IEEE 9th International Conference Cybernetic Intelligent Systems (CIS), pp. 1–6. IEEE (2010) 20. Kasabov, N.K.: Neucube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data. Neural Netw. 52, 62–76 (2014) 21. Oniz, Y., Kaynak, O.: Variable-structure-systems based approach for online learning of spiking neural networks and its experimental evaluation. J. Franklin Inst. 351(6), 3269–3285 (2014)
