An Automatic Image Segmentation Algorithm Based on Spiking Neural Network Model Xianghong Lin1, Xiangwen Wang1, and Wenbo Cui2 1
School of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China 2 Linyi Branch Corporation, China Telecom, Linyi 276000, China
[email protected]
Abstract. Inspired by the structure and behavior of the human visual system, an automatic image segmentation algorithm based on a spiking neural network model is proposed. At first, the image pixel values are encoded into the timing of spikes of neurons using the time-to-first-spike coding strategy. Then the segmentation model of spiking neural networks is applied to generate the matrix of spike timing for the visual image. Finally, using the maximum Shannon entropy as the fitness function of genetic algorithm, the evolved segmentation threshold is obtained to segment the visual image. The experimental results show that the method can obtain the optimum segmentation threshold, and achieve satisfactory segmentation results for different images. Keywords: spiking neural network, image segmentation, genetic algorithm, segmentation threshold, maximum Shannon entropy.
1
Introduction
Humans show a remarkable ability to extract salient information from visual inputs. This has inspired researchers to develop new computational visual models to help understand the biological mechanisms that underlie image processing in the brain. The process of segmenting images is one of the most critical ones in automatic image analysis whose goal can be regarded as to find what objects are presented in visual images. The project described in this paper discusses image segmentation in the visual system and aims to build a computational model based on spiking neural networks (SNNs). SNNs are artificial abstract models of different parts of the brain or nervous system, featuring essential properties of these systems using biologically realistic models. They are often referred to as the third generation of neural networks in which computation is performed in the temporal domain and relies on the timings between spikes, and have potential to solve problems related to biological stimuli [1], [2]. In general a spiking neuron operates by integrating spike responses from presynaptic neurons and generating an output spike when the membrane potential reaches a threshold value. In recent years, researchers used SNNs to segment visual images, and achieved better results [3], [4], [5]. Meftah et al. [6] used the SNN based on spike response D.-S. Huang et al. (Eds.): ICIC 2014, LNCS 8588, pp. 248–258, 2014. © Springer International Publishing Switzerland 2014
An Automatic Image Segmentation Algorithm Based on Spiking Neural Network Model
249
model to segment image and detect edge. The image pixel values are encoded into spike sequences using the population coding strategy in the input layer, then the unsupervised or supervised learning algorithm is applied to adjust the connection weights in the network, and the results of image segmentation or edge detection are obtained in the output layer. Soni et al. [7] proposed a SNN model which is constituted of the leaky integrate-and-fire neurons to segment gray-scale images, and clustering approach is used for the segmentation of visual images. After the selection and the extraction of the image features, the feature samples are grouped together in compact but well-separated clusters corresponding to each class of the image. Similar work was done in [8]. However, the result of image segmentation is encoded by the spike firing rate. Wu et al. [9] used integrate-and-fire neuron network model for color image segmentation. In their work, the network is constructed in the two parts. The first part is a SNN which extract color features through different ON/OFF pathways. The second part is a BP neural network which is trained to recognize the color features and segment the visual image. In essence, the approach is still based on the traditional rate coding neural networks for image segmentation. In this paper, we propose an automatic image segmentation algorithm based on a SNN model. The characteristics of this image segmentation method are: (1) the SNN is constructed in the three layers using the leaky integrate-and-fire neuron model, the input and output of the entire network adopts the time-to-first-spike coding strategy. That is, the image pixel value is encoded into the timing of spike in which each neuron fire only one spike, the information encoding of images is more close to the basic physiology of human visual nervous system [10], [11]; (2) the effect of image segmentation depends on the parameters of the network model, especially adaptive selection of the segmentation threshold. So that, the genetic algorithm can be applied for choosing the optimum segmentation threshold to achieve the better accuracy in image segmentation.
2
Spiking Neural Network Model for Image Segmentation
2.1
Spike Coding Method of Image Pixel
It is well known that biological neurons use pulses or spikes to encode information. Neurological research shows that the biological neurons store information in the timing of spikes [12]. SNN cannot directly calculate analog, so we first need to encode the analog value to timing of spike and input it into spiking neuron network, then output the calculated results in the spike mode. The time-to-first-spike coding strategy is an information encoding method of converting the analog value to spike timing, it assumes that the neurons fire only one spike, and the timing of spike is proportional to the analog value. The time-to-first-spike coding stresses that the first spike is generated in the neuron which received the stimulation signal [11]. In this paper, the time-to-first-spike coding strategy is applied to SNN for image segmentation, and use a nonlinear function that realizes the corresponding relationship from pixel values to the spike times of the neurons [13]. The conversion between gray value of image pixels and firing time of neuronal spikes using the Sigmoid function is represented as follow:
250
X. Lin, X. Wang, and W. Cui
t=
Tmax 1 + exp (σ (128 − p ) )
(1)
where p is the gray value of pixels in the interval [0, 255], t is the firing time of spikes, σ is the nonlinear coding parameter, Tmax is the maximum firing time, and Tmax=10ms. Fig. 1 shows the diagram of the time-to-first-spike coding strategy as the nonlinear coding parameter σ=0.05. The nonlinear corresponding relationship from pixel values to spike times of the neurons is shown in Fig. 1(a). When the gray values of pixels 25, 128, and 250 are inputted into a neuron, the neuron generate the output spikes at 0.06 ms, 5 ms and 9.98 ms respectively (Fig. 1(b)). The firing time of spikes (ms)
10 25
2.5
8 128
2
6 1.5
4
250
2
0.5 Stimulation signal
0
1
0
51 102 153 204 The gray value of pixels
255
(a) The nonlinear corresponding relationship
0
0
2 4 6 8 The firing time of spikes (ms)
10
(b) The examples of spike coding
Fig. 1. The diagram of the time-to-first-spike coding strategy
2.2
Spiking Neural Network Model
The human visual system can efficiently perform image segmentation and recognition, neuroscientists have found that there are a variety of neuronal receptive fields from simple cells to those cells in the retina and lateral geniculate [14]. On the basis of the receptive field mechanism and the leaky integrate-and-fire neuron which is probably the best-known example of a formal spiking neuron model [15], the threelayer SNN model for image segmentation is shown in Fig. 2. The first layer is lightsensing input layer, all image pixels are converted to spike times using the time-tofirst-spike coding strategy. In the middle layer, each neuron corresponds to a receptive field of the input layer and integrating the spike responses from presynaptic neurons in the receptive field. The third layer is the output layer, each neuron stores the spike timing of the corresponding neuron in the middle layer, and the matrix of spike timing in output layer represents the result of image segmentation through the segmentation threshold.
An Automatic Image Segmentation Algorithm Based on Spiking Neural Network Model
251
Receptive field Spike coding
Input layer
Ai
Threshold potential
Middle layer
Bi
Segmentation threshold
Output layer
Ci
Fig. 2. Spiking neural network model for image segmentation
Firstly, the original image needs to convert the gray values of image pixels to spike times in the input layer, we adopts the nonlinear method of time-to-first-spike coding strategy [13]. Assuming that the size of the original image is W×H, and therefore there are W×H neurons in the input layer, where W and H is the width and height of the image respectively. Each neuron Ai (i=1, 2, ..., W×H) in this layer as the center forms a rectangular receptive field whose size is 3×3. If neuron is located in the edges of the input layer, then not constitutes a complete 3×3 size of the receptive field, and the vacant positions in matrix will not fire spikes. In the middle layer, each neuron Bi receives the corresponding neuronal inputs in the input layer where the neuron Ai is the center of the receptive field, the connection weight wij between each neuron Rj (j=1, 2, …, 3×3) in the receptive field and the neuron Bi in the middle layer is calculated as follows:
F A −F R 2 X ( Ai ) − X ( R j ) ( i) ( j) + wij = Wmax exp − d d
2
(2)
where Wmax is the interaction strength or the weight scale between neurons, in our simulation Wmax=10 mV. F(Ai) and F(Rj) are the pixel gray values of the neuron Ai and Rj respectively, ‖F(Ai)‒F(Rj)‖ is the difference of the pixel gray values. X(Ai) and X(Rj) are the pixel coordinate of the neurons respectively, ‖X(Ai)‒X(Rj)‖ is the Euclidean distance between the pixel coordinates. d is a constant and we set d=3. After the firing times of spikes and the connection weights between the neurons are known, the membrane potential of neuron Bi can be calculated. The spike sequence in the receptive field is transmitted to neuron Bi according the order of timing of spikes. When the membrane potential reaches the threshold potential, the first spike is
252
X. Lin, X. Wang, and W. Cui
generated, and ignore the rest of spike inputs. If all spikes are inputted to the neuron Bi but the membrane potential of the neuron Bi still have not reached the threshold potential, it means that the neuron do not fire spike, the spike time of this neuron is represented as +∞. Each neuron Ci in the output layer corresponds to the neuron Bi in the middle layer. In this layer we set a segmentation threshold Tthresh, when the spike time of the neuron in the middle layer exceeds the threshold value, the neuron Ci do not fire spike, and labeling the corresponding pixel gray value as 255. When the firing time of the middle layer neuron does not exceed the threshold value, the neuron Ci fire a spike and labeling the corresponding pixel gray value as 0. So that, according to the segmentation threshold Tthresh, the segmented binary image can be obtained.
3
Automatic Image Segmentation with Genetic Algorithm
Genetic algorithm [16] is a random algorithm that imitating the biological group evolution. It starts from a population of potential solutions of the representative problems. A population consists of a certain number of genes encoding individuals, each individual is an entity with a characteristic chromosomal. At beginning we need to achieve the mapping of phenotype to genotype, namely genetic coding. In this paper we choose the binary coding, that is chromosome genetic code consisting of binary strings as the operation object of genetic algorithm. For the original image, we can obtain the spike timing matrix of the image using the SNN image segmentation model. That is, the matrix stores the firing times of spikes of all neurons in the middle layer. Then, a certain number of individuals generated randomly constitute the initial population, decoding each individual in the population can get the segmentation threshold. Using the segmentation threshold for the spike timing matrix, the binary image can be obtained. For the segmented binary image, we evaluate the fitness using the fitness function. Applying the selection operator based on the individual fitness, and crossover and mutation operators, the new population can be reproduced. Therefore, the segmentation threshold is evolved, and ultimately get the ideal segmented image. The SNN image segmentation process with genetic algorithm is shown in Fig. 3. For each generation, the individuals in the population are decoded to generate segmentation threshold, and they are used to the matrix of firing times to get the segmented binary image. According to the Shannon entropy [17], the fitness of the individual is calculated as follow:
Fitness = − P0 log 2 P0 − P1 log 2 P1
(3)
where P0 and P1 are the probability that the pixel gray values of segmented binary image are 0 and 255 respectively. For most images, Shannon entropy represents the amount of information of the image. If the Shannon entropy of the segmented image is larger, the segmented image will get the greater amount of information from the original image, better result is obtained for high detailed image, and the overall segmentation effect will be better accuracy.
An Automatic Image Segmentation Algorithm Based on Spiking Neural Network Model SNN image segmentation model
Genetic algorithm operation
Original image
Initialize population
SNN image segmentation
Fitness evaluation
Spike Timing Matrix
Selection operator
Segmented binary image
Crossover and mutation operators
Yes
253
Meet the optimization criteria No
Final segmentation result
Generate new population
Fig. 3. The automatic image segmentation with genetic algorithm Table 1. The SNN image segmentation with genetic algorithm (1) Apply the SNN image segmentation model to original image with the size of W×H, and obtain the spike timing matrix whose size is W×H. (2) Initialize a segmentation threshold population with a certain size (popsize). (3) For each segmentation threshold in the population do the follow: (a) Divide the spike times in the matrix into two categories by the segmentation threshold that decoding from the individuals in the population and output the binary image. (b) Evaluate this binary image and obtain the fitness according to formula (3). (4) Repeat until the new segmentation threshold population with popsize is generated. (a) Select two individuals in the population by the tournament selection. (b) Apply single-point crossover operator according to a certain probability to generate new individuals. (c) Apply single-point mutation operator according to a certain probability. (5) Replace the current population with the new generated population. (6) If the termination condition is satisfied, end the loop; otherwise, go to step 3.
254
X. Lin, X. Wang, and W. Cui
Therefore, generating the next generation individuals under the influence of genetic operators, the specific evolutionary processes are: (1) according to the fitness of the individual, select two individuals in the population by the tournament selection method; (2) for the two parent individuals, using the single-point crossover and the single-point mutation operators [18], the two new individuals are generated. In addition, our evolutionary algorithm uses elitist strategy where the elite ratio is 20%. Table 1 shows the steps of the SNN image segmentation with genetic algorithm.
4
Experimental Results
In our experiments, the resting potential value of the leaky integrate-and-fire neuron model is -70 mV, the threshold potential value is -55 mV, the membrane time constant is 20 ms. Comparing the coding methods in [13], we found that the method of non-linear coding is easier to select optimal parameters, and acquires the better segmentation result of image. Therefore, we used nonlinear coding method and set σ=0.05, and the gray value of pixels corresponds to the maximum firing time of spikes Tmax=10 ms. To test the effect on the adaptive selection of segmentation threshold, the population size is set to 20, and each evolution iterates 100 generation. The genetic algorithm executes 50 times, crossover rate is 0.5, and mutation rate is 0.05. Reference images use the Lena image with size of 256×256 and the camera man image with size of 256×256. Firstly we evolve the segmentation threshold of the Lena image, and finally achieve the optimal segmentation threshold Tthresh=7.713077, and the corresponding Shannon entropy is 0.99997, in [13] where the segmentation threshold is artificially selected, the Shannon entropy is 0.99992. Moreover, we evolve the segmentation threshold of the camera man image, and finally achieve the optimal segmentation threshold Tthresh=6.657836, and the corresponding Shannon entropy is 0.99998, in [13] where the segmentation threshold is artificially selected, the Shannon entropy is 0.999894. From the experimental results we find that we can achieve the better accuracy in image segmentation by evolving the parameter of segmentation threshold. Compared with the artificial multiple test method for obtaining segmentation threshold, the new algorithm is a high efficient and automatic image segmentation method. Fig. 4 shows the fitness change curve of the segmented image using genetic algorithm. It can be seen in Fig. 4(a) that the Shannon entropy of segmented Lena image gradually become larger, on average the genetic algorithm can converge and achieve the optimal fitness at around 20th generation, that is to get the maximum Shannon entropy of the segmented image. The average fitness value in general is also showing a rising trend. Fig. 4(b) shows the fitness change curve of the camera man image. The change trend is similar to Fig. 4(a). On average, the genetic algorithm can converge and achieve optimal segmentation threshold at around 17th generation. Fig. 5 shows the results of Lena image segmentation in an evolution process of genetic algorithm. Fig. 5(a) is the original Lena image. Fig. 5(b) is the optimal segmentation threshold Tthresh=6.475695 at the 5th generation, the Shannon entropy of the segmented image is 0.9679625. Fig. 5(c) is the optimal segmentation threshold Tthresh=7.148962 at the 10th generation, the Shannon entropy of the segmented image is 0.982473. Fig. 5(d) is the optimal segmentation threshold Tthresh=7.713077 at the 50th generation, the Shannon entropy of the segmented image is 0.99997. No matter the Shannon entropy or directly observed from the segmented image, it can be seen clearly that the result of Fig. 5(d) is better than one of Fig. 5(b) and Fig. 5(c).
An Automatic Image Segmentation Algorithm Based on Spiking Neural Network Model
1.0
1.0 Optimal fitness Average fitness
Optimal fitness Average fitness
0.96
Fitness
0.96
Fitness
255
0.92
0.92
0.88
0.88 0.84
0.84 0
20
40 60 Generation
80
(a) Lena image
100
0
20
40 60 Generation
80
(b) Camera man image
Fig. 4. The fitness change curve of the image segmentation
(a) Original image
(c) 10th generation, Tthresh=7.148962
(b) 5th generation, Tthresh=6.475695
(d) 50th generation, Tthresh=7.713077
Fig. 5. Results of Lena image in an evolutionary process
100
256
X. Lin, X. Wang, and W. Cui
(a) Original image
(c) 15th generation, Tthresh=6.436875
(b) 5th generation, Tthresh=5.013658
(d) 45th generation, Tthresh=6.6578
Fig. 6. Results of camera man image in an evolutionary process
Fig. 6 shows the results of camera man image segmentation in an evolution process of genetic algorithm. Fig. 6(a) is the original camera man image. Fig. 6(b) is the optimal segmentation threshold Tthresh=5.013658 at the 5th generation, the Shannon entropy of the segmented image is 0.976646. Fig. 6(c) is the optimal segmentation threshold Tthresh=6.436875 at the 15th generation, the Shannon entropy of the segmented image is 0.998953. Fig. 6(d) is the optimal segmentation threshold Tthresh=6.657836 at the 45th generation, the Shannon entropy of the segmented image is 0.99998.
5
Conclusions
In this paper, we propose an automatic image segmentation algorithm based on a SNN model, in which the key parameter of segmentation threshold is evolved using genetic algorithm. The experimental results show that the optimum segmentation threshold is obtained rapidly through the evolution process, the maximum Shannon entropy corresponding to the segmented image, namely the ideal segmentation effect is achieved. It should be noted that in this paper we only select segmentation
An Automatic Image Segmentation Algorithm Based on Spiking Neural Network Model
257
threshold adaptively in the SNN image segmentation model, the main reasons are that: (1) segmentation threshold is the key parameter of the image segmentation model, select the optimal value by its evolution process, the segmented image with the maximum Shannon entropy can be achieved; (2) encoding the image pixel values into spike times, the corresponding spike timing matrix can be obtained by inputting the spikes into the SNN, it needs to be executed only once. Each evolved segmentation threshold in the genetic algorithm is applied only on the spike timing matrix, thus it is to improve the image segmentation efficiency. Acknowledgement. The work is supported by the National Natural Science Foundation of China under Grants No. 61165002 and No. 61363059.
References 1. Ghosh-Dastidar, S., Adeli, H.: Spiking neural networks. International Journal of Neural Systems 19(4), 295-308 (2009) 2. Paugam-Moisy, H., Bohte, S.M.: Computing with spiking neuron networks. In: Rozenberg, G., Bäck, T., Kok, J.N. (eds.) Handbook of Natural Computing, pp. 335-376. Springer Berlin Heidelberg (2012) 3. Wu, Q.X., McGinnity, M., Maguire, L.P., Belatreche, A., Glackin, B.: Processing visual stimuli using hierarchical spiking neural networks. Neurocomputing 71(10-12), 20552068 (2008) 4. Borisyuk, R., Kazanovich, Y., Chik, D., Tikhanoff, V., Cangelosi, A.: A neural model of selective attention and object segmentation in the visual scene: An approach based on partial synchronization and star-like architecture of connections. Neural Networks 22(56), 707-719 (2009) 5. Finger, H., König, P.: Phase synchrony facilitates binding and segmentation of natural images in a coupled neural oscillator network. Frontiers in Computational Neuroscience 7, (2013) 6. Meftah, B., Lezoray, O., Benyettou, A.: Segmentation and edge detection based on spiking neural network model. Neural Processing Letters 32(2), 131-146 (2010) 7. Soni, C., Meftah, B., Khurshid, A.A.: Image segmentation using leaky integrate and fire model of spiking neural network. International Journal of Wisdom Based Computing 2(1), 21-28 (2012) 8. Buhmann, J.M., Lange, T., Ramacher, U.: Image segmentation by networks of spiking neurons. Neural Computation 17(5), 1010-1031 (2005) 9. Wu, Q.X., McGinnity, T.M., Maguire, L., Valderrama-Gonzalez, G.D., Dempster, P.: Colour image segmentation based on a spiking neural network model inspired by the visual system. In: Huang, D.S., Zhao, Z., Bevilacqua, V. (eds.) Advanced Intelligent Computing Theories and Applications, pp. 49-57. Springer Berlin Heidelberg (2010) 10. Thorpe, S., Fize, D., Marlot, C.: Speed of processing in the human visual system. Nature 381(6582), 520-522 (1996) 11. Wysoski, S.G., Benuskova, L., Kasabov, N.K.: Brain-like system for audiovisual person authentication based on time-to-first spike coding. Computational Modeling and Simulation of Intellect: Current State and Future Perspectives 384, (2011) 12. Bohte, S.M.: The evidence for neural information processing with precise spike-times: A survey. Natural Computing 3(2), 195-206 (2004)
258
X. Lin, X. Wang, and W. Cui
13. Cui W.B., Lin X.H., Xu M.Y.: Coding method of spiking neural networks for image segmentation. Computer Engineer 38(24), 196-199 (2012) 14. Izhikevich, E.M., Piekniewski, F., Nageswaran, J., Petre, C., Richert, M., Sokol, S., Szatmary, B.: Spontaneous emergence of simple and complex receptive fields in a spiking model of V1. BMC Neuroscience 14(Suppl 1), O6 (2013). 15. Burkitt, A.N.: A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input. Biological Cybernetics 95(1), 1-19 (2006) 16. Sivanandam, S.N., Deepa, S.N.: Introduction to genetic algorithms. Springer Berlin Heidelberg (2008) 17. Barbieri, A.L., De Arruda, G.F., Rodrigues, F.A., Bruno, O.M., Costa, L.D.F.: An entropy-based approach to automatic image segmentation of satellite images. Physica A: Statistical Mechanics and its Applications 390(3), 512-518 (2011) 18. Holland, J.H.: Outline for a logical theory of adaptive systems. Journal of the Association for Computing Machinery 9(3), 297-314 (1962)
