A CNN Based Algorithm for the Automated Segmentation of Multiple

A CNN Based Algorithm for the Automated Segmentation of Multiple Sclerosis Lesions Eleonora Bilotta1 , Antonio Cerasa2 , Pietro Pantano1, Aldo Quattrone2 , Andrea Staino1 , and Francesca Stramandinoli1 1

2

Evolutionary Systems Group, University of Calabria, 87036 Arcavacata di Rende, Cosenza, Italy {bilotta,piepa}@unical.it, {andreastaino,francescastramandinoli}@gmail.com Neuroimaging Research Unit, Institute of Neurological Sciences, National Research Council, 88100 Catanzaro, Italy {a.cerasa,a.quattrone}@isn.cnr.it

Abstract. We describe a new application based on genetic algorithms (GAs) that evolves a Cellular Neural Network (CNN) capable to automatically determine the lesion load in multiple sclerosis (MS) patients from Magnetic Resonance Images (MRI). In particular, it seeks to identify in MRI brain areas affected by lesions, whose presence is revealed by areas of lighter color than the healthy brain tissue. In the first step of the experiment, the CNN has been evolved to achieve better performances for the analysis of MRI. Then, the algorithm was run on a data set of 11 patients; for each one 24 slices, each with a resolution of 256 × 256 pixels, were acquired. The results show that the application is efficient in detecting MS lesions. Furthermore, the increased accuracy of the system, in comparison with other approaches, already implemented in the literature, greatly improves the diagnosis for this disease. Keywords: Cellular Neural Networks, Genetic Algorithms, Automated Magnetic Resonance Imaging Analysis, Multiple Sclerosis lesion load.

1

Introduction

Cellular Neural Networks (CNNs), first introduced by Leon O. Chua and Lin Yang [3] in 1988, are an array of nonlinear programmable analog processors, called cells, that perform parallel computation. Each cell is a dynamical system whose state evolves in time, according to a specific mathematical model, and whose output is a nonlinear function of the state. Unlike artificial neural networks, in a CNN interconnections among cells are local, that is each processing unit directly interacts only with the neighboring cells, located within a prescribed sphere of influence. For image processing purpose, the most usual CNN architecture is a regular two dimensional grid. Given a CNN of M × N cells, the neighborhood Sij (r) of radius r ≥ 0 for the cell Cij is the set of cells satisfying the following property: Sij (r) = {Ckl : max (|k − i|, |l − j|) ≤ r}

1 ≤ k ≤ M, 1 ≤ l ≤ N

C. Di Chio et al. (Eds.): EvoApplications 2010, Part I, LNCS 6024, pp. 211–220, 2010. c Springer-Verlag Berlin Heidelberg 2010 

(1)

212

E. Bilotta et al.

In the original model [3], each CNN cell is a simple nonlinear analog circuit (Fig. 1), composed of a linear capacitor, an independent current source, an independent voltage source, two linear resistors and at most 2m linear voltagecontrolled current sources, m being the number of neighbors cells of the considered unit. The voltage vxij (t) across the capacitor is called the state of the cell Cij , while vuij and vyij (t) represent the input and the output respectively. The characteristics of the generators Ixy (i, j; k, l; t) and Ixu (i, j; k, l) are defined as: Ixy (i, j; k, l; t) = A(i, j; k, l)vykl (t),

Ixu (i, j; k, l) = B(i, j; k, l)vukl

(2)

By setting the coupling parameters A(i, j; k, l) and B(i, j; k, l), it is possible to control the strength of interactions between cells. The output vyij (t) is determined by the nonlinear voltage controlled current source Iyx that is the only nonlinear element of the cell. It is characterized by the following equation: Iyx =

 1  f vxij (t) Ry

(3)

where f is the characteristic function of the nonlinear controlled current source, defined as:  1 |vxij (t) + 1| − |vxij (t) − 1| (4) f (vij (t)) = 2 Using the Kirchhoff laws, the state of a CNN cell can be described by the following nonlinear differential equation: C v˙ xij (t) = −

1 vx (t) + z + Rx ij



(A(i, j; k, l)f (vxkl (t)) + B(i, j; k, l)vukl ) (5)

Ckl ∈Sij (r)

where f holds the nonlinearity. Therefore, given input, initial state for each cell Cij such that 1 ≤ i ≤ M , 1 ≤ j ≤ N , and boundary conditions, the dynamics of a two-dimensional standard CNN are uniquely specified by the synaptic weights between a cell and its neighbors. These parameters, together with a bias value z, define a CNN template that can be expressed in the form {A(i, j; k, l), B(i, j; k, l), z}. The process performed by the system on the input image is fully defined by the set of coefficients in the CNN template. An evolutionary approach can be used in order to find a template that allows to obtain a desired process. In [2], CNNs are proposed as a parallel computing paradigm especially suited for processing analog array signals, with important applications in image processing, pattern recognition, numerical solution of PDEs and investigation of nonlinear phenomena. CNNs have been successfully applied in various image processing applications, especially because of the high pay-off offered by the CNN based architectures [1]. Neuroimaging is one of the most important area in which CNNs were also used in order to support medical diagnosis, both with magnetic resonance imaging (MRI) and computed tomography (CT) [5][8]. In the last years, there has been increased interest in developing novel techniques for automated multiple sclerosis (MS) lesions segmentation. MS is a demyelinising disease of the central nervous system that leads to inflammatory pathology. MS

A CNN Based Algorithm for the Automated Segmentation of MS Lesions

213

Fig. 1. Original CNN cell model

pathology is primarily expressed as focal lesions in the white matter of the brain. Because of its superior contrast, MRI is the modality of choice for clinical evaluation of MS. Generally, manual delineation of MS lesions is a time-consuming task, as three-dimensional information, from several MR contrasts, must be integrated. In [5], a CNN based approach to classify MR images with respect to the presence or absence of mesial temporal sclerosis has been proposed, using a genetic algorithm-based learning procedure in order to optimize the networks’ parameters, concerning the assigned classification task. By developing a new and efficient CNN based approach of MR images processing and by using a genetic algorithm which improves the technique developed in [5], this paper presents a fully automated method for the segmentation of MS lesions. The paper is organized as follows. After the introduction, the second section deals about the GA formal aspects to evolve the CNN templates. The third section reports about the CNN simulation to test the system’s implementation. The fourth and the fifth sections present the experiments we have performed and some results. The final remarks close the work.

2

GA Methods for Evolving CNN

From the work of J.H. Holland in 1975, Genetic Algorithms (GAs) are computational bio-inspired methods for solving problems. To evaluate the performance of each individual in relation to the problem, it is possible to define an appropriate fitness function, which quantitatively measures the performance of each individual, in a given generation, for all the generations [1]. The standard method for developing a GA is to choose a genetic representation, a fitness function and then it proceeds with the following steps: 1. Generating a random number of strings (initial population), that encode possible solutions to the problem. 2. Decoding of the genotypes of the population and assessment of each individual (phenotype), according to the fitness function. 3. If the current population contains a satisfactory solution, the algorithm stops. 4. If the system doesn’t find a “good” solution, a new evolution starts, generating a new population of individuals, by applying the operators of selection, crossover and mutation.

214

E. Bilotta et al.

Fig. 2. Marked areas corresponding to MS lesions in the white matter of the brain. Lesions are brighter than other tissues on MRI.

The process continues with the evaluation of new individuals through the fitness function and continues cyclically in this manner until a satisfactory solution to a given problem is obtained. GAs and, more generally, evolutionary computing, have been successfully applied to image processing tasks related to medical images classification [10]. In this paper, genetic algorithms have been applied in order to evolve a CNN, capable of detecting MS lesions from MRI. The main issue is to develop a CNN algorithm for automated segmentation of MS lesions, whose presence is revealed by regions in the brain that are brighter than their surroundings (Fig. 2). In image processing applications, a neighborhood of radius r = 1 is commonly used and, in most cases, space-invariant templates are chosen, that is the operators A(i, j; k, l) and B(i, j; k, l) depend only on the relative position of a cell with respect to its neighbors. With such assumption, the whole system is characterized by a 3 × 3 feedback matrix A, a 3 × 3 control matrix B and a scalar z and so 19 parameters are needed to “program” a cellular neural network; this means that, once initial state and boundary conditions have been assigned, the operation performed by the CNN on a given input image is determined only by 19 real values that completely define the properties of the network. For our aims, to design a genetic algorithm to search the weights of a standard twodimensional space invariant CNN, in which each cell has a radius of influence r = 1, it is convenient to adopt a representation of templates in vector form. To this end, the 19 parameters that define the triple {A, B, z} are arranged in an array consisting of 9 feedback synaptic weights, defining the A matrix, 9 control synaptic weights, defining the B matrix, and the threshold z (Fig. 3). These 19 coefficients represent a gene for the CNN, which is associated with a particular function performed by the network. The genetic algorithm has been designed to get template to be used for image processing applications. For this reason, we chose to impose that the matrices A and B are symmetric with respect to the central element, respectively. In this way, we set the conditions for the stability of the CNN, provided in the complete stability theorem [2], which ensures the convergence of the network. It also reduces the computational load of the algorithmic search, since it is necessary to determine only 11 coefficients, 5 belonging to the matrix A, 5 to the

A CNN Based Algorithm for the Automated Segmentation of MS Lesions

215

Fig. 3. Representation of a CNN template in vector form

matrix B and one corresponding to the threshold z. Each genotype is represented by a vector G of 11 elements: G = [a11

a12

a13

a21

a22

b11

b12

b13

b21

b22

z]

(6)

while the corresponding phenotype is the actual cellular neural network, configured by using the parameters in the genotype. The key step in the genotype-tofenotype mapping is the construction of the CNN template {A, B, z} from the elements of a given vector G; this can be easily accomplished by re-arranging genotype coefficients as shown in Fig. 3. To assess the fitness of a CNN gene compared to an assigned problem, we introduce a target image T of M × N pixels to be used for training the network. Applying the template corresponding to G to the input image to CNN, it generates an image I G which can be compared with T , through the cost function: dif f (G) =

M  N 

G Iij



Tij

(7)

i=1 j=1

where the operator ⊕ denotes the logic xor between the element in position (i, j) of the target image and the corresponding pixel in the CNN output. The fitness function for each phenotype CN N G , then, is evaluated by calculating the number of pixels equal between T and the CNN output: f itness(CN N G) = M × N − dif f (G)

(8)

Hence, the fitness measures the number of equal pixels between the target image and that obtained from the CNN simulation. In this way, higher values of fitness are associated with phenotypes corresponding to templates that produce outputs with a high number of pixels, that in turn coincide with the image target.

3

Experiments on the CNN Performance

The algorithm proposed for the segmentation of the MS lesions consists of three principal steps:

216

E. Bilotta et al.

(a)

(b)

Fig. 4. Input (a) and target (b) image used in the CNN training

– Step 1: Lesions detection and their binarization, – Step 2: Segmentation of the white matter of the brain, – Step 3: Lesions extraction and isolation. Genetic algorithms have been applied to determine the CNN synaptic weights that, for a given MR Image, perform a binarization in such a way that only MS lesions are detected inside the brain area. The subsequent steps have been necessary in order to remove the skull and other unwanted pixels. Because of different shapes and intensity of the lesions, it has been necessary to train the genetic algorithm on images presenting different characteristics; for this reason, the network was evolved using Fig. 4(a) as input and Fig. 4(b) as the corresponding desired output. The evolution of the CNN has been carried out using CNNSimulator, a software environment for the analysis of CNN dynamics; at each step of the training process, the error function to be minimized by the GA was the number of different pixels between the desired and the actual output of the CNN. In our implementation, we ran an initial random population of 35 individuals, making them evolve for 300 generations; the number of individuals was kept constant during the evolutionary process, weighted roulette wheel selector was used as selection method, mutations and elitism strategies were applied. In order to reduce the computational effort due to the large search space, we chose to constrain the elements of each genotype to be in the range [−8, 8]. The GA was conducted as follows: after evaluating the fitness of each phenotype, the elite individual, i.e. the most performant one, has been directly copied in the next generation; a number of single-point crossover operations, depending on the population size, has been performed. In our experiments, we used a crossover percentage of 30%, meaning that the number of genetic crossing over operations has been 0.3 multiplied by the population size. Mutations have been randomly applied in order to prevent trapping into local minima. The elements of the genotypes in the population have been randomly mutated according to a given mutation rate, that is each coefficient had a given probability of being changed by a randomly selected real number that falls in the chosen interval [−8, 8]. Using a mutation rate of 0.05, each component had 5% probability of being changed, resulting in 1/20 parameters being mutated on average. Once genetic operators have been applied, a fixed number of genotypes has been selected and moved on the next generation population. Obviously, the selection has been guided by the fitness,

A CNN Based Algorithm for the Automated Segmentation of MS Lesions

217

Fig. 5. Evolution of the Cellular Neural Network

i.e. higher probabilities of survival have been associated to phenotypes providing higher fitness values. At the end of the training process, the following template matrices were found: ⎡

⎤ −3.51879 3.42019 −3.48386 ⎣ 6.47032 7.75293 6.47032 ⎦ A= −3.48386 3.42019 −3.51879

⎡

⎤ 1.33076 −3.86887 1.53728 ⎣ B = −2.30849 −7.76398 −2.30849⎦ 1.53728 −3.86887 1.33076

z = −4.81797

(9)

Figure 5 shows the output generated by the CNN corresponding to the highest fitness value, together with the fitness achieved by the best individual in each generation. The removal of the skull and other unwanted features has been achieved by an AND operation between the output of the evolved CNN and the corresponding white matter of the brain, segmented in the second step of the algorithm. It gave the MS lesions in output, while the remaining parts have been removed. We used SPM8 [7] for white matter segmentation, while greyscale image binarization and logic AND operation could be easily performed, by using the templates proposed in the CNN software library [6]. The third step of the proposed algorithm is shown in Fig. 6. Once the pixels corresponding to the lesions have been extracted for each slice of a given MS patient, knowing the voxel size in the acquired MRI sequence, it is possible to perform a quantitative evaluation of the MS total lesion load (TLL). The performances of the process have been quantitatively evaluated by comparing the CNN output and the expert’s manual delineation of MS lesions, using the Dice coefficient [4] as a metric. The Dice coefficient D is a statistic measure used for comparing the extent of spatial overlap between two binary images. It is commonly used in reporting performance of segmentation and its values range between 0 (no overlap) and 1 (perfect agreement). In this paper the Dice values, expressed as percentages, are computed as follows:



2 LCN N ∩ LG

× 100 (10) D = CN N |L | + |LG | where LCN N is the automated segmentation result and LG the manual one. We applied the algorithm to a data set of real MR images acquired from 11 MS patients, for whom 24 slices were taken to cover the whole brain. Each slice

218

E. Bilotta et al.

Fig. 6. Skull and unwanted features removal

had a resolution of 256 × 256 pixels and voxel size is 0.94mm × 0.94mm × 5.00mm. The simulation results showed that the CNN based system is effective in segmenting MS lesions in fast fluid-attenuated inversion-recovery (FLAIR) axial images.

4

Results

The exposed method gives satisfactory results, showing that after the learning process the cellular neural network is capable of detecting MS lesions with different shapes and intensities, even in MRI slices with different contrasts between white and grey matter with respect to the images used during the genetic training process. The vast majority of the lesion load, detected by CNN for the described sample, ranges from D = 0.7 to D = 0.8. The technique we propose for segmenting white matter lesions in MS is a fully automatic method and does not require manually segmented data; in fact, while semiautomatic methods [9] are highly dependent on the choice of an appropriate threshold to effectively detect lesions (threshold that usually may vary between different slices even for the same patient, thus leading to a time consuming task), our algorithm allows for obtaining the desired output by programming a fully automated strategy on the entire data set, without the need of external calibration. Simulations have allowed to verify the validity of the above described algorithm. The output generated by the CNN can be viewed in MRIcro medical image viewer (www.mricro.com), as shown in Fig. 7. Calculating the number of pixels corresponding to the injury

A CNN Based Algorithm for the Automated Segmentation of MS Lesions

219

Fig. 7. Results of lesion-CNN segmentation on one of the MS patients

Fig. 8. A 3D reconstruction of lesion-CNN segmentation results. 3D reconstruction shows the typical spatial dissemination of lesions affecting white matter in MS.

and knowing the size of the voxel which scanning uses, it is possible to estimate the TLL for any patient. This method provides an important parameter to monitor the progress of the pathological disease. It should be emphasized that the results (Fig. 7) were obtained without changing the template from one slice to another. In fact, no manual thresholding is required during the segmentation process. Obviously, operating with an ad hoc manual calibration of the network on any single slice, the algorithm is able to produce more precise results. Overlapping the slices and the output of CNN, it is possible to obtain a 3D reconstruction of the brain of the patient (Fig. 8), which displays the region of the brain tissue that presents multiple sclerosis.

5

Conclusions

In this paper we have presented an innovative CNN based method for automatically detect MS lesions. The results obtained by applying the proposed algorithm

220

E. Bilotta et al.

have been very convincing, since CNN can determine most of the lesions in all the patients. The system could provide a useful MR support tool for the evaluation of lesions in MS, particularly to assess the evolution of the lesions. From a comparison with other existing methods in the literature on this topic, we can say that the results obtained with our method are effective and the threshold of recognition is currently at 70%. Furthermore, it should be emphasized the real improvement of the proposed method with respect to [5] for the greater accuracy of the system, its adaptation to different conditions of the stimuli, its ability to create 3D images of the injured areas of the brain, thus effectively supporting medical diagnosis.

References 1. Bilotta, E., Pantano, P.: Cellular Non-Linear Networks as a New Paradigm for Evolutionary Robotics. In: Iba, H. (ed.) Frontiers in Evolutionary Robotics, Vienna, Austria, pp. 87–108 (2008) 2. Chua, L.O., Roska, T.: Cellular Neural Networks and Visual Computing: Foundations and Applications. Cambridge University Press, Cambridge (2004) 3. Chua, L.O., Yang, L.: Cellular Neural Networks: Theory. IEEE Transaction on Circuits and Systems 35(10), 1257–1272 (1988) 4. Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26, 297–302 (1945) 5. D¨ ohler, F., Mormann, F., Weber, B., Elger, C.E., Lehnertz, K.: A Cellular Neural Network based Method for Classification of Magnetic Resonance Images: Towards an Automated Detection of Hippocampal Sclerosis. In: Proceedings of the 7th IEEE International Workshop on Cellular Neural Networks and their Applications, pp. 579–586 (2004) 6. Kek, L., Karacs, K., Roska, T.: Cellular Wave Computing Library (Templates, Algorithms and Programs), ver. 2.1. Cellular Sensory Wave Computers Laboratory, Hungarian Academy of Sciences (2007) 7. SPM8 - Statistical Parametric Mapping, http://www.fil.ion.ucl.ac.uk/spm/software/spm8/ 8. Szabo, T., Barsi, P., Szolgay, P.: Application of Analogic CNN algorithms in Telemedical Neuroradiology. Journal of Neuroscience Methods 170(7), 2063–2090 (2005) 9. Valentino, P., Cerasa, A., Chiriaco, C., Nistic` o, R., Pirritano, D., Gioia, M., Lanza, P., Canino, M., Del Giudice, F., Gallo, O., Condino, F., Torchia, G., Quattrone, A.: Cognitive deficits in multiple sclerosis patients with cerebellar symptoms. Multiple Sclerosis 15, 854–859 (2009) 10. V¨ olk, K., Miller, J.F., Smith, S.L.: Multiple Network CGP for the Classification of Mammograms. In: Giacobini, M., Brabazon, A., Cagnoni, S., Caro, G.A.d., Ekart, A., Esparcia-Alc´ azar, A.I., Farooq, M., Fink, A., Machado, P., McCormack, J., O’Neill, M., Neri, F., Preuß, M., Rothlauf, F., Tarantino, E., Yang, S. (eds.) EvoWorkshops 2009. LNCS, vol. 5484, pp. 405–413. Springer, Heidelberg (2009)