Retinal vessel extraction using lattice neural networks with dendritic processing Roberto Vegaa , Gildardo Sanchez-Antea,∗, Luis E. Falcon-Moralesa , Humberto Sossab , Elizabeth Guevarab a Tecnol´ ogico
de Monterrey, Campus Guadalajara Computer Science Department Av. Gral Ramon Corona 2514 Zapopan, Jal, M´ exico b Instituto Polit´ ecnico Nacional-CIC Av. Juan de Dios Batiz S/N, Gustavo A. Madero 07738 M´ exico, Distrito Federal, M´ exico
Abstract Retinal images can be used to detect and follow up several important chronic diseases. The classification of retinal images requires an experienced ophthalmologist. This has been a bottleneck to implement routine screenings performed by general physicians. It has been proposed to create automated systems that can perform such task with little intervention from humans, with partial success. In this work, we report advances in such endeavor, by using a Lattice Neural Network with Dendritic Processing (LNNDP). We report results using several metrics, and comparing against well known methods such as Support Vector Machines (SVM) and Multilayer Perceptrons (MLP). Our proposal shows better performance than other approaches reported in literature. An additional advantage is that unlike those other tools, LNNDP requires no parameters, and it automatically constructs its structure to solve a particular problem. The proposed methodology requires four steps: 1) Pre-processing, 2) Feature compu∗ Corresponding
author Email addresses:
[email protected] (Roberto Vega),
[email protected] (Gildardo Sanchez-Ante ),
[email protected] (Luis E. Falcon-Morales),
[email protected] (Humberto Sossa),
[email protected] (Elizabeth Guevara)
Preprint submitted to Elsevier
November 7, 2014
tation, 3) Classification and 4) Post-processing. The Hotelling T 2 control chart was used to reduce the dimensionality of the feature vector, from 7 that were used before to 5 in this work. The experiments were run on images of DRIVE and STARE databases.The results show that on average, F1-Score is better in LNNDP, compared with SVM and MLP implementations. Same improvement is observed for MCC and the accuracy. Keywords: Pattern recognition, machine vision, blood vessel segmentation, diabetic retinopathy, neural networks, dendritic processing 2000 MSC: 68T10, 68T35, 68T45
1. Introduction Diabetes is rapidly emerging as a global health care problem that threatens to reach pandemic levels by 2030; the number of people with diabetes worldwide is projected to increase from 171 million in 2000 to 366 million by 2030 [1]. Diabetic Retinopathy is considered as the most common diabetic eye disease that affects up to 80% of diabetics and causes blindness in many cases [2]. Arteriosclerosis is considered as the leading cause of death in people over age 45. According to [3] and [4], its prevalence is about 30%. High blood pressure, or hypertension is a condition present in about 26% of the total population [5]. The three above mentioned diseases can be diagnosed and monitored via the retina observation [6]. The retina is the only place in the human body where blood vessels can be directly visualized non-invasively and in vivo [7]. Among the different elements of a retina image, the vascular structure is a very relevant one. The segmentation of blood vessels is a required step for further analysis that allows measuring attributes such as length, width, branching factor and tortuosity. With them, it is possible to diagnose and evaluate the evolution of several ophthalmologic and cardiovascular diseases [8], [9]. Besides, the pattern of the vascular structure of the retina of an individual is unique, 2
making it one of the best choices for biometric systems [10]. Although this segmentation process can be performed manually, it is a long and tedious work that requires experience [11]. It has been suggested that screening programs, in which retinal images are captured and analyzed, should be implemented on a regular basis. This may certainly help to detect some of these complications earlier. However, it also implies an additional workload on specialists not only to get the image, but to analyze it [12], [13]. Nowadays, it is easier to capture retina digital images due to the development of digital ophthalmoscopes. This opens the possibility of performing automated image analysis, helping physicians to accomplish screening processes of their patients. This could also enable other applications for telemedicine. For instance, a nurse or a general doctor could capture the image of a patient in a remote place, the image can be analyzed by the intelligent system and then, a recommendation can be made on whether the patient needs to see an specialist or not. There are basically three kinds of retina images: Color photography, red-free photography, and fluorescent angiograms. In this work, we concentrate on the first kind. Segmentation in retinal images is a challenging task because retinal images usually have low contrast and are highly noisy. This effect is caused by the safety limitations used when obtaining the image, making the boundaries of the blood vessels unclear. Also, small depth of focus in the camera as well as the motion of the eye cause different degrees of blur. On the other side, some vessels are only a few pixels in diameter and some images have substantial pathology, making it difficult to accomplish proper segmentation. Finally, there are lots of vessel crossing and branching, as well as a phenomenon called central vessel reflex which makes difficult to extract this structure [14]. This work reports the utilization of a Lattice Neural Network with Dendritic
3
Processing (LNNDP) to automate the segmentation and extraction of the blood vessels structure from fundus images. As far as we know, this is the first time such kind of neural network is used for this purpose. This is an extended version of [15], where our first results were presented. In this new version, we extend our previous work by: a) Describing the theory behind LNNDP and their different training methods, offering implementation details that were not covered in [15], b) Implementing an adaptive adjustment of parameter called margin (M ) to improve the performance in the training phase, which automates a process that in [15] had to be adjusted manually, c) Presenting a comparison of our approach against other common machine learning algorithms, such as the multilayer perceptron (MLP) and the support vector machine (SVM)[16], d) Including other metrics to compare against recent approaches, and e) We have added data from the DRIVE dataset [9] to show the robustness of LNNDP for this task. The remainder of the paper is organized as follows: Section 2 describes current advances in retinal image processing. Section 3 introduces Lattice Neural Networks with Dendrite Processing. Section 4 presents our methodology, Section 5 describes experiments and results and finally, Section 6 presents the conclusions and future work. 2. Previous Work Although the idea of automating the analysis of eye fundus images has attracted the attention of many research groups during the last years, the problem has still not been completely solved [17], [18]. In general, computed aided diagnosis via medical images is a complex task where there are usually four steps involved: preprocessing, segmentation, classification and recognition. The work reported here is focused on preprocessing and segmentation (via a classification subproblem) of retinal blood vessels. 4
According to [19], the developed methods for vessel detection can be roughly classified into three categories: the edge detection-based, the segmentationbased, and the probing-based. The edge detection-based algorithms use gradient masks to identify edge points between regions in a retinal image. Then, the boundary of the vessel is identified by means of intensity discontinuity. For the segmentation-based methods, a good number of options have been considered, being one of the simplest the use of a single threshold to partition a retinal image into background and vessel. Finally, the probing-based methods utilize a profile model to incrementally step forward along the inspected vessel, and identify vessel boundary during probing. The three most important areas of active research in retinal imaging include: development of cost-effective digital equipment to capture retinal images, development of techniques to study the retina function using oximetry or nearinfrared analysis, and development of image processing and analysis algorithms that allow the classification of retinal images for automated diagnosis. In many cases, a set of features of the retina vascular structure can establish a probable diagnosis. Parameters such as diameter, color, curvature and opacity of blood vessels may serve as a basis for diagnosis, treatment, and monitor of the aforementioned diseases [12], [20]. In [21] a system named STARE was introduced. The name stands for STructured Analysis of the REtina. The authors describe in theory what steps could be required to analyze retina images, although they did not include experimental results. In a different approach, Hoover et al. [22] define a method based on threshold probing. Such method was compared against a matched filter response (MFR), giving an improvement of 15% on the true positive detection rate. According to [9], several other methods have been proposed to solve the
5
problem of blood vessel segmentation. These methods can be divided in two general categories: rule based methods, and supervised learning methods. The rule based methods comprise vessel tracking, matched filter responses, grouping of edge pixels, thresholding methods, and morphology based techniques. In the case of supervised learning methods, the general approach is to use image processing techniques to enhance blood vessels, and then use of a classifier to discriminate between blood vessels and background pixels. Among the most commonly used classifiers are the multilayer perceptron (MLP), K-nearest neighbors, and support vector machines (SVM) [8], [9] and [23]. Some of the open problems in this matter are: vessel segmentation for vessels that have only a few pixels in diameter, vessel segmentation in images with substantial pathology, differentiating arteries from veins, and assessing vessel tortuosity [14]. In an effort to improve the accuracy of the currently available methods, Ramlugun et al. [24], proposed the use of Gabor filters and vessel enhancement by contrast-limited adaptive histogram equalization followed by a double sided thresholding scheme. Addressing the problem of the segmentation of small blood vessels, Li et al. [25] proposed a multi-scale vessel extraction scheme by multiplying the responses of matched filters at three scales. In a work reported by Staal et al. [9], the authors introduce the Primitive-Based Method (PBM). PBM uses image primitives extracted from image ridges that are grouped and given to a classifier. The results were interesting although the authors point out the need to improve certain aspects of the processing to decrease the computational time required. Recently, in [26] the authors use a modified matched filter, called MF-FDOG, which basically extends the original MF by using a zero-mean Gaussian and its first-order derivative. The main advantage of such method is the competitive results it reaches, but with lower complexity.
6
Other authors have been interested in developing classifiers able to distinguish among arteries and veins. For example, in the work by Kondermann [27], the authors performed a comparison of two feature extraction methods and two classifiers based on support vector machines and neural networks. The best combination is able to reach a 95.32% of correct classifications. Color has also been considered as an important feature for segmentation. For example in [13] and in [28]. Those approaches usually detect the dark background and remove noise, then a further step of fine segmentation is applied. The authors report up to 95.43% of correctly classified samples. Approaches based on morphological operations have also been reported [29], [30]. Statistical based methods have been considered, such as linear discriminant analysis [6]. The work reported by Lau et al. [31] considers two steps. One is a segmentation of the vascular structure and the second one is a post processing to identify true vessels based on a graph tracer. The results are very good, reaching up to 98.7% of true vessels classification. In [32], the authors use an SVM to perform the classification of retinal images. The proposed method includes a preprocessing step to remove the optic disk edges, a normalization of the green channel and the computation of wavelets to generate a vector of 12 features. This method falls in the category of semi-supervised learning. In [33] a new segmentation method is proposed. Such method is mainly focused on solving problems related with central reflex pixels, with the error of considering two close pixels as being part of one wide vessel, and disconnected pixels. The method is based on a linear combination of linear detectors at different scales. The main drawback is that the method does not work that well on images of non-healthy retinas. It has trouble with images like the ones contained in the STARE database, due to the presence of dark and bright regions.
7
An excellent, and recent survey of methods for blood vessel segmentation is offered in [11]. According to that paper, regardless of current advances and the big amount on research on this area, there is still room for improvement. Pathological and noisy images are still a challenge, as well as images where the central reflex is present. Moreover, there are few public databases and those are comprised of a very limited number of samples, making complicated to evaluate the methodologies. More research on taking the methods to parallel implementation, and the creation of larger datasets with available ground truths is also an opportunity. Although each published work reports improvement over previous approaches, the methods proposed become more and more complex. Therefore, they might not be completely understood by people who is not an expert in the author’s field. On the other side, the training methods commonly used for supervised learning (Support Vector Machines and Neural Networks, for example) are difficult to understand. We propose the use of standard image processing techniques, along with a powerful, and easy to understand, classifier: Lattice Neural Networks with Dendritic Processing (LNNDP). The training method used for the LNNDP is transparent and it requires no parameter configuration. We consider this as an important aspect when solving medical problems, because more often than not medical experts are not experts in mathematical and computational techniques. It is our hope that this work encourages people to use LNNDP for tackling not only blood vessel segmentation in retinal images, but also other medical problems. 3. Lattice Neural Networks It is possible to state the analysis of retinal images as a classification problem. As such, each pixel can be labeled as blood vessel or non blood vessel. Thus, one possibility is to use artificial neural networks (ANN) as the classifier. The 8
features considered could be the pixel itself as well as other characteristics of its neighborhood. Artificial Neural Networks seem appropriate given its ability to extract the inherent model of a system represented through a set of attributes or features. If images in which blood vessels are hand labeled are provided as training examples, then the network might learn how to classify new images. The most well known models of ANNs are derivations of the perceptron. Those models consider that the processing happens in the body of the neuron, requiring a two-steps process. The first one computes a weighted sum and the second one applies an activation function, that usually is a non-linear model. However, it has been argued that this is not exactly what happens in natural neurons. Thus, new architectures for neural processing have been explored. One of them view the dendrites as the primary basic computational unit of the neuron [34]. It is believed that dendrites perform logical operations such as XOR, AND, and NOT [35] [36]. This idea has given place to a new generation of neural networks: Lattice Neural Networks (LNN) [36], Morphological Neural Networks (MNN) [37] and Spiking Neural Networks (SNN), among others. The absence of dendrites in artificial neural networks presents certain problems that can be avoided by the inclusion of these structures [38]. The main problem that multilayer perceptrons present is their inability to perform reliable pattern rejection. In order to model properly the probability distribution of patterns, the classifier must draw closed separation surfaces on the pattern space. However, Gori and Scarselli proved that when multilayer perceptrons with sigmoidal units have less or equal hidden units than input units, the network creates open surfaces in the pattern space [39]. If the number of hidden units is greater than the number of inputs, the separation surfaces might be closed, but determining it is NP-hard. Lattice Neural Networks exhibit certain interesting properties, such as: it
9
has been observed that there are no convergence problems with a single layer feedforward neural networks and a speeded-up learning process. Also, LNN do not require hidden layers, and are capable of multi-class discrimination, to mention a few [38]. This new model of neural networks is called Lattice Neural Networks with Dendritic Processing (LNNDP). In a traditional artificial neural networks the activation function f receives a linear combination of the weights and the input vector, and a term called “bias”, as given by:
f
n X
! xi ωij − θj
(1)
i=1
where xi denotes the value of the ith neuron, ωij denotes the synaptic strength between the ith neuron and the jth neuron, θj is called bias, and n is the number of neurons. On the other hand, in the case of a Lattice Neural Network, the activation W V function needs to compute the lattice operations (maximum) and (minimum), as well as the sum.
f
pj
n _
! rij (xi + ωij )
(2)
i=1
where rij = ±1 denotes the excitatory or inhibitory pre-synaptic influence of the ith neuron on the jth neuron, and pj = ±1 denotes the post-synaptic response of the jth neuron on the total input received. In lattice algebra the lattice operations maximum or minimum, and addition substitute their corresponding arithmetic operations of addition and multiplication. This substitution allows extremely fast neural computation and easy hardware implementation. With this new paradigm, convergence problems and complex training algorithms are seldom present [34]. As well as with traditional multi-layer perceptron neural networks, there are
10
different training methods for LNNDPs. In this work, we use a modified version of the hyperboxes partition method introduced in [40]. A former version of this modification was presented in [15], and some of the first steps are described again here for the sake of completeness. If we are given a set of n input neurons N1 , . . . , Nn , where n is the number of features in the input vector, m output neurons M1 , . . . , Mm , where m is the number of classes, and a finite number of d = {1, 2, . . .} of dendrites. Each dendrite is the connection point between input neurons and output neurons. The exact number of neurons created by the network will depend on the distribution of the training set. The number of dendrites in each neuron Mj may be different. Figure 2 shows a representation of the LNNDP. In the LNNDP model, a neuron Mj receive information from the input neurons {Ni }i∈I , I = {1, 2, . . . , n} through a finite number of dendrites Djk . Each neuron Ni will have at most two connections on a given dendrite Djk . If two connections are present, then one connection is assumed to provide an inhibitory input, while the other will provide excitatory input. The reason behind having at most two connections on a given dendrite is that the computation of the response of each dendrite is based on maximum and minimum values. Therefore, having more than one excitatory or inhibitory connections will not provide any additional useful information [36]. The weight of the axonal branch of neuron Ni terminating on the dendrite l Djk of Mj is denoted by ωijk , where the superscript l ∈ {0, 1} distinguishes
between excitatory (l = 1, marked as a black dot on Fig. 2), and inhibitory (l = 0, marked as an open dot on Fig. 2) input to the dendrite. The computation of the dendrite Djk of Mj is given by [15]:
τkj (x) = pjk
^
^
i∈I(k) l∈L(i)
11
l (−1)1−l (xi + ωijk )
(3)
where τkj (x) denotes the output of the dendrite Djk ; x = (x1 , . . . , xn ) denotes the input value of the neurons N1 , . . . Nn ; I(k) ⊆ {1, . . . , n} corresponds to the set of all input neurons with terminal fibers that synapse on the k-th dendrite Djk of Mj ; L(i) ⊆ {0, 1} corresponds to the set of inhibitory and excitatory fibers of Ni that synapse on Djk ; and pjk ∈ {−1, 1} denotes inhibitory or excitatory response of the dendrite Djk to the received input. The response of the neuron Mj , τ j (x), is given by the equation proposed in [36]: Kj _
τ j (x) = pj
τkj (x)
(4)
k=1
where Kj represents the total number of dendrites in the neuron Mj ; τkj (x) is the output of the dendrite k of neuron Mj computed using equation 3; pj = 1 to denote that the particular input is accepted; and pj = 0 to denote that the particular input is rejected. The output of the neuron is then determined by the following activation function: 1, if τ j (x) ≥ 0 f τ j (x) = 0, if τ j (x) < 0
(5)
If we take a system that consists only of one input neuron, which receives input x, connected to one output neuron through a single excitatory response (r = 1), and we denote the weight of this connection as ω = −α, α ∈
