Aug 1, 2017 - Network (CNN) to classify fundus retinal images and accurately categorize .... 3) Software: The deep learning package Keras is used to build up our ... conducting research on machine learning and deep neural net- works.
1
Automated Detection of Diabetic Retinopathy Parismita Bharali, Chandrika Bhuyan, Moirangthem T. Singh Department of Computer Science & Engineering, DUIET, Dibrugarh University
Abstract—Diabetic Retinopathy(DR) is a disease which is a growing cause of concern for medical science. Patients get affected by this disease after suffering from Diabetes for a longterm and eventually lose their eyesight completely. The diagnosis of DR, however, is still a long and manual process. In this paper, we make an attempt towards using Convolutional Neural Network (CNN) to classify fundus retinal images and accurately categorize them into five stages of the disease(0-no DR, 1-mild DR, 2-moderate DR, 3-severe DR, 4-proliferative DR). The CNN used in our work promises an impressive performance. Index Terms—Diabetic Retinopathy, Neural Networks, Supervised Learning, Image Classification
I. I NTRODUCTION
D
IABETIC Retinopathy (DR), a disease that leads to vision loss among the diabetic patients all over the world, is currently one of the most important challenges in the medical field. Long-term diabetes causes diabetic retinopathy. Over millions of people in India are affected by Diabetes. Approximately 425 million adults are estimated to be living with diabetes, however, by 2045, the number may even rise to 629 million [6]. With an increase in the number of diabetic patients all over the world, chances of them diagnosed with diabetic retinopathy increases as well. Patients usually do not experience symptoms until late in the course of this disease and delayed treatment is usually ineffective. Regular screening of patients for Diabetic Retinopathy has proved to be an effective and important factor of cure. An early diagnosis of retinopathy can increase the chance of preventing blindness. The current method of DR detection, however, is time-consuming and a manual process which requires a professionally trained clinician to examine
Fig. 1: Normal retina
and evaluate color fundus images of the patient’s retina. According to study, real fundus images are sent to clinicians for grading. By the time a clinician submits the reviews, which often takes a day or two, the delay leads to lost follow up, miscommunication, and delayed treatment. The disease is clinically classified into five stages based on the weighting of numerous features and determining the location of them [1] - Stage0: No DR, Stage1: Mild DR, Stage2: Moderate DR, Stage3: Severe DR and Stage4: Proliferative DR. The diagnosis comprises of detecting the presence of features responsible for the disease and further classifying the patient to its correct stage of severity, which while doing manually usually takes more time than desired. With recent advancements in computers that have developed to perform quick classifications once trained, it has opened ways to help clinicians in real-time. An automatic screening of Diabetic Retinopathy is regarded as a solution to the existing traditional diagnosis, and a few such works using image classification, pattern recognition, and machine learning, in this field, have made good progress. We propose a method of classifying fundus images into its stages of DR by using Deep Convolutional Neural Networks. We train our model with supervised learning on input images that gives the output indicating to which of the five stages of DR is the retina affected. This paper is further organized in the following order. Section II presents an overview of related works, Section III describes the proposed method and shows the training of models used, Section IV presents the results of our experiment, Section V brings in a conclusion of the paper with discussion.
Fig. 2: DR affected retina
2
II. R ELATED W ORK Many advancements are made on methods for classification of DR and these works yield encouraging results. Researchers from all over the world have been constantly working in this area. Several approaches are available in the field of diabetic retinopathy detection and classification. Techniques based on mathematical morphology, neural networks, pattern recognition, region growing techniques, fuzzy C-means clustering are available from the literature [1]. At Center for Artificial Intelligence Technology, Malaysia, Ali Shojaeipour, Md.Jan Nordin and Nooshin Hadavi [3] designed a CAD system for diagnosis of DR by using digital retinal images and classifying them into two categories, which are, eyes with DR and non-DR images (healthy eyes). In this study, the Gaussian filter was used to enhance images and Adaboost algorithm is used to classify the images after extracting vessels, optic disc, and exudate regions. They achieved a system sensitivity of 87.50% with 93.75% accuracy. Gardner, Keating, Williamson, Elliott, [2] at Tennent Institute of Ophthalmology, Glasgow, used Neural Networks and pixel intensity values to achieve sensitivity and specificity results of 88.4% and 83.5% respectively for yes DR and no DR. A small dataset of around 200 images splitting each image into patches was used. Firstly, with the help of a clinician, they classified the patches for features, followed by SVM implementation. At Indian Institute of Technology, Kanpur, Mohit Singh Solanki, [4] in his attempt to find an automated way to detect this disease in its early phase, used supervised learning methods to classify a given set of retinal images into 5 classes of retinopathy. Various image processing techniques and filters were used to enhance important features. The neural networks built for classification gave 55% accuracy on 500 data. Convolutional Neural Networks have also been used to classify fundus images according to the severity of Diabetic Retinopathy. Pratt, Coenen [1] built a network with CNN architecture. Data augmentation was performed to identify the features involved in the classification task such as microaneurysms, exudate, and haemorrhages on the retina. Their method provided a diagnosis automatically without user input. The network was trained on the publicly available Kaggle dataset which achieved 95% specificity, 75% accuracy and 30% sensitivity. Alex Tamkin, Iain Usiri, Chala Fufu [5] at Stanford University also performed classification of retinal images using convolutional neural networks. Their model was trained with transfer learning for binary classification of the images. They extracted the best results using the Inception V3 model used together with ImageNet pre-trained weights in addition to dense layers on top of the model. The investigative research on the available literature on detection and classification of Diabetic Retinopathy reveals a wide scope of deep convolutional neural networks to outperform any traditional method of the same. III. P ROPOSED M ETHODOLOGY We used deep Convolutional Neural Networks in our study to classify retinal images based on presence and absence of
Diabetic Retinopathy. The whole work can be divided into two main parts, the first being a classification of fundus images into the two categories of a retina having DR and not having DR. The second part of our work comprises of building a model that classifies the retinal images into five classes - no DR, mild DR, moderate DR, severe DR and proliferative DR. We used Convolutional Neural Networks in our research work because of the growing popularity of its success in the domain. CNNs are primarily comprised of neurons that selfoptimise through learning like the traditional ANNs do. Each neuron basically receives an input and performs an operation to produce output [7]. The major difference between simple ANNs and deep CNNs is that the latter is used extensively in the field of pattern recognition and image classification. CNN, like all artificial neural networks, emulates features of the visual cortex. For an image classification task specifically, a human develops the skill of recognizing the features of what he/she sees and specify it in a class of objects. The CNNs use the same technique but under different regulations as the computer can see numerical representation of the image unlike humans and therefore learns the numerical characteristics of the data. Every CNN takes an image as input in the form of matrix and pixel value. The basic CNN architectures are comprised of stacks of three types of layers - convolutional layers, pooling layers and fully-connected or dense layers. The hyperparameters of the network controls the performance and yields the result. A. Dataset, Hardware and Software The hardware and software parameters used during our experiment are thoroughly described in this section [Table I]. 1) Dataset: For the Binary classification, we used images from the open Messidor dataset [8] and parts of the openly available famous kaggle dataset [9]. For the Five class classification problem, however, we used the kaggle dataset in particular. For convenience, we separated the entire dataset, in both the cases, into Train and Valid directories inside one Data directory. In the two-class classification, both the Train and Valid directories consisted of two directories, No-DR and YesDR, with images separated accordingly. The model is trained on the images in folders No-DR and Yes-DR of Train directory whereas tested on the images in folders No-DR and Yes-DR of the Valid directory. For the five-class classification, the Train and Valid directories consisted of five directories No-DR, Mild-DR, Moderate-DR, Severe-DR and Proliferative-DR. 2) Hardware: The CNN models are trained on a highend GPU, the NVIDIA 940mx which contains 384 CUDA cores with the NVIDIA CUDA Deep Neural Network library (cuDNN) for GPU learning. 3) Software: The deep learning package Keras is used to build up our CNN models, on top of the Tensorflow machine learning framework [10]. Python libraries NumPy [17] and matplotlib [18] are used for preprocessing and graph plotting respectively. Tensorflow was originally developed by researchers and engineers of the Google Brain Team for
3
conducting research on machine learning and deep neural networks. It is a library used mostly for deep learning, although its computational model allows its use in a wide range [12]. Keras is a high-level machine learning API written in python and is capable of running on top of deep learning frameworks such as Tensorflow, CNTK or Theano.
1) Binary Classification: A stack of 3 convolutional layers with 32, 32 and 64 number of neurons respectively is used for the binary classification model. Each layer in the network is
TABLE I: Harware & Software parameters
Hardware/Software
Parameters
Operating System Processor Graphics Environment Dataset
Ubuntu 16.04.2 LTS (64 bits) Intel core-i5 NVIDIA GeForce 940mx Keras, Tensorflow Messidor, Kaggle
B. Data Preprocessing and Augmentation The data collection usually is followed by a number of preprocessing steps before feeding them into the neural network model. But here we have not performed any preprocessing manually before training our models. However, since the models are trained using Keras, preprocessing in Keras is done during training itself via a built class keras.preprocessing.image.ImageDataGenerator. This class performs random transformations and normalizations on the image dataset during the training itself. The data provided by kaggle is highly unequal in number for all the five classes. Maximum data in the dataset belongs to class 0 - no DR and only a very few belong to class 4 - proliferative DR. So to maintain an equilibrium, we augmented the dataset by performing rotations of the images into angles 45, 90, 120, 180 and 270 degrees. Augmenting a dataset, alongwith enlarging the dataset, reduces the chances of overfitting of the data in the model as well.
Fig. 3: Model for binary classification
C. Training We built supervised learning models precise enough to perform classification of input images, i.e, the CNN models are trained with labelled datasets to learn the attributes and classify the retinal fundus images accordingly. There are various configurations of state-of-the-art pretrained and tested neural networks architectures available such as the AlexNet, GoogleNet, VGG16. All these architectures have shown excellence in different image recognition and classification tasks. For our experiment, we tried building customized CNN models based on the basics of CNN architecture. We developed multiple CNN models, by changing the hyperparameters in a trail and error method resulting in different architectures. All these models resulted in varying performances and we proceeded with two of our best performing models; one each for both binary classification and five class classification problems.
Fig. 4: Model for multiclass classification
4
activated with ReLU activation function followed by a maxpooling layer of kernel size 2x2 [Fig 3]. Two fully-connected layers are added to the small network. The model ends with a single unit and a sigmoid activation, which is needed for a binary classification. It was trained with a set of around 9600 train image data and 4080 validation data for 100 epochs. 2) Five Class Classification: For the multiclass classification, a set of 24,000 training images and 4,800 validation images is trained using the architecture shown in [Fig 4]. It has 4 convolutional layers with 32, 64, 128 and 512 filters in each respectively. With the increase in convolution layers, the network is allowed to learn deeper features, which is necessary for a multi-class classification. After the last convolutional layer, the network is flattened to one dimension followed by dense layers with dropouts. The final dense classification layer with five nodes, has a softmax activation function which is generally used for multiclass classification.
IV. R ESULTS AND D ISCUSSIONS Each model was built and evaluated using Keras on top of TensorFlow GPU backend. Training these CNN models for fundus image classification using supervised learning, we obtained the accuracy and loss count for both training and validation data. The model ccuracy is the fraction of right predictions that a model makes. It is usually measured in percentage of the all the possible predictions using the numerical formula that states [13]: Accuracy =
Number of correct predictions Total number of predictions
The model loss, however, unlike accuracy, is not calculated as a percentage. It is interpreted as how well the model performs for the training and validation sets of data. It is therefore calculated as the summation of the errors made by the model for each data sample in training and validation sets.
Fig. 5: Binary Classification Model Accuracy
Fig. 7: Multiclass Model Accuracy
Fig. 6: Binary Classification Model Loss
Fig. 8: Multiclass Model Loss
5
We plotted both the accuracy and loss as model loss and model accuracy graphs. This gave the performance of the models in accordance with every epoch. The results of all the trained models show a promising sign of dealing with the issues related to detecting Diabetic Retinopathy. A. Two classes Different models were trained to classify images into diabetic retinopathy affected and not affected classes, one among them gave us significant results. The model trained a dataset of 9600 train data and 4080 validation data which led us to a training accuracy of 83% and a validation accuracy of 71% after 100 epochs [Fig 5 and Fig 6]. B. Five classes For the multiclass classification, a dataset of 24,000 training images and 4,800 validation images were fed to the CNN model and it gave a training accuracy of 60% and validation accuracy of 56% after 100 epochs [Fig 7 and Fig 8]. Our models obtained promising results without any prior preprocessing of the input images, which is efficient in both the dynamics of time and human involvement. V. C ONCLUSION This research work is an approach to the implementation of deep learning techniques in medical science for the diagnosis of a disease based on images. We tried to come up with automatic detection and grading of Diabetic Retinopathy into its degree of severity. Deep Convolutional Neural Networks has been used as our approach. We built our own neural network models to classify retinal fundus images collected from online repositories. Different models have been trained for both binary classification (No DR and Yes DR) and five class classification (No DR, mild DR, moderate DR, severe DR, proliferative DR). It gave clarity about Five-class classification problem of DR along with the Two class classification. These trained CNNs are found to be capable of performing classification over thousands of images within a small amount of time making it effective in real-time. Our networks have shown signs of being able to learn the features required to classify from the fundus images, and accurately classifying the cases with no DR and if there is DR then further classifying them into mild, moderate, severe or proliferative stages of DR. All the different network models that we built showed encouraging performances with no preprocessing of data required for the same. Unlike most of the works in this domain that performs rigorous data cleaning, data segmentation and other preprocessing steps prior to training the model, our experiment involved no preprocessing at all before training the models. However, augmentation of the dataset was performed in order to avoid overfitting. An analysis of the entire work is successful in stating the significance of difference in model architectures. Adding deeper layers and adjusting hyperparameters across the network builds separate models with difference in results. This
opened a way to further research on deep neural network architectures. The basic limitations of our project were dataset and hardware specifications. A higher GPU specification is believed to build more precise CNN models if trained with a larger and cleaner dataset. This leaves us with the scope to develop and train better CNN models to improve results in the future. Finally, it increases the chance of treating Diabetic Retinopathy with artificially intelligent medical treatment, thus making a difference. R EFERENCES [1] Pratt, Harry & Coenen, F..B.D..P.H.S..Z.Y.: Convolutional neural networks for diabetic retinopathy. Procedia Computer Science (2016). [2] Gardner, G., K.D.W.T.E.A.: Automatic detection of diabetic retinopathy using an artificial neural network : a screening tool. [3] Shojaeipour, A., N.M.J..H.N.: Using image processing methods for diagnosis diabetic retinopathy. EEE International Symposium on Robotics and Manufacturing Automation (ROMA).doi:10.1109/roma.2014.7295879 (2014). [4] Solanki, M.S.: Diabetic retinopathy detection using eye images (April 18, 2015). [5] Alex Tamkin, Iain Usiri, C.F.: Deep cnns for diabetic retinopathy detection. [6] Piemonte, Lorenzo. Facts & Figures. International Diabetes Federation Home, idf.org/52-about-diabetes. [7] O’Shea, Keiron & Nash, Ryan. (2015). An Introduction to Convolutional Neural Networks. ArXiv e-prints. [8] Methods to evaluate segmentation and indexing techniques in the field of retinal ophthalmology (messidor), http://www.adcis.net/en/DownloadThird-Party/Messidor. [9] Diabetic Retinopathy Detection — Kaggle, www.kaggle.com/c/diabeticretinopathy-detection/data. [10] Keras: The Python Deep Learning Library. Keras Documentation, keras.io/. [11] Deep learning-moving beyond shallow machine learning since 2006, http://deeplearning.net/software/tensorflow/. [12] Admin. (2018, April 18). What are the advantages and disadvantages of using TensorFlow over Scikit-learn for unsupervised learning? Retrieved from https://cesconi.com/advantages-disadvantages-usingtensorflow-scikit-learn-unsupervised-learning/ [13] Classification: Accuracy — Machine Learning Crash Course — Google Developers. (n.d.). Retrieved from https://developers.google.com/machine-learning/crashcourse/classification/accuracy [14] Geitgey, Adam. What Is Keras? Lynda.com - from LinkedIn, 1 Aug. 2017, www.lynda.com/Google-TensorFlow-tutorials/WhatKeras/601801/642171-4. [15] Keras: Building Deep Learning Applications with High ... (n.d.). Retrieved from https://opensourceforu.com/2018/01/keras-building-deeplearning-applications-with-high-levels-of-abstraction/ [16] Francois Chollet. The Keras Blog. The Keras Blog ATOM, June 2016, blog.keras.io/building-powerful-image-classification-models-usingvery-little-data. [17] Page. NumPy - Python Wiki, wiki.python.org/moin/NumPy. [18] Python Programming Tutorials, pythonprogramming.net/matplotlibintro-tutorial/. [19] []Top 8 Deep Learning Frameworks. (2018, March 07). Retrieved from https://www.marutitech.com/top-8-deep-learning-frameworks/ [20] S. Ben Driss, et al. A Comparison Study between MLP and Convolutional Neural Network Models for Character Recognition. doi:10.1117/12.2262589.5460687319001.
