3D Virtual Colonoscopy for Polyps Detection by Supervised Artificial Neural Networks Vitoantonio Bevilacqua1,2,*, Domenico De Fano1, Silvia Giannini1, Giuseppe Mastronardi1,2, Valerio Paradiso1, Marcello Pennini1, Michele Piccinni1, Giuseppe Angelelli3, and Marco Moschetta3 1
Dipartimento di Elettrotecnica ed Elettronica, Politecnico di Bari, Bari, Italy 2 e.B.I.S. s.r.l., Politecnico di Bari Spin-Off, Bari, Italy 3 Dipartimento di Medicina Interna e Medicina Pubblica (Di.M.I.M.P.), Sezione di Diagnostica per Immagini, Università degli Studi di Bari, Bari, Italy
[email protected]
Abstract. The occurrence of false-positives (FPs) is still an important concern and source of unreliability in computer-aided diagnosis systems developed for 3D virtual colonoscopy. This work presents three different supervised approaches, based on supervised artificial neural networks (ANNs) architectures tested on 16 rows helical multi-slice computer tomography. The performance of the best ANN architecture developed, by using the volumes belonging to only 4 of 7 available nodules diagnosed by expert radiologists as polyps and non-polyps were evaluated in terms of FPs and false-negatives. It revealed good performance in terms of generalization and FPs reduction, correctly detecting all 7 polyps. Keywords: Computer-aided diagnosis, 3D virtual colonoscopy, supervised artificial neural network, colonic polyps detection.
1
Introduction: Materials and Methods
The colon and rectal cancers are estimated to be the third carcinoma death cause in western countries. Every year approximately 678.000 new cases are diagnosed in the world and 150.000 in Europe. Although this form of cancer is more curable than other forms of digestive apparatus carcinoma, the possibilities of 5 years surviving from the diagnosis stands at 40-50%, reaching 80-90% in early cases. These statistics show how important is to detect colorectal neoplasia at an early stage in order to ensure the effectiveness of the therapies and reduce the risk of death. Screening programs are, in this perspective, fundamental instruments of diagnosis. Computed tomography colonography (CTC), also known as virtual colonoscopy, is one of the most recent screening test techniques. Although many computer-aided diagnosis (CAD) architectures have been investigated, the occurrence of false-positives (FPs) is still a problem that can lead to less confidence of behalf of technicians in the system and to the eventuality of non-distinction. The aim of this work is to develope a CAD system for CTC that could automatically detect polyps and, in the future, interact with the 3D *
Corresponding author.
D.-S. Huang et al. (Eds.): ICIC 2011, LNBI 6840, pp. 596–603, 2012. © Springer-Verlag Berlin Heidelberg 2012
3D Virtual Colonoscopy for Polyps Detection by Supervised Artificial Neural Networks
597
reconstruction and rendering of the colon lumen, in order to display polyps and fasten radiologists reviews. The data set available, obtained by using a 16-row helical-CT multi-slice scanner with a 1 mm resolution, consists of 10 volumetric regions diagnosed as polyps by expert radiologists in 6 different patients, and of a number of several regions belonging to the same patients correctly detected as colon folds and used as samples of non-polyps. Only 7 polyps were useful for the analysis, excluding 2 tumors, with a diameter bigger than 1 cm, and one polyp hardly recognizable due to fecal stool. The DICOM data have been provided by the operative unit called "Sezione di diagnostica per immagini" of "Dipartimento di medicina interna e medicina pubblica" of Policlinico of Bari. The principal techniques developed for colon polyps detection are based on characteristics, such as tissue density, shape and edges, and involve the analysis of volumetric and surface data, geometric and texture features or intensity values distribution. All of these can be led to a specific pattern recognition problem, solvable by ANNs [2,4]. This study focused on assessing the opportunity of using supervised ANNs approach to detect polyps and reduce FPs in 3D virtual colonoscopy after an opportune preprocessing phase. Starting from this assumption, three methods based on three different architectures have been investigated. At the end of this work, we can state that ANNs appear to provide robust performance in terms of classification, sensitivity and specificity. Moreover, the reduction of FPs findings has been achieved, from a theoretical point of view, thanks to the supervised ANNs ability of generalising their knowledge, acquired through a proper training set sampled by two radiologists among a restricted number of cases previously collected. In particular, with the limited set now available, our approach seems to show its better performance, in terms of correct detection of true polyps and no misclassification of any colon folds. Although our method appears promising, an extension to a larger database, retrospectively and prospectively, will be needed to confirm the usefulness of the method. In the following sections the tree methods are explained. The first uses a cascade of two ANNs working both with 3D input data with particular attention to the evaluation of the effectiveness of the shape feature for the recognition task. The second method uses a sequence of two ANNs: a 2D one scans CT slices in order to find possible polyps centers, and a 3D one processes the spheres centered where stated by the first ANN. For the realization of this system, the massive-training artificial neural network (MTANN) approach has been investigated to reducing FPs, based on the construction of a polyp model through a 3D gaussian density distribution, introduced by [3]. Finally the third method implemented works exclusively on 3D images, using a single ANN trained to recognize polyps and discard all the other structures. The results of the third method, being the better performing, are presented quantitatively at the end of the paper. 2.1 First Method: 3D ANN Approach The aim of this approach was to test the performance of using a cascade of two ANNs, working on 3D data, especially focusing on the shape recognition skill of ANNs. Considering the 7x7x7 pixels cubic volume containing a polyp, it is reasonable to think that the volume useful to detect the nodule is the inner one. In order to decrease the computational load, the three innermost slices of the cube are convoluted. The result is a matrix analyzed by means of its entropy variation. This
598
V. Bevilacqua et al.
approach is justified by the empirical verification that polyps, having denser nuclei, tend to originate a greater entropy than normal colon walls. Those matrices, whose entropy is greater than a 0.5 threshold value, are organized in a vector of 343 elements and then put as input of the ANN. 2.1.1 Implementation of the First Volumetric ANN The first ANN was trained using vectors extracted from real polyps and FPs. Its architecture consists of three layers (see tab. 1). A positive result of such an ANN is that it successfully recognizes all the polyps, including those not included in the training set, returning no false-negatives and showing good generalization skills. 2.1.2 Implementation of the Second Volumetric ANN The second ANN implementation tested if the only shape properties of a polyp can well perform the recognition task, using the sole shape as a training parameter for the ANN. An OR function has then been applied to the sequences of 7 slices, passed through a Sobel edge filter. The final image exalts the polyp shape and its variations considering the entire volume. The input for the second ANN is a 49 elements vector (fig. 1), representing shape variations in a 7x7x7 volume, selected on the basis of the first ANN results. The scan of an entire exam, with the architecture explained in tab. 1, was performed in a short time. As a final stage, centers recognized by both the first and the second ANN were filtered to eliminate replicas.
Fig. 1. Image processing for the second ANN implementation
2.2
Second Method: 3D ANN Approach
2.2.1 Implementation of the Second Volumetric ANN The first ANN works on single CTC preprocessed slices. A 7x7 pixels square window moves along the borders of the lumen, assuming as central pixel those returned by the segmentation process. The 49 pixels contained in each window are linearized by columns and stored in a vector constituting the input for the ANN. Aiming at creating the training set, the polyps diagnosed in the whole dataset have been manually divided into four classes on the basis of their shape and position. For each class, the pattern having the most defined shape has been chosen and it has been extracted the 7x7 pixels square window that best showed the polyp shape (the same technique has been used for the FPs). We use area dimension according to the idea that a nodule is easily recognizable when its diameter is larger than 6 mm. The fundamental analysis
3D Virtual Colonoscopy for Polyps Detection by Supervised Artificial Neural Networks
599
of polyp gray-scale values has been focused at this stage on the experimental observation that 7x7 pixels polyps matrices include only a specific range of gray-scale values, experimentally estimated to be a mean value floating in the range [0.1, 0.45]. The assignment parameters about the ANN structure, layers transfer function, the training algorithm and the best training set have been chosen upon evaluation of the ANN performance changes, gradually varying the parameters. As a final model, it was selected a feed forward multi-layer perceptron (see tab. 1). A positive result of such a ANN is that it successfully recognizes all the polyps in the dataset, including those not included in the training set, returning no false-negatives and showing good generalization skills. Table 1. Projected ANNs First method 3D ANN
First method “OR” ANN
Second method 2D ANN
Second method 3D ANN
Third method 3D ANN
neurons
45
15
49
171
65
activation function
logsig
logsig
tansig
tansig
logsig
neurons
9
9
3
/
37
activation function
logsig
logsig
tansig
/
logsig
neurons
1
1
1
171
1
activation function
tansig
tansig
purelin
purelin
purelin
Learning algorithm
Resilient backpropagation
Resilient backpropagation
Resilient backpropagation
Resilient backpropagation
Resilient backpropagation
Output threshold
0.9
0.9
0.7
mse = 0.04 avg = 0.45
0.98
Polyp
1
1
1
See fig.4
1
Lumen
0
0
0
See fig.4
-1
FPs
0
0
0.5
See fig.4
-0.5
First hidden layer Second hidden layer Output layer
Output training value
2.2.2 Design of a Volumetric ANN Architecture A 3D MTANN is a supervised volume-processing technique capable of directly operate on image data. The process involves the comparison between an input volume and a teaching volume, in order to facilitate the task of distinguishing a specific opacity from other opacities in medical images. In our study, starting from the hypothesis brought forward by [3], the teaching volume has been modelled using a 3D gaussian distribution for polyps and 0 values for FPs. The second ANN worked on volumetric data obtained considering the entire slices sequence, analysed through a window containing 7 slices at a time and moving with a step of 1 slice. In detail, spheres having a diameter of 7 pixels have been extracted for each window (fig. 2),
600
V. Bevilacqua et al.
Fig. 2. The teaching model for the MTANN
Fig. 3. Number of pixels falling in each of the 4 intensity values classes considering all the real polyps in the dataset. On the right graph it is shown the same distribution calculated for the model sphere confirming the hypothesis that polyps gray-scale values distribution can be associated to a 3D gaussian distribution. On the abscissa mean values are reported for the corresponding intervals.
in correspondence to the candidate polyps centers returned by the previous analysis. For each sphere, the pixels were rearranged in a vector containing 171 elements. It has been preferred to use a spherical volume instead of a cubic one since the sphere better reproduces a polyp 3D shape, allowing at the same time a 48% reduction of computational costs. The MTANN returns 171 output values, a volume reproducing the input forced to be more similar to the correspondent teaching volume. The result is a neural filter that exalts the polyps shape by making them brighter while at the same time darkening the FPs. The polyp teaching volume has to correctly reproduce a generic polyp physical conformation and its density distribution. To achieve this, intensity histograms have been analysed for each polyp in the dataset, grouping all the gray-scale values in 4 intensity classes ([0, 0.31], [0.31, 0.372], [0.375, 0.398] and [0.398, 0.45]). Following [3], the model sphere has been built using the of a normalized 3D gaussian distribution. The brightest pixel is the central pixel of the central square and darker gray-scale values are assigned to external pixels (fig. 2), according to the fact that polyps are thicker in their center and degrading towards the borders. This assumption allowed the computation of a 0.969 value for the standard deviation σ and the absolute value of 0.346 that had to be assigned to x, y or z in one pixel shifts from the central one. Further tests led to a multiplication of all the values in the model by a factor of 2.5 in order to move the gray-scale values in a range between 0.5 and 1. This operation allowed a better definition of the spherical shape and at the same time
3D Virtual Colonoscopy for Polyps Detection by Supervised Artificial Neural Networks
601
forced the MTANN to assume higher output values for polyps, increasing the filtering efficiency. Experimental tests on the ANN architecture, varying layers transfer functions, allowed the selection, as a final model, of a feed-forward multi-layer perceptron, with the parameters shown in tab. 1. It has been applied an output threshold based on the mean square error calculated between the model and the output volume. Low error values indicate a greater probability of being in presence of a polyp. Another threshold calculating the mean value of the entire output volume has then been added. 2.3 Third Method: ANN Approach Unlike the previous method, the following has been developed with the purpose of understanding if a single ANN can be capable of correctly identify polyps. The implementation of a 3D ANN has been chosen, in order to provide a bigger amount of distribution-related information, while at the same time reducing the computational complexity that would have been generated from the analysis of single 2D images using a 2D ANN. 2.3.1 ANN Development This time the attention has been focused on the density values composition of cubes containing polyps. Such an analysis led to the finding that, as the size of the cube extracted is increased, starting from the central position, the distribution of densitometry values becomes more similar to that of the lumen. Following this strategy, the slices of the cubes are ordered on the basis of their position (fig. 4). Then the vector creation process considers each slice of the cube as a matrix composed by the centre and three concentric circumferences. The linearisation of the cubic structure is realized through an algorithm that, by making use of the standard circumference formula, orders the elements, placing the ones at shorter distance from the centre at the beginning of the vector. The final result is a linear structure of 343 elements (fig. 4). All the other information useful to correctly train the ANN (lumen and other intestinal structures) have been obtained in the same way as mentioned above. In this case too the ANN structure that proved to be the most efficient for the accomplishment of the task was a 3-layered supervised ANN. Representative parameters used for the building process of the final ANN are shown in tab. 1. By experimental results on the available dataset, the finale ANN developed was able to correctly detect all the polyps while returning no FPs.
Fig. 4. Graphical representation of the matrices and the vector
602
V. Bevilacqua et al.
3 Results and Conclusions Finally we can conclude that the first method resulted to be the fastest. A complete scan took about 50 minutes, mostly due to the fact that the second ANN only examined a single image for each volume. However it also returned the worst results, since even though it has been capable of detecting all the polyps in the dataset, it returned also an average number of 25 FPs per exam. It has been then concluded that even though shape information is fundamental to the pattern recognition task, it is still important to consider other factors such as the density. Results and comparisons between the second and third method tested have been evaluated in terms of sensitivity, false positive rate (FPR), accuracy (ACC) and specificity (SPC), defined by receiver operating characteristic (ROC) analysis. Table 2. Performance indexes extracted for the ANN architecture by the scan of the entire dataset TPR
FPR
ACC
SPC
2D ANN
100%
0.4%
99.6%
99.6%
MTANN
100%
0.08%
99.2%
99.2%
Third method ANN
100%
0%
100%
100%
Aiming at demonstrating the efficiency of the MTANN even with the limited dataset available, random noise has been introduced in real polyps to generate new examples, and the MTANN has been tested on these new ones. The MTANN resulted to be robust, always detecting the polyps. In one case the polyp was discarded by the output filter, though it had a diameter smaller than 4 mm, and therefore not included in the considered range. The performance of the system, involves a 90 minutes complete scan time on a desktop PC equipped with a Pentium dual core CPU running at a frequency of 2 GHz and 384 MB RAM. The simulation output of the third method described, as previously reported, gave no FPs and detected all the polyps. The execution time was slightly longer than the first method, employing 110 minutes on the same PC. In this paper three different supervised ANNs approaches have been presented for automatic polyp detection in 3D virtual colonoscopy. We are limited in explaining from a practical point of view the achieved generalisation about the detection accuracy of our CAD schemes, due to the small test set available since now. Anyway both 2D and 3D techniques proved to be efficient and complementary, although trained with a small number of cases; the generalization skill still remains to be confirmed, especially regarding the second and third methods developed. An interesting aspect of the work is that it allows the confirmation of the existence of an analogy between polyps density values and those of a 3D gaussian distribution. This finding proved useful to the robustness and generalization skills of the system and helped reducing the number of FPs, together with other CAD algorithms that operated by identifying the shape and CT numbers variation to classify voxels into polypoid or non-polypoid areas.
3D Virtual Colonoscopy for Polyps Detection by Supervised Artificial Neural Networks
603
References 1. O’Connor, S.D., Summers, R.M., Yao, J., Pickhardt, P.J., Choi, J.R.: CT Colonography with Computer-aided Polyp Detection: Volume and Attenuation Thresholds to Reduce False-Positive Findings Owing to the Ileocecal Valve. Radiology 241, 426–432 (2006) 2. Wang, Z., Liang, Z., Li, L., Li, X., Li, B., Anderson, J., et al.: Reduction of False Positives by Internal Features for Polyp Detection in CT-Based Virtual Colonoscopy. Medical Physics 32(12), 3602–3616 (2005) 3. Suzuki, K., Yoshida, H., Näppi, J., Dachman, A.H.: Massive-training Artificial Neural Network (MTANN) for Reduction of False-positives in Computer-aided Detection of Polyps: Suppression of Rectal Tubes. Medical Physics 33(10), 3814–3824 (2006) 4. Suzuki, K., Horiba, I., Sugie, N.: A Simple Neural Network Pruning Algorithm with Application to Filter Synthesis. Neural Processing Letters 13(1), 43–53 (2001)
