Classification of Abnormal Endoscopic Images using RGB Color and Morphological Watershed Segmentation B.V.Dhandra, Ravindra Hegadi Department of P.G. Studies and Research in Computer Science Gulbarga University, Gulbarga-585106 Karnataka, INDIA E-mail:
[email protected],
[email protected]* Abstract The segmentation in RGB color space is motivated by the fact that the endoscopic images are rich in color where as segmentation of images by morphological watershed embodied many approaches such as detection of discontinuities, thresholding and region processing. In this paper the segmentation in RGB color space in combination with morphological watershed segmentation approach is implemented for the segmentation and analysis of possible presence of abnormality in endoscopic images. Here the preprocessed endoscopic image is segmented using color segmentation based on 3σ-intervals around mean RGB values. Its inverse transform is obtained for further processing and extended minima is imposed on the processed image using morphological reconstruction. Then the morphological watershed segmentation is carried out on this image and the number of regions is counted and is compared with the threshold value. The presence of number of regions more than the threshold value in the output image will indicate the presence of abnormality in the image. Keywords: Endoscopy, abnormality, color segmentation, Morphological watershed segmentation, extended minima transform, image imposition.
1. Introduction The visual analysis of endoscopic images carried out by medical experts has many constraints such as subjective diagnosis by the expert, interpretational variation, nonsuitability and time-consuming procedures for comparative evaluation [2]. These constraints and difficulties can be reduced considerably with the help of a computer-assisted scheme in the quantitative characterization of abnormalities increasing overall efficiency in managing the patient. Computer-assisted diagnosis in endoscopy consists of endoscopic image acquisition, image processing,
parametric feature extraction, and classification. A number of schemes have been proposed to develop the methods for computer-assisted diagnosis for the detection of abnormal images. The method proposed by Krishnan et.al. [3] is based on the segmentation of grayscale endoscopic image using Canny edge detection and curvature analysis of these detected edges. Another method proposed by Hiremath et.al. [4] is segmentation of endoscopic color image using 3σ interval around mean RGB values [9] followed by edge detection of segmented region using zero crossing method and curvature analysis of these edges. These methods are based on the analysis of edges in the endoscopic images. In this paper a new method is proposed which is based on the number of regions instead of edges, for the analysis of abnormality in endoscopic images by using the segmentation of images in RGB color space followed by morphological watershed segmentation technique. The introduction of the watershed transformation as a morphological tool is due to H. Digabel and Ch. Lantuejoul [5]. Their data were piles of binary images representing successive thresholds of a surface whose drainability was to be studied. Later a joint work by C. Lantuejoul and S. Beucher [6] led to the inversion of this original algorithm in order to extend in the more general framework of grayscale images. A fast and flexible algorithm for computing watersheds in digital grayscale images was introduced by Luc Vincent and Pierre Soille[7]. The presence abnormality such as the bleeding, growth of tumors, presence of polyps, polypoid lesions and cancerous growth will lead to the changes in the color and presence of roughness on the surface in the image captured by the endoscope. The color features in the images will be best extracted by the RGB color segmentation. The study reveals that the watershed segmentation will provide the better features for further analysis as compared to the simple edge analysis, since the morphological watershed segmentation embodies the basic principal concepts such as detection of discontinuity, thresholding and region processing, apart from producing more stable segmentation results, including the continuous segmentation boundaries [1]. Watershed technique is one of the classical techniques in the field of
topography providing a simple framework for incorporating knowledge based constraints in the segmentation process. When combined with other morphological tools, the watershed transformation is at the basis of extremely powerful segmentation procedures.
2. RGB color segmentation The endoscopic images are smoothed using average filter to reduce the effect of bright spots created by light reflection and small lumen regions created in the image. The RGB color image contains 3 times more data than a gray scale image. However, the three maps should not be processed independently because it appears that a strong spatial and chromatic correlation exists. The average filter is considered for smoothing multiple image [R(x,y), G(x,y), B(x,y)] component-wise. For the regions of interest in the test images, the mean and standard deviation of RGB values are determined using the statistical sampling technique [9]. The information is stored as the knowledge base for automatic segmentation of any given input image. Table 1 shows the estimated mean and standard deviation of RGB values for the region of interest for the test image.
Variables Mean Standard Deviation
Abnormal Region Values R G B 120.93 56.41 53.65 17.39 14.45 16.03
Table 1: Mean and standard deviation of RGB values for the region of interest. The segmentation is carried out by using 3σ-intervals around mean RGB values stored in the knowledge base. The output of this process will be the part of the image containing the abnormality.
3. Morphological Segmentation The image color containing the abnormal portion extracted after the color segmentation is converted into gray scale image. The compliment of this image is computed by subtracting every pixel of the image with 255, since, the intensity value in a gray scale images are ranging from 0 to 255. In the output image N, dark areas become lighter and light areas become darker. If this image is segmented using grayscale watershed segmentation technique then over segmentation occurs because of creation of large number of basins due to local minima. One way to deal this problem is to calculate a set of starting basin pixels and allow the segmentation to build off of those initial basins only. To overcome this problem the extended minima transform is applied on the complimented image. The extended minima transform is a kind of thresholding operation which will bring most of
the valleys present in the image terrain to zero. The compliment image N is subjected to extended minima transform with a threshold t, given by
E = EM ( N , t )
(1)
where E is output image which will be a binary image. The image E contains the pixels value as zero for all those pixels in compliment image N whose intensity values are less than the threshold value t. The threshold value t has to be chosen appropriately because the higher value of t will result in less number of regions and lower value of t will form huge number of regions. The output image E of the extended image transform will be imposed on the compliment image N given by
I = Im( N , E )
(2)
The combined process of extended minima transform and the image imposition will control the excessive over segmentation of the image when the watershed segmentation is carried out. The image I obtained after the image imposition step is segmented using the morphological watershed segmentation technique. A fast immersion based algorithm developed by Vincent and Soille [7] is employed here. In this algorithm all pixels in the image I are sorted in the order of increasing gray level values. Flooding is performed beginning from the global minimum. Suppose that the flooding reaches a height h, which is gradient magnitude of the pixel, the pixels at level h+1 have to be divided between the catchment basins of level h and the new basins corresponding to the local minima at level h+1. First the pixels at level h+1 that are neighbors to pixels from a catchment basin at level h are put into a FIFO queue. Under the constraint of considering only pixels at level h+1, the catchment basins are extended by propagation. When several catchment basins of level h are connected at level h+1, the resulting basins are separated along the geodesic skeleton by influence zone (SKIZ) [7]. The pixels at level h+1 that are not reached by one of the catchment basins must be local minima, and they become the seeds of new basins. Once the image is completely flooded, the SKIZ thus obtained will correspond to watershed lines. The watershed segmentation will generate a number of labeled watershed regions of the input image. The combinations of extended minima transform operation and watershed segmentation is equivalent to a watershed from markers operation. The whole processes can be explained in the following algorithm: Algorithm 1 i. Smooth the input image to eliminate the noise present in the image using average filter.
ii.
Find the region of interest using color segmentation based on 3σ-intervals around mean RGB values. iii. Convert segmented color image in to grayscale. iv. Find the compliment of the above image. v. Compute the extended minima transform which is the regional minima. vi. Find the impose minima which modifies the intensity image using morphological reconstruction. vii. Extract the watershed regions of the output image from the above process. viii. If the regions so obtained are more than the threshold value r, then classify such image as abnormal.
considered the image with no lumen regions. Figure 2 shows the abnormal images and their watershed regions.
Regions:
149
Regions:
33
Regions:
101
Regions:
84
4. Results and discussion The Olympus V70 endoscopic equipment is used for capturing the image. For the implementation of the algorithm Matlab software release 6.1 is used. Experimentation is carried out on 20 normal and 20 abnormal images. The empirical threshold t=30 is used for the computation of extended minima transform. Following figure shows the steps involved in the segmentation process.
[A]
[C]
[B]
[D]
Fig 1. Figure showing the segmentation process. [A] Abnormal Endoscopic color image, [B] Image after performing RGB segmentation, [C] Extended minima transform, [D] The watershed segments having 112 regions for the given input image. The threshold value for the number of watershed regions, r = 5, is empirically found for the classification of images. If the number of regions obtained is more than 5 then this image is likely to be an abnormal image. We have
Fig. 2 Abnormal images and their watershed regions.
Following Figure 3 shows the normal images and their watershed regions obtained by the proposed method.
160
Abnormal Images
No. of watershed Re gions
140
Regions:
1
Normal Images
120 100 80 60 40 20 0 1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 Image
Fig 4. Line graph showing the number of regions formed for normal and abnormal images.
5. Conclusion
Regions:
3
Regions:
1
A combination of the segmentation of endoscopic images by color image segmentation based on 3σ-intervals around mean RGB values and the morphological watershed segmentation is adopted in this paper. Since the abnormality in the endoscopic image lead to the change in its color, the color image segmentation process will extract the abnormal region from the image. This step will reduce the further processing of whole image thereby reducing the computation time. The morphological watershed segmentation algorithms have been proved to be a powerful tool for image segmentation because it embodies number of basic image processing approaches such as detection of discontinuity, thresholding and region processing. The over segmentation is avoided by adapting the extended minima transform and image imposition techniques. The output of the overall process indicates whether the endoscopic image is normal or abnormal based on the number of watershed regions present in the image.
Acknowledgements
Regions:
1
The authors are grateful to Dr. M. K. Ramakrishna, MS., Sri Lakshminarayana Nursing Home, Raichur, for providing endoscopic images and rendering manual segmentation for the present study. The authors are also indebted to Dr. P. Nagabushan, Dr. G. Hemanthkumar and Dr. D.S. Guru, Dept. of Studies in Computer Science, University of Mysore, Mysore, for their helpful discussions and encouragement during this work
Fig. 3 Normal images and their watershed regions.
Reference Following Figure 4 is the line graph showing the number of regions formed for the 20 abnormal images and 20 normal images for which experimentation is carried out.
[1] Refael Gonzalez, Richard E. Woods, “Digital Image Processing”, Pearson Edition Asia, 2nd Edition, 2002. [2] Kato H, Barron J P, “Electronic videoendoscopy”, Harwood Academic Publisher Japan, 1993. [3] S.M.Krishnan, X.Yang, K.L.Chan, S.Kumar, P.M.Y.Goh, “Intestinal Abnormality Detection from Endoscopic Images”, The
20th Annual International Conference of IEEE Engineering in Medicine and Biology Society (EMBS 98), Hongkong,1998.
Image processing, Real time edge detection and Motion Detection / Estimation, Rennes, France, Sept. 17-21, 1979.
[4] P.S.Hiremath, B.V.Dhandra, Ravindra Hegadi, G.G.Rajput, “Abnormality detection in endoscopic images using color segmentation and curvature computation”, 11th International Conference on Neural Information Processing, ICONIP-2004, ISI, Calcutta, India, ISBN-3-540-23931-6, Springer-Verlag Berlin Heidelberg, Germany, 2004.
[7] Luc Vincent and Pierre Soille, “Watersheds in Digital Spaces: An Efficient Algorithm based on Immersion simulations”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 6, June 1991.
[5] H. Digabel and C. Lantuejoul, “Iterative Algorithms”, Proceedings of 2nd European Symposium on Quantitative Analysis of Microstructures in Material Science, Biology and Medicine, Caen, France, Oct 1977. [6] S. Beucher and C. Lantuejoul, ”Use of Watersheds in Contour detection”, Proceedings of International Workshop on
[8] L. Vincent, “Morphological grayscale reconstruction in image analysis: Applications and efficient algorithms”, IEEE Transactions on Image Processing, Vol 12.
[9] P.S.Hiremath, B.V.Dhandra, Iranna Humnabad, Ravindra Hegadi, G.G.Rajput, “Detection of esophageal Cancer (Necrosis) in the Endoscopic images using color image segmentation”, Proceedings of second National Conference on Document Analysis and Recognition (NCDAR-2003), Mandya, India, 11-12 July 2003, pp. 417-422.
