An Optimized Superpixel Clustering Approach for High-‐Resolution Chest CT Image Segmentation Rafaelo Pinheiro da Rosaa, Marcos Cordeiro d’Ornellasa a
Laboratório de Computação Aplicada, Universidade Federal de Santa Maria, Santa Maria-‐RS, Brasil
Abstract Lung segmentation is a fundamental step in many image analysis applications for lung diseases and abnormalities in thoracic computed tomography (CT). However, due to the large variations in pathology that may be present in thoracic CT images, it is difficult to extract the lung regions accurately, especially when the lung parenchyma contains extensive lung diseases. A major insight to deal with this problem is the ex-‐ istence of new approaches to cope with quality and perfor-‐ mance. This paper presents an optimized superpixel clustering approach for high-‐resolution chest CT segmentation. The pro-‐ posed algorithm is compared against some open source su-‐ per-‐pixel algorithms while a performance evaluation is carried out in terms of boundary recall and under-‐segmentation error metrics. An experienced radiologist produces ground truth chest CT slices segmentation results. The over-‐seg-‐mentation results on a Computed Tomography Emphysema Database demonstrates that our approach shows better performance than other three state-‐of-‐the-‐art superpixel methods.
Despite a large body of knowledge in medical image [6], med-‐ ical image segmentation still faces many technical problems and still remains a challenge due to the large variability of body parts and image quality. Accurate 3D reconstruction models of the body anatomy from CT or MRI datasets rely on accurate image segmentation method and geometry pro-‐ cessing. For instance, Figure 1 displays a chest CT slice on the left side and its segmentation counterpart. 3D lung modeling involves analyzing 2D lung images and reconstructing a realis-‐ tic 3D model. Small inaccuracies in inclusion of juxtapleural nodules could lead to inaccurate volumetric measurement and estimation of malignancy and doubling times [7][8].
Keywords: Image Segmentation; Radiographic Image Interpretation; Tomography.
Introduction Medical imaging has developed steadily to become a perva-‐ sive component in virtually all branches of medicine and a fundamental diagnostic tool in every physician’s practice. Radiologists have long been considered the forefront of imag-‐ ing technology, specialized in a wide variety of imaging mo-‐ dalities [1]. Over the past years, diagnostic radiologists have turned into partners with primary care and specialty physi-‐ cians alike, to diagnose or treat conditions, enabling effective care [2]. Medical imaging is now used to provide interven-‐ tional treatments and diagnostic screening. It allow for sub-‐ stantial information about abnormal adjacent tissues, helping physicians to identify the appropriate treatment. As image processing and analysis techniques have broad the knowledge and skills of physicians, the size of images has also increase dramatically. This affected the storage costs and computer performance that is needed to process these imag-‐ es [3]. Automated medical image analysis supports unequivo-‐ cal, complex, and consistent measures that accelerate under-‐ standing of diseases [4]. It not only identifies abnormal pat-‐ terns but also highlight those patterns, which may lead to correct diagnosis of anatomical defects and function disor-‐ ders. These measures have been used to support radiologists’ workload and have been recognized to improve accuracy of diagnosis [5].
Figure 1 – A sample Chest CT image segmentation. Recently, superpixels have converted into an essential and fundamental approach for many imaging applications includ-‐ ing segmentation [9][10]. As superpixels are used as a pre-‐ processing step to lower the complexity of segmentation, they should be computationally efficient to avoid a negatively impact in overall performance. This paper presents an optimized superpixel clustering ap-‐ proach for high-‐resolution chest CT segmentation. The pro-‐ posed algorithm is compared against some open source su-‐ perpixel algorithms while a performance evaluation is carried out in terms of boundary recall and under-‐segmentation er-‐ ror metrics. An experienced radiologist produces ground truth chest CT slices segmentation results. The paper sections are structured in the following order: Chest CT Segmentation, Superpixel Algorithms for Image Segmentation, Methodology, Experimental Results and Dis-‐ cussion, and Conclusion and Further Results. Chest CT Seg-‐ mentation section describes the state of art in terms of seg-‐ mentation in medical imaging with respect to CT segmenta-‐ tion. The next section presents a number of superpixel algo-‐ rithms for image segmentation. A methodology section dis-‐ cusses the pros and cons of the optimized superpixel cluster-‐ ing approach. Experimental results and discussion are carried
out in the next section, by means of standard performance metrics. Useful conclusions and further research are drawn regarding evaluation procedures set forth in this paper.
Chest CT Segmentation Multi-‐slice CT scanning has changed the way physicians ac-‐ quire, process and interpret data originating from the human body and motivates the need for further image analysis tech-‐ niques. With respect to chest CT, lung segmentation is an important first step for quantitative lung CT image analysis and computer aided diagnosis (CAD). Appropriate and precise lung segmentation allows for the quantification and detection of lung malformations and abnormalities. Segmentation of lobes is essential to discover parenchymal diseases and to quantify its extension. Several methods for chest CT segmen-‐ tation, mainly with respect to lung segmentation have been proposed in the literature. Most of them are based on the fact that a healthy parenchyma displays a large difference in attenuation between the lung parenchyma and the surround-‐ ings. For instance, an adaptive border-‐matching algorithm was proposed in [11][12]. Nevertheless, the algorithm fails in some particular cases due some errors in the pre-‐ segmentation step. A refined segmentation approach was discussed in [13] and [14], which yield significant accuracy improvements in the segmentation but happened to be inef-‐ ficient due to some registration and classification processes. A segmentation based on the rib curvature was proposed in [15], yielding more accurate segmentation results. All of the-‐ se approaches have shortcomings that make them impracti-‐ cal for high-‐definition automatic processing of lung CT imag-‐ es. A novel approach to compute 3D supervoxels for radiological image datasets was proposed in [16]. It allows coping with the high levels of noise and low contrast encountered in clini-‐ cal multimodality data. Recently, a novel approach to patho-‐ logical lung segmentation using the keypoint sampling of su-‐ pervoxels in CT scan was proposed. Adapting the state-‐of-‐ heart Simple Linear Iterative Clustering (SLIC) method to gen-‐ erate supervoxel for CT images creates the near optimal grid for the keypoint sampling [17].
Superpixel Algorithms for Image Segmentation Several algorithms for image segmentation use the pixel-‐grid as the underlying representation [18][19]. The pixel-‐grid, however, is not a commonplace for medical imaging, being an image processing artifact in the medical imaging segmenta-‐ tion workflow. Therefore, it is more natural to deal with meaningful entities obtained from a suitable low-‐level group-‐ ing process. Superpixels have some properties: • • • •
superpixel should adhere to object boundaries; it is rather efficient, reducing the complexity of im-‐ ages from thousands of pixels to a regular mesh of few hundred superpixels; superpixels are perceptually meaningful since all pixels inside a superpixel are alike; Since the superpixels are the results of an over-‐ segmentation procedure, most structures are pre-‐ served in the image.
Superpixel algorithms are classified in graph-‐based algorithms and gradient-‐ascent based algorithms. The most common superpixel implementations are normalized cuts (NC00)[20], Turbopixels (TP09) [21], QuickShift (QS08) [22], and Simple Linear Iterative Clustering (SLIC12) [23]: •
•
•
•
NC00: Normalized Cuts is a computationally de-‐ manding segmentation method based on pairwise regional affinities. The normalized cuts algorithm is a graph-‐based algorithm using graph cuts to optimize a global energy function. TP09: Turbopixels is a fast geometric-‐flow based ap-‐ proach to over-‐segmentation of an image. After ini-‐ tial superpixel centers have been chosen, each su-‐ perpixel grows similar to a wave-‐front propagation. QS08: QuickShift is applied to model shifting means in a univariate setting. QuickShift is a 2D segmenta-‐ tion similar to meanshift, a robust method for image segmentation [24]. SLIC12: Simple Linear Iterative Clustering is rather simple and new algorithm. SLIC implements local K-‐ means clustering to generate superpixel segmenta-‐ tion with K superpixels.
Methodology The Algorithm The algorithm presented in this paper is an optimized version of the original SLIC algorithm [25]. The original version is con-‐ figured through two parameters: the first is related to the desired number of superpixels k and the second m is due to compactness. For gray scale images, the clusters are formed by pixels that depend on attributes such as intensity I and pixel position. During the initialization step, k initial clusters are positioned so as to occupy regularly sampled spaces. To this end, each superpixels have equal size, S = N K , were N is the number of image pixels and S S is the superpixel size. In order to decrease the chance that the center of a superpixel stands on the edge or noise, the centers of the clusters are moved to the lowest gradient values using a 3 x 3 mask. In the assignment step, each image pixel is labeled as belonging to a certain cluster. The labeling procedure is performed based on the distance of the pixel of interest with the nearest cluster, within a 2 S × 2 S limited region, enabling an en-‐ hanced performance when compared to other conventional cluster algorithms.
D determines the nearest cluster center for each pixel. Even when all images have both the same dimensions (512 x 512) and the same variations in gray scale (0-‐255), these values can vary for each cluster, which could lead to problems in calculating the distance. In order to reduce the chance of wrong classification of several pixels, spatial and intensity distances must be normalized with respect to the maximum value in each cluster. Therefore, Equation (3) shows the Eu-‐ clidean Distance:
dc = ( I j − I i )2 (1)
ds = ( x j − x i )2 + ( y j − y i )2 (2)
2
2
⎛d ⎞ ⎛d ⎞ D = ⎜ c ⎟ + ⎜ s ⎟ (3) ⎝ Nc ⎠ ⎝ Ns ⎠ Once all clusters must assume maximum values of the sam-‐ pled equivalent range size in the initial step, N s = S . For the definition of the maximum intensity variance value for each cluster, the parameter m is used. Thus, applying these values into the equation and performing the necessary simplifica-‐ tions, the calculation of distance D is given by Equation (4):
D = d c2 + f d s2 (4) where f = ( m s )2 . For higher m values, spatial distances start to have an impact in the equation, generating more compact superpixels. For smaller values, the intensities over-‐ lap, allowing superpixels adhere to object edges. In the final step, an update is carried out to determine the new cluster edges, based on an average of feature vectors [ I , x , y ] for each pixel in the cluster. Thus, the error E be-‐ tween the new cluster center locations and previous cluster center locations is calculated. Assignment and update steps are iteratively repeated until the error converges. A subjec-‐ tive evaluation states that 10 iterations are good enough for most images [23]. Algorithm Optimizations ESLIC optimizations are described as follows: • •
•
•
The optimized version (ESLIC) is tweaked to work on grey-‐scaled images. Therefore, similarity measures are greatly simplified. The original implementation makes use of f pa-‐ rameter in the calculation of Euclidean distance, to control compactness and adherence. Experiments have shown that this parameter may not be taken into account for chest CT slices, which are grey-‐ scaled images. To check whether it may or may not be included when calculating the distance, the algo-‐ rithm checks if the absolute difference between the maximum number of pixels with the same intensity within the cluster and the total number of pixels in this cluster is less than a specified threshold value, i.e., max− tp < T . In this paper T was set to 100. It was observed that 5 iterations suffice for our chest CT dataset. In a qualitative evaluation, it was ob-‐ served that the optimized algorithm produces suita-‐ ble results, besides improving algorithm perfor-‐ mance. At the end of SLIC processing, it is normal to get some pixels unlabeled, allowing them to be discon-‐ nected from the set of clusters. This is corrected, in the original version, by a connected components la-‐ beling algorithm. It was found, during the algorithm optimization, that this condition does not apply for the chest CT dataset. Therefore, the post-‐processing step was suppressed in the optimized implementa-‐ tion.
Experimental Results and Discussion A performance evaluation of the ESLIC algorithm is exhibited by comparing its accuracy against SLIC, NC00, QS08, and TP09 algorithm versions, both of which encode a compactness
constraint. The ESLIC algorithm was implemented in Java. While, in this paper, only comparisons with original algo-‐ rithms and extensions have been made, computer bench-‐ marks were performed in C using a commodity computer. Evaluation Metrics Superpixels are known to be an image over-‐segmentation procedure whose goal is to represent the image in such way as to reduce intra-‐class pixel spectral variability. Therefore, every superpixel is inside a meaningful image subset. Super-‐ pixel quantitative evaluation methods and metrics are exten-‐ sively described in the literature [18][21]. Boundary recall and under-‐segmentation error are standard metrics for boundary adherence. Boundary Recall The standard metric for boundary recall ( BR ) computes the portion of ground truth edges matches at least one superpix-‐ el boundary. High levels of boundary recall means that su-‐ perpixels accurately follow the edges of the objects in the segmented ground truth image. Given a ground truth bound-‐ ary G , the algorithms’ boundary image S and a maximum distance d = 1 . Then TP (True Positive) is the amount of boundary pixels in G for an existing boundary pixel in S with range d . FN (False Negatives) is the number of boundary pixels in G for whose does not exist a boundary pixel in S in range d . The Boundary recall metric ( BR ) is given by Equation (5):
BR ( S ,G ) =
TP ( S ,G ) (5) TP ( S ,G ) + FN ( S ,G )
Under-‐segmentation Error Under-‐segmentation ( USE ) occurs when a sensible gap amid two consecutive lines is narrow so that there is an overlap in local and global projection file. That notion is someway op-‐ posing to BR . High USE values relates to algorithms with high BR values. USE measures how well ground truth segments are recovered by grouping superpixels. In other words, USE impose penalties to superpixels that do not fit a ground truth segment boundary [21]. Equation (6) displays the under-‐segmentation metric ( USE ): USE ( S ,G ) =
1 N
⎛
⎞ min S j ∩ G i , S j − G i ⎟ (6) G i ∈G ⎝ S j ∩G i ≠0 ⎠
∑⎜ ∑
{
}
Dataset The Computed Tomography Emphysema Database (CTED) was chosen to evaluate the performance of ESLIC algorithm. CTED can be used free of charge for research and educational 1 purposes . The database base contains 115 high-‐resolution CT slices as well as 160 square patches, which were manually annotated by experienced chest radiologist and a CT experi-‐ enced pulmonologist. Slices were reconstructed using a high-‐ spatial resolution algorithm. These slices belongs to a study group consists of 39 subjects (9 never-‐smokers, 10 smokers, and 20 smokers with chronic obstructive pulmonary disease). The 512x512 slices were acquired in the upper, middle, and lower part of the lung of each subject [26].
1
image.diku.dk/emphysema_database/
Comparisons Although CTED provides the dataset, it does not provide ground truth segmentation for CT slices. A decision was made to produce a set of ground truth segmentation CT slices by a CT experienced radiologist. Segmented SLIC12 slices, with 100 superpixels and compactness set to 20, were shown to the radiologist, which evaluated the quality of segmentation and ordered the results, producing the ground truth set used in this paper. Segmentation results were compared on the basis of boundary recall and under-‐segmentation error, re-‐ garding the available ground truth. Figure 2 displays comparisons with respect to boundary re-‐ call. In this plot, ESLIC exceeds QS08 and TP09. It is also high-‐ er than NC00 when 50 superpixels are considered. However, as the number of superpixels increases, NC00 produces bet-‐ ter results. Figure 3 shows the results obtained with under-‐ segmentation error metric. It can be concluded from this plot that ESLIC exceeds the other algorithms after 900 superpixels. In the range [100-‐800] superpixels, TP09 is superior to the others algorithm versions. While informative, subjective, qualitative comparisons can be quite misleading. For the sake of clarity, figure 4 shows a visual comparison, produced by each one of the algorithms considered in this paper.
Figure 2 – Plot of BR with respect to number of superpixels.
Figure 4 – Segmentation results on CT sample for SLIC12 ESLIC, NC00, QS08, and TP09.
Conclusion and Further Work Although the proposed segmentation technique based on SLIC12 achieves results that are comparable only to some of the results published in the bibliography, the proposed methodology has large room for improvement. ESLIC was tweaked and tuned up for performance reasons. NC00, QS08, TP09, and SLIC12 algorithms were compared against ESLIC with respect to boundary recall and under-‐segmentation er-‐ ror. Results have shown that the proposed algorithm is supe-‐ rior, with respect to under-‐segmentation error, to the other algorithms. When considering boundary recall, ESLIC per-‐ forms better than QS08 and TP09 but is outperformed by NC00. The correctness of the results is relative to the correctness of the ground truth, established on CTED dataset. Despite the qualitative and quantitative comparisons, performed using a ground truth produced from CTED CT slices and its corre-‐ spondent metadata, there is no reason to believe that they are unreliable. The characterization and proper labeling of the slices becomes a problem that can take advantage of the large range of solutions and can be tackled in the future.
References [1] Smith NB and Webb A. Introduction to Medical Imaging – Physics, Engineering and Clinical Applications, Cambridge University Press, 2011. nd
[2] Suetens P. Fundamentals of Medical Imaging. 2 Edition, Cambridge University, U.K., 2013. [3] Rockall A, Hatrick, A, Armstron P, and Wastie M. Diagnos-‐ th tic Imaging, 7 Edition, Wiley-‐Blackwell, 2013. Figure 3 – Plot of USE with respect to number of superpixels.
[4] Rekik, I, Carpenter, TK, Wardlaw, JM & Allassonnière, S 2012, Medical Image Analysis Methods in MR/CT-‐Imaged Acute-‐Subacute Ischemic Stroke Lesion: Segmentation, Prediction and Insights Into Dynamic Evolution Simulation Models. A Critical Appraisal. NeuroImage: Clinical, vol 1, no. 1, pp. 164-‐178. [5] Rodríguez-‐González A, Torres-‐Niño J, Valencia-‐Garcia R, Mayer MA, and Alor-‐Hernandez G. Using Experts Feed-‐ back in Clinical Case Resolution and Arbitration as Accura-‐ cy Diagnosis Methodology. Comput Biol Med. 2013 Sep. 43(8):975-‐86. [6] Jones JL, Xie X, and Essa E. Combining Region-‐Based and Imprecise Boundary-‐Based Cues for Interactive Medical
Image Segmentation. Int J Numer Method Biomed Eng. 2014 Dec;30(12):1649-‐66. [7] Pu, J., Roos, J.E., Rubin, G. D., Napel, S., and Paik, D. S. (2008). Adaptive Border Marching Algorithm: Automatic Lung Segmentation on Chest CT Images. Computerized Medical Imaging and Graphics, Vol. 32(6), pp.452-‐462. [8] Zhou S, Cheng Y, and Tamura S. Automated Lung Segmen-‐ tation and Smoothing Techniques for Inclusion of Juxta-‐ pleural Nodules and Pulmonary Vessels on Chest CT Imag-‐ es. Biomedical Signal Processing and Control, Volume 13, September 2014, 62-‐70. [9] Bergh M, Boix X, Roig G, Capitani B, and Gool L. SEEDS: Superpixels Extracted via Energy-‐driven Sampling. Euro-‐ pean Conference on Computer Vision, pages 13–26, 2012. [10] Tang D, Fu H, and Cao X. Topology Preserved Regular Superpixel. International Conference on Multimedia and Expo, pages 765–768, 2012. [11] Brown MS, McNitt-‐Gray MF, Mankovich NJ, Goldin JG, Hiller J et al. Method for Segmenting Chest CT Image Data Using an Anatomical Model: Preliminary Results. IEEE Transactions on Medical Imaging 16(6) (1997) 828–839. [12] Pu J, Roos J, Yi CA, Napel S, Rubin GD, and Paik DS. Adap-‐ tive Border Marching Algorithm: Automatic Lung Segmen-‐ tation on Chest CT images. Computerized Medical Imaging and Graphic 32(6) (2008) 452–462 [13] Sluimer I, Schilham A, Prokop M, Ginneken B.Computer analysis of Computed Tomography Scans of the Lung: A survey. IEEE Transactions on Medical Imaging 25(4) (2006) 385–405. [14] Rikxoort EM, Hoop B, Viergever MA, Prokop M, Ginneken B. Automatic Lung Segmentation From Thoracic Comput-‐ ed Tomography Scans Using a Hybrid Approach With Error Detection. Medical Physics 36(7) (2009) 2934–2947. [15] Prasad MN, Brown MS, Ahmad S, Abtin F, Allen J et al. Automatic Segmentation of Lung Parenchyma in the Pres-‐ ence of Diseases Based on Curvature of Ribs. Academic Radiology 15(9) (2008) 1173–1180. [16] Holzer, M, Donner, R. Over-‐Segmentation of 3D Medical Image Volumes Based on Monogenic Cues, in Zuzana Kúkelová & Jan Heller, ed., 'Proceedings of the CVWW'14' , pp. 35-‐42 . [17] A. Mansoor , U. Bagci and D. Mollura Near-‐Optimal Key-‐ point Sampling for Fast Pathological Lung Segmenta-‐ tion", Proc. IEEEEMBC, pp.6032 -‐6035 2014. [18] Zhang Y, Hartley R, Mashfor J, and Burn S. Superpixels via Pseudo-‐boolean Optimization. International Conference on Computer Vision, pages 1387–1394, 2011. [19] Conrad C, Mertz M, and Mester R. Contour-‐relaxed Su-‐ perpixels. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, LNCS 8081, 280–293, 2013. [20] Shi J, and Malik J. Normalized Cuts and Image Segmenta-‐ tion. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2000; pp. 888–905. [21] Levinshtein A, Stere A, Kutulakos KN, Fleet DJ et al. Tur-‐ boPixels: Fast Superpixels Using Geometric Flows. Trans-‐ actions on Pattern Analysis and Machine Intelligence, 31(12):2290–2297, 2009. [22]Vedaldi A, and Soatto S. Quick Shift and Kernel Methods for Mode Seeking. In: ECCV, LNCS 5305, 705-‐718, 2008.
[23] Achanta, R, Shaji A, Smith K, Lucchi A, Fua P et al. SLIC Superpixels Compared to State-‐of-‐the-‐art Superpixel Methods. Transactions on Pattern Analysis and Machine Intelligence, volume 34, number 11, 2274 – 2281, 2012. [24] D. Comaniciu and P. Meer. Mean shift: A Robust Ap-‐ proach Toward Feature Space Analysis. IEEE Trans. Pat-‐ tern Anal. Machine Intell., 24:603–619, 2002 [25] Achanta R, Shaji, A, Smith K, Lucchi A, Fua P, and Süsstrunk S. SLIC Superpixels. Technical report, École Poly-‐ technique Fédérale de Lausanne, 2010. [26] Sørensen L, Shaker SB, and Bruijne. Quantitative Analysis of Pulmonary Emphysema using Local Binary Pat-‐ terns, IEEE Transactions on Medical Imaging 29(2): 559-‐ 569, 2010. Address for correspondence Rafaelo Pinheiro da Rosa
[email protected] Av. Roraima, 1000 Prédio 7 – Anexo B-‐ Sala 388 UFSM -‐ Campus Universitário 97105-‐900 Santa Maria-‐RS -‐ Brasil
