A REGION BASED ACTIVE CONTOUR METHOD ... - Semantic Scholar

A REGION BASED ACTIVE CONTOUR METHOD FOR X-RAY LUNG SEGMENTATION USING PRIOR SHAPE AND LOW LEVEL FEATURES P. Annangi, S.Thiruvenkadam, A.Raja, H. Xu

XiWen Sun, Ling Mao

Imaging Technologies, GE Global Research

Shanghai Pulmonary Hospital , China

ABSTRACT In this work, we propose a level set energy for segmenting the lungs from digital Posterior-Anterior (PA) chest x-ray images. The primary challenge in using active contours for lung segmentation is local minima due to shading effects and presence of strong edges due to the rib cage and clavicle. We have used the availability of good contrast at the lung boundaries to extract a multi-scale set of edge/corner feature points and drive our active contour model using these features. We found these features when supplemented with a simple region based data term and an average lung shape, able to handle the above local minima issues. The algorithm was tested on 1130 clinical images, giving promising results with a mean overlap value of 0.88 and a standard deviation of 0.07 between the manual and automated masks.

disjoint components of different sizes) directly to the evolution equations. The input of the seed mask to the evolution equations has to be explicitly handled in case of parametric curve based methods. The major challenges in using active contours for lung field segmentation are local minima due to shading effects, ability to segment points of high curvature like the castrophenic angle (CP angle), shape variations due to varying heart dimensions, and presence of strong edges due to the rib cage and the clavicle bone. The anatomical features of the chest x-ray PA view are shown in Fig. 1. Region based active contours as compared against edge based methods offer robustness against initial curve placement and noise. However, methods using global statistics are usually not ideal for segmenting objects with multi modal intensity as in the case of images with shading effects.

Index Terms : Lung Segmentation, Level sets, Pneumoconiosis, Chest X-ray, Canny edge, Region based active contour 1. INTRODUCTION Segmenting the lungs from digital Posterior-Anterior (PA) chest xray images is a very important first step towards any chest x-ray based computer aided diagnosis application. Pneumoconiosis is one such application, which needs the segmented lung masks to look for abnormalities present inside the lung. Various methods have been applied to segment the lungs from PA chest radiographs in the literature. These roughly fall into the following four categories [1] 1) rule based segmentation methods used to detect the outline of ribcage or the diaphragm, 2) pixel-based methods to classify each pixel of an image into either lung field or background based on a multi-scale filter bank of Gaussian derivatives and a K-NN classifier, 3) hybrid methods that combine rule-based and pixel-based methods, 4) deformable model-based methods, such as active shape model and active appearance model that have also been successfully applied to lung field segmentation. In this work, we propose a variational energy based framework, with the lung boundaries defined using the level set representation. Rule based segmentation methods like thresholding, morphology and connected components are used to generate a seed mask which initializes the level set evolution. The level set framework is commonly used in segmentation [2, 3, 4, 5, 6, 7] due to the following favorable properties: it provides an implicit boundary representation that is free of parameterization, easily deals with topological changes of the boundary such as splitting and merging, and can be naturally extended to any dimension. Although topological changes in the actual lung boundary is not expected, it is easier to input the initial seed mask (which may have This work was supported in part by the Science & Technology Commission of Shanghai Municipality, China (Grant No. 074107022).

Fig. 1. Chest X-Ray PA view Recent efforts have addressed this issue by estimating statistics in a neighborhood around each pixel [8, 9] but a higher computational cost results. There are also efforts that introduce prior shape information into segmentation schemes [10, 11, 12] based on active contours. These methods have been reasonably successful in handling low contrast, boundary gaps. However, for the above methods to be effective, they need to be carefully trained to acquire a rich description of shape statistics. In particular for lung field segmentation, the training step becomes challenging due to large non-linear shape variations that occur especially at the CP angle region and heart. Recent works on lung segmentation using Active Shape Models [13, 14] have used a rich shape description utilizing both population based and patient-specific shape statistics. However, we felt that the contrast near the lung boundary is by itself strong, and does not motivate use of rich shape descriptors, to deal with local minima issues due to rib cage edges and shading effects. This leads us to see if low level features such as edges/corners can be used to drive our segmentation away from the above local minima. Of course, one has to handle outlier feature points which might influence segmentation results, either by pre-processing or through other energy terms. In our approach, a set of extracted edge/corner feature points drive the active contour evolution, implemented in a multi resolu-

tion framework . We present a variational energy that uses these features, along with a region based data term using global statistics similar to the Chan-Vese approach [6], and a prior shape term as in [10]. The algorithm has been tested on 1130 images obtained from multiple scanners and multiple clinical sites on patients at different age groups and was validated against expert drawn ground truth segmentation. 2. LEVEL SET FORMULATION For simplicity, we use the following energy separately for the right and left lungs, without loss of generality let us consider the case of segmenting the left lung. Given the lung image I : Ω → R, we want to find a smooth closed curve C that lies on the lung boundary. Also, let S be a given binary prior shape of the lung extracted from a set of training data. We want the general shape of the curve C to be similar to S after factoring out an affine transformation T . Further, let E1 denote the set of extracted Canny edge points and E2 denotes corner features as obtained by the Alison Noble corner strength metric (defined in next section). The above features are used to drive the evolving contour away from the local minima, to the actual lung boundary. As indicated in the introduction, we represent C using a level set formulation through a function φ defined on Ω. Let H(t) denote the Heaviside function. We seek a closed curve C = {x|φ(x) = 0}, such that image statistics inside and outside C is Gaussian. Denote (c1 , σ1 ) and (c2 , σ2 ), the parameters of the distributions inside and outside C. We consider the following energy minimization: Z I − c1
2 H(φ)[ E[c1 , c2 , σ1 , σ2 , T ] = + ln(σ1 )]dx σ1 Ω Z I − c2
2 + (1 − H(φ))[ + ln(σ2 )]dx σ2 Ω Z Z + δ(φ)|∇(φ)|(dE 2 (x) + λ) + β (H(φ) − S ◦ T )2 dx (1) Ω

Ω

where dE = ωdE1 + dE2 denotes the sum of the distance transforms of the Canny edge map dE1 and the corner feature dE2 , for a weight ω. The first term drives the contour to seek a partition for two gaussian distributed regions given by estimates (c1 , σ1 ) and (c2 , σ2 ). The shape term controls the shape of the contour on a large scale (i.e. we set parameter β to be very small, close to convergence). Given the simple region based model used above, even for images with good contrast, minor shading effects can lead the contour to local minima. The length term, weighted using the extracted features by the function dE is meant to drive the contour away from such local minima. λ balances the feature based component dE within the length term. 3. NUMERICAL IMPLEMENTATION To minimize E, we use its Euler Lagrange equations to iteratively solve for c1 , c2 , σ1 , σ2 , T , and φ, using an explicit finite difference scheme, given a level set initialization for the lung contour φ0 . Also for robustness of extracted features to strong edges of small scale structures (e.g. rib/clavicle bone), the minimization is done in a multi resolution framework using two levels. 3.1. Seed Mask Generation Generating the seed mask is the first step in the algorithm work flow. A reasonable estimate of the initial seed is essential for successful

segmentation. The flowchart shown in Fig.2 depicts the various steps involved in the initial mask generation used to compute φ0 , the initial level set. The seed mask generation involves one level of Otsu thresholding on the input image to get the torso, and a second level of Otsu thresholding on Histogram equalized image to get a binary mask with the lungs segmented. The horizontal bounds which define the spatial extent of the lungs are obtained using an intensity projection plot of the chest x-ray. The seed mask is then obtained by connected components on the binary mask, and using the bounds and eroded torso to extract the component of interest for the left and right lungs.

Fig. 2. Flow chart of Initial Mask Generation. 3.2. Feature Computation : Canny Edges and CP Corner The edge map is obtained by the canny edge detection method on the histogram-equalized image. Here, the canny edge parameters are optimized, based on image mean intensity values to suppress spurious edges. Also, the finer resolution image is seen to be more susceptible to outlier edges than a low resolution image. So the initial mask is deformed in lower resolution initially to drive the level set out of local minima, and since the segmentation mask from the low resolution implementation is often expected to be in a close neighborhood of the actual lung boundary, canny edges outside this neighborhood are removed in the finer resolution step as shown in Fig. 3.

(a)

(b)

(c)

(d)

Fig. 3. (a) Low resolution canny edge (b) High resolution canny edge (c) High resolution edge processed using low resolution output (d) Final segmented mask . Note : Contour marked in black color indicates the segmented mask and Contour marked in dotted white color shows the ground truth. One of the major modes of shape variation of the lung is at the boundary separating the left lung and the heart. With varying heart size and shape, the width of the lung close to castrophenic angle might become as low as 10% of the average lung width. Under these circumstances, driving the contour to the castrophenic angle requires the level set to evolve through a narrow crevice. The edge energy in (1) uses the signed distance transform of the canny edge map to drive the contour towards the lung boundary. Thus, the distance transform of the edge map shows a local maximum (Fig. 4a) in such narrow regions. When the contour reaches this narrow region, there is no velocity component from the edge term, to drive the contour towards

the CP angle corner. Further, the curvature term in the resulting evolution equation will flatten out the contour, and the contour will not evolve any further. To handle this issue, the CP corner is extracted, and is integrated within the feature based term in the energy. In our approach, we have used a corner metric which is proposed in Alison Noble’s work at Oxford [15]. The computed corner image is normalized by local mean around each pixel to correct for illumination invariance. The corner computation is defined as follows

Corner Strength =

µ det(H) where H = (∇I)(∇I)T I trace(H)

(2)

A weighted function is then computed using the distance transforms of the canny edge and the corner point (Fig 4). The extent of the corner point’s distance transform is limited to a small neighborhood (updated once in every few iterations) which contains the evolving active contour.

(b)

(a)

resulted in a successful segmentation. As mentioned in the introduction, lungs in a chest x-ray exhibit wide non-linear variability in shape, and a single shape prior may not be sufficient to describe the lung. So, richer shape statistics may be additionally needed to take the evolving contour to the actual lung boundary. Alternatively, we have found the addition of the feature based term using canny edges and corner features to the energy (1), being able to handle the lung shape variations and few example are shown in Fig.7.

(a)

(c)

(b)

Fig. 6. (a) Input Image with seed mask overlayed (b) Segmented mask without the shape prior (c) Output of deforming the seed mask with the shape prior.

(c)

Fig. 4. (a) Distance transform of Canny Edge (b) Distance transform of detected corner (c) Combined weighted transform .

(a)

(b)

(c)

(d)

Fig. 7. (a) to (d) Images with varying lung shape . 4. RESULTS 4.1. Examples and Discussion In this section, we will present a few challenging cases and explain the robustness of our formulation in handling these cases. Fig.5(a) shows the input image with the initial mask overlayed where a significant portion of the initial mask is outside the actual lung boundary. Threshold based methods will not always give a good initialization and leakage issues as shown are common. A poor initialization may lead to undesirable local minima effects. To handle such initialization issues, the length term is weighted more for the first few iterations to refine the generated initial contour Fig.5(b).

The image in Fig.8 is a typical example where the lung region near the CP angle is narrow. Inspite of presence of a good diaphragm-edge, the contour fails to pass through the narrow region (c), due to inherent local maxima limitations of the distance function (b). The addition of the corner feature into the feature based term is necessary to deal with above variations of heart shape. (d) shows the output of deformation with the extracted CP corner (black colored ’+’) incorporated into the feature based term.

(a)

(a)

(b)

(c)

Fig. 5. (a) Input Image with seed mask overlayed (b) Output of deforming the seed mask with length term for 40 iterations (c) Final Segmented mask. The image shown in Fig.6 has an abnormality in the right lung. (a) shows the input image with the initial mask overlayed, and (b) shows an inaccurate result obtained using just the region and the feature based terms. It can be seen that the local minima has taken a lung shape vastly different from an expected lung shape. Thus, to preserve the overall shape of the lung under the influence of intensity/feature based terms, a prior mean shape is used, obtained by Procrustes analysis [16] on a training data set of 100 images. In (c), using the shape term has corrected the overall lung shape and has

(b)

(c)

(d)

Fig. 8. (a) Input Image with seed mask and detected corner overlayed(black colored ’+’) (b) Distance transform of the canny edge (c) Output of deforming the seed mask without the corner incorporated to the feature term (d) Output of deforming the seed mask with the corner incorporated to the edge term . The level set deformation is carried out in two resolutions (128× 128 and 256 × 256). At the lower resolution, edges of smaller scaled structures such as the clavicle bone and ribs cage, are suppressed. This drives the level set out of local minima. The example shown in Fig.9(b) is obtained by deforming the initial mask shown in (a) in a single resolution. The final contour is stuck at the strong clavicle edge. The output (c) is obtained by deforming the initial mask in a multi-resolution framework, and is seen to cross the clavicle bone onto the actual lung boundary.

5. CONCLUSION

(a)

(b)

(c)

Fig. 9. (a) Input Image with seed mask overlayed (b) Output single resolution deformation (c) Output of multi resolution deformation .

We have presented a multi resolution level set model in which the initial seed generated deforms hierarchically under the influence of a region based term in conjunction with a global shape prior and low level features like canny edge and the CP angle corner. The algorithm has been optimized and tested on a wide variety of images acquired from multiple clinical sites and multiple scanners. 6. REFERENCES

4.2. Experiments The algorithm was tested on a database of 1130 images with ground truth marked by expert radiologists. 400 of the images were acquired from Shanghai Pulmonary Hospital with an equal distribution of normal and pneumoconiosis effected patients. The remaining 730 images are acquired from 25 different clinical sites in China. It contains both normal and abnormal cases of various pulmonary conditions. The performance analysis of the algorithm is done based on the overlap between the manual(X) and automated masks(Y), and this is measured using DICE similarity coefficient (DSC) as shown in equation (3) . Fig.10a to 10d shows examples of segmentation output with varying DSC values. DSC =

(a)DSC=95

2|X ∩ Y | and0 ≤ DSC ≤ 1 |X| + |Y |

(b)DSC=90

(c)DSC=87

(3)

(d)DSC=81

Fig. 10. DICE Comparison The mean dice coefficient value obtained was 0.88 with a standard deviation of 0.07. The pie chart in Fig.11 shows the performance of the algorithm in terms of DSC vs % of images. The segmentation was observed to be visually accurate in images with DSC values greater than 0.88 . For pathologies such as Pneumoconiosis, the region of interest for feature computation is mainly concentrated towards the centre of the lung field and thus DSC values greater than 0.80 (93% cases) can be used for abnormality detection.

Fig. 11. Performance summary

[1] Bram van Ginneken and Max A.Viergever, “Computer-aided diagnosis in chest radiography-a survey,” IEEE Transactions on Medical Imaging, vol. 20, no. 12, pp. 1228–1241, 2001. [2] R. Malladi, J.A. Sethian, and B.C. Vemuri, “Shape modeling with front propagation: A level set approach,” IEEE Trans. on PAMI, vol. 17, pp. 158–175, 1995. [3] V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic active contours,” in ICCV, 1995, pp. 694–699. [4] S. Kichenassamy, A. Kumar, P. Olver, A. Tannenbaum, and A. Yezzi, “Gradient flows and geometric active contour models,” in ICCV, 1995, pp. 810–815. [5] N. Paragios and R. Deriche, “Geodesic active regions and level set methods for supervised texture segmentation,” IJCV, vol. 46, no. 3, pp. 223–247, 2002. [6] T.F. Chan and L.A Vese, “Active contours without edges,” IEEE Trans. Image Processing, vol. 10, no. 2, pp. 266–277, 2001. [7] A. Tsai, A. Yezzi, et al., “Model-based curve evolution technique for image segmentation,” in In Proc. CVPR’01, Dec. 2001, vol. 1, pp. 463–468. [8] T. Brox and D. Cremers, “On the statistical interpretation of the piecewise smooth mumford-shah functional,” in SSVM, 2007, vol. 4485, pp. 203–213. [9] W. Guo and Y. Chen, “Using non-parametric kernel to segment and smooth images simultaneously,” in ICIP, 2006, pp. 217– 220. [10] Yunmei Chen, Hemant Tagare, et al., “Using prior shapes in geometric active contours in a variational framework.,” IJCV, vol. 50, no. 3, pp. 315–328, 2002. [11] T. Chan and W. Zhu, “Level set based shape prior segmentation,” in Proc. CVPR’05, 2005, pp. 20–26. [12] D. Cremers, N. Sochen, and C. Schn¨orr, “Towards recognitionbased variational segmentation using shape priors and dynamic labeling,” in Proceedings of International Conference on Scale Space Theories in Computer Vision, 2003. [13] Yonghong Shi, Feihu Qi, Zhong Xue, Kyoko Ito, Hidenori Matsuo, and Dinggang Shen, “Segmenting lung fields in serial chest radiographs using both population and patient-specific shape statistics,” in MICCAI, 2006, vol. 1, pp. 83–91. [14] Yonghong Shi and Dinggang Shen, “Hierarchical shape statistical model for segmentation of lung fields in chest radiographs,” in MICCAI, 2008, vol. 1, pp. 417–424. [15] J.Alison Noble, “Finding corners,” Image and Vision Computing, vol. 6, no. 2, pp. 121–128, 1988. [16] C. Goodall, “Procrustes methods in the statistical analysis of shape series b,” Journal of the Royal Statistical Society, vol. 53, pp. 285–339, 1991.