A combined approach to 3D medical image segmentation using marker-based watersheds and active contours: the active watershed method Rudy J Lapeera and AC Tan b and Richard Aldridge a aSchool of Information Systems, University of East Anglia, Norwich NR4 7TJ bDepartment of Medical Physics and Bioengineering, University College London, London WC1E
6JA
Abstract. Active contours have been used extensively for medical image registration as they display good smoothing properties which allow to account for the discrete nature of the image and noise from the acquisition process. However, contours are generally difficult to initialise around the region of interest (ROI). The markerbased watershed segmentation can segment unique boundaries from an image or stack of images, however it has no smoothing/generalisation properties. Combining the two approaches results in a segmentation method which both solves the contour initialisation and generalisation problem. In this paper we briefly explain the marker-based watershed and active contour segmentation and the algorithm to convert watershed contours into active contours. We then illustrate the approach on the segmentation of abdominal organs from MR images and bony structures from CT images.
1 Introduction The segmentation of medical images in 2D, slice by slice, or directly in the 3D voxel dataset, has many useful applications for the medical professional: visualisation and volume estimation of objects of interest, detection of abnormalities (e.g. tumours, polyps, etc.), tissue quantification and classification, and more. Also, technical advantages can result from segmenting (or isolating) anatomical structures; for example the optimisation of the rendering process for virtual colonoscopy by segmenting the colon from the original dataset. However, as there is no ‘holy grail’ in the wide range of available segmentation methods, the choice of the most suitable method for a particular task is crucial. In medical image segmentation, the active contour method [1, 2] has been widely and successfully used for a variety of segmentation problems. The technique has been adapted, optimised and extended to 3D since its introduction by Kass, Witkin and Terzopoulos [1]. Nonetheless, it still suffers from initialisation problems, i.e. bringing the active contour (or deformable model, to use a more general term) sufficiently close to the boundary of interest to ensure convergence. The watershed segmentation method, which is a mathematical morphology based technique, has been widely used in geological and histological images. The concept was formalised by Buecher and Lantu´ejoul [3] and was later turned into an ‘immersion-based’ algorithm by Vincent and Soille [4]. The strength of watershed segmentation is that it produces a unique solution for a particular image. However, noise in the image results in over-segmentation, as illustrated in the MR image in Figure 1(c). The placement of internal and external markers into regions of interest (the approach is explained in the next section) can easily cope with the over-segmentation problem. Another disadvantage of watershed segmentation, again related to the image noise and the image’s discrete nature, is that the final boundaries of the segmented region lacks smoothness. The combination of watershed segmentation and active contours resolves the weaknesses of each method by using the watershed to initialise the active contour which then smooths the results from the watershed segmentation. The only intermediate step required, is to convert the watershed boundaries into a closed contour. In the next section we briefly discuss the different methods and algorithms used. The section continues by showing segmentation results from MR and CT images. Finally, the current state of this novel approach and future adaptations are discussed.
2 Methods and algorithms 2.1 Marker-based watershed segmentation The principle of the immersion-based watershed algorithm by Vincent and Soille [4] can be illustrated by imagining the (magnitude of the) gradient image of the (smoothed) original image as a relief, with the ‘height’ variable being the grey-value for each pixel position. Imagine, water immersing from the bottom of the relief (at grey-level 0). Every time the water reaches a minimum, which corresponds to a region in the original image, a catchment basin Contact:
[email protected]
is ‘grown’. When two neighbouring catchment basins eventually meet, a dam is created to avoid the water spilling from one basin into the other. When the water reaches the maximum grey-value, the edges of the union of all dams form the watershed segmentation. As this approach usually leads to over-segmentation, we place one or more internal marker(s) in the region of interest (ROI) and one or more external markers in the other region(s). Only catchment basins in regions with a marker are grown which results in a binary segmentation of the image, i.e. the ROI and the background. Figure 1(a) shows a slice from a 512 512 56 16 bit MRI dataset and its corresponding morphological gradient image (b). In (c), the over-segmentation from the standard immersionbased watershed algorithm is illustrated. This problem is resolved by using the manually placed markers as shown in Figure 1(d) and (f). Note that the liver segmentation (d) required more markers than the spleen segmentation (f) possibly because of the intensity non-uniformity in the liver region (which can be seen clearly in the gradient image (c)). As the algorithm is based on the ordering of pixels according to grey-value, the extension from 2D to 3D watershed segmentation is trivial.
2.2 A boundary-following algorithm for contour creation The contours resulting from the watershed segmentation may contain gaps, glyphs and may be several pixels wide. We therefore designed a boundary following algorithm to turn the watershed boundaries into a valid contour. The detailed description of this algorithm is beyond the scope of this paper and will be published elsewhere.
2.3 Active contours Since their introduction, active contours [1] have been adapted to include several optimisation models and extensions to 3D; a fairly recent update is given in [2]. At this stage of our research, we used the standard simplified active contour model (without inertial term):
X X
X = Fint (X) + Fext(X)
@ @t
(1)
with (s), the position vector of a contour element (s being the curve parameter), internal (smoothing) forces, Fint ( ), and external forces (Gaussian potential force derived from the image), F ext ( ). The factor is a damping coefficient. A finite difference scheme can be used for numerical implementation. Figure 1(e) shows an active contour of the liver image after applying the boundary-following algorithm onto the watershed boundary to produce an initial contour followed by smoothing (by setting appropriate internal, external and damping force parameters).
X
3 Results Figure 2 shows a segmented thigh bone from a 512 512 200 16 bit CT dataset. Figure 4 shows an axial slice of the previous dataset after 3D watershed segmentation and with active contour (superimposed on the original slice). Figure 3 shows segmentation of kidneys and colon from a 256 256 128 8 bit MRI dataset.
4 Discussion The results from Figure 2 show the strength of the combined approach of the two segmentation methods. The 3D watershed algorithm can easily segment the upper bony part of the thigh bone using just one internal and external marker across all slices, however the lower cartilaginous part cannot be fully captured. Figure 4(a) shows a slice with cartilaginous regions of the thigh bone. As illustrated in Figure 4(b), the watershed algorithm is only capable of partially segmenting these regions. Unless markers are carefully placed near the boundaries of the ROI, the region will never be fully captured. This effort would be a waste of user interaction time as it merely mimics what an active contour is capable of doing automatically. However, if the 3D watershed algorithm is used as a coarse filter to segment the ROI, followed by an active contour segmentation, fast and satisfactory results are achieved. This is illustrated in 3D in Figure 2(d) and Figure 4(c) in 2D with a superimposed active contour. In Figure 3, the structures are segmented from MRI datasets. It should be noted that here the 3D watershed algorithm gave already good results and further active contour segmentation was only necessary for smoothing. The ‘active watershed’ method, as it currently stands uses 3D gradient information for the watershed segmentation part but only 2D gradient information for the active contour segmentation. Further adaptation using a 3D
(a)
(b)
(c)
(d) (e) (f) Figure 1. The original axial slice from a 512 512 56 16 bit MRI dataset (a); the morphological gradient image (b); the over-segmented image using the standard Vincent and Soille algorithm (c); and avoiding this problem using manually placed internal (light) and external (dark) markers for liver segmentation showing the watershed boundary (d); the watershed contour converted to an active contour (e); spleen segmentation showing the internal and external catchment basins (f).
(a) (b) (c) (d) Figure 2. Coronal slice from a 512 512 200 16 bit CT dataset (a); The volume rendered dataset with translucency (b); Lower part of the thigh bone, segmented using 3D watershed algorithm - with original texture map in segmented region - (c); Same as in (c) but after using active contours (d).
(a) (b) (c) Figure 3. Segmentation (3D with texture map using original data) of kidneys (a) and two views, (b) and (c), of a segmented colon from a 256 256 128 8 bit MRI dataset.
(a)
(b)
(c)
Figure 4. Axial slice from a 512 512 200 16 bit CT dataset (a); After 3D watershed segmentation (with original texture map) (b); Active contour segmentation - contour superimposed on the original slice (c). deformable mesh model (e.g. as proposed by Park et al. [5]) can resolve this problem. The 3D watershed algorithm can also be speeded up by combining the immersion-based algorithm with a rainfall algorithm [6].
5 Conclusion We have proposed a novel approach for 3D medical image segmentation by combining 3D marker-based watershed segmentation with active contours. The combination of the two minimises user interaction and speeds up the entire segmentation process. Currently, the method has been successfully tested on abdominal structures in MR data and bony structures in CT data. Further testing on more demanding tasks such as brain segmentation and validation on more quantifiable applications such as (partial) volume estimation is being conducted.
References 1. M. Kass, A. Witkin & D. Terzopoulos. “Snakes: active contour models.” Int.J.Comp.Vis. 1(4), pp. 321–331, 1987. 2. C. Xu, D. Pham & J. Prince. “Chapter 3: Image segmentation using deformable models.” In J. Fitzpatrick & M. Sonka (editors), Handbook of Medical Imaging. Vol.2 Medical Image Processing and Analysis, pp. 175–272. SPIE, London, June 2000. 3. S. Buecher & C. Lantu´ejoul. “Use of watershed in contour detection.” In Proc. Int. Workshop Image Processing, Real-Time Edge and Motion Detection/Estimation, Rennes, France, pp. 17–21. September 1979. 4. L. Vincent & P. Soille. “Watersheds in digital spaces: An efficient algorithm based on immersion simulations.” IEEE Trans. Patt.Anal.Mach.Int. 13(6), pp. 583–598, June 1991. 5. J.-Y. Park, T. McInerney, D. Terzopoulos et al. “A non-self-intersecting adaptive deformable surface for complex boundary extraction from volumetric images.” Computers & Graphics 25, pp. 421–440, 2001. 6. P. D. Smet & D. D. Vleeschauwer. “Performance and scalability of a highly optimized rainfalling watershed algorithm.” In Proc. of the 1998 Int.Conf. on Imaging Science, Systems and Technology, CCIST’98 - Las Vegas, pp. 266–273. July 1998.
