Let C(Ri Rj) be the watershed contour be- ... computation of watershed contour dynamics can be done ... fpi j1 pi j2 ::: pi j Ng be a the wateshed countour be-.
HIERARCHICAL IMAGE SEGMENTATION BASED ON CONTOUR DYNAMICS K. Haris1 , S. Efstratiadis 21, N. Maglaveras1 1
Lab. of Medical Informatics, Faculty of Medicine, Aristotle University, Thessaloniki 54006, Greece 2 Dept. of Electronics, Technological Education Institute, Thessaloniki, 54639, Greece ABSTRACT
regional minima [4, 5, 6]. The remaining regional minima are referred to as markers. Although markers have been successfully used in segmenting many types of images, their selection requires either careful user intervention or explicit prior knowledge on the image structure.
In this paper, we propose an image segmentation method based on morphological decomposition and graph-based region merging using contour dynamics. The input image is initially decomposed into a set of primitive homogeneous regions through the morphological watershed transform applied to the image intensity gradient magnitude. This decomposition is represented by a Region Adjacency Graph (RAG) that is input to a hierarchical merging process in which neighboring regions of high similarity are merged. The region similarity criterion is based on the concept of watershed contour dynamics. The robustness of the segmentation to the presence of noise and/or low contrast is improved by a regularization of the contour dynamics. Experimental results on various kinds of synthetic and real images, as well as comparison of the proposed method with other wellknown region merging algorithms are presented.
An important method for the automated marker selection is based on the notion of regional minima dynamics [7] with main disadvantage its sensitivity to noise [6]. According to this approach, each regional minimum of the gradient image is characterized by the minimum contrast of the path leading to a regional minimum of lower height. The contrast of a path connecting two regional minima is defined by the maximum gradient magnitude value along it. A less noise sensitive contrast criterion for oversegmentation reduction is based on the dynamics of the watershed contours [6]. However, this criterion is still weak since the strength (saliency) of the watershed contours depends on the minimum gradient magnitude value along them. The variance of this minimum value is high as a direct result of the increase in the noise variance due to the differentiation operation.
1. INTRODUCTION Image segmentation is an essential process in computer vision that directly affects the performance of many subsequent higher level image analysis tasks [1, 2]. The general problem formulation involves image partitioning into a number of homogeneous regions, such that the union of any two neighboring regions yields a heterogeneous region. The ultimate goal is that such a partition identifies the objects of interest in the input image. Among the large number of proposed methods, morphological segmentation techniques are quite successful and promising [3, 4, 5, 6]. In morphological image segmentation, the watershed transform (WT) plays a key role as a tool for decomposing an image into primitive regions with certain properties. The WT inherently integrates edge- and region-based approaches by delivering an image partition to regions based on the image gradient magnitude information. The main problem with this approach stems from the WT sensitivity to intensity variations resulting in severe oversegmentation. Oversegmentation may be reduced by appropriately reducing the number of the image
0-7803-6725-1/01/$10.00 ©2001 IEEE
In this paper, we present an image segmentation method based on the WT and region merging guided by robust estimates of contour dynamics. An initial image decomposition is produced, with a large number of primitive homogeneous regions (oversegmentation), which is represented by an undirected Region Adjacency Graph (RAG). This RAG is input to a region merging process which produces hierarchical image partitions by iteratively merging similar neighboring regions. We propose a graph-based region merging image segmentation algorithm based on minimum local dynamics of regional minima and watershed contours in the RAG. In our work, the notion of watershed contour dynamics is modified in order to reflect the estimated contrast between the separated regional minima. In this way, the strength of watershed contours does not depend on the minimum gradient magnitude value along them but on their estimated (e.g. mean or median) value resulting in better segmentation results mainly in the presence of noise.
54
2. REGION DISSIMILARITY BASED ON CONTOUR DYNAMICS
For noisy images, the image gradient magnitude variations may cause large deviations of the watershed minimum intensities resulting in large deviations in the estimation of contour dynamics. We note that, independently of the watershed contour length, the watershed contour height is determined by the minimum value of the pixel intensities it contains. Therefore, a strong watershed contour, where most of its pixels have high values, may be valuated as weak due to existence of few pixels having small values. This estimation will cause the early elimination of the particular contour. In order to avoid the above situation, we propose the use of more robust contour dynamics estimations. Specifically, in Eq. (1), the mean and median watershed contour heights are used. The mean watershed contour height is defined as the average watershed pixel intensity and the median watershed contour height as the median of the watershed contour pixel intensities. Let C ij = C (Ri Rj ) = fpij1 pij2 : : : pijN g be a the wateshed countour between regions Ri Rj containing the pixels p ijk k 2 1 N ]. Then,
At the first step of the proposed method, the image gradient magnitude is input to the watershed transform algorithm which produces an initial image tessellation, R K , into a number of primitive regions
R = fR1 R2 : : : R g : Each premitive region R i 2 f1 2 : : : K g represents a K
K
i
regional minimum with its associated catchment basin. A (regional) minimum M at intensity level h in the image I is a connected set of pixels with intensity h, such that it is impossible to reach a pixel of intensity h 0 without having to pass from a pixel of intensity h 00 , where h0 < h < h00 . The catchment basin C (M ) associated with the minimum M is a set of pixels, such that, if a drop of water falls at any pixel in C (M ), then it will flow down to the minimum M . Watersheds are defined as the lines separating the catchment basins which belong to different minima. Due to the high sensitivity of the watershed algorithm to the gradient image intensity variations, the WT produces image partitions containing a large number of regions. This initial oversegmentation is input to a region merging process where at each the most similar pair of neighboring regions is merged. A contrast criterion that can be used as region disimilarity function is based on the dynamics of the watershed contours [6]. Let C (R i Rj ) be the watershed contour between regions Ri Rj and Ps C (Ri Rj ) the set of points having minimum intensity. Also, let B 1 B2 : : : Bn the connected components of the image points reachable from any point of P s with descenting intensity paths. The dynamics of the contour C (R i Rj ), dc (C (Ri Rj )), is defined as
( (
dc C Ri Rj
)) = min max f ( 1 ] k2
n p2Bk
W Ri Rj
) ( )g I p
mean (
W
( (
dc C Ri Rj
)) = (
W Ri Rj
) max( ( )
med (Ri Rj )
W
(
D Ri R j
)= ( (
dc C Ri Rj
))
I pijk
=1
)
= med( (
) (
)
(
I pij1 I pij2 : : : I pijN
))
:
The corresponding region disimilarity functions are mean (Ri Rj )
D
=
W
=
W
mean
max(M (R ) M (R )) i
j
(2)
and med (Ri Rj )
D
med max(M (Ri ) M (Rj )) :
(3)
3. REGION MERGING The image partition created by the watershed transform is represented by the Region Adjacency Graph (RAG) where each node represents a region and each edge represents the watershed boundary between the regions it connects. At each RAG-node, u i , the minimum intensity M (R i ) of the corresponding region R i is stored. At each RAG-edge, e ij = (ui uj ), the following data for the watershed pixels are stored:
In this case, the following region disimilarity function can be used
=N
and
( ))
M Ri M Rj
X( N
)=1
k
where W (Ri Rj ) is the height of the watershed contour and is equal to the minimum watershed pixel intensity. The computation of watershed contour dynamics can be done using a minimum spanning tree (MST) of the RAG where its edges are weighted by the minimum value of the watershed contour they represent [8]. In the context of hierarchical region merging, the region similarity criterion can be defined such that at each merge, the regions separated by the watershed contour of minimum dynamics are merged. This is justified by the observation that the dynamics of the minimum dynamics contour can be computed using the apparent contour dynamics defined as [8]
Ri R j
(1)
55
their minimum intensity, W (R i Rj ),
their number, kC (R i Rj )k,
the sum of their intensities, S (R i Rj ),
the histogram of their intensities, H (R i Rj ).
The watershed pixel intensity histogram is used for the computation of the median in linear time. During the RAG merging process, at each step, the minimum cost edge is selected using one of the disimilarity functions in equations 1,2,3. The regions linked by the minimum cost edge are merged and the minimum value of the resulting region is M (Ri Rj ) = min(M (Ri ) M (Rj )). This change may cause the increase of the cost of some edges which connect the resulting node with its neighbors. Also, merging the regions Ri Rj may cause the merging of some RAG edges pairs. These pairs correspond to RAG regions which are common neighbors of both R i and Rj . Let Rk be such a common neigbor. The RAG-edges (u i uk ) and (uj uk ) are removed and the edge (u ij uk ) is inserted with the following values
(
) = min( ( ) ( )) )k = k ( )k + k ( )k ) = ( )+ ( )
W Rij Rk
kC (Rij Rj S (Rij Rk
W Ri Rk W Rj Rk
C Ri Rk
S Ri Rk
C Rj R k
S Rj Rk
by simulated additive white Gaussian noise of standard deviation = 5. Figures 3 and 4 show the segmentation results using the segmentation algorithms based on minimum (Eq. (1)), mean (Eq. (2)) and median (Eq. (3)) contour dynamics for the original images and their noisy versions. In all experiments, the image gradient magnitude was computed using the dilation-erosion morphological gradient operator using a 3 3 flat structuring element, without any other preprocessing. It is clear that, the introduction of the proposed approach improves the segmentation performance by preserving certain important contours of low contrast (e.g. outline of the hat in the “Lena” case) which are lost using the minimum dynamics region merging. However, the method is completely dependent on the estimated image gradient magnitude and, therefore, in the case of noisy images its estimation is critical to the method performance.
:
5. CONCLUSIONS
The histograms of the watershed pixel intensities are stored as ordered lists and, therefore, their merge is accomplished in linear time on the list size. Each node of the histogram list represents an intensity and the number of pixels having this intensity. It is clear that the additional computational cost in order to implement the used disimilarity functions is small. This is due to the fact that, during the merging process, the edge costs are updated by simple operations. A fast implementation of the above process is obtained by using an auxiliary directed graph, the Nearest Neighbor Graph (NNG) [2]. In this graph each node has one edge pointing to its most similar neighbor. The regions of the most similar pair constitutes an NNG-cycle of length 2. Such NNG-cycles are kept in a priority queue data structure (heap) for the efficient detection of the most similar pair of regions.
In this paper, we presented an image segmentation method based on region merging applied to an initial morphological decomposition of the gradient image using the watershed transform. The region similarity criterion is based on the concept of watershed contour dynamics. The robustness of the segmentation to the presence of noise and/or low contrast is improved by a regularization of the contour dynamics through the mean and median filters applied on the contour intensities. Experimental results on various kinds of synthetic and real images show improved segmentation results compared to the original contour dynamics criterion. 6. REFERENCES [1] F. Moscheni, S. Bhattacharjee, and M. Kunt, “Spatiotemporal Segmentation Based On Region Merging,” IEEE Trans. on Pattern Anal. and Mach. Intell., vol. 20, no. 9, pp. 897–915, September 1998.
4. EXPERIMENTAL RESULTS The proposed disimilarity functions for region merging were applied to a significant number of synthetic and natural images. Figure 1(Top Left) shows a synthetic image which is piecewise constant, the background intensity level is 80, the object intensity level is 110 and contains simulated additive white Gaussian noise with standard deviation = 10. In Figures 1(Top Right, Bottom), the segmentation results obtained with the three dissimilarity functions in Eqs (1), (2) and (3) for 10 final regions are shown for the synthetic image. The improvement of the segmentation results is clear since the important image contours survive longer to the merging process. Figure 2 shows two commonly used natural 256 256 images (8 bits/pixel) to which the proposed disimilarity functions were applied. Also, the above images were corrupted
[2] K. Haris, S. Efstratiadis, N. Maglaveras, and A. K. Katsaggelos, “Hybrid Image Segmentation Using Watersheds and Fast Region Merging,” IEEE Trans. on Image Processing, vol. 7, no. 12, pp. 1684–1699, December 1998. [3] S. Beucher and C. Lantuejoul, “Use of Watersheds in Contour Detection,” in Proc. Int. Workshop Image Processing, Real Time Edge and Motion Detection / Estimation, Rennes, France, Sept 1979. [4] F. Meyer and S. Beucher, “Morphological Segmentation,” Journal of Visual Communication and Image Representation, vol. 1, no. 1, pp. 21–45, 1990.
56
[5] L. Vincent and P. Soille, “Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations,” IEEE Trans. on Pattern Anal. and Mach. Intell., vol. 13, no. 6, pp. 583–598, June 1991. [6] L. Najman and M. Schmitt, “Geodesic Saliency of Watershed Contours and Hierarchical Segmentation,” IEEE Trans. on Pattern Anal. and Mach. Intell., vol. 18, no. 12, pp. 1163–1173, December 1996. [7] M. Grimaud, “A New Measure of Contrast: Dynamics,” in SPIE Vol. 1769, Image Algebra and Morphological Processing III, San Diego, July 1992. [8] F. Meyer, “The Dynamics of Minima and Contours,” in Mathematical Morphology and its Applications to Image and Signal Processing (ISMM’96), Atlanta, May 1996.
Fig. 3. Segmentation results for “Lena” (Left) and its noisy version (Right) with 200 regions using Eq. (1) (Top), Eq.(2) (Middle) and Eq. (3) (Bottom).
Fig. 1. Segmentation results for the synthetic image (Top Left) with 10 regions using Eq. (1) (Top Right), Eq. (2) (Bottom Left) and Eq. (3) (Bottom Right).
Fig. 2. (Left): “Lena” (Right): “MIT” Fig. 4. Segmentation results for “MIT” (Left) and its noisy version (Right) with 500 regions using Eq. (1) (Top), Eq. (2) (Middle) and Eq. (3) (Bottom).
57
