Multiscale Image Enhancement and Segmentation Based on ...

Multiscale Image Enhancement and Segmentation Based on Morphological Connected Contrast Mappings Iv´ an R. Terol-Villalobos CIDETEQ,S.C., Parque Tecnol´ ogico Querétaro, S/N, SanFandila-Pedro Escobedo, 76700, Querétaro Mexico, [email protected]

Abstract. This work presents a multiscale image approach for contrast enhancement and segmentation based on a composition of contrast operators. The contrast operators are built by means of the opening and closing by reconstruction. The operator that works on bright regions uses the opening and the identity as primitives, while the one working on the dark zones uses the closing and the identity as primitives. To select the primitives, a contrast criterion given by the connected tophat transformation is proposed. This choice enables us to introduce a well-defined contrast in the output image. By applying these operators by composition according to the scale parameter, the output image not only preserves a well-defined contrast at each scale, but also increases the contrast at finer scales. Because of the use of connected transformations to build these operators, the principal edges of the input image are preserved and enhanced in the output image. Finally, these operators are improved by applying an anamorphosis to the regions verifying the criterion.

1

Introduction

Image enhancement is a useful technique in image processing that permits the improvement of the visual appearance of the image or provides a transformed image that enables other image processing tasks (image segmentation, for example). Methods in image enhancement are generally classified into spatial methods and frequency domain ones. The present work is focused on the spatial methods, and in particular, to the use of morphological image transformations. The methods presented here, not only have the objective of improving the visualization, but also to be used as a preprocessing step for image segmentation. In mathematical morphology (MM), several works have been focused on the contrast enhancement ([1],[2], [3], [4], [5],[6]). Among them, some interesting works concerning multiscale contrast enhancement were made by Toet ([1]), and Mukhopadhyay and Chanda ([2]). Toet proposes an image decomposition scheme based on local luminance contrast for the fusion of images. In particular, the use of the alternating sequential morphological filters as a class of low-pass filters was proposed in [1]. On the other hand, Mukhopadhyay and Chanda [2] propose a decomposition R. Monroy et al. (Eds.): MICAI 2004, LNAI 2972, pp. 662–671, 2004. c Springer-Verlag Berlin Heidelberg 2004

Multiscale Image Enhancement and Segmentation

663

of the image based on the residues obtained by the tophat transformation. However, the first formal work in morphological contrast was made by Meyer and Serra [6] who propose a framework theory for morphological contrast enhancement based on the activity lattice structure. In their work, the original idea of Kramer and Bruckner (KB) transformation [7] was used. This transformation, which sharpens the transitions between the object and background, changes the gray value of the original image at each point of the image for the closest of the dilation and erosion values. The contrast operators proposed by Serra and Meyer progress in the way suggested by KB, but the hypotheses are modified. They not only assume that the transformations are extensive and anti-extensive, but also that the transformations must be idempotent. The use of this last hypothesis to build contrast operators, avoids the risk of degrading the image. Another form that allows an attenuation of the image degradation problem in the KB algorithm was proposed by Terol-Villalobos [8]. In his work, the dilation and erosion transformations are also used as in the KB transformation, but in a separated way to build a class of non-increasing filters called morphological slope filters (MSF). On the other hand, the proximity criterion is not used, and a gradient criterion is introduced. An extension of this class of filters was proposed in [5]. In this case, the MSF are sequentially applied rendering a selection of features at each level of the sequence of filters. Recently, in [9], a class of connected MSF was proposed by working with the flat zone notion. The present work progresses in the same way suggested in [6], but using the opening and closing by reconstruction as primitives. However, in a similar manner for the dilatation and erosion in MSF, the opening and closing by reconstruction will be used separately. On the other hand, the proximity criterion will be avoided and a contrast criterion, given by the tophat transformation will be used for selecting the primitives. Furthermore, the use of a tophat criterion is combined with the notion of multiscale processing to originate a powerful class of contrast mappings. The use of filters by reconstruction, that form a class of connected filters, will allow the definition of a multiscale approach for contrast enhancement.

2 2.1

Some Basic Concepts of Morphological Filtering Basic Notions of Morphological Filtering

The basic morphological filters ([10]) are the morphological opening γµB and the morphological closing ϕµB with a given structuring element ; where, in this work, ˇ B is an elementary structuring element (3x3 pixels) that contains its origin. B ˇ is the transposed set (B = {−x : x ∈ B}) and µ is an homothetic parameter. The morphological opening and closing are given, respectively, by: γµB (f )(x) = δµBˇ (εµB (f ))(x)

and ϕµB (f )(x) = εµBˇ (δµB (f ))(x)

(1)

where the morphological erosion εµB and dilation δµB are expressed by: εµB (f )(x) = ∧{f (y) : y ∈ µBˇx } and δµB (f )(x) = ∨{f (y) : y ∈ µBˇx }. ∧ is the inf operator and ∨ is the sup operator. In the following, we will avoid the

664

I.R. Terol-Villalobos

elementary structuring element B. The expressions γµ ,γµB are equivalent (i.e. γµ = γµB ). When the homothetic parameter is µ = 1, the structuring element B will also be avoided (i.e. δB = δ). When µ = 0, the structuring element is a set made up of one point (the origin). Another class of filters is composed by the opening and closing by reconstruction. When filters by reconstruction are built, the basic geodesic transformations, the geodesic dilation and the geodesic erosion of size 1, are iterated until idempotence is reached [11]. Where the geodesic dilation and the geodesic erosion of size one are given by δf1 (g) = f ∧ δ(g) with g ≤ f and ε1f (g) = f ∨ ε(g) with g ≥ f , respectively. When the function g is equal to the erosion or the dilation of the original function, we obtain the opening and the closing by reconstruction: γ µ (f ) = lim δfn (εµ (f )) ϕ µ (f ) = lim εnf (δµ (f )) n→∞

2.2

n→∞

(2)

Connectivity and Connected Tophat Transformations

An interesting way of introducing connectivity for functions is via the flat zone notion and partitions. Concerning the flat zone notion, one says that the flat zones of a function are the largest connected components of points with the same gray-level value. On the other hand, a partition of a space E is a set of connected components {Xi } which are disjoint (Xi ∩ Xj = ∅) and the union is the entire space (∪Xi = E). Thus, since the set of flat zones of a function constitutes a partition of the space, a connected operator for functions can be defined as follows. Definition 1 An operator ψ acting on gray-level functions is said to be connected if, for any function f, the partition of flat zones of ψ(f ), is less fine than the partition f . The openings and closings by reconstruction (eqn. (2)) are the basic morphological connected filters. When applying these filters the flat zones increase the size with µ; the flat zones are merged. Based on these transformations, other connected transformations can be defined. In particular the connected tophat transformation is computed by the arithmetic pointwise difference of the original function from the opened one (or the difference of the closed function from the original one): T hwλ (f )(x) = f (x) − γ λ (f )(x) (and T hbλ (f )(x) = ϕ λ (f )(x) − f (x)). Below, the opening (closing) by reconstruction will be used as primitive to built the contrast operator, while the connected tophat transformation will be used as criterion to select the primitives.

3

Contrast Mappings Based on Tophat Criteria

In this section, a study of the contrast mappings, using a contrast criterion given by the tophat transformation, is made. The interest in the use of this type of


665

criterion consists in knowing strictly the contrast introduced in the output image. Consider the two-state contrast mappings defined by the following relationships:  µ (f )](x) ≤ φ µ (f )(x) if [f − γ γ κγµ,φ (f )(x) =  f (x) if [f − γ µ (f )](x) > φ  µ (f ) − f ](x) ≤ φ µ (f )(x) if [ϕ ϕ (f )(x) = (3) κϕ µ,φ  f (x) if [ϕ µ (f ) − f ](x) > φ Both operators κγµ,φ and κϕ µ,φ are connected; the partitions of the output images computed by these operators are composed by flat zones of f and other flat zones merged by γ µ and ϕ µ . The first operator works on bright structures, whereas the second one on the dark regions. The use of a contrast criterion to build these operators permits the classification of the points in the domain of definition of f in two sets. A set Sµ,φ (f ) composed by the regions of high contrast, where for all points x ∈ Sµ,φ (f ) [f − γ µ (f )](x) > φ f or

κγµ,φ

and

[ϕ µ (f ) − f ](x) > φ

f or

κϕ µ,φ

c and the set Sµ,φ (f ) composed of weak contrast zones (the complement of c (f ) Sµ,φ (f )), where for all points x ∈ Sµ,φ

[f − γ µ (f )](x) ≤ φ

f or

κγµ,φ

and

[ϕ µ (f ) − f ](x) ≤ φ

f or

κϕ µ,φ

Remark 1 In general, the operator κγµ,φ using the opening as pattern will be analyzed. Then, for convenience, the notation κµ,φ will be used instead of κγµ,φ . When it is required, the type of primitive (opening or closing) will be specified. Now, let us show how the contrast is modified for obtaining the output image. By construction, the contrast mapping κγµ,φ = κµ,φ is an anti-extensive transformation. Thus, the following inclusion relation is verified: γµ (f ) ≤ κµ,φ (f ) ≤ f . Then, the operator increases the contrast of a region by attenuating the neighboring regions with the opening by reconstruction. On the other hand, the contrast operators based on a tophat criterion not only classify the high and weak conc trast regions ( Sµ,φ (f ) and Sµ,φ (f ) ) of the input image, but also impose a well defined contrast to the output image, as expressed by the following property: Property 1 The output image computed by κµ,φ has a well-defined contrast. For all point x of its domain of definition, the tophat transformation value of κµ,φ (f )(x) is: [κµ,φ (f )(x) − γ µ (κµ,φ (f ))](x) > φ [κµ,φ (f )(x) − γ µ (κµ,φ (f ))](x) = 0

for all points x ∈ Sµ,φ (f ) c for all points x ∈ Sµ,φ (f )

and

Now, since Sµ,φ (κµ,φ (f )) = Sµ,φ (f ), one has: Property 2 The two-state contrast operator κµ,φ is an idempotent transformation; κµ,φ κµ,φ (f ) = κµ,φ (f ).

666


Similar results can be expressed for the contrast operator using the closing by reconstruction and the original function as primitives. Finally, in a composition of two-state contrast mappings based on the parameter φ the strongest operator imposes its effects: κµ,φi κµ,φj = κµ,φj κµ,φi = κµ,max{φi ,φj }

4

Multiscale Morphological Contrast

In this section, the different scales (sizes) of the image will be taken into account for increasing the contrast of the output image. To introduce the scale parameter, a composition of contrast operators depending on the size parameter will be applied. In order to generate a multiscale processing method some properties are needed. Between them, causality and edge preservation are the most important ones [12]. Causality implies that coarser scales can only be caused by what happened at finer scales. The derived images contain less and less details: some structures are preserved; others are removed from one scale to the next. Particularly, the transformations should not create new structures at coarser scales. On the other hand, if the goal of image enhancement is to provide an image for image segmentation, one requires the edge preservation; the contours must remain sharp and not displaced. It is clear that openings and closings by reconstruction preserve contours and regional extreme. In fact, they form the main tools for multiscale morphological image processing. Consider the case of a composition of two contrast operators, and defined by the update equation:  µ2 (κµ1 ,φ (f ))](x) ≤ φ µ2 (κµ1 ,φ (f ))(x) if [κµ1 ,φ (f ) − γ γ κµ2 ,φ κµ1 ,φ (f )(x) =  κµ1 ,φ (f )(x) if [κµ1 ,φ (f ) − γ µ2 (κµ1 ,φ (f ))](x) > φ (4) Figure 1 illustrates an example of a composition of two-state contrast mappings with parameters µ1 < µ2 and φ. In Fig. 1(b) the opening size of the function in Fig. 1(a) is illustrated in dark gray color, whereas the regions removed by the opening are shown in bright gray color. In Fig. 1(c) the output funtion κµ1 ,φ (f ) is shown. Finally, in Figs. 1(d) and 1(e) the opening γ µ2 (κµ1 ,φ (f )) and the output image computed by the composition κµ2 ,φ κµ1 ,φ are illustrated. Observe that µ2 as shown in Fig. 1(e). the high contrast regions of γ µ1 are not modified by γ In the general case, when a family {κµk ,φ } of contrast operators is applied by composition, the following property is obtained. Property 3 The composition of a family of contrast operators {κµk ,φ } , with µ1 < µ2 < · · · < µn , preserves a well-defined contrast at each scale. For a given µi , such that 1 ≤ i ≤ n, and for all points x ∈ Sµi ,φ (κµn ,φ · · · κµ2 ,φ κµ1 ,φ (f )); µi (κµn ,φ · · · κµ2 ,φ κµ1 ,φ (f ))](x) > φ; [κµn ,φ · · · κµ2 ,φ κµ1 ,φ (f ) − γ and for all points x ∈ Sµc i ,φ (κµn ,φ · · · κµ2 ,φ κµ1 ,φ (f )),


667

Fig. 1. a) Original function, b) Opening by reconstruction γµ1 (f ), c) Output function κµ1 ,φ (f ), d) Opening by reconstruction γµ2 (κµ1 ,φ (f )), e) Output function κµ2 ,φ κµ1 ,φ (f )

[κµn ,φ · · · κµ2 ,φ κµ1 ,φ (f ) − γ µi (κµn ,φ · · · κµ2 ,φ κµ1 ,φ (f ))](x) = 0; and the structures at scale µi are preserved,Sµi ,φ (κµn ,φ · · · κµ2 ,φ κµ1 ,φ (f )) = Sµi ,φ (κµi ,φ · · · κµ2 ,φ κµ1 ,φ (f )) For a composition of a family {κµk ,φk } of contrast operators the structures at scale µi are preserved under the condition φ1 ≥ φ2 ≥ · · · ≥ φn . Finally, the composition of contrast operators increases the contrast at finer scales as expressed by the following property: Property 4 In a composition of a family of contrast operators {κµk ,φ } , with µ1 < µ2 < · · · < µn , the following inclusion relation can be established. For a given µi such that 1 ≤ i ≤ n, and for all points x ∈ Sµi ,φ (κµn ,φ · · · κµ2 ,φ κµ1 ,φ (f )); [κµn ,φ · · · κµ2 ,φ κµ1 ,φ (f ) − γ µi (κµn ,φ · · · κµ2 ,φ κµ1 ,φ (f ))](x) ≥ µi (κµi ,φ · · · κµ2 ,φ κµ1 ,φ (f ))](x) [κµi ,φ · · · κµ2 ,φ κµ1 ,φ (f ) − γ Figure 2(b) shows the output image κµ2 ,φ κµ1 ,φ (f ), with µ1 = 64, µ2 = 96, and φ = 15, while Fig. 2(d) and 2(e) illustrate the binary images computed by a threshold between 10 and 255 gray-levels of the internal gradients of the original image and the output image κµ2 ,φ κµ1 ,φ (f ). Figures 2(c) and 2(f) show the output image κµ3 ,φ κµ2 ,φ κµ1 ,φ (f ) with µ1 = 32, µ2 = 64, µ2 = 96, φ1 = 10, φ2 = φ3 = 15, and its binary image computed by means of a threshold between 10 and 255 gray-levels of its internal gradient. Finally, Figs. 2(g)-(i) illustrate the segmented images obtained from the images in Figs. 2(d)-(f). By comparing Figs. 2(h) and 2(i) observe the structures at scale µ1 = 32 introduced in the

668


(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

Fig. 2. a) Original image, b), c) Output images κµ2 ,φ κµ1 ,φ (f ), κµ3 ,φ κµ2 ,φ κµ1 ,φ (f ) d), e), f), Gradient contours of images, (a),(b),(c), and g), h), i) Segmented images

output image 2(f). For a complete study of image segmentation in mathemathical morphology see [13].

5

Some Improved Multiscale Contrast Algorithms

The above-described approach presents a main drawback; this approach does not permit to increase the gray-levels of the image. To attenuate this inconvenience, some modifications to the multiscale contrast operator above-proposed are studied. On the other hand, when a family of contrast operators is applied by composition, one begins with a smallest structuring element. Here, one also illustrates the case of a composition of contrast mappings beginning with the greatest structuring element. 5.1

Linear Anamorphosis Applied on the Tophat Image

Consider the following residue: aµ,φ (f )(x) = 0 if f − γ µ (f ) ≤ φ and aµ,φ (f )(x) = [f − γ µ (f )](x) if f − γ µ (f ) > φ. Then, the output image κµ,φ (f ) can be expressed in the form: κµ,φ (f ) = γ µ (f ) + aµ,φ (f ) Now, let us take the linear anamorphosis: αaµ,φ (f ), where α is a positive integer. Then, a new two-state contrast mapping will be defined by; κµ,φ (f )(x) = γ µ (f )(x) + αaµ,φ (f )(x)

(5)

When the parameter α is equal to one, we have κµ,φ = κµ,φ . Take α as the minimum integer value that enables us to increase the contrast (α = 2 ). Let us


669

consider some conditions for this last operator. If the parameter φ takes the zero µ (f ))] is obtained, and if φ takes value, the extensive operator κµ,φ = [f + (f − γ the maximum value of function aµ,φ (f ), one has that for all points in the domain µ (f ). Now, for other φ values, one can define the set of of definition of f, κµ,φ = γ high contrast points Sµ,φ (κµ,φ (f )) that satisfy: for all points x ∈ Sµ,φ (κµ,φ (f )), κµ,φ (f )(x) = [f +aµ,φ (f )](x) and κµ,φ (f )(x)− γµ (f )(x) = 2aµ,φ (f )(x) ≥ φ. With c (κµ,φ (f )), one has: for all points regard to the set of weak contrast points Sµ,φ c µ (f )(x) and κµ,φ (f )(x) − γ µ (f )(x) = 0. Thus, x ∈ Sµ,φ (κµ,φ (f )), κµ,φ (f )(x) = γ the operator κµ,φ increases twice the contrast of the output image with respect to the operator κµ,φ . If a composition of two contrast operators κµ2 ,φ2 κµ1 ,φ1 (f ) (µ1 < µ2 ) is applied, the parameter φ2 can take twice the value of φ1 without affecting the structures preserved by the first operator. But this is not the only advantage, because this operator increases the contrast of a region using two ways: by attenuating the neighboring regions and by increasing its gray-levels. The images in Fig. 3 (b) and 3(c) illustrate the performance of this operator. A composition of three contrast mappings κµ3 ,φ3 κµ2 ,φ2 κµ1 ,φ1 , with µ1 = 8, µ2 = 16, µ2 = 48, and φ1 = 0, φ2 = 3, φ3 = 7 was applied to the original image in Fig. 3(a) to obtain the image in Fig. 3(b), whereas the composition κµ1 ,φ1 κµ2 ,φ2 κµ3 ,φ3 , with the same parameters was used to compute the image in Fig. 3(c). 5.2

Contrast Operator on Bright and Dark Regions

All the contrast operators, introduced in this paper, work separately with bright or dark regions. Here, the main interest is to introduce a contrast operator that permits the process of both regions. Consider the following operators κγ µ (f )(x) + αaγµ,φ (f )(x) κϕ µ (f )(x) − αaϕ µ,φ (f )(x) = γ µ,φ (f )(x) = ϕ µ,φ (f )(x) These contrast operators will be used as the primitives for building a new contrast operator. Observe that symbols γ and ϕ are now introduced in the contrast operators. In order to build such an operator, another criterion to choose the primitives must be introduced. The natural criterion is the comparison between the tophat on white regions with that on black regions. Thus, the contrast operator, working on bright and dark regions, will be given by:  γ µ2 (f ) − f ](x) ≤ [f − γ µ1 (f )](x)  κµ1 ,φ (f )(x) if [ϕ γϕ κµ1 ,µ2 ,φ (f )(x) = (6)  ϕ κµ2 ,φ (f )(x) otherwise Figure 3(d) shows the output image computed from the image in Fig. 3(a) by γϕ the contrast mapping κγϕ µ1 ,µ2 ,φ2 κµ1 ,µ2 ,φ1 . The selected sizes for the primitives were µ1 = 16, µ2 = 8 for the first operator and µ1 = 48, µ2 = 16 for the second one, while the parameter values φ1 , φ2 were taken equal to zero. A similar composition of contrast operators was applied on Fig. 3(a) for obtaining the image in Fig. 3(e), but in this case the parameter values φ1 and φ2 were selected as φ1 = 0 and φ2 = 7. Compare this last image with that in Fig. 3(d), and observe how the

670


(a)

(b)

(c)

(d)

(e)

(f)

Fig. 3. a) Original image f , b), c) Output images κµ3 ,φ3 κµ2 ,φ2 κµ1 ,φ1 (f ) and κµ1 ,φ1 κµ2 ,φ2 κµ3 ,φ3 (f ), with parameters µ1 = 8, µ2 = 16, µ2 = 48, and φ1 = 0, φ2 = 3, φ3 = 7, d), e) Output images κγϕ κγϕ (f ) with parameters µ1 = 16, µ2 = 8, µ ,µ ,φ2 µ1 ,µ2 ,φ1 1

2

µ1 = 48, µ2 = 16 using φ1 = φ2 = 0 for (d) and φ1 = 0, φ2 = 7 for (e), f) Output γϕ image κγϕ µ1 ,µ2 ,φ1 κµ ,µ ,φ2 (f ) with the same parameters of (e). 1

2

contrast is increased when the gray-level of some regions is attenuated by the opening or closing. Finally, Fig. 3(f) illustrates the output image computed by γϕ κγϕ µ1 ,µ2 ,φ1 κµ1 ,µ2 ,φ2 . with the same parameter values used to compute the image in Fig. 3(e).

6

Conclusion and Future Works

In this work, a multiscale connected approach for contrast enhancement and segmentation based on connected contrast mappings has been proposed. For building the contrast operators, a contrast criterion given by the tophat transformation was used. This type of criterion permits the building of new contrast operators which will enable us to obtain images with a well-defined contrast. When applying by composition a family of contrast operators depending on a size parameter, a multiscale algorithm for image enhancement was generated. The output image computed by a composition of contrast operators preserves a well-defined contrast at each scale of the family. Finally, the use of anamorphoses was introduced to propose some improved multiscale algorithms. Future works on the multiscale contrast approach will be in the direction to extend the approach to the morphological hat-transform scale spaces proposed in [14].


671

Acknowledgements. The author I. Terol would like to thank Diego Rodrigo and Dar´ıo T.G. for their great encouragement. This work was funded by the government agency CONACyT (41170), Mexico.

References 1. Toet, A.: Hierarchical image fusion, Machine Vision and Applications, 3 (1990) 1–11 2. Mukhopadhyay, S., Chanda, B.: Multiscale morphological approach to local contrast enhancement, Signal Process., 80(4) (2000) 685–696 3. Schavemaker, J.G.M., Reinders, M.J.T., Gerbrands, J.J., Backer, E.: Image sharpening by morphological filters, Pattern Recognition, 33 (2000) 997–1012 4. Potjer, F.,K.: Region adyacency graphs and connected morphological operators. Mathematical Morphology and Its Applications to Image and Signal Processing P. Maragos, Schafer R.W., M.A. Butt,(Eds.), Kluwer, (1996) pp. 111–118 5. Terol-Villalobos, I.R., Cruz-Mandujano, J.A.: Contrast enhancement and image segmentation using a class of morphological nonicreasing filters, Journal Electronic Imaging, 7, (1998) 641–654 6. Meyer, F., Serra, J.: Contrast and activity lattice, Signal Process., 16(4) (1989) 303–317 7. Kramer, H.P.,Bruckner, J.B.: Iteration of non-linear transformations for enhancement of digital image, Pattern Recognition, 7(4) (1975) 53–58 8. Terol-Villalobos, I.R.: Nonincreasing filters using morphological gradient criteria, Opt. Engineering, 35, (1996) 3172–3182 9. Terol-Villalobos, I.R.: Morphological image enhancement and segmentation. Advances in Imaging and Electron Physics. Hawkes, P.W. (Ed.), Academic Press,118, (2001) pp. 207–273 10. Serra, J.: Image analysis and mathematical morphology vol. 2. Academic Press, (1988) 11. Vincent, L.: Morphological grayscale reconstruction in image analysis: Applications and efficient algorithms, IEEE Transactions on Image Processing, 2 (1993) 176–201 12. Meyer, F., Maragos, P.: Nonlinear scale-space representation with morphological levelings, J. Vis. Comm. Image Represent., 11(3) (2000) 245–265 13. Meyer, F., Beucher, S.: Morphological segmentation, J. Vis. Comm. Image Represent., 1 (1990) 21–46 14. Jalba, A.C., Roerdink, J.B.T.M., Wilkinson, M.H.F.: Morphological hat-transform scale spaces and their use in testure classification. In Proc. Int. Conf. Image Proc., Barcelona, Spain, Vol. I (2003) 329–332