Image Segmentation of Pearlite Phases Based on

Image Segmentation of Pearlite Phases Based on Directional Morphology and Watershed Transformations Luis Morales-Hernándeza, Federico Manriquez Guerrerob, Iván R. Terol-Villalobosb

a

Facultad de Ingeniería, Universidad Autónoma de Querétaro Río Moctezuma 249 San Juan del Río, Qro., México, 76807

b

CIDETEQ, SC., Parque Tecnológico Querétaro S/N Sanfandila Pedro Escobedo, Qro., México, 76700

Correspondence address: M.C. Federico Manriquez Guerrero Center: Centro de Investigación y Desarrollo Tecnológico en Electroquímica. Mailing address: Parque Tecnológico Querétaro, Sanfandila-Pedro Escobedo. CP.76700-APDO 064.Querétaro, México. Telephone: +52-442-2116016, +52-442-2116000 Telefax: +52-442-2116001 e-mail: [email protected] .

Abstract

The present paper is focused on the segmentation of images of pearlitic microstructures using directional morphological transformations. The proposed approach is useful not only in computing pearlite phase orientation patterns but also for characterizing other structures containing anisotropies (for example, fingerprint images). First, one investigates a global approach based on the directional granulometries using line segments as structuring elements. Next, a local approach using the concept of the maximum distance function is considered. The maximum distance function is computed from the supremum of directional erosions. The maxima of the maximum distance function contain the sizes of the longest lines that can be included in the structure. To determine the directions of the line segments, a second image containing the orientations is built after the maximum distance function is computed. Next, combining the two images, i.e., the maximum distance function and the orientation function, permits the construction of a weighted partition using the watershed transformation. Finally, the elements of the partition are merged according to directional and size criteria for computing the desired segmentation of the image.

Key words: Directional morphology, directional granulometry, maximum distance function, watershed transform, pearlite phases.

Resumen

El presente trabajo se enfoca a la segmentación de imágenes de microestructuras del tipo perlita usando transformaciones morfológicas direccionales. El método propuesto se puede aplicar no solamente en el cálculo de las orientaciones de laminillas de la fase perlita, sino también en otros tipos de estructura conteniendo anisotropías (i.e., huellas digitales por ejemplo). Inicialmente, se investiga un método global basado en granulometrías direccionales usando segmentos de rectas como elementos estructurales. Enseguida, se considera un método local usando el concepto de la función de distancia máxima.

La función distancia máxima se calcula usando el supremo de

erosiones direccionales. Los máximos regionales de la función distancia máxima contiene los tamaños de los segmentos de rectas más grandes que pueden estar incluidos en la estructura. Para determinar las direcciones de estos segmentos de rectas se construye una segunda imagen, que contiene las orientaciones, al momento de crear la función distancia máxima. Enseguida, la combinación de ambas imágenes, i.e., función distancia máxima y la función de orientaciones, permite la construcción de una partición ponderada usando la transformación de la línea divisoria de aguas (watershed). Finalmente, los elementos de la partición son fusionados de acuerdo a criterios de orientación y tamaño para obtener la segmentación deseada de la imagen.

Descriptores: Morfología direccional, granulometría direccional, función distancia máxima, línea divisoria de aguas (watershed), fase perlita.

PACS: 87.57 Nk; 07.05 Pj

1 Introduction

Since many physical and mechanical properties of materials are closely related to their microstructure, a great interest exists in the use of image-processing techniques for determining the relationship between microstructure and material properties [1]. It is clear that whatever the material, evaluation of its quality in relation to an application is necessary. Here, the morphological characteristics play a fundamental role in understanding the properties in practice. In fact, for materials selection, and especially for steels, a critical factor is that the material must possess specified microstructural characteristics such as the proportion of phases, grain size, grain-size distribution, and for some special applications, the phase orientations. Diverse commercial systems for image analysis exist, but none of them can discern grains in materials that contain pearlitic structures appearing as a single phase since they are based solely on the delineation of the grain boundaries and contrast differences [2]. The pearlite phase displays a morphology in the form of parallel lines as shown in Figure (1), and when forming another grain, these can change of direction. Such morphology can generate errors when the software attempts to delineate the grain boundary. Field extraction from an image is a good technique, and when it is associated with the orientation, field extraction becomes a superior method for the characterization of the pearlitic phase. Currently, field-orientation detection is an active subject of research in image processing for fingerprint recognition [3-6]. One of the most interesting works concerning this subject [7] is focused principally on developing efficient algorithms for morphological directional filtering, and the applicability of these tools (directional filters) in fingerprint recognition is suggested. Nevertheless, it is in the area of materials testing where field-orientation detection can play a deeper role because identifying structural features will permit better understanding of the

macroscopic material characteristics. Few studies based on mathematical morphology methodology have been focused on this problem. Given the great interest in orientation pattern models for characterizing microstructures, this paper investigates the use of the mathematical morphology methodology for modeling orientation fields. In particular, this study focuses on the field-orientation detection of pearlite phases by using directional morphological transformations and the watershed transform. In fact, since the pearlite phases can be considered as a structure composed by a set of line segments, a bank of filters which is composed by directional morphological transformations permits the extraction of the main orientations of the image. This is similar to the perception of the orientation of line segments by the human brain. Computer image processing of oriented image structures often requires a bank of directional filters or template masks, each of them sensitive to a specific range of orientations. In order to achieve an image processing that yields image structures, we propose a local approach using the concept of a directional maximum distance function combined with the watershed transformation. In our case, the maximum distance function is computed from the supremum of directional erosions. The maxima of the maximum distance function contain the information of the longest line segments that can be placed inside the structure. In order to know their orientation, a second image is defined by observing the construction of the maximum distance function and its evolution. This second image is computed by detecting the orientation of the supremum of directional erosions. These local descriptors, for the element size and the orientation, enable the identification of the orientation fields based on the watershed transformation. This allows one to describe pearlitic structure patterns in a piecewise manner. This paper is organized as follows. In Section 2, the concepts of morphological filter and directional morphology are presented. In Section 3, the directional granulometry and the

supremum of directional erosions are introduced. In particular in this section the idea of the maximum distance function is proposed. Finally, in Section 4 our proposition of working with directional morphology and the watershed transform for segmenting pearlite phases is introduced.

2

Some Basic Concepts of Mathematical Morphology

2.1 Basic Morphological Filters Morphological filters are considered to be increasing and idempotent transformations [8-10]. While the increasing property requires that the order is preserved, one says that a transformation

Ψ is idempotent if and only if, for all functions f , ψ (ψ(f )) = ψ (f ) . The basic morphological filters are the morphological opening γ µB and the morphological closing ϕµB within a given structuring element µB , where in this work B is an elementary structuring element ( 3× 3 pixels) ( ( that contains its origin. B is the transposed set ( B = {− x : x ∈ B} ), and µ is an homothetic

(scalar) parameter. In this work, the homothetic parameter takes on only integer values. The morphological opening is an anti-extensive filter and the morphological closing an extensive filter. These transformations are expressed by means of the morphological dilation δµB and morphological erosion εµB . Thus,

γ µB (f )( x ) = δ µB( (ε µB (f ))

and

ϕ µB (f )( x ) = ε µB( (δ µB (f ))

(1)

The

morphological

erosion

( ε µB (f ( x )) = ∧{f ( y); y ∈ µB x }

and

and

dilation

are

respectively

expressed

by:

( δµB (f ( x )) = ∨{f ( y); y ∈ µB x }, where ∧ is the inf

operator ( ∨ is the sup operator).

2.1 Morphological Directional Transformations

Morphological directional transformations are characterized by two parameters.

Then a

structuring element L depends on its length (size µ ) and on the slope (angle α ) of this element. Thus, the set of points of a line segment L(α, µ) is computed by two sets of points for α ∈ [0,90] . The set of points {( x i , y i )} computed by the following expressions:

If 0 ≤ α ≤ 45 then, y i = x i tan α for x i = 0,1,..., (µ / 2) cos α If 90 ≥ α > 45 then x i = y i cot α for y i = 0,1,..., (µ / 2) sin α

and by the set of points {( − x i ,− y i )} . This means that the structuring element is a symmetric set L(α, µ) = Lˆ(α, µ) . Similar expressions can be expressed for α ∈ (90,180] . Then, morphological opening and closing are given by:

γ L(α,µ ) (f ) = δ L(α,µ ) ε L( α,µ) (f )

ϕ L (α,µ) (f ) = ε L (α,µ) δ L(α,µ ) (f )

where the morphological erosion and dilation are given by:

(2)

ε L (α,µ ) (f )(x ) = min{f (y ) : y ∈ L(α, µ)(x )} δ L(α,µ ) (f )(x ) = max{f (y ) : y ∈ L(α, µ)(x )}

(3)

3 Directional Granulometry and Supremum of Directional Erosions 3.1 Directional Granulometry

In mathematical morphology there are used operators based on the detection of the residues of parametric transformations. Examples include the ultimate erosion, the skeleton by maxima balls, the granulometry function and the distance function [10]. These last two transformations are computed by means of the differences of successive openings and erosions, respectively. Generally, associated functions are linked to these transformations. For example, with granulometry the density and distribution functions are related to these residues. Granulometry (and anti-granulometry) was formalized by Matheron[11] for binary images and extended to complete lattices by Serra [8]. Granularity is defined as follows: Definition 1. A family of openings {γ λ } (or respectively of closings {ϕ λ } ) where λ ∈ {1,..., n} , is a granulometry (or respectively, anti-granulometry) if for all λ, µ ∈ {1,..., n} and all functions f, λ ≤ µ ⇒ γ λ (f ) ≥ γ µ (f ) (or respectively ϕ λ (f ) ≤ ϕ µ (f ) ) .

The granulometric curves (density and distribution functions) can be computed from the granulometric residues between two different scales [ γ µi (f ) − γ µ j (f ) ] with µ i ≤ µ j or between the original image and one scale [ f − γ µ j (f ) ]. Then, one says that [ γ µi (f ) − γ µ j (f ) ] contains features of f that are larger than the scale µ i but smaller than µ j , and the residue [ f − γ µ j (f ) ]

contains features smaller than µ j . To illustrate the use of the granulometry for detecting anisotropies inside a structure, the binary image in Fig. 2(a) was computed from the gray-level image of Fig. 1(a). A morphological contrast operator called the tophat transformation, given by the arithmetic difference between the closed image ϕ µ (f ) and the original image f, i.e., [ ϕ µ (f ) − f ], was first applied. Then a threshold on this image difference was applied between 4 and 255 gray levels to obtain the image in Fig. 2(a). Now, the following granulometric study, using directional openings was made. Figs. 2(b)-(c) illustrate the output images computed from the image in Fig. 2(a) using a directional opening size 80 at directions 55 and 65 degrees, respectively. Observe that this microstructure contains a main direction at approximately 55 degrees. To detect automatically the main direction in a structure one computes a granulometry. Since one has two parameters; size and direction, the size is fixed, and then the angles are varied from 0 to 180 degrees. Let X be the binary image of Fig. 2(a); then one computes Vol(X) − Vol( γ L(α,λ ) (X )) / Vol(X) , where Vol represents the volume (gray-level integration). This expression permits one to know the percentage of the structure removed by the opening. For some

angles

the

directional

opening

removes

all

of

the

structure,

and

Vol(X) − Vol( γ L (α,λ ) (X )) / Vol(X) = Vol(X) / Vol(X) = 1 , whereas in the direction of the longest structures Vol(X) − Vol( γ L(α,λ ) (X )) / Vol(X) < 1 . This enables us to obtain the curves shown in Fig. 3(a) for different sizes of the structuring element. The global minimum computed from one of these functions permits us to determine the direction of the main structures. The curve in Fig. 3(a), obtained by the directional openings at size 80, was used for detecting the minimum. Figure 3(b) illustrates the position of the minimum. It was computed by using the morphological transformations in one-dimensional case. To detect the minimum, the distribution function was

transformed into the interval [0,255] as integer numbers, and then the traditional tools for detecting minima in mathematical morphology were applied [10]. This method for computing the main direction of the structure is a global approach that does not identify the local direction in each region of the image. In the following section a local approach is introduced that will be used to segment the image.

3.2 Supremum of Directional Erosions (Maximum distance Function)

Since the notion of the distance function will be used to propose a maximum distance function, let us introduce this concept.

Definition 2. The distance function d X ( x ) is a transformation that associates with each pixel x of a set X its distance from the background.

The distance function can be computed by successive erosions of the set X. Now, let us consider also the following definition.

Definition 3. The maximum distance function Dm X ( x ) is a transformation that associates with each pixel x of a set X its maximum distance from the background.

The purpose of building this function consists in codifying the size information in such a way that local directional information can be accessed from each point of the function. This codification of the size information will be used to build a local approach for detecting

orientation fields on an image. The maximum distance function Dm is computed by using the supremum of directional erosions. To stock the size information for all λ values, a gray-level image Dm is used. Thus, one begins with a small structuring element by taking into account all orientations to compute the set

sup {ε L(λ,α ) (X)} . Then one increases Dm(x) by one at all

α∈[ 0,180]

points x belonging to the set

sup {ε L(λ,α ) (X)} . Next, one continues the procedure by

α∈[ 0,180]

increasing the size of the structuring element until the structure (the image) is completely removed. This means that the procedure continues until one has a

λ max value such

that sup {ε L (λ max ,α ) (X)} = ∅ . α∈[ 0,180]

Figures 4(a) and (b) show the output images computed from the original image in Fig. 2(a) for the λ values 40 and 60, respectively. The maxima of the function Dm are the locus of maximal structuring elements. Thus, one knows the position of the greatest structuring elements that can be included completely in the structure. However, the angles of these structuring elements (line segments) are not accessible from the image Dm. Therefore, one stocks the directions of the line segments in a second image Om when the maximum distance function is computed. The images in Fig. 5(c)-(d) illustrate the maximum distance function image and the image containing the orientation. Now, these functions can be used for computing the line segments that characterize the structure. To illustrate the information contained in these images, the maxima of Dm were computed for obtaining the locus of the maximal structuring elements as shown in Fig. 6(a). Next, a line segment was placed at each maximum point “x” of Fig. 6(a), with the size given by the image Dm(x) and with an angle given by the image Om(x). The longest line segments in the image are illustrated in Fig. 6(b). The maximum distance function and its associated orientation image containing the angles serve to suggest a method for segmenting images of perlite phases.

4 Image Segmentation Using Directional Morphology and the Watershed Transformation

Image segmentation is one of the most interesting problems in image processing and analysis. The main goal in image segmentation consists in extracting the regions of greatest interest in the image [12]. A segmentation method must allow the introduction of specific criteria to obtain the desired regions (e.g., gray level, contrast, size, shape, texture,…). In mathematical morphology, the watershed-plus-marker approach is the traditional image segmentation method [12]. This method has proved to be an efficient tool in many image-segmentation problems. The idea of a watershed is drawn from a topographic analogy: consider the gray-level intensity of an image as a topographic relief map. Find the minima and “pierce” them. Immerse the whole relief into water, and let the water flood into the areas through the piercing points. As the relief goes down, some of the flooded areas will tend to merge; prevent this from happening by raising infinitely tall dams along the merging boundaries. When finished, the resulting network of dams defines the watershed of the image. The regions separated by the watershed lines are called catchment basins, each one associated with a single minimum. The main drawback of the watershed method consists in the over-segmentation produced if the watershed transformation is applied directly to the images to be segmented. One solution to prevent this over-segmentation consists in a priori selection of a set of markers locating the regions to be extracted. Here, we apply an alternative approach for segmenting the images of pearlite phases. Instead of looking for a set of markers signaling the regions, the watershed will be directly applied for obtaining a fine partition, and then a systematic merging process will be applied to obtain the final segmentation. Fig. 7(a) shows the inverse image of image Dm in Fig. 5(c), while Fig. 7(b) illustrates its watershed. To realize the merging process it is preferable to

work with the catchment basins associated with the watershed image. Figure 7(c) shows the catchment basins weighted by the values of the angles of the regional maxima of the image Om in Fig. 5(c) (or the minima of. Fig. 7(a)). Now, by analyzing a region of the image in Fig. 7(c), one can identify the neighboring regions with more-or-less similar orientations. In order to take into account their neighborhood relationships, a region adjacency graph (RAG) must be computed. In fact, the RAG simplifies the merging process. Let us introduce some concepts concerning graphs. •

A graph G is a pair made of a set V of vertices and a family of arcs.

Here, one considers the case of a graph without loops. This means that there are no arcs connecting a vertex to itself. In the general case, arcs are oriented; in this work, however, one takes the case of non-oriented graphs:

•

If there exists an arc joining vertex v to vertex v’, then there also exists an arc joining

v’ to v. •

A vertex v’ is said to be a neighbor of a given vertex v if there exists an arc joining v

to v´. Thus, the set of the neighbors of v for a given set of arcs A is denoted N A ( v) and it is defined by: N A ( v) = {v'∈ V : ( v, v' ) ∈ A} . Figure 8(b) illustrates a region-adjacency graph (RAG) of the image in Fig. 8(a). Fig. 8(c) shows the process of merging regions with a difference of gray levels less than or equal to 40. The values inside the nodes represent the gray levels of the image in Fig. 8(a). In our case, the computation of the RAG, using the angles of the regions, guides the subsequent merging of regions and provides a complete description of the neighborhoods. The

RAG graph was constructed by use of the catchment basins of the image in Fig. 7(c). One takes a point from each minimum of the inverse maximum distance function for representing each catchment basin. Remember that the inverse distance function is used. Since the graph under study is a valued, one must introduce some numerical values. Each edge is then assigned a value given by the absolute value of the difference between the angles of two neighboring vertices, computed from the orientations image. The neighborhood graph of the maxima of the maximum distance function and the directional function synthesizes the directional field of the image. Two vertices of the graph are linked by an edge if the catchment basins are neighbors, and the value of the edge represents the directional similarity. Now, we can compute the orientation fields based on the valued graph. We illustrate the method by identifying the adjacent regions with more-or-less similar orientation by considering the image in Fig. 1, a micrograph of the pearlite structure in steel. To achieve such a goal, one merges the vertices (catchment basins) with a difference of angles smaller than or equal to a given angle value θ . Figure 9(a) shows the output image after the merging process for θ equal to 15o. The intensities of regions in Fig. 9(a) were taken to be proportional to the mean value of the merged region angles. To demonstrate the computed directional fields, the contours of the image 9(a) are superimposed on the original image as shown in Figs. 9(b).

5 Conclusion

This paper has shown the possibilities for application of the morphological directional transformations and the watershed transformation to segment images of pearlitic microstructures in steels. The approach involves a local analysis using the concept of the maximum distance function. The maxima of the maximum distance function were used for computing the locus of

maximal structuring elements, and a second function (orientation function) was used to obtain the angles of the line segments. These pairs of local parameters enable us to produce a good description of the image by means of line segment. Then, a partition of the image is computed by means of the watershed transform. This enables us to realize a neighborhood analysis that merges adjacent regions of the partition according to directional and distance criteria, thus segmenting the pearlitic microstructure. The results based on the algorithms presented in this paper show the good performance of the approach.

Acknowledgments

The author Luis Morales thanks the government agency CONACyT for the financial support. The author I. Terol would like to thank Diego Rodrigo and Darío T.G. for their great encouragement. This work was partially funded by the government agency CONACyT (41170), Mexico.

References

[1] A. Galtier, O. Bouaziz and A. Lambert, Mécanique & Industries,3, (2002), 457. [2] A. From, R. Sandström, Applied Materials Technology (1998). [3] A.K. Jain, L. Hong, S. Pankanti, R. Bolle, Proc. IEEE 85 (9) (1997) 1365. [4] D. Maio, D. Maltoni, IEEE Trans. Pattern Anal. Mach. Intell. 19 (1) (1997) 27. [5] J. Gu, J. Zhou, D. Zhang, Pattern Recognition 37 (3) (2004) ,543. [6] J. Zhou, J. Gu, Pattern Recognition 37 (2) (2004), 389. [7] P.Soille and H. Talbot, IEEE Trans. on Pattern Anal. Machine Intell., 23(11), (2001). [8] J. Serra, Image Analysis and Mathematical Morphology, Vol. II, (Academic Press, New York, 1988).

[9] H. J. A. M. Heijmans, Morphological Image Operators, (Academic Press, New York, 1994). [10] P. Soille, Morphological image analysis, (Springer-Verlag, Heidelberg, 2nd edition , 2003.) [11] G. Matheron, Random Sets and Integral Geometry, (Wiley, New York, 1975). [12] F. Meyer, S. Beucher, J. Vis. Comm. Image Represent., 1, (1990), 21.

Table of Figures

Figure 1 a) and b) Pearlite phases from 1080 steel images Figure 2. a) Binary image computed from image in Fig. 1(a), b) and c)Output images obtained by means of directional openings size 80 and angles 55 and 65 degrees, respectively. Figure 3 a) Granulometric curves computed from image in Fig. 1(a) at sizes 20, 40, 60 and 80, b) Minimum detection of the granulometric curve at size 80. Figure 4. a) and b) Output images computed by the supremum of directional openings with sizes 40 and 60, respectively. Figure 5. a) Original image, b) Binary image, c) Maximum distance function computed by the supremum of erosions, d) Image of angles associated to the maximum distance function. Figure 6 a) Regional maxima of the maximum distance function in Fig. 5(c), b) Line segments computed from images in Fig. 5(b)-(c). Figure 7. a) Negative (inverse) image of image in Fig. 5(c), b) Image partition of (a) computed by the watershed transformation, c) Catchment basins of the partition weighted by the orientation (angles). Figure 8. a) Original image, b) Region adjacency graph, c) Merging process Figure 9. a) Merging process of the image in Fig. 8(c) using orientation criterion, b) Segmented image.

(a)

(b) Figure 1

(a)

(b)

(c) Figure 2

1.2

Size distribution

1 0.8

size 80 size 60 size 40 size 20

0.6 0.4 0.2 0 0

50

100

150

200

Angles

(a) Main orientation of the structure 1.2 1 0.8 0.6 0.4 0.2 0 -0.2 0

50

100 Angles

(b) Figure 3

150

200

(a)

(b) Figure 4

(a)

(b)

(c)

(d) Figure 5

(a)

(b)

Figure 6

(a)

(b)

(c)

Figure 7

(a)

(b)

(c) Figure 8

(a)

(b)

Figure 9