Morphological Boundary Pixel Classification Marcin Iwanowski Warsaw University of Technology, Institute of Control and Industrial Electronics, Warszawa, Poland Email:
[email protected]
Abstract— This paper presents the method which allow classifying pixels of the internal boundary of objects on the binary image. The application of morphological tools allows grouping internal boundary pixels into four spatial classes: core boundary, isolated regions, branches and corridors. The classification of boundary pixels provides us with a tool for shape description both locally and globally. This description is used to characterize the pixels themselves as well as to describe the image objects. In the latter case the coefficients based on the quantity of pixels of different classes are calculated. These coefficients may be used as features for pattern recognition.
..................... .11111..1..1....1.1.. .11111..1111....1.11. .11111111111111.1..1. .11111111111111.1.11. .11111111111..11111.. .11111..111...11111.. .11111..11...111111.. .11111..1....111111.. ..............11111.. ...11..........11.... .....................
..................... .aaaaa..b..b....b.b.. .a111a..aaaa....b.bb. .a111aaaa11accc.b..b. .a111111111accc.b.bb. .a111aaaa1aa..aaaaa.. .a111a..aaa...a111a.. .a111a..bb...ba111a.. .aaaaa..b....ba111a.. ..............aaaaa.. ...dd..........bb.... .....................
Fig. 1. Test shape (left) and four types of boundary pixels: a - core boundary, b - branches, c - corridors (right), d - isolated region pixels.
Keywords— image analysis, mathematical morphology
I. I NTRODUCTION Mathematical morphology [6], [1] provides us with efficient tools for binary object boundary computing the morphological gradient, which can be computed in various ways. This paper focuses on one of two kind of boundaries - the internal boundary. The binary image considered in this paper consist of objects - connected components of pixels of value 1. The set of all objects will be also referred to as the foreground, while the rest of the image - the set of all pixels of value 0 as the background. By definition the internal boundary consists of such foreground pixels which have, within their neighborhood (defined by the structuring element), at least one pixel of belonging to the image background. An application of morphological erosion shrinks all the foreground areas on the binary image. The way of shrinking depends on the structuring element used. The eroded image contains the core region of the original image, which refers to the area not removed by the erosion. The difference between original and eroded image is the internal boundary of the original image. The proposed approach goes beyond the simple classification of pixel as boundary/non-boundary one. It focuses on the pixels belonging to the internal boundary and allows classifying every pixel belonging to this boundary into one of four classes: 1) Core boundary pixels - all pixels which belong to the external boundary of the core area. 2) Isolated regions’ pixels - pixels belonging to regions that does not contain core area. 3) Corridors - pixels which are not core boundary pixels but which connect two core areas 4) Branches - all the other boundary pixels. All these classes are shown in Figure I. In the proposed method the morphological image processing tools are applied to the detection of all four
classes. In order to find the corridors, in addition, the anchored skeletonization is applied. The proposed classification is a useful tool for pixel and region description and may be also used to perform the object recognition. The paper is organized as follows. Section II contains the basic notions. Section III describes the method itself. Section IV introduces the numerical coefficients based on the boundary pixel classification. Section V shows two examples and, finally, section VI concludes the paper. II. BASICS A. Morphological Gradient Morphological gradient is one of the most widely known morphological operator [6], [1]. It combines the morphological erosion, dilation and the image itself using the subtraction. In case of the binary images the word gradient is usually replaced by contour because it results in the obtaining the contour of image objects. Three types of morphological gradient (contour) are defined for binary images: gradient by erosion, by dilation and morphological gradient. They are defined respectively as follows: ρ− B (f ) = f \ (f B)
(1)
ρ+ B (f ) = (f ⊕ B) \ f
(2)
ρB (f ) = (f ⊕ B) \ (f B)
(3)
were f ⊕ B stands for the dilation of image f with structuring element B and f B stands for the erosion. Usually the gradient is computed with the elementary or directional structuring element. In the first case the closest neighborhood in a given pixel grid is considered. It results in either 4- or 8-pixel neighborhood (B = N4 or B = N8 , shown in Figure 3(b) and Figure 3(a) respectively). In case of the directional structuring element, the neighbor(s) located only in a given direction
are considered. Apart from these traditional structuring elements also the thick gradient is also defined [1]. In this case wider neighborhood is considered, for example B = nB = B ⊕ B ⊕ ... ⊕ B where B stands for an elementary structuring element (N4 or N8 ). The thickness and shape of boundaries, detected by the morphological gradient, is defined by the size and shape of the structuring element. In the part of the paper that follows the thick gradient by erosion is used. The pixels belonging to the internal boundary of given thickness are classified by the proposed method to one of four classes. B. Morphological Reconstruction Morphological reconstruction is the operation which allows extracting from the binary image only given objects (connected components). These objects are indicated using the supplementary image called mask image. Morphological reconstruction is defined as a series of geodesic (or conditional) dilations which are defined as follows: (f ⊕g B)(n) = (...(((f ⊕ B) ∩ g) ⊕ B) ∩ g)... ∩ g) (4) n−times
where n stands for the size of geodesic dilation, f is an initial image and g is a mask image. By performing geodesic dilations one after another, at certain moment the image under processing stops to change. This final image is defined as the result of reconstruction by dilation. It can be formulated as follows: Rg (f ) = (f ⊕g B)(k)
(5)
k = arg min (f ⊕g B)(i) = (f ⊕g B)(i+1)
(6)
where:
i
Image f used in the equations 5 and 6 is referred to as the marker image because it contains markers which indicate object on the mask image g. In other words, the marker image is reconstructed according to the content of the mask, which remains unchanged. The morphological reconstruction can be computed either using the above definition or using specialized algorithms, which provides us with much faster computation of the reconstruction [2]. C. Anchored Skeletonization Skeletonization is used in the proposed methodology to find the corridor areas i.e. these areas which connect at least two core regions. The way of performing skeletonization is based on the notion of simple (or deletable) pixel [7], [5] i.e. such a pixel that its removal does not change the homotopy of the binary image. A foreground pixel p belonging to the image f is simple if and only if it satisfies the following three conditions: 1. NG (p) ∩ f = ∅, 2. NG (p) ∩ f C = ∅, 3. ∃S ∈ CC G (N8 (p) ∩ f ) such that NG (p) ∩ f ⊆ S.
Fig. 2.
The hierarchy of spatial classes.
where f C stands for the complement of image f . NG and NG represents the closest neighborhood of foreground and background pixels respectively. Due to connectivity paradox, different connectivity should be used for the foreground and for the background. So either G = 8 and NG represents 8 closest foreground neighbors (horizontal, vertical and diagonal one) and G = 4 so that NG represents 4 closest background neighbors (without diagonal one), or inversely. Function CCG returns the set of G-connected components of its argument. The fast computation of simpleness can be achieved using the look-up table for the neighborhood configurations. By the successive removal of simple pixels the image is thinned. Image obtained by this thinning performed till idempotence is a skeleton which is also referred to as homotopic marking. In order to get more balanced skeleton it is advised to perform the thinning in two stage iterative process. In the first phase the simple pixels are detected within the whole image (but not yet removed). The removal is performed in the second phase. These two phases are performed iteratively until idempotence. In order to have the possibility to control the thinning process, the notion of anchor pixels has been introduced [4]. These pixels are defined separately and - by definition - cannot be removed during the thinning process even if they are simple. The anchored skeletonization requires two input images - the image to be thinned and the image containing anchor pixels referred to as anchor image. III. B OUNDARY C LASSES The pixels belonging to the internal contour defined by the Eq. 1 are classified into four spatial classes: core boundary, isolated region, corridor and branch. The rest of the object area, a part of initial image which does not belong to internal contour will be referred to as core regions. Classes are detected in a step-wise process. At each step one class is excluded from the internal boundary following the class hierarchy which is shown in Figure 2. A. Core Boundary Pixels and Isolated Regions’ Pixels First two detected classes are core boundary and isolated regions. The isolated regions are these image objects (connected components) which does not contain the core region - they are too small in relation to the size of the structuring element used. Core boundary consists is an external boundary of core regions - a set of pixels which
do not belong to the core region but which have within their neighborhood (defined by the structuring element) at least one pixel belonging to the core region. By using the morphological erosion with the given structuring element, the image is shrunk. The smallest objects (i.e. noise) can also disappear. This operation divides, in fact, objects of the input image into two groups - small (in reference to the size of the structuring element) objects that disappear after the erosion, and large objects that remains, but are shrunk. The shrunk areas of object are the core regions. In other words, large object contains core regions while small ones - not. All the pixels belonging to small regions belong to the internal boundary of the initial image. These pixels are classified as isolated regions’ pixels. The rest of boundary pixels are further classified into three groups. The first of them is the locus of all pixels that are neighbors of the core area. Such pixels are classified as core boundary pixels. In fact core boundary pixels can be detected as the difference of input image opening (erosion followed by the dilation) and erosion.
fcb = (f B) ⊕ B
T
\ (f B)
(7)
where f stands for initial image, and B T for the transposition of structuring element B (B T = {p : −p ∈ B}, for elementary structuring elements N4 and N8 obviously B T = B). Isolated regions’ pixels can be detected using the opening by reconstruction, i.e. reconstruction by dilation with the initial image used as the mask image. The core areas image i.e. the result of erosion of the original image is used as the marker image: fis = f \ Rf (f B)
(8)
B. Branches and Corridors The remaining pixels can belong to regions of two kinds. Both are characterized by its thickness relative to the size of the structuring element. They are too thick and elongated to belong to the core boundary. On the other hand a single connected component can consists of more than one core region. This also implies two kinds of ’thick and elongated’ regions. The first one is stretched between two core regions - is a corridor connecting them, while the second one is just a branch connected to a single core region. The remaining part of the contour is obtained by subtracting the already detected classes (isolated regions’ pixels and core boundary) from the internal contour: frc = f \ ((f B) ∪ fcb ∪ fis )
(9)
The pixels belonging to this remaining part are classified into two classes: corridors and branches. In order to find corridors, the anchored skeletonization is applied. This is due to the fact that when performing the anchored skeletonization of the input image with the core areas used as anchors, the parts of input image that
......... ......... ......... ...111... ...111... ...111... ......... ......... ......... (a)
......... ......... ..11111.. ..11111.. ..11111.. ..11111.. ..11111.. ......... ......... (b)
......... .1111111. .1111111. .1111111. .1111111. .1111111. .1111111. .1111111. ......... (c)
Fig. 3. Structuring elements covering the neighborhood of size 1 - N8 (a), 2 - N8 ⊕ N8 (b) and 3 - N8 ⊕ N8 ⊕ N8 (c) in 8-connected pixel grid (radius computed according to the max-norm). ......... ......... ......... ....1.... ...111... ....1.... ......... ......... ......... (a)
......... ......... ....1.... ...111... ..11111.. ...111... ....1.... ......... ......... (b)
......... ....1.... ...111... ..11111.. .1111111. ..11111.. ...111... ....1.... ......... (c)
Fig. 4. Structuring elements covering the neighborhood of size 1 - N4 (a), 2 - N4 ⊕ N4 (b) and 3 - N4 ⊕ N4 ⊕ N4 (c) in 4-connected pixel grid (radius computed according to the city-block distance).
connects core regions are detected. Since however, the result of skeletonisation is one-pixel thick line, it does not cover the whole corridor areas. In order to the complete areas, the reconstruction is used, as described by the following equation: fco = Rfrc (Skel(f, (f B) \ fis ) ∩ frc )
(10)
where Skel(f, g) stands for the operator of the anchored skeletonization of the image f with anchor image g and R stand for the reconstruction by dilation operator defined by the Eq. 5. Using the equation 10, the third class - corridors - is detected. The remaining part of internal contour are the branches of boundary: fbr = f \ ((f B) ∪ fcb ∪ fis ∪ fco )
(11)
C. Structuring Element The only parameter used in the above described approach is the structuring element B. Its choice results in the form of the internal object boundary. The most convenient is a structuring element which contains a pixel’s neighborhood of a given radius. For 4- and 8connected pixel grids, the radii are defined according to the distance formulation which is different than the Euclidean one. In the case of 4-connected grid the ’cityblock’ distance is implied, while for the 8-connected grid - the max-norm. Examples of structuring elements of sizes 1,2 and 3 in both grids are shown in Figs. 3 and 4. IV. S HAPE C OEFFICIENTS Thanks to the classification described in the previous section, four classes are detected. Pixels belonging to these classes are represented by separate images fis , fcb , fco and fbr . The pixel classification is used to compute the shape coefficients based on it. They refer to the ratio of pixels
Fig. 5.
Fig. 6. Pixel classification using the elementary structuring element of size 1 (shown in Figure 3(a)).
Test image 1.
belonging to different classes to all the internal boundary pixels. Thus, four coefficients are defined: cis =
|fis | |fis ∪ fcb ∪ fco ∪ fbr |
(12)
ccb =
|fcb | fis ∪ fcb ∪ fco ∪ fbr |
(13)
cco =
|fco | |fis ∪ fcb ∪ fco ∪ fbr |
(14)
cbr =
|fbr | |fis ∪ fcb ∪ fco ∪ fbr |
(15)
where |f | stands for the number of foreground pixels of the image f . Following its definition, every shape coefficient is a real number between 0 and 1. Coefficients may be applied both to describe the image globally and locally. In the latter case they are computed for every object (connected component) separately. V. R ESULTS The proposed methodology was applied to test image 1 shown in Figure 5. The only parameter of the method is the structuring element. Depending on this element various classification results are obtained. The experiments was performed using the elementary structuring element of various sizes in 8-connected grid. Results are shown in Figures 6, 7, and 8. The following regions are marked on these figures: core area in yellow, core boundary in blue, isolated regions in orange, corridors in violet and branches in green. The classification visibly depends on the size of the structuring element. For example, areas which are classified as corridors when structuring element of size 3 is
used, are not the corridors any more for the size equal to 2. The same property can be observed for the other boundary classes. In order to show how the shape coefficients based on pixel classification describe the shape of binary object, the second example is given. Test image 2 shown in Figure 9 contains four shapes representing shapes typical for card playing game: heart, diamond, spades and club. The whole image has a resolution of 120x120 pixels. Pixel classification was performed with the elementary structuring element of size 2. Result of the pixel classification is shown on the right-hand side of Figure 9. From now on every connected component was treated separately in order to compute the shape coefficients1 . The final coefficients of all four shapes shown on are given in Table I. TABLE I C OEFFICIENTS OF CONNECTED COMPONENTS FROM TEST IMAGE 2. heart diamond spades club
cbr
ccb
cco
0.0530 0.1416 0.0704 0.0495
0.9470 0.8584 0.8873 0.9271
0 0 0.0423 0.0234
As the table describes, the proposed coefficients allow to differentiate the shapes and can be effectively used as the input for classifiers. VI. C ONCLUSIONS In the paper the morphological method for classifying the pixels belonging the internal boundary of binary 1 In the separate treatment of connected components, isolated region coefficients are not computed
objects was proposed. It allows classifying the boundary pixels into four groups: core boundary, isolated regions, corridors and branches. The method is based on the morphological operators as erosion, dilation and reconstruction as well as the anchored skeletonization. The parameter of the classifier is the structuring element used. Depending on the choice of this element various classification results are obtained. The shape coefficients based on the proposed classification can be also computed. They can be later used for recognition of binary shapes. R EFERENCES
Fig. 7. Pixel classification using the elementary structuring element of size 2 (shown in Figure 3(b)).
Fig. 8. Pixel classification using the elementary structuring element of size 3 (shown in Figure 3(c)).
Fig. 9. Test image 2 - playing cards (left) and classification result(right).
[1] P.Soille, “Morphological image analysis”, Springer Verlag, 1999, 2004. [2] L.Vincent, “Morphological grayscale reconstruction in image analysis: applications and efficient algorithms”, IEEE Trans. on Image Processing, Vol.2, No.2, April 1993. [3] J.Serra, L.Vincent, “An overview of morphological filtering”, Circuit systems Signal Processing, 11(1), 1992. [4] L. Vincent,“Efficient computation of various types of skeletons”, In: Loew, M., ed. Medical Imaging V: Image Processing. Volume SPIE-1445 (1991) pp.297311 [5] T.Kong, A.Rosenfeld, “Digital topology: Introduction and survey”,Computer Vision, Graphics, and Image Processing, 1989, vol.48, pp.357-393 [6] J.Serra, “Image analysis and mathematical morphology, vol.1”, Academic Press, 1983. [7] A.Rosenfeld, “Connectivity in digital pictures”, Journal of the ACM, 1970, vol.17, no.1 pp.146-160
