[5] R.M. Haralick, K. Shanmugam and I. Distein. Textural. Features for Image Classification. IEEE Trans. SMC, 6(3), pp. 610-622, 1973. [6] K.I. Laws. Textured ...
Improving Texture Pattern Recognition by Integration of Multiple Texture Feature Extraction Methods Miguel Angel García Domènec Puig Intelligent Robotics and Computer Vision Group Department of Computer Science and Mathematics Rovira i Virgili University Av. Països Catalans 26. 43007 Tarragona, Spain {magarcia, dpuig}@etse.urv.es Abstract This paper proposes a pixel-based texture classifier that integrates multiple texture feature extraction methods in order to identify the regions of an input image that belong to a given set of texture patterns. Experimental results with textured images of outdoor scenes show that the proposed technique yields lower classification errors than widely recognized texture classifiers based on specific families of texture methods.
1. Introduction In general, textured images are constituted by regions of different uniform texture patterns. A texture classifier aims at recognizing some or all of those patterns, and, thereby, at identifying regions of interest in the image, such as urban soil or crops in an aerial image. In this scope, unsupervised texture segmentation algorithms (e.g., [4][8]) are not applicable on their own, since the tackled problem is not the segmentation of an input image into distinctive regions, but the localization of regions that belong to certain texture patterns of interest. Pixel-based texture classifiers [14] aim at recognizing the texture patterns to which the pixels of an input image belong. For every image pixel, they compute a set of texture features by evaluating some texture feature extraction methods (texture methods in short) in a neighborhood of the pixel. A wide variety of texture methods have been proposed in the literature (e.g., [5][6][7][15]). Unfortunately, no single method is good enough to completely characterize and, therefore, distinguish the different texture patterns that may appear in nature. Thus, several methods must be combined in order to obtain good classification results. The different proposals usually combine texture methods that belong to a same family in order to improve the final classification (e.g., [2][11]). This combination is commonly This work has been partially supported by the Government of Spain under the CICYT project DPI2001-2094-C03-02.
tailored to the chosen family of methods. However, every family of texture feature extraction methods is potentially useful for texture discrimination to a larger or lesser extent. This means that no texture methods should be discarded a priori given a general classification problem. This paper presents a pixel-based texture classifier based on the theory of ensemble methods [12], which integrates different families of texture feature extraction methods with the goal of identifying the regions of an input image that belong to a given set of texture patterns, and shows that this technique produces better classification results than widelyrecognized texture classifiers based on specific families of texture methods. The organization of this paper is as follows. Section 2 describes the proposed classifier. Section 3 shows experimental results of the integration of widely-used texture feature extraction methods with the proposed technique, as well as a comparison with a well-known texture classification framework (MeasTex [9]). Conclusions and further improvements are finally presented in section 4.
2. Pixel-Based Texture Classification Let { τ 1, … , τ T } be a set of T texture patterns of interest. Every texture τ j is described by a set of sample images, with each image containing a uniform pattern of that texture: 1 K I j = { I j , …, I j } . Let I be an input textured image whose pixels must be classified. In order to classify a pixel I ( x, y ) based on textural information, a feature vector is extracted from I ( x, y ) : F ( x, y ) = ( µ 1 ( x, y ), …, µ M ( x, y ) ) . Each feature in that vector is obtained by applying a certain texture feature extraction method µ i to the pixels contained in a neighborhood around I ( x, y ) . This neighborhood is usually a square window centered at I ( x, y ) , whose size is experimentally set for each method. M different texture feature extraction methods are considered. This section proposes a technique for integrating the M features obtained as indicated above in order to determine whether every pixel I ( x, y ) can be classified into one of the
1051-4651/02 $17.00 (c) 2002 IEEE
T given texture patterns { τ 1, … , τ T }. This technique consists of four stages that are described below.
2.1. Supervised Training Stage At this stage, every texture method µ i is evaluated at each of the pixels of the sample images I j associated with each τ j , and a frequency table (histogram) H ij of the values returned by that method is computed. M × T different histograms are thus generated. For each H ij , a basic likelihood function is then defined by evaluating µ i at pixel I ( x, y ) and then mapping the result to the frequency table: Pi ( I ( x, y ) τ j ) = H ij ( µ i ( x, y ) )
sky
trees stones
(1)
Pi ( I ( x, y ) τ j ) can be interpreted as the basic likelihood that pixel I ( x, y ) belongs to texture τ j according to the outcome of method µ i . Each basic likelihood function is 2 associated with an uncertainty measure σ ij calculated as the variance of the values of H ij .
2.2. Integration of Multiple Texture Feature Extraction Methods Given a set of M × T likelihood functions Pi ( I ( x, y ) τ j ) defined according to (1), each one associated with a variance 2 σ ij , the objective now is to optimally integrate the likelihoods corresponding to the M texture feature extraction methods associated with each texture pattern τ j . The result will be T combined likelihood functions P ( I ( x, y ) τ j ) . The combination of different basic likelihood functions can be modeled as a linear opinion pool [1], which simply consists of a weighted average of those evidences after having been normalized. The weight corresponding to each likelihood function in a linear opinion pool can be calculated by applying the Generalized Ensemble Method (GEM) estimator [12]: P ( I ( x, y ) τ j ) =
sky
trees side
stones
Figure 1. Test images and corresponding groundtruth classification. Black regions in the groundtruth image are considered to belong to an unknown texture pattern.
2.3. Maximum a Posteriori Estimation Given a set of T likelihood functions P ( I ( x, y ) τ j ) (2), 2 each associated with a variance σ j (3), T posterior probabilities are then computed by applying the Bayes rule: P ( I ( x, y ) τ j )P ( τ j ) P ( τ j I ( x, y ) ) = -----------------------------------------------------T
w ij Pi ( I ( x, y ) τ j )
i=1
σ ij w ij = ------------------M
P ( I ( x, y ) τ k )P ( τ k )
with the prior probabilities being defined as: –2
σj P ( τ j ) = ------------------T
–2
Pixel I ( x, y ) will belong to texture class τ j P ( τ j I ( x, y ) ) > P ( τ k I ( x, y ) ), ∀j ≠ k .
k=1
Such a formulation of the weights w ij leads to an optimal combination of the basic likelihood functions. Similar formulations have been previously shown [10] to yield optimal information fusion (attaining minimum uncertainty ellipsoids), as well as to be optimal in the Kalman’s filter sense. The combined likelihood function P ( I ( x, y ) τ j ) (2) is associated with an uncertainty measure expressed as the following variance: –1 2
σj =
M
i=1
–2
σ ij
(3)
(5)
–2 σk
k=1
(2)
σ kj
(4)
k=1
–2
M
road
iff
3. Experimental Results The proposed technique has been evaluated on real outdoor images containing natural surfaces. Fig. 1(top row) shows two of those input images. Five texture patterns of interest have been considered: “sky”, “road”, “trees”, “stones” and road “side”. A set of sample images representing each of those patterns (Fig. 2) was extracted from the image database and utilized as the training dataset for the classifiers. Fig. 1(bottom row) shows the ground-truth classification of the given input images by considering the five texture pat-
1051-4651/02 $17.00 (c) 2002 IEEE
Test Image
sky
Classifier Proposed classifier MeasTex (Gabor, MVG) MeasTex (Gabor, 3-NN) MeasTex (Gabor, 5-NN) MeasTex (Markov, MVG) MeasTex (Markov, 3-NN) MeasTex (Markov, 5-NN) MeasTex (Fractal, MVG) MeasTex (Fractal, 3-NN) MeasTex (Fractal, 5-NN) MeasTex (GLCM, MVG) MeasTex (GLCM, 3-NN) MeasTex (GLCM, 5-NN)
road
trees
stones
side
Figure 2. Sample images corresponding to each considered texture pattern. terns of interest. Black areas represent image regions that do not belong to any of the sought texture patterns—as mentioned earlier, a texture classifier aims at identifying a set of texture patterns in an input image, not at segmenting all of the image regions based on texture uniformity. Pixels that belong to those “unknown” texture patterns have not been taken into account in the classification rates presented below. Taking relatively recent surveys into account [14][15], several widely-used texture feature extraction methods have been chosen to be integrated with the proposed technique: Laws R5R5, E5L5, E5E5, E5R5, R5S5, E5S5, S5S5, L5S5, L5L5, S5E5 [6], Daubechies-4 wavelets [7], Gabor filters [13] with wavelengths {8, 4} and orientations {0º, 45º}, firstorder statistics (variance, skewness), second-order statistics based on co-occurrence matrices [5] with varying distances and orientations: entropy(2, 0º), contrast(2, 0º), contrast(5, 45º), homogeneity(2, 0º), homogeneity(5, 45º); and fractal and multi-fractal dimensions [3]. All those texture methods were evaluated over windows of 33x33 pixels. The proposed classifier has been compared to MeasTex, a widely recognized texture classification framework [9]. MeasTex provides a set of general pattern classifiers that can be applied to different families of texture methods. This framework is oriented to the classification of entire images instead of individual pixels. Therefore, in order to achieve pixel-based classification, MeasTex was utilized to classify every pixel of the test images given a subimage of 33x33 pixels centered at that pixel. The same sample images shown in Fig. 2 were utilized as the training dataset for MeasTex. The first row in Table 1 shows the pixel classification rates obtained after applying the proposed classifier to the test images shown in Fig. 1. The remaining rows show the
Fig. 1(left)
Fig. 1(right)
80 75 76 76 59 58 58 57 60 60 78 60 62
68 63 67 67 49 44 45 50 52 52 51 50 50
Table 1. Classification rates (%) for the test images shown in Fig. 2, considering the proposed classifier and different configurations of MeasTex, each corresponding to a family of texture methods (Gabor, Markov, Fractal, GreyLevel Co-occurrence Matrices) and a classifier (Multivariate Gaussian Bayes, 3-Nearest Neighbors, 5-Nearest Neighbors). results obtained by applying MeasTex with different combinations of texture families (Gabor, Markov, Fractal, Grey Level Co-occurrence Matrices) and classification algorithms (Multivariate Gaussian Bayes, 3-Nearest Neighbors, 5-Nearest Neighbors). In all cases, the proposed technique produced better classification rates than MeasTex. In a few cases, though, the difference is insignificant (e.g., Meastex with Gabor and KNN). However, qualitative results show that, even in those cases, the proposed classifier is able to better identify the regions that belong to the sought texture patterns. Some of those qualitative results are presented in Fig. 3. Fig. 3(top row) shows the classification maps corresponding to the proposed classifier after applying it to the two given test images in Fig. 1. Pixels were labeled with the same gray levels utilized in the ground-truth classification (Fig. 1). Black borders in these images correspond to pixels that could not be classified due to the impossibility of evaluating the texture methods on 33x33 pixel windows centered at them. On the other hand, Fig. 3(middle row) shows the classification maps corresponding to the combination that produced best quantitative results with MeasTex (Gabor, 5-NN) according to Table 1. This is the best qualitative result obtained with MeasTex. Indeed, the classification maps corresponding to the combinations that present lower classification rates in Table 1 are significantly worse, such as with (GLCM, 5-NN), Fig. 3(bottom row).
1051-4651/02 $17.00 (c) 2002 IEEE
5. References
Figure 3. (top) Classification maps with the proposed algorithm for the two input images shown in Fig. 1. (middle) Classification maps with MeasTex (Gabor, 5-NN). (down) Classification maps with MeasTex (GLCM, 5-NN).
4. Conclusions This paper proposes a pixel-based texture classifier based on the integration of multiple texture feature extraction methods. Experimental results with natural textured images show that this technique produces better quantitative and qualitative results than traditional techniques based on the utilization of specific families of texture methods. The proposed technique has been compared to MeasTex, a widelyrecognized texture classification framework. Future work will consist of the determination of confidence levels that allow to distinguish if the maximum posterior probabilities found after applying the Bayes rule (Section 2.3) are significant enough to infer that a pixel really belongs to a given texture pattern or must be classified as an unknown. Another line will consist of studying the behavior of the proposed classifier as new texture methods are integrated, with the purpose of identifying a minimum (or at least small) number of texture methods necessary to obtain reasonably good classifications.
[1]
J. Berger. Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, 1985.
[2]
T. Caelly and D. Reye. On the classification of image regions by colour, texture and shape. Pattern Recognition, vol.26, no.4, pp. 461-470, 1993.
[3]
B.B. Chaudhuri and B. Sarkar. Texture segmentation using fractal dimension. IEEE Trans. PAMI, 17(1), pp. 72-77, January 1995.
[4]
K.I. Chang, K.W. Bowyer and M. Sivagurunath. Evaluation of Texture Segmentation Algorithms. IEEE CVPR, Fort Collins, USA, June 1999.
[5]
R.M. Haralick, K. Shanmugam and I. Distein. Textural Features for Image Classification. IEEE Trans. SMC, 6(3), pp. 610-622, 1973.
[6]
K.I. Laws. Textured Image Segmentation. USC ISG-TR-IPI940, Ph. D.Thesis (EE), January 1980.
[7]
C.S. Lu, P.C. Chung and C.F. Chen. Unsupervised Texture Segmentation via Wavelet Transform. Pattern Recognition, 30(5), pp. 729-742, 1997.
[8]
J. Malik, S. Belongie, T. Leung and J. Shi. Contour and Texture Analysis for Image Segmentation. Perceptual Organization for Artificial Vision Systems, K.L. Boyer and S. Sarkar, editors. Kluwer Academic Publishers, 2000.
[9]
MeasTex Image Texture Database and Test Suite. http:// www.cssip.uq.edu.au/staff/meastex/meastex.html.
[10] Y. Nakamura and Y. Xu. Geometrical Fusion Method for Multi-sensor Robotic Systems. IEEE Int. Conf. on Robotics and Automation, pp. 668-673, 1989. [11] T. Ojala, M. Pietikainen and D. Harwood. A Comparative Study of Texture Measures with Classification Based on Feature Distributions. Pattern Recognition, 29(1), pp.51-59, 1996. [12] M.P. Perrone and L. Cooper. When networks disagree: ensemble methods for hybrid neural networks. In R.J.Mammone, editor, Neural Networks for Speech and Image Processing. Chapman-Hall, 1993. [13] O. Pichler, A. Teuner and B.J. Hosticka. A Comparison of Texture Feature Extraction using Adaptive Gabor Filtering, Pyramidal and Tree Structured Wavelet Transforms. Pattern Recognition, 29(5), pp. 733-742, 1996. [14] T. Tanden and J.H. Husoy. Filtering for Texture Classification: A Comparative Study. IEEE Trans. PAMI, 21(4), pp. 291-310, 1999. [15] T.R. Reed and J.M. Hans du Buf. A Review of Recent Texture Segmentation and Feature Extraction Techniques. CVGIP: Image Understanding, 57(3), pp. 359-372, 1993. We thank Prof. Carme Olivé for her useful comments on various mathematical aspects of this paper.
1051-4651/02 $17.00 (c) 2002 IEEE
