Texture Classi cation Using Nonparametric Markov Random Fields

Texture Classi cation Using Nonparametric Markov Random Fields R. Paget, I. D. Longsta, and B. Lovell The authors are with the Department of Electrical and Computer Engineering, University of Queensland, QLD 4072 AUSTRALIA. E-mail: fpaget,idl,[email protected] This work was supported by the Cooperative Research Centre for Sensor, Signal, and Information Processing.

Abstract | We present a nonparametric Markov Random Field model for classifying texture in images. This model can

capture the characteristics of a wide variety of textures, varying from the highly structured to the stochastic. The power of our modelling technique is evident in that only a small training image is required, even when the training texture contains long range characteristics. We show how this model can be used for unsupervised segmentation and classi cation of images containing textures for which we have no prior knowledge of the constituent texture types. This technique can therefore be used to nd a speci c texture in a background of unknown textures. I. Introduction phase discontinuities. In this paper, we present a new approach: using the Solutions to the problem of classifying textures in an nonparametric MRF model to model just one texture and image usually requires prior knowledge of all textures that then using that model to map the probability of each pixel may occur [4]. For this case texture models need capture in an image being of the given texture. This approach is only sucient characteristics to discriminate between the similar to that used by Greenspan et al. [11] to produce a set of known textures, and a simple discriminant analysis probability map locating all texture in an image similar to may be used [6]. However, for images where not all textural a given texture. We show that our model captures enough types are known, a texture model needs to capture all rel- relevant characteristics of a given texture to determine the evant characteristics that characterise a particular texture. probability of each pixel in an image being that texture Then in segmenting and classifying an image, the model without using discriminant analysis. With this model, we of a known texture can be statistically compared with re- can segment and classify an image containing an unde ned gions in the image to determine the probability that the number of dierent texture types. region matches the known texture. This sort of modelling is also required when discriminant analysis can not be used II. Texture Model because background textures in an image are non-uniform. MRF models have been used in image restoration, region As yet, texture recognition is not a fully understood segmentation, texture synthesis [5]. The property of a problem and the characteristics needed to dierentiate tex- MRF is that aand variable on a lattice S = fs = (i; j ) : tures are not fully explored. Therefore, deriving a model 0  i; j < N g may have Xits value x set to any value, but to capture all relevant characteristics that dierentiate a the probability of X = x is conditional upon the values particular texture for any other remains an open problem x at its neighbouring sites r 2 N . A Local Conditional [7]. Probability Density Function (LCPDF) de ned over these A reasonable way to test whether a texture model has neighbouring sites r 2 N determines the probability of captured all relevant characteristics is to synthesise a tex- X = x as, ture from the model and evaluate how similar it is to the original. Testing for similarity is an open ended problem P (X = x jX = x ; r 2 N ) s 2 S; (1) and a criteria for similarity is required. (Clearly white noise with odd parity would be statistically dierent from white which in turn de nes the MRF [1]. noise with even parity.) One benchmark test of similarity is To model an image as a MRF, we consider each pixel in a subjective comparison, by eye, of the synthesised texture the image to be a site on a lattice, and the grey scale value with the training texture. We assume that if the texture of that pixel as the value of that site. If the image is all synthesised from the model is indistinguishable by eye from of one texture, then the derived LCPDF is the model that the training texture then the model is adequate for most de nes the texture. applications. Current models such as auto-models, autoreWe used the neighbourhood N of Geman [7,10], de ned gressive (AR) models, moving average (MA) models and as autoregressive moving average (ARMA) models have not been shown to realistically synthesise natural textures [12] N = fr = (k; l) 2 S : 0 < (k ? i)2 + (l ? j )2  cg; (2) such as those in the Brodatz album [3]. However, we [14,15] have recently used a nonparametric Markov Random Field where c refers to the order of the neighbourhood system. (MRF) model to successfully synthesise realistic represent- Neighbourhood systems for c = 1; 2 and 8 are shown in ations of structured and stochastic textures with minimal Fig. 1 (a), (b), and (c) respectively. s

s

s

s

r

s

s

s

s

s

s

r

r

s

c s

c s

A. Parzen Window Density Estimator The Parzen-window density estimator [16] has the eect of smoothing each sample data point in a multi-dimensional histogram over a larger area. Denoting the sample data as Z = Col[x ; x ; r 2 N ] s 2 S; N  S , for a column vector z = Col[L0; L r ; r 2 N ], (a) (b) (c) the Parzen-window density estimated frequency F^ (z) of the Fig. 1. Neighbourhoods. (a) The rst order neighbourhood ( = 1) frequency F in (3) is or \nearest-neighbour" neighbourhood for the site = ( ) = `'   and = ( ) 2 Ns = `'; (b) second order neighbourhood ( = 1 X K 1 (z ? Z ) ; F^ (z) = (4) 2); (c) eighth order neighbourhood ( = 8). nh 2 Ns  h s

s

r

s

s

s

n

c

s

r

i; j

k; l

c

c

s

d

s

Given an image of a homogeneous texture and a prede ned neighbourhood system, we can gain a nonparametric estimate of the LCPDF by building a multidimensional histogram of the image. For example, if we choose a neighbourhood N = fs ? 1g as shown in Fig. 2(a). To estimate the respective LCPDF, we build a 2-dimensional histogram with dimensions L0 ; L1, where L0 represents the pixel value x and L1 represents the relative neighbouring pixel value x ?1. We initialise F (L0; L1 ) = 0 8 L0; L1 . Then by raster scanning the image, we increment the variable F (L0 = x ; L1 = x ?1) for each site s 2 S; N  S . The simple estimate of the LCPDF is then given by s

s

s

s

s

s

F (L = x ; L1 = x ?1) : P^ (x jx ?1) = P 0 F (L ; L = x ) s

s

s

xs-1 xs (a)

2

1

7 L1 0 0 (b)

Fig. 2. Neighbourhood and its 2-D histogram

s

s

d=

The size of K is de ned by the window parameter h. We aim to choose h so as to obtain the best estimate of the frequency distribution F^ for the LCPDF. Silverman [16, p 85] provides an optimal window parameter: h



opt

=  n(2d4+ 1)

1=(d+4)

(6)

;

where 2 is the average marginal variance. In our case, marginal variance is the same in each dimension and therefore 2 equals the variance associated with the onedimensional histogram. 7

L0

S

where n is the number of sample data Z , h is the window parameter, and d = jN j +1 equals the number of elements in the vector z [16, p 76]. The shape of the smoothing is de ned by the kernel function K . We choose K as the standard multi-dimensional Gaussian density function, 1 exp(? 1 zT z): (5) K (z) = 2 (2) 2

(3)

?1

s

F(L 0,L 1)

L

0

s

S;

III. Texture Classification

In our classi cation method, each individual pixel in an image is assessed for its probability of being the original texture. In a more sophisticated version of this method, we could reasonably assume that pixels close to each other exhibit similar probabilities. It may also be prudent to incorporate boundary detection as part of a constrained optimisation of the probability map as discussed by Geman et al. [8]. Such improvement are application-driven. Here, we limit ourselves to outlining the simple version of our classi cation method. For cases when the image was not all of one texture, Geman and Gragne [9] assumed that small areas of the image were homogeneous and of one texture. They classi ed on the basis that the product of the joint probabilities for a neighbourhood over the area of concern resembles the joint probability for the area, that is,

Data obtained from the image to build a multidimensional histogram are not independent and identically distributed (i.i.d.). However, the pseudo-likelihood estimate [2] of the LCPDF uses the same non i.i.d. data. Geman and Gragne [9] proved that the pseudo-likelihood estimate converged to the true LCPDF with probability 1 as the image size increased to in nity. We use this evidence to justify use of non i.i.d. data for estimating our non-parametric version of the LCPDF. The true LCPDF is given by a histogram built from Y an in nite amount of sample data. Therefore, the true P (x ; x ; t 2 N ); (7) (x ; r 2 W ) ' LCPDF needs to be estimated from a multi-dimensional :Nr  s histogram. Where a domain is only sparsely populated with sample data, it is advantageous to use a non- where W is the window of sites, centred at s, which are to parametric density estimator [16]. be used for the classi cation of x . r

r

s

r

W

s

s

t

r

A. Probability measurement The probability (x ; r 2 W ) as de ned by (7) was found to give poor classi cation results. This was because our nonparametric LCPDF tended to give low probabilities for the neighbourhood con gurations in the classi cation window, which resulted in (x ; r 2 W ) being too susceptible to any minor uctuations in these neighbourhood probabilities. Instead, we used the set of probabilities de ned by the LCPDF for the window W and compared them directly to the set of probabilities obtainable from the sample texture. Probability P (x ; x ; t 2 N ) is calculated from (4) and (5) as r

s

r

s

types of textures present in the image. As the segmentation/classi cation method is able to identify those textures similar to the sample texture from those that are dissimilar, this indicates that our model has captured all relevant characteristics to characterise a particular texture. References

[1] J. E. Besag, \Spatial Interaction and the Statistical Analysis of Lattice Systems," Journal of the Royal Statistical Society, series B, 36, pp. 192{326, 1974. [2] J. E. Besag and P. A. P. Moran, \Eciency of pseudolikelihood estimation for simple Gaussian elds," Biometrika, 64, pp. 616{618, 1977. [3] P. Brodatz, Textures - a photographic album for artists P (x ; x ; t 2 N ) = and designers, Dover, 1966.   X 1 1 exp ? 2h2 (z ? Z )T (z ? Z ) ; [4] R. Chellappa and S. Chatterjee, \Classi cation of texnh (2) 2 2 tures using Gaussian Markov Random Fields," IEEE y Transactions on Acoustics, Speech, and Signal ProNp  y cessing, ASSP-33, no. 4, pp. 959{963, 1985. (8) [5] R. C. Dubes and A. K. Jain, \Random eld models in where z = Col[x ; x ; t 2 N ] and Z are samples taken image analysis," Journal of Applied Statistics, 16, no. 2, from the sample texture y de ned on the lattice S . The pp. 131{164, 1989. samples of the LCPDF, taken from the window W  S , are [6] R. O. Duda and P. E. Hart, Pattern Classi cation the set of probabilities fP (x ; x ; t 2 N ); r : N  W g. and Scene Analysis. Stanford Reseach Institute, Menlo We calculate the probabilities for sample texture y for Park, California, John Wiley, 1973. every site q 2 S ; N  S in a similar fashion to (8), [7] D. Geman, \Random elds and inverse problems in imaexcept for a pixel q 2 S we do not include the sample data ging," Lecture Notes in Mathematics, 1427, pp. 113{193, Z = Col[y ; y ; t 2 N ] in the calculation. This is done to 1991. prevent the probability P (y ; y ; t 2 N ) from being biased. With the set of probabilities fP (x ; x ; t 2 N ); r : N  [8] D. Geman, S. Geman, C. Gragne and P. Dong, \Boundary Detection by Constrained Optimization," W g from the window to be classi ed, and the set of probIEEE Transactions on Pattern Analysis and Machine abilities fP (y ; y ; t 2 N ); q 2 S ; N  S g from the Intelligence, 12, no. 7, pp. 609{628, 1990. sample texture, we are now able to determine the classi cation probability. The null hypothesis is that the distri- [9] S. Geman and C. Gragne, \Markov random eld image models and their applications to computer vision," bution of probabilities from the window is the same as the Proceedings of the International Congress of Mathemdistribution from the sample texture. For this test we use aticians, pp. 1496{1517, Berkeley, California, 1986. the nonparametric Kruskal-Wallis test [13]. The sampling distribution of the Kruskal-Wallis statistic [10] S. Geman and D. Geman, \Stochastic Relaxation, Gibbs K is approximately chi-squared with 1 degree of freedom. Distributions, and the Bayesian Restoration of Images," Given K , the accepted practice is to accept or reject the IEEE Transactions on Pattern Analysis and Machine null hypothesis on the basis of a particular signi cance level Intelligence, 6, no. 6, pp. 721{741, 1984. . In our approach we wished to nd the con dence asso- [11] H. Greenspan, R. Goodman, R. Chellappa and C. H. ciated with accepting the null hypothesis. This con dence, Anderson, \Learning Texture Discrimination Rules in for a particular window W , is denoted as P s : Multiresolution System," IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, no. 9, pp. 894{ (9) P s = P (k  K ); 901, 1994. where k is chi-squared distributed with one degree of free- [12] M. Haindl, \Texture synthesis," CWI Quarterly, 4, pp. 305{331, 1991. dom. It is this probability/con dence P s with which we plot our probability map. [13] L. L. Lapin, in Probability and Statistics for Modern Engineering. Boston: PWS-KENT, 1990. IV. Results [14] R. Paget and D. Longsta, \A Nonparametric Multiscale Markov Random Field Model for SynthesWe use a neighbourhood system of order 2 in our model ising Natural Textures," Fourth International Symto obtain the probability maps of Fig. 3. These probability maps show that with our texture model it is possible to segposium on Signal Processing and its Applications, 2, pp. 744{747, 25-30th August 1996. ment and classify windows of texture with respect to just one sample texture and without prior knowledge of other s

r

t

r

t

r

p

S ;

r

d

p

d=

p

opt

S

r

t

r

p

y

s

r

y

q

t

r

r

s

y

y

p

q

t

q

q

t

q

r

t

r

s

q

t

q

y

q

y

s

W

W

W

r

(a.1)

(a.2)

(a.3)

(b.1)

(b.2)

(b.3)

(c.1)

(c.2)

(c.3)

Probability scale 0 1 Fig. 3. Probability maps of medical images: (a) lymphoid follicle in the cervix; (b) small myoma; (c) focus of stromal dierentiation in the myometrium [17]. Sample textures (a.1) (b.1) (c.1) were used to segment and classify medical images (a.2) (b.2) (c.2) producing the probability maps (a.3) (b.3) (c.3), respectively.

[15] R. Paget and D. Longsta, \Texture Synthesis via a [17] J. D. Woodru, T. L. Angtuaco and T. H. Parmley, Nonparametric Markov Random Field," Proceedings of in Atlas of Gynecologic pathology. New York: Raven DICTA-95, Digital Image Computing: Techniques and press, 1993, 2nd ed.. Applications, 1, pp. 547{552, 6-8th December 1995. [16] B. W. Silverman, Density estimation for statistics and data analysis. London, Chapman and Hall, 1986.