Random neural network texture model - UCF CS - University of ...

Random neural network texture model Erol Gelenbe, Khaled Hussain, and Hossam Abdelbaki School of Computer Science University of Central Florida Orlando, FL 32816

ABSTRACT

This paper presents a novel technique for texture modeling and synthesis using the random neural network (RNN). This technique is based on learning the weights of a recurrent network directly from the texture image. The same trained recurrent network is then used to generate a synthetic texture that imitates the original one. The proposed texture learning technique is very ecient and its computation time is much smaller than that of approaches using Markov Random Fields. Texture generation is also very fast. We have tested our method with dierent synthetic and natural textures. The experimental results show that the RNN can eciently model a large category of homogeneous microtextures. Statistical features extracted from the co-occurrence matrix of the original and the RNN based texture are used to evaluate the quality of t of the RNN based approach. Keywords: Neural networks, Random Neural Network, Texture Synthesis, Texture Generation, Markov Random Fields, Color Change Mechanism in Animals.

1. INTRODUCTION

Although there is no exact and generally accepted de nition of texture, texture analysis is considered one of the most important subjects in image processing and computer vision. The task of extracting texture features is crucial and if one could model and quantify the process by which the human recognizes texture, one could construct a highly successful recognition system. Unfortunately, the process by which we recognize textures is not fully understood, and researchers are left to consider some ad hoc techniques. Conventional texture feature extraction algorithms can be grouped into four classes: statistical,6 structural,7 spectral,8 and model based.9 Statistical approaches yield characterizations of textures as smooth, coarse, grainy, and so on using relationship between intensity values of pixels; measures include the entropy, contrast, and correlation based on the gray level co-occurrence matrix. Structural algorithms are based on the image primitives; they regard primitive as the forming element to generate repeating patterns. Spectral techniques are based on properties of the Fourier spectrum and are used primarily to detect global periodicity in the image. Model based texture analysis methods are based on the construction of an image model that can be used not only to describe the texture, but also to synthesize it. The various methods for modeling textures and extracting texture features can be applied in three broad categories of problems: texture segmentation, texture classi cation, and texture synthesis. Recently, many arti cial neural network (NN) architectures for texture analysis have been proposed (for example, see10{12 ). Almost all these architectures either apply a supervised NN model for texture classi cation or an unsupervised NN model for texture segmentation but a very little attention has been devoted to training a NN on a certain texture and then using the trained network to generate a synthetic texture. Regarding the texture synthesis problem, Markov random elds (MRFs) have been used extensively because they are able to capture the local (spatial) contextual information in a texture and generate a similar texture using the extracted parameters. However, the operations performed during texture generation are very time consuming. The idea of using the neural networks in learning and regenerating textures was basically inspired from the color change mechanisms of some kinds of animals which are able to alter their colors and their color patterns so that they can be eectively camou aged against a variety of dierent backgrounds. In this paper, we introduce a novel method for texture modeling (learning) and synthesis using the random neural network model.13,14 This model has been successfully applied in generating synthetic textures15 that have features similar to those generated by the MRF model, such as granularity and inclination but with tremendous reduction in the generation time over the MRF. The E-mail addresses : ferol, khaled, [email protected]

paper is organized as follows. Section 2 describes the color change mechanism in some animals. The RNN model used throughout the paper is described in section 3. Section 4 presents the texture learning and generation algorithm using the RNN. In Section 5, an experimental evaluation of the model is given using both synthetic and standard natural texture images.17 Finally, Section 6 gives conclusions and suggestions to future work.

2. COLOR CHANGE MECHANISM IN ANIMALS

The best known lizard able to change its color is the chameleon.1{5 Illumination of some parts of its retina initiates impulses in the optic nerve, which evoke some kind of hormones into the bloodstream. When they reach the pigment cell they are able to control its color. The chameleon is able to control the concentration of dierent pigments in its skin cells. Cells called chromatophores containing red, yellow, blue, and brown pigment granules lie in distinct layers of the skin. This pigment can be evenly distributed throughout the cell, or it can be concentrated in one small area in the center, and there can be stages between full distributed and full concentration. For example, if the red pigment is dispersed while the other pigments are concentrated, the lizard will appear red because the red color covers a greater surface area of the lizard's skin. In the deeper layers of the skin there are many cells called melanophores. These are large cells with many branches which extend among the surrounding cells, and they contain a brown pigment called melanin. When melanophores are stimulated, they disperse their brown pigment and often mask all the other colors. Pigment movements in chromatophores are regulated by pigment concentrating and pigment-dispersing neurosecretory hormones, which is secreted from the brain. Melanophors respond to light in a variety of ways. For responses not involving the eye, the term nonvisual response is used. Nonvisual responses are of two kinds; coordinated by nerves or hormones and uncoordinated, where the melanophore is its own receptor (the so-called independent eector). For visual responses, the terms white and black background responses are used. The black background response is evoked by placing an animal in light-absorbing surroundings; the response is darkening of the skin due to dispersion of melanin. The white background response is evoked by placing an animal in light-scattering surroundings. Since the eye is the receptor for both background responses, the question is how the eye achieves discrimination between these two sets of external lighting conditions. There are two localized areas of the retina, the stimulation of one leading to melanin dispersion and the other to melanin aggregation. Illumination of a certain area of the retina initiates impulses in the optic nerve, which evoke melanin-dispersing hormones (M.D.H.) secretion into bloodstream, and as a result, melanin disperses black background response. When illumination of the retina ceases, secretion of M.D.H. is reduced, and the M.D.H. already in circulation is destroyed in both the liver and the peripheral tissues; as a result, melanin aggregates. Illumination of another area of the retina evokes impulses in aggregating nerves causing secretion of a melanin-aggregating hormones. Obviously, The chameleon's central nervous system is also involved, but scientists don't yet know the exact mechanism, nor do they understand how the hormone might stimulate one layer and not another.5

3. THE RANDOM NEURAL NETWORK (RNN) MODEL

In the random neural network model13 signals in the form of spikes of unit amplitude circulate among the neurons. Positive signals represent excitation and negative signals represent inhibition. Each neuron's state is a non-negative integer called its potential, which increases when an excitation signal arrives to it, and decreases when an inhibition signal arrives. An excitatory spike is interpreted as a \+1" signal at a receiving neuron, while an inhibitory spike is interpreted as a \;1" signal. A neuron i emitting a spike, whether it is an excitation or an inhibition, will lose potential of one unit, going from some state whose value is ki to the state of value ki ; 1. The state of the n-neuron network at time t, is represented by the vector of non-negative integers k(t) = (k1 (t); :::; kn (t)), where ki (t) is the potential or integer state of neuron i. We will denote by k and ki arbitrary values of the state vector and of the i-th neuron's state. Neuron i will \ re" (i.e. become excited) if its potential is positive. The spikes will then be sent out at a rate ri , with independent, identically and exponentially distributed inter-spike intervals. Spikes will go out to some neuron j with probability p+ij as excitatory signals, or with probability p;ij as inhibitory signals. P n A neuron may also send signals out of the network with probability di , and di + j=1 [p+ij + p;ij ] = 1. Let wij+ = ri p+ij , and wij; = ri p;ij . Here the \w's" play a role similar to that of the synaptic weights in connectionist models, though they speci cally represent rates of excitatory and inhibitory spike emission. They are non-negative. Exogenous excitatory and inhibitory signals arrive to neuron i at rates i , i , respectively. This is a \recurrent network" model which may have feedback loops of arbitrary topology. Computations are based on the probability

distribution of network state p(k; t) = Pr[k(t) = k], or with the marginal probability that neuron i is excited qi (t) = Pr[ki (t) > 0]. As a consequence, the time-dependent behavior of the model is described by an in nite system

of Chapman-Kolmogorov equations for discrete state-space continuous time Markov systems. Information in this model is carried by the frequency at which spikes travel. Thus, neuron j , if it is excited, will send spikes to neuron i at a frequency wij = wij+ + wij; . These spikes will be emitted at exponentially distributed random intervals. In turn, each neuron behaves as a non-linear frequency demodulator since it transforms the incoming excitatory and inhibitory spike trains' rate into an \amplitude". Let qi (t) be the probability that neuron i is excited at time t. The stationary probability distribution associated with the model is given by: p(k) = tlim !1 p(k; t); +i =

Xq w j

+

j ji

qi = tlim !1 qi (t); i = 1; :::; n: ;i =

+ i ; qi =

ri =

X q w; +  j

j ji

i

+i ri + ;i

Xw i

+

ij

+ wij;

(1) (2) (3) (4)

4. RNN FOR MODELING AND SYNTHESIS OF TEXTURES

In this section we will describe the procedure used for extracting the features from a given texture image, through training the RNN, and encoding those features into the weight matrices of the network. The resulting weights can then be used for generating textures that have similar characteristics to the initial texture.

4.1. Texture Modeling (Learning) Procedure

Given an N  N image, of an arti cial or a natural texture with G gray levels, and a w  h window, the RNN model used for extracting the texture features will have n = w  h neurons connected according to a prede ned structure as will be described in the next subsection. For the n neuron network, the network parameters are those n  n weight matrices W + = wij+ and W ; = wij; which need to be learned from the input texture. Let K be the the total number of overlapping windows in the image and (k) and y(k) be the kth external training input pattern and the corresponding desired output pattern respectively (here the input and target patterns have the same values). The procedure of texture learning can be summarized in the following steps.

   

Initialize the weight matrices W0+ and W0; to random values between 0 and 1. Set (k) and y(k) to the normalized pixel values in the window. Set (k) to 0:0. Solve the nonlinear system given be Equations 2 and 3 to obtain the actual neuron outputs q. Adjust the network parameters to minimize the cost function Ek given by. Ek =

n 1X k k 2 2 i=1 (qi ; yi ) :

(5)

For each successive desired input-output pair, indexed by k, the n  n weight matrices Wk+ and Wk; must be adjusted after applying each input. The details of the weights updating algorithm is found in.14 Since the weight matrices represent rates, only nonnegative values of the matrices are valid. The general learning algorithm of the RNN is of the recurrent type, thus it is possible to de ne any kind of dependency between the pixels of the given window.

4.2. Texture Synthesis (Generation) Procedure

We restrict our attention to the generation of gray-level images. The random neural network which is proposed to generate arti cial textures associates a neuron I (i; j ) to each pixel (i; j ) in the plane. The state f (i; j ) can be interpreted as the grey level value of the pixel at (i; j ). The topology of the proposed random network for texture generation is shown in Figure 4.2. We see that each neuron in the network will be, in general, connected to at most eight neighbors. We shall use the notation shown in Figure 4.2 (with numbering of neurons from 1 to 8 around a given neuron) for the positions in the network, where x denotes any (i; j ). For instance, x1 denotes the (i ; 1; j + 1) pixel. I (i − 1), (j + 1)

Ii, (j + 1)

1

I (i + 1), (j + 1)

2

I (i− 1), j

3

I i, j 4

I (i +

1), j

5

x 6

7

I (i− 1), (j - 1)

Figure 1.

Ii, (j − 1)

8

I (i+ 1), (j - 1)

Topology of the random neural network used for texture modeling and generation.

The parameters used for generating synthetic textures are the excitatory and the inhibitory connection weight matrices (W + and W ; ). Let (x) = c, where c is a constants, and (x) = 0, for any neuron I (x) in the network. Rewriting Equations 2 and 3 with the current notations. + (x) =

X q(x )w 8

d=1

d

+

(x; xd ) + c; ; (x) =

X q(x )w;(x; x ) 8

d=1

d

d

(6)

where q(xd ) denotes the stationary probability of the state of neuron xd , and q(x) =

+ (x) ; r + ; (x)

(7)

We associate the gray value function f (x) of a pixel x, with q values, that is: f (i; j ) = f (x) = q(x) if q(x) 1 and

f (x) = 1 if q(x) > 1.

The choice of the parameter c is very important since a large value of c will lead to a lighter image, while a small value of c will lead to a darker image. The choice of this value can be made as follows.15 Equation 6 will lead to a single xed point q(x) for any x, call this value q, and rewrite 6 and 7 as: q + c q= ; or; bq2 + (r ; )q ; c = 0 (8) bq + r which solves to 2bq = ( ; r) + ((r ; )2 + 4bc) 21 : (9) q can be interpreted as the average gray level of the image which will be generated, and is given as a real number between 0 and 1; thus c must be chosen so as to bring q to the desired value. In other words, if we are given a desirable average gray level of q, then we use Equation 8 to choose c as follows: c = q(bq + r ; ): The procedure of texture generation can be summarized in the following steps.

 Choose c from a given value of q, and determine all of the weights wd .

 Generate at random a bitmap value between 0 and 1 for each pixel x = (i; j ) and assign it to the variable q (x) 0

for each x. In general, if the desired texture has G gray levels, we start with an image that is generated by coloring each point with level l, where l is chosen with equal probability from the set f0, 1, 2,


, G ; 1g.  Starting with k = 0 up to k = K (the stopping condition), iterate on equations +k+1 (x) = Pd qi (xd )w+ (x; xd )+ P + (x) c; ;k+1 (x) = d qi (xd )w; (x; xd ); and compute qk+1 (x) = r+k+1 ; (x) ;. k+1

It should be stated here that the network weights can be chosen according to some criteria to generate synthetic textures with prede ned characteristics15 or they can result from training the RNN to a certain texture image as illustrated in Section 4.1. An interactive Java applet19 has been written to demonstrate the texture generation using RNN. A sample texture is shown in Figure 2

Figure 2.

Interactive RNN based texture generation.

5. EXPERIMENTAL RESULTS

In this section, we present some examples of textures generated according to various setting of the RNN model weights. The generated textures have 256 levels of gray and each texture image is 128  128 pixels. We conduct some simulations on synthetic textures and another set of simulations on natural textures as will be illustrated in the following subsections.

5.1. Modeling Synthetic Textures

In this simulations, we begin by assuming arbitrary weights for the RNN and generate synthetic texture using the procedure in Section 4.2. The texture learning procedure presented in Section 4.1 is then applied to the generated synthetic texture image, with 3  3 training window, and the RNN weights are obtained. The weights resulting from training are used in generating another synthetic texture (again using the procedure in Section 4.2). Figures 3 and 4 gives the results for modeling binary and 256 gray levels textures respectively. The calculated features will be used to compare the similarity of the original and the generated textures. The original and nal textures are then compared. Although all our experiments yield visually similar realizations, as shown in the gures, we calculate some of the statistical features which are derived from the co-occurrence matrix such as energy, contrast, entropy and homogeneity of the textures.6 See Table 1. As an example, the texture in Figure 3 (a) is generated using the following parameters w2;x = 1, w7;x = 1, + w4x = 1, and w5+x = 1 (see Figure 2). The texture learning algorithm is applied to this texture image and after around 1000 iterations, the training is stopped and the resulting RNN weights are then used for generating another texture as shown in Figure 3 (b). The estimated weights after training are as follows: w1;x = 0:8791, w2;x = 0:8104,

20

20

20

20

40

40

40

40

60

60

60

60

80

80

80

80

100

100

100

100

120

120 20

40

60

80

100

120

120 20

40

(a) Figure 3.

60

80

100

120

120 20

40

(b)

60

80

100

120

20

(c)

80

100

120

Synthetic binary textures generated with speci ed ((a) and (c)), and estimated ((b) and (d)) parameters. 20

20

20

40

40

40

40

60

60

60

60

80

80

80

80

100

100

100

100

120

120 40

60

80

100

120

120 20

40

(a) Figure 4.

60

(d)

20

20

40

parameters.

60

(b)

80

100

120

120 20

40

60

(c)

80

100

120

20

40

60

80

100

120

(d)

Synthetic 256 gray levels textures generated with speci ed ((a) and (c)), and estimated ((b) and (d))

w3;x = 0:7626, w4;x = 0:0205, w5;x = 0:0104, w6;x = 0:8445, w7;x = 0:9023, w8;x = 0:8630 w1+x = 0:0636, w2+x = 0:0094, w3+x = 0:2013, w4+x = 0:9522, w5+x = 1:0200, w6+x = 0:1275, w7+x = 0:0868, and w8+x = 0:1206. The co-occurrence

features derived from the textures in Figures 3 and 4 are listed in Table 1. Table 1.

Statistical features extracted from the co-occurrence matrix for textures of Figures 3 and 4. Figure Description Energy Contrast Entropy Homogeneity 2(a) speci ed 69546512 8196 59296 12286 2(b) estimated 69345018 7177 36427 127955 2(c) speci ed 104043150 2119 61378 15324 2(d) estimated 109852614 1699 61763 15534 3(a) speci ed 5106981 1413060 35315 2807 3(b) estimated 6124335 1409055 36427 3644 3(c) speci ed 7121979 1642544 37282 2995 3(d) estimated 9000193 1752960 39325 2996

5.2. Modeling Natural Textures

In this simulations, we apply the texture learning procedure to a natural texture image and the RNN weights are obtained. The weights resulting from training are used in generating the synthetic texture. The natural and synthetic textures are then compared again by calculating the co-occurrence matrix features as in Section 5.1. See Figure 5 and Table 2.

20

20

20

20

40

40

40

40

60

60

60

60

80

80

80

80

100

100

100

100

120

120 20

40

60

80

100

120

(a)

120 20

40

60

80

100

120

120 20

40

(b) Figure 5.

Table 2.

60

80

100

120

20

40

(c)

60

80

100

120

(d)

Natural ((a) and (c)) and synthetic ((b) and (d)) 256 gray levels textures.

Statistical features extracted from the co-occurrence matrix for textures of Figure 5. Figure Description Energy Contrast Entropy Homogeneity 4(a) natural 16985393 10681 47020 11864 4(b) synthetic 18006535 9560 46672 12604 4(c) natural 11314241 35971 44000 9340 4(d) synthetic 13179535 37801 45444 9221

It should be noted here that in the case of modeling synthetic textures, the weights obtained from the training procedure need not to be close to the weights initially used for generating the texture although both textures are visually similar. This may be due to the fact that the initial texture generated using arbitrary weights, may be typical with respect to many distinct distributions and as a result of this, the weights resulting from learning can point to any one of these distributions.18 Also in the training phase, it is observed that it is not necessary to use all the overlapping 3  3 windows as training patterns. In most of the simulations, around 1000 iterations were sucient for the RNN to generalize and extract the inherent features of the texture.

6. CONCLUSIONS

This paper gives a comprehensive presentation of what we consider a new approach of modeling and generation of textures based on training neural networks. All prior studies in texture modeling using neural networks have considered a model adequate if its parameters yield good classi cation. In this feature-based approach, we do not have the capability to generate images from features because features do not uniquely de ne a texture class. The proposed random neural network (RNN) based approach for modeling textures has some advantages over the Markov Random Field model. From the generation point of view, the main advantage is that the time needed for generating textures using the RNN is much less than that needed when using MRF technique because MRF algorithm is essentially based on Monte Carlo simulations, which may be prohibitively long in many real applications. Although there have been attempts to parallize MRF algorithms16 so as to reduce the computational cost, parallel MRF algorithms still require a substantial amount of time as well as the use of machines which may not always be available. It should be noted also that the complexity of MRF algorithms is a function of the number of gray levels required in the synthetic texture. On the contrary, the complexity of the RNN texture generation algorithm does not depend on the number of gray levels of the required texture (a 256  256 pixels texture with 256 gray levels is generated in less than 3 seconds using the Java program on 233 MHz Pentium PC19 ). From the modeling (learning) point of view, the MRF process requires large images to get good parameters estimate (usually the MRF parameters are extracted using the least square technique). On the other hand, for the RNN, small images can be sucient for the learning and generalization. We present simulation for the learning and generation of synthetic and natural textures. In both cases, the original and synthetic textures were visually much alike. To test the similarity, the co-occurrence matrix statistical

features were calculated and compared. Finally, it should be noted that both MRF and RNN are unable to model some inhomogeneous natural textures. Further research may consider learning the RNN model with a relatively larger training windows trying to extract the inhomogeneity of the texture.

REFERENCES

1. L. Hogen and L. Mirvish, \The pigmentary eector system. V. The nervous control of excitement pallor in reptiles," J. Exptl. Biol., vol. 5, pp. 295-308, 1928. 2. L. Hogen and D. Slome, \The pigmentary eector system. VIII. The dual receptive mechanism of the amphibian background response," Proc. Roy. Soc., vol. B120, pp. 158-173, 1936. 3. L. Hogen and F. Landgrebe, \The pigmentary eector system. IX. The receptor elds of the teleostean visual response," Proc. Roy. Soc., vol. B128, pp. 317-342, 1940. 4. L. Hogen, \Chromatic behaviour," Proc. Roy. Soc., vol. B131, pp. 111-136, 1942. 5. C. Hughes, B. Sillery, G. Sinha, and F. Vizard, "How chameleons change color," Popular Science, vol. 253, no. 1, 1998. 6. R. M. Haralik, K. Shanmugam, and I. Dinstein, \Textural features for image classi cation," IEEE Trans. Syst. Man Cybern., vol. 3, 1971, pp. 610-621. 7. H. Voorhees and T. Poggio, \Detecting textons and texture boundaries in natural images," in Proc. rst Int. Conf. on Computer Vision, London, 1987, pp. 250-258. 8. J. M. Coggins and A. K. Jain, \A spatial ltering approach to texture analysis," Pattern Recognition Letters, vol. 3, 1985, pp. 195-203. 9. G. C. Cross and A. K. Jain, \Markov random eld texture models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 5, 1983, pp. 25-39. 10. R. Chellappa, B. S. Manjunath, and T. Simchony, \Texture segmentation with neural networks," in Neural Networks for Signal Processing, Ed. Bart Kosko, Prentice Hall, 1992. 11. A. Visa, \A texture classi er based on neural network principles," Proceesings of the International Joint Conference on Neural Networks, San Diego, vol. I, pp. 491-496. 12. N. Rao and V. Vemuri, \A neural network architecture for texture segmentation and labeling," Proceesings of the International Joint Conference on Neural Networks, Washington, DC, vol. I, pp. 127-133. 13. E. Gelenbe, \Random neural networks with negative and positive signals and product form solution," Neural Computation, vol. 1, no. 4, pp. 502-511, 1989. 14. E. Gelenbe, \Learning in the recurrent random neural network," Neural Computation, vol. 5, no. 1, pp. 154-164, 1993. 15. V. Atalay and E. Gelenbe, and N. Yalabik, \Texture generation with the random neural network model," International Journal of Pattern Recognition and Arti cial Intelligence, vol. 6, no. 1, pp. 437-446, 1992. 16. F. S. Cohen and D. B. Cooper, \Simple parallel hierarachical and relaxation algorithms for segmentation noncausal Markov random elds," IEEE Trans. Pattern Anal. Mach. Intell., vol. 9, no.1, (1987). 17. P. Brodatz, Textures: a photographic album for artists and designers, Dover, New York, 1956. 18. H. Derin and H. Elliott, \Modeling and segmentation of noisy and textured images using Gibs random elds," IEEE Trans. Pattern Anal. Mach. Intell., vol. 9, no. 1, pp. 39-55, (1987). 19. H. M. Abdelbaki and K. F. Haussain, Random Neural Network Based Texture Generator, Interactive Java applet available at http://www.cs.ucf.edu/~ahossam/texture/RnnTextureApp/RnnTextureApp.html