A Coloring Method of Gray-Level Image using

A Coloring Method of Gray-Level Image using Neural Networks Jang-Hee Yoo and See-Young Oh

Network Computing Department, Systems Engineering Research Institute P.O. Box 1, Yoosung, Taejeon, 305-600, Korea; e-mail : [email protected]

Abstract In this paper, we describe a coloring method of gray-level images in a restricted area based on neural networks. The coloring method employs color clustering and classi cation algorithms to images in an application area. In this research, the self-organizing feature map algorithm for clustering is applied to construction of a codebook. Variations of intensity in the gray-level image are classi ed into corresponding codevectors using the back-propagation algorithm. The coloring is accomplished by clustering the classi ed codevectors of a gray-level image into colorvectors of the constructed codebook. Also, the proposed method is demonstrated in experiments with portrait images.

1 Introduction Computers and communication network technology have led to complex and various application services. The demand of multimedia information service increases exponentially in a very wide variety of applications. Many of these application areas (such as remote sensing, medical image, digital libraries, entertainment and lm, videoconferencing, and videophone) are based on digital image processing technology. In many areas the applications deal with color images, but other areas of application (such as medical imaging, digital libraries and documents) deal with gray-level images. In addition, the human eye is more sensitive to color changes of image rather than to intensity changes [Cotton, 1996; Parker, 1994; Pitas, 1993]. Therefore, the encoding of the intensity of gray-level images into color changes is more natural and important for the quality of multimedia services. Currently, there is a pseudocoloring method that a perceptual dynamic range of a display can be eectively increased by coding complex information in colors. The pseudocolor image in which the colors displayed need have no relation to the color that appears in the actual scene [Parker, 1994; Pitas, 1993]. This method is applied to gray-level images which can not be captured in color, like medical imaging, for image enhancement through coloring. In fact, this method can not be used for restoration of actual color. Therefore, a variety of image enhancement and coloring techniques, which improve the quality and eective maintenance of transferred or input image data, are needed in order to develop the many kinds of

multimedia services through computers and communication networks. On the other hand, neural networks have been eectively applied in various applications of pattern recognition, signal and image processing [Zurada, 1992]. Neural networks have demonstrated good performance in recognizing hand-written characters, image compression, image restoration, and image recognition. The useful neural network models for image processing include LVQ (learning vector quantization), SOFM (self-organizing feature map), Hop eld, back-propagation. LVQ and SOFM are useful pattern clustering algorithms for vector quantization in image compression, and the Hop eld model is useful for image retrieval. In addition, back-propagation is the most widely applied neural network algorithm [Smith, 1993; Zurada, 1992]; it has many application areas including pattern classi cation and recognition. In this paper, we describe a method for the coloring of gray-level image in a restricted area using neural networks. The codebook is constructed by applying SOFM neural network algorithm to color portrait images. The back-propagation algorithm is employed for classi cation of intensity in gray-level portrait images to corresponding codevectors. Finally, the coloring is accomplished by encoding with the new colormap, which is acquired by mapping the classi ed codevector into the constructed codebook through SOFM algorithm.

2 Color Clustering using Neural Network One of the important components for coloring graylevel images is the codebook which is used as a reference to encode intensity variants in the image to colorvector. A large number of codebook construction methods exist such as k-means and various neural network models like LVQ, SOFM, and others [Fritzke, 1997; Schalko, 1992]. A well-known approach, however, is the LBG [Linde, Buzo, and Gray, 1980] algorithm. But, its required computation time becomes enormous as the vector dimension increases, and forecasting computation time is dicult. In addition, it is quite possible with the wrong initialization for the algorithm to converge into a local minimum [Fritzke, 1997; Linde, Buzo, and Gray, 1980]. In contrast, the Kohonen neural net-

work for SOFM algorithm has fast learning speed, selforganization, and continuous learning. Therefore, when statistical distribution of input data becomes changed with time, the network can adapt automatically to this change. Kohonen's network is basically one or two dimensional interconnected arrays of parallel operating neurons at the same level. Input vectors X(t) = [x1(t); x2(t); :::; xn(t)]T 2 Rn are fully connected with all output nodes via weights mi (t) = [mi1(t); mi2 (t); :::; min (t)]T 2 Rn . The learning of algorithm is performed within a winning node to the given input patterns. For this, after determining a winning node, Kohonen's algorithm adjusts the weight vectors as follows by a lateral inhibition scheme [Kohonen, 1984; Kohonen, 1990]:  mi (t) + (t)[x(t) ? mi (t)] if i 2 Nc (t) mi (t + 1) = m otherwise i (t) In the equation, (t) is the learning rate and decreases monotonically (0 <  < 1), and Nc (t) is the function for topological neighborhoods and decreases in time. Thus, update learning rate and topological neighborhoods decrease in time and only the winning node is updated nally. On the other hand, a color model is a coordinate system in which colors can be speci ed and measured. The most widely used color model in a computer device is the RGB model. It de nes colors in a 3-dimensional Cartesian coordinate system, in terms of the three primary components: red, green, and blue. When coded into digital representation using one byte for each primary, the RGB color space becomes a lattice with 256*256*256 color points. One of the weaknesses of the RGB model is its nonuniform nature; equal distances in the RGB color space do not generally correspond to equal dierences in color perception. To overcome this problem, a uniform color space such as CIE L*u*v mode is developed for applications. The CIE L*u*v color space is probably the best color model in terms of being approximately perceptually uniform [Cotton, 1996; Pratt, 1991; Xiang and Joy, 1994]. The problem of color clustering in 3-dimensional color space is de ned and shown to be a problem of assigning a large number of vectors with 3-components to a small number of clusters (codebook). \Vector quantization", \clustering", and \unsupervised learning" in codebook construction are all words which describe the same process: mapping of input vectors of a particular size into a nite set of codevectors in the images which are represented by these color models [Pratt, 1991; Snyder et al., 1991]. Figure 1 shows the procedure of codebook construction using SOFM algorithm. The size of codebook and codevector for a codebook construction is important issues. The size of codebook using SOFM algorithm is de ned by a number of neurons of the Kohonen neural network. That is, a neural network architecture as in the

16 16

Color Images

Normalization Color Clustering

Fig. 1. Codebook Construction using SOFM gure has 256 neurons of 16*16 for codebook construction of 256 colors. Also, the connection vectors and input vectors in the Kohonen neural network use normalized values of 0 to 1. Accordingly, the codebook is constructed by learning of the neural network using the normalized input data.

3 Color Classi cation using Neural Network The colors added to a gray-level image are meant for conveying information more clearly to a human rather than for further computer processing [Parker, 1994]. It is not a restoration of color in actual images but an image enhancement where colors have been added to a gray-level image. The pseudocoloring method is used for this kind of purpose; this is a digital image transformation of the following form [Pitas, 1993]:

c(x; y) = bfT (f(x; y)) In the equation, f(x; y) is a gray-level image, and c(x; y) is a color image. It is usually expressed in terms of its red, green, and blue components c = cR ; cG ; cB T which is described in the previous section. The transformation function T produces a three-channel output. As the result of pseudocoloring is usually judged only subjectively, the choice of the transformation function T is rather heuristic. Therefore, the results of pseudocoloring depend on the suitability of the function. There are two useful methods to nd pseudocoloring transformations. The rst method is based on intensity quantization (slicing), and the other method is based on a ltering approach [Parker, 1994; Pitas, 1993; Pratt, 1991]. Also, if computational speed is of vital importance as in video or animation applications, implementation is accomplished by using look-up tables. Consequently, coloring to image is performed possibly by choice of each entry in the colormap against desired values. This method has similar characteristic to pattern classi cation models which map intensity variations of a gray-level image mapping to match values of the color codebook. Pattern classi cation divides a collection of data sets into several classes based upon some degree of

similarity [Schalko, 1992]. This pattern classi cation model is the most familiar application area of neural networks [Schalko, 1992; Zurada, 1992]. Back-propagation has been known to be a useful algorithm in applications of pattern classi cation and recognition. In the back-propagation algorithm, weights are recursively adjusted with a set of pairs (input values and corresponding output values) until the value of the dierence between desired output and actual output is less than the predetermined tolerance value [Smith, 1993; Zurada, 1992]. Thus, in the rst step of the learning procedure is a forward pass procedure in which input vector or pattern is given to learning algorithm and output value for each neuron is calculated with input values to the neuron (the weighted linear sum of all the outputs from the previous layer) and activation function. The second step is a backward pass procedure in which the dierence between desired output and actual output is back-propagated and the weights between two sequential layers are adjusted. Target

Color Images

Normalization

Gray-level image

Colormap (codebook)

Neural Network Training for Color Classifiction

Color Classification using BP

Color Clustering using SOFM

Codebook Construction using SOFM

New (colored) colormap

Color Encoding

Fig. 3. Procedure of Coloring using Neural Networks

Normalization

Input Gray-Level Images

classi cation in application area. The construction of the codebook, as described in the previous section, is a mapping procedure of color, which is frequently used in an application area, into a nite set of the colormap using clustering algorithm. The pattern classi cation in the coloring procedure is assigning variants of intensity into the corresponding codevectors in order to encode. As a result, the codebook is constructed by clustering of colorvectors which correspond to each pixel in the actual image of the application area. Also, the colormap of a gray-level image can be obtained by classi cation of intensity variations in the colormap. Gray-level images are created by color images, to codevector in the colormap of corresponding actual images.

Color Classification

Fig. 2. Color Classi cation using Back-Propagation Figure 2 shows the procedure of color classi cation using back-propagation algorithm. The training set and preprocessing for classi cation are an important factor in training the neural network. A training set is made by normalizing each codebook (colormap) using a pair set of color and corresponding gray-level images. The graylevel image is made by gray-leveling of the color image. The back-propagation neural network algorithm usually uses the input vector of ?1 to +1. Then, training of the neural network for color classi cation is performed by using colorvectors as input data, which are normalized to a binary number in the codebook of gray-level image, and corresponding colorvectors as desired output in the codebook of color image.

4 Coloring of Gray-Level Image using Neural Networks The coloring of gray-level images using neural networks is accomplished by codebook construction and pattern

Figure 3 shows the procedure of coloring using neural network algorithms. The coloring procedure consists of four major steps as in the gure. First, make a colormap of gray-level images and normalize the colormap in order to process in the back-propagation neural network as input vectors. Second, classify the normalized colormap into the corresponding color codevector using trained back-propagation neural network. Third, make a new colormap for coloring. In this process, the codevectors of the classi ed colormap are clustered into codevectors of the constructed codebook using SOFM neural network. Finally, encode color using the new colormap. The important factor in the procedure is the quality of the color codebook and the performance of the trained neural network for classi cation. In this research, the 128*128 input portrait images through the scanner are used for the experiment. Each pixel value in the input images is normalized for the codebook construction by clustering. The SOFM neural network of 16*16 neuron structure is used for codebook construction of 256 codevectors with 8-bits. A pair of codebooks is made by the clustering of each pixel value in the color and the gray-level portrait image. Also, a binary number with 8-bits is used for normalized intensity values in the colormap. Codevectors of the codebook are encoded to 256 colors using the look-up table. The codevectors of the codebook (colormap) are binarized through preprocessing for the training set. The normalization of the training set is vector quantization of 8-bits against each pixel in color portrait images.

The back-propagation neural network uses the training set with input, which is the colorvector (codevector) of the codebook in gray-level images, and the desired output, which are colorvectors of the codebook in the corresponding color images. The back-propagation neural network for color classi cation is trained by the constructed training set. The number of images for the training set is four portraits. The back-propagation neural network has a structure of 16 input neurons, 24 hidden neurons, and 8 output neurons. The parameters for training the neural network are 0.3 for learning rate and 0.5 for momentum. However, the back-propagation algorithm requires a lot of training time. The training is performed until the neural network can classify 80% of input vectors. Also, the training program is executed for 2 weeks at SUN SparcStation 20.

(a) Source image

(b) Color image

Fig. 4. Experiment Result Figure 4 shows the result of the experiment. Part (a) in the gure is a source gray-level image for coloring, and part (b) in the gure is a color image made by the proposed method. The comparison between the original color image and colored image is tested by using Hammingdistance for evaluation of the proposed method. Image (b) in the result of the experiment has some noise. This problem related to the constructed codebook and used the color model. The RGB color model was used for the experiment. As described in the previous section, the RGB model, which has nonuniform space, has a sensitive feature to values of codevector. Therefore, the processing in uniform color space will be more useful for color transformation, also this method can get good results applied to coloring through image segmentation.

5 Conclusion In this paper, we described a coloring method of graylevel image using pattern clustering and classi cation technique by neural network algorithms. The SOFM neural network algorithm was used for codebook construction in the coloring application area. Also, variants of intensity are classi ed to codevectors for color encoding using a back-propagation neural network algorithm. The performance of the proposed method depends on the quality

of the constructed codebook and the performance of the trained neural network for color classi cation. In this research, portrait images are used for evaluation of the method through the experiment. In the experiment, the encoding of color in RGB space has sensitive features dependent on color values with a codevector. This research can be eectively applied to application areas that document data, videophone, videoconferencing, gray-level pictures, and lm, which need color restoration of actual images. However, a processing in uniform color space, useful codebook construction, and eective normalization method of training set are needed for enhancement of the performance in the method.

References [Cotton, 1996] Symon D'O. Cotton, Colour, colour spaces and the human visual system, Technical Report, B15-2TT, University of Birmingham, May 1996. [Fritzke, 1997] Bernd Fritzke, The LBG-U method for vector quantization - an improvement over LBG inspired from neural networks, Neural Processing letters, Vol.5, No.1, Kluwer Academic Publishers, 1997. [Kohonen, 1984] Teuvo Kohonen, Self-Organization and Associative Memory, Springer, New York Berlin Heidelberg, 1984 (3rd edition 1989). [Kohonen, 1990] Teuvo Kohonen, The Self-Organizing Map, Proceeding of the IEEE, Vol.78, No.9, pp.1464-1480, Sep. 1990. [Linde, Buzo, and Gray, 1980] Y. Linde, A. Buzo, and R. M. Gray, An algorithm for vector quantizer design, IEEE Transaction on Communication, COM-28 (1), pp.84-95, 1980. [Parker, 1994] Jim R. Parker, Practical Computer Vision using C, John Wiley & Sons, 1994. [Pitas, 1993] Ioannis Pitas, Digital Image Processing Algorithms, Prentice Hall, 1993. [Pratt, 1991] Willeam K. Pratt, Digital Image Processing, John Wiley & Sons, 1991. [Schalko, 1992] Robert Schalko, Pattern Recognition: Statistical, Structural and Neural Approaches, John Wiley & Sons, 1992. [Smith, 1993] M. Smith, Neural Networks for Statistical Modeling, Van Nostrand Reinhold, 1993. [Snyder et al., 1991] Wesley Snyder, Daniel Nissman, David Van den Bout, and Gri Bilbro, Kohonen networks and clustering: comparative performance in color clustering, Advances in Neural Information Processing Systems, Vol.3, pp.984-990, Morgan kaufmann Publishers, 1991. [Xiang and Joy, 1994] Zhigang Xiang and Gregory Joy, Color image quantization by agglomerative clustering, IEEE Computer Graphics and Applications, pp.44-48, May 1994. [Zurada, 1992] Jacek M. Zurada, Introduction to Arti cial Neural Systems, West Publishing Company, 1992.

References

[Cotton, 1996] Symon D'O. Cotton, Colour, colour spaces and the human visual system, Technical Report, B15-2TT, University of Birmingham, May 1996. [Fritzke, 1997] Bernd Fritzke, The LBG-U method for vector quantization - an improvement over LBG inspired from neural networks, Neural Processing letters, Vol.5, No.1, Kluwer Academic Publishers, 1997. [Kohonen, 1984] Teuvo Kohonen, Self-Organization and Associative Memory, Springer, New York Berlin Heidelberg, 1984 (3rd edition 1989). [Kohonen, 1990] Teuvo Kohonen, The Self-Organizing Map, Proceedings of the IEEE, Vol.78, No.9, pp.1464-1480, Sep. 1990. [Linde, Buzo, and Gray, 1980] Y. Linde, A. Buzo, and R. M. Gray, An algorithm for vector quantizer design, IEEE Transaction on Communication, COM-28 (1), pp.84-95, 1980. [Parker, 1994] Jim R. Parker, Practical Computer Vision using C, John Wiley & Sons, 1994. [Pitas, 1993] Ioannis Pitas, Digital Image Processing Algorithms, Prentice Hall, 1993. [Pratt, 1991] Willeam K. Pratt, Digital Image Processing, John Wiley & Sons, 1991. [Schalko, 1992] Robert Schalko, Pattern Recognition: Statistical, Structural and Neural Approaches, John Wiley & Sons, 1992. [Smith, 1993] M. Smith, Neural Networks for Statistical Modeling, Van Nostrand Reinhold, 1993. [Snyder et al., 1991] Wesley Snyder, Daniel Nissman, David Van den Bout, and Gri Bilbro, Kohonen networks and clustering: comparative performance in color clustering, Advances in Neural Information Processing Systems, Vol.3, pp.984-990, Morgan kaufmann Publishers, 1991. [Xiang and Joy, 1994] Zhigang Xiang and Gregory Joy, Color image quantization by agglomerative clustering, IEEE Computer Graphics and Applications, pp.44-48, May 1994. [Zurada, 1992] Jacek M. Zurada, Introduction to Arti cial Neural Systems, West Publishing Company, 1992.