BULETINUL INSTITUTULUI POLITEHNIC DIN IAŞI Publicat de Universitatea Tehnică „Gheorghe Asachi” din Iaşi Tomul LVII (LXI), Fasc. 3, 2011 SecŃia AUTOMATICĂ şi CALCULATOARE
UNSUPERVISED COLOR - BASED IMAGE RECOGNITION USING A LAB FEATURE EXTRACTION TECHNIQUE BY
TUDOR BARBU∗, ADRIAN CIOBANU and MIHAELA COSTIN Institute of Computer Science, Romanian Academy, Iaşi
Received: August 3, 2011 Accepted for publication: September 16, 2011
Abstract. We propose an automatic content-based image recognition technique in this paper using color features. Our intention is to cluster a set of digital images in several categories on the color similarity basis. The images are processed using LAB color space in the feature extraction stage. The resulted color-based feature vectors are clustered using an automatic unsupervised classification algorithm. Some experiments based on the proposed recognition technique have also been performed. The described recognition method can further be applied in content-based indexing and retrieval (CBIR) domains. Key words: image recognition, color-based feature extraction, LAB colorspace, unsupervised classification, K - means algorithm, validation indexes. 2000 Mathematics Subject Classification: 62H30, 68T10, 68T45, 68U10.
1. Introduction Digital image recognition represents an important computer vision domain (Ballard et al., 1982). An image recognition process consists of classifying a set of images on the basis of their content similarity. As any ∗
Corresponding author: e-mail:
[email protected]
34
Tudor Barbu, Adrian Ciobanu and Mihaela Costin
pattern recognition task, image recognition is characterized by a feature extraction step and a feature vector classification one (Duda et al., 2001). The feature extraction consists of an image content analysis. As one knows, the image content is provided by its color, texture and shape information. Numerous color, texture and shape analysis techniques have been developed in recent years (Dong et al., 2008; Pradhan et al., 2008; Milisavljevic, 1999). We propose a color-based image recognition approach in this paper. The image classification process may have a supervised character, using a template image set, or an unsupervised character. We provide an automatic unsupervised image recognition technique (Barbu, 2006; Barbu, 2007). Therefore, we consider the following automatic recognition task (Barbu, 2007). If {I 1 ,..., I N } represents the set of the color images to be recognized, these digital images have to be clustered automatically in a number of color-based classes. First, we propose an image feature extraction that uses LAB color space characteristics (Ciobanu et al., 2010). This featuring process is described in the next section. In the third section we present an automatic feature vector clustering technique based on a K-means algorithm (Kanungo et al., 2002). Some experiments performed with the proposed method are described in the fourth section. The article ends with a conclusion section. 2. Feature Extraction Using LAB Color Space The characteristics of the LAB color space makes it suitable for extracting global color features from a digital image. Therefore we convert the analyzed RGB images into LAB format. The characteristic known as luminance or intensity is represented on the axis named L, that is perpendicular on a pile of “ab” planes, each one containing all the possible colors for a given luminance (Fairchild, 2005). One of the most useful characteristic of the LAB color space is that the colors are uniformly distributed in an “ab” plane, from green to red along the “a” axis and from blue to yellow along the “b” axis (Fig. 1). One color is defined by a point (a1, b1) in an “ab” plane of a given luminance L, and colors are changing gradually and uniformly in the plane around this point. It is important that the human eye perceives this gradual change of color as an uniform one. The values of the coordinates L, a and b are real numbers when applying RGB to LAB mathematical conversion, but, for programming convenience, these values are mapped to 256 levels.
Bul. Inst. Polit. Iaşi, t. LVII (LXI), f. 3, 2011
35
Fig. 1 – Color Distribution in “ab” plane for L=134.
A precise color is defined in the LAB color space by a triplet (L,a,b), much like a triplet (R,G,B) defines a color in the RGB color space. However, in LAB the pair (a,b) can be viewed as a pure color, and the L coordinate gives only how darker or lighter that color is seen by the human eye. A reduction of the number of colors is needed in order to obtain global characteristics (Ciobanu et al., 2010). With the LAB color space, such a reduction can be done by considering only several “ab” planes from the total of 256. We choose to use 8 “ab” planes corresponding to the following central L values: 16, 48, 80, 112, 144, 176, 208 and 240. Any value of L in an image is forced to take the nearest value of this set of values, resulting in creating some regions with uniform color and some loss in texture. Then, for each of the 8 selected “ab” planes, we construct one planar histogram, having [256 x 256] bins, one for every possible (a,b) color pair. We add 1 to the bin corresponding to the color determined by the (a,b) pair if we found in the image a pixel having as color coefficients the pair (a,b) along with the L value corresponding to the luminance L. Browsing all the pixels of an LAB converted image results in an easy computation of the eight planar histograms (Fig. 2). To avoid working with insignificant colors we have cleared the bins containing less than 20 points and thus the planar histogram became scarce in values (with a lot of bins empty) and thus further processing become faster.
36
Tudor Barbu, Adrian Ciobanu and Mihaela Costin
Fig. 2 − Planar histogram for L = 80 in one of our test images, darker pixels meaning color bins with more pixels in the image, white pixels meaning there are no pixels in the image with that color.
Instead of computing important colors globally and give a list of 10 to 50 colors in their order of importance for an image, we compute one important color for several subsets of possible colors. In an “ab” plane (Fig. 1) we can see mostly red colors in the upper right quadrant (ok, some yellow too!), mostly nuances of violet in the lower right quadrant, mostly nuances of blue in the lower left quadrant and mostly nuances of green in the upper left quadrant. Therefore, we take into consideration each quadrant of an “ab” plane as an independent subset of colors. It is ok to consider quadrants for a global case of classifying or retrieving images based on their color content. However, for dedicated applications it would be possible to split the “ab” planes in more complicated shapes in order to depict some specific colors. As we take into consideration 8 “ab” planes, we have 32 quadrants where we have to identify the most important color. For each quadrant a list of (a,b) pairs with corresponding nonzero bins in the planar histogram is constructed. Then the (a,b) pairs are ordered according to the values in the bins, the colors with a greater number of corresponding pixels in the image being on top of this list. Then colors very close in the “ab” planes are merged in this lists, which became a list of fewer colors, but with larger counts of pixels. Finally, the top (a,b) pair is taken as the most important color in the corresponding quadrant. Collecting all the 32 important colors provides us the following feature vector: i a i … a ij … a 32 V ( I i ) = 1i i i b1 … b j … b 32
(1)
Bul. Inst. Polit. Iaşi, t. LVII (LXI), f. 3, 2011
37
where each column represents a winning (ai, bi) color pair. The color contents of two images can be easily compared on the base of the color features extracted as above. It is essential to use the fact that the 32 components of the feature matrix in fact 32 pairs of coordinates designating points in an “ab” plane of the LAB color system. Two colors in such a plane are similar if the distance between them is small enough. We use the Euclidean distance as a metric for these color feature vectors. The 32 pairs in the feature matrix are ordered from low to high “ab” luminance planes, and every four values corresponding to the same luminance are ordered from quarter 1 to 4 of their corresponding place in the “ab” plane. This assures that comparing two color components of two images means comparing the distance between two points in the same “ab” plane and the same quadrant. 3. An Automatic Feature Vector Classification Approach We propose an automatic unsupervised feature vector classification technique in this section. Our clustering approach groups the N computed feature vectors, V (Ii),…,V (IN), in a number of K color classes (Barbu, 2006; Barbu, 2007). The provided method uses a sequence of K-means procedures and identifies the optimal number of clusters using a validation index. The Kmeans algorithm, proposed by MacQueen in 1967, represents an unsupervised method for cluster analysis that aims to partition N observations into K clusters in which each observation belongs to the cluster with the nearest mean (Kanungo et al., 2002). The following objective function has to be minimized by the K-means procedure, as a within-cluster sum of squares: F (K ) =
∑ ∑ d (V ( I K
j
), C i )
(2)
i =1 j∈ Ind ( C i )
where: d is the Euclidian distance, C i − the centroid of the current cluster and Ind (C i ) represents the set of the indices of pixels corresponding to the feature vectors from that cluster. The clustering procedure uses an iterative refinement algorithm, containing the following steps: 1. Initialize K centroids, one for each cluster, each centroid representing a point in the feature vector space. We select them as much as possible far away from each other; 2. Each image feature vector is associated to the closest centroid; 3. The centroids of the obtained clusters are recomputed, as means of the contained feature vectors;
38
Tudor Barbu, Adrian Ciobanu and Mihaela Costin
4. Steps 2 and 3 are repeated until the centroids of classes no longer change their positions. The optimal number of clusters is determined using the Dunn cluster validity index (Dunn, 1974). It aims to maximize the inter-cluster distances and minimize the intra-cluster distances. The number K that maximizes the Dunn index is considered the optimal number of clusters. The main advantage of Dunn index is its linear time complexity, the main disadvantage being the vulnerability for noise in the data (Dunn, 1974). Therefore, if the obtained image feature vector classes, for a given K, are Cl 1 ,..., Cl K , then the optimal number of clusters is computed as:
K optim = arg max (D ( K ) ) K ∈[1, N ]
(3)
where D ( K ) = min min i∈[1 , K ] j≠i
d (C , C ) i j d (Cl t ) tmax ∈[1 , K ]
(4)
C i being the centroid of the cluster Cl i and d (Cl i ) the intra-cluster distance
of that cluster, representing the absolute/squared distance between all pairs of cluster elements (Dunn, 1974). As one can see in formula (3), the K - means algorithm is executed N times for determining the optimal number of clusters. If the number of images is very high, the computational cost increases very much, therefore we could use a lower number of running times, for example N/2. The obtained sequence of image classes Cl 1 ,..., Cl K optim represents the final pattern recognition result. 4. Experiments Numerous experiments have been performed on large image datasets using this color-based image recognition technique. Some satisfactory results have been obtained, a high image recognition rate, over 80%, being achieved. High values for Precision and Recall performance parameters have been also obtained. A simple experiment performed with our recognition technique is described in Fig. 3. There are 9 color images which are clustered into 4 classes.
Bul. Inst. Polit. Iaşi, t. LVII (LXI), f. 3, 2011
39
Fig. 3 − Images grouped in four colors - based classes.
As one can see in Fig. 1, each image is labeled with its class number. The first cluster contains two beach images, the second one is composed of two images representing horses, the third one contains two images representing red flowers, while the fourth cluster contains three images which represent vestiges. 5. Conclusions We have presented a pattern recognition approach for color images in this article. Our paper has brought original contributions in both feature extraction and classification stages. We have proposed a novel color feature extraction approach using LAB space. Some robust color-based feature vectors, based on (a,b) coordinates in this color space are obtained. Next, an automatic feature vector classification technique has been provided. It uses a sequence of K-means procedure and a validation index that computes the optimal number of clusters.
40
Tudor Barbu, Adrian Ciobanu and Mihaela Costin
The results described in this paper can be successfully applied in the image retrieval and indexing domains (Barbu et al., 2009). The unsupervised image recognition method could be used in the automatic development of a cluster-based image index. Image retrieval solutions can also be implemented using the proposed recognition method (Barbu et al., 2009). Our future research in this domain will focus on developing some colorbased image indexing and retrieval techniques and on improving our image recognition technique by modeling new color feature vectors and classification approaches.
REFERENCES Ballard D.H., Brown C.M., Computer Vision. Prentice Hall, 1982, 528. Barbu T., An Automatic Unsupervised Pattern Recognition Approach, Proceedings of the Romanian Academy, Series A, Vol. 7, 1, January -April 2006, 73−78. Barbu T., An Automatic Image Recognition Approach. Computer Science Journal of Moldova, Vol. 15, 2 (44), 2007, 202−211. Barbu T., Ciobanu A., Color-Based Image Retrieval Technique Using Relevance Feedback. Proceedings of Third International Conference on Electronics, Computers and Artificial Intelligence, ECAI 2009, Vol. 4, Piteşti, România, 35 July 2009, 105−108. Ciobanu A., Costin M., Barbu T., Extraction of Main Colors from a Digital Color Image. Proceedings of the International Multidisciplinary Scientific GeoConference SGEM 2010, Vol. 1, June 20-25, 2010, Albena, Bulgaria, 1069−1076. Dong W., Shisheng Z, Color Image Recognition Method Based on the Prewitt Operator. Proceedings of International Conference on Computer Science and Software Engineering, Vol. 6, 2008, 170−173. Duda R.O., Hart P.E., Stork D.G., Pattern Classification. 2nd Edition, Wiley, New York, 2001. Dunn J., Well Separated Clusters and Optimal Fuzzy Partitions. Journal of Cybernetics, Vol. 4, 1974, 95−104. Fairchild M.D., Color and Image Appearance Models. Color Appearance Models, John Wiley and Sons, 2005, 340−385. Kanungo T., Mount D.M., Netanyahu N.S., Piatko C.D., Silverman R., Wu A.Y., An Efficient K-Means Clustering Algorithm: Analysis and Implementation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, 7, 2002, 881−892. Milisavljević N., Comparison of Three Methods for Shape Recognition in the Case of Mine Detection. Pattern Recognition Letters, Vol. 20, 11-13, 1079−1083, November 1999. Pradhan R., Bhutia Z.G., Nasipuri M., Pradhan M.P., Gradient and Principal Component Analysis Based Texture Recognition System: A Comparative Study, Proceedings of the Fifth International Conference on Information Technology: New Generations, 2008, 1222−1223.
Bul. Inst. Polit. Iaşi, t. LVII (LXI), f. 3, 2011
41
RECUNOAŞTEREA NESUPERVIZATĂ A IMAGINILOR BAZATĂ PE CULOARE, UTILIZÂND O TEHNICĂ DE EXTRAGERE A TRĂSĂTURILOR COLOR ÎN FORMAT LAB (Rezumat) În lucrarea de faŃă se abordează recunoaşterea imaginilor digitale pe baza caracteristicilor de culoare ale acestora. Tehnica propusă clusterizează în mod automat imaginile dintr-un set în mai multe clase, pe baza similarităŃii de conŃinut a acestora. Extragerea trăsăturilor de culoare ale conŃinutului se efectuează în spaŃiul de culoare LAB, prin urmare o conversie din formatul RGB este necesară în prealabil. Pentru fiecare imagine color în spaŃiul LAB se calculează un vector al trăsăturilor de culoare de tip matriceal, pe baza perechilor (a,b) ale celor mai importante culori determinate în imagine. În scopul clusterizării acestor vectori de caracteristici, se propune în continuare o tehnică de clasificare nesupervizată automată. Metoda prezentată constă în aplicarea repetată a unui algoritm de clasificare de tip K-medii, până la determinarea unei valori optime K a numărului de clustere. În scopul optimizării procesului de clusterizare se utilizează un index de validare Dunn. Numeroase experimente au fost efectuate pe baza acestei tehnici de recunoaştere, câteva exemple fiind descrise şi în lucrare. Rata de recunoaştere obŃinută este de peste 80%. Ca o concluzie, articolul prezintă contribuŃii originale în recunoaşterea imaginilor pe bază de culoare, atât în faza extragerii trăsăturilor cât şi în etapa de clasificare a vectorilor de trăsături. Caracterul automat al tehnicii propuse este extrem de important în cazul procesării unor seturi vaste de imagini, din acest motiv metoda poate fi utilizată cu succes în scopul indexării bazelor de date conŃinând imagini color.
