2013 International Conference on Advanced Computing and Communication Systems (ICACCS -2013), Dec. 19 – 21, 2013, Coimbatore, INDIA
An Improved Edge Direction Histogram and Edge Orientation AutoCorrlogram for an Efficient Color Image Retrieval K.Seetharaman
S.Sathiamoorthy
Department of Comp. Sci. and Engineering Annamalai University Annamalai Nagar, India
[email protected]
Department of Comp. Sci. and Engineering Annamalai University Annamalai Nagar, India
[email protected]
Abstract—This paper advances the edge histogram descriptor (EHD) of MPEG-7 standards and the edge orientation autocorrlogram (EOAC) by using the Full Range Autoregressive model (FRAR) with Bayesian approach. The model is applied in the HSV color space for extracting the edges. The proposed method preserves the chromatic information and the edges that are loosed by the spectral variations. The extracted edges are used to construct the EHD and EOAC. The proposed EHD and EOAC are extensively tested with the various image databases. Precision and recall method is used to measure the accuracy of the proposed method. The results evident that the proposed method outperforms the existing methods. Keywords—Full Range Autoregressive model; Edge direction histogram; Edge orientation autocorrlogram.
I.
INTRODUCTION
Today, tremendous amount of digital images are generated by various fields such as medicine, agriculture, astronomy, geology etc. Therefore, there is a great need for efficient and effective image retrieval system. Since 1990’s, researchers focused their interest in the development of automated or semi-automated image retrieval techniques based on the textbased keywords [1,2], content of an image such as color, texture, shape etc. [3-10] or by combining the textual and visual features [11, 12]. However, with the vast image collections, text-based techniques become inefficient for providing sufficient and distinctive discriminatory power of the images and it holds the disadvantages of being subjective, laborious, tedious and time-consuming. Literature also reveals that text-based image retrieval techniques suffer due to the limitations of current artificial intelligence methods [13]. Despite the great effort, the performances of early content based image retrieval (CBIR) systems are not satisfactory due to the ineffective and inefficient representation of images and it leads to semantic gap between the low-level image features and high-level semantic concept, which is an important issue in the CBIR systems [14]. Though, the researchers employed the relevance feedback technique to bridge the semantic gap, inadequate discrimination of an image limited the power of relevance feedback technique also.
[978-1-4799-3506-2/13/$31.00 ©2013 IEEE]
Thus, this paper attempts to improve the efficiency of edge direction histogram / edge histogram descriptor (EHD) [5, 13, 15, 16] and edge orientation autocorrlogram (EOAC) [16] using the FRAR model and Bayesian approach [18] which outperforms the existing methods in terms of capturing the edge features of grey-scale images. The rest of this paper is organized as follows. Section II describes a review of proposed features. Section III explains the statistical model and feature extraction from the model employed. Similarity measure is discussed in section IV. Section V explains the proposed methodology. Experimental results described in Section VI. Finally, section VII concludes the paper. II.
REVIEW OF PROPOSED FEATURES
Shape features provide potential information for CBIR systems. Many research efforts have been taken to describe the shape features [19-23] since 1990’s. The effectiveness of shape-based image retrieval system depends on the type of shape representation used. In common, shape representation can be classified into region based and contour based. Region based approach may not be easy and reliable for a diverse collection of images due to the unavailability of fully automated generalized approach [24]. This implies edge detection as more reliable and it contains rich texture and shape information. Edge-detection based feature extraction method uses perimeter, curvature, edge direction etc. to denote the shape on the basis of edges. The MPEG-7 standard uses EHD, which is extensively studied for its simplicity and effectiveness [5, 13, 15, 16]. The MPEG-7’s EHD represents the edge features based on their orientations and it is invariant to translation and small rotation. Subsequently, Mahmoudi et al, (2003) [17] introduced an EOAC, which captures textures, shape and spatial information more effectively, and it describes the edge features based on their orientations and correlation between neighbouring edges, and it is invariant to translation, illumination, viewing position and small rotation. Both the EHD and EOAC uses the Sobel edge detector for its construction. Normalization of both MPEG-7’s EHD and EOAC leads to scale invariance. It is observed from the literature [17] that the retrieval accuracy of EOAC is higher than the MPEG-7’s EHD. However, the aforementioned
2013 International Conference on Advanced Computing and Communication Systems (ICACCS -2013), Dec. 19 – 21, 2013, Coimbatore, INDIA
methods adopted the edge detection algorithm designed for grey-scale images. Literature reveals that applying the edge detector on H, S and V components separately loses some edges caused by the spectral variations [13]. Likewise, extracting the edges of color images from its grey-scale version also loses chromatic information [13]. To overcome this drawback, the proposed method advances the EHD and EOAC by extracting the edges more effectively by using the FRAR model with Bayesian approach [18] in HSV color space. III.
FRAR MODEL
Recent literature reports that, the FRAR model [18] outperforms the existing methods in terms of capturing the edge features of grey-scale images. We explored an improved version of EHD and EOAC by using the edges detected by the FRAR modal. The FRAR model for edge extraction is given in equation (1) with the combination of two sets (all diagonal elements are one set and the remaining as another set in a 3 x 3 region of the image) of elements with the following constraints used in equations (1.1) and (1.2).
C1 = {p, q : −1 ≤ p, q ≤ 1; p ≠ q }
(1.1)
C 2 = {p, q : −1 ≤ p, q ≤ 1; p ≠ 0; q ≠ 0}
(1.2)
X ( k , l) =
1
1
∑ Γ r X (k + p, l + q )
∑
p = −1 q = −1 p ≠ q
+
1
∑
1
p = −1 q = −1 p≠0 q≠0
(1)
where Γ = K sin( r θ ) cos( r φ ) and r = p + q r r α
In the above equation (1), X(k + p, l + q) accounts for the spatial variation owing to image properties and ε(k,l) is the spatial variation owing to additive noise and the model coefficient Γ = K sin( r θ ) cos( r φ ) is the rth coefficient of αr
variation among the low-level primitives in the small image region. The coefficients are interrelated. The interrelationship is established through the model parameters K, α, θ, and φ which are real. The model parameters are estimated with the concept of the Bayesian approach [18]. For a detailed study refer article [18]. IV.
n
D(X, Y ) = ∑ (X i − Yi )
(2)
i =1
Where X i is a query image feature vector, Yi is a target image feature vector in the feature database and n is the size of the feature vector. V.
PROPOSED METHODOLOGY
The proposed system converts the color images from RGB color format into an HSV color format [26] because HSV complies with human perception well. The HSV color space is quantized and transformed into a Cartesian coordinates system as described in [13]. Let (H,S,V) be a point in the cylinder coordinates system, and (H′,S′,V′) be the transformation of (H,S,V) in the Cartesian coordinates system, H′=S. cos(H), S′=S. sin(H) and V′=V. Now, we have applied the FRAR model as in the equation (1) to H′, S′ and V′ of color image g(x,y) in the Cartesian coordinates system to extract the edges. The gradients along x and y directions can then a (H ′x , S′x , Vx′ ) and b( H ′y , S′y , Vy′ ) where H ′x denotes the gradient in a H′ channel along the horizontal direction, and so on. Their norm and dot product can be defined as in [13]
∑ Γ r X ( k + p , l + q ) + ε ( k , l)
r
distance is adopted here. Square or square root operation is not performed in the L1 distance metric. Hence, it can save much computational cost and is very suitable for large scale image database. The L1 distance between query and target images are computed as
SIMILARITY MEASURE
Similarity measure plays a critical role in the CBIR systems by computing the similarity between the query and target images in feature vector space. In the last decade, various similarity measures from computational geometry, statistics and information theory can be used in the CBIR. In the proposed system, the most frequently used geometrical similarity measure (L1) is used. Huang et al., [25] suggested that L1 distance performs better than L2. Hence, a simple L1
a =
(H ′x )2 + (S′x )2 + (Vx′ )2
b =
(H′y )2 + (S′y )2 + (Vy′ )2
ab = H ′x H ′y + S′x S′y + Vx′ Vy′
The angle between a and b is described as in [13] ⎡ a .b ⎤ θ = arccos ⎢ ⎥ ⎣⎢ a . b ⎥⎦
(3)
After computing the orientations of each edge, they are classified into five types; vertical, horizontal, 45-degree, 135degree and non-directional. The EHD of 72 bins is constructed. The edge orientation correlogram [17] is then formed which is a matrix, consisting of 72 rows and 4 columns. Each element of this matrix indicates the number of edges with similar orientation. The columns give the number of edges that are 1, 3, 5, 7 pixels apart. Each row of the matrix corresponds to the aforementioned orientations. The extracted EHD and EOAC are normalized to achieve scale invariant. The extracted features are stored in a separate feature vector database. The feature vectors in the feature vector space is indexed [25]. Finally, the L1 distance similarity measure [25] is used to measure the similarity between the query image and the indexed [27] images in the feature database.
2013 International Conference on Advanced Computing and Communication Systems (ICACCS -2013), Dec. 19 – 21, 2013, Coimbatore, INDIA
VI.
EXPERIMENTAL RESULTS
In order to implement the proposed technique, color image databases such as Freefoto (http://Freefoto.com), Wang’s database (http://wang.ist.psu.edu/docs/related.shtml) and UCID database (http://vision.cs.aston.ac.uk/datasets/UCID/ucid.html) is used. For a sample, some of them have been presented in Figure 1.
(a)
(b)
Fig.1. Sample images taken from the experimental databases. The precision and recall method is used to measure the performance of the proposed technique. Precision is defined as a percentage of the retrieved images that are also relevant whereas recall is described as a percentage of retrieved images that are relevant. A subset of 50 query images was selected at random from each database to evaluate the performance of proposed EHD and EOAC. In the experiment, the proposed EHD and EOAC is compared with the conventional EHD and EOAC. The average precision-recall curve for various image databases is shown in the Figure 2. The average retrieval precision attained by the proposed EHD and EOAC, and conventional EHD and EOAC is 53.30%, 64.29% and 49.19%, 60.18% respectively for Wang database, and 45.97%, 53.65% and 42.87%, 48.81% respectively for UCID database, and 44.14%, 53.04% and 40.02%, 49.35% respectively for Freefoto database. The experimental results clearly reveal that the retrieval performance of the proposed EHD and EOAC is significantly superior to conventional EHD and EOAC. Some of the images in the databases are duplicated and then scaled and rotated at different angles to measure the robustness of the proposed techniques to scaling and rotation. Images taken in different viewing positions and different lighting conditions also considered in our experiments for measuring the robustness of illumination and viewing changes.
(c) Fig. 2. Average precision verses recall of the proposed EHD, EOCA and conventional EHD and EOCA for a) Wang database; b) UCID database; c) Freefoto.com. We observed that the experimental results are more robust than the conventional methods with the scaling, rotation, illumination and viewing invariance. From the experiments, it is also confided that the proposed feature vector has significantly better discriminatory power than the conventional EHD and EOAC due to its preservation of chromatic information, the edges that are loosed by the spectral variations in HSV color space and the effective edge detection by the FRAR model with Bayesian approach, which is insensitive to noise in nature. VII. CONCLUSION In the proposed work, the potential of FRAR model and the Bayesian approach in capturing the edges of grey-scale images is successfully incorporated into color images. The proposed
2013 International Conference on Advanced Computing and Communication Systems (ICACCS -2013), Dec. 19 – 21, 2013, Coimbatore, INDIA
EHD and EOAC are rich in information and have high discriminative power than the MPEG-7’s EHD and EOAC. The reason lies in the preservation of chromatic information and the edges that are loosed by the spectral variations in HSV color space, and the effective edge detection by the FRAR model with Bayesian approach, which is insensitive to noise. The enhanced features will be very useful for the pattern classification and mining problems in fields such as medicine, astronomy, agriculture etc., where the content of an image is more complex.
[12] Burdescu D. D., C. G. Mihai, L. Stanescu, M. Brezovan. 2013. Automatic image annotation and semantic based image retrieval for medical domain. Neurocomputing 109: 33–48.
REFERENCES
[16] Chatzichristofis S.A., K. Zagoris, Y. S. Boutalis and N. Papamarkos. 2010. Accurate image retrieval based on compact composite descriptors and relevance feedback information. International Journal of Pattern Recognition and Artificial Intelligence 24(2) : 207-244.
[1]
Chang. S. K. and A. Hsu. 1992. Image information systems: Where do we go from here?. IEEE Trans. on Knowledge and Data Engineering 4(5): 431-442.
[2]
Jeon. J, V.Lavrenko V, R. Manmantha. 2003. Automatic image annotation and retrieval using cross-media relevance models. In ACM conference on special interest group on information retrieval (SIGIR), 119–126. Toronto, Canada.
[3]
Flickner. M, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafine, D. Lee, D. Petkovic, D. Steele, and P.Yanker. 1995. Query by image and video content: The QBIC system. IEEE Computer 28(9) : 23-32.
[13] Liu G.H., Z.Y. Li, L. Zhang, Y. Xu. 2011. Image retrieval based on micro-structure descriptor. Pattern Recognition 44(9) : 2123-2133. [14] Su Z., H. Zhang, S. Li, & S. Ma. 2003. Relevance feedback in contentbased image retrieval: Bayesian framework, feature subspaces, and progressive learning. IEEE Transactions on Image Processing 12(8) : 924–937. [15] Chang S.F., T. Sikora, and A. Puri. 2001. Overview of the MPEG-7 standard. IEEE Trans. Circ. Syst. Video Technol. 11(6) : 688–695.
[17] .Mahmoudi, F., Shanbehzadeh J., Eftekhari A.M., and Soltanian-Zadeh H., “Image retrieval based on shape similarity by edge orientation autocorrelogram”, Pattern Recognition, vol 36, pp 1725-1736, 2003. [18] Seetharaman K. and R. Krishnamoorthi. 2007. A statistical framework based on a family of full range autoregressive models for edge extraction. Pattern Recognition Letters 28(7) : 759–770. [19] Gagaudakis G., P.L. Rosin. 2002. Incorporating shape into histograms for CBIR. Pattern Recognition 35(1) : 81–91.
[4]
Jain A. K. and A. Vailaya. 1996. Image retrieval using color and shape. Pattern Recognit 29(8) :1233–1244.
[20] Gagaudakis G., P. L. Rosin. 2003. Shape measures for image retrieval. Pattern Recognition Letters 24(15) : 2711–2721.
[5]
Rahman M. M., P. Bhattacharya, B. C. Desai. 2007. A Framework for Medical Image Retrieval Using Machine Learning and Statistical Similarity Matching Techniques with Relevance Feedback. IEEE transactions on information technology in biomedicine 11(1): 59-69.
[21] Wong W.T., F. Y. Shih, J. Liu. 2007. Shape-based image retrieval using support vector machines, Fourier descriptors and self-organizing maps. Information Sciences 177(8) : 1878–1891.
[6]
Chun Y.D., N.C. Kim, and I.H. Jang. 2008. Content-Based Image Retrieval using Multiresolution Color and Texture Features. IEEE Transactions on Multimedia 10(6) : 1073-1084.
[7]
Chen H., Z. Gao, G. Lu, S. Li. 2008. A Novel Support Vector Machine Fuzzy Network for Image Classification Using MPEG-7 Visual Descriptors. International Conference on MultiMedia and Information Technology : 365-368.
[8]
Lin C.H., R.T. Chen, Y.K. Chan. 2009. A smart content-based image retrieval system based on color and texture feature. Image and Vision Computing 27(6) : 658–665.
[9]
Iqbal K., M. O. Odetayo, A. James. 2012. Content-based image retrieval approach for biometric security using colour, texture and shape features controlled by fuzzy heuristics. Journal of Computer and System Sciences 78(4) : 1258-1277.
[10] Seetharaman, K., Sathiamoorthy, S., 2013. A statistical model based color image retrieval. In: Proceedings of the International conference on Recent Trends in Computer Applications (ICRTCA’13), Chennai, India. 91-96. [11] Zhao Y., Y. Zhao, Z. Zhu. 2009. TSVM-HMM: Transductive SVM based hidden Markov model for automatic image annotation. Expert Systems with Applications 36(6) : 9813–9818.
[22] Drew M. S., T. K. Lee, A. Rova. 2009. Shape retrieval with eigen-CSS search. Image and Vision Computing 27 : 748–755. [23] Chiang C.C., Y.P. Hung, H. Yang, G. C. Lee. 2009. Region-based image retrieval using color-size features of watershed regions. J. Vis. Commun. Image R. 20(3) : 167–177. [24] Müller H., N. Michoux, D. Bandon, A. Geissbuhler. 2004. A review of content-based image retrieval systems in medical applications-clinical benefits and future directions. International Journal of Medical Informatics 73(1) : 1-23. [25] Huang J., S.R. Kumar, M. Mitra,W. Zhu, and R. Zabih. 1997. Image indexing using color correlograms. Proc. CVPR 762–768. [26] Karkanis S.A., D. K. Iakovidis, D. E. Maroulis, D. A. Karras, and M. Tzivras. 2003. Computer-Aided Tumor Detection in Endoscopic Video Using Color Wavelet Features. IEEE transactions on information technology in biomedicine 7(3) : 141-152. [27] White D.A., R. Jain. 1996. Algorithms and strategies for similarity retrieval Tech. Rep. VCL-96-101, Visual Computing Laboratory, University of California, San Diego, 9500 Gilman Drive, Mail Code 0407, La Jolla, CA 92093-0407.
