â«On the other hand, the visual patterns that have properties of homogeneity or not, ... by MPEG-7 [4], namely Color Layout Descriptor (CLD) and Edge. Histogram ...
Ema Rachmawati , Mursil Shadruddin Afkar, Bedy Purnama Telkom University, Bandung, Indonesia
Background Content Based Image Retrieval Feature: Color and Texture Distance function for the combination of feature vector Method Color Layout & Edge Histogram Descriptor Distance function for color feature vector and texture feature vector Late Fusion Method and adaptive weight Experiment: Dataset and Result Conclusion The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
2
PREVIEW Low-level image feature extraction is the basis of content based
image retrieval (CBIR) systems. In that process, the usage of more than one descriptors has tremendous impact on the increasing of system accuracy.
we combined color and texture feature in the feature extraction
process,
namely Color Layout Descriptor (CLD) for color feature extraction and
Edge Histogram Descriptor (EHD) for texture feature extraction.
We measure the system performance on retrieving top-5, top-10, top-
15, and top-20 relevant images. We successfully demonstrated in the experiment, that the combination of color and texture descriptor might be improved the performance of retrieval system, significantly.
In our proposed system, the combination of CLD and EHD reaches
72.82% in accuracy, using adaptive weight in Late Fusion Method
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
3
BACKGROUND In CBIR system, the user usually submits an example image,
and the system will search for the most similar images in the database, then retrieved those similar images to the user query by image
In order to provide the most similar/relevant images to the
query image, extracting proper features from the images should be conducted
Further, a suitable distance function must be defined in the
selected feature space that will measure the similarity between query image and the images in the database.
a proper feature vector is commonly used to represent the
visual content of an image
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
4
FEATURE: COLOR & TEXTURE Color is one of the most widely used visual features in
image and video retrieval
relatively robust to changes in the background colors and is also
independent of image size and orientation
On the other hand, the visual patterns that have properties
of homogeneity or not, that result from the presence of multiple colors or intensities in the image, is usually referred as texture
we proposed a combination of color and texture feature in
the feature representation of CBIR system
We applied color and texture feature representation standardized
by MPEG-7 [4], namely Color Layout Descriptor (CLD) and Edge Histogram Descriptor (EHD)
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
5
COLOR LAYOUT & EDGE HISTOGRAM DESCRIPTOR Color Layout Descriptor was recommended as one of good
color descriptor [6].
Reference [7] used color layout descriptor as a feature
description for high-speed image/video segment retrieval.
The image retrieval system introduced by [8] is based on a
query by layout method using CLD and EHD.
[9] combine CLD with texture descriptor (Gabor filters) to
construct robust feature set in CBIR system.
Edge Histogram Descriptor (EHD) is defined in MPEG-7 for
describing nonhomogeneous texture [5].
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
6
LATE FUSION METHOD CLD and EHD has different distance function, hence a
function to calculate distance between combined feature vectors is needed. we adapted the Late Fusion Method in [10] to calculate final similarity value of using combined color and texture feature. We proposed the use of adaptive weight instead of fixed weight as used in [10].
to give better proportion to the color and texture feature
representation.
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
7
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
8
CLD (Color Layout Descriptor) is the color descriptor in
Content-Based Image Retrieval which extracts spatial color information in image [7]
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
9
EHD (Edge Histogram Descriptor) is the texture
descriptor in Content-Based Image Retrieval which extracts spatial edge distribution in image [11]
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
10
if there is a CLD vector which contains 12 elements and an EHD vector contains 80 elements, the combined vector of CLD and EHD has 92 elements The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
11
Euclidean distance to calculate similarity of color feature vector To calculate similarity of texture feature vector, we use Manhattan distance
j
(YAi -YBi )2+(CbAi -CbBi )2+(CrAi -CrBi )2
D A, B = i=0
79
D A,B =
64
LocAi -LocBi + i=0
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
4
SGAj -SGBj +5× j=0
GloAk -GloBk k=0
12
It takes the normalized value of texture descriptor distance (5) and color
descriptor distance (6) and weighs both of the value to obtain the distance (7)
𝑇 is a texture distance between query image and database image, 𝐶 is a
color distance between query image and database image, 𝑛𝑇 is a normalized value of texture distance between query image and database image, 𝑛𝐶 is a normalized value of color distance between query image and database image, and 𝑤 is the weight for color distance. The weight for texture distance is set so that the summation of weight of color and texture distance always 1.
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
13
DATASET Wang dataset [12], which contains 1000 images, categorized
into 10 classes.
The classes in database are Africa, beach, monument, bus,
dinosaur, elephant, flower, horse, mountain, and food. I
mages in the dataset had either portrait or landscape layout. Portrait images had dimension 256 x 384 pixels and the landscape ones had dimension 384 x 256 pixels The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
14
SYSTEM ACCURACY OF USING FEATURE VECTOR
w = 0.5 15 The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
SYSTEM ACCURACY ON RETRIEVING TOP-5, TOP-10, TOP-15, AND TOP-20 IMAGES
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
16
SYSTEM ACCURACY OF USING COMBINED CLD AND EHD ON DIFFERENT WEIGHTS feature vector: CLD18 and EHD150
The adaptive weight would change according to characteristics of retrieval result. -if the accuracy result on retrieving images using CLD is larger than using EHD, the value of w become larger -if the accuracy result on retrieving images using EHD is larger than using CLD, the value of w become smaller.
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
17
we proposed the combination of color and texture feature in a
content based image retrieval system and demonstrated how feature vectors from Color Layout Descriptor (CLD) and Edge Histogram Descriptor (EHD) are combined.
we adapted the Late Fusion Method in the similarity calculation of
using combined color and texture feature.
the use of adaptive weight instead of fixed weight in the feature
representation and proved that it has a significant impact on increasing system accuracy.
The best accuracy achieved in the system we build is 72.82 % in
using the combination of cld18 and ehd150.
CBIR using combination more than one descriptor can increase the
performance of the system rather than using only one descriptor.
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
18
[1] R. Datta, D. Joshi, J. Li, and J. Z. Wang, “Image retrieval: Ideas, Influences, and Trends of the New Age,” ACM Comput. Surv., vol. 40, no. 2, pp. 1–60, 2008. [2] O. A. B. B. Penatti, E. Valle, and R. da S. Torres, “Comparative study of global color and texture descriptors for web image retrieval,” J.Vis. Commun. Image Represent., vol. 23, no. 2, pp. 359–380, Feb. 2012. [3] R. S. Choraś, T. Andrysiak, and M. Choraś, “Integrated color, texture and shape information for content-based image retrieval,” Pattern Anal.Appl., vol. 10, no. 4, pp. 333–343, 2007. [4] S. F. Chang, T. Sikora, and A. Puri, “Overview of the MPEG-7 standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 11, no. 6, pp. 688–695, Jun. 2001. [5] T. Sikora, “The MPEG-7 visual standard for content description-an overview,” IEEE Trans. Circuits Syst. Video Technol., vol. 11, no. 6, pp. 696–702, 2001. [6]
H. Eidenberger,“How good are the visual MPEG-7 features?,” SPIE Vis. Commun. Image Process., pp. 476–488, 2003.
[7] E. Kasutani and A. Yamada, “The MPEG-7 color layout descriptor: a compact image feature description for highspeed image/video segment retrieval,” in International Conference on Image Processing, 2001, vol. 1, pp. 674–677. [8] S. M. Kim, S. J. Park, and C. S. Won, “Image Retrieval via Query-by-Layout Using MPEG-7 Visual Descriptors,” ETRI J., vol. 29, no. 2, pp. 246–248, 2007. [9] H. a. Jalab, “Image retrieval system based on color layout descriptor and Gabor filters,” in 2011 IEEE Conference on Open Systems, 2011, pp. 32–36.
[10] M. Bleschke, R. Madonski, and R. Rudnicki, “Image retrieval system based on combined MPEG-7 texture and Colour Descriptors,” 2009 Mix. Int. Conf. Mix. Des. Integr. Circuits Syst., pp. 0–4, 2009. [11] 30, 2002.
C. S. Won, D. K. Park, and S. J. Park, “Efficient use of MPEG-7 edge histogram descriptor,” ETRI J., vol. 24, no. 1, pp. 23–
[12] J. Li and J. Z. Wang, “Automatic linguistic indexing of pictures by a statistical modeling approach,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 9, pp. 1075–1088, 2003.
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
19
The Second International Conference on Soft Computing and Data Mining (SCDM) August 18 - 20, 2016
20
