images having similar color but differing in brightness, shade or the angle of illumination. .... axis, one goes from black to white through various shades of gray.
Human color perception in the HSV space and its application in histogram generation for image retrieval
a
b
A.Vadivel , Shamik Sural and A. K. Majumdar
a
a
Department of Computer Science & Engineering b
School of Information Technology
Indian Institute of Technology, Kharagpur 721302 India. {vadi@cc, shamik@sit, akmj@cse}.iitkgp.ernet.in
ABSTRACT
We have done a detailed analysis of the visual properties of the HSV (Hue, Saturation and Intensity Value) color space and Human visual system. Using the results of these analyses, we determine relative importance of hue and intensity based on the saturation of an image pixel with respect to rod and cone cells excitation of retina. We effectively apply this method to the generation of a color histogram and use it for content-based image retrieval applications. A web-based system has been developed for demonstrating our work.
Keywords: HSV color space, Cone and Rod Cells, Human Visual Perception, Color histogram, Soft decision, Content based image retrieval, Precision.
1. INTRODUCTION
Content-based image retrieval (CBIR) is an area of research that encompasses multiple domains like image processing, computer vision, databases and human computer interaction20. In any multimedia application, images play an important role in information retrieval due to their extensive use in various forms including those in web pages, scanned documents and medical databases. QBIC14, VisualSeek19, NeTra12, MARS15, Blobworld2, PicToSeek5 and SIMPLIcity24 are some of the popular CBIR systems developed in the academia and the industry. Digital image databases have seen an enormous growth over the last few years both in domain-specific as well as in generalpurpose applications. The focus of our research is image retrieval from very large databases like the web through simple user-friendly interfaces. Our goal is to make efficient retrieval of a set of images that are similar to a given query image. The term “similarity” is in the sense of visually perceived similarity in content as felt by a user as the color is always considered to be an important low level feature6. Thus, our target application does not need to make a distinction between two images having similar color but differing in brightness, shade or the angle of illumination. After looking into the various approaches to image retrieval, we feel that color histogram based approach is well suited for our application since color matching generates the strongest perception of similarity to the human eye3. Also, it is important to retrieve images efficiently with visually similar colors than a comparatively slow matching of images having an exact color distribution as in a query image. Use of associated text and other application specific features for detailed semantic classification of images can be incorporated in our system, if required.
A standard way of generating a color histogram from an image is to concatenate ‘N’ higher order bits of the Red, Green and Blue values in the RGB space22. It is also possible to generate three separate histograms, one for each channel, and concatenate them into one as proposed by Jain and Vailaya9. In the QBIC system14, each axis of the RGB color space is quantized in a predefined number ‘K’ of levels, giving a color space of K exp (3) cells. After computing the center of each cell in the MTM (Mathematical Transform of Munsell) coordinates, a clustering procedure partitions the space in super-cells. The image histogram represents the normalized count of pixels falling in each super-cell. When performing a query, the input MTM image histogram is matched to the database image histograms. Smith and Chang19 have used a color set approach to extract spatially localized color information. In their method, large single color regions are extracted first, followed by multiple color regions. They utilize binary color sets to represent the color content. Ortega et al15 have used the HS co-ordinates to form a two-dimensional histogram from the HSV color space. The H and S dimensions are divided into N and M bins, respectively, for a total of NxM bins. Each bin contains the percentage of pixels in the image that have corresponding H and S colors for that bin. They use intersection similarity, which captures the amount of overlap between two histograms. Schettini et al17 present a survey of different methods of color image indexing and retrieval from databases. Brunelli and Mich have analyzed a number of histograms for image retrieval1. In contrast to these, we generate a one-dimensional histogram from the HSV space in which a perceptually smooth transition of color is obtained in the feature vector. We have developed a web-based application to show a comparative performance of our proposed histogram and a number of other existing histograms.
Since histogram components store the number of pixels having similar colors, it may be considered to be a signature of the complete image represented by a feature vector. During retrieval, a query image histogram is compared with all the histograms in an image database. As the process of histogram generation having ‘n’ components, converts an image into a point in the n-dimensional vector space, a distance measure like the Euclidean distance can be used for the comparison and ordering of these vectors. Research has also been done to improve the retrieval performance by using other distance metrics like the Manhattan distance, Histogram Intersection and the Earth Mover’s distance16. We have previously explored the possibility of using vector cosine angle for image databases, a metric commonly used in document/text searches21 .
The main contributions of this paper are: (i) A detailed analysis of the human visual perception of color and its relationship with the HSV color space (ii) A soft decision approach to histogram generation using these visual properties and (iii) An extensive comparison with other color histograms for various distance measures using a web-based application. The rest of the paper is organized as follows. Analysis of the human visual perception of color and its relationship with HSV color space is presented in the next section. We then explain the method of histogram generation in section 3. We briefly describe our web-based application in section 4, present experimental results in section 5 and finally draw conclusions from our work in the last section of the paper.
2. HUMAN VISUAL PERCEPTION OF COLOR AND ITS RELATIONSHIP WITH THE HSV COLOR SPACE
Sensing of light from an image in the layers of human retina is a complex process with rod cells contributing to scotopic or dim-light vision and cone cells to photopic or bright-light vision7. Excitation of the cone cells leads to the perception of color while that of rod cells helps in perception of various shades of gray. At low levels of illumination, only the rod cells are excited so that only gray shades are perceived. As the illumination level increases, more and more cone cells are excited, resulting in increased color perception. There are three types of cone cells whose absorption spectra peak at wavelengths corresponding to the colors red, green and blue, respectively. For a given wavelength of light, our perception of which color we are seeing is determined by which combination of cones is excited and by how much. Various color spaces have been introduced to represent and specify colors in a way suitable for storage, processing or transmission of color information in images. Out of these, the HSV model separates out the luminance component (Intensity) of a pixel color from its chrominance components (Hue and Saturation). This representation is more similar to the human perception of color through rod and cone cells. Hue represents pure colors, which are perceived by the excitation of cone cells when incident light is of sufficient illumination and contains a single wavelength. Saturation gives a measure of the degree by which a pure color is diluted by white light. Light containing such multiple wavelengths causes different excitation levels of the cone cells resulting in a loss of color information. For light with low illumination, corresponding intensity value in the HSV color space is also low. Only the rod cells perceive this with little contribution from the cone cells.
An image is a collection of pixels each having a particular combination of Hue, Saturation and Intensity Value in the HSV color space. A three dimensional representation of the space is a
hexacone18, where the central vertical axis represents intensity, I (often denoted by V for Intensity Value). Hue, H, is defined as an angle in the range [0,2π] relative to the red axis with red at angle 0, green at 2π/3, blue at 4π/3 and red again at 2π. For example, human eye perceives a light of wavelength 540 nm as green primarily by the green cone cells, which is represented by Hue = 2π/3 in the HSV model. Saturation, S, is measured as a radial distance from the central axis with value between 0 at the center to 1 at the outer surface. For S=0, as one moves higher along the intensity axis, one goes from black to white through various shades of gray. On the other hand, for a given intensity and hue, if the saturation is changed from 0 to 1, the perceived color changes from a shade of gray to the most pure form of the color represented by its hue. When saturation is near 0, all pixels of an image, even with different hues, look alike and as we increase the saturation towards 1, they tend to get separated out and are visually perceived as the true colors represented by their hues as shown in Figure 1 (a). Low saturation implies presence of large number of spectral components in the incident light. This causes the cone cells to be excited by various degrees causing loss of color information even though the illumination level is sufficiently high. In Figure 1 (b), we show the variation in perceived color with intensity for different values of hue having saturation = 1.0. When intensity is low, only rod cells are excited with little cone cell contribution resulting in achromatic perception. With increasing values of intensity, more and more cone cells get activated and the color perception becomes stronger.
1(a)
1(b)
Figure 1. Variation of color perception with (a) saturation (Decreasing from 1 to 0 right to left) for a fixed value of intensity and (b) intensity (Decreasing from 255 to 0 right to left) for a fixed value of saturation.
From the two figures, we see that for low values of saturation or intensity, we can approximate a pixel color by a gray level while for higher saturation and intensity, the pixel color can be approximated by its hue. For low intensities, even for a high saturation, a pixel color is close to the gray value and vice versa. We, therefore, use the saturation and the intensity values of a pixel to determine whether it can be treated as a true color pixel or a gray color pixel. We would like to emphasize the fact that our approach treats the pixels as a distribution of “colors” in an image where a pixel may be of a “gray color” (i.e., somewhere between black and white, both inclusive) or of a “true color” (i.e., somewhere in the redÆ greenÆ blue Æ red spectrum). The reason is that, for an observer, this is what an image represents – a collection of points having colors – red, yellow, green, blue, black, gray, white, etc. Thus, this representation is a natural way of describing the content of an image.
3. COLOR HISTOGRAM GENERATION USING THE HSV COLOR SPACE PROPERTIES
Based on the above discussions, we note that hard decision thresholds of intensity and saturation can be used to determine the color perception of a pixel. If the saturation and the intensity of the
pixel are above the respective thresholds, we may consider it as a true color pixel; else as a gray color pixel. However, this method does not completely model human perception of colors near the threshold values. This is because there is no definite level of illumination above which the cone cells get excited. Instead, there is a gradual transition from scotopic to photopic vision of human eye. Similarly, there is no fixed threshold for the saturation of cone cells that leads to loss of chromatic information at higher levels of illumination due to color dilution with white light. In order to capture this fuzzy nature of human perception of color, we feel the need for a soft decision23 in determining the dominant property of a pixel that can be used for histogram generation in image retrieval applications. We call this histogram as Human Color Perception Histogram (HCPH).
In a standard histogram generation technique, only one component is updated for each pixel. However, to generate HCPH, we update two components for each image pixel - one true color component and one gray color component. The quantum of update, i.e., weight of each component, is determined by the saturation as well as the intensity of the pixel. For low (high) saturation and low (high) intensity, true color weight is low (high) and the gray color weight is high (low). To incorporate dependency of the weight on saturation as well as intensity, we make true color weight a function wH(S,I) of two variables, S and I that must satisfy the following constraints:
a. wH(S,I) ∈ [0,1] b. For S1 > S2, wH(S1,I) > wH(S2,I) c. For I1 > I2, wH(S,I1) > wH(S,I2) d. wH(S,I) changes slowly with S for high values of I e. wH(S,I) changes sharply with S for low values of I
Properties (a)-(c) are self-explanatory. Properties (d) and (e) follow from the fact that when illumination is high, the loss of color perception is only due to dilution of color by white light. On the other hand, for low intensity, it is a combined effect of cone cell de-activation and color dilution.
Initially we considered various functions satisfying the above constraints. The following function was found to have the best retrieval performance.
0.2(255 / I ) wH (S,I) = S
1.5
0
for I ≠ 0 for I = 0
A plot of the above function is shown in Figure 2.
Figure 2. Variation of true color weight with saturation and intensity.
The gray color weight wI(S,I) is derived as follows:
(1)
wI (S,I) = 1 – wH (S,I)
(2)
Thus, for a given pixel, after determining its H, S and I values, we calculate the true color weight and the gray color weight using Eqs. (1) and (2) to update the histogram. The different true color and gray color components of the histogram are shown in Figure 3. The true color components in the histogram may be considered circularly arranged since hue ranges from 0 to 2π, both the end points being red.
Å
True Color Components Æ
ÅGray Color ComponentsÆ
Figure 3. Representation of colors in the histogram.
We have studied the effectiveness of HCPH in a content-based image retrieval application. We extract color histograms from all the available images and store them in a database after normalizing by the size of the image. During retrieval, color histogram of a query image is extracted and compared with the histograms in the database to determine k- nearest neighbors.
The weight of the intensity component wI(S,I) is derived as follows:
wI (S,I) = 1 – wH (S,I)
(3)
Thus, for a given pixel, after determining its H, S and I values, we calculate the true color weight and the gray color weight using Eqs. (2) and (3) to update the histogram.. The true color components in the histogram may be considered circularly arranged since hue ranges from 0 to 2π, both the end points being red. The histogram, thus, may be represented as a logical combination of two independent vectors.
An image in one of the standard file formats like JPEG is first read and the RGB values of each pixel is determined. The next step is to convert the RGB values into corresponding HSV values. The complete algorithm for generating the HSV color histogram with soft decision may be written as follows:
For each pixel in the image Read the RGB value Convert RGB to Hue (H), Saturation (S) and Intensity Value (I) Determine wH(S,I) and wI(S,I) using Eqs. (2) and (3). Update histogram as follows: HCPH[Round(H.MULT_FCTR)] = HCPH[Round(H.MULT_FCTR)]+ wH(S,I) HCPH[NT +Round(I/DIV_FCTR)] = HCPH[NT + Round(I/DIV_FCTR)]+wI(S,I)
In the above algorithm, NT is the total number of true color components which is given by:
NT = Round (2πMULT_FCTR) +1
(4)
Here MULT_FCTR determines the quantization level for the true colors. We typically choose a value of 8. The number of gray color components is given by:
I
max Ng = +1 DIV_FCTR
(5)
Here Imax is the maximum value of the Intensity, usually 255. DIV_FCTR determines the quantization level for the gray colors. We, typically, choose DIV_FCTR =16. Using MULT_FCTR = 8 and DIV_FCTR = 16, the total number of components in the histogram becomes 64.
In the algorithm, it is seen that one true color and one gray color component of the histogram is updated by the value of the corresponding weights. In table I, we show the color histogram components that will be updated by the “value” given. This exemplifies the algorithm for computing the weight of a particular HSV value. The value of r used is 0.2 and r1 is 1.5. For brevity, in the last column we show the index of the gray color component updated and not its exact position in the complete histogram.
Table I. Histogram components updated for different HSV combinations. Hue Saturation Intensity True Color Update
Gray Color Update
H
S
I
(component, value) (component, value)
0
0
255
0,0
5,1
0
1
255
0,1
5,0
0
0
0
0,0
0,1
0
1
0
0,1
0,0
2π/3
0
200
19,0
4,1
2π/3
1
200
19,1
4,0
2π/3
0.5
200
19,0.92
4,0.08
2π/3
0.5
100
19,0.86
2,0.14
4π/3
0
200
38,0
4,1
4π/3
1
200
38,1
4,0
4π/3
0.5
200
38,0.92
4,0.08
4π/3
0.5
100
38,0.86
2,0.14
2π
0
200
57,0
4,1
2π
1
200
57,1
4,0
2π
0.5
200
57,0.92
4,0.08
2π
0.5
100
57,0.86
2,0.14
We extract color histograms from all the available images and store them in the database after normalizing by the size of the image. During retrieval, color histogram of a query image is extracted and compared with the histograms in the database to determine k- nearest neighbors. A system performing such queries is briefly described in the next section.
4. WEB BASED IMAGE RETRIEVAL SYSTEM
We have developed a web-based application for content-based image retrieval using the proposed histogram as shown in Figure 4 (a). The application is available in the public domain25. In this section we describe some of the important components of this application.
4 (a)
4 (b)
Figure 4. (a) Web based image retrieval system and (b) Result set display in the system.
Query Specification: A query in the application is specified by an example image. The example image is selected either by clicking on any of the displayed images or by uploading an external image. Initially, a random set of 20 images is displayed. A new set of 20 random images may be displayed by refreshing the page on the web browser. The number of images to be retrieved and displayed can be selected as an input parameter from the page. The histogram type and the distance metric can also be chosen for retrieval. The Histogram type is one of the six color histograms including a standard HSV Histogram (HSVSN), a HSV Histogram with hard decision threshold (HSVHD), the RGB histogram (RGB), Jain and Vailaya’s histogram (JV), QBIC histogram (QBIC) and the proposed HSV histogram with soft decision (HSVSD). The standard HSV histogram is generated by choosing 8 bits to represent hue, 4 bits to represent intensity and 2 bits to represent saturation. The HSVHD histogram is generated using the hard decision threshold. The RGB histogram is generated by using two higher order bits from each of the three channels, R, G and B. Through our application we demonstrate the usefulness of our proposed histogram and compare it with the other standard histograms. The reason for choosing RGB is that it is one of the simplest and common methods for generating color histograms. The QBIC histogram was chosen since it is one of the most influential research projects that shaped the initial days of work in the area of contentbased image retrieval. JV has some important properties and was reported to have good retrieval performance. Four distance measures, namely, Euclidean (EU), Manhattan (MH), Vector Cosine Angle Distance (VCAD) and Histogram Intersection (HI) can be chosen for each of the color histograms except JV. For JV, we consider only Histogram Intersection and Euclidean distance as suggested by its proponents9.
Display of Result Set: The nearest neighbor result set is retrieved from the image database based on the query image and is displayed as shown in Figure 4(b). The distance value from the query image is printed below each image. The retrieval process considers the parameter values selected in the options boxes for the number of nearest neighbors, histogram type and distance metric.
Histogram Display: One of the objectives of our research is to study the properties of different color histograms and how they affect nearest neighbor query for a variety of distance metrics. To get an insight into this aspect, we have made a provision for displaying the histograms. The “Histogram” hyper link on the result page displays all the retrieved histograms. On each of these result set histograms, we also display the query image histogram for effective comparison.
External Image Upload: Users are often interested in retrieving images similar to their own query image. To facilitate this, we provide a utility to upload an external image file (currently we support JPEG images) and use the image as a query on the database. Thus, our application is different from all others since we provide a platform for repeating our experiments as well as for running new queries with images provided by the user.
5. RESULTS
A standard way of comparing the results of information retrieval systems is through the use of Recall and Precision measures, which are defined as follows.
Recall
=
No. of Relevant Objects Retrieved Total No. of Relevant Objects
(6)
Precision
=
No. of Relevant Objects Retrieved Total No. of Objects Retrieved
(7)
However, performance comparison of content-based image retrieval systems is a non-trivial task since it is very difficult to find the relevant sets for a large database of general-purpose images. In the absence of ground truth in the form of relevant sets, determining recall as a performance metric is not possible. One way of presenting performance objectively is through the use of precision. Even though we do not exactly know the relevant set, an observer’s perception of relevant images in the retrieved set is what can be used as a measure of precision. Thus, we re-define precision as “perceived precision” (PP) which is the percentage of retrieved images that are perceived as relevant in terms of content by the person running a query. Perceived Precision is defined as:
Perceived Precision =
No. of Retrieved Objects Perceived as Relevant Total No. of Objects Retrieved
(8)
In each of the experiments performed, we have calculated perceived precision for 50 randomly selected images of different content and taken their average. Our database currently contains 28,168 images downloaded from the web and IMSI master clips8. PP is shown for the first 2, 5, 10, 15 and 20 nearest neighbors (NN) of the result set images.
We have compared the performance of the HCPH histogram with a standard HSV histogram (HSVSN) and the HSV histogram with hard decision (HSVHD). We consider all the four distance
metrics for comparison. The results are plotted in Figures 5 (a)-(d). From the plots, it is seen that both HCPH and HSVHD outperform HSVSN in terms of perceived precision. Also, for Histogram Intersection distance, HCPH performance is significantly better than HSVHD. Another interesting observation is that Histogram Intersection distance gives the best average perceived precision for each of the histograms for most of the nearest neighbor points.
Perceived Precision
HSVHD HCPH
0.5
0 2
5
10
15
Perceived Precision
HSVSN
1
HSVSN
1
HSVHD HCPH
0.5
0 2
20
Nearest Neighbors : Euclidean Distance
5
15
20
Nearest Neighbors : Manhattan Distance
5 (b)
HSVSN
1
HSVHD HCPH
Perceived Precision
5 (a)
Perceived Precision
10
HSVSN
1
HSVHD HCPH
0.5
0.5
0 2
5
10
15
20
Nearest Neighbors : Vector Cosine Angle Distance
0 2
5
10
15
Nearest Neighbors : Histogram Intersection Distance
20
5 (c)
5 (d)
Figure 5 . Variation of Perceived Precision with Nearest Neighbor for HSVSN, HSVHD and HCPH using (a) Euclidean Distance, (b) Manhattan Distance, (c) Vector Cosine Angle Distance and (d) Histogram Intersection Distance.
With this observation, we next proceed to compare the performance of HCPH with the histograms generated from the other color spaces. The histograms considered are RGB, QBIC and JV. The performance of these four histograms for different distance measures is shown in Figures 6 (a)-(d). Again, as mentioned in the last section, we consider four distance metrics for all the histograms, except JV, for which we consider two distance metrics. This set of experiments is done to find the best distance metric for each of the histograms. Since the performance of JV is optimized for HI and EU by its authors, we do not repeat experiments with the other distance metrics.
It is observed that in all the figures, Histogram Intersection distance gives the best retrieval result. In Figure 7, we consolidate the results of Figures 6 (a)-(d) and show comparative performance of all the four histograms using this distance measure. It is seen that the proposed histogram has higher perceived precision compared to the other histograms at most of the nearest neighbor points. The performance of the QBIC histogram is comparable with HCPH at two of the nearest neighbor points.
EU
MH
VCAD
HI
0.5
0 2
5
10
15
Perceived Precision
Perceived Precision
1
1
MH
VCAD
HI
0.5
0
20
2
Nearest Neighbors : RGB Histogram
5
10
15
20
Nearest Neighbors : HCPH
6 (b)
EU HI
1
0.5
Perceived Precision
6 (a)
Perceived Precision
EU
1
EU
MH
VCAD
HI
0.5
0 2
5
10
15
Nearest Neighbors : JV
6 (c)
20
0 2
5
10
15
20
Nearest Neighbors : QBIC
6 (d)
Figure 6 . Performance comparison of the distance metrics. Precision variation with nearest neighbor for (a) RGB Histogram (b) HCPH Histogram (c) JV Histogram and (d) QBIC Histogram.
Mean Perceived Precision
RGB QBIC
1
JV HCPH
0.5
0 2
5
10
15
20
Nearest Neighbors
Figure 7. Average perceived precision for the four histograms using histogram intersection.
It should be noted that the results plotted in all the figures represent the average of 50 images used as queries. For a content-based image retrieval system, it is also equally important to have low values of standard deviations for perceived precision. We, therefore, plot the standard deviations of these queries for Histogram Intersection distance metric in Figure 8. HCPH is found to have a low standard deviation while QBIC has a much higher standard deviation value. From the figures 6 and 7, we can infer that while HCPH and QBIC have comparable average performance in terms of perceived precision, due to its lower standard deviation, HCPH has performance close to the average for most of the queries and hence is expected to satisfy most of the users performing queries on general purpose image databases.
SD of Perceived Precision
6
RGB
JV
QBIC
HCPH
4
2
0 2
5
10
15
20
Nearest Neighbors
Figure 8. Standard Deviation of perceived precision for the four histograms using histogram intersection.
We have also compared proposed method with recently proposed color and color texture features shown in table II and found the result is quite encouraging.
Table II. Precision of retrieval for HCPH and other methods.
Features
Local Fourier Transform (LFT)
P (10)
P (20)
P (50)
P (100)
%
%
%
%
27.59
19.76
13.42
9.89
32.36
25.16
16.87
12.29
Quantization (YUV)26 Color Texture Moments (HSV)26
Color Texture Moments (SvcosH, SvsinH,
35.81
26.59
18.24
13.40
34.00
30.00
15.48
12.01
2 Systems combSUM Merge4
40.00
33.00
20.00
17.48
Color-Spatial Feature (36 Colors)11
34.98
29.00
15.99
12.60
Human Color Perception Histogram
46.00
35.90
25.40
19.98
V)26 Multimedia Retrieval Markup Language with Four-Level Relevance Feedback13
(HCPH)
6. CONCLUSIONS
We have shown how the visual properties of an image pixel vary with changes in its hue, saturation and intensity. The result of this analysis has been effectively used to generate a color histogram for content-based retrieval of images. Our approach makes use of saturation to determine the degree by which a pixel appears to have a true color or a gray color. This method tries to capture human visual perception of the color of a pixel and grouping of similar pixels. The relative numbers of true color and gray color components can be tuned depending on the nature of the image database. If the database contains predominantly gray-scale images, we can increase the number of gray color components and reduce the number of true color components. This enables us to differentiate between images having similar gray level distribution and also to classify any given image as either a true color dominant or a gray color dominant image. In the limiting case when we have only gray
level images and no color images, all the components may be made to represent the gray colors. Since every pixel will have a saturation of 0, the histogram generation algorithm will work seamlessly with wH(S,I) always giving a value of zero and wI(S,I), a value of one.
It should be noted that the absolute value of the perceived precision is not remarkably high for any of the histograms for any of the distance measures. The primary reason is that we are running the queries on a large un-categorized database of more than 28,000 images with no semantic guidance and the retrieval is done solely with color information. Running the system on categorized and annotated image databases is expected to increase the precision significantly. Our histogram may be enhanced further by distributing wH(S,I) and wI(S,I) over a number of adjacent components for each pixel. This distribution is possible since there is a perceptually smooth gradation of colors between adjacent components in the histogram. Another possible direction of future research is the use of adaptive binning as suggested by Leow and Rui in which the number of bins is adaptively selected based on the distribution of color in the image11. The distance measure also needs to be selected accordingly.
While it is well established that color itself cannot represent semantic information beyond a certain degree, we have shown that retrieval results can be considerably improved by choosing a better histogram. This color-based approach may be combined with other features like texture and shape, depending on the application. The result of our analysis can be directly used to extract pixel features for clustering in image segmentation problems as well.
We would like to enhance our application by incorporating more number of images with categorization and other color histograms as a continuation of our research work.
ACKNOWLEDGEMENT
The work done by Shamik Sural is supported by research grants from the Department of Science and Technology, India, under Grant No. SR/FTP/ETA-20/2003 and by a grant from IIT Kharagpur under ISIRD scheme No. IIT/SRIC/ISIRD/2002-2003.
REFERENCES
1. R. Brunelli and O. Mich, “Histograms analysis for image retrieval”, Pattern Recognition, 34, 1625-1637, 2001. 2. C. Carson, M. Thomas, S. Belongie, J.M. Hellerstein, and J. Malik, “Blobworld: A system for region-based image indexing and retrieval”, Third Int. Conf. on Visual Information Systems, 217-225, 1999. 3. Y. Deng, B.S. Manjunath, C. Kenney, M.S. Moore and H. Shin, “An efficient color representation for image retrieval”, IEEE Transactions on Image Processing, 10, 140-147, 2001.
4. J. C. French, J. V. S. Watson, X. Jin and W. N. Martin, “Integrating Multiple Multi-Channel CBIR Systems”, Intl. Workshop on Multimedia Information Systems (MIS 2003), 85-95, 2003. 5. T. Gevers and A.W.M. Smeulders, “PicToSeek: Combining color and shape invariant features for image retrieval”, IEEE Transactions on Image Processing, 9, 102-119, 2000. 6. T. Gevers and H. M. G. Stokman, “Robust Histogram Construction from Color Invariants for Object Recognition”, IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 26(1), 113-118, 2004. 7. R. C. Gonzalez and R. E. Woods, “Digital Image Processing “, 2nd Ed. Pearson Education, 2002. 8. International Microcomputer Software, Inc., http://www.imsisoft.com/about/aboutimsi.cfm 9. A.Jain and A.Vailaya, “Image retrieval using color and shape”, Pattern Recognition, 29, 1233-1244, 1996. 10. Z. Lei, L. Fuzong, and Z. Bo, “A CBIR Method Based Color-Spatial Feature”, Proc. IEEE Region 10 Annual International Conference, 166-169, 1999. 11. W. K. Leow and R. Li, "The analysis and applications of adaptive-binning color histograms", Computer Vision and Image Understanding (CVIU), 94, No. 3-1, 67-91, 2004. 12. W. Y. Ma and B. S. Manjunath, "NeTra: a toolbox for navigating large image databases," Multimedia Systems, 7(3), 184--198, May 1999. 13. H. Mueller, W. Mueller, S. Marchand-Maillet, D. Squire and T. Pun, “ A Web-Based Evaluation System for CBIR”, Third Intl. Workshop on Multimedia Information Retrieval (MIR2001), 2001.
14. W. Niblack, R. Barber, W. Equitz, M. Flickner, E. Glasman, D. Petkovic, P. Yanker, C. Faloutsos and G. Taubin, “The QBIC project: Querying images by content using color, texture and shape”, SPIE Int. Soc. Opt. Eng., in Storage and Retrieval for Image and Video Databases, 1908, 173-187, 1993. 15. M. Ortega, Y. Rui, K. Chakrabarti, A. Warshavsky, S. Mehrotra, S. and T.S. Huang, “Supporting ranked Boolean similarity queries in MARS”, IEEE Transactions on Knowledge and Data Engineering, 10, 905-925, 1998. 16. Y. Rubner, L.J. Guibas and C. Tomasi, “The Earth Mover's Distance, multi-dimensional scaling, and color-based image retrieval”, In ARPA IUW, 661-668, 1997. 17. R. Schettini, G. Ciocca, S. Zuffi, “A survey on methods for color image indexing and retrieval in image databases”, in Color Imaging Science: Exploiting Digital Media, eds. R. Luo and L. MacDonald, J. Wiley, 2001. 18. L. Shapiro and G. Stockman, Computer Vision, Prentice Hall, USA, 2001. 19. J.R.Smith and S. -F. Chang, “VisualSeek: A fully automated content based image query system”, ACM Multimedia Conf., Boston, 1996. 20. A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta and R. Jain, “Content based image retrieval at the end of the early years”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 1-32, 2000. 21. S. Sural, G. Qian, and S. Pramanik, “Segmentation and histogram generation using the HSV color space for content based image retrieval”, IEEE Int. Conf. on Image Processing, Rochester, 589-592, 2002. 22. M.Swain and D.Ballard, “Color indexing”, Int. Journal of Computer Vision, 7, 11-32, 1991.
23. A. Vadivel, S. Sural and A. K. Majumdar, “Perceptually smooth histogram generation from the HSV color space for content based image retrieval”, Int. Conf. on Advances in Pattern Recognition (ICAPR2003), Calcutta, India, 248-251, 2003. 24. J.Z. Wang, J. Li and G. Wiederhold, “SIMPLIcity: Semantics-sensitive integrated matching for picture libraries”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 947-963, 2001. 25. Web-based Image Retrieval System, http://www.imagedb.iitkgp.ernet.in/hchp.php and http://www.imagedb.iitkgp.ernet.in. 26. H.Yu, M. Li, H J. Zhang and J. Feng, “Color Texture Moments for Content-Based Image Retrieval”, Proc. Int. Conference on Image Processing, Volume III, 929-931, 2002.
