Entropy-Balanced Bitmap Tree for Shape-Based ... - Semantic Scholar

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 5, MAY 2011

1603

Entropy-Balanced Bitmap Tree for Shape-Based Object Retrieval From Large-Scale Satellite Imagery Databases Grant J. Scott, Member, IEEE, Matthew N. Klaric, Student Member, IEEE, Curt H. Davis, Fellow, IEEE, and Chi-Ren Shyu, Senior Member, IEEE

Abstract—In this paper, we present a novel indexing structure that was developed to efficiently and accurately perform content-based shape retrieval of objects from a large-scale satellite imagery database. Our geospatial information retrieval and indexing system, GeoIRIS, contains 45 GB of high-resolution satellite imagery. Objects of multiple scales are automatically extracted from satellite imagery and then encoded into a bitmap shape representation. This shape encoding compresses the total size of the shape descriptors to approximately 0.34% of the imagery database size. We have developed the entropy-balanced bitmap (EBB) tree, which exploits the probabilistic nature of bit values in automatically derived shape classes. The efficiency of the shape representation coupled with the EBB tree allows us to index approximately 1.3 million objects for fast content-based retrieval of objects by shape. Index Terms—Content-based retrieval, image databases, knowledge-based indexing, object indexing, remote sensing.

I. I NTRODUCTION

A

S THE volume of remote-sensing earth imagery continues to increase, automated processes must be developed and refined, which can eliminate the requirement of a human-inthe-loop for creating large-scale searchable image repositories. Content-based image retrieval (CBIR) is an increasingly popular retrieval method for large-scale image databases. CBIR queries are not performed in a traditional relational database management system (RDBMS) of image metadata, e.g., sensor, location, or time, but instead use features extracted from image content to search. Traditionally, descriptive features are extracted to represent various discriminating properties of the image content. These features may represent global properties, e.g., color and texture, or collective localized features, e.g.,

Manuscript received July 7, 2009; revised November 30, 2009 and May 18, 2010; accepted August 22, 2010. Date of publication December 17, 2010; date of current version April 22, 2011. This work was supported in part by the National Geospatial-Intelligence Agency University Research Initiatives (NURI) under Grant HM1582-04-1-2028 and by the U.S. National Science Foundation under Grant IIS-0812515. G. J. Scott, M. N. Klaric, and C. H. Davis are with the Center for Geospatial Intelligence, University of Missouri, Columbia, MO 65211-0001 USA. C.-R. Shyu is with the Informatics Institute, University of Missouri, Columbia, MO 65211 USA. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TGRS.2010.2088404

the shape and color of segmented objects or the texture of partitioned image regions. Numerous CBIR systems have been reported in the literature, e.g., Query by Image Content (QBIC) [1], VisualSeek [2], Photobook [3], and PicToSeek [4]. In [5], Gevers and Smeulders offer a comprehensive overview of CBIR. In [6], Lew et al. provide a review of the state of the art in CBIR. In the remote-sensing domain, there are relevant contributions that focus on content-based retrieval and, oftentimes, image information mining. A notable contribution that has explored content-based retrieval of satellite imagery is the knowledge-based information mining (KIM) system by Datcu et al. [7]. With regard to CBIR, KIM exploits Landsat Thematic Mapper (TM), as reported in [8]. Li and Narayanan, in [9], used Land Cover and Land Use thematic maps as supervised training of support vector machines over the spectral information of an image. They also exploit Gabor wavelets for textural feature extraction to capture spatial information from an image. A necessity for developing a successful CBIR system is the extraction of discriminant features to describe the images in the database. As such, the development of feature extraction algorithms has dominated the literature in the field. These fundamental features are often assembled to model higher level human visual perception for CBIR, where the ultimate goal is to retrieve visually similar images. In addition, there exists a solid foundation of literature on the extraction and modeling of spatial components of imagery, e.g., objects or natural divisions (e.g., the horizon of a landscape photo or a foreground person). Shape analysis and retrieval have emerged as particularly important topics in CBIR, because visual knowledge is often related to shape characteristics of objects. In this paper, we are primarily concerned with object shape retrieval from large-scale remote-sensing imagery databases. Hence, we will focus on a shape feature set generated using objects that were automatically extracted from a large collection of satellite imagery. Two promising approaches for automatic shape extraction from large-scale satellite imagery databases include techniques based on transforms (e.g., Fourier [10] or wavelet [11]) and morphology [12]. For the reported research herein, we employ the latter approach, as described in Section II. There exists a plethora of research with regard to object shape feature extraction. Traditionally, shapes are conceptualized in the literature using a few broad categories as follows:

0196-2892/$26.00 © 2010 IEEE

1604

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 5, MAY 2011

1) contours; 2) regions; or 3) skeletons. Contour representations of shapes are typically outlines found with edge detectors and other similar image processing techniques. Recently, scalespace methods [13] have become very popular as shape descriptors. Using this approach, an object contour is continually smoothed by increasing Gaussian filters, building a hierarchy of salient inflection points [14]. In [15], Avrithis et al. utilize Fourier transforms and curve moments to build invariant curve representations for further feature extraction. Kunttu et al. present a method for encoding both color and shape information using intensity Fourier [10]. Other methods have been developed for nonrigid shapes, e.g., multiscale convexity concavity by Adamek and O’Connor [16]. Comparative literature, such as [17], provides a retrieval performance review of various contour-based descriptors using a standard data set, including curvature scale space, wavelet encoded contours, and visual correspondence. Other comparative literature includes Zhang and Lu’s [18] extensive review of Fourier, scale space, Zernike moments, and grid descriptors. Object skeleton-based features are somewhat rare in the literature. Skeletons can be derived using morphological image processing techniques or variations such as medial axis [19] or shock graphs [20]. In the CBIR of skeleton-encoded shape features, searching is equivalent to graph matching or computing transformation steps to achieve the second graph from the first graph. The complexity of ranking skeletons with these methods limits the efficiency of retrieval performance, because the similarity must be computed between the query and numerous candidates. Despite the computational cost, skeletal methods have been shown to particularly be robust with regard to object occlusion. Other shape descriptors include edge histograms combined with Fourier transforms as in [21], which exploit statistical information of the shape. Minimum bounding circle [22] and convex hull approaches rely on finding a circular or convex region to encompass the shape prior to feature extraction. The feature extraction then processes the object shape by also examining the regions not populated by the intersection of shape and bounding object. Along with the breadth of object shape feature spaces, there exists a healthy quantity of the literature focused on measuring similarity of shapes in the aforementioned feature spaces. Popular approaches depend, to a degree, on the feature space. In scale-space methods, the predominant methods involve finding inflection point correspondence between objects. Some approaches measure the similarity through deformation/ transformation steps to achieve the second shape from the first (e.g., [23]), or the second skeleton from the first [20]. Other approaches combine local and global invariants for computing similarity [24]. Utilizing local invariants is key to maintaining adequate retrieval of objects that are subject to occlusion. In CBIR, it is desirable to provide the results of the query in a similarity-ordered set. For this reason, CBIR is often cast as a problem of finding nearest neighbors in the feature space defined by the chosen object descriptor. Although there exist methods such as pruning to eliminate segments of the database, the most efficient approaches use indexing schemes to access the feature space [25]. Various indexing schemes have

been reported in the literature, including the containment tree for topological image structure [26], EBS k-D tree for highdimensional feature spaces [27], and sparse distributed memory structures for properties generated from principal component analysis (PCA) in [28]. In [29], Liu et al. construct a separate 1-D index for each feature in the feature set. Through their search algorithm, this method has the benefit of quickly returning empty sets if no objects are within the desired similarity radius. However, the algorithm generates a candidate point set in each dimension, requiring results to be merged into a final result set. In high-dimensional feature spaces, the number of candidate point sets may inhibit performance. In addition, dense feature spaces will compound this problem, because the candidate point lists will substantially increase. Note that the CBIR literature rarely has feature descriptors tightly coupled with indexing structures. To create a truly scalable system for CBIR, one should substantially increase the database size without equivalent retrieval performance decreases. This paper directly addresses this problem. We have designed an indexing structure, the entropy-balanced bitmap (EBB) tree, which is particularly suited to our chosen shape descriptor. Existing RDBMS indexing mechanisms are not suitable for our shape encoding data. In addition, common space/data partitioning indexing extensions for RDBMS are ill suited for this high-dimensional data. We explored the suitability of metric index approaches and found them inadequate for our data collection. By using a shape descriptor that provides a small fixed encoding size and developing a tightly coupled indexing and retrieval structure, we have developed a scalable approach for content-based retrieval of objects using shape. Our object shape database consists of 1.3 million objects, yet we can return thousands of the most similar ranked shapes in a few seconds. The remainder of this paper is organized as follows. In Section II, we explain our automatic object extraction, shape encoding, and data clustering as the index preprocessing steps. Section III describes the theoretical basis of the EBB tree, along with relevant algorithm details. Our experimental methods and results are detailed in Section IV. We conclude with discussion in Section V. II. O BJECT E XTRACTION AND P REPROCESSING We have developed an extensive geospatial imagery retrieval system, GeoIRIS [30], which employs numerous retrieval techniques. Currently, our image database contains 45 GB of highresolution orthorectified, georeferenced commercial satellite imagery. This imagery is five banded—0.6–1.0 m panchromatic and 2.4–4.0 m multispectral—with each band having an 11-b effective range. One of the latest extensions to GeoIRIS employs a scale-invariant shape descriptor to retrieve objects from the database. This paper is focused on retrieval applications with geospatial awareness; as such, a typical interaction for our system might be to submit an object as the query, along with geospatial constraints. For example, “Given a query image containing a baseball diamond, find all similar baseball diamonds in the database that are within 2 km of a radio broadcast tower.” With this goal in mind, we must efficiently

SCOTT et al.: ENTROPY-BALANCED BITMAP TREE FOR SHAPE-BASED OBJECT RETRIEVAL

1605

perform object-based retrievals from our database as the first step to incorporate other geospatial knowledge. The extraction and encoding of object shapes is described in the following sections. Note that we also store object spectral information and principal axis length for use in complex object queries. A. Multiscale Object Extraction and Shape Representation in Bitmaps Our automatic object extraction algorithms for highresolution satellite imagery [31] exploit the differential morphological profile (DMP) [12] to facilitate the processing of large quantities of imagery and efficiently discover objects. One of the current challenges in any automatic object indexing process is to extract the relevant objects from the imagery. In small image databases, edge detectors and segmentation are viable object location strategies. Our database of high-resolution satellite imagery contains numerous large scenes, with a total coverage of 3994 km2 . For a collection of satellite imagery of substantial scale, traditional object extraction methods are inefficient. Manual extraction of objects from an imagery collection of this scale is infeasible; as such, automated processes are necessary. To accomplish this difficult task, we process the scenes using the DMP on the panchromatic channel of the imagery. The DMP is a multiscale segmentation algorithm, which exploits contrast edges in imagery. Using geodesic morphology by reconstruction, objects that are lighter or darker than their surrounding image content generate response in the DMP. The intensity of the DMP is correlated to the difference in the contrast of the object and its surrounding. The resulting extractions are homogenous regions, each representing an object. The interested reader should refer to [12] for a detailed presentation of the DMP. The DMP produces a set of scaled contrast responses, referred to as DMP levels. Level m represents the possible objects detected using a geodesic disk of size rm , which were not detected with radius rm−1 . Each level in the DMP represents objects extracted after the transition from one geodesic scale to the next. During the processing of the DMP, we utilized a normalized difference vegetation index (NDVI) to filter out nonanthropogenic objects. Because the resulting objects are anthropogenic structures extracted from imagery using DMP responses, we use regionbased (solid) objects instead of applying additional processing to generate contours or skeletons. Therefore, we focused our approach on the region-based subset of all available shape descriptors. Seeking the most efficient method to represent region-based shapes and still have adequate descriptive power to identify general shapes, we chose grid descriptors [32]. Grid descriptors are effectively a sampling of an object shape into a matrix of fixed size. These grids provide natural scale invariance. In addition, immediately prior to sampling the object into the grid, we align the principal axis of the extracted object to the middle horizontal grid axis. Given a fixed-size grid that represents a shape, it is natural to represent the grid as a simple bitmap. In GeoIRIS, we used 322 b, representing 1024-D bitmap space, for an encoding size of 128 B per shape. Early empirical analysis revealed that this size of bitmap

Fig. 1. Grid descriptor of extracted objects. The object exists in the original image and can be extracted using the DMP. (a) and (b) Two regions of original imagery. (c) and (d) DMP-extracted objects from (a) and (b), respectively. (e) and (f) Encoding of the lighter extracted objects each of (c) and (d), the ones represent bits set on.

provided good balance of retrieval performance and shape discrimination. Fig. 1 shows two example chips from our database imagery, followed by a relevant level of the DMP, and finally the resulting bitmap encoded shape. This object-encoding scheme has some substantial benefits that we exploit. First, for our large-scale image database with 1.3 million extracted objects, all objects are encoded in less than 160 MB or 0.34% of the original 45 GB. Therefore, continuing to scale our database will not be limited by an increase in the number of encoded shapes. Second, an indexing scheme has been developed, which allows efficient ranked retrievals and exploits bit operations, instead of floating-point operations, during search and ranking. This indexing structure is developed in Section III, including algorithms for induction and search. Finally, revisiting Fig. 1(e) and (f), we can see that the encoding is very intuitive. In our GeoIRIS database, we have a mixture of scenes from urban, suburban, and rural areas of the world. On the average, we have 325 encoded objects per square kilometer. Because the balance of land cover type varies in the imagery, the number of extracted objects will necessarily vary. For

1606

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 5, MAY 2011

example, as the proportion of urban versus rural area increases, the number of objects can be expected to increase per square kilometer. This condition will have an effect on the portion of the original database size that is needed for object shape representation.

niques, i.e., the concept of inverse document frequency [33]. We defined the inverse bit frequency, ibf , of bit[k] to be

B. Bitmap Dissimilarity Measure

However, ibf alone proved insufficient for some key objects in our satellite imagery domain. This measure of bit relevance drove the weights near the center horizontal axis bits to effectively zero. Our final bit-weighting scheme is to use the following combination of (3) and (4):

One important factor of the indexing and retrieval scheme is the choice of an appropriate dissimilarity metric. We utilize one dissimilarity metric for initially clustering data, ranking the bitmap results, and building the priority queues for leaf traversal. We experimented with numerous dissimilarity measures, which rely on measuring the number of bits that varies between two bitmaps. We have d(B1 , B2 ) = |B1

XOR

B2 |

(1)

with XOR representing the bitwise exclusive OR operation. In (1) and the following discussions, we borrow from a mathematical set notation and use |B| to represent the count of bits on in a bitmap for equations (as well as the more general count of elements in a set) and refer to this value as the cardinality of a bitmap in the text. When measuring the dissimilarity of two shaperepresentation bitmaps, not all bits need to be treated equal. As expected, various bits in different objects can have significantly different relevance to the object shape. We therefore evaluated weighted bit dissimilarities using dwt (B1 , B2 ) =

K 

(B1 [k]

XOR

B2 [k]) ∗ wt[k]

(2)

k=1

where wt[k] is the weight assigned to bit k. In these dissimilarity measures, each bit that differs contributes to the dissimilarity by the amount of its weight. In (2), the significant step is assigning weights that accentuate shape differences. As described in Section II, all objects are aligned to the x-axis and centered at the y-dimension of the bitmap. This alignment, coupled with the scaling of all objects into a fixed bitmap size, implies that every object will horizontally span the center of the bitmap. Therefore, the bits along this center axis contribute less to the shape information than the top and bottom regions. With this condition in mind, we assume that bits farther from the center horizontal are more important in describing the object shape when they are set on. Our initial experimental bit-weighting approach used the square root of the absolute y-of f set from the center x-axis as  (3) y_off[k] = y_offset. Because we use an even number of bits for the edges, the two rows that form the center horizontal axis are counted as an offset one. Therefore, with our chosen bitmap size of 32 × 32, (3) has a range of [1, 4]. Empirical analysis revealed that bit-weighting schemes based on the y-of f set alone were not sufficient. Therefore, we borrowed ideas from established document retrieval tech-

 ibf [k] = log

DB Population Size DB objects with bit[k] = 1

wt[k] = max {ibf [k], y_off[k]} .

 .

(4)

(5)

During our analysis, to determine an appropriate bit weighting for dissimilarity measures, we examined the effects on different types of shapes. Fig. 6 has a sample of the variety of shapes that may be found in our database. Airplanes are representative of complex shapes, L-shaped buildings are representative of shapes with a few significant concavities, and baseball fields are representative of shapes that combine linear and curvature to form shapes that are difficult to distinguish from purely curve-based shapes without bit weighting.

C. Clustering Multiscale Objects in Bitmap Space In CBIR applications, it is generally expected that similar objects are close together in the feature space. In addition, we do not expect the feature space to uniformly be saturated with objects. Instead, we expect that similar database objects will tend to form high-dimensional clouds in the feature space. In effect, this case is a necessary requirement of successful feature extraction algorithms, low intraclass variance, and high interclass spread. With this requirement in mind, it is beneficial to apply clustering algorithms to automatically discover and label these dense regions. Clustering techniques are well established in pattern recognition and data mining. The complexity of clustering algorithms is heavily influenced by the size of the database, both in the dimensionality and number of objects. In-depth discussion of feature extraction philosophies and clustering techniques can be found in [34] and [35]. Once clusters are discovered in the database, the statistical properties of these clusters can be exploited to create efficient and accurate indexing of the feature space. The EBB induction algorithms rely on these clusters. To prepare our database for indexing, we adapted the densitybased spatial clustering of applications with noise (DBSCAN) [36] clustering algorithm for use with a large collection of bitmaps. We developed a sampling-based clustering approach that typically uses two passes through the database. This approach makes it particularly attractive for large-scale feature sets, e.g., the objects extracted from our satellite imagery. To generate clusters, we measure dissimilarity between any two bitmaps using (2). After clustering, each object belongs to a cluster of objects that are similar in bitmap space.

SCOTT et al.: ENTROPY-BALANCED BITMAP TREE FOR SHAPE-BASED OBJECT RETRIEVAL

1607

Fig. 2. Example EBB tree. Circular nodes are decisions in the search path, determined by maximizing (6), and square nodes are the leaves that contain the bitmap population, which exists in the nodes at the various stages of induction, concluding with the leaves. The leaf nodes are labeled with their class population for clarity.

III. EBB I NDEXING Bitmap indexing has many uses in retrieval and databases. In traditional RDBMS, bitmap indexes are utilized to partition relations into a relatively small number of disjoint sets using single attributes (e.g., gender). Another common usage is the bitmap index for term–document correlation in information retrieval (IR) schemes [33]. In IR, the bitmaps are typically documents of a collection, and individual bits represent the presence of a term in the document. In our current context, we are dealing with bitmaps that represent object shapes as a binary grid, using 32 × 32 b. If we attempt to grow a full bitmap index that covers all of the bitmap space, we would need 21024 − 1 internal nodes to accommodate 21024 leaves. What is required is a significantly smaller index. In the current discussion, we are dealing with a large collection of bitmaps, clustered in sets of naturally occurring groups of similar bitmaps. Our approach, the EBB tree, exploits these groupings found in the bitmap space to efficiently index the object shapes with a much smaller tree than would be necessary to cover the entire space. For retrieval efficiency, these clusters are further divided into a large number of leaves that contain a small group of very similar bitmaps. Furthermore, to accommodate large results sets, the leaves are linked together in priority queues for leaf navigation. For GeoIRIS, our bitmap index has 27 005 leaves, with an average leaf population of 47.49 and an average search depth of 14.72. A. EBB Tree Induction In previous work, the entropy-balanced statistical k-D tree [27], [37] was used to exploit knowledge about classes or groupings in a feature space when indexing continuous mul-

tidimensional feature sets. The motivation is to increase retrieval precision by lowering the entropy while simultaneously reducing the imbalance of the tree. Using statistical analysis of clustered or ground-truth labeled data, we exploit the statistical properties of clusters to induce an entropy-balanced tree that decreases the entropy from parent to child nodes. Statistical entropy, as defined by Shannon, is a measure of the randomness or variability of data [38]. Therefore, induction should seek to minimize leaf entropy, ensuring that leaf contents have a high degree of similarity. One desirable trait is to not greedily sacrifice the entropy of one node to lower the entropy of its sibling. The result is an efficient indexing structure, where searches reach leaves of low entropy, implying more certainty that the leaf contents are similar to the query. Fig. 2 illustrates the general concepts of EBB induction. Note that the bitmaps represented are 16 b in size, and in the following discussion, the bit positions start at zero in the top left and count across the bitmap rows. One traditional bitmap that covers this bitmap space would require 65 536 leaf nodes and 65 535 internal nodes, a total tree size of 217 − 1. At the root level, there exists a large collection of bitmaps organized into five classes, represented by the five grids. If the induction algorithm determines the decision bit as k = 12, then the three bitmaps with that bit on will be in the right child (A,C,D). The two bitmaps with that bit of f (B,E) will be pushed into the left child. At the second level, the initial left child may determine the best decision bit as k = 8, thereby splitting the two classes into the left (E) and right (B) child nodes. The root node’s right child determines the next split to be k = 1, creating a single class (D) leaf as its right child and a two-class node as its left child (A,C). Finally, the internal node at the third level will use bit k = 0 to separate its two classes into the final leaves. This

1608

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 5, MAY 2011

resulting EBB tree will allow navigation to the various classes using two to three bit comparisons. If we used a greedy maximum entropy reduction approach, without the balancing effect, we would have a tendency to make decisions that position leaves at higher levels in the tree. For example, again considering the root node in Fig. 2, a greedy decision may use bit k = 0 as the root node decision, creating a class-A leaf at level 1. This step would be followed by a series of similar decisions, resulting in a tree with leaves at levels 1, 2, 3, and two leaves at level 4. The resulting searches would take from one to four comparisons. Projecting this behavior out to much larger databases, with larger bitmaps, more classes, and less crisp class-bit probabilities accentuates the variability in search efficiencies. Another issue may arise when examining why a greedy decision will split off leaves higher in the tree, i.e., it sacrifices the entropy of one subtree for the gain in the other subtree. Another downside to this type of split decision is the effect of creating numerous low-probability high-entropy leaves. These leaves would be sufficient in a classifier, but in an indexing system that expects retrievals to require leaf traversals, this approach can result in traversing through numerous highentropy leaves. Fig. 2 represents a simplification of the data for illustration, where the classes have only 0 and 1 as bit probabilities. Actual data are significantly more complex, including more classes, more bits, and bit probabilities in classes that range between 0.0 and 1.0. The a priori class-based bit probabilities are a key part of the exploitation of database knowledge during index induction. In any particular leaf, during the induction of an index, we must estimate the probability of some Classi within the portion of the bitmap space that the leaf occupies. These classes represent our discovered clusters of bitmaps. The clustered bitmaps could be represented as prototype vectors of floating-point numbers, but in reality, statistics developed from these vectors have little significance in the binary space. Therefore, we use the probabilities of bits being on or of f for a given Classi in some bitmap space covered by some Leafj . We approximate this value by examining the members of each Classi in the leaf and tracking the occurrence of on and of f bits for each bit position. The use of these approximations are discussed as follows in the context of computing the conditional probability of Leafj , given Classi . The EBB tree is designed for very large collections of bitmaps, which lend themselves to exploiting the probabilistic tendencies of data. One critical design issue is the development of the split decision objective function, which can properly exploit these class-based probabilities. Our desire is to induce an index with a collection of low-entropy leaves. The result is then a collection of leaves of objects in the bitmap space, where each leaf represents a group of similar bitmaps. Therefore, we desire a decision criterion that allows the recursive induction algorithm to balance the reduction of entropy between each set of sibling subtrees whenever a split decision is made. This condition ensures that the entropy of one subtree is not sacrificed for the sake of the other subtree. The decision criterion is the bit k that maximizes γ = Hparent − σHR − σHL − ABS(σHR − σHL ).

(6)

Hparent is the entropy of the parent node, and σHR and σHL are the weighted sum components of the right and left children, respectively, where σH = P (Leafj )H(Leafj ). The first three terms on the right-hand side of (6) represent the reduction of entropy. The terms in the absolute value represent the balancing factor. We also constrain the node splitting to require at least a minimal entropy reduction from parent to children, such as a percentile decrease. We calculate the entropy of any Leafj as H(Leafj ) = −

L 

P (Classi |Leafj ) log P (Classi |Leafj )

i=1

(7) where L is the number of classes that exist in Leafj . To calculate the entropy of a leaf in a high-dimensional bitmap space, we define basic probabilities over a bitmap database as a foundation. Given a database D composed of a set of disjoint classes, we define the a priori probability of Classi as P (Classi ) =

|Classi | |D|

(8)

where |Classi | and |D| are the size of Classi and the database, respectively. To capture the aforementioned point-based probabilities, we calculate the a priori probabilities of each bit k in each class i as # off bits |Classi | # on bits P (Classi,k=1 ) = |Classi | P (Classi,k=0 ) =

(9) (10)

which represent the probability of Classi with bit k set off or on, respectively. To maintain an approximation of the class bit probabilities during tree induction, we calculate the probability of Classi ’s bit k in N odej using ⎧ if on & of f bits ⎨ 1.0 P (Classi,k,j ) = P (Classi,k=0 ) if of f bits (11) ⎩ P (Classi,k=1 ) if on bits. Equation (11) examines Classi ’s bit variations in some particular Leafj . If the class has bitmaps with k both of f and on, then the probability of reaching Leafj by a search with a Classi bitmap is approximated as 1.0. If Classi has only of f or on bits, this approximation is taken from (9) or (10), respectively. The probability of Leafj , given Classi in a bitmap of size K, is calculated as P (Leafj |Classi ) =

K

P (Classi,k,j ).

(12)

k=1

This approach allows us to calculate the probability of Classi , given Leafj , using Bayes’ theorem as P (Classi |Leafj ) =

P (Leafj |Classi )P (Classi ) P (Leafj )

(13)

where the probability of Leafj is the number of database objects in Leafj over the size of the database. Using (13), we can

SCOTT et al.: ENTROPY-BALANCED BITMAP TREE FOR SHAPE-BASED OBJECT RETRIEVAL

1609

Fig. 3. Two split decisions based on a three class example, with the result tree shown on the right. The grids shown are populated with the probability of on bits in each class, as defined by (10). Bit numbers start at 0 and fill rows to the right, ending at 8 in the bottom right corner. L and R are the left and right children of a possible split, respectively.

calculate the entropy (7) and thereby make the desired induction decisions to maximize (6). The EBB has a notable property related to the maximum height of the index. In particular, the maximum search depth of any path to an EBB leaf is K + 1, where K is the size of the bitmaps. As can be observed from Algorithm 1, a specific bit k can only be used a single time in one path. This condition limits the number of decision nodes to K and, therefore, the maximum search depth. The EBB tree is built with a recursive decision tree induction algorithm as detailed in Algorithm 1, SplitNode. Initially, the entire database D is evaluated as a root node R, and a decision bitmap is created, dcsn, with all bits set of f . SplitNode is then called with the root node and the blank decision bitmap. For each of f bit in the dcsn bitmap, divide the current node into two candidate child nodes. Each database object with the current bit off is assigned to left child; if the bit is set on, objects are assigned to the right child. Then, the split objective function (6) is evaluated, possibly updating the current best decision γmax and setting the decision bit kmax . For a node to split, the entropy must be reduced from the parent to its children by some threshold . When a split bit is determined, that bit is set on in dscn. After storing the kmax in the current node as the decision bit, SplitNode is called for both the left and right children, each using the new dcsn bitmap. The dcsn bitmap is passed to the recursive calls of the children to allow the induction to accelerate, because those bits must never be evaluated again. If some N odej uses bit k as the decision bit, then all bitmaps in the left subtree will have the bit k off, and all bitmaps in the right subtree will have the bit set on. Therefore, the maximum height of the EBB is the number of bits in the bitmap, as in the case in traditional bitmap index schemes. Fig. 3 provides a three-class example to illustrate the behavior of Algorithm 1 for making split decisions. On the left side are class representations of the probabilities of bit k being on, i.e., (10). In the middle table are two decision levels, the root node, Decision 1, and the root node’s right child as Decision 2. The bit index is the first column, followed by the probability and entropy of each child for a possible split at that bit. Finally, the decision value of (6) is shown in the last table column. The last part in Fig. 3 is the resulting tree produced from the table. For the computations in Fig. 3, P (Class1 ) = P (Class2 ) = P (Class3 ) = 0.333. Note that bits 0, 2, 6, and 8 never need evaluation, because the entire population has those bits set off. The root decision has four bits that need to be evaluated: 1) bit 1; 2) bit 4; 3) bit 5; and 4) bit 7. Bits 5 and 7 are equivalent but have different permutations of the class-to-child distribution

generated by bit 1. Bit 1 effectively partitions off class 3 into the left child, leaving classes 1 and 2 in the right. Bit 4 partitions class 1 between the left and right nodes, raising the entropy of the right node. The root node’s right child decision bits that must be evaluated are 4 and 5. Bit 4 will split class 1 between the left and right children, increasing the entropy of the right, because it will also have all of class 2. Bit 5, as a decision bit, partitions the node into class-homogenous leaves, which is the optimal split. Algorithm 1. SplitNode(N,dcsn): Recursive node-splitting algorithm for inducing the EBB tree. Parameters include the node N and previous decisions dcsn. 1: Calculate and store (7) for N ; 2: Initialize decision parameter γmax ; 3: for all bit k such that: (k AND dcsn) = k do 4: Partition N into lef tN and rightN , using bit k; 5: Calculate σHR and σHL 6: Calculate γk using (6) 7: if γk > γmax then 8: γmax ← γk ; 9: kmax ← k; 10: end if 11: end for 12: if Suitable Decision Found then 13: Create LChild and RChild with appropriate data. 14: dcsn ← dcsn OR kmax ; 15: Store kmax the decision of this node N.k; 16: SplitNode(LChild, dcsn); 17: SplitNode(RChild, dcsn); 18: end if

B. EBB Tree Search and Retrieval Searching the EBB tree is performed in the following two steps: 1) search into the bitmap space index and 2) generate ranked results of bitmaps in the leaves using a chosen metric. During the induction of the index, each node stores the decision bit k. Once the tree is induced, the leaves are analyzed to provide efficient nonlinear leaf traversal during searches. To accommodate the need to traverse the leaves, we build a neighbor priority queue for each leaf by calculating the probabilistic prototypes of the leaves (i.e., groups of bitmaps). The probabilistic prototype is calculated from the probability

1610

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 5, MAY 2011

that each bit k is on in the current leaf. When the probability of bit k being on is greater than or equal to 0.5, the prototype bit is set on. These prototypes are then used to compute neighbor priority queues based on leaf prototype similarity on a per-leaf basis. The generation of the leaf priority queues is O(m2 q), where m is the number of leaves generated in the tree, and q is the desired priority queue size. Given a query bitmap B, the navigation down the index is a series of simple bitwise operations. A search into the feature space for n results is performed following the recursive Algorithm 2. The search with bitmap B starts in the root node, specifying desire result size S. In each internal node N , steps 1–6 facilitate the recursive tree navigation. At each internal node, a decision bit k was stored during induction. If the query bitmap has that bit set on, the search continues in the right subtree of the current node N ; otherwise, it continues in the left subtree. Algorithm 2. Search(N,B,S): EBB tree searching in node N for S results from population D partitioned into leaves L using query bitmap B. 1: if N not a leaf then 2: if B AND k = k then 3: Return Search(N.RightChild,B,S); 4: else 5: Return Search(N.LeftChild,B,S); 6: end if 7: else 8: Rank destination leaf’s bitmaps in order of similarity into result R; 9: if |R| < S then 10: for L ∈ P riorityQueue do 11: Rank L bitmaps into R; 12: if |R| ≥ S then 13: Break; 14: end if 15: end for 16: end if 17: end if 18: RETURN R; When a leaf of the tree is reached, the leaf population is added to the ranked result set R. Oftentimes, a leaf may not have an adequate amount of data to satisfy the desired result set size, in which case the search must continue collecting results from additional leaves. At this point, searches must continue outward in the bitmap space from the initial leaf. The traversal of the leaves is codified starting at step 9 in Algorithm 2 when the current result set size is less than the desired size S. We first check that more results are needed, which is the expected case. We then examine the first neighbor in the priority queue, which is the most similar leaf in the tree, as measured by the chosen similarity metric. The bitmaps of this neighbor leaf L are added into the result set R. In an iterative fashion, the size of R is again checked, and the next neighbor leaf of the priority queue is processed, if needed. We currently build priority queues for each leaf that cover a portion of the database population.

Retrievals are conducted using the EBB tree, which partitions the feature space into relatively small groups of similar bitmaps. The extracted bitmap data resides in a dedicated data store and is used during similarity ranking, whereas the index resides in search agents. The search agents have a small memory footprint and can be distributed (i.e., replicated) across a network. The search agents hold just the navigation portion of the EBB, accessing the priority queues and data inside the data store. When traversal through the leaf population is required, a leaf’s priority queue is used for navigation through the data store. From a practical standpoint, we build priority queues to accommodate search result sizes that are a portion of the database size |D|. IV. R ESULTS The evaluation of a large-scale content-based retrieval system is often subjective. Different users may consider the various visual–perceptual characteristics to be of different importance. This case is a driving reason for concepts such as relevance feedback and customizable queries. In large-scale databases of satellite imagery with automatically extracted objects, it is infeasible to have the database ground truth labeled. Due to the subjective nature of content-based retrieval and the large scale of our database, we provide some example retrievals from the system, experiments using deformed shapes, and efficiency evaluations in the remainder of this section. A. Shape Deformation Effects We performed experiments to evaluate the effects of increasing levels of bit differences on the retrieval of objects. For these experiments, we eroded the encoded bitmap shapes and then used the eroded bitmap as a query into the database. Fig. 4 shows three example imagery objects, followed by the bitmap encoded shape, and the bitmap eroded by 2% and 5% of the pixels. The bitmaps simultaneously were eroded from the top and bottom. As noted in Section III, bits farther along the yoffset from the center x-axis are more heavily weighted than bits near the center. For this reason, we chose to first erode the bitmaps for this experiment in the significant weighted bits. This approach accentuates the effect of the shape change in the rankings. The first row in Fig. 4 shows our first airplane shape, and the effect of the erosion is the loss of the wings on the plane shape. The second row is the L-shaped building, were the erosion appears to effectively round off the corners while still preserving the general appearance of the L shape. The last row is the water treatment pool, which we basically begin to flatten. Note that the water treatment pools, followed by the baseball diamonds, had the largest bitmap cardinality. Therefore, the percentage-based erosion has the most drastic impact on these shapes. Given that all our objects are automatically extracted from the imagery, these experiments help demonstrate to what degree our retrieval ability is affected by bit differences that result from imperfect extraction algorithms. We evaluated the ranking position of the original encoded shape in the query results, as well as the dissimilarity trend with increasing erosion. Fig. 6 represents a subset of our test data, which includes 10 airplanes, 20 baseball diamonds,

SCOTT et al.: ENTROPY-BALANCED BITMAP TREE FOR SHAPE-BASED OBJECT RETRIEVAL

Fig. 4.

1611

Shape erosion examples: imagery object, extracted and aligned shape, and then erosion at 2% and 5% for the test of shape change on retrieval ranking.

10 L-shaped buildings, 10 water treatment pools, and numerous other shapes. We used 500 test shapes to erode for this experiment, but retrievals were conducted against the full database of 1.3 million objects. Fig. 5(a) provides a summary of the erosion of our test data with regard to bit cardinality, related to percentage. Baseball diamonds and airplanes averages provide the bounds of the bit erosion as the highest and lowest bit cardinalities, respectively. The average of all test objects is shown by the middle trend (All). Fig. 5(b) plots the average dissimilarity of the eroded shape versus the original encoded shape as the erosion increases. Note that this trend lacks the linearity of the bits versus percent erosion in Fig. 5(a) due to the bit-weighting scheme employed. As expected, the higher cardinality baseball diamond average significantly increases faster under percentage erosion due to both the increased number of bits that have changed and the propensity of those eroded bits to be weighted higher. Fig. 5(c) shows the average rank of an evaluation object when the system is queried with an eroded version of the object. It is observed that the airplanes retain the first result backup through 5% erosion due to the uniqueness of the shape and, as

can be observed in Fig. 4, the airplane shape is still observable in 5% of the eroded shapes. Baseball diamonds, on the other hand, quickly drop in average rank by 5% erosion. This result can be attributed to the fact that a given percentage erosion in baseball diamonds is more than three times the bits eroded from an airplane. In addition, as previously emphasized, the bits eroded are the highest weighted bits, thereby more significantly affecting the rank of the original object. We also expect that our database has more objects that loosely resemble baseball diamonds, e.g., buildings and water treatment pools. B. Content-Based Object Retrievals Currently, we support customizable object searches in our system by separately weighting the effect of the object’s shape and spectral features, as well as offering size constraints on the objects. Fig. 6 shows six example objects (top row) used as queries into GeoIRIS, followed in the next eight rows with ordered search results. In the clips of the satellite imagery shown, the relevant objects are bounded by yellow rectangles.

1612

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 5, MAY 2011

Fig. 5. Shape erosion effects. (a) Trends of the average quantity of bits eroded from shapes relative to the percentage of erosion. (b) Trend of average dissimilarity between test objects and their eroded counterpart. (c) Trend in ranking of the original object when queried with its eroded version.

The first column is a query using an airplane object. The search was customized to designate a shape-only search, without using object spectral characteristics and constraining search result objects to a size of 25–115 m. These results demonstrate rotational insensitivity. In this example, one of the challenging issues of using the DMP for automatic object extraction emerges. Results 3 and 4 are the same plane, extracted from neighboring levels of the DMP, but with shape encodings that vary enough to not be detected as duplicates by our existing algorithms (due to variations in the extracted shape at different levels). This case happens also with results 6 and 7. In each of these cases, it is shown that the extracted shape has variations just enough to change the dimensions and position of the bounding box. The second column is a similar query with a different plane. Note that the query plane from column 1 appears as the second to the last of the results for column 2. Column 3 is a search with a single baseball diamond among a complex of diamonds.

In this search, we equally weight the shape with the spectral characteristics of the object and, furthermore, constrain the size to a range of 30–80 m. We want to ensure that we get large areas of dirt, which are not very large or very small to be a baseball diamond. Note that, in these results, we have found four distinct baseball diamond complexes. In addition, in lower ranked results, grass infield baseball diamonds are discovered once we exhaust the database of dirt infield diamonds. Column 4 is an L-shaped building that was queried with no spectral characteristics and a size constraint of larger than 30 m. By not utilizing spectral characteristics, we can find objects of various colors. Column 5 represents a water treatment pool. These dark objects are not perfectly round but, instead, have a deep and narrow concavity from the light-colored catwalk (see Fig. 4, row 3). This query is performed with the shape weighted 75% and the spectral characteristics weighted at 25%, with the size constrained to 45–60 m. In this query, some of our results are not our conceptual water pool yet closely match in terms of shape and spectral characteristics. The final column results from querying the system with the dark circle on top of a cooling tower, with 60/40 shape/spectral weighting and the size constrained to 30–150 m. Of our imagery collection, we have three cooling towers, two of which are shown as the top two results. The third cooling tower, which is not returned, has large amounts of steam occluding the top opening of the tower in our imagery. The remaining results are similar in extracted shape and spectral characteristics. We conducted an additional experiment using a collection of ground-truth labeled baseball diamonds, L-shaped buildings, and water treatment pools. Our nonlabeled data consisted of 31 000 objects from a 2-km spatial proximity of 20 baseball diamonds, 10 L-shaped buildings, and 10 water treatment pools. For each test, we withheld 10% of the ground-truth objects for tenfold cross-validation. After building EBB for each test collection, we queried the remaining data to measure the recall in the top 50 of the desired test class. Our baseball diamonds had an average recall 77.87% in the top 50 results. In addition, four new baseball diamonds, which were not part of the ground truth, were discovered in the results. Our L-shaped buildings exhibited an average recall of 80.48%, and the water treatment pools exhibited an average recall of 70.04%. In all of these tests, neither object size nor spectral characteristics was not used to restrict the result set, as in typical queries in our GeoIRIS system. The use of object size would allow for improved recall in a smaller result set, because the unlabeled data averaged 36.25 m, whereas the minimum size of the ground-truth objects was 37 m. C. Efficiency With regard to query efficiency, the search time of shapebased queries is primarily dependent on the number of results desired and the associated cost of computing the bitmap dissimilarities. Recall from Section III that we build leaf priority queues to enable the nonlinear navigation in the highdimensional bitmap space. Our system utilizes priority queues that have a target object coverage for retrievals of approximately 20 000. We typically retrieve 6000–12 000 results from

SCOTT et al.: ENTROPY-BALANCED BITMAP TREE FOR SHAPE-BASED OBJECT RETRIEVAL

Fig. 6.

Shape retrieval with EBB. Top row images are query objects (in bounding box), and top ranked results follow.

1613

1614

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 5, MAY 2011

Fig. 8. Content-based retrieval interface using an airplane (left-hand side) that was automatically extracted from the image database. The large center region is the fourth ranked object retrieved from the database, shown in context and marked with the bounding box. The top of the retrieval results are shown on the right.

Fig. 7. Retrieval efficiency. (a) Average number of bitmap dissimilarities calculated using (2) to retrieve thousands of the results. (b) Average seconds to retrieve a result set of increasing result set size (thousands). (c) Timing comparison for 10-nearest neighbor searches of increasing database size for database versus EBB.

the search agent for the client interface. We performed retrieval efficiency experiments using 800 randomly selected objects in our database. Each object was used to retrieve results sets of size 1000 and increasing to 25 000, in steps of 1000. Fig. 7 summarizes the results of these experiments. For each retrieval, we recorded the time (in seconds) and the number dissimilarity calculations used. For example, the average number of bitmap dissimilarities computed, with (2), to retrieve the top 1000 results was less than 1053. This result is 0.081% of the comparisons that a brute-force search would require in our database of 1.3 million objects. Fig. 7(a) shows that the number of dissimilarity computations (solid line) closely follows the desired result size (plus marked trend). The number of comparisons needed for brute force is 1.3 million for any number of desired results. As aforementioned, our priority queues target

coverage of objects is 20 000, which causes the dissimilarity computations to begin to level off after 20 000 desired results as we exhaust the navigation priority queues. Fig. 7(b) shows the timing trend (in seconds) for the experiment. The average retrieval time for our typical query of 6000 shapes was 1.87 s, and for 12 000, it was 4.57 s. As previously discussed, the priority queue sizes that we use began to limit the number of retrievals possible, and therefore, the timing begins to level off after 20 000. Note that this priority queue size is simply a system parameter that can be adjusted to balance the expected search result sizes against the resources dedicated to managing priority queues. This level of efficiency for shape retrievals facilitates integration into complex systems, such as GeoIRIS. In addition, Fig. 7(c) provides a comparison in timing between 10-nearest neighbor searches against increasing database sizes for both a traditional RDBMS and the EBB. These results are average times for 300 test queries as the database size increases. The notable trends are that searches without the EBB linearly increase with the size of the database. In contrast, searches with the EBB are significantly faster and logarithmically increase, as expected. When considering the two types of efficiency experiments together, we see that query times are linear with respect to the desired size of a result set S and are logarithmic with respect to the number of leaves L. Overall, the efficiency of the retrieval is O(Slog2 (L)). For all these experiments, we used a dual quad-core (1.6-GHz) server with 8 GB of RAM, which was running the back-end PostgreSQL database, as well as the EBB index agents and search clients. V. C ONCLUSION In our GeoIRIS retrievals, we have not expected to retrieve objects using shape alone; instead, we have combined the results of shape retrieval with object spectral signatures and, possibly, size constraints. In the future, we expect to include object textural analysis or other relevant algorithms, with shape-based object retrieval being a component of a larger solution. Fig. 8

SCOTT et al.: ENTROPY-BALANCED BITMAP TREE FOR SHAPE-BASED OBJECT RETRIEVAL

shows the results from our GeoIRIS system when searching with an automatically extracted airplane. The upper left corner is a region with the query shape noted by a bounding box. The larger center region shows the currently selected result in its larger context, with the result object in a bounding box. This example shows the fourth ranked result to the query object. Note that the airplanes are facing in two directions. Given the alignment steps, we can expect objects, e.g., airplanes, to be aligned pointing either left or right. To facilitate retrievals with rotational insensitivity, we augment the retrieval by repeating the search with a mirrored bitmap of the query. Given our retrieval efficiency, the search time for 6000 shapes increases to less than 6 s for a second result set and the subsequent merge of the result sets. Our future work will continue toward increasing the scalability of the EBB. Research is needed to determine the practical limits on the scalability of the EBB for today’s hardware platforms and algorithms to overcome these limitations. We are also developing algorithms to provide dynamic manipulation of the EBB with data inserts, deletes, and updates. One of the challenges of dynamic manipulation is maintaining the efficiency of the tree after periods of manipulation. As the number of changes to the tree increases, relative to the original database size, the statistics of the data needs to be reevaluated and the tree possibly entirely rebuilt. In addition, we will explore applications of the EBB to other domains, possibly domainspecific text retrieval with a limited index term set. Another possible extension of this paper is the generalization of the EBB from bitmap space to arbitrarily discrete feature spaces. ACKNOWLEDGMENT The authors would like to thank DigitalGlobe for providing QuickBird imagery from the RADII development data set for use in this paper and the reviewers for their constructive comments, which have significantly helped improve this manuscript. R EFERENCES [1] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafher, D. Lee, D. Petkovie, D. Steele, and P. Yanker, “Query by image and video content: The QBIC system,” Computer, vol. 28, no. 9, pp. 23–32, Sep. 1995. [2] J. R. Smith and S.-F. Chang, “Visualseek: A fully automated contentbased image query system,” in Proc. 4th ACM Int. Conf. Multimedia, 1996, pp. 87–98. [3] A. Pentland, R. W. Picard, and S. Sclaroff, “Photobook: Content-based manipulation of image databases,” Int. J. Comput. Vis., vol. 18, no. 3, pp. 233–254, Jun. 1996. [4] T. Gevers and A. W. Smeulders, “PicToSeek: Combining color and shape invariant features for image retrieval,” IEEE Trans. Image Process., vol. 9, no. 1, pp. 102–119, Jan. 2000. [5] T. Gevers and A. W. Smeulders, “Content-based image retrieval: An overview,” in Emerging Topics in Computer Vision, G. M. S. B. Kang, Ed. Upper Saddle River, NJ: Prentice-Hall, 2004, ch. 8, pp. 333–384. [6] M. Lew, N. Sebe, C. Lifi, and R. Jain, “Content-based multimedia information retrieval: State of the art and challenges,” ACM Trans. Multimedia Comput., Commun., Appl., vol. 2, no. 1, pp. 1–19, Feb. 2006. [7] M. Datcu, H. Daschiel, A. Pelizzari, M. Quartulli, A. Galoppo, A. Colapicchioni, M. Pastori, K. Seidel, P. G. Marchetti, and S. D’Elia, “Information mining in remote sensing image archives: System concepts,” IEEE Trans. Geosci. Remote Sens., vol. 41, no. 12, pp. 2923–2936, Dec. 2003.

1615

[8] H. Daschiel and M. Datcu, “Information mining in remote sensing image archives: System evaluation,” IEEE Trans. Geosci. Remote Sens., vol. 43, no. 1, pp. 188–199, Jan. 2005. [9] J. Li and R. M. Narayanan, “Integrated spectral and spatial information mining in remote sensing imagery,” IEEE Trans. Geosci. Remote Sens., vol. 42, no. 3, pp. 673–685, Mar. 2004. [10] I. Kunttu, L. Lepisto, and J. Rauhamaa, “Fourier-based object description in defect image retrieval,” Mach. Vis. Appl., vol. 17, no. 4, pp. 211–218, Sep. 2006. [11] V. P. Shah, N. H. Younan, S. S. Durbha, and R. L. King, “A systematic approach to wavelet-decomposition-level selection for image information mining from geospatial data archives,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 4, pp. 875–878, Apr. 2007. [12] M. Pesaresi and J. A. Benediktsson, “A new approach for the morphological segmentation of high-resolution satellite imagery,” IEEE Trans. Geosci. Remote Sens., vol. 39, no. 2, pp. 309–320, Feb. 2001. [13] J. Weickert, S. Ishikawa, and A. Imiya, “Linear scale-space has first been proposed in Japan,” J. Math. Imaging Vis., vol. 10, no. 3, pp. 237–252, May 1999. [14] F. Mokhtarian and A. K. Mackworth, “A theory of multiscale, curvaturebased shape representation for planar curves,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 8, pp. 789–805, Aug. 1992. [15] Y. Avrithis, Y. Xirouhakis, and S. Kollias, “Affine-invariant curve normalization for object shape representation,” Mach. Vis. Appl., vol. 13, no. 2, pp. 80–94, Nov. 2001. [16] T. Adamek and N. E. O’Connor, “A multiscale representation method for nonrigid shapes with a single closed contour,” IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 5, pp. 742–753, May 2004. [17] L. J. Latecki, R. Lakamper, and U. Eckhardt, “Shape descriptors for nonrigid shapes with a single closed contour,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., 2000, vol. 1, pp. 1424–1429. [18] D. Zhang and G. Lu, “Review of shape representation and description techniques,” Pattern Recognit., vol. 37, no. 1, pp. 1–19, Jan. 2004. [19] H. Blum, “A transformation for extracting new descriptors for shape,” in Models for the Perception of Speech and Visual Forms, W. Whaten-Dunn, Ed. Cambridge, MA: MIT Press, 1967, pp. 362–380. [20] T. B. Sebastian, P. N. Klein, and B. B. Kimia, “Recognition of shapes by editing shock graphs,” in Proc. ICCV, 2001, pp. 755–762. [21] S. Brandt, J. Laaksonen, and E. Oja, “Statistical shape features for content-based image retrieval,” J. Math. Imaging Vis., vol. 17, no. 2, pp. 187–198, Sep. 2002. [22] M. Safar and C. Shahabi, “MBC-based shape retrieval: Basics, optimizations and open problems,” Multimedia Tools Appl., vol. 29, no. 2, pp. 189– 206, Jun. 2006. [23] S. Belongie, J. Malik, and J. Puzicha, “Shape matching and object recognition using shape contexts,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 4, pp. 509–522, Apr. 2002. [24] E. Rivlin and I. Weiss, “Local invariants for recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, no. 3, pp. 226–238, Mar. 1995. [25] R. Mehrotra and J. Gary, “Similar-shape retrieval in shape data management,” Computer, vol. 28, no. 9, pp. 57–62, Sep. 1995. [26] M. Kliot and E. Rivlin, “Invariant-based shape retrieval in pictorial databases,” Comput. Vis. Image Understanding, vol. 71, no. 2, pp. 182– 197, Aug. 1998. [27] G. Scott and C.-R. Shyu, “Knowledge-driven multidimensional indexing structure for biomedical media database retrieval,” IEEE Trans. Inf. Technol. Biomed., vol. 11, no. 3, pp. 320–331, May 2007. [28] R. Rao and D. Ballard, “Object indexing using an iconic sparse distributed memory,” in Proc. IEEE Int. Conf. Comput. Vis., 1995, pp. 24–31. [29] C.-C. Liu, J.-L. Hsu, and A. L. Chen, “Efficient near neighbor searching using multi-indexes for content-based multimedia data retrieval,” Multimedia Tools Appl., vol. 13, no. 3, pp. 235–254, Mar. 2001. [30] C.-R. Shyu, M. Klaric, G. J. Scott, A. S. Barb, C. H. Davis, and K. Palaniappan, “GeoIRIS: Geospatial information retrieval and indexing system—Content mining, semantics modeling, and complex queries,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 4, pp. 839–852, Apr. 2007. [31] M. Klaric, G. Scott, C.-R. Shyu, and C. Davis, “Automated object extraction through simplification of the differential morphological profile for high-resolution satellite imagery,” in Proc. IGARSS, 2005, pp. 1265– 1268. [32] G. Lu and A. Sajjanhar, “Region-based shape representation and similarity measure suitable for content-based image retrieval,” Multimedia Syst., vol. 7, no. 2, pp. 165–174, Mar. 1999. [33] R. A. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval. Reading, MA: Addison-Wesley, 1999. [34] K. Fukunaga, Introduction to Statistical Pattern Recognition. New York: Academic, 1990.

1616

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 5, MAY 2011

[35] S. Theodoridis and K. Koutroumbas, Pattern Recognition. New York: Academic, 1999. [36] M. Ester, H. Kriegel, J. Sander, and X. Xu, “A density-based algorithm for discovering clusters in large spatial databases with noise,” in Proc. Int. Conf. Knowl. Discov. Data Mining, 1996, pp. 226–231. [37] G. Scott and C.-R. Shyu, “EBS k-d tree: An entropy-balanced statistical k-d tree for image databases with ground-truth labels,” in Proc. Int. Conf. Image Video Retrieval, vol. 2728, Lecture Notes in Computer Science, 2003, pp. 467–476. [38] C. E. Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J., vol. 27, pp. 379–423, Jul.–Oct. 1948.

Grant J. Scott (S’02–M’09) received the B.S. and M.S. degrees in computer science and the Ph.D. degree in computer engineering and computer science from the University of Missouri, Columbia, in 2001, 2003, and 2008, respectively. He is currently serves an Assistant Research Professor with the Department of Electrical and Computer Engineering, University of Missouri. He conducts research as part of the Satellite and Remote Sensing Group, Center for Geospatial Intelligence (CGI). During his Ph.D. studies, he was a member of the Medical and Biological Digital Library Research Laboratory and the Center for Geospatial Intelligence, University of Missouri, conducting research in high-performance multimedia retrieval systems (databases), hybrid retrieval systems and protein structural retrieval/comparison engines, and highresolution satellite image processing. During the course of his M.S. degree, he was a member of the Computational Intelligence Research Laboratory, with research emphasis on computational intelligence, pattern recognition, neural networks, fuzzy systems, image processing/machine vision, and bio-medical image databases. His current research is focused on the automated exploitation of high-resolution satellite imagery, in particular geospatial database development, imagery feature-extraction algorithm development, and distributed automatic imagery processing orchestration architectures. His research interests also include high-dimensional indexing and content-based retrieval in biomedical and geospatial databases, as well as computer vision, pattern recognition, computational intelligence, databases, parallel/distributed systems, and information theory in support of media databases systems.

Matthew N. Klaric (S’06) received the B.S. (summa cum laude) degree in computer science from Saint Louis University, St. Louis, MO, in 2003. He is currently working toward the Ph.D. degree in computer science at the University of Missouri, Columbia. In 2004, he was a Research Assistant with the Medical and Biological Digital Library Research Laboratory and the Center for Geospatial Intelligence, University of Missouri. In addition, he has served as an Instructor for several undergraduate computer science classes. His research interests include geospatial content-based information retrieval, data mining, computer vision, and pattern recognition. Mr. Klaric annually reviews papers for the IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

Curt H. Davis (S’90–M’92–SM’98–F’08) was born in Kansas City, MO, on October 16, 1964. He received the B.S. and Ph.D. degrees in electrical engineering from the University of Kansas, Lawrence, in 1988 and 1992, respectively. He is currently the Naka Endowed Professor of electrical and computer engineering with the University of Missouri, Columbia (MU) and the Director of the Center for Geospatial Intelligence. His primary research involves the use of satellite microwave and optical remote sensing systems for applications in the areas of earth observation and science, ice sheet mapping and change detection, and urban area geospatial information processing. His ice sheet mapping and change detection research has been funded by the National Aeronautics and Space Administration (NASA) for more than a decade, and he is an internationally recognized expert in the measurement of polar ice sheet change using precision satellite altimeters, the influence of climate on these changes, and the impact of these changes on global sea levels. His urbanarea research focuses on the automated processing and development of highresolution geospatial information products. Examples include high-resolution digital elevation models, urban land cover maps, automated feature extraction of anthropogenic features, and automated change detection. His research results have been documented in more than 45 refereed journal publications and 70 symposia presentations and proceedings. His most significant scientific results have been published in top scientific journals such as Science, Nature, and the Journal of Geophysical Research. Dr. Davis has recently been named an IEEE Fellow for his “contributions to satellite remote sensing.” He has received numerous awards throughout his career, including the National Science Foundation (NSF) Antarctica Service Medal (1988 and 1989), the International Union of Radio Science (URSI) Young Scientist Award (1996), and the NASA New Investigator Program (1996–1999). He served as the Technical Program Cochair of the 2004 IEEE Geoscience and Remote Sensing Symposium held in Anchorage, AK. He is currently an Associate Editor for the IEEE T RANSACTIONS ON G EOSCIENCE AND R EMOTE S ENSING , in which majority of his technical contributions to remote sensing have been published.

Chi-Ren Shyu (S’89–M’99–SM’07) received the M.S.E.E. and Ph.D. degrees in electrical and computer engineering from Purdue University, West Lafayette, IN, in 1994 and 1999, respectively. Upon completing one year of postdoctoral training with Purdue, he joined the Department Computer Engineering and Computer Science, University of Missouri (MU), Columbia, in October 2000. He is currently the Paul K. and Diane Shumaker Endowed Professor of engineering and heads the MU Informatics Institute. His research interests include geospatial image information mining, visual knowledge understanding and retrievals, and biomedical informatics. Dr. Shyu is the recipient of the National Science Foundation Faculty Early Career Development (NFS CAREER) Award, MU College of Engineering Faculty Research Award, and various teaching awards. He is a member of the American Association for the Advancement of Science (AAAS) and the American Medical Informatics Association (AMIA).