Metrics of spectral image complexity with application to large area ...

Metrics of spectral image complexity with application to large area search David W. Messinger Amanda Ziemann Bill Basener Ariel Schlamm

Downloaded from SPIE Digital Library on 28 Mar 2012 to 129.21.58.44. Terms of Use: http://spiedl.org/terms

Optical Engineering 51(3), 036201 (March 2012)

Metrics of spectral image complexity with application to large area search David W. Messinger Amanda Ziemann Rochester Institute of Technology Chester F. Carlson Center for Imaging Science Rochester, New York 14623 E-mail: [email protected] Bill Basener Rochester Institute of Technology School of Mathematical Sciences Rochester, New York 14623 Ariel Schlamm Rochester Institute of Technology Chester F. Carlson Center for Imaging Science Rochester, New York 14623

Abstract. Spectral image complexity is an ill-defined term that has been addressed previously in terms of dimensionality, multivariate normality, and other approaches. Here, we apply the concept of the linear mixture model to the question of spectral image complexity at spatially local scales. Essentially, the “complexity” of an image region is related to the volume of a convex set enclosing the data in the spectral space. The volume is estimated as a function of increasing dimensionality (through the use of a set of endmembers describing the data cloud) using the Gram Matrix approach. It is hypothesized that more complex regions of the image are composed of multiple, diverse materials and will thus occupy a larger volume in the hyperspace. The ultimate application here is large area image search without a priori information regarding the target signature. Instead, image cues will be provided based on local, relative estimates of the image complexity. The technique used to estimate the spectral image complexity is described and results are shown for representative image chips and a large area flightline of reflective hyperspectral imagery. The extension to the problem of large area search will then be described and results are shown for a 4-band multispectral image. © 2012 Society of Photo-Optical Instrumentation Engineers (SPIE). [DOI: 10.1117/1.OE.51.3.036201]

Subject terms: remote sensing; spectral imagery; image complexity; search. Paper 111203 received Sep. 27, 2011; revised manuscript received Jan. 25, 2012; accepted for publication Jan. 26, 2012; published online Mar. 29, 2012.

1 Introduction Large area search of imagery is a challenging task. It requires a very high probability of detection (i.e., missed detections render the search a failure) while accepting a moderate number of false alarms. The goal is to “cue” an analyst to regions of interest in an image for further investigation. Several automated target recognition (ATR) algorithms have been proposed, but the large area search problem in general does not seek to identify specific targets of interest with a known spatial or spectral signature.1 Here, we are interested in identifying localized regions of a large multispectral or hyperspectral image that contain evidence of man-made activity. Typically, this problem has been addressed through the application of change detection methodologies1–3 which have their own challenges. The approach taken here is based not on statistical models of the data, nor is it based on detection of changes between imagery. Instead, the image is decomposed into spatial tiles and the “complexity” of each tile is estimated. The hypothesis posits that complexity is related to “interesting-ness” and the analyst seeks to investigate “interesting” areas. The processed image can then be presented to the user with individual tile brightnesses adjusted to the complexity of that tile relative to the rest of the image. Alternatively, individual image chips can be collected and ranked by their relative complexity and sorted for analysis. Spectral image complexity is related to the concepts of spectral image quality and utility. A spectral image quality, 0091-3286/2012/$25.00 © 2012 SPIE

Optical Engineering

or utility, metric has long been sought.4–8 Spectral image “quality” generally refers to the faithfulness with which a spectral image represents the truth on the ground and is not addressed here. Spectral image “utility” refers to how useful a specific image is with regard to the performance of a specific task. The two metrics are not mutually exclusive—for an image to be useful it generally has to be of high quality. However, a high quality Landsat image is not useful for identifying personal vehicles due to the lack of sufficient spatial resolution. The search for such a metric is motivated by the desire to characterize imagery for tasking and collection, archiving, and exploitation. Reference 5 describes a functional expression, similar to the national imagery interpretability rating scale (NIIRS),4 for spectral quality as related to object and anomaly detection in reflective hyperspectral imagery. The function takes into consideration spatial resolution, spectral resolution, and signal-to-noise ratio and was formulated at various Probabilities of false alarm (PFA) across several hyperspectral images with known target locations. In Ref. 7, the general spectral utility metric (GSUM) is presented, a combination of both the spatial and spectral information in an image. The method assumes that the spatial and spectral information are largely separable, but both contribute to the overall utility of an image. In Ref. 6, seven spectral quality/utility metrics are identified and their performance is compared to the GSUM metric with regard to the problem of target detection in a hyperspectral image. It is shown that there are inconsistencies across these metrics and future work remains. Reference 8 presents a method to assess the utility of a spectral image for the task of target detection by utilizing a “target

036201-1

Downloaded from SPIE Digital Library on 28 Mar 2012 to 129.21.58.44. Terms of Use: http://spiedl.org/terms

March 2012/Vol. 51(3)

Messinger et al.: Metrics of spectral image complexity with application : : :

implantation” method. In this way, the authors are able to determine the utility of a particular image for detection of a specific target signature without knowledge of the presence or absence of the target in the image. It is likely that all of these metrics are related to the overall image complexity at both local and image-wide scales. However, all of these methods consider the image in its entirety. While it is intuitively obvious that the more “complex” an image is the more difficult it will be to (semi-) automatically extract quantitative information from it, a measure of this complexity is still unknown. Here, we propose such a metric to quantify, within a single image, regions of relatively more or less spectral complexity. The method is based on the assumption that more complex regions of the image will (a) contain a relatively larger number of materials and (b) those materials will be relatively more spectrally diverse. Alternatively, regions that are less complex are composed of a small number of distinct materials that are more spectrally similar. The complexity is quantified through estimation of the volume of the convex set that is assumed to enclose the data in the full spectral space—a larger number of more spectrally diverse materials will result in a larger convex set volume. Estimation of this convex set is achieved through identification, at spatially local scales, of endmembers that represent the pure materials in that region. The volume of this convex set is identified using the Gram Matrix approach (see Sec. 2.2) that does not require dimensionality reduction of the endmembers. This approach is qualitatively demonstrated against a large area reflective hyperspectral scene. The methodology is then extended to the large area search problem and is demonstrated against 4-band Quickbird imagery of a mountainous region. Here, the complexity measure cues an analyst to regions of the image that are potentially “interesting” relative to the rest of the scene, aiding the large area search task by prioritizing regions of the image for visual inspection. This paper is organized in the following way. Section 2 describes the mathematics behind the spectral complexity measure and how it is estimated. Results will be shown for a hyperspectral scene. Section 3 presents the application of the complexity measure to the large area image search problem and results will be shown for a multispectral scene. Section 4 presents a summary of the work.

2.1 Volume Estimation The estimation of a convex set volume is in principle straightforward. The smallest convex set in k dimensions is defined by the k þ 1 endmembers that characterize the corners of the simplex.10 In the linear mixture model these corners are assumed to be the “pure” pixels in the scene and the mixed pixels (i.e., everything else) lie inside this convex set in the k-dimensional spectral hyperspace.11 The linear mixture model has been successfully employed for estimation of the sub-pixel fractions (abundances) of pure materials in mixed pixels for several applications in hyperspectral imaging. Here, we are not interested in the linear mixture model per se, or the process of spectral unmixing. Instead, we use the construct of an enclosing convex set to make estimates of the complexity of a collection of pixels relative to other collections of pixels in the image. In this case, as described below, we collect the pixels spatially through a tiling procedure to estimate the spectral complexity in spatial regions covering the image. The assumption is that more “complex” regions of the image will likely contain more distinct materials, and those materials will be more well separated in the spectral space. This is intuitively obvious—large, uniform regions will occupy the spectral space in a relatively tight, single material cluster with variability driven by the within-material spectral variations. More highly cluttered regions will have multiple materials that are distinctly different from one another and will be more widely separated in the hyperspace. Regions that contain multiple, similar natural materials (e.g., a forest—grassland boundary), will be somewhere in between on this scale of complexity. Figure 1 demonstrates this with a simple example. Image chips of various complexity were taken from a hyperspectral image and Fig. 1(a) shows two band projections (from the visible and SWIR) of the pixels in the chips [shown in Fig. 1(b)]. In this example, the region of the hyperspace covered by the small city (shown in blue) is significantly larger than the region covered by the chip containing grass, a road, & trees (green). Both occupy a larger volume in the hyperspace than that occupied by the grass pixels (red). Mathematically, a k-dimensional dataset needs k þ 1 corners to build an enclosing simplex. However, we hold that although the physical data may be collected in the full hyperspace, it typically is inherent to a much lower dimension

2 Spectral Complexity Measure The “complexity” of a region in a spectral image is assumed to be related to the number, and spectral diversity, of the materials in that region.9 More complex regions of the image are assumed to: 1) contain a relatively larger number of materials, and 2) contain materials that are relatively more spectrally diverse In this section we describe how we estimate the complexity for a collection of pixels taken from a spectral image and how we apply this technique to a large area coverage image. Results will be shown for small, illustrative image chips of varying complexity and then for a large, diverse, hyperspectral scene. Optical Engineering

(a)

(b)

Fig. 1 Examples of set volumes for various complexity in a hyperspectral image. (a) Two-band projection of pixels from various levels of complexity. Blue—city, Green—grass, road, and trees, and Red— grass. Pixel values are reflectance × 10; 000. (b) The corresponding image chips.

036201-2

Downloaded from SPIE Digital Library on 28 Mar 2012 to 129.21.58.44. Terms of Use: http://spiedl.org/terms

March 2012/Vol. 51(3)

Messinger et al.: Metrics of spectral image complexity with application : : :

(similar to how an inherently one-dimensional line and a two-dimensional square can exist in a three dimensional space). The convex set enclosing the data is defined by the endmembers describing the pure materials in the scene. Of course, the question always arises as to how many endmembers to use to represent a collections of pixels, as it is impossible a priori to know how many distinct materials are in this set.12 This highlights the difference between the mathematical definition of the convex set and the physical interpretation of the linear mixture model. Given a data set in k dimensions, we know how many vectors are required to create the simplex; but this has no bearing on the actual number of materials in the scene. Here, we take an iterative approach in that for a collection of pixels, we collect an abnormally high number of endmembers relative to the number of pixels in the collection and compute the convex set volume as a function of the dimensionality (i.e., number of endmembers) for the set of pixels. We use the Max-D algorithm13 to identify the endmembers for the pixels under test, although any method can be used. However, it is important to note that the Max-D algorithm determines the endmembers and returns them sorted in order of decreasing magnitude, a key component of the method presented here. Typically, the volume of the convex set enclosed by the set of endmembers is computed by taking the determinant of the set of endmembers. Unfortunately, this assumes that the m endmembers will form a square matrix, which in practice will require dimensionality reduction on the k dimensional vectors (the matrix will initially be k × m with k ≫ m). This is particularly severe when the material diversity (i.e., the number of distinct materials) in the data is small and the hyperspectral data are collected with hundreds of spectral bands. Here, we use a different technique, the Gram Matrix, to compute the volume without that limitation. In this study, the k-dimensional construct of a parallelotope (which is a convex set) was used to characterize the complexity of the data.14 A parallelotope is the higher dimensional analog of a parallelogram (2-D) and parallelepiped (3-D), and its definition varies slightly depending on whether it is given in terms of points or vectors. In 2-D, a parallelogram is defined by two points, P1 and P2 , as well as the origin O. Alternatively, the same parallelogram is defined by the two vectors OP1 and OP2 . Although it may seem counterintuitive that a four-corner shape is uniquely defined by either three points or two vectors, any additional information would be redundant because of the symmetric properties of the parallelogram. This same idea extends to hyperdimensions, in which it follows that a k-dimensional parallelotope (known as a k-parallelotope) may be uniquely defined by either k points (with the implicit addition of the origin), k þ 1 points (including the origin), or k vectors (based at the origin). These three definitions are equivalent, and here we use that of a k-parallelotope defined by k þ 1 points. In the context of spectral analysis, the data do not always live near the origin of the space. Dark pixels will be clustered very closely to the origin, while brighter pixels will be farther away. Hypothetically, if a bright image and a dark image correspond to the exact same scene, the bright image data will be a much greater distance away from the origin. When considering the parallelotope defined by the bright data, the Optical Engineering

inclusion of the origin as a corner will drastically increase the volume of that parallelotope (as compared to that of the darker data). This means the brighter data will have much larger volume-related metrics than the darker data, even though they are composed of the same material(s). To avoid this problem when performing a material-based analysis of a tile of data, we first identify the endmember nearest to the origin, and then shift the data by that amount. This results in the enclosing parallelotope having a true corner at the origin.

2.2 Gram Matrix The Gram Matrix,15 G, is linear algebraic in nature. It is an n × n matrix where n is the number of vectors in the test set, fxi gjni¼1 . The fi; jgth element of the Gram Matrix is given by the inner product between the ith and jth vectors, 

(1) Gi;j ¼ xi ; xj : The Gram Matrix can also be computed with respect to a particular element of the test set, and as such is termed the local Gram Matrix, here denoted by G 0. The local Gram Matrix as computed about the lth element is given by G 0 ðxl Þi;j ¼ hðxl − xi Þ; ðxl − xj Þi:

(2)

The local Gram Matrix will be ðn − 1Þ × ðn − 1Þ in size for a set of n vectors. One unique property of the Gram Matrix is that the determinant, the Gramian, is the square of the volume of the parallelotope formed by the vectors.16 Also, the vectors are linearly independent if and only if the Gramian is nonzero. Here, we compute the Gramian for the set of endmembers starting with three and iteratively increase the number of endmembers until the Gramian approaches zero. Because the endmembers are ordered by decreasing magnitude, the convex set volume generally reaches a peak at a small number of endmembers and is then a monotonically decreasing function. The relative change in volume of the set of endmembers is the same regardless of whether one uses the Gram matrix (G) or the local Gram matrix (G 0 ) computed relative to one of the endmembers in the set. Examples of these volumes as functions of the number of endmembers in the set are shown in Sec. 2.3. It is important to note that while the volume estimations made here are motivated by the construct of the simplex, the volume that we are actually measuring is that of the parallelotope whose edges are defined by a given set of endmembers. The volume of the simplex and the volume of the parallelotope17 are related by V simplex ¼

1 V : k! parallelotope

(3)

Here, it is highly advantageous to measure the volume of the parallelotope as it yields a metric that is much more sensitive to changes in the inherent dimensionality of the data. Additionally, we are primarily interested in relative changes in the enclosed data volume across the image, not the absolute volume, thus further motivating this choice of a metric.

036201-3

Downloaded from SPIE Digital Library on 28 Mar 2012 to 129.21.58.44. Terms of Use: http://spiedl.org/terms

March 2012/Vol. 51(3)

Messinger et al.: Metrics of spectral image complexity with application : : :

Fig. 2 RGB image of the full hyperspectral flight line used in the study.

2.3 Example Hyperspectral Results As an example, we estimate the relative complexity in a hyperspectral scene using the technique described above by applying the method to a reflective hyperspectral scene described in Ref. 18. An RGB image of the hyperspectral flightline (without roll correction applied) is shown in Fig. 2. The image is 512 × 2; 178 pixels with 126 bands covering 0.45 to 2.5 μm and was collected with the HyMAP sensor with an approximate ground sample distance (GSD) of 2.5 m. The image was processed in estimated surface reflectance. The location of the scene is Cooke City, MT. The scene content varies, with a small town at one end, a road dividing grasslands and forest running for the majority of the scene, foothills at the opposite end, and near the foothills, some small areas of residential development. The method used to estimate the relative local complexity within an image is outlined here: 1. tile the image 2. for each tile a. extract m endmembers (m ≈ 15) b. for i ¼ 3, m endmembers i. compute the parallelotope volume for i endmembers ii. store that volume as a function of i c. extract complexity metric from function 3. normalize image complexity measures by maximum tile metric in the image 4. scale each tile RGB brightness by normalized complexity measure 5. display result Before describing the results as applied to the entire flight line, we analyze several image chips of increasing complexity (as assessed qualitatively) to demonstrate the complexity

metric. The image chips are shown in Fig. 3 and include regions (of increasing qualitative complexity) characterized as “trees,” “grass,” “grass, road, and trees,” a “construction” site, and a “small city.” Each chip was analyzed in the full hyperspectral dimensionality and is approximately 4,000 total pixels. The results from estimating the complexity of each chip are presented in Fig. 4. Shown are the plots of the estimated volume as a function of increasing number of endmembers (dimensionality) of the test set. There are several points of interest in these plots. Most notable is that as the tile content complexity increases [from (a) to (e) in Figs. 3 and 4], the overall magnitude of the plots increase. However, regardless of scene content, the shape of the plot is relatively unchanged, achieving a maximum value at low dimensionality and then monotonically decreasing to a volume approaching zero. Only in the case of the small city does the plot initially increases, attributable to the presence of multiple, bright (i.e., large magnitude) endmembers. Additionally, one notices that the dimensionality at which the figures approach zero increases with complexity, although this number is necessarily quantized to integer values due to the method. Consequently, for all results below, the peak value in the volume plot as a function of the number of endmembers is taken as a relative measure of complexity for the pixels in the test set. The results of applying this process to the hyperspectral image in Fig. 2 are shown in Fig. 5 where the RGB values in each tile (40 × 40 pixels) have been scaled by the computed relative complexity in the scene (i.e., the most complex tile in the scene is the brightest, the least complex tile has a brightness of zero, and the remainder are linearly scaled between the two extremes). Here, we note that by this metric the most complex region of the scene is indeed the small city in the middle right portion of the image. The least complex area in the image is the forest, occupying the majority of the upper half of the image. The road that traverses the majority of the scene is highlighted, as are several smaller roadways. The foothills region is brightened a modest amount, as it is generally a mixture of grass and exposed soil and as such represents a measurable level of complexity relative to the rest of the image. Of most interest is that the method identifies regions in the image with small, isolated areas of development such as the tile containing the ongoing construction shown in Figs. 3(d) and 4(d) and analyzed previously. Note that this method is not attempting to identify pixel level anomalies, as the houses and construction areas most certainly are, but it is instead identifying regions of the image that are relatively more complex through characterization of

Fig. 3 RGB image chips of test areas: (a) trees, (b) grass, (c) grass, road, and trees, (d) a construction area, and (e) a small city.

Optical Engineering

036201-4

Downloaded from SPIE Digital Library on 28 Mar 2012 to 129.21.58.44. Terms of Use: http://spiedl.org/terms

March 2012/Vol. 51(3)

E

E

E

Messinger et al.: Metrics of spectral image complexity with application : : :

N

N

N

(b)

(c)

E

E

(a)

N

N

(d)

(e)

Fig. 4 Complexity plots for the test areas in the hyperspectral scene: (a) trees (b) grass (c) grass, road, and trees, (d) a construction area, and (e) a small city. Note change in scale of the y axis in (a)–(e).

the convex set that enclosed the spectral data in that region. This metric is related to a geometric estimation of the “size” of the distribution of the data and variation in the complexity metric represents variation in the spectral distribution of the data across the image.

only a small percentage of pixels in the collection of natural materials are replaced by man-made materials. In each of the three cases, the complexity measure shows a significant increase after only two or three percent of the natural pixels have been replaced. After around 5 to 7%, the measure levels off as the addition of more man-made pixels has little or no further impact on the size of the convex hull. For the images

2.4 Spatial Sensitivity Relative to Tile Size To understand the impact of tile size relative to the size of the “interesting” materials in the tile, a study was performed following the methodology in Ref. 19 Here, 10,000 pixels over natural materials (foothills, forest, and grassland, all taken from the image in Fig. 2), were considered separately, and are representative of “un-interesting” tiles. Man-made materials (roof tops, vehicles, etc.) were also extracted from this image. Individual pixels from the test set of natural materials were randomly removed and replaced by the man-made materials, at increasing percentages of the overall set of pixels. At each level of pixel injection, 100 trials were run replacing different natural material pixels with different man-made material pixels. At each level, the mean of the complexity measure was computed, and the results are shown in Fig. 6. From this study, one can observe that the complexity measure for the test set of pixels rises dramatically when

(a)

(b)

(c)

Fig. 5 RGB image of the full hyperspectral flight line (See Fig. 2) with tile brightnesses scaled by their relative complexity measure.

Optical Engineering

Fig. 6 Results from injection of man-made pixels into collection of natural materials. Y -axis is the peak volume and the x-axis is the percentage of pixels injected for backgrounds of (a) forest pixels, (b) foothills pixels, and (c) grass pixels.

036201-5

Downloaded from SPIE Digital Library on 28 Mar 2012 to 129.21.58.44. Terms of Use: http://spiedl.org/terms

March 2012/Vol. 51(3)

Messinger et al.: Metrics of spectral image complexity with application : : :

considered below, tiles were sized at 30 × 30 pixels, indicating that objects in the scene containing more than ≈15 − 20 pixels (i.e., ≈2% of 900 pixels) will dramatically increase the size of the convex hull and will be easily detected. The impact of sub-pixel targets on the complexity measure has not been investigated here as the primary application is not sub-pixel target detection. However, one can visualize that the impact on the size of the convex hull will be directly related to the percent abundance of the sub-pixel target and the “contrast” between the target signature and the background in which it is embedded. For high contrast targets, whose target signature is spectrally very different from their background, one need only have a relatively small percent abundance to change the nature of the convex hull. Alternatively, if the target signature is very similar to one of the endmembers describing the background, then even a large sub-pixel abundance would not have much impact on the volume of the convex hull as the target signature and the background endmember will effectively “point” in the same direction in the hyperspace. A further study of this effect is warranted. These results lead us to the application of this technique to the problem of large area search in remotely sensed imagery. 3 Application to Large Area Search As mentioned above, the large area image search problem is challenging in that the signature for which one is searching is not necessarily well defined. In particular, here we are interested in the search problem involving looking through a large area for new targets/regions of interest. This is different from other search problems, such as “find and count all the targets in this region.” Here, we assume a large area image has been collected over an area of interest with little or no knowledge about the content of that image, and new smaller regions of interest within that image are to be identified. This is the classic problem of finding a “needle in a haystack” without knowing what the needle looks like. The test image used is shown in Fig. 7(a). This is a four band image collected with the DigitalGlobe Quickbird sensor. The image has an approximate ground resolution of 2.5 m and a spectral response covering the standard blue, green, red, and near infrared bands. The image was collected of the Esperanza Forest Fire in October 2006 and the location is near Cabazon, CA. The image used here has been cropped to a 2k × 4k (pixels) region to remove the obviously developed areas and focus on the relatively undeveloped mountainous region. Fig. 7(b) shows the same image after processing with the standard RX anomaly detection algorithm10 with the results linearly stretched across the image. The original image was sectioned into 30 × 30 pixel tiles and the complexity of each tile was estimated using the procedure described above. In this case, due to the low spectral dimensionality of the data (four bands), only five endmembers per tile were extracted using the Max-D algorithm. Again, for each tile, the volume as a function of dimensionality was estimated and the maximum value was used as the measure of complexity for that tile. The tile brightnesses were linearly scaled and the resulting complexity image is Optical Engineering

Fig. 7 (a) DigitalGlobe Quickbird 4 band image near Cabazon, CA. RGB Bands shown here. (b) Results from processing with the RX anomaly detection algorithm. (c) The resulting complexity image identifying areas of interest. The white boxes highlight the region shown in Fig. 8.

shown in Fig. 7(c). The majority of the image has been darkened with a relatively small portion of the image highlighted by the complexity measure. The dry wash along the left side of the image is highlighted, as is a portion of the small city in the lower right hand corner. However, several smaller features in the image are identified in the remote regions of the mountains. The isolated bright tiles in the upper left corner are a region of obvious development when looking at the RGB image, but the area highlighted in the middle right portion of the complexity image is less obvious. This area is shown in more detail in Fig. 8 in the native RGB multispectral, after processing with the RX anomaly detection algorithm, and the tiled, complexity image. Here, we see that the complexity image generally identifies the road network in this portion of the image and in particular highlights a specific tile along the road. The RX result identifies

036201-6

Downloaded from SPIE Digital Library on 28 Mar 2012 to 129.21.58.44. Terms of Use: http://spiedl.org/terms

March 2012/Vol. 51(3)

Messinger et al.: Metrics of spectral image complexity with application : : :

Fig. 9 (a) High resolution panchromatic image of the area highlighted in Fig. 8. (b) Zoom of the panchromatic image of this area showing the presence of structures and possibly vehicles.

Fig. 8 (a) RGB zoom of the highlighted area in middle-right of complexity image in Fig. 7. (b) Results from processing with the RX anomaly detection algorithm. (c) Zoom of the tiled complexity image of this area.

several of these features as well, but additionally identifies a large number of individual pixels as anomalous. The co-temporal, high resolution panchromatic image of this area is shown in Fig. 9(a) and shows a region of manmade activity not obviously visible in the low resolution multispectral image. Fig. 9(b) highlights a specific region, the brightest tile in Fig. 8(c), containing structures and possibly vehicles. The material signatures of these manmade objects influence the local, relative, measure of complexity in this image. Consequently, this region of the image is highlighted and an analyst can be quickly cued to investigate the area. This is accomplished without knowing the signature of the target of interest and without investigating each pixel in the large area scene through visual inspection. Optical Engineering

4 Summary We have developed a new metric related to the “complexity” of a set of spectral image pixels. This metric is likely related to previous attempts to understand spectral image “quality” and “utility,” although here it is applied in a localized manner addressing the relative complexity of tiles within a large spectral image. The method developed here uses the mathematical concept of a convex set enclosing the data in the spectral hyperspace to assess complexity. A collection of pixels that are less complex will likely occupy a convex set in the hyperspace with a smaller overall volume, while more complex regions are more likely to have more diverse spectral materials. This will be manifested in the hyperspace through a larger convex set required to enclose the data. Here, the parallelotope volume is estimated as a function of dimensionality (i.e., number of endmembers used to describe the set) and the volume is computed using the Gram matrix. This volume estimation as a function of number of endmembers is a useful construct to understand the number of endmembers required to best model a set of pixels. References 20 and 21 demonstrate this use of the technique as applied to hyperspectral unmixing in very large area coverage scenes. The method described here has been applied to a large area hyperspectral image collected with the HyMAP sensor covering the Vis—NIR—SWIR spectrum. Results presented for image chips of increasing complexity, as assessed qualitatively, demonstrate the relationship between the metric and the scene content. Results from processing the entire flight line show that the metric

036201-7

Downloaded from SPIE Digital Library on 28 Mar 2012 to 129.21.58.44. Terms of Use: http://spiedl.org/terms

March 2012/Vol. 51(3)

Messinger et al.: Metrics of spectral image complexity with application : : :

correctly highlights regions within the image of higher complexity. The methodology was then applied to a large area, multispectral image collected with the Quickbird commercial remote sensing system over a mountainous region of California. The method correctly highlighted obvious features in the image, but was also able to identify smaller scale features not readily visible to the naked eye upon initial visual inspection. It is only through investigation of the co-temporal high resolution panchromatic image that surface features such as structures and vehicles are evident. It is proposed that the method could be implemented in large area collection platforms to sort data at the time of, or shortly after, collection to provide tipping and cueing information to image analysts tasked with exploiting large area image data sets. Note that this method is not an anomaly detection scheme and the different results from processing were demonstrated by comparison to the standard RX anomaly detection algorithm. There is no background model, such as the data mean and covariance used by RX, against which individual pixels are compared. Instead, it is directly quantifying the volume of the hyperspace required to enclose a set of pixels over spatially local scales, and ranks those regions based on the assessed complexity. The metric as employed returns a value for a collection of pixels, here image tiles, and does not function on a per-pixel basis. Also, note that because the metric relies on accurate extraction of endmembers from the collection of pixels under test it may be susceptible to noise or other artifacts in the data without sufficient preprocessing to mitigate these effects. The methodology presented here has also been applied to the problem of change detection22 where changes in the convex set volume as estimated here indicate changes in an image tile. Future work will continue to develop the methodology as an indicator of spectral image quality and utility. It is possible that methods such as this could provide the basis for spatially adaptive algorithms (classification, target detection, etc.) that utilize various approaches based on the local complexity. Additionally, as stated above this methodology provides a geometric means to estimate the appropriate number of endmembers required to best model a set of spectral pixels.20,21 Acknowledgments The authors wish to thank Dr. John Kerekes for providing the HyMAP hyperspectral data used in the study. For more information on the data please see http://dirsapps.cis.rit.edu/ blindtest/. The authors also wish to thank DigitalGlobe for the Quickbird imagery used in the study.

5. S. Shen, “Spectral quality equation relating collection parameters to object/anomaly detection performance,” Proc. SPIE 5093, 29–36 (2003). 6. J. Kerekes, A. Cisz, and R. Simmons, “A comparative evaluation of spectral quality metrics for hyperspectral imagery,” Proc. SPIE 5806, 469–480 (2005). 7. R. Simmons et al., “General spectral utility metric for spectral imagery,” Proc. SPIE 5806, SPIE, 457–468 (2005). 8. M. Stefanou and J. Kerekes, “A method for assessing spectral image utility,” IEEE Trans. Geosci. Rem. Sens. 47(6), 1698–1706 (2009). 9. D. W. Messinger et al., “Spectral image complexity estimated through local convex hull volume,” in WHISPERS 2010, Reykjavik, Iceland, IEEE (2010). 10. D. Manolakis, D. Marden, and G. Shaw, “Hyperspectral image processing for automatic target detection applications,” Lincoln Laboratory Journal 14(1), 79–116 (2003). 11. D. Gillis et al., “A generalized linear mixing model for hyperspectral imagery,” Proc. SPIE 6966, 69661B (2008). 12. P. Bajorski, E. J. Ientilucci, and J. R. Schott, “Geometric basis-vector selection methods and subpixel target detection as applied to hyperspectral imagery,” in International Geoscience and Remote Sensing Symposium (IGARSS) 2004, IEEE, Anchorage, Alaska, United States (2004). 13. J. R. Schott et al., “A Subpixel Target Detection Technique Based on the Invariance Approach,” in AVIRIS, 2003 AVIRIS Workshop, Pasadena, California (2003). 14. N. Balakrishnan, Advances on Theoretical and Methodological Aspects of Probability and Statistics, Taylor & Francis, New York, NY (2002). 15. L. Saul and S. Roweis, “Think globally, fit locally: unsupervised learning of low dimensional manifolds,” J. Mach. Learn. Res. 4, 119–155 (2003). 16. A. Kostrikin and Y. Manin, Linear Algebra and Geometry, Gordon & Breach, New York, NY (1989). 17. A. Hanson, Geometry for n-Dimensional Graphics, in Graphics Gems IV P. Heckbert, ed., Academic Press, San Diego, CA, pp. 149–170 (1994). 18. D. Snyder et al., “Development of a web-based application to evaluate target finding algorithms,” in International Geoscience and Remote Sensing Symposium (IGARSS) 2008, Boston, MA, IEEE (2008). 19. A. Schlamm, D. Messinger, and B. Basener, “Effect of manmade pixels on the inherent dimension of natural material distributions,” Proc. SPIE 7334, 73341K (2009). 20. K. Canham et al., “High spatial resolution hyperspectral spatially adaptive endmember selection and spectral unmixing,” Proc. SPIE 8048, 80481O (2011). 21. K. Canham et al., “Spatially adaptive hyperspectral endmember selection and spectral unmixing,” IEEE Trans. Geosci. Rem. Sens. 49(11) (November2011). 22. A. Ziemann, D. Messinger, and B. Basener, “Iterative convex hull volume estimation in hyperspectral imagery for change detection,” Proc. SPIE 7695, 76951I (2010).

David W. Messinger received a BS degree in physics from Clarkson University and a PhD in physics from Rensselaer Polytechnic Institute. He is currently an associate research professor in the Chester F. Carlson Center for Imaging Science at the Rochester Institute of Technology where he is the director of the Digital Imaging and Remote Sensing Laboratory. His research focuses on projects related to remotely sensed spectral image exploitation using physics-based approaches and advanced mathematical techniques. He is a member of SPIE.

References 1. M. Carlotto, “Detection and analysis of change in remotely sensed imagery with application to wide area surveillance,” IEEE Trans. Image Process 6(1), 189–202 (1997). 2. M. Carlotto, “A cluster-based approach for detecting man-made objects and changes in imagery,” IEEE Trans. Geosci. Rem. Sens. 43(2), 374–387 (2005). 3. M. Carlotto, “A synergistic exploitation concept for wide area search,” Proc. SPIE 6235, 623516 (2006). 4. J. Irvine, “National imagery interpretability rating scales (NIIRS): overview and methodology,” Proc. SPIE 3128, 93–103 (1997).

Optical Engineering

Amanda Ziemann received both the BS and MS degrees in applied mathematics from the School of Mathematical Sciences at Rochester Institute of Technology in 2010. She is now pursuing her PhD in imaging science also at RIT. Her research with the Digital Imaging and Remote Sensing Laboratory focuses on mathematical approaches to hyperspectral image processing and analysis. She is a member of SPIE.

036201-8

Downloaded from SPIE Digital Library on 28 Mar 2012 to 129.21.58.44. Terms of Use: http://spiedl.org/terms

March 2012/Vol. 51(3)

Messinger et al.: Metrics of spectral image complexity with application : : :

Bill Basener is a professor in the School of Mathematical Sciences at the Rochester Institute of Technology, co-director of the Center for Applied and Computational Mathematics, and co-director of the Consortium for Mathematical Methods in Counterterrorism. He received his PhD from Boston University and his research includes the textbook Topology and Its Applications. He is a member of SPIE.

Optical Engineering

Ariel Schlamm received a BS degree in imaging and photographic technology from Rochester Institute of Technology in 2006 and her PhD in Imaging Science from RIT in 2010. She is currently employed by The MITRE Corporation. She is a member of SPIE.

036201-9

Downloaded from SPIE Digital Library on 28 Mar 2012 to 129.21.58.44. Terms of Use: http://spiedl.org/terms

March 2012/Vol. 51(3)