Non-linear Spectral Unmixing by Geodesic ... - Semantic Scholar

1

Non-linear Spectral Unmixing by Geodesic Simplex Volume Maximization Rob Heylen, Member, IEEE, Dˇzevdet Burazerovi´c, Member, IEEE and Paul Scheunders, Member, IEEE

Abstract—Spectral mixtures observed in hyperspectral imagery often display non-linear mixing effects. Since most traditional unmixing techniques are based upon the linear mixing model, they perform poorly in finding the correct endmembers and their abundances in the case of non-linear spectral mixing. In this paper, we present an unmixing algorithm that is capable of extracting endmembers and determining their abundances in hyperspectral imagery under non-linear mixing assumptions. The algorithm is based upon simplex volume maximization, and uses shortest-path distances in a nearest-neighbor graph in spectral space, hereby respecting the non-trivial geometry of the data manifold in the case of non-linearly mixed pixels. We demonstrate the algorithm on an artificial data set, the AVIRIS Cuprite data set, and a hyperspectral image of a heathland area in Belgium. Index Terms—Hyperspectral imaging, Spectral analysis, Manifolds

I. I NTRODUCTION One of the inescapable implications of hyperspectral remote sensing is that a single pixel can often record only a mixed signature of distinct surface materials. This creates the need for unmixing [1], or decomposition of the observed pixel spectrum into its constituent spectra, ideally corresponding to individual materials. Besides identifying these pure spectra, or endmembers, a second important aspect is the estimation of their respective abundances in each observed pixel. Spectral unmixing is generally done under the assumption of linear mixing: the spectrum of each pixel consists of a linear combination of endmembers, with abundances that are positive and sum to one. In practice, such a model fits the situation where the endmember materials appear in the pixel as spatially segregated regions. This interpretation also yields a physical explanation for the constraints on the abundances: Any given material cannot have a negative contribution to a pixel, and the sum of all contributions has to equal one. In the literature, the linear mixture model has been prevalently used, and continues to be a popular starting point in formulating general frameworks [2] and dealing with specific problems [3], [4]. However, one often encounters situations where the linear mixing model is no longer adequate for describing the spectral mixing effects. Examples are secondary and higher-order reflections, shallow water environments (an example is shown in Fig. 1), intricate mineral mixtures [1], ... Unmixing in these cases has often been handled by extensively modeling the source of the non-linear effects (e.g. multiple reflectance and scattering [5], [6]), or by employing more model-independent R. Heylen, D. Burazerovi´c and P. Scheunders are with the IBBT-Visielab, University of Antwerp, Universiteitsplein 1, 2610 Wilrijk, Belgium

Fig. 1. Scatter plot of band 10 (710 nm) and band 16 (884 nm) of partly submerged grassland. The data manifold has a highly nontrivial shape, and does not resemble a linear simplex, indicating complex non-linear mixing interactions are present.

methods for dealing with non-linearity (e.g. kernel-based processing [7] and artificial neural networks [8], [9], [10], [11]). In [12] one proposed a combined approach, as it used the linear mixture model to find the endmembers, and a neural network to refine their abundances. An alternative strategy for dealing with non-linearities in hyperspectral data sets is performing a non-linear dimensionality reduction, yielding a linear space of reduced dimensionality, followed by traditional algorithms based on the linear mixing assumption. Most non-linear dimensionality reduction algorithms are data-driven and unsupervised, and use a geometrically oriented approach based on manifold learning [13]. Most of these approaches consist of a mapping that preserves some global or local relationship from a high-dimensional manifold (constituted by the source data) while projecting it to a lowerdimensional linear space. The subsequent linear operations may relate to any of the conventional unmixing, classification or compression techniques often performed on hyperspectral data, yielding a two-step process for coping with non-linear data sets. For instance, several non-linear projections (Local Linear Embedding [14], Laplacian Eigenmaps [15] and Local Tangent Space Alignment [16]) were recently compared as a precursor to a K-nearest neighbor classifier [17]. ISOMAP has been used as a preprocessing step in classification problems [18], and in combination with linear unmixing for target detection [19]. An important disadvantage of most non-linear dimensionality reduction techniques is their high computational cost and memory requirements, making then rather impractical for use with sizable hyperspectral scenes. This problem was

2

acknowledged in [18], [20], where some strategies were proposed for realizing a scalable operation of ISOMAP. The scalability was there mostly achieved by aligning parallel executions of ISOMAP on image tiles, while streamlining the computation of geodesic distances and definition of local neighborhoods on the manifold. The idea of tile alignment was also followed in [21], where ISOMAP was effectively replaced by a coordinate representation derived from diffusion maps. Other methods resort to explicit use of supervision, i.e. by allowing the manifold structure to be used as input in off-line learning of a classification model [22]. Despite these improvements, it can be fairly observed that non-linear dimensionality reduction of large hyperspectral data is usually only proposed for applications where sizable non-linear effects can be expected beforehand (e.g. in bathymetry [23]). In this paper, we present a new method for the spectral unmixing of non-linearly mixed hyperspectral data. We first reformulate the popular N-FINDR algorithm for linear unmixing in terms of distance geometry, where all properties are expressed in terms of pairwise distances. We derive distancebased equations for the calculation of simplex volumes and for solving the abundance estimation problem, while obeying some of the constraints on the abundances. Next, we introduce geodesic distances, defined as shortest-path distances along a nearest-neighbor graph in the data set. Under certain curvature conditions of the data manifold, we can use these geodesic distances in the distance-based unmixing algorithm so that the unmixing is performed taking the structure of the data manifold into account. This yields an unmixing algorithm that is capable of non-linear unmixing, and has a much better performance than the computationally-intensive solution where some standard linear unmixing algorithm would be simply preceded by a non-linear dimensionality reduction. The remainder of this paper is organized as follows: In section II, we first describe the differences between linear and non-linear spectral mixing. Then we introduce an algorithm that is able to unmix linearly mixed spectral data, and explain how such an algorithm can be extended to deal with data that is not linearly mixed. This is accomplished by introducing geodesic distances on the data manifold, approximated by shortest-path distances on a nearest-neighbor graph. Next, we formally present the algorithm, followed by a word on the algorithm parameters and the computational complexity. Section III contains the results: III-A describes the performance of the proposed non-linear algorithm when executed on a simulated non-linear hyperspectral data set. The results are compared to those obtained with a linear unmixing algorithm: the original N-FINDR algorithm combined with fully constrained leastsquares linear unmixing to determine the abundances. In subsection III-B, we execute the algorithm on the well-known AVIRIS Cuprite data set, and again compare the results with N-FINDR. Since the Cuprite data set can be well modeled via linear mixtures, we also present a hyperspectral data set obtained over a heathland area in Belgium, which contains significant non-linear mixing effects. The results obtained by our non-linear algorithm and the original N-FINDR algorithm on this data set are presented in subsection III-C. Section IV contains the conclusions, and future work.

II. G EODESIC SIMPLEX VOLUME MAXIMIZATION A. Non-linear mixing Suppose we have a hyperspectral image, described by a data set of N points in a d-dimensional spectral space: {x1 , x2 , . . . , xN }. Furthermore, suppose there are p endmembers present in the data set, identified by the index set {e(1), . . . , e(p)}: ei = xe(i) , i = 1, . . . , p. These endmembers represent pure spectra, corresponding to pixels containing a single surface material. The assumption that a pure pixel is present for every endmember is common in several endmember extraction algorithms (see e.g. [24] for a comparative study of 6 popular endmember extraction algorithms where this assumption is made), but is often not valid in practice. In this case, a pixel that is close to the endmember in spectral angle will be considered as candidate endmember. In the linear mixing model, the endmembers form a linear basis for the pixel spectra, with coordinates that are positive and sum to one (barycentric coordinates). These coordinates then correspond to the fractional abundance of every endmember in that pixel: xi =

p ∑

aij ej

(1)

j=1 p ∑

aij = 1,

∀i, j : aij ≥ 0

j=1

With this mixing equation, the data manifold will form a simplex spanned by the p endmembers in the d-dimensional spectral space. Many unmixing algorithms exploit this geometric notion (N-FINDR [25], [26], pixel purity index [27], simplex growing algorithm [28], convex cone analysis [29], ...), and search for embedded or enclosing simplices in the data. However, when the spectral mixing happens non-linearly, this geometric notion fails. Generally, every pixel spectrum is now a non-linear function of the abundance coefficients and the endmembers. We can still assume certain continuity conditions however: a pixel with a very large abundance for a given endmember ei , and almost zero abundances for all other endmembers, will have a spectrum that lies close to the spectrum of endmember ei in spectral space. Furthermore, when the abundance coefficients vary continuously from one set to another, we also assume that the corresponding pixel spectrum will vary continuously from an initial spectrum to a final spectrum. If this were not the case, one would observe discrete jumps in the observed spectra at certain sets of abundances, which does not seem physically plausible. One way to model such a non-linearity is to assume a non-linear but continuous bijective mapping F between the linear space of abundance coefficients and the spectral space:   p ∑ xi = F  aij ej  (2) j=1

This mapping F induces a manifold in spectral space, composed of the continuous projection of a linear simplex. In practice, we consider this manifold to resemble a non-linearly

3

transformed low-dimensional simplex embedded in the highdimensional spectral space, so that the endmembers still form the vertices or extreme points of the data manifold. The presented algorithm is a data-driven approach for endmember extraction and abundance estimation, which takes this manifold structure into account. The main idea is to express a linear unmixing algorithm in terms of distances in spectral space, and then use geodesic distances along the data manifold in this algorithm. B. Simplex volume calculation The basis of the algorithm is the linear unmixing algorithm N-FINDR [25]. In this algorithm, one searches for the simplex of largest volume in the data set. The data points that span this simplex then are the endmembers, under the assumption that a pure pixel is present in the data for every endmember, and that (1) holds. The volume V of a simplex spanned by points x1 , . . . , xp can be written in terms of the inter-vertex distances using the Cayley-Menger determinant. Let dij be the Euclidean distance between xi and xj : (−1)p 2p−1 ((p − 1)!)2 V 2

=

C 1,2,...,p

=

det(C 1,2,...,p ) ( 2 ) D 1,2,...,p 1 1 0

[ ] with D 21,2,...,p = d2ij i,j=1,2,...,p . We can rewrite (3) as ( ) T det(C 1,2,...,p ) = − d1 C −1 2,3,...,p d1 det(C 2,3,...,p )

(3) (4)

(5)

(d212 , . . . , d21p , 1).

with d1 = Using (3) and (5), one can derive that the orthogonal distance from a point x1 to the plane spanned by vertices (x2 , . . . , xp ) is given by ( ) 12 T d1 C −1 2,3,...,p d1 d⊥ (x1 ; x2 , . . . , xp ) = (6) 2 This equation allows us to orthogonally project a point x1 onto the plane spanned by (x2 , . . . , xp ), yielding a point x01 expressed in terms of its distances to the points x2 , . . . , xp : d2 (x01 , xi ) = d2 (x1 , xi ) − d2⊥ (x1 ; x2 , . . . , xp )

(7)

C. Introducing non-linearity The distances used in the simplex volume calculation are Euclidean distances. However, by using geodesic distances measured on the data manifold instead, one can use (3) as an estimation of the geodesic volume one would find if the volume was to be measured along this manifold. We can still use (3) for determining the volume of the simplex along a non-linear data manifold as long as this data manifold can be covered by Euclidean space. This is the case when the data manifold is completely flat, or has (intrinsic or Riemann) curvature zero in every point. In the remainder of this text, we assume that the data manifold is indeed completely flat. Intuitively, this corresponds to a data manifold that can be created by embedding a (subset of a) low-dimensional Euclidean space into a higher dimensional space by folding

or deforming this low-dimensional space, however without ”stretching”. In real non-linear data sets, this is often not the case. However, we expect the algorithm to still perform well when the manifold has a small non-zero curvature. In practice, we did not encounter situations where this posed a problem. A well-known data-driven approach for approximating geodesic distances on a manifold is constructing a nearestneighbor graph on the data, and measuring shortest-path distances along this graph. To generate such a graph, calculate the Euclidean distance between any two points xi and xj , and connect every point to the K nearest points, with K a parameter of the algorithm. The weight of every edge is the corresponding Euclidean distance. The graph needs to be symmetrized and connected. We then define the geodesic distance between two points as the shortest-path distance along the weighted graph between these two points. The Dijkstra algorithm [30] can be used to calculate the shortest-path distances from a point to all other points. By defining distances in this way, these distances will approximate the true geodesic distances as measured along the surface of the data manifold. D. The endmember extraction algorithm To find the largest-volume simplex along the data manifold, use the following algorithm: 1) Construct a weighted, symmetrical and connected Knearest neighbor graph on the data set. 2) Select randomly p points as initial vertices. Calculate the shortest-path distances from these p vertices to all other points with the Dijkstra algorithm. Use (3) to determine the simplex volume. 3) Pick a random point x, and calculate the simplex volume when any vertex is replaced by x. If a larger simplex is found, keep the new vertex x, calculate the distance of x to all other points with the Dijkstra algorithm, and recalculate all modified matrix identities in (5). 4) Perform the previous step until no larger simplex is found for any point. The final vertices are now known. This algorithm provides us with the vertices (xe(1) , . . . , xe(p) ) that span the largest-volume simplex, and the geodesic distances from these vertices to all other points. There are two parameters of the algorithm that still need to be determined: The number of endmembers p, and the connectivity of the graph K. Determining the exact number of endmembers in a hyperspectral image is a highly nontrivial task. In many seminal papers on endmember extraction, such as N-FINDR [25] and pixel purity index [27], one uses minimum noise fraction (MNF) or principal component analysis (PCA) to reduce the dimensionality of the data. By choosing an appropriate threshold for the eigenvalues of the dimensionality reduction, an appropriate corresponding number of endmembers is found. However, the problem is then moved from choosing the number of endmembers to choosing an appropriate threshold. More recently, a new technique called virtual dimensionality [31] has been successfully employed to more accurately estimate the number of endmembers needed to span a given data set.

4

Unfortunately most of these techniques are formulated in such a way that they cannot be used straightforwardly with distance geometry. One method that can be easily adapted to be used with distance geometry however is looking at the maximum simplex volumes: If a data set lies on a pdimensional plane (or simplex) in a higher-dimensional space, then any simplex of p + 2 points or more has zero volume. This means that if one plots the maximum simplex volumes for increasing p, one should see a sharp jump in these volumes at a given p, indicating the dimensionality of the data. In practice however this technique has several drawbacks: It is very computationally intensive, and due to noise and deviations in the distance matrix, the simplex volumes of any dimension are non-zero, which obscures the expected discontinuity and makes it impractical to use this method in reality. In the experiments performed further, we have chosen appropriate values for p by hand. The second parameter of the algorithm is the connectivity constant K. This constant cannot be too small, or one ends up with disconnected clusters in the graph, which obstructs the creation of (shortest) paths between such clusters. Artificially connecting disconnected clusters usually destroys much of the information on the manifold structure present in the distance matrix, due to the creation of shortcuts. On the other hand, a value for K that is too high will smoothen possible fine structures present in the data manifold, and here one looses information as well. In practice, we have observed that a connectivity K = 20 usually yields good results, and has never caused the problem of disconnected clusters in the data sets we used. E. Abundance estimation Once the endmembers have been found, the next step in the spectral unmixing algorithm is the estimation of the abundances of each endmember in each pixel. In the linear mixing assumption, this abundance estimation can be done by solving the linear mixing equation (1) for the abundance coefficients, yielding an over-determined set of equations. Since in general this equation cannot be satisfied for every data point due to noise or erroneous endmember estimation, one tries to minimize the reconstruction error between the data point and the point recreated from the endmembers and extracted abundances. This yields a least-squares problem, where the abundance coefficients need to obey the positivity and sumto-one constraints. In the original N-FINDR algorithm, and many other linear unmixing algorithms, this problem is solved by employing constrained least-squares techniques, obeying some or all of the constraints on the abundance coefficients depending on the implementation. If all the constraints are obeyed, one obtains a fully constrained least-squares solution for the linear mixing problem [32]. In the non-linear case however, every data point is expressed in terms of its distances to the endmembers instead of coordinates in spectral space, and we need to employ another technique to estimate the abundances. Because we assumed that the data manifold is completely flat, the mutual distances between the data points still obey all Euclidean

e1

e2 V3 V2

x V1

e3 Fig. 2. A point x lies inside a simplex S3 spanned by {e1 , e2 , eP 3 }. The abundance coefficient ai equals the volume ratio Vi /V , with V = i Vi .

geometric properties. Hence, to estimate the abundances, we require a technique for linear unmixing expressed solely in distance geometry, preferably also obeying the constraints on the abundances. We introduce a technique that allows us to find the abundances from the mutual distances, and that obeys the positivity constraint. It is based upon the following observation: The abundance coefficients of a point x constructed via the linear mixing equation (1) can be written as ai =

V(e1 ,...,ei−1 ,x,ei+1 ,...,ep ) V(e1 ,...,ep )

(8)

or ai is the volume of the simplex obtained by replacing the i-th vertex by x, divided by the total volume of the largest simplex (see Fig. 2). This property arises from the fact that the abundance coefficients effectively play the role of homogeneous barycentric coordinates in the endmember coordinate system, and the equivalency between homogeneous barycentric and areal coordinates in simplices [33]. The volumes in this expression can be calculated in terms of interpoint distances with (3). Because volumes are per definition positive, the positivity constraint on the abundances is automatically satisfied. When the point lies inside the simplex, also the sum-to-one constraint will be satisfied. However, if the point lies outside the simplex (due to noise or bad endmember selection), and hence cannot be written with (1), the sum-to-one constraint will be violated if (8) is used to determine the abundances. In order to use equation (8), the point x to unmix has to lie in the (p − 1)dimensional plane spanned by the p endmembers. This can be accomplished by removing possible orthogonal components with equation (7). F. Complexity In some hyperspectral classification algorithms that are able to deal with nonlinearly mixed data [18], one uses the ISOMAP non-linear dimensionality reduction algorithm [34] as a preprocessing step before executing a linear classification algorithm. An analogous algorithm for unmixing would then be obtained by replacing the linear classification algorithm by a linear unmixing algorithm. Only very recently, such an algorithm has been proposed in [19], where a combination of the ISOMAP and N-FINDR algorithm is used for the purpose

5

Fig. 4. The average minimum spectral angle A as a function of σ, with the N-FINDR algorithm (solid line), and the non-linear algorithm with K = 20 (circles). Fig. 3. The artificial data set for σ = 0.5 (left) and σ = π (right), for 5000 randomly generated abundances, and color-coded by the value of a1 .

of target detection. If the distances between all points in the linear lower-dimensional projected space correspond exactly with the geodesic distances in the higher-dimensional spectral space, such an algorithm would yield exactly the same results as our presented algorithm. However, even in that case, one large difference of such an approach with the presented algorithm is the computational complexity: To calculate the low-dimensional representation in ISOMAP, one requires the complete geodesic distance matrix. In the current case, this would mean N executions of the Dijkstra algorithm (which is a O(N log(N )) algorithm in itself), and a O(N 2 ) memory requirement. The presented algorithm however only calculates the distances from the initial p endmember candidates to all other points, followed by another execution of the distance calculation every time a larger simplex is found. Since the number of times a larger simplex is found is only a small fraction of N , the number of iterations of the Dijkstra algorithm required before finishing the algorithm is much smaller than N . Furthermore, only the distances from the p endmembers to all other points need to be stored at any point during the algorithm, reducing the memory requirements by a factor N/p. These observations allow us to execute the algorithm on large data sets without the need for segmentation or divide-conquer-merge strategies, and obtain results comparable to the results obtained by the two-step process of non-linear dimensionality reduction followed by a linear algorithm. III. R ESULTS A. Artificial dataset We have created an artificial data set that consists of random samples in a two-dimensional linear simplex, transformed into a three-dimensional non-linear manifold by projecting it on a Swiss roll structure. The non-linearity of the data set is parametrized by a variable σ. The endmembers are also added to the data set. See Fig. 3 for an example of this data set for σ = 0.5 and σ = π, for 5000 randomly generated abundances:

xi yi zi

= ai1 sin(σai1 ) + 1 = ai1 cos(σai1 ) + 1 = ai2 + 1

(9) (10) (11)

The endmembers can be found analytically from this expression by setting a single abundance aij equal to one, while setting the other abundances to zero: e1 e2 e3

= (sin(σ) + 1, cos(σ) + 1, 1) = (1, 1, 2) = (1, 1, 1)

(12) (13) (14)

We unmix this data set using p = 3 endmembers, with two different algorithms: The proposed non-linear algorithm, and a linear unmixing algorithm consisting of the original implementation of N-FINDR for locating the endmembers, followed by fully constrained least-squares linear unmixing for finding the abundances. This latter algorithm will be referred to as ”the N-FINDR algorithm” in the remainder of the text. In the first experiment, we search for the endmembers with both algorithms, and for each found endmember e0i , we calculate the minimum of the spectral angles with the known endmembers (12)-(14). This quantity is then averaged over all endmembers, and provides a measure for the average deviation of the found endmembers from the actual endmembers: ) ( 0 p 1∑ ei · ej A= (15) min arccos p i=1 j ke0i kkej k This average minimum spectral angle A is plotted in Fig. 4 for several values of the parameter σ. One can clearly see that the non-linear algorithm retrieves all endmembers perfectly for the entire range of σ. The N-FINDR algorithm fails to correctly retrieve all endmembers once σ becomes larger than 2, and the errors quickly increase with increasing σ. As a second experiment, we use the retrieved endmembers to determine the abundance maps, with the fully-constrained least-squares technique for the linear case, and the simplex volume ratios with geodesic distances in the non-linear case. The obtained abundance maps a0i are then compared to the

6

Fig. 5. The averaged absolute error E on the abundances as a function of σ, with the N-FINDR algorithm (solid line), and the non-linear algorithm with K = 20 (circles).

actual known abundance maps ai , and the following error measure is calculated: ( ) p N 1∑ 1 ∑ 0 E= min |aik − ajk | (16) p i=1 j N

Fig. 6. False-color image of the AVIRIS Cuprite data set. Bands (36,22,12) are chosen as RGB values, corresponding to (2.34, 2.20, 2.10) µm.

k=1

This error measure is plotted in Fig. 5 as a function of σ. For low σ, the data manifold does not deviate much from a linear simplex, and both algorithms perform well. For higher values of σ, the abundance maps obtained with either method start showing significant deviations from the actual abundance maps. The non-linear algorithm however performs clearly better. For values of σ larger than 2, the linear algorithm fails to correctly retrieve all endmembers, further exacerbating the quality of the abundance maps. We have regenerated these results for all values of the graph parameter K in the range 3−20. For K in the range 5−20 we have not found significant differences in the results, neither for endmember retrieval nor for abundance estimation. For K = 3 and K = 4 however, the algorithm performs worse. For these connectivities, the differences between the shortest-path and geodesic distances becomes too large. B. AVIRIS Cuprite data set The experiments on an artificial data set in previous section demonstrate that the algorithm can perform much better than a linear unmixing algorithm when the data manifold contains large non-linearities. Previous data set however was a highly idealized and unrealistic model of an actual hyperspectral scene. In this section, we compare the performance of the nonlinear algorithm to the N-FINDR algorithm on a well-known hyperspectral scene: The AVIRIS Cuprite data set, obtained over the Cuprite mining region in Nevada, USA. We have selected a 350 × 400 pixel subset from the 1995 reflectance data set, pruned to contain only 50 spectral bands in the range 1.99 - 2.48 µm, since this wavelength range allows for better discrimination of mineral signatures [35]. A similar data set was used in the original paper on N-FINDR [25]. This data set is also available in the ENVI software under the name ”cup95eff”. A false-color image of the region is shown in Fig. (6).

Fig. 7. Three (out of p = 14) extracted endmembers (dots) as found with the N-FINDR algorithm, and the library spectra of smallest spectral angle (solid line). Top: Kaolinite. Middle: Montmorrilonite. Bottom: Alunite. The spectral angles are 0.056, 0.048 and 0.043 respectively.

As a first experiment, we determine the endmembers with the N-FINDR algorithm and the proposed non-linear algorithm, and compare them to the mineral spectra of the freely available USGS ”splib04c” spectral database. Every endmember is identified with the library spectrum of smallest spectral angle. Both algorithms were run for all values of the number of endmembers p in the range [8, 16]. We noticed that the minerals that are most prevalently present in the scene [25], such as alunite, kaolinite, and calcite, were detected as endmember for any of these values for p, for both versions of the algorithm. Other minerals that are present in the scene to a lesser extent, such as buddingtonite, muscovite and montmorrilonite, where detected as endmembers for some of the values of p, but not all. There were no specific values of p that performed significantly better, for neither of the two algorithms. In Fig. 7 we present three of the endmembers obtained with the N-FINDR algorithm, out of a total of 14 extracted endmembers: Kaolinite, montmorillonite and alunite. The other endmembers are not displayed for brevity, but have been identified with alunite as well, two calcite endmembers, butlerite,

7

Fig. 8. Three (out of p = 16) extracted endmembers (dots) as found with the non-linear algorithm with K = 20, and the library spectra of smallest spectral angle (solid line). Top: Kaolinite. Middle: Montmorrilonite. Bottom: Alunite. The spectral angles are 0.070, 0.049 and 0.056 respectively.

gaylussite, augite, pyrite, ammoniummillsmec, lepidolite, dumotierite and a shade endmember, although the spectral match with several of the latter is not so good due to noise present in some endmember pixels. Several of these minerals are known to be present in the scene, and these results can also be found in [25]. The equivalent results for the non-linear algorithm are presented in Fig. 8. We have again plotted the results for kaolinite, montmorillonite and alunite, illustrating clearly that the nonlinear algorithm is capable of retrieving correct endmembers as well. We have chosen a connectivity constant K = 20, and p = 16 endmembers. Some of the other endmembers extracted with good spectral matches were calcite, diaspore, lepidolite, vermiculite, rutile, and andradite. Next, we have calculated the abundance maps for both algorithms. The abundance map for the endmember identified as alunite is shown in Fig. 9, for the N-FINDR algorithm (using fully constrained least-squares linear unmixing) and for the non-linear algorithm (using volume ratios with geodesic distances). The linear abundance map can be compared to those found in e.g. [25], and shows exact agreement. The nonlinear map shows large similarities with the linear abundance map, but has many pixels with higher abundances, due to the non-linear treatment. The non-linear unmixing result also displays some salt-and-pepper noise, probably caused by the sumto-one constraint that is not always satisfied in the used unmixing technique, causing some non-zero abundances that should ideally be zero. However, the lack of pixel-wise ground-truth data on the abundances restricts us from performing a real quantitative comparison between the abundance maps obtained with both methods. C. Heathland data set We have tested the algorithm on a hyperspectral image from a heathland area in Belgium called “Kalmthoutse heide”. The total image size is 4509 × 4359 pixels, with a pixel size of approximately 2.4 × 2.4m2 , but we focus only on a part of the total image due to computational constraints. The image consists of 63 bands in the wavelength range 0.456 - 2.55 µm, but there are some noise bands: band 22-24 and 56-63. We use

Fig. 9. The abundance maps for the alunite endmember. Top: The N-FINDR algorithm. Bottom: The non-linear algorithm.

all bands in the algorithm except the noise bands, resulting in 52 usable bands. For testing our algorithm, we took a subset of 500 × 500 pixels from the image, containing representative features for the entire image. This image is plotted in Fig. 10, and contains partly submerged grassland, forest, heathland, arable land, sand dunes and water bodies. The scatter plot in Fig. 1 is also taken from this data set, and already hints at the presence of large non-linearities. There is classification ground truth available for this data set: about 2000 points were determined by field measurements, resulting in 24 different classes. From this ground truth, classification maps have been created [36]. These classification maps can be helpful in identifying some of the endmembers and their respective abundance maps yielded by spectral unmixing algorithms. However, no one-to-one correspondence between endmembers and classes can be expected. Since we have no ground truth for the endmembers available, we will not attempt to identify the endmembers by

8

(a) Classification map of the class “grassland”. Fig. 10. The region under consideration from the “Kalmthoutse heide” data set as a RGB color image.

examining their spectra. Instead, we search for the endmembers completely unsupervised (without identification), generate their respective abundance maps, and study these. The number of endmembers to be extracted is empirically chosen p = 16 for both the N-FINDR algorithm, and the non-linear algorithm. A typical result is shown in Fig. 11: We have generated the abundance maps for all 16 endmembers with both algorithms, and picked out two maps that are visually very similar. Both abundance maps seem to show high agreement with the classification map for ”grassland”. The abundance map obtained by the non-linear algorithm shows better agreement with the classification map. Especially in the upper-left corner, where there is mixing of water and grassland present, one notices that the abundance map obtained by the non-linear algorithm consistently shows high abundances, while the abundances obtained by the linear algorithm are lower. Because this area contains large non-linear mixing effects (see the scatter plot of this area in Fig. 1), one can expect such deviations in a linear unmixing algorithm.

(b) Abundance map for the linear algorithm.

IV. C ONCLUSIONS AND FUTURE WORK . We have introduced a new method for non-linear unmixing of hyperspectral imagery, by combining a geometrical algorithm based upon simplex volume maximization with geodesic distances in the data manifold. These geodesic distances are approximated by shortest-path distances in a nearest-neighbor graph on the data. We have compared the performance of the algorithm with linear unmixing algorithms by execution on artificial data sets, and on real hyperspectral data sets of the AVIRIS Cuprite mining region, and of a heathland area in Belgium called “Kalmthoutse heide”. The results obtained on artificial data sets show promising improvements over linear unmixing. On the AVIRIS Cuprite data set, the differences between linear and non-linear unmixing are hard to

(c) Abundance map for the non-linear algorithm. Fig. 11.

Classification and abundance maps for the “grassland” class.

9

quantify due to the lack of ground-truth data. The Kalmthoutse Heide data set however shows improvement of the non-linear algorithm over the linear algorithm, on the level of comparing abundance maps with known classification maps. Furthermore, the computational requirements of the presented algorithm are such that execution on large data sets is feasible with regular hardware, as opposed to the alternative approach of a non-linear dimensionality reduction preprocessing step followed by a linear algorithm. Future work involves testing of the algorithm on non-linear mixtures of minerals with known abundances, to better assess the performance of the abundance estimation. We are also looking into techniques to include the sum-to-one constraint in the abundance estimation algorithm, to obtain a fullyconstrained least-squares unmixing algorithm expressed completely in distance geometry. Determining a solid mechanism for estimating the required number of endmembers in the case of non-linear mixtures is another goal. ACKNOWLEDGEMENTS We are indepted to the Flemish Institute for Technological Research (VITO) for acquisition and preprocessing of the ”Kalmthoutse heide” data set, and to the Research Institute for Nature and Forest (INBO) for the field survey and composition of the classification scheme, under the framework of the STEREO programme project HABISTAT. R EFERENCES [1] N. Keshava and J.F. Mustard, “Spectral unmixing,” IEEE Sig. Proc. Mag., vol. 19, pp. 44–57, 2002. [2] N. Dobigeon, S. Moussaoui, M. Coulon, J.-Y. Tourneret, and A. O. Hero, “Joint Bayesian endmember extraction and linear unmixing for hyperspectral imagery,” IEEE Trans. Signal Process., vol. 57, no. 11, pp. 4355–4368, 2009. [3] B. Somers, S. Delalieux, J. Stuckens, W.W. Verstraeten, and P. Coppin, “A weighted linear spectral mixture analysis approach to address endmember variability in agricultural production systems,” Int. J. Remote Sens., vol. 30, no. 1, pp. 139–147, 2009. [4] B. Luo, J. Channusot, and S. Doute, “Unsupervised endmember extraction: Application to hyperspectral images from Mars,” Proc. IEEE Int. Conf. Image Processing, pp. 2869–2872, 2009. [5] B. Somers, K. Cools, S. Delalieux, J. Stuckens, D. Van der Zande, W.W. Verstraeten, and P. Coppin, “Nonlinear hyperspectral image analysis for tree cover estimates in orchards,” Remote sens. environ., vol. 113, pp. 1183–1193, 2009. [6] K. Arai, “Nonlinear mixture model of mixed pixels in remote sensing satellite images based on Monte Carlo simulation,” Adv. Space Research, vol. 41, pp. 1715–1723, 2008. [7] J. Broadwater and A. Banerjee, “A comparison of kernel functions for intimate mixture models,” 1st Workshop Hyperspectral Image and Signal Processing: Evolution in Remote Sensing, pp. 1–4, 2009. [8] J.M.P. Nascimento and J.M. Bioucas-Dias, “Nonlinear mixture model for hyperspectral unmixing,” Proc. SPIE, vol. 7477, pp. 74770I, 2009. [9] J. Plaza, A. Plaza, R. P´erez, and P. Mart´ınez, “On the use of small training sets for neural network-based characterization of mixed pixels in remotely sensed hyperspectral images,” Pattern Recognition, vol. 42, no. 11, pp. 3032–3045, 2009. [10] W. Liu and E.Y. Wu, “Comparison of non-linear mixture models: subpixel classification,” Remote sens. environ., vol. 94, no. 2, pp. 145–154, 2005. [11] K.J. Guilfoyle, M.L. Althouse, and C.-I Chang, “A quantitative and comparative analysis of linear and nonlinear spectral mixture models using radial basis function neural networks,” IEEE Trans. Geosci. Remote Sensing, vol. 39, no. 10, pp. 2314–2318, 2001. [12] J. Plaza, A. Plaza, R. P´erez, and P. Mart´ınez, “Joint linear/nonlinear spectral unmixing of hyperspectral image data,” IEEE Int. Geoscience Remote Sensing Symp., pp. 4037–4040, 2008.

[13] M. Gashler, D. Ventura, and T. Martinez, “Iterative non-linear dimensionality reduction with manifold sculpting,” Advances in Neural Information Processing Systems, vol. 20, pp. 513–520, 2008. [14] S.T. Roweis and L.K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science, vol. 290, no. 5500, pp. 2323–2326, 2000. [15] M. Belkin and P. Niyogi, “Laplacian eigenmaps and spectral techniques for embedding and clustering,” Advances in Neural Information Processing Systems, vol. 14, pp. 586–691, 2001. [16] Z. Zhang and H. Zha, “Principal manifolds and nonlinear dimensionality reduction via local tangent space alignment,” SIAM J. Sci. Comput., vol. 26, no. 1, pp. 313–338, 2004. [17] B. Wu, Y. Zhou, L. Yan, J. Yuan, R. Li, and D.L. Wang, “Object detection from HS/MS and multi-platform remote sensing imagery by the integration of biologically and geometrically inspired approaches,” Proc. ASPRS Ann. Conf., Baltimore, Maryland, 2009. [18] C.M. Bachmann, T.L. Ainsworth, and R.A. Fusina, “Exploiting manifold geometry in hyperspectral imagery,” IEEE Trans. Geosci. Remote Sensing, vol. 43, no. 3, pp. 441–454, 2005. [19] A. Zare, T. Dubroca, P. Gader, and R. Hummel, “Explosives detection using non-linear spectral unmixing on hyperspectral differential reflectometry data,” IEEE Trans. Syst., Man, Cybern., (submitted for publication), 2010. [20] C.M. Bachmann, T.L. Ainsworth, and R.A. Fusina, “Improved manifold coordinate representations of large-scale hyperspectral scenes,” IEEE Trans. Geosci. Remote Sensing, vol. 44, no. 10, pp. 2786–2803, 2006. [21] J. He, L. Zhang, Q. Wang, and Z. Li, “Using diffusion geometric coordinate representation for hyperspectral imagery representation,” IEEE Geosci. Remote Sensing Letters, vol. 6, pp. 767–771, 2009. [22] X.R. Wang, S. Kumar, F. Ramos, T. Kaupp, B. Upcroft, and H. DurrantWhyte, “Probabilistic classification of hyperspectral images by learning nonlinear dimensionality reduction mapping,” Proc. 9th Intl. Conf. Information Fusion, pp. 1–8, 2006. [23] C.M. Bachmann, T.L. Ainsworth, R.A. Fusina, M.J. Montes, J.H. Bowles, D.R. Korwan, and D.B. Gills, “Bathymetric retrieval from hyperspectral imagery using manifold coordinate representations,” IEEE Trans. Geosci. Remote Sensing, vol. 47, no. 3, pp. 884–897, 2009. [24] A. Plaza, P. Mart´ınez, R. P´erez, and J. Plaza, “A quantitative and comparative analysis of endmember extraction algorithms from hyperspectral data,” IEEE Trans. Geosci. Remote Sensing, vol. 42, no. 3, pp. 650–663, 2004. [25] E.M. Winter, “N-FINDR: An algorithm for fast autonomous spectral end-member determination in hyperspectral data,” Proc. SPIE, vol. 3753, pp. 266–275, 1999. [26] A. Chowdhury and M.S. Alam, “Fast implementation of N-FINDR algorithm for endmember determination in hyperspectral imagery,” Proc. SPIE, vol. 6565, pp. 656526, 2007. [27] J.W. Boardman, “Geometric mixture analysis of imaging spectrometry data,” Proc. Int. Geoscience Remote Sensing Symp., vol. 4, pp. 2369– 2371, 1994. [28] C.-I Chang, C-C. Wu, W-M. Liu, and Y-C. Ouyang, “A new growing method for simplex-based endmember extraction algorithm,” IEEE Trans. Geosci. Remote Sensing, vol. 44, no. 10, pp. 2804–2819, 2006. [29] A. Ifarraguerri and C.-I Chang, “Multispectral and hyperspectral image analysis with convex cones,” IEEE Trans. Geosci. Remote Sensing, vol. 37, no. 2, pp. 756–770, 1999. [30] E.W. Dijkstra, “A note on two problems in connexion with graphs,” Num. Math., vol. 1, pp. 269–271, 1959. [31] C.-I Chang and Q. Du, “Estimation of number of spectrally distinct signal sources in hyperspectral imagery,” IEEE Trans. Geosci. Remote Sensing, vol. 42, no. 3, pp. 608–619, 2004. [32] D.C. Heinz and C.-I Chang, “Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery,” IEEE Trans. Geosci. Remote Sensing, vol. 39, no. 3, pp. 529 –545, 2001. [33] H. S. M. Coxeter, Introduction to geometry, Wiley, New York, 1963. [34] J.B. Tenenbaum, V. De Silva, and J.C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science, vol. 290, no. 5500, pp. 2319–2323, 2000. [35] R.N. Clark, Spectroscopy of Rocks and Minerals, and Principles of Spectroscopy, John Wiley and Sons, New York, 1999. [36] G. Thoonen, K. Hufkens, J. Vanden Borre, S. De Backer, and P. Scheunders, “Mapping the conservation status of heathland vegetation using hyperspectral remote sensing in a hierarchical classification framework,” IEEE Trans. Geosci. Remote Sensing (submitted for publication).

10

Rob Heylen (M’10) received the B.S. degree, the M.S. degree and the Ph.D. degree in physics, with work in the field of statistical mechanics, from the Katholieke Universiteit Leuven, Leuven, Belgium, in 2001, 2003 and 2008, respectively. In 2009, he became a post-doctoral researcher with the Vision Lab, Department of Physics, University of Antwerp, Belgium. His main areas of research interest are hyperspectral image processing and computational physics.

Dˇzevdet Burazerovi´c (M’00) received his M.S. degree in electrical engineering from Eindhoven University of Technology, in 2000. Since 2009, he has been with the Vision Lab, Department of Physics, University of Antwerp, where he is working as researcher and towards obtaining a PhD degree. His current research interests include hyperspectral image processing and data/pattern analysis.

Paul Scheunders (M’98) received the B.S. degree and the Ph.D. degree in physics, with work in the field of statistical mechanics, from the University of Antwerp, Antwerp, Belgium, in 1983 and 1990, respectively. In 1991, he became a research associate with the Vision Lab, Department of Physics, University of Antwerp, where he is currently a professor. He has published over 120 papers in international journals and proceedings in the field of image processing and pattern recognition. His research interest includes wavelets and multispectral image processing.