Level Set Hyperspectral Image Segmentation using Spectral Information Divergence-based Best Band Selection John E. Ball, Student Member, IEEE, Terrance West, Student Member, IEEE, Saurabh Prasad, Student Member, IEEE, Lori Mann Bruce, Senior Member, IEEE GeoResources Institute (GRI) Mississippi State University Starkville, MS, USA
[email protected]
Abstract—We present a supervised hyperspectral segmentation procedure, consisting of best band analysis (BBA), an initial distance-based segmentation, and level set segmentation enhancement by forcing localized vicinities to be more homogeneous. BBA uses the spectral information divergence (SID) to reduce each pixel’s high dimensional data to a scalar value, where the Bhattacharyya Distance (BD) is maximized. The initial segmentation is based on feature vectors created from the SID metric. The level set segmentation then enhances areas that do not have spatially homogeneous ground cover. The proposed method is tested on a 72-band Compact Airborne Spectrographic Imager (CASI) image of a farm area in northern Mississippi, U.S.A. The proposed method is compared to a BBA-based maximum-likelihood (ML) method. Quantitative results are compared using segmentation and classification accuracies. Results show that both the initial classification using BBA features and the level set enhancement produced high-quality ground cover maps and outperformed the ML method, as well as previous studies by the authors. The ML method resulted in accuracies ≥ 95.5%, whereas the level set segmentation approach resulted in accuracies as high as 99.7%. Keywords - band analysis, band selection, classification, Bhattacharyya distance, distance metrics, image processing, image region analysis, image classification, image segmentation, hyperspectral, level sets, segmentation, spectral information divergence, SID, pattern classification, remote sensing
I.
INTRODUCTION
H
yperspectral image ground cover classification involves analyzing each pixel and creating a thematic map (TM), where each pixel receives a unique label identifying its ground cover class. Two issues faced in these classification systems are the high data dimensionality and errors caused by mislabeling neighboring pixels which can cause regions of homogeneous ground cover to appear heterogeneous. These errors can be caused by inadequate training data, natural class overlap, or high dimensionality of the data, which is the well known Hughes phenomenon [1]. In order to mitigate effects of the Hughes phenomenon, a typical hyperspectral system will have a dimensionality reduction (DR) component, which seeks to simultaneously reduce the data dimensionality and maximize the class separability. Commonly used hyperspectral DR methods
1-4244-1212-9/07/$25.00 ©2007 IEEE.
include principal components analysis (PCA), various forms of linear discriminant analysis (LDA), independent component analysis, projection-based methods, wavelet-based methods, support vector machines, and methods that reduce the hyperspectral data to a single scalar measure for each pixel. In hyperspectral image classification, some popular approaches include metric-based methods, such as the spectral information divergence (SID) [2-5], contemporary methods such as maximum likelihood (ML), kernel-based methods such as support vector machines, and more recently, level set methods. The ML classification algorithm is arguably the most common method for remotely sensed image segmentation. ML uses class mean and covariance estimates to decide the most likely class membership [6,7].
II.
BACKGROUND
For two dimensional (2D) image segmentation, the level set boundary is the zero level set of an implicit function φ , and the level set method uses the partial differential equation (PDE) φt + F ∇φ = 0 , where φt is the partial derivative of φ with respect to time, ∇φ is the gradient of φ , and F is a speed function which controls the level set propagation [8]. The level set will propagate outwards (inwards) as long as the speed function is positive (negative). For multiple region segmentation, is defined as the region Rj R j = p φ j ( p ) ≤ 0; φk ( p ) > 0,k ≠ j} , where the other level sets are not permitted to
{
segment a previously segmented pixel. Thus for a C class segmentation problem, C level sets are utilized. A grid spacing ∆x = ∆y = 1 and time step ∆t = 0.8 is used to evaluate the numerical approximation to the PDE. Previous methods in hyperspectral level set-based classification include road extraction [9] and level set multi-class segmentation [10-12]. In this paper, the authors investigate SID and ML, where best band analysis (BBA) will be used to mitigate the Hughes phenomenon. In addition, a level set approach will be used to enforce regional homogeneity in the TM. This paper is an extension of the work of Ball and Bruce in [10-12], which are, to the best of the authors’ knowledge, the first multi-class hyperspectral level set segmentation methods published.
4053
T Let B be the number of spectral bands, x = [ x1 ,⋯ ,xB ] be an T unknown signature to classify, and r = [ r1 ,⋯ ,rB ] be a reference pixel taken from a spectral library. Then the SID metric is
SID =
∑
B
(
)
(
)
and
q j = rj
( r1 + ⋯ + rB )
p j log p j / q j + q j log q j / p j ,
j =1
p j = x j / ( x1 + ⋯ + xB )
with for
j ∈ {1,⋯ ,B} [2]. When the unknown signature is similar (not
similar) to the reference signature, then SID will have a relatively lower (higher) value.
III.
Selecting a value of β close to 1.0 will account for almost all of the training data, and the parameter ε = 0.02 is used to slightly boost the threshold. A sliding template is used to scan the area around each pixel in the segmentation thematic map. An 11 × 11 square window was chosen experimentally. For each class c, the template counts the total number of pixels assigned to that class divided by the total number of pixels. For pixels near the border, the total number of pixels is the sum of the template pixels that are in the original borders of the image. Thus, for each pixel (x,y) and a class c, the following is evaluated: c = Tx,y
METHODOLOGY
The training and testing signatures are extracted from the image. Then the following procedure is used to classify the farm image. First, BBA is performed using the SID values for each pixel. For each class, The BD is calculated using the SID values for that class versus the SID values for all of the other classes. This process is repeated for each band set using a sliding spectral window. The band sets are L0 − L8 , and have band lengths of 72 (all bands),4,8,12,16,20,40,60, and 70 bands, respectively. The band set with the highest BD is used for initial classification. Assuming the image training data has C classes, then for each training pixel, a 1 × C feature vector is created using the SID metric values as follows: 1 C T f x,y = f x,y ,⋯ , f x,y ,
(1)
smallest value of x with Fc ( x ) ≥ β , where β =0.98. Then define
x,y
}
−c
Ω( x,y ) ,
(2)
where TM x,y is the thematic map entry at location (x,y), Ω( x, y ) is the set of “on” pixels in the template, δ is the Dirac delta function, and Ω( x,y ) are the number of “on” pixels in the template which are inside or on the image borders. This creates a 2D image for each class. For each pixel, a threshold is applied to equation (2): Tɶ c = H T c − α , where the constant α =0.5 x,y
(
x,y
T
)
T
was experimentally determined. Thus, pixels with surrounding areas with less than a majority of pixels are forced to zero in the speed function. It is important to point out that the level set will not enter a region where the speed function is zero. In the following algorithm, the constant three in step 2 was determined experimentally. The level set segmentation algorithm is: 1. For each class c to be processed 2. For k = 1 to 3 c c 3. Compute Gɶ x,y and Tɶx,y
k where f x,y , the feature vector element for class k, is calculated
using the bands determined by the BBA algorithm. Initial segmentation is performed by using a minimum Euclidean distance classifier, where each pixel’s feature vector is compared to the training data feature vectors for each class. After the initial segmentation, there may be small patches where the segmented regions are not homogeneous. A level set method will be employed to clean up these regions. For each class to be processed with the level set method, a 2D stopping map is created by treating the BSID feature for each pixel as a random variable, and examining the BSID cumulative distribution function (CDF). Let Fc ( x ) be the CDF for class c. Let τ c be the
∑ δ {TM
Ω( x,y )
4. Compute the speed function 5. Run level set until no sign changes 6. End For 7. End For
Once the initial and level set-based segmentations are created, the results are analyzed using confusion matrices and overall accuracies (OA) [13]. A statistical analysis using the methods of Congalton and Green (pp. 49-51 of [13]) is employed to compute a z-statistic to compare the two confusion matrices. For a 95% confidence interval, a z-statistic with z > 1.96 indicates that two confusion matrices are statistically different.
a threshold γ c = τ c (1 + ε ) , where ε = 0.02 is a small positive
IV.
constant. The stopping map for class c is given by c Gx,y
=H
(
c f x,y
)
− γ c , where H ( ⋅) is the Heaviside step function,
which is one when the argument is ≥ 0 , and zero otherwise. The 2D image Gxc ,y will be one if the pixel’s feature vector for that class is suitably small, and zero otherwise. As a final c preprocessing step, the image Gx,y is convolved with an isotropic
Gaussian filter with unity variance and [11 × 11] region of support, denoted Gσ . This is a common step frequently used in level set application when forming the stopping map, and provides some smoothing. The final stopping map is Gɶ c = Gɶ c ⊗ G , where ⊗ is the convolution operator. x,y
x,y
σ
1-4244-1212-9/07/$25.00 ©2007 IEEE.
CASE STUDY
A case study utilizes a Compact Airborne Spectrographic Imager (CASI) image [14]. The original image was cropped to 315 rows by 1010 columns. CASI is a programmable instrument, and the settings for our image use 72 hyperspectral bands from approximately 414 to 954 nm, with a spectral resolution of 8 nm. The image was taken of a rural farm area in Brooksville, Mississippi, U.S.A., on July 1, 2002, during good weather conditions. This image is denoted as the “farm image”. The endmembers for this image are listed in Table I. It should be noted that this image is a scene of the Black Belt Branch Experiment Station where field-level agricultural experimental studies are conducted. As a result, the crops have a high level of natural and imposed variations, so the ground cover classes are
4054
very challenging to segment and classify. A false color-image is shown in Fig. 1(a).
V.
RESULTS AND DISCUSSIONS
The initial and level set segmentations are shown in Fig. 1 (b) and (c), respectively. The confusion matrices for the initial and level set segmentations are listed in Tables III and IV. The zstatistics for the different band sets are shown in Table II. The OA are shown in Table II, and Fig. 2, and the results show a drastic increase from the all bands (L0) case, for band sets L2–L6 (8,12,16,20, and 40 bands, respectively). Band sets L7 and L8 (60 and 70 bands) performed worse than bands sets L2–L6. This is most likely a manifestation of the Hughes phenomenon. Table III also shows the OA results for the middle band sets were all above 95%. In general, the initial segmentation and level set results were better than the ML results. It is clear from Fig. 2 and Table II, which shows the zstatistics, that the level set method clearly provided improvement in all cases, and also provided a much more stable result in terms of OA. Tables III and IV show the CM results are dramatically different. Specifically note the large off-diagonal differences in the cement (Cem), cotton (Cot), and the soybeans (Soy) columns in each table.
VI.
Dodds, Dr. Ted Wallace, and Dr. Erick Larson for providing CASI ground truth information, and Dr. Valentine Anantharaj and Lewis Wasson for assisting with the CASI imagery.
REFERENCES [1] [2]
[3]
[4]
[5]
[6] [7] [8]
CONCLUSIONS
For the Farm image, the best results were obtained for the middle band sets (8,12,16,20, and 40 bands). The best OA for the BSID initial classification is 98.24% and for the level set classification is 99.70%. In this image, the level set method provided substantial improvements over the initial segmentation, as evidenced by high statistical differences based on analysis of the corresponding CM. Again, the proposed methods consistently outperformed the ML approach, which attained a maximum accuracy of 95.52%. The level set method was able to improve areas by forcing spatial homogeneity, but it can not completely overcome bad results from the initial classifier. This method works best when the errors are small patches of incorrect pixels in a homogeneous region. In general, the level set method provided a convenient means to perform a form of constrained morphological processing.
[9]
[10]
[11]
[12]
[13]
[14]
ACKNOWLEDGMENT J. Ball would like to acknowledge the National Science Foundation and Honda for providing graduate research fellowships, and NASA for providing partial research funding. The authors thank Dr. David Shaw, Dr. Dan Reynolds, Darren
TABLE I NUMBER OF SAMPLES OF TRAINING AND TESTING DATA Class
Train
Test
Cement Cotton Pasture Pecans Pond
77 544 500 500 162
83 12,280 1198 849 162
1-4244-1212-9/07/$25.00 ©2007 IEEE.
Class Road Shadows Shrubs Soybeans TOTAL
Train
Test
376 65 500 758 3,482
459 101 136 17,700 32,968
4055
G. Hughes, "On the mean accuracy of statistical pattern recognizers," IEEE Trans. on Information Theory, vol. 14, no. 1, pp. 55-63, 1968. C.I. Chang, "An information-theoretic approach to spectral variability, similarity, and discrimination for hyperspectral image analysis," IEEE Trans. on Information Theory, vol. 46, no. 5, pp. 1927-1932, Aug. 2000. S.S. Chiang, C.I. Chang, and I.W. Ginsberg, "Unsupervised target detection in hyperspectral images using projection pursuit," IEEE Trans. on Geoscience and Remote Sensing, vol. 39, no. 7, pp. 1380-1391, Jul. 2001. C. Kwan, B. Ayhan, G. Chen, W. Jing, J. Baohong, and I.C. Chein, "A novel approach for spectral unmixing, classification, and concentration estimation of chemical and biological agents," IEEE Trans. on Geoscience and Remote Sensing, vol. 44, no. 2, pp. 409-419, Feb. 2006. S.A. Robila and A. Gershman, "Spectral matching accuracy in processing hyperspectral data," Proc. of the Intl. Symposium on Signals, Circuits and Systems, vol. 1, pp. 163-166, Jul. 2005. R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, 2nd ed. New York: John Wiley & Sons, 2001. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. San Francisco: Academic Press, 1990. J.A. Sethian, Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science, 2nd ed. Cambridge, UK: Cambridge University Press, 2002. T. Keaton and J. Brokish, "A level set method for the extraction of roads from multispectral imagery," Proc. of the 31st Applied Imagery Pattern Recognition Workshop, pp. 141-147, Oct. 2002. J.E. Ball and L.M. Bruce, "Level set segmentation of remotely sensed hyperspectral images," Proc. of the IEEE Intl. Conference Geoscience and Remote Sensing (IGARSS), Anchorage, AL, vol. 8, pp. 5638 - 5642, Jul. 2005. J.E. Ball and L.M. Bruce, "Accuracy analysis of hyperspectral imagery classification using level sets," Proc. of the 2006 American Society of Photogrammetry and Remote Sensing Annual Conference, Reno, NV, Apr. 2006. J.E. Ball and L.M. Bruce, "Level Set Hyperspectral Segmentation: NearOptimal Speed Functions using Band Selection and Scaled Spectral Angle Mapper," Proc. of the 2006 IEEE Intl. Geoscience and Remote Sensing Symposium, pp. 2596 - 2600, Jul. 2006. R.G. Congalton and K. Green, Assessing the Accuracy of Remotely Sensed Data: Principles and Practices. New York, NY: Lewis Publishers, 1999. "ITRES Inc. Compact Airborne Spectrographic Imager," 2007. available: http://itres.com/cgi-bin/products.cgi.
TABLE II FARM IMAGE EXPERIMENTAL RESULTS - OVERALL ACCURACIES (OA) IN PERCENT AND Z-STATISTIC VALUES. BEST OA ARE BOLD AND HIGHEST OA HAS BOLD BOX AROUND IT. STATISTICALLY SIGNIFICANT Z-STATISTIC VALUES ARE SHOWN IN BOLD IN RIGHTMOST COLUMN. OA (%) Band Set z -statistic Initial Level Set ML L0 90.48 96.86 91.22 34.16 L1 89.82 97.54 90.67 41.47 L2 97.28 99.64 93.46 24.79 L3 98.16 99.70 95.52 19.41 L4 99.56 98.24 95.52 16.35 L5 97.57 99.54 95.47 21.40 L6 96.04 99.26 94.30 27.61 L7 91.32 96.26 93.34 26.60 L8 90.55 96.55 91.63 31.77
(a)
(b)
(c)
Fig. 1. (a) Farm image in pseudocolor. Farm pseudocolor image uses bands 71,39,19 for R,G,B. Cotton appears green, and soybeans red. The shrubs are on the right side of the image, and the pecan orchard is in the upper left hand side. The small white dots in the soybean fields are painted markers used for designating on-ground experiment locations. (b) Thematic maps for the initial segmentation. (c) Thematic map for the level set segmentation. The legend for (b) and (c) is shown to the left of (c).
TABLE III CONFUSION MATRIX FOR FARM IMAGE INITIAL SEGMENTATION. DIAGONAL ELEMENTS ARE BOLD.
TABLE IV CONFUSION MATRIX FOR FARM IMAGE LEVEL SET SEGMENTATION DIAGONAL ELEMENTS ARE BOLD. Cem Cot Past Pec Pond Road Shad Shrb Soy 0 0 0 0 0 0 0 0 Cem 80 Cot 0 12280 0 15 0 0 1 0 42 Past 0 0 0 0 0 0 0 0 1183 Pec 0 0 2 0 0 0 20 0 832 Pond 0 0 0 0 0 0 0 0 162 Road 3 0 0 0 0 0 0 0 459 Shad 0 0 0 0 0 0 0 0 100 Shrb 0 0 13 0 0 0 0 0 116 Soy 0 0 0 2 0 0 0 0 17658 Cem=Cement, Cot=Cotton, Past=Pasture, Pec=Pecans, Shad=Shadows, Shrb = Shrubs, Soy=Soybeans
Cem Cot Past Pec Pond Road Shad Shrb Soy Cem 0 0 0 0 0 0 0 0 0 Cot 2 0 0 0 0 0 0 21 8964 Past 0 908 830 0 0 0 0 0 792 Pec 0 0 0 0 0 0 69 202 601 Pond 0 0 0 0 0 0 0 0 162 Road 81 33 0 0 0 0 0 0 459 Shad 0 2373 0 1 0 0 0 0 101 Shrb 0 0 0 43 0 0 0 47 1803 Soy 0 2 368 204 0 0 0 20 14882 Cem=Cement, Cot=Cotton, Past=Pasture, Pec=Pecans, Shad=Shadows, Shrb = Shrubs, Soy=Soybeans
100
OA (%)
95 90 ORIGINAL SEGMENTATION
85
LEVEL SET SEGMENTATION 80 L0
L1
L2
L3
L4
L5
L6
L7
BAND SET Fig. 2. Overall accuracies (OA) in percent for the Farm image plotted versus the band sets.
1-4244-1212-9/07/$25.00 ©2007 IEEE.
4056
L8
