Contextual Descriptors for Scene Classes in Very High Resolution

80

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 9, NO. 1, JANUARY 2012

Contextual Descriptors for Scene Classes in Very High Resolution SAR Images Anca A. Popescu, Inge Gavat, Senior Member, IEEE, and Mihai Datcu

Abstract—The new generation of spaceborne SAR instruments with meter or submeter resolution finds enormous applications for the observation of urban, industrial, in general of man-made scenes. Thus, targets are not any more observed in isolation, instead the groups of objects, e.g., house, bridge, and road, etc., need to be recognized in their spatial context. This paper proposes a feature extraction method for image patches in order to capture the spatial context. The method is based on the characteristics of the spectra of the SAR data, integrating radiometric, geometric, and texture properties of the SAR image patch. The method is demonstrated for TerraSAR-X High Resolution Spotlight data. To account for the spatial context in which a group of targets is located, it uses an image patch covering typically 200 × 200 m2 of the scene. A comparative evaluation of our descriptors and grey-level co-occurrence matrix (GLCM) texture features has been performed over a database of 6916 patches. The method allowed for the robust recognition of over 30 different scene classes, with precision between 50% and 93%. Numerical results show that our method is able to discriminate between scene classes better than GLCM texture parameters. Index Terms—SAR image classification, spectral analysis, very high resolution.

I. P RELIMINARIES

D

UE to their special capabilities, SAR sensors have always been a very important source of information for various remote sensing applications, ranging from target detection, surveillance, monitoring, to urban planning and damage assessment. One of the major problems concerned in almost every application is automatic and reliable classification. With the increased resolution offered by modern spaceborne SAR sensors, the amount of available information demands new mechanisms and processing techniques to handle, understand, and discriminate information. The number of individual objects that can be “seen” by modern SAR imaging systems becomes comparable to the number of objects distinguishable by the human eye, and therefore, a solid comprehension of a high resolution (HR) SAR scene demands that hundreds of object/scene classes be defined and possibly identified.The HR SAR scenes used for this analysis contain complicated arrangements of numerous and various objects, mostly man-

Manuscript received September 19, 2010; revised January 18, 2011 and May 17, 2011; accepted May 20, 2011. Date of publication August 4, 2011; date of current version December 23, 2011. A. A. Popescu and I. Gavat are with the University Politehnica of Bucharest, 060032 Bucharest, Romania (e-mail: [email protected]; igavat@ lpsv.pub.ro). M. Datcu is with the German Aerospace Center DLR, 82234 Wessling, Germany, and also with the University Politehnica of Bucharest, 060032 Bucharest, Romania (e-mail: [email protected]). Digital Object Identifier 10.1109/LGRS.2011.2160838

Fig. 1. At meter resolution, if context is ignored (the analysis windows used is relatively small as compared to the object scale), very different scene classes may be confused. The example shows a bridge (left) and buildings (right) which are comprehensible only when the window size is sufficiently large to incorporate relevant context. Otherwise, both scene classes “look” similar and can be confused.

made. It is very unlikely that a scene shall contain two or more identical objects, and in addition, the same object may have different meanings in different scenes or arrangements. Context becomes an important source of information. The most important issue to be considered is the choice of the information extraction method to be employed. Complex structures are usually a mixture of regions and cover many pixels; therefore, a pixel level analysis is preferred once an object is identified and outlined. Moreover, different distributions of the same objects can have different semantic meanings. At meter resolution, two very different objects can have regions that are similar within a small analysis window as compared to the object size. Fig. 1 gives an example of such a situation: the signature of a bridge at very small scale may be very similar to the signature of a building. The signatures can be differentiated at larger scales. Moreover, two objects belonging to the same scene class can be similar on a global scale, but differ significantly on a small scale (e.g., have different structure details). By scene class, we define a group of objects that share the same semantic (e.g., bridge) or geometrical, radiometrical, and texture properties. Therefore, the spatial arrangement of objects is best observed and understood within a surface of tens or hundreds of meters. Therefore, we suggest the use of an image-patch-based approach. In the remote sensing literature, a number of patchbased analysis methods have been proposed, although there is not a unified definition for the term. Usually, patches represent small regions of 10 to 30 pixels extracted from a singular object and from which the object features are computed. Bag of Words-based approaches employ this method for object definition [1]–[3], in the sense that the small patches represent the words in the vocabulary that is used to build the document (i.e., the object).

1545-598X/$26.00 © 2011 IEEE

POPESCU et al.: CONTEXTUAL DESCRIPTORS FOR SCENE CLASSES IN VERY HIGH RESOLUTION SAR IMAGES

Fig. 2. Left: Example of typical ATR image (MSTAR [4] database: T-72 battle tank): one isolated target on homogeneous background. Middle and right: HS TerraSAR-X [5] examples of industrial/urban sites; middle patch: broad view of a harbor; right patch: close view of a district with skyscrapers. Conversely, to ATR, we no longer have isolated objects, but rather complicated arrangements of targets. The number of individual objects is so high that we should work with scene classes (e.g., use rather generic denominations like bridge instead of specific nomenclatures for target identification) instead of specific Objects, and with tree-like structures to further separate generic classes into smaller subclasses.

In this paper, we propose a different usage of the term image patch, justified by the need to account for the scene context. We employ patches that cover areas of approximately 200 × 200 square meters. The patch dimension was chosen after testing various sizes of the image patch, ranging from 30 to 250 pixels with TSX data of ∼1 m resolution, the best results being attained for a patch size of ∼200 × 200 pixels. Such a patch can comprise both large and relatively small manmade structures, and is also useful for texture computation on homogeneous areas (e.g., green areas, sport yards, etc.). Our aim is to detect and classify scene classes based on their intrinsic content, and not individual objects, as a scene class may be composed of different objects of different sizes. This approach may be considered as an extension of the classical automatic target recognition (ATR) techniques which usually deal with a target located on a generally homogeneous background like vegetation or bare land. In some cases, the models of interest targets are known and the detected object can be matched to the prior known data set, which eases the recognition of the target. Our work focuses on urban areas where generic objects no longer follow the assumption of individuality or homogeneity as in ATR (Fig. 2). In most situations, ATR methods for target identification and classification make use solely of the intensity information in detected SAR images, [6], [7]. Most of the approaches detect targets based on a series of parameters employed in image processing, for example size, contour shape, diameter [8], texture, and grey tones, and work with very small analysis windows, for example 5 × 5 pixels [9], or 80 × 80 pixels [10], in order to localize the target features. Specific methods have been designed for complex data. An approach based on spectral analysis is given in [7] where the authors assume that clutter will have different responses in two different sublooks due to the lack of coherence of the speckle, while a target will have the same response. Another approach is based on the different behavior of the amplitude and phase statistics at different resolution scales. In [11], the discrete wavelet transform is employed to model the formation of multiresolution SAR images, both in range and azimuth. The method avoids aliasing and reduces the variance of the speckle. Wavelet transforms are also employed in [12], [13] to discriminate between different objects in the SAR scene. A

81

cepstrum-based approach is given in [14], where the authors use 2-D real cepstrum features to discriminate between clutter and target in an ATR application tested on MSTAR database. The authors obtained results similar to principal component analysis but with an improved computational cost. Many ATR techniques rely on a good segmentation of the data, and at the same time, the accuracy of the segmentation following the ATR method serves as a performance indicator [15]. Segmentation techniques have been developed both for detected and complex SAR data, and the approaches are various: in [5], the authors propose a multiresolution statistical representation, which applies to homogeneous regions of terrain; in [4], the multiscale stochastic structure of SAR imagery is exploited. These are some examples of approaches that work for a logmagnitude representation of the data. The phase information is exploited in [16], [17] where the physical model of speckle is combined with a probabilistic model of the distribution of region labels, and [18] where the complex data is represented by a two-level hierarchical random field model. However, these approaches are statistical and limited to uniform areas. The authors of [16] emphasize the idea that the contextual information is important for segmentation techniques applied to polarimetric complex data, although a spatially uncorrelated model is used for neighboring pixels. Each pixel is characterized by the polarimetric measurement vector and by a region label. The contextual information is represented through a Markov random fields approach. The SAR image classification methods proposed in the literature, whether they rely on textural features [19], intensity, or the data is modeled by statistical measures [20], yield a very small number of distinguished classes (∼5–6 classes), and the detected classes have a very low level of semantic meaning (e.g., ground, buildings, roads, vegetation, water body, roof, cropland). Moreover, most of the reported results are based on very small data sets, consisting of one or two scenes, making it hard to draw a valid conclusion on the effective classification accuracy. We propose a compact set of descriptors [21] which are capable to discriminate between different types of structures in terms of geometrical shape, intensity, and texture. The measures are robust and adapt to additional targets and environments, being suitable for any kind of scene classes, with brightness and feature variations. This is a very important consideration, given the selected size of the patch which can be modeled as a nonstationary stochastic process. The method is evaluated on a large database comprising ∼7000 image patches extracted from four large amplitude TerraSAR-X HR spotlight scenes, with high diversity of both urban and non-urban classes. The results show that the proposed method is able to discriminate between 30 different scene classes with high accuracy. We have compared the performance of our proposed features with the performance of 12 textural features extracted from greylevel co-occurrence matrices (GLCMs). Results show that our method outperforms GLCM parameters, being able to retrieve 30 classes, as compared to the ten classes distinguished by the GLCM texture features on the same database. In the next section, we will introduce the descriptive measures, and we will detail the feature extraction procedure. The

82

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 9, NO. 1, JANUARY 2012

brightness of the associated image patch: N  M    S(x,y) (r, a) · t

C(x,y) =

r=1 a=1 N M  

  S(x,y) (r, a)

 ,

with t =

r r → C(x,y) a a → C(x,y) .

r=1 a=1

(3) The Spectral Flux F (4) characterizes the smoothness of the spectra. This measure represents the degree of correlation associated with a patch spectrum. It is computed as the square differences between successive normalized amplitudes of the spectra, denoted by N(x,y) (r, a) (r,a)

F(x,y) =

N M   

N(x,y) (r, a) − N(x−1,y) (r, a)

2

.

(4)

r=1 a=1

Fig. 3. Patch extraction: regular grid with 50% overlap applied on test image with urban elements: river, residential area, and rectangular structures.

last section presents the method of performance evaluation and the results.

II. S PECTRAL C HARACTERIZATION FOR I MAGE PATCHES The scene is tiled by applying regular grids on the input data, with an overlapping factor of 50%, and stored into a database. The idea is to search through the database for patches that contain similar structures, in terms of geometry, radiometry, or texture. Given the sampling factor of the grid, it is expected that patches have high heterogeneity and that the majority will contain multiple secondary classes and a dominant class. In this paper, we perform the search based only on the dominant class. We consider the patch to be independent, and we extract the patch features regardless of the patch vicinities (Fig. 3). For each patch, we compute features based on the Fourier spectra: first- and second-order statistical moments, Spectral Centroid, Spectral Flux, Spectral Rolloff, Cepstral Coefficients [21], [22]. The patches will be described by the feature vectors which consequently transform the 2-D information contained in the original image into a 3-D data structure of depth equal r a and S(x,y) represent the 2-D to the feature vector. Let S(x,y) Fourier spectra of each patch in range (denoted by superscript r) and azimuth (denoted by superscript a) consequently. The patch position indicators are given by subscripts x and y. The patch size in pixels is represented by M and N , with M = N = 200 in this case, since one pixel covers 1 square meter on the ground. Thus, the first- and second-order statistics of the spectra are given by m(x, y) =

M N  1    S(x,y) (r, a) M ×N r=1 a=1

(1)

σ 2 (x, y) =

M N  2 1    S(x,y) (r, a)−m(x, y) . M ×N r=1 a=1

(2)

The Spectral Centroid C (3) represents the centroid of the amplitude of the Fourier spectra. It describes the spectral

The Spectral Rolloff R (5) represents the frequency bin below which 85% of the amplitude distribution is concentrated, giving a measure of the energy associated to the respective patch. It is expected that similar acquisition geometries with almost the same incidence angle will result in similar responses in terms of energy for targets that belong to the same scene class. The Spectral Rolloff in range and azimuth denoted by Rr and Ra satisfies the relation: Ra M Rr  M        S(x,y) (r, a) = 0.85 · S(x,y) (r, a) . r=1 a=1

(5)

r=1 a=1

The last set of parameters is represented by the Cepstral Coefficients. In order to compute the Cepstral Coefficients for a given patch, we have to take the logarithm of the modulus of the spectra of the respective patch and then decorrelate the cepstral vectors by using either a discrete cosine transform or an inverse Fourier transform. We retain only the first five cepstral vectors and compute the mean and variance of each, for all the patches. We wish to determine the maximum number of scene classes that can be defined within the database, based on the feature vectors associated to each element in the database. The proposed descriptive measures that form the feature vectors are capable to provide a data representation amenable to classification and definition of categories. III. P ERFORMANCE E VALUATION In the following section, we will present the setup used in our evaluation experiment. This includes the database and the classification method we employed to validate our method. Our approach can easily be integrated into a content-based information retrieval (CBIR) system. Since most CBIR systems are based on nearest-neighbors methods for classification, our choice was a K-nearest neighbor (NN) classification method. Comparative results with the GLCM are also presented. Our test data consists of four TerraSAR-X amplitude images (∼10 000 × 10 000 pixels). The data was provided by the German Aerospace Center (DLR) in the MTH-0302 scientific proposal. The data format is Multilook Ground Detected (MGD), Spatially Enhanced (SE) HS Spotlight images, with single polarization (HH), of 10 045 meters resolution in ground

POPESCU et al.: CONTEXTUAL DESCRIPTORS FOR SCENE CLASSES IN VERY HIGH RESOLUTION SAR IMAGES

Fig. 4. Scene classes used in the evaluation of the method. Patches cut from four MGD TerraSAR-X images with the same incidence angle (MTH-0302 scientific proposal). The examples show the variety of structures and details, ranging from large homogeneous areas (classes 1, 2, 9) to structures with clear textural patterns (classes 7, 16) and classes with high levels of details (classes 17, 18, 20). The database consists of 6916 patches, with class cardinality that ranges between ten and several hundreds of patches per class.

range and 11 512 meters resolution in azimuth. The number of looks is 0.995 looks in azimuth and 10 589 looks in range; thus, the equivalent number of looks (ENL) is 1. To ensure a large variety of objects and structures, the selected SAR images were acquired over urban areas on different countries and continents: Las Vegas (USA), Venice (Italy), Gizah (Egypt), and Gauting (Bavaria, Germany) [23]. We obtained a number of 6916 image patches. The variety of structures is depicted in Fig. 4 (30 visually annotated classes from the database used in our tests). On the one hand, scene classes 1, 2, 4, and 9 show extended homogeneous subclasses (bare land, large scale agriculture, tall vegetation, and sea). On the other hand, classes 17, 18, or 20 exhibit particular scattering effects due to different building architectures (hexagonal building, dome, and skyscraper). Patches 3, 5, 8, 7, 13, and 30 are examples of different types of high density residential areas. In order to perform the numerical evaluation, we compared our method with a texture feature extraction procedure, GLCM, widely used in remote sensing data classification. Using human expertise, we were able to identify 46 classes and group the input patches accordingly. This procedure allowed us to compare the classification results with our accepted truth. The spectral vectors were normalized to values in the interval [0, 1]. For the GLCM parameters, we computed the normalized GLCM with offset 1 and orientation 1. Our implementation allows for two values of the offset (1 and 2) and four options for the orientation (0◦ , 45◦ , 90◦ , and 135◦ ). Each GLCM feature vector is formed of 12 parameters computed based on the normalized matrix: mean, variance, entropy, angular second moment, energy, correlation, maximum probability, contrast, homogeneity, dissimilarity, cluster shade, and cluster prominence [24], [25]. We tested both methods using the nonpara-

83

metric classifier K-NN. The K-NN algorithm is a statistical instance-based learning method. It decides that a certain pattern X belongs to the same category as do its closest neighbors, given the training set of m labeled patterns [26]. The algorithm estimates the probabilities of the classes, using a Euclidian metric. A sample pattern is assigned to that category which maximizes the quantity p(i)p(X|i), where p(i) is the probability of category I, and p(X|i) is the probability of pattern X given that X belongs to category I[27]. In our implementation, the scene class is arbitrarily chosen by a user who should train the classifier with positive (patches that contain the desired structure/object of interest) and negative examples. For the training stage of the classification we chose random examples from the 46 annotated classes, both for the positive and negative examples. We used 40% training samples and 60% testing samples for each class. For the quantitative assessment, we compared the classification results with the annotated database. We computed the following measures that allowed us to assess the method’s performance [28]: • Precision—the ratio of true positives and the sum of false positives and true positives: P recision = T P/ (T P + F P ) • Recall—the ratio of true positives and the sum of false negatives and true positives. The measures show the probability that the classification assigns an object to its expected class: Recall = T P/(T P + F N ) For each scene class, we trained the classifier three times, each time with randomly selected training samples. If the classification results were similar in all three tests, then the class was considered valid. The classes that did not yield stable results were rejected. Next, the classes with very low recognition precision (i.e., smaller than 50%) were rejected. Thus, only 30 scene classes could be retrieved with precision better than or equal to 50%. Fig. 5 displays a comparative Precision–Recall graph between the classification based on the spectral parameters and the classification based on the GLCM parameters: using our procedure, we have been able to distinguish 30 scene classes that have recognition accuracy higher than 50%. Using the GLCM procedure, only ten classes could be retrieved with precision higher than 45%. The results show that the parameters are able to provide a good characterization both for large homogeneous classes that dominate a patch (classes 1, 2, and 3 are recognized with precision higher than 80%), and can also capture a strong signature from a small object compared to the patch size (classes 20 and 21 in Fig. 4 are recognized with precision higher than 70%). IV. C ONCLUSION We have presented a combined radiometry/structure-driven approach for HR SAR image classification, based on spectral descriptors, which allowed for the recognition of 30 scene classes on a database of 6916 patches. The numerical evaluation shows a recognition rate higher than 50% for 30 classes, mostly man-made urban objects. Our method outperforms the classification obtained by using GLCM texture parameters, which was able to distinguish between ten classes on the same

84

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 9, NO. 1, JANUARY 2012

Fig. 5. Comparative precision–recall graph between the classification based on the spectral parameters and on the GLCM parameters: using the spectral procedure, 30 scene classes have been distinguished with precision higher than 50%. Using the GLCM procedure, ten classes could be retrieved with precision higher than 45%. The graph shows the probability that the classification assigns an object to its expected class. The order of the first 30 classes (accepted classes) follows the order of the patches depicted in Fig. 4.

database. The performance could be increased if tree-structured multiclass cases are considered: a patch containing two or three types of objects, which can be assigned to more than just one scene classes. Ongoing activities aim at assessing the impact of additional despeckling and at understanding the physical origin of the observed phenomena. Also, some parameters are dominant for certain types of classes. Future work aims at evaluating the contribution of each parameter to each type of class. R EFERENCES [1] S. Xu and T. Fang, “Object classification of aerial images with bag-of-visual words,” IEEE Geosci. Remote Sens. Lett., vol. 7, no. 2, pp. 366–370, Apr. 2010. [2] M. Lienou, H. Matre, and M. Datcu, “Semantic annotation of satellite images using latent dirichlet allocation,” IEEE Geosci. Remote Sens. Lett., vol. 7, no. 1, pp. 28–32, Jan. 2010. [3] L. Weizman and J. Goldberger, “Urban-area segmentation using visual words,” IEEE Geosci. Remote Sens. Lett., vol. 6, no. 3, pp. 388–392, Jul. 2009. [4] “Moving and Stationary Target Acquisition Recognition (MSTAR),” Program Technology Review, Denver, CO, Nov. 1996. [Online]. Available: http://cis.jhu.edu/data.sets/MSTAR [5] Free TerraSAR-X Sample Data, 2010. [Online]. Available: http://infoterra. de/free-sample-data [6] Nat. Committee Radio Sci. Australian Acad. Sci., J. Schroeder, Automatic Target Detection and Recognition Using Synthetic Aperture Radar Imagery, Canberra, Australia, Sep. 2002. [Online]. Available: www.ips.gov.au/IPSHosted/NCRS/wars2002/invited.htm [7] C. Henry, J.-C. Souyris, and P. Marthon, “Target detection and analysis based on spectral analysis of a SAR image: A simulation approach,” IEEE Trans. Geosci. Remote Sens., vol. 41, no. 12, pp. 2725–2734, Dec. 2003.

[8] L. Novak, G. J. Owirka, W. S. Brower, and A. Weaver, “The automatic target-recognition system in SAIP,” Lincoln Lab. J., vol. 10, no. 2, pp. 187–202, 1997. [9] A. Filippidis, L. C. Jain, and N. Martin, “Fusion of intelligent agents for the detection of aircraft in SAR images,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 4, pp. 378–384, Apr. 2000. [10] Y. Sun, Z. Liu, S. Todorovic, and J. Li, “Adaptive boosting for synthetic aperture radar automatic target recognition,” IEEE Trans. Aerosp. Electron. Syst., vol. 43, no. 1, pp. 112–125, Jan. 2007. [11] C. Bhattacharya and P. R. Mahapatra, “A generalized approach to multiresolution complex SAR signal processing,” IEEE Trans. Aerosp. Electron. Syst., vol. 45, no. 3, pp. 1089–1103, Jul. 2009. [12] J. P. Ovarlez, L. Vignaud, J. C. Castelli, M. Tria, and M. Benidir, “Analysis of SAR images by multidimensional wavelet transform,” Proc. Inst. Elect. Eng.—Radar, Sonar Navig.: Special Issue on Time Frequency Analysis for Synthetic Aperture Radar and Feature Extraction, vol. 150, no. 4, pp. 234– 241, Aug. 2003. [13] M. Tria, J. P. Ovarlez, L. Vignaud, J. C. Castelli, and M. Benidir, “Discriminating real objects in radar imaging by exploiting the squared modulus of the continuous wavelet transform,” IET Radar Sonar Navig., vol. 1, no. 1, pp. 27–37, Feb. 2007. [14] A. Elyirdirim and A. Cetin, “Man-made object classification in SAR images using 2-D cepstrum,” in Proc. IEEE Radar Conf., Pasadena, CA, May 2009, pp. 1–4. [15] M. Cetin, W. Karl, and D. A. Castanon, “Feature enhancement and ATR performance using nonquadratic optimization-based SAR imaging,” IEEE Trans. Aerosp. Electron. Syst., vol. 39, no. 4, pp. 1375–1395, Oct. 2003. [16] E. Rignot and R. Chellappa, “Segmentation of polarimetric synthetic aperture radar data,” IEEE Trans. Image Process., vol. 1, no. 3, pp. 281– 300, Jul. 1992. [17] E. Rignot and R. Chellappa, “Segmentation of SAR complex data,” J. Opt. Soc. Amer., vol. 8, no. 9, pp. 1499–1509, Sep. 1991. [18] H. Derin, A. Kelly, G. Vezina, and S. G. Labitt, “Modeling and segmentation of speckled images using complex data,” IEEE Trans. Geosci. Remote Sens., vol. 28, no. 1, pp. 76–87, Jan. 1990. [19] A. Reigber, M. Jager, W. He, L. Ferro-Famil, and O. Hellwich, “Detection and classification of urban structures based on high-resolution SAR imagery,” in Proc. Urban Remote Sens. Joint Event, Paris, France, Apr. 2007, pp. 1–3. [20] C. Tison, J.-M. Nicolas, F. Tupin, and H. Maître, “A new statistical model for Markovian classification of urban areas in high-resolution SAR images,” IEEE Geosci. Remote Sens. Lett., vol. 42, no. 10, pp. 2046–2057, Oct. 2004. [21] A. Popescu, I. Gavat, and M. Datcu, “Complex SAR image characterization using space variant spectral analysis,” in Proc. IEEE Radar Conf., Rome, Italy, 2008, pp. 1–4. [22] T. Li and M. Ogihara, “Toward intelligent music information retrieval,” IEEE Trans. Multimedia, vol. 8, no. 3, pp. 564–574, Jun. 2006. [23] A. Popescu, C. Patrascu, J. Singh, and M. Datcu, “Spotlight TerraSARX data modeling using space-variant measures, for scene targets and structure indexing,” in Proc. 8th EUSAR, Aachen, Germany, Jun. 2010, pp. 1–4. [24] F. Albgretsen: “Statistical Texture Measures Computed from Grey Level Co-occurrence Matrices,” Department of Informatics, University of Oslo, Nov. 2008, http://www.uio.no/studier/emner/matnat/ifi/INF4300/ h08/undervisningsmateriale/glcm.pdf [25] “Mryka Hall-Beyer,” The GLCM Tutorial, Version: 2.10, Feb. 2007. [Online]. Available: http://www.fp.ucalgary.ca/mhallbey/ tutorial.htm [26] E. Alpaydin, Introduction to Machine Learning. Cambridge, MA: MIT Press, 2004. [27] T. J. Mitchel, Machine Learning. New York: McGraw-Hill Science/ Engineering/Math, 1997. [28] M. Gönen, “Receiver operating characteristic (ROC) curves,” presented at the 31st Annu. SAS Users Group Int. Conf., San Francisco, CA, 2006, Paper 210-31. [Online]. Available: http://www2.sas.com/ proceedings/sugi31/210-31.pdf