Automated detection of sand dunes on Mars Louren¸co Bandeira1 , Jorge S. Marques2 , Jos´e Saraiva1 and Pedro Pina1 1
CERENA, Instituto Superior T´ecnico, Av. Rovisco Pais, 1049–001 Lisboa, Portugal 2 ISR, Instituto Superior T´ecnico, Av. Rovisco Pais, 1049–001 Lisboa, Portugal
[email protected],
[email protected],
[email protected],
[email protected]
Abstract. In this paper we show that the detection of dune fields on images of the surface of Mars, however varied they are, can be achieved through the application of an automated methodology. The procedure is based on the extraction of local information from images after they are organized according to a regular grid which defines cells, in turn aggregated into larger regions (blocks) that constitute the detection units. A set of gradient features is extracted and tested with Boosting and Support Vector Machine classifiers. A detection rate of 98.7% was obtained for a 5-fold cross validation on a set of images captured by the Mars Orbital Camera on board the Mars Global Surveyor probe. Keywords: Gradient and HOG features, SVM and Boosting classifiers, Mars.
1
Introduction
Dunes are the most frequent aeolian features on the Martian surface, and their study contributes to the understanding of the interactions between the atmosphere and the surface of the planet, of the way the climate has evolved along the history of Mars and of how it works currently [1, 2]. Dunes on Mars were first observed in the early 1970s on Mariner 9 images, but only the largest kilometric fields were detected. In the late 1990s, with the orbital mission of the Mars Global Surveyor probe (MGS), equipped with a higher spatial resolution camera, many more dune fields were resolved and it was confirmed that the shapes visible showed many similarities with those occurring on Earth [3]. Recently, a group of planetary scientists created the Mars Dune Consortium (http://www.marsdunes.org) whose stated intention is to produce a catalogue containing all dune fields identifiable on the surface of Mars [4]. The results of their program of search and delineation of dune fields, which has been performed manually, are available online in a geographical database, the MGD3 -Mars Global Digital Dune Database [5]. This database only contains, at the moment of writing, information about the area between latitudes 65◦ N and 65◦ S, in which dunes cover an area of approximately 70,000 km2 . A rough estimation of the total area covered by dune fields on Mars gives about 120,000 km2 in the southern hemisphere and about 680,000 km2 in the northern [6]; thus, more than 90% of the Martian dune
2
fields have yet to be mapped. Furthermore, the remotely sensed images used so far to construct the database still leave outside this venture the smallest dune fields. Besides this huge quantity of dunes still unmapped, the detection of changes on the characteristics of these dynamic aeolian features is another issue that could also benefit a lot if an automated method were available to delineate them on remotely sensed images at different scales and moments in time. In the last years, some techniques have begun to be implemented to automatically detect structures on planetary surfaces but, so far, only the field of impact crater studies has achieved some maturity. A large variety of methodologies have been developed and tested, using the most recent and powerful tools, with steadily improving performances [7–14]. On the contrary, there is as yet no automated approach to deal with the identification of sand dunes. There are some applications dealing with temporal change detection or measuring the height of dunes, but they are restricted to geographically confined case studies. Thus, the objective of this paper is to test the adequacy of recent and upto-date machine learning methodologies for the detection of aeolian dunes on remotely sensed images of Mars. This work is partly inspired on some previous strategies and algorithmic sequences used for automated crater detection. For the purpose now considered, we have selected two types of features that work best in the extraction of the directional and periodic characteristics of the dunes (gradient and histogram-of-oriented-gradients features), and which were both used on Boosting and Support Vector Machine classifiers to indicate if a given region of the image contains dunes. The performance of those methods is evaluated with a set of high spatial resolution images acquired by the MOC camera of the MGS probe which represent the diversity of Martian dune types.
2 2.1
Formulation of the problem Dune types
A geological classification scheme of sand dunes was proposed by McKee [15] for terrestrial examples, mostly based on field work. It considers the different shapes that exist and relates them to specific environments of deposition and the factors acting upon it. The dunes so far identified on the Martian surface have been classified according to that scheme, and although most of them fit into the main types there are some undefined morphologies not known to occur on Earth [4]. On Fig. 1, we present some examples of the predominant Martian dune type (barchan dunes). From this, it becomes clear the multitude of factors that affect the visual aspect of dune fields - constituents, size, shape and density, association to seasonal advance and withdrawal of ice cover, angle of illumination, just to name some - and that must be tackled by any automated approach designed to detect their presence on an image. Thus, the nature and varied characteristics of occurrence of sand dunes on images of the Martian surface demand a learning strategy, able to adapt itself to distinct situations.
3
Fig. 1. Diversity of Martian sand dunes on portions of MGS/MOC images (from left to right): R18-01906, S01-00739, S01-00925, E23-00005 and R22-00101. Each image covers an area of 1500x3000 m2 [image credits: MSSS/NASA/JPL].
2.2
Image analysis
The procedure adopted for the identification of the dunes is based on the analysis of the local information of the image along a regular grid. For that purpose, an image is divided into cells (Fig. 2a) from which given features will be extracted. To increase the invariance to specific factors such as illumination and shadowing, an aggregation of the local features is performed within larger regions, blocks constituted by 3 × 3 cells, which are the detection windows (Fig. 2b). The displacement of the block along the entire image grid is performed with an overlapping between adjacent blocks equal to one cell side (Fig. 2c).
(a)
(b)
(c)
Fig. 2. Tiling an image in (a) cells and (b) blocks (3 × 3 cells, in red); (c) Block displacement with overlapping. This region corresponds to a sample of image E0201086 [image credits: MSSS/NASA/JPL].
4
3
Features and classifiers used
Important advances were achieved in the last decade in the field of computer vision, both in the type of features used to characterize objects and in the recognition methods which are needed to learn and classify the objects present in the images. From those recent contributions we selected the features we consider to be among the most appropriate to detect the patterns presented by sand dunes. We considered features based on the image gradient g(x) ∈ R2 computed at each image point x. The gradient vector is characterized by its amplitude |g(x)| and phase φ(x). These features are grouped into the following four sets: - HP(9): This corresponds to the features introduced by Dalal and Triggs [16] in their face detection problem, the histogram-of-oriented gradients or HOG features. They intend to capture the characteristic edge structure of the local shape of the dune, with a controlled degree of invariance to local geometric and radiometric factors. To obtain them, it is necessary to compute the weighted histogram for each cell which results from the multiplication of the gradient phase by its magnitude for each contributing point. Therefore, the histogram value associated to the k th cell C k is: hki = where bi (φ) =
|g(x)|.bi (φ(x)).
(1)
x∈C k
1, if φ ∈ ith bin 0, otherwise
For our problem, we defined an angular interval of 20◦ for the computation of the directional histograms, so we have a total of 81 features per block (9 histogram bins/cell × 9 cells). The phase histograms were not normalized. - HPM(9): This consists of a modified version of the HOG features, by using separately the phase histogram: hki =
bi (φ(x)).
(2)
˜bi (|g(x)|).
(3)
x∈C k
and the magnitude histogram: ˜k = h i where ˜bi (|g|) =
x∈C k
1, if |g| ∈ ith bin 0, otherwise
For the phase, with the same angular interval of 20◦ , we have 81 features (9 histogram bins/cell × 9 cells), and for the magnitude, considering 11 bins (resulting from a 4-unit interval between a minimum of 0 and a maximum of 40), we have 99 features (11 histogram bins/cell × 9 cells) Thus, for this set, a
5
total of 180 features per block are obtained. - HP: This refers to the histograms of the gradient phase for each image block B: hi =
bi (φ(x)).
(4)
x∈B
in the same 9 bins (angular interval of 20◦ ). Thus, 9 features (9 histogram bins/block × 1 block) are used in this situation. In this case, the phase votes were not weighted by the gradient magnitude. - HPM: It consists of using the histograms of phase, as defined previously for HP, and of magnitude of the gradient on each block separately. The histogram amplitude is given by: ˜i = h
˜bi (|g(x)|).
(5)
x∈B
Thus, it gives 9 features for the phase (1 histogram × 9 bins of 20◦ of angular interval) and 11 features for the magnitude (the same bins of the HP features). For this set, a total of 20 features are extracted. The size of each cell is the same for all images and is equal to 40 × 40 pixels. In order to have the features varying between 0 and 1, a normalization step was performed globally for each image and for each individual feature. For the classification of the blocks we used two of the most advanced and powerful classifiers that have already proven their ability in dealing with a variety of classification problems, namely in remotely sensed imagery of the Earth and other planetary surfaces: Boosting and Support Vector Machines (SVM). Boosting algorithms achieve remarkable results by combining a large number of weak classifiers, using weighted majority vote [17]. They are also able to performe feature selection i.e., to select a subset of informative features for a given problem. This can be done by assuming that each weak classifier depends on a single feature [18]. The application of boosting algorithms in object recognition lead to excellent results in, for instance, face [18] and impact craters [12] problems. SVM are kernel methods that use an implicit transformation to a higher dimensional space in order to achieve good separability by means of a linear classifier in the new space [19]. The hyperplane used for separation in the higher dimensional space is chosen in such a way that the so-called margin (the distance to the closest samples in each class) is maximized. The samples determining the margin are called the support vectors. Different transformation kernels, such as Gaussian, polynomial, linear and circular can be used, yielding different classifiers.
6
4 4.1
Results Dataset
To test our approaches, we have selected a set of 20 remotely sensed images captured by the Mars Orbiter Camera N/A (narrow angle mode) of the Mars Global Surveyor probe. Those images are from different locations on the planet, cover a total area of about 1320 km2 and are representative of the diversity of barchan dunes, one of the most common types on Mars that we have chosen to test these features and classifiers. These single-band images with 256 grey levels have a spatial resolution between 3.22 and 6.79 metres/pixel. Their dimension is variable, but a typical value is of about 1000 columns per 6000 lines. For each image we constructed ground-truth information, by manually delineating the dunes therein contained (examples are shown in Fig. 3a and Fig. 4a), indicating the ’dune’ and ’not-dune’ regions). The tiling of the ground-truth into cells was then performed. In this process, only the cells containing more than 30% of dune area were considered as ’dune’, whereas the cells with less than 10% of dune area were considered as ’not-dune’; the cells with dune areas comprised in the interval 10-30% were not considered (examples in Fig. 3b and Fig. 4b). 4.2
Evaluation
Every classifier was tested with each of the four sets of features using a 5-fold cross-validation, i.e., the total number of image blocks was divided into five subsets of the same size: four of them were used for training, the remaining one was used for testing. This procedure was repeated five times, so that each subset was used once for testing. For the tests with the SVM classifier we have used the freely distributed package SVMLight [20]. Several kernels were exploited, but among those with higher performances we chose the linear kernel since it is the most simple. The performance of each classifier with each one of the 4 sets of features is evaluated through the computation of the probabilities of false negatives (pF N = F N/(F N + T P )), false positives (pF P = F P/(F P + F N )) and of a global error (perror = pN .pF P + pP .pF N ), where F N stands for the number of false negative blocks, T N the number of true negative blocks, F P the number of false positive blocks, T P the number of true positive blocks, N the total number of negative blocks and P the total number of positive blocks. The classification output is illustrated in Fig. 3c and Fig. 4c with two distinct MOC images (R17-00333 and S01-00925). The overall performances obtained for all images are synthesized in Table 1. Globally, the values achieved are very good, with the majority of situations (6 out of 8) presenting probabilities of error below 0.024: these refer to the features HP(9), HPM(9) and HPM, with both classifiers. The exception is given by the HP features which, both for Boosting and SVM classifiers, attain a probability of error of 0.347 and 0.436. This means that the phase of the gradient is not, by itself, a discriminative feature.
7 Table 1. Performance of the two classifiers for the detection of Martian dunes. Boosting pF N pF P perror HP(9) 0.019 0.032 0.023 HPM(9) 0.017 0.016 0.017 HP 0.353 0.330 0.347 HPM 0.013 0.014 0.013
Features
pF N 0.0408 0.0362 0.6431 0.0341
SVM pF P perror 0.007 0.024 0.007 0.022 0.228 0.436 0.004 0.019
Although the best performances of each classifier are achieved with the same HPM features (0.013 for Boosting and 0.019 for SVM), their difference is not relevant when compared to the values obtained with HP(9) and HPM(9) features. There is some concordance in these results, since both classifiers perform excellently for the same sets of features (HP(9), HPM(9) and HPM) and both have a weak performance when using the HP features. The histogram of magnitude seems to be the most discriminative feature and no advantage is observed in this problem by splitting the image block into 9 cells.
5
Conclusions
The major conclusion put forth in this paper is that the adequacy of automated methods for dealing with the diversity of sand dunes on the Martian surface was verified, as correct detections with significant performances were achieved. The values obtained in this first experiment indicate that the key factor resides on the selection of the features that are more adequate for describing the characteristics of these aeolian structures. In particular, the amplitude of the gradient has proved to be the most informative feature. Although a set of powerful features and classifiers were successfully used on representative samples of the large diversity of Martian dune fields, we must remind that this is only a preliminary study. We have dealt with dune fields composed by individuals of different sizes, shapes and densities in distinct illumination conditions, but we are aware that many more different situations will have to be faced, namely considering the scale and the diversity of the Martian landscape where many other geomorphological features can and will sometimes be present. Nonetheless, we believe that the adaptive and learning nature of the methods we are using will be able to deal with those different circumstances. In future work we intend to greatly expand the datasets by incorporating images of every type of Martian dunes and testing on them the approaches we have employed here; we will also test additional types of features and classifiers. Moreover, and with the ultimate goal of making available a robust tool to be used in the cartography of Martian dunes at a planetary scale, we also intend to automatically classify the Martian dunes according to the scheme used in the classification of analogue terrestrial structures [15].
8
(a)
(b)
(c)
Fig. 3. First example of dune classification on part of the image R17-00333 (TP in green, TN in yellow, FN in red, FP in blue): (a) input image with overlapping of manual ground-truth; (b) ground-truth tiling in cells; (c) output of SVM classifier with HPM features [image credits: MSSS/NASA/JPL].
Acknowledgments. This work was partially supported by FCT Portugal under the pluriannual funding attributed to CERENA/IST and ISR/IST and the project PTDC/CTE-SPA/099041/2008. L. Bandeira (SFRH/BD/40395/2007) and J. Saraiva (SFRH/BD/37735/2007) acknowledge financial support by FCT Portugal.
9
(a)
(b)
(c)
Fig. 4. Second example of dune classification on part of the image S01-00925 (TP in green, TN in yellow, FN in red, FP in blue): (a) input image with overlapping of manual ground-truth; (b) ground-truth tiling in cells; (c) output of Boosting classifier with HP(9) features [image credits: MSSS/NASA/JPL].
References 1. Greeley, R., Kuzmin, R.O., Haberle, R.M: Aeolian processes and their effects on understanding the chronology of Mars. Space Science Reviews 96, 393-404 (2001). 2. Wilson, S., Zimbelman, J.: Latitude-dependent nature and physical characteristics of transverse aeolian ridges on Mars. Journal of Geophysical Research-Planets 109, E10003 (2004).
10 3. Edgett, K.S., Malin, M.C.: New views of Mars eolian activity, materials, and surface properties: three vignettes from the Mars Global Surveyor Mars Orbiter Camera. Planetary and Space Science 50, 151-155 (2000) 4. Hayward, R.K. Mullins, K., Fenton, L., Hare, T., Titus, T., Bourke, M., Colaprete, A., Christensen, P.: Mars global digital dune database and initial science results. Journal of Geophysical Research-Planets 112, E1107 (2007). 5. Hayward, R.K., Mullins, K.F., Fenton, L.K., Titus, T.N., Tanaka, K.L., Bourke, M.C., Colaprete, A., Hare, T.M., Christensen, P.R.: Mars global digital dune database (MGD3): User’s guide. In: Planetary Dunes Workshop: A Record of Climate Change, abs. 7013, Alamogordo, NM (2008). 6. Hayward, R.K., Mullins, K.F., Fenton, L.K., Titus, T.N., Bourke, M.C., Colaprete, T., Hare, T., Christensen, P.R.: Mars digital dune database: Progress and application. In: Lunar and Planetary Science XXXVIII, abs. 1360, Houston, TX (2007). 7. Barata, T., Alves E.I., Saraiva, J., Pina, P.: Automatic recognition of impact craters on the surface of Mars. Lecture Notes in Computer Science 3212, 489-496 (2004). 8. Bue, B.D., Stepinski, T.F.: Machine detection of Martian impact craters from digital topography data. IEEE Transactions on Geoscience and Remote Sensing 45(1), 265-274 (2007). 9. Bandeira, L. Saraiva, J., Pina, P.: Impact crater recognition on Mars based on a probability volume created by template matching. IEEE Transactions on Geoscience and Remote Sensing 45(12), 4008-4015 (2007). 10. Salamuni´ccar, G., Lonˇcari´c, S.: Open framework for objective evaluation of crater detection algorithms with first test-field subsystem based on MOLA data. Advances in Space Research 42(1), 6-19 (2008). 11. Salamuni´ccar, G., Lonˇcari´c, S.: GT-57633 catalogue of Martian impact craters developed for evaluation of crater detection algorithms. Planetary and Space Science 56(15), 1992-2008 (2008). 12. Martins, R., Pina, P., Marques, J.S., Silveira, M.: Crater detection by a boosting approach. IEEE Geoscience and Remote Sensing Letters 6(1), 127-131 (2009). 13. Urbach, E.R., Stepinski, T.F.: Automatic detection of sub-km craters in high resolution planetary images. Planetary and Space Science 57(7), 880-887 (2009). 14. Stepinski, T.F., Mendenhall, M.P., Bue, B.D.: Machine cataloging of impact craters on Mars. Icarus 203(1), 77-87 (2009). 15. McKee, E.D.: Introduction to a study of global sand seas. In: McKee E.D. (ed.): A study of global sand seas, pp. 1-19, University Press of the Pacific, Honolulu (1979). 16. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR2005-Computer Vision and Pattern Recognition Conference, Vol. 1, pp. 886893, IEEE Press, New York (2005). 17. Schapire, R.E., Freund, Y., Bartlett, P., Lee, W.S.: Boosting the margin: a new explanation for the effectiveness of voting methods. Annals of Statistics 26(5): 1651-1686 (1998). 18. Viola, P., Jones, M.: Robust real-time face detection. International Journal of Computer Vision 57(2), 137-154 (2004). 19. Vapnik, V.N.: The nature of statistical learning theory. Berlin, Springer (1995). 20. Joachims, T.: Estimating the generalization performance of an SVM efficiently. In: Proceedings of the Seventeenth International Conference on Machine Learning, pp. 431-438, Morgan Kaufmann Publishers Inc., San Francisco, CA (2000).
