A SHADOW DETECTION METHOD FOR REMOTE SENSING IMAGES USING VHR HYPERSPECTRAL AND LIDAR DATA G. Tolt*, M. Shimoni** and J. Ahlberg* * FOI (Swedish Defence Research Agency), SE-581 11, Linköping, Sweden -
[email protected];
[email protected] ** Signal and Image Centre, Dept. of Electrical Engineering (SIC-RMA), 1000 Brussels, Belgium
[email protected] ABSTRACT 1.1 Previous work In this paper, a shadow detection method combining hyperspectral and LIDAR data analysis is presented. First, a rough shadow image is computed through line-of-sight analysis on a Digital Surface Model (DSM), using an estimate of the position of the sun at the time of image acquisition. Then, large shadow and non-shadow areas in that image are detected and used for training a supervised classifier (a Support Vector Machine, SVM) that classifies every pixel in the hyperspectral image as shadow or nonshadow. Finally, small holes are filled through image morphological analysis. The method was tested on data including a 24 band hyperspectral image in the VIS/NIR domain (50 cm spatial resolution) and a DSM of 25 cm resolution. The results were in good accordance with visual interpretation. As the line-of-sight analysis step is only used for training, geometric mismatches (about 2 m) between LIDAR and hyperspectral data did not affect the results significantly, nor did uncertainties regarding the position of the sun. Index Terms— Shadow detection, hyperspectral, LIDAR, DSM, supervised classification, SVM. 1. INTRODUCTION Digital imagery with high spectral and spatial resolution acquired from aerial platforms provides urban land cover mapping at very fine scales as well as with detailed information about the cover materials. Such data thus give new possibilities for detection and classification, but it is also a challenge how the data should be exploited. One challenge is to detect and/or classify shadowed targets, where the shadowing results in reduction or total loss of spectral information [1-3]. The problem of shadowing is particularly significant in very high spatial resolution (VHR) imagery of urban environments, where elevation varies dramatically across short distances [1, 4]. With the dominance of elevated objects such as buildings, bridges, towers and trees in the landscape, the proportion of the imagery that is affected by shadowing is significant [3, 5].
Considerable research has been conducted to investigate shadow detection, and several algorithms are available in the literature, such as invariant color models [6, 7], histogram analysis/thresholding [1, 8], 3D modeling [2, 9] and hierarchical shadow detection algorithms [10]. For shadow restoration, a variety of radiometric enhancement (or restoration) methods have been used, including linear correlation correction (LCC) [1, 8, 11], Gamma correction [11], histogram matching [1, 7, 11], Markov random fields [17], invariant color models [7], affinity propagation algorithm [18] and companion area intensity mapping [2]. Multisource data fusion has also been applied to remove shadows from high resolution imagery while combining spatial information such as adjacency relations with spectral resolution [3, 5, 12]. Shadow removal/identification methods have been developed for atmospherically corrected hyperspectral data using linear unmixing [13], region growing [14] or matched filtering methods [15, 16]. Each of these methods has limitations such as undercorrection for shadow at long wavelengths and overcorrection at short wavelengths [13, 16], restriction to scenes where the percentage of cloud coverage is less than 25% [14] and declination of deep shadow condition [16]. 1.2 Contribution and outline In an effort to overcome the above-mentioned limitations of shadow detection, we propose the use of a complementary 3D LIDAR data set, providing additional insight about shadow delineation in the scene. The primary application for the shadow detection is to serve as input for detection of difficult targets in deep shadow areas [18]. The outline of the paper is as follows. Section 2 describes the data sets, Section 3 describes the proposed method for combining hyperspectral and 3D data, Section 4 shows the results, and finally Section 5 contains our conclusions.
2. STUDY AREA AND DATA SETS The study area of this work is the town of Norrköping, Sweden. The total data set covers an area of approximately 10 km2, comprised of private and shared residential areas at different heights, commercial areas, green areas (grass, bushes and trees), roads and a river. The area is characterized by flat topography and flourishing vegetation. Norrköping represents typical European urban cover characteristics. VHR hyperspectral and LIDAR data have been collected simultaneously on September 2nd 2010, 08:00–08:30 UTC, using the hyperspectral push-broom sensor Itres CASI 1500 and the LIDAR sensor Optech ALTM Gemini from an altitude of approximately 750 m. The CASI sensor collected data in 24 equally sized spectral bands from 381.9 nm to 1040.4 nm. The pixel resolution of the orthorectified hyperspectral data was 0.5 m. From the LIDAR 3D data a digital surface model (DSM) with 0.25 m resolution was created. 3. METHODOLOGY The proposed shadow detection method is designed to be a quite simple and intuitive yet effective way of fusing DSM and hyperspectral analysis tools. The method is divided into four steps: Initial shadow estimation using spatial analysis of the DSM, identification of shadow/non-shadow training regions, shadow detection based on supervised classification and, finally, post-processing of the classification result. 3.1 Initial shadow estimation through spatial analysis If an ideal 3D model of the scene was available, and if the registration between elevation data and imagery was perfect, the shadows at a certain time could be computed using geometrical calculations and then overlaid onto the hyperspectral imagery (HSI). However, the quality of the elevation data is typically not accurate or detailed enough to allow for an approach based only on geometry, especially not if the 3D data is acquired with airborne sensors. In addition, there is often a certain degree of geometric mismatch between the elevation data and the imagery. Still, even a coarser and less accurate Digital Surface Model (DSM) provides valuable information, since a rough geometry-based shadow estimate can serve as a valuable input to subsequent processing. The process starts by computing an initial shadow mask, Sinit, through straight-forward line-of-sight analysis using the DSM data and an estimate of the sun position. A pixel p in the DSM is marked as non-shadow if a straight line from p to the sun does not intersect the DSM. In order to suppress the influence of noisy elevation values, pixels within a short distance from p are neglected. The influence of noisy pixels is otherwise particularly noticeable for low slant angles.
3.2 Identification of training regions Although we cannot expect shadows estimated only through geometrical computations to correspond perfectly to the true shadows (i.e., as observed in the hyperspectral data), we note that the estimation is quite reliable for the interior of large shadow and non-shadow regions, even in case of moderately erroneously registration (about 1-2 m geometric mismatch). Thus, the initial shadow mask is quite useful for detecting regions of interest for the spectral processing that can be used for training shadow detector working in the spectral domain. Therefore, we detect the interior of large shadow and non-shadow regions by applying a standard distance transformation followed by thresholding. The final shadow and non-shadow regions form a shadow training image, Straining. 3.3 Shadow detection through spectral analysis The shadow/non-shadow regions in Straining are used for training a supervised classifier. Hence, the role of Straining is to “teach” a spectral shadow detector the spectral pattern of a shadow in the HSI data set. As a detector, we use a Support Vector Machine (SVM) as it produces satisfactory results even by using small training samples (only support vectors), less sensitive to space dimensionality and hence it overcomes the Hughes' Phenomenon and is an effective tool in classification of hyperspectral data [19]. The result of the classification step is a shadow image Sclass, in which every pixel is classified as either shadow or non-shadow. Potentially, a confidence measure from the classification could be used to identify pixels that have been classified with less certainty. 3.4 Post-processing of classification result In order to correct misclassified pixels, we post-process Sclass by filling small holes in shadow regions. Such small holes typically correspond to high reflectance objects (cars), which appear brighter than the majority of shadow pixels. 4. RESULTS In Figure 1, a result of the applied methodology is shown. We note that the initial shadow mask (a), obtained through line-of-sight analysis of the DSM, contains errors mainly in borders of large shadow areas and of small objects. Figure 1 (b) shows the result after thresholding the distance transform image, resulting in the detection of the interior of larger shadow and non-shadow regions, respectively. After the SVM classification (d) has been applied to the hyperspectral data using the regions from (c) as training areas, dark objects are well distinguished from shadows and the shapes of the segmented shadows are preserved well, as seen from a comparison with the image in Figure 1 (c). The factor that affects the choice of distance threshold value the most is the
degree of the mismatch between the DSM and the orthorectified hyperspectral data. For our data, choosing a threshold value of 2 m was satisfactory; small enough to provide shadow/non-shadow regions for successful training of the classifier, while large enough to cope with misregistration errors.
Four test regions, corresponding to a total of 1 km2 were evaluated and all major shadows were correctly identified (according to visual inspection). Moreover, the shape and extent of the detected shadows and the actual shadows in the hyperspectral imagery were in good accordance.
Figure 1. Top left: Initial shadows estimated through line-of-sight analysis of lidar DSM. Top right: Interior of large regions (black – shadow, white – non-shadow, gray –not used for training). Bottom left: RGB image (three bands of a 24 band hyperspectral image). Bottom right: Final shadow image after SVM classification and post-processing. The inset shows the (magnified) result for a small part of the image.
5. CONCLUSIONS This paper presents a new methodology for shadow detection based on initial shadow estimation using a LIDAR DSM data set, followed by refinement of the
result through supervised classification of the VHR hyperspectral imagery. Through line-of-sight analysis of the DSM data and standard image processing techniques, regions of interest are detected, that serve as training samples for the
supervised classifier (SVM). The results show that the proposed method can distinguish small dark objects from shadows and that the shapes of the segmented shadows are preserved well. The method is simple yet effective and gives the advantage of using a learning mechanism (a supervised classifier) and training samples that can be extracted quite accurately from the DSM data. The method is robust against moderate geometric mismatches between LIDAR data and optical imagery, as long there is overlap between large shadow and nonshadow areas in the spatial and spectral data. A limitation with the current approach is that it produces a binary shadow image; it does not capture the gradual transition zone between shadow and non-shadow regions that appear as an effect of the sun being a distributed light source. That phenomenon becomes more accentuated with increasing spatial resolution and whether it is a practical limitation depends on the intended use of the detected shadows. However, our primary application is the detection of difficult targets in shadow areas, and therefore we are primarily interested in deep shadows. The proposed method fulfils those requirements but for other applications, the ability to accurately model the degree of shadow between 0 and 1 may be more important and is therefore considered as a future improvement. 6. REFERENCES [1] P.M. Dare, “Shadow analysis in high-resolution satellite imagery of urban areas”, Photogrammetric Engineering and Remote Sensing, 71(2), pp. 169−177, 2005. [2] Y. Li, P. Gong and T. Sasagawa, “Integrated shadow removal based on photogrammetry and image analysis”, International Journal of Remote Sensing, 26(18), pp. 3911−3929, 2005. [3] W. Zhou, G. Huang, A. Troy and M.L. Cadenasso, “Object-based land cover classification of shaded areas in high spatial resolution imagery of urban areas: A comparison study”, International Journal of Remote Sensing, 113, pp. 1769–1777, 2009. [4] F. Yuan, “Land-cover change and environmental impact analysis in the Greater Mankato area of Minnesota using remote sensing and GIS modeling”, International Journal of Remote Sensing, 29(4), pp. 1169−1184, 2008. [5] F. Yuan and M.E. Bauer, “Mapping impervious surface area using high resolution imagery: a comparison of object-oriented classification to perpixel classification”, In Proceedings of American Society of Photogrammetry and Remote Sensing Annual Conference, Reno, NV, May, 2006. [6] E. Salvador, A. Cavallaro and T. Ebrahimi, “Shadow identification and classification using invariant color models”, In IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1545−1548, 2001.
[7] V.U.D. Tsai, “A comparative study on shadow compensation of color aerial images in invariant color models”, IEEE Transactions on Geoscience and Remote Sensing, 44(6), pp. 1661−1671, 2006. [8] Y. Chen, D. Wen, L. Jing and P. Shi, “Shadow information recovery in urban areas from very high resolution satellite imagery” International Journal of Remote Sensing, 28(15), pp. 3249−3254, 2007. [9] J.Y. Rau, N.Y. Chen and L.C. Chen, “True orthophoto generation of built-up areas using multi-view images”, Photogrammetric Engineering & Remote Sensing, 68(6), pp. 581−588, 2002. [10] J. Yao and Z. Zhang, “Hierarchical shadow detection for color aerial images”, Computer Vision and Image Understanding, 102, pp. 60−69, 2006. [11] P. Sarabandi, F. Yamazaki, M. Matsuoka and A. Kiremidjian, “Shadow detection and radiometric restoration in satellite high resolution images”, In Proceedings of IGARSS 2004, Anchorage, Alaska, September 2004. [12] W. Zhou, A. Troy and J.M. Grove, “Object-based land cover classification and change analysis in the Baltimore metropolitan area using multi-temporal high resolution remote sensing data”, Sensors, 8, pp. 1613−1636, 2008. [13] J.W. Boardman, “Automated spectral unmixing of AVIRIS data using convex geometry concepts. In Summaries of Fourth JPL Airborne Geoscience Workshop, Washington, USA, 95-26(1), pp. 11–14, October 1993. [14] M.R. Davenport and W. Ressl, “Shadow removal in hyperspectral imagery,” In Proceedings of the International Symposium on Spectral Sensing Research, Las Vegas, October 1999. [15] S.M. Adler-Golden, M.W. Matthew, G.P. Anderson, G.W. Felde and J.A. Gardner, “An algorithm for deshadowing spectral imagery”, In Proceedings of the 11th JPL Airborne Earth Science Workshop, Pasadena, USA, March 2002. [16] S. Le Hégarat-Mascle and A. Cyrille, “Use of Markov Random fields for automatic cloud/shadow detection on high resolution optical images”, ISPRS Journal of Photogrammetry and Remote Sensing, 64, pp. 351366, 2009. [17] H. Xia, X. Chen and P. Guo, “A Shadow detection method for remote sensing images using affinity propagation algorithm”, In Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, San Antonio, TX, USA, October 2009. [18] M. Shimoni, G. Tolt, C. Perneel and J. Ahlberg, “Detection of vehicles in shadow areas”, in Proceedings of IEEE WHISPERS 2011, Lisbon, Portugal, June 2011. [19] M. Pal and P. M. Mather, Assessment of the effectiveness of support vector machine for hyperspectral data, Future generation computer systems, 20(7), pp. 1215-1225, 2004.
