Validation of Geometric Accuracy of Global Land Survey (GLS) 2000 Data Rajagopalan Rengarajan, Aparajithan Sampath, James Storey, and Michael Choate
Abstract
The Global Land Survey (GLS) 2000 data were generated from Geocover™ 2000 data with the aim of producing a global data set of accuracy better than 25 m Root Mean Square Error (RMSE). An assessment and validation of accuracy of GLS 2000 data set, and its co-registration with Geocover™ 2000 data set is presented here. Since the availability of global data sets that have higher nominal accuracy than the GLS 2000 is a concern, the data sets were assessed in three tiers. In the first tier, the data were compared with the Geocover™ 2000 data. This comparison provided a means of localizing regions of higher differences. In the second tier, the GLS 2000 data were compared with systematically corrected Landsat-7 scenes that were obtained in a time period when the spacecraft pointing information was extremely accurate. These comparisons localize regions where the data are consistently off, which may indicate regions of higher errors. The third tier consisted of comparing the GLS 2000 data against higher accuracy reference data. The reference data were the Digital Ortho Quads over the United States, ortho-rectified SPOT data over Australia, and high accuracy check points obtained using triangulation bundle adjustment of Landsat-7 images over selected sites around the world. The study reveals that the geometric errors in Geocover™ 2000 data have been rectified in GLS 2000 data, and that the accuracy of GLS 2000 data can be expected to be better than 25 m RMSE for most of its constituent scenes.
Introduction
The Global Land Survey (GLS) 2000 data set is a collection of images acquired by the Landsat 7 ETM+ sensor, and is the geodetic reference for all Landsat products archived at the US Geological Survey Earth Resources and Observation Science (EROS) Center. The goal of this paper is to provide an assessment of the geometric accuracy of the GLS 2000 data set and compare its accuracy relative to the Geocover™ 2000 data set. The Geocover™™ data sets (1975, 1990, and 2000) were the predecessor to GLS data sets. Studies (Franks et al., 2009; Masek and Covington, 2007) reveal that Geocover™ 2000 data have substantial errors (>100 m) on a per-scene and per-pixel level (Masek and Covington, 2007). Therefore, the Geocover™ 2000 data set images were reprocessed using all available ground control points, Landsat-7 definitive ephemeris, and tie points in a block configuration to create the GLS 2000 reference data set. In addition to the Geocover™ 2000 Aparajithan Sampath, James Storey, and Michael Choate are with Stinger Ghaffarian Technologies (SGT), Contractor to US Geological Survey, Earth Resources Observation and Science (EROS) Center, Mundt Federal Building, 47914 252nd Street, Sioux Falls, SD 57198 (
[email protected]). Rajagopalan Rengarajan is a Graduate Student at the Rochester Institute of Technology, Rochester, NY 14623.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
data set, Geocover™ data sets for 1975 and 1990 were also reprocessed and registered to the GLS 2000 data set forming GLS-1975 and 1990 data sets. New GLS 2005 and 2010 data sets have since been created. The GLS data base is freely available (through EarthExplorer: http://earthexplorer.usgs.gov/) global data base selected for each epoch (1975, 1990, 2000, 2005, and 2010) for minimum cloud cover and at peak greenness. They are used in a variety of applications needing a global time series images and provides a baseline for many global change studies (Gutman et al., 2008). The GLS 2000 data set is also the geographic reference for Landsat Data Continuity Mission (LDCM) and serves as standard reference data for a number of satellite data providers around the world (Chander et al., 2010; Sampath, 2012). This makes the task of validating the accuracy of the GLS 2000 data set very important. The most commonly used methods of validating the geometric accuracy of remote sensing data sets involve comparing them against reference data sets of higher accuracy (Chander et al., 2010; Gianinetto and Scaioni, 2008; Li et al., 2007; Sampath, 2012), or using available ground control points (Aguiller et al., 2012; Muller et al., 2012). However, the global coverage of the data sets makes it hard to assess the accuracy, as other independent data sets of comparable accuracy and coverage, or ground control points are limited in availability. To address the challenges presented by the global coverage of the data, the validation task was divided into three parts. In the first part (or tier), the GLS 2000 data were compared with the Geocover™ 2000 data. In the second tier, the data were compared with Landsat-7 (L7) Level 1G systematic products (radiometrically and geometrically corrected products, without the use of terrain or ground control) acquired during April 2005 through December 2006 when the gyros of the satellite performed extremely well; the platform was considered stable, and the resulting data products were of high accuracy (Lee et al., 2004). The third tier of tests consisted of testing GLS 2000 data against higher accuracy reference data sets, wherever they were available. Most of the available ground control points were used to generate GLS 2000 data, hence they were not used in this assessment. The reference data sets included data from Digital Ortho Quads and SPOT orthorectified data from Australia’s National Earth Observation Group. In other parts of the world, the GLS 2000 data were validated using a triangulation-based satellite bundle adjustment algorithm. Similar techniques, based on rigorous sensor modeling, have been used by many researchers for improving the accuracy of satellite data (Lutes and Grodecki, 2004; Li Photogrammetric Engineering & Remote Sensing Vol. 81, No. 2, February 2015, pp. 131–141. 0099-1112/15/812–131 © 2014 American Society for Photogrammetry and Remote Sensing doi: 10.14358/PERS.81.2.131
Fe b ru a r y 2015
131
et al., 2007; Habib et al., 2008; Thoutin 2003). The algorithm uses observations from multiple L7 Level 1G data and GLS 2000 data to simultaneously adjust the platform’s trajectory, and attitude using least squares framework.
Global Land Survey Data
The GLS is a partnership between the US Geological Survey (USGS) and the National Aeronautics and Space Administration (NASA), supporting the US Climate Change Science Program (CCSP) and the NASA Land-Cover and Land-Use Change (LCLUC) Program (Gutman et al., 2008; NASA, 2008, USGS, 2008). Characterizing and monitoring the global changes in land cover at a moderate-resolution is the key application of the GLS data sets (NASA and USGS, 2008). The GLS data sets are global land cover mosaics of cloud-free Landsat data collected around epochs of 1975, 1990, 2000, 2005, and 2010. These freely available data sets provide the scientific community a comprehensive view of the Earth’s land over the past 40 years. The GLS 2000 data set consists of approximately 8,700 images collected with the Landsat-7 ETM+ sensor. The GLS 2000 data were generated, for the most part, using the same images/ scenes as in the earlier Geocover™ 2000 data set. The images/ scenes were acquired between the years 1999 and 2002 with only two scenes acquired in January of 2003 (Plate 1). The GLS 2000 data were generated after several independent studies revealed deficiencies with the geometric accuracy of the Geocover™ 2000 data (Smith et al., 2005; Dykstra and Storey, 2004). Substantial errors (>100 m) were discovered on a per-image and per-pixel level (Masek and Covington, 2007), which were attributed to the use of lower resolution digital elevation models (DEMs), use of spatially disconnected bundles (in the adjustment of blocks), and a lack of sufficient ground control points. It was decided to reprocess the entire GeoCove database (1975, 1990, and 2000) using Shuttle Radar Topography Mission (SRTM) DEM and additional ground control from ETM+ definitive ephemeris. The ETM+ scenes that make up the Geocover™ 2000 data set were
reprocessed using block triangulation, SRTM DEM, additional ground control, and Landsat-7 definitive ephemeris. The Geocover™ 2000 Landsat-7 data layer was used as the geometric framework for the readjustment because it was considered internally more consistent than the other layers (1975 and 1990) (Franks et al., 2009). The reprocessed data set was called the Global Land Survey.
GLS 2000 Data Geometric Accuracy Validation Methodology
The GLS 2000 data were generated with a stated geometric accuracy of 25 m RMSE or better on a per-image basis. A comprehensive testing scheme was devised to assure the geometric accuracy of the data. The testing scheme consisted of three tiers, and the analysis is divided into 11 blocks. These blocks represent large landmasses (mostly contiguous, with the exception of islands), and reflect approximately the triangulation blocks used to generate the data sets by MDA Federal, Inc. Figure 1 is a representation of the blocks. Each block is stored as a collection of WRS-2 path/rows that are distributed on a continental scale. The WRS (Worldwide Reference System) is a global notation used to spatially reference Landsat images. Since the Landsat satellite is sun synchronous, passing over the same location on the ground every 16 days, Landsat images for any portion of the Earth can be uniquely identified by designating a path/row combination (NASA, n.d.).
Tier 1 Test Methodology
Tier 1 testing was designed to provide a global guide to the locations where GLS 2000 and Geocover™ 2000 data are not co-registered. Since Geocover™ 2000 data were reprocessed to GLS 2000 data, the lack of co-registration is a likely indicator of locations where the reprocessing was most effective and hence data in these locations are likely to be the ones where Geocover™ 2000 had the most error. It is important to identify these regions and assure users that the errors in the Geocover™ 2000 have been fixed (Tier 2 and Tier 3 analysis). The tests consisted of comparing the Geocover™ 2000
Plate 1. The acquisition years of GLS-2000 images on a global scale.
132
Feb rua r y 2 015
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Figure 1. Blocks of GLS-2000 data designed for testing and analysis (different shades/patterns represent different blocks). data with the GLS 2000 data using image matching techniques between the two data sets. The comparisons provide an assessment of geometric characteristics of the imagery. For high-resolution remote sensing imagery, this is usually accomplished by using well surveyed photo-identifiable targets on the ground, and comparing their geodetic coordinates with those obtained from the image under study. For mediumto-coarser resolution images, the accuracy is best assessed by using automated cross-correlation based image matching techniques between a (well characterized) reference image and the image with unknown accuracy (search image). The ground coordinates of features from the reference image are compared with the corresponding coordinates obtained from the search image. Plotting the points measured between the two images helps assess any systematic bias or higher order distortion within the search image. The comparison between a reference image and a search image is performed using a similarity measure. In this case it is the normalized crosscorrelation measure: CC ( l, s ) =
∑ (I (i, j ) − I ) (I (i, j ) − I ) ∑ (I (i, j ) − I ) ∑ (I (i, j ) − I ) i, j
R
R
S
S
2
R
R
(1)
2
S
S
where, CC(l,s) is the normalized cross-correlation coefficient at location (l,s), IR and IS refer to the intensities associated – – with reference and search image windows, I R and I S refer to the mean intensities within the window. The image matching process is implemented in two different ways for the tests in this research, depending on the way test point locations are selected. In the first method, test points are selected uniformly in the reference image and in the second method test point locations are selected using the Modified Morvec Interest Operator algorithm (MMIO, USGS n.d). The MMIO algorithm selects points/pixels that are easily
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
identifiable in the reference and the search image. Once the test points are identified (using either uniform sampling or MMIO algorithm), image chips are also extracted from the search image around the pixel that corresponds to the sampled pixels in the reference image using image geolocation/ header information. In each case, a 64 × 64 (or 32 × 32) pixel sized image chip is extracted around the check points in the reference image, and a 3 × 3 window centered on the check point in the search image is used to determine the normalized cross-correlation value. The image chips are extracted to restrict the search space in the reference image. The location of the maximum normalized cross-correlation in the search image chip indicates the location of the pixel corresponding to the center of the reference image chip. The maximum (or peak) of the correlation is measured to sub pixel level by fitting a three-dimensional surface on the normalized crosscorrelation values and determining the location of the maximum of the surface. If the maximum value of cross-correlation is less than a threshold (0.55), the check point is rejected. The relative error between the reference and the search image is taken to be the offset between the peak’s location on the correlation surface and the center of the study image chip. In Tier 1 testing, GLS 2000 data were considered to be the reference data. In Tier 3, Digital Ortho Quads (DOQ) and SPOT ortho-rectified data over Australia were considered to be the reference data.
Tier 2 Test Methodology
Tier 2 tests measure the relative geometric discrepancy between Landsat-7 (L7) Level 1G systematically corrected images and GLS 2000 data. The tests consist of two parts. In the first part, the geodetic accuracy of L7 Level 1G data were analyzed (to ensure that their accuracy is sufficient to be used as reference data), and in the second part, the accuracy of GLS 2000 data with respect to L7 Level 1G data was analyzed. The on-orbit geometric accuracy of the ETM+ sensor has been monitored constantly from its inception and is an
Fe b ru a r y 2015
133
Figure 2. Plot of ETM+ Level 1G geometric accuracy (RMSE) values attained for data acquired from April 2005 through December 2006. Table 1. Quarterly Assessment of the On-orbit Geometric Performance of etm+ Sensor Quarter
Mean Along Track Mean Across Track (AT) in Meters (XT) in Meters
STD (AT) in Meters
STD (XT) in Meters
RMSE (AT) in Meters
RMSE (XT) in Meters
RMSE in Meters
2Q05
0.1
4.7
6.1
4.7
10.6
17.4
20.4
3Q05
0.1
8.1
6.3
4.3
11.2
15.3
19.0
4Q05
6.4
2.4
5.8
5.0
13.2
22.0
25.7
1Q06
-5.2
-2.4
5.4
4.3
10.5
16.7
19.8
2Q06
-11.3
-12.9
7.1
4.4
16.9
21.3
27.2
3Q06
-16.9
-15.1
6.9
4.5
21.6
24.5
32.7
4Q06
19.9
2.2
6.2
4.7
25.6
17.1
30.8
1Q07
7.2
2.0
5.9
5.1
19.8
23.4
30.6
ongoing quality control process at USGS EROS. The ETM+ sensor’s on-orbit geodetic accuracy has been reported to be around 54 m (Lee et al., 2004) from launch (against the designed 250 m), and in the period between April 2005 and December 2006 (Figure 2 and Table 1), the spacecraft experienced good gyro performance (e.g., stable drift rates) and the platform was considered to have excellent pointing knowledge. Table 1 shows the RMSE for each quarter from second quarter of 2005 (April 2005) through first quarter of 2007 (March 2007) from the geodetic accuracy characterization using the Landsat Image Assessment System (IAS) (USGS, 2006). The geometric algorithms of the Landsat-7 IAS map the coordinates from the ETM+ sensor image space (band sample, band line, etc.) to the geodetic object space (real world coordinates, i.e., latitude, longitude, and height). The geodetic accuracy characterization process estimates the accuracy of the L7 Level 1G products using the DOQ supersites as reference data. The spatial errors in the DOQ supersite reference data are known to be less than 7.6 m RMSE (Greenfeld, 2001). Figure 2 shows a plot of the RMSE values for 580 L7 Level 1G scenes/images obtained from the IAS geodetic accuracy characterization procedure. The X-axis represents the timeline for acquisition of the data, and the Y axis represents the RMSE (in meters) associated with each scene trended in IAS system. The diamond and the square dots represent the across track and the along track RMSEs. Table 1 shows that the ETM+ scenes were accurate to 30 m RMSE between the second quarter of 2005 through the fourth quarter of 2006. Since the accuracy of the ETM+ systematic products in this time period is very close to the expected accuracy of GLS 2000 data set, the ETM+ systematic products
134
Feb rua r y 2 015
were used for validating the GLS 2000 data. Once the accuracy of Level 1G data has been verified, they are compared with the GLS 2000 data, in much the same way as they are compared against the DOQs in the geodetic characterization procedure (USGS, 2006). This allows us to use the operational geodetic characterization process using the Landsat Product Generation System (LPGS) to test the GLS 2000 data on a global scale. The elevation for the GLS 2000 data are derived from the GLS DEM (USGS 2008) provided by MDA Federal, Inc. While the results of comparing the GLS 2000 and the L7 Level 1G data do not provide an absolute accuracy assessment of the GLS 2000 data, they do serve as an important indicator of their accuracies around the world. Since the trajectory and pointing knowledge of the ETM+ sensor is accurate, it can be expected that in most cases, the GLS 2000 and Level 1G data should be well aligned and (geometrically) consistent with each other. In these comparisons, multiple Level 1G images were used wherever possible (i.e., wherever multiple Level 1G data were available for a give path/row), to eliminate the possibility of an anomalous Level 1G image. Root Mean Square Differences (RMSD) between a Level 1G image and its corresponding GLS 2000 scene were generated and used as a measure of accuracy. In the case of multiple Level 1G images vis-à-vis a given path/row, the Median Absolute Deviation (MAD), which is a robust measure of standard deviation (Ellison et al., 2009), was used to eliminate anomalous images. The median RMSD value (RMSDMedian) determined from the results of multiple comparisons, and the MAD estimator was computed as σRMSDMAD = Median of |RMSDi – RMSDMedian|. The image comparisons are treated as anomalous (outliers) according to PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
(a)
(b)
(c)
Figure 3. A schematic representation of the triangulation based bundle adjustment: (a) before adjustment, (b) adjustment with attitude correction, and (c) adjustment with attitude and ephemeris correction. the following criterion: > Ze, image RMSD measurement is an outlier Ztei , ≤ Ze, image RMSD measurement is acceptable where Ztei =
(2)
RMSDi − RMSDMedian and Ze is a error threshold. σ RMSDMAD
The final RMSD for each path/row was computed as mean of RMSD of the images determined to be inliers.
Tier 3 Test Methodology
In Tier 3, the absolute accuracy of GLS 2000 images are confirmed by comparing them against data sets of higher accuracy, wherever they are available. These higher accuracy data sets consisted of DOQs over the United States, SPOT satellitebased data over Australia (supersites), and high accuracy check points generated through a triangulation-based bundle adjustment procedure. The DOQs and the Australian supersite data were tested against the GLS 2000 data using the image matching procedure detailed above. The bundle adjustment procedure is detailed below. The L7 Level 1G data sets used in Tier 2 testing were chosen during a period when the sensor platform was extremely stable. However, there will still be slowly varying errors in platform position and pointing knowledge, which cannot be eliminated. To reduce the effect of these errors, a triangulation-based bundle adjustment strategy, originally devised to generate high accuracy check points for calibration/validation of Landsat-8, was used. This strategy aims to simultaneously determine the solution to the orientation and the location information of the sensor, based on observations of a large number of points visible in multiple images. By utilizing observations from multiple images acquired from different orbits, ephemeris, and attitude knowledge errors may be reduced. The solution thus obtained is generally more geometrically robust. A least squares adjustment allows the PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
integration of a number of image point observations (i.e., measurements of the locations of selected ground targets in multiple satellite images), sparse ground control information, and our a priori knowledge of the Landsat spacecraft position and attitude, to compute corrections to these imprecisely known a priori ground data and image measurements. Figure 3 shows the process schematically. The point ‘g’ on the ground is observed in both the images shown in Figure 3 (i.e., g1, g2). The coordinates of the point ‘g’ can be obtained directly from the systematically corrected Level 1G products, based on the ephemeris (position and velocity), the attitude data, and using a known DEM. The accuracy of these points is dependent on the accuracy of the pointing knowledge and the elevation source used. If there is a significant systematic bias in the pointing knowledge, it will result in inaccurate horizontal and vertical coordinates. In the bundle adjustment procedure, conjugate ground points of ‘g’, i.e. g1, g2, etc. are identified in overlapping L7 Level 1G scenes/images, and the ephemeris and the attitude are adjusted simultaneously, to determine the “best” (in the least squares sense) estimate of the ground coordinate of ‘g’. Before any adjustment, as shown in Figure 3, the systematic coordinates of the same ground point (X shaped dot) are geographically located at distinct locations for the two images (dots without X). This difference in location is due to the systematic bias and random error in the platform’s pointing knowledge. When all the platform’s orientation parameters for ephemeris and attitude are simultaneously adjusted in bundle adjustment, the ground point coordinate’s uncertainty is reduced. This reduction in uncertainty helps to identify the net residual error in the ground point’s coordinates and thereby helps validate the ground reference data set. In actual tests, five to ten images (instead of two as shown in Figure 3) over a test area are selected, and their ephemeris and attitude are simultaneously adjusted. Each set of images uses several ground points (checkpoints) visible in all the images. These checkpoints are selected automatically using the Moravec point selection operator.
Fe b ru a r y 2015
135
While the triangulation based bundle adjustment method can be used to solve for all three coordinates of a checkpoint (latitude, longitude, and elevation/height), in this study, the heights were obtained from SRTM-based DEM because Landsat’s cross-path image intersection is weak, and can yield imprecise heights. Therefore, the least-squares solution was constrained with the elevation coordinates from SRTM DEM data. The triangulation procedure involves iteratively adjusting the spacecraft position, velocity (represented by X,Y,Z and Vx,Vy,Vz, respectively, in Figure 3), attitude, and attitude rate (represented by Ω, φ, κ and Ωr, φr, κr, respectively) corrections and the ground point coordinates so that the residual errors in the measured image points are minimized. The correction model thus consists of six ephemeris correction parameters (three position and three velocity) for each satellite pass that is observed, six attitude correction parameters (three bias and three rate) for each image that is observed, and three position correction parameters for each ground point. It should be noted that the velocity and attitude rate terms are part of the general triangulation model developed for Landsat-8. In this particular application those terms were given relatively high a priori weights and, so, were not allowed to adjust much. The initial estimates of the position and velocity and the attitude are obtained from the Landsat Mission Operations Control (MOC). The initial estimates of the coordinates of the ground check points extracted by the modified Moravec operator are obtained from the corresponding GLS 2000 scene. The error in checkpoints, therefore, is the difference in coordinates between the initial estimates and the final adjusted value. In our implementation of the bundle adjustment process, the ephemeris correction was limited to one set of parameters per pass to account for the fact that ephemeris errors are highly time correlated due to the dynamics of the spacecraft orbit. Restricting the ephemeris correction parameters to one set per pass also helps de-correlate the ephemeris corrections from the attitude corrections. Attitude corrections are introduced for each image, to account for the more rapidly varying attitude errors, but the parameters for all images from the same pass are linked using correlation observations to provide along-track continuity in the attitude correction solution.
Results and Discussion Tier 1 Testing and Results
Tier 1 testing involved a comprehensive (near 100 percent) geometric accuracy comparison of Geocover™ 2000 data with the GLS 2000 data. In the image matching comparisons, the MMIO algorithm (USGS, n.d.) was used to extract image chips in areas where uniform sampling of pixels failed to yield acceptable correlations. Since the pixel size of the images in Geocover™ 2000 data are different than those in GLS 2000 data (28.5 m versus 30 m), these chips were resampled to 30 m. A number of comparisons also failed because of UTM zone changes to the data, particularly in the higher latitudes. A few images in Geocover™ 2000 yielded larger than expected differences, which were traced to measurement problems due to the absence of land features. Table 2 shows the results of the analysis, in terms of the various analysis blocks. The results are expressed as Root Mean Square Differences (RMSD), instead of RMSE because the comparisons (in this context) only indicate the relative deviation of Geocover™ 2000 data from the GLS 2000 data. Since each image has an RMSD and a number of sample checkpoints associated with it, the RMSD values of Table 2 were arrived at in the following manner:
136
Feb rua r y 2 015
RMSDLineorSample =
∑ × ( RMSD N
2 LineorSample
i =1
Total RMSD = RMSD
2 Line
+ RMSD
2 Sample
) /∑ , N
i
(3)
i =1
where N is the total number of images, and ni refers to the number of points analyzed for comparisons in an individual image; RMSDLine and RMSDSample refers to the RMSD value obtained from the analysis of each image. It should be noted here that the results shown Table 2 do not reflect all the Geocover™ 2000 versus GLS 2000 comparisons. This was because a few of the Geocover™ 2000 images fell into UTM zones that were different from the UTM zones assigned for the corresponding GLS 2000 path/row, while a few other images could not be compared because of a lack of image detail available for reliable cross-correlation. The results (in Table 2 and Plate 2) are expressed in pixels (and not actual ground distances) because these comparisons are purely relative in nature, and serve only to guide the absolute accuracy assessment process for GLS 2000 data sets. The outliers are images with greater than five pixel standard deviation errors. In some instances (path/row locations, e.g., 197/56), there are two GLS images available. While they are geometrically consistent with each other, the results of image matching with the corresponding Geocover™ 2000 data were different. Such anomalies were found to be due to the presence of cloud, and were removed from the analysis. Plate 2 shows the results of the comparisons on a global scale. The Geocover™ 2000 to GLS 2000 comparisons were used as a guide for Tier 2 analysis. All path/row locations with RMSE greater than one pixel were flagged, and samples from those locations were chosen for Tier 2 analysis.
Tier 2 Testing and Results
The Tier 2 analysis provides an estimate of the accuracy of the GLS 2000 data, relative to L7 Level 1G data. The comparisons against the L7 Level 1G images involved a nuanced choice of images. In the initial design of tests, 10 percent of all the GLS 2000 images were identified, such that they cover each block uniformly. These images were further filtered based on the availability of cloud free L7 Level 1G images acquired between the epochs 15 April 2005 and 31 December 2006. Then, GLS 2000 images that are close to existing National Geospatial Intelligence Agency’s (NGA) and other L7 control sites were eliminated. A random sampling of those images that failed the Geocover™ 2000 to GLS 2000 Tier 1 analysis or were deemed to have high RMSE values (>1 pixel) were added. Subsequently, as more L7 Level 1G scenes/images were processed over the years through user requests (and for archival purposes) results of almost 80 percent of the GLS 2000 path/row locations became available. These added scenes were processed through the Landsat Product Generation System (LPGS), which incorporates the comparison of the L7 Level 1G data against GLS 2000 data as a part of generating Level 1T products. The results are presented in Plate 3 and Table 3. For a given (path/ row location) comparison of GLS 2000 scene, multiple L7 Level 1G images, acquired over the same location, were used for analysis whenever possible. This reduces the effect of a single anomalous Level 1G scene being used as reference data. In the case of multiple scenes, a single RMSD value for a path/row location was arrived at from multiple RMSD values in Equation 3. The outliers were removed using Equation 2. The threshold Ze, was set at 5 in this case. Once all the results were evaluated, they were categorized into blocks. The results shown in the last column of Table 3 are arrived at by analyzing the results at a block level and eliminating outliers for each block based on Equation 2.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Plate 2. Relative difference between GeoCover™2000 and GLS-2000 data sets. Table 2. Results of Geocover™ 2000 and GLS-2000 comparisons Block
Number of Images
RMSD after outlier removal (pixels)
North America
1181
1.051
South America
726
1.954
95
1.002
8
1.821
1216
1.592
Greenland Iceland Africa Madagascar Eurasia Japan Australia New Zealand Islands
43
1.414
2643
0.966
41
2.722
367
0.815
34
0.548
752
1.629
Table 3. Results of Comparisons of GLS-2000 Data with Landsat ETM+ Level 1G Scenes Block North America
Number of Scenes (Scene Selection) 337
Mean of RMSD after excluding outliers (Scene selection) (meters) 23.1
Number of Scenes (using all available scenes) 1355
Mean of RMSD after excluding outliers (meters) 23.3
South America Greenland Iceland Africa Madagascar Eurasia Japan Australia New Zealand Islands Overall
177 29 7 135 8 640 12 54 15 46 1460
32.0 31.5 26.5 24.7 25.0 24.7 34.8 32.7 33.5 38.1 26.4
748 77 16 1137 34 2918 38 301 32 380 7036
33.5 30.5 34.0 27.3 25.1 25.9 30.0 35.2 35.4 33.7 27.5
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Fe b ru a r y 2015
137
(a)
(b) Plate 3. Results of comparison of GLS-2000 data with L7 Level 1G scenes: (a) using the scene selection procedure, and (b) using all the available scenes.
Tier 3 Testing and Results
In Tier 3 testing, the GLS 2000 data were compared against reference data that are known to have better accuracy. The reference data were the DOQs (over the United States), orthorectified SPOT data or triangulation-based bundle adjusted L7 L1-G scenes. The comparisons of the GLS 2000 data against the DOQs over US supersites, the ortho-rectified SPOT data over path/row locations in Australia and those using the triangulation procedure are presented below. A total of 23 supersites was selected to compare the geometric accuracy of the GLS 2000 data versus DOQs (over the United States) and 12 sites were used to compare GLS 200 data against 138
Feb rua r y 2 015
data (over Australia). The results of the comparison are presented in Figure 4 and Table 4. The average error for the GLS 2000 data over the United States was found to be 13.1 m, while the maximum RMSE observed was 24.6 m (for WRS path/ row 22/39) (Table 1). All the results of GLS 2000 comparisons over the United States show an absolute RMSE less than 25 m, and hence meet the geometric accuracy requirement. The geometric accuracy of the GLS 2000 data versus SPOT was found to be, on an average, 15.8 m. The maximum RMSE observed was 23.0 m (for WRS path/row 97/72). Again, the GLS 2000 comparisons show that the data meet the geometric accuracy requirement of 25 m RMSE. SPOT
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
(a) Figure 4. Results of (a) gls-2000 versus doq data, and (b) gls-2000 versus spot data.
(b)
Table 4. Results from Comparisons of gls-2000 Data versus doq and spot Data Over Supersites Reference data DOQ SPOT
Region United States Australia
Number of sites 23 12
Average RMSE (meters) 13.1 15.8
Minimum RMSE (meters) 7.0 10.0
Maximum RMSE (meters) 23.5 23.0
Plate 4. Triangulation-based bundle adjustment test sites for Landsat-7 Level 1G scenes. Table 5. Summary Results for 43 Triangulation Test Sites Block Name
Number of Images
RMSE (meters)
Africa
7
24.4
Australia
7
29.7
Eurasia
13
23.8
Madagascar
1
25.2
North America
5
15.6
New Zealand
1
21.2
South America
9
30.6
All Blocks
43
25.3
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
In the bundle adjustment process the ephemeris, attitude data, and the initial approximations of the ground point coordinates were simultaneously adjusted. The initial observations were obtained from the GLS 2000 data, and the difference in the initial coordinates and the final adjusted coordinates were treated as estimates of errors associated with GLS 2000 data. This process was carried out in 43 sites (Plate 4), from different blocks to determine the absolute accuracy of GLS 2000 data set. The triangulation process was tested by comparing the coordinates obtained from the process with DOQs over US supersites (path/rows 43/34 and 45/25). The RMSE values for the two sites were 18.8 m and 13.2 m, which indicate that the triangulation process has the ability to provide reference data of higher accuracy than GLS 2000 and a single Level 1G image. Fe b ru a r y 2015
139
The results shown in Table 5 are from triangulation tests performed over 43 sites. Each test consists of identifying features on the imagery that are common across five to ten Level 1G ETM+ images. The images were acquired from different trajectories within April 2005 to December 2006 time frames, when the ETM+ was most stable. Table 5 shows the RMSE associated with each path/row combination, where the RMSE is computed from the difference between the initial approximation and final adjusted coordinates from the triangulation procedure. The range of RMSE obtained from the analysis is between 8 m and 60 m (on a per-scene basis), with the average error being 25.3 m.
Discussion
Results indicate that 71 percent of the Geocover™ 2000 data align well (within a pixel) with the GLS 2000 data (Plate 2). However, 29 percent of the data are also seen to have larger (more than a pixel) differences from the Geocover™ 2000 data, with almost 9 percent of the data having greater than two pixel differences. The GLS 2000 differs from Geocover™ 2000 in South America, West Africa, and parts of the Indian subcontinent. GLS 2000 data do not differ significantly from the Geocover™ 2000 data in most parts of the United States, Europe and Asia. Tier 2 tests conducted over regions of higher differences reveal that L7 Level 1G data align well with GLS 2000 data. Tier 3 analysis also indicates that the accuracy of GLS 2000 data is higher in these regions. This finding indicates that the absolute accuracy of Geocover™ 2000 data set improved significantly, when it was reprocessed to generate GLS 2000 data set, and that errors in Geocover™ have been fixed to a large extent. Tier 2 tests were conducted using 1,460 scenes selected through a scene selection process and also using all L7 Level 1G scenes (7,036 scenes) that were processed through the LPGS. All the GLS 2000 path/row locations cannot be tested through comparison to L7 Level 1G scenes. For many locations a sufficient number of relatively cloud free images were not acquired in the April 2005 to December 2006 time frame to provide an estimate of the relative accuracy. For most locations, multiple L7 Level 1G scenes were available and used to arrive at the RMSD values for the GLS 2000 path/rows. The results from single scene comparisons may not be reliable due to many reasons, including unexpected perturbations in the orbit of the sensor, presence of cloud in even small portions of the image, uncertainties in correlation due to radiometric differences, etc. The results (RMSD) from multiple scenes were reduced to a single RMSD value using Equations 2 and 3. Equation 2 was used to eliminate potentially anomalous scenes (outliers), and Equation 3 was used to calculate the RMSD values from the results of the remaining “normal” scenes. In the case of Gaussian noise, it is common to not consider data that fall outside the interval of μ±3σ, where μ is the mean of the values and σ is the standard deviation. In our case, we have used the μmedian±5σMAD as the robust estimator of the interval (where 5 corresponds to Ztei in Equation 2). This is because σMAD is known to be related to σ by a factor of 1.48 (Ellison et al., 2009). Tier 3 validates the absolute accuracy of the GLS 2000 data by comparing them against data known to be of higher accuracy. The Federal Geographic Data Committee (FGDC) recommends that the criterion for data sets to be used as absolute reference be at least three times more accurate than the data being tested (FGDC 1998). The DOQ as reference data meets the criterion (Greenfeld, 2001), but the SPOT data do not (USGS, 2012). While this criterion is practical for high-resolution aerial images flown over smaller areas, it is impractical to use the criterion for selecting reference data in this case. The triangulation procedure on the DOQ sites produced checkpoints with an RMSE of 13.2 to 18.8 m using 8 Level 1G systematic scenes that have geopositional error of 30 m. Therefore, all three Tier 3 reference data have higher accuracy than GLS 2000 data sets. 140
Feb rua r y 2 015
The results from Tier 3 analysis (Tables 4 and 5) indicate that the RMSE for GLS 2000 data is better than 25 m RMSE. In general, GLS 2000 data are well aligned with the control data (26 to 27 m RMSD for Tier 2 analysis and between 13.0 to 25.3 m from Tier 3 analysis). The improvement of geometric accuracy from Geocover™ data to GLS 2000 data is apparent in all the processing blocks. However, the results for the South American block from Tier 2 analysis indicate an RMSD of 33.5 m (or 1.11 pixels). Tier 3 analysis (triangulation) for this block indicates that the RMSE ranges from 10.2 m to 56.5 m, with an average RMSE of 30.6 m. This finding indicates that the GLS 2000 has improved accuracy in most regions. However, the higher RMSE indicates that a few GLS 2000 scenes are likely to not meet the 25 m specification (Plate 3 and Plate 4). The Australian block is interesting, with a high degree of alignment between the Geocover™ 2000 and GLS 2000 data. However, from Tier 2 results, the RMSD between GLS 2000 and L7 Level 1G data is higher (at 35.2 m or 1.17 pixels), particularly in the southeast region (Plate 6b). Tier 3 results over Australian supersites show an RMSE of 15.8 m but the triangulation-based RMSE is higher (29.7 m) and varies from 16.4 m to 60.1 m. The higher errors are found in the same southeast region (path/row 94/84) where Tier 2 results indicated higher errors. This finding indicates that there may be some systematic errors associated with the Landsat data sets over the region. Since the triangulation algorithm is not designed to correct systematic errors, a further test was conducted (for path/row 94/84) by comparing GLS 2000 scene against Advanced Land Observing Satellite (ALOS) data that had been verified using Global Positioning System (GPS) based observations to have better than 6 m accuracy. The test indicated that the GLS 2000 data were accurate to greater than 15 m. The New Zealand block also seems to follow the same trend. However, there are no independent data sets available to test the GLS 2000 data, and further triangulation-based tests could not be conducted because of the difficulty in obtaining multiple cloud free L7 Level 1G scenes over the block. The Islands block have a large number of path/row locations, but they are also the most difficult to assess, mainly because of failures of correlation. These failures occur because of higher percentage of cloud cover in these regions, as well as the presence of water. Hence, the number of GLS 2000 path/ rows for which multiple L7 Level 1G scenes were available to test (Tier 2) was consistently lower for these regions. This lack of scenes also affected the ability to conduct Tier 3 triangulation tests in these regions. The Japan block has the largest differences between Geocover™ 2000 and GLS 2000 data (2.72 pixels RMSD). However, Tier 2 results indicate that they are well aligned with L7 Level 1G data (30.0 m or 1.001 pixels RMSD), which indicates that there were errors in the Geocover™ 2000 data, which were fixed in the GLS 2000 data. As in the case of Islands, it was difficult to obtain multiple cloud free L7 Level 1G data to perform Tier 3 analysis.
Summary and Conclusions
A global geometric accuracy analysis of the GLS 2000 data sets is not a trivial task due to the unavailability of comparable data sets (at a global scale). To accomplish this analysis, a three tier process was developed. In the first tier, the GLS 2000 data sets were compared with the Geocover™ 2000 data. Since the GLS 2000 data were generated to reduce the perceived inaccuracies of the Geocover™ 2000 data in certain areas, this analysis provided an indication of the regions where the GLS 2000 differed from the Geocover™ 2000. Most differences were found to occur in areas where the Geocover™ 2000 data were perceived to be less accurate. For Tier 2 of the data analysis, the images were initially selected as a random selection of 10 percent of the GLS 2000 data, and images from regions of higher Geocover™ 2000 to GLS 2000 variations. The ETM+ sensor’s improved on-orbit geodetic accuracy between April 2005 and December 2006 allowed PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Level 1G images to be used as reference data. In Tier 3 analysis, the reference data were Digital Ortho Quads over the United States and SPOT data over Australian supersites, and block triangulated Level 1G images acquired between 15 April 2005 and 31 December 2006. The Tier 2 analysis reveals that the RMSD ranges from 23.0 to 35.0 m from the Level 1G images. This finding indicates that the GLS 2000 data do not deviate significantly from the specified accuracy level. Tier 3 tests reveal that the RMSE of GLS 2000 data were well within the requirements of 25 m RMSE (~13.1 m over the United States, 15.8 m over Australia, and 25.3 m from triangulation analysis). Tier 3 analysis provides an estimate of the absolute accuracy of the GLS 2000 data set. However, the analysis is limited to locations where reference data are easily available (United States and Australia), and sampling of locations for triangulation-based bundle adjustment. The triangulation-based bundle adjustment study areas were selected such that they represent a global sampling of GLS 2000 image scenes. However, it is not practical to conduct these tests for every GLS 2000 path/row combination due to the limitation in the availability of multiple (five to 10) cloud free scenes for each path/row acquired between April 2005 and December 2006. The GLS 2000 is the base reference data set for the other GLS products (GLS, 1975, 1990, 2005, and 2010). All the GLS data sets are geometrically within a pixel of each other (Gutman et al., 2013). Therefore, this study indirectly validates the absolute accuracy of other GLS data sets too. In the future, with the launch of the Landsat-8 satellite and other agile and accurate remote sensing constellations, more accurate data will be available at a global scale helping to improve on the validation results of the GLS 2000 data set. In the long term, the Landsat-8 derived reference database will be generated and used for future precision and terrain product generation.
Disclaimer
Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the US Government.
References
Aguilar, M., F.Aguilar, M. Saldana, and I. Fernandez, 2012. Geopositioning accuracy assessment of GeoEye-1 panchromatic and multispectral imagery, Photogrammetric Engineering & Remote Sensing, 78(3):247–257. Chander, G., A. Sampath, A. Angal, T. Choi, and X. Xiong, 2010. Monitoring the long term stability of the IRS-P6 AWiFS sensor using the Sonoran and RVPN sites, Proceedings of SPIE 7862, Earth Observing Missions and Sensors: Development, Implementation, and Characterization . Dykstra, J. D., and J.C. Storey, 2004. Landsat-7 definitive ephemeris: An independent verification of Geocover™-Ortho 1990 positional accuracy, Proceedings of the ASPRS 2004 Annual Conference, May, Denver, Colorado, unpaginated CD_ROM. Ellison, S.L.R., J. Vicki, T.J. Barwick, and D. Farrant, 2009. Practical Statistics for the Analytical Scientist: A Bench Guide, RSC Publishing, pp. 55–56. Gianinetto, M., and M. Scaioni, 2008. Automated geometric correction of high-resolution pushbroom satellite data, Photogrammetric Engineering & Remote Sensing, 74(1):107–116. Greenfeld, J., 2001. Evaluating the accuracy of digital orthophoto quadrangles (DOQ) in the context of parcel-based GIS, Photogrammetric Engineering & Remote Sensing, 67(2):199–205. Gutman, G., R. Byrnes, J. Masek, S. Covington, S.C. Justice, S. Franks, and R. Headley, 2008. Towards monitoring land cover and land-use changes at a global scale: The Global Land Survey 2005, Photogrammetric Engineering & Remote Sensing, 74(1):6–10. Gutman, G., C. Huang, G. Chander, P. Noojipady, and J.G. Masek, 2013, Assessment of the NASA-USGS Global Land Survey (GLS) datasets: Remote Sensing of Environment, 134:249-265, URL: http://dx.doi.org/10.1016/j.rse.2013.02.026 (last date accessed: 10 December 2014).
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Federal Geographic Data Committee, 1998. Geospatial Positioning Accuracy Standards, Part 3: National Standard for Spatial Data Accuracy, URL: http://www.fgdc.gov/standards/projects/FGDC-standardsprojects/accuracy/part3/chapter3, Federal Geographic Data Committee, Reston, Virginia (last date accessed: 10 December 2014). Franks, S., J.G. Masek, R.M.K. Headle, J. Gasch, and T. Arvison, 2009. Large area scene selection interface (LASSI): Methodology of selecting Landsat imagery for the Global Land Survey 2005, Photogrammetric Engineering & Remote Sensing, 75(11):1287–1296. Habib, A., S.W. Shin, K. Kim, C. Kim, K. Bang, E. Kim, and D. Lee, 2008. Comprehensive analysis of sensor modeling alternatives for high resolution imaging satellites, Photogrammetric Engineering & Remote Sensing, 73(11):1241–1251. Lee, S., J. Storey, and M. Choate, 2004. Four years of Landsat 7 onorbit geometric calibration and performance, IEEE Transactions on Geoscience and Remote Sensing, 42:(12):2786–2795. Li, R., F. Zhou, X. Niu, and K. Di, 2007. Integration of Ikonos and QuickBird Imagery for geopositioning accuracy analysis, Photogrammetric Engineering & Remote Sensing, 73(9):1067–1074. Lutes, J., and J. Grodecki, 2004. Error propagation in Ikonos mapping blocks, Photogrammetric Engineering & Remote Sensing, 70(8):947–955. Masek, J., and S. Covington, 2007. Mid-decadal Global Land Survey: Status Update, Landsat Missions Project Documentation, URL: http://landsat.usgs.gov/documents/Masek_mdgls_corvallis_jun07.pdf, US Geological Survey, Reston, Virginia (last date accessed: 10 December 2014). Muller, R., T. Krauss, M. Schneider, and P. Reinartz, 2012. Automated georeferencing of optical satellite data with integrated sensor model improvement, Photogrammetric Engineering & Remote Sensing, 78(1):61–74. National Aeronautics and Space Administration, n.d. The worldwide reference System, URL: http://landsat.gsfc.nasa.gov/about/wrs. html, NASA-GSFC, Greenbelt, Maryland (last date accessed: 10 December 2014). National Aeronautics and Space Administration and U.S. Geological Survey, 2008. GLS Data, URL: http://gls.umd.edu/index.html, NASA GSFC, Greenbelt Virginia and U.S. Geological Survey, Reston, Virginia (last date accessed: 10 December 2014). Sampath, A., 2012. Geometric Accuracy Assessment of Data from UKDMC-2, SPOT 4/5 and RapidEye, Joint Agency Commercial Imagery Evaluation Workshop, Reston Virginia, 2012. Smith, C.M., M. Pagnutti, K. Holekamp, V. Zanoni, and D. Fendley, 2004. Geopositional accuracy assessment of Earthsat Geocover™ Landsat Orthorectified Imagery, Proceedings of the ASPRS 2004 Annual Conference, May, Denver, Colorado, unpaginated CD-ROM. Toutin, T., 2003. Block bundle adjustment of Landsat 7 ETM+ images over mountainous areas, Photogrammetric Engineering & Remote Sensing, 69(12):1341–13491. Tucker, C.J., D. Grant, and J. Dykstra, 2004. NASA’s global orthorectified Landsat data set, Photogrammetric Engineering & Remote Sensing, 70(3):313–322. U.S. Geological Survey, n.d. Extraction of GCP chips from Geocover™ using Modified Moravec Interest Operator (MMIO) algorithm: U.S. Geological Survey, Landsat Missions, Project Documentation, Technical Documentation (Landsat 7), Geocover™ Control, URL: http://landsat.usgs.gov/tools_project_documents.php (last date accessed: 10 December 2014). U.S. Geological Survey, 2006. Landsat 7 (L7) Image Assessment System (IAS) Geometric Algorithm Theoretical Basis Document (ATBD), URL: http://landsat.usgs.gov/documents/LS-IAS-01_ Geometric_ATBD.pdf, U.S. Geological Survey, Reston, Virginia, (last date accessed: 10 December 2014). U.S. Geological Survey, 2008. GLS DEM, URL: http://gls.umd.edu/, U.S. Geological Survey, Reston, Virginia (last date accessed: 10 December 2014).
(Received 20 March 2013; accepted 23 September 2013; final version 24 September 2014)
Fe b ru a r y 2015
141
