rpc-servir-fire-um-final

Rapid Prototyping of NASA Next Generation Sensors for the SERVIR System of Fire Detection in Mesoamerica Joel S. Kuszmaul, Henrique Momm, Greg Easson University of Mississippi, University, MS 38677 Timothy Gubbels Science Systems and Applications, Inc., Lanham, MD 20706

Executive Summary The SERVIR Fire Rapid Response System web site displays and distributes daily images of near-real-time MODIS data processed by the MODIS Rapid Response System. The MODIS Rapid Response System was developed by Science Systems and Applications Inc. scientists at the Goddard Space Flight Center and implanted as the SERVIR extension for the use of personnel involved in environmental monitoring and disaster management in Mesoamerica. This Rapid Prototyping Capability Experiment is designed to examine the continued performance of SERVIR’s fire detection system associated with the anticipated transition from MODIS to VIIRS sensor data. There is an inherent limitation in attempting this analysis prior to the commissioning of the actual VIIRS-based sensors. Our efforts in simulating the VIIRS data require us to make substitutions of sensor type and sensor resolution that cause the comparisons to be subject to this limitation. Our comparisons can, however, identify issues to be addressed in the preparation for the new sensors. We compared MODIS- to VIIRS-based fire products directly for four study dates during the 2003 Guatemala fire season. We find that there is reasonable agreement between the MODISand VIIRS-based products on the basis of direct comparison. While agreement rates are extremely high, largely due to the fact that fires are spatially uncommon, the kappa statistic was used to provide a measure of agreement that takes account of chance agreement. The calculated values of revealed good to excellent agreement beyond chance agreement. The highest values were obtained when the MODIS- and VIIRS-based assessments of high confidence fires were compared. The VIIRS-based fire products result in relatively few nominal-confidence fires and almost no low-confidence fires. The excellent agreement on high-confidence fires provides encouragement regarding the continued value of SERVIR’s fire-detection program beyond the life of the MODIS sensors. The difference between the two fire product results was found to be most significant for low and nominal confidence fires, where the overall results revealed that the classifications failed goodness-of-fit tests applied to the MODIS- and VIIRS-based classification distributions. We also compared the accuracy of both the MODIS- and VIIRS-based products against fires observed in both ASTER and LANDSAT-7 imagery. The MODIS-based products compared reasonably well with the ASTER imagery, with results roughly comparable to those reported by other investigators. When the VIIRS-based products were compared to the same fires detected using ASTER imagery, distinctly poorer results were observed with lower values of the kappa statistic and higher omission error values. This indicates that the challenge of detecting small 1

fires, which presumably may translate to lower confidence fires, is problematic with the VIIRS data and the current VIIRS-based algorithm. Finally, the two sets of fire products were compared to fires identified by two experts working independently with LANDSAT-7 imagery from the four study days. A total of 60 fires were identified by both experts (each fire independently identified). When these fires were compared to fires detected with the MODIS- and VIIRS-based decision support tool, the results demonstrated moderate, but more importantly nearly equal, success rates for the Terra-based products. The validation fire set was compared to the results obtained using the Aqua-based products yielding distinctly poorer success rates, but those were not considered as significant given the time lag between the collection of the LANDSAT-7 imagery and the Aqua data. The overall outcome of this RPC is the demonstration that SERVIR’s fire-detection system is expected to continue to perform well detecting fires that are currently being detected at a highconfidence level. Small fires or fires of low intensity do not appear to be as readily detected using the combination of the planned VIIRS sensor and the fire-detection algorithm designed for that sensor. The limitations of this experiment must also be considered. First, we used simulated VIIRS imagery where the current MODIS sensor was used to generate the pseudo-VIIRS dataset. Second, we did not have a confirmed set of fires detected at the surface at the time of imagery collection. Finally, we did not examine the consequences of the improved resolution associated with the expected VIIRS sensor. 1.0 Introduction This Rapid Prototyping Capability (RPC) experiment explores the feasibility of using Visible/Infrared Imager Radiometer Suite (VIIRS) data from the future National Polar-orbiting Operational Environmental Satellite System (NPOESS) to support the Rapid Response Fire Products within the SERVIR architecture. The VIIRS data is intended to replace current Moderate Resolution Imaging Spectroradiometer (MODIS) products. SERVIR is a NASA managed program integrating satellite and geospatial data with the aim to develop scientific and decision making knowledge for issues affecting Mesoamerica. The issues addressed by this program include disasters, ecosystems, biodiversity, weather, water, climate, oceans, health, agriculture and energy of the region. The Rapid Response Fire Products include Mesoamerican Web Fire Mapper and MODIS Rapid Response Fire Mapper. Both have been effectively implemented in recent years to monitor hot spots and active fires that threaten the natural resources of Mesoamerica. The products were developed as part of a collaborative effort by researchers from the University of Maryland, NASA and Center for the Humid Tropics of Latin America and the Caribbean (CATHALAC). The MODIS-based Rapid Response Fire Products of the SERVIR program were selected for the application of a NASA-funded Rapid Prototype Capability experiment. Our goal in this experiment is to compare the value of the MODIS-based tool to the planned VIIRS-based algorithm that may one day be used as a substitute fire detection system for the SERVIR program. There is an inherent limitation in attempting this analysis prior to the commissioning of the actual VIIRS-based sensors. Our efforts in simulating the VIIRS data require us to make substitutions of sensor type and sensor resolution that cause the comparisons to be subject to this 2

limitation. These substitutions create a limitation to our analysis that is an inherent consequence of the design of this Rapid Prototyping Experiment. In particular, the resolution differences between the proposed VIIRS-based sensor and the MODIS-based sensors used to create the simulated VIIRS data contributes an approximation to our comparison that cannot be quantified. Our comparisons can, however, identify issues to be addressed in the preparation for the new sensors. Both the similarity and differences in the MODIS- and VIIRS-based DSTs are significant. A record of good agreement between the two DSTs will provide assurance that the comparison has significance, the disagreement found can be used to identify areas where a difference between the success of the two DST may be expected. 1.1 Relevance to NASA This RPC experiment is directed at the sustained performance of a decision support tool (DST) with a well established track record of credibility and utility. This RPC experiment directly contributes to the NASA Applied Sciences Programs mission to extend the results of NASA Earth Science Division’s (ESD) contribution to national priority applications by supporting the ongoing mission of SERVIR’s Rapid Response Fire Products relate to National Application Areas of air quality, disaster management, ecological forecasting, and public health. Within the six National Focus Areas established by NASA ESD, this RPC contributes to an improved understanding within the categories Atmospheric Composition, Earth Surface and Interior and Weather. 1.2 Introduction to the SERVIR Fire Rapid Response System The SERVIR Fire Rapid Response System web site displays and distributes daily images of near-real-time MODIS data processed by the MODIS Rapid Response System. The MODIS Rapid Response System was developed by Science Systems and Applications Inc. (SSAI) scientists at the Goddard Space Flight Center (GSFC) based on experience gained during the wildfires in Montana in 2000. The SERVIR extension of the MODIS Rapid Response System has been developed for the use of personnel involved in environmental monitoring and disaster management in Mesoamerica. The MODIS SERVIR Fire Extent Product was developed to meet the requirements of the Guatemalan Park Service and was later extended throughout Mesoamerica. During 2003, the fire season in Guatemala was particularly severe (Figure 1), necessitating the need for greater reporting and evaluation of fire locations and extent in the sparsely populated northern part of the country. In that year, the Guatemalan Park Service did not possess the capability to effectively evaluate fire potential and location because they relied on visual and oral reports.

3

Figure 1. Fires in Guatemala, Honduras, and Yucatan, Mexico on March 19, 2003 as detected by MODIS visible and thermal bands.

The Guatemalan Park Service requested the SERVIR project to provide MODIS fires data sets to significantly enhance their current system in providing a twice daily mapping capability that had higher temporal and spatial characteristics. The MODIS data sets were evaluated to see if they met the technical requirements of the Guatemalan Park Service. It was determined that the MODIS thermal anomalies data “burned” into a land cover map twice daily would enhance the country’s ability to identify and evaluate fire events in a timely and effective manner. This data was also merged into the Guatemalan Park Service’s existing GIS to allow them to respond and allocate their resources more efficiently to fire events. The SERVIR Fire Rapid Response System uses daily MODIS data from both the Terra and Aqua spacecrafts. The data are processed in the MODIS Rapid Response system into fire products within 3-4 hours after acquisition. Image subsets with active fire overlay are usually generated in approximately two hours. SERVIR’s Mesoamerican Web Fire Mapper, which was originally developed by the University of Maryland, NASA, and CATHALAC, now serves Guatemala and the rest of Mesoamerica with on-line, interactive maps of active fires in the region. 1.3 Partners in the Interdisciplinary RPC Experiment Our partners in this RPC Experiment bring a diverse set of skills. We have partnered with experts in the science and application of the SERVIR fire detection tools. These include Dan Irwin of NASA’s Marshall Space Flight Center (NASA-MSFC) and project manager for SERVIR, to provide an understanding of how SERVIR is applied. NASA-MSFC is partnered with CATHALAC to manage the SERVIR project and serve information products to users in Central America. For understanding of the current MODIS-based fire detection system and associated SERVIR fire products we have relied on Tim Gubbels, of SSAI. Tim Gubbels supplied the MODIS-based fire products used for the comparison discussed in Section 4.0. We also incorporated experts in the NASA next-generation sensors. Robert Ryan, SSAI, provided 4

the simulated VIIRS data that was used to create the results for the VIIRS-based fire detection system. 2.0 SERVIR and Fire Detection: the MODIS Active Fire Products for Mesoamerica The MODIS Active Fire Products for Mesoamerica are derived from reflected and emitted radiation received by the MODIS instrument from the Earth’s land surface. The reflected radiation provides a representation of surface features and is conveyed by the Rapid Response Corrected Reflectance Product. The emitted radiation provides the fire detections and is conveyed by the Rapid Response Fire Detection Product. The two products are described below. Both products are produced in near-real-time within the Rapid Response system from the MODIS L0 data stream. 2.1 The Corrected Reflectance Product The imagery product for the fire detection system is based on a modified version of the MODIS Corrected Reflectance product (MOD09). The MOD09 product includes the visible and nearinfrared bands (1 to 7). The transformation from radiance to reflectance uses the ratio of the measured at-sensor radiance and the known solar irradiance at the time of the observation. The MODIS Rapid Response processing algorithm (Giglio et al., 2003) corrects for molecular (Rayleigh) scattering and gaseous absorption by water vapor and ozone and does not require real-time input or ancillary data. The algorithm uses climatological values for gas parameters. The near-real time MODIS surface reflectance product is similar to MOD09 surface reflectance in clear atmospheric conditions (based on 5S/6S radiative transfer model), but departs from MOD09 in the presence of aerosols. In order to preserve the signature of smoke plumes caused by fires, no aerosol correction is made. For ease of interpretation, the Rapid Response reflectance product is depicted as “true-color” imagery. This means that the colors red, green, and blue as observed by the sensor, in MODIS bands 1 (670 nm), 4 (565 nm), and 3 (479 nm), are depicted in the same colors in the resulting digital color-composite image. The advantage of this method is that it produces intuitive, natural-looking images of land surface, oceanic and atmospheric features. This imagery serves as the backdrop, or reference image, upon which the fire outlines are “burned-in” as red polygons. 2.2 The Fire Detection Product The fire detection strategy is based on a combination of absolute detection of fire size and intensity and on detection relative to the background, to account for variability of the ambient surface temperature and reflection by sunlight. The principal inputs to the Enhanced Contextual Detection Algorithm are the 4 and 11-micrometer brightness temperatures. Additional MODIS channels are further used to condition the data and to provide numerous tests to identify and reject typical false alarms. Cloud masking is provided by a combination of bands 31 and 32, while reflected bands 1, 2, and 7, assist in cloud masking, sun glint, bright surface, and coastal false alarm detection (Giglio et al., 2003). 5

Because the Terra MODIS instrument acquires data twice daily (10:30 am and 10:30 pm), as does the Aqua MODIS (2:30 pm and 2:30 am), four daily MODIS observations are available to contribute to global fire monitoring. The MODIS/Terra Thermal Anomalies/Fire 5-Min L2 Swath product, MOD14, is the most basic data in which active fires and other thermal anomalies, such as volcanoes can be identified. It is used to generate all of the higher-level fire products. The MODIS Rapid Response image products are daytime only. The sensitivity threshold is quantified as the probability of detection, which is strongly dependent upon the temperature and area of the fire being observed. The detection of any given fire is dependent on its magnitude, which is the product of its temperature and size. Hot fires as small as ~10 m in diameter can be readily detected, while cooler fires of larger sizes, such as ~30 m in diameter, are also readily detected. In any given area, the detection of a fire is related to biome type and characteristics. As a consequence, it must be noted that this RPC experiment addresses fire detection in Guatemala, and the outcomes might be different than what could be expected in another region. The fire detection algorithm used in the MOD14 algorithm in both MODAPS and Rapid Response is an improved algorithm that offers increased sensitivity to smaller, cooler fires, as well as a significantly lower false alarm rate. Performance of both the original and replacement algorithm has been evaluated using theoretical simulations and ASTER scenes (Giglio et al., 2003). In general, the new algorithm can detect fires roughly half the minimum size that could be detected with the original algorithm while having an overall false alarm rate 10-100 times smaller. The algorithm performance has been measured by numerous field validation experiments (eg., Morisette et al., 2005a) 2.3 SERVIR Rapid Response Tailored Products The Corrected Reflectance true color image and the Fire Detection product are used as the primary inputs to the Fire Extent product as produced in MODIS Rapid Response. The daytime Terra and Aqua MODIS observations are used to make the morning and afternoon products, respectively. The outlines of the fires are depicted in red, superimposed on the true color image. This has the effect of showing the fire extent, clouds, and smoke. The L0 data stream upon which the products are based is intercepted at the NOAA Data server at the Goddard DAAC. The data are then ingested in the Rapid Response system and processed in the Rapid Response system into fire products within 3-4 hours after acquisition. Image subsets with active fire overlay are then generated after a short time delay (about two hours). To assist in the interpretation, the coastlines and political boundaries are fused into each map in blue and the resultant JPEG images posted to the web on the SERVIR site at http://rapidfire.sci.gsfc.nasa.gov/servir/. Georeferencing information is stored in JPEG World Files that may be downloaded to ease the import of the data into a Geographic Information Systems (GIS). It was determined by the SERVIR project team that Mesoamerica was simply too large to convey the fire locations in adequate detail in a single frame. Six subframes were designed to 6

capture the region in greater detail. They are named Mexico, GuatBelSal, Honduras, Nicaragua, Costa Rica, and Panama. When a region is selected, the archive of subsets is presented chronologically as a list of image dates, with the most recent image at the top of the list. By clicking on a date, the corresponding Terra and Aqua images are displayed with annotation of calendar date as well. Project information is available by clicking the More Info button. These products have been produced since March 1, 2004 for each Mesoamerican fire season. 2.4 Related Decision Support Systems The VIIRS data products may also serve as an input to other operational decision support systems that currently use MODIS products for fire detecting. Some of such systems are Web Fire Mapper (University of Maryland based), Mexico National Commission for the Knowledge and Use of Biodiversity (CONABIO) that monitors fires in Mexico, Belize and Guatemala, National Meteorological Service of Mexico, Wild-land Fire Information System in Mexico, and Wildfire Alternatives, which is a fire-monitoring research initiative taken up at the University of Arizona. 3.0 The Ongoing NASA Contribution to SERVIR’s Fire Detection Tools The ongoing contribution of NASA to SERVIR’s mission in providing fire detection tools depends upon the sustained performance of MODIS sensor on both the Terra and Aqua satellites. VIIRS represents the next generation of satellites designed to combine and to replace the current MODIS Terra and Aqua missions (Yu et al., 2005) VIIRS will have significant improvements over its predecessors that will lead to new and improved products. These improvements include reduced data delivery time, improved scan geometry, and the 22 channels designed based on experience acquired from past missions (Lee et al., 2006). 3.1 Application of Current SERVIR Fire Detection Tools to the 2003 Guatemala Fire Season One of the characteristics of the design of this RPC experiment is the incorporation of data from the 2003 Guatemala fire season. We used field information provided by the SERVIR project regarding the number and location of fires that were encountered and surveyed. Scene selection constituted a vital part for the success of the experiment. Four dates were selected for investigation during the 2003 fire season: March 20, April 21, April 28, and April 30. These dates were selected according to the following criteria: • Guatemala had to be entirely contained by a single MODIS scene; • The cloud coverage of the selected dates (specially over Guatemala) should be as limited as possible; • Guarantee of fire existence in the selected dates based on information obtained from different sources; and • Availability of higher spatial resolution imagery such as LANDSAT and ASTER data to be used as validation data. Table 1 lists the dates selected and the higher resolution imagery used for our validation analyses. Our goal is to compare the results of the MODIS-based fire detection system with the simulated-VIIRS-based fire detection system. We achieve the goal by comparing the 7

consistency of classification of the two systems (Section 4.1), then by comparing the success that each of the models has compared to fires detected in the ASTER imagery listed in Table 1 (Section 4.2), and finally by examining the frequency with which fires found using the LANDSAT imagery listed in Table 1 are detected by each of the systems (Section 4.3). This last comparison provides a check on the rate of missed fires, or a check on the error of omission. We also attempted to compare the rate of falsely detected fires or a check on the error commission, but that effort was not successful for the reasons discussed below (Section 3.2). The LANDSAT coverage within Guatemala is shown in Figure 2 for each of the four study dates. These images show that inconsistent coverage of Guatemala is available and the number of confirmed fires will not be consistent from one day to another as a result. Landfires in Guatemala can change significantly within a few hours. Some span a broad area for more than a week, while others flare and are extinguished quickly. This presents a challenge when seeking to use two sets of satellite-based sensors to compare fire detection. If the time of acquisition is too great, the possibility exists that the discrepancy may be attributable in part due to changes in the fire itself. Further, the intensity of fire burns is affected by weather (including rainfall, humidity and air temperature). As a result, the time of day at which fires are detected can play a role in the position of the fires as well as the potential for success of a fire-detection system, the time of acquisition of the LANDSAT and ASTER images are included in Table 2. This information will be considered in the evaluation of the fire-detection DSTs discussed in Section 4.

8

Table 1 – List of available LANDSAT and ASTER scenes for the selected study date Study Date March 20, 2003

April 21,2003

April 28,2003

April 30, 2003

Landsat or ASTER Scene ID L71020048_04820030320 or LE70200482003079ASN00 L71020049_04920030320 or LE70200492003079ASN00 L71020050_05020030421 or LE70200502003111ASN00 L71020049_04920030421 or LE70200492003111ASN00 L71020048_04820030421 or LE70200482003111ASN00 AST_L1B_00304212003164104_20061101192514_4525 AST_L1B_00304212003164113_20061101192734_5672 AST_L1B_00304212003164121_20061101192614_4850 AST_L1B_00304212003164130_20061101192724_5586 AST_L1B_00304212003164139_20061101192724_5582 AST_L1B_00304212003164148_20061101192724_5575 AST_L1B_00304212003164157_20061101192814_5958 AST_L1B_00304212003164206_20061101192724_5589 L71021049_04920030428 or LE70210492003118ASN00 L71021048_04820030428 or LE70210482003118ASN00 L71019049_04920030430 or LE70190492003120EDC00 L71019048_04820030430 or LE70190482003120EDC00 AST_L1B_00304302003163436_20061101193535_8916 AST_L1B_00304302003163444_20061101193425_8130 AST_L1B_00304302003163453_20061101193325_7631 AST_L1B_00304302003163502_20061101193325_7629 AST_L1B_00304302003163511_20061101193535_8914 AST_L1B_00304302003163520_20061101193535_8908 AST_L1B_00304302003163529_20061101193425_8128 AST_L1B_00304302003163538_20061101193535_8903 AST_L1B_00304302003163546_20061101193535_8899 AST_L1B_00304302003163555_20061101193545_9006

9

Acquisition Time (Metadata) 16:11:47 16:12:14 16:12:11 16:12:38 16:12:37 16:13:04 16:12:13 16:12:40 16:11:49 16:12:16 16:41:04.199 16:41:13.460 16:41:21.894 16:41:30.741 16:41:39.588 16:41:48.436 16:41:57.285 16:42:06.132 16:18:26 16:18:53 16:18:02 16:18:29 16:06:04 16:06:32 16:05:41 16:06:08 16:34:36.125 16:34:44.971 16:34:53.818 16:35:02.664 16:35:11.513 16:35:20.360 16:35:29.208 16:35:38.550 16:35:46.904 16:35:55.753

Acquisition Time (Local Time) 10:11:47 10:12:14 10:12:11 10:12:38 10:12:37 10:13:04 10:12:13 10:12:40 10:11:49 10:12:16 10:41:04.199 10:41:13.460 10:41:21.894 10:41:30.741 10:41:39.588 10:41:48.436 10:41:57.285 10:42:06.132 10:18:26 10:18:53 10:18:02 10:18:29 10:06:04 10:06:32 10:05:41 10:06:08 10:34:36.125 10:34:44.971 10:34:53.818 10:35:02.664 10:35:11.513 10:35:20.360 10:35:29.208 10:35:38.550 10:35:46.904 10:35:55.753

Figure 2. The LANDSAT and ASTTER imagery coverage for the four dates in 2003. The LANDSAT imagery is shown for each of the four dates. The ASTER imagery coverage area is shown as a blue polygon on the two dates where imagery is available. The LANDSAT imagery, with the more extensive coverage, is to be used to form a validation data set for comparing the error of omission for both DSTs. The ASTER imagery, is used in section 4.0 for more complete validation of both DSTs.

10

Section 3.2 Calculating the error of commission The commission error represents the situation in which fires are detected from remotely sensed imagery when actually there is no fire in the field. The commission error is also referred to as a false positive error. The approach taken to evaluate the commission error was to identify and select areas where there were no fires during the entire 2003 fire season in northern Guatemala. Thus, assuming normal conditions, no change in biomass would occur in areas not burned before and after the fire season. The indicator of the presence and the condition of the green vegetation used was the Normalized Difference Vegetation Index (NDVI). NDVI uses a mathematical combination of red and infra-red channels which yields high values for vegetated areas, negative values for features such as clouds, water and near zero values for features such as rock and bare soil. (Lillesand and Kiefer, 2000). MODIS NDVI 16 day composites at 250 meters spatial resolution covering Guatemala prior and after the 2003 fire season were selected. The composites were chosen to reduce the cloud coverage and to use 16-day averaged NDVI values instead of individual day values. These two scenes were preprocessed to remove the negative values representing non-vegetated areas. The green vegetation situation was evaluated by computing the NDVI difference by subtracting the pre-fire NDVI from the post-fire NDVI image. Values above zero in the NDVI difference image relates to vegetation changes or has increased in the timeframe considered. However, to capture even a small decrease in the vegetation (which could be related to season changes or other climate reasons) a threshold of 0.015 was used. Visual investigation of the final healthy vegetation image revealed that small scars (smaller than MODIS pixel resolution) were missed when comparing to ASTER and LANDSAT imagery. Thus, known fires in a location did not yield a significant NDVI difference. In addition, the small commission errors reported by published literature (Morisette et al., 2005a) led us to discontinue the commission error evaluation through this approach. 3.3 Plans to Utilize NASA Next-Generation Sensors The Visible Infrared Imager Radiometer Suite (VIIRS) is designed to collect visible/infrared and radiometric data to obtain measurements of Earth’s oceans, land surface and atmosphere with global coverage at least once per day, and with better than 1-km spatial resolution (Welsch et al., 2001). The type of environmental data measurements to be made by VIIRS include atmospheric, clouds, earth radiation budget, clear-air land and water surfaces, sea surface temperatures, ocean color, and low light visible imagery (Lee et al., 2006). VIIRS will fly on the National PolarOrbiting Operational Environmental Satellite System (NPOESS) in three polar sun-syncronous orbit planes (at an altitude of 833 km), with equator crossing times of 1730, 0930, and 1330. It will also fly on the NPOEESS Preparatory Project with a 1030 equator crossing time. VIIRS will have multi-channel imaging capabilities to support the acquisition of high resolution atmospheric imagery and generation of a variety of applied products including visible and infrared imaging of hurricanes and detection of fires, smoke, and atmospheric aerosols. It will acquire data in 22 spectral channels between 0.3 and 14 microns with a spatial resolution of 400 m to 800 m and 3040-km scan swath. VIIRS draws heavily on the experience gained in building and operating MODIS that also combined diverse functions into a single sensor. It also draws on the experience with various sensors with specific tasks such as the Sea-viewing Wide Field-ofview Sensor for measuring ocean color, the Along Track Scanning Radiometer for measuring sea 11

surface temperature, and the Operational Linescan System for terminator imaging. The difference in the spectral content in the expected VIIRS sensor and the current MODIS sensors is significant. MODIS is a 36 band spectrometer with a spatial resolution (pixel size at nadir) of 250m for channels 1 and 2 (0.6µm - 0.9µm), 500m for channels 3 through 7 (0.4µm - 2.1µm) and 1,000m for channels 8 to 36 (0.4µm - 14.4µm). The MODIS instrument consists of a cross-track scan mirror that generates a simple rectangular array of pixels following the time sequence of the scan mirror rotation and the spacecraft alongtrack movement. At nadir position, a nominal 1-km pixel has dimensions of 1 x 1 km but as the scan angle increases from nadir the pixel dimensions grows to approximately 4.8-km by 2-km along track (Masuoka et al., 1998). Due to this feature, a single swath covers 10-km along track at nadir and it expands to 20-km along track creating a “bowtie-shaped” footprint which overlaps the swath below and above it. In simple terms a single feature on Earth may appear in several scan lines if it is located near the edge of the scene. VIIRS is designed to optimize both spatial resolution and signal to noise ratio across the scan. In this unique scanning approach, at nadir three detector footprints are aggregated to form a single VIIRS “pixel”. As the scan angle increases to approximately 32 degrees, the aggregation scheme is changed from 3x1 to 2x1 and at a scan angle near 48 degrees the aggregation scheme changes again from 2x1 to 1x1 (Miller and Liu, 2002). As consequence VIIRS exhibits a pixel growth actor of two along and across track. Miller and Liu (2002) also explain that the increase in spatial resolution is beneficial to the identification of active fires however the same improved spatial resolution at the edge of the images increase the changes of band saturations. The algorithm described by Giglio et al. (2003) uses brightness temperatures derived from MODIS channels at 4-µm and 11- µm to detect active fires. MODIS has two 4-µm channels: bands 21 and 22. The difference between these two bands is the temperature saturation values: band 21 saturates at 500K while band 22 saturates at 331K. Because band 22 is less noisy and has a smaller quantization error (Giglio et al., 2003) the algorithm uses this band unless it saturates then band 22 is replaced by band 21. The channel 11-µm in MODIS is represented by band 31 and it saturates at 400K for Terra MODIS and 340K for Aqua MODIS. Table 2 shows a comparison between VIIRS and MODIS spectral bands. Table 2 – VIIRS spectral bands comparison to MODIS. Spectral bands used in active fire detection are highlighted in gray. Source: modified from Byerly et al. (2002)

12

VIIRS MODIS Spectral Bands Spectral Range (um) Nadir HDR (m) Spectral Bands Spectral Range (um) Nadir HDR (m) M1 0.402-0422 750 8 0.405-0.420 1000 M2 0.436-0.454 750 9 0.438-0.448 1000 3 0.459-0.479 500 M3 0.478-0.498 750 10 0.483-0.493 1000 4 0.545-0.565 500 M4 0.545-0.565 750 12 0.546-0.556 1000 I1 0.600-0.680 375 1 0.620-0.670 250 13 0.662-0.672 1000 M5 0.662-0.682 750 14 0.673-0.683 1000 M6 0.739-0.754 750 15 0.743-0.753 1000 I2 0.846-0.885 375 2 0.841-0.876 250 M7 0.846-0.885 750 16 0.862-0.877 1000 M8 1.230-1.250 750 5 1.230-1.250 500 M9 1.371-1.386 750 26 1.360-1.390 1000 I3 1.580-1.640 375 6 1.628-1.652 500 M10 1.580-1.640 750 M11 2.225-2.275 750 7 2.105-2.155 500 I4 3.550-3.930 250 20 3.660-3.840 1000 M12 3.660-3.840 750 21 3.929-3.989 1000 M13 3.973-4.128 750 22 3.929-3.989 1000 23 4.020-4.080 1000 M14 8.400-8.700 750 29 8.400-8.700 1000 M15 10.263-11.263 750 31 10.780-11.280 1000 31 10.780-11.280 1000 I5 10.500-12.400 375 32 11.770-12.270 1000 M16 11.538-12.488 750 32 11.770-12.270 1000

4.0 The SERVIR Fire Detection System Using NASA Next-Generation Sensors The simulated VIIRS data was produced by our SSAI-Stennis partners led by Robert Ryan. They used the MODIS datasets from both Terra and Aqua sensors for the four study dates to generate a simulated VIIRS or proxy VIIRS dataset. Data was simulated for five of the moderate-resolution VIIRS bands needed for the fire detection algorithm; the red and NIR bands, SWIR (2.25 μm) band, MWIR (4.05 μm) band, and LWIR (10.763 μm) bands. As these bands are very similar spectrally to the MODIS bands used in the algorithm, a spectral band matching technique was used for the simulation. In this technique, the most similar MODIS band is used create each band of the VIIRS proxy data. The change in ground sample distance (GSD) from 1km MODIS data to 0.762-km VIIRS data was not considered, with the exception of the noise model described below. Corrections to account for the increased noise predicted for the VIIRS sensor were applied to the MWIR and LWIR bands. The noise correction is based on the NEDTs (noise equivalent delta temperatures) of MODIS and VIIRS for both extended and point source targets. Because the GSD of VIIRS is smaller than that of MODIS, one noise correction model could not account for all different types of targets. Thus, best case and worst case noise models were developed to bound the simulations. The extended targets, which are larger than a pixel and for which the improvement in GSD has no effect, are represented with the worst case noise model. Point source targets, which have a higher signal to noise ratio due to the improved GSD, are represented with the best case noise model. The relation between the NEDT of the two systems for an extended target is given in Equation (1) and for a point source in Equation (2). 13

(1) where,

= VIIRS NEDT = MODIS NEDT = Difference between MODIS and VIIRS NEDTs (2)

where,

= VIIRS ground sample distance = MODIS ground sample distance

value was found and used to scale a matrix of From each of these equations, the normally distributed random numbers the size of the original image. This scaled matrix was added to MODIS data which was converted to brightness temperature to produce the simulated VIIRS data set with either point source or extended target noise. Noise corrections were not applied to the red, NIR, and SWIR bands, which were simulated solely via spectral band matching. For the fire detection algorithm, these bands are used to detect water, clouds, and sun glint, and differences in sensor noise was determined to have little or no effect on algorithm output. The SSAI recommended (Kara Holekamp and Robert Ryan, personal communication, May, 2007) error model was the point source model. Two independent VIIRS simulations were performed by SSAI-Stennis. The first one produced simulated VIIIRS data in grid format. This data was then used as input to a MODIS fire algorithm modified for VIIRS thresholds and to accept gridded data instead of swath format (MODIS original format). The second simulation was performed after considerable modifications to the simulation algorithm to accept as input MODIS in swath format and produce as output simulated VIIRS also in swath format. This allowed pixel-by-pixel comparisons since there were no resampling operations involved. The simulated VIIRS in swath format was a requirement of the VIIRS fire detection algorithm. The goal of this RPC Experiment was to compare the effect of the change in sensor on the SERVIR fire detection DST. Our goal was not to make any modifications to the DST itself, but simply to assess how the DST would perform with simulated VIIRS data instead of MODIS data. It is important to note, that the scientists working in cooperation with SERVIR have already modified the DST in anticipation of the expected VIIRS sensor characteristics. The VIIRS fire detection algorithm (NASA Goddard Space Flight Center, electronic communication, May, 2007) used in this project is very similar in design to the MODIS fire detection algorithm. However, there are some changes in the saturation thresholds to meet VIIRS requirements. VIIRS is currently designed to have only one channel at 4-µm: band M13. Band M13 saturates at 634K while M15 (VIIRS equivalent to MODIS band 31) saturates at 343K (Miller and Liu, 2002). This saturation difference and other algorithm differences make the application of the new DST and the current DST genuinely different, and the comparison of the results is considered in the next section.

14

4.1 Comparing MODIS- and VIIRS-based Detection Tools To compare the results obtained from the MODIS- and VIIRS-based fire detection tools, we applied both algorithms to the data from each of the four days for both sensors. Thus, a total of 8 MODIS fire products (one for Terra and Aqua on each of the four study days) were compared with the 16 simulated VIIRS fire products (with two simulated VIIRS products for every one MODIS product due to the two different errors models). As an example of how the MODIS product can be compared to the simulated VIIRS product, consider the results for March 20, 2003 using the Terra satellite data. The results can be compared using an error matrix (also known as the confusion matrix) as shown in Table 3. This matrix reveals the number of pixels in agreement or disagreement for each of four pixel classifications. For each of the pixels where both algorithms make a classification, the results are compared class by class. In these models, fires are classed as high-confidence fires (Class 9), nominal-confidence fires (Class 8) and low-confidence fires (Class 7). Pixels classed as no fire are assigned to Class 5. In the error matrix of Table 3, agreement between the algorithms results in pixels being recorded along the diagonal of the matrix. Pixels in disagreement are shown off the diagonal. One measure of agreement that can be applied in this type of case is the overall frequency of pixels in agreement, which is a measure of the fraction of total pixels in which both algorithms made the same classification. In the case shown in Table 3, the agreement rate is 0.99958 or 99.958%. While these results appear to indicate extremely close agreement, high agreement rates for the prediction of rare events such as fires is to be expected. Much of the agreement stems from the fact that most of both fire products show agreement in that most of Guatemala is not on fire. Instead of comparing the MODIS- and VIIRS-based products on the basis of agreement rate, we seek a more complete measure of the extent to which the two DSTs are or are not achieving the similar fire detection results. A more complete comparison of the data sets is possible using the kappa statistic. The kappa statistic, first proposed by Cohen (1960), is a variable designed to account for the rate of chance agreement. The kappa statistic varies between -1 and +1. For two classifications that agree merely by chance, a kappa value of 0 would result. Classifications with agreement poorer than chance would result in a negative kappa value, while agreement better than chance would result in a positive kappa value. Landis and Koch (1977) proposed the use of kappa values greater than 0.75 to represent excellent agreement beyond chance, kappa values between 0.4 and 0.75 to be fair to good agreement beyond chance, and kappa values between 0 and 0.4 to be poor agreement beyond chance. To calculate the kappa value in this and all other comparisons of MODIS and simulated VIIRS fire products, we used the ArcView extension supplied by the USGS (Jenness and Wynne, 2005). This module calculates the sample’s kappa statistic, , with other supporting statistics. For the March 20, 2005 Terra case shown in Table 3, the calculated kappa statistic is 0.6989, which reveals good agreement beyond chance between the models.

VIIRS

Table 3. The error matrix result comparing the MODIS and simulated VIIRS fire products for March 20, 2003, using the Terra server data and the extended error model for the simulated VIIRS data. 5 7 8 9

5 594467 0 7 28

MODIS 7 8 2 170 0 0 015 50 0 28

9 15 0 1 227

We applied the Jenness and Wynne analysis tool (2005) to all comparisons to yield the results shown in Figure 3, where values (along with the confidence interval about each value) are shown for all cases. These cases include the four study dates, both Terra and Aqua sensors, and both the extended and point errors models applied to the simulated VIIRS data. One observation that can readily be made is that the type of error model applied to the simulated VIIRS data does not significantly affect the results. The SSAI recommended (Kara Holekamp, e-mail communication, May 11, 2007) error model was the point source model as the most representative of the expected level of error in the VIIRS sensor’s. 0.9000 0.8697

0.8791

0.8500 0.8262 0.8000 0.7827 0.7655 0.7500 0.7332 0.7000

0.7371

0.8368 0.7945

0.7641

0.7430

0.7414

0.7204

0.7187

0.6989

0.7028

0.7011

0.7011

0.6645

0.6686

0.6684

0.6684

0.6357

0.6357

0.6500

0.6880

0.6560

0.6196

0.6219

0.7222

0.6988

0.6942

0.6990

0.6720

0.6660

0.6709

0.6758

0.6393

0.6331

0.6900

0.6538

0.7175

0.7047

0.6000

0.5500

0.5000

0.5443

0.5434

0.5093

0.5084

0.4743

0.4735

0.4500

Figure 3. The overall kappa calculations (with point estimate of and the associated confidence interval) from the Jenness and Wynne tool (2005).Demonstrates that the choice of the point or extended error models applied to the VIIRS data is not significant. Using only the point model of error, the associated estimates and confidence intervals are shown in Figure 4, where the results from the two sensors on each of the four dates are illustrated. One observation that can be made of the results shown in Figure 4 is that there seems to be a fairly consistent size of the confidence interval of in each case. In addition, the values are reasonably consistent from day to day and sensor to sensor, with the exception of April 28 Terra and April 30 Terra values. The April 30 Terra value may differ from other results reported in part due to the fact that this analysis does not fully cover all of Guatemala. For the April 28 Terra products, there are distinctly lower estimates of . No obvious explanation for this poorer performance has been identified, but one possible explanation is that more fires were detected on April 28, 2003 than on any other day. Whether the poor agreement results from the presence of many fires on that date, or whether the large number of ‘detected’ fires is a symptom

16

of the challenges to accuracy of the algorithm cannot be known. However, the association between many ‘detected’ fires and lower associated estimates seems more than a coincidence. 0.9000 0.8791 0.8500

0.8368

0.8000 0.7500

0.7945 0.7641 0.7371

0.7414 0.7222

0.7187 0.7000

0.7028

0.7011

0.6686

0.6684

0.6500 0.6357

0.6988

0.6900

0.6660

0.6560

0.6990 0.6758

0.6331

0.6219

0.6000 0.5500

0.5434 0.5084

0.5000

0.4735 0.4500

Figure 4. Results from the overall kappa calculations (with point estimate of and the associated confidence interval) from the Jenness and Wynne tool (2005) for the case of the point source error model. Only comparisons involving the point source error model are considered. Another way to compare the results of the fire detection algorithms is by the efficacy of their fire detection. The current MODIS detection algorithm produces three classes of fires: highconfidence fires (class 9), nominal-confidence fires (class 8), and low-confidence fires (class 7). In the current MODIS fire products, all three fire classes are grouped together and assigned the class of “fire.” We could also compare the efficacy in detecting fires as defined by current practice, by grouping pixels classed as 7, 8 and 9 into a new combined fire class, to compare fire versus no fire. It is important to note that many pixels are not assigned a classification for a variety of reasons such as excessive clouds, the presence of a water body, and other reasons that emerge in the application of the algorithm (Giglio et al., 2003). The previous kappa values discussed were based upon similarity in class of fire. For example, a pixel declared to be a class 7 fire (low-confidence fire) and a class 8 fire (nominal-confidence fire) would have been considered an error. If we group classes 7, 8 and 9 into a single fire class, the new kappa value should show improved success (as it would not distinguish between classes of fires). While we made this comparison, we also chose to make two additional comparisons: comparing fires detected at the nominal confidence level and above and comparing fires detected

17

at the high-confidence level only. In these last two comparisons, we simply disregarded the locations found to have ‘possible’ fires. The result of the comparison is shown in Table 4. Table 4. The overall results comparison for the ability to detect fires using the four different definitions of fires. The upper left case considers the differences between classifications on a class-by-class basis. The other cases restrict the definition of a fire as discussed in the text, with the bottom right comparing the ability to detect high-confidence fires. Overall DAY TERRA March 20, 2003 0.7028 April 21, 2003 0.6684 April 28, 2003 0.5085 April 30, 2003 0.7335

Low-confidence and higher DAY TERRA AQUA March 20, 2003 0.7351 0.7869 April 21, 2003 0.6920 0.6810 April 28, 2003 0.5215 0.7014 April 30, 2003 0.7503 0.7230

AQUA 0.7414 0.6560 0.6660 0.6990

Nominal-confidence and higher DAY TERRA AQUA March 20, 2003 0.7369 0.7893 April 21, 2003 0.6921 0.6811 April 28, 2003 0.5220 0.7022 April 30, 2003 0.7524 0.7238

High-confidence DAY TERRA March 20, 2003 0.9134 April 21, 2003 0.8677 April 28, 2003 0.6922 April 30, 2003 0.9182

AQUA 0.9145 0.8075 0.8291 0.8326

The upper left result, labeled “Overall,” considers each class distinctly so that a MODIS ranking of nominal-confidence fire and a VIIRS ranking of a high-confidence fire would be recorded as an error. If however, the distinction between different confidence levels in the fire classes (classes 7, 8 and 9 in the algorithm result) are ignored, a higher success rate results. This is the assumption in the upper right result shown in Table 4. If low-confidence fire classes are ignored and nominal- and high-confidence fire classes are grouped together, the result is shown in the bottom left portion of Table 4. While this comparison shows slightly improved results, there is little impact in ignoring low-confidence fire results because so few pixels are assigned this classification. If only high-confidence fires are considered (with low- and nominal-confidence fires being ignored), the result is shown in the bottom right portion of Table 4. The very high values of for the case of high-confidence fire detection suggests that the ability to assess highconfidence fire should remain unchanged with the introduction of the VIIRS sensor. Because the simulated VIIRS data was generated using MODIS data as a significant input, the difference in the 4-µm channels’ saturation temperatures (discussed in Section 3.3) is not fully examined in this analysis. Nonetheless, this aspect of the two algorithms appears to be extremely consistent across the sensors. We sought to examine this band-saturation issue more closely. We examined the extent to which difference levels of fire confidence were occurring in the MODIS- and VIIRS-based products, presented in Table 5. We discovered a significant difference with respect to low and nominal confidence fires. The VIIRS-based product yields nominal confidence fires at a rate of 20% less than the MODIS-based products. Further, the VIIRS-based products did not yield any low confidence fires for the data considered in this RPC. To further examine this difference the two TERRA-based results were compared using a Chi-Squared goodness-of-fit test and found to be significantly different (P < 0.0001). The two AQUA-based results were compared to find similar results. In each case, the largest contribution to the chi-squared statistic came from the 18

disagreement attributable to the nominal-confidence fires (class 8). This reveals a distinct difference between the MODIS and VIIRS-based fire products. Table 5. A comparison of the overall frequency of classes 5, 7, 8, and 9 across the four study dates for each of the data types. Comparison of Overall Classification Frequency Classification MODIS:Terra VIIRS:Terra MODIS:Aqua VIIRS:Aqua No Fire (5) 0.9991196 0.9995138 0.9986930 0.9989892 Low Conf Fire (7) 0.0000028 0.0000000 0.0000046 0.0000000 Nominal Conf Fire (8) 0.0004331 0.0000926 0.0004942 0.0000768 High Conf Fire (9) 0.0004445 0.0003936 0.0008082 0.0009340

4.2 Comparing Detection Tools Using Validation Data Sets: Results from the ASTER Imagery The MODIS fire product and the simulated VIIRS fire product were evaluated by comparison to ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) derived fire maps following the procedure described by Morisette et al. (2005a). In this procedure ASTER bands 3N and 8 were converted from calibrated radiance to top-of-atmosphere reflectance and band 3N was aggregated to 30 meters nominal spatial resolution to match the spatial resolution of band 8. Eleven scenes were used: four scenes of April 30, 2003 covering the western part of Guatemala and seven scenes of April 21, 2003 covering a north-south strip over Guatemala. The algorithm described by Morisette et al. (2005a) and others starts by creating two intermediate images: ratio and difference of band 8 and band 3N. The algorithm performs two passes over each pixel. In the first pass a simple threshold operation is performed over the ration and difference image. Pixels that exceed these thresholds are flagged as obvious fires and therefore excluded from the second pass. In the second pass the neighboring pixels are considered (61x61 pixel square window centered in each pixel) and another sequence of thresholds is applied. The final map is a binary image containing fire pixels either flagged as active fires or non-fires. The final active map was then visually inspected to make sure that the no false fire pixels were present. The VIIRS and MODIS fire products were then evaluated by using the ASTER fire map as reference. The VIIRS and MODIS fire products pixels had to meet two criteria: (1) the pixel must lie within the spatial areas represented by the ASTER footprint; and (2) the pixel value must be 5, 7, 8, or 9 to be considered in the evaluation process. For each pixel meeting these criteria, the number of ASTER fire pixels was then counted and values calculated for the associated statistic, the omission error rate, and the commission error rate for different ASTER threshold values, where the threshold value is the minimum number of fire pixels to consider fire. The results in Figure 5 indicate that an optimal threshold could be obtained with a number of ASTER fire pixels between 10 and 20. This method of assessing detection success is consistent with results described by Morisette et al. (2005a), except that we have added the statistics and the concept of an optimum threshold value.

19

VIIRS - TERRA

MODIS - TERRA 0.1600

0.3500 April, 21 2003

April, 21 2003

April, 30 2003

April, 30 2003

0.1400

0.3000

0.1200

Kappa Statistic

0.2500

0.1000

0.2000

pp

0.0800

0.1500

0.0600

0.1000

0.0400

0.0500 0.0000 1.0000

0.0200

0

10

20

30

50

60

70

80

90

100

ASTERThreshold

0.9000 Probability (Omission Error)

40

0.0000 1.0000

0.8000

0.8000

0.7000

0.7000

0.6000

0.6000

0.5000

0.5000

0.4000

0.4000

0.3000

0.3000

0.2000

0.2000

0.1000

0.1000

0.0000 0.0035 0

10

20

30

40

50

60

70

80

90

100

0.0000 0.0014

ASTERThreshold

10

20

30

40

50

60

70

80

90

100

70

80

90

100

70

80

90

100

ASTERThreshold

0

10

20

30

40

50

60

ASTERThreshold

0.0012

Probability (Commission Error)

)

0.0030

0

0.9000

0.0010

0.0020

0.0008

0.0015

0.0006

y(

0.0025

0.0010

0.0004

0.0005

0.0002 0.0000

0.0000 0

10

20

30

40

50

60

70

80

90

100

0

10

20

30

40

50

60

ASTERThreshold

ASTERThreshold

Figure 5 – Summary of the Kappa statistic, Omission Error Probability, and Commission Error Probability values for different ASTER thresholds 4.2 Comparing Detection Tools Using Validation Data Sets: Results from the LANDSAT Imagery To provide a further basis of comparison for of the success rate of the fire detection systems, we used the available LANDSAT imagery introduced in Section 3.1. Two different experts familiar with the task of classifying imagery were asked to identify wildfires in the LANDSAT imagery. Efforts to collect reliable groundtruth information (i.e., date, location, and area of active fire sites) from Guatemala did not yield promising results. Hence it was decided to create

20

groundtruth data by visually interpreting LANDSAT ETM and ASTER data that were collected for the same dates previously considered: 20 March, 21 April, 28 April, and 30 April. The LANDSAT data were imported and geometrically corrected using ERDAS Imagine. The false color composite (FCC) (B1-Blue, B2-Green, and B7-Red) for each date was loaded onto ArcMap and clipped along the boundary line of Guatemala. Each FCC was examined closely from left to right and top to bottom to look for active fires. An active fire was identified based on its association with smoke that was visible in the FCC. Figure 6 presents examples of active fires.

Figure 6. Examples of LANDSAT false color composite images showing active fires in Guatemala. Yellow color in the left image is a polygon representing an active fire. Polygon data of fire polygons were generated for all of the four dates using ArcMap. Finally, the area and the central coordinates of the polygons were determined using Arc tools. When active wildfires that both experts identified were compared, confirmed fires were defined as those cases where both fire fighters identified fires at a location. These confirmed fires were then compared to the MODIS- and simulated-VIIRS-based fire products. A total of 60 fires were identified by both of independent ‘expert’ classifiers. We then compared the validation data set of 60 fires to the classified pixels from the SERVIR DST results. For each fire location listed in the field data it should be assessed as to whether a fire was detected in the imagery or not and also the size of the detected fire. Because the real conditions of fire (or no fire) can clearly influence sensors readings in adjacent pixels (Townshend et al., 2000) we proposed that pixels be considered using a 5-by-5 pixel window centered at the location of interest, as shown in Figure 7.

21

Figure 7 – 5-by-5 MODIS pixels window centered at each field fire location If a recorded location of a confirmed fire (marked with * in Figure 7) falls within the 5-by-5 grid it would be considered a true positive. In other words for each ‘confirmed’ fire identified, the pixel information of the 25 nearest pixels are recorded. Where any of the 25 pixels have values of either 7, 8, or 9 a finding or agreement is indicated. Figure 8 shows an example of the proposed approach. The success rate of both of the DSTs were applied to the Aqua and Terra cumulative results for the four dates to produce the success rates shown in Table 6.

Figure 8 - Fire location and the 25 nearest MODIS pixels collected to investigate agreement with field data Table 6. Comparison of fires identified in LANDSAT imagery and fires detected by the MODIS- and VIIRS-based DST. March 20, 2003 All Fires Sensor No. Found Fires MODIS:Aqua 0 1 VIIRS:Aqua 0 1 MODIS:Terra 1 1 VIIRS:Terra 0 1

April 21, 2003 No. Found Fires 10 39 0 39 23 39 13 39



Overall No. Found Fires 10 60 13 60 30 60 27 60

Accuracy (%) 16.67% 21.67% 50.00% 45.00%

The results demonstrate inconsistent success across the four dates. Further, the success rate for data from the Aqua sensor appears to be significantly poorer than the Terra server. It must be noted, however, that the LANDSAT imagery was collected closer in time to the Terra-based sensor than the Aqua-based sensor. Both the MODIS-based data and the LANDSAT data are collected during morning hours (NASA, 2007), but the AQUA-based data is collected hours later 22

during the afternoon hours. It is not surprising, therefore, that the Terra-based data appears to show greater success rate in matching LANDSAT-based classifications. For the purposes of this RPC experiment, the most significant finding is that the MODIS- and VIIRS-based results were very similar. This provides a strong indication that the results of the DST will continue with the introduction of the VIIRS sensor. We also sought to examine whether the success in detecting fires depends upon the size of the apparent fire. To accomplish this, we calculated the mean fire size for the 60 confirmed fires and examined the success rate for those fires smaller than the mean (a total of 40 fires) and those fires larger than the mean (20 fires). These results are presented in Table 7, where the results appear to show roughly similar overall results for both large and small fires. It is somewhat surprising, however, that the success rates vary considerably from day to day. For example, on April 21, 2003 for the Terra-based data the MODIS-based DST appeared to have greater success than did the VIIRS-based DST. On April 30, 2003 for the Terra-based data the VIIRS-based DST appeared to have greater success than did the MODIS-based DST. These results were examined on the basis of the area of the identified fire with the expectation that larger fires, would be more likely to be found than small fires. Given the fairly consistent overall results for both large and small fires, there does not appear to be a significant relation between fire size and the success rate in fire detection. Previously investigators (eg., Giglio et al., 2003) have reported that the intensity of a fire may have a stronger influence on the success in detecting a fire. Table 7. Comparison of both large and small fires identified in LANDSAT-7 imagery and fires detected by the MODIS- and VIIRS-based DST. March 20, 2003 Small Fires Sensor No. Found Fires MODIS:Aqua 0 1 VIIRS:Aqua 0 1 MODIS:Terra 1 1 VIIRS:Terra 0 1





Overall Accuracy 15.00% 17.50% 45.00% 50.00%

Large Fires March 20, 2003 Sensor No. Found Fires MODIS:Aqua 0 0 VIIRS:Aqua 0 0 MODIS:Terra 0 0 VIIRS:Terra 0 0





Accuracy (%) 20.00% 30.00% 60.00% 35.00%

5.0 Conclusions and Recommendations This RPC Experiment examined the impact of the expected sensor transition on SERVIR’s fire detection DST. The simulated VIIRS data was generated using the available MODIS data, with the inevitable limitation that the simulated VIIRS data retained the spatial resolution of the MODIS data. In fact, the planned VIIRS sensor will have greater resolution than the current sensor. The consequence of this change would be expected to improve fire detection as each sensor is responding to a smaller area, providing the opportunity for a more responsive measurement of ground conditions. We were, however, able to assess the impact of the transition to the anticipated VIIRS sensor. SERVIR’s fire detection DST is also expected to require a change in the algorithm used for fire detection. We did not make changes to any algorithm, instead the scientific team that has been supporting SERVIR’s tools provided us with access to their algorithm as adapted for the anticipated VIIRS sensor. 23

5.1 Summary of Findings We compared MODIS- to VIIRS-based fire products directly for the four study dates during the 2003 fire season. In addition, we have used ASTER and LANDSAT imagery as the basis for comparing each set of fire products to a set of probable fires evident in each of the higher resolution imagery. We find that there is reasonable agreement between the MODIS- and VIIRS-based products on the basis of direct comparison. While agreement rates are extremely high, largely due to the fact that fires are spatially uncommon, the kappa statistic was used to provide a measure of agreement. The calculated values of revealed good to excellent agreement beyond chance agreement. The highest values were obtained when the MODIS- and VIIRS-based assessments of high confidence fires were compared. The VIIRS-based fire products results in relatively few nominal-confidence fires and almost no low-confidence fires. The excellent agreement on the location of high-confidence fires provides encouragement regarding the continued value of SERVIR’s fire-detection program beyond the life of the MODIS sensors. A different result was obtained when the overall distribution of classifications were compared. The difference between the two fire product results was found to be most significant for low and nominal confidence fires, where the overall results revealed that the classifications failed goodness-of-fit tests applied to the MODIS- and VIIRS-based distribution across classes 5, 7, 8, and 9. We also compared the accuracy of both the MODIS- and VIIRS-based products against fires observed in both ASTER and LANDSAT-7 imagery. The MODIS-based products compared reasonably well with the ASTER imagery, with results roughly comparable to those reported by other investigators (Morisette et al., 2005a; Morisette et al., 2005b). When the VIIRS-based products were compared to the same fires detected using ASTER imagery, distinctly poorer results were observed with lower values of the kappa statistic and higher omission error values. This indicates that the challenge of detecting small fires, which presumably may translate to lower confidence fires, is particularly problematic with the VIIRS data and the current VIIRSbased algorithm. Finally, the two sets of fire products were compared to fires identified by two experts working independently with LANDSAT-7 imagery from the four study days. A total of 60 fires were identified by both experts (each fire independently identified). When these fires were compared to fires detected with the MODIS- and VIIRS-based DST, the results demonstrated moderate, but more importantly nearly equal, success rates for the Terra-based products. The validation fire set was compared to the results obtained using the Aqua-based products yielding distinctly poorer success rates, but those were not considered as significant given the time lag between the collection of the LANDSAT-7 imagery and the Aqua data. The overall outcome of this RPC is the demonstration that SERVIR’s fire-detection DTS is expected to continue to perform well at detecting fire currently being detected at the highconfidence level. Small fires or fires of low intensity do not appear to be as readily detected using the combination of the planned VIIRS sensor and the fire-detection algorithm designed for that sensor. The limitations of this RPC must also be considered. We used simulated VIIRS imagery where the current MODIS sensor was used to generate the pseudo-VIIRS dataset, we did not have a confirmed set of fires detected at the surface at the time of imagery collection, and 24

we did not examine the consequences of the improved resolution associated with the expected VIIRS sensor. 5.2 Recommendations The outcomes of this experiment suggest that sustained performance of SERVIR’s fire-detection DST will require changes from the currently contemplated DST. Other investigators examining the utility of a 3.95-μm band for detecting fires had difficulty detecting small or low-intensity fires (Gao et al., 2007). Other researchers (eg., Miller and Liu, 2002) have proposed changes to the fire-detection algorithm to overcome the limitations of this thermal band. The outcomes of this RPC experiment are consistent with the recommendations of Gao et al. (2007) and Miller and Liu (2002) and provide incentive to alter the current sensor and algorithm combination that has been proposed for SERVIR’s fire-detection DST. 5.3 Future Steps to Application of VIIRS in SERVIR’s Fire Detection System We recommend the systematic collection of ground truth data by personnel in Central America during periods of active wildfires. During the course of this RPC experiment we have collaborated with SERVIR and CATHALC investigators to promote such efforts. With this proposed dataset, the validation and calibration of the algorithm being used in the SERVIR DST can be altered when real VIIRS data becomes available. 5.4 Lessons Learned in this Rapid Prototyping Experiment In the process of completing this Rapid Prototyping Experiment, we learned a number of lessons that can contribute to the smoother or more successful completion of our future experiments. First, we learned that working with different research groups leads brings with it the challenge of differences in terminology, experience, and procedure. The simple qualities that constitute a fire and therefore the records of a fire were found to be different between groups that we had planned to involve in this experiment. The implications led to significant challenges when interpreting a truth set with which to compare the DST results. In future, a more complete discussion and agreement in terminology and experiment goals will be used. Second, the difference in resolution between the current MODIS sensor and the planned VIIRS sensor creates more uncertainty than we would want in future RPC experiments. Our analyses are subject to the uncertainty that comes from using an existing sensor with a difference in resolution to simulate the anticipated data from a future sensor with a different resolution. We used similarity that we found between the DSTs when applied to existing and simulated sensor data to establish credibility of procedures. We used differences between the DST results to highlight areas where the transition to the new sensor may be expected to lead to changes in the DST results. In future RPC experiments, we will seek alternative methods to simulate sensor data that will not be subject to this resolution discrepancy problem. Third, the findings of this experiment constitute a successful application of the intended RPC experiment procedures. The methods of analysis that we have established in this experiment will be applied to future experiments. 25

6.0 References Byerly, W.P., Clement, J.E., Dorman, T., Engel, J., Julian, R., Miller, S., 2002. Radiometric Calibration: Visible/Infrared imager/radiometer suite algorithm – theoretical basis document, The Raytheon Company, Information Technology and Scientific Services, Lanham, MD. Gao, B.-C., Xiong, X., Li, R.-R. and Wang, D.-Y., 2007. Evaluation of the Moderate Resolution Imaging Spectrometer special 3.95-micron fire channel and implications on fire channel selections for future satellite instruments. Journal of Applied Remote Sensing 1: 13516 13524. Giglio, L., Descloitres, J., Justice, C.O. and Kaufman, Y.J., 2003. An enhanced contextual fire detection algorithm for MODIS Remote Sensing of Environment, 87(2-3): 273-282. Jenness, J. and Wynne, J.J., 2005. Cohen’s Kappa and Classification Table Derived Metrics: An ArcView 3x extension for accuracy assessment of spatially explicit models, USGS Southwest biological Science Center. Lee, T.F., Miller, S.D., Schueler, C. and Miller, S., 2006. NASA MODIS previews NPOESS VIIRS capabilities. Weather and Forecasting, 21(4): 649-655. Lillesand, T.M. and Kiefer, R.W., 2000. Remote Sensing and Image Interpretation. John Wiley and Sons, , Toronto. Masuoka, E., Fleig, A., Wolfe, R.E. and Patt, F., 1998. Key characteristics of MODIS data products IEEE Transactions on Geoscience and Remote Sensing, 36(4): 1313-1323. Miller, S.W. and Liu, Q., 2002. Active Fires: Visible/Infrared Imager/Radiometer suite algorithm Theoretical Basis Document, Raytheon Company, Lanham, MD. Morisette, J.T., Giglio, L., Csiszar, I., Setzer, A., Schroeder, W., Morton, D., 2005a. Validation of MODIS active fire detection products derived from two algorithms. Earth Interactions, 9(9): 1-25. Morisette, J.T., Giglio, L., Csiszar, I., Setzer, A., Schroeder, W., Morton, D., 2005b. Validation of the MODIS active fire product over southern Africa with ASTER data. International Journal of Remote Sensing, 26(19): 4239-4264. NASA, 2007. LANDSAT-7 Science Data User’s Handbook. NASA: Greenbelt, MD. Townshend, J.R.G., Huang, C., Kalluri, S.N.V., Defries, R.S., Liang, S. and Yang, K., 2000. Beware of per-pixel characterization of land cover. International Journal of Remote Sensing, 21: 839843. Welsch, C., Swenson, H., Cota, S.A., Frank DeLuccia, Haas, J.M., Schueler, C., 2001. VIIRS (Visible Infrared Imager Radiometer Suite): A next-generation operational environmental sensor for NPOESS, Geoscience and Remote Sensing Symposium, 2001. IGARSS '01. IEEE 2001, Sydney, NSW Australia, pp. 1020-1022 Yu, Y., Privette, J.L. and Pinheiro, A.C., 2005. Analysis of the NPOESS VIIRS land surface temperature algorithm using MODIS data. IEEE Transactions on Geoscience and Remote Sensing, 43(10).

26