Proceedings of the ASME 2009 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference Proceedings of the ASME 2009 International Design Engineering Technical Conferences & IDETC/CIE 2009 Computers and Information in Engineering Conference August 30 - September 2, 2009, San Diego, California, USA IDETC/CIE 2009 August 30-September 2, 2009, San Diego, USA
DETC2009-86725 DETC2009-86725
ACQUISITION OF HIGH QUALITY REMOTE SENSING DATA USING A UAV CONTROLLED BY AN OPEN SOURCE AUTOPILOT
Marc Schwarzbach∗
Uwe Putze
[email protected]
[email protected]
Ursula Kirchgaessner
Dr. Maria v. Schoenermark
[email protected]
[email protected] Insitute of Space Systems, Universitaet Stuttgart 70569 Stuttgart, Germany
1
ABSTRACT This paper describes the integration and application of the Paparazzi autopilot into an aircraft system for scientific research on remote sensing. The main purpose of the presented UAV, Stuttgarter Adler, is remote sensing of the environment for fundamental radiometric research as well as applications in agriculture, surveying and mapping. Employing a UAV for the retrieval of remote sensing data of quality comparable to data from manned missions represents a very flexible and inexpensive method of data acquisition. The sensors required for the intended resolution total several kilograms in weight and led to the construction of a matching airplane. All intended missions require a robust and precise control of the aircraft during image acquisition flights, which can only be achieved with an automated pilot assistance system. The Paparazzi system was chosen because its open source approach allowed to adapt the autopilot to the specific mission requirements. Interfaces to the digital remote control and to the camera and spectrometer payloads have been created. Test flights show good results in stabilizing the airplane and controlling the payload.
∗ Address
INTRODUCTION
Many applications and scientific questions ask for high quality remote sensing data, with high radiometric and geometric precision and accuracy. The UAV Stuttgarter Adler and its payload have been designed especially as a flexible and inexpensive, yet highly precise, platform to meet these needs for high quality data. Acquisition of environmental data in great detail is usually expensive and time-consuming. Ground sampling can only return data for selected points, and great effort has to be invested in geostatistical data assessment to achieve area coverage. Remote sensing allows direct acquisition of area information, since each pixel of a digital image represents an average of the corresponding ground spot. Satellites and manned aircraft are commonly used for this purpose. Geostationary satellites allow frequent observation but with poor spatial resolution and on a fixed schedule. Satellites on lower orbits achieve better spatial resolution but temporal and areal coverage may be insufficient. Compared to satellites, manned aircraft can be more flexible and higher spatial resolution can be achieved, but both satellite and airborne remote sensing are expensive, hence only feasible for large areas and projects. In many cases these shortcomings can be overcome by small unmanned aircraft, which are more flexible and
all correspondence to this author.
1
c 2009 by ASME Copyright
inexpensive. Furthermore, through observation from only 300m above ground, more accurate data can be obtained since the influence of the atmosphere is smaller than at higher altitudes. There have been many efforts to provide remote sensing data using low cost UAV, mostly with standard consumer cameras for imaging, e.g. [1, 2]. However, their sensors are covered with Bayer-filters, a mosaic of selectively transmissive filters, to produce color pictures. This means that each pixel is sensitive to only one color while the other colors have to be interpolated. Furthermore, the filters do not represent separate regions of the electromagnetic spectrum as they are largely overlapping. This kind of camera can only return very limited information about the observed surface and is not sufficient for our purpose. In order to achieve the required image quality, special payload instruments have been chosen, namely a spectrometer, separate cameras for the visible and near infrared wavelengths, and a thermal infrared camera. Figure 1 shows the tranmission of the selected green, red and near infrared filters. In order to acquire complete coverage of the desired area, automated aircraft control is necessary. With regular remote control from the ground it is not possible to fly the aircraft as steadily as desired for image acquisition. Both the large distance to the aircraft and the limited reaction time of the pilot lead to large deviations of the aircraft from the intended attitude. Moreover, the very accurate flight geometry achieved by an autopilot greatly simplifies and expedites subsequent image processing.
Growth and health parameters of plants can be derived from their spectral signature in the visible and near infrared wavelengths and from thermal infrared radiation temperature. For example vegetation can be classified or evaluated by vegetation indices, which compare image brightness in different radiation bands. One very simple but meaningful algorithm is the Normalized Differenced Vegetation Index (NDVI) NDVI =
(1)
where RED and NIR stand for the digital counts measured in the near infrared band and the red band respectively. Since healthy plants have a much stronger reflectance in the near infrared than in the red region, they return values close to unity. Low NDVI results from lack of chlorophyll, which can represent either not vegetated ground or stressed plants. This type of information is required for precision agriculture, that is, to improve crop yields by spatially varying irrigation, manuring, and pest control within fields according to local needs. Furthermore, precision agriculture can be used to predict crop yield at an early stage of growth. Another important factor in precision agriculture are topsoil characteristics. For example organic matter, nutrients and mineral content, texture, and moisture are all related to soil fertility. These properties can also be derived from spectral signatures and radiation temperature. Precision agriculture not only improves turnover rates but also contributes to the protection of the environment by reducing pollution of ground water with fertilizer and pesticides by minimizing the applied amount. Last but not least, this reduction constitutes a financial benefit [3, 4]. The first application of the Stuttgarter Adler is a study carried out in cooperation with the Institute of Soil Science and Land Evaluation at the University of Hohenheim, Germany, with the goal to develop fine resolution maps of soil properties such as surface albedo, topsoil organic matter, and texture based on the remote sensing data. Starting in April 2009, remote sensing flights will be conducted while ground samples will be taken simultaneously for calibration and validation. Experiments are being carried out on several fields of the Ihinger Hof, a research station for crop production and crop protection near Stuttgart. Further feasible applications of the UAV with the current scientific payload are, for example, surveying, monitoring and mapping of landfill sites, surface mining sites, forest vitality, forest fire damage, animal populations, and soil sealing.
2 APPLICATIONS 2.1 Precision Farming One main purpose of the UAV Stuttgarter Adler is the remote sensing of the environment for applications in agriculture.
100 90 80
Transmission (%)
NIR − RED , NIR + RED
70 60 50 40 30 20 10 0 400 450 500 550 600 650 700 750 800 850 900 950 1000 1050 1100
2.2
BRDF Research The second major task of the research program is the measurement of the Bi-directional Reflectance Distribution Function (BRDF), from an altitude of 300m. The BRDF describes the reflectance of a surface depending on the illumination and viewing
Wavelength (nm) GREEN
Figure 1.
RED
INFRARED
GREEN, RED, AND NEAR INFRARED FILTERS.
2
c 2009 by ASME Copyright
Table 1.
Figure 2.
VISUALIZATION OF BRDF ANGLES [5].
LV (λ, Θi , φi , Θr , φr ) , E(λ, Θi , φi )
Spectrometer
Thermal camera
Manufacturer/ Model:
Vision Components VC4068
Avantes AvaSpec 128
Dias Infrared Compact 320L
Sensor:
1280 x 1024 pixel CCD
128 photo array
320 x 240 pixel uncooled bolometer
Spectral ranges:
520, 630, 880 nm
400 - 1000 nm
8 - 14 µm
Ground resolution at 300 m:
8 cm
50 cm
21 cm
Swath width:
ca. 80 m
50 cm
80 m
Mass:
460 g per camera
150 g
1.6 kg
pixel diode
IR
3 RESULTING REQUIREMENTS 3.1 Payload The payload used for this research consists of optical sensor systems for the visual and near-infrared spectral ranges, as well as a thermal infrared camera, as listed in Table 1. Three industrial CCD cameras, each equipped with a different spectral filter, provide matrix images of the target area with a ground resolution of approximately 10 centimeters at a targeted flight altitude of 300 meters. To complement the matrix image data, a spectrometer is included to provide a continuous spectrum, captured at a single point located at the center of the image area with a ground resolution of approximately 50 centimeters. The cameras and the spectrometer are swivel-mounted, to allow for the tilted viewing angle required for the BRDF measurements. Since no tilting is required for the thermal infrared camera, it is mounted looking nadir.
geometry. It is described as f by the equation
f (λ, Θi , φi , Θr , φr ) =
PAYLOAD INSTRUMENTS.
VIS/NIR Cameras
(2)
where E represents the irradiance from the light source, in this case the Sun, and L is the radiance measured by the sensor. Θ describes the zenith angles and φ the azimuth angles of the Sun, i, and the sensor, r. The geometry is illustrated in figure 2, with φi set to zero. The function is also dependent on the wavelength λ. Applications of BRDF data are the correction of air and spaceborne images and the improvement of classification results when data from more than one observation angle is available. BRDF effects have become important since the end of the 1990s, when high precision instruments were starting to be used in remote sensing.
3.2
Autopilot During the mission, the autopilot shall navigate the airplane along its predetermined path. To allow for precise control of the camera viewing angle as well as the timing of the image capturing, those functions are maintained by the autopilot as well. For mapping and surveying missions, the airplane will have to follow a systematic path covering the whole target area with a given image overlap. The camera release has to be timed accordingly by the autopilot. For BRDF measurements the airplane will fly in a circular pattern centered on the target area. The tilting mechanism will be used in this case to capture the area at a given zenith angle, while the complete range of azimuth angles is covered by the circular flying pattern. The camera release as well as the viewing angle have to be adjusted by the autopilot accordingly in this case.
Since then, most BRDF data sets of different surfaces have been acquired by ground measurements [6]. The data base is nevertheless too small to perform intensive comparisons [7]. In most cases, BRDF models are used when processing remote sensing data [5]. The use of UAV-based measurements offers the chance to expand the overall quantity of data. Additionally the influence of the atmosphere can be derived from parallel measurements with a ground based instrument already in use at the institute [8]. The definition of the BRDF assumes a point source. When measuring under natural conditions the additional diffuse radiation leads to the measurement of the hemispherical reflectance distribution function (HRDF). This can then be converted to a BRDF [9]. 3
c 2009 by ASME Copyright
Figure 3.
eter on their swivel-mount, the thermal infrared camera, as well as all necessary batteries and electronics. The twin-engine design allows for a redundancy in case of an engine failure. This system has been tested in flight and it has been shown that even with only one active motor a climb is possible. Using 35MHz RC channels interference is a possible threat, therefore a system transmitting on two separate channels is used. In the airplane a diversity dual receiver system is implemented as shown in Fig. 4 which checks the validity of each channel’s signal. Also digital encoding is used for the control link adding check sum validation and higher resolution. The Trainer airplane is built in a simple plywood construction, allowing for low cost and ease of repairs. Its wingspan measures 3 meters. Since the main purpose of this aircraft is to familiarize the pilot with controlling a larger scale airplane, it was designed for flying characteristics that match the larger Stuttgarter Adler as closely as possible. It uses airfoils and wing proportions similar to the Stuttgarter Adler and is equipped with the same electrical engine. It also serves as a testbed for tuning of the Paparazzi autopilot system before its implementation into the larger Stuttgarter Adler. Due to the limited size and payload capacity, the Trainer can only carry a single camera and the spectrometer as payload. A special mounting device has been constructed and manufactured combining the optical path of the camera and spectrometer, see Fig. 5. A rotatable mirror allows control of the viewing angle, as required by the mission. The third airplane is a commercially available Easy Glider Pro, manufactured by Multiplex. It is used for testing new Paparazzi hardware and software as well as the automatic triggering of a payload by the autopilot. Due to its very limited size and capacity, the Easy Glider can only carry a simple webcam.
THE STUTTGARTER ADLER.
4 UAV SYSTEM 4.1 Airplane Models This research is conducted using a fleet of three airplane models, see Table 2. The Stuttgarter Adler is the largest airplane of the three and it is capable of carrying all necessary payload instruments. Because of its size, manually controlling this airplane requires experience in flying large scale model aircraft, a skill which is rather uncommon with most hobby model aircraft pilots, due to the costs involved in building or purchasing such an airplane. To allow a potential pilot to familiarize oneself with the control of larger scale airplanes, a Trainer was constructed, which is smaller than the Stuttgarter Adler but still larger than the airplane types most commonly found in the hobbyist field. The Trainer is not able to carry all the payload required for the planned research projects, however it can still be used for missions requiring smaller payloads and as a test aircraft for electronic systems and especially the autopilot system. A third, commercially available, airplane is used in this project. This simple polystyrene airplane serves as a starting point for testing and learning to use the selected Paparazzi open-source autopilot system, since so far most of the development and demonstration of this system has been conducted using similar sized airplanes. The three airplanes used in this research project are described as follows. The Stuttgarter Adler was designed for remote sensing missions using the payload described above [10]. It is an electrical twin-engine aircraft with a wingspan of approx. 4.3 meters and is capable of carrying a total takeoff mass of 25 kilograms to an altitude of 300 meters. The wing was designed to allow for a low speed of 10 - 15 meters per second during the measurement, while still providing a tolerable flight stability. All wings are constructed of polystyrene and wood, the body and engine nacelles are made of glass fiber reinforced plastic. All electronic systems related to flight control are placed in the nacelles and main wing. For varying missions, multiple bodys can be equipped with the according payloads using the same wings and motors. The fuselage is large enough to hold the three cameras and the spectrom-
4.2
Autopilot System Setup The autopilot selected for this research project is the Paparazzi open-source autopilot system. This system consists of Table 2.
4
OVERVIEW OF AIRPLANE MODELS. Stuttgarter Adler
Trainer
Easy Glider
Wingspan:
4.33 m
3m
1.77 m
Overall length:
2.66 m
2.1 m
1.1 m
Max. weight:
25 kg
14 kg
1.2 kg
Payload capacity:
7 kg
4 kg
300 g
Engines:
2x PolyTec C42-62
1x PolyTec C42-62
Himax C 28161220
Batteries:
2x 10s2p 8000 mAh LiPo
1x 10s2p 8000 mAh LiPo
1x 2s1p 2100 mAh LiPo
Duration:
30+ minutes
30+ minutes
30+ minutes
takeoff
c 2009 by ASME Copyright
Figure 4.
SYSTEM OVERVIEW OF AUTOPILOT INTEGRATION IN THE STUTTGARTER ADLER.
digital, pulse code modulation (PCM), link. To adapt the Paparazzi system to this link, a special electronics board had to be designed and built. It uses the serial port of the receiver containing the control data and converts it to a pulse pause modulation (PPM) signal. Since the Stuttgarter Adler uses a dual receiver configuration, the serial port cannot be used to extract the control signal in this case, since this port is used by the two receivers to communicate with each other. Both receivers exchange their PCM signals and an internal logic then decides which receiver delivers the best result. This is then used as the output to the servo channels. Intercepting the PCM signal of each receiver would require a rebuild of this internal logic externally and thus defeat the purpose of the diversity system. Therefore a different approach is used in this case to connect the autopilot to the receiver system, by including an electronic board that unifies the servo channel signals generated by the diversity system into a single PPM stream which can be used by the Paparazzi. An overview of the system setup is shown in Fig. 4. To ensure a correctly timed release of all payload instruments, a separate electronics board was created. This board interfaces with all payload systems and is itself triggered by the autopilot. Upon receiving this signal, the cameras and the spectrometer are triggered immediately. The board then receives the recorded data from the spectrometer and stores it on an integrated memory card. The cameras have a built-in flash memory, which allows them to store the image data until retrieval after the airplane has landed. Since the thermal infrared camera can only be used connected to a PC, a netbook computer is included in the payload. Nevertheless the camera is triggered by a signal from the autopilot. All data is time stamped and can later be matched to the position and orientation data stored by the autopilot.
designs for the autopilot hardware, as well as all necessary software for mission planning, a ground station, and the autopilot itself [11]. It is supported by a growing community and is freely available on the internet. Due to this open-source nature, the Paparazzi system can be adapted to the specific needs of this research project. The Paparazzi system has already been used successfully in meteorological research [12, 13]. The integration of the Paparazzi system into the small Easy Glider Pro proved to be straightforward, since this system has been used in similar airplanes in the past and this was documented by the community. Also, examples showing the shutter release of an attached camera could be used. For safety reasons the Trainer airplane is controlled by a
Figure 5. THE SPECTROMETER AND A CAMERA ON THE MOUNTING DEVICE.
5
c 2009 by ASME Copyright
5 RESULTS Flights to get used to the autopilot system were performed throughout the summer of 2008 with the described Easy Glider airplane. Stable flight in automatic mode was achieved on the first day. The following tests focused on flight strategies, especially for surveying specific fields, as this will be one of the main tasks. A simple camera was integrated in the airplane and modified to be released by the autopilot. Wind proved to be the most critical factor using such a light airplane. Crosswinds have a large influence on the roll angle which prevent the camera from looking straight down as desired. However, these inaccuracies are marginal compared to flights controlled manually. The good alignment of the images was then used in the postprocessing for mosaicking, to obtain a single image of the whole area. For this purpose the position of the airplane during each image recording can be read from the autopilot logfile. Since the accuracy of the attitude measured by the paparazzi system is not sufficient, the SIFT algorithm is used for image matching [14]. As an example the resulting mosaic from images of one flight is shown in Fig. 6. In total, several hours of flight time were logged. The autopilot performed well and no problems occured. The next step was to test the autopilot in the bigger Trainer airplane. As the influence of the high power electric drive on the system was uncertain, extensive ground tests were performed, but interference was not observed. Adjustment of parameters in the control loops was the next task, which required some extrapolation since the existing configuration files only cover airplanes weighing less than 1 kilogram. The flights showed good results when switching to the manual rate control mode, meaning the inner control loops were able to stabilize the big airplane. In the autonomous navigation mode however, an oscillation around the desired course could be observed. Unfortunately, the test flights were interrupted by bad weather and could not be completed before winter. Parallel flights were conducted with both the Trainer and Adler airplanes using manual control. The primary goal was to test the different payloads in flight. Several fields of the Ihinger Hof were used as sample targets. The combined spectrometer and camera assembly was used in most cases. The images of the cameras taken with the different filters showed good ground resolution as shown in Fig. 7. Some problems occured with the sensitivity of the spectrometer. In order to obtain a useful signal to noise ratio, very long integration times had to be used. A manufacturing error could be accounted for this problem. After repair, the integration times are now expected to be in a useful range.
Figure 6.
MOSAICKING EXAMPLE.
the flexibility in building interfaces confirms this decision. The individually proven components for imaging and flight control will now be assembled into one system to carry out remote sensing missions. The cooperation with Ihinger Hof will provide opportunities to validate UAV based measurements by soil samples and crop analysis. The installation of a thermal camera will allow a wider range of applications like animal counting or landfill inspection. The extension of the capabilities of the UAV will continue. Recent development of small hyperspectral instruments [15] will be used to evaluate if implementation of such an instrument is possible. The benefit of the generated data
6 CONCLUSION The data obtained shows that high quality remote sensing data can be obtained using an inexpensive UAV. The quality of the data can compete with that of manned aircraft. The implementation of an open source autopilot returned good results and 6
c 2009 by ASME Copyright
Figure 7.
[4] Fox, G. A., and Metla, R., 2005. “Soil property analysis using principal component analysis, soil line, and regression models”. Soil Science Society of America Journal, 69, pp. 1782–1788. [5] Beisl, U., 2001. “Correction of bidirectional effects in imaging spectrometer data”. Remote Sensing Series, 37. [6] Schoenermark, M., Geiger, B., and Roeser, H., 2004. Reflection Properties of Vegetation and Soil. Wissens. und Technik Verl., Berlin. [7] Bohl, P., 2004. “Auswertung einer BRDF Datensammlung”. Diploma thesis, Institute of Space Systems, Universitaet Stuttgart. IRS 04-S-21. [8] Schwarzbach, M., 2005. “Conception of an apparatus for ground based measurements of the bi-directional reflectance distribution function under natural conditions”. Diploma thesis, Universitaet Stuttgart, Institute of space systems. IRS-05-S48. [9] Schopfer, J., Dangel, S., and Kneubhler, M., 2007. “Dual field-of-view goniometer system figos”. Intl. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XXXVI, Part 7/C50, p. 6. ISPRS, Davos (CH). ISSN 1682-1777. [10] Alt, B., and Schwarzbach, M., 2004. “Auslegung eines Modellflugzeuges fuer die Fernerkundung”. Study thesis, Universitaet Stuttgart, Institute of Space Systems. IRS-04S35. [11] Brisset, P., 2006. “The Paparazzi solution”. In Micro Air Vehicle Conference 2006. See also URL www.recherche.enac.fr/paparazzi/ papers_2006/mav06_paparazzi.pdf. [12] Jonassen, M., and Reuder, J., 2008. “Determination of temperature and humidity profiles in the atmospheric boundary layer by fast ascending UAVs”. Geophysical Research Abstracts, Vol.10(EGU2008-A-00698). [13] Reuder, J. e. a., 2008. “SUMO: A small unmanned meteorological observer for atmospheric boundary layer research”. 2008 IOP Conf. Ser.: Earth Environ. Sci., Vol.1. [14] Nomura, Y., Zhang, L., and Nayar, S., 2007. “Scene Collages and Flexible Camera Arrays”. In Proceedings of Eurographics Symposium on Rendering. [15] Oppelt, N., and Mauser, W., 2007. “Airborne visible / infrared imaging spectrometer AVIS: Design, characterization and calibration”. Sensors 2007, Vol.7, pp. 1934–1953. [16] KVH Industries, I. Cns-5000, kvhs gps/imu continuous navigation system. Tech. rep. See also URL www.kvh.com/cns5000.
ADLER GROUND RESOLUTION EXAMPLE.
would vastly improve classification capabilities. Since hyperspectral cameras can only be built as line scanning devices, the acquisition of high precision and high frequency inertial data is neccessary for the alignment of the recorded lines in the postprocessing. Inertial measurement units (IMU) providing this accuracy have recently become small and lightweight enough to be used in a UAV the size of the Stuttgarter Adler [16]. Tests will start as soon as the manufacturing company provides first results and devices. Implementation of an accurate IMU would also allow the use of its data as input for the autopilot for better attitude estimation and stabilization.
ACKNOWLEDGMENT The authors gratefully acknowledge the generous assistance of the staff of the Institute of Space Systems, especially the mechanical workshop and the electronics laboratory. Thanks also go to all people committed to the developement of the Paparazzi system.
REFERENCES [1] CropCam. A new altitude in agriculture. See also URL www.cropcam.com/pdf/brochure-cropcam.pdf. [2] Chao, H., Baumann, M., Jensens, A., Chen, Y., Cao, Y., Ren, W., and McKee, M., 2008. “Band-reconfigurable multi-uav-based cooperative remote sensing for real-time water management and distributed irrigation control.”. In In Proceedings of the IFAC World Congress, Seoul, Korea. [3] Hatfield, J. L., Gitelson, A. A., Schepers, J. S., and Walthall, C. L., 2008. “Application of spectral remote sensing for agronomic decisions”. Agronomy Journal, 100, pp. 117–131. 7
c 2009 by ASME Copyright
