Evaluation of UAV based schemes for forest fire monitoring. V. C. Moulianitis1, 2, G. Thanellas2, N Xanthopoulos2, N. A. Aspragathos2 1Dept
of Product and Systems Design Eng, University of the Aegean, Syros, Greece Eng and Aeronautics Dept, University of Patras, Patras, Greece
[email protected],
[email protected],
[email protected],
[email protected] 2Mechanical
Abstract. This paper presents a mechatronic evaluation of forest fire monitoring systems based on UAV. To begin with, a mapping of the requirements to the mechatronic abilities, which should be embodied by these systems, is presented. The enabling technologies that support these abilities are briefly reported. The evaluation of these systems’ architectural schemes is accomplished with the discrete Choquet integral. As a result, UAV based schemes are found to be better than other proposed schemes for forest fire monitoring. Keywords: Forest fire monitoring, UAV, Mechatronic evaluation.
1
Introduction
The applications of aerial robots are extensive and can be categorized as [1]: (a) Inspection and Maintenance, with the largest portion of applications, (b) Logistics and Delivery, (c) Search and Rescue and (d) Environmental Monitoring. In this paper, the environmental monitoring and more specifically, the early detection of wildfires using aerial robotics is mainly considered, while other existing methods are also analyzed and compared. Forests provide a diversity of ecosystem services (converting carbon dioxide into oxygen, acting as a carbon sink, aiding in regulating climate etc.) and serve as a source of lumber and recreational areas. Conventional forest monitoring is mainly based on fire watchtowers manned by permanent or temporary staff during the summer period, aerial and ground patrols, as well as public reporting [2]. Towards the direction of improvement of the efficiency of current fire-fighting operations, EU has funded projects such as AF3 [3]. In the recent years, emerging technologies are starting to be involved in monitoring, with spearhead Unmanned Aerial Vehicles (UAV) equipped with multiple types of sensors. There has been already noticeable research in the field of forest fire monitoring systems [2, 4, 5]. Some of the existing fire detection and suppression techniques rely on watch towers, spotter planes, water tankers, optical smoke and lighting sensors, and weather forecasts [2]. In an effort of early fire detection, the autonomous systems that have been proposed are satellite surveillance, local optical cameras installations, wireless sensor networks in the monitored area [2, 5], and systems based on UAVs [4].
2
Computer vision methods combined with UAVs, which achieve monitoring (potential fire finding), detection (alarm triggering, which intend to inform firefighters), diagnosis (localization of fire and tracking its evolution) and prognosis (fire progress estimation according to weather conditions or firefighting system reaction), are presented in [4]. This paper introduces, among others, an extensive analysis of UAVs forest fire monitoring systems in order to specify the desirable level of abilities and their enabling technologies. Towards this direction, a matching between the abilities and the requirements is defined and evaluated. Schemes based on UAVs are found to be the most promising solution for early forest fire detection. The paper is organized as follows: the requirements and their mapping to the corresponding abilities are described in the second section. In section three, various existing forest fire monitoring schemes are assessed. The fourth section deals with the technologies that render those systems feasible, while the fifth section depicts concluding remarks.
2
Requirements and mechatronic abilities of the fire detection systems.
The major requirements of an autonomous early detection system of forest fires are summarized as: • • • • •
Robust continues monitoring of the forest area (CMO). Fast Detection of Fire (FDF) Determination of the Exact Location of Fire (ELF). Early Notification (ENO). Minimization of Faulty alarms (FA).
They indicate a system with mechatronic abilities, and therefore, as stated earlier, a mapping between requirements and the abilities should be developed. Then, this mapping will be used to evaluate concepts/schemes of proposed systems. A new index based on Multi-Annual Roadmap’s (MAR for Robotics in Europe [1]) collective knowledge for the conceptual design evaluation of mechatronic product and systems has been presented in [6]. According to MAR, core system abilities are classified as Configurability (CA), Adaptability (AD), Interaction Ability (IA), Dependability (DA), Motion Ability (MA), Manipulation Ability (MnA), Perception Ability (PA), Decisional Autonomy (Daut) and Cognitive Ability (CgA). Some of these abilities are considered minor and are excluded. For example, manipulation ability and adaptability are fundamentally unnecessary. On the other hand, interaction, perception and decisional autonomy (including cognitive capabilities) are very important, while configurability, dependability and motion ability are less significant. Configurability is a parameter which defines the ability of the system to be customized to different tasks. The monitoring has to be robust and continuous, under any condition. Interaction ability is quite significant since guarantees the secure, non-faulty, on-time communication between the components of the system. Data must be sufficient
3
to accomplish automated early notifications of fires (or fault alarms), in order to correctly mobilize, redirect or immobilize the fire brigades, while they are assisted by high coverage, real-time video streaming on regions of interest. Interaction ability between the components of the system can also improve the determination of fire’s exact location. Dependability specifies the level of trusting upon the system, therefore is very important for CMO and FA. Motion ability is required in systems with moderate number of components. Motion ability is important to prevent faulty alarms, to provide enough autonomy when range of communication is not enough, to navigate and hover in strong winds. Perception is vital, since one of monitoring’s most important goal is data acquisition. The fast detection of fires and their exact location determination are based on data acquired by sensors. Finally, decisional autonomy (including cognitive capabilities) is needed to identify whether visual data correspond to real physical fire or fake incidents. The mapping of the important abilities to the requirements are summarized in table 1. Table 1. Abilities of a potential forest fire monitoring system. Abilities CA
CMO X
FDF
IA
3
DA
X
MA
X
PA
X
DAut
X
ELF
ENO X
FA X
X
X
X X
X
X
X
X
X
X
X
X
X
X
X
Schemes of fire monitoring systems
Any component that will be used to build a forest fire monitoring system should have a specified level of the above abilities. The components that consist a fire forest monitoring system are networks of cameras, satellite-based systems, wireless sensor networks, systems based on UAV and combinations of them. There are various architectural combinations which compose a forest fire monitoring system as they are shown in Fig. 1. Each architectural scheme implementation needs a central ground station to collect data of monitoring equipment, enabling direct management and interaction with the forest surveillance network. Architecture 1 (A1) is composed by optical, thermal sensors or satellite-based systems [2, 5]. Satellite-based systems are expensive, however they are suitable for large areas of forests. The detection results depend highly on weather conditions and the satellite period orbit. Architecture 2 (A2) consists of UAV with embedded sensors which patrols independently or cooperatively the target area [4]. However, flight time is limited by the battery' s capacity, creating restrictions of continuous forest monitoring. Architecture 3 (A3) is a hybrid scheme including stationary cameras which continuously monitor an area and UAV which are deployed in case of alarm. In addition, this
4
architecture should include communication technologies like LTE to inform firefighters on time for the validity of the alarm. In case of a fully autonomous forest monitoring system, UAV confirms the alarm, and in case of emergency triggers an alarm. Architecture 3 can be cost effective, if its components placed properly accordingly to forest area surveillance needs, enabling collaborative surveillance with ground cameras and aerial cameras.
Fig. 1. Architectures of fire monitoring schemes.
In Table 2, an evaluation is presented of the necessary abilities’ levels that these architectures should embody. Concerning the practicality and efficiency, the topography as well as the size of the forest is taken into account. Some systems are very sensitive to weather conditions (such as the satellite-based systems). Taking into account, the above levels, we can derive ranges for efficiency and practicality (EP), as well as Budget (B) indexes. Table 2. Abilities of the three architectural schemes CA
IA
DA
MA
PA
DAut
Efficiency & practicality (EP)
Budget (B)
A1
L1
L1
L1
L1
L2
A2
L2
L4
L1
L5
L4
L1
Low
Very High
L3
High
High
A3
L2
L4
L1
L5
L4
L3
High
Medium
Table 3. Scores of the three architectural schemes CA
IA
DA
MA
PA
DAut
EP
Cost
A1
0.25
0.2
0.17
0.2
0.29
0.1
0.3
0.3
Score 2.152
A2
0.5
0.8
0.17
1
0.57
0.3
1
0.6
5.223
A3
0.5
0.8
0.17
1
0.57
0.3
1
1
5.803
An evaluation of the cost is also included and the score of each criterion is shown in Table 3. The determination of the scores of the criteria and the evaluation of the final score of the architectures using the discrete Choquet integral is based on [6]. The
5
abilities are formulated as criteria and their importance of each criterion is specified by two values (0.2 and 0.1) corresponding in high and low importance, respectively (𝜇(𝐶𝐴) = 𝜇(𝐷𝐴) = 𝜇(𝑀𝐴) = 0.1, 𝜇(𝐼𝐴) = 𝜇(𝑃𝐴) = 𝜇(𝐷𝐴𝑢𝑡) = 𝜇(𝐸𝑃) = 𝜇(𝐶) = 0.2). The 2-additive fuzzy measures are used to deal with the Choquet integral complexity risen from the considered interactive criteria. The importance of the interactions of the pairs are the following and are based on the interactions presented in [1]: 𝜇({𝐶𝐴, 𝐶}) = 𝜇({𝐼𝐴, 𝐶}) = 𝜇({𝐷𝐴, 𝐶}) = 𝜇({𝑀𝐴, 𝐶}) = 𝜇({𝑃𝐴, 𝐶}) = 𝜇({𝐷𝐴𝑢𝑡, 𝐶}) = 𝜇({𝐸𝑃, 𝐶}) = 0.5, 𝜇({𝐼𝐴, 𝑃𝐴}) = 𝜇({𝐼𝐴, 𝐷𝐴𝑢𝑡}) = 𝜇({𝐷𝐴, 𝑃𝐴}) = 𝜇({𝐷𝐴, 𝐷𝐴𝑢𝑡}) = 𝜇({𝑃𝐴, 𝐷𝐴𝑢𝑡}) = 0.3. The rest of pairs are equal to the sum of the individual fuzzy measures. The Choquet integral makes the evaluation process more objective, since it considers the interactions between the criteria as well as the collective knowledge of MAR for Robotics in Europe, despite the fact that some values are based on the experience of the design team. The final score of the evaluation based on the Choquet integral is shown in the last column of Table 3. Using the proposed evaluation, the architectural schemes based on UAV are found to be the most promising solutions concerning the forest fire monitoring.
4
Enabling technologies for UAV based forest fire monitoring
In this section the UAV’s enabling technologies are presented briefly in order to justify the selection of the appropriate level of the ability that it is supported by the corresponding enabling technology. 4.1
Types of UAVs and their components
Currently there are three major categories of UAVs [7]. Rotary wing drones that feature maneuverability, easy control and hovering flight skills with limited flight time. In contrast, fixed wing drones are ideal for long-distance missions, designed for higher altitude – more “stable-flight” missions. Flapping-wing drones are a new design of UAV, which intend to imitate flying pattern of birds or insects. Forest fire monitoring based on UAV requires maneuverability to avoid obstacles, low altitude flight and hovering enough autonomy for take-off, navigation and docking, and enough payload capability for carrying the appropriate sensors. Open source/hardware projects, microelectronics, microcontrollers, Lithium polymer (LiPo) batteries and brushless motors have made feasible the development of lowcost UAVs meeting the criteria for monitoring applications. The diversity of their inner architecture -for example communication system, number of motors, sensors integrated, degrees of freedom, capacity of battery, on-board processors, ability to coordinate with other UAVs- offers a variety of customizable capabilities. A rotary wing UAV is composed of at least one microcontroller, a battery, electronic speed controllers (ESCs), motors, inertial measurement unit (IMU), communication/telemetry and other integrated sensors. The firmware is stored in the microcontroller
6
which works as its “mind”, taking input from sensors, processing data and controlling the motors through ESCs. LiPo batteries provide higher storage of energy and less weight than other lithium batteries. Brushless motors have long lifespan, low maintenance and high efficiency. Six DoF (Degrees of Freedom) IMUs provide 3 axis gyroscope and accelerometer, while nine DoF have additionally a 3-axis magnetometer for more stable compass/yaw estimation. UAVs that are controlled directly through human input are using radio frequencies (RF) mainly at 2.4Ghz, with a range of some kilometers. Telemetry systems work at 5.6Ghz and provide visual feedback with some other type of system information (at even shorter distances), such as battery life. Satellite connection can be embedded in order to provide a more distant connection and higher data bandwidth; many open-source, hobby and enterprise UAV projects have 4G circuits on board. Ultrasonic sensors are attached on top, front and bottom of UAVs’ frame to detect short distance (~40cm-5m) obstacles. Finally, the payload capacity of a rotary wing UAV is enough to carry additional sensors and modules like thermal camera. UAVs present a start-up level of configurability (Level 2). They can be configured prior to each mission in order to be suitably customized for the monitoring tasks. The power autonomy and measurements offer a low level of dependability since it can be known when the batteries will be drained. Fault detection can be compensated by a fleet of UAVs. 4.2
Fire detection equipment and methods
Vision systems, which include conventional and/or thermal imaging and video recording are used to perceive a fire in its early stage. Several publications suggested methods for fire detection based on flame and smoke recognition, using color, motion and geometry features in offline videos [4, 5, 8]. A method for forest fire detection underlying the importance of multi-sensor camera (thermal and vision sensor) for localization of fire in image processing was proposed [9]. In addition, the idea of Forest Fire Detection Index, which relates to Vegetarian Index was introduced. This method examines an image pixel by pixel and can be used in real time applications. Thus, success of fire detection relies on vision systems and the reliability of the algorithm. These technologies can embody level 2 perception ability concerning the detection of the fire. 4.3
Equipment and methods for motion planning, navigation, SLAM, and control of UAVs
The literature for motion planning, navigation, SLAM and control UAV is quite rich. In cases of unknown environment, it can be mapped beforehand or by using online techniques (SLAM). In [10], a combination of sensors for fire detection and obstacle avoidance was proposed using a non-contact IR heat sensor, a gas sensor, an ultrasonic distance sensor, a GPS and a compass provided by an IMU. The importance of motion planning algorithms can be derived from [11], where a well-organized idea for aerial surveillance based on a fixed-wing UAV was presented. This algorithm maximizes the covered area and provides obstacle avoidance for fixed wing UAVs. However, this
7
approach is semi-automatic; the interaction between a human operator and the drone for docking purposes is necessary and sometimes dangerous. A reactive path planning algorithm, using wave front algorithm as the local path planning algorithm for static obstacles was proposed [12]. Sometimes GPS denied environments like forests, the effective SLAM can be achieved by fusing IMU’s data, visual odometry and GPS measurements [13].Under the term of an effective UAV automation four groups of functionalities are necessary: sensing (camera, inertial sensor etc.), navigation (perception and estimation), guidance (path planning, mission planning and exploration) and control (attitude, position, velocity and acceleration) [14]. Robust control is necessary for bad weather conditions that act as disturbances to the system [15]. These technologies can embody perception ability of level 4 concerning the needed features for simultaneously localization and mapping. These technologies also support the motion ability of the UAV. 4.4
Communication model networks
Nowadays communication plays a fundamental role in many applications, including UAVs. LTE-networks are a promising communication model for unmanned aircraft systems, achieving transmission of telemetry and sensor data even in environments of limited transmittance [16]. This technology enables the annihilation of distances in data transmission; if the forest area is covered by the network, data can be transmitted with minimal delay everywhere in the world. The bandwidth provided by this generation of cellular networks is extremely satisfying for video streaming and navigation. However, LTE-networks could be combined with new wireless telecommunication system such as LoRa [17], which is the prevailing technology in IoT networks, achieving maximum coverage and reliability. Integration with robotic systems is already well established, small size circuits dedicated on connecting to these networks can easily be attached and function on all kinds of UAVs. Interaction ability of level 4 can be achieved using these technologies.
5
Conclusions
This paper presents a mechatronic evaluation of forest fire monitoring systems by mapping the requirements of such a system to mechatronic abilities. The minimum levels of these abilities are determined by enabling technologies which are also briefly reported. The evaluation was accomplished using the discrete Choquet integral taking into account interactive criteria. Schemes based on UAV technology are found to be the best solution. It is evident that the enabling technologies have the appropriate matureness to support the required abilities for the development of forest fire monitoring systems based on UAVs. Based on these results, a forest fire monitoring system using UAVs and static cameras is under implementation for the project SFEDA funded by Interreg BalkanMediterranean.
8
Acknowledgement This research is a part of work of the project funded by the Interreg Balkan-Mediterranean program:“ SFEDA-Forest Monitoring System for Early Fire Detection and Assessment in the Balkan-Med Area (MIS: 5013503)”.
References 1. euRobotics aisbl. Robotics 2020 multi-annual roadmap for robotics in Europe 2015. https://www.eu-robotics.net/eurobotics/topic-groups-/index.html. 2. Ahmad, A., Alkhatib A.: A Review on Forest Fire Detection Techniques. International Journal of Distributed Sensor Networks, (2014). 3. Advanced Forest Firefighting homepage, http://af3project.eu/, last accessed 2018/03/10. 4. Yuan, C., Zhang, Y., Liu, Z.: A survey on technologies for automatic forest fire monitoring, detection, and fighting using unmanned aerial vehicles and remote sensing techniques. Canadian Journal of Forest Research, 45(7), 783-792, (2015) 5. Waqas, A.A.I.: A Survey on WSN-based Forest Fire Detection Techniques. International Journal of Recent Trends in Engineering & Research, 1(1), 60-65, (2016). 6. Moulianitis, V.C., Zachiotis, G.-A.D., Aspragathos, N.A.: A new index based on mechatronics abilities for the conceptual design evaluation, Mechatronics, 49, 67-76, (2018). 7. Gachoki, N.M., Muhia, A.M., Kiio, M.N.: A Review of Quad-rotor UAVs and their Motion Planning. In Proc. of Sustainable Research and Innovation Conference, 117-121, (2017) 8. Bosch, I., Serrano, A., Vergara, L.: Multisensor network system for wildfire detection using infrared image processing. The Scientific World Journal, (2013). 9. Cruz, H., Eckert, M., Meneses, J., Martínez, J. F.: Efficient forest fire detection index for application in unmanned aerial systems (UASs). Sensors, 16(6), 893, (2016). 10. Abdullah, M. F., Wijayanto, I., Rusdinar, A.: Position estimation and fire detection based on digital video color space for autonomous quadcopter using odroid XU4. In Control, Electronics, Renewable Energy and Communications (ICCEREC), 2016 Int Conf. on, 169-173, (2016). 11. Darbari, V., Gupta, S., Verma, O.P.: Dynamic motion planning for aerial surveillance on a fixed-wing UAV. 2017 International Conference on Unmanned Aircraft Systems (ICUAS), Miami, FL, USA, 488-497, (2017). 12. Hebecker, T., Buchholz, R., Ortmeier, F.: Model-based local path planning for UAVs. Journal of Intelligent & Robotic Systems, 78(1), 127-142, (2015) 13. Balamurugan, G., Valarmathi, J., Naidu, V.P.S.: Survey on UAV navigation in GPS denied environments. In Signal Processing, Communication, Power and Embedded System (SCOPES), 2016 Int. Conf. on, 198-204, (2016). 14. Kanellakis, C., Nikolakopoulos, G.: Survey on Computer Vision for UAVs: Current Developments and Trends. Journal of Intelligent & Robotic Systems, 1-28, (2017). 15. Zulu, A., Samuel, J.: A review of control algorithms for autonomous quadrotors. Open Journal of Applied Sciences, 4, 547-556, (2014). 16. Maiwald, F., Schulte, A.: Using LTE networks for UAS-communication. In 36th European Telemetry and Test Conference, 3958-3963, (2016). 17. Augustin, A.; Yi, J.; Clausen, T.; Townsley, W.M. A Study of LoRa: Long Range & Low Power Networks for the Internet of Things. Sensors, 16, 1466 (2016).
