A platform for the design, simulation and development of quadcopter

A Platform for the Design, Simulation and Development of Quadcopter Multi-Agent Systems Rúben Veloso, Zafeiris Kokkinogenis, Lúcio S. Passos, Gustavo Oliveira, Rosaldo J. F. Rossetti Artificial Intelligence and Computer Science Lab. (LIACC) Department of Informatics Engineering (DEI) Faculty of Engineering, University of Porto, Portugal {ei11001, pro08017, pro09026, ei08125, rossetti,}@fe.up.pt

Abstract— Unmanned Aerial Vehicles (UAVs) are gaining the attention of researchers around the world. Recently researchers apply the multi-agent paradigm to such domain taking advantage of its research in cognition, deliberation, and mainly coordination mechanisms. The aim of our research is to contribute to the multiple quadcopter simulation area introducing a platform for the design, simulation, and development of quadcopter MultiAgent Systems (MAS). Specifically for this paper, we minutely discuss the proposed architecture and present the implemented prototype focusing our experiments on the scalability and the resource usage of it. The results demonstrate that the centralised perspective limits the scalability mainly when the physical engine is being used; also, we identify that the method responsible for the 3D update is the platform's bottleneck. After this work different type of studies in social interaction of quadcopters will follow up as well as the improvement of the proposed architecture, for instance, using a multi-threading approach. Keywords- Multi-Agent Unmanned Aerial Vehicles.

I.

Systems;

Symbiotic

Simulation,

INTRODUCTION

The simulation is essential while developing complex systems because avoids failures in real-world tests, especially to Unmanned Aerial Vehicles (UAVs); in these applications, the failure usually means permanent damage to the equipment. The UAV simulation provides a vast range of possible analysis, for instance, the physical aspect (stability and equipment design) as well as the cognition capabilities (flight control based on learning). On the top of this, from the several aspects of UAVs field, much research has been done on the introduction of autonomy and cooperation features within the vehicle. A recent approach to encompass these features is to apply the multi-agent paradigm and inherent its research in cognition, deliberation, and cooperation mechanisms. Our research focuses on coordinated usage of autonomous quadcopters in virtual environments. This type of aerial vehicle is able to perform (i.e. it changes direction and altitude faster than aeroplanes), as well as it easily stabilizes compared to other types of UAVs. On the other hand, quadcopters usually struggles with their flying time as designers have to optimize flying weight versus battery life. From the simulation perspective, all the aforementioned aspects must be present in the virtual domain so quadcopters’ planning, learning, and

Joaquim Gabriel IDMEC – FEUP Campus Faculty of Engineering, University of Porto, Portugal [email protected]

other features might be put under test accounting for realistic conditions. A careful analysis in current aerial simulators reveals some of the following issues. Those that simulate a single quadcopter usually exclude the autonomy and cognition processes. Furthermore, the majority of simulators do not instantiate multiple quadcopters, which exclude their use to coordination and collaboration scenarios. Besides, those that simulate multiple UAVs constraint the number of simultaneous entities in the virtual environment and their interaction with the real world; therefore, there is a lack in frameworks able to model and simulate multiple 3-Dimensional (3D) autonomous quadcopters with the ability to interaction and possibly coordinate strategies. By proposing a framework that overcomes this issue, it opens the possibility a new perspective of implementing UAVs for real applications where the designer can develop the quadcopters and test them in virtual environments without prototyping the real aerial vehicle. This envisioned scenario poses advantages such as the reduction of production costs inherited from equipment damage and, for the case of multiples entities, spending proportional to the number of UAVs. In this paper we intend to contribute to the multiple quadcopter simulation area introducing a platform for the design, simulation, and development of quadcopter MultiAgent Systems (MAS). We study the requirements of our quadcopter context from the simulation perspective and thus present the developed prototype. Such a framework would allow the scientific community to validate planning and cooperation algorithms in a more realistic domain and practitioners to have means to easily design different configurations of structure, motors, and so forth aiming to observe the quadcopters’ response to given commands. We assess the scalability and the resource usage of the developed platform and identify some issues regarding the centralized architecture. The remainder of the paper is organized as follows. In Section 2 we discuss the related work centring in the simulation tools that share some of our desired features. Section 3 describes the conceptual architecture while Section 4 discusses the main technologies and component as well as

overall functioning; furthermore, in Section 5 we present the methodology of the performed tests and discuss their results. Section 6, finally, reviews the accomplishments and identified issues and points out future steps to our research. II.

RELATED WORKS

Our work is related to the simulation of autonomous quadcopter and thus we study methods and tools that are within this area. We consider the paradigm of symbiotic simulation as its approaches combine simulation of virtual and physical systems: the so-called mixed reality, co-simulation, and on-line simulation. This choice can be justified by Aydt et al. [1] definition of symbiotic simulation classes. There are several tools that simulate UAVs including quadcopters (see [6] for a deeper study) and, as this work does not intend to exhaustively survey this area as we focus our discussion on the following ones. Flight Simulator [9] is a simulation tool used for certification of complex flight systems; however, this tool does not instantiate customized model of quadcopters nor the needed indoor environments. A commercial tool, Microsoft Flight Simulator [6], is one of the most well-known around the globe. It is based on flight models constituted of vehicles and generic environments; it also integrates modelling of hardware failure in a generic and imprecise way. This is an interesting platform for outdoor applications. At last, the X-Plane 1 uses an UAV model of a geometric form, which it divides it in sub-parts and then calculates the forces at each sub-part. It is a professional simulator and the Federal Aviation Administration (FAA) for flight training and testing approves it. Simulators of the robotic area are also being used to simulate aerial vehicles. USARSim [5] is an open-source simulator that is appropriate when simulating competitive robots and the application needs a high fidelity with regard to sensors uncertainties. Gazebo [12] is a simulator designed to model 3D environments with multiple robots; it is open-source and simulates remote environments. Studied platforms that focus on robotic simulation usually concern with realism of sensors and actuators adding uncertainty, however they do not pre-process this information to deliver for a higher level reasoning process. Some works benefit from the information exchange between simulated and real systems, for instance, Mancini et. al. [13] proposed a modular and layered architecture allowing such interaction. Likewise, IntelWheels [4] is a wheelchair with a high level of cognition, developed using MAS and simulation; it intends that the virtual chair mimics the behaviour of the real one to test used control algorithms. Specifically to UAVs context, Kamrani and Ayani [10] discuss a symbiotic simulation approach in UAV path planning. Authors assessed the benefit of their framework comparing the performance to other off-line simulation approaches. The existence of a feedback loop in the symbiotic simulation case provides UAV with the necessary adaptively to perform better. AgentFLY [14] integrates real UAVs into a multi-agent simulator already populated with virtual UAVs achieving a mixed reality system where both entities coexist. Similarly to 1

http://www.x-plane.com/desktop/home/

this project, the planning and reasoning are computed on-board and the coordination and cooperation at the virtual. Also in a recent endeavour, Michal Jakob et al. [8] introduced a platform for incremental development using test beds, which were instantiated with a multi-UAV system tracking a moving target; strategies are tested to improve advanced human-agentrobot teamwork. Finally, Simbeeotic [11] project developed a framework to simulate environments with thousands of agents while some virtual entities represent avatar for physical helicopters flying in a controlled room. As regards the implementation of a virtual agent, Hama and Bordini [7] also intended to benefit from an agent-oriented language when developing UAV behaviours; although this work resembles our architecture when integrating agent-based simulation tool embedding an agent into physical device, they do not implemented any feature of the symbiotic simulation. Therefore, the literature review corroborates the requirements we are addressing with the proposed architecture, such as the use of symbiotic simulation to test UAVs and endow a quadcopter with higher-level cognitive capabilities using an agent-oriented approach. However, to the best of our knowledge, no platform was able to merge all these features under the same tool and thus we believe that this paper is a first step to bridge this gap. III.

THE PROPOSED ARCHITECTURE

Features of a platform capable of simulating multiple instances of autonomous quadcopters (or generally UAVs) are discussed below as it is depicted in Figure 1. The conceptual architecture of the quadcopter simulator consists of three major modules:

Figure 1. Conceptual Architecture

Quadcopter Simulator module implements a graphical and analytical model of the quadcopter along with the virtual model of the desired environment. The UAV model reflects both the dynamic and kinematic behaviours of a real one. The simulator allows for instantiation of multiple UAV models all of them sharing the same vision of the world. Moreover it performs the simulation of all UAVs in the environment, along with all their sensors and actuators, and reflects every surrounding object. It also features a game engine for immersive 3D animation through both powerful physics and visualization modules; this latter allows implementing the operation of the UAV in realistic condition where different dynamic control schemes can be simulated during tasks execution.

Deliberation and autonomous decision-making module: The architecture must divide the basic functions of UAV’s between low- and high-level controls. As we said in the previous paragraph the low-level operation control is implemented within the UAV simulator module. The highlevel control intends to implement the cognitive and deliberative functions of the UAV following the agent metaphor. It is the module where action planning and deliberation take place. Thus the module (and consequently the platform) should support the implementation of agents and thus is vital to combine a reasoning engine, e.g., by using the Belief-Desire-Intention (BDI) architecture. In this way the agent-UAV has a higher level of decision-making and a set of plans allowing different approaches to UAVs' individual and collective decisions. Symbiotic Simulation module: this module builds a close association between a virtual (simulated) UAV model and a physical UAV system. The mutual benefits that can emerge from this synergy vary according to the what-if scenarios analysis we need to perform. In such a relationship, the simulation system benefits from data of sensors while the physical system may be improved by conclusions drawn based on the simulation results. Thus, the virtual UAV might receive perceptions from the real one and send back a set of commands to the real one relying on the information of the simulated environment. To accomplish this envisioned scenario a communication layer between the simulation and the real system is essential. We can acknowledge the bidirectional communication between each module, whereas the UAV simulator provides its kinematic variables perceiving the world and the agent-UAV estimates both its internal state respect to world state and return back its beliefs and commands to the UAV. All of these transactions should occur in the same time step. IV.

PROTOTYPE DEVELOPMENT

This proposal intends to reveal the multifaceted nature of the simulation platform. It intends to explore different operational aspects such as design and evaluation of

coordination actions and coalition formation, testing of agent implementation, decision-support for real UAVs system applications among others. The architecture implements the conceptual description of the previous paragraph and it presents its layered nature, as each main component demonstrate different types of functioning of this proposal. Figure 2 minutely depicts the proposed software simulation architecture based on the symbiotic simulation paradigm. To put together the proposed approach a set of technologies have been chosen that are presented in the following subsection. Having defined the main modules of the architecture in Section III, we continue the discussion by presenting the software (and hardware) platforms that compose the simulator. Virtual environment: The virtual 3D environment builds the “world” and also implements the UAV’s physical model. The 3D environment uses the jME3, an open-source Javabased software development kit, and is divided in three essential parts: the scenario, the physical engine, and the manual control (for testing purposes). The first supports a 3D representation of the simulated environment aiming to supply perceptions for both virtual and real agents; the jME3 also provides a reliable physical engine essential to test low-level control and stability. Furthermore, the virtual UAV splits in the graphical model where one can define the generic model of the quadcopter or import a 3D model of the real quadcopter, and the analytical model, where all parameters for the quadcopter’s parts are defined according to their datasheets. The jME3 engine is integrated with CArtAgO (Common ARTifact infrastructure for AGents Open environments) framework [15]. CArtAgO is a general-purpose infrastructure that makes possible to program and execute computational environments for multi-agent systems by implementing the agents and artefact meta-model. According to this approach artefacts are resources and tools in the environment that agents can use and manipulate to support individual and collective activities or decisions. The platform also has to implement virtual sensors that gather information of the environment so the Quadcopter agent may take the best action. This

Figure 2. Architectural Design

interface is based on XBee2 module that establishes a radio link to send virtual sensorial data and receives sensor information or commands from the real quadcopter. The communication protocol is based on the UAVTalk implementation3 that it has been adapted to support FIPA ACL messages.

architecture mimics the real functioning of a quadcopter where a set of sensors acquire all information about the environment. With this perspective, we implement general interface (using UAVTalk) that modularises the addition of different and possibly real sensors. Furthermore the simulator implements two interfaces to the other components of architecture: An internal interface between jME3 and a MAS development platform permits agents to be instantiated in the agent-oriented engine that will be discussed later in this paper. The agents represent the cognitive layer of the UAV in the jME3. Through a GUI, the user can set some parameters for each agent model. At the same time, there is a 3D representation of jME3 updates the internal state of each agent with perceptions while the agent-oriented engine sends back agent's actions through internal actions. The internal actions are a set of methods to map high-level actions of agents into lowlevel commands in Java language. An external interface allows the communication between the simulated entities and a physical system. Multi-agent system for UAVs: This module focuses in the implementation of software agents. This layer is responsible for the multiples instances of Quadcopter agents and multi-agent communication. The agents will implement the reasoning layer (module) of the UAV following a deliberative BDI architecture. For this purpose it is used the Jason platform for multi-agent system development [3]. Jason it is an open source java-based interpreter of the AgentSpeak language [16]. Jason/AgentSpeak joins declarative logical programming language with BDI architecture; it has high-abstractive levels of expressiveness when describing autonomy and intelligent behaviours and supports integration with others agent-oriented platforms and tools. An agent, in this context, is defined by a set of beliefs, goals, and its library of plans that adopts BDI model. The multi-agent communication is supported by an inter-agent communication protocol based on speech acts included in AgentSpeak. Agents in Jason act upon a virtual environment through high-level commands and the environment maps them to low-level methods; therefore, the solution is to create (what Jason names) internal actions, which connect agents to the environment. These internal actions are developed from the agent's viewpoint, as the internal engine of Jason will perform them where the AgentSpeak semantics is interpreted.

To describe the proposed framework a prototype was developed using jME3 and Jason. The methodological approach is many-folds: 1) Extend the jME3 platform: jME3 is not prepared to accommodate UAVs. The integration of the platform with the necessary set of virtual sensors and 3D models of the quadcopter it is necessary to be implemented. The platform has been extended with the classes of the CArtAgo infrastructure as well as; 2) Integrate the jME3 with the Jason framework: apply the required modifications to integrate both platforms; 3) Implement an agent-based quadcopter: to successfully validate the proposed architecture and prototype in its essence, a simple BDI agent was devised with a simple plan library. The control algorithm moves the quadcopter through a trajectory, possibly deviating from random objects; 4) Implement an interface and a protocol for communication with real-world devices. V.

EXPERIMENTAL RESULTS AND DISCUSSION

To properly validate the implementation of the general architecture proposed in this project, we perform some tests to address aspects related with the performance of the platform. The first is a scalability test to establish the number of agents that can be simultaneously instantiated in the platform; and, moreover, the memory use and processor consumption per process that highlight the issues to be addressed in future versions. This scalability evaluation test was performed on two different computers available in our department denominated PC 1 and PC 2. Table I illustrates the computers specifications, which should be considered in the critical assessment of the result values. The experiment aims to check the level of processing (including graphical) through the frames per second (FPS) that the computer is able to render. Several quadcopters instances TABLE I.

Quadcopters (Physical): this module allows the connection between a real and a simulated quadcopter. The association is performed considering an avatar in jME3 that models the real entity in the virtual environment. The goal of this extension of the simulation platform is to provide a decision support tool both in phase of design and testing but also in the phase of deployment following the symbiotic simulation paradigm as it is described in [1]. To achieve this synergy between simulated and physical device a communication interface and protocol have been implemented. The 2 3

TWO COMPUTER SET-UPS USED IN THE EXPERIMENTS

PC Features

PC 1

PC 2

CPU

Dual CPU T2410

i5-2400

Clock CPU

2.00GHz

3.10GHz

Memory

3.00GB

8.00GB

Graphical Device

NVIDIA GeForce 9300M GS

AMD Radeon HD 6570

OS

Windows 8.1 Pro

Windows 7 Pro

http://www.digi.com/xbee/ http://wiki.openpilot.org/display/Doc/UAVTalk

were introduced to the simulated scenario; they were not executing missions or interacting among themselves. Moreover, these tests were executed with and without the physical engine. With the physical engine, the agents are able to detect collisions, carry objects, and be influenced by external objects. By turning off the physical engine, we intended to mimic a usage of the platform only to design and evaluate cognitive/deliberative strategies, which does not require the simulation of the quadcopter forces. The steps to be taken into account in this scalability test are: 1) 2) 3) 4)

Start the platform in PC 1 and then PC 2; Start a quadcopter in a static virtual environment; Wait until the platform stabilizes during a few seconds; Record both instantaneous FPS and number of agents.

Figure 5 depicts the achieved results from the tests with the physical engine turned on. For both computers, the scalability curve presents an exponential tendency and the platform stops to properly work after 10 instantiated quadcopters. Two facts contribute to this result: first, this more realistic mode demands for more processing as the forces of all physical models need to be calculated in a time step; second, the physical engine centrally calculates all the aforementioned forces and does not take advantage of the multiple cores of the processor. Figure 3 plots the results acquired when the physical engine is turned off that could be used to test coordination and cooperation of UAVs. Differently from last results, these have a logarithm tendency. As expected, the number of agents is considerably higher and it improves to be around 100 agents with increasing the memory threshold to the Java Virtual Machine. From the practical results of the scalability experiment, we can infer that the processing capability of the physical engine is the bottleneck of the platform. Nevertheless, there are some solutions to these problems as the use of distributed processing for the case of multiple instances of quadcopters. The computational resources test aims to analyse the rate of memory usage since the start of the platform until the instantiation of five quadcopters; furthermore, we are able to identify the most demanding methods of the proposed platform. Figure 4 presents the memory usage throughout a complete run of the experiment. During the platform initialization, there is a substantial consumption of memory due to the loading of

Figure 5. FPS vs. Number of Agents with the physical engine

Figure 3. FPS vs. Number of Agents without the physical engine

all the components from the interface to the virtual environment. After that, the system releases disposable resources as can be seen when the usage went down to 150MB. Following, five quadcopters were added at short intervals of time for stabilization of each quadcopter, which are represented by stages from A1 to A5. Each quadcopter instance allocates on average 9.5 MB of memory including the graphical and analytical model for same agent. Again near the end of the graph, the memory usage drops because no quadcopter in the screen and it does not handle the rendering the graphical model to optimise system resources. The last set of experiments intends to verify the methods in the platform that use more resources regarding processing time and data flow; this aims the systems hotspots. The critical method is the simpleUpdate(), which updates the internal state of the quadcopter relying on the 3D model information. This was an expected result because as aforementioned the centralise architecture and it is the cause main cause of the bottleneck effect observed in the scalability test.

VI.

CONCLUSION AND FUTURE WORKS

This paper has presented the conceptualization and development of a platform to design, implement, and simulate, multiple instances of autonomous agent-based quadcopters. In this context autonomy is intended not only as making decisions but mainly as cognitive process. The envisaged architecture is a hybrid approach; low level decisions (actions) are executed in a reactive fashion, while processes such as can be the one of planning follow a deliberative reasoning. The capacities of the platform are extending with the concept of symbiotic simulation. Therefore we have create a synergy between virtual models of agent-based quadcopters and real agent-based devices, where one can explore various what-if scenarios for the definition of the application aspect one intends to explore.

Figure 4. Memory Usage During the Experiment

The purpose of this paper is to propose a simulation architecture that highlights important aspects towards an efficient development of agent-based quadcopters that miss from the current available simulation tools. Furthermore, the experiments revealed some issues regarding the scalability of the proposed platform, especially when the physical engine was being used. The number of virtual quadcopters that the platform supports (at the minimum) doubled when testing it without the physical engine. As this first version relies on a centralised processing of the physicalrelated calculus, we identify the bottleneck of the platform being the simpleUpdate() method that updates the internal state of the quadcopter relying on the 3D model information. The memory usage is optimised when there is no 3D model rendering and on average each instance of the quadcopter allocates 9.5 MB including graphical and analytical models. Both scalability and resource usage help us to assess the implemented tool and to pinpoint the weaknesses of our architecture taking into account its main purpose. After the analysis described in this paper, future works will move towards two different directions. First, we will investigate the improvements that can be done in the platform to support a high number of agents so researchers might use it to apply for instance swarm intelligence algorithms. Moreover, we intend to increase realism of the analytical model including all quadcopter components such as motors, sensors, structure, and so forth; this opens a new line of research where already developed models should be imported to our proposal and a cross-validation of the virtual UAV might be possible. The second direction of future research is exploring the multi-agent paradigm. Several works in the multi-agent field propose mechanisms to coordination, collaboration, and even competition among UAVs; in our perspective it is interesting to experiment those technique in a simulated scenario where UAVs may encounter unexpected events and, in its current development state, our platform is able to provide such scenario.

[2]

[3]

[4]

[5]

[6]

[7] [8]

[9]

[10]

[11]

[12]

[13]

[14]

ACKNOWLEDGMENT This project has been partially supported by FCT (Fundação para a Ciência e a Tecnologia), the Portuguese Agency for R&D (PhD Scholarship grants SFRH/BD/66717/2009 and SFRH/BD/67202/2009). REFERENCES [1]

H. Aydt, S. J. Turner, W. Cai, and M. Y.-H. Low. “Symbiotic simulation systems: An extended definition motivated by symbiosis in biology,” 22nd Workshop on Principles of Advanced and Distributed Simulation, PADS'08, pp. 109-116. IEEE, 2008.

[15] [16]

[17]

H. Aydt, S. J. Turner, W. Cai, and M. Y. H. Low. “Research issues in symbiotic simulation,” Proceedings of the 2009 Winter Simulation Conference (WSC), pp. 1213-1222, Dec. 2009. R. H. Bordini, J. F. Hübner, and M. Wooldridge. “Programming MultiAgent Systems in AgentSpeak using Jason (Wiley Series in Agent Technology),” John Wiley & Sons, 2007. R. A. Braga, M. Petry, A. P. Moreira, and L. P. Reis. “A development platform for intelligent wheelchairs for disabled people,” International Conference on Informatics in Control, Automation and Robotics, pp. 115-121, 2008. S. Carpin, M. Lewis, J. Wang, S. Balakirsky, and C. Scrapper. “Usarsim: a robot simulator for research and education,” IEEE International Conference on Robotics and Automation, 2007, pp. 1400-1405. IEEE, 2007. R. Gimenes, D. C. Silva, L. P. Reis, and E. Oliveira. “Flight simulation environments applied to agent-based autonomous uavs,” ICEIS, vol. 4, pp. 243-246, 2008. M. T. Hama, R. H. Bordini, and R. S. Allgayer. “Agentspeak for uav development: Towards a technology bridge,” M. T. Hama, R. S. Allgayer, C. E. Pereira, and R. H. Bordini. “Uavas: Agentspeak agents for unmanned aerial vehicles,” In II Workshop on Autonomous Software Systems, vol. 7, 2011 M. Jakob, M. Pechoucek, M. Cap, O. Vanek, and P. Novak. “Towards incremental development of human-agent-robot applications using mixed-reality testbeds,” 2012. E. N. Johnson and S. Fontaine. “Use of flight simulation to complement flight testing of low-cost uavs,” Conference and Exhibit AIAA Modeling and Simulation Technologies, pp. 6-9, 2001. F. Kamrani and R. Ayani. “Using on-line simulation for adaptive path planning of uavs,” Proceedings of the 11th IEEE International Symposium on Distributed Simulation and Real-Time Applications, DSRT '07, pp. 167-174, Washington, DC, USA, IEEE Computer Society, 2007. B. Kate, J. Waterman, K. Dantu, and M. Welsh. “Simbeeotic: a simulator and testbed for micro-aerial vehicle swarm experiments,” Proceedings of the 11th international conference on Information Processing in Sensor Networks, IPSN '12, pp. 49-60, New York, NY, USA, ACM, 2012. N. Koenig and A. Howard. “Design and use paradigms for gazebo, an open-source multi-robot simulator,” Proceedings 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004. (IROS 2004). Vol. 3, pp. 2149-2154, 2004. A. Mancini, A. Cesetti, A. Iuale, E. Frontoni, P. Zingaretti, and S. Longhi. “A framework for simulation and testing of uavs in cooperative Scenarios,” Unmanned Aircraft Systems, pp. 307-329, Springer, 2009. M. Selecky and T. Meiser. “Integration of autonomous uavs into multiagent simulation,” Acta Polytechnica, vol. 52(5), 2012. A. Ricci, M. Viroli, and A. Omicini. “CArtAgO: A Framework for Prototying Artifact-Based Environments in MAS,” Environments for MultiAgent System III, vol. 4389, pp- 67-86, Springer, 2007. A. Rao. “AgentSpeak(L): BDI agents speak out in a logical computable language.” Proceeding of the 7th European workshop on Modelling autonomous agents in a multi-agent world, pp. 42-55, Scaucus, USA, Springer, 1996.