Developing a distributed real-time monitoring system ...

DEVELOPING A DISTRIBUTED REAL-TIME MONITORING SYSTEM TO TRACK UAVs Diogo Branquinho Ramos, Denis Silva Loubach, and Adilson Marques da Cunha Brazilian Aeronautics Institute of Technology Electronics & Computer Engineering Department São José dos Campos, São Paulo, Brazil

Abstract Over the last two decades, the development of Unmanned Aerial Vehicles (UAVs) has shown to be feasible in order to perform several tasks with certain degree of risk. This kind of tasks was previously held by conventional manned aircrafts. Nowadays, current UAVs employment has brought economic benefits in agricultural, energy, public safety and telecommunication areas, among others, considering operational costs, maintenance, and safety. The Air Force, Army and Navy under the Brazilian Department of Defense are developing a UAV in the so-called Veículo Aéreo Não Tripulado (VANT) Project. The Brazilian Aeronautics Institute of Technology (Instituto Tecnológico de Aeronáutica - ITA) in its Software Engineering Research Group worked in a Preliminary Testing Monitoring Station (PTMS) (Estação de Monitoramento de Ensaios Preliminares - EMEP). The PTMS is considered a key part of the VANT Project. Therefore, the major contribution of this paper reports a complete development life-cycle using state-of-the-art practices provided by System Engineering, Rational Unified Process (RUP), Military Standards, Model-Driven Development (MDD), and Integrated – Computer Aided Software Engineering – Environment (I-CASE-E) tools. Finally, the effective use of those modern approaches has enabled the Unmanned Aircraft System (UAS) to be developed, integrated on-time and to work as expected and specified.

Introduction The development of Unmanned Aerial Vehicle (UAV) has shown to be feasible to perform several tasks with some degrees of risk. Hence, considering this scenario, the Brazilian Department of Defense is developing a UAV in the so-called Veículo Aéreo Não Tripulado (VANT)

Project. The Brazilian Aeronautics Institute of Technology (Instituto Tecnológico de Aeronáutica ITA) in its Software Engineering Research Group worked in a Preliminary Testing Monitoring Station (PTMS) which is considered a key part of the VANT Project. PTMS’ main goals are getting technological knowledge to provide UAV navigation, guidance, and control. In order to prove the feasibility of technical concepts for autonomous flight capability, both autopilot and navigation are performed through waypoints, and flight supervision through ground control stations. PTMS is a distributed real-time monitoring system used to track UAV. This system comprises a collection of independent computers presenting a single and consistent interface to end-users. PTMS is composed of the following Computer Software Components (CSC): Telemetry Data Extraction (TDE); Antennas and Cameras Pointing (ACP); and Panoramic Video Digitalization (PVD). Being interconnected into a computer network through software, the system allows resources sharing hardware, software, and data. The approach taken in this work was driven by System Engineering using the best practices of software development. Those are recommended by Military Standards, N2 Chart, Rational Unified Process (RUP), Unified Modeling Language (UML) and Model-Driven Development (MDD). This work also includes automatically code generation and testing technologies through a set of Integrated – Computer Aided Software Engineering – Environment (I-CASE-E) tools. Firstly, this article presents the main Goals. Next sections talks about PTMS Requirements and Applicability. In the System Engineering section it is presented process, tools and best practices of systems’ development. System architecture section

presents how PTMS is composed, and then System Parts, Modeling and Integration section details all its components. Flight Test Setup and Results sections show how PTMS was tested in the VANT Project. Finally, Major Contributions and Conclusion sections report the main achievements, learning and recommendations experienced when working with this kind of system.

Goals, Requirements and Applicability PTMS has monitored the UAV during flight testing to acquire knowledge about the platform (aircraft) behavior regarding its operations. It recorded both on-board and ground parameters during the UAV flights, with enough accuracy. By doing so, it allowed platform stability derivatives and control refinements. Furthermore, video cameras have captured the platform behavior including command areas during flights. In sum, PTMS was able to provide the acquisition of telemetry data, show UAV position in a 2D-map, point the antennas and cameras direction to track UAV, and receive panoramic video from UAV, allowing flight command by Head-Up Guidance System (HGS) containing the main telemetry information. Also, PTMS was able to provide records from each flight for later analysis. This helped to obtain technological knowledge by collecting sufficient information from the flight platform in order to develop the autonomous UAV control, through its navigation, guidance and control. An important UAV application in Brazil has been in the Research and Development (R&D) area involving some important national research centers. It has been enhanced by courses taught at ITA, and the provision of undergraduate and graduate scholarships.

practices considering: requirements management, visual modeling, configuration management, and change management.

Process The development process had to drive the project during its life-cycle to provide scheduled reviews and audits, according to MIL-STD-1521B. The VANT Project had also used the MIL-STD-498 as a guideline helping to define a process model in which PTMS was described as a Computer Software Configuration Item (CSCI). It has represented a software aggregation satisfying enduser functions assigned to a distinct configuration management [2]. During the PTMS development, Rational Unified Process (RUP) was adopted. RUP is a Software Engineering Process which aims to increase the productivity of development teams, offering the best practices related to software through guidelines, templates, and critical activities orientation for software development [4]. In this case, RUP was tailored to fit this medium-size project. It also aimed to help PTMS CSCI management, once the VANT Project had to be split up into different teams. The RUP employment was synchronized with other CSCIs Project. This was done by means of reviews and audits (system requirements review, system design review, software specification review, preliminary design review, critical design review, test readiness review, functional configuration audit, physical configuration audit and formal qualification review) [4] and the main four RUP phases (inception, elaboration, construction and transition), as shown on Figure 1.

System Engineering A system is an interdisciplinary and multidisciplinary collaborative engineering approach to verify and evolve a balanced solutionsystem during its life-cycle, meeting stakeholders’ expectations and public acceptances [1]. To do so, it is necessary to adopt a main process and its best

Figure 1. Phases, and reviews and audits

The main pillar of RUP is the concept of best practices, which are rules/practices to reduce the existing risk in any software project, making more efficient development through six well-known best practices [4], as following: 1. 2. 3. 4. 5. 6.

Develop software iteratively; Manage requirements; Use component-based architectures; Visual model software; Verify software quality; and Control changes to software.

The main best practices and its I-CASE-E tools applied to this case are depicted on the next sections.

Requirements Management Requirements management is a systematic model used to find, organize, document and trace system requirements in which the model must be able to: • Identify, organize and document the system requirement; and • Establish and keep deals between client and team project in the system requirements. In this work, the tool employed to implement the best practices for Requirements Management was the IBM-Rational RequisitePro [5].

Visual Modeling Visual modeling is a way of thinking about problems by using organizational models around ideas of the real world. One of the most used patterns notation to represent software abstractions has been the UML. Essentially, it defines two sets of diagrams to show relationships between computer systems objects and elements, namely structural and behavior diagrams [6]. Another technique related to visual modeling is the MDD. It allows all models built to be fully exploited in the development. Consequently, automatically source-code generation is performed using those models.

Some advantages in the use of models can be listed as follows: • It facilitates the understanding of problems; • It helps the communication between engineers and users; • It prepares documentation for the system; • It prepares the software design; and • It should be used for automation tests. The tools employed to implement the best practice for Visual Modeling were the IBMRational Rose Real-Time [7] and IBM-Rational Test Real-Time [8].

Configuration management Configuration management is a key to control many artifacts (e.g. documents, software or hardware) produced by different people working in the same project. This approach helps to avoid confusion among teams. It guarantees resulting artifacts to not come into conflict, since the following difficulties do exist: • Simultaneous updating; • Notification limitation; and • Several versions. The tool employed to implement the best practices for Configuration Management was the IBM-Rational ClearCase [9].

Change Management Change management provides controlled monitoring of verification process and approval of requests for change. It also coordinates approved changes to be implemented. Those changes may occur due to defects or requests for project improvement. Moreover, it helps investigating defects reasons and improving requirements elicitation. The tool employed to implement the best practice for Change Management was the IBMRational ClearQuest [9].

Figure 2 shows the integration among the best practices of software development and tools that support these activities.

with regular operating systems (MS Windows and Linux). PTMS is ranked as a soft real-time system. Also, the use of commercial off-the-shelf (COTS) components was one of the PTMS main goals. Next section describes the PTMS operation.

Operation

Figure 2. Best Practices vs. Tools

System Architecture Considering system architecture, the PTMS CSCI was comprised of the following three CSC: • Telemetry Data Extraction (TDE) – that receives GPS data, inertial sensor (x and y axis) and anemometrics data from the UAV telemetric unit. These features are mirrored into CSC TDE2, doubling GPS data and inertial sensor (z and y axis); • Antennas and Cameras Pointing (ACP) – that controls the antennas direction pointing in order to track the UAV. Besides, it shows UAV position in a 2Dmap in real-time (1 Hz rate); and • Panoramic Video Digitalization (PVD) – that receives in real-time the aircraft front-line image sight, and adds an overlapping layer containing the main telemetry information.

PTMS as a distributed system

In this architecture the operation is started by TDE and TDE2 receiving telemetry data from the aircraft at 10Hz rate. Each TDE had its own telemetric unit. Then, it processes and sends telemetry data to other CSC connected to it at a rate of 5Hz. Notice TDE operation is mirrored. Therefore, it assumes the system clock role. PTMS uses a directional antenna, which receives signals from telemetry data, video and radio control. Consequently, when the UAV are on operation, it is necessary that antennas to point to track it. That function is assigned to ACP, which receives TDE data from the network. Its focus is on longitude and latitude data, calculated by using predictor algorithms. To accomplish this operation, the PVD receives panoramic video from UAV, and adds an overlapping layer named HGS containing the main telemetry information. Finally, each PTMS CSC generates log files. This helps to evaluate UAV on-the-flight behavior by doing post-flight analysis in order to improve navigation, guidance and control. Figure 3 shows PTMS system architecture.

System Parts, Modeling and Integration

A distributed system is a collection of independent computers which are presented to the end-user as a single and consistent system. It is interconnected in a computer network through software system allowing resources of hardware, software and data sharing [10].

Considering the well-defined and stable PTMS requirements, each CSC had been developed and integrated without major difficulties. The Elaboration RUP phase was very important to assure successful components integration.

On PTMS the main sharing resource was telemetry data among the CSC. Architecturally, PTMS uses Ethernet network to share its resources. Its nodes are computers based x86 architecture,

Another useful applied technique was the N2 Chart [11]. It helped to verify and validate the information input/output on each CSC. Next sections show how each CSC was developed.

Figure 3. PTMS System Architecture

CSC TDE TDE receives and manages the CSC TCP/IP connections. Then, it distributes relevant information to each of them. For its development visual modeling and MDD were used. This enabled 80% automatic source-code generation from this component. The other part was hand-made into the models [12]. The application of MDD was performed by using Rational Rose Real-Time. For the TDE development some active classes was needed. They were instances classes having the ability to concurrently operate with each other. Consequently, it was assumed that some active classes groups were mapped into different threads. Therefore, this approach ensures concurrency, so that the best processor usage was done by these instances. The communication, reading and writing files instances were mapped into different threads, since they are the most important ones. On the other hand, other instances were mapped into the default main thread, automatically created by the modeling tool.

Figure 4. State diagram with real-time mode and simulator mode Regarding this scenario, another important UML diagram is the structure diagram. It shows the instances class organization, helps to dynamically enable/disable instances, and represents messages exchange among them. Figure 5 shows a TDE structure diagram.

The behavior of each active class was modeled through states diagrams, which follows the pattern of Mealy-Moore state machines [13]. On Mealy machine, states outflow are associated with transitions. Conversely, on Moore outflows are associated with states. Figure 4 shows one of TDE state diagrams. Figure 5. TDE Structure diagram

CSC ACP After ACP gets connected to TDE or TDE2, it begins to receive UAV positioning data. Then, it starts to plot UAV coordinates on a georeferenced 2D-map, and calculates the command to pointing antennas in order to receive telemetry data, videos and radio control signal. The PTMS development aimed to use cheap solutions. An example was the reuse performed in ACP, more specifically, the use of a satellite georeferenced image obtained from the Google Earth tool [14], as shown on Figure 6.

pixelY =

(−UAVLat + iniLatPos ) * imgHeight (iniLatPos − finLatPos )

Equation 2. Pixel Y position calculation To point antennas to track the UAV, ACP uses miniature pan-tilt units that provide accurate realtime positioning of cameras, lasers, antenna, or other small-medium payloads. The pan-tilt unit was controlled through TCP/IP with instructions varying the movement direction of pan, tilt, motion in degrees, speed and acceleration. To do so, it was necessary the use of a mathematical predictor that calculates the UAV future position, one second ahead, so that the delay in pointing was compensated. The adopted mathematical prediction algorithm was the Recursive Least Squares (RLS) [15]. Basically, the RLS consists of five main steps: 1. Gain determination; 2. Parametric estimate update; 3. Co-variance matrix update; 4.

Figure 6. The main ACP screen To extract an interest image to the UAV testing from Google Earth, a procedure was performed. It consisted in setting initial and final latitude and longitude positions of the image (iniLonPos, finLonPos, iniLatPos, and finLatPos). Also, the image scale (imgHeight and imgWidth) had to be stored. Thus, it was possible to relate the UAV latitude and longitude positions (UAVLat and UAVLon) with the image pixels, consequently making a direct UAV location on it (pixelX and pixelY). The Equation 1 and Equation 2 express the calculations done.

pixelX =

(iniLonPos − (−UAVLon)) * imgWidth ( finLonPos − iniLonPos )

Equation 1. Pixel X position calculation

φ vector regression update; and

5. Algorithm iteration (recursive).

CSC PVD After PVD gets connected to TDE or TDE2, it can already add an overlapping layer (telemetry information), in real-time, to the aircraft front-line image sight. The development of this component was performed by using Linux Operating System, distribution Ubuntu 7.10 [16]. One of the challenges faced by the authors of this paper was to add an overlapping layer of telemetry data over the live video. It worked out by using a development library for editing live video named hairCapture. It uses libraries of Video for Linux (V4L) and Simple DirectMedia Layer (SDL) [17]. Figure 7 shows an example of the HGS display developed using those libraries.

• Timeliness; • Show information on screen, in soft realtime; • Antennas pointing; • Videos digitalization and recording; • Telemetry data processing; and • Functions performance.

Flight Test Results Figure 7. HGS layout Video signals were received by directional antennas, and digitalized through capture-cards installed on the PVD computer. The HGS experience was really useful, so that UAVs pilots could have notion of how to remote fly the UAV.

Flight Test Setup So far, on the Flight Test Setup (FTS), it was proved that the use of the development process (shown in this paper) and its reviews and audits really have worked as expected. This process was successfully used on the entire VANT Project, in flight testing, also named preliminary testing, and also in other CSCIs integrating their products to work together.

Integration testing The PTMS integration testing in FTS was submitted to an integrated testing sequence which had included the following items:

After all PTMS’ units had being tested, integrated and all integration testing had being performed with other CSCIs of the VANT Project, it was possible to perform the field testing part. Those testing were performed in large open area far from cities. The flight scenario consisted of a 1.3 miles (diameter) circle area, where the UAV was supposed to flight in circles. Each flight should last about half an hour, considering a high of about 1000 feet at 90 knots. Therefore, the PTMS had monitored a total of 9 successful flights. Each flight was performed in order to vary the UAV behavior changing its speed; high; and roll, pitch and heading angles. Each flight had its own log (e.g. telemetry data, video, and signal quality) collected by the PTMS. By carrying out this entire process, it was possible to analyze data and generate a set processed information (e.g. plain text and graphical). Figure 8 shows an example of graphical information processed by PTMS, exhibiting a UAV 3D trajectory.

• Ground tests with simulated aircraft; • Ground tests with UAV, nearby PTMS; and • On-the-flight tests with UAV.

Functionalities verification The integration tests sequence, mentioned in section “Integration testing”, had supported the critical functionalities verification, regarding the PTMS. Those functions comprise of: • Communication interface with the UAV; • Data and video signals reception;

Figure 8. UAV 3D Trajectory

Figure 9 shows time vs. high of the UAV during one flight.

The best practices’ application was possible since I-CASE-E tools were also applied, supporting the whole process of software configuration and management. The PTMS architecture and its modularization had helped the development, componentization and integration of all its parts. Moreover, the use of COTS has contributed to cost reduction and efficiency improvement. That brought up a significant research and development effort economy.

Figure 9. UAV altitude and speed The analysis process of all this information together was able to help getting some knowledge considering UAV navigation, guidance, and control.

Major Contributions The major contribution of this paper was the report of a complete development life-cycle using state-of-the-art practices provided by System Engineering, RUP, Military Standards, MDD, and I-CASE-E tools. The effective use of those modern approaches presented here in this paper had enabled the UAS to be developed, integrated on-time and to work as specified and expected. After all this process gets done, the PTMS has helped to test other UAVs. It is also getting improved. The authors of the PTMS believe that, as a spin-off, it can become a full functional ground control station, in a short term period.

Conclusion and Remarks The System Engineering approach adopted in this work to multidisciplinary projects had shown to be quite feasible and efficient. Within the System Engineering, it was defined a development process, audits and review phases, and I-CASE-E tools utilization were performed.

The use of post-flight processed information has helped the VANT Project to get its main goals, improving technological knowledge to provide UAV navigation, guidance, and control. Finally, PTMS worked successfully as expected performing all of its required functions. In a near future, PTMS can become a complete ground control station, however, nowadays it just has all components it needs to get there.

Acknowledgements The authors of this paper thank the following Institutions for their support to this work: the General-Command of Airspace Technology (Comando-Geral de Tecnologia Aeroespacial – CTA); the Aeronautics and Space Institute (Instituto de Aeronáutica e Espaço – IAE); the Aeronautics Institute of Technology (Instituto Tecnológico de Aeronáutica – ITA); the Brazilian Funding for Studies and Projects (Financiadora de Estudos e Projetos – FINEP); the Brazilian National Council for Research and Development (Conselho Nacional de Pesquisa e Desenvolvimento – CNPq); the Casimiro Montenegro Filho Foundation (Fundação Casimiro Montenegro Filho – FCMF); and the Brazilian Department of Science and Technology (Ministério da Ciência e Tecnologia – MCT).

References [1] The Institute of Electrical and Electronics Engineers, Inc., 1995, IEEE trial-use standard for application and management of the system engineering process, IEEE Std 1220-1994.

[2] Department of Defense, 1994, MIL-STD-498, Software Development and Documentation, Washington, USA.

[10] A S Tanenbaum, M V Steen, 2001, Distributed Systems: Principles and Paradigms, Upper Saddle River, USA, Prentice Hall PTR.

[3] Ivar Jacobson, Grady Booch, Jim Rumbaugh, 1999, The Unified Software Development Process, Boston, MA, USA, Addison-Wesley.

[11] M Naci Akkøk, 2004, Refining Compositional Structure Using the N2 Chart for Identifying Components and Design Patterns in ComponentBased Systems, Izmir, Turkey, IMPROQ 2004: Impact of Software Process on Quality Workshop.

[4] Department of Defense, 1985, MIL-STD1521B, Technical Reviews and Audits for Systems, Equipments, and Computer Software, Washington, USA. [5] Zielczynski, Peter, 2007, Requirements management using IBM Rational RequisitePro, Boston, MA, USA, IBM Press. [6] Loubach, Denis Silva, 2007, ASSETRE: Uma Arquitetura de Software para Sistemas Embarcados de Tempo Real, Master Thesis, São José dos Campos, Brazil, Instituto Tecnológico de Aeronáutica. [7] Walter A Santos, Denis S Loubach, Marcos R Nascimento, João C Nobre, 2006, Aerospace software prototype with IBM(R) Rational Rose(R) Realtime. In: IBM Rational Software Development, Orlando, USA, Software in Concert. [8] Denis S Loubach, João C Nobre, Adilson M Cunha, Luiz A V Dias, Marcos R Nascimento, Walter A dos Santos, 2006, Testing Critical Software: A Case Study for an Aerospace Application, Portland, USA, 25th Digital Avionics Systems Conference, 2006 IEEE/AIAA, pp.1-9. [9] David E Bellagio, Tom J Milligan, 2005, Software Configuration Management Strategies and IBM Rational Clearcase – A Practical Introduction, Second Edition, Indiana, USA, Addison Wesley.

[12] Denis S Loubach, Diogo B Ramos, Osamu Saotome, Adilson M Cunha, 2008, Comparing Source Codes Generated by CASE Tools with Hand Coded, Las Vegas, USA, Information Technology: New Generations, ITNG 2008, Fifth International Conference on , vol., no., pp.1292-1292. [13] Avnur, A, Finite state machines for real-time software engineering, 1990, Computing & Control Engineering Journal, vol.1, no.6, pp.275-278. [14]Google, 2008, Google Earth, Online, http://earth.google.com. [15] Yaakov Engel, Shie Mannor, Ron Meir, 2004, The kernel recursive least-squares algorithm, Signal Processing, IEEE Transactions on, vol.52, no.8, pp. 2275-2285. [16] Ubuntu, 2008, Operation System Ubuntu 7.10, Online, http://www.ubuntu.com. [17] Faria, Alessandro de Oliveira, 2008, Library hairCapture, Online, http://lhaircapture.sourceforge.net.

27th Digital Avionics Systems Conference October 26-30, 2008.