Development of a VRML/Java Unmanned Airship ... - CiteSeerX

IEEE/RSJ International Conference on Intelligent Robots and Systems, Kyongju, Korea, October 1999.

Development of a VRML/Java Unmanned Airship Simulating Environment Josué Jr. G. Ramosa,b Silvio M. Maeta a, Marcel Bergerman a, Samuel S. Bueno a, Luiz G. B Mirisola a, Augusto Bruciapagliab a

Automation Institute CTI -Technological Center For Informatics Foundation CP 6162 Campinas Brazil email: [email protected] b

UFSC-DAS - Federal University of Santa Catarina System and Automation Department Santa Catarina Brazil

Abstract We present in this paper one of the first Internetaccessible airship simulators, based on a comprehensive airship dynamic model. The simulator is meant to be used as a tool for the development of control and navigation methods for autonomous and semiautonomous robotic airships and as testbed for airship pilot training. Realistic views of both the airship in flight and that of a virtual pilot are provided, as are commands to apply thrust to the engines, swivel them up and down, and deflect the control surfaces. This work is significant for providing robotics researchers with means to safely experiment with airship control.

1. Introduction Several researchers have been pursuing in the past few years the development of autonomous and semiautonomous aerial vehicles for such diverse tasks as exploration, surveillance, transportation, or simply to function as advanced technology demonstrators. In this context the Automation Institute proposed the AURORA project, where automatic and semi-autonomous navigation strategies are being developed for unmanned airships. The project’s goal is to provide a low-speed, low-altitude aerial vehicle for environmental monitoring missions. The choice of an airship is based on the fact that these vehicles outrank airplanes and helicopters for this type of missions [5] mainly because their aerostatic

VRML world

Pan Tilt Vectorization Wind Propellers Simulator Command Ballonets Heaviness

Instruments panel

Pitch and Yaw

Figure 1: A general view of the Internet Accessible Airship Simulator.

IEEE/RSJ International Conference on Intelligent Robots and Systems, Kyongju, Korea, October 1999.

lift makes them less intrusive, capable of hovering, and able to stay in the air for longer periods. Among the project objectives, one is to develop a realistic visual simulator of the airship from take-off to flight and landing. Such simulator is the focus of this paper. It is meant to be a tool for both control and navigation methods development and for pilot training. At the simulator’s core lies a comprehensive dynamic model of the airship, including inertial, aerodynamic, Coriolis, centrifugal, gravitational, buoyant and propulsion forces. Such dynamic model is based on 600 hours of wind tunnel data collection, considered to be the most complete airship aerodynamics database available [6][7]. The simulator also comprises all the technical stuff today available for airship operation, such as propeller’s speed and vectorization, aerodynamic surfaces for pitch and yaw commands, means for splitting the air volume between fore and aft ballonets and heaviness condition as well. These airship operation aspects are described in [7]. The airship simulator is based on our previous experience with the development of an unmanned helicopter simulator [10]. To increase its accessibility, it has been developed using Java and VRML, which makes it available world-wide through any VRML97 compatible WWW browser [13]. By making the simulator available on the Internet, the authors hope to contribute to research and development on unmanned robotic airships. As far as we know this simulator is one of the first integrating an airship dynamic model in a browser / VRML / Java environment. The NPS virtual environment for Underwater Vehicles [2] is another example of a virtual environment using VRML. This paper is divided as follows: Section 2 presents the simulator’s software components. In Section 3 we present a pilot training tool. Section 4 discusses the development issues associated with such environment. In Section 5 we briefly mention the ongoing work and in Section 6 we present our conclusions.

2 Simulator Software Components In this section we describe the simulator’s components, including its human machine interface, its main software components and how the system works. The simulator user interface is composed of a Java Applet [11] integrated in single HTML page, depicted in figure 1. It allows a user, located anywhere in the world, to access the simulator’s through its URL [1]. The interface loads all files and Java classes necessary for the simulator onto the user’s computer, and the simulation can be started. Figure 2 shows the simulator’s structure, with its main software components. It is divided in two boxes: box 1

shows the features seen by the user and box 2 shows features related to the simulator’s software structure. Browser Window

1

VRML virtual World EAI

2

VRML Environment Control Applet simvrml - simulator's components interfacing

Mouse/ Keyboard Events (user)

Command and Instruments Panel ( User graphic Interface )

Geometric Transformations (Rotation, Translations)

CONT Socket Conection (Class sockcomm) Simulator controlled via Radio Control

Instruments COM Panel Updating and Commands Interfacing STAT (Thread console)

Interface (Class interfac) store states and commands

COM

STAT

Mathemathic Modelling (Thread runmod)

Browser Virtual JAVA Machine SOCKET

Command Flow

CONT : simulation control variables

Data Flow

STAT : SImulation state variables COM : commands variables

Figure 2: Simulator structure and main components. The components included in box 1 of figure 2, are associated to the HTML page, this includes: text, the VRML world, the command and instruments panel, and the results of the HTML page loading is shown in figure 1. The reference to the VRML world and to Java applet are made in the following way inside a HTML page: a) Reference to the VRML file: b) Reference to Applet responsible for the simulation: The reference to the VRML file, ASvrml.wrl, in “a)”, results in the VRML plug-in calling to show the VRML world. The reference to the Java classs, simvrml.class, in “b)”, results in the Java Virtual Machine calling, to interpret and execute the Java program; since this Java program is executed inside a Browser, it is called an Java Applet. Box 2 in figure 2, shows features related to simulator’s software structure. It is composed of a set of Java classes, responsible for: the simulation process, user commands processing, instruments panel and virtual world visualization.

IEEE/RSJ International Conference on Intelligent Robots and Systems, Kyongju, Korea, October 1999.

For virtual world visualization, it is required a connection between the Java Applet and the VRML world. This connection is made by the use of the EAI (External Authoring Interface) [3]. EAI is an interface that allows programs written in Java and JavaScript to control the contents of a VRML world. The controls available include: object creation and removing, operations as rotation, translations, user’s point of view changing, properties changing, etc. The following subsections describe the main simulator’s components. 2.1 VRML World It is seen at the upper left corner of figure 1. The airship is geometrically modeled as an elongated ellipsoid, with cylindrical propellers on each side and polygonal control surfaces. By default, the VRML [12] world displays the aircraft as if it were being seen from ground; alternatively, user can embark in a virtual airship flight by switching to a virtual pilot’s view. The VRML world displays also a tessellated flat ground and a cloudy sky for more realism. The VRML world displays the airship based on the position and orientation provided by the time integration of the airship’s dynamic model. 2.2 VRML world control - class simvrml Simvrml is the main and first class invoked by the browser when the simulator’s page is accessed. and it is responsible for all other classes initialization and for connecting the VRML to the Java Applet, beginning the simulation. This component includes methods responsible for: body positioning and orientation updates (inside the virtual world) and user point of view definition at every simulation step. This allows the user to visualize the movements being executed by the airship. 2.3 Airship’s console - class console Console is the Java class of the airship’s human-machine interface. It includes the simulator commands, airships’ commands, simulator’s instruments, ballonets and heaviness commands, as well as data acquisition and plotting commands. The Simulator Commands, seen at the lower left corner of figure 1, allow the user to run the simulation step-bystep or continuously, acquire data for further plotting, and reset the simulation. It is also possible to exchange from a ground view to a virtual embarked camera view (with the Inside check-box); this last one can be further commanded by pan-tilt buttons so that the camera is moved up-down or right-left. Additionally, the user receives information on the current simulation cycle time

(in ms/cycle), simulation overall time and number of frames per second being displayed in the VRML world in figure 1. Airship Commands seen at the lower right corner of figure 1 allow the user to: 1. pitch the ship up and down and to yaw it to the left and to the right, by holding the mouse button on the leftmost circular region of the window; 2. adjust the engines (propellers) speed by dragging the center cursor up and down on the rightmost region; 3. swivel the engines up and down (vectorization) by dragging the cursor on the top of this region. In any case, the mouse coordinates are transformed into the corresponding commands sent as inputs for the dynamic model calculation. Ballonets and heaviness are commanded by two slide bars, shown in the left down side of figure 1, that can be move continuously during simulation, The upper bar allow for changes on the percentage of volume division among fore and aft ballonets. This values affects the airship pitch equilibrium. The lower bar is used to define the airship heaviness or buoyancy, which corresponds to the difference between the upperward lift force and the airship weight. In figure 1 the airship is flying with neutral heaviness (“H 0” in the figure) and the fore ballonet has 50% of the volume of both ballonets combined. The Simulator Instruments’ panel includes methods that show the airship’s flight instruments, emulating altimeter, compass, artificial horizon, etc., allowing to see its behavior during a flight, and a commands panel to actuate over the airship. A data acquisition window (not shown in figure 1) allows for plotting of variables recorded during simulation. The current implementation provides for plotting of the airship’s position and orientation, velocities and accelerations. Console class is a Java thread responsible for the interpretation of user’s generated events from the mouse and keyboard and captured by the browser. These events will be transformed in the of the simulator’s commands described above. This class is also responsible for the instruments’ panel updating, using the simulated data. The Simulator Instruments’ panel is presented in graphic context created by simvrml, but its updating is done by console. A picture that contains all the drawing of the panel is loaded when the simulator is being loaded. The updates are only carried on the changeable parts of the instruments (numerical data, and clocks), aiming performance profits.

IEEE/RSJ International Conference on Intelligent Robots and Systems, Kyongju, Korea, October 1999.

2.4 Mathematical Model execution - class runmod The class responsible for running the airship’s dynamic mathematical model is runmod. It is a Java thread responsible for the execution of a simulation step of the selected mathematical model, giving as result new airship’s states. This execution can be continuous or step by step, depending on the user’s configuration. This class is also responsible to command the virtual world updating. The mathematical model is a Java object which at every cycle, receives as inputs the pitch and yaw commands, engine speed and angle, ballonets volume sharing and heaviness, and outputs the airship’s position and orientation. It is a 6 degree of freedom system whose non- linear differential equations are numerically integrated through a 4th order Runge-Kutta algorithm to avoid numerical instabilities. The output of the dynamic model is used to update the view in the VRML world. 2.5 Simulator’s states and commands - class interfac This is a class shared by all components, allowing these classes to exchange data without communication overheads. This class contains a set of state variables that stores the simulator’s operation mode (for example step by step), data acquisition state (on/off), etc., and a buffer of passed states, that allows for the aircraft behavior analysis. 2.6 Additional Components a) Graph plotting: provides graphical representation of the behavior of the aircraft (speed, acceleration, etc.). The plotting class opens a window to the user and presents the listing of the graphs that can be plotted. The user selects one of the possible graphs and sets its presentation. b) Control over environmental features: the user can define the earth wind direction and speed that will act on the airship’s flight.

initiates the simulation process, and makes the threads runmod and console run. During the simulation process we have: - thread runmod: executes a simulation every cycle. It updates the data of interfac and the state of the virtual world (airship position and orientation); - thread console: from interfac data it updates the instruments panel. It treats the new user commands and modifies the corresponding variables in interfac; - applet simvrml: it receives user events (keyboard or mouse) and sends them to be treated by console. It sends to the browser the control sent by runmod on the virtual environment, and by console for the instruments panel.

3 The Simulator as a Training Tool On AURORA project, an onboard system is being developed for unmanned airships, rendering them robotic aerial vehicles capable of autonomous operation. But, for safety reasons, a remote control unit (RCU) must be available to steer the airship at all times, in case there is a malfunction or collapse of the onboard control system. This means that a few integrants of Project AURORA must be capable of piloting the airship safely to ground. Since the easiest and cheapest way of practicing with aerial flight is using a flight simulator, and because there are no known commercially available simulated airship for piloting training, we developed additional software and hardware components to make our simulator a pilot training tool. Figure 3 presents the scheme developed. Roughly speaking, the simulator reads data from the computer serial port, data which has been sent from the RCU via a microcontroller. PWM Signals

Microcontroller (Measures PWM Signals data sent by serial line) Serial

2.7 How the Simulator Operates

8 channels Radio Control Cable - PWM Sinal

Serial Cable - ASCII Data

The process of simulation initialization is carried in the following way:

Serial

- the HTML page with the Applet simvrml and the virtual world references are loaded by the browser. The VRML plugin loads the virtual environment that contains the airship;

Browser RadCon with VRML plug in ( C++ Program ) Radio Reads serial line Commands running JAVA Simulator Sends data via sockets

-simvrml carries the connection between Java and VRML (through the EAI - External Authoring Interface), calls the other components (interfac, runmod, console)

PC - Windows 95

Sockets

Figure 3: Additional software and hardware components.

IEEE/RSJ International Conference on Intelligent Robots and Systems, Kyongju, Korea, October 1999.

The RCU sends data to the microcontroller in PWM (pulse width modulation) form. The microcontroller measures the pulse’s width and converts them into ASCII data, which is sent to the computer via a serial interface. In the computer, a C++ program continuously monitors the serial port, reads the data received, and forwards it via a socket to the Java-based simulator. This implementation was chosen because of the strict security characteristics of the Java machine running in a browser, which does not allow for direct communication to a serial port unless one is highly familiar with the security mechanism of today’s browsers and is able to override them. Currently, the pilot training tool allows any integrant of project AURORA to pilot a simulated airship as realistically as if it were the real vehicle, and using exactly the same RCU that is utilized to fly it.

4 Development Issues In this section we discuss several issues related to the system presented, and why it is still limited to PC-based or Silicon Graphics Inc. (SGI) computers.

The simulator was developed for Netscape due to the lack of a VRML plug-in for Microsoft’s Internet Explorer. It was only recently that a functional version for IE 4.0 became available. 4.5 Browser Evolution Serious difficulties were faced when the authors upgraded the simulator from Netscape 3.0 to version 4.0, indicating that there is still a need for a more useroriented software products. The solution adopted consisted of combining several applets into a single one, which Netscape 4.0 accepted. 4.6 Multiplatform Issues Even though it is the authors’ wish to make a platformindependent system, we have had no success in our search for VRML/EAI for Linux, Sun or AIX environments. Currently the simulator is available for PC-based and SGI architectures.

4.1 Standardization Issues

4.7 Connection to peripheral hardware

VRML97 became an ISO standard as late as August, 1997. It was only after this that ISO-compliant plug-ins started to be developed. EAI, on the other hand, is expected to become an ISO standard only by January 2000 [3]. These two facts limit the number of computer platforms which attend the simulator’s specifications.

As mentioned in section 3, the intricate security mechanism of today’s browsers led us to create an additional C++ file to allow the simulator to communicate with an external microcontroller. We believe this to be a solution with a very satisfying cost/benefit relationship.

4.2 The evolution of Java

5 Ongoing work

Java is a programming language with a growing number of users, and rapidly evolving itself. During the development of our simulator, based on Java 1.0, version 1.1 was released. This made our human-machine interface, based on the 1.0 AWT (abstract window toolkit), obsolete. We are currently porting the system to the new version.

The simulation environment is being adapted to become a tool in project AURORA for safe and low cost development, test, and validation of both the onboard and ground hardware and software infrastructures, as well as of the control and navigation algorithms, before they are implemented in the actual vehicle. In this sense, it is being integrated to our Simulink-based control system development environment [8], [9]

Regarding computational efficiency, one must keep in mind that Java is an interpreted language, which makes it slower to run than compiled languages like C++. Nevertheless, it suits very well our need for an Internetaccessible tool without being slow. 4.3 Installation Issues The lack of maturity of the available VRML plug-ins used, make installation of the simulator a frustrating experience, mainly for new users. As the plug-ins have stepped out of the beta stage in the past few months, ease of installation has improved tremendously. 4.4 Netscape and Internet Explorer

The VRML world can include 3D maps constructed from real digital topographic data. This feature will be used in the mission planning phase to specify a flight profile, including points to be visited, the trajectory to be followed between these points and also the time schedule of the flight.

6 Conclusion The VRML/Java-based airship simulator presented in this article is one the first to provide to the robotics community an Internet-accessible tool for realistic aerial vehicle control. Its main distinguishing features are the

IEEE/RSJ International Conference on Intelligent Robots and Systems, Kyongju, Korea, October 1999.

underlying 6-DOF dynamic model, VRML world, flight console, and flight controls.

International Conference on Advanced Intelligent Mechatronics, Atlanta, USA, September 1999.

The ongoing work described above will greatly extend the use of this simulation environment.

[10] Ramos, J.; Neves O., "Environment for Unmanned Helicopter Control System Development: Application Examples", 4th IEEE Conference on Control Applications, Albany, USA, September 1995.

7 Acknowledgments This work is partially supported by the Brazilian funding agencies CNPq and FAPESP (grant no. 97/13384-7).

[11] SUN- Java Home Page: http://www.javasoft.com .

8 REFERENCES

[12] The VRML consortium site: http://www.vrml.org .

[1] AURORA’s Internet Accessible Airship Simulator URL: http://www.ia.cti.br/~josue/asim. [2] Brutzman, D. P. , “A virtual world for an Autonomous underwater vehicle”, Ph.D. Thesis, Naval Postgraduate School, USA, Dec. 1994.

[3] EAI Specification: http://www.vrml.org/WorkingGroups/vrml-eai/ Specification/ [4] EAI Working Group Home Page http://www.vrml.org/WorkingGroups/vrml-eai [5] Elfes, A.; Bueno, S.S.; Bergerman, M.; Ramos, J.J.G. “A semi-autonomous robotic airship for environmental monitoring missions.” IEEE International Conference on Robotics and Automation, Leuven, Belgium, May 1998, pp. 34493455. [6] Gomes, S. B. V., “An Investigation of the Flight Dynamics of Airships with Applications to YEZ2A”, Ph.D. Thesis, Cranfield Institute of Technology - College of Aeronautics, 1990. [7] Gomes, S.B.V.; Ramos, J. J.G.,“Airship Dynamic Modeling for Autonomous Operation”, IEEE International Conference on Robotics and Automation, Leuven, Belgium, May 1998. [8] Paiva, A.; Bueno, S. S.; Gomes, S.B.V.; Bergerman, M.; “A Control System Development Environment for AURORA’s Semi-Autonomous Robotic Airship”. IEEE International Conference on Robotics and Automation, Detroit, MI, USA, May 1999. [9] Ramos, J. J. G.; Maeta L. G.; Mirisola, L.G.B; Bergerman, M.; Bueno, S.S.; Pavani, G. S.; Bruciapaglia, A.; “A Software Environment for an Autonomous Unmanned Airship” IEEE/ASME

[13] VRML97 Specification: http://www.vrml.org/Specifications/VRML97