th
In Proc. of IECON’99, The 25 Annual Conference of the IEEE Industrial Electronics Society, pages 821-826, San Jose, CA, IEEE
RoboSiM: Java 3D Robot Vizualization Andreas Speck Universität Tübingen Wilhelm-Schickard-Institut für Informatik Sand 13 D-72076 Tübingen, Germany
[email protected]
Herbert Klaeren Universität Tübingen Wilhelm-Schickard-Institut für Informatik Sand 13 D-72076 Tübingen, Germany
[email protected]
Abstract – The programming language Java has made it possible to develop device independent software. We introduce in this paper a robot visualization system implemented in Java. This graphical user interface is part of our robot simulation and monitoring system (RoboSiM). Currently two versions of the visualization exist, a Java 1.1 based system and its successor realized in Java 3D. Both applets display real or virtual robot arms and enable the user to manipulate this arm. Due to support of distributed computing by Java and its platform independence the Java GUI may run on different local and remote systems. Therefore the same robot may be controlled by a local user or via Intranet or Internet with the same applet. The GUI applet communicates directly with the control or simulation computer. In contrast to the Java 1.1 user interface the Java 3D robot visualization is much more detailed. Compared with other visualization systems the Java 3D applet is very realistic. Therefore the Java display is a capable alternative to the traditional visualization.
I. INTRODUCTION In its early days Java was developed as universal programming environment for embedded systems like set-top boxes on TV screens. Then Java started to conquer the Internet. All over the world Java applets are distributed via the WWW. Many people who were aware of the object-oriented software development now came in contact with object-oriented Java software. Since the simple usage and the effective way of programming many of Java's critics have been convinced. Today Java becomes more and more interesting for other application domains like the industrial automation. Parallel to the raise of Java another trend is emerging: The use of powerful standard computing platforms such as PCs or workstations with POSIX compliant operating systems instead of proprietary control systems. In this paper we introduce a robot simulation and monitoring system (RoboSiM) which is based on the HighRobot control [5, 13] - a robot control running on standard platforms (SPARC workstations, Windows NT PCs or Linux PCs). The core of RoboSiM is the Java visualization of a robot arm. Currently two releases of
Speck, Klaeren / 1
RoboSiM exist: a Java 1.1 version and Java 3D system [11]. RoboSiM combines a device simulation with the monitoring of a real existing robot at the same time on the same piece of control hardware (as well as on remote control stations). In contrast to other robot simulation systems, RoboSiM runs on the real control using the real control software. Hence this simulation concept runs Simulation Tasks under very realistic conditions. Robot programs controlling a virtual robot have a similar temporal behavior as robot programs controlling a real device because these tasks are processed on the real control.
I.A RoboSiM Applications Due to the fact that RoboSiM is a combination of simulation and monitoring it covers a wide range of applications [12]. Some examples may be: 1. Training in education and research: • The simulation may completely substitute a robot, e.g. demonstrations of the virtual robot can be given within lectures. • Real and virtual devices may be blended. Simulated robots can be used in combination with real robots. Therefore it is possible to build and test virtual production lines with one real robot which is monitored and other collaborating virtual robots. 2. Off-line robot programming This is the development of robot programs without teaching the arm positions. The motions of the robot are simulated and displayed on the screen. The robot model is either moved manually (by keyboard or mouse commands of the programmer) or according to position data from a CAD database [8]. 3. Monitoring of a real device The Java 3D visualization displays the motions of
Paper Id: 275
the robot arm very realistically. Hence the representation of RoboSiM may be compared with video camera recordings. Moreover it may exceed such video pictures due to the fact that in the visualization displays important things (e.g. small devices) even if they are hidden by other hardware. The Java Graphical User Interface may be exported on other systems via Intranet or Internet. Therefore the monitoring and simulation system can be accessed from remote sites. Of course this new concept for the simulation and monitoring system may be extended to all types of automation controls such as PLCs (Programmable Logic Controllers) or NCs (Numeric Controls).
parts of the original control code, but this is just an approximation of the behavior of the real control. In comparison with these simulation systems RoboSiM has the advantage, that there is no gap between the simulation and the control system, because both may run on the same piece of hardware. Therefore the behavior of the robot control is part of the robot simulation model. The behavior of the simulated robot model can be directly compared to a real robot arm.
I.B Other Visualization Systems Visualization systems are used for both simulation and monitoring of devices. Some new control systems (e.g. the Windows 95 interface of the Kuka robot control KR C1) provide graphical user interfaces (control panel) displaying the current state for an interactive control by the user. More complex systems like Win CC from Siemens display the devices and their behavior itself. These systems mostly run on special devices or screens. On contrast to these new systems the majority of traditional control systems (e.g. the Bosch rho 3 robot control) provide only very limited (or no) process visualizations functions.
konventionelle Robotersteuerung z.B. Bosch rho 3
Roboterzelle mit Industrieroboter z.B. Bosch SR 60
Off-line Simulation z.B. RobCAD auf SGI Workstation
Fig. 2. Java GUI of RoboSiM Simulationssystem enthält Robotersteuerungs-Code Control Panel with Switches and Light Bulbs
Fig. 1. Traditional Process Control and Simulation Systems
Of course simulation systems provide the visualization of their virtual devices. Examples for such simulation systems may be the Robotic HyperBook (RHB) [7] which is optimized for computer based training, the offline programming systems ROPES [3] and MOSES [4] or systems like RobCAD which can be used for both purposes (training or off-line programming). These systems usually run on a special computer such as an SGI workstation and not on the real robot control. Some of these systems like RobCAD already include
Speck, Klaeren / 2
II. GRAPHICAL USER INTERFACE The Graphical User Interface (GUI) is of course the most important part of RoboSiM. Currently there exist two different versions of this interface (Java 1.1 [1, 2] and Java 3D [11]). In contrast to other user interfaces like X Window System or OSF Motif both Java versions have the advantage that they may run on different platforms. Additionally the Java applets can be displayed on remote hosts. Both GUIs are used for simulation as well as for monitoring which makes it possible to compare the virtual model with the reality.
Paper Id: 275
II.A. Java 1.1 Interface This user interface is kept simple. In contrast to the Java 3D version the presentation of the robot arm is not very detailed. Due to this the Java user interface applet costs only little computing power. It is supported by many virtual machines, e.g. the Java appletviewer as well as the Netscape and Internet Explorer browsers. Therefore the simple Java visualization may be used universally on a large number of platforms.
The system has a layered architecture (cf. Fig. 4) which is a common approach in the development of object-oriented real-time systems [10, 9]. All layers except the Java GUI layer are implemented in C++.
The Java GUI (cf. Fig. 2) displays the robot arm and provides a command field with buttons and scroll bars. The robot is presented as a solid or wire frame model (like in other simulation systems [6]) with the current coordinates of the tool center point (coordinates of the robot's end effector) displayed at the bottom of the robot. In the command field the user can interactively move the robot and start or terminate a robot application program. Additionally the perspective and the direction of the robot arm may be changed by the user.
II.B Java 3D Interface The Java 3D user interface represents the latest version of the RoboSiM display (RoboSiM 2.0). The development of this GUI has started since Java 3D has been available in summer 1998. Compared with the first approach this presentation of the robot is much more realistic. Currently the Java 3D visualization is realized on a Windows NT PC (SPARC workstations with Solaris 2.6 are the alternative platform supported by Java 3D). The Java 3D interface (cf. Fig. 3) displays the same robot arm as the simple Java applet. In the same way the robot may be presented as solid or wire frame model. The options of the command field are adopted from the first GUI.
III. SYSTEM ARCHITECTURE The robot simulation and monitoring system is based on an existing open object-oriented robot control running on SPARC multiprocessor workstations or PCs. Detailed descriptions of the control may be found in [5, 13]. Therefore RoboSiM reuses components of the robot control system, however the visualization components of the simulation and monitoring are additionally developed.
Speck, Klaeren / 3
Fig. 3. Java 3D GUI (RoboSiM 2.0)
• Device and Simulation Layer The lowest layer consists of real devices and simulated devices. Both real and virtual devices may exist in the same RoboSiM system, which means that real as well as simulated robots can be visualized at the same time. The real devices are connected to the Communication Layer via the field-bus CAN (Controller Area Network). The Simulation Component is executed on the control hardware. Instead of a field-bus, shared memory is used to connect the simulation devices with the Communication Layer. • Communication Layer This component is responsible for the communication with the real and simulated devices. The Communication Subsystem cyclically receives and transmits the messages from and to the device. It reads the received messages from the real or virtual devices and stores them in a receive buffer (a shared memory region which can be accessed by the Application Tasks and Interconnection Task). After the reception of all messages the Communication Subsystem sends the messages of the transmit buffer (also a shared
Paper Id: 275
memory region which can be accessed by the Application Tasks and Interconnection Task) to the devices. Besides the exchange of data with the devices the Communication Subsystem synchronizes all the other components.
The Graphical User Interface component is located at the highest layer. Due to the socket connection this component may run on another computer than the control system. With this interface the user has direct access to the robot control. The robot may be manipulated by pushing the appropriate buttons on the screen.
IV. EXPERIENCES In this section we would like first to compare the two versions of the RoboSiM visualization. Secondly we discuss the alternative of using video pictures for process visualization and finally introduce the platforms on which RoboSiM has been tested.
IV.A Java 1.1 Comparison between the Java and the Java 3D Visualization Currently both the Java 1.1 and the Java 3D GUIs may be regarded as equivalent. Some requirements are better met by the one version while other necessities are fulfilled by the other system. The most important differences between the two visualization variants are:
Fig. 4. Simulation and Monitoring System Architecture
• Application Task Layer The Interconnection Task and Application Tasks are two components on the same level. Both use control components (which provide the actual control methods) to control and communicate via the Communication Subsystem with the devices. However they have different functionality. The Interconnection Task is a special type of application. It connects the Graphical User Interface with the Communication Subsystem. The Interconnection Component receives commands from the Graphical User Interface and interprets these which are either requests for manipulating a special device (e.g. move the robot arm) or requests to start or terminate Application Tasks. Bidirectional stream sockets are used as connection between the Interconnection Task and the Graphical User Interface.
Speck, Klaeren / 4
• Currently Java is more portable than Java 3D. Development systems and virtual machines for Java 1.1 are currently running on most platforms. Moreover Java 1.1 is supported by almost all web browsers. At the moment Java 3D development environment is only provided for Solaris and Windows NT systems. However Sun has announced to build a Linux version of Java 3D developers kit. Some browsers like the Netscape may be extended by a Java 3D plugs-in in order to execute these applets. • Java costs less performance than Java 3D. On a Windows NT PC (AMD K6II 3D with 64 MB) the Java interface causes considerably less than 50 while a Java 3D uses the full performance available. • Compared with Java 3D, Java 1.1 supports only very limited graphical features. Java 3D provides different rendering modes, a scene graph based programming model, methods for the three dimensional visualization of geometric objects like spheres, cubes and motion interpolation methods. Due to the superior quality of the Java 3D device visualization it may be expected that most systems will be based on Java 3D. The disadvantages of a higher consumption of resources and the limited portability will be overcome by the permanently increasing computer
Paper Id: 275
power and additional virtual machines on more platforms.
IV.B Java 1.1 Video Visualization of Devices Despite the visualization with proprietary systems (cf. section I.B) the video visualization is an alternative to the Java system. Compared to all graphical solutions the video pictures provide the most realistic representation. The installation of a camera is very simple and may be used completely independent from a control system. Video pictures may be transported via networks to remote monitoring station. Unfortunately the transmission of video pictures costs a high network bandwidth. When moving pictures have to be transmitted the load of standard networks such as Ethernet is extreme high and via Internet it is very expensive to transport moving video pictures. Another disadvantage of a video camera is that it can not visualize sensor data or the states of I/O units. A Java GUI displays such data much better. Due to these disadvantages a pure video process control seems to be not satisfying. The Java GUI displays selected data out of the incoming stream of signals which makes it easier for the user to control the process. Moreover the transfer of data to a Java applet costs only very little bandwidth.
IV.C Java 1.1 Platform Independence The Java 1.1 RoboSiM version has been tested on different platforms. The GUI applet is supported by the Microsoft Internet Explorer and Netscape Navigator browsers. The Netscape browser has run on different platforms such as HP Workstations, IBM PowerPCs, Silicon Graphics Workstations and PCs with Microsoft Windows NT and Windows 95/98 or LINUX). The simulation and monitoring system has been tested on different types of SPARC workstations and PCs (with Windows NT or Linux as operating system). The platform independence of the RoboSiM GUI enables to graphical interface be exported to remote hosts via Intranet or Internet. This allows to use the user interface for various purposes such as the visualization at the control cabinet itself on teach boxes connected with the control via a TCP/IP connection or on a remote monitoring or programming station.
Speck, Klaeren / 5
RoboSiM Java Robot Visualization
Visualization on a remote Computer e.g. via Intranet
Visualization on the real Control System
Visualization on a Teach Box
Fig. 5. RoboSiM
Due to the fact that the communication protocol is TCP/IP the connection between the control and a remote computer is rather slow. The average response time in an Intranet (within one 10 MBit/s Ethernet segment) is less than 2 s. The response time of an Internet connection (Reutlingen to Tübingen with a 2 MBit/s connection, distance 10 miles) is between 1 s and 10 s. However this delay is not caused by the net communication time. The time is lost by the processing of the TCP/IP protocol stack.
V. CONCLUSION The robot simulation and monitoring tool RoboSiM provides two versions of a Java based robot visualization: Java 1.1 and Java 3D which may be regarded as successor of the first one. The advantage of the Java 1.1 GUI is that it costs less performance than the Java 3D applet. Additionally Java 1.1 is available on much more platforms than Java 3D. Hence the Java 1.1 user interface may be the more common solution at the moment. In near future however Java 3D will support the same systems than Java 1.1 today. Moreover Java 3D allows to build a GUI which may be regarded equivalent to today's visualization (e.g. RobCAD). The RoboSiM user interface may be used for different applications such as education purposes or as GUI of an industrial control. Both Java GUIs may be executed on the local control system as well as on remote monitoring sites. Therefore RoboSiM allows to gain profit from all possibilities given by the Intranet or Internet, e.g. remote control or remote maintenance. The simulation and control parts of RoboSiM are implemented in C++. If powerful Java processors will be available the entire system may be implemented in Java. This would lead to a complete new generation of control systems based on a language which supports portability
Paper Id: 275
and distributed computation in a way that has not been reached yet.
[11] H. Sowizral, K. Rushforth and M. Deering, The Java 3D API Specification, Addison-Wesley, Sydney, Wokingham, Reading MA: 1998
V. REFERENCES [1] The HotJava Browser: A White Paper, Sun Microsystems: 1995. [2] The Java Language: Microsystems: 1995.
A
White
Paper,
Sun
[3] P. Arjanne and J.J. Suutarinen, "Fully automatic offline programmable robotized press brake controlled with two transputer based multi axis controllers," in Proceedings of 25th, Int'l Symposium on Industrial Robots, Robotics '94 Flexible Production Flexible, Hanover, Germany, 1994.
[12] A. Speck, "Robot Simulation and Monitoring on Real Controllers (RoboSiM)," in Proceedings of ESS'98, 10th European Simulation, Symposium and Exhibition, SCS, Nottingham, UK, 1998, pp. 481487. [13] A. Speck, G. Gruhler and W. Küchlin, "ObjectOriented Robot Control Framework," in Proceedings of IECON'98, The 24th Annual Conference of the IEEE Industrial Electronics Society, IES, Aachen, Germany, 1998, pp. 16631666.
[4] J. Bickendorf, "Full-Automatic Off-Line Programming of Complex Cutting Paths - A Contribution to the Economic Production of "Lotsize 1"," in Proceedings of 25th, Int'l Symposium on Industrial Robots, Robotics '94 - Flexible Production Flexible, Hanover, Germany, 1994. [5] W. Küchlin, G. Gruhler, A. Speck and T. Lumpp, "HIGHROBOT: A High-performance Universal Robot Control on Parallel Workstations," in Proceedings of ECBS'97, Int'l IEEE Symposium and Workshop on Engineering of Computer Based Systems, Monterey, pp. 444-451. [6] C. Laloni and E. Wahl, "Principles of Robot Simulation and their Application in a PC-based Robot Simulation System," in W. Straßer and E. Wahl, editors, Graphics and Robotics, Springer, Heidelberg, Berlin, New York, 1995, pp. 1-30. [7] R. Maglica and N. Martenson, "Teaching Robot Programming Using CAL," in Proceedings of 26th, Int'l Symposium on Industrial Robots, Robotics '95 - Flexible Production Flexible, Singapore, 1995, pp. 111-116. [8] P.J. McKerrow, Introduction to Robotics, AddisonWesley, Sydney, Wokingham, Reading MA: 1995 [9] D. Morris, G. Evans, P. Green and C. Theaker, Object Oriented Computer Systems Engineering, Springer, London: 1996. [10] B. Selic, G. Gullekson, and P.T. Ward, Real-Time Object-Oriented Modeling, Wiley: 1994.
Speck, Klaeren / 6
Paper Id: 275
