VECIMS 2004 – IEEE International Conference on Virtual Environments, Human-Computer Interfaces, and Measurement Systems Boston, MD, USA, 12-14 July 2004
Development of Man Machine Interface Software For an Industrial Robot E. Ilhan Konukseven1, Anas Abidi2 1
Department of Mechanical Engineering, Middle East Technical University 06531 Ankara, Turkey Phone: +90-312-2105242, Fax:+90-312-2101275, Email:
[email protected] 2 Department of Mechanical Engineering, Middle East Technical University 06531 Ankara, Turkey Phone: +90-312-2105242, Fax:+90-312-2101275, Email:
[email protected]
Keywords – Robotics, Computer Aided Graphics and Animation, Collision Detection, Serial Port Communication, Robot Programming
makes a great economic sense when applied to automated factories. A greater percentage of production level can still be maintained since the robot (production equipment) will not necessarily be tied down when the need for reprogramming or testing arises. Also the verification of early object collision detection within simulation systems provides a higher security and can prevent serious damages to real robot and its environment. In this paper the development and implementation of a Man Machine Interface Software for an Industrial Robot is described. Through this software application it would be possible to program an industrial robot on-line or off-line. This software is developed as open source software. This would allow further development by adding or modifying the source code of the software.
I. INTRODUCTION
II. SOFTWARE DESIGN
Industrial Robots have interfaces (man machine interfaces) to allow communication with them. These man machine interfaces can be either hardware or software. Through a robot’s man machine interface it would be possible to control and program the robot to perform a specific task. Programming robots to perform specific tasks has been one of the main aspects in the field of robotics research. Robot programming can be classified into two main categories,
In this section we describe the design of the software package called “Man Machine Interface (MMI) Software” which handles the visual simulation, collision detection (in virtual environment) and control of the ABB IRB-2000 robot.
Abstract – This paper describes the design and implementation of a man machine interface software for the ABB IRB-2000 industrial robot. The main aim of this software is to provide features which would facilitate on-line/off-line programming of the robot. The software would provide a three dimensional graphical simulation of the robot and its environment, which also allows loading custom designed objects into the simulated environment. The software also offers collision detection capability between graphically simulated objects, an interpreter/ editor to interpret/write robot programs in BASIC like language enhanced with robot commands. The facility to control the actual robot through the computer’s serial communication port is also provided.
1. 2.
On-line Programming Off-line Programming
In On–line programming the actual robot is utilized while the program is being created. In Off-line programming, the program is done independent of the robot. After developing an off-line program, it is loaded to the control system of the robot for execution. The robot programs done off-line can be tested and debugged on a graphical simulated model of the robot before being executed on the actual robot itself. The wisdom of this lies in the fact that possible damage to the robot or to objects in the robot environment can be averted in the event of any mishap arising from undetected bugs in the programs [1]. The practice of off-line Robot Programming and Simulation also
The simulator in the MMI software calculates the Forward Kinematics and implements the Inverse Kinematics for the robot. The 3-D Graphical Display and User Interface allow users to observe and interact effectively with the ABB IRB2000 simulation. At the same time the whole process can be saved, stopped and repeated. The modules comprised in the system are: 1.
2.
3. 4.
5.
A virtual environment graphics simulation environment module for simulating and executing the robot programs and graphically performing the robot operations on the screen. A Wavefront Obj File Format generic parser module for parsing and displaying 3-D models in Wavefront Obj File Format [2] into the graphical simulator. A RS232 communication module which allows the software to communicate with the actual robot. A collision detection module for detecting any physical interactions with the environment during the movement of the robot arm. A robot-language interpreter module which interprets the source robot programs.
The MMI also provides a program to provide the robot programmer with a text editor, debugging tool for writing robot programs. A. Graphical Simulator The graphical simulator engine in the software was done by using OpenGL. In order to use OpenGL in Visual Basic environment, which is the programming language used for developing the system, an ActiveX control [3] called ‘glxCtl’ was used. This ActiveX control is a library to support higher level routines for OpenGL, handles the setup of OpenGL window and fires various events. In our virtual environment of the ABB IRB-2000, each joint of the six joints of the robot is accurately modeled as a Wavefront Obj File. The integration of these modules into the Virtual Environment is done by using the parser module. The simulator also allows the user to integrate additional user defined 3D models in the simulated environment. The modeling of the robot has two operational modes for the robot’s motion: Robot Coordinates where the robot moves based on its joint coordinate; the other is the in Rectangular Coordinates where the gripper of the robot can be moved in X, Y, Z world coordinate and its orientation is in Euler 321 sequence. Collision detection is coupled with the simulator to give warnings of possible collision errors in the simulation. It will automatically stop further execution of the robot motion once interaction is detected to ensure the safety of the robot and its environment. The system also has a status display capability which enables the user to debug and interact with the graphical simulator. B. Robot Language The Robot Language is based on Microsoft’s Script Control. This control acts as an engine to interpret and execute VBScript code (VBScript is very similar to BASIC language, for further information on how to write VBScript code, refer to [4]). This code can be executed at run time of an application, because of this ability it is possible to load and execute programs at the run time of the MMI software. Besides executing VBScript code, the software offers the flexibility to the user to define new commands and add them to the interpreter (the Script Control). For example the command ‘move x, y, z, t1, t2, t3’ was defined in the software, by using this command it is possible to move the robot’s hand to a specified position and orientation. The command takes x, y, z, t1, t2 and t3as inputs. The values (x, y, z) define the position and (t3, t2, t1) define the orientation in Euler 321 angle sequence. Fig. 1 shows an example of a VBScript Code with the move command. for i = 0 to 360 step 5 move(950,150+i,150,0,0,0) next Fig. 1. A VBScript Code with ‘move’ Command.
As it is clear in the above Figure by using VBScript it is possible to write high level routines easily. Programs written in VBScript are either executed on the real robot by using the interpreter module and RS232 module or on the virtual simulation by using the interpreter module and the graphical module. C. Collision Detection The ability for checking collision detection in the graphical simulation would allow us to check for any interactions between the Robot links, Robot environment and objects which are placed in that environment. The collision detection module is based on the Bounding volume hierarchies (BVHs) which are one of the simplest and most widely used data structures for performing collision detection on complex models. The type of bounding volumes used is the Oriented Bounding Boxes (OBB) and Convex Hulls. Oriented bounding boxes fit their underlying geometry as tightly as possible and are quite fast for detecting collision between two objects. For this reason OBB-tree collision detection algorithm was used. The algorithm used is based on the discussion forwarded by [5]. D. Software Structure The software developed is made of a graphical user interface (GUI) which is based on eight main objects defined through classes. The eight main classes are, • cObjReader: Through this class it is possible to parse (read) Wavefront’s object file format files (.obj) for solid models. • cVSimulation: This class handles all the graphical operations required by the software. Solid models’ information is loaded into this class. The output of this class is the graphical scene of the robot and its environment. • cAbbKinematics: This class calculates the forward and inverse kinematics solution for the ABB IRB-2000 robot. • cModel: This class builds a collision model for a given solid model. The solid model should be in the form of a triangular mesh. • cCollideResult: This class performs collision checks for given two collision models along with their orientation and position in world coordinates. • cRT_Abb: Through this class it is possible to give commands to the ABB IRB-2000 robot through the serial port. • cQuaternion: This class handles all the required calculations for converting / obtaining Euler angles in the sequence 321 to/from quaternion, or rotation matrix to / from quaternion. • cCompiler: By using this class it is possible to load programs for the robot. These programs can be off-line
(executed by the virtual robot) or online (executed by the real robot).
into the system (Fig. 2); and phase two: where the user can interact with the system (Fig. 3). Fig. 4 shows the finished software’s interface.
The running of the Software is divided into two phases: phase one, where all the models are initialized and loaded
.obj files
cVSimulation
Base.obj Link1.obj Link2.obj Link3.obj Link4.obj Link5.obj Link6.obj Floor.obj
parse
cObjReader
‘TModel’ type variables
cModel ( Collision Models Constructed )
Collision Models
Graphical User ( the visual Interface model of ( View Scene robot and Window ) environment is constructed ) Position and Orientation of all models cCollideResult
Fig. 2. Phase 1 of the MMI Software.
User
Robot Joints’ angles Robot Hand’s Position (X, Y, Z)
GUI Updated or Unchanged Scene + Errors (if any)
Robot Hand’s Position (X, Y, Z)
Robot Joints’ angles cVSimulation (Check Joints’ angles limit)
cAbbKinematics
Rotation Matrix
Rotation Matrix
Robot Hand’s Orientation ( Euler 321 angles)
Robot Joints’ angles (calculated by using inverse kinematics)
Message
Collision Models’ Position and Orientation
cCollideResult (Check for Collision)
cQuaternion Robot Hand’s Orientation (Euler 321 Angles)
Fig. 3. Phase2: Red Arrows executed when Robot is in Robot Mode, Blue Arrows when Robot is in Rectangular Coordinates Mode, Black Lines are Executed in Both Modes
III.
SAMPLE RUN
A program was written to move the robot’s hand in a semicircular motion in the YZ-plane, the semi-circle’s radius is 150 mm. The program is shown in Fig. 5. The figure also
shows the editor debugger which is used to write and debug the programs written for the MMI software. The execution of the program in the graphical simulation is shown in Fig. 6.
Fig. 4. MMI Software’s Interface
Figure5. A Program to Move the Robot’s Hand in a Semi-Circular Motion
IV.
CONCLUSION
An experimental on-line off-line graphical robotprogramming simulator has been successfully designed and implemented for the ABB IRB-2000 Industrial Robot. This developed system is in fact an integrated robot-programming system. The system offers the following to the user:
Control of the industrial robot ABB IR-2000 on-line through a portable computer. Possibility to write off-line programs for the robot, debug them on a graphical simulation of the industrial robot and then execute them on the real robot. A Graphical User Interface (GUI) which is responsible for providing the user with a way to interact with the graphical simulator of the robot and its environment. User specified solid models can be imported into the simulator’s simulated environment. Detection and warning of any possible collision between the objects in the simulation.
The robot-programming language has been implemented as an editor-interpreter system, which is based on Microsoft’s VBScript. The language is not specific to any particular robot type, and all the commands are configuration independent. This frees the language-processor form the underlying robot hardware and preserves its portability to other robot systems. The graphics simulator allows the work environment to be viewed in different positions, angles and scales for a more thorough investigation of the operation. Moreover, programs may also be traced, interrupted and debugged with the facilities of the interpretation part of the simulation. In the future the software can be enhanced further by doing the following suggestions:
Collision detection capability between the robot and its environment in the MMI software can be extended to collision avoidance. Moving objects can be loaded in the simulated environment. A library of different robots can be incorporated into the software. External devices like sensors, haptic devices, etc… can be fitted in the system for feed back. This would enhance interaction with the external world.
b)
c)
d)
e) Fig. 6. Five Screen Shots from the ‘semi_circle’ Program Simulation
REFERENCES [1]
[2] [3] [4]
A library of different tools which can be fitted to the robots wrist can be added to the software, for different simulation purposes.
a)
[5]
[6]
[7]
Zhou, R. and Sha, X., “Development of An Off-line Graphical PUMA 560 Robot Simulator”, November 2002 http://www.cs.msstate.edu/~rzhou/-Simulator.htm Wavefront Obj File Format, http://www.mediatel.lu/workshop/graphic/3D_fileformat/files/obj_spec.pdf Edx, “Programming OpenGL with Visual Basic”, August 1999 1s6.pacific.net.hk/~edx/ VBScript User's Guide http://msdn.microsoft.com/library/default.asp?url=/library/enus/script56/html/vtoriVBScript.asp Gottschalk, S., “Collision Queries using Oriented Bounding Boxes”, PhD. Thesis, Department of Computer Science, University of North Carolina, Chapel Hill, 2000. Gottschalk, S., M. Lin and D. Manocha, “OBBTree, A Hierarchical Structure for Rapid Interference Detection”, Proceedings of ACM Siggraph, pp. 171-180, 1996. Gottschalk, S., “Separating axis theorem”, Technical Report TR96024, Department of Computer Science, UNC Chapel Hill, 199
