A multimedia environment for supporting the teaching of ... - CiteSeerX

Proceedings of the Sixth International Workshop on Multimedia Network Systems and Applications" (MNSA2004), held during the 24th International Conference on Distributed Computing Systems (ICDCS), 2004, ISBN: 0-7695-2087-1, pp. 280-285

A multimedia environment for supporting the teaching of robotics systems José C. Freire Jr São Paulo State University, CP 205, 12516-410, Guaratinguetá, SP, Brazil [email protected] Rodrigo A. Neves São Paulo State University, CP 205, 12516-410, Guaratinguetá, SP, Brazil [email protected]

Abstract This paper aims at describing the building of an educational system for teaching and learning robotics systems. Actually, multimedia resources are used to construct a virtual laboratory where users are able to utilize functionalities of a robotic arm, by moving and clicking the mouse without caring about the detailed internal robot operation. The engineering students are its target public. The final product was a Virtual Laboratory prototype of a teaching and learning environment that can be used to study robotics topics, to manipulate a virtual robot and to communicate with a real one. It socializes the students’ synthesis based on questioning and contributions, clarifying doubts and posing new questions about robotic systems. First, this paper introduces the metaphor of Virtual Laboratory used in the application. Next, it is described the Graphical and Multimedia Environment approach, an interactive graphic user interface with a 3D environment for simulation. Design and implementation issues of the real-time interactive multimedia learning system, which supports the W3C SMIL standard for presenting the real-time multimedia teaching material on the class, are used and described. With content-sharing and interactive capabilities provided by this system and its tools, students can devote themselves on the learning process just as they are in the traditional face-to-face classes. Finally, we mention the conclusions and possible future works from this research.

MNSA-2004

José V. de Lima Federal University of Rio Grande do Sul, CP 15064, 91501-970, Porto Alegre, RS, Brazil [email protected] Galeno J. Sena São Paulo State University, CP 205, 12516-410, Guaratinguetá, SP, Brazil [email protected]

Keywords: Teaching and Learning Robotics Systems Environment, Multimedia System, Virtual System, Java 3D, Robot’s control.

1. Introduction Education and research in the area of didactic robots have become very popular [1] [2]. New developments and improvements in hardware technology have made didactic robots [3] [4] pertinent to a growing number of problem domains. Following this trend, education in robotics related disciplines has found its way into many engineering, computer science and cognitive science curriculum. From an education standpoint, the ability to offer students access to didactic robots related courses and materials enables the student to participate in a number of practical areas, including engineering, computer programming, artificial intelligence, project management and technical writing. Engineering is about solving problems of practical import, and traditional classroom instruction alone may not be sufficient to educate a practicing electrical or computer engineer. Typical classroom exercises are usually designed to illustrate a point or demonstrate a technique rather than to approximate problems one will really face. Robotics is the science that seeks to forge an intelligent, computational connection between perception and action. In this project, we use robot algorithms to solve tasks in robotics arm, manipulation, simulation and control in order to integrate these algorithms in a multimedia environment for teaching and learning robotics systems. In developing the

1


project, it was also of our concern the use of Educational Technologies [5] in Engineering Courses, considering the importance that investigations related to the use of new technologies in Education have nowadays. In the work [6], for instance, it was emphasized the importance of the development of Hypermedia tools to support the teaching activities in subjects of the Engineering. Other examples of tools related to the teaching of Engineering subjects were presented in [7], [8]. The main objectives of this work were to develop a multimedia environment able to communicate and interact with a real robot arm for didactic use in laboratory activities of engineering students. It presents a multimedia system, developed with the Java language [9], which provides a dynamic environment for teaching and learning concepts on robotic systems and communication of a PC microcomputer with external devices. Such system, to have increased their qualities, should be used with an electro-mechanic robotic arm also developed by the authors and presented in [10]. Concisely, this electro-mechanic robotic arm (nowadays almost on plain function) is composed of step motors and its respective drivers and sensors, assembled in a plate and connected to the PC parallel port. Through the parallel port, the system dynamically interacts with a robotic system under student control, following a preprogrammed content or by means of an application developed by the students. To do this, the system must activate devices like step motors, and read the status of sensors connected to parallel port.

2. Multimedia environment 2.1. The graphical presentation Visualizing a simple virtual world can be very difficult in a 2D projecting monitor. 3D graphic tools that have been developed and are being enhanced every day create a more realistic interface. Some of these environments are more dedicated to high precision and dedicated modeling like ACAD [11] and Solid Works 3D Modeling Software [12]. Some other like VRML [13], OpenGL [14], and Java 3D [15] are more graphics oriented. High precision 3D modeling and design software are usually more expensive, while some graphics oriented environments are available through the Web free of cost. OpenGL, VRML and Java 3D are three examples for this case. Yet OpenGL is a C++ graphics library, and therefore the final application would be platform dependent. VRML is platform independent, and it runs usually as a plug-in to standard web

MNSA-2004

browsers (Microsoft Internet Explorer, and Netscape Navigator). The plug-in is available for most systems, and can be downloaded from the Internet for free. The most widely used plug-ins are Cosmoplayer [16], and Blaxxun [17]. Both support Microsoft Internet Explorer and Netscape Navigator, and they have common standard functionality, but different interface. They both have some non-standard features that add to their capabilities. Another platform independent environment is Java 3D. The Java 3D 1.1 API is a set of classes for writing three-dimensional graphics applications and 3D applets being an addition to Java for creating and presenting three-dimensional graphics. Programs written with Java 3D, can be run on several different types of computer and over the Internet. The Java 3D class library provides a simpler interface than most of other graphics libraries, but has enough capabilities to produce good applications and animations. Application developers can describe very large virtual worlds using these constructs, that provide Java 3D with enough information to render these worlds efficiently. Java 3D builds on existing technology such as DirectX [18] and, as specifically in this case, OpenGL [14], so the programs do not run so much slowly. Also, Java 3D can incorporate objects created by 3D modeling packages likes VRML models.

2.2. The development language The widespread acclaim that Java is receiving in both academic and industry is well justified. The language cannot truly be said to incorporate the very latest advances in programming language theory, but nonetheless it manages the rare feat of providing a base for robust, stable programs while giving programmers many casual similarities to C++. As a programming language, Java eliminates many of the criticisms by the previous industry language standards, C and C++. For example, Java makes better the support for threads and runtime exceptions. It compiles to a platform-independent bytecode that makes porting to other hardware nearly trivial. It is fully object-oriented, not hybrid like C++. Another remarkable point refers to Memory Management. Java programs do not care about memory. The Java Virtual Machine (JVM) provides applications with all the memory they need. The JVM handles the nuances of obtaining more memory from the underlying host operating system. Java programmers do not even need to concern themselves with returning memory they no longer need to the system, because of the JVM's automatic garbage

2


collection mechanism, which automatically deallocate objects with no more references. One of the main concerns in the development of this system was to minimize its implementation cost. Taking into account this concern and its excellent techniques characteristics, the Java programming language was chosen to develop the multimedia system. This choice led the API Java 3D utilization for build the graphical simulation environment (the robotic arm).

2.3. The developed multimedia system As can be noted in Figure 1 (a detail of Figures 2), the developed tool has 4 tabs that offers different utilization possibilities. The ‘Robot Arm Simulation’ tab permits access to the 3D arm simulator. The ‘Dynamic Presentation’ gives a presentation about the robotics developed under the SMIL technology (described below). In the ‘Static Presentation’ tab the user can find topics related to robotics in a static way. Finally the ‘Notepad’ tab is a simplified text editor that could be used as a notebook by the students during the use of the multimedia system.

Figure 1 – Multimedia system tabs possibilities. 2.3.1 Robot Arm Simulation. The generated virtual world, the ‘Robot Arm Simulation’ tab, enables the user to continuously interact with a graphical 3D representation of the robot, to observe its reaction to commands inputs, and to understand its movement behavior and interaction with the others objects in the scene (Spheres, Cubes and etc). In addition, the system includes several tools to help the user during the stage of developing a new robot sequence commands, and then analyze to simulate all the structure. To assist the user in building simulations, the user can access the Static Presentation tab where he can find a lot of information about robotics systems. Furthermore, using the real time integrated numerical solver, we allow dynamic changing of link parameters. This means that the user can change any link parameter (for example extends it) during the simulation. For this purpose we have created a special dialog box to adjust the parameters of a link (e.g. the right panel on Figure 2). This allows the user to see directly the influence of a parameter on the structure. The purpose of the virtual

MNSA-2004

world, however, is to go further than just presenting reality to the user: we can add 3D graphic information representing things the user cannot see normally. For example, we can show robot internal parameters such as joint speeds or accelerations, the load on a link, the overheating of a ball bearing, or external measures (e.g. distance to an obstacle). Using adequate representations (changing form or size of an additional object, color or texture of a link), we can ‘attach’ the information about one parameter to its associated mechanical part. Thus the user can obtain a direct feeling visual about the behavior of the system and the relations between the different parts. In Figure 2 we present a snapshot of the Robot Arm Simulation in the multimedia tool.

Figure 2 - Environment of simulation 3D. 2.3.2. Robotics System Dynamic Presentation. Rich and interactive multimedia applications, where audio, video, graphics and text are precisely synchronized under timing constraints are becoming ubiquitous. Multimodal applications further extend the concept of user interaction combining different modalities. However, authoring dialog-capable multimodal, multimedia services is a very difficult task. Fortunately, the W3C has sponsored the development of SMIL [19], an elegant notation for multimedia applications, which has been embraced by several enterprises like Microsoft or Real Networks. SMIL already accommodates a feature often considered essential for interactive multimedia applications: the ability to rewind a presentation according to synchronization points. The emphasis of SMIL is on defining hyperlinks so that user of a media player may jump into the middle of presentations. The multimedia environment integrates a dynamic presentation (e.g. Figure 3) of robotics system topics. This presentation can be started in the ‘Dynamic Presentation’ tab and shows lessons, developed with

3


SMIL technology, about robotics systems. Actually this presentation is made by using the RealOne Player [20]. We intend to integrate a SMIL presentation player inside our Java application using the X-Smiles API [21] developed at Helsinki University of Technology.

In our project to implement the ‘multimedia environment/real arm’ communication, the simulator module must activate the registers of the parallel port of the computer (PPI) [24]. Due to the problems presented above, we chosen to implement this communication using a direct interaction with the firmware. Knowing that the objective is to send and extract data from PPI, our efforts were oriented in this line, not adding any other type of functionality on the driver. A windows library able to establish communication with the driver and return the useful parameters was developed, focalizing a future union with Java applications (e.g. Figure 4). The library uses the driver’s capabilities to interact with the firmware, manipulating it to obtain the desired information. Java Application 1 Java Application 2 Java Application 3

Library

Driver

Parallel Port

Figure 4 - Driver Linkage. Figure 3 - Smil teaching and learning integrated environment. 2.3.3. Robotics System Static Presentation. The multimedia environment also includes a set of HTML pages, the ‘Static Presentation’ tab that presents the robotics system topics, like in the SMIL presentation. These set of pages are presented in an integrated browser that can be reached by the ‘Static Presentation’ tab. Besides the robotics system content, in these pages the students can access some practical exercises either to simulate in the 3D simulator or to activate in the real arm. 2.3.4. Interaction with the real Robotic Arm. The Multimedia environment will interact through the parallel port with a real robotic arm [10]. Dealing direct with the hardware is always a delicate task, knowing the vast quantity of possible problems, which varies from simple software requisition conflicts to system’s memory violation. To protect the system and increase its stability, Microsoft implemented in its the Windows 2000, a privilege level scheme inherited from Windows NT. This scheme has only two I/O privilege levels, level 0 and level 3. User mode programs will run in privilege level 3, while device drivers and the kernel will run in privilege level 0, commonly referred to as ring 0. As result, the I/O can be touched only by the operation system and drivers, not allowing any user to read or write it. All user mode programs should talk to a device driver [22], [23] which arbitrates access.

MNSA-2004

4. Discussion and Related Work Much work has been done in the domain of construction of synchronized didactic materials. Rousseau et al. have defined one generic solution for user adaptation of synchronized multimedia presentations [25]. The adaptation was considered as a transformation problem in which the user specifies a predicate that being applied to a generic multimedia presentation yields a customized view of presentation. Holtman et al. have defined extensions to HTTP to handle transparent content negotiation [26]. The extensions allow the client to specify which version of a given object a server should deliver. Similar functionality has been proposed by Kamada and Miyazaki [27]. Fox et al. have developed a distillation proxy that degrades several media types (images, video, postscript) to decrease the download time [28]. Our work aims at synchronizing the medias as didactic material, without worry with the media adaptable problem. However, we are studying these problems for a future work development. Thus, we have proposed here a virtual laboratory capable to show the synchronized operations with a robotic arm. Also, the presentation could be made by synchronizing videos, slides and operations with the robotic arm functions. In its current state, the system has been used to support the teaching activities of the subject matter of the Electrical Engineer Course in São Paulo State University at Guaratinguetá. With respect to this use, specific evaluation instruments will be developed and

4


applied, so that one can have feedback concerning its impact on the improvement of the teaching of that subject matter.

robot arm. A 3D model of the robot, as well a video camera, enables the user to visualize and control the motion of the arm.

5. Conclusions and Future Plans

6. References

This work demonstrates that it is possible to build a real time simulator, using appropriate methods, for general and specific robot structures. In addition, using a virtual environment interface enables users to have direct interaction with simulated robot. This allows the user to ‘feel’ the simulated structure, and to act directly in the control loop. It should be noted that the program developed is not intended to compete with mechanical modeling software (which allows analysis of kinematics, dynamics, etc.), but rather is a supplementary tool for didactic robot simulation and control. The current implementation uses an open architecture, which easily allows new ‘plug-in’ modules, and will be available for distribution to interested research groups as soon as possible. Future implementations like a camera system to transmit streams broadcasting on the Internet should be employed to make the environment a really powerful tool on teaching and learning robotic systems. Then, after having a fully functional Multimedia Environment for Teaching and Learning Robotics Systems we intend to develop a web-based environment that requires platform independent software or server based applications. An example of a such system with Internet connection is shown in Figure 5.

[1]

Rohrmeier, M; Web Based Robot Simulation Using VRML, in: 2000 Winter Simulation Conference, WSC 2000, Orlando/USA, 2000, pp. 1525-1528.

[2]

Dunn T, L; Wardhani, A; A 3D Robot Simulation for Education in: 1st International Conference on Computer Graphics and Interactive Techniques in Austalasia and South East Asia, Melbourne/Australia, 2003, pp. 277-278.

[3]

Robix RCS-6: Robot http://www.robix.com/.

[4]

Lynxmotion - Lynx 5 http://www.lynxmotion.com/.

[5]

Akamatsu, J. I.; Sena, G.J. Tecnologias de Aprendizagem para o Ensino de Engenharia In Loco e à Distância. Ouro Preto, MG, in: Proc. of the XXVIII Congresso Brasileiro de Ensino de Engenharia COBENGE'00 (in CD-ROM), 2000.

[6]

G. J. Sena; R. A. C. Silva; O. C. Winter; F. T. Mitsuyassu; E. L.C. Moura; M. A. F. Barbosa. Hypermedia-based Tools: Development and Usage to Support Teaching Activities. Rio de Janeiro-RJ, in: Proceedings of the ICECE: Int’l Conf. on Engineering and Computer Education, 1999, pp. 330-334 (in CDROM: paper no. 502).

[7]

G. J. Sena, R. M. Bittencourt, O. P. B. Teixeira, A. D. Velasco, M. A. R. F. Gonçalves, J. I. Akamatsu. Desenvolvimento de um Software Multimídia para o Ensino de Tópicos de Eletrônica Básica. Natal, RN, in: Proc. of the XXVII Congresso Brasileiro de Ensino de Engenharia - COBENGE'99, 1999, pp. 318-325 (also in CD-ROM).

[8]

Sena, G.; Alves, M. N. An Induction Heating Software to Simulate Special Heat Treatments. in: Proceedings of the International Seminar on Heating by Internal Sources – HIS-01, ISBN: 88-86281-64-1 Padua, September 12-13-14, 2001.

[9]

The Source for Java Technology - http://java.sun.com/.

Figure 5 - Robotics system teaching and learning web-based environment. This figure presents a server connected to the robot arm. The server has access to the Robot I/O ports. The user can login to the control server from any computer via Internet. The web based control software is loaded at the user’s computer, and the user can control the

MNSA-2004

Construction Robotic

Set Arm

Kit,

[10] Kina, R. B. Relatório da Bolsa de Iniciação Científica FAPESP – Desenvolvimento de um Braço Robótico para Utilização Didática. UNESP/FEG, 2003. [11] Frey, D; AutoCAD 2002: No Experience Required, Sybex, 2001. [12] SolidWorks Office Professional, http://www. solidworks.com/html/Products/swofficeprof.cfm.

5


[13] The Virtual Reality Modeling Language http://www.web3d.org/technicalinfo/specifications/ISO _IEC_14772-All/index.html.

[23] Microsoft Driver Development Kit documentation: http://www.microsoft.com/ddk

http://www.opengl.org/users/about/

[24] Beyond Logic – Portalk, http://www.beyondlogic. org/porttalk/.

[15] Java 3D API - http://java.sun.com/products/javamedia/3D/.

[25] Rousseau F., García-Macías J. A., Lima J. V. And Duda A., User Adaptable Multimedia Presentations for WWW, Proc. WWW-8, Toronto - Canada, 1999.

[14] OpenGl, index.html.

[16] Cosmo Player - http://www.cosmosoftware.com/

[26] Holtman K. and Mutz A., Transparent Content Negotiation, In IETF RFC 2295, ftp://ftp.isi.edu/innotes/rfc2295.txt.

[17] Blaxxun Technologies, http://www.blaxxun.com/ [18] Microsoft DirectX, http://www.microsoft.com/ windows/directx/default.aspx. [19] Synchronized Multimedia Integration Language (SMIL 2.0) Specification’- W3C Working Draft 01 March 2001 - http://www.w3.org/TR/smil20/ [20] ReanOne index.html.

Player,

http://www.real.com/realone/

[21] X-Smiles, http://www.xsmiles.org/index.html. Open XML Browser for Exotic Devices.

Na

[27] W3C, Client-Specific Web Services by Using User Agent Attributtes, W3C NOTE –agent-attributes19971230, http://www.w3.org/TR/NOTE-agentattributes-971230.html [28] Fox A. And Brewer A., Reducing www latency and Bandwidth Requirements by Real Time destiollation, Proc. 5th International WWW Conference, May, Paris France, 1996.

[22] Windows Developer Magazine - Sending IOCTLs to Windows NT Drivers: http://www.wd-mag.com/

MNSA-2004

6