An Interactive Virtual Reality-based e-Education System Leith K. Y. Chan and Henry Y. K. Lau Department of Industrial and Manufacturing Systems Engineering The University of Hong Kong Pokfulam Road, Hong Kong, PRC Tel: +852 2241 5178 Fax: +852 2858 6535 Email:
[email protected]
Abstract A fully immersive and interactive visualization system that provides extremely vivid stereoscopic views of sceneries in 3-D is developed based on the technologies of distributed virtual reality and computer networking. The system, known as the imseCAVE, is a low cost, high performance generic Cave Automatic Virtual Environment (CAVE) system developed by the researchers at the University of Hong Kong for teaching and research. imseCAVE is a versatile and powerful virtual reality platform with the capability to provide a highly cost effective e-education media for teaching and learning. The system can be readily configured to provide virtual tours for students to obtain first hand experience such as deep sea and space exploration, journey inside human body, visit to the pyramid, etc. without the limitations of time and space. The system can also be tools for complex system analysis and design such as in the development of products, buildings, and other facilities. In addition, the computer generated images can be tailored made to produce any simulated environment for training and skill evaluation purposes within a short period of time. One of the novel features of the imseCAVE is its design and implementation. Low cost computer systems are clustered with high bandwidth communication network to achieve a fully distributed computing environment to realize the concept of distributive virtual reality. In addition, the application software of the system enables real and virtual sceneries to be seamlessly integrated to generate an environment by which application scenarios can be built in a highly flexible way. Furthermore, the imseCAVE can be readily interfaced to a host of input and output devices including 3-D tracker, joysticks and motion platform for real time interacting with the virtual environment. With the addition such devices, not only visual effect is reproduced, other sensation including motion and sound can be provided whereby enhancing the immersiveness of the overall system. In the Department of Industrial and Manufacturing Systems Engineering, the imseCAVE has been deployed in the simulation of very large and complex logistics systems including the automated air cargo terminals and port container terminals. Such simulations provide a means for students to study their operations without the need to physically visiting these facilities. More importantly, experiments can be performed on the virtual simulation systems whereby operation and design parameters can be changed in a safe and controlled environment. In addition, we also use the imseCAVE for engineering product design and visualization; and for setting up as training systems for machineries.
Keywords: Virtual Reality, Interactive, Immersive, Education, CAVE
1 Introduction At the Department of Industrial and Manufacturing Systems Engineering at the University of Hong Kong, an interactive virtual reality-based (VR) simulation and visualization system was developed called the imseCAVE based on the concept of the Cave Automatic Virtual Environment (CAVE TM ). The system provides users with 3-dimensional (3D) computer generated or video images with the capability of allowing the users to interact with these images in real-time through customized input devices such as joysticks and other movement capture sensors. This system is a versatile and powerful virtual reality platform with the capability to provide a highly cost effective e-education media for teaching and learning. The system can be readily configured to provide virtual tours for students to obtain first hand experience such as deep sea and space exploration, journey inside human body, visit to the pyramid, etc. without the limitations of time and space. The system can also be tools for complex system analysis and design such as in the development of products, buildings, and other facilities. In addition, the computer generated images can be tailored made to produce any simulated environment for training and skill evaluation purposes within a short period of time. P
P
One of the novel features of the imseCAVE is its design and implementation. Low cost computer systems are clustered with high bandwidth communication network to achieve a fully distributed computing environment to realize the concept of distributive virtual reality. In addition, the application software of the system enables real and virtual sceneries to be seamlessly integrated to generate an environment by which application scenarios can be built in a highly flexible way. Furthermore, the imseCAVE can be readily interfaced to a host of input and output devices including 3-D tracker, joysticks and motion platform for real time interacting with the virtual environment. With the addition such devices, not only visual effect is reproduced, other sensation including motion and sound can be whereby enhancing the immersiveness of the overall system This paper the system design, architectural, configuration and characteristics that differentiate the imseCAVE from other existing systems are presented. A number of interesting applications is highlighted and its benefits will be discussed in the following sections.
2 The imseCAVE Design and Architecture The CAVE environment, originally developed by the Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago, produces a 3-dimensional stereo effect by displaying in alternating succession the left and right eye views of the scene as rendered from the viewers perspective [2]. These views are then seen by the viewers through a pair of LCD shutter glasses whose lenses open and close at high frequency in synchronization with the left and right eye views that are projected via cathode-ray projectors onto the translucent walls acted as screens. Since the CAVE system provides a highly versatile platform for 3-D visualizing complex concepts and systems, it has been deployed to explore new statistical graphics applications [4], simulate complex molecular dynamics and interactions between atomic particles [3], virtual exploration and analysis of archaeological site [5], perform assembly planning [6], and for collaborative product design and development [1]. The imseCAVE is designed based on the specification of the EVL CAVE system that facilitates the creation of an interactive immersive 3-D environment. The system consists of three walls of 10 x 8 foot acting as projection screens, and a 10 x 10 foot silver screen lying on the floor that acts as the fourth projection screen (Figures 1 and 2). The major advantages of the imseCAVE are its extremely low cost, high performance and versatility in 3-D model creation and visualization. With the use of high resolution LCD projectors as compared to costly cathode-ray projectors and special polarized lenses that simultaneously project the alternating stereo images onto a screen to recreate 3-D images, passive 3-D visualization hardware can be deployed as compared to using shutter glasses. In addition, with the use of high performance PCs (2.0 GHz Pentium 4 PC with high performance graphics adaptors) that are clustered
and linked together using high bandwidth Ethernet connections, high resolution computer models can be animated, distributed efficiently to appropriate PCs in the cluster through the network, and then projected to the corresponding screens forming the distributing virtual environment (Figures 3 and 4). With this novel architecture of the imseCAVE, a user configurable, flexible and low-cost CAVE is developed for diverse applications for effective e-education such as the design of automated warehouses and distribution centers, simulation study of the operation container terminal cranes systems, 3-D product development, and architectural design for urban development, etc. as highlighted in this paper.
Figure 1 The imseCAVE
Figure 2 The imseCAVE Virtaul Reality Engine
LCD Projectors (for Left Screen)
LCD Projectors for Right Screen)
PC Cluster and Virtaul Reality Engine
Translucent Screen
LCD Projectors for Front Screen
Pentium PC
Pentium PC
LEFT EYE IMAGE
Pentium PC
RIGHT EYE IMAGE
Pentium PC
LEFT EYE IMAGE
Pentium PC
RIGHT EYE IMAGE
Pentium PC
LEFT EYE IMAGE
Pentium PC
User 10 Feet
LCD Projectors (for Front Screen) LCD Projectors (for Floor projection via mirror reflection) LCD Projectors (for Right Screen)
Figure 3 The design of the imseCAVE
LCD Projectors for the Floor
RIGHT EYE IMAGE
Pentium PC Remote Workstation
Ethernet
10 Feet
LCD Projectors for Left Screen
LEFT EYE IMAGE
RIGHT EYE IMAGE
Ethernet
Pentium PC USB Bus
Joysticks I/O
RS232 Serial link
Tracker & Snesors Motion Platform
IPC
Figure 4 The architecture of the virtual reality engine and the interactive visualization system of the imseCAVE
While the virtual reality application software manages the animation of the distributed graphical model, the system also interfaces to external input devices that enable the real-time interaction with the graphical model. In the imseCAVE, a number of joysticks, digital I/O switches and indicators, tracking devices as well as the control of motion devices are incorporated. In particular, a motion platform is designed that interfaces directly to the virtual reality software such that users of the imseCAVE can experience the motions that are associated with the virtual scenarios. The incorporation of the motion platform is particularly useful in simulating the motion of a container terminal quay crane in operation where the sense of acceleration and deceleration that is generated can be exerted to the users.
3 Content Authoring for the imseCAVE Creating content of virtual world for the imseCAVE is divided into 2 phases: 3-D modeling and 3-D simulation. During the phase of 3-D modeling, sophisticated 3-D modeling software packages such as Alias|Wavefront MAYA are used to create realistic 3D objects and the surrounding environment. Usually, in order to create a compelling 3-D scene, highly detailed 3-D models should be used. However, restricted by limited 3-D computer graphics processing capability and the requirement of high frame rate, low polygon counted models with photorealistic texture mapping is indeed more desirable. In the phase of 3-D simulation, behaviors are embedded into different objects so that they can behave and interact with each others. For example, collision detection prevents objects from penetrating each others and implementation of physics properties such as gravity helps to create a convincing virtual world. Moreover, the interaction between users and the virtual world has to be programmed in this stage. In order to optimize the performance, technique of Level of Detail (LOD), that is, by displaying closer objects with more details and further objects with fewer details are also employed. In the following sections, a number of applications that are currently investigated by our department with the imseCAVE are described.
4 Container Terminal Simulator As the container handling business is one of the most important components in Hong Kong logistics industry, efficient terminal operation is essential to sustain the quality of service and growth in the future. Despite some automation has been adopted in modern container terminals, crane operations are currently fully manual. To maintain the required performance and quality of the operation, highly skilled crane operators are employed to operate different crane systems to load and unload containers to and from vessels. Traditionally, crane operators are trained on the actual crane systems and their skill evaluation are carried out in an ad hoc manner. With the aid of imseCAVE, an interactive virtual reality-based (VR) simulation and visualization training system was achieved.
4.1 Modeling of Crane Systems in a Container Terminal To generate suitable virtual scenery and interfaces for simulating the container crane control cabin for operators to perform container handling, the followings are developed: •
A graphical model of the container terminal that provides a realistic virtual environment for the simulator.
•
A graphical model of the crane system that consists of the movable objects that are associated with the system. These include the containers, spreader, control cabin, flippers, etc.
•
Other container handling equipment, in particular the vessel and the trucks that are deployed in a terminal to transport the containers between the berth and the yard.
For the simulator that we developed for simulating the quay crane operation using the imseCAVE, the computer software MAYA is used to model the various graphical entities. Figures 5, 6, and 7 show the MAYA models of the quay crane, rubber-tyre gantry crane, and the spreader for holding the containers of these crane systems respectively. After the models are being created, they are exported to the virtual reality software and various physical properties are defined before they are linked together. In order to attain an efficient virtual reality model for real-time animation in the imseCAVE, these graphical models are optimized for their details and complexity (for example, in terms of number of polygons) before they are exported
4.2 Simulation of Quay Crane Operations Using the developed imseCAVE, a container terminal quay side crane system is studied. The imseCAVE simulator is configured to a crane operator cabin of a quay side crane (Figure 8). A container terminal scenery is built with a container vessel berth at the quay side with containers ready for unloading. Container trucks are also created for transporting the container away from the quay crane. Inside the cabin, a twin joystick control panel with control buttons and indicators lights (Figure 9) is setup that interfaced directly to the virtual reality engine for the control of the quay crane, the spreader and various devices that are found in a common quay crane.
Figure 5 A Quay crane model
Figure 6 A rubber-tyre crane model
Figure 7 A spreader model
Trials of unloading containers from the vessel to the trucks with the quay crane were being undertaken according to common practices (Figures 10 and 11), and the actions and behaviors of the users are captured (Figures 12 and 13). Different types of containers and configurations are set to study the users’ reactions. The information obtained can be used for subsequent study on the performance of the crane system under different control strategies used by the mechatronics system; the ergonomics considerations of the cabin design including the operator seat design and the control panel arrangement; the users’ behaviors in respond to changes of operating conditions such as lighting and weather conditions; and the overall design of the crane system in terms of the structural design and optimum operating modes.
Figure 8 A virtual quay-side crane control cabin generated by the imseCAVE, an immersive 3-D scenery of the container terminal and a vessel is visualized by the users seated at the center of the system
Figure 9 The user interface of the quay-side crane control system that consists of a number of control buttons, indicator lights, and a joystick by which the user interacts with the virtual environment and the crane in real-time
Figure 10 A illustration showing the berth, the quayside crane and the container yard where the container will be loaded and unloaded to and from the vessel
Figure 11 The loading of a container to the truck that is parked at the loading zone as seen from the control cabin. The quay-side crane spreader is used to manipulate the container
Figures 12 and 13 The operator using the quay-side crane simulator to perform container loading and unloading operations
5 Air Cargo Terminal Located at the south east corner of the Hong Kong International Airport, the cargo terminal – SuperTerminal 1 is one of the largest air cargo terminals in the world. This six-level terminal building with a total floor area of over 288,000 sq. meters has the potential to handle 3.5 million tones of air cargo per annum. The SuperTerminal 1 is highly automated in order to provide the efficient and reliable cargo handling services to its airline customers. Simulation of SuperTerminal 1 is initially a project for the purpose of education. By penetrating right into the heart of this giant automated warehouse and to experience its operation, students are able to study, understand and appreciate the working principle of this massive structure.
5.1 Modeling of SuperTerminal 1 SuperTerminal 1’s Automated Cargo Handling Systems (CHS) mainly consists of three systems. They are Container Storage System (CSS), Box Storage System (BSS) and Bulk Cargo Distribution System (BCDS) (Figure 15). •
The Container Storage System is located at the east and west sides of the terminal building, the multifloor CSS has direct and sheltered interface with the airside, allowing units and pallets to be transferred from tarmac straight into the system. - Fully automated with more than 3,500 storage positions - 12 computer-controlled stacker cranes for moving cargo efficiently within the system - Linkage with workstations in the terminal building by 72 overhead bridges located on 2/F, 3/F and 4/F
•
The Box Storage System is located at the heart of the terminal building. The BSS provides about 10,000 storage positions for bulk cargo awaiting import, export and transshipment in one central location. Compartments inside the system are allocated by computer at random, ensuring that they are evenly utilized. - 2 Box Storage Systems providing about 10,000 storage positions for bulk cargo - Each system has 6 aisles and 2 stacker cranes in each aisle, providing maximum back-up even in the unlikely event of one crane failing
•
The Bulk Cargo Distribution System is a sushi-bar style conveyer loop system that moves cargo between floors and workstations within the terminal building. It links the Box Storage System with workstations, the Customs Examination Hall and import and export truck docks, enabling the boxes stored in the BSS to be routed by computer to any specific point in the terminal promptly and securely. Boxes can hence be routed directly to the assigned truck dock, minimizing cargo movements within the truck dock area, and at the same time enhancing cargo handling flexibility within the terminal.
Due to the scale and complexity of the structure of SuperTerminal 1, building of the 3-D models can be challenging. In order to minimize the number of polygons, a large number of texture maps are used. Similar to the modeling of the Container Terminal, 3D models of various graphical entities are first constructed in MAYA, and then they are exported to the virtual reality software. Figures 14 - 17 show the MAYA models of SuperTerminal 1.
Figure 14 A birds eye view of the SuperTerminal 1
Figure 15 Locations of CSS, BSS and BCDS in the SuperTerminal 1
Figure 16 An illustration showing an air cargo being Figure 17 An illustration showing the conveyer loop moved to the CSS system in BCDS
5.2 Virtual Tour of SuperTerminal 1 After completion of modeling virtual scenery of SuperTerminal 1, a variety of operations are programmed and attached to the corresponding 3D objects to mimic their functions in real life. For example, uploading/downloading air cargos by crane and transportation of air cargos by conveyer belt are some of the most common activities. During the virtual tour inside imseCAVE, users can move freely to any location within the virtual scenery. For example, the users can choose to follow one of the air cargo and experience the complete processing cycle of cargo handling.
Figure 18 Using the imseCAVE for the design and evaluation of the performance of the SuperTerminal 1 material handling systems
6 Conclusion In this paper, the design and architecture of the imseCAVE virtual reality system is presented. The system provides a versatile platform for interacting and visualizing 3-dimensional computer generated sceneries that recreate real world scenarios in real-time. In our research, the imseCAVE is deployed to study the operation of crane systems that are commonly found in container terminals, as well as the operations of an automated warehouse – the SuperTerminal 1. A virtual container terminal together with a simulated interactive crane system is developed. Through the performance of container handling trials undertaken by different parties including our students, observation and data are collected for subsequent analysis of system control and ergonomics issues. From the feedbacks obtained from these users, the virtual crane system simulator implemented with the imseCAVE provides a realistic simulated environment and human-computer interface for learning and training, crane control and operating system design, performance evaluation, and skill quantification. On the other hand, by creating the virtual SuperTerminal 1 with the imseCAVE, a new way of studying the operation of an automated warehouse is developed. The benefit of eliminating the limitation of on site visit and getting close access to automated equipment, students can explore and study this automatic warehouse without the constraint of location and availability. Moreover, different scenarios can be simulated and investigated, which may not be possible to be carried out in real world. In fact, the use of the imseCAVE is extremely versatile and only bounded by our imagination. Figure 19 and 20 show more examples of our current deployment of the system in applications such as product and architectural designs to enhance the effectiveness of teaching and learning.
Figure 19 Design and evaluation of a robotic arm
Figure 20 Simulation of running is a virtual township with historical buildings
References 1. Bochenek, G. M., and Ragusa, J. M. (2001) Virtual collaborative design environment: a review, isues, some research, and the future, Proc. Int. Conf. Management of Engineering and Technology (PICMET’01), vol. 1, pp. 726 – 735. 2. Cruz-Neira, C., Sandin, D. J., and DeFanti, T. A. (1993) Surround-screen projection-based virtual reality: The design and implementation of the CAVE, in Computer Graphics (Proc. of SIGGRAPH ‘93), ACM SIGGRAPH, pp. 135 – 142. 3. Disz, T., Papka, M., Stevens, R., and Pellegrino, M. (1995) Virtual reality visualization of parallel molecular dynamics simulation, Proc. Simulation Multi-conference Symposium, Phoenix, Arizona, pp. 483 – 487.
4. Symanzik, J. S., Cook, D., Kohlmeyer, B. D., and Cruz-Neira, C. (1996) Dynamic statistical graphics in the CAVE virtual reality environment, Proc. Dynamic Statistical Graphics Workshop, Sydney, Australia, pp. 1 – 11. 5. Vote, E. Feliz, D. A., Laidlaw, D. H., and Joukowsky, M. S. (2002) Discovering Petra: archaeological analysis in VR, IEEE Computer Graphics & Applications, September/October, pp. 38 – 50. 6. Ye, N., Banerjee, P., Banerjee, A., and Dech, F. (1999) A comparative study of assembly planning in traditional and virtual environments, IEEE Transactions on Systems, Man and Cybernetics, vol. 29, no. 4, pp. 546 – 555.
