Teaching Structured Development of Virtual Reality Systems using P-VoT Jinseok Seo1 and Gerard J. Kim2* 1
2
Dept. Of Game Engineering, Dong-eui Univ., Busan, Korea Dept. of Computer Science and Engineering, Korea Univ, Seoul, Korea
[email protected]
Abstract. Learning to build and test virtual reality (VR) systems is difficult due to the many required knowledge (e.g. computer graphics, sound processing, simulation, interaction, etc.) and subsystems to worry about (e.g. various sensors, displays, computers, graphics board, etc.). Furthermore, virtual reality contents have to be optimized according to different goals such as its basic function, usability, and presence. Thus, learning and applying a structured approach to designing VR systems is very critical to a successful completion of a meaningful class project. In this paper, we report our experiences in using a development methodology and an authoring support tool called the CLEVR/P-VoT to teach virtual reality to engineering students of advanced levels. CLEVR’s central concept is to refine and validate forms, functions and behaviors of the virtual objects and scenes incrementally and hierarchically. P-VoT helps students interactively try out and explore different virtual object/scene configurations and immediately see their impact with respect to system performance, interaction usability, realism, and presence. P-VoT, used in the first stage of the class, is designed at an abstraction level appropriate for even non-computer science major students to quickly learn and understand the need of a structured development approach. Having learned the merits of the structured approach firsthand, the students effectively put it to use in the second stage of the course for implementing a more sophisticated class project. Keywords: Virtual Reality, Structured Development, Methodology, Authoring, Teaching, Learning, Virtual Objects.
1
Introduction
Learning (or teaching) how to build and test virtual reality (VR) contents and systems is very difficult. For instance, students must study and understand many different topics such as computer graphics, modeling, physical simulation, interaction design, and VR library programming. There are many hurdles to actually carrying out a class project, for instance, having to deal with various types of sensors, interaction *
Corresponding author.
devices, display systems (visual, aural and even haptic), and not to mention, computers. Many students are often overwhelmed by the level of the complexity they have to somehow manage (in one semester) and fail to demonstrate a meaningful class project in the end (e.g. ending up with a keyboard/mouse based 3D graphic program). What makes this job more difficult is that, on top of having to tackle with the traditional computational and logical errors, designing a virtual reality contents/system is also an exploration (or optimization) task. Developers (students) must find the right combination of various types of constituents of the virtual environment like objects’ forms, simulation details, interaction style and modalities, etc. From author’s experience, it became apparent that the students needed hands-on training in structured application development to embark on a large-scale project, a usual required item in a typical introductory VR class. Most VR contents/system developments are still based on traditional programming with application programming interfaces or libraries. It is difficult to quickly and concretely (e.g. in a month or so) instill the concept of a structured approach around such a development framework. Thus, we have a used a development/authoring tool called the P-VoT based on a structured approach called the CLEVR, both developed by the authors [1]. CLEVR’s central concept is to refine and validate forms, functions and behaviors of the virtual objects and scenes incrementally and hierarchically. P-VoT helps students interactively try out and explore different virtual object/scene configurations and immediately see their impact with respect to system performance, interaction usability, realism, and presence. In P-VoT (See Fig. 1) most system development activities occur through the drag-and-drop GUI with some amounts of rudimentary level of programming without the gory system details like device interfacing or API syntax (i.e. students can concentrate only on the logic of the object).
Fig 1. A snapshot of P-VoT in use. The left window shows the execution results and the right window contains the interactive and interpretive development drag-and-drop GUI.
In an actual class, P-VoT is used during the first month (without learning any API details) to quickly establish the need and appreciation of a structured development approach (like CLEVR), applying P-VoT to a small scale project. Having learned
CLEVR, the students effectively put it to use in the second stage of the course for implementing a more sophisticated class project using real world VR devices and programming libraries (without P-VoT). In this paper, we describe the details of how P-VoT was put to use to teach structured approach to virtual reality system development to advanced level engineering students (who may not be of computer science major, but possess basic programming skills). In the next section, we first review other attempts to teach system or program development using high level design tools. We briefly introduce the CLEVR methodology as the major background for our work. Then, we describe the teaching process with actual examples used in the class and discuss our experiences. Finally, we conclude the paper with a summary and directions for future work.
2
Related Work
The use of highly abstracted design tools to teach difficult concepts is not new [2][3][4]][5][6]. In particular, Alice, to which P-VoT resembles, has been used to teach not just 3D interactive graphics (its original target domain), but also basic object-oriented programming [7]. It has been reported that the match between the concepts of computational objects and the concrete graphic objects (in using Alice) has helped novices understand object oriented programming concepts better than the traditional approach. Alice’s interpretive environment has allowed the students to try and design different versions of 3D graphic contents very quickly [7]. In the general software area, computer aided software engineering (CASE) tools are used for both teaching and development. Harel [8] developed a graphical CASE tool, called STATEMATE, for specification, analysis, design, and documentation of large and complex reactive system. In this tool, he proposed to use “Statecharts” as a visual formalism (which is also used as a way of behavior specification in this work). Gaskell and Phillip’s [9] Executable Graphical Specification (EGS) system uses the Data-Flow Diagram (DFD) notation to describe functional models. These formalisms can be executed in an interactive manner for the functionality of the system to be observed and checked. Behavior or function authoring using script languages has also been attempted in many tools and products, because scripting enables developers e.g. to build virtual worlds easily and rapidly. This script-based approach is best exemplified in Alice [4]. Alice also provides a predefined set of functions applicable for any generic objects which can be reused to create high-level behaviors. However, because the major objective of Alice is to design interactive 3D graphics, it lacks support for VR specific features (3D multimodal interaction, rigorous performance issues and presence). In the area of entertainment, many 3D gaming engines have also adopted this scripting approach [10][11][12]. Scripts are usually associated with an interactive environment in which it is executed. Such interactive programming saves a lot of time and labor. We can confirm the potential utility of such an interactive kernel approach from the great debate about LISP vs. C as a prototyping language [13], and even more so from the proliferation of many visual Rapid
Application Development (RAD) tools for 2D WIMP user-interfaces and game prototyping examples.
3
CLEVR Approach and P-VoT
The work described in this paper originated from an effort to establish a systematic engineering approach for VR systems called the CLEVR. The three philosophies that underlie the CLEVR are (1) concurrent consideration of form, function, and behavior, (2) hierarchical and incremental development (exploration), and (3) the spiral development process that addresses performance, interaction modeling and presence enhancement in turn. P-VoT refers to an integrated set of tools that supports the aforementioned engineering philosophies. P-VoT’s is based on the clear separation of form, function and behavior (philosophy 1), which in turn promotes reuse (mixing and matching) of object components at this level dynamically and incrementally (philosophy 3). Thus, to understand how to use P-VoT, the concept of form, function and behavior must be made clear. The application developer must “design” three things that are interrelated, namely, “form,” “function,” and “behavior” at the same time. Form refers to the outer appearance of virtual objects, their structure (for composite objects) and the scene structure of the virtual world. Other physical properties (which may be required for physical simulation) such as mass, material property, velocity, acceleration may be added as a part of the form information. Function refers to encoding what virtual objects do (i.e. primitive tasks) to accomplish their behavior, whether autonomously, or in response to some external stimuli or events, while behavior refers to how individual virtual objects dynamically change and carry out different functions over a period of time, usually expressed through states, exchange of events, and inter-state transitions. Since virtual environment construction is a design and an explorative task, the usual hierarchical and incremental approach is even more fitting than for the conventional softwares that are more algorithmic in nature. Moreover, employing the hierarchical style in the incremental development process promotes a performance conscious design by forcing the developer to focus on the more critical features in the form, function/behavior in a top-down manner. For example, intermediate models obtained from the hierarchical modeling approach can be used for level of detail (LOD) models as well. At each abstraction level (development stage), in order to validate the models (e.g. behavioral correctness, visual effect, performance) and carry out further refinements, they must be simulated or executed. With P-VoT’s interactive and interpretive kernel, developers can interactively build, deploy and test virtual objects immediately (without compilation) and practice the CLEVR methodology. Fig. 2 shows the overall structure of the P-VoT’s integrated development environment (IDE). The IDE includes many support tools such as the (simplified) Statecharts behavioral editor, performance monitoring tool, scripting (we chose “Python” [14]) window, GUI based scene and object design and execution tool, and version control. Currently, on-line and off-line Python scripts and an XML based
mark up language called the PVML can be used to specify virtual objects and construct a virtual world. Note that the above IDE is not just a development environment, but also an execution environment of the same virtual world.
Fig 2. The overall structure of P-VoT. The kernel can take script commands interactively or offline. It can also interpret mark-up language style specifications called the PVML.
3.1
Form Modeling
In the current P-VoT kernel, several primitive geometries (such as cube, sphere, cone, and cylinder) are available for direct creation and a variety of geometry file formats can be imported. One POM (P-VoT Object Model) object can have a form hierarchy using a mixture of primitive geometries and database files. Functions can access and manipulate the form (i.e. represented as nodes in the form hierarchy). 3.2
Dynamic function specification
The PVObject (the top-most class of POM class hierarchy) provides predefined primitive and useful functions (or methods) to save development time (similarly to Alice [4]). These primitive functions include manipulation of the form attributes of the object and basic animation routines (e.g. move and turn). Currently, the PVObject has seven attributes to control the position, orientation and scale. They are x, y, z, h, p, r, and scale, and their names themselves become the setter and getter functions. For example, a statement “z = object.z” invokes the getter function and “object.z = 3” does the setter, internally. Thanks to the dynamic binding feature of the Python language [14], developers can write their own functions by composing these primitive functions (or from scratch).
When writing new functions, it is also possible to utilize rich standard libraries that Python already provides. These user-defined functions can be added to the existing objects or classes, and also replace existing functions. 3.3
Behavior authoring using Statecharts
The behavior part of an object can be specified using a simplified version of Statecharts formalism[8] and it manages the lifecycle of an object from its construction to destruction. This behavior specification is translated into a hierarchy of instances of PVSTM, PVState, PVTransition, and PVEvent classes. The PVSTM objects have a set of PVState objects, and can have links to a set of PVSTM objects. In this way, the behavior of an object can express parallelism and hierarchy, which are the highlighted features of the Statecharts (and underlying philosophies of the CLEVR). A PVState object has three attributes: onEnter, onDuring, and onExit. Each attribute is defined through a function call. The function on the onEnter attribute and the one on the onExit attribute are invoked only once, when a transition to this state occurs and when transition to another state occurs, respectively. The function on onDuring is executed at every simulation tick. Though the state transition formalism in Statecharts is more complex, the PVSTM only currently supports event based transitions. There are four types of events: message, timer, collision, and button (e.g. buttons of mouse, keyboard, and other devices). We are planning to add more types of events and logical operators. 3.4
Supplemental VR Functionalities
In order to make P-VoT more useful and powerful in developing “VR” contents, we implemented several supplemental functionalities such as sensor interfacing, display support, object encapsulation or reuse, performance tuning, and physical simulation. By embedding and wrapping the functions from the Virtual Reality Peripheral Network [15], students can easily connect various sensors to their contents and carry out interaction design. P-VoT also supports a diversity of display configurations (e.g. 2 channel stereoscopic HMD, shutter glasses, and multi-channel CAVE-like systems). In P-VoT, hardware specific requirements are designed separately from the contents. The device configuration specification consists of the information about display systems and input devices, and it uses an in-house XML based format called the PVDC. Developers or students can easily reuse various pre-defined hardware configurations and retarget existing VR contents to different hardware platforms. P-VoT also can display various run-time performance statistics (e.g. rendering time, simulation time, system overhead, frame rate, etc.) at different levels of granularity, e.g. at individual object level, at group object level (represented as a sub-tree in the scene tree), or at the whole scene level. The statistics are useful in identifying the possible sources of performance bottlenecks and making level of detail decisions.
Finally, P-VoT offers VR related reusable object libraries. The reusable objects may be purely functional, such as those for physical simulation (implemented using the Open Dynamics Engine [16]), or concrete objects as various interaction metaphors (like virtual rays, Arcball [17], Homer/Go-Go [18]). Efforts are under way to add more functionalities for character animation, augmented reality and 3D sound support.
4
Teaching Example
In this section, we step through a small example project used in the introductory VR class at our university 1 using only P-VoT. As already emphasized, the main objective is let the students gain an experience in applying a structured method and successfully completing a VR contents considering a multitude of issues like the basic functionality, the level of realism and expected user-felt presence, trade-offs with performance, interaction design, and project management. 4.1
First Stage: A Simple Modeling Exercise
Make a “simple” specification (e.g. storyboard, class diagram, message, Statecharts, scripts, etc) for a simple basketball shooting game. For example, one possible scenario is: (1) when the system starts, a hand/lower arm appears with the ball, (2) the hand/arm can move left and right by keyboard arrows, (3) user can press “s” to shoot, (3) when shooting, the arm can have simple animation, (4) when “s” is pressed, the ball travels from the hand toward the basket by some motion algorithm. Remember this exercise is to be simple. Do not include too much details (e.g. don't have to give details like the finger movements and so on). (a) Use MultiGen to model the needed objects, like the hand/arm, ball, and basket. (b) Use P-VoT to implement and simulate your specification. (c) Propose how this simple scenario can be extended to become “virtual reality.” Fig. 3. The first homework, a modeling exercise for a simple basketball shooting simulation (from Spring 2005 Intro. to VR class).
After two weeks into the course, during which basic object/scene modeling techniques and needs for structured development were taught by standard lectures and illustrations, the following first homework was given (See Fig. 3). The main purpose of this exercise was to first to produce a clear requirements specification and assess the feasibility with a “Level 1” implementation. Fig. 4 shows samples of the specification and implementation run (using P-VoT) from an actual student. Two weeks were given to finish this exercise considering the time to learn how to use the
1 “Introduction to Virtual Reality” (listing: CSE-511) was offered in Spring/Fall 2005 and Spring 2006 semesters. The course homepage can be found at http://www.postech.ac.kr/~gkim/cse511/cse511homepage.html.
geometric modeler (to create the geometry files for the objects) and learn the specifics of the P-VoT and Python language. Fig. 4 shows part of the homework submitted by one student showing the scenario board, message diagram and actual implementation results.
Fig. 4. The first homework submitted by a student. The upper part shows part of the specification using story boards and message diagrams (drawn manually). The lower part shows the implementation in P-VoT (reproduced with the courtesy from Joosung Yoon).
4.2
Second Stage: Further Refinement and Performance Tuning
In continuation to the first homework, a second assignment was given immediately after the completion of the first. The second homework asked the students to further refine the specifications of the basketball shooting contents (by introducing more
objects, more detailed simulation and interactive features, etc.) and come up with the “Level 2” implementation. In addition, students were asked, as a separate exercise, to deliberately populate the scene with (many) copies of objects to bring down the system performance and monitor it with the performance tuning tool of P-VoT. Parts of the “Level 2” specification are shown in Fig. 5.
Fig. 5. The second homework submitted by a student. The upper part shows part of the refined Statecharts (behaviors) of the player and ball. Compared to the “Level 1,” “Level 2” has a “jumping” stick figure, a nicer floor, and a new functionality to control the shooting time. The lower figure shows snapshots of the implementation in P-VoT (reproduced with the courtesy from Wontae Jang).
4.3
Third Stage: Tour into the “Gymnasium”
The theme of the third stage of this continuing homework was “image based modeling.” The basic approach of creatively using textures and image based modeling was taught by presenting the Quicktime VR [19] and Tour into the Picture (TIP) [20]. The third homework exercise asked students to a take a picture of the school gymnasium, use it to create a rectangular cubed TIP gymnasium environment (using Photoshop and a geometric modeler), and integrate it into the “Level 2” P-VoT implementation.
Fig. 6. The third homework submitted by a student. An image based gymnasium environment was created by learning the Tour into the Picture (TIP) technique. The TIP model was imported into P-VoT and integrated with the “Level 2” implementation only with a few mouse clicks and position adjustments (reproduced with the courtesy from Wontae Jang).
4.4
Fourth/Fifth Stage
The final stages this continuing mini-project was to use P-VoT to (1) set up the basketball contents for stereo display and connect it to an HMD (which only required just a few clicks and connection), and (2) set up the basketball contents to use a 3D tracker and/or button device (e.g. for view point control). This way, the students needed not to bother with device specific programming and easily gain a feel for using VR devices with their contents. Finally, the whole P-VoT written program was to be ported to a real world implementation engine (e.g. SGI OpenGL Performer, OpenSceneGraph, etc.).
5
Conclusion
In this paper, we introduced a development methodology and an authoring support tool called the CLEVR/P-VoT and presented our experience in using them to teach structured approach to developing virtual reality systems. While it is difficult to quantitatively show the effectiveness of the proposed methodology and tools in terms of educational attainment in a controlled situation, P-VoT clearly helped students gain a hands-on experience in applying structured approach to system development
(compared to the previous classes which were taught without such a tool). P-VoT has been used in the “Introduction to Virtual Reality” course three times since spring of 2005. As already mentioned, most students had basic understanding of how to program, but had little or no experience in computer graphics, 3D programming API’s, dealing with various sensors, devices and displays. Most students had no difficulty in finishing the mini project in time and gain concrete results and an appreciation for the structured approach to the system development. Some of the keys to the success seemed to be the following. The level of what the tool expected of the user was just right. P-VoT was designed for advanced undergraduate students with programming background (but not necessarily computer science students). Many authoring tools work purely through mouse clicks and drag-and-drops to eliminate programming entirely and can only offer very limited functionalities. The scripting approach combined with the state-based (or eventbased) representation (and built-in state machine) gave the right level of flexibility for the target students. Consistency and concreteness in abstraction also simplified the understanding of the overall system. For instance, interaction design could be abstracted as a virtual interaction object design (e.g. Arcball), creating its form (e.g. sphere) and its function (e.g. device mapping function) and behavior (e.g. interaction modes). Finally, it was the elimination of the API idiosyncrasies (rather than programming) that invited the students to try many different things (e.g. sensors, interaction style, model objects, their configurations, etc.) and zoom in on a good content design within a given time. Though we have showed that our approach using CLEVR/P-VoT was promising in helping students learn and practice to build and test VR systems, many improvements (as observed and suggested by students) are necessary as well. We are now planning to add more easy-to-use drawing/parsing tools for various visual formalisms such as message sequence diagrams, class diagrams, and Statecharts and improve the specification methods and tools for managing complex interactions between objects.
Acknowledgement This work was supported by grant No. R01-2006-000-11142-0 from the Basic Research Program of the Korea Science and Engineering Foundation (“Teukjung Gicho” project).
References 1. Seo, J.S., Kim, G.J.: Design for Presence: A Structured Approach to Virtual Reality System Design. Presence: Teleoperators and Virtual Environments, 11(4), 378-403, 2002. 2. The MathWorks, Inc.: The MathWorks – MATLAB and Simulink for Technical Computing. Retrieved Jul. 23, 2006 from the World Wide Web: http://www.mathworks.com, 2006. 3. Pattis, R.E.,: Karel the Robot: A Gentle Introduction to the Art of Programming. John Wiley & Sons, Inc., New York, NY, 1994. 4. Carnegie Mellon University: Alice: Free, Easy, Interactive 3D Graphics for the WWW. Retrieved Jul. 23, 2006 from the World Wide Web: http://www.alice.org, 2006.
5. Myers, B.A., Giuse, D.A., Dannenberg, R.B., Zanden, B.V., Kosbie, D.S., Pervin, E. Mickish, A., Marchal, P.: Computer, 23(11), 71-85, 1990. 6. Epic Games: Powered By Unreal Technology. Retrieved Jul. 23, 2006 from the World Wide Web: http://www.unrealtechnology.com, 2006. 7. Cooper, S., Dann. W., Pausch, R.: Teaching objects-first in introductory computer science. ACM SIGCSE Bulletin, 35(1), 191-195, 2003. 8. Harel, D.: STATEMATE: A Working Environment for the Development of Complex Reactive Systems. IEEE Transactions on Software Engineering, 16(4), 403-414, 1990. 9. Gaskell, C., Phillips, R.: Software Architecture of the executable graphical specification tool EGS. Software-Concepts and Tools, 16(3), 124-135, 1995. 10. Conitec Corporation: 3D GameStudio. Retrieved Dec. 16, 2003 from the World Wide Web: http://conitec.net/a4info.htm, 2003. 11. Radon Labs GmbH: The Nebula Device. Retrieved Dec. 16, 2003 from the World Wide Web: http://www.radonlabs.de/nebula.html, 2003. 12. Tyberqhein, J.: Crystal Space 3D, Retrieved Dec. 16, 2003 from the World Wide Web: http:// crystal.sourceforge.net/tikiwiki/tiki-view_articles.php, 2003. 13. Gat, E.: Lisp as an alternative to Java. Intelligence 11(4): 21-24, 2000. 14. Python Software Foundation: Python Programming Language. Retrieved Jul. 23, 2006 from the World Wide Web: http://www.python.org, 2006. 15. VRPN. Retrieved Jul. 23, 2006 from the World Wide Web: http://www.vrpn.org, 2006. 16. Smith, R.: Open Dynamics Engine. Retrieved Jul 23, 2006 from the World Wide Web: http://www.ode.org, 2006. 17. Shoemake, K.: ARCBALL: a user interface for specifying three-dimensional orientation using a mouse. Proceedings of the conference on Graphics interface, 151-156, 1992. 18. Bowman, D., Hodges, L.: User Interface Constraints for Immersive Virtual Environment Applications. Technical Report of Graphics, Visualization, and Usability Center, Georgia Institute of Technology, GIT-GVU-95-26, 1995. 19. Chen, S.E.: QuickTime VR: an image-based approach to virtual environment navigation. Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, 29-38, 1995. 20. Youichi, Horry, Ken-Ichi, Anjyo, Arai, K.: Tour into the picture: using a spidery mesh interface to make animation from a single image. Proceedings of the 24th annual conference on Computer graphics and interactive techniques, 225-232, 1997.
