Software Engineering of Virtual Worlds - CiteSeerX

Software Engineering of Virtual Worlds G. Jounghyun Kim, Kyo Chul Kang, Hyejung Kim and Jiyoung Lee Dept. of Computer Science and Engineering Pohang University of Science and Technology (POSTECH) San 31, Hyoja-dong, Pohang, 790-784, Korea [email protected]

Abstract

developer, hiding and abstracting out low level details and providing easy-to-use API's for the programmers. Despite such tools that are becoming more available to application programmers, developing and maintaining a virtual reality application remains to be more dicult than usually expected. The main diculty lies in the management of complexity. The complexity arises from the fact that the application developer must \design" three things that are interrelated, namely, \form", \function" and \behavior" at the same time. Form refers to the outer appearance of virtual objects, their structure (for composite objects) and the scene structure of the virtual environment. Other physical properties (which may be required for physical simulation) such as mass, material property, velocity, acceleration may be part of form information. Function basically refers to encoding of what virtual objects do to accomplish their behavior whether autonomously or in response to some external stimuli or event, while behavior refers to how individual virtual objects dynamically change and carry out dierent functions over some period of time, usually expressed through states, exchange of events, and inter-object constraints. Most VR applications are developed in sequence of form design, then function/behavior programming. That is, virtual objects are modeled using the CAD systems, then converted into the appropriate le formats or data structure to be used by the run-time virtual environment. The application program is usually written by start lling in code template in Figure 1 [1]. For performance reasons (e.g. to manage dierence in simulation rate and rendering rate), the rendering loop may be split into multi server-client processes. After this point, no particular methodology or guidelines exists for further software development. The result is spaghetti-like code in which information regarding function, behavior, and constraints among them are all mixed in together, not readily visible for easy maintenance.

Developing a virtual reality application is a very dicult task. For one thing, it requires an in-depth knowledge and understanding in many disciplines. However, the most dicult part lies in the eective management of simultaneous design of form, function and behavior of virtual objects. Up to now, very little focus has been given to methods and tools for rapid prototyping VR software and modeling virtual objects. Most VR applications are developed by modeling the needed objects on conventional CAD systems, then using low-level graphics libraries or high level simulation packages. In this paper, we advocate for a structural and pedagogical approach to developing VR applications, and add the component of \form" to an existing real-time system development methodology so that the resulting methodology (and the prototyping environment) supports simultaneous and hierarchical engineering of form, function and behavior. Using the proposed methodology and supporting tool, it is our hope that VR applications can be produced and maintained eectively.

1 Introduction 1.1 Motivation: Requirements Engineering for VR Software Developing a virtual reality application is a very dicult task. For one thing, it requires an in-depth knowledge and understanding in many disciplines such as sensing and tracking technologies, stereoscopic display, multimodal interaction and processing (e.g. 3D menus and metaphors, voice, haptic, gesture), computer graphics and geometric modeling, dynamics and physical simulation, performance tuning, etc. Several software packages have been developed to help alleviate this burden for the everyday application 1

method that incorporates form design in addition to behavior and function, embedded in a tool called the ASADAL/PROTO [9]. This new method takes advantages of both the behavior/function speci cation formalisms called the Data Flow Diagrams and Statecharts, and an object-oriented form representation called the Visual Object Speci cation (VOS), and is designed to support simultaneous, incremental and hierarchical speci cation of form, function and behavior of a virtual environment.

main () { ... Initialize Run-time Environment Set Environment Condition ... Import Objects (or Create Objects) Create Composite Objects Create a Scene Structure ... /* infinite rendering loop */

VOS describes virtual object's physical properties (e.g. shape, dimensional parameters, mass, material), con guration (e.g. position, velocity), inter-object spatial constraints, and their encapsulated primitive low level behaviors (explained later). The dynamically changing states of the virtual world is represented through the Statecharts, and the functional structure of the virtual objects are expressed through the Data Flow Diagrams.

While (1) { Poll Trackers and Other User Input Check Interference Compute One Simulation Tick Render Graphics } }

Figure 1: A typical core process in a virtual reality software (adapted from [1]).

ASADAL/PROTO oers two methods for verifying or validating its intended behavior and functionality. One method involves specifying important real-time properties in temporal logic and applying the theorem proving techniques (not described in this paper), and the other is to run simulation of the speci cations. In order to carry out the simulation, the speci cations are interpreted, activating and rendering VOS objects, communicating with an optional but separate physical simulation (e.g. interference checking, gravity dynamics).

Construction of a virtual world often requires many revisions, and changing one aspect of the world will undoubtedly aect other aspects. For instance, dierent shapes and con gurations (positions and orientations in space) can result in dierent dynamic behaviors. A jet ghter has a dierent aerodynamic characteristics from that of a passenger airplane. Form can also aect functionality. For instance, two different robot manipulators diering in size may have dierent work volumes and capabilities. Such a development cycle is dicult to handle when working in a single level of abstraction and considering these design spaces in isolation. A preliminary and structured process of \requirements engineering" can help by gathering and organizing user/system requirements, making explicit representations of them and making sure they are accounted for during the course of software development.

Using the proposed method and the supporting tool, it is our hope that VR applications can be speci ed and validated at early development stages and that the validated speci cation will serve as a basis for actual code generation (for later stages) and history data for software maintenance. This paper is organized as the following. First, in the following section, we review several related research in modeling and developing VR worlds. Section 3 explains the main concept behind our modeling approach and section 4 describes ASADAL/PROTO's modeling process, namely, the incremental requirements speci cation, stage-by-stage validation process and the speci cation constructs for form, function, and behavior with an example of a speci cation of a virtual navigation system. Then, we conclude our paper with a critical summary and report of our current status.

1.2 This Paper In this paper, we advocate for a structural and pedagogical approach to developing VR applications, and introduce current approaches to software engineering techniques to making speci cations for real-time control systems. This is based on our observation that VR software requirements must include speci cations for virtual object models and their real-time control within the environment. We describe a speci cation 2

2 Related Research

another degree of freedom) during maintenance.

Traditionally, the software engineering research community has focused on speci cation (and its simulation) of embedded control systems in terms of their behavior and functionality, while researchers in computer graphics and mechanical design concentrated their eorts on visualization of form data. Harel [4] developed a graphical CASE tool, called STATEMATE, for speci cation, analysis, design, and documentation of large and complex reactive systems. STATEMATE has features such as batch mode simulation, a set of testing procedures (for reachability, nondeterminism, and deadlock), three dierent speci cation methods to support multiple views, and easy-to-understand semantics. ASADAL by Kang et al. [5], the predecessor of ASADAL/PROTO, is also a CASE tool designed for real-time systems, but have comprehensive simulation facilities to support incremental development practice. However, all of above approaches make no provision for specifying meaningful environmental behavior with physical properties. One of the rst researchers who noticed the need for software engineering methods for building \physical" systems was Paul Fishwick. In [8], he proposes for a convergence of systems and software engineering for building physical simulation models by use of objectoriented methods and multimodels, and separation of static/dynamic models. However, his model of engineering is strictly sequential: modeling for static objects, then dynamic behavior. Our modeling process is concurrent and iterative, than sequential, allowing the user to freely moving between the function, form, and behavior spaces, and re ning each spaces in an appropriate manner. Real-time computer graphics or animation programs are developed using low-level simulation programming packages constructs or libraries [6, 11, 10, 13, 2, 14]. Such software packages allow encoding of the functional/behavioral aspect of the virtual environment, and usually support object-oriented programming, communication with popular VR devices, and importing of various model le formats. Even with these types of graphics packages, incremental development and eective maintenance of graphics/animation applications is yet to be supported in full scale. Graphic objects are created using geometric modeling or computer-aided design systems by users who must think ahead and remember how they are used when used or controlled. Then, simulation programs are written (to fair amounts of detail), compiled, debugged, run, and modi ed. Sometimes, it is often required to modify the graphic object (e.g. add

3 Incremental/Hierarchical Modeling ASADAL/PROTO's main system modeling philosophy is based on that of the incremental and hierarchical development paradigm. The incremental development refers to practices that allow a program to be developed, validated, and delivered in stages, and it is supported by many lifecycle models such as the spiral, evolutionary-prototyping, and the staged-delivery [3]. Advantages of incremental development are many. For instance, it reduces risk by breaking the project into a series of smaller subprojects and focusing on the more risky subprojects rst, and increases progress visibility by providing nished operational pieces of a system long before you could make the complete system operational. The incremental development approach is meant to eliminate the necessity to possess a complete set of requirements speci cations for a system before the start of its implementation, allowing a lot of changes to be applied afterwards in simple fashion. Physical properties, con guration, interactive behavior and other form related attributes should evolve incrementally together with its behavior and function. The underlying theme is that the speci cation in each dimension aects each other along the way. Figure 2 illustrates a possible virtual object model as a result of our integrated modeling approach with form, function and behavioral description in place. Object ports channels states, transitions functional attributes(data, signals) spatial properties(forms) state management methods state access methods functional methods spatial methods constraints behavioral/functional constraints spatial constraints

Figure 2: A virtual object model in ASADAL/ PROTO. Depending on the level of abstraction and division 3

of computation, the behavioral/functional description of a virtual object may be explicitly modeled and be included in its place or be \hidden" in the form speci cation as low level primitive behaviors. For instance, at a gross level, the function speci cation of a robot controller might simply send a \move" command with a target position. The form speci cation may possess as a built-in primitive (not to be modi ed by the programmer at that moment) that, for instance, performs inverse kinematics, calculate the joint angle error matrix, multiply it by a preset gain, and actuate each joint in constant speed. Alternatively, at a more detailed modeling level, this primitive may be taken out for modi cation and placed into the virtual object's behavior/function representation. For instance, a new type of trajectory pro le may be described, and with this newly added speci cation, the virtual object is activated not only with a simple \move" command, but also with acceleration/deceleration/velocity data. The behavior and function models are constructed using formalisms of the Statecharts and the Data Flow Diagram (DFD) respectively, both available from speci cation methods of ASADAL/PROTO. A DFD speci es a single process or a function in terms of how it can be decomposed into a number of subprocesses or objects and their input/output relationships. A DFD may have a corresponding Statechart that speci es how the process behaves, that is, it speci es when to execute the processes according to the state of the system. For a more detailed explanation of their exact formalisms, please refer to [5]. Form is expressed using the Visual Object Speci cation (VOS). Its primary purpose is to describe physical properties and con guration of a physical entity. VOS also speci es explicit physical relationships or constraints among physical objects (e.g. A block's location follows the tip of the end-eector once grabbed.). Ideally, these relationships should be inferred automatically from the physical simulation computation. Realistically, however, the environment simulation cannot be run with every little physical detail, and thus some reasonable level of abstraction should be set. The dynamic behavior of the controlled system is, in most cases, coupled with physical laws, and thus this is the middle ground where the intended behavior and observed behavior are resolved. Behavioral, functional, and spatial speci cations share data within its object de nition as shown in Figure 2. For example, a VOS object may contain spatial properties in terms of control data and signals updated by the functions which are in turn scheduled to be invoked by the behavioral model.

For validation purpose, ASADAL/PROTO is equipped with an engine that simulates the form, function, and behavior speci cation. The resulting interactive system behavior is observed and analyzed using graphical facilities provided by ASADAL/PROTO. This speci cation and simulation-driven validation process is continued in a step-wise and incremental fashion. For instance, a form object may be further re ned into a set of constituent subobjects (e.g. a robot broken down into links and joints), and their corresponding behaviors and functions would be re ned accordingly. For further detailed code generation, the speci cations are transformed, with user's help, into a canonical object representation, which can be translated into platform, hardware, and language speci c programs to be used in the real VR system. This mechanism facilitates speci cation-level reuse of behavior/function and modi cation. Details about the translation process into common languages are out of the scope of this paper. In this paper, we focus on validating the form/function/behavior of virtual worlds at the speci cation level.

4 Modeling/Validation Process and Speci cation Constructs In ASADAL/PROTO, behavior and function speci cations start with drawing several Message Sequence Diagrams (MSD). The MSD depicts typical scenarios of external behavior of a real-time system in terms of a sequence of data or control signals exchanged among the objects in the system (See Figure 5.). With consideration of form, this facilitates the object-orientation of the overall modeling process. That is, by use of MSDs and consideration of form, objects that correspond to real world objects are easily identi ed and further re ned. It is important to start the overall modeling with a consistent view of object-orientation as seen by the behavior/function speci cation process and form design, as each may proceed, at rst, separately and merged later. Figure 3 shows the modeling process to specify and validate a system. In the following subsections, we illustrate the overall process and speci cation constructs used with an example of building a speci cation for a virtual walkthrough system.

4.1 System Speci cation At the very beginning, one might start out with a requirement: \A navigator navigates around the 4

Navigation Engine

system objects (MSD)

Observer obs_position, obs_view_direction angular_velocity

for spatial objects

for functional objects

obs_view_direction

behavior/function specification(DFD, Statecharts) behavior/function class

behavior/function class

functional objects

velocity

form specification(VOS)

obs_position

form class

Figure 5: The Message Sequence Diagram (MSD) of the walkthrough system.

spatial objects

POSTECH

visual simulation system

PIRL 6 EB

Figure 3: Process of form, function, behavior speci cation EB2

Library

Auditorium

SB

Figure 6: A spatial structure diagram (SSD) of the virtual environment. The gure merely depicts the containment relationship between POSTECH and its subobjects (e.g. academic buildings) denoted with surrounding rectangles. Numbers indicate number of subobjects. Expression of other types of physical relationships is possible, but not shown here for brevity.

Library

SB

6 RIST

RIST

PIRL

lationship Diagram (SRD) and Spatial Structure Diagram (SSD) (See Figure 6.) are used to capture the spatial and composition relation between objects which we are interested in. With a clear picture of a con guration of the system in terms of constituent objects and information ows between them, detailed behavior, function, and form speci cation of each object may start. We explain the behavior and function speci cation rst in the next section.

Figure 4: A top-level path speci cation for POSTECH walkthrough system. POSTECH buildings" and, based on this, a simple navigation path can be designed as shown in Figure 4. The gure shows that the navigator is to start at the RIST building, pass by the PIRL building, Library, Engineering Building 2 (EB2), and nally arrive at Student Union Building (SB). As mentioned earlier, the modeling process starts with constructing the Message Sequence Diagrams (MSD) that describe typical scenarios of external behavior of the virtual world, in order to further clarify the system objects needed to ful ll the required system objective. The MSD of Figure 5 shows typical messages exchanged among the identi ed objects. For example, the NavigationEngine sends angular velocity for the Observer to rotate its view, and upon completion of this task by the Observer, sends velocity for the Observer to move to its next target position. While the MSD clari es information exchanges among these objects, Diagrams called the Spatial Re-

4.2 Behavior and Function Speci cation obs_position obs_view_direction

Observer

Navigation Engine

velocity angular_velocity

Figure 7: A top level DFD of the walkthrough system. The big circles denote computational processes, and the arrows with text represent data ow. 5

obs_position

calculate_ motion_ vectors

next_dest

Navigation Controller

move_vector

compute_ observer_ velocity

velocity

NavigationController /next_dest := (90, 120, 10) ; stop_time := 5

Moving toRIST

turn_and_move_end /next_dest := (130, 100, 20) ; stop_time := 10

Moving toPIRL

dest_direct

compute_ observer_ angular_ velocity

Navigating

Turn_and_Moving tmd(compute_observer angular_velocity) /rotate

Idle

Turning

tm(en(stop), stop_time) /turn_and_move_end

Moving

Stop Moving toSB

obs_view_direction

turn_and_move_end /next_dest := (120, 60, 20) ; stop_time := 5

turn_and_move_end /next_dest := (70, 80, 20), stop_time := 15

angular_velocity

Moving toEB2

Figure 8: The re ned DFD of the NavigationEngine

ex(moving)

Rotating Moving toLibrary

turn_and_move_end /next_dest := (95, 60, 20) ; stop_time := 10

Stop [obs_position rotate = next_dest] move

Rotating

Now, we may proceed to specify the intended behavior and re ne the functional structure of each object in MSD using the Statechart and DFD. There is one functional object in this example, NavigationEngine, responsible for moving the observer through the campus. At the top level, it is represented as one entity in Figure 7, then further re ned into a more detailed form in Figure 8 with needed data and control signals among its subprocesses. Figure 9 shows the intended behavior of the NavigationEngine process, named NavigationController, divided into four possible concurrent sub-behaviors: one that sequences through states of moving to RIST, then moving to PIRL, and so on (shown in the left part of Figure 9), and three modes of navigation (\rotate in xed location", \translate", and \rotate as translate"). This Statechart speci es when the processes in the DFD are to be activated as the states change. The rounded rectangles represent the states and the dotted line divides the overall behavior into four concurrent states. The computations that the process blocks in the DFD carry out are speci ed using the Computation Speci cation Language(CSL). For example, shown in Figure 10 is the computation speci cation of the process compute observer velocity that computes the velocity of the observer given the vector from source to target location. Once a behavior speci cation is made for objects using Statechart and DFD, the logical consistency and correctness, and the timing behavior of the speci cation can be analyzed. The speci cation simulator engine of ASADAL/PROTO can execute these DFD and Statecharts and visualize the rough navigation scene from these behavioral and functional data. This is realized by activating and rendering the graphic object speci cation of the Observer de ned by the VOS described in the next section. The simulator, in addition,

ex(rotating) /move

Moving Stop [obs_view_ direction = dest_direct]

Moving

Figure 9: A Statechart of the NavigationController. The rounded rectangles and arrows represent states and and their transitions. The transitions are described with transition condition (left side of the slash) and with computational actions taken (right side of the arrow). This Statechart is divided into four concurrent processes parted by the dotted lines, and initial states for each concurrent sub Statechart are indicated with the \small circles with arrow pointing out". is capable of performing stochastic data ow analysis, reachability analysis, and nondeterminism analysis. The analysis results, in addition to manually checking the visual simulation (e.g. checking for observer's collision), can be used to identify and correct problems with the behavior and function speci cation.

4.3 Visual Object Speci cation (VOS) Speci cation of form may proceed independently from that of behavior and function. For instance, upon system speci cation explained in section 4.1, form speci cation would start by capturing the initial approximate shape/volume and con guration information of those objects in the campus sketch of Figure 4. Like behavior or function, form is further re ned in an incremental fashion: e.g. by breaking RIST into its six constituent buildings, re ning the interior of the Library, etc. Currently, ASADAL/PROTO does not have a geometric modeller, and thus, such a re nement is realized by selecting domain objects (e.g. buildings, car, classrooms, trees, etc.) from a library. The library objects described by VOS include the following information. Physical Properties and Con guration 6

Class SpatialObject { Shape shape ; public Coord location ; }

Process compute_observer_velocity { Input Vector3D move_vector ; Output Vector3D velocity ; Data const Float gain = 5 ;

Class DynamicObject : SpatialObject { public Vect velocity ; public Vect accel ;

ComputationalSpecification OnArrival(move_vector) ProcessTime(50ms) { velocity := gain * move_vector / abs(move_vector) ; Generate(velocity) ; }

public Vect public Vect ... Eq location Eq velocity ...

angularVel ; angularAccel ; := location + velocity * time ; := velocity + accel * time ;

}

} Class Observer : DynamicObject { ... }

Figure 10: Process speci cation for computing the Observer's velocity.

Figure 11: Form speci cation using VOS.

Typical types of attributes of a VOS include shape, material, position, velocity, acceleration, force, angular velocity, angular acceleration, torque, etc. Users can de ne additional properties as needed.

between the conveyor and the screw. Spatial constraints can be used to de ne a structure similar to a scene graph [7]. A scene graph structure organizes a composite object by a tree, where the coordinate system of children objects (constituents of the parent object) is de ned relative to that of the parent object. When a motion is applied to the parent, all of its children are aected by it as well. To later seamlessly merge the behavior/function speci cation with the object-oriented form speci cation, namely VOS, the DFD and Statechart are transformed into an object-oriented representation as seen in Figure 2. This transformed object has behavior as speci ed in its Statechart, calling appropriate functions (realized as class methods) when an action should be executed, and sends a signal through channels when an event occurs by a state transition. The broadcasted events in Statechart are assigned to \channels" between objects to exchange data or control signals for synchronization.

Primitive Encapsulated Behavior/Function The primitive encapsulated behavior/function refers to the low level behavior/function that the user does not care to control. For instance, a moving object might have a primitive \constant speed" behavior controlled with target location only. Another moving object might have a primitive \accelerate-constant speed-decelerate" behavior instead. The VOS library objects conveniently provide object classes for representing objects with typical spatial motions such as linear acceleration, linear velocity, angular acceleration, angular velocity, etc. Figure 11 shows two meta class templates representing spatial (i.e. occupies some space) and dynamic (changes its properties) objects respectively. Notice that the Observer is a subclass of the DynamicObject which is a subclass of the SpatialObject.

4.4 Hierarchical Re nement and Incremental Validation

Inter-Object Spatial Constraints VOS also speci es explicit physical relationships or constraints among real world objects (e.g. A screw's location follows the tip of the end-eector once grabbed.) It is through the inter-object spatial constraints that make physical objects aect one another dynamically like placing a screw on a conveyor belt realized by explicitly de ning a structural constraint

With what has been explained so far, a simple system validation is possible through simulation of a navigating observer following the path described in Figure 4. The overall re nement and stage-by-stage simulation can proceed iteratively, until the nal desired 7

behavior is obtained.

MovingtoPIRL /next_dest := (130, 102, 20) ; stop_time := 5 ; move_mode := CONST

Observer Velocity Controller

move_vector

Moving toDoor turn_and_move_end

Stop T

profile

/next_dest := (134, 110, 20) ; stop_time := 10 ; move_mode := ACCEL

tm(en(stop), 5) /turn_and_move_end

generate_ velocity_ profile

Stop

Moving toRoom

turn_and_move_end

generate_ velocity

Moving toDoor turn_and_move_end /next_dest :=(130, 102, 20) ; stop_time := 10 ; move_mode := CONST

velocity

Figure 12: The newly re ned DFD for of the process

NavigationController

MovingtoRoom

Turn_and_Moving

Navigating /next_dest := (90, 120, 10) ; stop_time := 5 ; move_mode := CONST turn_and_move_end /next_dest := (130, 100, 20) ; stop_time := 10 ; move_mode := ACCEL

Moving toPIRL Moving toSB turn_and_move_end /next_dest := (70, 80, 20), stop_time := 15 ; move_mode := ACCEL

Moving toEB2

Moving toElevator

Figure 14: A re ned Statechart for MovingToPIRL. Now the Observer enters the PIRL building and looks around.

compute observer velocity

Moving toRIST

turn_and_move_end /next_dest := (140, 105, 20) ; stop_time := 7 ; move_mode = ACCEL

tmd(compute_observer angular_velocity) /rotate

Idle

T

ex(rotating) /move

tm(en(stop), 5) /turn_and_move_end

Rotating

turn_and_move_end

Moving

Moving toDoor

Moving toLibrary

Stop rotate

[obs_position move = next_dest]

Rotating

Moving toBlackboard

Stop

ex(moving)

Stop

turn_and_move_end /next_dest := (137, 114, 20) ; stop_time := 7 ; move_mode := ACCEL

turn_and_move_end /next_dest :=(134, 111, 20) ; stop_time := 10 ; move_mode := CONST

[obs_view_ direction = dest_direct]

Moving

Moving toDoor

turn_and_move_end /next_dest := (134, 113, 20) ; stop_time := 10 ; move_mode := ACCEL

Moving

Stop

turn_and_move_end /next_dest := (120, 60, 20) ; stop_time := 5 ; move_mode := ACCEL

turn_and_move_end /next_dest := (95, 60, 20) ; stop_time := 10 ; move_mode := CONST

Stop

Turning

tm(en(stop), stop_time) /turn_and_move_end

/next_dest := (134, 111, 20) ; stop_time := 5 ; move_mode := CONST

Moving toChair

Figure 15: A re ned Statechart for MovingToRoom. Now the Observer roaming the PIRL building, enters a classroom.

Figure 13: The updated Statechart of NavigationThe added/modi ed portion is shown within the long rectangles.

Controller.

can expect that the virtual campus model to grow into a more complex one, adding environment objects like classrooms, hallways, etc. This will in turn require revision/re nement to the navigation behavior. In the original navigation path speci cation, the Observer only passed by the PIRL building. We might opt have the Observer to enter the PIRL building and navigate through its hallways and visit a classroom. A re nement to the state, MovingtoPIRL, of the Statechart in Figure 13 is shown in Figures 14, which speci es the navigation path within the PIRL building. Figure 15 re nes one of the states in Figure 14, the MovingtoRoom, and speci es the navigation path within the classroom. Figure 16 shows a snap shot of views of the Observer in simulation according to

For instance, we might decide to re ne the \constant speed" movement in a way that a pro le with an initial acceleration and nal deceleration can be de ned. The newly re ned DFD with a subprocess block that computes the velocity pro le is shown in Figure 12. The behavior of the velocity controller will have to change so that depending on a new datum, move mode, it either uses the \constant speed" or \accel constant decel" mode. The original higher level DFD and Statechart would be modi ed accordingly, to accommodate the new datum. The above example illustrated a re nement of the motion model. Another possible behavioral re nement opportunity lies in the path speci cation. One 8

those speci cations.

ing this work, for instances, user-assisted code generation (from the given speci cations), checking consistency among information across form, function, and behavior through expression and satisfaction of dynamic constraints, integration of a geometric modeller for more exible form re nement, veri cation of realtime properties using temporal logic and etc.

Acknowledgments The research described in this paper has been supported in part by the Science and Technology Policy Institute (STEPI) of the Korean Ministry of Science and Technology under the project named \Real-Time Simulation."

References [1] R. Pausch. The Software Development Environment. Implementing Virtual Reality, SIGGRAPH 93 Course Note 43. pages 1.6.1-1.6.6, 1993 [2] C. Shaw, J. Liang, M. Green, Y. Sun. The Decoupled Simulation Model for Virtual Reality Systems. Proceedings of the ACM Computer Graphics Conference (SIGGRAPH '92), 1992

Figure 16: Behavioral simulation with the speci cation of Figure 15. The Observer is at the door (upper left), then moves toward the board hanging in the corner of the classroom (upper right), moves to the chair (lower left), and nally, returns to the door for exit (lower right).

[3] B. W. Boehm. A Spiral Model of Software Development and Enhancement. IEEE Computer, pages 61-72, May 1988 [4] D. Harel, H. Lachover, A. Naamad, A. Pnueli, M. Politi, R. Sherman, A. Shtull-Trauring and M. Trakhtenbrot. STATEMATE : A Working Environment for the Development of Complex Reactive Systems. IEEE Transactions on Software Engineering, Vol. 16, No. 4, pages 403-414, April 1990

5 Conclusion and Future Work Development of virtual reality software with complex virtual environment is very dicult because the programmer must manage and design three things that are interrelated, \form", \function" and \behavior," at the same time. In this paper, we have presented and demonstrated a software engineering approach to building speci cations for form, function, and behavior of a VR application. The proposed methodology is embedded in a support tool called the ASADAL/PROTO with representation constructs such as the Statechart, DFD, and VOS. The speci cations can be made in an hierarchical and incremental fashion, and be simulated for validation purpose. The main objective of applying such a method would be to increase our level of con dence at early stages of development and be able to later modify, revise, and reuse the software. As for future research topics, we are currently exploring several avenues for extend-

[5] K. C. Kang and G. I. Ko. PARTS : A Temporal Logic-Based Real-Time Software Speci cation and Veri cation Method. Proceedings of the 18th International Conference on Software Engineering '95, pages 169-176, Seattle, U.S.A., April 23-30, 1995 [6] P. Strauss and Rikk Carey. An Object-Oriented 3D Graphics Toolkit. Proceedings of the ACM Computer Graphics Conference (SIGGRAPH '92), pages 341-349, 1992 [7] VRML Architecture Group (VAG). The Virtual Reality Modeling Language Speci cation Version 2.0. VRML Architecture Group (VAG), Aug, 1996 (Available at www.vrml.org/VRML2.0/FINAL) 9

[8] P. Fishwick. Extending Object-Oriented Design for Physical Modeling. To Appear in the ACM Transactions on Modeling and Computer Simulation, 1996 (Draft available at www.cise.u .edu/~fishwick) [9] J. Lee, H. Kim, K. C. Kang, G. J. Kim. Prototype = Function + Behavior + Form. ACM Software Engineering Notes, To Appear in July 1998 [10] Vega User's Manual. Paradigm Simulation, Inc., 1996 [11] IRIS Performer Programming Guide. Silicon Graphics, Inc., 1994 [12] R. Mascarenhas, D. Kurumuri, U. Buy, R. Canyon. Modeling and Analysis of Virtual Reality Systems with Time PetriNets. Proceedings of the Intl. Conference on Software Engineering, 1998 [13] WorldToolKit Refernece Manual. Sense8 Corperation, 1996. [14] R. Pausch. Alice: Easy to Learn Interactive 3D Graphics. http://alice.cs.cmu.edu

10