Structured Design of Interactive Virtual and Augmented ... - CiteSeerX

Structured Design of Interactive Virtual and Augmented Reality Content C. Geiger, V. Paelke, C. Reimann, W. Rosenbach C-LAB Fürstenallee 11 33102 Paderborn +49 +5251 60 60 47 {chris, vox, reimann,bobka}@c-lab.de

1. INTRODUCTION Our paper presents work on user centred prototyping and evaluation of virtual and augmented reality (AR) applications. Most existing VR/ AR research and development has focused on pure technology development and highly specialised applications in the professional domain. We focus on the specific and distinct requirements of end-user applications. It is apparent that end-user VR and AR applications will have to differ significantly from existing demonstrators in both content and technology to become successfull mass-market appliances. However, the constituents of successfull end-user applications still need to be discovered. In order to explore the design space of end-user VR/AR applications we have developed a design process that allows active participation of end-users in the conception, design and evaluation of VR and AR applications. Based on our previous experience in tool support for interactive 3D worlds we developed a framework for the structured design of animated VR/AR content based on reusable elements (components) that can be used in a large number of applications ranging from PC desktop 3D, virtual reality applications to augmented reality scenarios. Our approach includes 1) a design methodology for interactive VR/AR content that should be extended to arbitrary smart 3D environments, 2) high level tools for the design of visual 3D components and 3) a number of relevant application scenarios where we applied our concepts for validation. The combination of the design process with various existing, adapted and newly developed tools enables fast and cost-effective experimentation with new application concepts to accelerate the discovery process. The paper provides a motivation for the adaption of VR and AR technologies and concepts to the end-user domain, provides a detailed description of our design and development process including tool support - and closes with the presentation of some early results of end-user participation in conception, design, implementation and evaluation of end-user applications.

2. Design of interactive 3D graphics Our design methodology [1] combines scenarios, hierarchical decomposition and iterative prototyping with the conceptual model of actors to address the different design requirements like short turn around times, design and test, and usability aspects. A central requirement was the identification of a conceptual model that fits the dynamic and proactive content of smart 3D environments. Since typical conceptual models taken from

multimedia (e.g. based on books, cards, slides) do not match the dynamic and three dimensional nature of virtual environments we have selected an actor based model that is loosely based on the computers as theatre model by Laurel [4] and well suited to communicate about the design. In this anthropomorphic model components of the virtual environment are viewed as actors that are acting out roles specified by the designer. At the same time the technical concept of actors as parallel entities that achieve goals through coordinated actions is powerful enough for a direct mapping to an agent-based implementation of abstract and concrete elements.

3. I4D - A Component Based High Level Library for Virtual 3D Content The design of a 3D animation library for smart virtual environments has to fulfill a number of requirements: 1) actor based metaphor supporting a high level design view, 2) extendible by components (even third-party), 3) independence from operating systems and underlying graphics library, 4) efficient and easy design by providing a programming API, a scripting interface and visual tools

3.1 Actor Based Metaphor An i4D scene consists of a number of actors placed on stages. Actors can be lights, cameras, 3D objects or even software elements without a visual representation. Actors perform actions (e.g. animations, sending messages, sound) by continuously modifying their attributes. A stage is viewed by a number of cameras and displayed on monitors. The assignment of monitors,

Conceptual Model

Structured Design Process

Implementation and Reuse

CONTENT Idea

I4D De sign and Development

cameras, stages and actors can be altered interactively during runtime. Similar to scene graphs, actors can be hierarchically structured. In opposite to scene graphs attributes are parts of i4d actors (e.g. color, transformation). To ensure locality1 when executing complex animations, we introduced a special component called I4D High Level Attribute (HLA) that allows to encapsulate the dependencies of attributes in a local component similar to a directed acyclic graph. For simple example, a virtual spider walk cycle can be animated by construction a HLA WalkPosition. Continuously setting the X/Y/Z-value of this attribute results in setting the orientation of each leg including the relevant joints and links of upper and lower parts – the spider walks. Components for continuously modifying attributes of actors are called I4D actions. Combined with HLAs it is often sufficient to have simple actions (e.g. a keyframing animation ChgVec3fAbs (Change 3D vector absolutely) that animates the WalkPosition attribute from the start position to a new end position with a given speed or time. Once this action has been realized, it can be assigned to all attributes of type 3D float vector (e.g. color, size, position). A number of such simple components have been designed including a LookAt action that makes an object always look at another object or a FollowActor action that follows other actors. Because actors can be 3D objects, cameras or lights this component allows to build complex camera movements or moving spotlight very easily and thus could build a base for intelligent virtual cameras based on cinematographic knowledge or smart spot lights that always highlight important scene features.

library which can be loaded and unloaded, the adaption to new configurations requires only the modification of the corresponding I0 components. The current i4D prototype runs on Win NT and Irix using Open GL Optimizer. A component for pure OpenGL is nearly completed and a Performer port for Linux / Irix and a Direct3D component is under development.

3.4 Scripting vs. Programming Many scene graph API’s offer an efficient but less intuitive C++ API. For the design of interactive virtual worlds we found the technique of rapid prototyping extremely useful. To support this, a script binding (e.g. for Tcl/Tk, Python, Perl) is extremely useful to allow an interactive design of the virtual environment without compiling. Thus, we developed a scripting interface for i4D that allows to integrate i4D in arbirary scripting languages. So far, we use Tcl/Tk and completely integrated our i4D prototype in this language. Based on this integration the development of visual tools for the design of i4D worlds is also very efficient.

4. Visual Tools Using the Tcl/Tk Binding a visual editor for designing i4D-Scenes is currently under development. I4D worlds can be saved and restored using an XML compliant file format.

3.2 Extendibility Our animation system was designed to work seamlessly with a number of different systems (e.g. simulation systems, AI based approaches like multi agent systems) even across networks. Thus it was important to develop a framework that allows to manage different components in a flexible way and to provide a structured and open interface for external (third party) components. To achieve this we implemented a management unit that enables us to dynamically load and unload components even at run time and keep i4D open for third party components. For our projects we are currently building complex components for animated textures, import of different 3D models, physical simulation [5] and augmented reality toolkits [8]. For an easy reuse in other projects i4D worlds can be saved in a XML compliant format and can be loaded from any i4D application. For the integration in existing (Microsoft) applications we developed a prototype for an ActiveX wrapper for i4D and successfully extended a Macromedia Director Application with animated 3D content.

3.3 Independence Due to the large number of combinations of operating systems (Irix, Solaris, Linux, Windows 98/NT/2000) and graphics libraries (OpenGL, Optimizer, Inventor, Performer, Direct3D, XSG...) we decided to build i4D independent from a given combination and designed i4D to consist of two major parts (see fig 1), the main part (80%) consists of a platform / graphics library independent management system called I-layer. The other layer I0-layer encapsulates platform and library specific code. As the I0-layer also consists of components for operating system and graphics

1

Often a problem in scene graph based approaches like Java3D, Performer, Open Intentor,

5. Architecture The design and reuse of i4D components for expressive VR/AR worlds requires a dedicated management unit for handling various issues like selection, parametrization and execution of components. This unit extends concepts of standard 3D graphics based on retained and immediate mode approaches like Java 3D, Direct3D or OpenGL based libraries [2]. Elements of this management unit include: Platform dependent layer: A major part of the management unit was developed independently from existing standard libraries and only a small part provides the functionality to access specific base libraries. Due to the large number of combinations of operating systems (Irix, Solaris, Linux, Windows 98/NT/2000) and graphics libraries (OpenGL, Optimizer, Inventor, Performer, Direct3D, XSG...) most of the i4D core system was built independently from a given combination. The platform independent layer encapsulates platform and library specific code and consists of components for accessing operating system and graphics library. Similar to abstract classes (e.g. ani4D 3D primitive "cube") and concrete implementations (e.g. an Open GL Optimizer Cube) the i4D framework allows the adaption to new configurations simply

by modifying the corresponding platform dependent components (e.g. switching to a Performer "cube"). For certain functionalities a direct access from an ATMIRE applications to the underlying base libraries may be necessary. A special "tunnel" component was designed that circumvents the management framework. Basic

Advanced Domain Specific Components Script ActX

message handler

execution unit

repository unit

FI L E F O R M A T

component loader

Platformdependent layer 3D 3D 3D Libraries Libraries Libraries

MM MM MM Libraries Libraries Libraries

Other MM MM Libraries Libraries Libraries

API

Access Tunnel

Message Handler: Access to components is basically provided using structured messages. The message handler identifies the components that are being addressed, delivers the message and maps it to corresponding function calls. From the sender’s point of view each component offers a number of services that can be addressed by sending an appropriate command. Adding new functions to a component simply increases the set of available services. Repository Unit: All used objects and object types that are presently available in the system are registered by storing their name, unique identifier and access path. This enables broadcast and multicast messaging as well as the forming up of virtual groups of objects that can be addressed as a whole. Component Loader: This unit allows the insertion of new objects and object types into the system and to delete them. A main requirement for this unit is that loading and unloading can be done dynamically during run time. This requires special solutions for dynamic type loading and registering. This approach will extend the design pattern "factory" regarding the special requirements for components in VR and AR domains. Execution Unit: This unit handles the execution of all actions that appear in the system including time consuming functions (e.g. animations) and instant modifications of system elements. It will also employ a suitable mechanism for priority scheduling of actions ensuring a proper handling of execution requests with different priorities. The synchronization of parallel and sequential actions will be organized and a conflict resolution scheme will be designed for parallel actions having shared access to an object’s attribute. Virtual Time Unit: By providing a virtual time component the i4D execution unit has full control over all actions in the system. This enables any i4D application to squash and stretch the system time for arbitrary reasons, e.g. to realize slow motion and time laps or to guarantee predefined frame rates for continuously rendering frames into bitmaps or video files.

6. Examples We are currently working on a number of demo examples that show special features of our current prototype.

I4D was used to prototype an information visualization client for a feasibility study concerning new approaches for sales planning. Within two weeks an interactive 3D applications was developed that allows tp modify the different sales parameter and watch the changes in the 3D visualization. Information filtering and efficient navigation in real time was also supported. For a simulation of interactive 3D content on mobile devices a small 3D application was realized that allows to navigate through a small village, collect items and interact with some objects. Some special i4D components (for collision detection, off-screen rendering and sound processing) were developed for this application.

content we want to supply the designers and content experts of complex virtual environments with tools for the efficient design of future VR/AR applications. The final version will describe our design method and the core concepts of the i4D library in more detail and present the current status of the application scenarios mentioned above. We will also relate our approach to existing related work.

8. REFERENCES [1] Geiger, C., Paelke V., Reimann, C., Rosenbach, W.: A Framework for the Structured Design of VR/AR Content, "Proc. of the ACM Symposium on Virtual Reality Software & Technology, VRST 2000", Seoul, Korea, 2000

The most recent project integrates the augmented reality toolkit ARToolkit from Hitlab [5], University of Washington in i4D. In combination with the other components that have already been developed we aim at providing a high level support for the design of augmented reality illustrations that can be used for interactive explanations of real artefacts (e.g. in museums), technical devices (e.g. for maintenance) and wayfinding (e.g. navigational aids).

[2] Vision 2000. IEEE Computer Graphics and Applications. Vol 20,

7. Summary

[5] http://www.hitl.washington.edu/

With the development of i4D we aim at high level support for the visual components of smart 3D virtual environments. Combined with our structured design approach for interactive animated 3D

Nr. 1, 2000

[3] V. Paelke. Systematic Design of Interactive Illustration Techniques for User Guidance in Virtual Environments. Proc. of the IEEE Int. Conf. On Virtual Reality, IEEE VR 2000, March 2000

[4] B. Laurel. Computers as theatre, Addison Wesley, New York,1991