Published in Proceedings of the VRML’97 Symposium (Monterey, Ca., Feb. 24-26, 1997), ACM SIGGRAPH, 87-94
Populating the Internet: Supporting Multiple Users and Shared Applications with VRML Wolfgang Broll GMD — German National Research Center for Information Technology Institute for Applied Information Technology (FIT) D-53754 Sankt Augustin, Germany email:
[email protected] www: http://orgwis.gmd.de/VR
Abstract VRML —the Virtual Reality Modeling Language— has established as the standard description language for 3D worlds on the Internet. Although it recently has been extended by features in order to support behaviors and user interactions, enabling authors to realize interactive virtual worlds, there is still a lack of support to share such worlds with other users. In this paper we will present our approach on supporting large-scaled shared virtual worlds based on VRML by an appropriate network infrastructure as well as mechanisms to partition these worlds. We will show how virtual worlds can be populated by world-wide distributed users and applications, and the requirements to share behaviors and interactions between those. CR Categories and Subject Descriptors: C.2.2 [Computer Communication Networks]: Network Protocols; C.2.4 [Computer Communication Networks]: Distributed Systems - Distributed applications; H.5.1 [Information Interfaces and Presentation] Multimedia Information Systems - Artificial Realities; I.3.2 [Computer Graphics]: Graphics Systems - Distributed/network graphics; I.3.6 [Computer Graphics]: Methodology and Techniques - Interaction Techniques; I.3.7. [Computer Graphics]: Three-Dimensional Graphics and Realism Virtual Reality; Additional Keywords Virtual reality modeling language (VRML), multiuser environments, subdivision of shared virtual worlds, multicasting.
1
INTRODUCTION
VRML has established as the standard for the exchange of virtual worlds on the Internet. While the first version of VRML [2] was a static scene description language, its second version [3] Permission to make digital/hard copies of all or part of this material for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication an its date appear, and notice is given that copyright is by permission of the ACM, Inc. To copy otherwise, to republish, post on servers or to redistributed to lists, requires specific permisission and/or fee.
VRML '97, Monterey CA USA 1997 ACM
[12] [22] as well as other VRML related extensions [1] [7] provide the possibility to create interactive virtual worlds. However the VRML community has already started to go for the next step, allowing several users distributed world-wide to share contents of virtual worlds and VR applications on the Internet. While the realization of shared VRML worlds within the current VRML 2.0 standard might only be realized by a skilled programmer (if the browsers supports a scripting language such as Java which provides appropriate network communication mechanisms), a more general approach is required to give a large number of users the ability to author shared VRML worlds. Several prototypes already exist, providing certain multi-user extensions to VRML [6] [8] [13]. Recently a new initiative called Living Worlds [15] was started by a couple of companies in order to provide a framework for networked VRML based on individual (company specific) extensions. In its initial draft it does not define standards for the exchange of data over the network, but allows each provider of multi-user technology (MuTech) to set up appropriate servers and define appropriate network protocols. To provide a uniform node interface to VRML worlds, the required extensions are encapsulated by rather complex mechanisms based on prototyping and scripts. Since network communication of each shared object might be handled by its particular multiuser extension, a single shared worlds containing several shared objects, might require several multi-user extensions of several vendors running as part or beside the local browser at the same time. Additionally the realization of avatars and shared objects or applications is difficult to handle for the author of a virtual world. While this approach provides a suitable platform for providers of network services as well as related extensions (avatars, applications, authentication services), it is far to complicated in its current version to be used as a new standard for multi-user distributed VRML. In our strong opinion all multi-user, multi-application and multi-author extensions to VRML should be based on simple vendor independent mechanisms. For authors of virtual worlds it should be a minimal effort to make these worlds sharable among several users. The same applies for applications. Users should be able to represent themselves by simple but powerful mechanism, in order to support a wide range of possible avatars. In this paper we will present our approach to extend VRML and related network communication by simple mechanisms, in order to support large scaled virtual worlds populated with users and applications. In the second section of this paper we will focus on the support of large virtual worlds distributed world-wide on the Internet. We will present the mechanisms used in our current prototype to support a flexible communication infrastructure as well as the partitioning of virtual worlds into several regions. In the third section of this paper we will introduce our approach to rep-
W. Broll
Populating the Internet: Supporting Multiple Users and Shared Applications with VRML
resent users and applications in shared VRML worlds and show how this approach applies to mechanisms introduced before. The final section of the paper deals with the synchronization mechanisms required in order to share virtual world contents and how these mechanisms might be applied to VRML.
2
•IP multicasting is not a reliable protocol. For that reason packets might get lost or duplicated. This is not critical for the transmission of avatar positions, however it is relevant for the transmission of single events based on user interaction. •Simple resend mechanisms fail for multicasting connections, since transfer failures often depend on the individual recipient. •Existing extensions on top of IP multicasting (e.g. RMP [19] or [23]) do not seem to fit very well for the task. They usually assume a rather static set of participants, but not a continuous flow of joining and leaving users. Additionally they usually require additional peer-to-peer connections between all participants to realize reliability. •There is still a large number of potential users which do not have access to IP multicasting. Thus a general architecture has to consider these user as well. •Update messages sent to other participants during the initial download phase of the scene description do not reach the new participant. Again, this is not a problem for simple avatar updates, but causes serious consistency problems in interactive worlds.
SUPPORTING LARGE-SCALED DISTRIBUTED WORLDS
In the near future people from all over the world will meet in shared virtual worlds to participate at events, to go shopping or just to meet friends. Some of these places will become as popular as real world places and thus be similar crowded. Thus a communication infrastructure for world-wide distributed virtual worlds has to provide mechanisms to support hundreds or even thousands of users. This requires a flexible and scalable network protocol as well as the possibility to subdivide large virtual worlds into several parts.
2.1 A Scalable Communication Infrastructure
2.1.2 Our second prototype
Currently VRML worlds are downloaded from HTTP daemon servers using the HTTP protocol. While this TCP/IP based protocol is very suitable for reliable file transfers, it does not seem to be an appropriate basis for sharing virtual worlds. Shared multi-user worlds require synchronization messages to be sent from each client to all other clients in order to keep the shared environment consistent. While direct connections between all clients, or a central server and all clients have failed in earlier work, existing systems such as NPSNET [24] have already shown, that a large number of users can share a distributed virtual world if supported by a scalable communication infrastructure.
The second version of our prototype (see figure 1) provides several extensions to overcome these problems and to allow the full support of interactive shared virtual worlds.
2.1.1 Our first prototype The first version of our multiuser VRML prototype [6] was already based on IP multicasting connections [14] in order to realize a scalable communication infrastructure. Nevertheless the approach was straight forward: •the communication parameters were transmitted as part of the VRML scene file by an traditional HTTP daemon server, •after joining the multicast group, each new participant sent his or her avatar to all other participants via the multicast connection, •all updates on the avatars’ positions and orientations were transmitted via this connection as well, •a second daemon at the server site added the avatars of current users to the scene file and updated them according to movements of the participants. Thus new participants as well as users of standard VRML browsers got a snap-shot containing all currently present avatars when downloading the world file. Although this approach works pretty fine for entities which frequently send updates on their current state (such as avatars), a number of problems were encountered when extending our prototype to support interactions and behaviors:
Figure 1: SmallView —our prototype of a multi-user VRML browser The HTTP daemon server and the second server daemon used in the first prototype are replaced by a single multi-user daemon. This daemon provides TCP/IP connections for reliable initial download of shared virtual worlds. In order to prevent a gap between the state of the transmitted world and the shared world, the daemon keeps incoming events and sends them to the new participants after the transmission of the contents of the world is completed (see figure 2). The connection parameters are still transmitted as part of the scene description. They are now part of
-2-
W. Broll
Populating the Internet: Supporting Multiple Users and Shared Applications with VRML
TCP/IP connection world contents and events(2)
connection, applying them to the virtual world and distributing the world’s contents to new participants. Additional server daemons to support unicast connections may or may not provide their own local copy of the shared virtual world. In the first case (proxy daemons) they can be used as additional dial-in points for the virtual world and by that providing full recovery services. However, all additional server daemons require access to the multicast group in order not to slow down the performance of the multi-user daemon.
multi user daemon
connect to world(1)
new participant
new connection to be established(3)
multicast group
Figure 2: Connecting to a virtual world
2.2 Sub-Dividing Large VRML Worlds
additional fields added to the VRML 2.0 WorldInfo node. Individual multicast connections for the transmission of world updates and audio transmissions might be specified. Additionally one or several unicast connections can be provided for non-multicast capable browsers and for recovery purposes. The participant (client) will than either connect to the multicast group or to one of the unicast addresses (if it is not multicast capable). As soon as this connection has been successfully established the initial TCP/IP connection is closed. The participant will now send and receive events via the new established connection. If the participant wants to be represented by an avatar, this is usually transmitted first. The multi-user daemon acknowledges all events sent to the multicast group by consecutively numbered messages. Each message might acknowledge several distributed messages. If the sender of an event does not receive the acknowledge message within a certain time it should resend the message. If a recipient of a message receives an acknowledge but has not yet received the full message, or if it detects missing acknowledge messages, it has to contact one of the central daemons providing unicast connections to receive the missing messages.
In large virtual worlds the number of objects which require a certain synchronization or update messages to be transferred over a network can slow down the interaction of the individual user with the shared world in an unreasonable way. On the other hand, the user usually realizes only some parts of his or her surrounding. A solution for this problem is the subdivision of large virtual worlds into several regions or zones [4] [10] [20], and to link several worlds together by mechanisms which do not disturb the user during navigation. Only those parts of the virtual world currently visible to the user require to receive updates on their contents. By that, the amount of required network messages can be reduced significantly.
2.2.1 Regions To achieve a reduction of the required network traffic, each region needs to provide its own communication parameters. In NPSNET [24] for example, the landscape is subdivided into hexagonal regions, each with its own multicast group. An area of interest manager (AOIM) [16] manages the regions currently visible to the user. However, this approach does not fit very well to general purpose virtual environments. In our approach a region allows the subdivision of a large virtual worlds into arbitrary parts. For example in a virtual town the streets as well as each building might be represented by individual regions. Regions might even build hierarchies, e.g. a lecture room might be a sub-region of the building. Our approach on regions is based on a new VRML Region node. This node specifies •the transformation,
multi-user daemon
participant participant participant
participant participant participant
multicast group
multicast capable
participants
multicast capable participants
proxy daemon
relay daemon
participant participant participant
participant participant participant
•the radiation, •the horizon, •the hull, •the external representation, •and the contents of the region. The contents are specified by an URL. By pointing to virtual worlds specifying their own communication parameters within WorldInfo nodes, a virtual world can easily be split into several independent communication groups (e.g. one multicast group for each region). Providing an additional scaling factor as part of each WorldInfo node, allows us to realize a micro or macro world by scaling all entities (objects, user avatars, applications) entering such a world. The transformation specifies the position and orientation of the region within the current scene. The radiation defines a space. Users whose viewpoints are inside this space can realize the contents of the region. Users outside this space will only realize the external representation of the region. Regions are realized by grouping nodes [22]. The children of the region node define its external representation. Different external representations can be applied by using the VRML level of de-
non multicast capable participants IP multicasting unicast (UDP/IP)
Figure 3: Multi-daemon architecture Although our multi-user daemon also supports unicast connections, we usually use additional server daemons (see figure 3) (located on different hosts) to support this kind of communication. The reason is, that with the increasing number of unicast connections, the multi-user daemon would become the bottleneck of the system. Without that burden it is still responsible for receiving and acknowledging events via the multicast
-3-
W. Broll
Populating the Internet: Supporting Multiple Users and Shared Applications with VRML
2.2.2 Portals
tail mechanism (LOD nodes). The horizon of a region defines the space which can be realized from within a region. Finally the hull defines a space which has to be larger than the radiation. As soon as an user’s viewpoint enters the hull, the browser starts to load the contents of the region. By that the contents of the region can be displayed immediately to the user when entering the radiation space. If the hull is not specified, browsers should estimate when to start loading the contents of the region. The Region node might be defined by the VRML 2.0 prototyping mechanism using ProximitySensor nodes to detect the viewpoint of the user and scripts to check if a user is within a specific space (hull, horizon, radiation), to load the region contents into a Switch node, and make them visible to the user. This would allow regular VRML 2.0 browsers to parse such worlds and make use of regions. However to reduce network traffic by different communication groups, as well as to realize unloading of regions no longer visible, an extended VRML browser is required.
Portals or gateways are another mechanism to support large virtual worlds. Similar to regions, portals provide their own transformation, a hull, an external representation as well as a URL to its contents. However in contrast to regions, portals rather realize a more sophisticated kind of anchors. As soon as the user’s viewpoint or avatar collides with the portal representation, the user is beamed to the specified world. Portals might also be used for transitions within the local scene. The hull of portals is used similar to the hull defined by regions. Portals could also be approximated by VRML 2.0 prototypes. However the portal contents could only be read as an inlined world, since there is not support for prefetching virtual worlds.
3
user 1 (4)
SUPPORTING MULTIPLE USERS AND SHARED APPLICATIONS
In order to make shared virtual worlds attractive to users as well as service providers, a flexible mechanism is required to represent users and applications. In this section of the paper we will introduce our extensions to support these features.
user 1 (3)
user 1 (2)
3.1 User Representation
user 2 region contents
In order to realize the impression of sharing a single virtual world, each user or participant requires an adequate representation. This representation —usually called avatar— allows users to be aware of other users as well as their interactions. Avatars might depend on the virtual world or on the user represented by it. In a common meeting environment each user would like to be represented by a very individual avatar in order to make other users remember him. In certain applications, e.g. a football game, a training environment, or a combat simulation, however, the user might only be able to select an avatar provided by the virtual world author or use avatars of certain types. Avatars should be able to provide their own behaviors. Users might want to define some specific interactions independent of the individual virtual world. Many of these aspects have also been discussed as part of the Universal Avatars proposal [18]. Our approach to realize user representation with VRML is based on further extensions of the WorldInfo node as well as two new VRML nodes: the User node and the Avatar node. All these nodes should be defined using the VRML 2.0 prototyping mechanisms in order to keep world definitions consistent with the current standard.
radiation user 1 (1)
horizon hull
Figure 4: Different spaces defined by regions Figure 4 shows the different spaces defined by a region node and how they influence the view of a user. From a viewpoint outside the region user 1 realizes the external representation only. When user 1 enters the hull of the region (1), the browser will connect to the region, i.e. starting to load the contents of the region. At position (2) user 1 has entered the horizon of the region — he is now visible by user 2, who is within the region, but still cannot realize anything from the region except its external representation. After entering the radiation space of the region (3) user 1 can see user 2 as well as the contents of the region. At position (4), user 1 has left the hull of the region and the browser will disconnect from the region. If the region provides its own communication group, it will disconnect from the multicast group or the unicast daemon. A time out mechanisms is used to prevent flickering when a user crosses the border of the hull frequently. The relation between the contents of the region and its radiation and horizon spaces will differ according to the part of the virtual world represented by the region. In a room, the radiation and the horizon will usually have the same size as the room. In a building, the horizon might be larger than the contents and the radiation in order to see people from inside, without being realized from outside. For landscapes, radiation and horizon usually will have the same size and be much larger than the actual contents of the region. It is also possible to realize some special effects by regions, e.g. by using a radiation larger than the horizon or by regions providing an external representation but without contents.
3.1.1 Users In our approach users are represented by User nodes. These nodes rather contain information on the individual user than on his or her representation. Thus the attributes specified within the User node are shape or avatar independent. In our prototype, User nodes provide the following information: •a unique identifier,
•the name of the active avatar, •the position and orientation, •the default navigation style, •items and personal belongings,
-4-
W. Broll
Populating the Internet: Supporting Multiple Users and Shared Applications with VRML
•default interaction parameters, •shape independent behaviors and •avatars descriptions as children.
•the relative position and orientation, •the navigation style, •items, •interaction parameters •avatar specific behaviors and •avatar appearance as children.
Each user requires a world-wide unique identifier in order to distinguish any two users meeting in an arbitrary shared virtual world. The avatar currently used to represent the user is specified by its name. Each user might define an individual navigation style. Items and belongings are virtual world objects, which the user is carrying, but which are not part of the avatar. While items are visible to all other users, personal belongings are only visible to the user. Users might take or leave items and belongings while moving between several worlds. By that, applications such as a virtual shopping center or market place can be realized. Browsers might provide the possibility of displaying belongings in a separate window. Interaction parameters allow to provide a more realistic interface to virtual world objects as well as to other users. Based on the spatial model for interactions [5] these parameters define three spaces (see figure 5). An interaction range object 2
The Avatar node is a grouping node. The children of the Avatar node define the actual appearance of the avatar as well as avatar dependent behavior (e.g. a walking animation to realize a human avatar). The interaction parameters and the navigation style of the Avatar nodes overwrite the default settings in the User node. Avatars might have their own items. It is possible to transform the avatar of a user in relation to the position and orientation of the user. This is for example necessary to adapt the viewpoint (eye position) to the eye level of the avatar. The avatar type is used to determine a valid avatar, if restricted by the virtual world author. Avatar nodes are bindable nodes, for that reason only one of them can be active at any time at a particular browser.
interaction range audio radiation object 1
3.1.3 Virtual Worlds
audio radiation
We extended VRML 2.0 WorldInfo nodes in order to allow the authors or designers of virtual world applications to provide default avatars or to restrict the valid avatar types of a certain world. The virtual world author may provide an arbitrary number of obligatory as well as default avatars as children of the WorldInfo node. By default it is up to the individual user to use one of his or her own avatars, or one of the default avatars. However if the valid avatar types are restricted, only those types are allowed. At least one default avatar of each restricted type has to be provided in order to make sure, each user can be represented. If the author provides such an avatar as an obligatory avatar, the user has to use the predefined avatar. Further extensions might allow to restrict the total number of concurrent visible and not visible users, or avatars of a certain type, and assign avatars or avatar types to new users automatically.
user 2
audio perception
interaction range
user 1
audio perception
Figure 5: Spaces limiting user interactions which limits the number of objects the user can interact with (user 1 might interact with object 1 but not with object 2), an audio perception space and an audio radiation space. A user can receive audio data from another user of the same virtual world, when the radiation space of the other user and his or her reception space overlap. Thus a communication might be unidirectional only. In the example, user 1 can receive the audio output of user 2, since the audio radiation space of user 2 and the audio perception space of user 1 intersect, but not vice versa. The volume depends on the amount of overlapping and can additionally be influenced by an input and output level. Finally the avatar independent behavior might be specified within the User node. Usually one or several Avatar nodes are provided as sub nodes (children) of User nodes. Each user should have a personal VRML user file, containing a special header as well as a single user node. When connecting to a shared virtual world, this file is transmitted by the local browser to all other participants as well as to the central daemon. When a scene file is received from the multi-user daemon, the current users are transmitted as part of this file. However, they are not part of the original scene graph, but define an individual scene graph for each user. Nevertheless the user’s viewpoint can be influenced by binding the User node (i.e. the user scene graph) to a Viewpoint node. By this, it is for example possible to realize a user within an elevator.
3.2 Shared Applications Our approach allows applications to participate in shared virtual worlds similar to users. Thus service providers may simply connect to a shared virtual world. We realize this approach by Application nodes, which define the transformation of the application in the virtual world, the unique application id as well as an event interface to the application. The child nodes of the Application node realize the appearance and the local behavior of the application object. Application nodes realize there own scene graph —thus they are independent of other virtual world contents. By changing the current position and performing interactions on other virtual world contents, applications might be used to realize autonomous agents. Figure 6 shows a Web browser application, displaying a texture of the current HTML document. From the technical point of view, applications are actually simplified participants of a shared virtual world. They connect to a shared world and use the browser event interface but usually would not display (render) the current scene at their local workstation. They catch events sent to the specified application event interface and can send events themselves to the shared copies in order to modify their appearance or to influence other objects or users.
3.1.2 Avatars Our approach uses Avatar nodes to define the shape dependent attributes of the user representation: •the avatar type
-5-
W. Broll
Populating the Internet: Supporting Multiple Users and Shared Applications with VRML Autonomous behaviors are either completely deterministic or independent of the state of their shared copies. Examples for autonomous behaviors are a clock, a blinking light, the rotation of windmill rotor, or sparkling stars. These behaviors —once defined within the scene graph— usually do not require any synchronization during execution. However, such behaviors might be influenced by user interactions, e.g. activated, deactivated or modified. These cases of course require a resynchronization. Synchronized behaviors are not completely deterministic. This means they can be treated as deterministic for a certain time or within certain threshold values. However, such behaviors require a synchronization from time to time to prevent different users from getting different views of the shared virtual world — i.e. to guarantee consistency. Examples for this type of behaviors are a bouncing ball, a vehicle or a flying bird. Even certain deterministic behaviors require resynchronization, since the behavior of the shared copies might deviate after some time due to the use of discrete time steps and floating point precession. Additionally synchronized behaviors might be computed at a single site and only be approximated at all other sites. This includes dead reckoning mechanisms [21], which also apply to this type of behaviors. Independent interactions are interactions which do not depend on any other interaction performed concurrently. In contrast to behaviors, interactions require an immediate synchronization in order to preserve consistency among the shared worlds. Interactions are independent if they might be executed by different users in arbitrary orders at each site or if they can only be executed by a single user. One example for the first sub-type is a button to ring a bell. This button might be pressed several times by several different distributed users. Even if the actual ringing is performed in a different order the result will be the same at each site. The second sub-type applies, if the behavior is only reachable by one user’s avatar. For example since it is the only one within the local region or the only one with an interaction range including the behavior. Shared interactions occur when several users have the possibility to perform a certain behavior. That means, there avatars share a single region and the behavior is located within their interaction range. In this case concurrent access has to be prevented or resolved in addition to immediate synchronization required for all user interactions. Existing distributed VR systems [11] usually use tokens, master copies or locks based on a perobject basis to prevent concurrent access. However these approaches usually reduce the interactivity of the individual user in an unreasonable way.
Figure 6: A WWW browser application Users might take applications into other shared worlds and even drop them there. This means, applications can be bound to other virtual world entities. The application mechanisms additionally allows us to support agent or soft-bots. Actually there is no difference to other applications (they are represented by a certain shape —their user interface— and perform certain action). However agents might not be restricted to a certain virtual world but follow up hyperlinks and collect or leave items and return to a specific place afterwards. Additional mechanisms are required to position application objects within a virtual world and to allow or forbid all or certain applications to connect to a particular virtual world.
4
SHARED BEHAVIORS AND INTERACTIONS
Since we have now introduced the basic concepts of partitioned virtual worlds and how they might be populated, we will explore the additional requirements to realize the impression of a single shared world. To achieve this, all behaviors performed in any local copy of the shared virtual world have to be synchronized with all other copies of the world. It does not seem appropriate to leave this task to the skilled programmer of behavior scripts. Additionally any approach based on sharing only certain objects as well as their state (as proposed by the current LivingWorlds proposal [15]) instead of synchronizing the whole virtual world would limit the interactivity of the local user or allow partial inconsistencies. In our opinion a simple but comprehensive mechanisms is required to support shared behaviors and interactions between VRML worlds. Messages distributed to achieve this synchronization use the communication infrastructure described in the second section. Nevertheless it is necessary to reduce the required network traffic in order to maximize local interactivity. Different types of behaviors however, require a different amount of synchronization. We further explored this by classifying shared behaviors. We identified four general classes of shared behaviors: •autonomous behaviors
4.1 Our Original Approach In our original approach shared behaviors and interactions are represented by nodes. These nodes contain two or more component nodes which further specify the behavior. Behavior nodes can receive and distribute events. These are either events generated by user input (e.g. mouse, key) or by other behavior nodes. Some behavior nodes e.g. for realizing animations do not require input but issue events depending on the elapsed time. Events may be sent to regular virtual world nodes or to other behavior nodes, resulting in further events and thus realizing an event cascade. To synchronize such worlds in a shared environment, we send copies of each event influencing an virtual world object to its shared copies. However, if the origin of the event cascade was not a user interaction, but an autonomous behavior, the event is marked as already synchronized. Thus, when reaching the virtual object, no synchronization is required, since the same events
•synchronized behaviors •independent interactions •shared interactions -6-
W. Broll
Populating the Internet: Supporting Multiple Users and Shared Applications with VRML
were also created in the remote copies of the shared virtual worlds by the corresponding autonomous behaviors. In order to support synchronized behaviors, behavior nodes require the possibility to specify conditions for additional resynchronizations. If the conditions for resynchronization are fulfilled, the behavior nodes sends an appropriate event to its shared copies. However, which of the shared behavior nodes is the master node in this approach? In our approach it is the copy of the behavior node which was activated by the local user. This means if a synchronized behavior is enabled by a user interaction, this instance becomes the master. If the behavior is already active in initial world, the multi-user daemon server is the owner of the master copy. However, the server might use strategies to migrate such master instances to new clients. To minimize the network traffic required for immediate synchronization of user interactions, our model allows to force behavior objects to distribute their outgoing events to all copies of the virtual world. The local events are then marked as synchronized as well. This mechanism allows to distribute a single event or a small number of events instead of synchronizing the events of each object, reducing the network traffic significantly. We realize shared interactions by a locking mechanism based on a per-interaction basis instead of the widely used per-object basis. In VRML, an object refers to a sub-tree of the scene graph. Thus locking an object would also lock all sub-objects in this part of the scene graph. If one user, for example grabs a briefcase, it would be impossible for any other user to take a book out of it. This however, is no problem with a per-interaction based locking scheme. Since taking the book and taking the briefcase is realized by different interactions, different locks are used as well. In many existing systems a lock can only be achieved by one site, if acknowledged by all other sites. This mechanism is not suitable for distributed virtual worlds containing a large number of objects and users. In our approach locks are either only acknowledged by the multi-user daemon or are not acknowledged at all, i.e. only if a participant tries to achieve a lock on an object, which has already been locked, the lock is rejected. Which mechanisms to use, merely depends on the kind of interaction to be performed. Interactions which require high interactivity (immediate feedback) and will not start a new event cascade or realize significant changes to the virtual world, might use the faster mechanisms without acknowledge, while all other interactions will use the first mechanism. A time out mechanism prevents participants from keeping a lock indefinitely. Locks are also released if the interaction is deactivated by another user interaction. This allows us to realize even soft locks [9]. In addition to these mechanisms, our user and avatar model provides the possibility to further increase the interactivity. As in the real world users can interact with objects only if they are within a certain distance. Our user model takes this into account by the interaction range. We can now forego acknowledge messages if the interaction is attached to an object, which is not part of the interaction range of any other user.
In VRML 2.0 worlds, events are initiated by sensors and distributed along routes. Although the event model is very different from our approach, some of the basic mechanisms to synchronize behaviors can be applied to this standard. However, due to the synchronous event model used, behaviors cannot directly depend on different types of user input or on user input combined with the current state of autonomous behaviors. On the one hand this limits the number of possible behaviors to be modeled without scripting, on the other hand it simplifies their synchronization. Sensors can be subdivided into those depending on user interactions (geometric sensor nodes, e.g. Anchors, ProximitySensors, TouchSensors) and those independent of user input (time sensors). While geometric sensors can realize independent interactions, time sensors can be used to model autonomous behaviors. A first step would now be to send copies of all outgoing events of geometric sensor nodes to the shared copies of the virtual world. Thus a sensor at the beginning of the event cascade, potentially influencing a large number of nodes by its event outputs requires only a single event for synchronization. Anchors will cause a quit message to be sent to the multi-user daemon as well as all other participants, at least if the destination is not a part of the current world (e.g. a region). The daemon and all other recipients will then remove the user (and the associated avatar) from their local copies of the virtual world. Timer sensors do not require any synchronization events to be sent. Since all modifications to nodes are originated on events issued from sensors (except scripts), even changes to sensor fields do not require any further synchronization. However, these simple extensions would not allow to realize synchronized behaviors or shared interactions. The current sensor mechanism actually does not allow synchronized behaviors e.g. the presented examples. Such behaviors have to be realized within script nodes. Synchronization of time sensors requires only to have synchronized clocks at each participant. This however, is not a VRML specific problem and might be solved by existing solutions such as NTP [17]. Our model for preventing several users from performing a shared interaction can be adapted to geometry sensors. Instead of using locks or equivalent mechanisms on per objects basis, each such sensor would be extended by appropriate fields to provide the locking information. The sensor node will try to apply a lock, when the user starts input or when the sensor becomes part of the interaction range of the user’s avatar. Since script nodes might issue events on their own, they may require a synchronization mechanisms similar to sensors. However since scripts might realize any of the four behavior classes, no default synchronization strategy can be used.
4.2 Applying Our Approach to VRML 2.0
In this paper we showed the basic requirements for realizing multiple users and shared applications within distributed VRML worlds. For that reason we presented our approach of a flexible and scalable network structure to support the communication of hundreds or even thousands of users. We further showed how VRML can be extended in order to support subdivision into regions, allowing large-scaled worlds and populating these worlds with users and applications. We introduced our approach to minimize the required network traffic for realizing consistent shared worlds by considering the individual requirements of par-
5
Although our current prototype meanwhile supports most features of the VRML 2.0 (Moving Worlds) standard, our multiuser extensions are currently still based on the concepts of our original proposal for VRML 2.0 [7]. However, some of the mechanisms discovered can be adapted to the new standard. In this subsection we will show, how the results obtained from our prototype can be applied to VRML 2.0.
-7-
CONCLUSIONS AND FUTURE WORK
W. Broll
Populating the Internet: Supporting Multiple Users and Shared Applications with VRML [11] O. Hagsand. Interactive Multiuser VEs in the DIVE System. IEEE Multimedia Magazine, Vol 3, Number 1, 1996. [www] http://www.sics.se/dce/dive/dive.html
ticular behavior types. Finally we showed the basic concept of applying this approach to the VRML 2.0 standard. In our future work, we will further explore the mechanisms required to support large distributed worlds. This includes mechanisms for automatic partitioning. We will further consider requirements such as privacy, access rights and authentication. Especially a more flexible approach to influence the connection of applications to a certain world has to be developed. We will also investigate the area of agents and how they might be used within multiple connected shared worlds. Finally we will further explore how different types of distributed behaviors can be better supported by minimal extensions to VRML, minimizing the effort of the individual world author to allow other users to share the world.
[12] J. Hardman, and J. Wernecke. The VRML 2.0 Handbook, Building Moving Worlds on the Web. Addison-Wesley, 1996. [13] Y. Honda, K. Matsuda, J. Rekimoto, and R. Lea: Virtual Society: Extending the WWW to support a Multi-user Interactive Shared 3D Environment. In Proceedings of the VRML’95 Symposium, (San Diego, Ca., Dec. 13-15, 1995) ACM, 1995, 109-116. [www] http://vs.spiw.com/vs/ downld1.html [14] Vinay Kumar. MBone: Interactive Multimedia on the Internet. New Riders, Indianapolis, Indiana (1995)
References
[15] Living Worlds. Making VRML 2.0 Applications Interpersonal and Interoperable. Draft 1, Oct. 1996. [www] http:// www2.blacksun.com:1080/livingworlds/spec/draft_1/ index.html
[1] Active Virtual Reality Modeling Language, Microsoft, 1996, [www] http://www.microsoft.com/intdev/avr/avrml.htm [2] A. L. Ames, D. R. Nadeau, and J. L. Moreland. The VRML Sourcebook. John Wiley, New York (1996).
[16] M. R. Macedonia, M. J. Zyda, D. R. Pratt, et al.: Exploiting Reality with Multicast Groups: A Network Architecture for Large-Scale Virtual Environments. In Proceedings of the Virtual Reality Annual International Symposium. VRAIS ‘95. (pp. 2-10). Los Alamitos, CA: IEEE Computer Society Press (March 1995).
[3] A. L. Ames, D. R. Nadeau, and J. L. Moreland. The VRML 2.0 Sourcebook. John Wiley, New York (1996). [4] J. W. Barrus, R.C. Waters, D.B. Anderson. Locales and Beacons: Efficient and Precise Support for Large Multi-User Virtual Environments, Proceedings of the IEEE VRAIS’96 Conference, IEEE Computer Society Press, Las Alamitos, CA, March 1996.
[17] D. L. Mills. Network Time Protocol (Version 3) specification, implementation and analysis. DARPA Network Working Group Report RFC-1305, University of Delaware, March 1992, 113 pp.
[5] Steve D. Benford, and Lennart E. Fahlén. A Spatial Model of Interaction in Large Virtual Environments. Proceedings of the Third European Conference on CSCW (ECSCW’93), Kluwer, Milano, Italy, 1993.
[18] M. Moses, D. Greening, M. Marvit, A. Brush. Universal Avatars. On Overview of Universal Avatars. [www] http:// www.velo.com/avatar/avatar.htm] [19] Robbert van Renesse, Kenneth P. Birman and Silvano Maffeis. Horus, a flexible Group Communication System. Communications of the ACM, April 1996.
[6] W. Broll, and D. England. Bringing Worlds Together: Adding Multi-User Support to VRML. In Proceedings of the VRML’95 Symposium, (San Diego, Ca., Dec. 13-15, 1995), pages 87-94. ACM SIGGRAPH, ACM Press, Dec. 1995.
[20] Bernie Roehl. Distributed Virtual Reality —An Overview. In Proceedings of the VRML’95 Symposium, (San Diego, Ca., Dec. 13-15, 1995) ACM, 1995, 39-43.
[7] Wolfgang Broll, David England, Jürgen Fechter, Tanja Koop. Towards Interactive Virtual Environments: Interaction and Behavior Extensions for VRML, Report WSI96-11, University of Tübingen, Germany.
[21] S. Singhal. and D. Cheriton. Exploiting position history for efficient remote rendering in networked virtual reality. Presence. Vol. 4 No. 2. Spring 1995. pp. 169-193. MIT Press.
[8] CyberHub, Multi-User VR browser, Black Sun Interactive. [www] http://www.blacksun.com
[22] VRML 2.0 (Moving Worlds). [www] http://vag.vrml.org/ VRML2.0/FINAL/
[9] P. Dourish. Consistency Guarantees: Exploiting Application Semantics for Consistency Management in a Collaboration Toolkit. Rank Xerox Technical Report EPC-1995-106, Cambridge, 1995.
[23] Brian Whetten, Todd L. Montgomery, and Simon Kaplan. A High Performance Totally Ordered Multicast Protocol, Theory and Practice in Distributed Systems, Springer Verlag LCNS 938.
[10] T. A. Funkhouser. RING: A Client-Server System for MultiUser Virtual Environments. ACM SIGGRAPH Special Issue on 1995 Symposium on Interactive 3D Graphics, New York, 1995, 85-92.
[24] M. J. Zyda, D. R. Pratt, J. S. Falby, C. Lombardo, and D. M. Kelleher. The Software Required for the Computer Generation of Virtual Environments. Presence 2, 2 (1994).
-8-
