a multi server multi user approach for distributed virtual ... - CiteSeerX

WORKSHOP ON VIRTUAL REALITY IN PRODUCT ENGINEERING AND ROBOTICS: TECHNOLOGY AND APPLICATIONS Special issue of: BULLETIN OF THE TRANSILVANIA UNIVERSITY OF BRAŞOV ISSN 1221-5872

A MULTI SERVER MULTI USER APPROACH FOR DISTRIBUTED VIRTUAL ENVIRONMENTS C. RUNDE * , F. DECKER * , D. TALABA**, F. GIRBACIA** Abstract: In this paper a VR architecture is described that enables an easy and very scalable way to build up multi user and multi wall projection systems for distributed applications and generic VR interface integration. The approach is based on a multi server architecture and a strict separation of two layers of control of the distributed virtual environment. The 3D representation is full VRML2.0 compliant thus all VRML sensors, events and sounds can be used. The same interactive 3D worlds run in full immersive VR theaters as well as on desktop systems. The virtual environment includes also a wired haptic system and a voice command system, constituting thus a multimodal interface with 3D visualization and haptic as output modalities and tracking, speech and gesture as input modalities.

Keywords: virtual reality, multi user, multi server, multi channel, multi wall projection system, CAVE, VRML, scalable 1. Introduction Modern manufacturing engineering (ME) has become an interdisciplinary domain that incorporates numerous engineering skills, ergonomists, logistics experts and architects. Their common focal point, the digital factory, is not only intended for digital testing of various aspects of planning but also for serving as a common discussion platform for any intersecting work field [7,9]. The strong intuitive aspects of virtual reality (VR) offer easy communication and interaction means through various modalities [2]. To avoid consistency problems it is desirable

*

to access only one 3D model for all representations [3,10]. Rendering techniques like level-of-detail (LOD), clipping and area-of-interest management may meanwhile be used to ensure reliable frame rates. From the point of view of typical engineering tasks [5, 6, 15] applied to this 3D model, a variety of virtual reality technologies need to be used: − for workplace ergonomics design: a holo bench system with body tracking and data gloves − for layout planning: a non-immersive factory planning table [4,11] with a 2D layout, a 3D camera and brick tracking

Fraunhofer Institute for Manufacturing Engineering and Automation (IPA), Stuttgart/Germany

**

Dept. of Product Design and Robotics, Transilvania University of Braşov/Romania

46

Virtual Reality – Technology and Applications

−

for robot teach-in: a stereoscopic desktop visualization with a 6-degreeof-freedom force feedback haptic device, e.g. a PHANToM − for factory virtual walkthrough: a VR theater for group discussions and presentations Thus the challenge is to provide a number of task related, distributed VR interfaces to one common 3D model, even sharing in real-time. 2. State of the art Collaborative virtual environments have now been developed for many years [12,15]. Overviews and comparisons are given by Bartlett, Manninen [1,8]. Shared high level virtual environments do exist, but those VR systems lack a generic architecture. 3. A multi server multi user approach Concept overview The presented system architecture is based on the strict separation of its VR system management into two layers: (i) a Multi User Server performs the administration of the 3D model and the users; (ii) Virtual Environment Servers coordinate local projections and navigation devices. Multi User Server (MU Server) The multi user server (fig. 1) provides for all distributed clients coordination services concerning the 3D scene: − This server administrates 3D files and distributes them to all connected clients. If clients have the appropriate rights, they can upload new 3D data via the multi user server to the scene. Each client holds the entire 3D scene, just positions and perspectives differ. − The MU Server is responsible for the management of the users, roles and user rights. Users may be allowed to modify a scene, they may just be passive visitors to a 3D scene, or they

−

−

may have no access rights at all. Roles and rights are specific to 3D scenes. Modifications within the 3D scene are a result of engineering work. This requires a management of different 3D scene files, representing different states of development. There is also a dedicated right to store 3D scenes, given to certain users. The MU Server distributes all dynamic events throughout the 3D scene. As VRML2.0 [16] is the data format applied, VRML events are used as the basis for communication between the clients. With this approach, all VRML sensors, i.e. environmental sensors (time, proximity, visibility, collision node), pointing device sensors (plane, cylinder, sphere, anchor, touch) and embedded JavaScript/ECMA Script [16] are used.

Virtual Environment Servers (VE Servers) A number of VE Servers (fig. 1) control subgroups of clients linked to the Multi User Server: − The VE Servers combine subgroups of clients to one virtual environment to be used at one location. Like this, several rendering clients can be coordinated to form a stereoscopic VR theater or work bench. The VE Servers calculate the viewing transformations needed for all projections (front, side, ground, offcenter). − Virtual camera parameters are managed by the VE Server. The eye distance can be varied as well as the amplification of tracked movement. A perspective projection or planar projection can be selected. − Tracking and navigation devices are directly linked to the VE Server. In our application case we used electromagnetic tracking and a space mouse for navigation. Based on this information of tracking and

A Multi Server Multi User Approach for Distributed Virtual Environments

−

−

navigation, the resulting avatar position is calculated. The avatar position is again liable for the VRML acoustic rendering. As each client has loaded the same entire scene, each client also contains the sound files. But for multi wall projection environments it is recommended to use just one acoustic output. In the VR theater at Fraunhofer IPA one multi channel sound system is plugged to the front rendering client to get stereophonic sound in this environment. The VE Server contains a mouse hook function able to redirect incoming mouse signals to the VE Server’s clients. The receiving client can be chosen, depending on the movements of the mouse pointer: If the pointer excesses a display’s edge towards another display, the latter one is set active. With this function a radiofrequency (RF) trackball can be attached to the VE Server computer and to achieve a virtual mouse for all its rendering clients. Being supported by the standard VRML pointing device sensors (see above), it is a

47

great benefit using a trackball as mouse input signal. Scalability and VRML compliance In our environment we use pure VRML but one not VRML compliant extra camera node. This extra camera node is needed for off center projection. The parallel projection mode is approximated by using an extreme narrow camera opening angle and an extreme distance between the camera and the scene. All object functionality (e.g. kinematics) is implemented in VRML embedded JavaScript/ECMA Script functions. We use a standard VRML plug-in as ActiveX component for Microsoft Windows development environments. All interaction sensors, graphical rendering and sound rendering are therefore based on VRML concepts and VRML standards. This also means that the 3D scene can be run in any environment managed by the two servers – from desktop systems up to VR theaters. The VR front-end is thus highly scalable. Recently developed modules These components are clients to the Multi User Server:

Figure 1. The architecture of the distributed VR system

48 − − −

−

Virtual Reality – Technology and Applications

A collision detection based on FreeSOLID [19] and the CyberVRML parser [20] is now used. To incorporate simulations of material flow and robotics, connectors to simulators have been built. For Fraunhofer IPA’s VR theater a 3 degree-of-freedom spatial input force feedback device SPIDAR has been implemented. A PHANToM client is in use for smaller workspaces.

SPIDAR with 8 wires [13, 14], allowing thus a bi-fingered haptic interaction (Fig.3). Completed with a voice recognition system and a tracking device (Flock of birds) the achieved VR system represents a multimodal interface, for Product engineering applications including the following modalities: • 3D visual and haptic as output modalities • Tracking, speech and gesture as input modalities

4. A multipurpose VR system Two types of large scale 3D visualisation systems are currently available on the market: Holo-bench and CAVE. The first system is destined to the study of technical tasks that are performed by a human operator in the posture “seated”. For this reason, it usually has two projection screens, one in front and the other in the horizontal plane. In the case of the CAVE, the user is performing tasks that require a “standing” posture. A multipurpose architecture was developed at Transilvania University of Brasov, able to provide both possibilities for the 3D visualization: CAVE and holobench functionality. Therefore the system is called “holo-CAVE” (fig.2).

Figure 2. Holo-CAVE The visualisation system is accompanied by a wired haptic device type

Figure 3. Haptic device 5. Discussion of results There is only very little synchronization within this architecture. Incoming events from external simulators are forwarded into the scene and each rendering computer works independently form that time on. It must be expected, that the frame rate varies among the clients due to different work load according to the screen content. The worst case in this sense is an event generated by activating a VRML sensor on one client. On this particular client the processing of this event begins immediately. The event is also sent to the Multi User Server and from there forwarded to all other clients. To our own surprise we found that latency effects were very little up to negligible for our applications. An important issue in this context is surely to choose the network

A Multi Server Multi User Approach for Distributed Virtual Environments

structure and network protocols carefully: within our architecture we are using the UDP protocol [17] for the VE Server communication and the TCP protocol [18] for the MU Server communication. In conclusion we may state that this approach offers a very affordable, feasible and flexible way to built up both multi channel projection systems as well as multi user VR systems. 5. Acknowledgements Research described within this paper was supported through specific support activities within the project “VEGA – Virtual Reality in Product Design and Robotics” (FP6 – SSA project ACC contract no. – 16565) funded by the European Commission in the period 20052008. References 1. Bartlett, R.: A Categorisation Model for Distributed Virtual Environments, ipdps, 18th Int. Parallel and Distributed Processing Symp., 2004, p. 231 ctd. 2. Boud, A.C.; Steiner, S.J.: Review of virtual reality applications in manufacturing. Proc. of Second World Manufacturing Congress, Millet, Alta., Canada: Int. Computer Science Convention, 1999, 31-36 3. Craig, D.; Zimring, C.: Support for collaborative design reasoning in shared virtual spaces. Automation in Construction, Volume 11, Issue 2, 2002, 249-259 4. Fjeld, M.; Morf, M.; Krueger, H.: Activity theory and the practice of design: evaluation of a collaborative tangible user interface. Int. Journal of Human Resources Development and Management, Vol. 4 (2004), No. 1; 94116 5. Hindmarsh, J.; Fraser, M.; Heath, C.; Benford, S.; Greenhalgh, C.: ObjectFocused Interaction in Collaborative Virtual Environments, ACM Transactions on Computer-Human Interaction, 7, 4, 2000, 477-509

49

6. Killich, S., Luczak, H., Schlick, C., Weissenbach, M., Wiedenmaier, S., Ziegler, J.: Task Modelling for cooperative work. Behaviour & Information Technology, Vol. 18, No. 5, 325-338 7. Klumpp, B.; Runde, C.: Advanced Solutions for Planning and Control of Fabrications: The Communication Platform Digital Factory. Int. CIRP Design Seminar Design with Manuf. 2000, 175-180 8. Manninen, T.; Pirkola, J.: Comparative Classification of Multi-User Virtual Worlds. In: http://www.tol.oulu.fi/~tmannine/game _design/multi-user_virtual_worlds.pdf 9. Runde, C.: Vertrauen und Transparenz. Kooperatives Engineering am Fraunhofer IPA. Digital-Engineering-Magazin 7 (2004), No .6 10. Runde, C.; Shligerskiy, M.: Kooperation in der Digitalen Fabrik mit VR. wt werkstattstechnik online, Jahrgang 95 (2005); Issue 1-2 11. Sihn, W.; Bischoff, J.; Briel, R. von; Josten, M.: Team Table: A Framework and Tool for a Continuous Factory Planning. Intelligent Systems in Design and Manuf. III Proc., 2000 12. Singhal, S.; Zyda, M.: Networked Virtual Environments: Design and Implementation. Addison-Wesley, 1999 13. Antonya Cs. Butnaru T., Talaba D. Design and development of a wired haptic system for the interaction with virtual mechanical systems. Euromech Colloquium 476, Ferroll, 2006. 14. Talaba D., Antonya Cs. Dynamics of articulated mechanical systems for simulation in virtual environments. Proceedings of the TMCE 2006 Horváth and Duhovnik (eds), April 18–22, 2006, Ljubljana, Slovenia, 15. Tromp, J.; Steed, A.; Wilson, J.R.: Systematic Usability Evaluation and Design Issues for Collaborative Virtual Environments, Presence, 12(3), 2003, 241-267 Specifications 16. Specification of the Virtual Reality Modeling Language VRML2.0 and the ECMA (European association for

50 standardizing information and communication systems) Script (ISO/IEC 14772-1:1997): http://www.web3d.org/x3d/specificatio ns/vrml/ISO-IEC-14772-VRML97/ 17. Specification of User Datagram Protocol UDP: http://www.ietf.org/rfc/rfc768.txt 18. Specification of Transmission Control Protocol TCP: http://www.ietf.org/rfc/rfc793.txt

Virtual Reality – Technology and Applications

Internet resources 19. Internet resources of FreeSOLID: http://sourceforge.net/projects/freesoli d/ 20. Internet resources of the CyberVRML parser: http://www.cybergarage.org/vrml/inde x.html