Overlapping and Synchronizing Two Worlds Daesung Jang
Joon-Seok Kim
Chi-Hyun Joo
Department of Computer Science and Engineering Pusan National University South Korea
Department of Computer Science and Engineering Pusan National University South Korea
Department of Spatial Information Engineering Pusan National University South Korea
[email protected]
[email protected] Ki-Joune Li
[email protected]
Department of Computer Science and Engineering Pusan National University South Korea
[email protected]
ABSTRACT
Keywords
Recently social network services in virtual space become popular. In particular, virtual reality services such as Second LifeTM provide a very realistic environment for social networking. With recent advances in sensor technologies and mobile devices, several services of augmented reality are also being developed and provided. In this demo, we show a prototype, which connects these two different worlds - virtual reality and augmented reality - so that a user in real world and an avatar in a virtual space communicate for location-based social network. The prototype is implemented with Second Life for virtual space and augmented reality functions of smart phones and tracking sensors. It provides three basic functions. First, a given indoor space in real world is reflected in virtual space. Second the user in the real indoor space is tracked and her/his location is mapped into the virtual space, and at the same time, the location and movement of avatar in the virtual space are also mapped to the real world and shown via smart phone. Third, two users in the same area, a user in the real world and an avatar in the virtual space, can communicate by exchanging messages via dialog box in virtual space and short message service of smart phones. We expect that this prototype will be extended to provide enhanced and flexible social networking services.
Augmented Reality, Augmented Virtuality, Mixed Reality, Synchronized Indoor Space
Categories and Subject Descriptors
1.
INTRODUCTION
In this demo paper, we present an integration of two promising technologies; online services for social networking in virtual space and augmented reality. Firstly, online virtual world services such as Second LifeTM [1] provide realistic environments for social networking where users meet and socialize with each other, and participate in group activities through avatars. Secondly, augmented reality services become popular with recent progress on sensor and mobile technologies, particularly smart phones equipped with several types of sensors such as digital compass, gyroscope, and accelerometer. While the former type of services reflects the real world into a virtual space, the later services overlay a virtual space on a real world. For this reason, they are called augmented reality (AR) and augmented virtuality (AV) respectively. An excellent taxonomy is given in [2], where the authors give a taxonomy on augmented reality and augmented virtuality in terms of virtuality continuum. As AV and AR are positioned at the two opposite extremities, several types of intermediate technologies are in the middle of this virtuality continuum as shown in figure 1, which is called mixed virtuality.
H.2.8 [Database Management]: Database Applications— Spatial databases
General Terms Applications
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ACM SIGSPATIAL GIS ’11 November 1-4, 2011. Chicago, IL, USA Copyright 2011 ACM ISBN 978-1-4503-1031-4/11/11 ...$10.00.
Mixed Reality Real Environment
Augmented Reality
Augmented Virtuality
Virtual Environment
Figure 1: Virtual continuum and mixed world [2] However, the mixed reality given in [2] is merely a separated world, which may have properties of AR and AV but no mutual interactions between two worlds are allowed. An advanced mixed reality is implemented with a system called cAR/PE, which allows primitive interactions between users from different worlds [3]. Multiple users in different world may have a video conference in a virtual space given by this
system. Several systems have been also implemented to allow users to experience an integrated world of real world and virtual reality. For example, in XIM [4], Dual Reality [5], and TwinSpace[6], a user may interact with others from different worlds in a merged space.
User
User
Reflection of AR to AV Real World
Augmented Reality
Augmented Virtuality
Virtual World
Reflection of AV to AR Synchronized Indoor Space
• location-based communication between users in virtual world and real world. It is also required to implement the functions listed above in easily accessible environments such as desktop PC or mobile phones without any specialized or expensive equipments such as head-mounted display. The environment that has gained the popularity for social networking may be preferred.
3.
ARCHITECTURE
The overall architecture of SyncIS is show in figure 3 and it has been developed with the following environments; SyncIS Server
Figure 2: Integration of Augmented Reality and Augmented Virtuality
Real World
Indoor Context Manager Indoor Spatial Data Manager
Indoor Static Spatial Data Tracking Module
Indoor Dynamic Data Manager
A common weakness of these approaches is that the prototypes or systems were developed with dedicated environments and equipments, which cannot be widely used. Moreover, it is impossible to provide public users with social networking services in these environments. In this demo paper, we present a prototype system called SyncIS (Synchronized Indoor Space), which integrates two worlds -AR and AV-, and allows users from different worlds to communicate for social networking as depicted in figure 2. It is developed with Second LifeTM for the virtual world with AV and smart phone for the real world with AR. The major differences of SyncIS from the previous approaches are summarized as; • with easily accessible devices and services, it provides social networking services to public users, and • the social networking service is offered by locationbased approach with a common spatial reference system. In the demo, we will show the basic functions of SyncIS including firstly the reconstruction of a real indoor space into virtual world of Second LifeTM , secondly the real-time reflection of what is going on in the virtual world on a smart phone screen with augmented reality and vice versa, thirdly the social networking between users in virtual space and real world respectively. In the rest of this paper, we will briefly survey the functions and requirements of SyncIS prototype, present the overall architecture and discuss in implementation issues. A demo example will be given and we will conclude this demo paper.
2. REQUIREMENTS AND FUNCTIONS In order to integrate AR and AV, we need a real-time reflection of a virtual world into a real world with augmented reality and vice versa. And the communication media is also required between the user in the real world and the avatar in the virtual world. The functions of SyncIS are designed to meet these basic requirements as summarized below; • construction of real indoor space into a virtual world, • real-time reflection of the real world into the virtual world and vice versa, and
Virtual World Context Manager
Indoor Dynamic Data Sensors
Social Networking Module Second Life Server
Smart Phone Indoor Positioning
Virtual World Camera Second Life Viewer Augmented Virtuality
2D Bar Code
Augmented Reality User
Figure 3: System architecture of SyncIS
• Second LifeTM : a popular service of virtual world for social networking developed by Linden Lab. • smart phone: a popular mobile device with several types of sensors such as digital compass, gyroscope, camera, and accelerometer • 2D bar code: a simple and efficient positioning technology for indoor space.
(a) Floor plan
(b) Reflection in 2nd Life
Figure 4: Floor plan of test bed site and reflection in the Second Life space The SyncIS server is composed of three major modules, Indoor Context Manager, Virtual World Context Manager, and Social Networking Manager as illustrated in figure 3, each of which corresponds with the functions defined in the previous section. The users of augmented reality and augmented virtuality interact with the smart phone and the viewer client of the Second Life. Note that RFID and 2D
(a) Juliet logs into SyncIS
(b) Juliet enters 2nd Life
(c) Romeo sends a message
(d) Juliet gets the message
(e) Juliet recognizes Romeo via AR
(f) Juliet sends a message
(g) Juliet moves to another room Figure 5: Demonstration Scenario
bar code technology is used for indoor positioning in this architecture, any other positioning technology may replace them if it would provide sufficient accuracy with reasonable cost.
4. EXAMPLES In this section, we briefly present a scenario for demonstration of SyncIS, which was carried out in an indoor environment implemented at Sejong u-City test bed site in Korea. The floor plan of the indoor space is shown in figure 4-a. In this indoor space, RFID readers are installed to track the movement of each user with RFID tag. The virtual space reflected into the Second Life is shown in figure 4-b. In this scenario, two users, Juliet and Romeo exchange messages for social networking, where Juliet is a user in the real world and Romeo is in the virtual world. The scenario is given as following; • step 1: a user, Juliet in the real indoor space logs in SyncIS via her mobile phone (figure 5-a) and enter the Second Life space as an avatar at the corresponding position (figure 5-b). • step 2: another user, Romeo in the the Second Life space, realizes that Juliet is within his chatting area and talks to her via chatting box (figure 5-c). Then this message is sent to Juliet via mobile phone (figure 5-d). • step 3: After Juliet receives this message, she recognizes via her mobile phone with AR function that Romeo is in the same room(figure 5-e). Then she sends a reply message to Romeo, which is to be sent to Romeo via chatting box in the Second Life space (figure 5-f). • step 4: When Juilet moves to another room, her locations are tracked by RFID sensors and reflected into the Second Life space (figure 5-g).
5. CONCLUSION In this demo paper, we present a prototype SyncIS(Synchronized Indoor Space), which integrates virtual world and real world, and provides a social networking function. The contributions of our work are summarized as follows. First, this system allows users from different worlds - virtual world and real world - to communicate each other for location-based social networking, by integrating two technologies of augmented reality and augmented virtuality. Second, it is implemented with the Second LifeTM and smart phones, which are being widely used by public users.
Acknowledgement This work was supported in part by the Brain Korea 21 Project in 2011 and a grant (06KLSGB01) from Cuttingedge Urban Development - Korean Land Spatialization Research Project funded by Ministry of Land, Transport and Maritime Affairs.
6. REFERENCES [1] Second Life. the official website of the second life. http://secondlife.com/.
[2] P. Milgram and A. F. Kishino. Taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems, E77-D(12):1321–1329, 1994. [3] H. Regenbrecht, C. Ott, M. Wagner, T. Lum, P. Kohler, W. Wilke, and E. Mueller. An augmented virtuality approach to 3d videoconferencing. In Proceedings of the The 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, pages 290–291. IEEE, 2003. [4] U. Bernardet, S. B. i Badia, and P. F. Verschure. The experience induction machine and its role in the research on presence. In Proceedings of the 10th annual international workshop on Presence, pages 329–333, 2007. [5] J. Lifton and J. A. Paradiso. Dual reality: Merging the real and virtual, in facets of virtual environments. 33(1):12–28, 2010. [6] D. F. Reilly, H. Rouzati, A. Wu, J. Y. Hwang, J. Brudvik, and W. K. Edward. Twinspace: an infrastructure for cross-reality team spaces. In UIST ’10 Proceedings of the 23nd annual ACM symposium on User interface software and technology, pages 119–128. ACM, 2010.
