Multi-user Networked Interactive Augmented Reality Card Game

Multi-user Networked Interactive Augmented Reality Card Game Marissa D´ıaz, Mois´es Alencastre-Miranda, Lourdes Mu˜noz-G´omez, Isaac Rudomin Tecnol´ogico de Monterrey - Campus Estado de M´exico (ITESM-CEM) Virtual Environments and Robotics Laboratory (VERLAB) Carretera Lago de Guadalupe Km. 3, Atizapan de Zaragoza, 52926, Estado de M´exico, M´exico [email protected], [email protected], [email protected], [email protected] Abstract Many efforts to improve interaction in virtual and augmented reality applications do so by including tactile elements to create a link between virtual objects and actions. It is also important to add collaboration, expanding single user interaction to networked user to user interaction. In this paper, we describe a networked virtual card game for multiple users based on a collaborative virtual environment. This game is a networked augmented reality game that uses an inexpensive tangible card flipping device allowing the system to be used in homes,museums, schools. The game is easy to set up and capable of hosting several remote users in a distributed system. We constructed two card flipping devices so we can show complete experimental results with two computers and their corresponding hardware interfaces.

1. Introduction Collaborative games that use tangible interfaces as Human Computer Interfaces (HCI) with objects enhanced by computation, try to link and create a separate virtual environment for the users in order to expand the user’s capabilities through internet communication. Interaction [10] must be made clear by using self explanatory components and must create the illusion that the place where actions are taken is the virtual world. Augmented reality (AR) collaborative games like [7, 6] that place the user in an augmented world empower the players to believe and to enjoy the joint experience freely. The success of a game that has interaction between users depends in great measure, on the communication’s system speed, robustness and the correct replication of events for each player in the game. There is already a number of people working in collaborative AR systems [3, 4, 11, 8]. The main idea in this

kind of systems is to allow multiple users to interact with the same augmented environment. Collaborative AR systems are also used for different games [4, 11, 8] like virtual hockey, virtual tennis and ”Concentration”. However most work mentioned in the previous paragraphs, use face to face collaboration. This means that users interact in the same physical environment in the same location, so it is not remote collaboration. On the other hand, there are several Networked Virtual Environments (NetVEs) or Collaborative Virtual Environments (CVEs) that allow multiple users to collaborate, interact and communicate in a common Virtual Environment (VE) regardless of the geographical locations of the users [5, 12]. In [2] live video is integrated into a NetVE system (called VLNET), so the participants are sharing a mixed environment with real and virtual objects in a teaching application. In this paper, we propose a networked augmented reality game with cards. The application was developed based on a distributed, multiplatform and modular CVE with the capability of mixing real video with the virtual environment. Each user interacts physically with its own tangible HCI and interacts virtually through the CVE with other remote users, all feeling as if they are playing the card game in the same location.

2. Multi user augmented card game People around the world have each day less time to interact physically with other people; multi user games try to provide a safe environment to interact with others. “Memory” is a commonly played card game that can be played alone or with many people at the time. It is simple but enjoyable, and also works as a brain exercise. As in other virtually enhanced games [6, 13], we wanted to create a vast playground that serves as a place to detach from each day activities, but we wanted to take a different

approach. We want to create a game that handle tens of players around the world. In our card game we want to construct a robust application that can adapt to the user’s budget by making the main HCI components low cost and all the elements modular.

2.1. Game description We use a set of cards that when flipped change the virtual world in memory game. Each user has a real board with the set of cards connected to a computer. Each card has a sensing device, which lets the application know when the user flips a card. When a real card is flipped its corresponding image is displayed represented by a 3D model in a virtual board. The virtual model can be seen by all users playing the game on his/her computer. The goal of the game is to find pairs of cards having the same 3D model. Multiple users can play the game, each user has a turn. On each user’s turn, two cards have to be flipped to see the 3D models. If both 3D models are the same, this user earns a point; if not the user must flip the cards again to hide the models. The game ends when all pairs are found. The user with the highest number of points wins the game. Each time that the game is restarted, all the locations of the 3D models are distributed randomly below the cards.

3. Card Flipping Game Interface We wanted to design an interface that allows users to interact remotely with each other and to provide this platform to as many users as possible. Therefore we need the platform to be simple, reliable and low cost. We needed to detect when the user flips the cards.

3.1. Building the flipping interface We decided to use Hall effect switches to sense card flipping. We also use a camera, but did not want to use it to determine the flipping of the cards. The markers would be used as a reference for the player giving him/her a clue of the position of the element and allowing him/her to cheat in the game. Therefore, we want the interface to react instantaneously to the user actions and those actions to be sent to all the players at the same time. Hall effect technology allows us to know the characteristics of a magnetic field giving intensity, polarity and direction. In this particular application, we only need a hall effect switch that reacts when a field of certain polarity (in this case south) is near. INFINEON Hall effect switches 3240 and 921 are extremely stable to temperature and are suited to switch

rapidly. They have their own regulator and chopper stabilization, so they can operate from 4.2 to 24 volts. Each switch operates giving an open collector low output when a magnetic field perpendicular to the Hall sensor exceeds the operate point threshold and it can sink 25mA. This works perfectly to activate systems that have higher or lower voltage operation ranges.

Figure 1. Card magnet arrangement and simplified electrical configuration In our application there is a Hall effect switch for each card placed in the game board, the cards are made with two layers of printed material and a magnet between them as seen in figure 1 left. Applications using magnets have to take into account interference and the characteristics of the environment to avoid noise. In this case this problem is not present due to the magnetic force of magnets nearer to the card prevailing while the forces generated by the other magnets are not strong enough to activate the 3240 Infineon Hall effect switch. It is possible to build a larger board with a grid of Hall sensors depending of the microcontroller resources. For this paper, we built a grid of 3 by 3 Hall sensors, therefore, we have nine cards with 4 pairs and one card without pair. The simplified electrical configuration can be seen in figure 1 right, where the rectangle between the Hall sensors circuit and the serial port of the computer represents the used microcontroller. There we can see the nine hall effect switches that are part of one of the card game boards. Each user or player has his/her own card board connected to his/her computer. The array of sensors is connected to a microcontroller that buffers the information and transmits it to each computer in the system. We use a USB interface to connect the interactive card pad to each PC in the system.

The system for each PC costs around $35 USD and can be migrated to a system that costs around $15 USD if we use the PIC16C432 that is a one time programmable device suitable for production but not experimentation.

We capture the information at 2400 bauds and we setup the internal clock in the microprocessor to send the data each 1.5 ms allowing the communication buffer to be read and USB operations to be performed on real time and the information of each user’s board to be replicated to the other players.

running on a computer connected to the network. Entities are all the objects within the VE. The architecture is designed to have multiple entities in a distributed VE where the complete visualization can be accomplished by multiple participants running in several machines (see figure 2). Each participant can add one or more entities in the VE.For each entity, there is a ’local entity’ and several ’remote entities’, called ’proxies’. A proxy entity is in charge of the render of the virtual object that represents that entity in each participant (as in the figure 2 where a virtual snowman is displayed by all the corresponding proxies). The local entity is only in the participant that inserts the entity in the VE, and is in charge of updating all proxies state.

4. Building multi user capability to the game

For more details about the implementation of the architecture see [1, 9].

3.2. Guaranteeing interactivity

In order to have several users remotely interacting and playing the same game, we integrated the previously described interface into a CVE system. Each user has a card game board connected to his/her computer, as the input device for the CVE system. The CVE system that we developed is based on the Object Oriented Distributed Virtual Reality Architecture (OODVR#). OODVR# is designed for running virtual reality applications in a distributed visual simulation system. Originally OODVR# was designed and implemented for several robotics applications [1, 9]. To our knowledge, this is the first augmented reality game with these features. This architecture was implemented as modules in Java, using Java RMI (Remote Method Invocations) and sockets for communication, Java JMF (Java Media Framework) for video capturing, Java 3D for rendering the common VE with all virtual models, and Java comm for connecting the HCI. This allows us to build a multi-platform system that can be run on several operating systems.

In the case of the “memory” game, each card in the environment is an entity and when a user flips a real card, the corresponding virtual card is also flipped and the 3D model is shown in all the participants. When a user hides a model, all participants also see the model disappear. In the original system, we had developed the capability to transmit video streaming to all participants from each video source connected to any computer in the CVE. In this case, we modified it in order to display the video of the camera of each user as a live texture in the background of the virtual board.

5. Experimental Results We tested our game with two players in two computers and their corresponding flipping interfaces. This is because we only built two card boards. However, this game is scalable, it could be played by multiple users. How many depends on the network and computer speed resources available. We took some pictures during the game. In figure 3 one can see two computers (one desktop and a laptop) running the CVE system with the card game. Each computer has connected its own flipping interface with the real cards and its camera capturing the real board. In that figure, the player on the desktop computer is flipping two cards with different 3D models (a worm and a spaceship).

Figure 2. OODVR# Architecture OODVR# is a multiuser system with ’participants’ and ’entities’. A participant is a new instance of the software

The exact moment when the player flips a card is shown in figure 4 left where the hidden model on that card was a virtual apple. In figures 4 center and right one can see two pairs found in two different games (a pair of snowmen found in the HCI corresponding to the laptop, and a pair of green worms found by the player on the desktop computer). Snapshots showing the VE of the game were taken for the hand interaction, for the pair of snowmen and the pair of worms (figure 5 left, center and right).

Using this low cost interaction equipment, we can give an added dimension to traditional games. The card flipping hardware is useful for many of these and other games and applications.

References

Figure 3. Test with two player with their respective virtual and real boards with cards.

Figure 4. Interaction with a real card is replicated in the corresponding virtual card.

Figure 5. Augmented environment

6. Conclusions In order to transform ordinary objects into interaction tools it is important to maintain conceptual concordance between the actions performed in the virtual world and the real object itself. In this paper, we have shown an interface that introduced new accurate interaction hardware intuitive to the game and a collaborative platform that empowers users to interact remotely. Our system’s tangible interface is a virtual board, the user relates the visual stimuli with the physical sense of flipping the cards in a direct manner, the Hall Effect sensors provided a low cost alternative. The user by moving, flipping and discovering the function of the cards enhances the collaborative experience. All in all we have developed a low cost multi-user networked interactive augmented reality card game that is distributed, scalable and multiplatform.

[1] M. Alencastre-Miranda, L. Munoz, and I. Rudomin. Teleoperating robots in multiuser virtual environments. Mexican Int. Conf. on Computer Science, pages 314–321, Sep. 2003. [2] S. Balcisoy and D. Thalmann. Hybrid participant embodiments in networked collaborative virtual environments. Int. Conf. on Multimedia Modeling, pages 130–137, Oct, 1998. [3] H. Benko, E. Ishak, and S. Feiner. Collaborative mixed reality visualization of an archeological excavation. IEEE Int. Symp. on Mixed and Augmented Reality (ISMAR), pages 132–140, Nov, 2004. [4] M. Billinghurst, I. Poupyrev, H. Kato, and R. May. Mixing realities in shared space: An augmented reality interface for collaborative computing. IEEE Int. Conf. on Multimedia and Expo (ICME), pages 1641–1644, Aug, 2000. [5] T. K. Capin and et. al., editors. Avatars in Networked Virtual Environments. Wiley, New York, 1999. [6] A. D. Cheok and et. al. Touch-space: Mixed reality game space based on ubiquitous, tangible, and social computing. Personal and Ubiquitous Computing, 5(6):430–442, 2002. [7] A. D. Cheok and et. al. Human pacman: A mobile entertainment system with ubiquitous computing and tangible interaction over a wide outdoor area. Int. Symp. on HumanComputer Interaction with Mobile Devices and Services, 8:209–223, Sep. 2003. [8] A. Henrysson, M. Billinghust, and O. M. Face to face collaborative ar on mobile phones. IEEE Int. Symp. on Mixed and Augmented Reality (ISMAR), pages 80–89, Oct, 2005. [9] L. Munoz, M. Alencastre-Miranda, and I. Rudomin. Defining and executing practice sessions in a robotics virtual laboratory. Mexican Int. Conf. Computer Science, pages 159– 165, Sep. 2003. [10] P. G. Ogden. Human computer interaction in complex process control: developing structured mental models that allow operators to perform effectively. IEE Conf. on People in Control, (481):120–125, Jun. 2001. [11] T. Ohshima, K. Satoh, H. Yamamoto, and H. Tamura. Ar 2 hockey: A case study of collaborative augmented reality. IEEE Virtual Reality Annual Int. Symp. (VRAIS), pages 268– 275, Mar, 1998. [12] S. Singhal and M. Zyda, editors. Networked Virtual Environments. Addison Wesley, Reading, MA, 1999. [13] Z. Y. Zhou, A. D. Cheok, T. Chan, and J. H. Pan. Interactive entertainment systems using tangible cubes. IE Australian Workshop on Interactive Entretainment, pages 19–22, 2004.