Interactive Mobile Augmented Reality System Using a Vibro-Tactile Pad Moon-Sub Jin*, Jong-Il Park† Hanyang University
ABSTRACT Smart phone is good platform for hand-held Augmented Reality. As smart phone users grow, Augmented Reality applications are increasing. This paper proposes an interactive Mobile Augmented Reality system using a vibro-tactile pad. The proposed system can provide vibro-tactile feedback for realistic immersive experience. For interactive Mobile Augmented Reality, We focus on providing expressions augmented object’s movements and location information using vibration motors. Through simple memory test application, we prove that the proposed system is useful for providing intuitive knowledge for information of augmented object’s movements and location. KEYWORDS: Haptic I/O, Human factors, User Interfaces, Virtual Reality INDEX TERMS: H.1.2 [Information Systems]: User/Machine System—Human factors; H.5.2 [INFORMATION INTERFACES AND PRESENTATION]: User Interfaces—Haptic I/O: C.3 [SPERCIAL-PURPOSE AND APPLICATION-BASED SYSTEM]: Microprocessor/microcomputer applications: I.3.7 [COMPUTER GRAPHICS]: Three-Dimensional Graphics and Realism—Virtual reality 1
INTRODUCTION
Smart phone functions as cellular phone, television, camera, mp3, Bluetooth and navigator. We can do it many things using smart phone. As smart phone users grow, smart phone applications are increasing especially augmented reality. Smart phone is a good device for implementing mobile augmented reality. Many researchers have already demonstrated mobile augmented reality applications. [1][5] However, most of mobile augmented reality applications provide just visible information such as visible object, sound effect. It is not enough to provide variety of realistic and immersive experience. In this paper, we propose interactive augmented reality system using vibro-tactile pad. Existing vibro-tactile device which is developed by company or other researchers has a glove type device. [2] This device is inconvenient, because user must suit it their hands and device have to be located nearby PC. So we choose a portable type device T-Mobile that is developed by KIST. [3] This device has many advantages that a size of pad is small, weight is low. The developed device consists of 12pieces of vibro-tactile pad, each of which components consists of a vibration isolator and linear vibrator. Above all, T-Mobile can
display both spatial and directional information together through the vibro-tactile display while interacting with mobile applications. We add more hardware functions such as vibration intensity, direction and movement to fit interactive mobile augmented reality system. This proposed system is tracking a visual marker on tactile pad. According to information of augmented object’s movements and location, mobile phone provides variety of vibro-tactile patterns to human palm. The haptic display which is located on palm area was developed by Tan et al.[4] They implemented direction information based on sensory saltation. In their paper, vibro-tactile system is useful to detect a change in a virtual space. We implement one interaction mode in application which provides different vibration pattern based on sensory saltation. Also, according to the z-axis in the coordinates of the augmented object, an intensity of vibration is generated. 2
IMPLEMENT ION
The goal of interactive Mobile augmented reality system is that provide a tactile feedback about movements of augmented object using vibro-tactile pad to user. We focus on providing expressions augmented object’s movements and location information using vibration motors. We implement two kinds of applications. For one, one application express the position information of boxes. For the other one, application which is moving character using touch panel expresses directions information. We classify augmented character’s movements according to eight directions. The augmented character can move eight directions in our application. Each of directions has different vibro-tactile feedbacks based on sensory saltation. 2.1 Overview of Interactive Mobile reality system In this paper, We use a vibro-tactile pad,T-Mobile which is made by KIST. Figure 2 is inserted vibro-tactile pad for our system. It consists of micro controller, numbers of 12 linear vibration motor and Bluetooth. When mobile phone send commands to pad using connected Bluetooth, the pad operates saved tactile patterns. Figure 1 shows the structure of proposed system. When mobile phone AR application executes, mobile phone identify the maker on the vibro-tactile pad. According to the position and orientation of augmented object, vibration patterns are generated.
* email :
[email protected] † email :
[email protected] (corresponding author)
IEEE International Symposium on Virtual Reality Innovation 2011 19-20 March, Singapore 978-1-4577-0037-8/11/$26.00 ©2011 IEEE
Figure 1. Vibro-tactile pad which is made by KIST. The maker is located the front part of the pad patch..
333
3
EXPERIMENT
We conducted a memory test of 8 males and 2 females aged 20 to 25 to verify this system. They were asked to arrange an order when random 9 green boxes came up on mobile phone screen. We did two kinds of experiment. For one, participants had to put boxes in right order only after seeing visible boxes on the screen. For the other one, on the other hand, they could see random boxes and also vibration was added by tactile pad when each set of boxes was generated. After two sets of experiment for each one, we found that the chance for correct answers with the latter was 90% due to visual aid and vibration effect while it amounts to only 60% with the former.
First experiment (only visual effect)
Figure 2. Structure of proposed interactive mobile augmented reality system using a vibro-tactile pad. User can feel the position of displayed green box.
2.2 Application We demonstrate two applications. Figure 3-(a) shows that mobile phone can provide tactile feedback according to the position of green boxes. We divide total numbers of 9 areas which are expressed vibration. If there is only one green box, pad vibrates just one area according to exist a position of one green box. Figure 3-(b) shows the directional information of augmented character. If user touches a screen, character can move to touched point of screen. When screen is touched, we classify what kinds of directions and orientations, and then send commands to pad. As a result, the tactile feedback is in sync with display. Both Figure 3(a) and Figure3-(b) are generated intensity of stimulation by the zaxis in the coordinates of the augmented object. If object is located far from base position, the intensity of stimulation is relatively weak. We classify 5 level of intensity according to the z-axis. The calculated power is changed by a Pulse Width Modulation (PWM) command programmed microcontroller functions in tactile pad.
Correct people
Incorrect people
6
4
Second experiment 9 1 (tactile effect and visual effect) Table 1. The result of memory experiments for proving a proposed system
4
CONCLUSION
We have developed interactive mobile augmented reality using a vibro-tactile pad that can provide augmented object’s spatial and directional information. We prove that the proposed system is more intuitive than before mobile augmented reality system though a memory tests. We want to utilize the system for navigation or education. It will be intuitively achieved. As future work, we will add modules which express other sensations for providing variety of realistic feeling. For example, we will ready to add feedback modules, stiffness and the thermal of environment. ACKNOWLEDGEMENTS This research was supported by Ministry of Culture, Sports and Tourism (MCST) and Korea Creative Content Agency (KOCCA) in the Culture Technology (CT) Research & Development Program 2009 (3rd year). REFERENCES [1]
[2]
[3]
Figure 3. Demonstrate two applications. Left image is (a). Right image is (b). We implement augmented reality application using GalaxyS base on Android, Which is made by Samsung electronics. (a)Display position information of augmented area on the palm. (b)Display directional information of augmented character.
334
[4]
[5]
Georg Klein, David Murray. Parallel Tracking and Mapping on a camera phone. Proceedings of IEEE International Symposium on Mixed and Augmented Reality 2009, pages 83-86, October 2009. Kikuo Asai, Hideaki Kobayashi and Norio Takase. Parm-on haptic environment in augmented reality. Proceedings of International Conference on Human Computer Interaction (IASTED-HCI2008), pages 68–73.2008. Gi-hoon Yang, Moon-sub Jin, Yeonsub Jin and Sungchul Kang. TMobile: Vibrotactile Display Pad with Spatial and Directional Information for Hand-held Device. Proceedings of The 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems. October 2010. H.Z. Tan, R. Gray, J. Jay Young, and R. Traylor. A Haptic Back Display for Attentional and Directional Cueing, Proceedings of Haptics-e. Vol.3. No.1, 11-June-2003. Byung-Kuk Seo, Junjeoung Choi, Jae-Hyek Han, Hanhoon Park, and Jong-Il Park. “One-Handed Interaction with Augmented Virtual Objects on Mobile Devices”. Proceedings of ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry (VRCAI’08). December 8-9 2008
