Mobile Indoor Navigation System in iOS Platform Using Augmented Reality Ibrahim Arda Cankaya Department of Computer Engineering Suleyman Demirel University Isparta, Turkey
[email protected]
Arif Koyun Department of Computer Engineering Suleyman Demirel University Isparta, Turkey
[email protected]
Abstract— Usage areas of mobile phones have increased in the last 10 years. Although there have been improvements in many areas, most of the developments are in the field of positioning systems. Although the people’s lives continue in indoor environments, location-based information system receives data from the satellites, which can detect a person's location in outdoor areas. In indoor areas, satellite signals can cause false information or can be interrupted by the objects in the area. In this study, an indoor navigation system has been designed and developed that only uses the accelerometer, the camera and the compass components on the phone and does not require satellite signals for positioning. To provide independence from the map in this application, augmented reality is applied during the routing process by utilizing built-in camera of the phone and no map is used. Index Terms—Indoor navigation, augmented reality, step detection
I. INTRODUCTION In the 21st century, the most widely used form of communication is the mobile communication that is an indication that smart phones have an important role in people’s lives. At first, using the phones as mobile suggested that land telephones could be portable devices. However, their development speed is high and they are benefited in many different areas. Sensors (accelerometer, compass, etc.) that are located on the smart phones, additional equipment (wristband, Bluetooth receivers, modems and so on) that communicates with telephones and the software developed by users for phones have been an indicator for telephones that they can be used in various areas. People whose works requires mobile phones and people who develop applications can create different applications (games, social media, advertisement, communication) for mobile phones. However, most of the research are conducted in routing and positioning systems [1]. In the last years, location-based systems have become an important topic for researchers. Due to the improvements on smart phones’ battery power capabilities and increased cellular
Tuncay Yigit Department of Computer Engineering Suleyman Demirel University Isparta, Turkey
[email protected]
Asim Sinan Yuksel Department of Computer Engineering Suleyman Demirel University Isparta, Turkey
[email protected]
coverage, applications are being developed in many different areas (roads, restaurants, museums, hospitals, buildings). Guidance and positioning functions of navigation systems enabled different application fields to make use of these systems and occupy an important role in daily life. Positioning and navigation systems presents necessity for various fields of profession [2]. Additionally, users can be informed about helpful activities, announcements and opportunities that provides additional information that are collected from users due to navigation services. Map databases of involved terminals should be established in order to determine the shortest way from current location of user to his/her destination point. Outdoor navigation systems can determine the location of user by Global Positioning System (GPS). Applications that offers outdoor navigation contains preinstalled maps and provides latest updates for map databases. GPS technology is generally used outdoors. However, 95.5% of a user’s daily life is spent in indoor areas [3]. GPS can not provide accurate information for navigation systems in indoor areas. GPS signals are weak or not available. One of the reasons for the lack of signals is blocking objects which are located in the buildings. This blocking objects can be walls and roofs. Also the proximity of the corridors to each other in the building may result in errors in routing [5]. Despite these problems, it should be noted that calculating the present position in indoor navigation is required as it is required in normal navigation systems. Accelerometer that is embedded in smart phones can be used as a function of pedometer and individuals who suffers from immobility can take advantage of it by using it for calculating the calorie intake to tackle the health problems such as obesity [2]. Besides the health assistance, accelerometer feature in smart phones can be used for guidance in indoor navigation services by utilizing pedometer function. Augmented reality is the perception of virtual and real objects at the same time by user [6]. With augmented reality, the following features are included in the scene: • Virtuality and reality are combined. • It interacts in real time.
•
It has 3D utilization
This study presents the application of indoor navigation system that only utilizes smart phones’ accelerometer, compass and camera features and augmented reality without the need for additional hardware or GPS module. II. RELATED WORK Link et al. used the accelerometer and compass of the Android based smartphone and they developed the Footpath navigation system that enables walking to target point from a starting point in an indoor area. Firstly, they created the map of the indoor area where they apply the routing algorithm. Map creation process was done in OpenStreetMap application which is easy to use and allows additional enhancements. After map creation process, user starts walking and steps are identified by using an algorithm that counts steps and users are directed to the target on the map. Since the directions in Footpath application are predetermined on the map that are defined on OpenStreetMap, consistency is provided by comparing expected direction and the direction where the user goes. Incoming data from walk process is filtered by First Fit algorithm [7]. Ravi et al. found user’s current position in the indoor area with smartphones’ cameras in their studies. In their application, they did not use an additional hardware except the features of the mobile phone. The parts that requires additional costs are the remote server, image data and a database where location information of images are stored. Photos of the moving user that are taken by a phone and the location information are instantly sent to the remote server. Photos and the location information were compared in database and when there is a match, the location is determined. Furthermore they considered the person’s height that affects the walking process and they toke photos from 10 different angles and 10 different heights of each point and stored them in the database. Their results showed that the accuracy of the process was 80% [8]. Kolbe reviewed the studies related to navigation for pedestrians and focused on two new approaches by evaluating the previous approaches. One of his ideas is to record the videos of the way that the pedestrian is going and the way that the pedestrian is returning. After this process, they show the information of important locations on the way. The other one is storing the location information of all the intersections on the city to use the navigation system. This information is then integrated and a panoramic view of the area is created. As a result of Kolbe's research, there are two events that can be used in navigation process for pedestrians; a location-based navigation and an augmented reality based navigation for outdoor environments. In the location-based navigation, a person’s location information is received instantly, and the buildings, the ways and the objects around are displayed on the user’s mobile phone. In the augment reality based navigation, the person’s current location is determined by the GPS, the camera, and the compass devices that are located on the person’s helmet. The location information of the buildings, the
roads and the objects in the fields are stored in the application [9]. Liu and Zlatonava developed a new way of finding solutions for indoor navigation systems. They designed their maps in 2D and they employed Dijkstra’s algorithm to find the shortest path [10]. Jimenez et al. attempted to find the best solution for mobile phones with MEMS sensors by comparing step count algorithms which can be used in outdoor and indoor areas. They used MEMS sensors since they were small and lightweight. They placed MEMS sensors to user’s right foot and analyzed the incoming data [11]. Kim and Jun developed and indoor navigation system by using augmented reality. They placed a screen on a helmet, a camera was mounted and a tablet was given to the user. They stored images that was taken from any location of the building and sent to the remote server. Images from the camera along the path of user walking with navigation were send to a remote server with a wireless connection and before the comparing images in database, these images were analyzed by image processing techniques. If the images in database matched with the image of user’s location, they identified the user’s location as correct. They also added the routing function to this system, and showed the direction information on the screen for guidance. They found that the success rate of their study was 89% [12]. III. SYSTEM ARCHITECTURE Our augmented reality based indoor navigation system is composed of two parts. In the first part, the user enters information about the starting location and the target location. After this process, augmented reality section is displayed and the routing process starts and the shortest path is calculated. The system architecture is shown in Figure 1.
Fig. 1. System architecture
A. Pedometer Algorithm According to the user's movements, we need to know the location change to update the momentary location. The user's location changes are calculated with the accelerometer sensor on the mobile phone. Values of three different frequencies that are X, Y and Z axis values are received from accelerometer. Figure 2 shows the frequency values of the accelerometer while the user is walking.
Fig. 2. Unfiltered Accelerometer Data
Z axis shows the changes in accelerometer during the user is walking forward. Lowpass filter has been applied to smoothen the values that is shown in Figure 3.
point where user is heading. Additionally, calculation of the angle between two points is done. D. Finding the Shortest Path with Dijkstra 19 different points have been identified for this study in Computer Engineering, Chemical Engineering and Geological Engineering departments that are located in Suleyman Demirel University. Connections of each points to others and the distance of each connections were calculated. Dijkstra's algorithm is applied to calculate the shortest distance between two points. The reason to choose this algorithm is because it works faster than other algorithms. If the weights between the points take negative values, Dijkstra’s algorithm can face significant problems. Therefore the distance between connected points are never negative which makes the algorithm work quickly and without any problems. 19 points are stored in the database to be used in the application. When the application is executed, registered location information is retrieved to be used in Dijkstra’s algorithm. E. Adding Augmented Reality to the Application Map independency is achieved through augment reality technique. After adding the augmented reality libraries to the application, virtual guidance is provided on the user’s camera. The direction and distance to the target are information that are displayed to user.
Fig. 3. Filtered Accelerometer Data
IV. APPLICATION
Considering the values of the Z axis, it is seen that the portion which have the highest values correspond to each step. However this portions can be counted as step when the user does not take a new step in less 0.4 seconds. Thus, the application prevents wrong step information for short periods.
The user enters required information of his/her gender and the height for calculating the step length in application screen. Furthermore, the user specifies the start and end points. After specifying these points, the augmented reality based routing process is started on built-in camera. The application assigns new target point when user arrives each new point and assign previous target point as the starting point. The routing process continues in this way until user arrives to the target point. The routing screen of the application is shown in Figure 4.
B. Step Length Calculation After determining the number of steps, step length is calculated. The static step length calculation was done for this purpose. For static step length calculation, information of the step length depends on user’s gender and height. Equation 1 is used for calculation of the female users. H represents female user’s height information. Equation 2 is used for calculation of the male user’s step length. H represents the user’s height information. Αω = Η ∗ 0.415
(1)
Αμ = Η ∗ 0.413
(2)
C. Calculation of the Distance and the Angle Between Points During the user routing process, the distance between two points must be known and the distance of the user to point must be updated. For this purpose, the distance calculation between two points in a coordinate plane is performed. With the calculation of the distance between two points, the user’s location must be directed angularly according to user’s target
Fig. 4. Navigation Screen
V. RESULTS To measure the success rate of our pedometer algorithm, we examined the steps (100 steps taken) of 5 different male and 5 different females. Results are shown in Table 1. The average success rate was found to be 95.4%. TABLE I. PEDOMETER ALGORITHM TESTS Person
1. Person 2. Person 3. Person 4. Person 5. Person 6. Person 7. Person 8. Person 9. Person 10. Person
Gender
Male Male Male Male Male Female Female Female Female Female
Real Step Count 100 100 100 100 100 100 100 100 100 100
Number of Step Counted By Pedometer Algorithm 102 91 95 99 99 106 101 96 99 96
Success Rate 98% 91% 95% 99% 99% 94% 99% 94% 91% 94%
Static step lengths are calculated both for male and female users during the routing process. Necessary step count to reach 500 meter distance and user’s step count when he/she reached 500 meter distance are calculated. The results for males and females are shown in Table 2 and in Table 3 respectively. TABLE II. STEP LENGTH TEST FOR MALES Person 1. Person 2. Person 3. Person 4. Person 5. Person
Height(cm) 183 175 177 190 184
Calculated Step Length 75.945 72.625 73.455 78.85 76.36
Expected Step Count 65.83 68.84 68.06 63.41 65.47
Real Step Count 67.25 70.50 66.00 61.75 69.50
Considering the distance between two points does not exceed 100 meters in routing process, we encountered no errors in calculation and the results were not affected. TABLE III. STEP LENGTH TESTS FOR FEMALES Person 1. Person 2. Person 3. Person 4. Person 5. Person
Height(cm) 165 162 167 170 169
Calculated Step Length 68.145 66.906 68.971 70.21 69.797
Expected Step Count 73.37 74.73 72.49 71.41 71.63
Real Step Count 75.25 77.00 73.50 73.75 73.50
VI. CONCLUSION In this study, we have developed indoor navigation application using augmented reality for indoor areas in IOS platform. We developed the application with mapindependence in mind. Therefore, the user is directed by looking at the way with the virtual components thate are placed
on camera and without having to look at the map on the mobile phone. Furthermore we developed a novel pedometer algorithm and employed Dijkstra’s algorithm to find the shortest way to the target location. As a future work, we are planning to use Bluetooth beacon devices in important points on the walking path to verify and correct the user’s location. Thus, user’s location will not only depend on pedometer data but also the data retrieved from beacon devices. As a result, we will collect more precise data and the routing process will be more efficient. REFERENCES [1] O. Köroğlu, "En Yaygın Yeni İletişim Ortamı Mobil İletişim Ortamında İçerik ve Yayıncılık", Civilacademy Journal of Social Sciences, vol. 8, no. 2, pp. 55-86, 2010. [2] N. Z. Naqvi, A. Kumar,A. Chauhan and K. Sahni, “Step Counting Using Smartphone-Based Accelerometer,” International Journal on Computer Science and Engineering, vol 4, no 5, pp. 675-681, May, 2012. [3] J. Chon and H. Cha, Lifemap: “A smartphone-based context provider for location-based services,” Pervasive Computing, IEEE, vol 10, no 2, pp. 58-67, April – June, 2011. [4] R. T. Azuma, “A survey of augmented reality,” Presence
Teleoper Virtual Environ, vol. 6, no.4, pp. 355–385, 1997. [5] M. Werner, M. Kessel and C. Marouane, “Indoor positioning using smartphone camera,” In Indoor Positioning and Indoor Navigation (IPIN), International Conference on IEEE, pp. 1-6, September 21-23, 2011. [6] P. Milgram, H. Takemura, A. Utsumi and F. Kishino, "Augmented reality: a class of displays on the reality-virtuality continuum", Telemanipulator and Telepresence Technologies, Vol. 2351, pp .282-292, December, 1994. [7] J. B. Link, P. Smith, N. Viol and K. Wehrle, “Footpath: Accurate map-based indoor navigation using smartphones,” In Indoor Positioning and Indoor Navigation (IPIN), International Conference IEEE, pp. 1-8, September 21-23, 2011. [8] N. Ravi, P. Shankar, A. Frankel, A. Elgammal, and L. Iftode, “Indoor localization using camera phones, In Mobile Computing Systems and Applications,” WMCSA’06, pp. 1-7,August 1-1, 2005. [9] T. H. Kolbe, “Augmented videos and panoramas for pedestrian navigation”, 2nd Symposium Location Based Services and TeleCartography, January 28-29, 2004.. [10] L. Liu, S. Zlatanova, “ A “door-to-door" Path-finding Approach for Indoor Navigation,” Proceedings Gi4DM: GeoInformation for Disaster Management, May 3-8, 2011. [11] A. R. Jimenez, F. Seco, C. Prieto and J. Guevara, “A comparison of pedestrian dead-reckoning algorithms using a low-cost MEMS IMU,” In Intelligent Signal Processing, WISP, IEEE International Symposium, August 26-28, 2009. [12] J. Kim, H. Jun, “Vision-based location positioning using augmented reality for indoor navigation”, Consumer Electronics, IEEE Transactions on, vol. 54 no. 3, pp. 954-962, 2008.
