However, while it could be said that email and websites have most of ... mobile devices and augmented reality techniques to create a social network that allows ...
LINK2U: AN AUGMENTED SOCIAL NETWORK ON MOBILE DEVICES Davide De Chiara, Marco Romano, Monica Sebillo, Genoveffa Tortora, Giuliana Vitiello. DMI, Università di Salerno, Italy {ddechiara, marromano, msebillo, tortora, gvitiello}@Unisa.it
ABSTRACT Nowadays, modern mobile devices are becoming true personal computer that allow us to exploit new typologies of applications by combining several technologies such as camera, GPS, 3D graphics and permanent Internet connection. The resulting integration of such technologies allows to run complex applications, such as augmented reality and Social Networks. The goal of our present research is to support users’ daily activities, by developing advanced solutions which take into account principles of human-computer interaction and usability. In this paper we propose to join the suitability of mobile devices and augmented reality techniques to create an advanced social network. Besides common functions, it combines the potential of the augmented reality with the ability to “communicate” of the social network. In fact, each community user may exploit advanced location based services, such as navigation through a two dimensional map, exploration of an area through a camera mode, and identification of points of interest embedded in an augmented reality. KEYWORDS Augmented reality, social network, mobile devices, geo localization, web services, human computer interaction.
1. INTRODUCTION Recently, more and more people have realized that the entertainment is a key factor in the improvement of the technology, due to the stimulus it offers to develop innovative solutions for a world-wide community of users. Many efforts are then devoted towards this industry, and it has grown dramatically as a topic of research interest. Among the most pervasive applications which result from these joint forces, a relevant role is being played by social networks and services they provide. A social network focuses on building online communities of people who share interests and/or activities, and are interested in exploring the interests and activities of others. At the present, social networks are being used regularly by millions of people, thus encouraging the research interests towards new ways to communicate and share information. It is in fact undeniable that the validity of a pioneering solution can be confirmed only through the robustness of a methodical approach. As for services, most of them are web-based and offer a variety of ways for users to interact, such as email and instant messaging services. However, while it could be said that email and websites have most of the essential elements of social network services, recently the idea of proprietary encapsulated services has gained popular uptake. The goal of the research we are carrying out is meant to design and develop advanced solutions to support users’ daily activities. In particular, we aim to provide users with services where mobility comes into, such as services to support user’s navigation and location based services (LBS). In order to achieve this goal our studies are focused on human-GIS interaction, advanced modalities of geo-localization and augmented reality. In this paper we present Link2U, an integrated solution for mobile devices which combines the potential of augmented reality with the ability to “communicate” of the social network. In particular, the rationale underlying Link2U is to meet mobile users’ requirements of continuously acquiring information about the
surrounding environment, essential for their activities. To this aim, we propose to join the suitability of mobile devices and augmented reality techniques to create a social network that allows members of a group (SNU: Social Network User) to invoke several functionalities, ranging from traditional functions such as messaging, to the calculation of a route on a map and the identification of points of interest (POIs) through the support of the augmented reality. In this case, the POIs may represent both a SNU and a traditional POI, such as a building and a bus stop. A prototype of Link2U is being tested at the University of Salerno (this notation will be used to preserve anonymity, due to the double-blind review). A general-purpose community has been involved, which includes students and teachers. It allows them to determine their indoor/outdoor position and calculate the path to follow for reaching a target. To build Link2U we exploit a modern mobile device that combines an integrated camera for the video-image capturing, a Global Position System (GPS) device to detect the position, a compass and motion sensors to detect the user point of view. Moreover, WiFi allows to identify user position with a good accuracy inside the area not covered by GPS signal. The remainder of this paper is organized as follows. In Section 2 we recall some recent applications which take advantage of similar technological solutions. In Section 3 we describe our project, which has been experimented through the prototype described in Section 4. Conclusions are drawn in Section 5.
2. RELATED APPLICATIONS: GOOGLE LATITUDE AND ENKIN Nowadays, modern mobile devices have been developed as a true personal computer that increase the ability of building / extending existing applications by combining camera, GPS, 3D graphics and permanent Internet connection, thus providing an intuitive and light navigation system for mobile devices. In the following we recall two applications recently developed, namely Google Latitude [5] and Enkin [2], as examples of a social network and an augmented reality application for mobile devices, respectively. The rationale behind this choice is that they are representative of the aforementioned categories. In particular, the former has been chosen thanks to both its wide world diffusion, the latter is validated by a scientific approach. Google presents a new concept of mobile social network through the service Latitude. When connected, a SNU is allowed to display on a map the position of other members of the group she/he belongs to, as well as she/he can invoke the calculation of the fastest route to achieve them starting from a focus point which corresponds to her/his position. Google Latitude is freely distributed on the latest version of Google Map for mobile devices. Enkin offers a new concept of pedestrian navigation (handheld navigation concept) based on the augmented reality that combines reality with a typical display of maps. In particular, on the screen and superimposed to video image, the user is capable to see the POIs represented in 3D, and can recognize their location outside the view limits of the camera. From a technological point of view, Enkin takes advantage of the camera eye, the GPS, the compass and the motion sensors to detect the point of view. Moreover, through a little radar view-like frame visualized on the display, Enkin also allows a 2D vision of POIs by using the classic map view. Several developers are going in the same direction of Google (for mobile social network) and Enkin (for augmented reality system for navigation). They are realizing applications which benefit from advances in technology and are meant to a wide-world community of users. Our proposal is based on geolocalization technique which integrates GPS and WiFi Positioning System (WPS). A WPS deals with accurately determining the location of wireless access points (APs) to estimate the physical location of a WiFi enabled device, thus overcoming limits of GPS in terms of coverage and accuracy. Nowadays, several geolocalization services based on WPS are available, such as Intel’s Place Lab [3][9] and the Skyhook Wireless WPS [11]. In the next Section, we describe the design underlying Link2U and illustrate its functionality. Moreover, we describe the analysis of the resulting system made in terms of Norman’s interaction model and principles followed in order to realize a system that would work as human expect them to.
3. DESIGNING LINK2U The study carried out on mobile applications offering advanced mobility functionality has stimulated us to design the integrated solution that we propose in this Section. Link2U consists of a project which merges the new concepts introduced by Google Latitude and Enkin to create a social network that exploits potential of both and adds a service characterized by a strong geographic component and by an augmented reality solution that improves the perception of the SNUs. In Link2U SNUs are provided with different functionalities. Among the basic ones, they can belong to some groups, known as contact list, they have a status which is shared with other SNUs of the same contact list, they can invoke the route calculation towards a target. As for the advanced functionalities, SNUs may benefit from two different visualization modalities, namely Map mode and Live mode. The former corresponds to the classic two-dimensional view where POIs are drawn on a map. The latter exploits the augmented reality to improve the users’ sensory perception about POIs. In fact, POIs are designed in a three dimensional environment and are superimposed on the image captured by the camera phone. A SNU can switch between the Map mode and the Live mode by laying down and drawing up the device. These two different visualization modalities allow to provide users with advanced functions which combine the augmented reality support and the global view of a two-dimensional map. It is worth to note that in Live mode only POIs within the field of vision of the camera can be detected and then identified. Such a field of vision has a predefined radius, which can be fixed by the user, on demand. Moreover, in order to avoid confusion due to SNUs crush on the screen, a limited number of them may be displayed by using a visual aggregation technique on both visualization methods. The approach is inspired by the Framy visualization technique [10], which can be used to visually summarize information distributed within an area of interest. The idea is to map meeting places on the basic cartography, such as library, classroom and swimming pool, where SNUs are used to gather. Such places can be detected as Area of SNU Aggregation (ASA) through the icon. When the SNU number / ASA surface ratio is bigger than a threshold value, the icon evokes the presence of a set of SNUs as shown by the associated label. This approach is exploited in Map mode when the zoom level is within a specific range, and in Live mode when the user position is outside the ASA. Figure 1 depicts the visual representation of this visual aggregation technique.
Figure 1. Visual aggregation technique The resulting benefits of LINK2U offer users the capability of both navigating a map towards a target, and recognizing POIs by simply pointing the phone camera in the right direction. In particular, in Map mode a calculated route is displayed as a 2D polyline, along with a label on the target icon indicating the distance in kilometers between the involved points. Differently, a route in Live mode is displayed as a set of waypoints along with labels that contain a progressive number to reach the next waypoint. A little radar view-like frame visualized on the screen displays the near waypoints. Once a waypoint has been passed through its color is modified.
The interaction task that the user performs when achieving a goal, can be modeled in terms of the Norman’s execution-evaluation interaction model which was explicitly conceived to model interaction. The steps characterizing the interaction cycle are: 1. Establishing the goal - At this stage the user is willing to locate POIs and eventually interact with them. 2. Forming the intention - The user wishes to interpret the information about the POI distributions, as conveyed by Link2U. 3. Specifying the action sequence. The user may exploit two alternative human input modalities, namely Live mode and Map mode. 4. Executing the action - The user lays down the device and observes the whole map. The user draws up the device, points it to the area of interest and looks at the scenario. 5. Perceiving the system state. The user switches to the Map Mode and captures the location of POIs within a 2D view. The user switches to the Live Mode and observes POIs featuring the area of interest through a camera mode. 6. Interpreting the system state - Both modalities of visualization allow the users to locate and identify POIs. In particular, the Live mode facility provides users with a more accurate and realistic positioning of POIs by exploiting a best placement thanks to the availability of the third dimension. 7. Evaluating the system state with respect to the goals and the intentions. At this stage the user is capable to evaluate whether the localization and identification of users coming from Link2U allow her to decide with whom to interact. Of course a good mapping should be achieved between the execution and the evaluation phases in order to bridge what Norman calls the gulf of execution (i.e., the distance between user’s specification of the action and the actions allowed by the system) and the gulf of evaluation (i.e., the distance between user’s perception of the new state and her expectation).
3.1 Hardware architecture From a technological point of view, Link2U combines an integrated camera for the video-image capturing, a GPS device to detect the position, a compass and motion sensors to detect the user point of view. Moreover, WiFi allows to identify user position with a good accuracy inside the area not covered by GPS signal. This is possible through a technique of triangulation of the access point positions. Obviously, this service can be offered only within WiFi covered areas, where positions of access points are known. The architecture of the system is client-server based, where the server is responsible for coordinating and updating the positions of the various clients. SNU personal data are stored in the database A. In our case, the server works as a broadcast node, receives information from clients related to their position and their ID, and rebroadcasts such information to all clients. The client represents the SNUs currently connected to the network, recovers their position to make them known to the server and collect information related to other SNUs from the server. The update of a SNU position is delegated to a server that receives the new position and sends it to all SNUs. When the position of a SNU moves significantly from the last recorded position, its device registers the new location and sends it to the server. As shown in Figure 2, the localization of a client is obtained through two methods. The former corresponds to the classical localization through the GPS standard. The latter calculates the position through the triangulation of the WiFi hotspot signals. The technology works by sniffing for the surrounding WiFi networks, then measuring signal strength and comparing those results with a database. This implies that a service provider would need to conduct periodic sweeps of an area to catalog the WiFi hotspots that populate the database B. It worth to note that the client connection is not really required, i.e., the client only scans the hotspots to measure relevant factors used for the database lookup. The location accuracy is proportional to the number of hotspots present in the area around the device. The client collects data related to the signal strength and its SSID, and transmits them to the server, which in turn calculates the position of client according to the data contained in the database and sends an “Update Position” to all clients.
The GeoServices block embeds two servers which offer external employments targeted at generic users, thus reducing the computational work load on the mobile devices. The block communicates via HTTP with the clients, no intermediate node is needed.
Figure 2. Hardware architecture In the next Section we exemplify the usage of Link2U through a scenario where the advanced functionality is illustrated.
4. LINK2U@UNISA The study carried out on mobile applications which offer advanced mobility functionality has stimulated us to design the integrated solution that we propose in this Section. Link2U consists of a project which merges the new concepts introduced by Google Latitude and Enkin to create a social network that exploits potential of both and adds a service characterized by a strong geographic component and by an augmented reality solution that improves the perception of the SNUs. Link2U@Unisa is an initial version of a prototype realized in order to validate some of the functionalities underling the approach previously described. The application has been developed on a G1 HTC DREAM that incorporates the needed hardware along with a Linux embedded system (Android [1]) which allows great freedom in the development of applications. Android includes a set of core libraries that provides most of the functionality available in the core libraries of the Java programming language. It provides applications with access to the location services supported by the device through the classes in the android.location package. The central component of the location framework is the LocationManager system service, which provides an API to determine location. The access to a map is provided by MapView that displays a map with data obtained from the Google Maps service. MapView captures key-presses and touch gestures to pan and zoom the map automatically, including handling network requests for additional maps tiles. It also provides users with all of the UI elements necessary for the map control. The applications can also draw a number of Overlay types on the top of the map. In general, the MapView class provides a wrapper around the Google Maps API [6]. Android also supplies 3D libraries, an implementation based on OpenGL ES 1.0 APIs. The libraries use either hardware for 3D acceleration (where available) or the included highly optimized 3D software rasterizer. Link2U@Unisa uses LocationManager for identifying the current position when the GPS signal is accessible, otherwise it exploits the WiFi hotspot triangulation technique. Map mode uses the MapView framework to view data either on a street plan or on a satellite image. Live Mode displays data in a 3D environment by using the OpenGL ES and superimposes them on video-captured by camera.
Typically mobile devices have some computational limits that can slow the application performances. To overcame this limits we exploit some Web Services, thanks to the Internet permanent connection of the device. In particular, for outdoor navigation a route from a user to a POI is provided by the GPX Driving Direction [8]. As for the indoor mobility, an ad hoc campus cartography and a service have been realized. However, both services receive two couples of coordinates representing source and target points, calculate the path between them and transmit it in GPX format (the GPS Exchange Format)[7]. GPX is a light-weight XML data format for GPS data interchange between applications and Web services on the Internet, such as waypoints, routes and tracks. In Map mode a route is designed according to the list of waypoints in the GPX file. In Live mode, more information are necessary, namely the height of point. Due to the low accuracy of the height detected with the GPS, it can be achieved through the geo-service GeoNames [4]. Finally, by sending the coordinates of each point belonging to the waypoint list the corresponding heights computed at sea level are obtained.
4.1 Scenario In order to prove the effectiveness of the Link2U@Unisa, the following scenario of a mapping navigation and social interaction between SNUs has been run. The scenario is set at the Fisciano campus of the University of Salerno. It consists of four activities, each describing a specific functionality offered by Link2U. Task 1: Monica, a SNU of Link2U@Unisa, looks for her friends present at the campus. Task 2: Monica demands the route calculation towards one of her located friends. Task 3: Monica looks for one of her friends Task 4: Enzo, a foreign student, invokes the route calculation towards a secretariat. In order to accomplish the first task, Monica accesses to Link2U@Unisa in Map mode. The application provides her with a distribution map of the SNUs belonging to her group. Figure 3 shows the resulting map where Marco and Davide are visualized. In particular, Marco is in classroom and his status is busy, while Davide is in the parking area of the swimming-pool and his status is free. In this case, the Map mode is useful to obtain a global view of the campus.
Figure 3. Monica’s friends and POIs at Fisciano Campus The relevant functionality within task 2 is represented by the route calculation. Such a service may be invoked by both modalities according to the user’s purpose, such as car navigation and pedestrian exploration.
Once invoked this functionality in Map mode, Monica obtains the route toward Davide and starts navigation, as shown in figure 4.
Figure 4. Route towards a SNU The third task benefits from the innovative services provided by Link2U, namely detecting and indoor identifying a SNU through the augmented reality. In this case, Monica inserts Luca in her contact list and switches to Live mode. This helps her to recognize Luca within a building through the properly arranged hotspots, as shown in figure 5.
Figure 5. SNU identified by Live Mode The last task involves Vincent, a foreign student who aims to reach the student secretary. He interacts with Link2U@Unisa in Live mode which supports him to explore an unknown place. Figure 6 illustrates the scenario that Vincent observes during his walking.
5. CONCLUSION The goal of the research we are carrying out is meant to realize advanced solutions to support users’ daily activities. In particular, we aim to provide users with services to support user’s navigation and location based services. In this paper, we have presented Link2U, a project which integrates advanced, but distinct, technological solutions and principles of human-computer interaction. In particular, it combines benefits from augmented reality solutions with the advanced location based services. A scenario is described to illustrate some functionalities of a prototype built following the previous approach. The prototype Link2U@Unisa has been tested with a general-purpose community, whose interests may range from a simple contact list and a messaging to the detection and identification of POIs, to the route calculation and exploration. In the future, we plan to complete the prototype and perform a usability study meant to improve the application embedding Link2U. In particular, we aim to automate its behavior in reply to some stimuli, such as ambient brightness, user’s mobility and screen shape.
Figure 6. Live mode exploration
REFERENCES [1] Android platform. Website: http://www.android.com/ [2] Max Braun, Rafael Spring. Enkin. Technical report, University of Koblenz-Landau, Website: http://www.enkin.net/Enkin.pdf [3] Cheng, Y.-C.; Chawathe, Y.; Lamarca, A.; Krumm, J. (2005). Accuracy Characterization for Metropolitan-scale WiFi Localization. In Proc. of the 3rd Int.l Conference on Mobile Systems, Applications, and Services (MobiSys 2005), 233-45. [4] Geonames website: http://www.geonames.org/ [5] Google Latitude. website: www.google.com/latitude [6] Google Maps API website: http://code.google.com/intl/it-IT/apis/maps/ [7] GPX website: http://www.topografix.com/gpx.asp [8] GPXdrive direction website: http://gnuite.com/cgi-bin/gpx.cgi [9] Lamarca, A. et al., (2005). Place Lab: Device Positioning Using Radio Beacons in the Wild. In Proc. 3rd Int’l Conf. Pervasive Computing (Pervasive 05), LNCS 3468, Springer, pp. 116-133. [10] L. Paolino, M. Sebillo, G. Tortora, G.Vitiello, “Framy: Visualizing Geographic Data on Mobile Interfaces”, in Journal of Location Based Services, vol. 2, n. 3, 2008, Taylor & Francis, pp. 236-252. ISSN 1748–9725 print/ISSN 1748–9733 online. 10.1080/17489720802487949 [11] Skyhook Wireless WPS. Commercial service available from http://www.skyhookwireless.com/
