ST-Diary: A Multimedia Authoring Environment for Crowdsourced Spatio-Temporal Events Akhlaq Ahmad1, 2, Imad Afyouni3, Abdullah Murad3, 4, Md. Abdur Rahman5, Faizan Ur Rehman3, 6, Bilal Sadiq3, Saleh Basalamah3, 5, Mohamed Ridza Wahiddin1 1Kulliyah of Information and Communication Technology, International Islamic University, Malaysia, {2College of Engineering and Islamic Architecture, 3KACST GIS Technology Innovation Center, 4Innovation and Entrepreneurship Institute, 5College of Computer and Information Systems} Umm Al Qura University, Makkah, Kingdom of Saudi Arabia, 6Dept. of Computer Science, LIG, University of Grenoble Alpes, France
Email: {aajee, aamurad, marahman, fsrehman, smbasalamah}@uqu.edu.sa, {iafyouni, bsadiq}@gistic.org,
[email protected] ABSTRACT The intensive use of social media through mobile devices has leveraged the development of digital diary applications that keep track of social events as well as geotagged multimedia content. In a large crowd where users with cultural diversity perform spatiotemporal activities, such geotagged multimedia content facilitates users’ navigation through points of interest (POI) based on their preferences. This work presents a crowdsourced geo-spatial multimedia data aggregation tool that allows users to develop diary chapters relevant to forthcoming users’ spatio-temporal activities. Our proposed solution provides users with the ability to add POIs through an authoring environment with multiple dimensions, such as spatio-temporal filters, multimedia categories, and event types. Specific application domains such as emergency situations, leisure trips, journalism, and tourism can take benefit of this technique. This authoring environment also visualizes geo-spatial multimedia content for collocated points of interest (CPOI) with moving users’ timelines. We plan to integrate our proposed authoring environment as a proof of concept into our existing large-scale crowdsourcing environment that is envisioned to support millions of users during the Hajj 2015 event.
Categories and Subject Descriptors H.2.4 [Systems] Multimedia Database H.2.8 [Database Applications] Spatial Database and GIS. H.3.5 [Online Information Services] Data sharing and web based services.
General Terms Design, Human factors
Keywords
Crowdsourcing, Spatio-Temporal Events, Multimedia, Aggregation, Visualization
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]. LBSN'15, November 03-06, 2015, Bellevue, WA, USA © 2015 ACM. ISBN 978-1-4503-3975-9/15/11$15.00 DOI: http://dx.doi.org/10.1145/2830657.2830664
1. INTRODUCTION
Recording interesting spatio-temporal events of daily life in a diary used to be favorite pastime for many individuals. Regular diary writing, being a reflective exercise to learn, requires figuring out what to write and how to write, which is somehow a laborious and distressing activity in today’s busy life. Modern digital resources are replacing such exercises by introducing different mobile applications for social data sharing such as Facebook and Twitter with a wide range of multimedia fields. The users collect their memories with timeline into their personal multimedia diaries. Adjoining a temporal dimension to the co-located multimedia data helps in segregating it from the rest of the data collected during multiple visits to the same location. This could happen in routine life when users visit a multi-story shopping mall, a park or a library, physically or virtually [13][15]. Semantics added to collected POIs from abundant heterogeneous user generated content help to explore multimedia data with location similarity [19][20]. The collocated multimedia content can then be integrated to understand the recurring pattern of similarities and dissimilarities in users’ activities [8][4]. This could help in large events with a culturally diverse profile, where different spatio-temporal activities such as locating lost friend or navigating to points of interests (POIs) are of common interest [1]. In such gatherings where users are participating in important social events, those users are keen to perform and share data about their activities within defined spatiotemporal constraints. Integrating users’ spatio-temporal activities with multimedia content into diary chapters empowers the navigation to desired POIs, thus improving forthcoming users’ spatio-temporal activities [18][5]. This work presents an authoring environment to mobile users by leveraging the ubiquity of geo-spatial multimedia data, and by providing them with the ability to add POIs with multiple dimensions, such as spatio-temporal filters, multimedia categories, and event types [6]. This authoring tool allows users to aggregate one’s own spatial media with that of publicly available geo-tagged multimedia artifacts to develop diary chapters relevant to forthcoming users’ spatio-temporal activities. Users can also add multimedia metadata to POIs collected earlier or instantly [11]. Combining such crowdsourced geo-tagged multimedia information provides a semantically rich multimedia diary, such as a favorite shopping diary, an emergency evacuation diary, a health diary, etc. The compiled digital diary can then be saved into a data repository for personal use as well as for sharing with a community of interest (COI). It is obvious that personal diaries remain private with limited circulation among people of interest; however, one can establish a shareable multimedia diary with selective attributes, so that
beneficiaries can speed up their learning processes and perform well in some specific tasks such as emergency situation, leisure trip, journalism, tourism, etc. domains [18]. During the authoring process, our proposed authoring environment allows for a novel visualization technique of browsing collocated points of interests (CPOI) with users’ timeline, thereby exploring POIs in a multistory building, shopping mall, etc. To facilitate efficient and userfriendly discovery of CPOIs, the authoring environment also supports spatio-temporal multimedia queries such as range and KNN (K nearest neighbor) queries. We plan to integrate our proposed authoring environment as a proof of concept for our existing large-scale crowdsourcing environment that is envisioned to support millions of pilgrims during the Hajj1 2015 event. This paper is organized as follows. Section 2 discusses the related work on mobile recommender systems and multimediaenhanced location-based services. Section 3 is about the motivation to this work; particularly, we explain the Hajj Scenario and elaborate different services required by participants of such a large crowd for their spatio-temporal activities. Section 4 presents our methodology and the system architecture. In section 5 we provide the implementation details. Section 6 discusses three different scenarios. Finally we conclude our work and discuss our vision for future work.
2. RELATED WORK Traditional diaries not only require time to write [14][9], needs skills, and makes it difficult to search and deduce some knowledge from previous findings. These limitations detracts many people to record their daily activities because of their routine busy life. To support their main objective regarding creation of digital diary, smartphone usage has been chosen as platform to save more information in less time. Quick and spontaneous addition of information, availability of cheaper devices has attracted a significant number of population to start recording their interesting events [9]. Hammerl, S. et al. 2012, talk about maintianing digital diaries that are replacing the traditional writing system. Carter, S. and Mankoff, J. 2005, presented a diary study technique in order to understand participants’ behavior. Their
three different studies include photo diary study in an everyday setting, transit decisions diary study, and festival diary. They also elaborated the diary impact for feedback
Hyldegård, J. 2006, presents diary information by combining groups of users’ interests. The proposed diary was supposed to focus on group members’ activities and emotional experiences for multiple assignments. They presented two case studies, between structured and unstructured diaries with fixed response and no response categories, respectively. Also, diaries compiled electronically and/or manually for a span of one to four weeks were introduced [12]. Our platform differs from the previously described related work, by providing users with an authoring environment to create and organize their own diary chapters related to their real time experiences during trips, study tours, and or special life events. Using our platform, they can edit, search, annotate, and partially/fully share such diaries with other users. The past users experiences are thought to be reliable for learning process through recursive activities similar to Hajj gatherings, where users go through a series of spatio-temporal events. Moreover our work provides users with a service to visualize their collocated interesting moments within specified timelines.
3. MOTIVATION Living in the information age requires an extensive use of multimedia data, thus raising many challenges with respect to its computation, management, and storage, among others [17]. In a large crowd, users share multimedia information while moving through different spatio-temporal zones, interacting with their friends and family members, or to find other individuals, hotels, parking, etc. Users carrying smartphones equipped with high speed internet can produce and share data in different scenarios such as sharing information about a road accident or finding a missing family member within a crowd by sharing their locations, audio clips, video shots, tweets, and images. As a use case of our study on large crowd management, Figure 1 indicates the number of pilgrims who performed Hajj in the holy city of Makkah, Saudi Arabia, during last two decades. In this yearly gathering, pilgrims are culturally diverse, being from more than 140 countries all over the world.
analysis and for learning about some specific tasks like interviews. Their study revealed a need for situated annotation of captured events and usefulness of different medias in different situations [7]. Sohn, T. 2008, conducted a two-week diary study to understand mobile information needs and explained that people use a variety of strategies to obtain the desired information. Moreover the authors agreed that users need specific applications and services to understand their diaries while taking users’ contexts into consideration [16]. Hess, J. and Wulf, V. 2009, developed a structured media diary. Their aim was to ease the understanding of social activities through rich media consumption (through TV, PC and cinema). They offered a diary package including pens and digital camera that allows participants to document their social events in their daily life. Their interest was in a more structured way of users’ feedback and documentation. The evaluation yielded two goals, the acceptance of such structured diaries and its significance towards adding value to previous findings [10].
Hajj is a yearly gathering for a very short duration of about a week, where pilgrims are bound to perform Hajj rituals restricted to certain spatio-temporal zones3[1]. Within this large crowd pilgrims are from different backgrounds, spoken languages, and literacy
1
3
2
http://en.wikipedia.org/wiki/Hajj Central Department of Statistics and Information, KSA
Figure 1: No. of Pilgrims in Saudi Arabia during past two decades2.
Haram, Mina, Muzdalifah, Arafat, Jamarat are located in Makkah, Kingdom of Saudi Arabia
rates. Mostly old aged pilgrims are more challenging for local authorities to provide them with a customized set of location based services. During their stay, different spatio-temporal activities such as finding lost users or navigating through points of interest (POIs) are of common interest. For most of the users it is once in their lifetime, thereby users are more curious about their performance. Local authorities offer them their needed services at their best however crowd diversity still needs modern methods to address their requirements. In Hajj 2014, we have deployed a mobile framework that captures users’ mobile sensory data; encounter their personal profile and social interaction to fine-tune their contexts. Once a user’s context is defined, the framework offers adapted services like navigation to favorite POIs, boundary services, ritual guide, traffic update etc., to name a few[1][2]. Table 1, below gives a statistical view of different services used by the pilgrims during Hajj 2014 gathering. Weather
4.2%
Find Favorite Friend
30.4%
Prayers Time
6.7%
Find Favorite POI
1.1%
Hajj Rituals
13.5%
Find POI
4.0%
Umrah Rituals
5.6%
Add New POI
4.1%
Hajj & Umrah News
6.1%
Complaints
0.3%
Translation
4.9%
Update Traffic
1.2%
Emergency SMS
0.6%
Currency Exchange
3.2%
Twitter Search Map
0.4%
Add to Favorite List
0.4%
Health Form
0.7%
Out of Boundary
12.6%
Table 1: A statistical analysis of services used in Hajj 2014. Users exchange multimedia information with in their community of interest (COI) and save to their personal profile as well as share publically with their crowd fellows. Sharing such information helps users to get prior or instant guidance with personal and other past users’ experiences in real time. For Hajj 2014 pool of services, Hajj Messenger was one of the most promising service that was used by users frequently. Pilgrims were provided with a facility to add friends and family member within their community of interest (COI). They were able to chat and share multimedia messages with each another. Hajj messenger has generated a social network of pilgrims where they can share realtime data about their spatio-temporal activities related to performance of Hajj rituals. Initially a small social network was observed between pilgrims after 15 days of observation, where more than 371 users shared text and multimedia messages. Mostly they were sharing their locations, were finding one another by exchanging multimedia information [3]. This trend has motivated us to enhance the multimedia services environment to present in a modern style as an automated multimedia diary platform. Users can record and enrich their real time experiences with multimedia information in the form of diary chapters. This could help their followers to speed up their learning process and perform well in some specific tasks such as emergency situation, leisure trips, emergency situation, journalism, and tourism domains [18].
The compiled digital diary can then be saved into a data repository for personal use as well as for sharing with a community of interest (COI). One can establish a shareable multimedia diary with selective attributes, so that beneficiaries can speed up their learning processes. During the authoring process, our proposed authoring environment allows for a novel visualization technique of browsing collocated points of interests (CPOI) with users’ timeline, thereby exploring POIs in a multi-story building, shopping mall, etc. To facilitate efficient and user-friendly discovery of CPOIs, the authoring environment also supports spatio-temporal multimedia queries such as range and KNN (K nearest neighbor) queries.
Figure 2: High Level System Architecture A detailed system architecture is presented in Figure 3. The system is composed of three different layers: the Presentation layer, the Business layer and the Data tier layer. The system is implemented using Model View Controller approach. The responsibility of the Presentation layer is to control user requests by sending commands to the Model in order to update its state. On the order hand user send control commands to its associate View to change the views presentation of the Model. The Business tier is responsible to communicate with Data Tier to store and retrieve data as per controller‘s request and display it on the View. The model retrieve data from the Data Tier and sends it to View for visualization once requested from Presentation layer.
4. METHODOLOGY AND SYSTEM ARCHITECTURE As illustrated in Figure 2, our aim is to collect crowdsourced geotagged multimedia information from social media in order to provide a semantically rich multimedia diary, such as a favorite shopping diary, an emergency evacuation diary, a health diary, etc.
Figure 3: Server Side Architecture.
REST is platform and language independent and also HTTP based standard service which is easy to use even in the presence of firewall. We used the REST FULL API’s to communicate with the third part system e.g. Twitter server on the Business Tier. The Data Tier is responsible for data storage and retrieval of data. We segregate the Data Tier into two diverse categories of RDBMS e.g. In Memory Database and PostgreSQL. All the open source technologies are used to reduce the overall cost of implementation. The benefit of using this configuration is to satisfy high scalability, high performance, distributed and high availability for robust environment. This hybrid approach enhance sorting, storing, and retrieving specified data and also is cost effective as less hard disks can be substituted by more memory. Persistence is another feature for this hybrid approach. Hence manufacturing efficiency offers additional motivation for selecting a combined in memory and on disk database system. In Memory Database and PostgreSQL contain all the relational data sets and synchronized with each other on receiving requests from Model. The synchronization is needed whenever a new relation record is entered or relation is formed. In Memory Database contain the active user sessions which demands high efficiency and PostgreSQL have all the relation data. Suppose whenever there is need to add a friend in a live session, the query fetches that requested data from permanent storage if relation satisfied the condition. The Hadoop Distributed File System (HDFS) is used to store the huge multimedia data and also synchronized with the In Memory Database and PostgreSQL implicitly as required.
Java 8 environment. We have also used Servlet version 3.1 in the project to build the project accordance with servlet architecture used in Tomcat 7.x +. Servlet 3.0 allow asynchronous request processing but only traditional I/O was permitted, which can restrict scalability of applications. Non-blocking I/O feature of Servlet 3.1 allow to build scalable applications. Android and IPhone Mobile applications sends asynchronous web service requests to business tier, in order to response the request asynchronously. We have used asynchronous response handling feature of servlet 3.1. Client Application For multiple points of interest at a single location we propose a treebased visualization as shown in Figure 4. This visualization is proposed to elaborate co-located multimedia data objects collected by users at the same location such as a multistory building or a shopping mall, etc. A branch (Level 1 in Figure 4) that comes out of the stem represents the temporal dimension of a single POI with chronological order of the multimedia objects. The view the above concept, we have developed an Android smart phone app4 to collect the multimedia POIs and designed a web interface (see Figure 7) for authoring and sharing a collection of customized POIs. To collect the POIs from pilgrims, we deployed the system during Hajj 2014 where pilgrims from more than 95 countries downloaded our app.
5. IMPLEMENTATION In order to implement our proposed system, we used Java technology to create highly scalable, secured, multi-threaded, high performance, distributed, and robust server application. Table 2 below illustrates the technologies used to develop the complete system. Technology
Java 8, Servlet 3.1.0. Spring MVC (4.1.6.RELEASE), Spring Security (4.0.1.RELEASE), Spring Security OAuth2 (2.0.7.RELEASE), Spring based REST web-services, Java Bean. Hibernate (4.3.10.Final), Hibernate-spatial (4.3.1-SNAPSHOT), C3P0 connection pooling, EhCache, Envers auditing, PostgisJDBC, JDBC. PostgreSQL 9.4, Postgis 2.1, Apache Geode, Hadoop.
Business Layer
Database Layer Database Web Server
Apache Tomcat 8.x.
Platforms
Linux, Windows.
IDE
Eclipse Luna 4.4.2.
Others
Maven, Log4J, Jackson-databind 2.5.2, HTTP-Client 4.4, GCM server, APNS. Table 2: Server Implementation Table.
We preferred Java 8 over 6/7 because multiple security issues are resolved, and is highly configurable and efficient in a multithreaded environment. Also Spring 4.1.x works more efficiently in 4
https://play.google.com/store/apps/details?id=com.hajjandumrah&hl=en
Figure 4: A Multilevel Co-located POI Browsing Interface The front end of the system is a web application that communicates with the backend through HTTP REST API, which passes and receives the parameters in JSON. The backend comprises of a big data repository that is deployed in Amazon EC2 c3.4xlarge machines for processing crowdsourced and user generated multimedia data. SYSTEM ALGORITHM: Figure 5 shows the algorithm used for collecting co-located POIs. The algorithm consists of the following steps. 1) Adding POIs on a single location, 2) Save the data in a private diary and 3) If the public flag is true, save the data in public diary and publish it. Current location, Current Level, Meta data and multimedia data of the POI and flag for public data are the inputs of the algorithm. Specifically, the algorithm starts by the initialization of the variables (Line 1 and 2). Then the “Fetch_POIs_by_Location” is called to check for any existing POI on the current location and return the value of the function in Boolean variable (Line 3).
6. EXAMPLE SCENARIOS-USE CASES The example scenarios are based on data that collected through our developed android application for smartphones running on Android 4,4.x (Kitkat) and 5.x (Lollipop). Our system stores all the geotagged data and store it publically or privately based on user preference. User can also share diary chapters partially or completely with other users. In this section, we first show how to collect geo-tagged multimedia POIs using crowdsourcing architecture and save these in a big data environment. Our proposed authoring tool can then be used to search these private or publicly shared POIs using KNN and range spatial queries to aggregate and add multiple attributes to compile the geo-tagged multimedia diary, save it in a big data environment and share it with one’s COI. The second scenario takes into consideration multiple points of interests located at a single GPS location. The tool allows for visualizing, aggregating, and saving these multiple POIs in a big data environment as a multimedia diary, and then sharing it with one’s COI. The third scenario is about sharing location based multimedia information within one’s social network that could server for guidance purpose to users in a new environment. Figure 5: The algorithm for adding POI After that, at (Line 4) the value of the Boolean variable is checked with false meaning that there is no POI at the current location and it will go to (line 9). Otherwise, the function will be executed meaning that there is one or more existing POI at the current location. The function fetches all the POIs of current level and assigns the first POI to the head of the linked list (line 6). This linked list is then traversed until reaching its last POI (Line 7 and 8). Then the memory is allocated for the POI and function to insert the POI data in the allocated memory is called (Line 9 and 10). After that, the data gets saved as a private diary and inserted into a linked list as well (Line 11). Then if the public flag is checked and returned true (Line 12), the data will also be stored in the public database (Line 14) and published (Line 15).
Use Case-I: Data Collection: We have developed a smartphone based application to generate Multimedia Diary chapters. For each diary chapter, a user can collect Audio, video, images and can log textual information for all interesting events with time line. For all collected objects, the components will appear in cluster as show in Figure 6 (a). Once the user zooms in this cluster is spreads in an exploded view Figure 6(b). User can select one object to view, with option to add/edit/delete share privately/to COI options. The user can also share the whole chapter with his friend(s) list. Figure 6(c) shows components of two multimedia diary chapters with multiple options to edit/share privately and/or publically. Figure 6(d) shows sharing a diary chapter with one’s COI. Diary owner can share a diary chapter to a single or multiple users. Upon sharing the recipient receives shared diary chapter notification that upon acceptance is added to his profile.
(a) (b) (c) (d) Figure 6: (a) A complete journey that shows multiple multimedia components captured at different locations. The numbers greater than 1, are showing collocated components. The menu at the top helps in filtering these components to display only text, images, Audios, Videos and/or all media components. (b) Shows collocated components in an exploded view. One can select by tapping on a single component to see the details. (c) Show a media component selected by a user. It shows its location, caption with multiple options of editing/deleting and or sharing privately with community of interest COI and or publically. (d) Shows a friends (COI) list. A user once selects any diary chapter partially/fully to share with COI, he can send request which upon acceptance shows the shared component(s).
Use Case II: Partial/Fully Diary Sharing with COI: Existing multimedia location-based diary of one user will help users’ in their respective COI to plan their trip. For example, in Figure 6(d), UserA travel to a location L for example, and creates multimedia location-based data throughout his trip. One of the user (say UserB) in COI wants to travel to same location L in future. User-A share his experience with user-B using multimedia diary as shown in Figure 6(d). User-A have the option to share complete chapter or partial chapter with user-B. Sharing of diary chapter will help userB to understand the environment and nearby places in her trip. User B now can move accordingly once reached physically to specific location. He can now see the locations live with the help of Diary and can add his experience and can further distribute to his COI with segregated privacy details. In this way user can have
multidimensional multilayered Diary, which is timestamp based division and aggregated COI Diary. Use Case-III: Data Visualization: On the web version, a user logs in and starts interacting through web-based authoring tool that provides the user with an interface to select the POIs by dragging the mouse over the map in a spatial browser (Figure 7 (a) A). The user can select the date range (Figure 7 (a) B) and event type or journey (Figure 7(a) C). He has the option to drag the selected multimedia objects (Figure 7(a) D) into spatial browser (7(a) A). In this way by selecting different authoring options shown in Figure 7(a) E, one can author the diary and produce an output as a diary chapter as shown in Figure 7(a) F for a specific event such as shopping, rituals, vacations, etc., to be viewed on smartphones, tablets and or PC.
(a) (b) Figure 7: (a) The web authoring interface. User can filter media components as per date range, spatial zone and or keyword search. (b) Smartphone authoring interface that is used to search specific media components by keywords, location and or date range. Below shows the diary chapters that has been created in between the selected date range.
(a) (b) Figure 8: (a) & (b) A user selects a single among many collated multimedia components to see details. The authoring environment allows user to edit/delete/share privately and or with community of Interest.
At the end of authoring process, one can save the diary privately or share it with a community of interest. The authoring tool is customized for desktop and smartphone web browsers. A user collecting multimedia information about his visit to certain shopping Mall, a multistory building etc., where the collected images, audios, videos etc., appear at a single GPS location. Once the user views the diary chapter by selecting certain spatial zone, these collocated multiple multimedia objects appear at a single GPS location. For all such collocated POIs, the visualization layer will show all POIs in an expanded view as shown in Figure 4. Figure 8 shows a screen shot of the implemented web interface where a single multimedia object has been selected among the exploded view to see the details. The authoring environment facilities the users to share/delete/edit the selection.
ACKNOWLEDGMENT We are thankful to King Abdul Aziz City for Science and Technology (KACST), Kingdom of Saudi Arabia for funding this research project through NSTIP grant number 11-INF1683-10, 11INF1700-10 and 13-INF2455-10. We extend our thanks to Mohammad Delwar and Syed Abdullah for their kind support on the technical implementation.
[7]
[8] [9]
[10]
[11] [12]
[13]
CONLUSION AND FUTURE WORK In this paper we proposed an authoring environment to record/view/share multimedia information about users’ past and real-time life experiences. This implemented system allows users to make diary chapters. It also allows them not only to save such information in their personal profile but also to choose to share it fully and/or partially, and privately and/or publically. For collocated multimedia objects, we have presented an exploded view of POIs that enables users to visualize each and every media component separately. Such multimedia diary can be a worthful tool for large crowds especially when participants perform important spatio-temporal activities. Our ongoing work consists in designing a comprehensive evaluation of our platform with respect to its performance and usability within large crowds. Preliminary results are encouraging. Future work will include utilizing multimedia collection tools like GoPano, 360 Video Camera, Google glass, etc. to record a whole journey with fully geo-tagged helps users to practice their activities using a cyber-physical virtual reality system prior to their trips.
REFERENCES [1]
[2]
[3]
[4]
[5] [6]
Ahmad, A. et al. 2014. A Framework for Crowd-Sourced Data Collection and Context-Aware Services in Hajj and Umrah. Computer Systems and Applications (AICCSA), 2014 IEEE/ACS 11th International Conference (2014), 405–412. Ahmad, A. et al. 2014. Towards a mobile and context-aware framework from crowdsourced data. The 5th International Conference on Information and Communication Technology for The Muslim World (ICT4M). (Nov. 2014), 1–6. Ahmad, A. et al. 2015. Visualization of a Scale Free Network in a Smartphone-Based Multimedia Big Data Environment. 2015 IEEE International Conference on Multimedia Big Data (Apr. 2015), 286– 287. Alvarez-lozano, J. et al. 2012. User Location Forecasting at Points of Interest. Proceedings of the 2012 RecSys workshop on Personalizing the local mobile experience. (2012), 7–12. Ave, S.M. and Floor, F. 2013. Multi-Day and Multi-Stay Travel Planning using Geo- Tagged Photos. c (2013), 1–8. Bao, X. et al. 2009. Integration of multimedia and location-based services on mobile phone tour guide system. Network Infrastructure
[14]
[15]
[16]
[17]
[18]
[19] [20]
and Digital Content, 2009. IC-NIDC 2009. IEEE International Conference on. (2009), 642–646. Carter, S. and Mankoff, J. 2005. When Participants Do the Capturing : The Role of Media in Diary Studies. Human-Computer Interaction. 05, (2005), 899. Clarkson, B. et al. 2001. The familiar. CHI ’01 extended abstracts on Human factors in computing systems - CHI '01. (2001), 271. Hammerl, S. et al. 2012. Towards a semi-automatic personal digital diary. Proceedings of the 5th International Conference on PErvasive Technologies Related to Assistive Environments - PETRA ’12. (2012), 1. Hess, J. and Wulf, V. 2009. Explore social behavior around richmedia: a structured diary study. Proceedings of the seventh european conference on European interactive television conference. (2009), 215–218. Hutchison, D. and Mitchell, J.C. Lecture Notes in Computer Science. Hyldegård, J. 2006. Using diaries in group based information behavior research: a methodological study. … of the 1st international conference on Information …. (2006), 153–161. Rahman, M.A. and Anwar Hossain, M. 2004. Lornav: A demo of a virtual reality tool for navigation and authoring of learning object repositories. Proceedings - Eighth IEEE International Symposium on Distributed Simulation and Real-Time Applications, DS-RT 2004. (2004), 240–243. Rieman, J. 1993. The Diary Study: A Workplace-Oriented Research Tool to Guide Laboratory Efforts. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’93. (1993), 321–326. El Saddik, A. et al. 2005. Authoring multimedia objects in collaborative ambient intelligent virtual environment. HAVE 2005: IEEE International Workshop on Haptic Audio Visual Environments and their Applications. 2005, October (2005), 159–164. Sohn, T. 2008. A diary study of mobile information needs. The 26th Annual CHI Conference on Human Factors in Computing Systems CHI 08. (2008), 433. Tekiner, F. and Keane, J. a. 2013. Big Data Framework. 2013 IEEE International Conference on Systems, Man, and Cybernetics. (Oct. 2013), 1494–1499. Vazirgiannis, M. et al. 1998. Spatio-temporal composition and indexing for large multimedia applications. Multimedia Systems. 6, 4 (1998), 284–298. Wang, X. et al. 2015. Semantic-Based Location Recommendation With Multimodal Venue Semantics. 17, 3 (2015), 409–419. Wu, F. et al. 2015. Semantic Annotation of Mobility Data using Social Media. (2015), 1253–1263.
