Location-based service using ontology-based ...

Computers, Environment and Urban Systems 62 (2017) 41–52

Contents lists available at ScienceDirect

Computers, Environment and Urban Systems journal homepage: www.elsevier.com/locate/ceus

Location-based service using ontology-based semantic queries: A study with a focus on indoor activities in a university context Kangjae Lee a, Jiyeong Lee b, Mei-Po Kwan c,⁎ a

Illinois Informatics Institute, University of Illinois at Urbana-Champaign, 255 Computing Applications Building, MC-150, 605 E Springfield Ave., Champaign, IL 61820, USA Department of Geoinformatics, University of Seoul, The 21st Century Building, 163 Seoulsiripdae-ro, Dongdaemun-gu, Seoul 130-743, South Korea Department of Geography and Geographic Information Science, University of Illinois at Urbana-Champaign, 255 Computing Applications Building, MC-150, 605 E Springfield Ave., Champaign, IL 61820, USA b c

a r t i c l e

i n f o

Article history: Received 14 July 2016 Received in revised form 23 September 2016 Accepted 17 October 2016 Available online xxxx Keywords: 3D GIS Indoor spaces 3D network-based topological data models Ontology Semantic queries

a b s t r a c t Much research on three-dimensional (3D) indoor geographic information systems (GIS) to date has been focused on 3D topological modeling in the context of emergency management and response. Besides emergency situations, however, little is known about other human activities and the effective use and retrieval of semantically relevant information about such activities based on route analysis in complex buildings regarding 3D indoor GIS. This study proposes a location-based service (LBS) using ontology-based semantic queries with a focus on the indoor activities in a university context. An ontology model called ‘University activity ontology’ is designed with regard to the indoor activities at a university for sharing, managing and querying data semantically. In particular, reasoning rules are created for semantic queries to retrieve and provide information about places relevant to a destination with keywords given by users. A 3D network-based topological data model is generated by connecting a road network model and indoor topological network model to calculate the shortest path from an outdoor/indoor location to an indoor destination of interest selected by users among suggested choices. For the implementation, a location-based GIS application is developed based on the Android operating system (OS) with interactive two-dimensional (2D) and 3D visualization. © 2016 Elsevier Ltd. All rights reserved.

1. Introduction The growing interest in three-dimensional (3D) geospatial information since the 1990s has led to a turning point in geographic information science (GIScience) toward complex 3D environments (Lee, 2004; Stoter & Zlatanova, 2003). With regard to three-dimensional geographic information systems (3D GIS), 3D topological modeling that seeks to define the relationships between geometric primitives for representing 3D objects has been a challenging and intriguing issue in 3D data modeling (Zlatanova, Rahman, & Shi, 2004). Many scholars in this area focused on the development of 3D topological models for various purposes (Arens, Stoter, & Van Oosterom, 2005; Molenaar, 1990; Shi, Yang, & Li, 2003; Zlatanova, 2000). 3D GIS have provided opportunities for extending the scope of study to micro-scale settings like indoor spaces, with a focus on the visualization and topological analysis for various applications, especially emergency response based on 3D network models (Kwan & Lee, 2005; Lee, 2004; Lee, 2007; Lee & Kwan, 2005; Lee & Kwan, 2014; Meijers, Zlatanova, & Pfeifer, 2005; Thill, Dao, & Zhou,

⁎ Corresponding author. E-mail addresses: [email protected] (K. Lee), [email protected] (J. Lee), [email protected] (M.-P. Kwan).

http://dx.doi.org/10.1016/j.compenvurbsys.2016.10.009 0198-9715/© 2016 Elsevier Ltd. All rights reserved.

2011; Vanclooster et al., 2012; Yang & Worboys, 2015; Zhou et al., 2015). Besides emergency situations, however, little is known about other human activities and the effective use and retrieval of semantically relevant information about such activities based on route analysis in complex buildings regarding 3D indoor GIS. People spend more than 90% of their time in indoor spaces (EPA, 1989) and thus human activities in indoor environments are of great importance. Human indoor activities can be described by various semantic information, like type (e.g., work or recreational activity), date, participants or duration. The amount of semantic information of human activities in a complex area like a university campus is especially huge and may change frequently, requiring efficient data management and query process. As smartphones have been widely used, people in such complex areas can easily reach destination buildings in which activities of interest happen, using commercial mapping applications (e.g., Google Maps). However, information retrieval for finding destinations through such mapping services is limited to a few ways of describing locations (e.g., building name, street address). Thus, the needs of mobile users who would like to search places with specific activity information are not met. In addition, providing information about not only destinations but also other places relevant to the destinations which users might want to visit will enrich the information content of the responses to users' queries. For this kind of

42

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

Fig. 1. Network-based topological model design by Yang and Worboys (2015).

applications, ontology-based semantic queries can greatly facilitate the search for relevant places in mobile computing environments through better querying indoor places and activities that people would like to engage in. An ontology model is a framework for describing domain knowledge, taking into account pertinent concepts and their relationships (Gruber, 1995). The structured knowledge model in a specific domain can be shared, reused, and merged with other ontology models in different domains. Concepts and their relationships in an ontology model are represented as classes and properties with a hierarchical structure. The Semantic Web is implemented based on ontology models. It provides relevant information retrieval capabilities through various reasoning rules (Berners-Lee, Hendler, & Lassila, 2001; McIlraith, Son, & Zeng, 2001). The role of the Semantic Web has become important in the provision of more accurate and pertinent information. The use of Semantic Web technology based on ontology models may thus be effective in mobile computing environments. Many past studies focused on the usability of ontology techniques for indoor navigation for production assets in manufacturing environments (Scholz & Schabus, 2014), seamless navigation in indoor and outdoor spaces (Yang and Worboys, 2011; Worboys, 2011), and indoor route planning (Anagnostopoulos, Tsetsos, & Kikiras, 2005; Dudas, Ghafourian, & Karimi, 2009; Kikiras, Tsetsos, & Hadjiefthymiades, 2006). Existing research, however, tend to neglect

further implementation and the practical use of ontology models, and the proposed ontology models do not address human activities in indoor spaces. In addition, with the widespread use of smartphones, the integration of ontology and semantic techniques with location-based end user services needs to be considered. Thus, this study focuses on the implementation of an ontology model and the integration of the model with semantic queries related to human activities in indoor spaces. Compared to path suggestion or selection based on user profiles (e.g., pedestrians or the disabled) for route planning and navigation in existing ontology research, this study addresses the problem of providing smartphone users with a service that suggests a destination of interest and relevant places in indoor spaces and to provide them with more choices based on rich indoor activity information structured in an ontology model. The purpose of this study is to propose a location-based service (LBS) using ontology-based semantic queries with a focus on indoor activities, using a university campus as a case for its development and implementation. A framework architecture is designed to describe the entire LBS system and its components. An ontology model called ‘University activity ontology (UAO)’ is designed with regard to indoor activities at a university for efficient semantic data management and semantic queries. In particular, reasoning rules are created primarily for complex semantic queries, which involve deduction that takes into account the multiple relationships among different concepts, to retrieve

Fig. 2. Framework architecture of the LBS using ontology-based semantic queries.

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

and provide information about relevant places with keywords given by users. A 3D network-based topological data model (3D NTDM) is used to calculate the shortest path to a destination location of interest. This study contributes to the development of an ontology model that encompasses potential activities at a university and application of ontology reasoning based on semantic information of indoor activities in mobile computing environments. By using Semantic Web technology, the proposed LBS can provide mobile users with rich semantic information about their primary destinations and other places relevant to the destinations for more choices, which they might be also interested in. For the implementation, a location-based GIS application is developed based on the Android operating system (OS) with interactive two-dimensional (2D) and 3D visualization. The remainder of this paper is organized as follows. Existing research on network-based topological data models for route analysis and applications of ontology models in route analysis is discussed in Section 2. The framework architecture of the proposed LBS is described in Section 3, and our pilot university activity ontology model is presented in Section 4. The design of user-defined reasoning rules for semantic queries, route analysis with the semantic queries, and implementation of the LBS are described in Section 5. Section 6 summarizes this work and discusses directions for promising future work in this area. 2. Related work 2.1. Network-based topological data model Routing analysis is executed based on topological data models. Many studies have been conducted on 3D network-based topological data models (3D NTDM) using a graph structure to represent each room and the connected sections of corridors as nodes and the links between the nodes as edges (Gilliéron & Merminod, 2003; Lee, 2004; Lee & Kwan, 2005; Meijers et al., 2005). Poincaré duality (Munkres, 1984) is a fundamental concept for transforming 3D objects and its relationships into nodes and edges. In this study, an indoor topological network model for 3D NTDM is generated following the approach of ‘supergraph’ for node and edge subdivision of corridors by Yang and Worboys (2015) as shown in Fig. 1. Connectivity relationships between nodes are considered in the calculation of the shortest route. When compared to previous research (Lee, Kang, & Lee, 2013), the proposed 3D NTDM in this study provides routes not only in multi-story buildings but also between different buildings by connecting a road network model and an indoor topological network model. 2.2. Ontology model and its application to route analysis Ontology modeling plays an important role in designing well organized and conceptualized formal models in knowledge domains with a hierarchical structure. The procedure of ontology modeling consists of multiple steps (Noy & Deborah, 2001; Uschold & Gruninger, 1996). For instance, Noy and Deborah (2001) suggested a set of standard procedures to design ontology models through seven steps from the decision of a target domain to the creation of instances. Many studies on the use of ontology models and their application to route analysis have been carried out to date with multiple purposes. The interest in mobile environments and context awareness in complex and large areas gave rise to the development of various location-based service systems using various ontology models to provide personalized information about tour services and multiple events at the right place and time (Kim et al., 2005; Schmidt-Belz et al., 2002; Weißenberg, Gartmann, & Voisard, 2006). Ontology techniques were also applied to take into account private information and personal preferences for specific transportation and road characteristics in the development of route planning systems (Chang et al., 2007; Niaraki & Kim, 2009). Furthermore, the scope of ontology studies was expanded to indoor navigation. Existing ontology models focused

43

Table 1 Ontology modeling for university activities. Ontology modeling process

Application to university activities

Step 1: Determining the range of an ontology model and utilizing relevant models

• Determining the range of a university activity domain with the use of competency questions • Examining relevant existing ontology models and utilizing some parts of the existing models • Designing classes, data and relation properties and a hierarchy of classes • Generating individual instances of classes with property values for implementation

Step 2: Designing a class hierarchy and properties Step 3: Generating instances

mainly on indoor navigation of production assets in manufacturing environments (Scholz & Schabus, 2014), seamless navigation in indoor and outdoor spaces (Yang and Worboys, 2011; Worboys, 2011), and route planning in indoor spaces for handicapped people considering building user profiles and indoor contexts, like passageways and elevators as well as legally sanctioned standards (Anagnostopoulos et al., 2005; Dudas et al., 2009; Kikiras et al., 2006). In terms of human activities, only few studies (e.g., Zlatanova, 2010) have attempted to design formal models for different emergency respond tasks for emergency situations in indoor spaces, and the implementation of ontology models and their integration with information retrieval and provision through analytic processes based on these models for end users has not been addressed yet. This study thus focuses on human activities in indoor spaces at a university and the development of a location-based service using ontology-based semantic queries. 3. Overview of the architecture for the LBS The framework architecture of the LBS is described in this section and illustrated in Fig. 2. An object-relational database management system (ORDBMS) stores a geometry model, a 3D NTDM, an ontology model, and a set of reasoning rules. An ORDBMS that supports spatial data types and the capabilities of semantic data management and rule-based query, like Oracle 11 g, should be employed in the server side as the database management system. Particularly, the geometry model is built based on geometric primitives (e.g., line, polygon) supported by the spatial data types in the ORDBMS to represent 3D buildings or footprints, which assist users to visually recognize their current locations and destinations as well as possible routes connecting them. The 3D NDTM, the construction of an ontology model, and the creation of instances and the reasoning rules will be discussed in Sections 4 and 5 below. A web environment plays an important role in the communication and data transfer (shown as thin arrows) between the databases on the server side and the components on the client side. The smartphone application helps mobile clients to interact with the database system to retrieve information and obtain routes to destinations via a high-level user interface (Fig. 2). The smartphone application is comprised of three components – Indoor & Outdoor Localization,

Table 2 Examining existing ontology models. Existing ontology model

Scope

University ontology focusing on education domain University ontology for courses

Hierarchy of affiliated institutes and campus administration (Malik et al., 2010) Course registration and restrictions (Ameen et al., 2012) File management, data retrieval and work processing (Dwivedi & Kumar, 2013) Publication and research (Heflin et al., 1999)

University ontology for paperless work processing University ontology focusing on academic activities

44

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

Fig. 3. Hierarchical view of upper-level university activity ontology (UAO).

Visualization, and Semantic Query. The Indoor & Outdoor Localization component detects the current location of a user in either indoor or outdoor space using computational processes, like geocoding, and positioning capabilities of a smartphone. In this study, a geocoding method and global positioning systems (GPS) are applied to indoor localization and outdoor localization, respectively. Determined location is sent to the databases on the server side and displayed in the user interface. The Localization and Visualization components will be discussed in greater detail in Sections 5.2 and 5.3. The Semantic Query component is responsible for repeating the processes of receiving keywords from a user, sending it to the ontology database, and receiving semantic information about a destination and relevant places until a user finally determines a destination of interest to visit. Once a destination is determined, it is mapped onto the destination 3D building or its footprint and the calculated optimal route is visualized. The creation of the reasoning rules and the semantic query process will be discussed in greater detail in Section 5.

4. A university activity ontology model 4.1. Procedure of ontology modeling This research uses a university campus as a case to develop and implement the ontology-based method. The development of the ontology model, called the university activity ontology (UAO), follows the procedures suggested by Noy and Deborah (2001). The ontology modeling process in this research is reduced to three steps as shown in Table 1, yet it includes all of the seven original steps by Noy and Deborah (2001). In the first step, the range of university activities in indoor spaces is determined, and helpful existing ontology models for developing the UAO model are examined. Setting boundaries of a specific domain is important for determining what and how much information is considered in the design, and ‘competency questions’ (Gruninger & Fox, 1995) help to determine the range of the ontology model. By listing potential

Fig. 4. Hierarchical view of lower-level UAO.

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

questions, the kind of information that should be answered through the ontology model is specifically determined. Utilization of existing models facilitates data compatibility among relevant ontology models and the extension of the model to other domains by filling the gap between the ontology models in different domains. Following this, the knowledge on concepts, its characteristics and relations is defined and structured hierarchically by using classes and properties in the second step. With respect to the two kinds of properties, data properties are defined to store attribute values of each instance whereas object properties are established to indicate connections among concepts. In the last step, individual instances from defined classes are generated with property values. The UAO model is built and visualized with classes, object properties and data properties in the next sections. The UAO model in this study focuses more on coursework-related activities and academic research activities for a few cases to explore its application to semantic queries in a university context. 4.2. Range of the UAO and the utilization of relevant models The scope of the UAO covers potential activities in university buildings as well as the physical places where these activities happen. These indoor activities include not only coursework-related activities and academic research activities but also a variety of events and everyday activities, like shopping and eating. In order to assist the development of the ontology model, competency questions are designed. This study puts emphasis on addressing relatively complex competency questions which involve logical reasoning to show the capabilities of semantic queries, and the following three questions are examples: (1) Where is the office of the teaching assistant (TA) of a certain class? (2) Where is the office of a researcher in a specific meeting? (3) How can I get to the office rooms of the students working with a certain professor? Existing university ontology models are searched and investigated for reuse and modification. As shown in Table 2, the investigated models focus largely on coursework, academic activities, the organization of universities, and paperless work processing (Ameen, Khan, & Rani, 2012; Dwivedi & Kumar, 2013; Heflin, Hendler, & Luke, 1999; Malik, Prakash, & Rizvi, 2010). Among the existing ontology models, only fundamental concepts and relationships concerning professors, students, and information about their affiliation and associated courses are adopted in the UAO because this study only focus on the activities in university buildings and the places where these activities take place. 4.3. Designing the class hierarchy and properties The UAO model is hierarchically designed as shown in Fig. 3 using a modeling tool called Enterprise Architect. The structure of the UAO model is described via two sub-ontologies – the upper ontology and the domain ontology (Guarino, 1998). The upper ontology indicates higher-level concepts like locations, whereas the domain ontology defines lower-level concepts in an application (e.g., a specific university activity). The proposed ontology model adopts the upper ontology of the context ontology by Wang et al. (2004) because the upper ontology clearly defines significant abstract concepts for the UAO (e.g., location, person and activity) and their relationships. Sub-classes of the ‘Activity’ and ‘Person’ classes are defined in the domain level because their specificity is bounded by the university activity domain. Since this study focuses solely on rooms and buildings in terms of indoor spaces in which various activities occur, other indoor spaces including corridors are not defined in the UAO. The relations between rooms and buildings are important for supporting the common mechanism through which people find a route to a specific room by initially identifying a specific building in which the destination room is located. The domain ontology defines the hierarchy of a variety of activities, types of rooms, and people at a university. It is partially illustrated in

45

Table 3 User-defined reasoning rules based on UAO. Competency question

Reasoning rule

Where is the office of TA in a certain class? Where is the office of a researcher in a specific meeting?

‘hasTaIn’ (?x :support ?y) ^ (?x :locatedIn ?z) ⇒ (?y :hasTaIn z) ‘hasParticipantsIn’ (?x :participateIn ?y) ^ (?x :locatedIn ?z) ⇒ (?y :hasParticipantsIn ?z) ‘hastudentsIn’ (?x :assist ?y) ^ (?y :locatedIn ?z) ⇒ (?x :hastudentIn ?z)

How can I get to the office rooms of students working with a certain professor?

Fig. 4, which shows the relevant academic-related activities more specifically. Various indoor activities including shopping, eating, and attending classes or meetings are incorporated in the ‘Activity’ class. A room is classified into three categories (office, classroom, and conference room) by its common usage and is characterized by its name, number, floor and telephone number to provide semantic information to users. Student, staff, faculty, and researcher are sub-classes of the ‘Person’ class to describe the types of people in a university which users might be interested in. In addition, because buildings are composed of partitioned rooms, there is the ‘compose’ object property between ‘Building’ and ‘Room’ classes. The ‘compose’ property can help to deduce the spatial relationship concerning where a room is located and in which building. Among various activities, the attending activity includes participating in events, joining meetings, and attending classes (or taking courses). Particularly, with respect to taking courses, a student can be assigned as a teaching assistant for a class that a certain professor teaches. The ‘Course’ class therefore has relations with the ‘Student’ and ‘Faculty’ classes. In addition, the ‘assist’ object property is defined between the ‘Student’ and ‘Faculty’ classes because of professors' relationships with students as advisors or employers.

Input: 3D NTDM N, Node s, t; Room t_room; RoadNetwork s_vertex, ArrayList e; Procedure Route_analysis(N, s, t){ ior CALL IndoorOutdoorRecognition() IF(ior = indoor) s CALL 3DIndoorLocalization() END IF ELSE IF(ior = outdoor) s_coord CALL OutdoorLocalization() s CALL MapMatching(s_coord) END IF t_room CALL SemanticQP() For each room_node rn in V(N) IF(rn = t_room) t rn END IF END FOR e CALL Dijkstra(N, w, s, t) return e; } Fig. 5. Route analysis algorithm based on semantic queries.

46

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

Fig. 6. Illustration of five buildings and their floors in 3D as a case study area.

5. An LBS using ontology-based semantic queries The implementation of an LBS using ontology-based semantic queries based on the proposed framework architecture is described in this section. Generated instances from the UAO model function as base data for processing not only simple but also complex semantic queries. Reasoning rules are keys to facilitate the semantic queries based on semantic relations among the classes in an ontology model. Particularly, user-defined reasoning rules are designed in this study to take into account potential questions that users might ask. An algorithm to integrate the semantic query process based on the reasoning rules with route analysis is developed, taking into account indoor and outdoor localization methods that determine the current location of a user.

5.1. User-defined reasoning rules for semantic queries User-defined reasoning rules support extensible logical deduction combining a set of object properties between classes to answer semantic queries. Some examples of the user-defined reasoning rules for the UAO model are represented in Table 3. They aim at answering the three questions derived from the first step of the ontology modeling processes in Section 4.1. Among the three defined reasoning rules, the ‘hasParticipantsIn’ rule is defined to infer the location of the office rooms of researchers who participate in a specific meeting by linking ‘participateIn’ and ‘locatedIn’ object properties. These user-defined reasoning rules are pre-defined and stored beforehand in a database to be used for semantic queries. The results of semantic queries using these user-defined reasoning rules are provided as a form of extra information

(a) Captured room number

about relevant places because it is assumed that users don't directly ask questions that involve complex logical reasoning. Provided that a user wants to find a destination of interest, relevant information with regard to the rooms that might be related to the activities or persons in the destination room is searched automatically and offered as additional information.

5.2. LBS algorithm using localization and semantic queries In order to calculate outdoor/indoor-to-indoor routes for the LBS, the 3D NTDM should be a seamlessly connected model of a road network model and an indoor topological network model for both indoor and outdoor spaces. For indoor spaces, a simplified indoor topological network model based on the geometric network model (GNM) developed by Lee (2004) is employed. All the rooms and subdivided corridors and stairways are represented as nodes and edges between nodes respectively, based on the concepts of node relation structures (NRS). Additionally, the indoor network model in this study is generated manually following the approach of ‘supergraph’ by Yang and Worboys (2015). In order to visualize a route through stairs more realistically, each stair way is represented as a slant straight line rather than just a simple vertical line like that used to represent an elevator. In order to consider people's strong preference for convenient routes, smaller distance values are assigned to the vertical edges of elevators rather than actual measured distances. A road network model for outdoor spaces is connected to the indoor topological network model by generating lines between the exits of a building and the closest road segments, perpendicular to the exit doors. The data structure of edges stores the connectivity relationship

(b) Determination of a source location

Fig. 7. 3D indoor localization using an indoor geocoding method.

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

(a) Outdoor localization using GPS unit

47

(b) Map matching on road network

Fig. 8. Outdoor localization and visualization.

and distance between two nodes. The connectivity relationship is a key to implement route analysis. Each node is represented by 3D coordinates and the type of space in which the node indicates. An LBS algorithm based on semantic queries is developed to calculate the shortest path in indoor and outdoor spaces as shown in Fig. 5. Basically, the outdoor/indoor localization method is used by the algorithm to determine the current location of a user. One of the commonly used outdoor localization methods is GPS. Specifically, a map matching method is applied to match a user's detected outdoor location with a proximate road vertex in this study. For indoor localization, various positioning techniques, such as wireless local area networks (WLANs) or Bluetooth, can be selected. In this study, the indoor geocoding technique using a string matching method is used (Lee & Lee, 2013). Dijkstra algorithm (Dijkstra, 1959) is used to calculate the shortest path between the determined source node s and target node t. Once a

user is determined to be located indoor or outdoor by the IndoorOutdoorRecognition function, the source node s is determined either by the 3DIndoorLocalization method for indoor spaces or by the MapMatching algorithm that helps to find a road vertex close to the determined point s_coord derived from the OutdoorLocalization method. After the current location of a user is determined, the target room t_room is derived through the semantic query process, SemanticQP, and then the target room is matched with the representative target node t. Dijkstra algorithm finally provides a list of edges e of the shortest path from the source node s to the target node t. 5.3. Implementation The LBS based on ontology-based semantic queries is implemented in this study. An Android application is developed for the client side

--Creating table CREATE TABLE univ_onto (id Number, triple SDO_RDF_TRIPLE_S); --Creating table space for semantic network CREATE TABLESPACE rdf_tblspace ...... SEGMENT SPACE MANAGEMENT AUTO; EXECUTE SEM_APIS.CREATE_SEM_NETWORK('rdf_tblspace'); --Creating model EXECUTE SEM_APIS.CREATE_SEM_MODEL('university_model','univ_onto','triple'); --Defining classes INSERT INTO univ_onto VALUES(1, SDO_RDF_TRIPLE_S('university_model', ‘http://www.university.org/activity/Activity’, ‘http://www.w3.org/1999/02/22-rdf-syntax-ns#type’, ‘http://www.w3.org/2000/01/rdf-schema#Class’)); --Creating instances INSERT INTO univ_onto VALUES (108, SDO_RDF_TRIPLE_S (‘university_model’, ‘http://www.university.org/space/cab272’, ‘http://www.w3.org/1999/02/22-rdf-syntaxns#type’, ‘http://www.university.org/space/Office’)); --Defining object properties INSERT INTO univ_onto VALUES (206, SDO_RDF_TRIPLE_S (‘university_model’, ‘http://www.university.org/space/cab272’, ‘http://www.university.org/space/compose’, ‘http://www.university.org/space/cab’)); --Defining data properties INSERT INTO univ_onto VALUES (346, SDO_RDF_TRIPLE_S(‘university_model’, ‘http://www.university.org/space/cab272’, ‘http://www.university.org/space/hasNumOf’,‘272’)); Fig. 9. PL/SQL statements to store UAO model and its instances and properties.

48

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

--Semantic query ----Find the place for a certain meeting SELECT dt, ogz, ...... ad FROM TABLE(SEM_MATCH('{ ?x meeting:hasDateOf ?dt. ?x meeting:hasOrganizationOf ?ogz. ...... ?z space:hasAddressOf ?ad.}', SEM_Models('university_model'), SEM_Rulebases('owlprime','activity_rb'),"+"SEM_ALIASES(SEM_ALIAS('', 'http://www.university.org/activity/'), SEM_ALIAS('space','http://www.university.org/space/'), SEM_ALIAS('meeting','http://www.university.org/meeting/')), null))"+"WHERE x = 'http://www.university.org/meeting/"+namevar1+"\'; ----Office information of participants in the meeting SELECT rid, nid, rna, rn, fn, bd, bn, ad FROM TABLE( SEM_MATCH('{?x :hasParticipantsIn ?y. ?y space:hasIdOf ?rid. ?y space:hasNidOf ?nid. ...... ?z space:hasAddressOf ?ad.}', SEM_Models('university_model'), SEM_Rulebases('owlprime','activity_rb'), SEM_ALIASES(SEM_ALIAS('', 'http://www.university.org/person/'),SEM_ALIAS('space','http://www.university.org/spa ce/')), null)) WHERE x = 'http://www.university.org/meeting/"+namevar1+"\'; Fig. 10. Statements for semantic query process.

utilizing available features of smartphones and implementing various functions based on the features – indoor and outdoor localization functions to determine the initial positions of users, user interface for semantic queries, and 2D and 3D visualization of optimal routes with building footprints. The hierarchical structure and instances of the proposed ontology model, the user-defined reasoning rules, the 3D NTDM, and the geometry model are stored in Oracle 11g R2.1 Visualization of the geometry model of building footprints and the shortest path is implemented using OpenGL. Five buildings at the University of Illinois at Urbana-Champaign are selected as a case study area for this research as illustrated in Fig. 6. CAD data are used to extract the footprints of these multi-story buildings and to construct the 3D NTDM. The current indoor location of a user is determined by the OCRbased indoor geocoding method. Built-in digital camera devices of smart phones are exploited to capture images of descriptive information, like room numbers and room names attached on doors. The OCR module2 extracts text data from the descriptive information (Fig. 7(a)). Eventually, the location of the room that the descriptive data indicate is determined as shown in Fig. 7(b) by matching the extracted text data and corresponding attributes in the database. JaroWinkler string matching algorithm (Winkler, 1990) is used to reduce recursive queries and the number of access to the database during the matching process. JaroWinkler algorithm quantifies similarities between the text data and room numbers based on the length of words and different characters. It has great performance when short strings are compared (Cohen, Ravikumar, & Fienberg, 2003). The JaroWinkler algorithm is implemented in this study using SimMetrics library.3

1 See http://docs.oracle.com/cd/E11882_01/appdev.112/e25609/sdo_rdf_concepts. htm#RDFRM100 2 See https://github.com/rmtheis/android-ocr. 3 See http://sourceforge.net/projects/simmetrics.

When a user is in outdoor space, the person's geographic coordinates are obtained from the GPS unit in his/her smartphone. Whether a user is located in indoor or outdoor space can be recognized through the button on the lower right side of the user interface (Fig. 8(a)), and his/her current location is determined by pushing the button on the upper left side. The determined outdoor location is then mapped with Google Map. A map matching method, which retrieves the closest road vertex from the obtained geographic coordinates, is used to set a source location on the road network for route analysis (Fig. 8(b)). The SDO_NN function in the SDO_GEOM package and spatial operators of the Oracle database support finding the closest node from the detected GPS point in the map matching process (bottom of Fig. 8). The processes for creating the UAO model with classes and properties as well as instances are executed using the procedural language (PL)/ structured query language (SQL) of Semantic Technologies in Oracle database. As shown in Fig. 9, first, the table ‘univ_onto’ and its storage space ‘rdf_tblspace’ are created. The resource description framework (RDF) format is the basic structure for the table to store classes, instances, and properties. Second, the ontology model ‘university_model’ is generated and classes of the model are defined based on the RDF triple structure. Further, instances and their properties are created based on the structured ontology model. For example, the instance ‘cab272’ is created from the class ‘Office’ using the RDF syntax ‘rdf:type’ as shown in Fig. 9. The created room instance is linked to the building instance ‘cab’ (which stands for Computing Applications Building) through the object property ‘compose’ and also, has the text value ‘272’ as the data property for its room number. Semantic queries by users are processed based on the defined query statements as shown in Fig. 10. The OWL Prime rulebase is used to handle semantic relationships and user-defined reasoning rules. Semantic queries are executed on the basis of the pure structure of the UAO model. For example, data properties that the corresponding instance has, like ‘hasDateOf’ or ‘hasOrganizationOf’, are entailed to retrieve the

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

49

Fig. 11. User interface to select a destination of interest among choices through semantic queries.

semantic information about a specific meeting place. On the other hand, relevant information that entails complex logical reasoning is retrieved through semantic queries using user-defined reasoning rules. For instance, the office information of participants in a specific meeting can be searched using the ‘hasParticipantsIn’ rule, which is defined in Section 5.1. The semantic query process is handled in the activity window of the developed application as shown in Fig. 11. A type of activities is selected first, as illustrated on the upper left side of Fig. 11. Then, keywords are entered, and the query using the keywords as input data is executed based on the defined semantic query statements. The result with regard to the keywords is provided in the ‘Result’ row, and relevant

(a) Source location view

information associated with the result based on the query using userdefined reasoning rules is shown in ‘Relevant locations’ row. For example, information about the office of James Smith who is planning to join the ‘Urbinsure’ meeting is retrieved as well as the meeting information including the location, date, and topic about the ‘Urbinsure’ meeting. Once one of the rows is clicked, the location that the selected row indicates (e.g., the conference room or the office room of James Smith) is determined as a destination. The resulted shortest path from the determined current location to the destination is computed based on the 3D NTDM and displayed with building footprints based on the geometry model in the database. The

(b) Target location view

(c) Resulted indoor-to-indoor shortest path in 3D Fig. 12. Visualization of shortest path in 2D and 3D.

50

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

(d) Resulted outdoor-to-indoor shortest path in 3D

(e) Resulted shortest path in 2D on Google map Fig. 12 (continued).

shortest path is visualized in both 2D and 3D, aiming at the view on Google Map with other contextual information (Fig. 12(e)) and the detailed view of the relevant indoor space (Fig. 12(a)–(d)). In other words, 2D and 3D visualizations are mutually complementary in that the 2D view with geographic information on Google Map shows outdoor routes more recognizably, whereas the 3D view is designed to display complicated indoor routes inside buildings. Users can switch between the 2D and 3D views by pushing the button on the lower right side on the user interface. With the support of various touch gestures in Android, users can zoom in and out, shift, and rotate the 3D view to see with different points of view. Based on the geometric shape of the path, the elevator with the straight vertical line in the source building is distinguished

from the stairways in the target building as shown in Fig. 12(a) and (b). The shortest path can be calculated and visualized not only for indoorto-indoor routes (Fig. 12(c)) but also for outdoor-to-indoor routes (Fig. 12(d)). The total calculated distance of the shortest path is displayed at the bottom of the 3D viewer (Fig. 12(c), (d)). 6. Discussion and conclusions This study proposes an LBS architecture using ontology-based semantic queries with a focus on the indoor activities at a university. The 3D network-based topological data model (NTDM) considers both indoor and outdoor spaces, and the university activity ontology (UAO)

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

model and reasoning rules are implemented for semantic queries. An ORDBMS stored the geometry model, the 3D NTDM, the UAO model, and the reasoning rules. An Android application was developed to retrieve a destination of interest based on indoor activity information and to visualize building footprints with an optimal route to the destination by exchanging data between the databases in the ORDBMS through a web environment. This study contributes to the further implementation of ontology models and their application to semantic queries using ontology reasoning techniques that are integrated with LBS. A variety of smartphone features are helpful for implementing the LBS, including 3D indoor geocoding, network communication with ORDBMS and visualization. In addition, the study shows that ontology modeling that focuses on indoor activities has considerable potential for expanding the limited scope of existing studies on indoor ontology models to include people's various activity patterns in different domains, like indoor environmental problems. The ORDBMS and its capabilities of semantic queries facilitate complex information retrieval that conventional database management system cannot cope with, since the deduction process in the ORDBMS takes into account multiple relationships between concepts. The utilization of user-defined reasoning rules also enhances the richness of semantic queries for suggesting other places for more choices, relevant to destinations of interest. This research, however, has some limitations. For instance, the query processing of the LBS presented in the paper needs to be improved using a search algorithm and more reasoning rules. The ORDBMS simply receives and uses the keywords that users enter to match values of corresponding attributes in the ontology database. In order for the system to understand users' intent based on a combination of keywords, more advanced search algorithm, like semantic search, should be exploited to take into account synonyms and to perform inference on the relationships between given keywords. In addition, to provide rich information to meet users' needs, more reasoning rules need to be designed and incorporated into the system. Further, the 3D visualization should be optimized and refined. Controlling the 3D view like zooming in or out was smooth in the tested device. However, loading 3D building footprint data took considerable time. If a large number of buildings are displayed simultaneously, the performance of the 3D visualization will be much lower. Given that only five buildings are displayed in the developed application, an optimization process needs to be considered to efficiently visualize a much larger volume of 3D building data by using an indoor level-of-detail model and an adaptive real-time rendering approach for the level of detail (Hagedorn et al., 2009; Xia, El-Sana, & Varshney, 1997). Lastly, the proposed ontology model should be extended to cover more activities, consider user preference, and support temporal reasoning and various spatial analysis. Since the proposed UAO model focuses mainly on coursework-related and academic research activities, a more comprehensive ontology model needs to be designed by improving and refining the proposed model with more diverse and specific concepts on and properties of not only other activities in a university context (e.g., facility management activity). The extension of the model to other contexts is also worth being studied. The improvement of the UAO model and data sharing among different academic/research groups, departments, and campuses would bring in different levels of interoperability issues (e.g., semantic interoperability), and thus, the interoperability issues need to be addressed (Harvey et al., 1999). Further, users often have different preferences for travel modes or routes in outdoor spaces (e.g., scenic views and clean sidewalks) and indoor spaces (e.g., elevators for going upstairs for the disabled). Such user preferences should thus be added to the ontology model in future work. Temporal reasoning should also be used when providing information about an activity or a place that is temporally specific or time-sensitive. For instance, temporal information about a coffee shop which opens during 11 a.m. to 7 p.m. or a workshop which will last for a few specific days should also be provided to users during the pertinent time period. Supporting spatial

51

analysis based on an improved ontology model, like finding activities in the same building in which a user is located, would provide users with more fruitful information and may enhance the functionalities of a location-based service.

References Ameen, A., Khan, K. U. R., & Rani, B. P. (2012). Construction of university ontology. In A. Abraham, S. M. Thampi, S. Pal, E. Corchado, V. Snasel, S. Abraham, & S. Ramakrishnan (Eds.), World congress on information and communication technologies (pp. 39–44). Washington: IEEE Computer Society Press 30 October – 2 November 2012 Trivandrum. Anagnostopoulos, C., Tsetsos, V., & Kikiras, P. (2005). OntoNav: A semantic indoor navigation system. 1st workshop on semantics in mobile environments (5 May 2005 Ayia). Arens, C., Stoter, J., & Van Oosterom, P. (2005). Modelling 3D spatial objects in a geoDBMS using a 3D primitive. Computers & Geosciences, 31(2), 165–177. Berners-Lee, T., Hendler, J., & Lassila, O. (2001). The semantic web. Scientific American, 284(5), 28–37. Chang, Y. J., et al. (2007). Ontology-based personalized wayfinding system using deviation detecting for individuals with cognitive impairments. In Y. J. Na, F. I. S. Ko, J. Lee, D. P. Truex, S. Sohn, & N. T. Nguyen (Eds.), International conference on convergence information technology (pp. 1844–1848). Washington: IEEE Computer Society Press 21–23 November 2007 Gyeongju. Cohen, W., Ravikumar, P., & Fienberg, S. (2003). A comparison of string metrics for matching names and records. In S. Kambhampati, & C. A. Knoblock (Eds.), IJCAI-03 workshop on information integration on the Web (pp. 73–78) 9–10 August 2003 Acapulco. Dijkstra, E. W. (1959). A note on two problems in connexion with graphs. Numerische Mathematick, 1(1), 269–271. Dudas, P. M., Ghafourian, M., & Karimi, H. (2009). ONALIN: Ontology and algorithm for indoor routing. 10th international conference on mobile data management: Systems, services and middleware (pp. 720–725). Washington: IEEE Computer Society Press 18– 21 May 2009 Taipei. Dwivedi, S. K., & Kumar, A. (2013). Development of university ontology for aSPOCMS. Journal of Emerging Technologies in Web Intelligence, 5(3), 213–221. Environmental Protection Agency (1989). Report to congress on indoor air quality: Volume II assessment and control of indoor air pollution. Washington: Office of Air and Radiation. Gilliéron, P. Y., & Merminod, B. (2003). Personal navigation system for indoor applications. 11th IAIN world congress (pp. 21–24) 21–24 October 2003 Berlin. Gruber, T. R. (1995). Toward principles for the design of ontologies used for knowledge sharing? International Journal of Human-Computer Studies, 43(5), 907–928. Gruninger, M., & Fox, M. S. (1995). Methodology for the design and evaluation of ontologies. Workshop on basic ontological issues in knowledge sharing 1995 Montreal. Guarino, N. (1998). Formal ontology and information systems. International conference on formal ontology in information systems (pp. 3–15). Amsterdam: IOS-Press 6–8 June 1998 Trento. Hagedorn, B., et al. (2009). Towards an indoor level-of-detail model for route visualization. 10th international conference on mobile data management: Systems, services and middleware (pp. 692–697). Los Alamitos: IEEE Computer Society Press 18–20 May 2009. Harvey, F., et al. (1999). Semantic interoperability: A central issue for sharing geographic information. The Annals of Regional Science, 33(2), 213–232. Heflin, J., Hendler, J., & Luke, S. (1999). SHOE: A knowledge representation language for internet applications. University of Maryland at College Park Available from: http://www.cs.umd.edu/projects/plus/SHOE/pubs/techrpt99.pdf (Accessed 23 July 2015) Kikiras, P., Tsetsos, V., & Hadjiefthymiades, S. (2006). Ontology-based user modeling for pedestrian navigation systems. Workshop on ubiquitous user modeling (28 August 2006 Riva del Garda). Kim, J. W., et al. (2005). Location-based tour guide system using mobile GIS and web crawling. In K. J. Li, & C. Vangenot (Eds.), 5th International Workshop on Web and wireless geographical information systems (pp. 51–63). New York: Springer 15–16 December 2005 Lausanne. Kwan, M. P., & Lee, J. (2005). Emergency response after 9/11: The potential of real-time 3D GIS for quick emergency response in micro-spatial environments. Computers, Environment and Urban Systems, 29(2), 93–113. Lee, J. (2004). A spatial access-oriented implementation of a 3-D GIS topological data model for urban entities. GeoInformatica, 8(3), 237–264. Lee, J. (2007). A three-dimensional navigable data model to support emergency response in microspatial built-environments. Annals of the Association of American Geographers, 97(3), 512–529. Lee, J., & Kwan, M. P. (2005). A combinatorial data model for representing topological relations among 3D geographical features in micro-spatial environments. International Journal of Geographical Information Science, 19(10), 1039–1056. Lee, J., & Kwan, M. P. (2014). Spatiotemporal routing analysis for emergency response in indoor space. Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, 32(6), 637–650. Lee, K., & Lee, J. (2013). A geocoding method on character matching in indoor spaces. Journal of Korea Spatial Information Society, 21(1), 87–100. Lee, K., Kang, H. Y., & Lee, J. (2013). Topological analysis in indoor shopping mall using ontology. Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, 31(6–2), 511–520.

52

K. Lee et al. / Computers, Environment and Urban Systems 62 (2017) 41–52

Malik, S. K., Prakash, N., & Rizvi, S. A. M. (2010). Developing a university ontology in education domain using protégé for semantic web. International Journal of Science and Technology, 2(9), 4673–4681. McIlraith, S. A., Son, T. C., & Zeng, H. (2001). Semantic web services. IEEE Intelligent Systems, 16(2), 46–53. Meijers, M., Zlatanova, S., & Pfeifer, N. (2005). 3D geoinformation indoors: Structuring for evacuation. International workshop on next generation 3D city models (pp. 21–22) 21– 22 June 2005 Bonn. Molenaar, M. (1990). A formal data structure for 3D vector maps. 1st European conference on geographical information systems (pp. 770–781) 10–13 April 1990 Amsterdam. Munkres, J. R. (1984). Elements of algebraic topology. Menlo Park, CA: Addison-Wesley. Niaraki, A. S., & Kim, K. (2009). Ontology based personalized route planning system using a multi-criteria decision making approach. Expert Systems with Applications, 36(2), 2250–2259. Noy, N. F., & Deborah, L. M. (2001). Ontology development 101: A guide to creating your first ontology. Stanford Medical Informatics, Stanford University Available from: http:// liris.cnrs.fr/~amille/enseignements/Ecole_Centrale/What%20is%20an%20ontology% 20and%20why%20we%20need%20it.htm (Accessed 20 July 2015) Schmidt-Belz, B., et al. (2002). Personalized and location-based mobile tourism services. Workshop on “Mobile Tourism Support Systems” in conjunction with Mobile HCI (17 September 2002 Pisa). Scholz, J., & Schabus, S. (2014). An indoor navigation ontology for production assets in a production environment. International conference on geographic information science (September 2014 Austria). Shi, W., Yang, B., & Li, Q. (2003). An object-oriented data model for complex objects in three-dimensional geographical information systems. International Journal of Geographical Information Science, 17(5), 411–430. Stoter, J. E., & Zlatanova, S. (2003). 3D GIS, where are we standing? ISPRS joint workshop on spatial, temporal and multi-dimensional data modelling and analysis (October 2003 Quebec). Thill, J. C., Dao, T. H. D., & Zhou, Y. (2011). Traveling in the three-dimensional city: Applications in route planning, accessibility assessment, location analysis and beyond. Journal of Transport Geography, 19(3), 405–421.

Uschold, M., & Gruninger, M. (1996). Ontologies: Principles, methods and applications. The Knowledge Engineering Review, 11(2), 93–136. Vanclooster, A., et al. (2012). Measuring the excitability of buildings: A new perspective on indoor accessibility. Applied Geography, 34, 507–518. Wang, X. H., et al. (2004). Ontology based context modeling and reasoning using OWL. 2nd IEEE international conference on pervasive computing and communications (pp. 18–22). Washington: IEEE Computer Society Press 14–17 March 2004 Orlando. Weißenberg, N., Gartmann, R., & Voisard, A. (2006). An ontology-based approach to personalized situation-aware mobile service supply. GeoInformatica, 10(1), 55–90. Winkler, W. E. (1990). String comparator metrics and enhanced decision rules in the FellegiSunter model of record linkage. In: Survey Research Methods Section, American Statistical Association, 354–359. Worboys, M. (2011). Modeling indoor space. 3rd ACM SIGSPATIAL International workshop on indoor spatial awareness (November 2011 Chicago). Xia, J. C., El-Sana, J., & Varshney, A. (1997). Adaptive real-time level-of-detail based rendering for polygonal models. IEEE Transactions on Visualization and Computer Graphics, 3(2), 171–183. Yang, L., & Worboys, M. (2011). A navigation ontology for outdoor-indoor space: (workin-progress). 3rd ACM SIGSPATIAL international workshop on indoor spatial awareness (pp. 31–34). New York: ACM 01-04 November 2011 Chicago. Yang, L., & Worboys, M. (2015). Generation of navigation graphs for indoor space. International Journal of Geographical Information Science, 29(10), 1737–1756. Zhou, Y., et al. (2015). Enhanced 3D visualization techniques in support of indoor location planning. Computers, Environment and Urban Systems, 50, 15–29. Zlatanova, S. (2000). 3D GIS for urban development. Thesis (PhD) the Netherlands: ITC. Zlatanova, S. (2010). Formal modeling of tasks to support search of geo-information in emergency response. International symposium climate change & disaster management. 2010. Noida: GIS Development Pvt. Ltd. Zlatanova, S., Rahman, A. A., & Shi, W. (2004). Topological models and frameworks for 3D spatial objects. Computers & Geosciences, 30(4), 419–428.