Automation in Construction 17 (2008) 737–748
Contents lists available at ScienceDirect
Automation in Construction j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / a u t c o n
Ubiquitous location tracking for context-specific information delivery on construction sites Amir H. Behzadan a,1, Zeeshan Aziz b,2, Chimay J. Anumba b,2, Vineet R. Kamat a,⁎ a b
Department of Civil and Environmental Engineering, University of Michigan, 2340 G.G. Brown, 2350 Hayward, Ann Arbor, MI 48109, USA Department of Civil and Building Engineering, Loughborough University, Loughborough, Leicestershire LE11 3TU, UK
a r t i c l e
i n f o
Article history: Accepted 11 February 2008 Keywords: Augmented reality Construction Context awareness Information delivery Location tracking
a b s t r a c t Construction projects are information-intensive in nature and require site personnel to have continuous ondemand access to information such as project plans, drawings, schedules, and budgets. Awareness of a user's context (such as user profile, role, preferences, task, and existing project conditions) can enhance the construction project delivery process by providing a mechanism to determine information relevant to a particular context. Context awareness can also be used to improve security, logistics and health and safety practices on construction sites. Location is an important aspect of context awareness. A location aware application can utilize the knowledge of the user/object location to provide relevant information and services. This paper argues that a successful and reliable location tracking system must be able to track a user's spatial context and deliver contextual data continuously in both outdoor and indoor environments to effectively support construction projects. Research describing the use of Wireless Local Area Network (WLAN) for indoor tracking and Global Positioning System (GPS) for outdoor spatial context tracking is presented, and an integrated tracking technique using WLAN and GPS for ubiquitous location sensing is introduced. The key benefits and technical challenges of such an integrated approach are also highlighted. The presented tracking techniques have been validated in both indoor and outdoor environments to ensure their practical implementation on real construction jobsites. © 2008 Elsevier B.V. All rights reserved.
1. Introduction The information-intensive nature of construction projects requires the site staff to have on-demand access to construction project data such as plans, drawings, schedules, and budgets. The unprepared and dynamic nature of a construction site, and the hazards and difficulties presented by the on-site work, also necessitate the use of intelligent ways to support on-site construction staff and personnel. Context aware information delivery provides the ability to intelligently capture and interpret the user context, and delivering data and services to the mobile worker based on the user's context. In this way, it is possible to eliminate distractions for mobile workers, related to the volume and level of information. Also, user interaction with the system can be reduced by using context as a filtering mechanism to deliver only context relevant information to users. This has the potential to increase usability, by decreasing the level of interaction required between the mobile devices and the end users. The emergence of complementary
⁎ Corresponding author. Tel.: +1 734 764 4325; fax: +1 734 764 4292. E-mail addresses:
[email protected] (A.H. Behzadan),
[email protected] (Z. Aziz),
[email protected] (C.J. Anumba),
[email protected] (V.R. Kamat). 1 Tel.: +1 734 764 4325; fax: +1 734 764 4292. 2 Tel.: +44 1509 222615; fax: +44 1509 223982. 0926-5805/$ – see front matter © 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.autcon.2008.02.002
technologies such as user profiling, ubiquitous computing and sensor networking enables the capture of many other context parameters. Location is an important and probably the most widely used aspect of context awareness. Location aware applications utilize the knowledge of the user/object location to provide relevant information and services, control labor inputs, and measure project productivity [1]. Accurate and timely identification and tracking of construction components are critical to operating a well managed and cost efficient construction project [2]. Location tracking technologies are often classified as indoor (i.e. location tracking in indoor environments) and outdoor (i.e. location tracking in outdoor environments). A variety of indoor and outdoor location tracking technologies exist with significantly different characteristics, infrastructure, and device requirements. Although some researchers have demonstrated the potential of various indoor and outdoor positioning technologies for location tracking [3–5], the benefits of integrating the two categories to develop a robust position tracking platform capable of delivering context aware information have not been widely investigated in the construction industry. This paper argues that both indoor and outdoor positioning technologies are important to support construction projects. It presents current research on outdoor as well as indoor position tracking for context aware information delivery on construction sites. It describes the rationale behind the integration of these two methods and how this adds value to current practice of position tracking. The key benefits and technical challenges are also highlighted.
738
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
2. Related work 2.1. Context aware computing Context aware computing is defined as the use of environmental characteristics such as the user's location, time, identity, profile and activity to inform the computing device so that it may provide information to the user that is relevant to the current context [6]. Context aware computing enables a mobile application to leverage knowledge about various context parameters such as who the user is, what the user is doing, where the user is and what mobile device (if any) the user is using. The application adapts services to the interpreted context, thereby ensuring that the busy user gets highly specific data and services [7]. Pashtan [8] describes four key partitions of context parameters, including user static context (i.e. user profile, user interests, and user preferences), user dynamic context (i.e. user location, user current task, and proximity to other people or objects), network connectivity (i.e. network characteristics, mobile terminal capabilities, available bandwidth, and quality of service), and environmental context (i.e. time of day, noise, and weather). These various context types can be tracked and used to filter the delivery of information and services to mobile workers in a variety of industry sectors. The use of context awareness for mobile users has been demonstrated in a large number of applications, including fieldwork [9,10], museums [11,12], route planning [13], libraries [14], and tourism [15,16]. Other projects that have specifically focused on location-based data delivery include the GUIDE project [17] and the Mobile Shadow Project (MSP) [18]. The MSP approach is based on the application of agents to map the physical context to the virtual context. Context aware applications are also being investigated in other fields of computer science research including mobile computing, wearable computing, augmented reality, ubiquitous computing, and human–computer interaction. 2.2. Location tracking for context aware computing A context aware data delivery system retrieves and displays data to the users based on their latest position on a site and the level of detail they request. As a result, position tracking is a crucial task in almost all applications designed to obtain, maintain, and deliver context aware information on a continuous basis. Good examples of such information are material, labor, and equipment tracking on a site. Accurate and effective data delivery to the site personnel is the key for higher productivity and faster service. Different tracking technologies with different implementations and hardware installation requirements are currently available in the market. Radio Frequency Identification (RFID) technology, Wireless Local Area Network (WLAN), Global Positioning System (GPS), and ZigBee are amongst the most common tracking technologies. Their application has been investigated in different fields and with different levels of detail and functionality. For example, Ergen et al. [19] studied the use of RFID technology to track the status of different facility components during operation and maintenance for an extended period of time. They connected RFID tags to a number of fire valves in a facility to conduct a longevity test for sixty consecutive days by simulating tag identification, data access, and entry in real life conditions. Song et al. [20,21] developed an RFIDbased method to automate the task of tracking, delivery, and receipt of fabricated pipe spools in lay down yards and under shipping portals. In another study, Caldas et al. [22] investigated the use of GPS integrated with a handheld device to track the position of pipe spools on lay down yards in order to improve the process and reduce the number of lost items. Jang et al. [23] developed an Automated Material Tracking system (AMTRACK) based on ZigBee localization technology with two different types of query and response pulses. They installed ZigBee routers at different locations on a construction site to detect the events associated with the movements of distributed sensors.
Although some of the above tracking techniques provide satisfactory level of accuracy, there are two major drawbacks that limit their use in wide range applications such as a real construction site. First, except for the GPS-based tracking techniques all others are either dependent on pre-installed infrastructure (e.g. routers) or controlled environmental conditions. Second, their data transmission capability is usually limited to a small range compared to a wide area in which a large scale operation (e.g. a construction project) typically takes place [24]. 3. Indoor tracking with WLAN-based positioning 3.1. Overview There are various indoor location tracking technologies including RFID, WLAN, Bluetooth and those based on a Dedicated Spectrum. Key features of these technologies are summarized in Table 1. From this Table, it is apparent that there is a clear tradeoff between the accuracy and operational range expected from an indoor tracking technology and the cost of the required components to achieve that level of accuracy. For example, while a relatively cheaper RFID-based system can only cover a sub-meter range in terms of operational space and provide a low accuracy, application of a more accurate WLAN-based system which can work in a wider area (up to 100 m) requires a more expensive infrastructure setup. In addition, the decision to use a specific indoor positioning technique depends, to a large degree, on the interaction level (i.e. one way or two ways) between the user and the system. If a one way communication (i.e. from the user to the infrastructure) is desirable, RFID-based techniques can be applied whereas if a two way communication is essential, WLAN-based and Bluetooth-based techniques become more attractive options to the system developer. 3.2. Technical approach A context aware system using a WLAN-based positioning technology was developed and tested at Loughborough University (UK). A key reason for choosing WLAN-based positioning technology was that it Table 1 Comparison of various indoor tracking technologies Indoor tracking technology
RFID-based tracking
Description The location of the moving RFID tag is deducted from the location of the reader
WLAN-based tracking
Bluetooth-based tracking
Dedicated spectrum based tracking
Measures the signal strength data, which is then correlated with location
Works on the same principle as WLAN-based positioning using Bluetooth technology
Up to 1 m, provided three or more access points are provided Supports 2 way data e.g. from WLAN enabled mobile device to backend infrastructure and vice versa
Up to 1 m, provided there are a number of access points available Supports 2 way data (e.g. from PDA to infrastructure and vice versa)
Works on same principle as WLAN-based positioning using protocol, to minimize power consumption Very high accuracy up to 0.3 m
Accuracy
Relatively low — depends on sensing of tags by RFID readers
Ability to send and receive data from the tag
Poor — RFIDbased system primarily supports one way communication i.e. from tag to infrastructure About 30 cm (for 100 m passive tags) and 90 cm (for active tags) Very cheap Relatively (less than 5$) expensive (about 60$)
Range
Tag costs
Does not support data transfer to tag
50 m
50–100 m
~ 10$
~ 10$
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
739
does not require additional infrastructure provided good accuracy could be achieved through existing WLAN deployment. Also, WLAN provided considerably higher bandwidth of up to 54 mbps enough to support data, voice communications and tracking features. In the implementation, user context was used as a filtering mechanism to deliver relevant applications and services to users. The WLAN-based tracking system from Ekahau [25] was used to track user location. This system makes use of the signal strength to determine the actual position of the target device, and then reports the tag coordinates as well as area, direction, and speed within the WLAN coverage area. It consists of the following software components:
requirement of having an adequate view of the sky is a vital part of a GPS-based tracking system since, for uninterrupted GPS data communication between GPS satellites and the receivers, the data signals have to be continuously received and interpreted with an acceptable level of accuracy. In order for a GPS receiver to obtain accurate positioning data (i.e. longitude, latitude, and altitude), it has to be visible by a certain minimum number of GPS satellites orbiting the earth. This number and the location of the satellites relative to the earth is a function of where the receiver is being used.
• A client component which is a small program that runs on a WLAN enabled client device (PC laptop, PDAs, Wi-Fi Tag, etc.) and send positioning information to the server. • The positioning engine that runs on a server and calculates the client device location. It provides location coordinates and relevant information to other applications through a Java-based API. • A manager application for recording the calibration data for a positioning model, tracking client devices on a map, and analyzing the positioning accuracy.
A GPS-based position tracking system was developed and successfully tested at the University of Michigan [27,29]. GPS data signals follow certain data transmission standards, the most common of them being the National Marine Electronics Association (NMEA) standard. Under this standard, each sample of GPS data is encoded as a sentence starting with a data type indicator which defines the interpretation of the rest of the sentence. Each data type indicator has its own unique interpretation and is defined in the NMEA standard. Different sentences with different data type indicators may repeat some of the same information but will also supply new data. Depending on what data elements are needed, a GPS-based positioning application can receive and process appropriate data sentences and ignore other sentences. The sentence that contains all three pieces of positional data (i.e. longitude, latitude, and altitude) for a location starts with indicator $GPGGA. A set of string manipulation statements are used inside the tracking application to extract the necessary parts of a GPS sentence and store the values in a usable format. A sample sentence starting with this type of indicator as well as the algorithm used in the presented research to obtain and extract the GPS data is shown in Fig. 1.
4. Outdoor tracking with GPS-based positioning 4.1. Overview For indoor environments, the area in which the user moves is well confined. This situation which is often referred to as a prepared environment, simplifies the task of position tracking. An extensive list of popular indoor position tracking methods was previously presented in Table 1. Additional approaches already studied by researchers include mechanical, electromagnetic, optical, wireless signal and infrared tracking. In a mechanical tracking system the user stands inside an articulated frame consisting of a number of mechanical arms all connected together. The user grabs the grippers by two hands and starts moving them to adjust the position of the display as s/he changes position or direction of view. Based on the latest arm configuration (i.e. angles and lengths) the final position of the user is calculated. In an optical tracking system, the position of an object (including the user) in the scene is calculated based on the visibility and relative distance between a number of pre-installed optical markers on the site. Using wireless signal tracking, the position of an object (including the user) on the site is calculated based on the absolute position of a number of pre-installed benchmarks. Each benchmark is usually equipped with a signal transmitter that continuously sends signals to the surrounding space. Objects are equipped with signal receivers and as they move on-site, their absolute position is calculated based on the signal strength received from the benchmarks. Although all these methods provide acceptable level of accuracy, their application is limited to relatively small environments. Thus far, significant research effort has been devoted to improving performance, precision, robustness, and affordability of these tracking methods. What makes most of these methods inappropriate for a construction site is the fact that they all depend on pre-installed infrastructure (e.g. articulated frame, optical markers, and signal transmitters). Construction projects usually take place in a dynamic environment where equipment and materials are being moved, attached together, or taken apart, and the terrain itself can change shape [26]. Such an environment does not present ideal conditions for the installation of tracking equipment, particularly if the work is primarily outdoors and spans large areas. Hence, field personnel in an outdoor unprepared environment such as a construction site must be provided with a positioning system capable of being set up and run rapidly and without any dependence on pre-installed infrastructure. GPS is an effective tracking tool on construction sites since a significant amount of work takes place in outdoors environments where there is a clear line of sight (LOS) to the sky [20–22,27,28]. The
4.2. Technical approach
5. Head orientation tracking 5.1. Overview In all context aware information delivery applications developed thus far (both indoor and outdoor), the spatial context is defined solely by the location (position) of the user. Another major attribute, the 3D head orientation, is ignored in the computations. As depicted in Fig. 2, a user's three dimensional head orientation is defined by three angles (yaw, pitch, and roll) and fully describes the user's line of sight (i.e. the direction in which the user is looking). Together with position, 3D orientation can define a user's spatial context with much greater precision than is possible with position alone. For example, tracking only an engineer's position on a construction site might help determine which floor of a building the engineer is located on [30]. However, this information is not sufficient to conclude which part or section of the room, or what particular component or object in that room the engineer is currently looking at or is interested in. While the global position of the user is being tracked by a set of hardware devices (e.g. WLAN, GPS) and designed software implementation, head orientation data is an essential piece of information that also needs to be continuously tracked. Only by knowing both the exact position and head orientation, the context aware application can deliver precise information to the user. Head orientation is usually captured using a 3D head orientation tracker connected directly above the user's head which continuously sends rotational values of the head in the form of the three angles shown in Fig. 2. 5.2. Technical approach From the different head tracking technologies available in the market, magnetic head trackers are widely used since they are easily applicable to both indoor and outdoor applications. Fig. 3 illustrates
740
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
Fig. 1. Sample GPS sentence and data extraction algorithm.
the six degrees-of-freedom (DOF) parameters required by a GPS-based context aware information delivery system to precisely locate a user in an outdoor environment. In order to track the user's head orientation in a 3D space, a TCM5 magnetic head orientation tracker was used in the presented research [29]. The orientation data coming through different head trackers follow different data transmission standards based on the brand and model of the tracker. In the presented work, a binary data transmission protocol was used to obtain and extract the tracker data. Each data packet contains a Frame Type ID which describes the contents of the packet. Based on this ID, the packet may contain each of the 3D rotational angles as well as the current temperature. These values are stored in the packet Payload. Although the protocol takes advantage of fast data transmission due to the fact that all data is in binary format, it introduces more risk of defective data due to signal interferences and unwanted environmental noise signals. As a result, a mechanism called Cyclic Redundancy Check (CRC) is used to distinguish between useful and corrupted binary data packets. In general, a CRC is a mathematical transformation applied to a series of bytes that produces an integer result that can be used for error detection. Upon receiving data from the orientation tracker, the tracking application calculates the CRC value of the packet using its actual contents and then compares this value to the received CRC. If the two do not match, the packet is concluded to be corrupted and no longer reliable for use. Hence, the packet is ignored and the application waits for the next incoming data packet. Otherwise, the data stream is error-free and will be extracted into its components. A set of binary data manipulation statements are used inside the tracking application to convert, extract, and store the necessary parts of a head orientation data packet. A sample head
orientation tracker binary data packet as well as the data extraction algorithm used in this research is shown in Fig. 4. 6. Indoor and outdoor tracking systems integration rationale A construction site typically consists of a relatively wide area occupied by workers, raw material, and equipment. Existing structures
Fig. 2. Definition of yaw, pitch, and roll angles.
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
741
less routers, etc.). As a result, an integrated hybrid tracking system equipped with both GPS and indoor positioning system hardware is a promising approach to the problem. 7. Example applications 7.1. Indoor applications
Fig. 3. Six necessary DOF parameters for outdoor user tracking.
such as trailers, temporary on-site inventory, and emergency shelters, as well as natural obstacles such as water ponds, and trees may also be part of a construction site. Hence, depending on the nature of the operation, site personnel may be working in various indoor or outdoor locations as well as continuously changing their location. As a result, an ubiquitous tracking system with reliable position and head orientation tracking capability should be able to continuously deliver contextual data in both indoor and outdoor environments. For example, an engineer who is in charge of monitoring the inventory level of a certain type of raw material, and consequently controlling how it is being handled and delivered to the installation area may periodically walk indoors and outdoors. As a result, the corresponding position tracking system should be capable of identifying the nature of the environment (i.e. indoors or outdoors) the engineer is located in, and accordingly switching to the appropriate tracking method in order to immediately provide positional information to the system. Fig. 5 shows an example of a timber construction project in which the engineer has to control the inventory level, monitor the actual construction operation on-site, and conduct the final inspection when the work is complete. At each stage and depending on where the engineer is located, appropriate contextual data has to be presented through the mobile computer. For example, when the engineer is inside the inventory room, there may be a need to know the exact location of certain wood sections. As the engineer walks outside to monitor the actual progress of the work, the data delivery system may be asked to display information about the crew and equipment, and how they should be staged on the site. At the final inspection stage, the engineer may require detailed information about a wood connection such as the number and type of joints, and connection angles so that it can be compared to what has been specified in the drawings and what has been actually done at the jobsite. In each step, the environment the engineer is located in is different. In the particular example of Fig. 5, the engineer starts the job from an indoor material inventory, moves outdoor to monitor the actual operations, and again walks inside the completed structure to do the final inspection. Other examples are excavation and tunnel boring operations in which the workers and material are in constant transition between indoor and outdoor environments. As discussed earlier, GPS-based tracking systems are only functional where there is a clear line of sight to the sky (i.e. outdoor conditions) while indoor positioning techniques can be effectively used in environments that are deployed with tracking infrastructure(e.g. optical markers, wire-
The implementation of the indoor position tracking technology introduced in Section 3 was conducted on a simulated construction site. Four logical areas were defined within the simulated construction site, including a site office, a site warehouse, a walking track, and site operations area. The positioning engine was first calibrated. This involved walking around a particular point on the floor map and recording signal strengths for the point. Measurements were taken at every two steps. A similar procedure was repeated for a number of points. When all the points were recorded, the calibration data was stored in the positioning model. The positioning engine compared the measurements made during runtime with those stored in the positioning model to determine the real time position of the user. The object's location was updated after a fixed time interval. Once the location was calculated the position was shown on the map. Fig. 6 shows the tracking of a notebook (local-host) and a WLAN tag (IP Address 192.168.1.101). During trials conducted in a stationary office environment an accuracy of up to 1 m was achieved in 90% of the total 44 readings taken. However, in the trials conducted in the actual construction site, the measurements were not stable enough. As WLAN-based tracking is based on calibration and signal strength measurement, any change in site conditions because of the ongoing work (such as changes in soil, structure, plant and equipment, sitelayout, etc), will require calibration after regular intervals to maintain high level of accuracy. Such regular calibration requirements may make the system difficult to manage. During trials WLAN-based tracking system has shown its capability for location determination in indoor
Fig. 4. Sample head orientation tracker binary packet and data extraction algorithm.
742
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
Fig. 5. Sample contextual data delivery cycle in a timber construction project.
stationary environments but further development is needed before it is possible to use it in dynamic construction site environments. A key advantage of using a WLAN-based positioning engine is that it has considerably less infrastructure requirements, compared to other real time indoor location tracking systems. This makes it affordable for deployment in a site environment. Also, it is appro-
priate for use both during the construction process and within the constructed facilities. Fig. 7 shows the overall deployment architecture which is based on three tiers, context capture (captures user location and context), context-inference (reasons about the captured context) and context-integration (retrieves information based on the captured location and
Fig. 6. Tracking a notebook device and a WLAN tag.
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
743
Fig. 7. Overall deployment architecture.
context). User context was drawn from indoor location tracking system and other sources. Examples include obtaining user profile through the unique IP address of user device and log-in information, user device type through RDF-based Capability/Preference Profile (CC/ PP) [31], user activity through integration with the project management application, and time through computer clock. Based on the user profile, a set of services were pushed to the user device. Contextinference tier provided the ability to reason about the captured context using a Semantic-Web based model to describe a knowledge model for a corresponding context domain, thereby helping context description and knowledge access (by supporting information retrieval, extraction and processing) based on the inferred context. The understanding of semantics (i.e. meanings of data) enables the creation of a relationship between the context parameters and available data and services. Output from the context-inference tier is passed into applications to make them aware of events on the site. The context adapter converts the captured context (e.g. user id, user
location, time, etc.) into semantic associations. Different levels of semantic mapping included: • User profile to project data: Mapping of data, based on the role of the user on-site; • Location to project data: Mapping user location to project data (e.g. if an electrician is on floor 3, the user probably requires floor 3 drawings and services); • User task to project data: Mapping the task the user is currently involved in order to provide appropriate data. The project database acted as a shared repository for all project related data (e.g. project documents and drawings) which could be accessed by all project partners. Semantic annotation using ontology as shown in Fig. 8 were developed for all project documents and drawings, and were used to develop the project repository. These annotations facilitated indexing and searching. It also enabled improved ways of information submission and retrieval, by describing
Fig. 8. View of team profile ontology.
744
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
Fig. 9. Delivering information to users based on their context.
resources, and links between them. Such semantic description enabled synthesis of content from multiple information sources based on user context. Based on the captured context, context-integration tier helps in service discovery and integration. Changes in the context prompt the context broker to trigger the preprogrammed events which may include pushing certain information to users or an exchange of information with other applications using Web Services, to make them aware of the events on the site. Web-services standards are used to allow applications to share context data and dynamically invoke the capabilities of other applications in a remote collaboration environment. As the user context (i.e. location, task) changes, services available to users are calculated in real time. Context information is then used to support both pull (e.g. an exchange of information including project documents with the backend
system) and push-based (e.g. health and safety warnings) information delivery. Fig. 9 shows an example of how the information delivered to a user changes based on the user's context. 7.2. Outdoor applications One of the emerging fields which uses position tracking as an important building block is interactive Augmented Reality (AR) visualization of simulated construction processes. AR is a technique in which computer generated data (i.e. graphics, text, diagrams, etc.) are embedded into the views of the real environment. Compared to Virtual Reality (VR) in which the synthetic world completely generated by computer plays the dominant role, AR takes advantage of the existing elements of the surrounding 3D space as the real background. This
Fig. 10. An illustrative snapshot of an AR-based animation.
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
745
performed using this data to compute the updated position of each virtual object. Obtaining real time positional data of the user is another vital part of the calculations. This can be done using either outside–in or inside–out approaches. In brief, an outside–in tracking approach is when the user's motions are tracked by means of a number of pre-installed sensors on the site whereas an inside–out tracking approach uses sensors connected to the moving user (e.g. a GPS device) to obtain positional data in real time. Depending on the availability of the preinstalled infrastructure and the range over which the objects and the user are expected to move in the scene, either of these approaches can be adapted and used for tracking purposes. AR has been widely used in several engineering as well as non engineering fields ranging from medical and automotive to military and gaming. However, the application of AR in construction has been limited to a few number of systems developed by researchers for very specific purposes. To the
Fig. 11. Profile of an AR system user equipped with position tracking devices.
reduces the amount of time and effort required for modeling and rendering the graphical contents of the scene. At the same time, the fact that a mixed environment consisting of both real and virtual objects is much more difficult to compose compared to a fully synthetic world introduces new challenges into AR-based visualization applications. A well designed and fully functional AR-based application must create and maintain both spatial and contextual links between superimposed computer generated data and the views of the real world as observed by the user. This link is often called registration. Accurate and consistent registration between computer generated information and the real world as seen by the user is one of the most important challenges in AR. For example, if the AR system is designed in a way that it allows the user to walk in the augmented world with little physical constraints, then it should be capable to continuously obtain the user's position and orientation within the environment in order to display appropriate superimposed data in real time [27,29]. The precise, fast, and robust tracking of the user as well as virtual and real objects in the scene is a critical task for creating convincing AR applications. As discussed in Section 4, GPS-based tracking is an attractive option to track a user's outdoor position because it does not rely on any pre-installed infrastructure and instead depends on direct satellite communication. Thus it can be set up and used immediately in almost any outdoor location. Fig. 10 is an illustrative AR scene in which computer generated models of construction equipment are superimposed on top of live scenes of an airport construction site. As mentioned earlier, there are several ways to track the position of the virtual and real objects in an AR environment. For the specific case of Fig. 10, the position of the aircraft (i.e. real object) can be conceptually obtained by reading real time positional data coming through the GPS device of the aircraft. The position of the virtual excavator and trucks can be obtained by keeping track of their initial position (i.e. the position they were assigned right before they were displayed on the screen) and their physical movement history in the scene. Vector calculations can be
Fig. 12. Screenshots of a sample outdoor case study in ARVISCOPE.
746
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
authors' best knowledge, a general purpose AR-based animation tool in construction with full user interactivity has not been developed and implemented. This has led the authors to apply the concepts of position tracking in an AR-based visualization application which is capable of animating simulated construction operations. The key factor in developing such a system was to obtain a user's position and 3D head orientation to construct the augmented viewing frustum, and update its contents in real time. CAD objects are displayed on top of the real background through a Head Mounted Display (HMD) if and only if they are visible to the user based on the current location and line of sight. A preliminary prototype called UM-AR-GPS-ROVER was implemented and successfully tested [29]. The results were very promising and confirmed the ability of GPS-based user tracking to animate simulated tasks in outdoor environments. The design and implementation of ARVISCOPE, a more advanced application, is also in progress at the University of Michigan [32]. ARVISCOPE is capable of creating real time dynamic augmented scenes of construction operations while allowing the user to walk freely on the site with minimum constraints and look at the animation from different perspectives. It continuously updates the contents of the augmented view based on the latest position and orientation data obtained through the GPS receiver and the 3D orientation tracker. Fig. 11 shows the profile of a user in ARVISCOPE equipped with trackers and the mobile computing backpack. To validate and verify the capability and reliability of the GPSbased tracking system, the established data communication and extraction methods were integrated into ARVISCOPE. The user of the system wears an AR backpack which is equipped with a GPS receiver. A hard hat is also provided to the user inside which a 3D head orientation tracker is installed and secured. Other hardware devices used in ARVISCOPE include a video camera to capture live scenes of the construction site, a HMD to view the final augmented display, and a miniature keyboard together with a touch pad for user input. A laptop computer inside the backpack is the main computation center of the system. As the animation runs, the user can walk freely on the site and change head orientation to view the augmented animation from different perspectives and locations. A major part of the application is a module responsible for tracking the user's movements in the 3D outdoor space, and using the tracking information to update and modify what the user sees inside the HMD accordingly. The heart
of this module is the implementation of the algorithms discussed in Sections 4 and 5. At each frame, the user's real time global position and head orientation angles are obtained, extracted, and stored to construct an updated augmented viewing frustum in front of the user's eye. Several case studies were conducted to test the validity of the presented tracking algorithms. Each case study consisted of a certain outdoor construction operation such as steel erection, excavation, and concrete delivery on a body of water. Fig. 12 shows snapshots of an outdoor test visualizing a steel erection operation. In each case study conducted using ARVISCOPE, the user was allowed to walk freely in the animation and view the animated scenes from different angles and locations. The delivered graphical data at each frame was completely a function of the user 6 DOF spatial context tracked in real time and as a result, the accuracy level achieved by both the GPS and 3D head orientation tracker was a major concern in conducting these tests. For proof-of-concept validation tests, free publicly available GPS signals were used which provided the application with a sub-meter positional accuracy. However, the algorithms and methods developed in this research are generic enough that other types of GPS signals with higher data accuracy can be deployed without any need to modify or change the existing positional data extraction methods. The head orientation tracking data accuracy was also very high (much less than one degree in all three directions) which is completely acceptable for the type of applications introduced in the presented work. 7.3. Potential applications of hybrid location tracking 7.3.1. Micro and macro level services delivery to program managers Managers of construction projects spend a considerable amount of their time generating, managing, sending, collecting, and analyzing project data. It is, therefore, imperative that only the most relevant information is made available to a project manager at any given time so as to reduce information overload. Context aware information and service delivery provides a mechanism to do this by filtering data, information and services based on the construction program manager's current context. It is important that the filtering mechanism takes into account both macro and micro perspective in delivering information and
Fig. 13. Navigation guidance in first response.
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
services as the project manager travels between different jobsites. The macro level relates to the project management level, which covers a wide geographical area (e.g. national or regional territory) and is the level at which the project manager is more interested in a portfolio of projects than individual projects. At the micro level, the project manager is only interested in the local site that is being visited. Fathi et al. [33] briefly described the enabling technologies for such scenarios. The integration of outdoor (GPS-based) and indoor (WLAN-based) tracking is seen as an effective means of providing context aware support to project managers. GPS-based tracking could be used to track the project manager's position at the macro level (as he or she travels between sites), with control being transferred to a WLANbased tracking system when the manager arrives at a given project site.
747
matically switch from one technique to the other based on the jobsite configurations. In particular, if the tracking device has clear sight to the sky, it can conveniently use GPS signals to track a user. If these signals seem to fade as the user walks indoor the application immediately switches to the WLAN signals in order to avoid any interruption in obtaining the user's position in real time. There are a number of technical challenges in developing such an integrated tracking system. These are currently being explored within the research projects highlighted in this paper and include but are not limited to identification of the most appropriate contexts to be tracked, seamless transition between outdoor and indoor tracking, determination of the most appropriate transition point, and improving the accuracy of both positioning systems. Acknowledgments
7.3.2. Emergency response and management Emergency response and management is a critical field of research since the inability to identify and access relevant information is the primary obstacle that prevents rapid and optimal decision-making by emergency responders (e.g. firefighters, civil engineers) who respond to natural and manmade disasters [34]. A potential application of ubiquitous hybrid position tracking currently being investigated at the University of Michigan is the development of a self-contained, location aware technology that can automatically provide engineers and first responders with accurate, prioritized contextual information for making critical, real time decisions in chaotic, post-disaster environments. For such an application, a mobile user context-sensing framework can be developed that will accurately track an engineer's or first responder's 3D spatial context in any indoor and/or outdoor environment without relying on a pre-installed network of sensors or trackers. Specific information of an engineer's or first responder's interest at a given time can then be retrieved by interpreting spatial context with a level of precision sufficiently high to accurately prioritize identified contextual data. An example of such information is a 3D plan of a building on fire in which the exact locations of emergency exits and fire extinguishers are marked. If this plan is superimposed on top of what the firefighter is observing at the location, the augmented view can be very helpful in quick navigation through the building, finding potential victims, and bringing them out of the fire zone. Fig. 13 shows an example of contextual data displayed to a first responder during a fire emergency scenario. 8. Discussion and conclusions Awareness of the user context (such as user profile, role, preferences, task, location, and existing project conditions) can enhance the construction project delivery process by providing a mechanism to determine information relevant to a particular context. Context aware information and services delivery offers the following benefits [27]: • Delivery of relevant data based on the worker's context thereby eliminating distractions related to the volume and level of information. • Reduction in user interaction with the system by using context as a filtering mechanism. This has potential to increase usability by making mobile devices more responsive to user needs. Context awareness, through improved sensing and monitoring of a user's context parameters, can also be used to improve security, logistics and health and safety practices on construction site. A successful and reliable ubiquitous tracking system with guaranteed tracking capability should be able to track a user's position and deliver position-based contextual data continuously in both indoor and outdoor situations. In this paper, the potential and requirements of an integrated tracking system was introduced which uses WLAN for indoor and GPS for outdoor tracking purposes. A major advantage of such a system is the fact that the tracking application can auto-
The presented work has been partially supported by the US National Science Foundation (NSF) through grant CMS-0448762, and by the UK Engineering and Physical Sciences Research Council (EPSRC). The authors gratefully acknowledge the supports of NSF and EPSRC. Any opinions, findings, conclusions, and recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the NSF or EPSRC. References [1] R. Navon, E. Goldschmidt, Can labour inputs be measured and controlled automatically, Journal of Construction Engineering and Management, vol. 129, No. 4, ASCE, Reston, VA, 2003, pp. 437–445. [2] K. Furlani, W.C. Stone, Architecture for discrete construction component tracking, Proceedings of the 16th IAARC/IFAC/IEEE International Symposium for Automation and Robotics in Construction [ISARC' 1999], Madrid, Spain, 1999, pp. 289–294. [3] A. Meissner, L. Baxevanaki, I. Mathes, C. Branki, T. Bozios, W. Schonfeld, M. Crowe, R. Steinmetz, Integrated mobile operations support for the construction industry: the COSMOS solution, Proceedings of the 5th World Multi-Conference on Systemics, Cybernetics and Informatics, International Institute of Informatics and Systemics, Orlando, FL, 2001). pp. 248–255. [4] F. Peyret, J. Jurasz, A. Carrel, E. Zekri, B. Gorham, The Computer Integrated Road Construction Project, Journal of Automation in Construction, vol. 9, No. 5–6, Elsevier Science, New York, NY, 2000, pp. 447–462. [5] R. Sacks, R. Navon, E. Goldschmidt, Building Project Model Support for Automated Labor Monitoring, Journal of Computing in Civil Engineering, vol. 17, No. 1, ASCE, Reston, VA, 2003, pp. 19–27. [6] J. Burrell, G.K. Gay, Collectively defining context in a mobile, networked computing environment, Short Talk Summary in CHI 2001 Conference on Human Factors in Computing Systems, Seattle, WA, 2001. [7] B. Schilit, N. Adams, R. Want, Context-aware computing applications, Proceedings of the 1st IEEE Workshop on Mobile Computing Systems and Applications, Santa Cruz, CA, 1994, pp. 85–90. [8] A. Pashtan, Mobile Web Services, Cambridge University Press, Cambridge, MA, 2005. [9] G. Kortuem, M. Bauer, Z. Segall, NETMAN: The Design of a Collaborative Wearable Computer System, Journal Mobile Networks and Applications (MONET), vol. 4, No. 1, Springer, London, UK, 1999, pp. 49–58. [10] J. Pascoe, D.R. Morse, N.S. Ryan, Developing Personal Technology for the Field, Journal of Personal Technologies, vol. 2, No. 1, Springer, London, UK, 1998, pp. 28–36. [11] M. Fleck, M. Frid, T. Kindberg, E. O'Brien-Strain, R. Rajani, M. Spasojevic, From Informing to Remembering: Ubiquitous Systems in Interactive Museums, IEEE Journal of Pervasive Computing, vol. 1, No. 2, IEEE, Piscataway, NJ, 2002 (pp. 1321). [12] O. Lassila, M. Adler, Semantic gadgets: ubiquitous computing meets the semantic web, in: D. Fensel, J.A. Hendler, H. Lieberman, W. Wahlster (Eds.), Spinning the Semantic Web: Bringing the World Wide Web to Its Full Potential, 2003, MIT Press, MA, 2003, pp. 363–376. [13] N. Marmasse, C. Schmandt, A User-Centered Location Model, Journal of Personal and Ubiquitous Computing, vol. 6, No. 5–6, Springer, London, UK, 2002, pp. 318–321. [14] M. Aittola, T. Ryhänen, T. Ojala, SmartLibrary — location-aware mobile library service, Proceedings of the 5th International Symposium on Human Computer Interaction with Mobile Devices and Services, Udine, Italy, 2003, pp. 411–416. [15] M. Laukkanen, H. Helin, H. Laamanen, Tourists on the move, Lecture Notes in Computer Science: Proceedings of the 6th International Workshop of Cooperative Information Agents VI, Vol. 2446, Springer-Verlag, London, UK, 2002, pp. 36–50. [16] S. Long, R. Kooper, G. Abowd, C. Atkeson, Rapid prototyping of mobile contextaware applications: the cyberguide case study, Proceedings of the 2nd International Conference on Mobile Computing and Networking (MOBICOM '96), Rye, NY, 1996, pp. 97–107. [17] N. Davies, K. Cheverst, K. Mitchell, A. Friday, Caches in the air: disseminating information in the guide system, Proceedings of the 2nd IEEE Workshop on Mobile Computing Systems and Applications (WMCSA'99), New Orleans, LA, 1999, pp. 11–19.
748
A.H. Behzadan et al. / Automation in Construction 17 (2008) 737–748
[18] S. Fischmeister, G. Menkhaus, W. Pree, MUSA-shadows: concepts, implementation, and sample applications; a location-based service supporting multiple devices, Proceedings of The 40th International Conference on Technology of ObjectOriented Languages and Systems (TOOLS Pacific 2002), Sydney, Australia, ACS, 2002, pp. 71–79. [19] E. Ergen, B. Akinci, B. East, J. Kirby, Tracking Components and Maintenance History within a Facility Utilizing Radio Frequency Identification Technology, Journal of Computing in Civil Engineering, vol. 21, No. 1, ASCE, Reston, VA, 2007, pp. 11–20. [20] J. Song, C.T. Haas, C.H. Caldas, Tracking the Location of Materials on Construction Jobsites, Journal of Construction Engineering and Management, vol. 132, No. 9, ASCE, Reston, VA, 2006, pp. 911–918. [21] J. Song, C.T. Haas, C.H. Caldas, E. Ergen, B. Akinci, Automating the task of tracking the delivery and receipt of fabricated pipe spools in industrial projects, Journal of Automation in Construction, vol. 15, No. 2, Elsevier Science, New York, NY, 2006, pp. 166–177. [22] C.H. Caldas, D.J. Torrent, C.T. Haas, Using Global Positioning System to Improve Materials-Processing on Industrial Projects, Journal of Construction Engineering and Management, vol. 132, No. 7, ASCE, Reston, VA, 2006, pp. 741–749. [23] W.S. Jang, M.J. Skibniewski, Wireless sensor technologies for automated tracking and monitoring of construction materials utilizing Zigbee networks, ASCE Construction Research Congress: The Global Construction Community, Grand Bahamas Island, 2007. [24] R.T. Azuma, The challenge of making augmented reality work outdoors, in: Y. Ohta, H. Tamura (Eds.), Mixed Reality: Merging Real and Virtual Worlds, 1999, SpringerVerlag, London, UK, 1999, pp. 379–390. [25] Ekahau (2004). Ekahau Positioning Engine: http://www.ekahau.com [26] R. Lipman, K. Reed, Using VRML in construction industry applications, Proceedings of the 5th Symposium on Virtual Reality Modeling Language (Web3D-VRML), Association for Computing Machinery, New York, NY, 2000, pp. 119–124.
[27] A.H. Behzadan, V.R. Kamat, Georeferenced Registration of Construction Graphics in Mobile Outdoor Augmented Reality, Journal of Computing in Civil Engineering, vol. 21, No. 4, ASCE, Reston, VA, 2007, pp. 247–258. [28] F. Peyret, D. Betaille, G. Hintzy, High-Precision Application of GPS in the Field of Real-Time Equipment Positioning, Journal of Automation in Construction, vol. 9, No. 3, Elsevier Science, New York, NY, 2000, pp. 299–314. [29] A.H. Behzadan, B.W. Timm, and V.R. Kamat (in press). “General Purpose Modular Hardware and Software Framework for Mobile Outdoor Augmented Reality Applications in Engineering”, Journal of Advanced Engineering Informatics, Elsevier Science, New York, NY. [30] Z. Aziz, C.J. Anumba, D. Ruikar, P.M. Carrillo, D.N. Bouchlaghem, Context Aware Information Delivery for On-Site Construction Operations”, Proceedings of the 22nd CIB-W78 Conference on Information Technology in Construction, Institute for Construction Informatics, Technische Universitat Dresden, Germany, 2005, pp. 321–332, CBI Publication No.: 304. [31] CC/PP (2003). Composite Capability/Reference Profiles: Structures and vocabularies 1.0: http://www.w3.org/TR/2003/PR-CCPP-struct-vocab-20031015/ [32] A.H. Behzadan, V.R. Kamat, Enabling smooth and scalable dynamic 3d visualization of discrete-event construction simulations in outdoor augmented reality, Proceedings of the 2007 Winter Simulation Conference, Institute of Electrical and Electronic Engineers (IEEE), Piscataway, NJ, 2007. [33] M.S. Fathi, C.J. Anumba, P. Carrillo, Context awareness in construction management: key issues and enabling technologies, Proceedings of the Joint International Conference on Construction Culture, Innovation and Management (CCIM), Dubai, UAE, 2006, pp. 425–435. [34] National Research Council (Nrc) Board of Natural Disasters, Reducing Disaster Losses through Better Information, National Academic Press, Washington, D.C., 1999
