Context Awareness: a Practitioner's Perspective - CSE, IIT Bombay

Context Awareness: a Practitioner’s Perspective Hui Lei IBM T. J. Watson Research Center, Yorktown Heights, NY 10598, USA [email protected] Abstract Context awareness allows applications to adapt themselves to their computing environment in order to better suit the needs of the user. Such a capability is critical to pervasive computing because it promises to reduce the demand for human attention, which arguably is the most limited resource in an environment saturated with computing and communication capability. This paper reflects on our experiences in enabling and exploiting context awareness. These include a general context service that provides infrastructure support for context collection and dissemination, a programming abstraction for supporting the composition of context data and the continual rebinding of context sources, and a series of context-aware systems that enable the unified communication between people and between people and computers. Conducting the work in an industry research lab, we have taken an incremental approach to tackling the problem space and had it as our goal to affect both the state of the art and the state of the practice. The paper concludes with a discussion on some of the challenges in the wide deployment of context-aware technologies.

1. Introduction In a world with pervasive computing, networked computers are embedded in everyday objects and places; sensors and actuators maintain rich connections between the physical and virtual worlds; and computing is spread throughout the environment and yet gracefully integrated with it. A critical element of the pervasive computing paradigm is context awareness. Just as humans use any situation’s surroundings to appropriately gauge behavior, applications can be augmented to react in ways suitable to their computing environment. Plenty of debate goes on about what context means to those who claim to work on context-aware computing. Philosophical complaints aside, most would agree that context refers to aspects of the environment in which a computation takes place [9, 10]. We define context in the broadest possible sense. Context attributes may include aspects of the physical

world, such as a user’s location or activity. They may describe conditions in the virtual world, such as a user’s online presence and availability. Further, context can be transient or persistent. Transient context reflects the environment state during a single occurrence of time (for example, the current traffic condition). Persistent context, such as traffic patterns, is associated with a long-term trend. By recognizing and obtaining relevant context information, applications may adapt themselves to better suit the needs of the user and the tasks. Contextaware computing supports personalized access by offering the right information and services in the right places and at the right time. It allows useful resources in the local environment to be discovered and exploited for the purpose of improving the user’s comfort. It enables proactive decisions and actions by the computing system in anticipation of user needs. The most interesting kinds of context-aware systems are those that automatically collect context information via sensing and perceiving the physical environment, rather than relying on humans’ explicit input. Such systems reduce user distraction, allowing the user to interact nearly at a subconscious level. Ultimately, context awareness is the key to making pervasive computing a “technology that disappears” [19]. At the IBM T. J. Watson Research Center, we have conceived and developed a number of projects for enabling and exploiting context awareness. This paper reflects on some of those efforts. While context-aware computing encompasses various disciplines, including hardware technologies, software infrastructure, sensing and influencing the physical world, and human computer interaction, our efforts focus on the middleware infrastructure and applications. Working in an industry research lab, we always seek to affect both the state of the art and the state of the practice. Therefore, we have adopted the following guiding principles for our work: x Build systems in an incremental fashion, providing early opportunities for assessment and refinement.

Proceedings of the 2005 International Workshop on Ubiquitous Data Management (UDM’05) 0-7695-2411-7/05 $20.00 © 2005

IEEE

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on May 1, 2009 at 06:21 from IEEE Xplore. Restrictions apply.

x Always keep in mind end-to-end solutions and their requirements to ensure that the technologies developed are useful in practice. x When possible, leverage existing and standard technologies, which helps to preserve customers’ prior technology investments and ease the migration to the new technologies. In this paper, we provide a retrospective overview of a general context service that provides infrastructure support for context collection and dissemination, a programming abstraction for supporting the composition of context data and the continual rebinding of context sources, and a series of contextaware systems that enable the unified communication between people and between people and applications. We conclude with a discussion on some of the challenges in the massive deployment of context-aware technologies. Due to lack of space, we omit discussions of related work. Interested readers are referred to our papers on individual projects.

2. Context Service We started our exploration of context awareness in the late 1990s. Although various applications had been developed by that time that used context information to enhance the user experience [1, 17, 18], most of them built custom mechanisms for their context needs, and were typically limited in the kinds of context used (e.g., location awareness). New applications were hard to develop, because application developers had to spend significant programming effort on deriving and managing context data. In addition, we were not aware of any prior work that had addressed the issue of giving users control over the release of their own context information. To make context information easily and widely available, we have investigated a context service [11] to provide the function of context collection and dissemination at the infrastructure level. Essentially we have taken a service-oriented approach to enabling context awareness, which offers a higher level of abstraction for application developers. Applications can interact with the Context Service to obtain the information they need, without worrying about the details of context management. Furthermore, the costs associated with introducing new context sources can be amortized across many applications. Our context service provides the additional benefits of an extensible framework to support heterogeneous context sources and integrated support for user privacy. The overall architecture of our Context Service, shown in Figure 1, consists of the Dispatcher, a configurable set of Context Drivers, and a collection of

utility components. In addition, there are three programmatic interfaces: the Client API, the Context Push Interface, and the internal Context Driver Interface. Client API

Dispatcher

Context Driver Interface


Instant Messaging Context Driver

Context Driver

…

Internal Utilities ____________ Context Cache Work Pace Event Engine Privacy Engine



Calendar Context Driver

Location Context Driver

Context Push Interface

Figure 1. The Context Service architecture

The Dispatcher maintains information on the capabilities of individual Context Drivers. Using this information, it routes incoming application requests to the appropriate Context Drivers, through the Context Driver Interface. A Context Driver is a “plug-in” component that encapsulates the details of interaction with context sources to gather and distribute a specific type of context information. It may either pull information from context sources or let context sources push updated information to it via the Context Push Interface. Four utility components are accessible to Context Drivers. The Context Cache retains recently accessed context information in main memory for performance reasons. The Work Pacer schedules the pulling of information from passive context sources to avoid overloading the context sources. The Event Engine matches context events with registered application interests (callback requests). The Privacy Engine authorizes the release of context information based on policies defined by individual context subjects. The programming model of our Context Service supports different types of context information based on a forms metaphor. Each context form describes one type of context information and embodies a collection of related context attributes (i.e., form fields). For example, one form describes current user activity, contact means, and location, while another form describes users’ online presence and availability. A form is further associated with meta-data on the quality of information (QoI) contained in the form. Currently two QoI metrics are used: timestamp for indicating data freshness, and confidence as asserted by the data source. To obtain context information, a client partially fills out a form template, selects the requested fields, and specifies the desired QoI, if any. The Context Service


IEEE


then responds with the form data that match the application’s query template. Applications may query for the current context information synchronously or may submit a callback request to be notified asynchronously when a particular condition is met. A callback request can additionally specify the number of callbacks desired and the frequency of notifications. Both queries and callback requests are subject to an application-supplied or default expiration time. A context source may push new or updated context information into the Context Service by passing the latter a context form filled with the latest information. The context source also indicates whether the update is partial or complete. A partial update will substitute information in the filled form fields only, keeping information that arrived prior to the current update that corresponds to the unfilled fields. In comparison, a complete update will nullify information that corresponds to the unfilled form fields. The Context Service respects people’s privacy by allowing individuals to determine for themselves when, how, and to what extent context information about them is communicated to others. Privacy protection is implemented in the Privacy Engine and is based upon the technique of RBAC, or Role Based Access Control [15]. Context subjects assign potential users of context information to roles and associate access permissions to roles. The permissions are specified at the granularity of form fields: the right to access a specific field in a context form is considered one permission. Privacy policies can also be predicated on the requestor’s context, resulting in very flexible privacy policies. One example privacy policy may state that only those colleagues of Peter who are actually at work can determine his exact location in the office building. The design of the Context Service has been influenced by several considerations. First, although context data is heterogeneous, the ways in which applications would like to obtain it are often similar. Therefore, our client API encompasses only a small number of operations such as queries and callbacks. Data heterogeneity is accommodated by allowing for an extensible set of form types on which the operations may be performed. Second, context information often involves real world entities. Thus, it makes sense to measure the quality of context information, i.e., the extent to which the data corresponds to the real world [2]. We have provided clear interfaces for context sources to characterize the inaccuracies and uncertainty in the data they provide, via QoI metrics, and for applications to specify the quality of information requirements. Third, intended to support diverse applications and services, a general context

service needs to be easily extensible to accommodate new context sources and to cope with new forms of context data. New context sources can be integrated into our system by simply defining a new context form and writing a Context Driver that handles the form. Fourth, people’s reaction [7] to previous research in the area has indicated that, without support for adequate handling of private context data, people are likely to be deterred from adopting context technologies. Based on this finding, our Context Service has had an integrated privacy control mechanism from the outset. The prototype implementation of the Context Service supplies a variety of context information. The instant messaging context specifies whether and how long a user has been active on instant messaging. The calendar event context includes information on a user’s activity and contact means, as derived from the user’s calendar entries. The location context may be reported by various sources, including cellular carriers, GPS devices, Blackberry devices, and 802.11 networks. The Location Context Driver is able to resolve conflicts and ambiguities in data from these different sources, using a mechanism developed in [13]. The desktop context describes what applications the user is actively using on his computer. Since instrumented physical artifacts and spaces are still rare, we have largely relied on the use of virtual sensors (such as calendars) and virtual context (such as instant messaging online status) that are part of a user’s personal computing space. Still, such exploitation can go a long way in offering an enhanced user experience, as illustrated by our own context-aware applications described later in this paper. The current prototype implementation consists of a single service instance. To achieve better scalability and privacy, however, multiple instances of the Context Service may be deployed, which maintain context data for disjoint subsets of users.

3. Context Mediation The initial Context Service masks the heterogeneity in context sources by supporting the automatic resolution of context sources and unified interoperation with the context sources. Experiments with the first prototype implementation have led us to the recognition of additional functions critical to context-aware applications. Applications typically are not interested in “raw” data from individual context sources, but rather correlated or aggregated data that are endowed with higher-level semantics. Further, mobility is an integral part of everyday life. As the user moves from place to place, the providers of her personal context information are likely to be different


IEEE


and applications must switch to new context sources accordingly. We use the term context mediation to refer to the filtering, correlation and aggregation of context data, from potentially multiple sources, and the continual rebinding of context sources. Expecting that context mediation will be a function common to many contextaware applications, we have decided to provide explicit support for context mediation in the Context Service by augmenting the original programming model. Our augmented programming model builds upon the notion of composers [6]. Composers act as providers of higher-level context data. They accept particular kinds of input values and transform them into a particular kind of output value. Providers of input data can be external context sources or other composers. The composers in an application conceptually form a hierarchy of data-reduction transformations. Raw, low-level, voluminous context data, closely aligned with the characteristics of data sources, enters the hierarchy from the bottom, and comes out from the top as refined, correlated, aggregated data, closely aligned with the concerns of the application. An application declares a composer’s behavior in a composer specification. The composer specification defines the requirements for input values, the context form type, or schema, of output data, and an expression that maps input values to an output value. To obtain data produced by the composer, the application simply fills out a context form template in the usual way (c.f. Section 2) and submits it to the extended Context Service. We have designed a language, called iQL1, for writing a composer specification. We use an example mobile shopping composer from [6] to illustrate the language features of iQL. The mobile shopping composer reports matches between a mobile user’s shopping list and ads from a local Yellow Pages service, assuming that each locality has its own Yellow Pages service. Below is a specification of the composer: type UserShoppingList schema(“http://myService.com/shoppingList”); type Location schema(“http://xml.org/location”); type LocalAds schema(“http://xml.org/localAds”); type MyAds schema(“http://myService.com/myAds”); 1

The iQL language as described in [6] is de-coupled from the Context Service. This paper presents iQL in a slightly modified form to better integrate with the Context Service.

composer mobileShopping (MyAds m: m.user as user) { UserShoppingList shoppingList is input(UserShoppingList u: u.userid=user); String zip is input(Location l: l.userid=user).zip; LocalAds localAds is input(LocalAds a: a.zipCode = zip); output join(shoppingList, localAds.adList); }

A composer specification may include a series of type definitions for the context forms that represent the composer’s input data and output data respectively. Each type is defined by an XML schema. Following the type definitions, the composer interface identifies one context form type for composer output; some fields in the form may be declared as request parameters that must be supplied by a context requestor. The mobile shopping composer uses three forms of context data – UserShoppingList, Location, and LocalAds. It produces data that are instances of the MyAds form, with the field user being a request parameter. The body of a composer specification is essentially one expression. For better readability, however, the expression may be broken up into several named subexpressions. In our example, the first sub-expression obtains the user’s shopping list. The second subexpression obtains the zip code of the user’s current location. The third sub-expression obtains the local business ads. The final sub-expression returns the local ads that match the shopping list, using a custom join function. The collection of sub-expressions can be easily rewritten as a single expression that is functionally equivalent. A collection of powerful operators is defined in iQL for mediating context data. The input operator, as shown in the example, evaluates to data of a particular type that satisfies a given predicate. The Context Service runtime discovers a context source that produces requested data and binds the expression to that source. In cases where a series of context data is requested, the Context Service performs continual rebinding as appropriate to cope with context sources that fail, or whose properties change, or whose relevance changes due to a user’s mobility. Other iQL operators include those that manipulate streams of context data (for example, filtering based on the instance value or a change of state) and those that recognize patterns of context events, such as sequences and repeats, from one or more sources. In addition,


IEEE


application developers can implement their own custom operators as Java methods. This allows the reuse of library code or application-specific procedural algorithms. To support context mediation at runtime, the original Context Service architecture has been augmented. The Dispatcher is extended with logic for continual rebinding. A new component, the Composition Engine, has been implemented, which manifests itself as another Context Driver. The Composition Engine registers the currently deployed composers with the Dispatcher. When the Dispatcher determines that the context data requested by an application can be produced by a composer, it directs the application request to the Composition Engine. The engine then instantiates and executes the composer. The main contribution of our effort on iQL is a nonprocedural programming abstraction for context mediation. Composer specifications are based on expressions, not statements. They can use expressive iQL operators to filter, correlate and aggregate context data. Input expressions state requirements on data, not names or locations of context sources. The details of discovering, accessing, and rebinding data sources are all implicit. Our programming model also promises flexibility. One composer specification can be written to work with a variety of context sources. The runtime system can select the best available source, based on system loads and the quality of information. Applications are more likely to sustain resource changes in the environment and be portable across environments.

4. Context-Aware People Communication To utilize and validate the Context Service, we have designed and implemented a series of context-aware systems. The first one, dubbed Mercury, enables users to communicate with each other across heterogeneous endpoints [11, 14]. A wide variety of mechanisms, such as email, instant messaging (IM), cellular phones, landline phones, short-messaging service (SMS), voice-mail, and pager, are now available for people-to-people communication. Each means of communication has its own sets of features and limitations. Although a person typically has multiple communication tools, he may have access to only a subset of them at a particular time. Depending on his context, he may also have a preference on which of the available tools to use. For example, a person may prefer chatting with somebody using an IM client when he is working on something else or in the middle of a meeting. But when the meeting is over or if he has to leave the room, he may want to continue chatting with the other party on his

cell phone or via SMS. Hence, there is a need for a unified communication system that allows a person to communicate using the most convenient device at the time. The Mercury system embodies two main thrusts: integration and context sensitivity. It integrates heterogeneous devices in various aspects of communication, including reachability status, call announcement, call routing and call migration. In addition, the policies that drive all the integration features are user-supplied preferences predicated on user context, which makes the integration adaptive and flexible. The Mercury architecture comprises an extensible set of Device Agents, the Routing Engine and the Context Service. Each Device Agent serves as an access point for one type of communication devices and manages communication sessions (i.e. calls) involving those devices. A Device Agent is addressable in the network of the corresponding communication network. For example, the IM Device Agent is a user in the instant messaging system. The user may use the IM interface to communicate with others by first establishing an IM session with the IM Device Agent. Similarly, the Phone Device Agent may be reached at a standard telephone number. The user dials this number to use Mercury. The Routing Engine makes call routing and migration decisions based on context-sensitive user preferences and the dynamic context information that is provided by the Context Service. It also maps between a user’s device-specific addresses and provides information on a user’s reachability. Mercury uses SIP as the underlying mechanism for creating, maintaining and terminating calls. In fact, Device Agents and the Routing Engine are all SIP entities: the former are SIP endpoints and the latter acts as a SIP server. A critical issue in any unified communication system is the selection of an appropriate communication device, which is often manual in conventional communication systems. Mercury exploits user context information and automatically routes an incoming call to the callee’s most convenient device. It further monitors the conversation session and may proactively transfer the on-going call to a more appropriate device, for either the caller or the callee. As an example, the case of call migration is illustrated in Figure 2. When a call involving a user is first set up, the Routing Engine creates a subscription with the Context Service for changes in the user’s context (Step 0). Later on if a change occurs, the Context Service sends the Engine a callback (Step 1). The Engine checks the user’s preferences to determine whether the context change warrants a switch of user device and, if


IEEE


so, looks up the address of the new device to which the call should be transferred (Step 2). The Engine then sends to the old Device Agent of the user a NOTIFY message containing the address of the new device (Step 3). The user is asked on the old device if he wants to move the call to the new device (Step 4). If the user accepts the transfer (Step 5), the Device Agent sends a REFER message to the other party’s Device Agent (Step 6). A SIP message for transferring calls, the REFER request instructs the receiver to start a new session with the address referred to. After the other party gets the request, it sends an INVITE to the new device and starts a session with the new device using the SIP 3-way handshake (Steps 7-10). Once the new session is set up, the old session is terminated by the other party’s Device Agent sending a BYE message to the user’s old Device Agent (Step 11), which then replies with a positive response (Step 12). The old Device Agent also sends a NOTIFY to the Routing Engine informing it of the successful call transfer (Step 13). Party 1 Device

Preference Profiles 2

Party 1 Device Agent 2

Routing Engine

10 9

7

8

11 12

Address Book

6

1 13

Party 2 New Device Agent

Party 2 Old Device Agent

Party 2 Old Device

Party 2 Old Device

5

3

0 Context Service

4

Figure 2. Call migration in Mercury

Mercury distinguishes between messaging devices, such as pagers, SMS and email, and conversation devices like telephones and IM. In addition to providing an overarching framework that integrates both messaging and conversation devices, Mercury also leverages messaging devices to enhance conversation. When a conversation request arrives at the Routing Engine, it is possible that the callee is not reachable on any conversation device. For example, he may be away from his office phone, and is not running the IM client. However, he may still be reachable through some messaging device such as his pager. The soft ring feature of Mercury notifies the callee of the incoming call via an appropriate messaging device. If the callee desires to have a conversation with the caller, he can make himself available on a conversation device. For instance, he can log into his IM client, or go to his office, or supply an alternative phone number

to Mercury. Mercury then directs the call to this conversation device. The selection of an appropriate messaging device for soft ring is again based on the callee’s context. Information on a person’s reachability status helps others to decide whether it is possible to place a call. Since Mercury proactively selects communication devices for users, it is natural for the system to present a unified picture of people’s reachability. Mercury integrates a person’s presence information from all his devices and is able to tell a requestor whether the person is reachable on any communication device. To achieve that, Mercury takes into account the following information of the subject: connectivity state on various devices, context attributes such as location, activity and whether the person is currently in a call, preferences applicable to the current context and the requestor. The connectivity state, like other context attributes, is maintained by the Context Service. The connectivity state may be either automatically sensed or manually asserted. Unified reachability status simplifies the task of potential callers and in the meantime better preserves the privacy of the callee. Mercury promises to enhance user experience and offer more opportunities for inter-person collaboration by integrating disparate communication endpoints. The Context Service plays an instrumental role in Mercury’s functionality. It allows Mercury to obtain diverse information about users and make informed decisions without having to painstakingly access heterogeneous context sources. As more and more context information is plugged into the Context Service, Mercury can easily broaden its use of context with little change to its system architecture. The builtin pub/sub function (i.e., callback requests) in the Context Service also substantially simplifies the logic of the Routing Engine for supporting call migration and unified reachability status. On the other hand, there is also limit to the use of context data. Sensed context may be ambiguous or incomplete, and high-level context such as user intention may not be perfectly inferable. Therefore, Mercury allows a user to provide explicit input to override or supplement the system’s decision. If necessary, the user can request a specific device for the callee when initiating a call, decide whether to accept the system’s recommendation for transferring a call, and manually register his connectivity state. Such accommodation of user assistance, in our opinion, will be very important for making context-aware systems useful in practice.


IEEE


5. Context-Aware User Interaction The Mercury system has demonstrated that people can be enabled to communicate with each other in a context-sensitive manner. We have taken this idea further in our follow-on efforts to enable people and applications to communicate in a context-sensitive manner. Specifically we have investigated how communication tools such as IM, SMS, email and telephones, which were initially designed for ad hoc, free-form communication between people, may be exploited for structured interaction between people and computer applications. Our efforts have been motivated by a number of considerations. Using communication tools as interface to applications extends the reach of the applications beyond desktop computers and conventional, browserequipped, mobile computers. Many built-in features of communication tools can also be very handy for human-computer interaction. For example, applications can take advantage of the call-receiving capability of communication tools to proactively initiate and push an interaction to end users. It is also conceivable to transfer an on-going user interaction session to another device or to another user, analogous to the way people transfer a phone call today. Using the email modality, people may be able to interact with applications in an asynchronous and disconnectable fashion. Finally, a hybrid interaction scenario can be supported, where the server-side participant can be seamlessly switched between an application and a human being, without the end user on the client-side taking any explicit actions. Although there have been various point solutions that use a single communication modality for application access, our goal was to provide an overarching framework that seamlessly integrates a wide range of communication modalities while leveraging the capability of context awareness. Our initial investigation, carried out under the project name PerCollab, has focused on one class of applications - workflow systems [3, 4]. Workflow systems support the modeling and execution of business processes. They have been selected as the initial target for mobilization due to two reasons. First, business processes can last days or even longer. Traditional poll-based user-interaction model is thus not desirable or feasible; user involvement is needed only when certain events happen and/or when the business process state satisfies a predefined condition. Workflow systems can directly benefit from a pushbased user-interaction model. Second, business processes are typically modeled as a collection of business partners that exchange messages according to

predefined sets of rules, where the partners can be either software entities or human users. We have thus conjectured that the pervasive enablement of workflows can be easily supported on top of the Mercury unified communication system. Our conjecture has turned out correct. The PerCollab implementation simply uses a component called Interaction Controller to bridge a workflow system and Mercury. The Interaction Controller serves as a proxy to all human participants in the workflow system and receives human tasks, or manual process steps, dispatched by the workflow engine. The Interaction Controller also acts as a virtual Device Agent in the Mercury framework, representing the business processes. To execute a manual task in a business process, the Interaction Controller initiates a round-trip exchange with the responsible user on a context-sensitive user device. It pushes the task data to the user, gathers user input and relays it back to the business process. Encouraged by positive responses to PerCollab both inside and outside IBM, we have decided to broaden our probe to support the mobilization of arbitrary Web applications. That has resulted in the Puma project [5]. Puma borrows a number of ideas from its predecessors Mercury and PerCollab, including an extensible architecture for integrating disparate communication devices, application-initiated user interactions, and context-sensitive selection of an appropriate interaction modality. However, Puma has a much broader scope than PerCollab. Besides enabling pervasive access to applications beyond workflow systems, Puma supports both push-based and pullbased interactions, as opposed to push only in PerCollab. While user interaction in PerCollab is based on a simple message-exchanges model, Puma allows the explicit handcrafting and customization of the user interface, which enables additional support data to be presented to the user and leads to a friendlier user interface. One new challenge Puma has addressed is supporting structured user interfaces on a wide spectrum of communication devices. Assuming applications are based on the Model-View-Controller (MVC) design pattern, Puma allows the View components of an application to be represented in a modality-independent format - XForms. The XForms representation describes the intent behind the user interaction rather than the actual physical representation of UI controls. It is rendered at runtime based on the characteristics of the particular modality. The use of XForms in fact serves two purposes. First, it allows an application to capture the basic interaction structures that can be reused across multiple devices


IEEE


and modalities as appropriate. Second, it provides a way to describe the structure and content of the user interface for originally unstructured communication modalities, without resorting to some home-grown or modality-specific representation. Puma also allows application developers to handcraft the layout and style of the user interface to specific client modalities if so desired, giving rise to highly customized and usable interfaces. The Puma architecture is shown in Figure 3, with the direction of arrows describing the flow of control between components. The system accommodates a diverse and extensible set of clients, including conventional Web browsers and communication tools. The communication tools are integrated into the system via Modality Bots. The Dispatcher serves as a single point of entry for user requests from various clients. It interprets those requests and routes them to corresponding applications, based on the navigation information contained in a configuration file. The Pusher component handles application-initiated user interactions. It receives user interaction descriptors from applications, consults with the Context Service to determine appropriate interaction modalities, and forwards the interaction descriptors to the corresponding Modality Bots. Application 1

Application 2

Dispatcher

Pusher

PUMA

Web Browsers IM Bot

Context Service

IM Bot

Email Bot

Phone Bot

IM Server

Email Server

Phone Gateway

IM Clients IM Bot

Email Clients IM Bot

Phone Clients IM Bot

Agent receives descriptors of application-initiated interactions via the Pusher. It obtains an initial view specification through the Dispatcher so as to trigger an interaction with the user on her communication tool. Although similar in spirit, a Modality Bot is different from the Device Agent in Mercury. A Modality Bot must interpret view specifications in a modalityindependent format and render them in a modalitysensitive manner. While a Device Agent interacts with the core of Mercury in SIP, a Modality Bot interacts with the core of Puma in HTTP and SOAP. This is mainly to preserve the current Web applications framework and the existing applications themselves. The Pusher component is essentially a scaled-down version of Mercury’s Routing Engine. The Routing Engine is responsible for the routing of one-way messages as well as the routing, migration and termination of two-way conversation sessions. In comparison, the Pusher is only required to route oneway messages, which have interaction descriptors as payload, to the context-appropriate Modality Bots. It is then up to the Bots to establish and control the interaction sessions with the user. The application of context in Puma is relatively straightforward. This is partly because Puma follows the same vein of thought as our earlier efforts and partly because the Context Service has effectively provided reusable assets and simplified application design. Even though it is easy to implement, the context-awareness feature in Puma is crucial to the entire system. Without it, it would be hard to enable push-based interactions in a truly pervasive and integrated manner. To the best of our knowledge, Puma is the first generic system for accessing Web applications from arbitrary communication modalities, and the first system that offers the capability of applicationinitiated, two-way interactions.

6. Conclusions Figure 3. Puma architecture

At the next level of detail, a Modality Bot performs three kinds of functions. First, it manages user interactions that go through communication tools of a particular type. It receives one view specification at a time and renders it in a modality-specific manner. Second, it acts as a Web client and communicates with the Dispatcher. Each request from the Modality Agent is a bundling of user input collected from the communication tool. The response from the application contains a new view specification. Third, the Modality

Earlier in the paper, we alluded to three guiding principles for our work: incremental development, addressing end-to-end concerns, and leveraging standard technologies whenever possible. Applications of these principles are abound. First, as an example of incremental development, the Context Service initially focused on providing a uniform framework for interacting with individual context sources and had the more complex issue of context mediation addressed in a later stage. As another example, the investigation of context-aware user interaction started with workflow systems in PerCollab and grew to support arbitrary Web applications in Puma. Second, since we expected that the privacy of


IEEE


context subjects would be an important issue in any end-to-end context-aware solutions, we provided privacy control as an integral part of the Context Service rather than as an afterthought. Also, a good part of the Puma effort was spent on making sure that the user interface could be properly specified and rendered on various communication modalities. Although it is an issue orthogonal to context awareness, having a customizable and friendly user interface is crucial to the usability and adoption of the overall system. Finally, with regard to the use of standard technologies, the management of communication sessions in Mercury is based on SIP, rather than some home-grown protocol. In addition, the Puma architecture builds upon the existing Web application framework so that legacy applications can be easily mobilized. We have gained valuable insight through our pursuit of context awareness. Obviously there are many open issues we haven’t been able to address. In fact, we consider those issues part of the challenges in the wide deployment of context-aware technologies. Our efforts follow an evolutionary path and so far have targeted relatively contained domains such as smart offices and enterprises. We have used the abstraction of context forms to represent heterogeneous context data. To support context awareness in an open and much larger setting, a reflective, or self-describing, context model is required. Such a context model will provide standardized vocabulary for context attributes and their interrelationships. It should support context representation in different granularity to match the human perception of the environment and to suit the diverse needs of applications. Furthermore, it will allow context sources, context services, and aware applications to interoperate spontaneously without prior knowledge of each other. It should enable applications to interpret new types of context data, and to easily map context data of different granularity. The use of ontologies seems to be a promising approach to defining the desired context model. We have developed a non-procedural approach to context mediation. The current set of mediation operators include those for the continual rebinding of context sources, the manipulation of context data streams, and the detection of context event patterns. In order to raise the abstraction level of context data and better align with the user’s needs, more sophisticated mediation may be necessary, such as mining historical context data to reveal persistent context, fusing data from competing sources to resolve conflicts and ambiguity, and inferring a user’s activity and intention from low-level context states. Supporting such context

mediation opens up the following questions: Are there generic enough algorithms that can implement each kind of mediation? What metrics should be used to assess the quality of the derived context data and how should they be computed? How can these forms of context mediation be integrated into a programming model such as iQL? As mentioned earlier, context monitoring and reporting raises huge privacy concerns. Our Context Services provides a mechanism for people to control the release of their private context information, possibly in very sophisticated ways. However, we have also realized that people often need assistance in forming their privacy policies. This is because people may be willing to trade some privacy for well-defined benefits. Therefore one crucial issue is to help people to understand and evaluate the trade-off between compromised privacy and augmented services [8]. Appropriate privacy control is also a function of the type of the environment in which context data is gathered and accessed. In a corporate environment, for instance, the privacy policies are primarily defined by the enterprise. Like it or not, employees have come to accept being monitored as part of the job. Simply consider the common practices of logging telephone uses and web accesses, recording phone conversation with customers, and using surveillance cameras. In comparison, users situated in their own house can be less concerned about personal data being distributed to other components in the house because the house is considered a private space. However, care must be exercised to confine personal data within the boundaries of the house. A foreign environment, on the other hand, presents most challenges. Ad hoc interaction with the environment is desirable and privacy must be preserved between the parties involved. The deployment of context-aware systems often requires a large array of interoperable components, including augmented physical artifacts, the ubiquitous sensing and networking infrastructure, and computational services. How do we convince the relevant businesses to make the necessary investment? Because it is impossible to have the buy-in of the businesses all at once, killer applications are being actively searched for. The hope is that killer applications will cause significant investment in the core infrastructure and a small set of components. This in turn will make it easy and cost-effective to deploy a wider variety of components, gradually moving towards the grand vision of pervasive computing. But what if the search for killer applications fails? A closely related issue has to do with business models [8]. The cost of providing context-aware


IEEE


systems is distributed among numerous components. How do the providers of these components recover their costs? What kind of business model should be used for appliance, network and service providers respectively? What billing and accounting support, if any, is required from cooperating components? Context awareness as a research notion has been in existence for over 10 years [16]. However, contextaware systems that have been widely deployed are still very rare. The commercial success of context-aware technologies and, more generally, pervasive computing will require that many technical, sociological and economic issues like the ones above be adequately addressed.

Acknowledgement Although this paper has a single author, it is based on collaborative projects with many colleagues and interning students at IBM Research. Specifically, the Context Service was joint work with Maria Ebling and Guerney Hunt. The iQL language is result of collaboration with Norm Cohen, Paul Castro, John Davis II, and Apratim Purakayastha. The contextaware systems Mercury, PerCollab and Puma have been jointly developed with Anand Ranganathan, Dipanjan Chakraborty, and Badrish Chandramouli respectively. Any errors or omissions are, of course, entirely this author’s responsibility.

7. References [1] G. D. Abowd et al. Cyberguide: A Mobile ContextAware Tour Guide. Wireless Networks, 3(5):421-433, October 1997. [2] P. Castro and R. Muntz. Managing Context Data for Smart Spaces. IEEE Personal Communications, pp. 44–46, October, 2000. [3] D. Chakraborty and H. Lei. Extending the Reach of Business Processes. IEEE Computer, 37(4), April 2004. [4] D. Chakraborty and H. Lei. Pervasive Enablement of Business Processes. IEEE International Conference on Pervasive Computing and Communications, Orlando, Florida, March 2004. [5] B. Chandramouli, H. Lei, K. Bhaskaran, H. Chang, M. Dikun, T. Heath. Pushing the Envelope of Pervasive Access.

IEEE International Conference on Pervasive Services, Santorini, Greece, July 2005. [6] N. Cohen, H. Lei, P. Castro, J. Davis II, and A. Purakayastha. Composing Pervasive Data Using iQL. IEEE Workshop on Mobile Computing Systems and Applcations, Callicoon, New York, June 2002. [7] P. Coy. Big Brother Pinned to your Chest. Business Week, 3279, August 17, 1992. [8] N. Davies and H. W. Gellersen. Beyond Prototypes: Challenges in Deploying Ubiquitous Systems. IEEE Pervasive Computing, 1(1):26-35, January – March, 2002. [9] A. K. Dey. Understanding and Using Context. Personal and Ubiquitous Computing, 5(1):4–7, 2001. Issues for Context Services for Pervasive Computing [10] M. Ebling, G. Hunt and H. Lei. Issues for Context Services for Pervasive Computing. Middleware’01 Advanced Workshop on Middleware for Mobile Computing. Heidelberg, Germany, November 2001. [11] H. Lei and A. Ranganathan. Context-Aware Unified Communication. 2004 IEEE International Conference on Mobile Data Management, Berkeley, California, January 2004. [12] H. Lei, D. Sow, J. Davis II, G. Banavar, and M. Ebling. The Design and Applications of a Context Service. ACM Mobile Computing and Communications Review, 6(4), October 2002. [13] J. Myllymaki and S. Edlund. Location Aggregation from Multiple Sources. International Conference on Mobile Data Management, Singapore, January 2002. [14] A. Ranganathan and H. Lei. Context-Aware Communication. IEEE Computer, 36(4), April 2003. [15] R. S. Sandhu, E. J. Coyne, H. L. Feinstein, and C. E. Youman. Role-based Access Control Models. IEEE Computer, 29:2:38–47, February, 1996. [16] B. Schilit, N. Adams, and R. Want. Context-Aware Computing Applications. IEEE Workshop on Mobile Computing Systems and Applications, Santa Cruz, California, December 1994. [17] R. Want, A. Hopper, V. Falcao, and J. Gibbons. The Active Badge Location System. ACM Transactions on Information Systems, 10(1):91-102, January 1992. [18] R. Want et al. An Overview of the PARCTAB Ubiquitous Computing Experiment. IEEE Personal Communications, 2(6):28-43, December 1995. [19] M. Weiser. The Computer of the 21st Century. Scientific American, 265(3):66-75, September 1991.


IEEE
