IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 11, NO. 3, JULY 2014
715
Energy-Responsive Aggregate Context for Energy Saving in a Multi-Resident Environment Ching-Hu Lu, Member, IEEE, Chao-Lin Wu, Member, IEEE, Tsung-Han Yang, Hui-Wen Yeh, Mao-Yung Weng, Li-Chen Fu, Fellow, IEEE, and Tsung-Yuan Charlie Tai
Abstract—Human activity is among the critical information for a context-aware energy saving system since knowing what activities are undertaken is important for judging if energy is well spent. Most of the prior works on energy saving do not make the best of context-awareness especially in a multiuser environment to assist the energy saving system. In addition, they often ignore whether appliances are operating implicitly or explicitly related to the context. These factors may compromise the practicality and acceptability of most of the currently available energy saving systems, thus failing to meet real user needs. Therefore, we propose Energy-Responsive Aggregate Context (ERAC) to model multi-resident activities and their associated energy consumption. Based on the relationship, implicit or explicit, between a given appliance and its associated context, an energy saving system and its users can better determine whether the power consumed by the appliance is wasted. Our experimental results demonstrate the effectiveness of the proposed approach. Note to Practitioners—This paper was motivated by the practical problem of enabling a traditional energy-saving (ES) system to know what activities are being undertaken in a multi-resident environment. This problem is important for industrial practitioners to improve product readiness for home automation related applications since it helps us put users in the center of an automation process for providing more attentive services. Existing approaches focused more on detecting locations or presence of a single resident to achieve simple yet less human-centric energy saving. Moreover, they often ignored the relationship between users’ ongoing activities and the associated energy consumption, which may limit Manuscript received May 07, 2013; revised September 05, 2013; accepted November 01, 2013. Date of publication December 16, 2013; date of current version June 30, 2014. This paper was recommended for publication by Associate Editor W. Shen and Editor J. Wen upon evaluation of the reviewers’ comments. This work was supported by the National Science Council, the National Taiwan University, and Intel Corporation under Grant NSC101-2911-I-002-001, Grant NSC 102-2221-E-155-073, and Grant NTU102R7501. C.-H. Lu is with the Department of Information Communication, Yuan Ze University, Taoyuan 320, Taiwan (e-mail:
[email protected]). C.-L. Wu is with the Intel-NTU Connected Context Computing Center, National Taiwan University, Taipei 10617, Taiwan (e-mail:
[email protected]). T.-H. Yang was with National Taiwan University, Taipei 10617, Taiwan. He is now with Research and Development, Acer, Taipei 221, Taiwan (e-mail:
[email protected]). H.-W. Yeh was with National Taiwan University, Taipei 10617, Taiwan. She is now with Research and Development, Chunghwa Telecom Co., Ltd., Taipei 100, Taiwan (e-mail:
[email protected]). M.-Y. Weng was with National Taiwan University, Taipei 10617, Taiwan. He is now with Research and Development, Mediatek, Taipei 11442, Taiwan (e-mail:
[email protected]). L.-C. Fu is with the Department of Computer Science and Information Engineering and the Department of Electrical Engineering, National Taiwan University 10617, Taipei, Taiwan (e-mail:
[email protected]). T.-Y. C. Tai is with Intel Corporation, Santa Clara, CA 95054 USA (e-mail:
[email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TASE.2013.2290312
other potentials for energy saving and cannot scale to multiuser solutions. To address these issues, the authors proposed a new approach to recognize multi-resident activities and to analyze various energy consumption information associated with the activities. The approach can be integrated into a home gateway to change the operating modes of those appliances which are not or less related to the undertaken activities, thus achieving energy saving without interfering with users. The approach therefore can improve energy efficiency and the quality of ES services, and both are important factors for real-life scenarios. Preliminary evaluations suggest that the approach is feasible and can achieve energy saving by about 30%. However, since we care more about what activities are undertaken rather than who undertakes the activities (i.e., ignoring data association), this may lead to the limitation of non-personalizable ES services, but the limitation can be mitigated by incorporating user identification techniques such as RFIDs, face recognition, etc. In future research, experiments in real and more houses will be conducted for more realistic evaluations. In addition to the application in a smart home, we can extend the approach to smart office, hospital, and other context-aware applications. Index Terms—Energy-responsive aggregate context (ERAC), energy-tagged aggregate context (ETAC), multi-resident activity recognition.
I. INTRODUCTION
H
UMAN activity is one of the major causes to bring about energy consumption, and hence ignoring the relationship between user related contexts (especially users’ ongoing activities) and their energy consumption often makes it hard to find effective energy-saving (ES) policies without interfering with users. For example, if a person in the living room is undertaking a very energetic activity (e.g., playing xBox using Kinect), the system knows that the energy used by HVAC is likely not wasted. However, if the person is taking a nap or is absent while the HVAC is operating, it is obvious the energy is wasted. Therefore, context-awareness, especially activity recognition (i.e., knowing users’ ongoing activities), is important for many real-life context-aware applications [1]–[3], including energy saving in smart homes. Therefore, in this study, the terms “context” and “activity” are used interchangeably. However, for smart homes to achieve energy saving based on activity recognition, we need to address two critical issues: how to recognize activities in a multi-resident environment, and how to determine whether energy is wasted or not based on activity-recognition (AR) results. There are already lots of studies on activity recognition in smart homes, but most of these works primarily focused on single-user environments. Since it is common that there are multiple residents in a real home environment, single-user activity
1545-5955 © 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
716
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 11, NO. 3, JULY 2014
recognition becomes impractical in a multi-resident environment especially due to the following two challenges. • Complicated data association: Data obtained from a multi-resident environment have to be associated with their corresponding users who trigger the data before applying single-user activity recognition, and thus lots of costs are required to deal with the high complexity of tracking multiple users and the difficulty in modeling interpersonal interactions among users. • Inevitable labeling noises: An activity-recognition approach usually needs users to correctly label each data instance. However, it is natural that multiple activities occur simultaneously in a multiuser environment, and it is very error-prone and unlikely for users to label every occurring activity at each time instance. Consequently, such a misannotated instance causes noises, and the performance of the overall activity recognition will thus be inevitably degraded. In order to overcome these two problems, we proposed the concept of Aggregate Context, which tries to group all individual activities into an aggregated entity, to address the two aforementioned challenges for multiuser activity recognition. For example, although two residents in the living room undertake different activities (e.g., “watching TV” and “reading books”), they as one group share usage of the same service (e.g., “light”) in this space. Therefore, occurrence of several individual activities in the living room can be abstracted as occurrence of an aggregate activity (e.g., ”), which can be regarded “ as a more coarse-grained activity undertaken by a group of users in the living room sharing usage of the same set of appliances. Even with the inferred aggregate contexts, a smart home needs to know the relationship between an aggregate context and its energy consumption to determine if the energy is well spent, like the example depicted in the beginning. This issue has created a barrier for energy-saving technologies developed so far to be actually deployed in smart homes and thus may hinder their further advancement progress. We thus refer an aggregate context with its associated appliances and their power consumptions to Energy-Responsive Aggregate Context (hereafter referred to ERAC for convenience). These ERACs, when being fed back to users in a visualizable manner, will become one type of eco-feedback to help users spontaneously save more energy [4]. Such design should refer to the earlier research [5], which indicates that user’s behavioral changes in the overall electricity usage given the (static) power consumption information of every single electrical appliance may contribute to 5% 15% power saving. Moreover, if a smart home can utilize ERACs with context-aware energy-saving optimization, autonomous energy saving beyond human’s interventions can be achieved. In summary, ignorance of the correlation between a context and energy consumption and difficulty of multiuser activity recognition make most of the prior works fail to accommodate real-life scenarios nor to scale to a multiuser solution for a practical smart home. To address the above challenges, this study proposed ERAC to model aggregate contexts of interest (e.g., activities undertaken by one or multiple residents that
may incur significant power consumption) and to analyze their energy consumption information. With information about all these contexts, smart homes can switch off or change the operating modes of those appliances which are not or less related to the undertaken activity, thus to achieve energy saving without interfering with their users. The remainder of this paper is organized as follows. Section II overviews the prior works on energy saving at home and on multiuser activity recognition. Section III introduces the proposed approach, including how to learn and recognize the aggregate contexts of interest and how to construct ERACs. Section IV summarizes our experimental results about the energy-saving performance of the proposed approach. Finally, Section V concludes this work. II. RELATED WORK Although energy saving has attracted attention in various domains [6]–[11], this study focuses on its applications in a smart home. We briefly classify prior home energy-saving systems into two categories based on how energy saving can be achieved, namely, one with human intervention and the other is with technology intervention. Solutions in the first category are mainly based on the information from smart meters [12], [13], which make energy consumption information available to users and help to decrease the standby power of home appliances which are not being used. However, due to limited capacity of human’s attention, the energy saving may not be thorough, and hence becomes less optimal. Solutions in the second category are mainly based on single-user contexts and they provide automatic appliance control, which can be further divided into two subcategories: non-context-aware and context-aware. For solutions in the former “non-context-awareness” subcategory, they provide energy-saving control based on some fixed and simple rules (maybe according to the sensed information). These solutions are less flexible for a dynamically changing home environment, and likely will render the resulting systems unacceptable to users at the end since the solutions are not adaptive enough to various changes caused by the residents or the environment. As a result, solutions in the latter “context-awareness” subcategory appear to be a more promising design. Among the examples of context-aware energy saving, the Adaptive House project [14] focused more on the adaptability of the house to a user based on his/her occupancy patterns, preferences, and schedules. This project developed an energy monitoring system based on reinforcement learning to estimate user comfort, and then used it to control air heating, lighting, ventilation, and water heating via the actuators in the environment [15]. Another example is the European AIM [16], [17] project, which used wireless sensor networks (WSNs) to monitor and manage the energy consumption according to users’ preferences and their previously observed behaviors. More specifically, this project tried to predict users’ behaviors and to optimize the energy consumption primarily based on users’ daily routines, which are synthesized from users’ locations and house states (such as temperature, lightness, etc.) in the sensed environment. The AIM then provided users with several energy-saving services including intelligent power management, power planning, as well as remote power monitoring and control. Another work
LU et al.: ENERGY-RESPONSIVE AGGREGATE CONTEXT FOR ENERGY SAVING IN A MULTI-RESIDENT ENVIRONMENT
is from Davidsson and Boman [18], [19] and they proposed a Multi-Agent System (MAS) to monitor the users’ locations and utilize user’s preferences to provide appropriate lighting and temperature related services. So far, these works focus more on single-user based energy saving, and often considered saving of power consumption and user-related contexts separately. Therefore, we propose the ideation of ERACs and utilize them in a multi-resident environment so as to reduce more energy waste and also to better accommodate real-life scenarios. An energy-saving approach can utilize the results of activity recognition to justify if the energy is in good use. In order to do activity recognition, sensing devices are chosen to collect information for later feature handling and activity inference. For example, Logan et al. [20] deployed various types of sensors, including RFIDs, ambient sensors, cameras, and microphones to infer user activities using Naïve Bayes and C4.5 decision trees. The other commonly used activity models included Bayesian network [21], support vector machine (SVM [22]), Conditional Random Field (CRF [23]), and Hidden Markov Model (HMM [24]), etc. Kim et al. [25] compared the performances of the four activity-recognition methods including HMM, CRF, skip-chain CRF, and emerging patterns (EP). However, most of these works primarily focus on single-user environments but it is common that there are multiple residents in a real home environment. As for activity recognition in a multi-resident environment, one of the most critical issues is the challenge of data association, which often refers to an effort of associating a sensor event, often triggered by an interaction with the environment, with its corresponding user who initiates the event. Data association becomes even harder when the physical distance of any two users interacting with each other gets closer. It is straightforward that better data association can improve the performance of multiuser activity recognition [26] since there exists positive correlation between the accuracy of data association and that of activity recognition [27]. So far, in the literature carrying wearable sensors is one feasible method to address this challenge, but it may cause discomfort to users. Chiang et al. [23] used non-obtrusive sensors and CRF to solve the problem of data association. On the other hand, to directly cope with the challenge of recognizing multiuser activities, probabilistic graphic models have been proposed to build the engine for activity inference, including Coupled Hidden Markov Model (CHMM) [3], [28], [29], Dynamic Bayesian Network (DBN) [21], and Emerging Patterns (EP) [30]. However, this challenge becomes intractable when the number of resident increases. If the accuracy of multiuser activity recognition is not high enough, the energy-saving services provided by system may become annoying. Therefore, we propose aggregate contexts as an alternative and use them to improve the performance of energy saving in a multi-resident environment.
III. ENERGY-RESPONSIVE AGGREGATE CONTEXT INFERENCE Traditional energy-saving systems often provide energy consumption information and user related contexts separately. That is, they decouple user related contexts from their energy usage, and also ignore those implicitly (or indirectly) related appliances while constructing an energy-saving policy. Such energy-
717
saving systems may fail to make the best of the useful information about the inferred contexts to derive more optimal energy-saving policies. This section will elaborate how the proposed ERAC models the relationship between an aggregate context and all associated appliances. In this work, an aggregate context here primarily refers to an aggregate activity, involving more than one individual activity, and we regard the case with only one activity as a special case. Before we proceed, we assume that various sensors have been deployed in the home environment to constantly extract features (such as temperature, lighting, sound, power consumption level of an appliance, etc.) relevant to the contexts of interest. The flowchart of the training and testing of the proposed approach is as shown in Fig. 1. The training phase contains two major steps, namely, “Context Aggregation” (named as “Context Generalization” in our previous work [31]) and “Model Construction.” The first step abstracts or groups individual activities to form an aggregate-context (AC) layer by layer via aggregating activities with similar feature combinations and power-usage signatures, whereas the second step constructs three kinds of models for aggregate contexts for each layer, namely, context models (CMs) and correlation models (two types: ERACs and ETACs), where each of the latter captures the relations between a given aggregate context and its associated power-usage signature. As shown in Fig. 1, an ERAC is an inferred aggregate context embedded with information about energy consumptions of all the associated electrical appliances. Note that all appliances including those with zero power consumption (e.g., a light whose state is “off”) are all taken into consideration when an ERAC is formed. Note that here we assume that the power consumption of an appliance can be measured by a smart meter, and the correlation between an aggregate context and an appliance can be learnt from the data collected by a smart home system. To facilitate humans to decode useful information from an ERAC, we also propose Energy-Tagged Aggregate Context (ETAC) as a visualizable and simplified graphical representation of an ERAC. Besides visualization, the main difference between the above two is that ETAC ignores those appliances with zero power consumption whereas ERAC does not. The reason of the neglect is twofold: one is that there are often many appliances of this kind (with zero power consumption) for an aggregate context (or an activity), and the other is that humans are usually interested only in those appliances which consume power. More details about ERACs and ETACs will be elaborated in Section III-C. In the following, we delve into the details of the process in Fig. 1. A. Context Aggregation By utilizing the features extracted from the sensor data collected from the sensed environment, the activity-recognition system can first train the model of each individual activity using Bayesian Networks (cf. [32]) and then detects the occurring activity. Once the models of all individual activities are trained, the system will iterate context aggregations. The first step is to determine all possible combinations of contexts that can be aggregated satisfying the minimum pairwise “distance,” and then generate aggregate contexts. The next step is to iterate the process introduced in previous step by repeatedly clustering aggregate contexts and the leftover contexts from the previous
718
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 11, NO. 3, JULY 2014
Fig. 1. The flowchart of training and testing Aggregate Contexts and construction of energy-responsive aggregate contexts (ERACs).
Fig. 2. The concept of Context Aggregation.
step, thus producing a new set of aggregate contexts unless no new aggregate context can be further generated. An example to illustrate this concept is shown in Fig. 2. The initial context (activity) set for the first layer context aggregation, denoted as AC(0), consists of seven individual activities with their corresponding context models, namely, to . In the next layer, the resulting context set denoted as AC(1) consists of aggregate contexts and/or original contexts based on the aforementioned minimum pairwise distance and are clustered criterion. In this example, , whereas and being clustered into into . This figure illustrates the results from a three-layer clustering, and finally totally 15 aggregate contexts/contexts can be found. It is worthwhile to note that each higher level aggregate context often contain at least one lower level aggregate context. Generally, the higher the layer goes, the less specific the resulting context will become. For example, each con-
text in the layer of AC(0) is as specific as each individual activity originally defined, and once it is detected, the exact corresponding services will be launched. In contrast, at the layer of AC(2), we can aggregate them into three high-level contexts, and each of them can hardly be specific as an individual activity . For example, the coarse meaning of some may except be just like “people perform some activities in the living room” or “people undertake activities in the kitchen.” In fact, the advantage of having aggregate contexts is that, since the associated context model comprises more lower level context models, success of recognition of an occurring activity obviously is higher, which facilitates the smart home to issue some general (none specific) energy-saving services with higher probability, leading to higher energy saving. Semantically, the aforementioned distance between two aggregate contexts actually refers to the similarity between their corresponding ERACs. Since an ERAC includes the correlation between an aggregate context and power-usage signature of its associated appliances, the distance between two ERACs can be calculated by the following equation shown at the bottom of the next page, where is the correlation confidence between an (Aggregate) (e.g., “WatchTV”) and an appliance (e.g., “TV”) whose state is (e.g., “on”). The correlation confidence is represented using the likelihood of co-occurrence [33]. Apparently, if the distance is shorter, the two ERACs appear more similar, and hence are likely to be clustered. As mentioned earlier, this will increase the chances of providing more general (i.e., less specific) energy-saving services based on aggregate contexts. Note that the level numbers of context aggregation is determined by
LU et al.: ENERGY-RESPONSIVE AGGREGATE CONTEXT FOR ENERGY SAVING IN A MULTI-RESIDENT ENVIRONMENT
a predefined threshold with respect to the minimum pairwise distance defined in (1), and the distance is to measure the divergence between two contexts to be aggregated. A higher predefined threshold for the correlation confidence will result in an earlier termination of context aggregation, whereas a lower one may adversely cause all activities to be clustered into one single aggregate context. The threshold of correlation confidence can be determined by users, and the value of represents the false positive rate of inferring contexts that users can tolerate since each false positive may lead to undesirable services applied to users. The detailed procedure of iteratively clustering aggregate contexts is shown in Algorithm 1. The precondition that any can be clustered to a higher two aggregate contexts in is that the two in should incur level one in AC at least one common energy-saving service (via turning on/off or up/down some common appliances) based on their respective ERACs and/or some attributes defined by the adopted energy-saving policies. This way, once any individual member of an aggregate context being detected takes place, the aforementioned common energy-saving service of all the members, instead of the service specific to that particular member, will be delivered to the users.
// means measuring distance based on ERAC is reckoned using (1)} else { } if } } for { if (
where
for { if
{ aggregate new AC as put
into
and
into a }
} }
: the set of context models for
}
: the set of ERAC models for AC(0)=the set of all individual activities in the beginning Train CM(0) and ERAC_AC(0) for each individual activity in AC(0) // the index of the current layer while (there exist common ES services between two members in ) or attributes
{
for { for { means having common ES services &&
}
{
: the th aggregate context in
if (
){
else
: the set of aggregate contexts in the th layer
//
into
put
Algorithm 1. Pairwise Hierarchical Clustering for Context Aggregation
//Common ES services are based on defined by ES policies
719
){
train
and
for
}
After all members of each aggregate context in are determined, each data instance within the newly clustered aggregate context will be relabeled using its corresponding label ” as the label for the aggregate (e.g., “ context comprising “WatchTV” and “ReadBooks”). Note that the symbol “ ” in this work can be semantically interpreted as “OR” for easy interpretation. With this as an example, the system will likely provide energy-saving services to reduce energy waste as long as an activity resembling (but not identical to) “WatchTV” or “ReadBooks” is detected. In addition to the training of context models for all individual activities, each aggregate context will also have its corresponding model facilitating later detection of an aggregate context of interest. Here, we use a Dynamic Bayesian Network (DBN) for modeling an individual activity or an aggregate context since it has proved effective in our previous work [32].
(1)
720
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 11, NO. 3, JULY 2014
In summary, inferring aggregate contexts gives the system extra opportunities to successfully enforce some energy-saving services. In the next section, we will continue to discuss how to construct relevant models based on the results of Context Aggregation. C. Construction of ERAC Model and ETAC Graph
Fig. 3. AC enhances ES performance by providing more general ES services.
B. Inference of Aggregate Context The detailed procedure of inference is in Algorithm 2, and is briefly explained as follows. denote the th aggregate context in , the set Let of aggregate contexts in the th layer, 1) Use all activity models in every layer to infer whether an occurs. aggregate context is determined to be positive, 2) If the occurrence of check whether all the individual activities within have not been determined in some lower layer. have not 3) If all the individual activities included by into the set of inference been determined yet, put results. Otherwise, ignore the inference result of . In short, the system will use the inference results of context models whose AC layers are as lower as possible since the system under that circumstance can provide more specific energy-saving services rather than general services to the users in order to achieve higher energy-saving performance. For example, based on the ERAC of an aggregate context in the lower layer, say “ReadBooks,” an energy-saving system can determine both TV and Wii/xBox as potential appliances to be turned off since both appliances are not related to the “ReadBooks” activity. On the other hand, based on the ERAC of an aggregate ,” only context in the higher level, say “ Wii/xBox can be determined as the potential one to be turned off since this appliance is not related to the “ ” activity. Apparently, the energy saving effect of the latter is less than that of the former. As a result, the energy-saving system prefers to provide energy-saving services based on finer grained contexts whenever possible, and such contexts can be detected by the AC models resided in the lower layers. On the other hand, in the situation where there is only one resident in the environment, it is often every individual activity can be detected and hence the system will provide energy-saving services based on its ERAC. However, there are also cases where some individual activities fail to be detected due to interference from other activities, and thus no energy-saving services can be issued, causing zero energy saving. To prevent suffering from this difficulty, our system will continue to check if any aggregate context is in the higher layer, and less context granularity, can be recognized. If affirmative, the system will launch more general energy-saving services, better than no energy-saving services, and therefore achieve some energy saving effect still. Such ideation is illustrated in Fig. 3.
After aggregate contexts are determined, the system in the training phase will build three kinds of models for each aggregate context, as shown in Fig. 1. One of them is Context Model, which can be successfully built by using DBNs in our previous work [32], and the other two, ERAC and ETAC, are detailed in this section with examples. Algorithm 2. Recognition of Aggregate Context : the set of aggregate contexts in the th layer : the th aggregate context in : the aggregate-context model for AC(0)=the set of all individual activities in the beginning : the set of all individual activities been inferred as Positive : the set of the inference results
while (there is
)
{ has not been used)
while ( {
if (system determines )
occurs according to
{ AC(0) and
={activities belonging to both }
// convert the inference result into a set of individual activities if {// all the individual activities included by the inference result are newly inferred as Positive // put those newly inferred individual activities into // put the inference result } } } } return
into
LU et al.: ENERGY-RESPONSIVE AGGREGATE CONTEXT FOR ENERGY SAVING IN A MULTI-RESIDENT ENVIRONMENT
1) Energy-Responsive Aggregate Context (ERAC) Model: An ERAC model is used to estimate the correlation confidence between an aggregate context of interest and the states of its associated appliances. Besides that, an ERAC also models different types of relations between them. More specifically, according to the types of power consumption with respect to an aggregate context, we divide the power consumption into two types. • Explicit power consumption: The power consumption of an appliance is directly triggered by or closely related to an activity. We therefore name an appliance explicitly related to a given context as an explicit appliance. That is, the states of all explicit appliances are likely to be chronologically synchronized with their corresponding activity. In other words, when an activity takes place, all explicit appliances change their states within a predefined time window. In terms of mathematical formulation, an appliance belongs to explicit power consumption in an ERAC if its mutual information is greater than a predefined threshold. Given and an appliance , their mutual inan AC formation can be calculated as follows:
721
Fig. 4. Examples of (a) explicit power consumption and (b) implicit power consumption.
(2) where is one of the possible states for (e.g., Positive or Negative), and refers to one of the possible states for (e.g., ON, OFF, or Standby). • Implicit power consumption: such power consumption is made by an appliance that is indirectly triggered by an activity of interest. Therefore, the operating interval of an implicitly related appliance (aka an implicit appliance in this work) is often much longer than that of the activity to be detected. Unlike the mathematical formulation for an explicit appliance, an appliance belongs to implicit power consumption in an ERAC if it does not belong to explicit power consumption and its co-occurrence probability with a given context is greater than a predefined threshold. and an appliance More specifically, given an AC , their co-occurrence probability can be calculated as follows:
(3) where the meaning of and are the same as those defined in (2). The concepts of explicit and implicit power consumption are illustrated in Fig. 4, and more details can be found in our previous work [34]. Most of the prior works made efforts to detect explicit power consumption since it is obvious and intuitive for an energy-saving application. However, the advantage of detecting implicit power consumption is to provide additional opportunities to achieve more energy saving, but such type of power consumption is often neglected in most of the prior works. To benefit from this type of energy saving, our system first checks that, for a given implicit appliance, whether all the aggregate contexts associated with the implicit appliance
Fig. 5. An example of an ERAC model for “WatchTV.”
are not detected (i.e., all those corresponding activities which may use this implicit appliance are not undertaken at the moment), then the system can achieve energy saving by switching the operating mode or shutting down the standby power of those implicit appliances. More details about how to save energy according to ERAC can be found in our previous work [34]. An example of “WatchTV” ERAC is shown in Fig. 5, which is represented by a profile of an activity with all related energy-usage information categorized under the explicit and implicit power consumption. 2) Energy-Tagged Aggregate Context (ETAC) Graph: As mentioned earlier in the previous section, an ETAC is one graphically represented model of an ERAC to represent an aggregate context visually “tagged” with all energy related information, which provides a more intuitive way to interpret an ERAC. As a matter of fact, an ETAC graph can be regarded as a simplified version of a Bayesian network, but it provides much richer information to users. An example of our prototyped ETAC graph to represent the “Watch TV” ERAC in Fig. 5 is shown in Fig. 6, and different representations of an ETAC graph can be designed in the future to further improve its readability. In an ETAC graph, the central circle represents an aggregate context, which is surrounded by some smaller satellite circles representing the associated appliances of the aggregate context. The solid and dotted links represent explicit and implicit power consumption, respectively. The annotation beside a satellite circle denotes the power consumption of the appliance and its correlation confidence associated to the aggregate context. Note that the thickness of a link represents the strength of the aforementioned correlation confidence between the appliance
722
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 11, NO. 3, JULY 2014
Fig. 6. An example of an ETAC graph.
TABLE I TABLE OF POWER CONSUMPTION LEVEL
and the aggregate context. In other words, the higher the confidence is, the stronger the appliance is correlated to the given aggregate context represented by the central circle, and hence the thicker the link becomes. Since the range of power consumption varies significantly with different appliances, for an easier way to represent power consumption in an ETAC, this study quantizes power consumption into several discrete levels, as shown in Table I. Each level is reflected by its corresponding length of a link. In order to take location of an appliance into consideration, the power consumption in this study is further classified into “local” and “remote” types. More specifically, if the appliance is located in the same region (e.g., the TV and the air conditioner in the living room) where the aggregate context takes place, then its power consumption is deemed local; otherwise, it is deemed remote. Given the aforementioned categorizations on an appliance and its power consumption, it is intuitive that a remote and implicit appliance with a weak correlation to an aggregate context will be a highly prioritized candidate appliance whose operational mode can be changed (i.e., turn on/off or up/down) for achieving more energy saving. In summary, unlike most of the traditional energy-saving approach, the hereby proposed ERACs and ETACs additionally takes energy usage of all associated appliances into account, thus enabling the energy-saving system to identify the main causes of energy waste, especially based on the knowledge of the implicit appliances. In this regard, the proposed energysaving approach is able to provide users with more comprehensive information about their routine energy usage. Also, the system can control energy consumption more accurately and can facilitate more spontaneous energy saving via the proposed ERACs. In addition to the aforementioned benefits, the system can utilize an ERAC model or an ETAC graph to find alternative energy-saving service for every associated appliance. For
Fig. 7. Layout of Smart-Home Lab and its sensor deployment.
example, based on the ETAC graph in Fig. 6, the energy-saving system can try to dim the light or open the windows of the living room to increase the indoor illumination. The energy-saving system also can switch the water heater to an energy-saving mode when currently no aggregate context is explicitly associated with that appliance (e.g., no one is using hot water). IV. EXPERIMENTS The results of this study belong to a multiyear interdisciplinary project (kicked off in 2010) titled “M-CHESS: M2M-based Context-Aware Home Energy Saving System,” which is one of the ongoing projects in Intel-NTU Connected Context Computing Center at National Taiwan University (NTU). The goal of this project is to build upon an M2M (machine to machine) infrastructure the context-aware home energy-saving system which tries to reduce the power consumption at a home to its minimum but not to compromise the user comfort in the singleor multiuser environment. In this work, the metrics of the system evaluation is divided into two parts: 1) the accuracy of activity recognition enhanced via aggregate contexts and 2) the degree of energy saving after deployment of the hereby proposed context-aware energy saving measures. The former is evaluated via a real-user experiments conducted in our simulated home, which is a physical home-like lab located at NTU, and is referred to as Smart-Home Lab afterwards. Since one commonly faced problem in activity recognition is that we cannot find a baseline dataset that can be utilized to compare accuracy among different models [35] or under any circumstances, we need to collect data from scratch in the Smart-Home Lab to evaluate the proposed system. Fig. 7 shows the layout of the Smart-Home Lab, in which several wireless multimodal sensors are deployed, including current-flow, light, microphone, and touch sensors, to collect data. Note that for the privacy issue and that our system does not need multiuser tracking due to the omission of data association, the four cameras are primarily for
LU et al.: ENERGY-RESPONSIVE AGGREGATE CONTEXT FOR ENERGY SAVING IN A MULTI-RESIDENT ENVIRONMENT
723
TABLE II OCCURRING PERCENTAGE OF EACH INDIVIDUAL ACTIVITY IN THE SMART-HOME LAB
Fig. 8. The ETAC graphs for all tested activities in the Smart-Home Lab.
collecting ground-truth activities for labeling training data afterwards. We also ignored the power usage of these wireless sensors in the current phase since their total power consumption is relatively much less than that of all home appliances. Besides, the power consumption of the sensors can be effectively reduced using approaches proposed in [36] and [37]. The latter metric, which is the degree of energy saving, is first evaluated via an experiment also conducted in the Smart-Home Lab. However, due to the layout constraints of the lab, some home activities (e.g., bathing, washing, etc.) cannot be undertaken, and thus difficult to conduct long-term multiuser experiments with 24/7 living scenarios. Therefore, for conducting the experiment for further evaluating the performance based on the second metric, we have created a home simulator to better reflect a real-life scenario for evaluating how much the energy consumption can be saved.
A. Evaluation of Real-User Activity Recognition in the Smart-Home Lab In order to evaluate how an aggregate context enhances the performance of activity recognition, we invited five groups of subjects, each of which has two persons, to join the experiment, and collected multiuser activity data about two weeks for training and testing the context models. All subjects are with technology background, and their ages range from 22 to 32. They were asked to read a brief instruction before the experiment as rough guide and they undertook activities based on a predefined scenario to avoid unbalanced data collection. Table II lists the average percentage of occurrence of each activity for the experiment. Note that the difference between the two activities, “Sleep” and “All_Sleep,” is that the former
implies that at least one subject is not sleeping, but is doing another activity, while another subject is sleeping. To simulate a real-life scenario in the Smart-Home Lab for system evaluation, the percentage of the total instances of an activity undertaken by the subjects in the testing scenario over the entire collected activity instances is set roughly identical to the counterpart that happens in our real life, referring to our pilot survey. For example, the subjects need to spend roughly one fourth of time duration in sleeping during our experiment since an ordinary person spends that percentage of time in sleep in his life time. Note that although our scenario involved two people, sometimes there is only one person in the Smart-Home Lab (e.g., the other person went shopping), and sometimes two people were doing the same activity together (e.g., watching TV together). As a result, more than one activity may take place at every time instance, and this makes the total number of collected instances of that particular activity exceed the pre-set number of that activity. Consequently, the more frequently an activity is undertaken by two people together, the lower its percentage of occurrence would become. For example, because both of the subjects often rested together, the total percentage of instances of the activities, “Sleep” and “All_Sleep,” in Table II is lower than one fourth, which is the supposed percentage of occurrence of sleeping for a single subject under our scenario design. According to the testing scenario, these target activities and their ETAC graphs, which are the simplified graphic representation of ERACs, are shown in Fig. 8. From these ETAC graphs, a user can more readily learn how an activity consumes energy and such information may potentially motivate users to seek more spontaneous energy saving. For example, the system can additionally “tag” each appliance in an ETAC with, say, “dollars/Watts,” or even with some energy-saving tips (not derived from an ERAC but from the Internet). Such additionally tagged
724
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 11, NO. 3, JULY 2014
RESULTS
OF
TABLE III CONTEXT AGGREGATION
TABLE IV PERFORMANCE OF ACTIVITY RECOGNITION FOR EACH ACTIVITY
information may incur more spontaneous energy-saving measures from users, which certainly requires deeper user study to verify its effectiveness in the next phase. Based on Algorithm 1 and formula (1), our system performs the proposed Context Aggregation and the results are shown in Table III (more details can be found in our previous work [31]). The two activities, “Watch TV” and “Play xBox,” have been clustered into an aggregate context at layer one, i.e., AC(1), since according to their ERACs, their power-usage signatures are very similar (please refer to Fig. 8). Likewise, “No One at Home” and “All Sleep” are clustered in to the same aggregate context in the next layer. The higher the layer goes, the more coarse-grained the activities become. For example, in the layer of AC(0), the system can successfully distinguish each individual activity. In contrast, at the layer of AC(7), we can generalize all activities into three aggregate contexts, each of which semantically may represent “people perform activities in the living room,” “people undertake activities in the study room or bed room,” or “someone is sleeping but someone is not.” This demonstrates that the system can provide more general energy-saving services to users based on aggregate contexts. The performance of activity recognition is evaluated via fourfold cross validation, and the results are shown in Table IV. F-measure, a harmonic combination of precision and recall, is used for performance evaluation and its equation is shown below (we set in this experiment) (4)
The results under the header: “Without Label Noises” in Table IV refer to the cases where users can provide perfect labeling. Although these results are obtained by using only the individual activity models (i.e., without the help of Context Aggregation yet), the accuracy of recognizing activities can be as high as 90% even if most of them were undertaken by more than one subjects. One exception is that “Sleep” has a lower recall value due to the data generated by the other subject, which refers to interference problem for context recognition. The results under the header: “With Label Noises” in Table IV refer to the cases where users are assumed able to provide only 20% perfect labeling; that is, for all the instances that involve two simultaneous activities, 20% of them are labeled accurately, whereas the remaining 80% are only given one of the labels. The reason to look into such labeling noise issue is that we are interested in knowing what the impact to activity recognition will be for a user tends to provide error-prone data annotation especially in a real-life application (i.e., this is an important issue for improving practicality of the energy-saving system). The results show that the precision values almost remain the same, but the recall values significantly degrade. This is why this work makes use of aggregate contexts to reduce degradation of the performance. The results under the header: “AC enhanced (w/ Label Noises)” in Table IV show the performance of aggregate-context enhanced activity recognition for the cases simulating 20% of perfect labeling mentioned earlier. Note that since an aggregate context may include multiple activities, the system
LU et al.: ENERGY-RESPONSIVE AGGREGATE CONTEXT FOR ENERGY SAVING IN A MULTI-RESIDENT ENVIRONMENT
with the support of aggregate contexts need to redefine the precision and recall as follows. • If a ground-truth activity is recognized in any layer through context inference, there is a True Positive. For example, given that the ground truth activity is “Watch TV,” the system will get a True Positive if any model whose output label consisting of “Watch TV” (e.g., “ ” in the 2nd layer) infers Positive for the “Watch TV” activity. • If all models fail to recognize the ground-truth activities, all estimates in the recognition result will be marked as False Positive. For example, the system infers “ ” (at the lowest layer), but neither “Watch TV” nor “Play xBox” is a ground-truth activity, then both “Watch TV” and “Play xBox” get False Positives. The results show that even with the “labeling noises” problem which usually happens in a practical multiuser environment, AC-enhanced activity recognition can still achieve high performance. B. Discussion of the Real-User Experiment A lesson learned from the experiment is that it is better to set a high threshold against which the estimated probability of the activity-recognition models, generated by Context Aggregation, can be compared in order to determine if an activity really occurs or not. This way, the system can reduce the chances of false positive. When the system is executing the Pairwise Hierarchical Clustering (i.e., Algorithm 1) and it reaches the higher layer of clustering, the resultant activity-recognition models will recognize more general rather than specific activities, as shown in Table III. This leads to pros and cons, where the former is that models at the higher layer can be used to recognize those activities which cannot be recognized in the previous lower layers, whereas the latter is that recognition precision can be lower due to more false positive in the results. One should beware that aggregate contexts at the higher layer may become much more general and hence their mutual dissimilarities become much lower, which unfortunately may more easily induce possible wrong inference later on. Therefore, it is advisable to set a high threshold for activity-recognition models generated by Context Aggregation, which thus lowers the probability of generating false positive estimates. Another lesson learned from this experiment is that utilizing aggregate contexts can help to avoid the dilemma of setting an appropriate threshold for performance evaluation in activity recognition. For a context-aware energy-saving system, the precision of activity recognition is closely related to how well the energy-saving system can correctly provide energy-saving services, whereas the recall of an activity-recognition system is proportional to how often the system gets to provide corresponding energy-saving services. Therefore, a context-aware energy-saving system needs both high precision and high recall so as to provide correct energy-saving services whenever possible. Generally speaking, it is a tradeoff between precision and recall given a fixed threshold for performance evaluation. A
725
TABLE V RECOGNITION ACCURACY OF INDIVIDUAL ACTIVITIES
TABLE VI RECOGNITION ACCURACY OF CONTEXT AGGREGATION
higher threshold results in higher precision but lower recall, and vice versa. From the human-centric perspective, a system providing services to humans has to consider how the users feel about the service quality. Since the lower threshold results in the lower precision, which increases the probability of annoying users with wrongly provided services, it is better to choose a higher threshold yet at the price of lower recall. Nevertheless, this price can be greatly reduced by the proposed AC-enhanced activity recognition because it can increase the recall by utilizing the more general activity-recognition models generated at the higher AC layers. Consequently, by setting a high threshold for each layer of aggregate contexts, a context-ware energy-saving system can assure most of its energy-saving services will be correctly provided. In addition, with the more and more general activityrecognition models supported by aggregate contexts, the system can manage to provide correct energy-saving services as frequently as possible. In terms of the potentials of utilizing aggregate contexts to increase the chances of providing more generalized energy-saving services, Tables V and VI show the accuracy of recognizing individual activities and aggregate contexts on the M-CHESS. The results show that the accuracy can be improved using context aggregation, which in turn increases the opportunities for the smart home system to provide more generalized energysaving services. In other words, like what has been stressed in the previous section, an aggregate context, which often comprises several lower-level context models, can improve the success rate of recognition of the occurring activities. This facilitates the smart home to issue more generalized rather than specific energy-saving services with higher probability, leading to higher energy saving. Since the energy saving is achieved based on the results of activity recognition, activity recognition error occurs may mislead the energy saving system. In case of wrong activity recognition, the system may provide incorrect services. In this regard, we have implemented a web-enabled interface to help users correct the wrong provided services. In the next phase, we will use the feedback of correction to detect the situation of wrong activity recognition and then adapt the currently working models accordingly.
726
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 11, NO. 3, JULY 2014
C. Evaluation of ES Performance on the Home Simulator This section will show that, with AC enhanced activity-recognition models and the energy-saving services based on ERAC models, how much energy consumption can be saved. We will present the energy-saving performance of the proposed energysaving system (i.e., the M-CHESS) in the Smart-Home Lab first, and then the performance on a home simulator. 1) ES Performance in the Smart-Home Lab: With the AC enhanced activity-recognition models built in the previous experiment, we invited 11 subjects to undertake activities by following non-predefined single-user scenario, and then we evaluated the energy saving without and with the intervention of the M-CHESS. In this experiment, the subjects are the adults whose ages range from 22 to 40 since we believe they are more open-minded to new smart-home technologies. In addition, the subjects within this age span may need to pay utility bills, which may foster them to care about money spent on electricity. Half of these subjects come from technology background, and the other half come from business management background. More specifically, this experiment includes two stages. In the first stage, the M-CHESS does not provide any assistance for energy saving. In the second stage, the M-CHESS will provide energy-saving services whenever necessary. Like the experiment in the previous subsection, all subjects were asked to read a brief instruction as rough guide, but during this experiment, a subject can undertake a preferred order of activities (i.e., in an arbitrary order) rather than based on a predefined scenario in the previous experiment. The only constraint is that the scenarios of the two stages for each subject should be highly similar to provide a fair ground for energy-saving performance comparison. One simple and intuitive way to achieve energy saving at home is utilizing automatic lighting control by a location-aware energy-saving system since it can switch off lights whenever a user leaves a sensed space. However, a further enhanced energy saving at home can be done via appliance control, and this kind of energy saving can hardly be achieved without assistance from a context-aware system because users may undertake activities in the same location (and thus cannot trigger conventional location-based energy-saving system). The latter kind of energy saving is what the M-CHESS focuses on, and in the current phase, the energy-saving control policies are some rule-based control commands derived from the proposed ERAC and triggered by the inferred aggregate contexts. These simple rules are meant to shut down the standby power or to switch off unused appliances whenever possible. For evaluating the performance of the M-CHESS under different lengths of time, each stage lasts for about 3 hours for 4 of these 11 subjects, and lasts for 8 hours for the rest of the subjects. As has been illustrated in Fig. 9, with the M-CHESS involved, the average energy consumption can be reduced by about 33.3% for those 4 users participating in the 3-hour experiments. As for the rest of 8 users participating in 8-hour experiments, the average energy consumption is reduced by about 33.7% with the M-CHESS involved. The energy-saving performance varies a lot depending on the energy-saving habits of the subjects (e.g., whether they are used to turning off the unused appliances every time) and
Fig. 9. The comparison of energy consumption without/with M-CHESS intervention (the unit is Kwh).
Fig. 10. The interface of the home simulator.
their personal preferences (e.g., some of them frequently undertook activities that consume lots of energy whereas some of them did not). Nevertheless, the M-CHESS shows promising performance. 2) ES Performance on the Home Simulator: The energysaving performance of the M-CHESS is further conducted on our home simulator to better reflect a real-life scenario with multiple users living 24/7 in a home environment. The interface of the home simulator is shown in Fig. 10. In this simulator, we simulate an apartment which has a common indoor layout for a two-person residence in Taiwan. For a more realistic simulation, we have created a virtual couple as two subjects for the experiment. In order to make this simulation more realistic, we collected a weekly real-life routine from a couple. Their routine schedule is shown in Fig. 11 where each activity has its corresponding ID. The schedule also has been transformed into user scripts. That is, according to these user scripts, our home simulator can simulate the life of the couple and thus enables us to estimate the total energy consumption. In addition to simulating activities undertaken by executing the user scripts, the preferences of the couple towards appliance configuration (such as temperature thresholds) were also encoded in the user scripts. After executing the user scripts, the simulator began measuring the energy usage of the couple. With the measurement, we can compare the energy consumptions
LU et al.: ENERGY-RESPONSIVE AGGREGATE CONTEXT FOR ENERGY SAVING IN A MULTI-RESIDENT ENVIRONMENT
727
TABLE VII RESULT OF ENERGY SAVING
Fig. 11. The weekly routine of the couple.
ness and the simple energy-saving policies, the total amount of power consumption is reduced to 41.392 kWh, as shown in Table VII. From the simulation, the energy-saving system helps to reduce energy waste by about 28.6% on average. From this simulation, we learned that the energy consumption caused by those unused or unrelated appliances can be reduced via assistance of context-awareness. Even though the simulation can save extra energy by more than 28%, it is also important that we need to keep user comfort in mind [38]. Therefore, both energy saving and user comfort will be taken into account in our next phase of the system enhancement. V. CONCLUSION
Fig. 12. The power consumption of all appliances on the simulator.
with and without activation of the context-aware energy-saving system. Like the two-stage experiment in the Smart-Home Lab, we have a control group (as the baseline which deactivates the context-aware energy saving) and an experimental group (with the context-aware energy-saving system activated) for evaluating the energy-saving performance through Home Simulator. Both groups sequentially undertook all activities assigned by the user scripts. For quantitative evaluation, the power consumption of every appliance was measured a priori using a smart meter just as that used in our lab, and the results are listed as in Fig. 12. That is, the data regarding the power consumption of each appliance on the home simulator were actually obtained based on the readings of the smart meters from the Smart-Home Lab. There are three states on each appliance, namely, ON, OFF, and Standby. The energy-saving control policies of the simulator are the same as those in the Smart-Home Lab. Before applying context-aware energy saving system, the power assumption is about 57.445 kWh, but after applying the context-aware-
In order to make multiuser activity recognition far less intractable, we have reformulated this challenging problem and proposed an aggregate rather than individual based activity-recognition system to greatly reduce the complexity inherent in a multiuser context-aware problem for energy-saving applications. The first contribution of this work is to regard multiple users as a whole and aggregate them for obtaining an aggregate context to increase the success rate of the provided energy-saving services from our energy-saving system and its service time-span. When an individual basic activity cannot be identified, an aggregate context enables the system to provide a more general energy-saving service since an aggregate context is more abstract or has a higher abstraction level than any individual activity. In the experiment, it has been successfully shown that the accuracy is improved via Context Aggregation, which also enables a context-aware energy-saving system to provide a coarse-grained energy-saving service before any individual activity can be detected. In short, our proposed context-aware energy-saving system will exhibit more promising performance by obtaining extra opportunities to provide more appropriate energy-saving services. With the proposed context-aware energy-saving system, we have achieved energy saving by about 28.6% 33.7% in our preliminary evaluation in a semi-real home environment and on the home simulator. We are seeking a real house for the next phase more realistic evaluation. Another contribution of our work is to propose the EnergyResponsive Aggregate Context (ERAC) model, which relates each aggregate context with its associated explicit and implicit appliances as well as their power usage signature. Such model
728
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, VOL. 11, NO. 3, JULY 2014
can also be transformed to a graphical model, named EnergyTagged Aggregate Context (ETAC) graph, which is convenient for user’s visualization. We have shown that the former documents rich information about a context with various types of power consumption, whereas the latter provides more visualizable information to user to provoke more spontaneous energy saving. In the next phase, we will conduct some user study to gain user feedback whereby we can further improve readability of the current ETAC graph. Furthermore, we will provide various representations of an ETAC graph by working with experts with design expertise. REFERENCES [1] O. P. Popoola and K. Wang, “Video-based abnormal human behavior recognition—A review,” IEEE Trans. Syst., Man, Cybern., Part C: Appl. Rev., vol. PP, pp. 1–14, 2012. [2] G. J. Burghouts and J. W. Marck, “Reasoning about threats: From observables to situation assessment,” IEEE Trans. Syst., Man, Cybern., Part C: Appl. Rev., vol. 41, no. 5, pp. 608–616, Sep. 2011. [3] D. Trabelsi, S. Mohammed, F. Chamroukhi, L. Oukhellou, and Y. Amirat, “An unsupervised approach for automatic activity recognition based on hidden Markov model regression,” IEEE Trans. Autom. Sci. Eng., vol. 10, no. 3, pp. 829–835, Jul. 2013. [4] T. Peffer, D. Perry, M. Pritoni, C. Aragon, and A. Meier, “Facilitating energy savings with programmable thermostats: Evaluation and guidelines for the thermostat user interface,” Ergonomics, vol. 56, pp. 463–479, 2012, 2013/03/01. [5] T. Ueno, F. Sano, O. Saeki, and K. Tsuji, “Effectiveness of an energy-consumption information system on energy savings in residential houses based on monitored data,” Appl. Energy, vol. 83, pp. 166–183, 2006. [6] A. Vergnano, C. Thorstensson, B. Lennartson, P. Falkman, M. Pellicciari, F. Leali, and S. Biller, “Modeling and optimization of energy consumption in cooperative multi-robot systems,” IEEE Trans. Autom. Sci. Eng., vol. 9, no. 2, pp. 423–428, Apr. 2012. [7] M. Dominguez, A. Fernandez-Cardador, A. P. Cucala, and R. R. Pecharroman, “Energy savings in metropolitan railway substations through regenerative energy recovery and optimal design of ATO speed profiles,” IEEE Trans. Autom. Sci. Eng., vol. 9, no. 3, pp. 496–504, Jul. 2012. [8] P. Tae-Jin and H. Seung-Ho, “Experimental case study of a BACnetbased lighting control system,” IEEE Trans. Autom. Sci. Eng., vol. 6, no. 2, pp. 322–333, Apr. 2009. [9] W. Zhu and W. Lingfeng, “Occupancy pattern based intelligent control for improving energy efficiency in buildings,” in Proc. IEEE Int. Conf. Autom. Sci. Eng. (CASE), 2012, 2012, pp. 804–809. [10] M. Endo, H. Nakajima, and Y. Hata, “Simplified factory energy management system based on operational condition estimation by sensor data,” in Proc. IEEE Int. Conf. Autom. Sci. Eng. (CASE’12), 2012, pp. 14–19. [11] M. Tien-Yan, L. Chin-Yang, H. Shu-Wei, H. Che-Wei, and H. TingWei, “Automatic brightness control of the handheld device display with low illumination,” in Proc. IEEE Int. Conf. Comput. Sci. Autom. Eng. (CSAE’12), 2012, pp. 382–385. [12] E. Williams, S. Matthews, M. Breton, and T. Brady, “Use of a computer-based system to measure and manage energy consumption in the home,” in Proc. IEEE Int. Symp. Electron. Environ., 2006, pp. 167–172. [13] T.-S. Choi, K.-R. Ko, S.-C. Park, Y.-S. Jang, Y.-T. Yoon, and S.-K. Im, “Analysis of energy savings using smart metering system and IHD (in-home display),” in Transm. Distrib. Conf. Expo.: Asia and Pacific, 2009, pp. 1–4. [14] M. C. Mozer, “Lessons from an adaptive house,” in Smart Environments: Technologies, Protocols, and Applications, D. Cook and R. Das, Eds. Hoboken, NJ, USA: Wiley, 2005, pp. 273–294. [15] M. C. Mozer, “The neural network house: An environment hat adapts to its inhabitants,” in Proc. Amer. Assoc. Artif. Intell. Spring Symp. Intell. Environ., 1998, pp. 110–114. [16] M. Barros, H. Hrasnica, S. Tompros, and M. Caragiozidis, “AIM architecture evaluation and validation testbed,” in Proc. IEEE Int. Conf. Ultra Modern Telecommun. ICUMT, 2009, pp. 1–5.
[17] S. Tompros, N. Mouratidis, M. Draaijer, A. Foglar, and H. Hrasnica, “Enabling applicability of energy saving applications on the appliances of the home environment,” IEEE Network Mag. (Special Issue on Digital Home Services), vol. 23, no. 6, pp. 8–16, Nov.-Dec. 2009. [18] P. Davidsson and M. Boman, “Saving energy and providing value added services in intelligent buildings: A MAS approach,” in Proc. 2nd Int. Symp. Agent Syst. Appl. 4th Int. Symp. Mobile Agents, 2000, pp. 79–143. [19] M. S. Ryoo and J. K. Aggarwal, “Recognition of high-level group activities based on activities of individual members,” in Proc. IEEE Workshop on Motion and Video Computing (WMVC’08), 2008, pp. 1–8. [20] B. Logan, J. Healey, M. Philipose, E. Tapia, and S. Intille, “A longterm evaluation of sensing modalities for activity recognition,” Lecture Notes in Computer Science, vol. 4717, pp. 483–495, 2007. [21] D. Youtian, C. Feng, X. Wenli, and L. Yongbin, “Recognizing interaction activities using dynamic Bayesian network,” in Proc. 18th Int. Conf. Pattern Recognit., Hong Kong, 2006, pp. 618–621. [22] T. Zhang, J. Wang, L. Xu, and P. Liu, “Fall detection by wearable sensor and one-class SVM algorithm,” Intell. Comput. Signal Process. Pattern Recognit., pp. 858–863, 2006. [23] K.-C. Hsu, Y.-T. Chiang, G.-Y. Lin, C.-H. Lu, Y.-J. Hsu, and L.-C. Fu, “Strategies on training conditional random fields for multiple resident activity recognition in a Smart Home,” in Proc. 23rd Int. Conf. Ind., Eng. Other Appl. Appl. Intell. Syst. (IEA-AIE’10), Córdoba, Spain, 2010, pp. 417–426. [24] N. Vaswani, A. K. Roy-Chowdhury, and R. Chellappa, “Shape activity: A continuous-state HMM for moving/deforming shapes with application to abnormal activity detection,” IEEE Trans. Image Process., vol. 14, p. 1603, 2005. [25] E. Kim, S. Helal, and D. Cook, “Human activity recognition and pattern discovery,” IEEE Pervasive Comput., vol. 9, pp. 48–53, Jan.–Mar. 2010. [26] D. H. Wilson and C. G. Atkeson, “Simultaneous Tracking and Activity Recognition (STAR) using many anonymous, binary sensors,” in Proc. 3rd Int. Conf. Pervasive Comput., 2005, pp. 62–79. [27] D. J. Cook and S. K. Das, “How smart are our environments? An updated look at the state of the art,” Pervasive and Mobile Comput., vol. 3, pp. 53–73, 2007. [28] L. McCowan, D. Gatica-Perez, S. Bengio, G. Lathoud, M. Barnard, and D. Zhang, “Automatic analysis of multimodal group actions in meetings,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, pp. 305–317, 2005. [29] M. Brand, N. Oliver, and A. Pentland, “Coupled hidden Markov models for complex action recognition,” in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit., San Juan, Puerto Rico, USA, 1997, pp. 994–999. [30] T. Gu, Z. Wu, L. Wang, X. Tao, and J. Lu, “Mining emerging patterns for recognizing activities of multiple users in pervasive computing,” presented at the Int. ICST Conf. Mobile and Ubiquitous Syst.: Comput., Netw. Services, Toronto, ON, Canada, 2009, unpublished. [31] C.-L. Wu, M.-Y. Weng, C.-H. Lu, and L.-C. Fu, “Hierarchical generalized context inference for context-aware Smart Homes,” in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS ’12), Vilamoura, Algarve, Portugal, 2012, pp. 5227–5232. [32] C.-H. Lu and L.-C. Fu, “Robust location-aware activity recognition using wireless sensor network in an attentive home,” IEEE Trans. Autom. Sci. Eng. (IEEE T-ASE), vol. 6, no. 4, pp. 598–609, Oct. 2009. [33] D. Cohn, Z. Ghahramani, and M. Jordan, “Active learning with statistical models,” J. Artif. Intell. Res., vol. 4, pp. 129–145, 1996. [34] M.-Y. Weng, C.-L. Wu, C.-H. Lu, H.-W. Yeh, and L.-C. Fu, “Context-aware home energy saving based on energy-prone context,” in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS’12), Vilamoura, Algarve, Portugal, 2012, pp. 5233–5238. [35] L. Atallah and G.-Z. Yang, “Review: The use of pervasive sensing for behavior profiling—A survey,” Pervasive and Mobile Computing, vol. 5, pp. 447–464, 2009. [36] Y. Liang, J. Cao, L. Zhang, R. Wang, and Q. Pan, “A biologically inspired sensor wakeup control method for wireless sensor networks,” IEEE Trans. Syst., Man, Cybern., Part C: Appl. Rev., vol. 40, no. 5, pp. 525–538, Sep. 2010. [37] R. V. Kulkarni and G. K. Venayagamoorthy, “Particle swarm optimization in wireless-sensor networks: A brief survey,” IEEE Trans. Syst., Man, Cybern., Part C: Appl. Rev, vol. 41, no. 2, pp. 262–267, Mar. 2011. [38] L. L. Fernandes, E. S. Lee, and G. Ward, “Lighting energy savings potential of split-pane electrochromic windows controlled for daylighting with visual comfort,” Energy and Buildings, vol. 61, pp. 8–20, 2013.
LU et al.: ENERGY-RESPONSIVE AGGREGATE CONTEXT FOR ENERGY SAVING IN A MULTI-RESIDENT ENVIRONMENT
Ching-Hu Lu (M’06) received the B.S. and M.S. degrees in the electrical engineering from the National Taiwan University of Science and Technology, Taipei, in 1993 and 1995, respectively, and the Ph.D. degree in computer science and information engineering from National Taiwan University, Taipei, in 2010. Since 2011, he has been a faculty member and Assistant Professor with the Department of Information Communication, Yuan Ze University, Taiwan. His research interests include human computer interaction, intelligent spaces, machine to machine, context-aware technologies, and topics related to them. In 2010, he began to participate in a M2M-based context-aware home energy saving project sponsored by the National Science Council and the INTEL-NTU Connected Context Computing Center. The objective of the project is to minimize the power consumption at home but not to compromise the user comfort using machine learning built upon an M2M infrastructure.
Chao-Lin Wu (M’03) received the B.S. degree in industrial technology education from National Taiwan Normal University, Taipei, in 1996, and the Ph.D. degree in computer science and information engineering from National Taiwan University, Taipei, in 2009. His research interests include artificial intelligence, intelligent agents, smart homes, intelligent spaces, context-aware systems, human-computer interaction based on ubiquitous computing, human-centric computing, and human computation.
Tsung-Han Yang received the B.S. and M.S. degree in computer science from National Taiwan Ocean University, Keelung, and National Taiwan Normal University, Taipei, in 2007 and 2009, respectively. Currently, he is working towards the Ph.D. degree at National Taiwan University. His research interests include smart environments, wireless sensor network, and multicamera surveillance system.
Hui-Wen Yeh received the B.S. and M.S. degree in computer science from National Cheng-Chi University, Taipei, Taiwan, and National Taiwan University, Taipei, in 2008 and 2011, respectively. She is now an Assistant Researcher with Chunghwa Telecom Co., Ltd. Her research interests include smart environments, affective computing, and multimedia.
729
Mao-Yung Weng received the B.S. degree in computer science from National Chiao-Tung University, Hsinchu, Taiwan, in 2010. He is currently working towards the M.S. degree at the Department of computer Science and Information Engineering, National Taiwan University, Taipei. His research interests include smart environments and activity recognition.
Li-Chen Fu (F’04) received the B.S. degree from National Taiwan University, Taipei, in 1981, and the M.S. and Ph.D. degrees from the University of California, Berkeley, Berkeley, CA, USA, in 1985 and 1987, respectively. Since 1987, he has been a member of the faculty, and is currently a Full Professor with the Department of Electrical Engineering and the Department of Computer Science and Information Engineering, National Taiwan University. He was awarded Lifetime Distinguished Professorship from his university in 2007. His research interests include precision motion control, robotics, smart home, visual detection and tracking, intelligent vehicle, evolutionary optimization, virtual reality, and nonlinear control. Dr. Fu has received numerous academic recognitions, such as the Distinguished Research Awards from the National Science Council, Taiwan and the Irving T. Ho Chair Professorship. He currently serves as Editor-in-Chief of the Asian Journal of Control and had been invited to serve as Distinguished Lecturer of the IEEE Robotics and Automation Society during 2004–2005, 2007, and Distinguished Lecturer of the IEEE Control Systems Society during 2013~2015.
Tsung-Yuan Charlie Tai received the B.S. degree from National Taiwan University, Taipei, and the M.S. and Ph.D. degrees in computer science from the University of California, Los Angeles, Los Angeles, CA, USA. He is Principal Engineer and Manager of the Energy Efficient Systems and Communications Laboratory at Intel Corporation. He joined Intel Corporation in 1992, and has held a number of research, architecture development and management position in driving strategic research in energy-efficient communications and mobile platform architecture. He holds 11 U.S. patents. Dr. Tai has received two Intel Achievement Awards. He has represented the Intel Corporation in standard organizations such as IEEE, IETF, USB Forum, and served on the UPnP Forum Steering Committee and Board of Director for the UPnP Implementers Corporation.
