The wrong idea of grouping nail clipper with a pencil is always possible. Figure 3. Figure 3: Swiss Army Knife from Victorinox (http://www.victorinox.com) ...
Sensible Appliance: applying context-awareness to appliance design SungWoo Kim1, SangHyun Park2, JungBong Lee3, YoungKyu Jin4, Hyun-mi Park5, Amy Chung6, SeungEok Choi7, WooSik Choi8 6th Floor, Apkujong Building, 599-4, Shinsa-dong, Kangnam-gu, Seoul, Republic of Korea, 135-893 {caerang.kim1, sh0121.park 2, jbong_lee3, jinyk4}@samsung.com {
hyun_mi.park5 , amychung6, sechoi7, woosik.choi8}@samsung.com
Fax: +82-2-3416-0400 Tel: +82-2-3416-0426
Abstract. Sensible appliance is an appliance which understands user’s situation and takes appropriate actions to provide a high-quality user experience. A sensible appliance should be able to collect and infer relevant contexts to understand the user’s situation. It is important for designers to understand contextawareness and its impact on design. In this paper we introduce what context and context-awareness are from our survey on existing work in the area. Then we present our categorization of existing appliances based on their contextualness. Finally, as a design case study of context-aware appliances, we describe our work on the Gate Reminder - a ubiquitous and context-aware reminder appliance built into a front door to remind users of things they should take with them when leaving home, and messages they should be aware of before going out.
Keywords: context-awareness, context-aware computing, design methodology
1. Introduction: Sensible Appliance As ubiquitous computing[20] arrives, everyday appliances are becoming important, partly based on the influence of embedded computing and home networking. The scope of these new appliances – often referred as ‘smart appliances’ or ‘information appliances’ - is constantly expanding - from electronic appliances to furniture and household utensils. This phenomenon has brought conventional appliances enormous new possibilities for providing high-quality user experiences. As one of the possible high-quality user experiences, our research group is focusing on the “sensible” aspect of appliances”. By “sensible” we mean “reasonable, tactful, witty, smart, wise, and able to judge situation well.” A sensible person is normally quick at grasping the situation and takes an appropriate action to resolve given issue. Like a sensible person, a sensible appliance will grasp the user’s situation and take an appropriate action (including not taking any action) to provide a high-quality experience. We are currently working on a project named SeASeA (Sense-Added Sensible Appliance). In the project we are exploring various ideas on sensible home appliances by “adding sense” to them. How do we add sense to appliances and make them sensible? We as humans have five senses. Through these senses we collect information on the given situation. These pieces of information collected by each sensor can be referred as contexts; who I am talking with, what time it is, noise-level of the surroundings, my preferences, etc. Based on the collected contexts, we are able to understand the situation correctly. Once we understand what is going on, we decide what to do and take an action.
Similarly, making an appliance sensible is to equip appliances with sensors that collect pieces of information, or contexts. Once contexts are collected, an inference is made to understand the given situation. In summary, a sensible appliance is an appliance built upon context-awareness to collect relevant contexts to its use, infer the user’s situation from collected contexts, decide what to do and take an action to provide a high quality service to users.
2. Context and Context-awareness Basic understanding on context and context-awareness are required to apply contextawareness to appliance design. A number of papers from ubiquitous and context-aware computing field have contributed in defining context and context-awareness.
2.1 Context The term “context-aware” was introduced by Schilit and Theimer[17]. They defined context through giving a number of examples of contexts - location, identities of nearby people and objects, and changes to those objects. Similar attempts to define context by example list elements like season, date and time, lighting, temperature, user’s location, objects, etc,. There are also works that provide synonyms of context [3][9] [10] [21]. These works refer to context as environment, surroundings, or situation of either user or application. While these definitions somewhat limit themselves to static entities, definitions that focus on dynamic aspects of context are also being introduced, focusing on the constantly changing aspect of context [8][15][16].
Dey and Gregory addressed the limits of the definitions introduced above in their paper and defined context as ‘any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.’[5][6] This definition points out another important aspect of context; relevance. Obviously the scope of context should be limited to an acceptable level, and a good criterion for sorting out meaningful contexts is measuring their relevance to the application and its use. To summarize, context has been defined as an element in the user and application’s environment including time and date, location, temperature, lighting etc, that may constantly changes. It is the designer’s responsibility to select contexts that have relevance to the task at hand. One way to establish basic list of relevant context is to use the ‘five W’s’:
‘who’,
‘what’, ‘when’, ‘where’ and ‘why.’[6]. Here, ‘who’ will be identification of the user, ‘what’ will be the given task, ‘when’ will be time information, ‘where’ will be location information and lastly ‘why’ will be the user’s intention.
2.2 Context-Awareness The simplest definition of context-awareness is ‘acquiring and applying context.’ Applying context includes ‘adapting to context’ [16][17][21] and ‘using context’. [10][15] A mobile phone that changes its screen’s brightness as the user moves from one location to another is adapting itself to given context. A computer system that checks on available printers and lists them to the user is using context.
Dey and Gregory defined context-aware application as ‘a system is context-aware if it uses context to provide relevant information and/or services to the user, where relevancy depends on the user’s task.’[5][6] To summarize, context-awareness is to acquire context and utilize it to provide relevant information and service to users.
3. Context-awareness and Design Applying context-awareness into design is not a new idea. In fact, many successful commercial products have already made use of context-awareness to provide a highquality service to their users. Any product with minimal common sense built into its design is context-aware product because designers consider basics contexts like ‘who the main target user is’, ‘when and where the user will use the product’ and so on. During our background research on SeASeA, we selected a number of products that we thought are designed with more context-awareness than others and classified them into three categories. The three categories are: non-computational contextual design, noncomputational context-aware design and computational context-aware design.
3.1 Non-computational Contextual Design Products that are designed to conform to context during design fall into this category. These products do not acquire contexts in actual use, but the designer acquires predicted contexts and designs them to support their possible use scenarios based on these contexts. The product is not in itself context-aware but context-awarely designed by the designer. Any product that is designed with minimal common sense is a contextually designed product in a broadest sense. Nevertheless, we can still evaluate products based on levels
of contextual-ness: how much they are designed in the light of contexts they will encounter. A pencil with an attached eraser is a good example of a contextually designed product. The main use of pencil is to write, but there are times the writer needs to erase what she just wrote. Writing and erasing occurs often within the same use context and thus it makes more sense to attach an eraser to the end of the pencil than, say, a nail clipper. Although someone could need a nail clipper when writing, it is not a usually the case. Similarly, a hammer with a nail-pull (figure 1) or a stapler with a staple-pull (figure 2) is designed in relation to its use context.
Figure 1 Figure 1: a hammer with a nail-pull
Figure 2 Figure 2: a stapler with a staple pull
The examples above imply that making a product multifunctional is one approach to making a contextually designed product. Ted et al.[12] state that integration by combining many functions into one design is one technique to make context-aware product. A good example is the Swiss Army Knife (figure 3). While this is true, it is the designer’s responsibility to correctly figure out the correct use context of the product and decide what functions to combine. The wrong idea of grouping nail clipper with a pencil is always possible.
Figure 3 Figure 3: Swiss Army Knife from Victorinox (http://www.victorinox.com)
The correct use context is eventually decided by the user; no matter how much effort the designer puts to predict the product’s use scenario, it is the user who decides in what contexts it is used in real life. A pencil with an attached eraser, although generally can be considered to be contextually well designed, is not contextually designed for the user who needs detachable pencil sharpener attached to the end of a pencil for frequent pencil-sharpening. Designing a context-ready product, in this sense, is a tough task that requires careful design decisions based on extensive user research and analysis of possible use scenarios.
3.2 Non-computational Context-aware Design Products that belong to this category acquire contexts and react to them at their in realtime. Dynamic acquirement and application of context (context-awareness) is required. A kettle that signals the user with whistle-like sound when the water is boiled is a typical example of this category. It acquires the context (the water is boiling) in realtime. Such behavior matches our definition of context-awareness. The Oral B Indicator toothbrush is another good example (figure 4). It calculates how long it has been used and indicates the time to change to a new one.
Figure 4 Figure 4: Oral-B Indicator from Braun (http://www.braun.com/)
Another good example is the popular Hite Beer sold in Korea (figure 5). The label on the bottle of the Hite Beer has a mark that looks like a traffic light which turns to green from red when it is the right temperature for the best flavour, indicating the best time to drink it. It uses a special ink of which changes color depending on the temperature.
Figure 5 Figure 5: the temperature indicator mark in the Hite Beer
Perhaps the old Asian silverware is the oldest context-aware product in Asia. Spoons and chopsticks that royal families used in eastern Asian kingdoms were made in silver because silver reacts chemically to poisons like arsenic, which were frequently used by assassins. Work on OOBE (Out-Of-Box-Experience) by Ted et al. shows how context-awareness can be applied to even simple wrapping and packaging of a product [12][19]. He suggests that the setup tutorial in a manual or as a form of multimedia software that can be executed only after the computer is set up is highly context-inappropriate, and that setup instructions should be presented in order in which they occur and in close physical proximity to the product they describe. In OOBE the wrapping paper puts the instructions on strips of paper that are inserted in the machine during packaging and presented to the user only when they are useful. As examples above show, non-computational products make use of unique characteristics of materials or the natural phenomena - like steam, physical strength, abrasion and chemical reaction to acquire contexts. It is the designer’s role to make use of these properties to empower non-computational products to acquire and apply contexts in run-time.
3.3 Computational Context-aware Design Computational context-aware products are products equipped with computing power to acquire contexts and provide information or services to users. Many futuristic ideas in the area of ‘smart appliances’ fall into this category.
One of the most popular context-aware applications we meet in our everyday life is the context menu from GUI. In Windows, for example, when the user clicks on the right mouse button the system displays a list of menu relevant to the object in focus (figure 6).
Figure 6 Figure 6: context menu in Microsoft Windows
The automatic urinal flushing in the bathroom and the street lamp which gets turned on automatically when it is dark are other popular context-aware products we see everyday. As these examples indicate, automation plays important role in context-awareness. However, there’s a clear distinction between simple automation and context-awareness. In simple automation there’s no intelligence in inferring the situation. A street lamp set to turn on at 7p.m every evening is less effective in providing optimal service on a day when it is still bright at 7 p.m. or when it already became dark before 7 p.m.
3.4 Freeing Designers from Resource Limits for Context-aware Design The goal of designing a sensible appliance is to infuse sense into everyday objects to enhance user experiences. Designers can make non-computational products context-aware through using unique properties of materials or natural phenomena. However, the scope of avaiable resources for this approach is extremly limited and restricted by other design factors such as aesthetics or price. Adding computing power to handle contexts to everyday objects gives far more freedom to the designer, widening the scope for imagination and creativity. As
ubiquitous computing[20] becomes the new computing paradigm, embedded computing and sensor technology will advance further, providing possibilties to empower many oridnary products with context-awareness. One of our brainstorming-tasks conducted during the ideation phase in SeASeA was to list the everyday non-computational objects around us and then enumerate the contexts that we want them to be aware of. Then we assessed each item to check on possibility of implementation with current or near-future embedded computing/sensor technologies and evaluate which ideas are reasonable.
3.5 Context-awareness & Interaction Design As an HCI research group, we particularly focued on the interaction design aspect of SeASeA. From the perspective of HCI, the purpose of using context-awareness is to reduce the user’s effort in theb use of a product. The contextual menu in the Windows GUI clearly shows how many interaction steps can be eliminated. Even in human-tohuman communication, context-awareness reduces explicit communication. Indeed, a sensible person quickly captures the other’s thoughts and needs and proactively takes appropriate action before that person explicitly expresses them. Explicit communication requires concentration on sending andd receiving messages and an effort to understand the delievered content. Most conventional desktop computers demand user’s full attention during the interaction and the user needs to understand what the machine is trying communicate. Through context-awareness new interaction styles and techniques that require less concentration and effort will emerge. Researches on transparent interaction [1] or zero input interaction techniques are the signs of a new interaction paradigm that minimizes explicit interaction between the user and the
computer. In fact, it is well known that the the next core technology requirement in HCI is to move away user’s full attention from the user interface and make user interaction implicit [2][18]. In a ubiquitous computing environment where many computers are embedded in the user’s surroundings, it may become chaotic when each appliance competes for a user’s attention. Thus, techniques to balance between release and capture of the user’s attention are necessary, and ideally the user interface should require minimum attention from the user. Such an interaction style well conforms to Mark Weiser’s vision of calm technology [20].
4. Design Case: The Gate Reminder In this section we introduce a design case study of the Gate Reminder, a context-aware appliance we have developed from our Smart Home project conducted in 2002 [14]. Although the Gate Reminder is not an outcome of SeASeA, it clearly shows our goal to design sensible appliances.
4.1 Effective Reminding From our user study during the Smart Home project we noticed that reminding is one of the most common issues users deal with every day. Stories such as “often I forget to take my house key or cellular phone when I leave home in the morning” and “I had to buy some bread on my way back home after work. I passed by the bakery twice but I didn’t remember it until I walked into the kitchen” are common reminding problems we all encounter in our daily life. Don Norman decribes two types of aspects of reminders: the signal and the message [13]. “Tying a string around your finger” provides only the signal. It incicates you that
there’s something to remember but does not tell you what it is that you must remember. A memo written in the notebook is only a message. It has content but it has no way to indicate it to you. Therefore, a good reminder should have both signal and message. A scheduler in a PDA that notifies you through beeps or vibration with the reminding message displayed on its screen is a typical example of signal and message reminder. But is a PDA as effective as we expect? To provide a more effective reminding service the reminder needs to serve the users within user’s correct contexts [4][7][11]. That is, the reminder should choose the best moment to act. For the user in the latter story above the best moment to remind her is when she is just passing by the bakery on her way back from work and not when she is having a weekly meeting with her boss, or when she is driving to a client’s office for a sales meeting at 3pm and happens to be passing by the bakery. Thus an effective reminder should consider context in addition to signal and message. Using ubiquitous and context-aware reminding is one approach to build effective reminders: reminders that are pervaded into our surroundings and serve the users within correct contexts.
4.2 The Gate Reminder: outline. The Gate Reminder is a reminder located at the front door to remind users of (1) objects like mobile phones and wallets that users should take with them when leaving home (2) messages like “Don’t forget to take out the laundry” or “Return the Titanic DVD!” that users need to be informed before levaing home (3) today’s weather and recommends the user to take umbrella when it is raining or going to rain soon.
Figure 7
Figure 7: the Gate Reminder
Figure 7 shows its three main components: (1) a fixed message input system in the home with which users enter the reminder message (2) a mobile message input system, an application on a mobile phone that allows the users to enter messages when they are away from home (3) the reminder server that does context acquisition, inference, and reminding at the front door. The display screen connected to the reminder server is in the front door and displays reminding messages. We have built a working prototype and a front door mockup called virtual front door (figure 8) to simulate authentic environment for usability and user experience evalaution.
Figure 8 Figure 8: the Virtual Front Door
Figure 9 shows a use scenario of the Gate Reminder. The user enters a reminder message with a particular date. For those objects that user wants the Gate Reminder to check, the user attaches RFID tag on each of those objects, and registers them on the system. When the user approaches the fron door to go outside, the Gate Reminder detects the user, identifies who that users is, and brings up all the messages for that day. At the same time it checks if the user is missing any registered objects and displays icons of undetected objects on the screen to alarm the user. It also displays today’s weather as basic information and recommends the user to take umbrella by highlighting the umbrella icon on the screen if it is raining or going to rain soon.
Figure 9 Figure 9: Use Scenario of the Gate Reminder
4.3 Context-awareness in the Gate Reminder A number of contexts are handled in the Gate Reminder. Being built into the front door, in itself, is a form of contextual awareness of the right time and right place. Various user behaviors are possible in the front door area. Is someone in the front door? Is that person’s intention to go outside? Is the user coming in from outside? Is the user retrieved back to the living room? What direction is that person heading? The Gate Reminder needs to deal with these possible scenarios. To do so, it needs to acquire basic contexts such as if a person (and not a pet) is at the front door, which way the user is heading, if the door is opened or closed and so on. Because the Gate Remidner is a family and home appliance, it also needs to identify the family member in the front door and bring up the messages that relevant to that person only. We use passive-type RFID system, face and speaker recognition technologies, motion detector and door sensors to acquire these contexts. RFID plays important role in both detecting missing objects and identifying the person at the front door. Through empirical studies, we found that most of the objects users want the Gate Reminder to check for are personal belongings such as cellular phones, wallets, house keys, office keys, etc. Thus RFID ID tags attached to these personal belongings act as ID tags attached to the users themselves. Of course, there are exceptions like in a scenario where the husband mistakenly takes his wife’s house key. We use context fusion [2] to achieve the best recognition and inference rate. In user identification, for instance, the reminder server collects opinions from speaker, face and RFID identifiers, and makes an inference as to who the peson at the front door is.
All these behind-the-scene processes should happen with minimum effor from the user. Speaker and face identification, RFID technology and motion detection are used to provide natural and transparent interaction[18] to prevent the user being interrupted at the front door longer than necessary. For face identification, for example, after extensive user evaluation we decided to add a mirror just above the web-cam to lead the user to more naturally show her face to the web-cam.
4.4. Gate Reminder: Brief Summary Many existing reminders are not as effecitve as we expect because they are lack an ability to handle contexts to remind the user at the right moment. In order to judge the right moment for optimal remiding, a reminder should be able to acquire contexts and infer the user’s situation. One approach is to make effective reminders is to have reminders pervaded in the user’s surroundings and have each of them function specifically in a given context. The Gate Reminder at the front door has demonstrated that ubiquitous and context-aware reminders are one of the promising directions towards a more effective reminding mechanism.
5. Summary Context-awareness opens possibility for the next generation of appliance that understands the user’s context and provides appropriate services to enhance user experience and maximize user satisfaction. Embedded computing and sensor technologies will widen the availability of resources to infuse sense into everyday products – making them “sensible”. Also, context-awareness opens up possibilities for
new types of interaction techniques and styles that require less attention from users, supporting Mark Weiser’s vision of calm technology. SeASeA is an ongoing project. We will focus on devising ideas on new appliances and interaction styles based on the findings we report in this paper.
6. References 1. Abowd, G. D. Software Engineering Issues for Ubiquitous Computing. In proceedings of ICSE'99, (May 1999) 2. Abowd, G. D., Mynatt, E. D., Rodden, T. The Human Experience. IEEE Pervasive Computing,1(1), (JanuaryMarch, 2002). 48–57 3. Brown, P.J. The Stick-e Document: a Framework for Creating Context-Aware Applications. Electronic Publishing ’96 (1996) 259-272 4. DeVaul, R., Pentland, A. The memory glasses: towards a wearable, context aware, situation-appropriate reminder system. In Proceedings of Workshop on ‘Situated Interaction in Ubiquitous Computing’ at CHI 2000 (CHI ’00) (The Hague, The Netherlands, April 1-6, 2000) 5. Dey, A. K. Understanding and using context. Personal and Ubiquitous Computing 5(1) (2001). 4-7. 6. Dey, A. D., Abowd, G. D. Towards better understanding of context and context-awareness. In Proceedings of the CHI 2000 Workshop on The What, Who, Where, When and How of Context-Awareness (CHI ’00) (The Hague, The Netherlands, April 1-6, 2000) 7. Dey, A. K., Abowd, G. D. CybreMinder: A Context-Aware System for Supporting Reminders. HUC 2000. 172186 8. Dey, A.K., Abowd, G.D., Wood, A. CyberDesk: A Framework for Providing Self-Integrating Context-Aware Services. Knowledge-Based Systems, 11 (1999) 3-13 9. Franklin, D., Flaschbart, J. All Gadget and No Representation Makes Jack a Dull Environment. AAAI 1998 Spring Symposium on Intelligent Environments, Technical Report SS-98-02 (1998) 155-160
10. Hull, R., Neaves, P., Bedford-Roberts, J. Towards Situated Computing. 1st International Symposium on Wearable Computers (1997) 146-153 11. Kargl, F., Dong, B., Illmann, T., Weber, M. Smart Reminder – Personal Assistance in a Mobile Computing Environment. (Pervasive 2002) (Zurich, Switzerland, August 26-28, 2002) 12. Lieberman, H., Selker, T. Out of context: Computer systems that adapt to, and learn from, context. IBM Systems Journal 39(3&4.) 617- (2000) 13. Norman, D. A. the Design of Everyday Things. Doubleday Publishing Group, New York, NY, 1988 14. Park, SH., Won, SH., Lee, JB., Kim, SW. Smart home – Digitally Engineered Domestic Life. In the Proceeding of First Appliance Design (1AD) (HP Labs, Bristol, UK, May 6-8, 2003). 15. Pascoe, J. Adding Generic Contextual Capabilities to Wearable Computers. 2nd International Symposium on Wearable Computers (1998) 92-99 16. Schilit, B., Adams, N., Want, R. Context-Aware Computing Applications. First International Workshop on Mobile Computing Systems and Applications (1994) 85-90 17. Schilit, B., Theimer, M. Disseminating Active Map Information to Mobile Hosts. IEEE Network, 8(5) (1994) 2232 18. Schmidt, A., Gellersen H-W., Merz, C. Enabling Implicit Human Computer Interaction: A Wearable RFID-Tag Reader. Fourth International Symposium on Wearable Computers (ISWC'00)(Atlanta, GA, October 18-21, 2000). 193-194 19. Selker, T., Burleson, W. Context-aware design and interaction in computer systems. IBM Systems Journal 39(3&4). 880- (2000) 20. Weiser, M.: The Computer for the 21st Century. Scientific American, Vol. 265, no. 3 (Sept. 1991), 66-75. 21. Ward, A., Jones, A., Hopper, A. A New Location Technique for the Active Office. IEEE Personal Communications 4(5) (1997) 42-47
