Acceptability of Nomadic User Interfaces: An ...

Acceptability of Nomadic User Interfaces: An investigation upon user perceptions

Evangelos Karapanos

Project report submitted in part fulfilment of the requirements for the degree of Master of Science (Human-Computer Interaction with Ergonomics) in the faculty of Life Sciences, University College London, 2005.

NOTE BY THE UNIVERSITY This project report is submitted as an examination paper. No responsibility can be held by London University for the accuracy or completeness of the material therein.

Acknowledgements To both Prof. Ann Blandford from UCL and Walter Dees from Philips Research for their excellent supervision and advice during this project. To my parents Thomas and Kostoula and my brother Kostas, for their enthusiastic support during my studies.

2

Abstract In-home nomadic user interfaces are expected to enhance the quality of domestic life. They will increase mobility as users will be able to move freely within the home, maintaining the technology-mediated activity regardless of devices’ user interface attributes, and enhance the interaction in terms of efficiency, experience, and sociability, as users will be able to move applications to other, than the predefined devices. This approach may take advantage of distributed user interface (UI) elements rather than restricting the user to the range of UI elements offered by each device. However, nomadic user interfaces are confronted with the complexity of inhome social interactions. Recent research attempts have illustrated the uniqueness of the home setting. Homes are not designed to accommodate technology, while in-home interactions are highly distributed in time and space. Families are not structured organisations: users range from babies to old age pensioners, while there are no clear rights in domestic interactions and the notion of training does not exist. Furthermore, domestic technologies change and are changed by the social relations that they mediate. Thus, the perceived usefulness, ease of use, and implications of nomadic user interfaces in the home life are expected to be crucial to the concept’s acceptability within this highly technology-defended setting. A study was conducted to investigate the user perceptions of in-home nomadic user interfaces. Eight pairs of users were recruited for two-hour sessions that were conducted at Philips HomeLab, a laboratory that feels and looks like a real home. Emphasis was paid towards couples, as it is believed that such sessions are accompanied by richer reflections. The study showed that user perceptions may significantly restrict the added value of nomadic user interfaces for given activities, in given regions, or with given devices. In addition, nomadic user interfaces bring a radically different interaction paradigm. Thus, we need to consider the conceptual model that is communicated to the user and take into account the nature of the given activity, as well as the interaction/social interaction affordances of domestic spaces and devices. Nomadic user interfaces will enhance the quality of domestic life only if their development is driven by human needs. Further attempts need to introduce the concept to real life and examine its acceptance through long-term studies.

3

Table of Contents 1. Introduction 1.1 BACKGROUND QUESTION...................................................................................6 1.2 AIM OF THE STUDY ...............................................................................................7 1.3 IMPORTANCE OF THE STUDY ............................................................................7 1.4 OUTLINE OF THE REPORT ..................................................................................8

2. Background 2.1 HISTORY AND FUTURE OF USER INTERFACES ............................................9 2.2 DESIGNING USER INTERFACES FOR THE HOME.......................................11 2.2.1 WHAT IS HOME? ...................................................................................................12 2.2.2 WHAT MAKES HOME DIFFERENT FROM OTHER SETTINGS? ....................................12 2.2.3 IN-HOME USER INTERFACES .................................................................................13 2.3 NOMADIC USER INTERFACES..........................................................................14 2.3.1 RELEVANT APPROACHES ......................................................................................15 2.3.2 TECHNICAL CHALLENGES OF NOMADIC USER INTERFACES ...................................16 2.3.3 A NEW INTERACTION PARADIGM?.........................................................................18

3. Seeking values 3.1. SENSING THE FUTURE.......................................................................................20 3.2. METHODS FOR SENSING...................................................................................22 3.2.1 OBSERVATION VS. INTERVENTION ........................................................................22 3.2.2 SITUATED VS. ABSTRACT ......................................................................................23 3.2.3 CONCURRENT VS. RETROSPECTIVE .......................................................................23 3.2.4 1-PARTICIPANT VS. 2-PARTICIPANT.......................................................................25 3.3. CONCLUSION ........................................................................................................25

4. The experiment 4.1 INTRODUCTION ....................................................................................................27 4.2 THE SYSTEM ..........................................................................................................28

4

4.3 THE PROCESS ........................................................................................................29

5. Results 5.1. INTRODUCTION ...................................................................................................32 5.2. FINDINGS ...............................................................................................................33 5.2.1 PAIR ONE: THE CONSERVATIVES ...........................................................................33 5.2.2 PAIR TWO: THE EARLY ADOPTERS ........................................................................34 5.2.3 PAIR THREE: THE FINANCE CONSCIOUS CONSUMERS ............................................34 5.2.4 PAIR FOUR: THE YOUNG COUPLE ..........................................................................35 5.2.5 PAIR FIVE: THE PROGRAMMERS ............................................................................35 5.2.6 PAIR SIX: THE COLLEAGUES .................................................................................35

6. Discussion 6.1. INTRODUCTION ...................................................................................................37 6.2. NATURE OF ACTIVITIES ...................................................................................39 6.3. IN-HOME PRIVACY .............................................................................................40 6.4. PERCEIVED DEVICE ROLE, CAPABILITIES AND RESTRICTIONS........40 6.5. PARTIAL MIGRATION........................................................................................41 6.6. REVEALING THE SESSION................................................................................42 6.7. MIGRATION PARADIGM ...................................................................................42

7. Conclusion 7.1 FINDINGS SUMMARY ..........................................................................................44 7.2 FUTURE WORK......................................................................................................45

5

Introduction CHAPTER 1 ________________________________________________________________________ 1.1 BACKGROUND QUESTION...................................................................................6 1.2 AIM OF THE STUDY ...............................................................................................7 1.3 IMPORTANCE OF THE STUDY ............................................................................7 1.4 OUTLINE OF THE REPORT ..................................................................................8 ________________________________________________________________________

1.1 BACKGROUND QUESTION Since the early 90’s that ubiquitous computing was first defined, the role of computing has changed considerably. At the same time we began shifting our focus from user to activity-centred design. As computing pervades into our life, it becomes more activity-oriented. However, understanding how to design tools that are able to fit into human activities is not a trivial task. Industries have already started realising the need for additional knowledge on the use of the product. Figure 1.1 (Brombacher, 2004) illustrates the increasing rate of No-Fault-Found on product returns. No-Fault-Found refers to the situation where a given customer complaint can not be traced back to a non-compliance with product specifications. These problems are called soft reliability problems and are often attributed to lack of knowledge on the use of the product. This knowledge is mostly gained after the initial launch of the product. However, during the conceptualisation phase the problem is even more visible. As technologies pervade into our everyday life, their role becomes unclear, while artificial usability studies are unable to capture the complexity of social interactions. Long term studies in the field are called to fill this gap of knowledge. However, users’ perceptions are being challenged through the introduction of new systems, while ethnographic studies can only

6

capture the current practice. Thus, the need to study the acceptance of new concepts by challenging users’ assumptions becomes critical. ________________________________________________________________________

Figure 1.1 Percentage No Fault Found in field return reports of modern high-volume consumer electronics (Brombacher, 2004)

________________________________________________________________________ 1.2 AIM OF THE STUDY Philips Research defines nomadic user interfaces as “the user interfaces offered by an application in the network that can be adapted and delivered to any stationary, portable, and wearable UI device inside the network when needed”. These are expected to enhance domestic life in two ways: increase mobility as users will be able to move freely within the home maintaining the technology-mediated activity regardless of each device’s user interface attributes, and enhance the interaction in terms of efficiency, experience, and sociability, as users will be able to move applications to other than the predefined devices and take advantage of distributed user interface (UI) elements rather than restrict to the range of UI elements offered by each device. To realise such a vision we need to overcome numerous practical issues such as device independence, smooth integration of modelling in the design process, and the crucial shift from user-centred design to activitycentred design. However, as Venkatesh (1985) noted, “we cannot assume that what technology can do in the household is the same as what the household wants to do with the technology”. Thus, this study aims to understand the activities that this technology is called to mediate, as well as their social attributes that are likely to influence the acceptance of this technology in the home setting. 1.3 IMPORTANCE OF THE STUDY Despite the fact that ubiquitous computing is constantly gaining research interest, the lack of a theoretical framework is more than visible. As the computer pervades the human life to support the distributed human activities, we realise a need for designing in context. History has illustrated

7

that home is not a trivial context for design. But, this setting has only recently been a point of research attention. In addition, ethnographic studies are able to capture only the current practice and not how users’ perceptions change after the introduction of new systems. This study investigated the role as well as the implications of nomadic user interfaces in home life. The study affirmed Venkatesh’ argument that what technology provides is not necessarily what home needs and revealed the complexity of home life. It identified the role of nomadic user interfaces in home life and revealed the perceived distinctions regarding home regions, home activities, and technological devices. Thus, it managed to act as an informative point regarding user perceptions of domestic distributed technological artefacts. 1.4 OUTLINE OF THE REPORT Chapter 2 reviews research that is relevant to the context of this thesis. The history and the current debates regarding user interfaces are presented, paying attention to the shift to distributed user interfaces. The home context is analysed and results from relevant studies regarding domestic technologies are presented. Finally, this chapter introduces and describes the vision of nomadic user interfaces, briefly mentioning the technical challenges while paying attention to novel interaction paradigms for distributed user interfaces. Chapter 3 defines the purpose of the study and describes the rationality behind the experiment’s procedure. It first distinguishes this study from other usability studies and subsequently describes the experiment’s main concerns in 4 dimensions. It pays attention to the need for the experimenter’s intervention, the need for situated observation and introspective interviewing for minimising the cognitive latency period between the occurance and the user’s feedback, and finally the need for engaging the user in rich reflection. Chapter 4 describes the experiment with regard to the setting, the user group, the system, and the experiment’s procedure. Chapter 5 outlines the analysis procedure as well as the initial findings within their context. Thus, each section covers the main findings that were revealed during the corresponding experiment session. The first section describes the findings’ context in detail while the remaining sections describe the additional findings that each session revealed. Chapter 6 discusses the results in a level of abstraction and inference. The perceived added value of nomadic user interfaces is first described as derived from what users said in the study, while distinctions are made across the range of domestic places, devices and activities regarding two main variables. Finally, the user preferences regarding issues of privacy and experience are described, as identified through the comprehension of users’ perceptions. Chapter 7 summarises and concludes this dissertation and gives an outlook of further research.

8

Background CHAPTER 2 ________________________________________________________________________ 1. HISTORY AND FUTURE OF USER INTERFACES ..............................................9 2. DESIGNING USER INTERFACES FOR THE HOME.........................................11 2.1 WHAT IS HOME? ......................................................................................................12 2.2 WHAT MAKES HOME DIFFERENT FROM OTHER SETTINGS? .......................................12 2.2 IN-HOME USER INTERFACES ....................................................................................13 3. NOMADIC USER INTERFACES............................................................................14 3.1 RELEVANT APPROACHES .........................................................................................15 3.2 TECHNICAL CHALLENGES OF NOMADIC USER INTERFACES ......................................16 3.3 A NEW INTERACTION PARADIGM?............................................................................18 ________________________________________________________________________

2.1 HISTORY AND FUTURE OF USER INTERFACES The term “user interface” is one of the most common terms in the field of Human-Computer Interaction (HCI). In fact, Kuutti and Bannon (1993) note that the whole field of HCI is synonymous with the design and use of interfaces to computing systems. However, while efforts trying to define this term are more than visible, there is still no clear, unique definition of the “user interface”. This is due to the interdisciplinary background of the studies related to the design and use of interfaces. As Robinson (1990) notes, “the interface stands as the boundary between what is the work of HCI and what is the work of software engineering”. From a historical perspective, it is clear that the user interface has not significantly evolved through the last two decades. A. van Dam (1997) argues that the history of user interfaces can be characterised using evolutionary biologist Steven Jay Gould’s notion of punctuated equilibrium: long period of stability interrupted by rapid change. Grudin (1990) describes the history of user interface as the story of “computer reaching out”, starting from the “interface as hardware” and ending with the “interface at the work setting”. Initially, the user interface was located at the hardware setting – only engineers could interact with the system. However, the shift from hardware to software development enabled computer engineers to focus on the programming task independently from the hardware. The next phase enabled end-users to interact with the system through specialised

9

input and output devices such as the keyboard, mouse and screen. The 4th phase focused on the conceptual representation of the system in the users’ minds while the 5th reached out even further to the work setting, studying how the system can become embodied in the social and work setting. Brodersen and Kristensen (2004) add a 6th level called “interface as everything”, paying attention to the ongoing trend towards systems that pervade and disappear into the environment. Dourish (2001) presents the stages in the historical development of user interfaces in terms of the different sets of human skills they are designed to exploit, shifting from the electrical to the symbolic, the textual, the graphical and the peripheral attention. There is a belief that the current graphical user interface has exploited all of its potentials, which has led to research efforts for novel user interfaces that exploit more human senses, including tangible, post-WIMP and enactive user interfaces. As Bill Buxton points out, WIMP graphical user interfaces based on the keyboard and the mouse are the perfect interface only for creatures with a single eye, one or more single-jointed fingers, and no other sensory organs. But, what is the role of the user interface? Is it to communicate the system’s functionalities to the user? Is it thus, only a surface of the system? The system’s role is to support us in performing specific tasks. Thus, the way in which we can achieve our goals through the system defines our conceptual model of the system and thus, the system’s interface. Andersen (1991) argued that “it does not make sense to say that a system in isolation has an interface; interfaces “emerge” when the system is used…”. This perspective can be traced back to Heidegger’s rejection of Cartesian dualism, supporting the view that thinking and being are fundamentally intertwined (Dourish, 2001). He made a distinction between ready-to-hand and present-at-hand. The artefact is ready-to-hand when it allows us to perform specific tasks through itself. However, on many occasions breakdowns occur in our activity and the artefact forces us to focus on how to use it; in this situation the artefact is present-at-hand. In Heidegger’s perspective, the artefact exists for us as an entity only because of the way in which it can become present-athand, and becomes equipment only through the way in which it can be readyto-hand (Dourish, 2001). These ideas lead to the current trend characterised by the term “invisible computing”. An artefact becomes invisible when it is ready-to-hand; it allows us to focus on the activity rather than the artefact. Weiser (1991) noted that “the most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.”. He added that “such a disappearance is a fundamental consequence not of technology, but of human psychology. Whenever people learn something sufficiently well, they cease to be aware of it”. Thus, the concept of information appliances (Norman, 1998) is integrated into the notion of invisible computer. Norman (1998) argues that “the primary motivation behind the information appliance is clear: simplicity. Design the tool to fit the task so well that the tool becomes part of the task, feeling like a natural extension of the work, a natural extension of the person”. On the other hand, there is a current trend towards access to information from everywhere. More than one device has the ability to access the same information and thus, each

10

device is capable of supporting multiple activities. Thus, it is clear that there is a tension between integration and separation. The first provides convenience while the second provides simplicity. A broader tension in the field of user interfaces relates to the interaction paradigms that are called to support and provide consistency across an increasing diversity of devices and contexts of interaction. Beaudouin-Lafon (2004) argues for three primary interaction paradigms: computer-as-tool, which extends human capabilities with sophisticated tools, computer-aspartner, which embodies anthropomorphic means of communication in the computer, and computer-as-medium, which uses the computer as a medium by which humans communicate with each other. The two interaction paradigms computer-as-tool and computer-as-partner are represented by two currently dominant approaches named direct manipulation and computer agents. Reeves and Nass (1998) argue that individuals’ interactions with computers, television, and new media are fundamentally social and natural, just like interactions in real life. Therefore, the agent paradigm intends to enhance the communication between the user and the system by adding intelligence in the system, enabling it to understand the undergoing activity, interpret her goals and achieve a human-human interaction with the user. Turk (2000) defines perceptual user interfaces as “highly interactive, multimodal interfaces modelled after natural human-to-human interaction, with the goal of enabling people to interact with technology in a similar fashion to how they interact with each other and with the physical world”. Dey and Abowd (1999) distinguish between implicit and explicit interaction; in implicit interaction the users’ concrete actions are not directed primarily towards a computer, but are interpreted by so called ‘context-aware’ technology in the surroundings. However, as Brodersen and Kristensen (2004) argue, implicit interaction easily becomes a nuisance rather than a desirable feature because it removes control from the users, and Shneiderman (1997) argues against anthropomorphic interfaces, emphasising the importance of direct, comprehensible and predictable interfaces which give users a feeling of accomplishment and responsibility. 2.2 DESIGNING USER INTERFACES FOR THE HOME However, all the previous issues raise questions on how we want to interact in given settings; still, the answer will not come through computer science studies in isolated environments but, rather through long term sociological studies of human behaviour with current and coming technologies. Arthur (1990) points out that technologies typically improve as more people adopt them. Technologies are understood only after they are exposed to wide use. However, despite the large efforts that industries spend on producing fastfeedback procedures during pilot releases, user involvement needs to take place in the product conceptualisation phases, while minimising the artificial parameters. Thus, we need to study people using the technology in realistic, non-laboratory settings for long periods of time and then measure if it is worth bringing something new to these settings (Intille, 2002). As Venkatesh (1985) noted, “we cannot assume that what technology can do in the household is the same as what the household wants to do with the

11

technology”. Despite the significant effort spent on understanding user needs in work settings from an ethnographical standpoint, the home setting has only recently become a focus of research attention (Bannon and Schmidt, 1991; Baecker, 1993). Understanding the role of technologies in the home requires studies in the current patterns of activities being carried out in such settings and a deeper conceptualisation of what people want from their home and what “home” means to them. 2.2.1 What is home? Home is not a place; it is more like a state of mind (Pallasmaa, 1992). Microsoft Research considers five perspectives in the meaning of home life: a place; an emotion or feeling about that place that is more than something merely physical; a set of relationships; activities related to the home wherever members of that home may be; ideals and aspirations for what people want their home life to be. Eggen et al. (2002) conducted user studies to understand what “home” means to people. Participants responded that “home is a feeling; it is a cosy, trusted, and safe place, a place to return to. It is a place for relaxation, a place where you are doing what you want, a place where you have your own stuff – the things you are attached to, a place where you get together with your family and friends”. The study resulted that people are prepared to defend their homes and are very reserved with things involving home; “not every new idea is taken along and carried to home. Home is like a nest that is protected and that protects its inhabitants from the outside world”. Prompted to express their opinion about the “smart home”, people explicitly discussed negative aspects: “make sure I can trust it”. The notion of automation was rejected by the participants. People want to be in control and be the ones that make the decisions (Eggen et al., 2002). But does intelligence implies automation? Aldrich (2003) defines the “smart home” as a residence equipped with computing and information technology which anticipates and responds to the needs of the occupants, working to promote their comfort, convenience, security and entertainment through the management of technology within the home and connections to the world beyond. Who does have the control in such a setting? Does context awareness necessarily need to lead to automation? Do we necessarily need to remove the control from the users in order to provide intelligence? Berg (1994) notes that the smart home is a “gendered socio-technical construction” developed in line with the interests of its male designers. She adds that smart homes are a typical case of “technology push” rather than “consumer pull”, motivated principally by what is technically possible rather than what is desirable. Such approaches do not tend to consider why people use technologies and the relevance the technology has in people’s everyday lives (Norman, 1998). People consider the home of the future “as it is now, but better”. They want technology to move into the background and the focus to shift from function to experiences (Eggen et al. 2002). 2.2.2 What makes home different from other settings? Aldrich (2003) argues that “homes are not workplaces – unlike workplaces, they are not designed to accommodate technology, they are not networked, nor do they have benefit of professional planning, installation, and

12

maintenance of technology and infrastructure. Harper (2003) adds that families are not structured like organisations. Users go from babies to old age pensioners. He reports on a study funded by Orange, which resulted the understanding of the complexity of the domestic setting. There are no clear rights or roles in home settings or as he describes “children may want to override mum’s selection to music”. There is no existence of the notion of training in domestic technologies and artefacts use; people want simpler technologies: “not control panel, but switches for the lights”. In-home activity is not focused around a desk with a networked computer and all artefacts situated nearby; it is rather distributed across the whole space of home. As new technologies are being adopted and adapted in the home, they change and are changed by the social relations that they mediate. As Petersen (2002) argues, expectations become challenged in the meeting with new products and they are formed and modified as we gain experience with using the new technology. Thus, domestic technology should evolve together with its users, just as the building grows with its inhabitants. Finally, the home setting provides an unusual difficulty in the design and evaluation of domestic technologies; no one would ever want researchers hiding in the home, and the use of video in domestic environment has been widely discussed. Introspective approaches (e.g. incident diaries) are not easily applied in long term studies while the retrospective ones (e.g. occasional interviews) are the most feasible but not the most reliable ones. 2.2.3 In-home User Interfaces If we can consider history as a resource of knowledge, we realise that we cannot predetermine the use of the artefact prior to its exposition in wide use. Brown and Perry (2000) demonstrate through an example of a pervasive every-day in-home technology that use can rarely be predicted prior the launch of the product. Engestrom (1990) noted that an artefact implies more possible uses than the original operations that led to its creation; thus we are able through sociological analysis to understand the potential role of the artefact within a given setting, and communicate a repertoire of purposes which as Petersen et al. (2002) argue, concern the development of motives, in Activity Theory terms, of what we do. Recent studies in Interactive TV (iTV) (e.g. Daly-Jones and Carey, 2000) revealed that viewers find Electronic Program Guides (EPGs) difficult to operate, while Maguire and Batters report a study that found that almost one third of video owners almost never programme their machines. The questions that were raised tried to reason about these difficulties. Was it because of bad usability of the interface? Was it because of input restrictions through the remote control? Or was it due to the already existing perception of TV as a passive medium? Taylor and Harper (2003) noted that people choose information sources that require the minimum effort to make transition from viewing to choosing and programme, and added that an EPG will only be a viable solution if it can limit the disruption to people’s sense of what television watching is about. A significant number of research studies has paid attention to the “relaxed” and social nature of TV. Peters (2003) stresses the need to consider the emotional significance of the set itself, or as I would describe it, the perceived role of each device in our “domestic life”. He argues that the television is by

13

its nature a social screen, designed for entertainment and to coincide with the interests of millions of viewers. He adds that the television should be seen as a medium which offers an enriched experience for the viewer and not necessarily as a means of accessing the Internet. Television has been also seen as a focal point for families; it is used as a form of bonding for many families who all sit down to watch together (Norton, 2000). Clawson (2002) argues that due to the TV’s relaxed personality people will never want to interact with the TV – they want to watch it and not have to think. Taylor and Harper (2002) note that rarely does TV content engage or absorb members of the household’s attention in the way that a video or the Internet might. They note that the TV is often turned on to unwind, to start the process of relaxing or as a form of distraction, undertaken alongside other activities. They add that television viewing appears to be curiously “unplanned”. Unplanned in the sense that though they might know what they are about to watch, they do not at any particular point settle down and plan that activity with reference to programme guides. There is nothing that one might call a rational decision-making process. McLuhan (1964) distinguished between “hot” and “cool” media, concerned about the sensory involvement that the medium requires. Thus, TV watching brings a notion of “cool” medium to the TV while iTV requires higher sensory involvement and thus it demands a shift in to the perception of TV as a “hot” medium. Peters (2003) pointed out that we need to consider where each technology is situated within the home as different places involve different activities and thus they contain a different meaning. Du gay et al. (1997) makes a distinction between public and private spaces, while Goffman (1959) introduces the notion of regions which he defines as any place that is bounded to some degree by barriers of perception. However, our perceptions become challenged through the introduction of new services through those media. New services like gaming and interactive access to content are likely to bring a shift in the users’ perceptions of passive or active roles of each device, while technologies are being converged and enhanced (e.g. readability of latest TV displays becomes comparable to this of PC monitors). We need to reconsider the usability issues of the interaction and not focus solely on sociability issues. iTV studies revealed difficulties in changing those perceptions but, is it solely due to the users perceptions of the role of each device or also due to the interaction restrictions of each device? iTV studies showed that consumers are beginning to expect devices to “talk to each other” (Peters, 2003) and use devices in conjunction with others. In addition, perceptions of the mobile phone are already changing; the mobile is taking on a new meaning and has superseded its utility as a medium solely for voice telephony; it is increasingly perceived as a multi-purpose device (Peters, 2003). However, bringing change in the social distribution of in-home activities requires scepticism. Silverstone (1994) supported that the household communication and information technologies are not only objects of use like dishwashers or video recorders, they are media. Thus, they have the capacity to threaten the social stability of coexistence. 2.3 NOMADIC USER INTERFACES

14

In the above paragraph we discussed how users’ perceptions change through the introduction of new technologies, and the need for networking in home environments is gradually increasing. Consumer Electronics companies have recently recognised the need for “seamless integration of portable devices in home entertainment systems” (Philips CE, 2003), that lead to CE devices capable of sharing content within a network. ________________________________________________________________________

UI Devices

Figure 2.1 Nomadic UI vision (Philips Research)

________________________________________________________________________ Philips Research defines a concept called “nomadic user interfaces” as “the user interfaces offered by an application in the network that can be adapted and delivered to any stationary, portable, and wearable UI device inside the network when needed”. Core assumptions in this vision are the remote user interface protocol, the device independence of user interfaces as well as session continuity. In the nomadic user interface vision, devices do not share only content but also their user interface. In a home context, this will allow users to access applications and legacy devices from anywhere, both within the home network and outside (e.g. through cell network or WiFi hotspots). In addition, it is expected to open a new market for UI-less devices (without any local UI) as well as UI-only devices (e.g. iPronto). This is subsequently expected to lead to a change in our perceptions of devices and in-home technologies (e.g. where should the user interface of the home cinema speakers be?). 2.3.1 Relevant Approaches

15

Other companies have also directed commercially towards media connectivity for in-home environments. Products from Hauppauge and Linksys (fig. 2.2) are capable of remotely accessing and controlling applications through the TV. Microsoft has a product called "Media Center". This is a hardware/software platform that is oriented towards having a very silent PC in a consumer-electronics like casing, connected to a TV, that enables users to access, store and share media content such as photos, music, videos, recorded TV shows and applications (including applications on the Internet), through a TV-like user interface. In order to access the content and applications of such Media Center PC from other places in the home, they provide low-cost devices, called media extenders, which can act as a client to the Media Center PC. Although limited to other TV sets in the home, it provides a first step towards having access to your favourite content and applications from anywhere. ________________________________________________________________________

Figure 2.2 Microsoft’s vision (left) and Remote UI products (right)

________________________________________________________________________ However, in the nomadic UI vision, every device – portable, wearable or situated, will be able to share its applications within the network. In addition, user interfaces will be capable of migrating partially to two or more devices. For example, a user might want to interact with her portable device, to navigate within a set of pictures while displaying them selectively on the TV set so as to share them with other family members or friends. However, we realise that while the nomadic UI vision allows for a truly connectivity within a networked home, it lacks in feasibility as it is based on assumptions that still involve significant technical challenges. 2.3.2 Technical challenges of nomadic user interfaces One of the key assumptions of nomadic user interfaces is the ability of the user interface to migrate to a range of devices with diverse UI attributes, while at the same time maintain a sufficient level of usability and consistency across these rather diverse devices. Currently existing devices in the market demonstrate diversity in input (e.g. keyboard, remote control, touch/mouse based, speech based) and output (graphics, speech, ambient – lighting, sound). Within this group of available modalities a wide diversity exists, with

16

the current emphasis directed towards graphics output diversity with regard to the screen size, the resolution and the aspect ratio of the display. The transition, however, from one device to one other with different UI elements will certainly have an impact on the UI design starting from the layout and style, pagination/navigation, the widget choice – so as to fit the interaction style of each device, and the content of the presentation. In order to be able to deal with this wide diversity in terms of interaction modalities and UI features of the devices, UIs will have to be described in a more abstract, device independent way. Shrubsole and Dees (2003) distinguish between four different types of abstract models: • • • •

Application models, which model the internals of an application (data, states, functions) and how these internals are exposed to the outside world (i.e. the API) Task models, which model the tasks a user can perform with an application and which steps he/she has to go through to reach his/her goals Abstract UI models, which model the user interface in terms of abstract interaction models. Presentation models, which model the user interface in concrete terms, e.g. style

Dees (2004) proposes a technique, called “multi-level style-sheets” (fig. 2.3) which involves specifying style attributes on different levels of abstraction for supporting runtime migration of user interfaces. ________________________________________________________________________

Figure 2.3 Multi-level style-sheets (left) and example of a task model in Concur Task Trees (right)

________________________________________________________________________ Bandelloni and Paterno' (2004) introduce the notion of flexible interface migration paying attention to run-time migration and interaction continuity. They note two types of information needed for such a migration: static information referring to the device features, and runtime information that refers to the state of the migrating application. They also make a distinction

17

between three types of migration: total migration where the client interface migrates totally from a device to the other, partial migration where the client interface is divided into two parts, one for user interaction (control part) and one for information presentation (visualisation part), and where the control part remains on one device, while the presentation one migrates to the other device, and mixed migration, where the client interface is split into several parts, concerning both control and presentation and different parts are distributed over two or more devices. Vandervelpen and Coninx (2004) bring notions from the area of distributed systems while the make a classification of interaction elements in: Interaction Resources (IRs) which are atomic I/O channels that are available and that a user can exploit for executing a task; Interaction Devices (IDs) that are computing systems that handle the input or send output to individual IRs that are connected to it; and Interaction Groups (IGs) that are groups of interconnected IDs which are engaged in a particular distributed interaction session of one or more users. They also make a distinction between static UI distribution where it is known which IRs are available in the environment and to which IR every part of a UI is distributed during the design phase, and Dynamic UI distribution where UI components are allocated to IRs at runtime. Coutaz et al. (2003) introduce an ontology for multi-surface interaction and describe how IRs can have different roles: surface or instrument. Thevenin and Coutaz (1999) introduced the notion of plasticity as the capacity of an interactive system to withstand variations of context of use while preserving usability. 2.3.3 A new interaction paradigm? Despite the tremendous advantages that the vision of partial migration will bring to the interaction with in-home technologies, we need to consider usability and sociability issues of having multiple sessions accessible through multiple devices in a multi-user environment such as home. However, the concept of interacting with multiple devices is not new. Robertson et al. (1996) introduces a dual device system employing a PDA working in conjunction with interactive TV. However, the association between the PDA and the TV is predefined and static; the PDA only acts as a commander for the TV. The PARCTAB project (Schilit, 1993) integrated a palm sized computer device in an office network, while Bolt (1980) had introduced a multi-display system that enabled interaction through voice commands and hand pointing for the data management of spatially distributed data sources. Beigl (1999) introduces the point and click interaction for associating interaction resources with devices while Rekimoto (1997) introduces the “Pick-and-Drop” direct manipulation technique for transferring data between collocated networked devices. Both techniques liberate the user from the need to deal with symbolic concepts, and extend her abilities based on direct manipulation. However, despite the research significance of such early concepts in the demand for interaction with multiple devices, most of these approaches dealt with a single-user and single-application-and-context situation. However, multi-device interactions in the home environment bring into focus the social implications in home-life. Such interactions have to deal with privacy issues while the introduction of user recognition systems into the market is still

18

questioned, in an environment where the activities are highly distributed, and undertaken by a wide range of users with no distinct roles and authorities (§2.2). As Preece (2001) notes, ‘social computing’ applications should not be optimised only for usability but especially for sociability. Thus, this study pays significant attention to the complexity of in-home social interactions and the perceived implications of nomadic user interface in people’s lives.

19

Seeking values CHAPTER 3 ________________________________________________________________________ 3.1. SENSING THE FUTURE.......................................................................................20 3.2. METHODS FOR SENSING...................................................................................22 3.2.1 OBSERVATION VS. INTERVENTION ........................................................................22 3.2.2 SITUATED VS. ABSTRACT ......................................................................................23 3.2.3 CONCURRENT VS. RETROSPECTIVE .......................................................................23 3.2.4 1-PARTICIPANT VS. 2-PARTICIPANT.......................................................................25 3.3. CONCLUSION ........................................................................................................25 ________________________________________________________________________

3.1. SENSING THE FUTURE Referring back to Grudin’s five stages of “computer reaching out” (§2.1), software usability started becoming evident at the 4th stage where large scale systems were exposed to use by end users – people that were not involved in the development of the system and thus, would have to form by scratch a mental model of the system. The 5th and 6th phase (introduced by Brodersen and Kristensen, 2004, §2.1), bring into focus the social role of computer in mediating human activities. Usability studies in the 4th phase were mainly driven by a need for increased performance. Such studies represent the “hard” approach – where the goals and the problems were known and predefined, while the last two stages shift the focus to the “soft” approach, according to the Checkland’s (1981) distinction. Software or system’s usability becomes a rather narrow term; usability becomes distributed as the

20

effectiveness, efficiency, and satisfaction do not refer to goals internal within the use of the artefact but to activity goals that the artefact supports. Thus, it becomes crucial to understand the role of the artefact in supporting a given range of activities. However, history has illustrated that people’s needs change significantly through time, or as Petersen et al. (2002) argue, expectations become challenged in the meeting with new products and they are formed and modified as we gain experience with using the new technology. So, the question becomes: how can we sense the future, predict the future needs of people, and design products that do not currently exist in their perceptions? And what form should the user involvement have in the conceptualisation of those future solutions? This chapter reports on the undertaken approach during this explorative study. This approach is inspired by ethnomethodology (Suchman, 1987; Dourish 2001) and action research (Avison et al., 1999), participatory design, and change management theories (Senge et al., 2005; Hammond, 1997; Checkland and Holwell, 1997). Kensing and Munk-Madsen (1993) identify three different domains of discourse in design: users’ present work, technological options and new system. The first refers to ethnographic studies of current practices. Such studies may inform the design process about the current practice, what works and why it works, and may reveal potential areas of interest. What concerns me though is the applicability of data retrieved from such processes in the design of new solutions. History has proven that we tend to focus on what doesn’t work, which does not necessarily lead to positive evolution. Bodil et al. (2004) argue that change can be initiated in two different ways: either by reaction to a situation that we do not like, or by acting towards a desire or an imagined situation. Usability studies have been largely based on the first approach. However, the conceptualisation of new products that will come to meet the evolutionary human needs and perceptions requires not only privative usability studies. As Hammond (1997) argues, “if we focus on what is wrong or what is missing, we tend to see everything through a filter or frame. The filter or frame is our unconscious set of assumptions. We tend not to be aware of our frame, and we fail to notice that we disregard information that doesn’t fit our reality”. Senge et al. (2005) call the process of seeing beyond our assumptions, or “mental models of reality” as they call them, presence: a process that enables us to stop our habitual ways of thinking, “hang our assumptions in front of us”, and begin having less influence on what we see. Instead of searching for flaws, we can start identifying what works, why it works, and try to do more of that, as Hammond (1997) notes. However, drawing direct conclusions from ethnographic studies for the acceptability of future products seems rather superficial. We need to give users concrete and abstract experience of new technological options and new systems (2nd and 3rd domain of discourse, §3.1 – above). We need to challenge their assumptions and their own understanding of reality. This, as Bodil et al. (2004) argue, will inspire them to discover and explore new use situations and look at their own situation in a new way –with new glasses. Argyris and Schon (1991) noted that “causal inferences about the human behaviour are more likely to be valid and enactable when the human beings in question participate in building and testing them”. To do so, we need to change our

21

perceptions of the experimenter’s role during the observation. Solely observation will not bring valuable data for change. Action research (e.g. Avison et al., 1999) encourages researchers to experiment through intervention and to reflect on the effects of their intervention and implication of their theories. The view within action research is that complex social processes can be studied by introducing changes into these processes and observing the effects of these changes. Action researchers, however, have been criticised for being involved in such degree that they can no longer be objective and that the situation is so biased that it can no longer be scientifically studied. However, as Bodil et al. (2004) argue, being an intense listener and reflector on the one hand, and an interpreter, inventor and technological action researcher on the other is as delicate as it is important. 3.2. METHODS FOR SENSING As noted in the previous paragraph, this study is largely based on observations of interactions with future technology, accompanied by interventions of the experimenter in order to get a deeper insight. Baber and Stanton (1996) note the main problems of observation including the problem of causality and the question of validity. A significant number of research efforts have focused on the validity of results drawn from field studies, but not enough attention has been paid to the level of insight that each technique may offer. I realise that in studies such as the one reported here, the level of insight and the ability to reason and evaluate multiple future paths, challenging existing assumptions, is of critical importance. In this section I report on the social research methods and techniques that were considered and evaluated for their effectiveness in this study. Instead of presenting the techniques semantically or chronologically, I will try to present them according to four dimensions that I consider critical and of need for further study. These dimensions are: • • • •

observation vs. intervention, which refers to the level of intervention of the experimenter concurrent vs. retrospective, while there is also a mid-state – the introspective 1-participant vs. multi-participant, and situated vs. abstract, which is concerned about the visualisation of the future.

3.2.1 Observation vs. intervention Baber and Stanton (1996), as mentioned before, paid attention to the problem of causality. Verbal data have appeared to solve this problem to a great extent, but do verbal data have the same purpose in different experiments? Boren and Ramey (2000) illustrate why intervention was not initially considered by Ericsson and Simon (1993) and why it needed to be considered now for usability studies. They argue that usability practitioners must deal with a higher level of uncertainty and complexity than researchers conducting more formal experiments. In experiments such as the problem solving one with the Towers of Hanoi, the experimenters know how the puzzle works and the element under examination is human cognition. In

22

usability studies, on the other hand, the element under examination is the system itself which is most of the times under development, including much more complicated functionalities than the examined task. The aim of such study is mostly to reveal usability flaws that are unknown to the experimenters up to that time. Therefore, in many occasions, the experimenter needs to intervene and the question becomes how to control this intervention. I would further pay attention to the distinction between usability studies and the study reported here. While the aim of most usability studies is to reveal problems (hard approach), studies like the one reported here need to explore the changing perceptions, needs and values of the users, by letting them experience the future. Such studies would not be that interested in the cognitive processes of subject – such cognitive processes would be strongly tied to the implementation that the subjects experience and it is mostly the case that we want to evaluate the concept and extend to other situations, rather than evaluate a given implementation of the concept. Thus, I would argue that in such studies, the experimenter’s intervention is not only permissible, but imperative. 3.2.2 Situated vs. abstract In the previous paragraph I tried to distinguish this study from traditional usability studies. Instead of searching for problems, you are trying to identify values, perceptions, and their changeability. As I called it, it is an “ethnography in the future”. The difference is that you don’t observe how activities are being carried out, but you observe how perceptions develop. However, such process is in danger of producing superficial data. Thus, this dimension is concerned about the level of situatedness of the process of retrieving data. This depends largely on the experience of the future that the users receive. However, on the other hand, situatedness has its own tradeoffs as the more concrete experience the users get close to a prototype, the more tied the results will be to this particular implementation. Such studies tend to bridge the abstract data from ethnographic studies of current practices and the very specific data that can be retrieved through the observation of users interacting with potential future technologies. Thus, the balance between situatedness and abstract thinking seems of critical importance. Two conflicting trade-offs existing in this process: abstract thinking provides superficial data while data from concrete experience are tied to the given implementation. Therefore, two different approaches of visualising the future are distinguished: prototypes of future potential products, and scenarios visualised in story boards, in video mock-ups or in written text. 3.2.3 Concurrent vs. retrospective Verbal protocol analysis is defined as “a rigorous methodology for eliciting verbal reports of thought sequences as a valid source of data of thinking” (Ericsson and Simon, 1993). There are three main techniques in verbal protocols that have received considerable attention: concurrent verbalisations (also referred to as “think-alouds”), retrospective reports, and interviews. Interviews can be conducted during, after, or independent of task performance, and are focused on engaging in a conversation with the participant to determine the nature and scope of knowledge and thinking

23

strategies associated with performing a task or operating in a particular domain (Olson and Biolsi, 1991). Interviews are a fairly straightforward technique for gathering data, requiring little prior training, but they provide relatively superficial information. According to Olson and Biolsi (1991), this method can be useful in studying straightforward, general underlying relationships, but it is likely to be inadequate when one is to capture complex organisations or organisations that contain many exceptions arising from specific cases. In addition, as they noted, there are well-known limitations associated with data gathering methods that require the subject to verbalise thought sequences outside of the context of the task being performed. The cognitive latency period between performing the task and responding to interview questions on the performance is likely to result in steps of the cognitive process not being recalled upon being interviewed. Concurrent verbalisations involve asking the subject to verbalise their thoughts as they work on solving problems in a given domain area (Olson & Biolsi, 1991). The objective of the concurrent verbalisation data is to systematically study the thinking processes and strategies underlying the performance of a task (Yang, 2003). Participants are asked to concentrate on performing the task, and to verbally express only their thoughts. Participants are instructed to avoid engaging in introspective analysis or in explaining how they solve the problem (Ericsson, 2000). Ericsson and Simon (1993) state that concurrent verbalisations are an especially valid source of data on thinking processes in the context of tasks that take longer to complete. Retrospective reports are conducted after the participant has completed performing a task, and focus on having the participant explain the thinking and reasoning process they used in completing the task (Ericsson & Simon, 1993). Retrospective reports become less valid sources of data with longer performance tasks, but they can be used in conjunction with concurrent verbalisations. The verbal protocol analysis appeared as an evolution of the introspective method, seeking to overcome many of the issues that led to it being criticised. Concurrent verbalisations instruct the participant to simply verbalise their thoughts, without interpreting them (Ericsson & Simon, 1993), as it has been observed that interpretation has an effect on the participants behaviour during the interaction with the system, and that due to interpretation, the experimenter does not have access to the original thoughts of the participant, and therefore his ability to verify the accuracy of the participant’s interpretation is limited. This methodology requires from the experimenter to develop a clearly defined and constrained task that the participant is instructed to perform while engaged in verbalisation (Ericsson & Simon, 1993). The methodology requires the development of an a-priori task analysis. The task analysis provides a systematic representation of though sequences that one expected to be employed in the performance of a task (Ericsson & Simon, 1993). Verbal protocol analysis methodology provides a series of empirically validated procedures for the analysis of the verbalisation data to maximise the rigor and objectivity with which the data was analysed. Nevertheless, concurrent verbalisations have been widely criticised about their validity due to the effect of verbalisation on participants’ thought

24

processes. Schooler et al. (1993) argued that when participants try to verbalise processes that are not readily verbalisable, they are likely to focus on processes that are easier to verbalise but are not necessary helpful in solving the problem. It has also been observed that this effect, called verbal overshadowing, is more obvious in domains that appear to be nonverbal in nature. Other research has demonstrated that performance in several verbal tasks is actually facilitated by concurrent verbalisation (e.g Schooler & Engstler-Schooler, 1990). However, Ericsson & Simon (1993) argued that instructing participants to think aloud does not disrupt the sequence of the participant’s thought processes, provided that only items naturally attended to by the participants are verbalised. It is only when experimenters instruct participants to explain their thoughts that changes in the sequences of cognitive processes may occur. This is especially the case if it has not been made explicitly clear through instruction and practice that the participant should concentrate on talking to himself or herself and should not attempt to explain things to the experimenter (Ericsson & Simon, 1993). Excessive prompting by the experimenters, intended to facilitate participants’ verbalisation rates, can slow the rate of cognitive processing. Thus, it is important to ensure that excessive prompting is avoided during the collection of verbal protocols. 3.2.4 1-participant vs. 2-participant In two-participant methods the communication is not made for the investigator’s benefit but for the other participant. As O’Malley et al. (1984) argue, it is much easier to get a novice to articulate questions to a tutor – who is obligated to try and answer – than to “think aloud” in a way that benefits the experimenter (but not the participant). Therefore the interaction is more enjoyable for the participants and may reveal issues that the single user participant evaluation might not reveal. However, it has been argued that it is more difficult to watch two people collaborating and interacting with the system, than one interacting alone. The investigator is not necessarily observing the participants’ thoughts but on many occasions the articulated thoughts of the participants in the form of questions to each other. Thus, the experimenter is kept only level of abstraction away from the original thoughts and perceptions. Sluis et al. (2001) argued that recruiting pairs of persons from the same household makes them feel more comfortable, providing implicit feedback while talking to each other. There are two distinct types of two-participant evaluation: the participantparticipant evaluation methods such as the co-discovery learning (Rubin, 1994) or the constructive interaction (O’Malley et al., 1984), and the participant-expert evaluation such as the coaching (user can ask an expert questions) and the teaching (expert user teaches novice user) method. The main obvious difference between these two types is that the methods of the first type require participants that have comparable knowledge about the topic. In addition, as can be seen in constructive interaction, there is a need for the participants to want to solve the same problem, and the emphasis is on understanding or developing concepts, as opposed to learning procedures. 3.3. CONCLUSION

25

This chapter tried to define the purpose of the study and describe the rationality behind the experiment’s procedure. By referring to this methodology as “ethnography in the future”, I aim at emphasising two key features; the first part points to the understanding character of the study while the second part pays attention to perceptions instead of the current practice. The need for an understanding character is due to the futuristic type of study. The typical process paradigm build-deploy-evaluate cannot offer a deep insight and is subject to premature commitments. Such a study can be described by the paradigm design-experience-reflect. The first part aims at visualising the future for enabling users experience it and, by challenging their assumptions, we enable them to think beyond the limitations of the current practice. Furthermore, I tried to differentiate the role of verbal data in this type of study comparing to other studies as problem solving experiments or usability studies, and argued for a more active role of the experimenter in the observation process. The next chapter will describe the details of the experiment, including the system, the users, the context and the techniques used.

26

The experiment CHAPTER 4 ________________________________________________________________________ 4.1 INTRODUCTION ....................................................................................................27 4.2 THE SYSTEM ..........................................................................................................28 4.3 THE PROCESS ........................................................................................................29 ________________________________________________________________________

4.1 INTRODUCTION The reported study was conducted at Philips HomeLab (Aarts, 2002) where it is possible to simulate behaviour in a real home environment. This place looks and feels like an ordinary home and, thus, allows unobtrusive monitoring of users’ interactions and conversations while enabling them feel comfortable in a space where they can freely move, relax, interact and explore. This was considered as a critical factor of the effectiveness of the study, and therefore significant effort was spent on communicating an underlying aim of “showing what we want to develop” rather than “testing what we developed”. This enabled users feel more important for the process and therefore taking a more active role, revealing themselves from feelings of being tested or being a critical element of a highly controlled process. Philips HomeLab (fig. 4.1) is a two-floor house consisting of six spaces, including the living room and the kitchen in the ground floor, while the bedroom, the kid’s

27

room and the bathroom are located in the first floor. The reported experiment took advantage of three different spaces including the living room, the bedroom and the kitchen. ________________________________________________________________________

Figure 4.1 HomeLab control room (left) and view of the living room (right)

________________________________________________________________________ Eight pairs of users were recruited through internal advertising mechanisms within the High Tech Campus, located in Eindhoven. Nine users were male while seven were female. Their familiarity to networking concepts ranged from high understanding of the underlying technical mechanism to no prior experience of using networked systems, while there was a bias towards educated and financially fortunate people. Seven of the pairs were couples in life, six of them living together and having shared rights in the structure of the house. One pair was consisted of two colleagues. The initial interest towards pairs of users that live together and have shared rights in the structure of the house proved fruitful. Such users have common experiences, form consistent assumptions and have a common base for stating, debating, and reflecting on their preferences about their actual life. Furthermore, such studies may require significant reflection from the users which implies a need for feeling free to discuss and reflect on feelings related to their personal life. On the other hand, sessions involving users that do not live together can become fruitful at the stage where each user’s assumptions are challenged due to conflicting perceptions between the two users. In any case, the capacity of the session to gain insight in personal attributes of in-home life will largely determine the fruitfulness of the extracted data. 4.2 THE SYSTEM The experiment took advantage of three spaces of HomeLab. These were: the living room, the kitchen and the bedroom. Devices used within the experiment were three TV displays (fig. 4.3), a tablet pc connected through WiFi, as well as an Ericsson P900 smart mobile phone connected via Bluetooth. The application was consisted of an underlying shell (fig. 4.2) and two main applications running above the shell: a media player and a picture viewer. The shell’s as well as the applications’ user interface were specifically

28

adapted to each one of the device’s UI elements in terms of visual output. The output and input modalities were common across all devices while the interaction style did employ variations. The two TV sets located in the living room and the bedroom employed interaction through a conventional remote control, while the TV set located in the kitchen enabled direct manipulation through its touch sensitivity. Interaction with the tablet pc and the mobile phone were taking place through their touch screens. ________________________________________________________________________

Figure 4.2 Remote UI Shell for a mobile phone (left) and TV (right)

________________________________________________________________________ The Remote UI Shell (fig. 4.2) enables the migration/suspension of a session and represents a conceptual bottom level above which all applications are running. Two applications were available for use during the experiment: a media files player and a picture viewer. The media player application was aiming at focusing on the perceived usefulness and easy of use of session migration/suspension (extending to different domains than music), while the picture viewer was aiming at focusing on the social issues of interaction within the home setting. 4.3 THE PROCESS Each two-hour session was starting with a warm-up conversation and exploration of HomeLab which aimed at making participants feel comfortable within the space and with the experimenter. Therefore, a tour of Homelab, accompanied by concurrent introduction to the experiment and the system, aimed first at making sure that users know what the potentials of the system are, and secondly enabling them feel free to explore and interact within the space as well as communicate with the experimenter.

29

Subsequently, users were let free to explore the system. We realised that the level of thoroughness of the previous phase had a large effect on the users’ capacity to interact and explore, and therefore, on the experimenter’s ability to identify issues and engage the users in introspective discussion. During the exploration phase the experimenter was present, observing the users at a distance. Users were instructed to collaboratively interact with one device, while the conversation was constrained only between the two participants except the cases that the experimenter was explicitly interrupting them. The main difficulty in this particular type of study is the need to code and analyse the data in real time, while maintaining a neutral position. The experimenter needs to identify issues in real time and interrupt the participants in certain times in order to get a deeper insight into their perceptions. This is done by referring to prior behaviour which often acts as a trigger for the users’ to reason about the rationality behind unconscious actions. The time period between the actual behaviour and the discussion is of critical importance, while on the other hand, the experimenter needs to be controlling his intervention in order the discussion to be always situated on actual interaction experience. ________________________________________________________________________

Figure 4.3 Views from the experiment

________________________________________________________________________ Subsequently, certain scenarios, incorporating role playing of both participants and the experimenter, were employed to bring into focus the issues that were not covered during the exploration phase. A list of scenarios is described in Appendix. All scenarios were oriented towards role playing and having concrete experience so as to generate the desired level of situatedness. Figure 4.4 illustrates a mock-up that was developed during this study for communicating the added value of nomadic user interfaces in a given domain and generating discussion on issues of concern. The scenario involved three devices within the home: a laptop placed in the living room, the TV set placed within the same space as well as a PDA near by the sofa. The story was described as follows: “We have created a central message centre that has access to all your messages (e.g. emails, SMS). You can access the message centre from every device you want. You come back home and you are

30

expecting an important e-mail from a colleague. You access the message centre but you still haven’t received the expected message. Thus, you turn your message centre into a background mode, move to the sofa with your wife, and switch on the TV. Thus, the follow-me functionality of the message centre is attached to the TV set. After a while, a message is displayed in the TV set saying that you received the message you were expecting. You grab your PDA placed nearby and see the message while your wife continues watching the show on the TV”. This scenario was illustrated by three applications. A message centre application for the laptop, a message centre for the portable device and a follow-me application implemented for the TV including video content and a message pop-up functionality. Figures 4.4 and 4.2 illustrate how one application can be adapted to diverse devices with regard to UI elements. ________________________________________________________________________

Fig. 4.4 Message Center accessed from a PDA (left) and TV (right)

________________________________________________________________________ The last phase of the experiment aimed at revising the identified issues. This phase was used as a hint for the generation of further discussion on the same topic which might be helpful for validating expressed opinions.

31

Results CHAPTER 5 ________________________________________________________________________ 5.1. INTRODUCTION ...................................................................................................32 5.2. FINDINGS ...............................................................................................................33 5.2.1 PAIR ONE: THE CONSERVATIVES ...........................................................................33 5.2.2 PAIR TWO: THE EARLY ADOPTERS ........................................................................34 5.2.3 PAIR THREE: THE FINANCE CONSCIOUS CONSUMERS ............................................34 5.2.4 PAIR FOUR: THE YOUNG COUPLE ..........................................................................35 5.2.5 PAIR FIVE: THE PROGRAMMERS ............................................................................35 5.2.6 PAIR SIX: THE COLLEAGUES .................................................................................35 ________________________________________________________________________

5.1. INTRODUCTION An initial analysis of the data was taking place right after each session. This was because of the need for the experimenter to get immersed in reasoning about participants’ behaviour in real time. These pilot analyses were proved fruitful for identifying emerging themes while at the same time effort was being spent on maintaining a neutral position during the next session for avoiding a biased or incomplete view of the emerging themes. A grounded analysis took place after the end of the field studies. This process started by getting a first view of the emerging themes across all sessions. Due to technical problems and lack of time only six of the sessions were analysed. Transcripts were made only selectively; though, their coverage was significant enough to capture the whole context of the emerging themes. Considerable attention was paid to the context of the emerging themes in

32

terms of prior experience (regarding both interaction and discussion), of the assumptions that were communicated due to the given implementation and due to the experimenter’s bias while trying to get a deeper insight. 5.2. FINDINGS This section will try to describe the findings as they were emerged in the ongoing procedure. This is done in order to capture the context that might have potential influence on the emerging findings. Thus, this section is segmented on six subsections each one trying to describe the findings as they were emerging during each session. In each session, only the major themes are described here and theme repetition is avoided across the whole range of sessions. However, the experiment procedure was almost common in all sessions, though adaptable to the flow of each one. In order to ensure the coverage of all main themes, a list of role-playing scenarios (Appendix) was available for use after the exploration phase. 5.2.1 Pair One: the conservatives The first pair of users was a Dutch couple. The male participant had significant technical expertise in networking concepts while the female had no prior experience using networking products. An initial exploration of HomeLab aimed at creating a positive atmosphere in terms of feeling free to interact with the equipment, inspiring them and increasing their perceived importance in the study as well as feeling free to discuss their personal thoughts and perceptions both in front and with the experimenter. A subsequent interview tried to capture their practice in terms of in-home music habits, possession and use of domestic information and communication technologies, and the in-home use of mobile phone. They are called as “the conservatives” due to their limited use of technologies in home. They own a 10-years old personal computer which is placed in the work room and is turned on only when there is a need for work. Most of the activities that accompany the computer need concentration and are only for short periods of time, while when they were asked about the level of satisfaction regarding their possession of technologies, they expressed the need of having a laptop so as to do take it on the living room when they want to be together or when they want to show something to their guests, like a picture or an internet site. Subsequently, we moved to the sofa and the basic structure of the system was explained in order be able to explore its potentials with minimal need for intervention from the experimenter. Afterwards, they were let alone in order to engage themselves in interaction and feel free to explore and criticise the system. They tried to choose a device to use first, including the mobile phone, the tablet pc and the TV, so they engaged in a discussion of perceived device advantages and restrictions. The tablet pc was considered as easy to use while the mobile phone was considered as more portable for use within the home. They started using the mobile phone and accessed a media player application. When they tried to migrate the session to the TV set they were surprise when the control was removed from the mobile device as it was totally migrated to the TV set. They were expecting to continue interacting with the portable device while gaining access to the TV set. At this time the experimenter interrupted the process of interaction in order to get a deeper

33

insight on the reasons for this breakdown. The participants demanded the need for partial migration. However, they considered that interacting with two displays has its own trade-offs, especially when the two displays are placed in a distance and they have considerable difference in size. Subsequently, a scenario tried to engage the users in interaction with a media player including the user explicitly suspend the activity for a significant amount of time. The participants were positively surprised by the fact that the application maintained the media playlist and the status of the session, and during a subsequent discussion they expressed a perceived distinction between different types of media in terms of their nature regarding time. 5.2.2 Pair Two: the early adopters The second pair of participants was constituted by two young people working in the industrial design and applied physics domains. They are described as early adopters due to the high amount of money they spend on technological artefacts and the high use of them. The process was maintained as in the previous session. While trying to view some pictures together on the mobile phone, they realised the added value of the large TV display. Therefore, they tried to migrate the session to the TV set but they were expecting a partial migration. They wanted to continue interacting with the portable device and “share” the pictures on the large display. Role playing was used to simulate a real need and identify the behaviour of the user. A subsequent discussion tried to reason about the user’s behaviour. The male user tried to “push” the romantic music playlist to the female participant as he already formed his goal due to the reason that it was the tasks’ goal. However, they both realised the added value of recalling a session from the target device as they don’t always plan their moves within the home. Regarding the email notification and “access from anywhere” scenario, they noted that the added value of such a system is mostly in entertainment applications. They preferred to separate these activities “just because the pc is more efficient”. 5.2.3 Pair Three: the finance conscious consumers This session gained a deeper insight in the nature of the in-home activities as well as the advantages and trade-offs of certain migration paradigms. They paid attention to the need for recalling the session from anywhere as they can move to one place only for a while, but then decide to stay there. In addition, they paid attention to the need for being able to explicitly suspend a session for supporting the feeling of being in control. However, they felt that having to push the session, although it might be good when they know that they are going to a certain place, most of the times they move around the home and they need to take their activities with them without having to engage in complex interactions, especially prior to the relocation. They noted that there might also be some “dead time” when relocating (e.g. brush teeth before going to the bedroom). Subsequently, the concept of tokens (Sluis et al., 2001) was communicated to the users. While they were impressed by the simplicity, they had a negative attitude due to various reasons. As they said, “I may loose it, I may go from place to place and I may sometimes have to look for the token in other places in order to migrate my session… The token requires

34

that you think of it beforehand… Your kids can play with it and take it in the garden… I don’t want to carry another piece of equipment all the time… What if we are watching TV together and she wants to take this session upstairs, while I want to stay downstairs and watch another program. So, she takes the token, but can I still watch TV? Is it switching off because the current application was taken away? ”. 5.2.4 Pair Four: the young couple This session overlapped issues that were covered in most other sessions but did not gain any significant additional insight or reasoning about their occurrence. When they were asked if they would find useful buying and a device like the ones used in this experiment, they had a negative attitude towards expensive devices that do not offer clear additional value. Portable devices tend to extend on the mobile phone concept while they add computing intelligence. However, concerning the concept of partial migration through personal, portable devices, they were wary regarding their concurrent use as a personal device and a common device for the household. 5.2.5 Pair Five: the programmers This session was valuable in terms of new ideas that were brought due to the experience of the participants in designing technologies. However, despite the value of creativity, there is often a need to decrease the level of abstraction and situate the discussion through concrete experience. To achieve this, we first tried to increase their perceived importance in the study due to their special background and subsequently pay attention to the need for situated discussion and access to their perceptions. Role playing to simulate flawed interactions enabled situated discussion on participants’ actual behaviour as well as reflection on this behaviour. Extensive discussion on the users’ perceptions of the networking concept took place. Users paid attention to the difference between device that they can interact with – the ones perceived as interactive, and devices that they can use for output modalities – the perceived as output devices (e.g. display, speakers). The complexity of partial migration was reinforced but further insight was gained. 5.2.6 Pair Six: the colleagues The last pair of users was constituted by two colleagues working in the area of lighting. They were first introduced to the HomeLab and the initial interview managed to bring into focus conflicts and different perceptions which was fruitful in terms of getting a better reasoning about these preferences. They started by collaboratively exploring the interface. Initial problems in understanding where the data come from and where the session is located, brought into attention interesting issues regarding the mental model that the device communicates and whether the intended use is being afforded by the given paradigm. Role-playing enabled users to experience situations where social conflicts are occurred due to the sharing of devices, as well as the privacy issues that are related to exposure of private information in social spaces and devices. While the one was performing one activity on one TV set, the other tried to push a session from another device within the

35

home. The participants were wary about issues of device occupancy. Users do not want to be annoyed by others sending them sessions and people use to check first whether the device is occupied in order to relocate there. Yet, the fact that a device is switched on, or even when a person is in the space of this device, does not necessarily mean that the device is occupied. On the one hand, they noticed the need for context awareness but on the other hand they realised that social practice is much more complicated than “device occupied” or “person interacting with device”. Furthermore, situated on the message center concept (figure 4.4), they were wary about the privacy of their personal data that is threatened by other users as well as the privacy of their personal life that is threatened by the “always attached” technologies.

36

Discussion CHAPTER 6 ________________________________________________________________________ 6.1. INTRODUCTION ...................................................................................................37 6.2. NATURE OF ACTIVITIES ...................................................................................39 6.3. IN-HOME PRIVACY .............................................................................................40 6.4. PERCEIVED DEVICE ROLE, CAPABILITIES AND RESTRICTIONS........40 6.5. PARTIAL MIGRATION........................................................................................41 6.6. REVEALING THE SESSION................................................................................42 6.7. MIGRATION PARADIGM ...................................................................................42 ________________________________________________________________________

6.1. INTRODUCTION Sluis et al. (2001) refer to freedom of movement as a crucial need in domestic environments. They argue that access to a particular service will no longer be restricted to a certain location or device. However, this study revealed a hesitant attitude towards the concept of nomadic user interfaces. One of the reasons for this, is that in order for nomadic user interfaces to work, a complete system of interconnected devices is needed. Also, users rarely have had the chance to interact with such a complete system before the purchase. Furthermore, people are used to buying individual products that provide clear benefits, not systems for which more than one device is needed. So,

37

since nomadic user interfaces constitutes such an unfamiliar concept to the users, this may be rephrased to providing no clear benefits to the users. The added value only becomes apparent once the user owns a number of devices that are attached to the network which are compatible with each other (important to note here is that devices will only be gradually added to the network, therefore it is important that new devices remain compatible with old devices). Thus, to provide users with clear benefits for nomadic user interfaces we need to investigate their needs in domestic environments and how these needs are currently fulfilled. Therefore, part of the study aimed at understanding the users’ needs in domestic environments through the comprehension of their perceptions and assumptions. The basic needs, relevant to Nomadic User Interfaces, were identified and classified as illustrated in figure 6.1. ________________________________________________________________________ Relocation

Mobility

Access from anywhere Suspension

Needs in domestic environments Enhanced capabilities Enhanced interaction

Appropriate style of interaction Collaboration

Figure 6.1 Classification of needs, relevant to Nomadic User Interfaces, in domestic environments

________________________________________________________________________ The need for mobility is partly realised through the need for relocation; this takes place when a user, immersed in a given activity, needs to move to another place, while relocating and maintaining the status of the ongoing activity. “Access to anywhere” refers to the ability to access applications that exist in the network from whatever place and device. However, users in domestic environments often have to perform multiple activities, and they need to maintain the status of their sessions (suspending) while performing other tasks. On the other hand, mobility is not the only need that nomadic user interfaces are called to fulfil. Enhanced interaction may be realised through the access to device-specific applications from other devices, that offer either enhanced capabilities in terms of I/O interaction resources, or a more appropriate style of interaction in given situations (e.g. lean back interaction for a more passive-relaxed interaction). Furthermore, as it will be described in the next sections, domestic technologies are also called to fulfil the need for collaboration – social interaction – in certain regions of the home.

38

6.2. NATURE OF ACTIVITIES However, in order to understand the domestic needs that nomadic user interfaces are called to fulfil, we need to understand the nature of the activities that those media support. The question raised was: what is the nature of in-home activities and what effect does it have on the users’ attitude towards migration and suspension? Figure 6.2 illustrates a classification of media supported activities as derived from what users said in the study. ________________________________________________________________________ High level of involvement

passive

Low level of involvement Media supported in-home activities

interactive

Figure 6.2 Classification of media supported activities in domestic environments

________________________________________________________________________ The first distinction corresponds to the level of interaction with the system that the activity requires, as perceived by the user. For example, a web browsing activity involves higher level of interaction comparing to watching a movie. This distinction is valuable when we need to consider the added value of migration across a range of diverse devices with regard to their UI elements. In addition, higher level of interaction corresponds to higher level of situation awareness. Thus, the migration of passive activities has an additional need for simplicity as there is lower level of goal awareness. Also, for such passive activities it is important to look at the level of commitment or involvement in the given activity. The higher the level of commitment will be in a given activity, the higher the need for continuing this activity will be. The effort for managing activities that involve lower level of commitment should be significantly lower. For example, listening to music is considered as a background activity, an activity that involves lower level of commitment, or “something you can do while you are doing something else”, while watching a film involves higher level of commitment. Furthermore, it needs to be noticed that this distinction is different to McLuhan’s (1964) distinction between “hot” and “cool” media. McLuhan referred to the level of sensory data that the medium contains. Therefore, a hot medium, which contains relatively complete sensory data, requires lower sensory involvement from the user and thus it is synonymous to lower participation. However, our distinction takes into account the current social practice, not focusing solely on the cognition side. Thus, it is different in the sense that, even though a “cool” medium may in-principle require higher level of commitment, this might not be observed in the current social practice. Such a distinction is expected to lead to an understanding and reasoning for the added value of nomadic user interface for different types of applications.

39

During the experiment, we identified a hesitant attitude towards activities that involve higher level of interaction. Users believed that the migratability of the user interface adds value mostly to entertainment applications which involve lower levels of interaction. As one participant noticed, “for the email type of thing, I have a computer that does it much better”. Activities that require higher levels of involvement need a different treatment with regard to the time for which a session may remain suspended. One participant noticed: “you want to suspend your movie only for a short time. The play-list is different; it’s like your private collection…”. 6.3. IN-HOME PRIVACY Furthermore, nomadic user interfaces have to deal with the perception of inhome regions. Goffman (1959) defined a region as any place that is bounded to some degree by barriers of perception. During this experiment Du Gay’s (1997) distinction between public and private spaces was confirmed. The living room is the default social space and what is needed is some computational intelligence there. As one participant noticed, “if you want to show something to other people, you don’t have to drag everyone upstairs where the computer is located… show some internet site, or if you have a disc with pictures…”. The computer is often placed in private places: “some times I prefer to concentrate… some other times I might want a laptop to sit in the living room; NOT always, but sometimes”. There is also a distinction between personal and common devices. The first provide a better connection with you; you form them as you want, you have your personal data, and you do not want everyone else to have access to this device. In the same sense, you do not want other users to have access to your personal data and applications. Home networking needs to take this into account. An important question for further investigation is whether the current technology affords privacy. This question should be studied in a much more social context than solely technical. Even if existing technology facilitates secure access to personal data, if the interaction is not in line with the domestic needs and the social interaction that the home place incorporates, this is unlikely to result to perceived privacy. 6.4. PERCEIVED DEVICE ROLE, CAPABILITIES AND RESTRICTIONS Furthermore, the perceived role of a given device determines the way of its use. It has been widely discussed that the TV represents a social device in the users’ perceptions, while the personal computer is a private device, highly associated with work tasks or situations that a person requires isolation to focus on a certain activity. On the contrary, TV often constitutes a form of bonding for the family and serves most of the time as a background activity. This has direct implications on nomadic user interfaces: every device has a certain role within the home and migratability will be constrained by the users’ perceptions regarding the role of each device. As noted by one participant: “we have so many devices that I would prefer to leave the TV for other things”. TV is capable of providing multimodal content without requiring interaction by the user; TV is “lively”; it is capable of providing companionship and raising the level of energy within the home. Portable

40

devices are perceived as highly interactive. A smart mobile phone is perceived as more portable but is associated with worse interaction comparing to the “easy to use” tablet pc. Data retrieved from this experiment regarding the current use of mobile phones within the home revealed that in-home mobile phone use is not significant. Mobile phones are placed near by the kitchen, near by a computer or even in bags. However, most of the participants did not own any type of smart mobile phone or personal digital assistant. They were mainly using their mobile phones only for communicating and they could not find any significant reason for in-home use as they could be reachable through their home telephone. Therefore, it is likely that the current trend towards multipurpose portable devices will bring a shift in the users’ perception and therefore in the actual use of the personal device. An interesting distinction that exists in the users’ perceptions is the one of interactive devices against output devices. As noted before, portable devices are considered as highly interactive while TV is considered as a display rather than a device. As noted by one participant “it is a display, it doesn’t have applications”. This distinction was revealed in most sessions; they argued that “we have devices to interact with, the ones I have in mind to do something with, and devices for output stuff…”. The whole history of TV is accompanied by the passive activity of watching television; iTV brings the notion of interaction to this medium but iTV studies revealed significant difficulties in achieving user acceptance of such interactive applications. The question becomes: was is solely due to passive role of activities accompanied by the use of TV or was it due to interaction restrictions of the remote control? Nomadic user interfaces are likely to enhance the interaction within this space by employing portable devices as controlling devices for the TV display. In this sense, even though TV will gain access to a wide range of applications, the interaction paradigm will enforce the users’ perception of TV as a display. 6.5. PARTIAL MIGRATION Partial migration will enable users to have access to distributed interaction resources rather than interaction devices. Users will be able, for example, to create a media playlist and migrate the audio output to the home cinema audio system. Viewing pictures with friends can be another demonstrating scenario. Taking advantage of the social TV display is an added value of nomadic user interfaces. However, the issue of interacting with two devices requires investigation. Users preferred to interact with a portable device for navigating through the pictures, rather than the TV remote control. However, the questions regarding the association between the portable device and the TV set did not seem to have a clear answer. They liked the idea of navigating through the list of pictures and projecting only the ones they want to share when being with visitors in the home, but they felt that having to switch their focus between those two screens is tiresome, and added that it also isolated them from the social activity as it requires significant concentration on the interaction with the portable device. Thus, they realised the advantage of interacting with a remote control or “scrolling with my

41

mobile phone without having to look at it” as they noted. In addition, they felt that having a portable device in their hands gives them the possibility to point within the picture so as to enhance their collaborative session. 6.6. REVEALING THE SESSION However, as UI elements become distributed, this is very likely to cause conflicts and confusion. For example, consider the following scenario: one starts interacting with a tablet pc, opens a media player session and migrates the audio output to the stereo speakers placed in the room. Should the tablet pc then be reserved, or what should happen if the tablet pc is no more available and the music is playing on the speakers? We might want to distinguish between media types with regard to the using modalities (e.g. video, audio, etc.). Thus, different media types should have different rights in occupying available interaction resources in the room. For example, audiovisual content should have the right to occupy the TV set as well as the speakers placed in the room. Audio content, on the other hand, should have the right to occupy only the speakers available in that room. The question becomes: which should be our target when migrating sessions? Should we have the speakers resource as a different entity from the TV set and assign rights so that the TV set can “command” the speakers when having multimodal content? Sluis et al (2001) introduced, in a similar way, the notion of “interacting with the room”, liberating users in such way from the additional cognitive effort as well as the complexity brought by the UI element distribution. The experiment simulated a situation where users had to deal with an interaction flaw, as described above, and users were asked questions in order to understand their perceptions, while their behaviour was observed. These investigations revealed that the session was located, according to users’ perceptions, in the device they started interacting with, even though they had partially migrated the main output modality to another interaction resource; in spite of being able to perceive the activity (music playing) only through the sound, they were not ready to accept that the session was located in the speakers: “the session is in the device you are interacting with”. In addition, the notion of networked activities is not readily comprehensible by all users. Therefore, there is a need to provide a stable and easily comprehensible conceptual model, even thought it may not correspond to reality. This might either correspond to a central place where everything is located, or to application distribution within the range of available devices; Even a led blinking on the device might enable users to perceive that the data are coming from this device. 6.7. MIGRATION PARADIGM It is obvious from the previous section that partial migration adds significant complexity, but unclear issues are evident even in much simpler situations, regarding the migration paradigm. Thus, multiple migration paradigms were experienced generating points of conflict and identifying and reasoning about preferences. A critical question was whether people prefer to “push” a session from the device they are currently towards a target device or relocate and then “pull” the session towards the new device. Although in some situations they argued that, they know that they want to move to a certain place and

42

therefore have formed their goal and is more convenient to “push” the session to the target device, the “push” method in most cases appeared to be accompanied by many trade-offs. As it was extensively argued, we are rarely moving rationally within the house. Our mobility behaviour would be more appropriately described by Suchman’s (1987) notion of situated decision making rather than by absolute rationality. As it was mentioned, “I may go upstairs for a while, but then decide to stay there. So, I should be able to recall my session…”. Access from anywhere is critical as people rarely program their behaviour in home. In addition, we need to consider the dead time. Users mentioned that “I may want to transfer my movie to the bedroom TV but first I want to go and brush my teeth…”. The “push” paradigm may often be perceived as a force towards making premature decisions. On the other hand, people want to be sure that the status of their session will be remained. Automatic suspension does not give that feel of control, and therefore, although users soon realised the existence of automatic suspension felt that it was removing control from them: “I want to be in control; I want to say pause! Hold what you are doing…”. The “pause” metaphor is very familiar to the users and therefore should be considered for the suspension of networked activities. This metaphor might be applied to applications other than video content as well. The question becomes: how should the effect of this action be then visualised? According to the video domain, the picture freezes and a message may appear. However, is it suitable for other applications? Other applications might not contain moving visual content and a suspended session should optimally not occupy the device. People felt the need for explicitly suspending their session: “somebody ringing on the bell, so I am saying: do nothing until I come; if you press the pause button, you have a better feeling of control”. In addition, users felt that in some situations they need to mirror the session in two or more devices: “I might be cooking and want my music both in the kitchen and the living room”. With regard to the token paradigm, users were hesitant due to various reasons. They felt that it could be easily lost or that they might have to look for the token, which might be placed in another TV set, in order to move the activity. Although the token has the potential to deal with the “dead time” problem, it is still a “push” activity as you have to think of it beforehand. Furthermore, they were concerned about the fact that they have to carry the token with them, while they might decide to do something else before going to the final destination. In conclusion, it seemed that the token does not cope well with the complexity of in-home interactions (e.g. “What if we are watching TV together and she wants to take this session upstairs, while I want to stay downstairs and watch another program. So, she takes the token, but can I still watch TV? Is it switching off because the current application was taken away? ”).

43

Conclusion CHAPTER 7 ________________________________________________________________________ 7.1 FINDINGS SUMMARY ..........................................................................................44 7.2 FUTURE WORK......................................................................................................45 ________________________________________________________________________

7.1 FINDINGS SUMMARY Nomadic user interfaces will enhance the home life. People will be able to move freely from space to space while maintaining the technology-mediated activity regardless of devices’ UI attributes. They will experience an enhanced interaction with domestic technologies: they will be able to utilise distributed user interface elements (e.g. interact with the tablet pc and output sound to home cinema speakers), achieve more appropriate style of interaction in given situations (e.g. take advantage of TV’s lean-back interaction style to access the pictures that she captured today with her mobile phone), or adapt the use to given situations (e.g. share pictures on the large display with guests). However, we need to consider the users’ current practice and their needs as they perceive them. This study showed that, users’ perceptions may significantly restrict the added value of nomadic user interfaces for given activities, in given regions, or with given devices. In addition, nomadic user interfaces bring a radically different interaction

44

paradigm. We first need to consider the provided conceptual model, as the current one of applications-associated-with-devices is not necessarily adequate. Vander Wal (2005) in a different context introduced the notion of personal InfoCloud which might, in the nomadic UI context, represent a “cloud” that contains all the applications attached to the user. Furthermore, we need to take into account the nature of activity. iTV studies revealed a passive character of the TV watching experience, but is it only due to the affordance of the medium and the space? New devices and applications may bring richer interaction in those spaces and change the users’ perceptions of certain activities. TV often acts as a form of bonding for the family and users often “do their own thing” even when collocated. People want to access applications from anywhere. 7.2 FUTURE WORK Ubiquitous computing is not only a trend. It represents the maturation and evolution of computing from the point where one uses a technology to perform a given task to the point where one uses webs-of-technology (Nielsen, 2002) to support his activities. Activity Centred Design (Gay & Hembrooke, 2004; Norman, 2005) or Ubiquitous User Interface Design (Brodersen & Krisensen, 2004; Nielsen, 2002; Bertelsen & Bødker, 2002), based on core ideas of Activity Theory, is a very promising approach that tries to bridge the gap between design and use. During this project we realised the gap between design and use even in the theoretical level. The development of future pervasive technologies demands a theoretical background. Furthermore, industry will need knowledge to be modelled into tangible, industrial processes. Actual use needs to be captured during the initial phases of the product and become incorporated, through fast-feedback processes, into industrial models (e.g. Venkatesh, 2003). The product conceptualisation phase needs to be supported by activity based ethnographic studies that will go beyond the communication paths of Distributed Cognition and provide a better structure than Activity Theory. Significant technical problems need to be overcome in order to realise the vision of nomadic user interfaces. Device independence is not a trivial requirement and the question becomes more complicated when it comes to the design of device independent user interfaces. Our distinctions between designers, developers and HCI consultants need to be questioned. HCI community is talking about model based design for quite a long but we need to question ourselves whether these modelling techniques and approaches have been adopted in industrial, time-pressured environments. Furthermore, despite the extended work on model-based device-independent user interfaces, according to the author’s knowledge none of these studies takes into account the different role that each device might have in the completion of one task. Consider the below scenario: “Jennifer is leaving for work. While she is picking up her keys, the system recognises that she is leaving (e.g. through RFID technology) and a touch screen that is placed in the wall near by her apartment’s door displays her meetings. She realises that she will be late, so she transfers the activity to her mobile phone where she can easily send a voice message to the other members of the meeting, informing them that she will be 10 minutes late”. The underlying problem is twofold: (a) current

45

infrastructure (e.g. Remote User Interface Protocol) does not support such a scenario, and (b) how do we incorporate different task models that represent the use of each device, and how do we employ them in an iterative design and evaluation process? This study focused on user perceptions regarding in-home nomadic user interfaces. However, the need for further study is obvious as the reported one was limited by significant restrictions such as time, limited number of participants and more importantly, by limited provided interaction. The effectiveness of such studies depends significantly on the level of experience that the users receive. Thus, I would argue for the importance of understanding the use of the product in actual environments for long periods of time. Long term studies in situ will enable us to observe how the users’ behaviour changes over time. This will subsequently provide valuable information relevant to the learning curve regarding the use of the technology as well as the actual needs of the users. In a similar context, a commercial company (Orange) conducted long term studies where the users were provided with the maximum currently available technological options. This enabled them understand what users do not want and what their real needs are. This study was an explorative investigation on user perceptions of in-home nomadic user interfaces. The results of this study need to be validated while they can act as an informative point for further research. Further attempts need to be situated in actual environments and for longer periods of time.

46

References A. van Dam. Post-WIMP user interfaces, Communications of the ACM, Vol. 40, No. 2, Pages 63-67, 1997. Aarts E., Ambient intelligence in homelab, ‘www.research.philips.com/technologies/misc/homelab/downloads/ambientintelligence.pdf’, 2002. Aldrich, F.K. Smart Homes: Past, Present and Future, in Inside the Smart Home, R. Harper (ed.). London: Springer-Verlag, 2003. Andersen, P.B. A Semiotic Approach to Construction and Assessment of Computer Systems. In Proceedings of ISRA-90 Conference. Elsevier, pp. 465-514, 1991. Argyris, C., and Schon, D. A.. Participatory action research and action science compared—A commentary. In Whyte, W. F. (ed.), Participatory Action Research, Sage, Beverly Hills,CA, pp. 85-96, 1991. Avison, D., Lau, F., Myers, M. and Nielsen, P. A. Action Research, Communications of the ACM, Vol. 42, No. 1, pp. 94-97, 1999. Baber, C. and Stanton, N.A. Observation as a technique for usability evaluation, In P.W. Jordan, B. Thomas, B.A. Weerdmeester and I. McClelland (eds.) Usability Evaluation in Industry London: Taylor and Francis 85-94, 1996. Baecker, RM. Readings in Groupware and Computer-supported Cooperative Work, Morgan Kaufmann, 1993. Bandelloni, R. and Paterno, F.` Flexible Interface Migration, in Proc. Of Intelligent User Interfaces (IUI ’04), pp. 148-157, 2004.

47

Bannon, L. & Schmidt, K. CSCW: Four Characters in Search of a Context. In J. Bowers & S. Benford (Eds.) Studies in Computer Supported Cooperative Work: Theory, Practice and Design. pp. 3-16, 1991. Beaudouin-Lafon, M. Designing Interaction, not Interfaces. Proc. Conference on Advanced Visual Interfaces. ACM Press. pp.15-22, 2004. Beigl M. Point & click – Interaction in smart environments. In: Proceedings of HUC’99. Berlin, Springer-Verlag 1999. Berg, A.-J. A gendered socio-technical construction: the smart house, in: C. Cockburn and R. Furst-Dilic (Editors), Bringing Technology Home: Gender and Technology in a Changing Europe (Open University Press, Milton Keynes) pp. 165-180, 1994. Bertelsen, O. & Bødker, S. Interaction through multi-artifacts. In Bagnara, S., Pozzi, S., Rizzo, A. & Wright, P. ECCE 11 - Cognition, Culture and Design Eleventh European Conference on Cognitive Ergonomics, Rome: Instituto di scienze e technologie della cognizione consiglio nazionale delle ricerche, pp. 103-111, 2002. Bodil, J., Malmborg, L., Svensk, A., Anderberg, P., Brattberg, G., Breidegard, G., Eftring, H., Enquist, H., Flodin, E., Gustafsson, J., Magnusson, C., Mandre, E., Nordgren, C. and Rassmus-Gröhn, K. 2004. “Situated research and design for everyday life”. [Online]. Lund University, Sweden, Accessed May 2004. Bolt, R. A. Put-That-There: Voice and gesture at the graphics interface. ACM SIGGRAPH Comput. Graph., Vol. 14, No. 3, pp. 262-270, 1980. Boren, M. and Ramey, J. Thinking aloud: Reconciling Theory and Practice, IEEE Transactions on Professional Communication, 43(3): 261-278, 2000. Brodersen, C. and Kristensen, J.F., Interaction through negotiation, In Proceeding of NordiCHI’04, ACM, 2004. Brombacher, A.C., Sander, P.C., Sonnemans, P.J.M., Rouvroye, J.L.; Managing product reliability in business processes ‘under pressure’, Reliability Engineering and System Safety 2004 Brown, B. and Perry, M. Why don't telephones have off switches? understanding the use of everyday technologies., Interacting with computers, 12, 623-634, 2000. Checkland, P., Systems Thinking, Systems Practice, John Wiley & Sons, Ltd, Chichester, 1981. Clawson, T. Boxing Clever, E-bussiness, January 2002. Coutaz, J., Lachenal, C., Dupuy-Chessa, S. Ontology for Multisurface Interaction. Proceedings INTERACT, 2003. Daly-Jones, O., & Carey, R. (2000). Interactive TV: A new interaction paradigm. Paper presented at the CHI 2000. Dees, W. Handling Device Diversity through Multi-Level Stylesheets, in Proc. Of Intelligent User Interfaces (IUI ’04), 2004.

48

Dey AK, Abowd GD. Towards a better understanding of context and contextawareness. CHI’2000 Workshop on the What, Who, Where, When, and How of Context-Awareness, 2000. Dourish P. Where the action Is: the foundations of embodied interaction. MIT Press, Cambridge, MA, 2001. Du Gay, Paul, Hall, Stuart, Janes, L., Mackay, H., Negus, Keith, and Tudor, A. Doing cultural studies: The story of the Sony Walkman. London: Sage, 1997. Eggen, B., Hollemans, G., Sluis v.d. R. (2002). Exploring and Enhancing the Home Experience. Journal on Cognition Technology and Work, (5), pp. 4454, Springer-Verlag, London Ltd. Engestrom, Y. When is a tool? Multiple meanings of artifacts in human activity. In Learning, Working and Imagining, Orienta-Konsultit, Helsinki, Finland, 171–195, 1990. Ericsson, K. A. & Simon, H. A. Protocol Analysis: Verbal Reports as Data (revised edition). MIT Press, Cambridge, Mass, 1993. Gay, G., and Hembrooke, H. Activity-Centered Design: An ecological approach to designing smart tools and usable systems. Cambridge, MA, MIT Press, 2004. Goffman, E. The Presentation of Self in Everyday Life, Penguin, New York, 1959. Grudin, J. The Computer Reaches Out: The historical continuity of user interface design. In Proceedings of CHI '90, ACM SIGCHI Conference, ACM, 1990. Hammond, S. A. The Thin Book of Appreciative Inquiry. Kodiak Consulting, Plano, Texas, 1996. Harper, R. Inside the Smart Home, Springer-Verlag, London, 2003. Intille. S.S., Designing a Home of the Future. IEEE Pervasive Computing, pp. 80-86, 2002. Kuutti, K., and Bannon, L.J. Searching for unity among diversity: Exploring the“interface” concept. In Proceedings INTERCHI’93, p. 263–268, April 1993. McLuhan, Marshall. Understanding Media, New York: McGraw-Hill, 1964. Nielsen, C. Designing to support Mobile Work with Mobile Devices. PhD dissertation, University of Aarhus, 2002. Norman D.A. The Invisible Computer, MIT Press, 1998. Norman, D.A., Human-Centered Design Considered Harmful, Interactions, 12. 4, ACM, 2005. Norton, C. Two in Ten Watch TV 36 Hours Every Week, The Independent, 19 May 2000.

49

Olson, J.R., & Biolsi, K.J. Techniques for representing expert knowledge. In K.A. Ericsson and J. Smith (Eds.), Toward a general theory of expertise: Prospects and limits. Cambridge: Cambridge University Press, 1991. O’Malley, C., Draper, S. and Riley, M. Constructive interaction: A method for studying human-computer-human interaction, in Proc. IFIP Conf. HumanComputer Interaction: Interact’84, Amsterdam, The Netherlands, 1984, pp. 269–274, 1984 Pallasmaa, J. Identity, intimacy and domicile - Notes on the phenomenology of home, The Consept of Home: An Interdiciplinary View -symposium at the University of Trondheim, 21-23 August 1992. Peters, S. Emotional Context and “Significancies” of Media, in Inside the Smart Home, R. Harper (ed.). London: Springer -Verlag, 2003. Petersen, M.G., Madsen, H.K. & Kjær, A. The usability of everyday technology: emerging and fading opportunities. ACM Transactions Computer-Human Interaction 9(2): p74-105, 2002. Preece, J. Sociability and usability: Twenty years of chatting online. Behavior and Information Technology Journal, 20, 5, 347-356, 2001 Reeves, B., and Nass, C. The Media Equation: How People Treat Computers, Television and New Media Like Real People and Places. New York: CSLI, 1998. Rekimoto, J. Pick-and-Drop: A direct manipulation technique for multiple computer environments. Proceedings of UIST’97. ACM, 3 I-39, 1997. Robertson, S., Wharton, C., Ashworth, C., and Franzke, M. Dual device user interface design: PDAs and interactive television. In CH1’96 Proceedings, pp. 79-86, 1996. Robinson, H. Towards a Sociology of Human-Computer Interaction. In P. Luff & N. Gilbert & D.Frohlich (eds.) Computers and Conversation, Academic Press, London, pp. 39-50, 1990. Rubin, J. Handbook of Usability Testing, Wiley, 1994. Schilit, B. N., Adams, N., Gold, R., Tso, M. M. and Want, R. The ParcTab mobile computing system. Proceedings IEEE Workshop on Workstation Operating Systems, Napa, California, 1993. Schooler JW, Engstler-Schooler TY. Verbal overshadowing of visual memories: some things are better left unsaid. Cognitive Psychology 17: 36– 71, 1990. Schooler JW, Ohlsson S, Brooks K. Thoughts beyond words: when language overshadows insight. Journal of Experimental Psychology: General 122: 166–183, 1993. Senge, P., Scharmer, C.O., Jaworski, J. and Flowers, B.S. Presense – exploring profound change in people, organisations and society, Nicholas Brealy Publishing, 2005.

50

Shneiderman, B. & Maes, P. Direct Manipulation vs. Interface Agents," Interactions (ACM), 4(6 (Nov)), pp. 42 – 61, 1997. Shrubsole, P. & Dees, W. A survey of technologies for Device Independent User Interfaces, Philips Report Technical Note, PR-TN-2003/00396, 2003. Silverston, R. Television and Everyday life, Routlege: London, 1994. Suchman, L. A. Plans and Situated Action. Cambridge University Press, Cambridge, 1987. Taylor, A. S. & Harper, R. Switching on to switch off, in Inside the Smart Home, R. Harper (ed.). London: Springer-Verlag, 2003. Thevenin, D. and Coutaz, J., Plasticity of User Interfaces: Framework and Research Agenda, Proc. of INTERACT’99 (Edinburgh, August 1999), IOS Press, 1999. Turk, M. & Robertson, G. Perceptual user interfaces. Communications of the ACM, 43(3), 33-34, 2000. Van de Sluis, R., Eggen, B., Jansen, J., and Kohar, H. User Interface for an In-Home Environment. Proceedings of the Eighth IFIP TC.13 Conference On Human-Computer Interaction, 2001. Vandervelpen, C. and Coninx, K. Towards Model-Based Design Support for Distributed User Interfaces. 2004. Accepted for publication at NordiCHI, 2004. Venkatesh, A. A conceptualisation of the household/technology interaction. In Advances in Consumer Research, 1985. Venkatesh, M., Davis & Davis, User Acceptance of Information Technology: Toward a Unified View, MIS Quaterly, vol. 27. pp. 425-478, 2003. Weiser, M. The computer for the 21st century. Scientific American, 265, 3, 94104, 1991. http://www.ubiq.com/hypertext/weiser/SciAmDraft3.html Yang, S. C. Computer-mediated history learning: Spanning three centuries project. Computers in Human Behavior, 19(3), 299–318, 2003.

51

Appendix

52

Informed Consent Field Test of Networked Applications in Domestic Environment I freely and voluntarily consent to participate in the Philips Research project involving a field-test of networked applications in domestic environments. I understand that the experiment may last up to two hours and involves my and my partner’s usage of the system as well as interview sessions both before and after my experience with the system. I understand that all interactions and discussions will be monitored and video as well as audio data will be captured. I have been told that every effort will be made to keep my responses and any other information that is collected during the study confidential. I understand that I will not be liable for any damage caused while using the equipment, apart, of course, if caused by any deliberate actions with the intend of misuse of the system. I acknowledge that participation in this study may include access to Philips' proprietary information and agree that I will maintain full secrecy and confidentiality with regard to Philips' proprietary information which has explicitly been indicated as such or which should reasonably have been understood by me to be of Philips' proprietary/confidential nature. I have been given the chance to ask questions about the study, and any questions have been answered to my satisfaction. I have also been given a copy of this consent form. I have read and understand this before: -----------------------------------------------------------------------------------------------------------Participant's Signature and Date

-----------------------------------------------------------------------------------------------------------Participant's Name (printed) A member of the research team has explained the procedures and offered to answer questions: Evangelos Karapanos -----------------------------------------------------------------------------------------------------------Researcher's Signature, Name and Date Philips Research Prof. Holstlaan 4 (WDC 2054) 5656 AA Eindhoven Tel. 040-274 3003 / email: [email protected]

53

Philips HomeLab

Is it still your dream to visit Philips HomeLab?

This is the opportunity for you! We need your opinion about our new technologies. The experiment will last no more than 2 hours. We need people, preferably couples to perform this experiment. Therefore, if possible, bring with you your friend, wife, husband, or anyone you feel that you can easily discuss with. The communication will be done in English. Every participant will receive a present for offering his time and thoughts. The experiment will take place in the last week of July and the first week of August. If you are interested in participating, or you have any questions, contact us via telephone (tel. 040-2743003) or email at [email protected] Evangelos Karapanos

54

Scenarios Tom and Karen, 36 and 32 years old, live in Eindhoven, Netherlands. It is 10 o’clock in the evening and they are watching their favorite movie in the living room. However, Karen is getting tired so they prefer to watch it in their bedroom. So, Tom transfers the session to the TV set placed in the bedroom and they continue watching the movie there. Peter is on his summer vacation with his wife and their two kids, enjoying freedom in the place where they live. However, peter has to inspect a colleague’s paper before he submits it to a conference, while his wife is calling him to the garden to enjoy the sun and see their children playing. So, Peter transfers the session to his tablet pc and continues working in the garden together with his family. Dan is a 16-year-old student at London, enjoying his weekend freedom as a student. He is writing a message to a friend, while he is laying back on the couch in the living room. However, the screen on his mobile phone device is frustrating him as it is too small and forces him to hold his mobile phone in a not ergonomically optimal position. So, he prefers to transfer his session on the TV set placed on the living room while the keypad of the mobile phone remains active. Therefore he takes advantage of a lean-back interaction with a big screen while he types in via his mobile phone’s keypad. Philip is organising his schedule on his tablet pc, and he receives a call from the later. So he suspends the session, which will be maintained as it is until Philip recalls it. company where he works. So he has to go immediately to the office, but he wants to continue on his schedule In the living room a family is watching TV. One family member uses a handheld screen for searching an alternative program. This can be by zapping, using an Electronic Program Guide, or scanning the content of a Personal Video Recorder. When he has found a suitable program, he proposes the alternative to the watching family members: he creates a PIP (Picture In Picture) window to show his proposal. After acceptance, the proposed program is taken over, either immediately, or by setting a timer, if the start time of the selected program is in the future. Paul stops working on his computer in order to go to the bathroom. In the meanwhile, his wife reserves the computer for an important task. Paul returns, and as the computer is reserved from his wife, he starts working on their tablet pc, from where he gains access to the most recently accessed activities in several devices

55

56