Technical note

Technical Note PR-TN 2009/00637

Issued: 11/2009

Experiencing Control

G. Monaci; R.A.C. Braspenning; B.W. Meerbeek; P. Bingley; R. Rajagopalan; M. Triki Philips Research Europe

Unclassified Koninklijke Philips Electronics N.V. 2009

PR-TN 2009/00637

Authors‟ address

Unclassified

G. Monaci

HTC36-01

[email protected]

R.A.C. Braspenning

HTC36-01

[email protected]

B.W. Meerbeek

HTC34-51

[email protected]

P. Bingley

HTC34-61

[email protected]

R. Rajagopalan

HTC36-01

[email protected]

M. Triki

HTC36-02

[email protected]

© KONINKLIJKE PHILIPS ELECTRONICS NV 2009 All rights reserved. Reproduction or dissemination in whole or in part is prohibited without the prior written consent of the copyright holder .

ii

Koninklijke Philips Electronics N.V. 2009

Unclassified

PR-TN 2009/00637

Title:

Experiencing Control

Author(s):

G. Monaci; R.A.C. Braspenning; B.W. Meerbeek; P. Bingley; R. Rajagopalan; M. Triki

Reviewer(s):

IPS Facilities, Sarroukh, B.E.

Technical Note:

PR-TN 2009/00637

Additional Numbers:

ID671374

Subcategory: Project:

Experiencing Control (2008-324)

Customer:

Keywords:

User interaction interaction

concepts,

Abstract:

This report describes the activities carried out in the first part of the Experiencing Control project (2008-324). The guiding idea of the project is to make control part of the experience, exploring new interaction solutions for complex, engaging interactions with Philips devices in the living room. In particular we focus on a collaborative content creation scenario, for which a concept prototype, the Q-ball, has been built and demonstrated.

Conclusions:

In the Interactive Living field we see that our devices are becoming connected to Internet. This means an increase in the amount and diversity of content is coming to these devices compared to few years ago. This presents a challenge for the way we as Philips offer interaction possibilities for devices, since people are used to interact with such content from the PC. However the interaction solutions from that domain (keyboard and mouse) do not match well the environment and setting in which the Philips consumer devices are used. A PC is a single user device, while Philips, on the other hand, brings people together and that is exactly what we want to do for interaction as well, sharing the interaction. And we want to do it in such a way that the interaction becomes part of an enjoyable and fun experience. Philips is uniquely positioned to leverage its stronghold in the living room stemming from the TV domain and to create successful „social interaction solutions‟. The Q-ball concept is a first prototype to enable collaborative interaction in a comfortable (relaxing) setting for co-browsing, co-selection and co-manipulation of content databases. It addresses the trends mentioned above and fits well with the Philips brand positioning and product portfolio.


interaction

technologies,

human

machine

iii

Unclassified

PR-TN 2009/00637

Contents 1.

Introduction .......................................................................................................................... 6

2.

Workshop ............................................................................................................................. 9

3.

4.

5.

2.1.

Goal and setup ............................................................................................................ 9

2.2.

Results ......................................................................................................................... 9

2.3.

Creating content scenario ......................................................................................... 10

Project concept: the Q-ball ............................................................................................... 13 3.1.

Q-ball prototyping ...................................................................................................... 13

3.2.

Applications ............................................................................................................... 15

Benchmark: Technologies enabling collaborative content creation ........................... 17 4.1.

Multi-player gaming solutions .................................................................................... 17

4.2.

Collaborative content creation and manipulation ...................................................... 18

4.3.

Ball-shaped controllers .............................................................................................. 19

4.4.

Conclusions from technology scouting ...................................................................... 20

Relevance for Philips and Conclusions .......................................................................... 21

Bibliography............................................................................................................................... 23 Appendix A................................................................................................................................. 24


5

PR-TN 2009/00637

1.

Unclassified

Introduction

There is a clear trend towards an increasing amount of technology in the home. Consumers experience an increase in both the number of the devices, their complexity, and the variety in interaction modalities for controlling them. In particular in the living room, a discrepancy is emerging between the way people interact with their advanced devices, and the actual user experience the device is trying to deliver. While a user effectively controls the complex functionalities of her/his PC sitting at the desktop, issues arise when the same advanced functionalities are brought into the living room. First of all, the living room is an intrinsically social environment, where several activities are carried out by several persons at the same time. The typical 1:1 relationship between the user and the PC cannot be assumed in the living room setting. Besides, we assume that the interaction style with devices in the living room is significantly different from the interaction one has with the PC. The atmosphere in the living room is typically relaxed and laid-back (when watching TV for example) or rather exciting and enjoyable (when playing video games or sharing moments with friends and family). We believe the interaction with devices should match the overall user experience the device aims to deliver. In this line of thinking, we assume that in the living room of the future a keyboard and mouse might not be desirable, as these are 1:1 communication systems and strongly associated with a working environment, from which one might want to retreat once back home. Alternative input modalities and interaction styles have to be designed. Starting from these observations and assumptions, how can we design optimal interaction styles between users and devices in the living room? According to Rozendaal (1), an optimal experience in interaction with products is achieved through optimizing richness and control given the product, the person, and the task, which have the following characteristics: Product: o Amount of possibilities. o Amount of sensorial stimulation. o Connections between product features. Person: o Level of expertise that is considered to increase with time. Task: o Having an experimental or goal-directed orientation on interaction. o Amount of time available to pursue goal. The control that a user experiences while interacting with a device is an important factor in the overall user experience. Skinner defined the „Experienced Control‟ as the process of action and action feedbacks in which an individual intentionally exerts effort towards a goal and can feel the energy of the effort transmitted into the environment to produce the outcome (2). Important factors influencing control are: the amount of choice available to the user, the information that is available, and the predictability of the outcome. The guiding idea of the Experiencing Control project is to make control part of the experience. From the above-mentioned considerations, it emerges that enjoyable, rich control should match or enhance the level of engagement that the user wants to have with the overall application/experience. This involves not only a new “skin”, but a radical new thinking of interaction styles and the technologies enabling them. Think for example of the successful Wii console. When playing with the Wii, the experience is enjoyable and engaging, with high hedonic valence, which is matched by an engaging control paradigm. The same level of engagement and fun in the control is probably not needed when setting the heating system. In this case a more pragmatic approach is required, where the temperature is set quickly and easily. This suggests that the richness of the interaction should reflect the richness of the experience: Low for simple controls like channel zapping, but for instance also automatic turn-off if there is nobody watching (presence detection). 6


Unclassified

PR-TN 2009/00637

High for complex controls, e.g. multi-device, browsing, viewing, manipulating large content databases. Rich interaction: multi-modal, multi-sensorial, variety in input and output, etc. In the Experiencing Control project we focus on interaction solutions for high complexity tasks. Reasons for this choice are manifold: These tasks typically cause higher user frustration. They involve engaging and exciting interaction. Devices with more advanced functionalities are more and more common in the living room. We can address multi-device and cross-device interaction tasks. These interaction solutions will enable strong differentiation for Philips. In this project, we explore new interaction solutions for complex, engaging interactions with Philips devices in the living room. The way we interact with technology in general or with Philips devices in a living room setting more in particular, is undergoing some profound changes. The first lines of the Microsoft Research report on the 2-days workshop “HCI 2020” (3) state: “The environment we live in has become suffused with computer technologies. They have created change and continue to create change. It is not only on our desktops and in our hands that this is manifest; it is in virtually all aspects of our lives, in our communities, and in the wider society of which we are a part”. In this workshop, five major trends in our interaction with technology emerged: Digital devices are presented to us in other ways: o No more single interface, but multiple in various forms. o In our personal spaces or even embedded in us. Growth in techno-dependency: o Use technology in almost every aspect of our lives. o Technologies become more intelligent and autonomous. Growth of hyper-connectivity: o More time and effort spend on being in touch. o Global communication. End of the ephemeral (temporary): o Desire to capture more information about our lives. Growth of creative engagement: o Technology to work, play, and express ourselves in new ways. These trends suggest that more and more digital devices are going to be present in our living rooms, they will come in different, unconventional forms, they will be interconnected between them, they will be personal and personalized and they will be used to express our-selves, i.e. we will not passively experience these devices, but we will rather use them to create things and concepts, to play and to interact. In a study clearly targeting the living room setting and focusing on the interaction with the principal technological device traditionally present in the living room, the TV (4), some major trends emerge: Technology becomes part of this personal space only when it supports the feeling of being at home, to be private and to relax. TV viewing is strongly related with other activities, doing other things in the background or doing things at the same time, like watching TV and talking to somebody on the phone.


7

PR-TN 2009/00637

Unclassified

TV is perceived as a medium for recreation and information, depending on the context, including time as a major contextual factor. Remote controls are often perceived as unusable and participants complain about the large number of multiple remote controls in their homes and their complexity. From these observations, it emerges that while technology has still not fully pervaded the living room, still the conventional TV viewing is also changing, and it is merging with other activities and it is integrated with other devices. A similar trend emerges from the Technographics Survey of Forrester Research (5) on media fruition. TV watching and Internet-mediated social interaction are merging into a single living room experience: 48% of Europeans aged 12-24 sometimes watch TV while using the Internet; 38% of Europeans 25 and above do so. Of the people that sometimes watch TV while using the Internet, 60% (12-24y) respectively 34% ( 25y) regularly talk to others online about topics unrelated to the broadcast. From these studies it emerges that in the future the interaction with technology in our living rooms will be more and more frequent, and already today TV viewing is changing into a more sophisticated and interactive process involving internet browsing for example. The growth of hyperconnectivity will involve that more time and effort will be spent on being in touch. We expect that this social aspect of interaction with technology will become more important in a living room scenario, and will be a central topic of the project. Finally, the desire to capture more information about our lives, to personalize our experiences, to be creative and engaged, to use technology to work, play, and express ourselves fits particularly well with the project goal of designing interaction concepts and technologies for engaging, complex and advanced tasks in the living room.

8


Unclassified

2.

PR-TN 2009/00637

Workshop

2.1. Goal and setup We organized a workshop in which the project members and interaction experts from Research, Design, Consumer Lifestyle, Lighting and AT Labs participated. The goal of the workshop was to create and discuss scenarios that describe future interactions of people with technology in their living room in line with the project vision (see Figure 1).

Figure 1: Project vision Experiencing Control: making control part of the experience. The workshop consisted of two parts. In the first part, the participants created a list of (emerging) activities in the living room. For each activity, a list of tasks and interactions that are currently needed were described. Furthermore, the optimal experience for this activity was depicted in a group mindmap. During the second part of the workshop, the participants were asked to think how to make the interactions / control tasks match or enhance the desired experience for a number of selected activities and write an interaction scenario, focusing on interactions between user, context, and devices.

2.2. Results The output of the workshop was a set of seven scenarios selected at the end of the brainstorm based on the votes of workshop participants. These scenarios were: 1. 2. 3. 4. 5. 6. 7.

Eating with friends. Watching TV. Relaxation. Socializing with friends (physically present). Information gathering for decision making. Content creation. Socializing with friends (not physically present but connected remotely).

Each scenario has an associated list of tasks, a typical usage scenario and a mind map. An example of mind map for the content creation scenario is shown in Figure 2. All seven scenarios are included in Appendix A. Koninklijke Philips Electronics N.V. 2009

9

PR-TN 2009/00637

Unclassified

Figure 2: Example of mindmap elaborated during the workshop. The scenarios were ranked by project members based on their: Technical feasibility. Business relevance. End-user value. Innovativeness. Match with the project goals. „wow factor‟. Among the seven scenarios, the project members judged the content creation scenario to be the most interesting and suitable to pursue the project goal of defining new, engaging interaction paradigms and technologies. The project members decided to consider a collaborative content creation and sharing context. Reasons for that choice were manifold: Content creation and personalization is one of the main trends in interaction with technology and there are few solutions tackling this aspect. The scenario allows developing concepts for social interaction with other people. This is a very relevant trend as well, in particular in the living room scenario. Again, there is a lack of solutions in this field. Collaborative content creation is a naturally engaging, enjoyable, enriching activity. There is a clear need for new interaction paradigms and technologies enabling engaging, rich interaction.

2.3. Creating content scenario In the creating content scenario five main activities were identified in the mind map: 1. 2. 3. 4. 5.

Sharing content (including up- and downloading content). Collecting, editing, and merging content. Exploration of talents and content. Reliving memories and shared experiences. Engagement: all participants should feel involved.

The mind map suggested that content creation involves a multi-user aspect (simultaneously browsing and editing content) as well as a cross-device aspect (content is retrieved from multiple sources). Based on these observations, on the relevant trends foreseen and on the relevance for 10


Unclassified

PR-TN 2009/00637

Philips, a detailed creating content scenario was elaborated:

A few weeks after a great holiday in South-east Asia, John invites his friends to share some nice photos, videos, and memories in the comfort of his living room.

Eric takes his camera and laptop to John‟s place, as he has some photos and videos stored on them that he would like to share with his friends. Being the first to arrive, he wants to give John a „preview‟ of his best shots while they are waiting for the others. As it‟s inconvenient to use the small LCD screen of his camera, Eric picks up the remote of John‟s TV, pointing first at his camera and then at the TV to transfer the view (UI and photo) from the camera to the TV. A short while later David arrives. He made some short videos of the joint trip with his smart phone, which he now wants to share with David and John. He starts up the media browsing application on his phone and points at John‟s TV to show the application and video on the TV screen. John, Eric, and David are browsing together through the videos that David recorded with his phone, sharing and reliving the good moments of their holiday. Eric notices that some of David‟s short videos would perfectly fit with the holiday movie he has been editing over the past week. He wants to explore the possibility of merging his movie with the material of David. To not bother John and David with the detailed movie editing process, he copies the movie browsing application to his laptop by selecting it with the remote on the TV and pointing to his laptop.


11

PR-TN 2009/00637

Unclassified

Making a compilation fun for all

Peter couldn‟t join for the evening as he lives a two-hour drive away, but he has the time and mood to join his friends through his Net TV. While communicating with his friends via video chat, he can watch the media that is shown on John‟s TV, share his photo and videos, and even interact with the „shared‟ media browsing application using his own remote control.

12


Unclassified

3.

PR-TN 2009/00637

Project concept: the Q-ball

It is clear that to enable such a scenario, new technological solutions are needed. Keeping in mind the project scope, which is to define new interaction concepts and supporting new technologies, here we do not tackle the challenges related to remote communication (cobrowsing etc.) and multi-devices integration and communication. Furthermore, the interaction concept and associated technologies should take into account the context of the foreseen collaborative content scenario: Non-professional setting, location is the living room. Private (family/friends scope). Enjoyable, engaging, stimulating, but also natural and not too complex (not gaming scenario). The project members decided to realize a collaborative content creation interface where the interaction is made possible by a set of soft squeezable balls that can each concurrently control an application. We call the prototype Q-ball. The design, look and feel of the Q-ball should fit with the image of Philips. Thus, it should not be too toy-like, but be a design gadget, pleasant to see, touch and feel. Its form factor should stimulate people to play with the Q-ball also in a lean-back, relaxed situation.

Figure 3: Users enjoying interaction with Q-balls There are several advantages of having a tangible interaction device. First, the Q-ball can embed all kind of sensors. No extremely complex and expensive sensors are required, but only robust, cheap, well-known hardware such as accelerometers and magnetic field sensors. Second, feedback mechanisms like lights, vibrations, temperature changes can be embedded in the Q-ball, enabling rich feedback which is essential in multi-user interaction. Multi-user feedback is a complex problem to face with solutions based only on device-less control (correspondence problem, robustness to ambient conditions etc.). Finally, a set of Q-balls can be sold separately as a design gadget for Philips products, allowing Philips to have a strong differentiator and to earn larger margin on such devices.

3.1. Q-ball prototyping Keeping the above mentioned attributes of the Q-ball in mind, various existing hardware platforms were considered, namely the Sun SPOT (http://www.sunspotworld.com/), the Philips piNode (6) and the Accelerometer-hardware from a Nintendo Wii-Nunchuk Koninklijke Philips Electronics N.V. 2009

13

PR-TN 2009/00637

Unclassified

(http://www.nintendo.com/wii/what/controllers). Important considerations while deciding a platform were the size and the ergonomics of the tangible prototype. The Wii-based set-up was chosen accordingly for its ease of interfacing, cost, and compactness, making it possible to embed it within a simple tennis ball (see Figure 4).

(a)

(b)

(c)

Figure 4: Q-ball prototypes. (a) Set of 4 Q-balls, (b) Tennis ball with prototyped Accelerometer-hardware, (c) Embedding of hardware within the ball. The latest versions of the Nunchuk can be connected wirelessly with the main Wii Remote (Wiimote) controllers. The Wiimotes can then be accessed via Bluetooth on any WindowsPC (see Figure 5). Using input emulators like GlovePIE (Glove Programmable Input Emulator created by Carl Kenner http://carl.kenner.googlepages.com/glovepie), which include support for Wiimote, a range of Microsoft Windows applications can be controlled using GlovePIE scripts. Like the Wiimote, the Nunchuk provides a three-axis accelerometer for motion-sensing and tilting, which can be translated (mapped) onto any application (control PC games, for example). Furthermore, the hardware within the ball was positioned to also enable the usage of the „C‟ button of the Nunchuk as an option to squeeze (click) the Q-ball thereby triggering a command/execution within an application. Two applications were interfaced to the Q-ball prototype, which are discussed in the next section.

Figure 5: Basic structure of Q-balls prototype.

14


Unclassified

PR-TN 2009/00637

3.2. Applications The Q-balls were used to control two different applications involving media content manipulation, exploration and creation. One application controlled was the SpacePlayer, a spatial audio-visual interface for exploring media collections developed by Aki Haarma within the AMUSE project. A snapshot of the application is shown in Figure 6. In the interaction several users navigate in the music space, each one contributing to the direction of the browsing. Every user can select at any time a piece of music which goes into the playlist which is built in a collaborative and playful way. Browsing is enhanced with audio feedback as the users can hear the albums “fly by”; the music of an album fades in when the album comes “closer” to the user and fades out when an album moves away. The albums were ordered by year (“recent” at the front and “old” at the back) and by tempo (“slow” at the left and “fast” at the right).

Figure 6: A snapshot of the spatial audio-visual interface of the SpacePlayer. Four Q-balls were interfaced to a typical WindowsPC after pairing their respective Wiimotes via Bluetooth. GlovePIE scripts were written which map the rotation of the Q-ball about the Z-axis (yaw) to the horizontal panning within the music space, and the rotation about the X-axis (roll) to zooming in/out within the music space. The squeeze (click) option was used to trigger the selection of a particular album in the music space, while hovering over it. The selected album is then added to the playlist pane, on the right side of the screen. Users (see Figure 7.a) were given individual Q-balls with the task of choosing playlists of various music albums. Every user had overall control of the entire application, thereby stimulating a very social and playful mode of music browsing. This demonstrator was shown at Innovate! 2009. The concept was also adapted for a second application involving collaborative creation of photocollages. The Q-ball setup was interfaced to a typical online photo-album service (www.picture2life.com) and the three users (illustrated in Figure 7.b) were asked to make a collage from a set of holiday pictures uploaded online. The rotational movements of the Q-ball were mapped onto typical mouse-like navigation behaviour on the screen. In other words, the yaw and roll were translated to the horizontal and vertical movement of the mouse, and a combination of two resulting in diagonal movement of the mouse. The squeeze (click) option triggered a left-click (select) operation of the mouse, which when done while hovering over a picture in the online album would select it to be added to the collage. In a next step the selected pictures could then automatically be combined into a picture collage.This also brought about an enthusiastic and fun feeling to interacting within such an otherwise passive, living-room scenario. Koninklijke Philips Electronics N.V. 2009

15

PR-TN 2009/00637

Unclassified

The collaborative photo-collage creation demonstrator was shown at the Lifestyle Experience Research day 2009.

(a)

(b)

Figure 7: Applications controlled with the Q-balls: (a) Co-browsing of music playlists using SpacePlayer (b) Collaborative photo-collage creation from online photo albums. The feedback received during Innovate! and LER-day showed enthusiasm for the concept, mainly because of the playfulness and low thresholds for multi-user interaction. The interest for this type of new interaction paradigm is witnessed also by the very recent proposition of the interactive ball concept by other companies and research institutes, as we will discuss in the next section. While the impressions received at research fairs are important and confirm the validity of the idea, at this stage a rigorous user study with the Q-balls prototypes should be performed to validate the concept and tune a series of parameters, from the different control options to the feedback options.

16


Unclassified

4.

PR-TN 2009/00637

Benchmark: Technologies enabling collaborative content creation

In the market and in literature there are several examples of multi-user and collaborative interaction. In the following we briefly present an overview of these products and technologies.

4.1. Multi-player gaming solutions In the video games market there are several multiplayer solutions, although they typically do not involve collaborative creation aspects. Most of the existing solutions in this area are PC based desktop solutions which are not suitable for the social living room setting we have in mind. However in recent years very relevant ideas and products have popped-up. In 2003 Sony launched the EyeToy (see Figure 8.a) for its Play Station 2. The EyeToy is a camera that uses computer vision algorithms to segment players‟ silhouettes from the video and interpret movements and gestures. The EyeToy enables multiplayer interaction in an unconventional living room setting, increasing the fun factor of gaming by having the users themselves be the characters. However, this solution had several drawbacks that limited its diffusion. For example, people segmentation accuracy is greatly influenced by the illumination conditions, distance, occlusions. The EyeToy only supports simple games and commands, since the conventional video camera cannot capture complex body gestures. Furthermore, the screen can get very crowded when 4 people play at the same time. As in almost all gaming settings, there is no real concurrent interaction, but rather sequential interaction.

(a)

(b)

(c)

(d)

Figure 8: Social multi-player gaming solutions: (a) Sony EyeToy, (b) Nintend Wii, (c) MERL table-top multi-player strategy game, (d) Consigalo. Koninklijke Philips Electronics N.V. 2009

17

PR-TN 2009/00637

Unclassified

More recently, Nintendo presented its hugely popular Wii (shown in Figure 8.b). The Wii uses smart controllers with a camera, a 2D gyroscope (optionally a 3D accelerometer can be added with the Nunchuk) and feedback systems (vibration and sound). It is well suited for a living room scenario, although the device has a strong gaming characterization. Despite the huge success of the Wii, we can still highlight few drawbacks of the solution. First, there is no intuitive correspondence between players (positions, characteristics…) and characters. Further, there is no real simultaneous interaction between players. For example, in the tennis game users have control one after the other. However, the Wii controls can be easily hacked and they are used for many diverse applications, including a collaborative art demonstrator (7). MERL (8) developed a research prototype featuring a complex table-top multi-player strategy games (see Figure 8.c). The interaction is based on touch, speech and gaze. It is structured as a table game, thus if gets quickly crowded and it is not suitable for a living room setting. Again, there is no intuitive correspondence between players (positions, characteristics…) and characters and the Interaction between players is achieved using interleaved commands. Another research prototype is Consigalo (9). Consigalo (Figure 8.d) is an interesting academic demonstrator which investigates ways to enrich the feedback given to the users and ways players can interact among each others. The prototype has a dual-sided immaterial display with a complex setting. It features simple games but with advanced interaction concepts. Users are the characters, not their silhouette extracted from a video. There is real-time interaction between users and between users and game. There is an advanced localized audio and video feedback mechanism where video information is used to localize the players (3D tracking with stereo cameras). Interestingly, it has been noted that this feedback mechanism is very important in multi-user setting, so that people can understand who is interacting and how.

4.2. Collaborative content creation and manipulation In recent years several multi-user and multi-device solutions have been proposed by major companies (Philips, Sony, Mitsubishi and Microsoft among others) for interaction on the same “desktop”. Typically the setting is professional, thus focusing on productivity, and interaction concepts are not very advanced. Besides, these products and prototypes (e.g. Philips Entertable, Microsoft Surface or GestureTek Illuminate) are conceived as expensive smart table solutions, which do not fit with the consumer, living-room focus of this project. However, recently some innovative collaborative concepts have started to pop-out, coming from world-class research institutes and companies. A collaboration between Stanford and Harvard research groups produced a collaborative gestures platform (10) that uses a multi-user, touch-sensing input device (see Figure 9.a). This is a table-top structure, thus is not suitable for a living room application. However, it features some interesting concepts. For example, gestures of more than one user contribute to a single, combined command. Actually, the goal of the prototype is to enhance teamwork, requiring combined, involving interactions and gestures. Overall however the interaction looks cumbersome and little natural. Researchers at MIT and Tufts University developed a collaborative video editing system (11) (Figure 9.b) that is based on video object recognition algorithms and tangible artefacts as tokens to control the editing application. The researchers involved in the projects decided to use nice smart artefacts to make the interaction fun and informal. Different users place together the artefacts on a surface, and the content and disposition of the artefacts determine the final editing. ATR Media Information Science Research Labs presented an interesting prototype for the collaborative annotation of pictures in a multi-user environment, called SenseWeb (12) (Figure 9.c). SenseWeb is a collaborative environment to browse and edit images, it uses touch and gesture based interaction. It is a road-show prototype, with a huge wall screen usable simultaneously and concurrently by several users.

18


Unclassified

PR-TN 2009/00637

(a)

(b)

(c)

(d)

Figure 9: Collaborative content creation solutions: (a) Collaborative gestures platform, (b) Collaborative video editing, (c) SenseWeb, (d) Lunar SoftRemote. The design studio Lunar proposed in 2008 SoftRemote (13), a design concept for gesture based TV remote controls (see Figure 9.d). It is a set of soft squeezable balls that can all concurrently control a TV. Lunar foresees remote controls as nice objects that make control relaxed and fun, but still powerful. Each remote has a unique color and a unique correspondence on the screen, so that the correspondence problem in multi-user interaction is simply overcome. In this concept, users can share, record, vote, take snapshots, save, and browse content using the squeezable balls. This concept is interesting and very close to our Q-balls; however there is no information or suggestion about possible ways of implementing a prototype.

4.3. Ball-shaped controllers While the Experiencing Control project was approaching its end, two concepts of ball-shaped, soft, squeezable controllers were presented. Interestingly, research centers and companies are proposing new solutions to control technology in the living room. In November 2009 a squeezable game controller called Blobo (http://www.bloboshop.com) was presented (see Figure 10.a). Blobo is roughly the size and shape of a golf ball, and “it measures hundreds of times per second the prevailing magnetic field and its direction, acceleration and its direction, the air pressure inside the ball, as well as other physical quantities”. This information is communicated to the PC via Bluetooth and used to controlled compatible games, like the Fun&Fit game package. At ACM Siggraph Asia, in December 2009, students of the University of Tsukuba (Japan) presented Puyo-con (http://www.entcomp.esys.tsukuba.ac.jp/puyo), a soft, squeezable, ballKoninklijke Philips Electronics N.V. 2009

19

PR-TN 2009/00637

Unclassified

shaped input peripheral (see Figure 10.b). Puyo-con has 14 pressure sensors and a three-way acceleration sensor. This information is sent to the PC using Bluetooth to control the cursor, pretty much like one Q-ball prototype.

(a)

(b)

Figure 10: Blobo (a) and Puyo-con (b), two recent ball-shaped squeezable input peripherals. Neither of these devices targets explicitly a collaborative, social scenario and they are still mainly conceived as gaming input peripherals. However the interaction paradigm, the technological solution and the form factor are similar to those developed for our Q-balls. The buzz created by these devices in popular technology blogs reveals the interest for new, alternative, more intuitive and playful interaction concepts.

4.4. Conclusions from technology scouting There exist several examples of collaborative content creation and sharing solutions, but these are typically desktop solutions targeting a professional setting. There are also few nonprofessional examples that target a more entertaining, laid-back scenario. These prototypes are all coming from academia or research institutes. By far the majority of multi-user interaction technologies consist of table-top surfaces or very large screens with close-by touch-like interaction. These solutions do not fit with the living room scenario: new interaction concepts and supporting technologies need to be conceived. Several studies highlighted the importance of having a clear correspondence between screen character and actual user in the multi-user scenario. Localized feedback (audio, video, etc.) is important in multi-user setting so that users understand who is doing what and how they are interacting. This is one of the main aspects that will have to be investigated in the future. No major company or research center has proposed a solution for collaborative content creation in a social, multi-user setting. However, several interesting concepts that investigate multi-user content creation and browsing (e.g. the Lunar Softremote) and social, playful control of devices (e.g. the Blobo) start to pop-out.

20


Unclassified

5.

PR-TN 2009/00637

Relevance for Philips and Conclusions

Interaction technology and design is a wide multi-disciplinary field, important for virtually all consumer products of Philips. In this project we have focused on the areas where new interaction technology is needed and can make a big difference. This is typically the area of „rich‟ interaction (complex controls, see Chapter 1) and can be found primarily in the traditional consumer electronics domain (TV, AVM) or Interactive Living theme as it is called now. For Domestic Appliances it is most often sufficient to limit the interaction to a few buttons and status lights, and hence this was not the focus of this project. In the Interactive Living field we clearly see that devices are becoming connected to Internet, examples are the NetTV and the Streamium audio product line. This leads to an increase in the amount and diversity of content that is coming to these devices compared to a few years ago. This presents a challenge for the way we as Philips offer interaction possibilities for devices, since people are used to interacting with such content from the PC domain. However the interaction solutions from that domain (keyboard and mouse) do not match very well with the type of environment and setting in which the Philips consumer devices are used. A PC is a single user device, used in a 1:1 relationship between user and device. Philips, on the other hand, brings people together and that is exactly what we want to do for interaction as well, sharing the interaction. And we want to do it in such a way that the interaction becomes part of an enjoyable and fun experience. Furthermore, the atmosphere in the living room is typically relaxed and laidback (when watching TV for example) or rather exciting and enjoyable (when playing video games or sharing moments with friends and family). Philips is uniquely positioned to leverage its stronghold in the living room stemming from the TV domain and to create successful „social interaction solutions‟. The Q-ball concept is a first prototype to enable collaborative interaction in a comfortable (relaxing) setting for co-browsing, co-selection and co-manipulation of content databases. It addresses the trends mentioned above and fits well with the Philips brand positioning and product portfolio. To summarize the relevance for Philips: Market Trends o

Internet in combination with our devices is a great opportunity for us. One opportunity is bringing people together around content.

o

New interaction technology can be differentiator, e.g. the recently introduced multi-touch technology used by Apple.

o

The onset of NetTV demands rich interaction

o

People own larger and larger content databases

o

Social activities and contacts are under time pressure.

NPS o

UI and Interaction technology are the interface to the customer, doing it right can lead to high NPS scores (like e.g. for Apple), doing it wrong can make for a significant NPS detractor.

Brand Pillars: the concept of collaborative interaction fits very well with the Philips brand and its pillars o

Easy to Experience: intuitive control without cumbersome devices nor passing them around.

o

Designed Around You: brings you and your family and friends together

o

Advanced: powerful sensor technology makes interaction simple.

Positioning


21

PR-TN 2009/00637

Unclassified

o

The PC is a single user device and PC style interaction is not convenient and suited for (social) living room settings. Hence collaborative interaction is a means to differentiate ourselves from PC-type companies such as Microsoft and Apple.

o

Philips is uniquely positioned to leverage its stronghold in the living room stemming from the TV domain and to create successful „social interaction solutions‟.

Proposition o

The Q-balls can be sold as accessory to a NetTV. This way it puts less pressure on the price of the NetTV, and consumers can decide for themselves how many Q-balls they need

o

The Q-balls could be combined with a video telephony function on the TV as part of a larger special package. This could be introduced as a special model NetTV (at a higher price) with the marketing focussed on the social aspect, bringing people together both in the living room, as well as at a distance.

Opportunity o

We see from the state-of-the-art that collaborative interaction for living room spaces is largely unaddressed, giving still quite some room for differentiating propositions and unique IP.

To further explore collaborative user interaction the following steps could be taken: Develop a dedicated application that better exploits the interaction abilities of the Q-balls. Perform a user study with the Q-balls prototype(s). Investigate alternative algorithms for more intuitive translation of Q-ball movements into application actions. Develop dedicated hardware for Q-balls (e.g. based on PI-nodes) that enables providing feedback to users (e.g. via vibration or lights). Explore business opportunities for collaborative interaction. Explore alternative interaction concepts for collaborative interaction (i.e. other than Qballs).

22


Unclassified

PR-TN 2009/00637

Bibliography 1. Rozendaal, Marco. Designing engaging interactions with digital products. TU Eindhoven. 2007. PhD Thesis. 2. A guide to construct of control. Skinner, E. 3, 1996, Journal of Personality and Social Psychology, Vol. 71, pp. 549-570. 3. Richard Harper, Tom Rodden, Yvonne Rogers and Abigail Sellen. Being Human: HumanComputer Interaction in the Year 2020 . Cambridge, England : Microsoft Research, 2007. 4. Trends in the Living Room and Beyond: Results from Ethnographic Studies Using Creative and Playful Probing. R. Bernhaupt, M. Obrist, A. Weiss,E. Beck, M. Tscheligi. 1, 2008, ACM Computers in Entertainment, Vol. 6, pp. 5:1 - 5:23. 5. Research, Forrester. European Technographics Media, Marketing, And Social Computing Online Survey, Q3 2008. s.l. : Forrester, 2008. 6. Acht, V.M.G. van, Bongers, E.G.J.M. and Lambert, N. The pi-Node. 2009. PR-TN 2007/00513. 7. WiiArts: Creating collaborative art experience. Hyun-Jean Lee, Hyungsin Kim, Gaurav Gupta, Ali Mazalek. 2008. International Conference on Tangible and Embedded Interaction. 8. MULTIMODAL MULTIPLAYER TABLETOP GAMING. Edward Tse, Saul Greenberg, Chia Shen, Clifton Forlines. 2, 2007, ACM Computers in Entertainment, Vol. 5. 9. An Immaterial, Dual-sided Display System with 3D Interaction. Olwal, A., DiVerdi, S., Candussi, N., Rakkolainen, I., and Höllerer, T. 2006. Proceedings of IEEE Virtual Reality Conference. pp. 279-280. 10. Cooperative Gestures:Multi-User Gestural Interactions for Co-located Groupware. Meredith Ringel Morris, Anqi Huang, Andreas Paepcke, Terry Winograd. 2006. CHI 2006. pp. 12011210. 11. The Tangible Video Editor:Collaborative Video Editing with Active Tokens. Jamie Zigelbaum, Michael S. Horn, Orit Shaer, Robert J.K. Jacob. 2007. International conference on Tangible and Embedded Interaction. pp. 43 - 46. 12. SenseWeb: collaborative image classification in a multi-user interaction environment. Roberto Lopez-Gulliver, Hiroko Tochigi, Masami Suzuki, Norihiro Hagita. 2004. ACM international conference on Multimedia. pp. 456 - 459. 13. Pluralizing the Screen: Converging Gesture, Environment & Interface. Ron Goldin, Alex Rochat, Gretchen Anderson. 2008. International Conference on Designing Interactive User Experiences for TV and Video.


23

PR-TN 2009/00637

Unclassified

Appendix A

1. Eating with friends Interactions / tasks Set table Set background music create/recall playlist Set light (candles…) Select drinks and food Unpack, put food on table Serve clean plates Photos on frame adjusted to guests or theme etc. Take pictures or videos during the dinner Go to kitchen (between courses) Prepare menu on paper (for fancy dinner!) Prevent being disturbed (mute phone, …) Entertainment during dinner Chatting Recall shared memories (photos, videos,…) Serve coffee and tea End dinner, say goodbye Clean the table Get rid of leftovers

Scenario Remco decides to organize a dinner with eight friends he doesn‟t see since a long time. But since there are many tasks involved in the preparation, he gets help from his personal planning device. Before shopping, he easily selects people joining the dinner using his contact list on Facebook. From that he automatically gets some menu suggestions based on their preferences and on what Remco previously prepared to them. Next, Remco also receives suggestions on which music to play. He forwards this list to his music player by a simple “drag-and-drop” gesture using the pointing device. The system has also found a set of pictures with most of friends present. Remco forwards those to his beamer such that it will be played as a slideshow during or after the dinner. The funny thing is that everybody can interact with the slideshow, pause it to discuss and laugh together about the current picture. The planning device also helps Remco to select the light setting for the evening. Based on some general questions that the personal planner asks Remco and the music selection, the lights settings are created.

24


Unclassified


PR-TN 2009/00637

25

PR-TN 2009/00637

Unclassified

2. Watching TV Interactions / tasks (description 1) 1/ take the RC (remote control) 2/ switch on TV

3/ find channel: directed (news, series) browse (movie) Insert disk (movie) 4 / switch on the sound system 5/ set lighting for movie 6/ adjust audio/video settings Desired: Automatic input switching Single RC Automatic adjustment of (sound system/light/video settings) depending on the media type (movie) Better way to brose content (categorized, recommended...)

Interactions / tasks (description 2) Switch on TV Switch on the home-cinema Set lighting Select program : zapping / EPG / list Get drink Zapping / Skipping commercials Planning next program Had control over the remote

Scenario Title: Finding the best program

John returns home after work and starts cooking his meal. While cooking, he wonders what is on TV tonight!? John asked the TV: “is there anything interesting tonight?” Based on John‟s preferences, the TV shows a recommended program and a short description from the EPG. John isn‟t convinced yet and wants to browse through the suggestions. The TV suggests a football match between Holland and Tunisia. Because he knows that his friend Mahdi may be interested and in order to be sure that Mahdi doesn‟t miss this program, he sends a notification. Mahdi receives John message about the football match. Mahdi is glad with this notification, and asks his TV to alert him when the game starts (in 1 hour). The system suggests John a documentary about earthquakes. John is very interested and starts watching: the curtains close, the light dims, and the TV settings adapt automatically to the content. John enjoys the show.

26


Unclassified


PR-TN 2009/00637

27

PR-TN 2009/00637

Unclassified

3. Relaxation Sheet 1: Interactions / tasks: 1. Select appropriate Music – specify „easy listing‟, radio channel or disc 2. Setting the lighting Potentially involve activity that „defocuses‟

Sheet 2: Interaction / tasks: Very solitairy activity Set background Music Dim the light Take a nap Read book, newspaper, … Watch news, TV (passive) Watch kids play Close curtains Switch off Phone (Mute) Set the alarm (Wake-up …) Get a hot drink

Sheet 3: Scenario Title: Relaxation  Powernap chair Scenario: John comes home from work. He does the obligatory chores in the house. Finally time to relax… He sits down in his favorite relax chair (this is one is nicely designed: i.e. not ugly). He needs a powernap. So he starts “nap mode”, by moving the chair in resting position: Music level goes down Lights dim gradually TV fades off Chair goes into comfy mode Telephone mutes automatically After 20 minutes: “Wake up sequence”(lights, sounds, chair  subtle vibrations?) Tea is ready Phone switches back on Chair massage optional 

28


Unclassified


PR-TN 2009/00637

29

PR-TN 2009/00637

Unclassified

4. Socializing with Friends that are present Sheet 1: Interactions / tasks: Get drinks Welcome people Set-up lighting Set-up Music Clean up Close curtains Fresh air control (fragrance) Temperature Fireplace activation Get snacks Bring children to bed Arrange flowers Showing pictures (digitally)

Sheet 2: Interaction / tasks: Make yourself comfortable in couch / chair Offer drinks (“indrinken”) Watch sports Play cards Telling jokes Dancing Select Music to play Get some snacks Talk about old / past memories (shared) Watch vacation video / Photo Play games Planning “going out” activity

Sheet 3: Scenario Title: Socializing with Friends (while present) Scenario: John visits Tom & Linda to watch a soccer match. They catch up with recently visited matches and other interests they have in common (using photos, videos, Music). When the match starts the scene is set for an immersive TV watching experience (using self made “preset” that controls multiple aspects of the room): Lights dim (or change to favorite “club colors”) Surround sound Stadium noises Beer and snacks Photoframes adapt content to theme During halftime break focus shifts from screen to room: o Lights dim up again 30


Unclassified

PR-TN 2009/00637 o o o o

Sound goes down to accommodate conversation Stadium noises continue to “stay in the mood” Get a drink, make a burger, take a leak Discuss the match (replay highlights) – i.s.o. having the TV commentators do the talking

Drawing(s):


31

PR-TN 2009/00637

32

Unclassified


Unclassified

PR-TN 2009/00637

5. Info gathering for decision making Interaction steps (description 1) -

-

-

Turn on PC or Internet-connected device Find related materials present in the home Navigate to different sources of (web-based) information o Finding URL o Select/enter URL Create short overview of information Set lights for good readability Render associated product content (digital samples) to see if you like it o Move trailer to AmbiLight / Home Cinema set o Advertisements o Impressions of locations/places Enter personal preferences to narrow down search / selection results Ask advice from other persons / friends Select candidate products Bookmark and store already selected information / choices / candidate products Make final choice Attach personal notes (judgement) to candidates Share your final decision Buy / book / … the final candidate

Interaction steps (description 2) -

Switch on device Adapt lighting Get drink Get rid of people that are present (unless it‟s a group decision) Search / select program Browse Consult forums Align / influence „stakeholders‟ Share experiences Order List options

Scenario description: “EVA cares” Eddine, wanting to go on holiday, enters the living room and speaks out lout “I want to go on holiday”. Automatically the light and music are adapted and the big screen is switched on. The lovely virtual EVA appears on the screen. She asks about Eddine‟s preferences and guides him through the process of finding and selecting a nice holiday. He is presented with the possibility of making virtual trips (as some kind of preview). Eddine likes books a trip by confirming his choice to Eva; subsequently it‟s paid automatically. His agenda will also be updated accordingly. The scenario description is also applicable to a „virtual holiday‟ situation.


33

PR-TN 2009/00637

34

Unclassified


Unclassified

PR-TN 2009/00637

6. Content Creation Interactions / tasks [multimedia] Share experiences Create photo albums / movies Edition (movies) Uploading (pictures/movies) Downloading (pictures/movies) Share (pictures/movies) Set-up/control lighting Set-up/control ambiance Link ambiance to content Share with community Order and pay

Scenario Title: Making a joint multimedia show of holiday trip

Anna invites her friends over to make a nice joint multimedia show of their holiday trip to Tunisia. Her friends bring their own photos and video cameras with the content recorded during the trip. Attendees put their equipments (cameras, mobile phones, USB sticks, flash cards, etc) in a box near the TV. All the multimedia content is then uploaded to the TV, and automatically sorted in chronological order. The system gathers similar pictures and asks the group what to do: which pictures should be included? Do you want to add a verbal description, some memories or stories (audio tag)? Anna and her friends have a lot of fun while sharing and reliving their holiday experiences. Everybody has a say in what should be included in the end video. The system integrates the selected media in a single presentation. Particular attention is paid to unify image quality (resolution, size, etc) and smooth photo settings (correct for image brightness, sharpness, etc mismatch). While the system displays the combined presentation Anna and her friends can give feedback and make final adjustments. Anna and her friends were very satisfied with their joint holiday trip video. They decided to upload it in their Facebook profiles and share it with other friends. More important, they had lot of fun while making it and relived their vacation experiences.


35

PR-TN 2009/00637

36

Unclassified


Unclassified

PR-TN 2009/00637

7. Socializing with friends (non present) Interactions / tasks Text Use multiple chats in parallel Multi-tasking Start PC Open buddy list Type message Use speech to text to replace typing

Audio Pick the phone Dial the number Switch off noise sources (Music, TV, vacuum cleaner…) Start PC Log on to computer Select contact Check if person is online Call Spend 5 minutes checking “can you hear me?” Push-to-talk (preselected buddies)

Video Get dressed Clean up Check lights Dim ambient sounds Turn on video call device (TV, PC) Log on to account Select contact Check if person is online Call

Spend 5 minutes checking “can you see me?” scenario Mark comes home from work and puts his touch screen phone in the charging station in the living room. The phone displays his main contacts and their status (availability, context, potential activity). He can select one of them or any other person to “start” the communication. He may do this from the device itself or from any other device, potentially through speech. Communication can happen at various levels such sms on the device using a keyboard on the table, phone calls, or video chat, and he can seamlessly change between them without manual setup requirements. He can also switch seamless between a private/personal mode or an ambient communication mode. Finally, he can combine the call with his primary activity (through e.g. PIP). Mark‟s status is also visible to his friends. He can easily change his status or current activity (displayed by any device) and customize/adapt this status and displayed activity to individual friends, including a “wish to communicate” status for individuals. Some status updates should happen automatically. Koninklijke Philips Electronics N.V. 2009

37

PR-TN 2009/00637

38

Unclassified


Technical note

Technical note

Suggest Documents

NOTE TECNICHE - TECHNICAL NOTE

TECHNICAL NOTE

TECHNICAL NOTE

Technical Note

Technical Note

TECHNICAL NOTE

Technical Note

Technical Note

TECHNICAL NOTE

Technical Note

technical note

Technical note

TECHNICAL NOTE

Technical Note #623 - Sixnet

Technical Note

technical note

Technical Note

jaas technical note

Writing a technical note

technical note - USDA [PDF]

Model Performance Technical Note

Technical Note - ACPD

technical note 1432000147 - EJMINT

technical note