Computers in Human Behavior 50 (2015) 148–158
Contents lists available at ScienceDirect
Computers in Human Behavior journal homepage: www.elsevier.com/locate/comphumbeh
Playing with multiple wearable devices: Exploring the influence of display, motion and gender Yubo Zhang, Pei-Luen Patrick Rau ⇑ Department of Industrial Engineering, Tsinghua University, Beijing 100084, China
a r t i c l e
i n f o
Article history:
Keywords: Wearable device Smart watch Smart bracelet Multitasking Uses and gratifications
a b s t r a c t In order to explore the influence of display, motion and gender on the human computer interaction with wearable devices, we recruited participants to wear a smart watch and a smart bracelet to complete tasks including exercise, operating the smart watch and managing fitness information via the smart bracelet. The results showed that females’ needs of information acquisition and emotional experience were more gratified compared with males’ needs in the experiment, and interacting with wearable devices while jogging increased users’ cognitive workload and perceived difficulty and decreased users’ level of flow experience compared with interacting while walking. Despite the superiority of the bracelet with a screen, indicated by the results, it is worth noting that displaying information on a mobile app has a good chance of improving the usage experience of the bracelet without a screen. Ó 2015 Elsevier Ltd. All rights reserved.
1. Introduction The advance of emerging interactive and computing technology has enabled users to step into the era of smart human–computer interaction (HCI). It is no longer a novelty for users to interact with multiple smart devices like tablets and smartphones. Recently the interaction with wearable devices has become the focus of manufacturers and users. Wearable devices are reported to create a massive new market after the wave of smartphones. IMS Research forecasts that there are going to be as many as 171 million wearable device units shipped by 2016 and ABI Research estimates that this market will reach 485 million annual device shipments by 2018 (Ballve, 2013). The wearable devices differ in use scenarios with smartphones and tablets. Since users can interact with wearable devices in various states of motion and the devices themselves are limited in size, how to design the display of wearable devices to match the use scenarios is an important issue. Currently, some smart bracelets like UP by Jawbone (Wikipedia, 2015a) have no display screen and focus on collecting users’ fitness data as a sensor. In contrast, some smart bracelets like Nike + FuelBand (Wikipedia, 2015b) have screens, which can be used to display fitness information. The pros and cons of the two designs have not been deeply examined in empirical studies, and both categories of products have a large market share. Display design is a classic issue in HCI studies ⇑ Corresponding author. Tel.: +86 10 62776664. E-mail address:
[email protected] (P.-L.P. Rau). http://dx.doi.org/10.1016/j.chb.2015.04.004 0747-5632/Ó 2015 Elsevier Ltd. All rights reserved.
and investigating this factor in new scenarios provides an extension to the field’s theories. Users’ motion is another important issue in the ubiquitous computing environment, for users’ interaction with wearable devices can happen almost anywhere at any time. For instance, users’ motion can influence their performance of reading information from smart watches because it is more difficult for people to read in the state of moving than standing still. However, motion is rarely considered as an important factor in traditional HCI studies because people commonly sit in front of a computer to operate it. Another important perspective about wearable devices is the gender difference. Wearable devices embed sensors and computing components into daily clothing or ornaments. Women usually wear more accessories and care more about dressing than men, and their clothing is less likely to have smartphone-bearing pockets compared with men (Elgan, 2013). Hence, it is reasonable to distinguish males and females separately from the beginning of the product design. Besides, users tend to conduct multitasking interaction activities with multiple wearable and portable devices simultaneously. Previous studies have indicated that females outperform males in some multitasking paradigms (Stoet, O’Connor, Conner, & Laws, 2013). Hence, gender difference may exist in the driving factors and usage patterns of wearable devices between male and female users. The objective of this study is to shed light on the role of display, motion state and gender difference in interactions with wearable devices. We care about the usage experience, users’ different gratification levels and their acceptance attitudes toward wearable
Y. Zhang, P.-L.P Rau / Computers in Human Behavior 50 (2015) 148–158
devices. Figuring out the influence of display, motion and gender can benefit the design of wearable devices and provide helpful design suggestions for related manufacturers, which is of both theoretical and practical importance. 2. Related work and research questions 2.1. Display design on wearable devices Well before the appearance of today’s wearable devices, Billinghurst and Starner (1999) pointed out that wearable computers have limited screen space and their output interface design should consider how much user attention is required. Researchers have proposed various ways to output information for wearable devices. For example, ‘Tacton’ (Brewster & Brown, 2004, 2004) is a non-visual way to display information for mobile and wearable devices. It communicates information via tactile output. It is more ambient and peripheral, and demands less user attention than full visual and audio multimedia (Billinghurst & Starner, 1999). Besides replacing one modality with another, aggregating more than one modality to solve the problem of limited display space is also a solution. Brewster, Lumsden, Bell, Hall, and Tasker (2003) propose two ‘eyes-free’ interaction techniques, with the addition of audio feedback, as effective alternatives to visual-centric interfaces. Some manufacturers have also applied for patents related to output on wearable devices, such as Google’s ‘‘Wearable device with input and output structures’’ patent for its Google Glass (Olsson, Martin, Hebenstreit, & Cazalet, 2014). Personal computers and mobile phones have a mature form of output interface, but wearable devices vary in ways of displaying information. Current products for sale vary in information display design. One way is to regard the smart bracelet as a pure sensor and require users to read information via an associated smartphone app, like UP by Jawbone. According to Jawbone’s Vice President of Product Management and Strategy Travis Bogard, ‘‘the lack of screen will induce users to check the associated smartphone app and receive more detailed data and keep pushing toward change’’ (Rubin, 2014). The other design is to show fitness information on the screen of the smart bracelet such as energy consumption (e.g. calories burnt) or the number of steps, like Nike + FuelBand. Although controversies exist in the industrial circles regarding the two designs, there are no studies that compare the two designs and investigate their influence on usage experience and attitude. Hence, the first set of research questions is as follows: RQ 1-1. Which display design on wearable devices better gratifies users’ needs to acquire fitness information?
RQ 1-2. Which display design on wearable devices better gratifies users’ emotional experience (e.g. enjoyment) during the interaction? RQ 1-3. Which display design on wearable devices is perceived as more useful and easier to use?
149
such as walking, running or even cycling. Johnson (1998) once claimed, ‘‘HCI methods, models and techniques will need to be reconsidered if they are to address the concerns of interaction on the move’’ and Wobbrock (2006) has pointed out the importance of considering context in the future of mobile HCI research with the increasing amount of personal computing done away from the desktop. Iwata, Yamabe, and Nakajima (2010) identified two issues affecting users’ attention in the mobile computing environment: situational disabilities and fragmented attention. Situational disabilities refer to the degraded interaction performance due to postural changes and environmental transition. It is conceptualized as SIID, which stands for situationally induced impairments and disabilities (Sears, Lin, Jacko, & Xiao, 2003). SIID can cause higher cognitive workload and divert more attention, which will influence interaction experience and performance of both input and output activities (Barnard, Yi, Jacko, & Sears, 2005; Bastian & Enrico, 2010; James, Thad, Daniel, David, & Scott, 2014; Lin, Goldman, Price, Sears, & Jacko, 2007; Price et al., 2006). From another perspective, interaction activities in motion essentially are multitasking activities because motion itself is a task. In the mobile context, users’ limited attention resources are distributed to multiple tasks according to Wickens’ multiple resource theory (Damos, 1991; Wickens, Sandry, & Vidulich, 1983). Besides the attentional resources expended to multiple tasks, interaction behavior and motion will compete for limited visual attention (Oulasvirta, Tamminen, Roto, & Kuorelahti, 2005). Xie and Salvendy (2000) concluded that in multitasking settings, in addition to the workload attributed to each single task, the management workload, which is the extra mental effort expended to control the concurrent tasks and task scheduling (switching from one to another), also contributes to the overall cognitive workload. Pellecchia (2003) found that devoting effort to cognitive or motor control can influence the other available resources. Hence, interacting with wearable devices while coordinating motion can bring extra management workload. On the other hand, distributing attention resources to different tasks may decrease the level of flow experience, which describes the ‘‘optimal experience’’ when an individual completely concentrates on an activity (Csikszentmihalyi, 1975). If operating the wearable devices causes too much interruption to motion, it is difficult for users to engage in the state of flow. Among current related works, some studies focus on the performance degradation of input and output activities when interacting with portable electronic devices (Barnard et al., 2005; Lin et al., 2007; Price et al., 2006). Studies focusing on smart mobile devices (Conradi & Alexander, 2014) are rare. Some other studies focus on the replacement of or compensation for information modalities of mobile interaction, such as deploying an audio notification system for mobile device users (Iwata et al., 2010) or comparing single and multiple communicative modality conditions in a mobile interaction context (Hooten, Hayes, & Adams, 2013). Although researchers have been considering ‘‘nomadic usage’’ (Price et al., 2006), i.e. use while in motion, the devices they care about are mostly portable electronic devices like portable digital assistants and the motion is merely limited to walking. The lack of caring about the stateof-the-art smart wearable devices and various motion states and emerging wearable devices promoting fitness and sports motivate us to study the influence of motion on the usage experience of wearable devices. Therefore, the second set of research questions is as follows:
RQ 1-4. Which display design is more acceptable? 2.2. Influence of motion on interaction with wearable devices When users interact with wearable devices, they are not in a stationary situation like sitting, but possibly in various states of motion
RQ 2-1. What is the influence of motion on gratifying users’ various needs?
RQ 2-2. What is the influence of motion on the perceived usefulness and ease of use in interacting with wearable devices?
150
Y. Zhang, P.-L.P Rau / Computers in Human Behavior 50 (2015) 148–158
RQ 2-3. What is the influence of motion on the subjective workload of nomadic usage of wearable devices? RQ 2-4. What is the influence of motion on the flow experience of interacting with wearable devices?
2.3. Uses and gratifications in interaction with multiple devices As mentioned above, nomadic usage of wearable devices essentially is a multitasking activity. Interaction with wearable devices and maintaining the motion state are both tasks in the scenario. Besides, users can interact with multiple smart devices simultaneously, which is one type of media multitasking. According to the theory of uses and gratifications, needs are ‘‘the combined product of psychological dispositions, sociological factors, and environmental conditions’’ (Katz, Blumler, & Gurevitch, 1973) which motivate media uses. Gratifications are the ‘‘perceived fulfillment’’ (Palmgreen, 1984) of the needs through media uses. Needs drive multitasking media uses and media uses generate gratifications. Multitasking uses may deliver none of, some or all of the gratifications sought and those gratifications obtained in turn can influence user needs. Previous studies accumulated various typologies of user needs in media multitasking. Jeong and Fishbein (2007) identify information processing and enjoyment experience as two types of needs in the context of media multitasking. Leung, Sheng, and Cruickshank (2007) summarize four types of needs in the context of user-generated content on the Internet: cognitive needs, social needs, recognition needs and entertainment needs. Zhang and Zhang (2012) focus on the scenario of multitasking with computers and provide a summary of three groups of needs: convenient/easy/ instant; control/habitual; and social/affective/relaxation. Another typology of user needs includes emotional, cognitive, social and habitual needs for the context of media and non-media mixed multitasking (Wang & Tchernev, 2012), and social media and other media mixed multitasking (Wang, Tchernev, & Solloway, 2012).
and time frames’’ (Mäntylä, 2013), whether gender differences exist in interaction with wearable devices is within our research interests. Thus, the next research question is formulated as follows: RQ 3-1. Is there gender difference in perceived difficulty (i.e. perceived ease of use) when users are involved in multitasking activities concerning wearable devices? On the other hand, previous research studies have shown that women and men differ in terms of intention to use technology. As an antecedent for intention to use mobile chat services, intrinsic motivation, or ‘‘the pleasure and inherent satisfaction derived from a specific activity’’, is more salient for women than men, like perceived enjoyment (Nysveen, Pedersen, & Thorbjørnsen, 2005). Similar results have also been found in behaviors of online gamers. Williams, Consalvo, Caplan, and Yee (2009) found that among game players who are in romantic relationships, female game players have higher levels of happiness than their male counterparts. In the study conducted by Nysveen et al. (2005), they found that perceived enjoyment played a stronger influence on the intention to use mobile chat services for female users than male users. Previous researchers have pointed out that women tend to be more process oriented (Venkatesh, Morris, & Ackerman, 2000) and men tend to be more task oriented (Minton & Schneider, 1980). Nysveen et al. (2005) thought that would be the foundation of women’s higher intrinsic motivation to use a technology. Similarly, we want to explore whether males and females show different patterns in gratification of different needs when interacting with wearable devices. Hence, the final research question is formulated as follows: RQ 3-2. Is there gender difference in gratification of user needs when users are involved in multitasking activities concerning wearable devices?
3. Experiment design 3.1. Independent variables
2.4. Gender difference in multitasking and gratification of needs The emergence of wearable devices makes the interaction pervasive and fragmented temporally and spatially. Moreover, interaction behavior has been integrated into daily life more than before. There has been evidence of women’s superiority compared with men in performing multiple tasks simultaneously in daily life. In a study by Stoet, O’Connor, Conner, and Laws (2013), women outperformed men in a task-switching paradigm and the interleaved tasks slowed down men’s speed more than women. Ren, Zhou, and Fu (2009) conducted a frequently adopted cognitive ability test and observed significant deterioration of men’s performance when coordinating a primary test with a simple secondary test. This superiority proves the Hunter-Gatherer hypothesis in evolutionary psychology, which states that the natural selection processes favoring hunting skills in men and gathering skills in women result in the gender difference in task performance, and that women’s superiority results from conducting multiple activities simultaneously, such as gathering food while feeding the offspring (Ren et al., 2009). However, this conclusion has been debated. Fasanya, McBride, Pope-Ford, and Ntuen (2011) found that there are no differences between genders when performing combined tasks, such as solving algebraic problems while listening for changes in auditory signals. Mäntylä (2013) found the opposite case to be true, that men are superior at multitasking, attributing it to men’s more powerful spatial ability. Since multitasking is a ‘‘loosely defined construct that covers a wide spectrum of activities
There were four independent variables in the experiment: two between-subject variables and two within-subject variables. The two between-subject variables were display and gender. Display referred to the bracelet type with two categories: bracelet without a screen and bracelet with a screen. Participants assigned the bracelet without a screen needed to plug the bracelet into the earphone jack of a smartphone, synchronize data between the two devices and then read the information about fitness via the smartphone. Participants assigned the bracelet with a screen were able to read the information on the screen of the bracelet by pressing a button. Two within-subject variables were motion and sharing operation. Motion included two categories: walking and jogging. Sharing operation referred to whether the participant shared the fitness information via social media or not. 3.2. Apparatus Each participant wore one smart watch and one smart bracelet, and carried one smartphone in their pocket in the experiment. The smart watch was a Sony SmartWatch MN2SW (Wikipedia, 2015c), which was connected to an Android smartphone via Bluetooth. The watch can alert users to incoming phone calls and allow replying to messages with pre-customized content. The experimenter customized the reply messages before the formal experiment: ‘‘I am in motion. I’ll call you back later.’’ We adopted two smart bracelets and assigned one type to each participant. They were Codoon S
Y. Zhang, P.-L.P Rau / Computers in Human Behavior 50 (2015) 148–158
with no screen and Codoon 2 with a screen (Codoon, 2015). The two bracelets are the same in terms of brand and basic functions. They record users’ energy consumption and allow synchronization of data via an associated mobile app. Users can read fitness information and share it via social media on the mobile phone. We adopted WeChat, which is a popular mobile social networking service in China, in the experiment. In WeChat, users can post on the ‘moments’ to communicate with their friends. Participants were required to share information about fitness after the exercise on the ‘moments’ of WeChat. 3.3. Dependent variables The dependent variables were all measured by paper-based questionnaires after each round of exercise. All the questions were in Chinese and responses were coded using a 7-point Likert scale. Users’ gratifications of different needs were measured according to four dimensions: emotional experience, information acquisition, social contact and self-recognition. They were derived and adapted from a study by Katz, Haas, and Gurevitch (1973). Questions about perceived ease of use and perceived usefulness were derived from a study by Davis (1985). Questions about participants’ acceptance were derived from a study by Wang, Rau, and Salvendy (2011). Questions about the subjective cognitive workload were derived from a study by Zhang et al. (2013). Questions about the flow experience were derived from a study by Agarwal and Karahanna (2000). All the variables were repeatedly measured for smart watch usage and smart bracelet usage respectively, except that flow experience was measured only for smart watch usage because it described the experience in respect to motion. Detailed descriptions concerning user gratifications are shown in Table 1. 3.4. Tasks and procedures We conducted an experiment to explore answers to the research questions. In total, 36 participants were recruited via social network sites. They were all university students with routine sportive habits. None of them owned a wearable device at the time. Controlling the ownership of wearable devices was aimed at ensuring the consistency of participants’ usage experience of wearable devices. Another requirement was that they should own a WeChat account, because they were required to share information about fitness via WeChat in the experiment. Each participant needed to complete a two-phase experiment, each phase of which included two sequential rounds of exercise: walking and jogging on a standard sports field. The sequences of walking and jogging were balanced among all the participants. In the round of walking, the participant walked around the sports field once (400 m) with
Table 1 Measurement of user gratifications. Dependent variables
Question descriptions
Emotional experience Information acquisition Social contact
Using the smart watcha in motion keeps me entertained
Self-recognition
a
151
natural walking speed, while in the jogging round the participant jogged two laps (800 m) with a comfortable speed. The participant wore a smart bracelet on the dominant hand, a smart watch on the other hand and had a smartphone in a pocket during the exercise. Although there are uncontrolled confounding factors involved, conducting such an experiment on the sports field instead of on a treadmill has been proven to guarantee ecological validity (Barnard et al., 2005). Before the exercise, the participants signed an informed consent form and were taught how to check energy consumption with the bracelet and how to deal with incoming calls and reply to a message via the watch. During the exercise, the participants received one phone call and one short message from the experimenter at a random time. The message concerned a time and location change to a class or a party. The participant was required to hang up the call, read the message and reply to it with the pre-customized content in the smart watch, saying that he/she was exercising and would call back later. After walking or jogging, the participant checked the energy consumption with the bracelet. In order to guarantee that the participant had read the message, the experimenter provided an answer sheet and asked the participant to recall the message and fill in some blanks about the message. After that, the participant filled in a questionnaire regarding the dependent variables. That was the whole procedure of the first phase. The second phase was conducted at least five days later to eliminate the influence of the former phase. This procedure was similar to the first one, but contained one more task: sharing the fitness information via the Codoon application on the ‘moments’ of WeChat. The sequence of the two phases was balanced among all the participants. At the end of the second phase, a short interview was conducted concerning participants’ experience in the two motion states; their acceptance of wearable devices; and attitude toward sharing fitness information online. The whole procedure is depicted in Fig. 1 and a picture of the experimental scene is shown in Fig. 2. 4. Results There were 36 participants in the experiment: 18 males and 18 females. Their average age was 24.2 years old (SD = 2.3 years). A total of 15 participants used the smart bracelet without a screen, eight of whom were males and seven females, and 21 participants used the smart bracelet with a screen, 10 of whom were males and 11 females. The experiment utilized a mixed factorial design with both within-subject variables and between-subject variables. Motion and gender were designed as independent variables of those dependent variables concerning smart watch usage because the interaction with the smart watch happened in motion. Display, gender and sharing operation were designed as independent variables of those dependent variables concerning smart bracelet usage because the interaction with the smart bracelet happened after being in motion. 4.1. Gratifications of user needs
Using the smart watch in motion gratifies my need to acquire information Using the smart watch in motion strengthens my contact with friends Using the smart watch in motion gives me confidence Using the smart watch in motion makes me feel influential Using the smart watch in motion makes me feel higher quality of life
The question is repeated for smart watch and smart bracelet.
4.1.1. Emotional experience The data on emotional experience was not normally distributed, so a nonparametric test was conducted to analyze it. For the emotional experience of smart watch usage, a Wilcoxon signed ranks test indicated a significant main effect of motion (Z = 3.553, p < .001) and a Mann–Whitney test indicated a significant main effect of gender (Z = 2.259, p = .024), where participants reported being more entertained when walking than when jogging and female participants reported being more entertained than male participants. When we analyzed cases in different
152
Y. Zhang, P.-L.P Rau / Computers in Human Behavior 50 (2015) 148–158
Fig. 1. Experimental procedure.
pattern of simple effects of gender on emotional experience regarding different motion states. The detailed statistics of main effects are shown in Table 2.
Fig. 2. Experimental scene.
genders separately, we also found that both males (Z = 3.213, p = .001) and females (Z = 3.553, p < .001) reported being more entertained when walking than when jogging, which showed the same pattern of simple effects of gender on emotional experience regarding different motion states. For the emotional experience of smart bracelet usage, a Mann– Whitney test indicated that the main effects of display (Z = 3.136, p = .002) and gender (Z = 3.425, p = .001) were significant but Wilcoxon signed ranks tests indicated the main effect of sharing operation was not significant (Z = .098, p = .922). Participants who used the bracelet with a screen and female participants reported being more entertained than their counterparts. When we analyzed cases in different genders separately, we also found that both males (Z = 2.312, p = .021) and females (Z = 2.045, p = .041) who used a bracelet with a screen reported being more entertained than their counterparts, which showed the same
4.1.2. Information acquisition The data on information acquisition was not normally distributed, so a nonparametric test was conducted to analyze it. For information acquisition through smart watch usage, a Wilcoxon signed ranks test indicated a significant main effect of motion (Z = 2.737, p = .006) and a Mann–Whitney test indicated a significant main effect of gender (Z = 4.805, p < .001), where walking participants and female participants reported that their need to acquire information was more gratified than their counterparts. When we analyzed cases in different genders separately, we found different patterns between males and females regarding motion states. Males reported that their need to acquire information was more gratified when walking than when jogging (Z = 2.327, p = .020), but females did not report any significant difference in gratification of information acquisition between two motion states (Z = 1.470, p = .142). For information acquisition through smart bracelet usage, a Mann–Whitney test indicated that the main effects of display (Z = 3.027, p = .002) and gender (Z = 3.366, p = .001) were
Table 2 Statistics of emotional experience. Independent variable
N
Emotional experience of smart watch usage Motion Walking 72 Jogging 72 Gender Male 72 Female 72 Emotional experience of smart bracelet usage Display Without 60 screen With screen 84 Gender Male 72 Female 72
Mean
SD
Z
p
5.10 4.40 4.50 5.00
1.19 1.46 1.39 1.31
3.553
