Effects of Displaying Trust Information on Mobile Application Usage Zheng Yan1, Conghui Liu2, Valtteri Niemi3, and Guoliang Yu2 1
Nokia Research Center, Helsinki, Finland
[email protected] 2 Institute of Psychology, Renmin University of China, China
[email protected];
[email protected] 3 Nokia Research Center, Lausanne, Switzerland
[email protected]
Abstract. As trust helps users overcome perceptions of uncertainty and engages in usage, visualizing trust information could leverage usage behavior and decision. This paper explores the effects of trust information’s visualization on mobile application usage in Finland and China with three steps: (1) Studying users’ opinion on mobile application’s importance. (2) Evaluating a trust indicator’s effect on mobile application usage. (3) Evaluating a trust/reputation indicator’s effect on mobile application usage. Although the results achieved in above two countries showed differences, both indicated that displaying an application’s reputation value and/or a user’s individual trust value could assist in the usage of mobile applications with different importance. We also discuss the possible reasons that caused the different effects in two countries.
1 Introduction A mobile device has been becoming an open computing platform to run various applications. A mobile application is a software package that can be installed and executed in a mobile device (e.g. a mobile phone). Generally, this software package developed by various vendors can be downloaded for installation. A number of similar functioned mobile applications would be available for selection and usage. Future market could be very competitive. Which mobile application is more trustworthy for a user to consume becomes a crucial issue that impacts both the user’s usage and the application’s final success. A user’s decision is required during practical consumption of a mobile application, for example, whether it is safe to conduct an important task via using an application; if it is worth continuing usage, etc. Nowadays, mobile users are not given any cue that could help their usage decision. In fact, trust is an important factor that impacts usage. It helps consumers overcome perceptions of uncertainty and engages in trust behavior (i.e. a trusting subject’s actions to depend on, or make her/him vulnerable to a trusting object). In the literature, numerous researchers have found a positive correlation between trust and use [1-4]. Some have conceptualized trust as a behavior [8-10]. Prior research has also confirmed a strong correlation between behavioral intentions and actual behavior, especially for human - software system interaction [11, 12]. However, B. Xie et al. (Eds.): ATC 2010, LNCS 6407, pp. 107–121, 2010. © Springer-Verlag Berlin Heidelberg 2010
108
Z. Yan et al.
few existing studies explore the effects of visualizing trust information in the context of mobile application usage. In this paper, we study the effects of displaying trust information on mobile applications. We hypothesize that visualizing trust information could leverage users’ usage decision. We validate that displaying a mobile application’s reputation (public trust) value and/or a user’s individual trust value could enhance the user’s ability to perform appropriate actions in various usage contexts. Detailed trust information about how trust/reputation values are generated can be accessed in an additional way during application execution for the purpose of usability. Our validation is based on a threestage experiment conducted in both Finland and China. In the rest of this paper, we firstly give a brief review on related work. Then, we introduce our research questions and experiments followed by experimental details and results. We further discuss the possible reasons that caused the difference of results in two countries. Finally, conclusions are presented in the last section.
2 Background and Related Work Trust is firstly a social phenomenon. With the rapid growth of computer and networking technology, human – computer trust has been paid attention to. Trust has been defined by researchers in many different ways, which often reflect the paradigms of particular academic disciplines. In this paper, we derived our understanding of trust in mobile applications that has its roots crossing multiple disciplines [6]. A user’s trust is defined as his/her belief on a mobile application that could fulfill a task as expectation. Reputation is public trust derived from direct and indirect knowledge or experiences. In our study, it is defined as the public belief on a mobile application that could fulfill a task according to many people’s expectations. Trust is an integral component in many kinds of human interaction, allowing people to act under uncertainty and with the risk of negative consequences [14]. Recently, researchers in Human-Computer Interaction (HCI) and human factors have studied trust in an on-line context, but few in the mobile application domain. The realization that design can affect the trust of a user has had implications for user interface design, web sites and interactivity in general [15]. Some researchers examined the cues that may affect trust. These cues range from design and interface elements, to perceived website credibility, to the extent to which the technology is perceived and responded to as a social actor (e.g. photographs and other indicators of social presence) [16]. Research focuses on the cues that convey trustworthiness to users. Interface design can give cues about trust or signal trustworthiness [16, 17]. However, this part of research did not consider how to confer computational trust values to the users. Highly related to our work, notification systems attempt to deliver current, important information to users in an efficient and effective manner without causing unwanted distraction to ongoing tasks [18]. McCrickard et al. commented that the effects of incoming notifications on ongoing computing tasks have been relatively unexplored [18]. Notification without interruption of usage still lacks investigation, especially for mobile and ubiquitous devices that include a small display element. Antifakos, et al. conducted experiments to show that displaying confidence information increases the user’s trust in a system in various contexts classified by criticalness [19].
Effects of Displaying Trust Information on Mobile Application Usage
109
But Rukzio et al. proved that the user needs slightly more time and produces slightly more errors when the system confidence is visualized [20]. The contradictory results implied that the visualization of system confidence seems questionable or works differently in different situations. Recently, trust and reputation mechanisms have been proposed in various fields of distributed systems, such as ad hoc networks, peer-to-peer systems, Grid and pervasive computing, web services and e-commerce [6]. In many existing web services (e.g. eBay.com and Amazon.com), reputation values (mostly in a Likert scale) are displayed based on rating in order to assist users’ decision. However, few work in the past, as we know, studied mutual effects of visualizing both individual trust value and public trust (i.e. reputation) value to the mobile application users during its execution. In addition to all above, the third party, and cultural factors are also important variables affecting trust and its development [21]. It was found that Japanese have a generally low level of trust compared with Americans [22]. Karvonen found that Finnish consumers are the most cautious and Icelandic consumers are the most trusting in ecommerce in Nordic countries [23]. A lot of work has been conducted regarding user interface design in order to improve user’s trust, mainly for web sites and in the context of e-commerce. Still, prior art left rooms for further studies on the effects of trust information on usage behaviors and, in particular, on how to provide trust information for mobile users. In the work presented in this paper, we use a trust indicator to indicate a user’s individual trust value and a trust/reputation indicator to indicate both the individual trust value and reputation value. They are interface design elements that provide the cue of trust information through a mobile application trust management system [7]. Particularly, a user’s individual trust value could play as valuable credibility for the user’s contribution (e.g. his/her rating) to the reputation of a mobile application. In addition, the individual trust value and the reputation value could serve as credibility with each other. But no previous researches investigated these two pieces of information’s mutual effects on users, which is one of our research targets. What is more, the above effects could also be impacted by region and culture (e.g. western and eastern cultures), and context (e.g. the importance of the mobile application), which is also what we aim to explore in our study.
3 Research Questions and Experiment Overview 3.1 Displaying Trust Information for Mobile Application Usage A user’s trust in a mobile application is built up over time and changes with the use of the application due to the influence of many factors. We developed a system to calculate the device’s estimate on its user’s individual trust based on his/her usage behaviors (e.g. using behavior about normal usage statistics, reflection behavior related to application performance and user experience, and correlation behavior regarding similar functioned applications) [7, 13]. We achieved an individual trust model by mathematically formalizing trust behavior measures that were investigated in a large scale user study [13]. The individual trust is calculated by the user’s mobile device based on trust behavior observation.
110
Z. Yan et al.
The application’s reputation value is generated through aggregating the users’ individual trust and ratings together. It is issued by a reputation service provider, as designed in [7]. We use a number to indicate the individual trust value and the reputation value, respectively. Their value range is from 0 to 1, representing from full distrust to full trust. We use a rectangle bar to indicate the trust value and a trapezoid bar to indicate the reputation value in our experiments. Concretely, the user’s individual trust value is presented by the trust indicator. The trust/reputation indicator presents both values. We also assumed that detailed trust information is available through an additional access, either from a device menu or by touching the indicator or via a short-cut key. This is because providing comprehensive information and necessary explanation could increase users’ trust [24]. But directly displaying the detailed trust information (e.g. how these values are generated and who provides this information) could influence the user’s usage experience due to the limited screen size of mobile devices. Although application reputation is very helpful for the users to select an application for purchase and installation, we still faced an important system design issue: is it helpful for the users if we display the real time trust information (i.e. trust and/or reputation values) during the application usage or consumption? The work presented in this paper aims to evaluate the effects of displaying trust information on mobile application usage. Concretely, we try to explore the following research questions: 1. How important do people think in using various mobile applications? 2. How does the individual trust information affect mobile application usage? 3. How does the individual trust and public reputation information affect mobile application usage? 3.2 Experiment Overview We developed an experiment toolkit and conducted a three-stage user study to investigate the above research questions in both China (CN) and Finland (FI). We adopted 48 mobile applications and recorded their usage videos. The 48 mobile applications were selected from real products or prototypes, which provide a diverse sample of various mobile application scenarios and usage contexts. Examples include mobile enterprise solutions (e.g. corporate email checking); mobile personal business solutions (e.g. mobile payment, wallet, and safe box); mobile entertainment solutions (e.g. mobile TV, mobile video/audio/radio/music, gaming and camera); mobile life and social networking solutions (e.g. mobile search, location based services, maps, instant messaging and VoIP applications, travel aids, and mobile diary); mobile education solutions (e.g. e-book reader and multi-language translator); and integrated applications that provide an easy access to various mobile Internet services, e.g. Nokia WidSets; and Yahoo! Go. In the first experiment we assessed the importance of mobile application scenarios. Using this result we selected a small set of 9 applications based on their importance rates (low, medium and high) and showed them in the later two experiments. In the second and the third experiments with different participants, the trust indicator or the trust/reputation indicator was displayed in one experiment block, while in the other block no information about trust was given. For each application usage scenario (showed as a video) we asked the participants to rate their willingness to continue
Effects of Displaying Trust Information on Mobile Application Usage
111
consumption and check trust information. Thereby, the effects of three variables: the availability of trust information, trust information itself (either a trust value indicator or a trust/reputation values indicator), and the specific mobile application scenario (with different importance rates) were studied, respectively. Meanwhile, the results achieved in both countries were also compared. Table 1 shows the design of Experiment 2(E2) and Experiment 3 (E3). Table 1. The design of Experiment 2 and 3
E2
E3
Experiment variables Indicated trust value Indicator availability Application importance Region (test sites) Indicated (trust value) × (reputation value) Indicator availability Application importance Region (test sites)
Block 1 Low, mid, high No Low, mid, high Finland & China (low, mid, high) × (low, mid, high) No Low, mid, high Finland & China
Block 2 Low, mid, high Yes Low, mid, high Finland & China (low, mid, high) × (low, mid, high) Yes Low, mid, high Finland & China
4 Experiments and Results 4.1 Experiment 1: Application’s Importance We define importance as how important a specific task that can be fulfilled by a mobile application is to the participant concerning its usefulness and personal interest. To determine it, each participant’s preferred scale is assessed in Experiment 1. Experimental Toolkit, Participants and Procedure The experimental toolkit contains a number of testing slides. Each testing slide is divided into four parts under the application name, as shown in Fig. 1. Part 1 is a brief introduction of the mobile application. Part 2 is the explanation of assumed application scenario and an underlying task. Part 3 is a video record of the application usage. The videos last from 48 seconds to 3 minutes and 17 seconds. Part 4 is a feedback area where the participant can use the scroll bars to provide his/her feedback. The experiment data were automatically recorded. In Finland, 26 participants mainly from a university participated in this study. We selected 23 valid samples (39.1% female) for data processing. This group ranged in age mostly from 19-29 years old, and 87.0% majored in science and technology, but also arts (8.7%) and business (4.3%). In China, we selected 26 valid samples (61.5% female) from a total of 31 participants for data processing. The sample was composed of 13 graduates and 13 undergraduates. This group ranged in age from 19-29 years old, and 7.7% majored in science and technology, while 57.7% in arts, and 34.6% in business. Each participant in both countries was rewarded a gift after the testing.
112
Z. Yan et al.
Fig. 1. Example testing slide of experiment 1
In this experiment, each participant completed a series of application scenarios to assess his/her opinion on application importance. For each scenario, shown as a text description and a video, the participant was asked to answer the following question: How important is this application scenario to you? A scroll bar was provided for the experiment participants to indicate the importance rate (IR) from ‘not at all’ to ‘very much’. The corresponding values attached to them are 0 and 1, respectively. After answering these two questions, the participant pressed “Confirm” button. Then the test automatically went to the next scenario. After each user test, the display order of the application scenarios was shifted by two. Meanwhile, all the scroll bars’ values were reset to 0. Results The importance rates of the 48 application scenarios varied between .224 and .777, with an average of .487 in Finland, and between .314 and .887, with an average of .553 in China, respectively. This showed that the applications adopted in the experiment covered the continuum of importance well. Table 2. Selected mobile applications (FI and CN) Application groups Application names
IR average IR stdev
IR low FI CN LifeBlog LifeBlog Video TextTV Downloa Webd Browser Shozu .292 .341 .010 .038
IR medium FI CN Gizmo MobiYahoo!Go; Reader MYahoo!Go realplayer Mrealplayer .478 .539 .059 .008
IR high FI CN Nokia Camera Maps Fring Gmail Music player WebBrowser .719 .773 .042 .014
Effects of Displaying Trust Information on Mobile Application Usage
113
We grouped and selected the applications based on the importance rate. As shown in Table 2, the results in Finland and China are different, implying that people’s opinions on mobile applications’ importance could be different in different regions. Surprisingly, one of the high important applications (i.e. mobile web browser) in Finland was treated unimportantly in China although its usage experience rate is not low (73.9% in Finland and 49.2% in China). We further conducted paired samples t test to evaluate our grouping. The t and p values of high importance group and medium importance group are 25.666 and
