TSINGHUA SCIENCE AND TECHNOLOGY ISSN 1007-0214 06/26 pp293-299 Volume 13, Number 3, June 2008
An Laboratory Experiment for Comparing Effectiveness of Three Types of Online Recommendations* SHI Lin (ದ ॾ), WANG Kanliang (ฆ࢜॥)** School of Management, Xi’an Jiaotong University, Xi’an 710049, China Abstract: The widespread use of Internet accelerates the rapid development of business to customer electronic commerce. To reduce information overload and help their customers to make better purchase decisions, e-commerce websites are beginning to use online recommendations. This paper compares the effectiveness of three types of online recommendations, the personalized recommendation, best sellers, and consumers’ reviews, which are widely used in e-commerce. This research used a laboratory experiment combined with a questionnaire. This paper also establishes an integrated model of the facts that influence recommendation effectiveness. Key words: online recommendation; recommendation adoption; consumer perception
Introduction The rapid expansion and penetration of the Internet have brought us to the era of network economy. Traditional shopping approaches are facing unprecedented challenges. The product categories provided by business to customer (B2C) e-commerce websites are far beyond the traditional brick-and-mortar stores due to unlimited cyberspace. This results in the rapid growth of information available to customers, increasing the probability of selecting satisfactory goods[1]. Therefore, online consumers have to deal with vast amounts of information with limited recognition capability[2,3]. More and more e-commerce websites are adopting online recommendation technologies to solve this problem and help online consumers make effective purchase decisions. Online recommendation refers to approaches used by e-commerce websites to help online consumers Received: 2007-06-06; revised: 2007-12-20 γ Supported partially by the National Natural Science Foundation of China (Nos. 70372049 and 70121001) γγTo whom correspondence should be addressed. E-mail:
[email protected]; Tel: 86-29-82668748
make effective purchase decisions by recommending products and providing them with product information[1]. The recommendation technologies adopted by the well-known e-commerce websites (such as Amazon.com, eBay, dangdang.com, joyo.com) are based on best sellers, demographics, and purchase history of consumers to predict future purchase behavior[4]. The commonly used recommendation technologies are personalized recommendation[5-9], best sellers, and consumers’ reviews[10,11]. There are various products sold over e-commerce websites. Product category may influence the way that consumers deal with different sources of product information and the role of this information in making purchase decisions[12,13]. Most of the relevant research in the field of consumer behavior adopts the product categorization criteria proposed by Nelson in 1970. Products are categorized by their search attributes and experience attributes[14,15]. The differences between searched products and experienced products may lead to the different degree of difficulties in obtaining product-relevant information when consumers make purchase decisions. Compared with search products, the search cost of experienced products is much higher due
Tsinghua Science and Technology, June 2008, 13(3): 293-299
294
to the difficulty in evaluating before purchasing or using this kind of product[16]. Therefore, consumers tend to depend on the recommended information when buying this kind of products[17]. Many researchers investigated the factors that may significantly influence the adoption of online recommendations by online consumers (Table 1). The current research focuses on the degree of personalization, perceived usefulness, and trustworthiness on the effectiveness of recommendations. Table 1 A brief summary of factors influencing the effectiveness of recommendations Literature Gershoff et
Influencing factors Consistence of recommendation and con-
al.[10]
sumer’s opinion [18]
Tam and Ho
Personalization
Senecal and
Personalization, trustworthiness, and the
[17]
Nantel
Wang and Benbasat[19] Tam and Ho[20]
characteristics of recommended products Perceived usefulness and perceived ease of use, trustworthiness Degree of preference fitness, size of recommendation, sequence hint
(1) Personalization Personalization refers to the capability of providing tailored contents and services to an individual based on the knowledge of preferences and behavior about this individual[21]. The three objectives of Internet personalization are increasing the awareness of consumers to products and services, implanting information, and trying to persuade consumers to make a purchase[18]. Personalized information affects consumers much more than that of non-personalized information[17]. By studying the downloads of several music websites, Tam and Ho[18] found that customers who adopted personalized services downloaded more than the controlled group. The number of items downloaded each time was less than the controlled group and more energy was saved when making purchase decisions. Moreover, customers who adopted personalized services perceived much higher satisfaction. Online recommendation is a major approach of ecommerce website personalization, especially for personalized recommendation. This paper defines personalized recommendation as an approach which can recommend suitable products for consumers according to their various needs.
(2) Technology acceptance model (TAM) The aim of TAM proposed by Davis[22] is to explain and predict the adoption of new technologies by users. Two important constructs were proposed in TAM ü perceived usefulness (PU, users’ belief that their performances can be improved using a specified technology) and perceived ease of use (PEOU, users’ belief that a specified technology can be used easily). Much research on the adoption of information technology has been based on TAM, several of which were research on online shopping and online recommendation. Koufaris[23] regarded online consumers as double role individualsüonline shoppers and computer usersüat the same time. He integrated TAM, consumer behavior, and flow theory to investigate online consumer behaviors. Wang and Benbasat[19] verified the effectiveness of TAM in interpreting consumers’ adoption of online recommendation by introducing the construct of trustworthiness into TAM. (3) Credibility Online consumers face much higher uncertainty since they are unable to give precise evaluations of a specified product or the credit of a seller before the completion of a purchase[24]. In addition, in the context of online shopping, it is difficult to control the behavior of a seller since the legislation on online shopping is on the way to maturity[25]. In the virtual environment characterized by high risk and high uncertainty, the credibility of online recommendation may become an important factor affecting its adoption by consumers. The concept of credibility was originally used in the context of information source, which is defined simply as “believable”[26,27] and has two dimensions of trustworthiness and expertise[28]. Trustworthiness refers to the perception that the information source is unbiased in delivering facts, while expertise is defined as the ability of the information source that can obtain the right answer[29]. There are positive correlations between expertise, trustworthiness of the information source and the attitude, intention, and actual behavior of consumers towards a specified brand[30-33]. Since this research regards online recommendation as the source of information, the concepts of trustworthiness and expertise are introduced as two independent constructs. Here, trustworthiness implies that recommended information is realistic and dependable, while expertise implies that the recommendation is
SHI Lin (ದ ॾ) et alġAn Laboratory Experiment for Comparing Effectiveness of …
professional in nature and the quality of products can be guaranteed. This paper compares the characteristics and the effectiveness of the three most popular recommendation technologies, i.e., personalized recommendations, best sellers, and consumers’ reviews. We are trying to explore the factors that influence the effectiveness of recommendation.
1
Model and Hypotheses
The effectiveness of recommendation can be measured by whether consumers purchase the recommended products. The approaches to recommend are different in the recommendation creation and the information sources. Therefore, we have the following hypothesis. H1 There are significant differences among different online recommendation approaches. Previous research showed that product category affects the use of different sources of product information by consumers[12,13], and therefore, the adoption of different recommendation approaches. Besides, it is difficult to obtain information related to the attributes that determine the quality of experienced products directly. Consumers need recommended information to help them make purchase decisions. However, consumers tend to make purchase decisions by themselves when they evaluate search products[13]. H2 It is more effective to make online recommendation for experienced products than for search products. Consumers may have different perceptions on the recommended information by different approaches when they make purchases. Thus, different recommendation approaches may fit for different categories of products. H3 There are significant differences on the effectiveness of recommendation approaches for experienced products and search products. This study used a questionnaire survey to investigate the factors affecting the effectiveness of recommendation. We separated the effectiveness of recommendation into two parts: perceived effects of recommendation and recommendation adoption. Perceived effect of recommendation is defined as the perceived effects of online recommendation to the purchase decisions of consumers. According to the relationship between perception and actual behavior, we have H4. H4 The perceived effect of recommendation of
295
consumers has significant influence on their adoption of recommendation. In this research, online recommendation is regarded as both a decision aid and the information source. TAM was used to analyze the adoption of online recommendation by online users. Only differences in recommendation creation exist for the three approaches. Both the approaches of best seller and consumers’ reviews create recommendation contents directly, while personalized recommendation needs consumers to show their preferences explicitly. Thus, there are no differences on the perceived ease of use among the three recommendation approaches. In this research, perceived usefulness is defined as “the belief of consumers that use of online recommendation technology may improve the performance of their purchase.” Since perceived usefulness has significantly positive effects on the attitude of users towards technology[22], we have H5. H5 The consumers’ perceived usefulness of a recommendation approach increases with the perceived effects of recommendation. However, recommendation was regarded as the information source; trustworthiness and expertise are the key factors that affect the adoption of online recommendation. H6 The consumers’ perceived trustworthiness of a recommendation approach increases with the perceived effects of recommendation. H7 The consumers’ perceived expertise of a recommendation approach increases with the perceived effects of recommendation by consumers. “Expertise is defined as the perceived capability of the information source in obtaining the correct answer”[29], and we have H8. H8 The consumers’ perceived expertise of a recommendation approach increases with the perceived usefulness of recommendation by consumers. In addition, personalization has significant effects on the shopping behavior of online consumers. Personalized information has much higher effects on consumers’ buying decisions than that of non-personalized information[34]. H9 The personalization of a recommendation approach has a positive effect on the perceived effects of recommendation by consumers. H10 The personalization of a recommendation
Tsinghua Science and Technology, June 2008, 13(3): 293-299
296
approach has a positive effect on the perceived usefulness of recommendation by consumers.
According to the above discussions, we summarized the research model as shown in Fig. 1.
Fig. 1 Research model of recommendation
2
Methodology
Students from four universities at Xi’an (Shaanxi, China) participated in the experiment. Questionnaires were distributed and collected on a website. We obtained 349 questionnaires. Among them, five questionnaires were discarded due to inconsistency. Therefore, 344 valid questionnaires were used. To motivate the respondents to participate in the experiment and behave as they would in the real world, all respondents were given a small gift and joined a lottery, and the lucky respondents got the products they chose. All the respondents logined the website designed for the experiments according to the guidebook using a specified username and were exposed to different experiment scenarios. In the scenario of personalized recommendation, respondents were required to fill in a form to express their preferences relevant to the specified product after they logined, which would be used to calculate the fitness of product and their preferences. In the other two scenarios, respondents got to the homepage of the website directly after they logined. Each respondent had to choose one and only one product. Information on the product chosen by the respondents was recorded in the database. Questionnaires were distributed to the respondents after the shopping task was completed. After that, all the respondents were invited to join a lottery. The experiment was designed as a 2h 3 factor online experiment. The shopping process of the
respondents was consistent with a realistic scenario except for payment. (1) Products Considering the characteristics of the university student respondents, this study used books and MP3 players to represent the experienced product and search product. Since most of the respondents were studying management or business, most of the books were in the fields of economics, business, and management. The books and MP3 players chosen were real products. To specify the product list, we chose 30 book titles in business and management and 24 different styles of MP3 players from several shopping websites with national brands, with the principle of minimizing the difference in price and brand for similar products. Thirty respondents were exposed to a pilot study without any kind of recommendations. They were required to choose a preferred product from the above product list. Eight books and eight MP3 players were selected into the final product list to ensure no dominant products in similar products so that this study focused on the recommendation effects on the online purchase decision by the respondents. (2) Experiment scenarios Since this is a 2h3 factor experiment, we have six scenarios (Table 2). Table 2 Experiment scenarios Product category
Recommendation approach Consumers’
Personalization
Best sellers
Books
Scenarioĉ
Scenario Ċ
Scenario ċ
MP3 players
Scenario Č
Scenario č
Scenario Ď
reviews
SHI Lin (ದ ॾ) et alġAn Laboratory Experiment for Comparing Effectiveness of …
MySQL4.1.8 was used to establish databases and Micromedia Dreamerwaver MX2004 and PHP were used to implement the website. The questionnaire was built on the basis of classical instruments adopted in the precious research (Table 3), which had passed the reliability test and validity analysis. Table 3
Source of subscales
Subscale
Literatures
Trustworthiness
Ohanian[35]
Expertise
Ohanian[35]
Perceived usefulness
Davis[22]
Subjective knowledge of
Flynn and Goldsmith[36]
product Susceptibility to interpersonal
Smith and Menon[11]
influence
3
Results and Discussion
The descriptive statistics showed that 53.9% of the respondents were females and 46.1% were males among the 344 valid respondents; the youngest was 18 years old, the oldest was 29, and the average age was 21.41. For the respondents exposed to the books buying scenario, 52.8% had online shopping experiences, while for the respondents exposed to the MP3 players buying scenario this ratio was 51.3%. We adopted dichotomy logistic regression to test hypothesis H1 through H3. Recommendation adoption was the dependent variable, and recommendation approach and product category were the independent variables. Individual differences among the respondents (such as product experiences, online recommendation experiences, and personality of respondents) that might affect recommendation adoption
Fig. 2
297
were controlled by randomly assigning respondents to different scenarios. The results show that the interaction effect of the recommendation approach and product category on recommendation adoption is not significant (Wald Ȥ2=0.241, p=0.887), which implies that there is no significant difference on the effectiveness of recommendation approaches for these two kinds of products. Product category has no significant effect on recommendation adoption (Wald Ȥ2=0.213, p=0.644). The differences of recommendation effects are significant for the three approaches (Wald Ȥ2 ˙ 40.801, p
