Electronic Commerce Research and Applications 14 (2015) 58–74
Contents lists available at ScienceDirect
Electronic Commerce Research and Applications journal homepage: www.elsevier.com/locate/ecra
Visualizing market structure through online product reviews: Integrate topic modeling, TOPSIS, and multi-dimensional scaling approaches Kun Chen a,1, Gang Kou b,⇑, Jennifer Shang c,2, Yang Chen b,3 a
School of Management and Economics, University of Electronic Science and Technology of China, Chengdu, Sichuan 610054, China School of Business Administration, Southwestern University of Finance and Economics, Chengdu, Sichuan 610074, China c Katz Graduate School of Business, University of Pittsburgh, Pittsburgh, PA 15260, USA b
a r t i c l e
i n f o
Article history: Received 28 September 2013 Received in revised form 20 November 2014 Accepted 21 November 2014 Available online 12 December 2014 Keywords: Market structure Text mining Topic modeling Ranking of products TOPSIS
a b s t r a c t Studies have shown that perceptual maps derived from online consumer-generated data are effective for depicting market structure such as demonstrating positioning of competitive brands. However, most text mining algorithms would require manual reading to merge extracted product features with synonyms. In response, Topic modeling is introduced to group synonyms together under a topic automatically, leading to convenient and accurate evaluation of brands based on consumers’ online reviews. To ensure the feasibility of employing Topic modeling in assessing competitive brands, we developed a unique and novel framework named WVAP (Weights from Valid Posterior Probability) based on Scree plot technique. WVAP can filter the noises in posterior distribution obtained from Topic modeling, and improve accuracy in brand evaluation. A case study exploring online reviews of mobile phones is conducted. We extract topics to reflect the features of the cell phones with a qualified validity. In addition to perceptual maps derived by multi-dimensional scaling (MDS) for product positioning, we also rank these products by TOPSIS (Technique for Order Performance by Similarity to Ideal Solution) so as to visualize the market structure from different perspectives. Our case study of cell phones shows that the proposed framework is effective in mining online reviews and providing insights into the competitive landscape. Ó 2014 Elsevier B.V. All rights reserved.
1. Introduction Online product reviews offer businesses the opportunity to economically and expediently perform in-depth and comprehensive market landscape analysis. These consumer-generated data have received increasing attention from academics and practitioners in recent years (Laroche et al. 2005; Gopal et al. 2006). Although online voices of the consumer (VOC) are in the form of free-text, they have proven to reflect essential characteristics of the market (Campbell et al. 2011; Godes and Mayzlin 2004; Duan et al. 2008; Shao 2012), and they can leverage changes of traditional marketing activities (Onishi and Manchanda 2012). Nevertheless, there remains an obstacle which prevents online product reviews from reaching its best potential. Namely, the online user-generated content is enormous and qualitative in nature, making it difficult to
⇑ Corresponding author. Tel.: +86 15982498501. E-mail addresses:
[email protected] (K. Chen),
[email protected] (G. Kou),
[email protected] (J. Shang),
[email protected] (Y. Chen). 1 Tel.: +86 28 87512680. 2 Tel.: +1 412 648 1681. 3 Tel.: +86 18828041300. http://dx.doi.org/10.1016/j.elerap.2014.11.004 1567-4223/Ó 2014 Elsevier B.V. All rights reserved.
analyze the data quantitatively and attain meaningful knowledge therein (Godes et al. 2005). Due to lack of effective methods to extract key features of these texts, businesses were unable to obtain useful information to develop a market structure map. Yet, the understanding of these viewpoints and market structure is crucial for product development, pricing, promotion/campaign, and brand positioning. Companies thus resort to product ratings as proxies for product reviews. For example, Chintagunta et al. (2010) use product ratings to examine the relationship between consumer reviews and sales in the movie industry, while Chevalier and Mayzlin (2006) use them to study the book industry. Market structure is the depiction of relationship among brands based on various approaches such as consideration set (Urban et al. 1984), brand-switching data (Cooper and Inoue 1996) and brand associative networks (John et al 2006). With the advance of text mining techniques based on natural language processing (NLP) (Feldman et al. 2007), researchers have begun to elicit structured and quantitative information of brands in the market from online product reviews. For instance, Lee and Bradlow (2011) developed a text-mining algorithm for online product reviews. And Netzer et al. (2012) proposed a hybrid text-mining and semantic network analysis tool for market-structure surveillance. However, based on
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
the ‘‘Bag of Words’’ assumption, these methods require human intervention to distinguish similar product features and cannot perform the task of eliciting market structure in a completely automatic manner. For example, Lee and Bradlow (2011) use manual reading to identify 39 distinct clusters of product attributes from 99 clusters extracted by K-means. Their studies motivate us to tap into the online product reviews through text-mining techniques. In our study, we propose a hybrid method to look into the online product reviews. The contributions of this research are multifold: (1) We utilize topic modeling to save efforts of manual reading to identify product features with synonyms which are likely to distort the elicited market structure (Lee and Bradlow 2011), and thus enhance the automatic level of visualizing market. Traditional text classifying methods assume that a term in a consumer’s review represents a unique concept or level of attribute dimensions, thus creating a superfluous number of product features when synonyms are utilized by different product reviewers. In response, Topic modeling technique is a generative probabilistic model of latent dirichlet allocation (LDA), which is capable of grouping synonyms into the same topic and derive the probability distribution of a topic across all documents simultaneously (Blei et al. 2003). (2) To exploit the strength of topic modeling, a method named WVAP is developed based on Scree plot technique to filter noises in the outputs of Topic modeling, and to elicit valid weight matrix of alternative brands. Since noises result from Topic modeling will accumulate and obscure the hidden differences among valid posterior probabilities, they prevent Topic modeling from accurately evaluate competitive brands. Furthermore, there exist serious format discrepancy among Topic modeling, TOPSIS, and MDS in inputs and outputs. With the help of the WVAP method, these three frameworks can be better integrated to automatically perform market structure visualization. In short, to tackle the data congruence issue, we use WVAP to transform the results of topic modeling into inputs feasible for other methods so as to efficiently implement the proposed hybrid model. (3) We adopt the TOPSIS method to evaluate products and derive product rankings (Chen and Hwang 1992). Although other MCDM (Multi Criteria Decision Making) methods such as AHP (Analytic Hierarchical Process) are also suitable (Kou et al., 2012), TOPSIS is computationally simple once the weights of each criterion are known by WVAP method. Besides developing perceptual maps of market structure, our ranking of products helps marketers better grasp products’ performance in general. The remainder of the paper is organized as follows. In Section 2, we review methods for obtaining market structure data. Section 3 proposes the WVAP model to integrate topic modeling, TOPSIS, and MDS. Section 4 presents a case study to illustrate the application of
59
the proposed framework. We summarize the case study and generalize its managerial implications in Section 5. In Section 6, we discuss the limitation of this research and future research direction. Finally, the conclusion of this research is given in Section 7.
2. Literature review Market structure research has played a central role in providing marketers with insights into competitive landscape and marketing opportunities, and in turn prompting marketing actions (Elrod et al. 2002). Marketing strategies, such as market segmentation, brand positioning, pricing, and new product development, are developed based on market structure research (Erdem and Keane 1996). In order to infer market structure, researchers have applied multi-dimensional scaling, conjoint analysis, and cluster analysis to analyze survey data (Elrod 1991; DeSarbo et al. 1991; Green and Srinivasan 1978). Although the validity of these methods is often supported, there are limitations. For example, due to accessibility of data and prolonged data collection process, samples collected by survey or focus group tend to be in small size and cross sectional. It is thus extremely time-consuming to infer dynamic market structure utilizing survey and focus group. Another limitation of these methods is that the product features need to be predetermined before initiating the data collection process. To address the deficiency of the conventional methods in the Internet era, new methods have been developed to analyze the market structure by automatically extracting product characteristics and brand positions from online customer reviews. The online user-generated data provides a new opportunity to observe the market (Urban and Hauser 2004). Through text mining, researchers can discern patterns and trends hidden in the abundant product reviews (Lee and Bradlow 2011; Netzer et al. 2012). By ‘‘listening in’’ online voice, researchers have made significant improvement in tapping into user-generated content to extract product features and build competitive market landscape. Armed with online product reviews and text-mining techniques, dynamic market structure could be derived more conveniently due to the growing body of unsolicited user-generated online content. Also, product attributes hidden in online consumer discussion need not be predetermined. With the help of an unsupervised learning algorithm, product features which concern consumers most could emerge from the large-scale, unstructured text data automatically. Despite breakthroughs in mining market structure from online product reviews, several limitations are present: (i) Most text-mining algorithms, which adopt ‘‘Bag of Words’’ assumption, are likely to assign similar words to different categories, and require additional human reading to combine them. Specifically, these algorithms assume that every word or term is a unique concept of a product attribute. Given that synonyms are prevalent in many text documents, the assumption would lead to superfluous product features and inaccurate market structure.
Table 1 Literature on market structure most relevant to our studies. Market structure research methods
References
Strengths
Weaknesses
Traditional approaches Text-mining techniques
Elrod (1991), DeSarbo et al. (1991), Green and Srinivasan (1978) Lee and Bradlow (2011), Netzer et al. (2012)
Reliable methods of inferring market structure from survey-based data Deriving dynamic market structure and extracting product attributes from user-generated content
Unsuitable for deriving dynamic market structure; product attributes need to be predetermined ‘‘Bag of Words’’ and unique term assumption; human intervention in identifying crucial product attributes; excluding rankings of products
60
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
(ii) The literature derives brand positioning without considering the rankings of the products. Through both the positioning and ranking information, marketers can develop a better view of their own product and give a comprehensive assessment of competitors’ products. The strengths and weaknesses of the current literature are summarized in Table 1, which justifies the need of introducing topic modeling and TOPSIS into the proposed model. By grouping synonyms together in one topic, Topic modeling can be a more efficient method to evaluate brands. In addition, the TOPSIS framework is particularly suitable for deriving rankings of products or brands in studying the market structure. 3. The proposed method To better exploit the user-generated data and visualize the market structure, we now propose a framework to integrate topic modeling, MDS and TOPSIS techniques. The main obstacle is that outputs of Topic modeling, which assign tiny probabilities to unimportant words in a topic, are not valid inputs to TOPSIS and conventional market structure research methods. To overcome this difficulty and derive the perceptual map with product attributes and rankings, we propose the WVAP method and decompose the overall process into six steps (see Fig. 1): Step 1: Collect online product reviews by web scraping techniques. We use Python language to code and perform the task due to its power and simplicity. Within Python, we install a BeautifulSoup module to take advantage of its pre-built web scraping feature. We then scrape text data of interest such as date of review, pros, cons, and review summary off the target web page. A number of loops are created to repeat the above tasks so as to maximize the reviews collected. Step 2: Prepare qualified data for text mining by data-cleaning process. Since the raw data scraped off the target web page usually contains unwanted characters, it is necessary to remove the punctuations, numbers, and stop words following text-mining procedure (Weiss et al. 2004). To avoid tallying one word in various grammar contexts, we only retain the stem of a word. Finally, comments of interest such as pros and cons were split from the text and transformed into Corpus format which is ready to be processed by statistical mining methods.
Fig. 1. The proposed framework to map unstructured text data.
Step 3: Extract essential topics from product reviews by topic modeling method. Topic modeling is a probabilistic generative model, which assumes that each text is a sample of joint distribution of hidden topics, topic proportions, and topic assignment (Blei et al. 2003). As a process of latent dirichlet allocation, topic modeling is capable of discovering latent topics among a colossal quantity of text data and identifying most frequently adopted words in each topic. Therefore, applying topic modeling to product reviews can discern key product features from digital communication in the form of free texts. Based on the prior probability assumption of dirichlet distribution, we identify hidden topics and present two posterior probability matrices similar to Geman and Geman (1984). One matrix is the converged distribution of extracted topics for all documents, while the other is the converged distribution of words within each topic. Step 4: Determine the weights of topics according to the valid posterior probabilities. Because a data format gap exists between the outputs of topic modeling and the inputs of market structure analysis, to transform the topic modeling outputs into legitimate inputs for TOPSIS and MDS, we derive Weights from Valid Posterior Probability (WVAP). Each text document in the corpus of online product reviews has a relatively small number of valid terms, as opposed to long news articles or academic papers. Thus, the online review tends to be brief with only several major topics standing out (i.e., with valid posterior probability), while the posterior probabilities of other minor topics are negligible and behave as noises. Thus, we apply Scree plot technique, which is utilized in PCA (Principal Component Analysis) to obtain valid attributes by calculating the change of their variances to filter noises. Otherwise, when we later adopt TOPSIS technique by summing up all posterior probabilities, those noises (although approximate zero) will accumulate and cover valid probabilities, making the evaluation results among brands obscure. Let (Pij)mn be the posterior distribution matrix of documents across all topics, where m is the number of documents and n is Pn the number of topics. Thus, 0 6 Pij 6 1 and j¼1 P ij ¼ 1. Let Pi be the ith row in (Pij)mn and Gt be the tth (t = 1, 2, 3, , n) largest value in Pi. We define Mk (k = 1, 2, 3, , n) as a sequence of sets and M k ¼ Pi n fG1 ; ; Gk g, where ‘‘n’’ represents the difference operation of sets. For convenience, we denote M0 = Pi, which means
Fig. 2. Steps to implement the WVAP method.
61
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
M0 does not exclude any element from Pi. Thus, the variance of M0, denoted by Var(M0), is the same as the variance of Pi. Consequently, Var(Mk) represents the variance of Pi excluding the k 1 largest elements. From the Scree plot technique, we detect noises by observing the change of Var(Mk). It is obvious that Var(Mk) is a decreasing function of Mk, since the retained probabilities will become similar after larger values are ruled out. After calculating Var(Mk) for each k, we can draw a Scree plot of Var(Mk) with respect to each Mk. The Scree plot should be a decreasing line and become flatter close to zero while k gets larger. Thus, the corresponding posterior probabilities of the steeper part of the line could be regarded as being valid, while those of the flatter part of the line ought to be considered as noises. The elbow point which differentiates the steeper part and the flatter part is detected by choosing the point which has the maximum change rate of Var(Mk). Consequently, we retain the valid posterior probabilities in (Pij)mn and replace those noises
in (Pij)mn with zeros. To offset the possible disturbance caused by different amounts of product reviews of each product, we scaled the retained posterior probabilities by multiplying each row the proportion of amount of reviews of each product. The updated matrix of posterior probabilities is denoted as ðP0ij Þmn . Next, we add up each column of ðP 0ij Þmn and select a number of important columns (or topics) according to the obtained sums. In the following experiments, five important topics are chosen since their sums are over 70 percent of the sum of all topics. We also test the results of derived market structure based on six topics, and no significant difference is found. Then, only those important columns of ðP 0ij Þmn are retained and the updated ðP0ij Þmn is denoted as ðP 00ij Þmn . Further, we identify rows of ðP 00ij Þmn which correspond to the reviews of the same brand and split ðP00ij Þmn by those indentified rows into a number of sub-matrices. For example, if the reviews between the ath row and the bth row are about the same
Reviewer ID: 23 Date of Review: Dec 25 '11 Pros: Small, lightweight, sturdy, quick, easy to use, network, call quality, battery life, good camera. Cons: Not the fastest, no memory slot, no user-replaceable battery, no dedicated call buttons. Summary: I would recommend the iPhone for someone who likes a dependable, easy-to-use, stylish smartphone. Also, anyone who loves music and taking pictures. Recommended: Yes Amount Paid (US$): 199.00 Recommended For: Stylish Trendsetters - Hip and Trendy Fig. 3. An example of a product review downloaded from www.epinions.com source (accessed October 3, 2012): http://www.epinions.com/review/Apple_iPhone_4_Black_16_GB/content_574456696452.
750
Count 500
250
0 2000
2002
2004
2006
2008
2010
2012
Date of Review
(a) Conventional Phones
200
150
Count 100
50
0 2000
2002
2004
2006
2008
2010
Date of Review
(b) Smart Phones Fig. 4. Frequency distribution of the number of reviews.
2012
62
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
brand, a sub-matrix ranging from the ath row to the bth row is extracted from ðP 00ij Þmn . Therefore, those sub-matrices refer to distinct brands, since reviews on different brands are assembled into the overall corpus in order. Finally, for each sub-matrix, we sum each column to derive the weights of a brand which eventually constitutes the weight matrix of all brands. The flowchart for implementing the WVAP method is given in Fig. 2. Step 5: Build the perceptual map of ranked market structure by combining the method of MDS. We first calculate the Euclidean distance between two products based on the weight matrix of products derived by WVAP. Then, we apply the MDS to establish a perceptual map of market structure by projecting the relative positions of products onto a two-dimensional plane. Step 6: Derive ranking of alternative products by TOPSIS. Hwang and Yoon (1981) and Chen and Hwang (1992) developed the TOPSIS method to rank alternatives over multiple criteria. In the TOPSIS method, the positive (or negative) ideal solution is defined as a vector consisting of maximum (or minimum) in each column of the weight matrix obtained in step 4. Then, TOPSIS suggests that the ranking of alternatives could be obtained by measuring the distances of each alternative to positive ideal solution and negative ideal solution (Opricovic and Tzeng 2003). Accordingly, the relative closeness of a product (or alternative) Ei with reference to positive ideal solution P+ is calculated as follows:
CC i ¼
þ di
di þ di
ð1Þ
þ
where di and di represent the n-dimensional Euclidean distance of Ei to positive ideal solution P+ and negative ideal solution P, respectively. Thus, the greater the value of CCi is, the better evaluation the product would obtain. Finally, with the aid of ranking results of the products, both the relative positions and priorities of products could emerge from the unstructured text data.
4. The case study To validate the proposed model for deriving dynamic market structure, we examine the consumer-generated reviews of cell phones. A total of 10,134 reviews, among which there are 7874 conventional phone reviews and 2260 smart phone reviews, were crawled off of the Epinions website (www.epinions.com) on October 3, 2012. The text dataset includes 19 conventional phones and 17 popular smart phones, each of which is reviewed more than 30 times. Fig. 3 shows a typical review of the iPhone 4. Although this review may seem straightforward, the task of comprehending thousands of documents like this is not trivial and is beyond the normal capacity of a human brain. An individual reviewer’s comment is classified as positive or negative according to its polarity. The frequency distribution of 10,134 reviews over time is summarized for conventional phones and smart phones, respectively in Fig. 4. Fig. 4 suggests that there is only one product life cycle for conventional phones, while there are four such cycles for smart phones. Furthermore, the oscillation pattern divides the smart phone market into four periods: 2000–2004, 2005–2006, 2007–2009, and 2010–2012, approximately corresponding to the number of reviews over time. The first period (2000–2004) is the introductory phase, while the others reflect the different stages of the product life cycle. We believe grouping smart phones according to life cycle stages provides a better understanding of the dynamics of the market. This case study is technically divided into three steps. First, we group product reviews into pros and cons since the polarity of each review has already been predefined. Second, we elicit key product attributes of cell phones from the positive and negative reviews, respectively. Finally, we derive the market structure based on the identified product attributes. 4.1. Mining online reviews of conventional phones
Fig. 5. Topic distribution matrix derived by topic modeling.
Following step 1 in Fig. 1, we clean the product reviews by removing punctuations, numbers and stop words, and transform them into the corpus data format, so as to qualify them for topic modeling. As a result, we are left with 7851 valid reviews on conventional phones. Further, to match the review text with topic modeling, we need to decide on the optimal number of topics in advance. For a given number of topics, we use perplexity and log likelihood to measure its fitness. The perplexity, defined as an exponent function of the negative log likelihood per term, is a monotonically decreasing function of log likelihood. It reaches its minimum when log likelihood arrives at its maximum. We experiment with different numbers of key review topics (attributes), ranging from 5 to 150 topics with an increment of 5 topics. After applying topic
Fig. 6. Topic distribution matrix with filtered noises.
Table 2 Key topics in positive reviews of conventional phones. Topic 19 Speaker Function Low Full Feature
Topic 7 0.455 0.172 0.075 0.073 0.042
Light Weight Girl Buyer Decent
Topic 14 0.509 0.417 0.008 0.003 0.003
Screen Clarity Outlook Good Option
Topic 50 0.347 0.265 0.119 0.1 0.024
Use Easily Small Cheap Adequate
Topic 21 0.252 0.217 0.169 0.126 0.034
Battery Long Life Last Cover
0.463 0.131 0.096 0.071 0.051
63
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74 Table 3 Key topics in negative reviews of conventional phones. Topic 56 Life Battery Full Storage Massive
Topic 46 0365 0.357 0.112 0.087 0.043
Friend Use Stylish Box Multifunction
Topic 6 0.235 0.147 0.113 0.066 0.056
Navigation Touch screen Accuracy Factor Interface
modeling method to positive review data, we designate the number of topics with the lowest perplexity value as the optimal one. It is found that there are 50 topics hidden in the consumers’ positive reviews of conventional phones, with perplexity of 166.02 and log likelihood of 58,994. Next, WVAP method is applied to filter noises. The scaled posterior distribution matrix derived by Topic modeling is presented in Fig. 5. Note every row is a topic distribution scaled by multiplying the proportion of amount of reviews of each product. For instance, the 147th product review refers to the cell phone LG enV2 whose updated proportion is
amount of product reviews of LG enV2 total amount of product reviews
Topic 5
72 ¼ 7851 ¼ 0:009. Hence, the row corre-
sponding to document No. 147 is multiplied by 0.009. The Scree plot technique is then adopted to identify noises in the matrix and assign each noise zero. We denote the number of zeros assigned for each row as Zi, where i is from 1 to 7851. The mode of Zi is 45 and its percent is 0.426. This shows that only five valid probabilities are retained for nearly half of the entire rows. We add up each column of the updated matrix and select five most important topics according to the column sums of valid posterior probabilities (Fig. 6). Note that the variance of column sums in Fig. 5 is 0.315 and that in Fig. 6 is 1.072. This major enhancement of variance demonstrates that the process of filtering noises is able to unfold the true weights differences among extracted topics. Table 2 displays the words of five selected topics to mark the distinct features of conventional phones. Under each topic, the chosen words are listed on the left and the corresponding probabilities are on the right. The results indicate that the main concerns of users of conventional phones include speaker, weights, screen, easy-to-use, and battery life. In order to study the conventional phone market without prejudice, negative online reviews are also examined. Unlike positive reviews which clearly specify preferred attributes of a smart phone, a handful of negative reviews are surprisingly nonspecific. Examples of vague statements include ‘None at all’, ‘Everything else’, and ‘Nothing so far’, etc. For these negative reviews, an extra data cleaning step is applied to remove these terms. Once the negative reviews are cleaned, the steps in Fig. 1 are applied again to identify the optimal number of topics based on model fitness
0.147 0.123 0.114 0.097 0.069
Player Software Carry Around DVD
Topic 57 0.274 0.094 0.066 0.066 0.056
Graphic Operation System Camera Stability
0.203 0.184 0.135 0.058 0.048
indices. We find there are 60 topics hidden in the reviews with perplexity of 305.96 and log likelihood of 42880.22. As indicated in Table 3, the key topics in negative reviews are similar to those in positive reviews except that consumers complain about software, graphics and camera. The validity of the extracted topics must be verified before using these topics to visualize market structure. Thus, we compare the results with those derived by K-means which is a typical and well-known clustering method. In order to classify words, K-means calculates distances (or similarities) between columns of the corpus matrix since each column refers to a word. The results of Kmeans indicate that there are 45 and 50 extracted features among positive reviews and negative reviews of conventional phones, respectively. Features similar to topics in Table 2 and Table 3 are found and presented in Table 4. It supports the validity of topics obtained by topic modeling. Further, synonyms which are supposed to be grouped into the same feature are assigned to different
Table 5 The weight matrix of conventional phones based on positive reviews. Conventional phones
Topic 19
Topic 7
Topic 14
Topic 50
Topic 21
Kyocera 2760 LG enV2 LG VX6000 Motorola V70 Nokia 3360 Nokia 5160 Nokia 5185i Nokia 6610 Nokia 8210 Nokia 8260 Nokia 8290 Palm 90 Rim 9000 Samsung 6100 Samsung A500 Sony Ericsson R280 Sony Ericsson T68 Sony Ericsson T616 Sony Ericsson w580
0.471 0.000 0.000 0.162 0.311 0.456 0.061 0.203 0.406 0.369 0.000 0.000 0.055 0.216 0.000 0.430 0.153 0.153 0.056
0.219 0.163 0.055 0.100 0.000 0.541 0.109 0.000 0.165 1.182 0.315 0.103 0.000 0.300 0.108 0.055 0.108 0.056 0.064
0.105 0.096 0.097 0.266 0.251 0.853 0.000 0.000 0.240 0.836 0.201 0.000 0.000 0.048 0.113 0.049 0.103 0.000 0.048
0.255 0.000 0.058 0.148 0.513 0.612 0.217 0.000 0.503 0.361 0.175 0.050 0.000 0.048 0.000 0.367 0.051 0.000 0.000
0.049 0.098 0.000 0.098 0.100 0.457 0.097 0.049 0.203 0.804 0.247 0.049 0.049 0.000 0.000 0.049 0.066 0.157 0.000
Table 4 Features extracted by K-means from reviews of conventional phones. Extracted features of positive reviews similar to topics in Table 2 Feature 3 Speaker Function Low Full Perfect
Feature 8 Compact Weight Girl Buyer Decent
Feature 23 Resolution Screen Appearance Good Option
Feature with synonyms to feature 3 Feature 26 Usage Easy Small Cheap Versatile
Feature 33 Battery Life Durable Last Cover
Feature 18 Software Take Around Provide DVD
Feature 45 Camera System Reliable Graphic Operation
Extracted features of negative reviews similar to topics in Table 3 Feature 5 Life Battery Last Storage Massive
Feature 12 User Friendly Stylish Box Simply
Feature 17 Navigation Touch screen Accuracy Factor Interface
Feature 30 Phone Headset Ringtone Quality Fantastic Feature with synonyms to feature 45 Feature 32 Picture Quality Image Device Application
64
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
features in the results of K-means. For example, in the extracted features of positive reviews, feature 30 with words about phone and headset is obviously similar to feature 3 of speaker function. In the case of negative reviews, both feature 45 and feature 32 focus on camera, graphics, picture and image. Thus, manual reading is needed to identify features with synonyms and combine them together. Otherwise, with redundant features, the map of market structure is likely to be distorted. With the introduction of topics between words and documents, topic modeling is capable of grouping synonyms under the same topic, and therefore saves lots of efforts of obtaining distinct product features. For example, although words, such as phone, headset and ringtone, have
relatively small assigned probabilities, they are under the same topic with words like speaker and function in the results of topic modeling. Retaining only the five selected topics in Fig. 6 and adding up documents which belong to the same product as suggested by the last two steps in Fig. 2, we obtain the weight matrix of conventional phones based on positive reviews and present it in Table 5. When applying the WVAP method, we update row indexes belonging to a product in the corpus matrix if any reviews of the product are deleted during the data cleaning process. The weight matrix is a vital intermediate result which serves as the input to market structure analysis. Each entry in the matrix is the value assigned
Nokia 8260
0.6 Nokia 5160
0.4
F2 (17.15%) 0.2 Nokia 8210
Kyocera 2760 Nokia 8290
Nokia 3360
0.0
Samsung 6100 Motorola V70Sony Ericsson T68
Sony Ericsson R280
Samsung A500 LG VX6000 Palm 90 Rim 9000 Sony Ericsson w580
Nokia 6610
-0.2 -0.2
-0.1
LG enV2
Sony Ericsson T616
Nokia 5185i
0.0
0.1
0.2
F1 (75.33%) Fig. 7. MDS map of conventional phone market based on online positive reviews.
1.0 Nokia 8260
0.8 Nokia 5160
Ratings 0.6
0.4
Motorola V70
0.2
Nokia 8210
Nokia 3360
Kyocera 2760
Samsung 6100
Nokia 5185iNokia 6610
LG enV2 LG VX6000
5
Sony Ericsson R280
Nokia 8290
Palm 90 Rim 9000
10
Samsung A500
Sony Ericsson T68 Sony Ericsson T616 Sony Ericsson w580
15
Conventional phones Fig. 8. Ratings of conventional phones based on online positive reviews.
65
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
to a conventional phone under a particular key topic in the period, and the column sum equals the topic’s weight, which can be used to understand the strengths and weaknesses of each conventional phone. The market structure of smart phones can be visualized in two ways. First, the perceptual map of product positioning is built to reveal the mutual distances among smart phones. Second, rankings of smart phones are derived to show the overall performance of each product. We adopt the MDS method to plot the perceptual map. As key topics differ in importance, the coordinates of the conventional phone matrix are derived by multiplying the weight matrix with the topic weights. Afterwards, the coordinate matrix becomes the Euclidean distance matrix, which forms the perceptual map of conventional phones. The best-fitting 2-dimensional MDS map of conventional phones from positive online reviews is illustrated in Fig. 7. It is a 2-dimensional representation of the
Table 6 The weight matrix of conventional phones based on negative reviews. Conventional phones
Topic 19
Topic 7
Topic 14
Topic 50
Topic 21
Kyocera 2760 LG enV2 LG VX6000 Motorola V70 Nokia 3360 Nokia 5160 Nokia 5185i Nokia 6610 Nokia 8210 Nokia 8260 Nokia 8290 Palm 90 Rim 9000 Samsung 6100 Samsung A500 Sony Ericsson R280 Sony Ericsson T68 Sony Ericsson T616 Sony Ericsson w580
0.368 0.356 0.358 0.267 0.359 0.114 0.258 0.423 0 0.105 0.332 0.213 0.309 0.308 0.261 0.157 0.251 0.315 0.267
0.213 0.258 0.296 0.195 0.256 0 0.241 0.361 0.159 0 0.198 0.3678 0.382 0.251 0.417 0.365 0.267 0.237 0.117
0.269 0.113 0.168 0.143 0.301 0 0.211 0.216 0.135 0 0.206 0.207 0.412 0.127 0.326 0.218 0.198 0.214 0.235
0.315 0.212 0.212 0.264 0.158 0.101 0.301 0.368 0.164 0.034 0.268 0.194 0.291 0.135 0.309 0.197 0.32 0.264 0.31
0.401 0.204 0.318 0.138 0.197 0.051 0.289 0.159 0.204 0.062 0.301 0 0.307 0.229 0.336 0.194 0.157 0.317 0.241
original Euclidean distance matrix based on two eigenvectors: F1 and F2. Note that MDS projects a high-dimensional data matrix into the 2-dimensional plane: F1 and F2, which do not correspond to specific features of products. They are chosen because they have the largest eigenvalues and can thereby form the best-fitting representation of the original Euclidean distance matrix. The adequacy of the 2-dimensional representation can be assessed by the sum of eigenvalues of F1 and F2 as discussed in Mardia et al. (1979). To demonstrate the validity of the proposed method, we also adopted the method based on bag of words assumption (BWA) presented by Lee and Bradlow (2011) to obtain the MDS map and compared the sum of eigenvalues of F1 and F2 between these two methods. As shown in Fig. 7, the sum of eigenvalues of F1 and F2 equals 92.48% while the sum becomes 88.75% by BWA method. Since greater value of the sum means better accuracy, the finding indicates that the proposed method is more effective in building the MDS map. The MDS map of conventional phone market could reveal competitive strategies adopted by different products. For example, Nokia, as the leader brand of conventional phones, has grouped their products into two categories. A handful of products, such as Nokia 8260 and Nokia 5160, were introduced to the market by marketing strategy of differentiation, while Nokia 6610 and Nokia 8290 were made to resemble their competitors. In the second part of visualizing market structure, rankings of smart phones are computed by TOPSIS based on the weights in Table 4. Unlike MDS maps, which help to identify competitors in different segmented markets, the TOPSIS gives the rating of each product (Fig. 8). The finding suggests that Nokia 8260 is the most favorite cell phone while Rim 9000 is the least popular one. More insights about the conventional phone market can be attained by analyzing the market structure derived from negative reviews. Thus, the weight matrix of conventional phones from negative online reviews is obtained similarly (Table 6). The MDS map elicited from the weight matrix reveals which conventional phones receive complaints of similar extent (Fig. 9). For example, the positions of Nokia 8260 and Nokia 5160 are far away from other conventional phones. It implies that Nokia 8260
Rim 9000 Samsung A500
0.2
Kyocera 2760
Nokia 6610
Sony Ericsson T616 LG VX6000 Nokia 3360
Sony Ericsson T68 Sony Ericsson R280
0.0
F2
Palm 90
Nokia 8290
enV2 Motorola Samsung 6100
V70
(16.26%)
-0.2
Nokia 5185i LG
Sony Ericsson w580
Nokia 8210
Nokia 5160
-0.4
Nokia 8260
-0.2
-0.1
0.0
0.1
F1 (62.42%) Fig. 9. MDS map of conventional phone market based on online negative reviews.
66
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
and Nokia 5160 have either less or more online consumer complaints (based on the TOPSIS analysis) than those of other phones. Similarly, we calculated the sum of eigenvalues of F1 and F2 by the BWA method to check the validity of the proposed method. As shown in Fig. 9, the sum of eigenvalues of F1 and F2 is 78.68% while the sum is 76.54% by the BWA method. The finding suggests that the proposed method generates a more accurate MDS map than the BWA method. The ranking results (Fig. 10) derived by TOPSIS suggest that most conventional phones receive high rating except for Nokia 8260 and 8210. Since the ratings are from negative reviews, it implies that Nokia 8260 and Nokia 8210 have less online consumers complaints than others. In short, the results from positive and negative reviews are generally consistent. For instances, conventional phones such as Nokia 8260 and Nokia 8210, which are popular based on the market structure extracted from positive reviews, are also less complained. On the contrary, Rim 9000 which is least preferred in the positive reviews is found to have more flaws than other conventional phones.
4.2. Mining online reviews of smart phones As aforementioned, there are four distinct periods in the case of smart phones. Through eliciting key topics in positive reviews of smart phones, we identify the corresponding indices of optimal number of topics and summarize them in Table 7. Table 8 displays the five topics of the highest frequencies in each period to mark the distinct features for that time. Under each topic, the chosen words are listed on the left and the corresponding probabilities are on the right. The meaning of most words is obvious, while a few technical terms such as AIM and Siri need to be clarified by retrieving the original review contexts. For example, in the period of 2000–2004, topic 18 focuses on the convenience of carrying a smart phone, topic 37 centers on the integrated functions, topic 61 emphasizes the surfing task on the Internet, topic 50 underscores the design and outlook features, and finally topic 44 highlights the importance of keyboard and screen.
Table 8 also shows the evolution of these topics, i.e., the key topics hidden in online product reviews tend to vary over time. For instance, unlike the key aspects discovered in 2000–2004, online consumers in 2010–2012 are passionate about tech-savvy functions such as application store access, Siri, and Retina Displays. These changes are attributed to the rapid technological advancement of smart phones in recent years. Conversely, some topics remain stable over time, e.g., user friendliness and ease of use. These findings suggest that recent smart phones are probably more difficult to maneuver than conventional phones due to their increasing complexity. Given the optimal number of negative topics for each period presented in Table 9, we derive key topics through topic modeling and summarize them in Table 9. Interestingly, the key topics extracted from negative reviews appear stable across all four time periods, as opposed to those extracted from positive reviews, which are more likely to reflect newly developed functions of smart phones. For example, Topics 56, 27, and 28, in the 1st, 2nd, and last periods, respectively, share similar attributes of smart phones that indicate poor antennae or negative experience of dropped calls. Topics 3, 109, and 56 in the 1st, 2nd, and last periods, respectively, concern the size and weight of a smart phone. Additionally, battery life of smart phones has been a serious problem since 2007 as shown by topic 24 in the 3rd period and topic 20 in the last period. Similar to the case of conventional phones, we compare the results with respect to K-means to check the validity of the extracted topics. For each period, features similar to topics presented in Tables 8 and 10 are found, and the validity of elicited
Table 7 The optimal number of topics based on positive reviews. Period Optimal number of topics Perplexity Log Likelihood
2000–2004
2005–2006
80
60
52.04 2971.89
119.55 6242.72
1.0
Rim 9000
0.8
Kyocera 2760 LG VX6000
Ratings 0.6
Nokia 6610
Sony Ericsson Nokia 8290
Nokia 5185i
Nokia 3360
Samsung A500
LG enV2 Motorola V70
Palm 90
T616 Sony Ericsson T68 Sony Ericsson Sony Ericsson w580 Samsung R280 6100
Nokia 8210
0.4
Nokia 5160
0.2
5
Nokia 8260
10
15
Conventional phones Fig. 10. Ratings of conventional phones based on online negative reviews.
2007–2009 70 186.87 12773.93
2010–2012 40 142.03 9480.93
67
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74 Table 8 Key topics in positive reviews of smart phones. Period
Topic 18
2000–2004
Phone PDA Carry Afford Hardware
Topic 37 0.5 0.463 0.036 0 0
Topic 23 2005–2006
Look Cool Amazing Intend Ability
Easy Usage Eye Fall Simply
Application Store Wi-fi Accessory Airplay
Web Email Aim Instant Browser
Battery Life Super Future Provide
User Friendly Fantastic Take Absolute
Table 9 The optimal number of topics based on negative reviews.
Optimal number of topics Perplexity Log likelihood
2000–2004
2005–2006
100
120
29.58 2310.08
50.77 4948.35
2007–2009 80 147.1 11299.94
Program Available Fine Install Unlimited
Screen Touch Attractive Crisper Choose
2010–2012 85 91.74 6322.04
Keyboard Android Qwerty Overall Market
Sync Design Outlook Camera Concept
Topic 44 0.176 0.118 0.118 0.059 0.059
Topic 16 0.269 0.115 0.077 0.077 0.038
User Friendly Slim Decent Fun
Application Store Incredible None Launch
Siri Faster icloud Core Accurate
0.167 0.167 0.111 0.111 0.111
Speaker Organize Sync Calendar Direct
0.269 0.154 0.115 0.115 0.077
Topic 2 0.55 0.078 0.025 0.025 0.025
Topic 22 0.176 0.137 0.137 0.118 0.078
Keyboard Screen AIM Color Cool Topic 52
0.304 0.261 0.087 0.087 0.043
Topic 1 0.442 0.423 0.154 0.019 0
Topic 19 0.34 0.319 0.085 0.064 0.043
topics by Topic modeling is supported. Meanwhile, features with synonyms to one another are also detected. Specially, the result of period 2010–2012 is chosen as an example and shown in
Period
0.387 0.38 0.232 0 0
Topic 62 0.429 0.321 0.071 0.036 0.036
Topic 7 0.867 0.044 0.033 0.022 0.011
Browser Web Price Fast Mail
Topic 50
Topic 33 0.188 0.188 0.156 0.094 0.063
Topic 64 0.947 0.011 0.011 0.011 0.011
Topic 35 2010–2012
0.316 0.316 0.105 0.053 0.053
Topic 54 0.323 0.29 0.065 0.065 0.065
Topic 49 2007–2009
Integration Palm Phone Afford Final
Topic 61
Function Form Load Versatile Perfect
0.571 0.167 0.048 0.024 0.024
Topic 14 0.457 0.142 0.063 0.047 0.047
Phone Retina Talk Feel Aesthetic
0.644 0.051 0.051 0.017 0.017
Table 11. In the context of positive reviews, both features 21 and 17 concentrates on keyboard, Qwerty, button and touch, whereas, features 28 and 30 indicate similar online consumer complaints about signals of antenna. Since smart phones are more complex than conventional phones, we also find the extent to which key topics obtained from online user-generated content indeed represent the preferences of consumers at large and their off-line shopping behaviors (Pan and Zhang 2011). Lee and Bradlow (2011) maintain that text-mining techniques complement rather than substitute for traditional methods of inferring market structure. We thus compare the extracted topics in this study with the survey-based results and expert buying guides (Table 12). We find that topics derived from
Table 10 Key topics in negative reviews of smart phones. Period
Topic 56
2000–2004
Poor Antenna Coverage Address Appall Topic 94
0.094 0.053 0.053 0.053 0.053
Topic 16 Sync Cradle Serial USB Backlight Topic 75
0.333 0.333 0.167 0.167 0
Color Screen Expansion Capability Slot Topic 109
0.359 0.193 0.179 0.089 0.089
Hard Button Black Cause Delay Topic 27
0.125 0.063 0.063 0.063 0.063
Bulky Phone Little Drive Easy Topic 19
0.231 0.154 0.154 0.077 0.077
2005–2006
Buzz Memory Occasion Crash Card
0.14 0.14 0.14 0.087 0.07
Reset Device Difficult Annoy Applications
0.15 0.10 0.10 0.05 0.05
Size Phone Require Adapt Larger
0.191 0.095 0.095 0.048 0.048
Drop Call Experience Function Miss
0.286 0.214 0.143 0.071 0.071
Internet Little Slow Pocket Compute
0.211 0.158 0.158 0.105 0.053
Topic 24 2007–2009
Life Battery Margin Security Adobe
Topic 14 0.5 0.472 0.139 0.139 0
Topic 20 2010–2012
Life Battery Adapt Application Bill
Topic 35
Keyboard Horrible Bold Crazy Cripple
Topic 49 0.73 0.081 0.027 0.027 0.027
Topic 56 0.509 0.49 0 0 0
Heavy Pricey Sexy Plan Addict
Topic 17
Curve Learn Trackball Cheap Manage
Topic 68 0.143 0.143 0.114 0.057 0.057
Topic 1 0.438 0.314 0.188 0.063 0
Easily Broken Scratch Amount Holder
Topic 3
System Slow Buggy Operation Laggy
Topic 75 0.128 0.081 0.077 0.077 0.051
Topic 28 0.56 0.12 0.12 0.04 0.04
Antenna Death Grip Issue Else
Video Capability Record Wifi Movie
0.485 0.152 0.152 0.061 0.03
Topic 67 0.421 0.14 0.14 0.088 0.07
Protect Depend Handle Mode Delay
0.20 0.10 0.10 0.10 0.05
68
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
Table 11 Crucial features elicited by k-means from reviews of smart phones. Extracted features of positive reviews similar to topics in Table 8 Feature 8 Application Email Wifi Accessory Airplay
Feature 10 User Compute Friendly ipod Take
Feature 21 Keyboard Android Qwerty Nice Market
Feature with synonyms to feature 21 Feature 35 Siri Fast icloud Camera Core
Feature 38 Phone Retina Amazing Feel Aesthetic
Feature 28 Antenna Death Grip Indoor Issue
Feature 41 Protect Support Handle Mode Delay
Extracted features of negative reviews similar to topics in Table 10 Feature 5 Life Battery Adapt Application Charge
Feature 9 Heavy Pricey Afford Plan Addict
Feature 16 Easily Broken Scratch Amount Holder
Feature 17 Button Touch Dial Fade Cramp Feature with synonyms to feature 28 Feature 30 Signal Wireless Fragile Call Flaw
Table 12 Key attributes of smart phones derived by traditional approaches. Evaluated by traditional approaches
Type
Key attributes of smart phones
InsightExpress (2004)
Survey sample size: 600 Survey sample size: 23,000 Expert buying guide
Ordered by importance: brand, extra connectivity options (bluetooth, infrared, Wi-Fi), price, operating system, multimedia, device size, keyboard, computer integration Ordered by importance: app availability, ease of typing, battery life, screen size, ease of use, appearance, price, manufacturer brand Not ordered by importance: apps (bluetooth, camera, document editing, GPS navigation, hearing-aid compatibility, media player), ease of use (preset and customer text messages, programmable shortcuts), QWERTY keyboard, accessories (speakerphone, Standard headset connector), touch screen, voice command, Wi-Fi
TouchType and Smartphone Experts (2011) ConsumerReports (2012)
Table 13 The weight matrix of smart phones based on positive reviews. Periods
Smart Phones
Topic 18
Topic 37
Topic 61
Topic 50
Topic 44
2000–2004
Kyocera 7135 Kyocera 6035 Palm 600 Palm 650 RIM 8100 Sharp Sidekick II
1.313 1.259 0 0 0 0
0.847 1.275 0 0 0 0
0 0.351 0 0 0 0.989
0 0.238 0.806 0 0 0.862
0 0 0 0 0 1.862
Topic 23
Topic 54
Topic 33
Topic 16
Topic 52
0.437 0.193 0 0 1.386 0.747 0.57
0 0 0 0 0 0 2.52
0 0 0 0.677 0.95 0.825 0
0 0.193 0.352 0.274 0 0.847 0.475
0 0.125 0 0.968 1.007 0 0
Topic 49
Topic 64
Topic 62
Topic 1
Topic 2
1.192 0.477 0 0.247 0 0.085 0.236 0.606 0 0.27 0
0.513 0 0.411 0 0 0.124 0 0.777 0.273 0.479 0
1.013 0.514 0 0 0 0.183 0.175 0.36 0.194 0 0
0.827 0.706 0 0 0.222 0.132 0.092 0.246 0 0 0
0.241 0 0 0 0.275 0.292 0.132 1.018 0 0 0
Topic 35
Topic 7
Topic 9
Topic 22
Topic 14
0 0.428 1.232 0.358 0 0 0 0.238 0
0.125 0.125 0.728 0.42 0.176 0 0.133 0.192 0
0 0 1.44 0.259 0 0 0 0 0
0 0.32 0.6 0 0.081 0 0 0 0.516
0 0 0.216 0.194 0.094 0.44 0.102 0 0.467
2005–2006
Kyocera 7135 Motorola Moto MWG Dash Palm 600 Palm 650 RIM 8100 Sharp Sidekick II
2007–2009
iPhone 3G iPhone 3GS Motorola Droid Motorola moto MWG Dash Palm Centro Palm 650 RIM 8100 RIM 9530 RIM 8310 Sharp Sidekick II
2010–2012
iPhone 3G iPhone 3GS iPhone 4 iPhone 4S Motorola Droid Palm Centro RIM 9530 RIM 8310 Samsung Galaxy S II
69
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
online product reviews greatly match the product attributes attained from the traditional approaches. Minor discrepancies are also observed and can be attributed to different research contexts. First, given a specific product, online reviewers tend to overlook the effect of brand in general, which is otherwise considered as a key product attribute by the surveybased approach. Second, the same product feature is phrased differently in expert buying guides and online product reviews. For example, the term ‘‘Siri’’ is synonymous with voice command in Apple products. Third, crucial attributes discovered by conventional methods, such as battery life and price, are found on the list of extracted topics from negative reviews. Therefore, the external validity of the extracted topics from user-generated online reviews is substantiated and can be utilized for inferring market structure of smart phones next (Erdem and Keane 1996). Further, unlike traditional approaches, the framework we proposed not only obtains crucial features of products, but also groups these extracted characteristics into positives and negatives. Similarly, the weight matrix of smart phones from positive reviews are derived and summarized in Table 13. It shows that the Samsung Galaxy S II has very high scores on topics 22 and 14 in 2010–2012, which correspond to Siri and Retina, respectively as two new functions of smart phones in Table 8, while iPhone 4 has high marks across all key topics. The MDS map is displayed in Fig. 11. First, the four graphs represent the evolutionary stages (industry life cycle) of the smart phone market. The period of 2000–2004 is the introductory stage since there were only six smart phones in the market, and they belonged to only four brands. The market attracted more players as it moved to the growth period in 2005–2006. For example,
Motorola and MWG entered the market, and different brands positioned their products against one another. During the period of 2007–2009, the competition continued to intensify. The number of smart phones increased to 11, and they were even closer to each other as compared to those in the earlier periods. The total horizontal and vertical distances in the perceptual map in 2005–2006 are approximately 1.5 and 2.3, respectively, while the corresponding numbers shrank to 1 and 1.5 in the perceptual map in 2007– 2009, signaling the rise of competition. Although competition remained fierce in 2010–2012, the iPhone as a single brand dominated the smart phone market and accounted for four of the nine smart phones on the market. Second, MDS maps can show how effective a smart phone is at differentiating itself from its competitors. For example, Sharp Sidekick II was highly distinct from its competitors in both 2000–2004 and 2005–2006. However, this distinctness disappeared as an increasing number of competitors entered the smart phone market in 2007–2009. Another interesting finding is that iPhone products are likely to advance along a similar R&D direction since all of their products in the last two stages tend to stay around a vertical line. Other smart phones such as RIM 8100 and Samsung Galaxy S II seem unwilling to compete directly with iPhones and instead position themselves in other areas of the market. Similarly, we compared the sum of eigenvalues of F1 and F2 between the BWA method and the proposed method. The results are presented in Table 14. It shows that the proposed method is more accurate in deriving MDS maps in most cases except the period from 2007 to 2009. From the TOPSIS results in Fig. 12, we notice that market leaders in the smart phone industry are likely to change over time. The
Sharp Sidekick II Sharp Sidekick II
0.15
0.15
F2(25.15%)
F2(33.32%)
0.10 0.05 0.00
Palm 600 RIM 8100 Palm 650
0.10
0.05
0.00
-0.05 Kyocera 6035
-0.10
-0.05
Kyocera 7135
-0.05
0.00
0.05
0.10
Motorola moto Kyocera 7135 MWG RIM 8100 Dash Palm 600
-0.05
F1(60.42%)
0.05
0.10
F1(58.87%) (b) Perceptual Map from 2005 to 2006
(a) Perceptual Map from 2000 to 2004
0.20
iPhone 3G
0.10
0.00
Palm 650
iPhone 4
RIM 8100
0.05
F2(9.94%)
F2(22.79%)
0.15
iPhone 3GS
0.00
-0.05
Palm Centro Palm 650 MWG Dash RIM 8310 Motorola moto Motorola Droid RIM Sharp Sidekick II 9530
-0.04
-0.02
0.00
0.02
0.04
0.10 0.05 0.00 -0.05
0.06
F1(66.06%) (c) Perceptual Map from 2007 to 2009
RIM iPhone 4S iPhone 3GS 8310 Motorola Samsung Galaxy S II iPhone 3G RIM Droid Palm Centro 9530
-0.02
0.00
0.02
0.04
0.06
F1(83.46%) (d) Perceptual Map from 2010 to 2012
Fig. 11. MDS Map of smart phone market based on online positive reviews.
70
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
Sharp Sidekick II was prioritized as the most valuable product in the market in the first two periods, while iPhones were regarded as the most important smart phones during the last two periods. In terms of competitiveness, iPhone products decisively outperformed others more so than Sharp Sidekick II. Also, we found that market challengers could be identified by the ranking results. For example, the RIM brand has been present in the market in all four periods, and RIM 8100 placed second in the third period. It suggests that RIM has undergone intense competition from iPhones and could have become the market leader if iPhones had not been introduced to the market. Finally, from the last period we found that iPhone 4S and Samsung Galaxy S II are two challengers that have the potential to overtake iPhone 4 as the market leader in the next period. Similarly, for the negative reviews of smart phones, we derive the weight matrix of smart phones with WVAP method and summarize it in Table 15. Contrary to the weight matrix obtained from the positive reviews, higher values refer to worse performance since each topic extracted from the negative reviews is undesirable
Table 14 Validity comparisons of MSD maps of smart phones based on positive reviews. The sum of eigenvalues of F1 and F2 2005– 2006
2007– 2009
2010– 2012
93.74 92.79
84.02 82.85
88.85 90.5
93.4 91.36
0.8
0.6 Kyocera 7135
Sharp Sidekick II
Sharp Sidekick II
Kyocera 6035
Ratings
Ratings
0.8
0.4
0.6 Palm 650
0.4
0.2
0.2
1
2
Palm 650
Rim 8100
4
5
3
Kyocera 7135
6
MWG Dash
Motorola Moto
2
(a) Smart Phones from 2000 to 2004
Rim 8100
Palm 600
Palm 600
4
6
(b) Smart Phones from 2005 to 2006
iPhone 4
0.8
0.8
iPhone 3G Rim 8100
0.6
Ratings
Ratings
Proposed method (%) BWA method (%)
2000– 2004
in nature. The negative weight matrix signifies the deficiencies of smart phones. For example, in 2010–2012, iPhone 4 receives high negative scores in all key topics, while Samsung Galaxy S II is criticized profoundly in topic 20, battery life. The perceptual maps of smart phones from negative reviews are displayed in Fig. 13. We find that in terms of market evolution, the MDS maps derived from negative reviews exhibit the same characteristics as those derived from the positive ones. For instance, smart phones in Fig. 13(c) face harsher competition than those in the earlier periods, because they were much more similar to each other in 2007–2009 than before. In terms of product differentiation and positioning, the findings from the negative perspective are consistent with those from the positive one. For example, both the MDS maps of the positive and negative reviews suggest that RIM 8100 and Samsung Galaxy S II prefer to avoid direct competition with iPhones. As another example, Sharp Sidekick II in the second period (identified as a unique product in positive reviews), appears much more similar to other smart phones from the negative review perspective. This is because that Sharp Sidekick II became a mature product with unique features and acceptable defects in the second period. Similarly, the validity of the proposed method in establishing MDS maps is checked by comparing the sum of eigenvalues of F1 and F2 with the BWA method. The results are presented in Table 16. It shows that the proposed method is consistently more accurate in obtaining MDS maps. Finally, the rankings of smart phones based on negative reviews are summarized in Fig. 14. Since the evaluation is based on negative reviews, high ratings imply significant improvements needed. For example, both RIM 8100 and iPhone 4 are the highest-rated smart phones in the third and the last periods. This indicates that
iPhone 3GS
0.4
0.2
Palm Motorola Centro Droid
MWG Dash
Palm 650
Rim 9530
4
6
8
0.4 iPhone 3GS
Rim 8310 Motorola Moto
Sharp Sidekick II
2
0.6
10
(c) Smart Phones from 2007 to 2009
0.2
Palm Centro
Rim Rim 8310 9530
Motorola Droid
iPhone 3G
2
Samsung Galaxy S II
iPhone 4S
4
6
8
(d) Smart Phones from 2010 to 2012
Fig. 12. Ratings of smart phones based on online positive reviews.
71
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74 Table 15 The weight matrix of smart phones based on the negative reviews. Periods
Smart phones
Topic 56
Topic 16
Topic 35
Topic 17
Topic 3
2000–2004
Kyocera 7135 Kyocera 6035 Palm 600 Palm 650 RIM 8100 Sharp Sidekick II
0.603 0 0 0 0 1.63
0 1.287 0 0.394 0 0
0 2.313 0 0 0 0
0.948 0.917 0 0 0 0
0 1.458 0 0 0 0.88
Topic 94
Topic 75
Topic 109
Topic 27
Topic 19
2005–2006
Kyocera 7135 Motorola Moto MWG Dash Palm 600 Palm 650 RIM 8100 Sharp Sidekick II
0 0 0 1.537 0.234 0 0
0 0.386 0 0 1.93 0 0
0 0 0 0.347 0.848 0.613 0.519
0 0 0 0.266 0.266 0.361 0.834
0 0 0 1.167 0.77 0 0.347
Topic 24
Topic 14
Topic 49
Topic 68
Topic 75
1.025 1.722 0 0.297 0.347 0.259 0 0.878 0.229 0 0
0.268 0.588 0 0 0 0.53 0 1.042 0.619 0 0
0 0.703 0.78 0 0 0.518 0 0.347 0.832 0 0
0 0 0.425 0.173 0 0 0.455 0.78 1.137 0.392 0
1.189 0 0 0 0 0 0.151 0.308 0.511 0.568 0.419
Topic 20
Topic 56
Topic 1
Topic 28
Topic 67
0 0.323 2.064 1.508 1.184 0 0.219 0 2.63
0.314 0.314 0.519 0.824 0.894 0 0 0 0
0 0.714 1.323 0.474 0 0 0 0 0
0 0 1.672 0 0 0 0 0.314 0
0 0.483 0.784 0.811 0 0.392 0 0 0
2007–2009
iPhone 3G iPhone 3GS Motorola Droid Motorola moto MWG Dash Palm Centro Palm 650 RIM 8100 RIM 9530 RIM 8310 Sharp Sidekick II
2010–2012
iPhone 3G iPhone 3GS iPhone 4 iPhone 4S Motorola Droid Palm Centro RIM 9530 RIM 8310 Samsung Galaxy S II
these two products received the most criticism, and they have flaws with which most users were unsatisfied. On the other hand, although low ratings from the negative perspective suggest that consumers are not too unhappy with the products, they have delicate and complex reading. Low negative ratings may be due to no potential for product improvements or users giving up on rectifying the defects. It turns out that the latter interpretation is sensible for the smart phone market. For instance, the rating of Sharp Sidekick II from negative perspective decreased continuously from the first period to the third period, and surprisingly it ended up being eradicated in the last period. The only reasonable explanation of the seemingly negative correlation between the rating of Sharp Sidekick II and its fate is that its low rating from negative perspective in the third period did not mean it’s flawless, but simply indicates it is outdated and less relevant. Combing the market structures elicited from both Pros and Cons of smart phones, we find that no smart phones in the collected sample could gain both high ratings on positive reviews and low ratings on negative reviews. Thus, compared with conventional phone market, the smart phone market has more uncertainties. For example, from the second period to the third period, the positive rating of Sharp Sidekick II dropped from first place to last place, while its negative rating also decreased from third place to fifth place. A similar pattern can be found in the rating changes of iPhone 3G and iPhone 3GS from the third period to the last period. This trend suggests that smart phones starting with popular features and only minor defects tend to be defeated by competitors in the future; the succeeding high negative ratings reflect lack of
innovation, while competitors developed solutions to fix previous flaws. Based on this observation, we predict that iPhone 4, as the market leader in 2010–2012, is going to be replaced by iPhone 4S or Samsung Galaxy S II after year 2012. Further, smart phones with both low positive and negative ratings are likely to be eliminated from the market when the competition intensifies. For instance, in the first period, Palm 600, Palm 650, and RIM 8100 ranked as the three least negatively reviewed phones, and at the same time the three least positively reviewed phones. They all survived in the second period due to low competition in period 1 (2000–2004). However, the survival rate of the same category decreased dramatically in a more competitive environment. Out of seven smart phones with both low positive and negative rating in the third period, only Motorola Droid and Palm Centro remained in the market in the last period. We thus anticipate that five products in the last period (i.e., iPhone 3G, iPhone 3GS, Palm Centro, RIM 9530, and RIM 8310) will be excluded from the market after 2012.
5. Discussions and managerial implications From the methodology perspective, our research contributes to the development of the WVAP method to lessen manual interventions in the process of uncovering market structure from online product reviews. The main advantage of the proposed WVAP model is combining the topic modeling, multi-dimensional scaling, and TOPSIS rating methods to extract customers’ preferences from
72
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
0.05
Kyocera 7135
Sharp Sidekick II
RIM 8100 Palm 600 Palm 650
Sharp Sidekick II
0.00
Motorola moto
0.00
F2(37.07%)
F2(23.27%)
MWG Dash Kyocera 7135 RIM 8100
0.05
-0.05 -0.10 -0.15
-0.05
Palm 600
-0.10
-0.20 -0.15
Kyocera 6035
-0.15
0.00
0.05
-0.15
-0.10
-0.05
0.00
0.05
F1(50.04%)
(a) Perceptual Map from 2000 to 2004
(b) Perceptual Map from 2005 to 2006
Motorola Droid
Palm 650 Sharp Sidekick II RIM 8310 Motorola moto MWG Dash
iPhone 3G Palm RIM 9530 Centro
RIM 8310
0.05
iPhone 3GS
Palm Centro
0.00
F2(27.41%)
-0.05
F1(68.86%)
RIM 9530
-0.02 iPhone 3G
-0.04
F2(21.97%)
0.02
-0.10
Palm 650
0.00
Motorola Droid iPhone 4S
-0.05
RIM 8100 Samsung Galaxy S II
-0.06 iPhone 3GS
-0.10
iPhone 4
-0.08 -0.06
-0.04
-0.02
0.00
0.02
0.04
F1(45.36%) (c) Perceptual Map from 2007 to 2009
-0.05
0.00
0.05
F1(65.54%) (d) Perceptual Map from 2010 to 2012
Fig. 13. MDS map of smart phone market based on online negative reviews.
Table 16 Validity comparisons of MSD maps of smart phones based on negative reviews. The sum of eigenvalues of F1 and F2
Proposed method (%) BWA method (%)
2000– 2004
2005– 2006
2007– 2009
2010– 2012
92.13 91.86
87.11 85.33
72.77 72.65
87.51 86.98
online product reviews and present rankings of products in a completely automatic manner. From the application perspective, our research advances the values of online product reviews and enhances the knowledge of market landscape. Specifically, the managerial implications of the proposed hybrid text mining method are threefold: (1) The extracted dynamic topics can provide marketers with insights about changes in consumers’ preferences over time. A serious challenge marketing managers usually face is monitoring the dynamic changes of consumer brand preferences (Sriram et al. 2006). In our smart phone case study, with the dynamic-topic extraction method, marketers can understand which product features consumers value most, and how their emphases vary over time. Compared with traditional survey-based studies (Geir and Jens 2009; Leclerc et al. 2005), our approach has the advantages of higher resolution and lower cost. (2) The MDS map of each period derived from online reviews helps marketers understand competitors without conducting additional market surveys. The strength of the MDS
map lies in its ability to elucidate the landscape of product position. Unlike the traditional market study using a survey-based approach, the proposed hybrid text mining method can obtain MDS maps automatically from online product reviews. Armed with MDS maps based on dynamic topics, marketing managers can recognize their product position and recognize competitors who could pose direct threats to them. (3) The weight matrix and final rankings method help management monitor effectiveness of their ongoing marketing strategies. These results are beneficial for marketers to discover effective ways of improving their products because product performance is assessed from the perspective of key topics.
6. Limitations Although the proposed hybrid method is able to exploit online product reviews by providing additional insights into the market, there is a limitation in terms of its application. Note that in our case study, topical modeling is applied to process documents of positive and negative reviews, respectively. In order to integrate the results of topic modeling with TOPSIS by the newly developed WVAP method, the product reviews need to be grouped into separate positive and negative reviews in advance. Otherwise, product ranking is not meaningful due to the confusion caused by mixing positive topics with negative topics. However, some online review websites only present summary comments of products where positive comments and negative
73
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
1.0
1.0
0.8
Kyocera 6035
0.6 Sharp Sidekick II
0.4
Ratings
Ratings
0.8
Sharp Rim Sidekick II 8100
0.4
Kyocera 7135 Palm 650
0.2 Palm 600
1
Palm 650 Palm 600
0.6
2
3
Motorola Moto MWG Dash
0.2 Rim 8100
4
5
Kyocera 7135
6
2
(a) Smart Phones from 2000 to 2004
4
6
(b) Smart Phones from 2005 to 2006
1.0
1.0
0.8
0.8
0.6
0.4
0.2
Rim Rim 8100 9530
iPhone 3GS
iPhone 3G
Motorola Droid Motorola Moto
2
Palm Rim Centro Palm 8300 650
MWG Dash
4
6
8
Ratings
Ratings
iPhone 4
0.6
0.4 Sharp Sidekick II
10
(c) Smart Phones from 2007 to 2009
0.2
Samsung Galaxy S II
iPhone 4S Motorola Droid iPhone 3GS
Palm Rim Rim Centro 9530 8310
iPhone 3G
2
4
6
8
(d) Smart Phones from 2010 to 2012
Fig. 14. Ratings of smart phones based on online negative reviews.
comments are mingled together. Therefore, in order to extend the proposed method into such context where the polarity of text data is not predefined, new methods need to be developed to split the overall product review into pros and cons. 7. Conclusion The amount of online user-generated content is growing exponentially and presents both a challenge and an opportunity for those in both academia and industry. Marketers can make use of such a huge supply of free text data to discern trends in consumers’ preferences and in patterns of the competitive landscape. The proposed model automates the derivation of market structure and provides maps of product positioning like those available in the current literature. In addition, it offers product rankings based on their performances. The case study of smart phones illustrates that the proposed framework is a powerful tool for exploring online reviews and evaluating ongoing marketing activities. Acknowledgements This research was partially supported by grants from the National Natural Science Foundation of China (#70901014 for Kun Chen, #71222108 and #71471149 for Gang Kou), and Postdoctoral Science Foundation of China (#2011M501299 for Kun Chen). References Blei, D.M., Ng, A.Y., Jordan, M.I., 2003. Latent dirichlet allocation. Journal of Machine Learning Research 3 (1), 993–1022. Campbell, C., Pitt, L.F., Parent, M., Berthon, P.R., 2011. Understanding consumer conversations around ads in a web 2.0 world. Journal of Advertising 40 (1), 87– 102. Chen, S.J., Hwang, C.L., 1992. Fuzzy Multiple Attribute Decision Making: Methods and Applications. Springer-Verlag, Berlin.
Chevalier, J.A., Mayzlin, D., 2006. The effect of word of mouth on sales: online book reviews. Journal of Marketing Research 43 (3), 345–354. Chintagunta, P.K., Gopinath, S., Venkataraman, S., 2010. The effects of online user reviews on movie box office performance: accounting for sequential rollout and aggregation across local markets. Marketing Science 29 (5), 944–957. ConsumerReports, 2012. Cell phone & service buying guide. Retrieved from: . Cooper, L.G., Inoue, A., 1996. Building market structures from consumer preferences. Journal of Marketing Research 33 (3), 296–306. DeSarbo, W.S., Howard, D.J., Jedidi, K., 1991. Multiclus: a new method for simultaneously performing multidimensional scaling and cluster analysis. Psychometrika 56 (1), 121–136. Duan, W., Gu, B., Whinston, A.B., 2008. The dynamics of online word-of-mouth and product sales—an empirical investigation of the movie industry. Journal of Retailing 84 (2), 233–242. Elrod, T., 1991. Internal analysis of market structure: recent developments and future prospects. Marketing Letters 2 (3), 253–266. Elrod, T., Russel, G.J., Shocker, A.D., Andrews, R.L., Bacon, L., Bayus, B.L., et al., 2002. Inferring market structure from customer response to competing and complementary products. Marketing Letters 13 (3), 221–232. Erdem, T., Keane, M.P., 1996. Decision-making under uncertainty: capturing dynamic choice processes in turbulent consumer good markets. Marketing Science 15 (1), 1–20. Feldman, R., Fresko, M., Goldenberg, J., Netzer, O., Ungar, L.H., 2007. Extracting product comparisons from discussion boards. In: Proceedings of the 2007 Seventh IEEE International Conference on Data Mining. Association for Computing Machinery, New York, pp. 469–474. Geir, S.-G., Jens, ø., 2009. Consumer evaluation of unbranded and unlabelled food products: the case of bacalhau. European Journal of Marketing 43 (1/2), 213– 228. Geman, S., Geman, D., 1984. Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transaction and Pattern Analysis and Machine Intelligence 6 (6), 721–741. Godes, D., Mayzlin, D., 2004. Using online conversations to study word-of-mouth communication. Marketing Science 23 (4), 545–560. Godes, D., Mayzlin, D., Chen, Y., Das, S., Dellarocas, C., Pfeiffer, B., Libai, B., Sen, S., Shi, M., Verlegh, P., 2005. The firm’s management of social interactions. Marketing Letter 16 (3), 415–428. Gopal, R.D., Pathak, B., Tripathi, K.A., Yin, F., 2006. From fatwallet to eBay: an investigation of online deal-forums and sales promotions. Journal of Retailing 82 (2), 155–164. Green, P.E., Srinivasan, V., 1978. Conjoint analysis in consumer research: issues and outlook. Journal of Consumer Research 5 (2), 103–123.
74
K. Chen et al. / Electronic Commerce Research and Applications 14 (2015) 58–74
Hwang, C.L., Yoon, K., 1981. Multiple Attribute Decision Making Methods and Applications. Springer, New York. Kou, G., Lu, Y., Peng, Y., Shi, Y., 2012. Evaluation of classification algorithms using MCDM and rank correlation. International Journal of Information Technology & Decision Making 11 (1), 197–225. One Fourth of Americans Say a Smart Phone is on Their Wish List; New InsightExpress Conjoint Analysis Survey Reveals Important Attributes for Smart Phones, 2004. Business Wire, 12, 1. John, D.R., Loken, B., Kim, K., Monga, A.B., 2006. Brand concept maps: a methodology for identifying brand association networks. Journal of Marketing Research 43 (4), 549–563. Laroche, M., Yang, Z., McDougall, H.G.G., Bergeron, J., 2005. Internet versus bricksand-mortar retailer: an investigation into intangibility and its consequences. Journal of Retailing 81 (4), 251–267. Leclerc, F., Hsee, C.K., Nunes, J.C., 2005. Narrow focusing: why the relative position of a good in its category matters more than it should. Marketing Science 24 (2), 194–205. Lee, T.Y., Bradlow, E.T., 2011. Automated marketing research using online customer reviews. Journal of Marketing Research 48 (5), 881–894. Mardia, K.V., Kent, J.T., Bibby, J.M., 1979. Multivariate Analysis. Academic Press, London. Netzer, O., Feldman, R., Goldenberg, J., Fresko, M., 2012. Mine your own business: market-structure surveillance through text mining. Marketing Science 31 (3), 521–543.
Onishi, H., Manchanda, P., 2012. Marketing activity, blogging and sales. International Journal of Research in Marketing 29 (3), 221–234. Opricovic, S., Tzeng, G.H., 2003. Compromise solution by MCDM methods: a comparative analysis of VIKOR and TOPSIS. European Journal of Operational Research 156 (2), 445–455. Pan, Y., Zhang, J.Q., 2011. Born unequal: a study of the helpfulness of user-generated product reviews. Journal of Retailing 87 (4), 598–612. Shao, K.-H., 2012. The effects of controversial reviews on product sales performance: the mediating role of the volume of word of mouth. International Journal of Marketing Studies 4 (4), 32–38. Sriram, S., Chintagunta, P.K., Neelamegham, R., 2006. Effects of brand preference, product attributes, and marketing mix variables in technology product markets. Marketing Science 25 (5), 440–456. TouchType & Smartphone Experts, 2011. What do smartphone users really want? Retrieved from: . Urban, G.L., Hauser, J.R., 2004. ‘‘Listening in’’ to find and explore new combinations of customer needs. Journal of Marketing 68 (2), 72–87. Urban, G.L., Johnson, P.P., Hauser, J.R., 1984. Testing competitive market structures. Marketing Science 3 (2), 83–112. Weiss, S., Indurkhya, N., Zhang, T., Damerau, F., 2004. Text Mining: Predictive Methods for Analyzing Unstructured Information. Springer-Verlag.
