Sentiment Classification of on-line Products based on Machine Learning Techniques and Multi-agent Systems Technologies Mohammed Almashraee1,2? , Dagmar Monett Diaz2 , and Rainer Unland1 1
2
University of Duisburg-Essen, Institute for Computer Science and Business Information Systems (ICB), Schtzenbahn 70, D-45117 Essen, Germany Department of Computer Science, Berlin, School of Economics and Law, Alt Friedrichsfelde 60, 10315, Berlin, Germany {almashraee,rainer.unland}@icb.uni-due.de
[email protected]
Abstract. The rapid spread of different social media applications provides a new way for people to interact and to share information on-line all over the world. This massive and enormous volume of on-line information needs to be structured and organized in a useful way for users to get oriented opinion summary from text related to their search. In this paper we consider the problem of sentiment classification of on-line product features to obtain their overall semantic orientation. We present a novel method for extracting product features and to obtain their associated sematic orientations. Our proposed method is based on multi-agent technology with emerging machine learning classification techniques that together promise a better level of efficiency. Keywords: multi-agent systems; machine learning; Nave Bayes; sentiment classification; product features
1
Introduction
Initially, collecting opinions for a particular product was done either by asking people around or by carefully constructing a questionnaire and letting people express their opinions as positive or negative. Such kind of collected information does not convey people’s real opinions due to the limitation of information sources. Recently, the spread of the Internet scope contributed to the existence of many social media such as Twitter, Facebook, YouTube, the blogosphere, hi5, Flicker, LinkedIn, etc., which in turn play an important role in serving users in many different aspects of their daily life including economic, political, security, scientific, government, social, and many other aspects. A massive number of online products are become available in different social media networks. This number is getting higher over and over related to time and it results in a vast amount of data on the internet which is increasing dramatically and significantly. ?
e-mail:
[email protected]
Collecting information about product features in different user-generated information sources is an important request for many parties especially for providers and customers. It is for providers to keep track and to have a clear feedback concerning their products (what others say about their products), and for customers, on the other side, to collect a comprehensive overview over products that they attempt to buy or investigate. Users face two problems when they attempt to get information concerning a particular product. First, they cannot read the huge amount of available information in its unstructured or semi-structured formats. Second, it is difficult to trace all the available user-generated content for a related product. In this work, we describe a novel system that combines mechanisms from different computer science disciplines such as Machine Learning (ML), Information Retrieval (IR), text mining (TM), Natural Language Processing (NLP), and Multi-agent Systems (MAS) technologies. The main goal is to collect information about on-line products from different social media networks and to provide the user with an overall reliable summary for the collected product features from differ-ent social media sources. This paper is organized as follow. In Section 2, we discuss the state of the art and depict the different mechanisms used for sentiment analysis. In Section 3, we introduce the research problem description. The project structure and the proposed solution are in Section 4. Finally, the conclusion and future work are presented in Section 5.
2
State of the art
Sentiment analysis is first introduced in [1] in which it is used to analyze in an automatic fashion social media sources, blogs, and so on. Sentiment analysis expresses opinions, evaluations, relations, attributes, sentiments, attitudes, and emotions mentioned by bloggers, concerning a specific topic of interest. In [2], Hu proposes a set of techniques for mining and summarizing product reviews based on data mining and Natural Language Processing methods. Dave [3] focuses on extracting useful features from opinions (feature-based opinion summarization) in different online source, and in evaluating sentiment orientation for each opinion. Different methods and techniques to obtain opinion orientation from text are discussed in a survey conducted by Khan [4]. Machine Learning techniques including both supervised and unsupervised learning approaches are also used for sentiment classification. They attempt to estimate the sentiment orientation of documents and provide, for example, a positive or a negative orientation for a given topic. Pang discusses in [1] the major issues related to social media analytics and the use of different methods concerning ML techniques. He studies the effectiveness of some algorithms from ML techniques in sentiment classifications such as Bayesian models [5] especially the Naive Bayes (NB) method [6], the maximum entropy classification, and the support vector machines (SVM) which is first proposed by Vapnik [7]. Liu proposes in [8] a framework to analyze and to compare consumer opinions of competing products; the authors also implement a system prototype called Opinion Observer, which uses supervised rule-discovery techniques to extract a product features and their corresponding
pros and cons. Another survey is conducted in which the authors discuss and compare the effectiveness of several ML techniques for opinion mining. Their investigation and results indicate that the most commonly used ML mechanisms are SVM and NB and that they outperform several other techniques. A unsupervised information extraction system called OPINE is introduced by Popescu [9], it uses the relaxation-labeling technique to determine the semantic orientation of opinion words according to the extracted features and to the specific review sentences. OPINE system takes a particular product and its related set of reviews as an input, it solves the opinion mining tasks (identify product features, identify opinions regarding product features, determine the polarity of opinions, rank opinions based on their strength), and it produces a set of product features, each with its associated list of opinions that are ranked based on strength. The multi-agent system technology also contributes to the development of opinion mining and sentiment analysis [10–12]. Fujimoto uses a MAS approach to provide the users with a fast and high search efficiency. He only guarantees the relatedness between gathered pages and user’s request, and do not go for further analysis. Garcia and Schweitzer went for further, they define an agent-based model of the users of product review communities using a modeling framework. Feedback rating about the quality of the reviews is provided by the users. The helpfulness and unhelpfulness of reviews is calculated according to the amount of positive and negative votes. Large number of research efforts has been done to provide solutions to the issue of extracting and analyzing opinion mining for on-line products. However, most existing product features detection proposals are either biased in collecting their information from one or from a limited number of on-line social sources or they do not consider some important factors like effective feature coverage, date, geographical area, among others, which are very important in giving more accurate results for the social networks users.
3
Problem statement
The document and the sentence level of classification can only define the general polarity or the orientation of any review as positive or negative. Thus, we will not be able to reveal the final and clear decision of any product over its mined attributes. Researchers mostly poured their attention in collecting product features from text based on one or on a limited number of social media sources. They pay little attention to resolve the features coverage effectively. This is considered a very important issue for the social media networks users to get a comprehensive summary to support decision making. Recently, some research efforts offered proposals in this direction like Aleebrahim in [13], in which the author provides a solution to the sentiment classification for product features. He uses the most frequent names and name phrases and put them as features instead of using n-grams. We share some Aleebrahim ideas but in our case we are going to use the MAS technology with NB classifier based on not only the most frequent nouns and noun phrases, but all of them instead.
4
Project structure and proposed solution
In our work, we introduce the idea of using both multi-agent system technologies and machine learning techniques to provide a solution to the problems of sentiment classification of on-line product features. Our solution is characterized by its ability to performing an automatic classification, and the ability to deal with partial information. Our proposed idea takes advantage of the expendability (easy to expand) and the interaction from the multi-agent techniques, and the classification performance from the machine learning methods. Sentiment analysis is the main aim of our research. It focuses on automatically extracting people’s opinions from the Web and on providing users with a structured text summary related not only to a specific product. For any given product, to find what have been told about it. This can be realized by collecting opinions from different social media networks about that product. Our main focus is to cover all the available set of features for a given product on which social media users wrote their opinions and their opinion orientation or sentiments. Using these technologies, we will be able to extract data, to analyze sentiments, and to provide a learning mechanism for future prediction, as well as to facilitate cooperation and interaction among different available social media nets, as illustrated in Figure 1. We deal with different types of agents. This is because agents consult different social media networks to collect the data. Each social media differs from others in representing the data, the types of libraries that are used to consult it, the type of the classifier used, and so on. Each agent for example, Twitter-opinion-mining-agent (TOMA), deals with certain social media source (in this case Twitter); it collects all the relevant documents, extracts the available features for the given product, and it provides their semantic orientation as positive or negative. Each agent will be able to add information from other agents to its repository, this facilitates the learning capabilities. On the other hand, the system facilitates an easy way to be extended, in order to include any number of new social media. This can be done by simply adding a new agent to the system, and by defining the way and form it should communicate with the others.
Opinion Miner Agent (OMA) is responsible for gathering all the features that are extracted from the different social media networks, and then it summarizes them in its list. OMA maintains a blackboard that makes available that list. Other agents learn and update their feature lists with new features that are not included in their own lists. Twitter-opinion-mining-agent (TOMA), Wikiopinion-mining-agent (WOMA), and Facebook-opinion-mining-agent (FOMA) are responsible for collecting features each from the social media that is associated to its name. Each agent collects information according to different criteria such as searching opinions in different time periods, in different geographical areas, and so on. Each criterion is tackled by a set of related agents. The following example shows how our project works. Suppose that we need to investigate a
Fig. 1. Product sentiment classification based on multi-agent systems.
Doc1 Doc2 Doc3 product name 1 0 1 picture quality 1 -1 1 weight 0 0 0 battery life 1 1 -1 price 1 0 1 zoom -1 1 -1 Table 1. Investigation results of TOMA
camera. Using the twitter agent TOMA, we could have the following features: product name, picture quality, weight, battery life, price, and zoom as shown in Table 1. Each agent is represented by a vector of different semantic orientation such as positive (1), negative (-1), and neutral (0). The columns represent the documents and the rows represent the features. Another agent such as Facebook agent FOMA on the other hand could have other results that are shown in Table 2. The weight feature does not exist within the feature list summary of TOMA agent, but it exists within the FOMA agent feature summary. In this case, the TOMA agent should learn from FOMA agent by adding the missed features through the blackboard. The same case occurs with FOMA agent, it does not have the zoom feature, while exists within FOMA list. Table 3, shows the final result after both agents learned from each other via the Blackboard. In more details, each agent performs the following tasks
Doc1 Doc2 Doc3 product name 1 0 1 picture quality 1 -1 1 weight 1 1 -1 battery life 1 1 -1 price 1 0 1 zoom 0 0 0 Table 2. Investigation results of TOMA
Doc1 Doc2 Doc3 Doc4 Doc5 Doc6 product name 1 0 1 1 0 1 picture quality 1 -1 1 1 -1 1 weight 1 1 -1 1 1 -1 battery life 1 1 -1 1 1 -1 price 1 0 1 1 0 1 zoom -1 1 -1 -1 1 -1 Table 3. Investigation results of TOMA
4.1
Product information retrieval
In this stage, we build our corpus by aggregating the most relevant documents to the query. This is done using term frequency - document inverse frequency (tf-idf) weighting scheme as in Equation (1) Wt,d = (1 + log tft,d ) × log10
N dft
(1)
where W(t,d) represents the weight of in common feature between the document and the query All document vectors are further normalized since not all the collected documents have an equal length Equation (2) | x |=
sX
x2i
(2)
i
The most relevant documents are are selected according to the similarity measure between the collected documents and the given query. Cosine measure is used as document scoring measure see Equation (3)
cos(q, d) = q.d =
|v| X i=1
qi di
(3)
4.2
Product features extraction
Before extracting the product features we apply some preprocessing steps such as, tokenization. In which we divide documents into separate words (tokens). Each token is presented to a Part Of Speech (POS) to obtain its type. Nouns and noun phrases are extracted as features that are considered the classifier input. From each document, we extract the opinion expressed about that particular feature and then assign its polarity as positive or negative based on either a lexicon that maintains almost all the positive and negative words, or by using WordNet.
4.3
Classification
The Nave Bayes (NB) is the most commonly used classifiers for classifying text. NB is based on the Bayes theory in which we have a set of documents D = d1, d2, . , dn, with n the number of documents, and a set of predefined classes C = c1, c2, .., cm, with m the number of the categories or the classes, such that P (ci /dj ) =
(dj /ci ) × P (ci ) P (dj )
(4)
where P (ci /dj ) is the probability that a document exist given a category.
5
Conclusions and future work
In this paper we are using sentiment analysis mechanism to main product features from text in different social media sources. Machine learning techniques such as Nave Bayes is proposed to classify user’s opinions. Different agents are used to deal with different kind of information from different social media networks. Agents communicate and interact with each other to learn new information. Different features for classification have to be considered and different classifiers are also need to be tried such as support vector machine (SVM). In addition to that, we are considering several kinds of available social media sources and intend to generalize to deal with different types of information (such as, political information, sport information, medical information, and so on). The project is still at its beginning. We will start with one source of social media which is Twitter. Lucene java libraries or R-projects libraries are going to be used for text processing. Nave Bayes is our used classifier. We are currently seeking suggestions and feedback that will help us to improve our work.
References 1. B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs up?: sentiment classification using machine learning techniques,” in Proceedings of the ACL-02 conference on Empirical methods in natural language processing - Volume 10, EMNLP ’02, (Stroudsburg, PA, USA), pp. 79–86, Association for Computational Linguistics, 2002. 2. M. Hu and B. Liu, “Mining and summarizing customer reviews,” in Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD ’04, (New York, NY, USA), pp. 168–177, ACM, 2004. 3. K. Dave, S. Lawrence, and D. M. Pennock, “Mining the peanut gallery: opinion extraction and semantic classification of product reviews,” in Proceedings of the 12th international conference on World Wide Web, WWW ’03, (New York, NY, USA), pp. 519–528, ACM, 2003. 4. K. Khan, B. Baharudin, A. Khan, and F. e Malik, “Mining opinion from text documents: A survey,” in Digital Ecosystems and Technologies, 2009. DEST ’09. 3rd IEEE International Conference on, pp. 217 –222, june 2009. 5. N. Samsudin, M. Puteh, and A. Hamdan, “Bess or xbest: Mining the malaysian online reviews,” in Data Mining and Optimization (DMO), 2011 3rd Conference on, pp. 38 –43, june 2011. 6. S. Abney, “Part-of-speech tagging and partial parsing,” in Corpus-Based Methods in Language and Speech, pp. 118–136, Kluwer Academic Publishers, 1996. 7. S. Abney, “Statistical methods and linguistics,” in THE BALANCING ACT: COMBINING SYMBOLIC AND STATISTICAL APPROACHES TO LANGUAGE, pp. 1–26, MIT Press, 1996. 8. B. Liu, M. Hu, and J. Cheng, “Opinion observer: analyzing and comparing opinions on the web,” in Proceedings of the 14th international conference on World Wide Web, WWW ’05, (New York, NY, USA), pp. 342–351, ACM, 2005. 9. A.-M. Popescu and O. Etzioni, “Extracting product features and opinions from reviews,” in Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT ’05, (Stroudsburg, PA, USA), pp. 339–346, Association for Computational Linguistics, 2005. 10. D. Garcia and F. Schweitzer, “Emotions in product reviews–empirics and models,” in Privacy, security, risk and trust (passat), 2011 ieee third international conference on and 2011 ieee third international conference on social computing (socialcom), pp. 483 –488, oct. 2011. 11. F. Schweitzer and D. Garcia, “An agent-based model of collective emotions in online communities,” The European Physical Journal B - Condensed Matter and Complex Systems, vol. 77, pp. 533–545, 2010. 12. K. Fujimoto, “A computational account of potency differences in ewom messages involving subjective rank expressions,” in Web Intelligence and Intelligent Agent Technology (WI-IAT), 2011 IEEE/WIC/ACM International Conference on, vol. 3, pp. 138 –142, aug. 2011. 13. N. Aleebrahim, M. Fathian, and M. Gholamian, “Sentiment classification of online product reviews using product features,” in Data Mining and Intelligent Information Technology Applications (ICMiA), 2011 3rd International Conference on, pp. 242 –245, oct. 2011.
