Finding Opinion Strength Using Rule-Based Parsing for Arabic ...

Finding Opinion Strength Using Rule-Based Parsing for Arabic Sentiment Analysis Shereen Oraby1 , Yasser El-Sonbaty2, and Mohamad Abou El-Nasr1 1

Departments of Computer Engineering 2 Computer Science Arab Academy for Science and Technology Alexandria, Egypt {shereen.oraby,yasser,mnasr}@aast.edu Abstract. With increasing interest in sentiment analysis research and opinionated web content always on the rise, focus on analysis of text in various domains and different languages is a relevant and important task. This paper explores the problems of sentiment analysis and opinion strength measurement using a rule-based approach tailored to the Arabic language. The approach takes into account language-specific traits that are valuable to syntactically segment a text, and allow for closer analysis of opinion-bearing language queues. By using an adapted sentiment lexicon along with sets of opinion indicators, a rule-based methodology for opinion-phrase extraction is introduced, followed by a method to rate the parsed opinions and offer a measure of opinion strength for the text under analysis. The proposed method, even with a small set of rules, shows potential for a simple and scalable opinion-rating system, which is of particular interest for morphologically-rich languages such as Arabic.

1

Introduction

The tasks of opinion mining and sentiment analysis have become essential points of research in recent years, particularly due to the wide array of applications and industrial potential involved in gathering informative statistics on the polarity of opinions expressed in various settings, ranging from social, to political, to commercial. The need for efficient, reliable, and scalable automatic classification systems is on the rise, in order to take advantage of the abundance of available opinionated web data for classification and analysis. While much work is available regarding the sentiment analysis tasks under various approaches in English, research is more limited for morphologically-rich languages such as Arabic, particularly with respect to rule-based approaches that attempt to determine syntactic patterns in the language to aid in the analysis tasks. This paper focuses on the task of sentiment analysis in Arabic through a rule-based approached using sentiment lexicons and modifying words lists to determine document sentiment on a standard opinion movie-review corpus. The considered documents are compared based on their sentiment strength (as compared to ratings given in the text), as opposed to a simple, crisp classification, thus offering a more detailed measure of opinion strength. F. Castro, A. Gelbukh, M.G. Mendoza (Eds.): MICAI 2013, Part II, LNAI 8266, pp. 509–520, 2013. c Springer-Verlag Berlin Heidelberg 2013 

510

S. Oraby, Y. El-Sonbaty, and M.A. El-Nasr

A survey of related work is detailed in Section 2, followed by introduction to the syntax of the Arabic language and the intuition behind a rule-based approach for rating in Arabic in Section 3. The proposed rule-based classification algorithm is introduced in Section 4, with results and evaluation presented in Section 5 and conclusions and future research points discussed in Section 6.

2

Related Work

The standard two-class problem of sentiment classification into positive and negative sets was addressed early on by Pang et al. (2002) for English movie reviews from the Internet Movie Database (IMDB) corpus, developing a baseline for the task employing machine learning techniques [1]. Yu et Hatzivassiloglou (2003) improved on the feature selection problem for the machine learning task by adopting N-gram based features and a polarity lexicon on the Wall Street Journal (WSJ) corpus [2], while Bruce and Wiebe (1999) employ additional lexical, part-of-speech (POS) and structural features to the same corpus, demonstrating variations in feature selection and optimization [3]. While supervised learning techniques often employ the use of classifiers such as Naive Bayes or Support Vector Machines (SVMs) to perform classification on sets of test samples given a tagged training set and rich feature sets for various tasks [4], such methods are often very accurate on the domain in which they are trained, and the most successful features reported are often simple unigrams, with performance often faltering when tested on different domains [5]. Because sentiment-bearing words and phrases have proven to be the “dominating factor” [6] in sentiment classification systems in general, samples of approaches focusing in on these features in particular are frequent in the literature. Turney (2002) uses fixed opinion-bearing patterns based on part-of-speech information to perform classification [6] by measuring the semantic orientation between terms [7]. For document-level classification dictionaries of sentiment words and phrases categorized by polarity and strength are employed with negations and intensifiers by Taboada (2011) to compute document-level sentiment scores [6], as compared to sentence and word level analysis. Kar and Mandal (2011) analyze product reviews by mining product features, extracting opinion-bearing phrases in the reviews, and measuring the strength of the extracted opinionphrases to determine overall product ranking [8]. Although research and corpora are more limited for more morphologically-rich languages such as Arabic, some interesting work in lexicon and rule-based classification does exist. An example of Chinese sentence-level sentiment classification presented by Fu and Wang (2010) takes advantage of very language-specific traits in Chinese to tailor the opinion-phrase extraction and score assignment task closely to the distinctive structure of the language, where words are segmented into their respective component sentiment-bearing morphemes, in a “fine-to-coarse grained” methodology [9]. Another notable contribution to lexicon-based sentiment analysis is the work of Syed et al. (2010) in Urdu, where shallow-parsing methods and a developed sentiment-annotated lexicon are used for the task, as a baseline

Finding Opinion Strength Using Rule-Based Parsing

511

development in a language which differs greatly from English in its scripture, syntactic formation, and morphological features [10]. With respect to the task of Arabic sentiment analysis in particular, AbdulMageed et al. (2011) present a manually-annotated corpus of Modern Standard Arabic (MSA) from the newswire domain, together with a tailored wide-scale polarity lexicon, to show the effects of using language-specific features with an SVM classifier on morphologically-rich languages, achieving results of 57.84% F and up to 95.52% F with a domain-specific lexicon [11]. Likewise, Abbasi et al. (2008) use an “Entropy Weighted Genetic Algorithm (EWGA)” that incorporates both syntactic and stylistic features, as well as the information-gain heuristic, to classify document-level text from Middle-Eastern web forums to a reported 93.6% accuracy, also using machine learning classifiers [12]. Rushdi-Saleh et al. (2011) introduce the Opinion Corpus for Arabic (OCA), which consists of movie reviews compiled from various Arabic web pages, and use combinations of N-grams, stemming, and stop-word removal pre-processing under both Naive Bayes and SVM classifiers, achieving a best result of 90% accuracy under SVM [13]. In comparison to the discussed work, the proposed method utilizes the OCA movie review corpus [13] to explore syntactic patterns specific to Arabic to develop a set of rules that are used in conjunction with a polarity lexicon and sets of fixed grammatical units to extract sentiment phrases from opinionated documents, and calculate opinion scores and ratings for each of them. This methodology attempts to take advantage of opinion-bearing language patterns in Arabic for a simple approach that can scale appropriately to different contexts and domains depending on the lexicons utilized, without the need to re-learn data features as with supervised learning methods, and in turn provides a means of determining the respective strength of the classified text.

3

Background Information

The complexity of syntax and structure in morphologically-rich languages makes the task of language modeling and representation a challenging one for natural language processing systems. In particular, the effect of a language’s morphology on its overall syntax is significant: in Arabic, the language’s rich morphology allows for a greater degree of freedom for word order and syntactic structure, as the morphology itself defines many aspects of the language’s syntax, and is an important consideration for language modeling [14]. 3.1

Basic Descriptive Sentence Structure in Arabic

In Arabic, the most basic sentences used to describe a subject are nominal, consisting of a definite noun, proper noun, or pronoun subject, with an indefinite noun, proper noun, or adjective predicate agreeing with the subject (in number ab ˇgadyd, meaning and gender) [15]. For example, the sentence       (alkit¯ “The book is new.”) consists of a simple definite noun,     (alkit¯ab, meaning

512

S. Oraby, Y. El-Sonbaty, and M.A. El-Nasr

“book”), with a simple adjective,   (ˇgadyd, meaning “new”), and constructs a full sentence in this sense [14]. Arabic adjectives in particular have a structured position in basic sentences: adjectives follow the nouns they modify, and agree with them in gender and number [14]. A particular exception to this rule is found with adjectives that modify plural irrational nouns, which are always feminine singular [14]. Still, the placement of the adjective with respect to the noun it describes is an important detail of descriptive sentence structuring [15]. 3.2

Other Morphological Components of Syntax

Not all Arabic part-of-speech units are distinct, independent word forms; rather, several grammatical units are more related to the morphology of word forms they are attached to than separate entities [15]. Clitics, or morphemes that syntactically define words but are often bound to other base words, serve to change the meaning of the inflections they attach to. aw, signiFor example, conjunctions are cliticization morphemes, such as  (w¯  fying “and”) and  (f¯ a, indicating “in”), that can attach to any part-of-speech. Negations can attach to various parts-of-speech, or they can be independent words or morphemes, as well, such as  (lam, signifying “did not”), while adverbs



of degree, such as   (ktyr¯ a, signifying “a lot”) and   (qalyl¯ a, signifying “a ¯ little”), can modify verbs, adjectives, other adverbs, and more complex phrases, and usually come after the modified word. 3.3

Intuition Behind the Rule-Based Method for Sentiment Analysis

The basic word forms and simple sentence structures described can be used to derive a simple system for a set of rules focused around descriptive sentences in Arabic. Because some of the most basic and essential units governing the existence of descriptive opinion-detail in sentences revolve around adjectives and their modifiers, including negations, conjunctions, and adverbial intensifiers, these basic units offer a simple structure to map out sentiment-bearing phrases and sentences, as is proposed and modeled in this paper. The motivation behind a rule-based method for Arabic sentiment analysis lies in the ability to use the basic rules introduced to score documents (and more particularly, reviews) based on the opinion-bearing units within them. While standard machine-learning algorithms can serve to classify documents based on rich sets of word features, the simple proposed method offers a way to capture the level of positive or negative sentiment within the analyzed text, and therefore rank the text within the observed classes, rather than declaring a strict membership to one or the other.

Finding Opinion Strength Using Rule-Based Parsing

4

513

Proposed Algorithm

The proposed rule-based opinion-strength calculation algorithm is presented in the following sections, which detail the system design, dataset, tools, derived rule-set and score calculation scheme utilized. 4.1

System Design

The experiments conducted on the dataset are carried out in a number of steps, as shown in Figure 1 and described in the following sections.

Fig. 1. Experimental system block diagram

Initially, the corpus documents and derived set of modifier word lists (negations, intensifiers, and conjunctions) are parsed using a set of developed grammar rules, mapping the documents into patterns of words and modifiers that are to be used to match with polar word sets. These polar word sets compose the four sentiment lexicons (weak and strong word sets for both the positive and negative classes), which are used to match words in the already rule-parsed documents in the opinion-score calculation phase. Each document receives both a positive and negative score, according to a scoring scheme based on the number of polar units and modifiers in each sentence of the document. Each of the documents is then given a document rating, depending on a measure of the ratio of polar items within them, resulting in a strength rating for each of the documents, which can be compared to the actual review rating for validation, and used to find system accuracy and other assessment metrics.

514

4.2

S. Oraby, Y. El-Sonbaty, and M.A. El-Nasr

Opinion Corpus

The dataset chosen for system implementation was the Opinion Corpus for Arabic (OCA) movie-review corpus [13], particularly for its highly subjective content and review rating scale, which was necessary to validate the proposed opinionstrength rating scheme. The corpus consists of 500 Arabic movie reviews, 250 positive and 250 negative, collected from 15 different Arabic web sites. Each of the written reviews are sets of text detailing the movie and author’s commentary, with most reviews followed by an overall numerical rating of the movie to supplement the written review. To prepare the corpus for analysis, considering the non-uniform nature of the different reviews, a process involving tokenization, basic stop-word, special character, and punctuation removal, and light stemming were performed. Additionally, considering that each site utilizes a different rating scheme to allow the author to give an overall numeric rating of the movie under review, normalization of the rating schemes (rating out of five versus rating out of ten, for example) was performed [13]. For the purposes of the rating-strength assessment conducted using the proposed algorithm, only documents with a review rating were considered for the opinion-strength rating task (to establish a means of validation), and so the actual number of documents used were 483 (250 positive, and 233 negative), with 17 negative reviews excluded due to unavailable ratings. 4.3

Resources and Tools

Several resources and tools were employed in the algorithm implementation to prepare the text and aid in the rule-based rating task. Sentiment Lexicons and Modifier Lists: As mentioned in the literature, sentiment-bearing words and phrases are the most crucial components for opinionrelated tasks [6], considering that the overall semantic meaning of the sentences is ultimately reliant on the collection of the individual polar units within it. This features lends to the use of a set of sentiment lexicons and modifier lists adapted for various natural language processing systems [16]. Sentiment Lexicons: The applied set of four sentiment lexicons were adapted from the Mutli-Perspective Question Answering (MPQA) Subjectivity Lexicon [17], an English set of subjective words that are part of the OpinionFinder project [18], used to identify various aspects of subjectivity in document sets. The words are divided into four sets, {positive weak, positive strong, negative weak, negative strong}, with additional “subjectivity queues”, such as part-of-speech tags, that could be relevant to usage. As described by Riloff and Wiebe (2003), the sets were collected from both manually and automatically-tagged sources [19]. In order to use the wide-scale lexicon set for the presented Arabic analysis task, a translated version of the lexicon, the Arabic MPQA Subjectivity Lexicon [20] was adapted for use in the proposed system. Because of the scale of the

Finding Opinion Strength Using Rule-Based Parsing

515

lexicon, problems with the relevance of many translated terms appropriated the task of manual filtering of the set for irrelevant terms, post-translation. Modifier Lists: In addition to the sentiment lexicons used, a set of Arabicspecific modifiers was also utilized to help define the rules applied for document parsing, as input to the opinion-scoring phase of the algorithm with the sentiment words. The modifier set consists of three word lists, {negations, intensifiers, conjunctions}, which represent either full Arabic words or clitic prefixes for polarity inversion, incrementing or decrementing adverbs, and connective conjunctions. The effects of each of the modifier types are articulated in the derived rule sets, with the general intuition that at the most basic form negations serve to flip (or, at least diminish) the polarity of the following word or set of words, intensifiers update the polarity of the preceding word, and conjunctions carry polarity over to the connected set of words. 4.4

Document-Parsing Rule Set

As detailed in set of rules derived for document parsing presented in Figure 2, the top-most classification of a document is a group of one or more terminating (ending with a terminal punctuation) sentences, which are composed of one or more phrases. A phrase may consist of one or more expressions, which may include a negated phrase (simply defined as a basic phrase prefixed with a negation from the negations modifier list). These phrase-forming expressions, in turn, are more basic building-blocks containing conjunction groups (defined as strings of words connected with a conjunction from the modifier list), incremented words (single words preceded or followed by an incremented from the modifier list), or simply single-word units (which are later interpreted as either polar or not, depending on whether they exist in the sentiment lexicons). Each of the terminals are either entries in one of the modifier lists, basic punctuation, or word units. The document rule-parsing phase works by tokenizing each entering corpus document into its respective grammar units, in accordance with the components defined. The parser then reduces each set of tokenized groups using the grammar rule set, such that each level of granularity within the document (word, conjunction and negation groups, expressions, phrases, and sentences) are clearly and uniquely defined, and the parse tree for each document can be traced back to each of its distinct components, resulting in a single parse per sentence that is ready to be mapped for polarity queues. Sample parse trees for two corpus sentences [13] are shown in Table 1, to be mapped for sentiment words in the subsequent interpretation and score-calculation phases. 4.5

Opinion-Score and Rating Calculations

Given a rule-parsed document, the opinion-score calculation algorithm can be applied to find a polarity score for both positive and negative sentiment, followed by subsequent rating calculations on the derived opinion scores.

516

S. Oraby, Y. El-Sonbaty, and M.A. El-Nasr

Fig. 2. Derived rule set for document parsing

Opinion-Score Calculation The basic positive and negative opinion score points are determined when the parsed documents are interpreted by attempting to match each of the terminal words with words from the four adapted sentiment lexicons. Likewise, as dictated by the rule set, each of the individual modifier terminals serve to affect the score of the polar words they operate on. Negations: A negation can invert the polarity of the word it modifies or decrease its polarity in the same class, just as    (lays ˇgayd, meaning “not good”) does not necessarily mean  (sy, meaning “bad”), but rather a diminished form



of   (ˇgayd, meaning “good”). More particularly because of the less predictable sentence structure of Arabic (where the negations may be further off from the modified polar word), negation words in the algorithm serve to halve the score of the phrase that they precede, where this phrase could be either a single negated word, or a longer negated phrase.

Intensifiers: Intensifiers (in the most general case, “incrementers”, or words that intensify the polarity of the word they modify) double the score of the polarity word they operate on in the algorithm, which can appear either before or after the modifier. The scope of the simple grammar only handles the case of a direct intensifier attached to the modified polar word, such as the case of

    (lays ˇgayd ˇgd¯a, meaning “not very good”).

Finding Opinion Strength Using Rule-Based Parsing

517

Table 1. Sample parse trees for two OCA corpus sentences [13] using the rule-based parser Sentence 1 with Parse Tree

Sentence 2 with Parse Tree

alqs.h .tryfh lil˙g¯ ayh wamosaleyh “The story is very funny and entertaining.”

lm ykon z.¯ ahr¯ a bqowh fy fylmah “[It] was not powerfully present in his film.”

 $  % &$

!"  #

(“phrase”, [ (“WORD”, “ &$ ”), (“conjunction group”, [ (“incremented”,  (“WORD”, “ $ %”),  (“INC”, “  # ”)), (“CONJ”, “”), (“WORD”, “ !"”)])])

   '   ()$  *% +

(“negated”, (“NEG”, “  ”), (“phrase”,  ”), [ (“WORD”, “ + (“incremented”, (“WORD”, “ *% ”), (“INC”, “ ()$”)),  (“WORD”, “ ”),

 

(“WORD”, “ ' ”),  (“WORD”, “, ”)]))

Conjunctions: In the simple grammar presented, conjunction groups are defined as sets of expressions (which can be larger conjunction groups, or single words) connected by a conjunction word, such as -."    (ˇgayd wa mohim, meaning “good and important”). In the case of conjunction groups containing only similar-polarity words with neutral intermediate words, the algorithm calculates the score of the conjunction as the combined count of all the similar-polarity connected words in the group (thus, “passing the polarity” through the group). Review Rating Calculation: By interpreting each of the parsed documents with the basic counting scheme proposed, a final positive and negative score is calculated for each of the documents. These scores are calculated as the word scores from the sentiment lexicons, with a single point for a weak lexicon words, and double points for strong lexicon words, along with the updates to the scores based on the surrounding modifiers, according to grammar rules described. pos score: count of the positive polar words, with modifier updates neg score: count of the negative polar words, with modifier updates From these two document polarity scores, the overall document rating, as shown in Equation (1), is calculated by taking the positive polarity score over the total document polarity score (for both positive and negative), to give an estimate of the document’s polarity score as a ratio of polar units.

518

S. Oraby, Y. El-Sonbaty, and M.A. El-Nasr

doc rating =

pos score (pos score + neg score)

(1)

This rating is then compared to the actual numerical rating given in the review to give a sense of the polarity strength of opinions in the document, measured by the average absolute difference between actual and calculated ratings.

5

Results and Evaluation

The results of the review-rating experiments conducted on the 483 rated documents in the Opinion Corpus for Arabic (OCA) dataset [13] under the proposed rule-based algorithm are presented in the following section. Based on Equation (1), the absolute difference between the actual document review and the predicted document rating was found for all of the documents. Figure 3 shows a frequency distribution of the absolute difference in points between actual and predicated ratings for the full set of documents.

Fig. 3. Frequency of absolute difference in rating points between actual and predicted ratings versus the number of documents with that rating difference

For example, a positive document with an actual rating of 9/10 and given a predicted rating of 7.5/10 and would get a difference score of 1.5 points. The graph shows the highest frequency in the 2-3 difference point bin, with a drop in frequency in either point direction in the range of a 0-10 point-scale difference. An average difference of 2.1 and 2.5 points (and standard deviation of 1.3 and 1.4) was found for the positive and negative reviews, respectively, giving an average of 2.3 point difference between the ratings for both classes, as compared to the ratings given in the reviews. This gives an indication of how closely the document opinion score matches the actual rating (in ratio between positive and negative scores) and reflects the existence of significant polar items in the document under consideration. As a novel approach, the results give good evidence of the potential of using a rule-based method for the Arabic opinion-rating task.

Finding Opinion Strength Using Rule-Based Parsing

6

519

Conclusions

Due to the morphological complexity of languages such as Arabic, rule-based schemes for modeling and parsing the language are often considered difficult tasks. The proposed algorithm, which offers basic decomposition and modeling of Arabic grammar structure as focused around simple opinion-phrase units, including polar words and basic negation, intensifier, and conjunction modifiers, shows that using these simple rules and adapted word lists achieve promising results in sentiment rating tasks. For further enhancement of the approach, the word lists and rules themselves can be expanded and more thoroughly modeled to the intricacies of the language. Likewise, the task of using more semantic information and modeling of relationships between words in the text is important for further research. The great variance of the semantic meaning of words even in the same polarity class makes the task of measuring opinion strength very relevant. The proposed system uses the polar scores derived from the rule-based method to determine an overall document polarity rating, which coincides to a high degree with the actual document rating given by the author. With this extra classification information, documents can be also be ranked according to their sentiment strength. The proposed rule-based classifier and novel opinion-strength calculation scheme for Arabic reviews thus provide a scalable approach to sentiment analysis to help further the possibilities of text classification and rating schemes for more complex languages, helping to model sentiment and opinion more closely and correctly.

References 1. Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up?: sentiment classification using machine learning techniques. In: Proceedings of the ACL 2002 Conference on Empirical Methods in Natural Language Processing, EMNLP 2002, vol. 10, pp. 79–86. Association for Computational Linguistics, Stroudsburg (2002) 2. Yu, H., Hatzivassiloglou, V.: Towards answering opinion questions: separating facts from opinions and identifying the polarity of opinion sentences. In: Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, EMNLP 2003. Association for Computational Linguistics, Stroudsburg (2003) 3. Bruce, R., Wiebe, J.: Recognizing subjectivity: a case study in manual tagging. Nat. Lang. Eng. 5, 187–205 (1999) 4. Saleh, S., El-Sonbaty, Y.: A feature selection algorithm with redundancy reduction for text classification. In: 22nd International Symposium on Computer and Information Sciences, Turkey (2007) 5. Taboada, M., Brooke, J., Tofiloski, M., Voll, K., Stede, M.: Lexicon-based methods for sentiment analysis. Comput. Linguist. 37, 267–307 (2011) 6. Liu, B.: Sentiment Analysis and Opinion Mining. Morgan & Claypool Publishers (2012) 7. Turney, P.: Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting of the Association of Computational Linguistics, Philadelphia, Pennsylvania (2002)

520

S. Oraby, Y. El-Sonbaty, and M.A. El-Nasr

8. Kar, A., Mandal, D.: Finding opinion strength using fuzzy logic or web reviews. International Journal of Engineering and Industries 2(1) (2011) 9. Fu, G., Wang, X.: Chinese sentence-level sentiment classification based on fuzzy sets. In: Proceedings of the 23rd International Conference on Computational Linguistics: Posters, COLING 2010, pp. 312–319. Association for Computational Linguistics, Stroudsburg (2010) 10. Syed, A.Z., Aslam, M., Martinez-Enriquez, A.M.: Lexicon based sentiment analysis of urdu text using sentiunits. In: Sidorov, G., Hern´ andez Aguirre, A., Reyes Garc´ıa, C.A. (eds.) MICAI 2010, Part I. LNCS, vol. 6437, pp. 32–43. Springer, Heidelberg (2010) 11. Abdul-Mageed, M., Diab, M., Korayem, M.: Subjectivity and sentiment analysis of modern standard arabic. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers, HLT 2011, vol. 2. Association for Computational Linguistics, Stroudsburg, pp. 587–591 (2011) 12. Abbasi, A., Chen, H., Salem, A.: Sentiment analysis in multiple languages: Feature selection for opinion classification in web forums. ACM Trans. Inf. Syst. 26 (2008) 13. Rushdi-Saleh, M., Martin-Valdivia, M., Urena-Lopez, L., Perea-Ortega, J.: Oca: Opinion corpus for arabic. J. Am. Soc. Inf. Sci. Technol. 62, 2045–2054 (2011) 14. Habash, N.: Introduction to Arabic Natural Language Processing. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers (2008) 15. Ryding, K.: A Reference Grammar of Modern Standard Arabic. Reference Grammars. Cambridge University Press, New York (2006) 16. Ezzeldin, A.M., Kholief, M.H., El-Sonbaty, Y.: ALQASIM: Arabic language question answer selection in machines. In: Forner, P., M¨ uller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 100–103. Springer, Heidelberg (2013) 17. Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phraselevel sentiment analysis. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT 2005, pp. 347–354. Association for Computational Linguistics, Stroudsburg (2005) 18. Wilson, T., Hoffmann, P., Somasundaran, S., Kessler, J., Wiebe, J., Choi, Y., Cardie, C., Riloff, E., Patwardhan, S.: Opinionfinder: a system for subjectivity analysis. In: Proceedings of HLT/EMNLP on Interactive Demonstrations, HLTDemo 2005, pp. 34–35. Association for Computational Linguistics, Stroudsburg (2005) 19. Riloff, E., Wiebe, J.: Learning extraction patterns for subjective expressions. In: Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, EMNLP 2003, pp. 105–112. Association for Computational Linguistics, Stroudsburg (2003) 20. ALTEC: Arabic mpqa subjective lexicon (2012), http://www.altec-center.org/