Computational approaches for emotion detection in text

4th IEEE International Conference on Digital Ecosystems and Technologies (IEEE DEST 2010) © 2010 IEEE.

Computational Approaches for Emotion Detection in Text

Haji Binali, Chen Wu,Vidyasagar Potdar Digital Ecosystems Business Intelligence Institute Curtin University of Technology Perth , Australia [email protected], chen.wu, [email protected] Abstract— Emotions are part and parcel of human life and among other things, highly influence decision making. Computers have been used for decision making for quite some time now but have traditionally relied on factual information. Recently, interest has been growing among researchers to find ways of detecting subjective information used in blogs and other online social media. This paper presents emotion theories that provide a basis for emotion models. It shows how these models have been used by discussing computational approaches to emotion detection. We propose a hybrid based architecture for emotion detection. The SVM algorithm is used for validating the proposed architecture and achieves a prediction accuracy of 96.43% on web blog data. Keywords- Emotion detection; Text classification; Emotion models; Sentiment analysis

I.

INTRODUCTION

Emotion involves feelings, experience, physiology, behaviour, cognitions and conceptualisation [4, 5]. According to WordNet Search 3.0 1, an emotion is “any strong feeling”. Wikipedia 2 defines an emotion as “a mental and physiological state associated with a wide variety of feelings, thoughts, and behaviour”. We notice that the aspect of feeling is common and can be characterised by accompanying physiological changes and facial expressions. Emotion detection in text aims to infer the underlying emotions influencing the author/writer by studying their input texts. This is based on the premise that if a person is happy, it influences them to use positive words. Likewise, if a person is sad, frustrated or angry, the kind of words they use can allow us to infer their underlying negative emotion. Emotion detection in text has a number of important applications. In the area of business development, emotion detection can help marketeers to develop strategies for customer relationship 1 2

http://wordnetweb.princeton.edu/perl/webwn http://en.wikipedia.org/wiki/Emotion

management, new product development and service delivery. Psychologists can benefit from being able to infer people’s emotions based on the text that they write which they can use to predict their state of mind. This knowledge can be practically applied to predict consumer behaviour and customer preferences for corporate financial gain. In the field of education, the ability of computers to automatically track attitudes and feelings with a degree of human intuition has contributed to the development of Text-to-Speech systems and Intelligent Tutoring Systems (ITS). Web communication can be facilitated through video, voice recordings, images and text. The most prevalent form of communication on the web exists in the form of text which offers a rich platform for expressing emotions. In the absence of face-to-face contact to detect facial expressions and intonations in voice, the alternative option is to decipher emotions from text in online forums. Emotion detection research studies have investigated emotion detection in prosody[9], changes in physiological state [10, 11], facial expressions [12]and text[13, 14]. However, there is a paucity of research in emotion detection from text in comparison to the other areas of emotion detection [14, 15]. Being stuck can lead to frustration or abandonment of a task altogether or induce motivation toward finding alternative means for progressing. Anger can intensify defensive behaviour that shuts the path toward rational thinking thus closing the mind to new possibilities and outcomes. We learn from biological psychology, a branch of psychology that studies the relationship between behaviour and the body, particularly the brain that “reason without emotion is inadequate for making the decisions that guide our lives, and in fact make up our lives” [5]. This provides motivation and a new perspective on incorporating emotional intelligence within decision making systems. The need to understand emotions is further supported by a recent report that “W3C is

172


to work on a web standard for Emotion Markup” [16]. Text based emotion detection as mentioned earlier can be used in business, education, psychology and any other field where there is paramount need to understand and interpret emotions. In this paper, we will investigate emotion detection methods from text, propose an architecture for the same and discuss our preliminary findings. This paper is structured as follows. Section II is related work that covers emotion theories, Information Retrieval as a tool for emotion detection and computational approaches for emotion detection. Section III describes the emotion architecture that has been proposed and validated. Section IV explains the experimental setup that we used and Section V discusses the results. The conclusion and our future work are in Section VI. II.

RELATED WORK

A. Emotion theories In this section, we will briefly mention Ekman’s emotion model and the OCC (Ortony/Clore/Collins) model. Emotion models stipulate needed knowledge to appraise events. Ekman’s emotion model consists of “sadness, happiness, anger, fear, disgust and surprise” [4]. It has been used in systems that recognise facial expressions related to these emotional states. The OCC model presents emotions generally expressed by an agent. It includes 22 emotion categories [1] designed to model humans in general. It is based on the premise that emotions “are not themselves linguistic things, but the most readily available non phenomenal access we have to them is through language” [1]. The major benefit of using the OCC model is that it provides a specification for different kinds of emotions with a relatively culture free footing. This eliminates the need to study emotions based on specific words but as classes of underlying emotion types. In other words, it is a theory of emotions and not a theory of the language of emotions. The six emotion types in Ekman’s model appear in the OCC model because the specific keywords specified in the former model can be mapped onto specific emotion types specified using tokens (e.g. happiness, sadness, anger, disgust, surprise and fear) in the latter. The OCC emotion types are: Happy-for, Sorry-for, Resentment, Gloating, Hope, Fear, Fears-confirmed, Relief, Disappointment, Pride, Self-Reproach, Appreciation, Reproach, Gratitude, Anger, Gratification, Remorse, Liking, Disliking, Shame, Admiration, Pity[1]. B. Information retrieval as a tool for emotion detection Information retrieval is the acquisition of specific information about a topic and is usually based on a query[17]. Text classification consists of categorising text at the document, sentence or token level (e.g. document classification, opinion recognition, word sense disambiguation)[18]. Classification is an information retrieval task used for analysing the content of unstructured data expressed in natural language on the web ( i.e. email, scientific documents and government reports)[19]. Text classification is currently the main method for emotion detection in text[14] . However, due to the highly subjective

nature of what actually constitutes an emotion, this method faces many challenges[14] which have generated interest among researchers. The major challenge identified is the inclusion of subjectivity detection mechanisms. Furthermore, Kao et al. [8] have specifically identified three issues associated with keyword spotting techniques. They are; “ambiguity in keyword definitions”, “incapability of recognising sentences without keywords” and “lack of linguistic information”. Experiments in recognising emotions from text by [13] lead to the conclusion that in-depth semantic analysis of text is necessary for more accurate results. Domain specific thesauri describing keywords and phrases have been used in information retrieval systems to improve information retrieval tasks[19]. However, the problem with this approach is that it could lead to non portability across domains resulting in poor emotion identification. Furthermore, their construction is costly as it involves human indexers. C. Computational approaches for emotion detection Three approaches currently dominate the emotion detection task; keyword based, learning based and hybrid based approach. These make use of features mainly selected from syntactic (e.g. n-grams, pos tags, phrase patterns) and semantic (e.g. synonym sets) data to detect emotions. For example, in Table 1, Chunling et al. [1] used WordNet 1.6 to extract synonym sets (i.e synsets according to wordNet terminology) from emotion seed words which are grouped according to their part of speech (pos) tag. This was used to construct an affective lexicon by using Open Mind Common Sense Knowledge base (OMCS) as a knowledge-rich linguistic resource since wordNet alone provides a general meaning of words. In this section, we will provide a brief description with particular emphasis on the pros and cons of each of these approaches. Table I is a compilation of recent studies that have used these approaches for emotion detection and we refer to it in our discussion below. 1) Keyword based approach This approach has a total reliance on the presence of keywords and may involve pre-processing with a parser and emotion dictionary. It is easy to implement, intuitive and straight forward since it involves identifying words to search for in text. As shown in Table 1, this has been applied in real time chat systems by [1-3]. However, it is domain specific, relies on the presence of keywords for accurate results and requires pre-processing for improved accuracy results. 2) Learning based approach This approach uses a trained classifier to categorise input text into emotion classes by using keywords as features. It is easier and faster to adapt to domain changes since it can quickly learn new features from corpora by supplying a large training set to a machine learning algorithm for

173


Author

M. Chunling et al. [1]

Task description

Emotion detection in chat system and displaying emotion using an avatar Ekman

Emotion model

Detection approach Features

Granularity

Keyword based WordNetAffect DB. wordNet1.6. Emotional weight, OMCS (Open mind common sense) knowledge base Sentence

TABLE I. J. T. Hancock et al. [2]

Emotion detection studies

H. Li et al. [3]

S. Aman and S.Szpakowicz [6]

C. Yang et al. [7]

E. C.-C. Kao et al.[8]

Laboratory controlled online chat experiment to enact sadness and happiness and reporting strategies that people employ to express emotions in text

Emotion detection incorporating personality factor in chatting system to improve accuracy results

Emotion annotation task by category, intensity and word/phrase

Emotion classification from training at sentence level and applying at document level with sentence level contextual meaning

A survey on detection methods and possible improvements

Social information processing model, happy and sad

OCC model, FiveFactor Model (FFM)

Ekman

Not specified

OCC

Keyword based

Keyword based

Hybrid

Learning systems

Hybrid

Linguistic Enquiry and word count program, LIWC

OpenMind Common sense knowledge base (OMCS), ConceptNet

WordNet-Affect, General Inquirer

Lexicon constructed from training set

Not specified

Sentence

Sentence

Sentence

Sentence

Sentence

building a classification model. However, acquiring large corpora may not always be feasible. Nonthetheless, this facilitates easy implementation of classifiers by novices who can then apply the learned model to new instances. The major drawback of this approach is that it leads to blurred boundaries between emotion classes and a lack of context analysis. Refer to Table 1 for an emotion corpus annotation task example by[6]that highlights this issue. To clarify the classification task, we introduce the support vector machine(SVM). SVM is a popularly used statistical learning theory based algorithm. It has achieved empirically good performance in many fields (bioinformatics, text, image recognition). For more detailed information on the use of svm for emotion detection, please refer to [20]. SVM assumes an even distribution between positive and negative training examples from the SVM hyperplane as shown in Fig1. Classification is based on the margin distance from the decision hyperplane(h) to the positive and negative training examples.

+

+

__ Margin h __ __ + + __ __ + Margin argin __ Fig 1: Distribution of even margins from svm hyperplane (h)

3) Hybrid based approach This approach consists of a combination of the keyword based implementation and learning based implementation. The main advantage of this approach is that it can yield higher accuracy results from training a combination of classifers and adding knowledge-rich linguisitic information from dictionaries and thesauri. The advantage of this is that it will offset the high cost involved in using human indexers for information retrieval tasks and minimise complexities encountered while integrating different lexical resources. In this section, we have outlined the major approaches for text based emotion detection and shown how syntactic and semantic information can be beneficial for emotion detection. Current methods are lacking in in-depth semantic analysis for detecting hidden phrase patterns and more investigations need to be done to identify, build and incorporate knowledge rich linguistic resources that have a focus on detecting emotions. The architecture proposed in section III is aimed at meeting these drawbacks. ARCHITECTURE FOR EMOTION DETECTION III. In this section, we will present a hybrid based architecture comprising of a keyword based component and learning system component. The solution is proposed to meet the following requirements: 1. Hidden phrase pattern detection: Subtle sentences expressing emotions should be identified based on syntactic and semantic details.

174


Emotion prediction: Identify the emotion being expressed based on knowledge rich linguistic resources and trained classifiers.

2.

To address the requirements above, we have chosen the following design decisions. 1.

Incorporate a mechanism to identify keywords.

2.

Examples of hidden patterns should be provided by a human annotator to assist in training the classifier.

Fig.2 is a hybrid emotion detection architecture that we propose and conceptually depicts the prevailing approaches to emotion detection as discussed in Section II. The keyword based component is largely based on the knowledge engineering approach for Information Extraction and is represented in Fig.2 by the tokeniser, sentence splitter, POS tagger and Emotion Intelligence PR. Our discussion on the architecture is based on the way in which it is implemented in GATE (section IV explains what GATE is). The first stage of pre-processing commences with reducing the text to its Web blog text

Tokeniser

Sentence splitter POS Tagger Emotion Intelligence PR Emotion Predictor Predicted Emotion

A. Keyword based component 1. Web based blog data is collected in a corpus. 2. The text is then split into tokens and sentences are identified. 3. The tokens are then annotated with Part of Speech (POS) tags. 4. Syntactic and semantic data is used for Postprocessing tasks before machine learning takes place. B.

Learning based component

1.

Supply corpus to be used as the training set.

2.

Perform pre-processing tasks as listed in A above.

3.

Select a classifier (algorithm) for classification.

4.

Generate feature vectors from the training set.

5.

Repeat steps 1-4 with the testing set.

6.

Convert feature vectors to LibSVM format.

7. Run LibSVM to measure prediction accuracy of classifier. IV.

Build learning model Apply learning model

addressed in the proposed architecture. Hidden phrase patterns can be detected and contextual analysis improved by using the sentence splitter, tokeniser and POS tagger to obtain syntactic and semantic information. We now summarise the key aspects of the keyword based component and learning based component which constitute the hybrid architecture.

Machine Learning PR

Figure 2: Emotion detection architecture

smallest processing component (token) using the tokeniser. The text is split into sentences using a sentence splitter. These sentences form the basic unit for sentence classification. Thereafter, we employ a POS tagger to annotate the data with syntactic information essentially identifying all verbs, nouns, adverbs and adjectives. This information is used in the Emotion Intelligence PR (e-Intel) to create new annotations within GATE. The e-Intel PR obtains its semantic information from gazetteer lists comprising of key terms that humans identified as being indicators to identify emotions in sentences. This knowledge is used to categorise emotions into Ekman’s basic emotions. Once this is completed, the last step in the knowledge processing component is to categorise the emotions into positive and negative classes. Using a machine learning PR involves two steps. The first is to build the training model used for emotion prediction. The second step is to apply the learned model to a testing dataset to predict the emotion class. Overall, you can see how the design requirements have been

EXPERIMENTAL SETUP

The conceptual framework for this research is based on a hybrid approach consisting of a knowledge based component and learning system component as proposed in section III. The output of the former is used as input for the latter in supervised machine learning. We introduce the main tool that we will be using in our research based on the GATE user guide [18]. The General Architecture for Text Engineering (GATE) is an all in one tool used for experimentation, development and deployment of Language Engineering processes. GATE uses a rule-based Information Extraction (IE) system that provides the low level features to handle common tasks in Natural Language Processing (NLP) and Language Engineering (LE). We use this IE system to implement the tasks of the knowledge based component of our architecture in conjunction with an emotion processing resource we developed. It consists of gazetteer lists that provide semantic information in conjunction with jape scripts that manipulate syntactic details (i.e. pos tags and phrase patterns) to create new annotations for emotion classification. This directly addresses our design requirements stated in section III. Among the several components that GATE has, the one of particular interest to our research is the one used for machine learning which we use for implementing the tasks of the learning component. The most important outputs of this component are feature vector files and learned models. The purpose for which we are using supervised machine learning in

175


this research is to detect patterns in a testing set that will most likely be representative of emotions in the training data. GATE allows the generation of feature vectors that can be tested in other systems. The feature vectors produced are in the format: 685 1 2 2:1.0 3:1.0 4:1.0 5:1.0 8:1.0 10:0.43972343 14:0.5392087 16:0.7182599. The first number (685) is an index number assigned to a sentence in GATE. The second number shows the number of class labels in the sentence. The third number is a class label. After the class label, there is a series of numbers separated by “:”. The interpretation of this is as follows: 2:1.0 means that 2 is the dimension of the annotation in GATE and 1.0 is its value. When converting to Libsvm 3 format, the first number and second number representing the index of the sentence and number of class labels respectively are ignored. The benchmark dataset developed by Aman and Szpakowicz [6, 21] comprising of web blog data from online forums was used as a training set. Based on the features (i.e. keywords) that the annotators’ reported as having provided clues to annotate emotions in sentences, we developed gazetteer lists and wrote Jape rules to automatically annotate and classify sentences into classes. The jape rules used syntactic information in the form of POS tags obtained using GATE’s POS tagger to identify adjectives, nouns, verbs and adverbs. In the first instance, we label the sentences into one of six categories based on Ekman’s basic emotions. We further split these into positive and negative classes for binary classification. The positive class consists of emotions happy and surprise whereas the negative class consists of fear, disgust, anger and sadness. The labelled data is the source of the feature vectors for the training set used in machine learning with GATE-SVM. The main outputs of this process in GATE are a learned model that is applied to the testing set to classify emotions and a feature vector file that can be used externally from GATE. The GATE generated feature vector file was converted to Libsvm format before being utilised for measuring the prediction accuracy. The testing set comprises of online student reviews collected from the web. The feature vectors were generated in the same manner as those of the training set. The training set is used to construct a classifier (model) based on annotated data. Determining the emotion of a sentence can be cast as a binary-class classification problem. Let T denote the text, and s a sentence, where s ∈ T. Let E ={empos,emneg}, where empos denotes a positive emotion and emneg a negative one. The goal is to determine a mapping function f: s → E , thus obtaining an ordered labelled pair (s, emn) where n = positive or negative. The mapping is based on F = {f1, f2,..., fn}, where F contains the features (i.e. keywords, semantic and syntactic data) derived from the training data. V.

DISCUSSION OF RESULTS

In this paper, we proposed an emotion detection architecture that can be used for emotion classification. As stated earlier, one of the major problems is the inclusion of subjectivity detection mechanisms. Our first step towards this end is the use of gazetteer list in conjunction with syntactic 3

data. Emotions were detected based on keywords obtained from gazetteer lists that specifically deal with discovering emotion keywords. Semantic analysis was performed to enhance the detection accuracy. We obtained a prediction accuracy of 96.43% with Libsvm which would suggest that SVM can accurately predict the emotion class if aided with some human annotated examples that are knowledge rich. Furthermore, these results confirm that the standard svm algorithm previously discussed is classifying data that is evenly distributed around the decision hyperplane. The test set comprised of 560 instances (sentences) of which 540 were rightly classified. The training set comprised of 2340 instances classified into positive and negative classes. We found the optimum cost value for the svm classifer to be 0.125. VI.

CONCLUSION AND FUTURE WORK

In this paper, we have demonstrated how emotion theories have been used as a basis for building models applied in computational approaches in emotion detection. We have presented a literature review of the latest research in this area. A hybrid based architecture for emotion detection has been proposed and validated with experimental results that suggest semantic and syntactic information can vastly improve the prediction accuracy. Our future research will focus on determining portability across domains and the development of heuristics that can be implemented in emotion detection to reduce classification bias.

VII.

REFERENCES

[1] M. Chunling, H. Prendinger, and M. Ishizuka, "Emotion Estimation and Reasoning Based on Affective Textual Interaction " in Affective Computing and Intelligent Interaction. vol. 3784/2005: Springer Berlin / Heidelberg, 2005, pp. 622-628. [2] J. T. Hancock, C. Landrigan, and C. Silver, "Expressing emotion in text-based communication," in Proceedings of the SIGCHI conference on Human factors in computing systems, 2007, pp. 929 - 932 [3] H. Li, N. Pang, and S. Guo, "Research on Textual Emotion Recognition Incorporating Personality Factor," in International conference on Robotics and Biomimetics, Sanya, China, 2007. [4] A. Ortony, G. L. Clore, and A. Collins, The cognitive structure of emotions: Cambridge University Press, 1988. [5] B. Garett, Brain and behaviour: an introduction to biopsychology, 2nd ed.: SAGE Publications, 2009. [6] S. Aman and S. Szpakowicz, "Identifying Expressions of Emotion in Text " in V. Matousek, P. Mautner (eds.): Proc 10th International Conf. on Text, Speech and Dialogue TSD 2007, Plzeň, Czech Republic, Lecture

http://www.csie.ntu.edu.tw/~cjlin/libsvm/

176


Notes in Computer Science 4629, Springer, 2007, 196205 [7] C. Yang, K. H.-Y. Lin, and H.-H. Chen, "Emotion Classification Using Web Blog Corpora," in IEEE/WIC/ACM International Conference on Web Intelligence, 2007.

[14] J. Read, "Recognising Affect in Text using PointwiseMutual Information ": Master of Science, University of Sussex, 2004. [15] A. Esuli and F. Sebastiani, "Determining Term Subjectivity and Term Orientation for Opinion Mining," in Proceedings of EACL 2006.

[8] E. C.-C. Kao, L. Chun-Chieh, Y. Ting-Hao, H. ChangTai, and S. Von-Wun, "Towards Text-based Emotion Detection," in International Conference on Information Management and Engineering, 2009, pp. 70-74.

[16] M. Schröder, "W3C to work on web standard for Emotion Markup ", Retrieved from http://emotionresearch.net/publicnews/w3c-working-on-web-standardfor-emotion-markup, Accessed on 1/10/2009.

[9] T. S. Polzin and A. H. Waibel, "Detecting Emotions in Speech," in Proceedings of the CMC,, 1998.

[17] H. H. Binali, V. Potdar, and C. Wu, "A State Of The Art Opinion Mining And Its Application Domains," in IEEE ICIT09, Gippsland, Victoria, Australia, 2009.

[10] K. H. Kim, S. W. Bang, and S. R. Kim, "Emotion recognition system using short-term monitoring of physiological signals " Medical and Biological Engineering and Computing, vol. 42, pp. 419-427, 2004. [11] A. Haag, S. Goronz, P. Schaich, and J. Williams, "Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System " in Affective Dialogue Systems. vol. 3068/2004: Springer Berlin / Heidelberg, 2004. [12] L. C. D. Silva, "Facial Emotion Recognition Using Multimodal Information," in Proc. IEEE Int. conf. on Information, Communications and Signal Processing (ICICS'97), Singapore, 1997, pp. 397-401. [13] S. Aman, "Recognizing Emotions in Text," Master of Computer Science,University of Ottawa, 2007.

[18] H. Cunningham, D. Maynard, K. Bontcheva, V. Tablan, C. Ursu, M. Dimitrov, M. Dowman, N. Aswani, I. Roberts, Y. Li, A. Sha, and A. Funk, "Developing Language Processing Components with GATE Version 5 (a User Guide)," 2009. [19] F. Sebastiani, "Machine Learning in Automated Text Categorization," ACM Computing Surveys, vol. 34, No.1, pp. 1-47, 2002. [20]Z. Teng, F. Ren, and S. Kuroiwa, "Recognition of emotion with SVMs," D.S. Huang, K.Li and G.W. Irwin (Eds.): ICIC 2006, LNAI 4114, pp. pp.701-710, 2006. [21]S. Aman and S. Szpakowicz, "Using Roget's Thesaurus for Fine-grained Emotion Recognition," in Third International Joint Conf. on Natural Language Processing IJCNLP 2008, Hyderabad, India, 296-302, 2008.

177