Mining User Personality in Twitter Fabio Celli CLIC-CIMeC, University of Trento
[email protected] September 8, 2011 Abstract The paper describes how we collected and annotated “Personalitwit”, a corpus of 25700 posts from the popular micro-blogging site Twitter, automatically annotated by user personality and by language with two computational linguistic tools. From the analysis of that data emerged how different writng styles and personality models are associated to different communitites using different devices to post to Twitter. Keywords: Social Network Sites; Personality Recognition; Information Extraction; Twitter
1
Introduction and Background
Twitter is one of the most popular micro-blogging site. It is a social network site, following Boyd and Ellison’s definition (see [1]). People use Twitter in different ways: Zhao and Rosson 2009 ([13]) highlight the fact that people use twitter for a variety of social purposes, for instance (1) keeping in touch with friends and colleagues; (2) raising visibility of interesting things to ones social networks; (3) gathering useful information for ones profession or other personal interests; (4) seeking for helps and opinions; and (5) releasing emotional stress. They also report that the way people use twitter can be grouped in three broad classes: (1) update personal life activities, a blog-like way of using Twitter; (2) do real-time information, a journalistic style and (3) follow people -based RSS feeds, which is a way to be informed about personal intersts. In recent years many scholars showed interest towards twitter, also from a Natural Language Processing perspective, for example Pak and Paroubek ([11]) built a sentiment analysis classifier from Twitter data, in order to recognize automatically when a post is about positive, negative or neutral emotions. Zhao et Al. (see [14]) proposed a ranking algorithm for extracting topic keyphrases from Twits. Finin et Al. (see [5]) performed Named Entity Recognition on Twitter using crowdsourcing services such as Mechanical Turk1 (see [12] for details) and Crowdflower2 in order to provide a first step towards semantic annotation in a Social Network Site domain. 1 https://www.mturk.com/mturk/welcome 2 http://crowdflower.com
1
In this paper we use a language recognition tool (see [2] and [3]) and a personality extraction tool (see [4]) to annotate a dataset sampled from Twitter. To the best of our knowledge it is the first time that a personality recognition tool is run on Twitter. In the next section we describe in detail the dataset, we provide a definition of personality and a description of the personality recognition tool and finally we report the results of the experiments.
2 2.1
Tools, Dataset, Experiments and Discussion Formalization of Personality, Description of the Personality Recognition Tool
Personality is a complex of attributes that characterise a unique individual. Psychologists formalize personality along five traits known as the “Big Five” (see Goldberg 1992 [6]), a model introduced by Norman 1963 [9], obtained from factor analysis of personality description questionnaires that has become a standard over the years. The five dimentions are the following: Extraversion (E) (sociable vs shy); Emotional stability (S) (calm vs insecure); Agreebleness (A) (friendly vs uncooperative); Conscientiousness (C) (organized vs careless); Openness (O) (insightful vs unimaginative). Mairesse et Al. 2007 [8] provides a long list of linguistic features that correlate with personality traits of the “Big Five” model in English. We selected 12 cross-linguistic features in order to develop our personality recognition tool, namely:(1) all punctuation (ap): the total of . , ; : in the sentence; (2) commas (cm): the total of , in the sentence; (3) reference to other users (du): the frequency of the pattern @... in the sentence; (4) exclamation marks (em): the total of ! in the sentence; (5) negative emoticons (ne): the total of emoticons expressing negative feelings in the sentence; (6) numbers (nb): the total numbers in the sentence; (7) parenthesis (pa): the total number of parenthetical phrases in the sentence; (8) positive emoticons (pe): the total of emoticons expressing positive feelings in the sentence; (9) question marks (qm): the total of ? in the sentence; (10) long words (sl): total number of words longer than 6 characters in the sentence; (11) type/token ratio (tt): ratio of new words used in a sentence, defined in formula below; (12) word count (wm): total words in the sentence. For the correlation between those features and the “Big Five” traits see Celli 2011 ([4]). The personality recognition tool do not need annotated data in order to modelize users’ personality, but it compares the posts of each user within them and provides accuracy and validity as evaluation measures. Accuracy gives a measure of the reliability of the personality model and validity gives information about how much the model is valid for all the user’s posts, in other words how much one user writes expressing the same personality traits in all his/her posts. The system can evaluate personality only for users that have more than one post, the other users (and their posts) are discarded. Personality models are represented in strings of five characters that stand for the traits of the “Big Five” and take 3 possible values: y, n, o. For example the string “ynoon” represents an extravert, insecure and unimaginative person.
2
2.2
Working Hypothesis
It is clear that under this computational perspective personality has much to do with writing style, and that there are some problems in the extraction of personality from twitter: for instance: (1) personality depends on culture. Correlations between features and personality traits were studied on English, and are supposed to work properly on western cultures, but slight changes in correlation might occur in different cultures. The fact that we have many features prevents cultural correlation changes. (2) Twits are very short and this influences the way people write, this means that the results we obtain in Twitter are not generalizable out of it. Our working hypothesis is that if it is true that twitter influences the way people write, it is also true that other media, such as Twitter for iPhone, Twitter for Blackberry or Twitter for Facebook, influence the way people write. And we hypothesize also that topics such as the royal wedding, seen as a special type of aggregation medium, have an influence on the way people write. In this paper we analyze those kind of influences.
2.3
Collection of the Dataset
The dataset, called “Personalitwit” corpus was collected via Twitter public timeline3 with a variable sample rate (20 to 80 posts per minute) between April 28th and May 2nd 2011. Different events happened in that period, for instance Bin Laden’s death, the Royal Wedding and Aruba’s crashdown. We annotated the corpus with the language identification and personality recognition tools, discarding all the posts of users that appeared only once because of the personality recognition evaluation process. The information provided in the annotation of the corpus includes: • username, • post, • date, • platform from which the user posted, • personality model, • accuracy of the personality model, • validity of the personality model, • language. Results of the evalution of the personality annotation are a mean accuracy=0.6651 and mean validity=0.6994.
2.4
Experiments and Discussion
Results show that in general the personality of people writing in Twitter is an introvert, secure and open-minded person (nyooy). Other frequent (above 1000 instances) models are reported in table 1. A difference is observed between 3 http://twitter.com/public
timeline
3
agreeble people participating to the discussion about the royal wedding with respect to people participating to the discussion about Bin Laden’s death. This could be a consequence of the large posting activity about the royal wedding. Table 1 also shows that unprecise people tend to retwit more than other people.
binladen royalwedding retweet tweet twitter blackberry iphone ubersocial facebook
nynny 0.0% 7.6% 17.5% 5.1% 6.8% 9.3% 3.9% 16.2% 8.7%
nynoy 37.5% 18.4% 15.5% 18.5% 17.0% 9.4% 12.8% 13.0% 22.4%
nyony 12.5% 8.4% 21.4% 8.6% 10.8% 18.6% 7.7% 21.2% 14.1%
nyooy 50.0% 39.6% 29.3% 44.8% 38.3% 29.1% 37.2% 27.2% 46.8%
nyyoy 0.0% 22.0% 16.1% 10.8% 13.0% 15.6% 16.9% 16.1% 7.6%
ooooo 0.0% 4.0% 0.2% 12.2% 14.0% 18.1% 21.5% 6.4% 0.4%
Table 1: Cooccurrence of topics, retweet and posting platforms with most frequent personality models Really interesting things can be observed taking Twitter interface users as a baseline and comparing personality of users writing from different platforms. For example the community of people posting from iPhone is less unprecise with respect to the community of people using Blackberry. We also note that Blackberry and iPhone communitites produce more flat personality types (ooooo), and this can be due to the fact that they post very short messages, probably with an SMS style, hence the system has some problems in the assignment of personality traits. The community of people using Ubersocial is the most unprecise, while people posting from Facebook tend to be more uncooperative and unprecise than people writing directly from Twitter. It is also interesting to note that people writing from Facebook have the lowest rate of flat personality models. This can be due to the fact that people in facebook is used to write longer sentences with respect to Twitter users, hence it is easier for the system to recognize their personality.
3
Conclusions and Future work
In this paper we produced a dataset called “Personalitwit”, sampled from Twitter and annotated with language and personality models. From the analysis of that data emerged how different writng styles and personality models are associated to different communitites using devices to post to Twitter. For the future we would like to sample more data from Twitter and improve the personality recognition system.
References [1] Boyd, D., Ellison, N. Social Network Sites: Definition, history, and scholarship. In Journal of Computer-Mediated Communication 13(1). 2007.
4
[2] Celli, F. SLIde: Simple Language Identifier. (technical report available at http://clic.cimec.unitn.it/fabio). 2009. [3] Celli, F. Improving Language identification performance with FriendFeed data. (technical report available at http://clic.cimec.unitn.it/fabio). 2009. [4] Celli, F. Unsupervised Recognition of Personality from Linguistic Features. (technical report available at http://clic.cimec.unitn.it/fabio). 2011. [5] Finin, T., Murnane, W., Karandikar, A., Keller, N., Martineau, J., Dredze, M. Annotating named entities in Twitter data with crowdsourcing. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk (CSLDAMT ’10). Stroudsburg, PA, USA, 2010. [6] Goldberg, L., R. The Development of Markers for the Big Five factor Structure. In Psychological Assessment, 4(1). 1992. [7] Mairesse, F., Walker, M. Words mark the nerds: computational models of personality recognition through language. In: Proceedings of the 28th Annual Conference of the Cognitive Science Society. Vancouver, Canada. 2006. [8] Mairesse, F., Walker, M. A., Mehl, M. R., Moore, R, K. Using Linguistic Cues for the Automatic Recognition of Personality in Conversation and Text. In Journal of Artificial intelligence Research, 30. 2007. [9] Norman, W., T. Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality rating. In Journal of Abnormal and Social Psychology, 66. 1963. [10] Oberlander, J., Nowson, S. Whose thumb is it anyway? classifying author personality from weblog text. In Proceedings of the 44th Annual Meeting of the Association for Computational Linguistics ACL. Sydney. 2006. [11] Pak, A., Paroubek P. Twitter as a corpus for sentiment analysis and opinion mining. In Proceedings of LREC 2010 Malta. 2010. [12] Snow, R., O’Connor, B., Jurafsky, D., Ng, A. Y. Cheap and fast—but is it good?: evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Honolulu. 2008. [13] Zhao, D., Rosson, M.B. How and why people Twitter: The role that micro-blogging plays in informal communication at work. In Proceedings of GROUP 2009 pp. 243–252. New York. 2009. [14] Zhao, W.X., Jiang, J., He, J., Song, Y., Achananuparp, P., Lim, E.P., Li, X. Topical keyphrase extraction from Twitter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1 (HLT ’11), Stroudsburg, PA, USA, 2011.
5
