Spam Profile Detection in Social Networks Based on Public Features ...

2017 8th International Conference on Information and Communication Systems (ICICS)

Spam Profile Detection in Social Networks Based on Public Features Ala’ M. Al-Zoubi∗ , Ja’far Alqatawna∗† , Hossam Faris∗ ∗ King

Abdullah II School for Information Technology Business Information Technology Department The University of Jordan, Amman, Jordan [email protected],{j.alqatawna,hossam.faris}@ju.edu.jo † Jordan Information Security and Digital Forensics Research Group (JISDF) Amman, Jordan Abstract—In the context of Online Social Networks, Spam profiles are not just a source of unwanted ads, but a serious security threat used by online criminals and terrorists for various malicious purposes. Recently, such criminals were able to steal a number of accounts that belong to NatWest bank’s customers. Their attack vector was based on spam tweets posted by a Twitter account which looked very close to NatWest customer support account and leaded users to a link of a phishing site. In this study, we investigate the nature of spam profiles in Twitter with a goal to improve social spam detection. Based on a set of publicly available features, we develop spam profiles detection models. At this stage, a dataset of 82 Twitter’s profiles are collected and analyzed. With feature engineering, we investigate ten binary and simple features that can be used to classify spam profiles. Moreover, a feature selection process is utilized to identify the most influencing features in the process of detecting spam profiles. For feature selection, two methods are used ReliefF and Information Gain. While for classification, four classification algorithms are applied and compared: Decision Trees, Multilayer Perceptron, k-Nearest neighbors and Naive Bayes. Preliminary experiments in this work show that the promising detection rates can be obtained using such features regardless of the language of the tweets. Keywords—Spam; Feature selection; Social Networks; Twitter; Classification

I.

INTRODUCTION

Online Social Networks (OSNs) such as Facebook, Twitter, and LinkedIn have become the most influential tools in the web. In 2016, these OSNs have combined more than 2 billion profiles [1]. With this large number of users, communication with friends has hit its highest peak ever. The increased popularity of the OSNs can be attributed to various reasons including the mimic of real social interaction along with the ability to instantly publish any user generated content using a very convenient web-based platform. From business perspective, these platforms provide both governmental and commercial agencies with efficient ways for marketing, customer behavior analysis and opinion mining [2]. All these attractive features have made OSNs everyone’s first choice to use in the internet [3]. However, everything with facinating features bring along its dark side as well. One aspect of this is the Spammers or Spam Profiles that take place in the dark side of the OSNs [4]. In the context of the OSNs, spam could take 978-1-5090-4243-2/17/$31.00 ©2017 IEEE

several forms, for example: fake profile, fraudulent reviews, clickjaking, advertisement, malicious links, and malicious files. The first appearance of spamming in the internet was in a shape of emails. Spam emails has stably grown since the 90s [5]. Spam can be defined briefly as sending unsolicited messages to other users; these messages could be a malware, advertisement, or link’s (URL) to other dangerous (malicious) websites [6]– [8]. Although, spam messages and spam profiles are treated differently in the literature, they have an inherent connection. For instance, spam profiles have a high probability to send spam messages; and spam messages are posted by spam profiles (users) [9]. In most cases, spammers are interested in generating vandalism, nuisance and anarchy by posting unsolicited or malicious contents, thus, leading to indignation from users. False advertisements, for instance, could misguide users and could cause unfavorable effects in fair trading. Spam does not even differentiate between adult and young, hence, the risk of exposing children to diverse pornographic links [10]. Accordingly, OSNs can impact both the virtual world and the real world [11]. Thus, the protection of these platforms is a real necessity. Both regulations and technical controls are important to thwart spammers and to increase users’ trust in these platforms. Although spam in the context of email system has been extensively studied, several recent security reports [12], [13] show that the volume of spam and the severity of the attacks it carries are still representing an issue especially with the emergence and the widespread of of OSN. As spam restrain the performance, productivity and trustworthiness of the OSNs leading to expose users to various security threats, it is important to limit the number of spam profiles or to completely eliminate them [14]. All the previous factors have motivated us to study the nature of spam profile in OSNs. This paper reports a preliminary result of a research project which investigates the nature of spam profiles in the context of Twitter OSN. The project’s aim is to improve social spam detection by utilizing a set of simple and publicly available features regardless the language of the profiles’ tweets. At this stage a dataset of 82 Twitter’s profiles of Arabic and English tweeters are collected and analyzed. With feature engineering we propose ten binary features that can be used to classify spam profile in the context of Twitter. Moreover, feature selection will be used to identify the most influencing factors in the process of identifying spam profiles. 130


The rest of this paper is organized as follows: Section 2 briefly reviews related work. Section 3 presents our methodology and the proposed approach. Classification methods and feature selection techniques utilized in this study are outlined in section 4 and section 5, respectively. In Section 6 we discuss the experimental results and finally the conclusion is provided in section 7. II.

RELATED WORK

Although spam is mostly recognized in email systems [7], [15]–[17], it has inherent relations with the spam in OSNs. In [9], these relations were exploited to improve the detection results. On the other hand, it was argued in [18] that web and email spams are different from social spam, therefore, handling this type of spam needs to be in a completely different fashion. Studies in OSNs spam detection indicated that most of detection process can be categorized through their features. In [19] it was suggested that detecting spammers’ profiles can be done by analyzing their social graph and determining the times when the edges (Likes) were created in the graph. Whereas, [20] pointed out that the nodes and the edges refer to the profiles and their interactions, respectively. The interactions are page likes, active friends and shared URLs. A work similar to our study were reported in [21]. However, it focused on detecting fake profiles in LinkedIn. They employed a set of public attributes such as number of languages a person can speak, presence of profile summary, number of education qualifications and number of connections. using data mining approach they have obtained an accuracy of 84% and only 2.44% false negative. Another study [22] analyzed Twitter’s profiles that send spam posts and how spammers exploit links in Twitter for phishing, malware distribution and fake ads. They have studied the effectiveness of spam blacklist features offered by Twitter. Their study indicated that spammers propagate their spam by misusing compromised accounts instead of creating new ones. The results revealed that 0.13% accounts were exposed to spam. A spam detection framework that is based on user judgements and their carefulness were proposed in [23]. Its approach depends on how careful the user is when he/she tries to avoid spammers. Several features were used to quantify the carefulness of user to classify the user to a spammer or a legitimate one these included clustering coefficients, PageRank and degrees . All Experiments were carried out on Weibo OSN, the results showed large difference over the genuine features from 0.1% to 2.8%, especially when they have tested on the link prediction measurement. Another study on Weibo [24] utilized Extreme Learning Machine classifier (ELM). It reported a very high spammer detection rate based on features such as Reposts, Followers, Messages, Messages containing URL were used to discriminate spammer from legitimate users. In the context of Twitter, feature set such as API ratio, Duplicate tweet count, URL ratio and a number of followers were used to detect spammers [25]. Results showed that the detection rate has increased with a percentage of over 10% after using the extracted features. An interesting honeypot-based approach for detecting spammer profiles in Twitter and MySpace has been proposed in [26]. The proposed social honeypot was used to harvest

potential spammers profiles which were analyzed to produce classification features. They managed to detect spammers with high precision and a low rate of false positives. Link farming in Twitter by which spammer acquire more followers in order to increase their tweets ranks has been investigated in [27]. To reduce the effect of this issue, the study proposed a ranking system termed as Collusion-Rank that penalizes the users for following unknown accounts which could be spammers profiles. In summary, there is a growing body of knowledge focusing on the issue of spam detection in the context of OSNs. Existing studies analyzed spammers behavior and proposed several data mining technologies to improve detection methods. However, with the fact that spammers keep developing their attack vectors and social engineering tricks, the door is still open for better feature engineering which would enhance detection methods. III.

METHODOLOGY

In this section, we present the proposed methodology for achieving the aim of the study. This methodology consists of five main processes as follows: 1)

2)

Analysis of profiles structure: In order to collect information, we have to analyze the related OSN. In this case, Twitter. We analyze what we are able to collect and what every profile has of public information. Feature engineering: According to the information we gathered from the analysis of the OSN, we were able to decide which feature is suitable to classify the profiles and which is not. In this work, the following ten features are proposed for identifying the spam profiles from legitimate ones: • Suspicious Words: Such as: Diet, Click Here, Health, Make Money, Give Me, Vote , Free, etc. Also, other similar terms in Arabic language are checked. • Default Image: If the profile still retains the same default image even after a while ( Three Months) [21], [27]. • Text to Links Ratio: High percentage of links (URLs) per tweet [26], [28], [29]. The threshold for this feature is set to (80%). • Following to Followers Ratio: A user’s following count is much higher than their followers count [29] (70%). • Repeated Words: High Percentage of duplicated words per tweet [30]. • Comments Ratio: A user has a really low count of replies to others [26], [29].(10%). • Tweet Time Pattern: Always tweets at the same time. [28], [31] • Different Description from Tweets : User’s description is different from their tweets’ general topic. • Different Following Interest from Tweets: A user is following people with different interests from what they generally tweet about [21]. 131


•

3)

4)

5)

6)

7)

Number of Tweets per Day : This number should theoretically not remain the same daily [24], [31]. Data collection: The data collection process is carried out on three stages; the first is to get the authentication key from the Twitter APIs by registering to Twitter Developer and using the OAuth endpoints, this step is necessary in order to communicate with twitter data directly. In the second step TwitterR and an R script are used to extract the exact profile data we need, whether they are spam or legitimate. Finally, save all the information in the CSV file for labeling. In this work, the features are extracted from 82 Twitter profiles who post in English and Arabic languages. A sample of the collected dataset is shown in Table I. Data labeling: Here we need to examine every profile data with these features and label it as Spam or Legitimate profile, where we mark them with (0) for legitimate and (1) for spam. Detection model development: in this stage, the features designed previously are used to develop the spam detection model. This is conducted by training different popular machine learning models like Naive Bayes, Decision Trees, Neural Networks and k-Nearest Neighbors on a part of the collected dataset. Then, the developed models are evaluated and tested on another unseen part of the dataset for final assessment. Evaluation and assessment: The developed detection models developed in the previous stage are assessed using different evaluation metric commonly used in the literature. Feature importance analysis: At this stage we conduct set of experiments to study the importance and influence of each feature on the accuracy of the spam profiles detection model. Two feature selection algorithms are used to rank the features according to some given weight by the algorithms.

Figure 1 depicts the stages of the methodology followed for the proposed detection system. IV.

C LASSIFICATION MODELS

A. Decision Trees C4.5 C4.5 is one of the decision tree family algorithms that generates trees from training data using information entropy. The decision tree C4.5 is an extension of the ID3 algorithm [32]. At each node, the algorithm select the attribute of the data by splitting the samples into a set of subsets using the information gain criteria, the highest attribute value will make the decision. Finally, the process is repeated on the smaller sublists.

does not need any model to fit therefore it is categorized as lazy learner, where the learner waits for the provided test dataset before doing any model generating. Lazy learners such as kNN make less work when a training dataset is provided, and extra work when making a numeric prediction or classification. C. Naive Bayes Naive Bayes is another simple algorithm based on the probability theory. The algorithm assumes that all features contribute independently to the output class. Naive Bayes does not use any explicit representation of the classifier. Although the algorithm is very simple with naive assumption, it showed high success in different real-world applications. D. Multilayer Perceptron (MLP) The Multilayer Perceptron is one of the most common types of artificial neural network. It can be defined as an information processing systems consist of a set of layers that map the inputs to a suitable set of outputs. Each layer has a number of simple processing elements called neurons. Neurons in each layer are fully connected with neurons in the next layer. Connections are represented as a set of weights. In MLP networks, every neuron has a nonlinear activation function except for the input nodes. The network is trained by updating the weights until the maximum number of iterations is reached or a certain level of error is reached. Conventionally, MLP networks use a gradient decent learning algorithm to train the network such as the popular Backpropagation algorithm. V.

F EATURE SELECTION METHODS

Feature selection is a technique to select the best subset of features to utilize them in the model construction process. In general, feature selection is performed by removing the irrelevant or redundant features without sustaining any loss of information while focusing on the relevant ones only. The goal of this process is to make the training time of the classification models shorter, improving the generalization ability of the model by reducing the overfitting and to help researchers and practitioners to simplify their models and make them easier to interpret [34]. A. Information Gain This is one of the most commonly used methods for feature selection. This method calculates the information gain for each input feature with respect to the output class [35]. The assigned IG value for each feature ranges from 0 which means no information up to 1 which represents maximum information. IG can be calculated as given in following equation, where C is the output class, Fi and H is the entropy. IG(C, Fi ) = H(C) − H(C|Fi )

(1)

B. k-Nearest Neighbors (k-NN)

B. ReliefF

k-NN is a very simple supervised learning algorithm. For classification tasks k-NN classifies a given instance based on the majority vote of its closest k instances [33]. Close instances can be determined using some distance measurement or function like the Euclidean, Minkowsky or mini- max. k-NN

This feature selection algorithm is an instance based method. It works frequently and randomly by sampling instances and then the neighboring instances of the same and different classes are checked. If the closest neighbour of the same class has different value for a given feature then the 132


TABLE I: Sample of the dataset. Name

Profile Profile Profile Profile Profile

1 2 3 4 5

Suspicious Words

Default Image

Text to Links Ratio

Following to Followers Ratio

Repeated Words

Tweet Time Pattern

Comments Ratio

Different Description from Tweets

F1 0 0 0 1 1

F2 0 0 0 0 0

F3 0 0 1 1 1

F4 0 1 1 0 1

F5 1 1 1 1 1

F6 0 1 0 0 0

F7 1 1 0 1 1

F8 0 0 0 1 0

Different Following Interest from Tweets F9 0 1 0 0 0

Number of Tweet per Day

F10 0 1 0 0 0

Type

Legitimate Spam Legitimate Spam Spam

Fig. 1: The methodology followed for developing the proposed system.

weight of this feature is decreased. On the other hand, if the closest neighbour of different class has different value the weight of the feature is increased [34]–[36]. VI.

E VALUATION RESULTS

In order to evaluate the spam detection models developed in this work, the confusion matrix will be referred to as a primary source for evaluation. The confusion matrix can be illustrated as shown in Figure 2.

P recision =

TP . TP + FP

(3)

Recall: the percentage of correctly classified spam profiles to the number of profiles originally categorized as spammers. Recall =

TP . TP + FN

(4)

F-measure: is measured as a weighted average of the precision and recall. F − measure =

2 × P recision × Recall . P recision + Recall

(5)

Area under ROC curve (AUC): AUC is widely used to evaluate probabilistic classifiers. A random classifier has an AUC equal to 0.5 while a perfect classifier has an AUC equals to 1. AUC can be given as shown Equation 6: Fig. 2: Confusion matrix

Accuracy: calculated by the total number of correctly classified instances of both classes over the total number of all instances in the dataset. Accuracyrate =

TP + TN . TP + FN + FP + TN

(2)

Precision: the ratio of profiles classified as spammers that actually are spammers.

AU C = VII.

1 0

TP FP 1 d = P N P ·N

1 0

T P dF P

(6)

E XPERIMENT AND RESULTS

The experiments in this work are carried out on two stages: first the basic classifiers including Decision Trees, k-NN, NB and MLP are applied then in the second stage the feature selection methods are used to reduce the set of features in the constructed dataset and select the most representative features. 133


All classifiers are applied using 10-folds cross validation with stratified sampling as a training/testing methodology. In this process the dataset is split into 10 parts so the training and testing process is repeated 10 times where in each time the classifier is trained using 9 parts and tested based on the 10th. Stratified sampling means that when the folds have the same class label distribution of the original dataset. This process is very important when the dataset is imbalanced by nature. The default parameters of classifiers are used as it was reported in previous studies that these values can lead to acceptable results. The evaluation metrics described in the previous section are used to evaluate the performance of the J48, k-NN, NB and MLP classifiers. Then the average and standard deviation are calculated over the 10 folds. The results of this stage are shown in Table II. According to the results all classifiers achieved high evaluation rates based on using the full set of proposed features, with a slight advantage for NB over the other classifiers. NB achieved an accuracy rate of 95.7%, a precision of 94% and a recall of 96%. After evaluating the basic classifiers, we apply the IG and ReliefF feature selection methods in order to identify the most influencing features on the spam identification process. Both methods assign a weight to each factor then the features are ranked in descending order according to their weights. To study the effect of the best ranked features on the performance of the classifiers we select the best 3, 5 and 7 features and train again J48 and NB using these sets. Table III lists the selected top 3, 5 and 7 features by IG and ReielfF, respectively. According to these results, it can be noticed that F1 and F5 (the suspicious words and the repeated words) are commonly selected among the top 3 features by both IG and ReliefF. In the top 5 sets, F2 and F4 (i.e. Default Image and Following to Followers Ratio) are commonly selected by both methods in addition to previous features. In the top 7, F7 and F10 (i.e. Comments ratio and Number of tweets per day) were commonly selected. Finally, all the selected sets in Table III are used to train J48 and NB classifiers for final evaluation. The results of the latter step are shown in Table IV. The results reveal that the best spam detection performance is obtained by using the top 7 features identified by ReliefF using the J48 decision tree classifier.

expected that developing spam detection models on huge data will be more challenging. Therefore, the efficiency of more scalable models will be investigated and applied. R EFERENCES [1] [2]

[3] [4]

[5] [6] [7]

[8]

[9]

[10] [11] [12] [13] [14]

[15]

[16]

[17]

VIII.

CONCLUSION

In this work, spam profiles detection models were developed based on a set of simple and publicly available features in Twitter. This set of features is extracted from the available profile information regardless of the language used in users’ tweets. Four machine learning algorithms are used to develop the detection models and two feature selection methods are applied to identify the most influencing features in the detection process. Preliminary results in the work show that promising results can be obtained using the Naive Bayes and Decision Trees classifiers. The results reveal also that suspicious words and the repeated words have a high influence on the accuracy of the detection process regardless of the language used in the tweets of the user. Since this work was a preliminary study, for future work, it is aimed to collect much larger dataset for different languages using the same methodology followed in this work. It is

[18]

[19]

[20]

[21] [22]

Statistic, Most famous social network sites — Global social media ranking 2016, 2016 (accessed November 15, 2016). J. Alqatawna, “An adaptive multimodal biometric framework for intrusion detection in online social networks,” IJCSNS International Journal of Computer Science and Network Security, vol. 15, no. 4, pp. 19–25, 2015. Statistic, Website Ranking, 2016 (accessed November 15, 2016). J. Alqatawna, A. Madain, A. Al-Zoubi, and R. Al-Sayyed, “Online social networks security: Threats, attacks and future directions,” in Social Media Shaping e-Publishing and Academia, Springer, 2017. R. Lieb, “Make spammers pay before you do,” 2002. J. J. Farmer, “3.4 specific types of spam,” FAQ). An FAQ for news. admin. net-abuse. email, 2003. J. Alqatawna, H. Faris, K. Jaradat, M. Al-Zewairi, and A. Omar, “Improving knowledge based spam detection methods: The effect of malicious related features in imbalance data distribution,” International Journal of Communications, Network and System Sciences, vol. 8, no. 5, pp. 118–129, 2015. R. A. Halaseh and J. Alqatawna, “Analyzing cybercrimes strategies: The case of phishing attack,” in 2016 Cybersecurity and Cyberforensics Conference (CCC), pp. 82–88, Aug 2016. F. Wu, J. Shu, Y. Huang, and Z. Yuan, “Co-detecting social spammers and spam messages in microblogging via exploiting social contexts,” Neurocomputing, 2016. Dont click that link! New Facebook spam features child porn and an account-controlling virus. R. Goolsby, L. Shanley, and A. Lovell, “On cybersecurity, crowdsourcing, and social cyber-attack,” tech. rep., DTIC Document, 2013. Kaspersky, Global IT Security Risks Survey 2015, 2015 (accessed November 15, 2016). Symantec, INTERNET SECURITY THREAT REPORT 2015, 2015 (accessed November 15, 2016). W. Herzallah, H. Faris, and O. Adwan, “Feature engineering for detecting spammers on twitter: modelling and analysis,” Journal of Information Science, vol. Accepted, 2017. J. Alqatawna, A. Hadi, M. Al-Zwairi, and M. Khader, “A preliminary analysis of drive-by email attacks in educational institutes,” in Cybersecurity and Cyberforensics Conference (CCC), pp. 65–69, IEEE, 2016. A. Zaid, J. Alqatawna, and A. Huneiti, “A proposed model for malicious spam detection in email systems of educational institutes,” in Cybersecurity and Cyberforensics Conference (CCC), pp. 60–64, IEEE, 2016. H. Faris, I. Aljarah, and J. Alqatawna, “Optimizing feedforward neural networks using krill herd algorithm for e-mail spam detection,” in Applied Electrical Engineering and Computing Technologies (AEECT), 2015 IEEE Jordan Conference on , vol., no., pp.1-5, pp. 1–5, IEEE, 2015. P. Heymann, G. Koutrika, and H. Garcia-Molina, “Fighting spam on social web sites: A survey of approaches and future challenges,” IEEE Internet Computing, vol. 11, no. 6, pp. 36–45, 2007. A. Beutel, W. Xu, V. Guruswami, C. Palow, and C. Faloutsos, “Copycatch: stopping group attacks by spotting lockstep behavior in social networks,” in Proceedings of the 22nd international conference on World Wide Web, pp. 119–130, ACM, 2013. F. Ahmed and M. Abulaish, “An mcl-based approach for spam profile detection in online social networks,” in 2012 IEEE 11th International Conference on Trust, Security and Privacy in Computing and Communications, pp. 602–608, IEEE, 2012. S. Adikari and K. Dutta, “Identifying fake profiles in linkedin.,” in PACIS, p. 278, 2014. C. Grier, K. Thomas, V. Paxson, and M. Zhang, “@ spam: the underground on 140 characters or less,” in Proceedings of the 17th ACM conference on Computer and communications security, pp. 27– 37, ACM, 2010.

134


TABLE II: Evaluation results of the basic classifiers. Classifier J48 K-NN=1 K-NN=3 K-NN=5 MLP NB

Accuracy + stdv 94.9 94.1 94.0 93.0 93.6 95.7

Precision + stdv

±7.91 ±8.83 ±8.58 ±8.83 ±8.29 ±7.18

0.95 0.95 0.95 0.94 0.95 0.94

±0.11 ±0.12 ±0.12 ±0.12 ±0.11 ±0.11

Recall + stdv 0.94 0.91 0.91 0.88 0.90 0.96

F measure + stdv

±0.13 ±0.17 ±0.15 ±0.17 ±0.17 ±0.11

0.93 0.92 0.92 0.90 0.91 0.95

±0.10 ±0.13 ±0.12 ±0.12 ±0.12 ±0.09

AUC + stdv 0.95 0.98 0.97 0.96 0.97 0.98

±0.08 ±0.06 ±0.07 ±0.07 ±0.07 ±0.05

TABLE III: Selected features using IG and ReliefF methods. Filter

Selected Features

InfoGainAttributeEval (Ranker = 3) ReliefFAttributeEval (Ranker = 3)

F1,F5,F2 F1,F5,F4

InfoGainAttributeEval (Ranker = 5) ReliefFAttributeEval (Ranker = 5)

F1,F5,F2,F4,F8 F1,F5,F4,F2,F9

ReliefFAttributeEval (Ranker = 7) InfoGainAttributeEval (Ranker = 7)

F1,F5,F4,F2,F9,F7,F10 F1,F5,F2,F4,F8.F10,F7

TABLE IV: Evaluation results of J48 and Naive Bayes classifiers after performing feature selection. Classifier InfoGain (Top 3) + J48 InfoGain (Top 5) + J48 InfoGain (Top 7) + J48 ReliefF (Top 3) + J48 ReliefF (Top 5) + J48 ReliefF (Top 7) + J48 InfoGain (Top 3) + NB InfoGain (Top 5) + NB InfoGain (Top 7) + NB ReliefF (Top 3) + NB ReliefF (Top 5) + NB ReliefF (Top 7) + NB

[23]

[24]

[25]

[26]

[27]

[28]

[29]

Accuracy + stdv 93.9 94.2 94.4 93.9 94.4 94.9 91.0 91.4 91.4 91.7 91.2 91.4

±8.33 ±8.32 ±8.17 ±8.33 ±8.31 ±7.91 ±9.09 ±8.85 ±8.85 ±8.94 ±8.99 ±8.85

Precision + stdv 0.95 0.95 0.95 0.95 0.95 0.95 0.91 0.92 0.92 0.93 0.91 0.92

±0.12 ±0.12 ±0.11 ±0.12 ±0.12 ±0.11 ±0.14 ±0.13 ±0.13 ±0.13 ±0.14 ±0.13

H. Fu, X. Xie, and Y. Rui, “Leveraging careful microblog users for spammer detection,” in Proceedings of the 24th International Conference on World Wide Web, pp. 419–429, ACM, 2015. X. Zheng, X. Zhang, Y. Yu, T. Kechadi, and C. Rong, “Elm-based spammer detection in social networks,” The Journal of Supercomputing, pp. 1–15, 2015. C. Yang, R. C. Harkreader, and G. Gu, “Die free or live hard? empirical evaluation and new design for fighting evolving twitter spammers,” in International Workshop on Recent Advances in Intrusion Detection, pp. 318–337, Springer, 2011. K. Lee, J. Caverlee, and S. Webb, “Uncovering social spammers: social honeypots+ machine learning,” in Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval, pp. 435–442, ACM, 2010. S. Ghosh, B. Viswanath, F. Kooti, N. K. Sharma, G. Korlam, F. Benevenuto, N. Ganguly, and K. P. Gummadi, “Understanding and combating link farming in the twitter social network,” in Proceedings of the 21st international conference on World Wide Web, pp. 61–70, ACM, 2012. G. Stringhini, C. Kruegel, and G. Vigna, “Detecting spammers on social networks,” in Proceedings of the 26th Annual Computer Security Applications Conference, pp. 1–9, ACM, 2010. A. H. Wang, “Don’t follow me: Spam detection in twitter,” in Security

Recall + stdv 0.91 0.92 0.92 0.91 0.92 0.94 0.88 0.88 0.88 0.88 0.88 0.88

±0.14 ±0.14 ±0.14 ±0.14 ±0.14 ±0.13 ±0.16 ±0.16 ±0.16 ±0.16 ±0.16 ±0.16

F measure + Std 0.92 0.92 0.93 0.92 0.93 0.93 0.88 0.89 0.89 0.89 0.88 0.89

±0.11 ±0.11 ±0.11 ±0.11 ±0.11 ±0.11 ±0.12 ±0.12 ±0.12 ±0.12 ±0.12 ±0.12

AUC + stdv 0.94 0.94 0.94 0.94 0.94 0.95 0.96 0.96 0.96 0.95 0.96 0.96

±0.09 ±0.09 ±0.09 ±0.09 ±0.09 ±0.08 ±0.07 ±0.08 ±0.08 ±0.08 ±0.08 ±0.08

and Cryptography (SECRYPT), Proceedings of the 2010 International Conference on, pp. 1–10, IEEE, 2010. [30] A. Java, X. Song, T. Finin, and B. Tseng, “Why we twitter: understanding microblogging usage and communities,” in Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 workshop on Web mining and social network analysis, pp. 56–65, ACM, 2007. [31] B. Krishnamurthy, P. Gill, and M. Arlitt, “A few chirps about twitter,” in Proceedings of the first workshop on Online social networks, pp. 19–24, ACM, 2008. [32] J. R. Quinlan, C4. 5: programs for machine learning. Elsevier, 2014. [33]

D. W. Aha, D. Kibler, and M. K. Albert, “Instance-based learning algorithms,” Machine learning, vol. 6, no. 1, pp. 37–66, 1991. [34] P. A. Castillo, A. M. Mora, H. Faris, J. Merelo, P. Garca-Snchez, A. J. Fernndez-Ares, P. D. las Cuevas, and M. I. Garca-Arenas, “Applying computational intelligence methods for predicting the sales of newly published books in a real editorial business management environment,” Knowledge-Based Systems, vol. 115, pp. 133 – 151, 2017. [35] I. H. Witten and E. Frank, Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann, 2005. [36] I. Kononenko, “Estimating attributes: analysis and extensions of relief,” in European conference on machine learning, pp. 171–182, Springer, 1994.

135