LNCS 8281 - Web Spam Detection: New Approach ... - NUS Computing

Web Spam Detection: New Approach with Hidden Markov Models Ali Asghar Torabi1, Kaveh Taghipour2,*, and Shahram Khadivi1 1

Human Language Technology Lab, Department of Computer Engineering and IT, Amirkabir University of Technology {a.torabi,khadivi}@aut.ac.ir 2 Department of Computer Science, National University of Singapore, 13 Computing Drive, Singapore 117417 [email protected]

Abstract. Web Spam is the result of a number of methods to deceive search engine algorithms so as to obtain higher ranks in the search results. Advanced spammers use keyword and link stuffing methods to create farms of spam pages. Most of the recent works in the web spam detection literature utilize graph based methods to enhance the accuracy of this task. This paper is basically a probabilistic approach that uses content and link based features to detect the web spam pages. Since we observe there is a high connectivity between web spam pages, we adopt a method based on Hidden Markov Model to exploit conditional dependency of a sequence of hosts and their spam/normal class distribution of each host. Experimental results show that the proposed method can significantly improve the performance of baseline classifier. Keywords: web spam, link spam, hidden Markov models, ant colony optimization.

1

Introduction

Given the vast amount of information on the web, users have to use search engines to locate the useful web pages that are relevant to their interests and inquiries. The goal of search engines as the main information retrieval machines for web is to provide higher ranks for pages that are most important and relevant to the users’ query. Therefore, search engines need to distinguish the normal web pages from Spam pages so as to prevent misleading of the users [1]. In order to find desired contents, Search engines use specific textual similarity measures for determining the relevancy of a page and a query. To measure the importance of the pages there are several global query-independent indicators like Page Rank [2] that often are calculated from web link structure [3]. While these two important criteria are used in search engines for evaluating web pages, a new industry of Search Engine Optimizers has developed (SEO) recently. Malaga and Ross [4] *

This research was conducted at the time Taghipour was in AmirKabir University of Tehran.

R.E. Banchs et al. (Eds.): AIRS 2013, LNCS 8281, pp. 239–250, 2013. © Springer-Verlag Berlin Heidelberg 2013

240

A.A. Torabi, K. Taghipour, and S. Khadivi

grouped the SEO methods into two categories: White hat SEOs that stay within the guidelines by search engines and Web Spams (also known as Black hat SEOs) that violate the rules and transgress accepted norms. Web Spamming means boosting the rates of web pages undeservedly, without improving the true value of a page. The web Spamming methods cause crucial problems for search engines, e.g., they tremendously waste the resources for indexing illegitimate web pages, unduly decrease the quality of retrieval process, damage search engines [5]. According to [3] Web Spam techniques can be categorized to the following types: • Content Spam: If Spammers target textual similarity measures it is a content spam generation method. The content of pages is filled by popular words so that they are relevant to more popular users’ queries. In [6] the term “keyword-stuffing” is used to refer to this method. • Link Spam: There is a general belief that pages with more incoming links are more popular and important than others. As mentioned, Search engines use some linkbased measures like Page Rank to assess importance of a web page. Spamming methods that intend to influence these algorithms are named Link Spam. Spammers create so many pages that link to the target page to increase its popularity. Extensive researches have been presented to reduce the impact of Web Spam. Most of the proposed solutions such as [6, 7] considered Web Spam detection as a classification problem. This research considers hosts as train/test instances and features are extracted from content of pages within the host and links among them. Previous experiences show that in Web Spam Detection, instances are not independent and data labels are unbalanced [8]. In this paper we present a new approach to handle biasness of data and model the dependency between hosts. To our knowledge this is the first time that Hidden Markov Model (HMM) is used to do this. The proposed system starts by building a classifier based on Aggregating One Dependence Estimators (AODE) [9]. Hidden Markov Model helps us to consider the dependency of web hosts during the prediction process and boost the performance of AODE. A simple method to adopt HMM for this task is to find the most frequent sequences of visiting hosts. In the proposed system, Ant Colony Optimization algorithm is used to generate the required sequences of hosts. The paper is organized as follows. In Section 2, we provide an overview of previous works. In Section 3, feature selection and classification method are described. In Section 4, we propose a method that extracts sequences of hosts that we need to apply HMM on these sequences. Finally, we conclude by summarizing key principals of our approach.

2

Related Work

Several automatic techniques for web spam detection have been presented in the literature. Fetterly et al. [10] justified the statistical difference between machine generated spam pages and normal web pages. They presented some features based on page content, linkage structure and page evolution. In their next following paper [6] they

Web Spam Detection: New Approach with HMM

241

also proposed several content based features and a decision tree, classifying Spam and Normal pages. Piskorski et al [11] studied on some linguistic features and discovered several discriminative features that are available publicly for others. Moreover in [12] Araujo et al, They offered a new approach rooted in combination of link based features and language model based ones. They observed the semantic relation between linked pages and found them to be useful to improve the performance of the classification task. In addition to traditional learning models many papers used graph based methods to boost the performance of Web Spam Detection by considering topological dependency between the hosts. Link propagation as one the most popular methods in graph based problems has been widely used in web spam detection. Becchetti et al. [13] performed a statistical analysis on link structure of web pages in a large collection. Their experiments show that link based metrics like TrustRank and Truncated PageRank can improve the performance of Web Spam classifiers. TrustRank separates an initial set of good pages. It starts with a seed of good pages and then follows the link structure to propagate the rank thorough the related pages. Implementation of this method is described in [14]. Truncated PageRank is a version of PageRank that ignores the direct contribution of near neighbors according to a damping function. Experiments by Becchetti and others in [13, 15] show that Truncated PageRank is a discriminative feature. Castillo et al [5] proposed stacked graphical learning for propagating labels across the web graph. In addition to content and link based features the average of probability of Spam for neighboring hosts is added to the feature vector of each host and thus is considered in decision process. Link refinement, elimination and regularization methods are other methodologies that exploit link structure to improve performance of basic classifiers. Elimination of Nepotistic links is one of the proposed method to reduce the impact of the link stuffing by removing certain links of the web graph [16]. Jacob Abernethy et al in [17] presented a graph regularization based algorithm, WITCH, that learns simultaneously from graph structure, content and link based features. There are also other works and experiences in this regard. For additional studies on the above mentioned topics, you can refer to the Survey on Web Spam Detection by Nikita Spirin and Jiawei Ha [8] , that is a good survey covering many papers and proposed systems to date.

3

Classification and Feature Selection

The following paper has tested and trained the proposed method by WebSpamUk2006 [18] which is a public Web Spam dataset. This collection contains 11402 hosts from the .uk domain. For each host 263 features have been extracted from links and content of pages. Additional information about feature types and list of them is available at [6, 13]. In this dataset, 7473 hosts are labeled by a group of volunteers into three categories of Spam, Normal and Borderline. Here, we use the first two categories to build a model that recognizes spam hosts from normal ones.

242

A.A. Torabi, K. Taghipour, and S. Khadivi

In this research several classifiers such as decision trees, neural networks and statistical classifiers were examined and compared against each other. We use F-measure value as a criterion to compare efficiency of different classification methods. The results showed that AODE was superior to other competitive methods. AODE is a statistical classifier that achieves higher accuracy by averaging over all naïve Bayes models. The core of above mentioned classifier is to consider weaker independency assumption for Naïve Bayes. Thus it has less error rate and still is as fast as possible in training and test phases. In comparison with other methods like Bayesian networks, AODE benefits from the advantage of not performing model selection while it’s accuracy is comparable with none parametric models like decision trees and neural networks [9]. The proposal has taken advantage of AODE by using its implementation in Weka [19]. A subset of two sorts of link based and content based features have been used in this paper. The paper approach is to use Wrapper Feature Selection method with which evaluates features by using a learning model [20] and chooses the most discriminative and relevant features. After feature selection process, only 22 of 263 features have been selected. By using cross validation and a bidirectional search an optimal feature subset has been found that results the best accuracy for AODE classifier. Table 1 shows the performance for different feature selection models and AODE classifier. In this paper "true positive" and "false positive" rates are respectively about "the rate of correctly detected spam hosts" and "normal hosts that are detected as spam incorrectly". Tables 1 and 3-5 are result of cross validation with 10 folds. Table 1. Comparision of feature selection methods

True positive rate False positive rate F-Measure

Correlation Attribute Evaluation

Principal Component Analysis

Wrapper feature selection

74% 9% 0.72

77.2% 10.4% 0.724

81.2% 4.9% 0.825

Reported results show that Wrapper feature selection improves the result by increasing true positive and reducing false positive ratio which has resulted in the selection of AODE and the new feature space to setup the proposed system.

4

Smoothing

In this Section, the research proposes a method to detect Web spam by using topological dependency between hosts. It started by representing the web spam detection as a graph based problem. In this presentation each host is a node in graph , of Web hosts and links . For each pair of nodes and we have , the number of links from host to . Most of the traditional classifiers presuppose that instances are independent and identically distributed. But in web spam problem, samples are topologically dependent [21, 22]. Therefore lots of latent information in the link structure between hosts

Web Spam Detection: New Approach with HMM

243

are missed if we confine ourselves to use only the base classifier in the previous Section. The proposed system is based on smoothness assumption of semi-supervised learning. According to this, nodes which are closer to each other are more likely to share a label [23]. In web graph for each pair of nodes the paper declared a closeness factor: (1) log 1 , On the other hand Castillo et al [5] showed in their experiments that normal nodes tend to be linked by very few Spam nodes and they mainly link to other normal nodes while Spam nodes are mainly linked to Spam nodes. In Table 2 the results of our study on WebSpam-Uk2006 dataset are shown. Probability of transition from a spam host to a normal host is 23% and that is much lower than transition to other spam host. Table 2. Dependency of Spam and Normal classes

Spam 77% 13%

Spam Normal

Normal 23% 87%

Considering the conditional dependency of Spam and Normal hosts to each other and topological dependency of hosts in the web graph, and also hidden category of each host; it is intuitively obvious that Hidden Markov Model would be a useful learning schema to build a pattern of dependency between nodes and handle imbalance between spam and normal labels. HMM is a probabilistic method to model sequence of data [24]. Indeed, to take HMM into use we need a sequence of connected hosts. This paper proposes Ant colony optimization algorithm to extract sequences of related hosts according to similarity measure (1). Fig. 1 presents the workflow of the proposed system.

Host Webgraph

ACO

Sequences for Baum Welch

Generating sequences for HMM

HMM

HMM parameters

Training on sequences

Train

Host Webgraph

ACO Target Host

Generating sequences for HMM

Sequences for forward algorithm

HMM Calculating probability of Spam and Normal on sequences

Test

Fig. 1. Web spam detection workflow

Average probability of Spam and Normal

244

4.1

A.A. Torabi, K. Taghipour, and S. Khadivi

Ant Colony Optimization

In computer science, Ant Colony Optimization refers to a general propose method of finding the best solution of an optimization problem. In ACO, artificial ants build solutions and exchange information by depositing pheromone on the ground in order to remark favorable path to the optimization problem [25]. To use ACO it is needed to represent the problem space as a graph and declare three fundamental components: • Edge selection: Artificial ants move from vertex to vertex along the edges of the graph. A stochastic mechanism is hired to guide each ant to choose edge , in each step of walk. This mechanism uses a probability distribution based on heuristic function and pheromone values . The probability function is defined as: ,

∑ 0

(2)

and are "the pheromone value on edge , " and "heuristic In equation (2) function", respectively. represents a set of the neighboring hosts pointing to host that has not yet visited by ant . • Heuristic function: The heuristic function has been defined using the assumptions we mentioned at the beginning of this Section. Equation (3) illustrates heuristic function that is the same as similarity measure in equation (1). Therefore an ant that is in the web host chooses an edge , that has more input links than others. 1

,

(3)

• Pheromone update: According to [25] each artificial ant should update pheromone on edge , after each step of walk to communicate with other ants. These pheromone's updates incrementally specify the best paths of connected hosts. 1

1

, ∑

,

(4)

This study combines offline and local pheromone updates [25] in one formula. Equation (4) explains the defined pheromone's update function. represents the value of pheromone on edge , in iteration and , is the number of links from to . Real number 0 1 is a decay coefficient. According to the proposed equation, amount of pheromone on each edge decreases over time. Higher

Web Spam Detection: New Approach with HMM

245

value of gives a greater chance to other paths to be selected by edge selection method in the next following iterations and as a result we will have more paths of connected hosts. 4.2

Hidden Markov Model

HMM is a stochastic extension of Markov Model with hidden states. In this model, states are not visible but probability of states and transition between them are given by state dependent functions [24, 25]. This paper defined two states Spam and Normal. The visible output and emission probability functions are respectively the 22 dimensional feature vectors and AODE model that was presented in Section 3. Since AODE is a probabilistic model [9] that here predicts posterior class probabilities, this model is appropriate to be used as emission probability of HMM.

Fig. 2. Sequences of hosts to host t

In training phase, Baum Welch algorithm is run on generated sequences from ACO to estimate the transition matrix and the initial probabilities . All HMM parameters are recalculated in maximization step of Baum Welch except emission probabilities that have already been estimated using an AODE classifier. In test phase, the label of host will be predicted by the proposed system. It first extracts sequences of hosts that are linked to the by ACO. Fig. 2 illustrates an example of host sequences with length three that are linked to the target web host. The forward algorithm, Equation (5) is used to calculate probability of normal and spam hosts according to each sequence. |

,

:

,

,

(5)

:

,

| ,

Z |Z

Z

,X

:

246

A.A. Torabi, K. Taghipour, and S. Khadivi

| spam is the probability of observing feature vector in the state spam. Z ,X refers to the initial probabilities of spam and normal and Z |Z is the transition probability distribution. Four possible transitions are as follows: ─ ─ ─ ─ Finally to predict the label of target host, average probabilities of spam| : and nomal| : over all sequences are used. In Table 3, the performance of the new classification method is shown. Table 3. Smoothing by one HMM

True positive rate False positive rate F-Measure 4.3

AODE

HMM

81.2% 4.9% 0.825

81.8% 4.3% 0.836

Multiple HMMs

So far, we only use the output of ACO algorithm in the system, i.e., we only use the best sequences to train a single HMM. In this Section, we use the values of pheromones to better estimate the HMM parameters. We assume the pheromone values of each edge as a measure of conditional dependency between two hosts. Here, we introduce a technique for label smoothing by using multiple HMMs. In Fig. 3, the results of our experiments on .uk 2006 dataset are shown. In these experiments, we first use ACO with 100 artificial ants and then we extract sequences with length 2. Then a discretization with 10 equal depth bins is performed on pheromone values. Afterwards, we train a Hidden Markov Model for each bin, so we have ten different HMMs. Prior probability of spam Z spam and transition probability Z spam|Z spam for each bin are presented in Fig. 3. According to the reported parameters, the label of destination point is conditionally more dependent on the source point when there is more pheromone on the edge between them. Furthermore, it illustrates that probability of spam has an inverse relation with amount of pheromone. The result of the above experiment is convincing enough to make use of different HMM components to model relation between points with different dependency values (Pheromones). For sequences with length two implementation of such a system with non-parametric models is straightforward. But for sequences with length of three or higher we should present a technique that considers pheromones in edges in depth two or higher in addition to pheromones in first step of sequences. Please note that since we aim to use non-parametric models in HMM, we need to discretize the edge values so as to decrease the amount of sparsity.

Web Spam Detection: New Approach with HMM

247

1 Probability

0.8 0.6

P(X1->Spam|X2->Spam)

0.4

Probability of Spam

0.2 3.09-inf

0.2

0.28

0.12

0.1072

0.1033

0.1015

0.1006

0.1002

0.05

0

Fig. 3. Dependency between HMM parameters and pheromone value

In this paper two approaches were examined for sequences with length three. In the first one, a weight for each sequence of two edges was defined by multiplication of pheromones of the edges. Afterwards the binning was performed on these weights. In the second approach, the binning is applied two times. First on the edges connected to the target host, and second time on the edges connected to the neighbors of the target host. The binning algorithm introduces ten bins in each run. Therefore, a 10 10 table of 100 HMM component was created. Each sequence was assigned to one of the HMM components according to the amount of pheromone on their edges. For example if the first edge of sequences belonged to bin 3 and second edge belonged to bin 6 the system assigned the sequence to the HMM number 18 in row 3 and column 6 of the pheromone table. The train and test phases are the same as before; unless spam| : and nomal| : should be computed according to the appropriate HMM component. Table 4 shows the result of the experiments. Table 4. Camparison of proposed approaches

True positive rate False positive rate F-Measure

HMM

Multiple HMMs of order 2

Multiple HMMs of order 3 first approach

Multiple HMMs of order 3 second approach

81.8% 4.3% 0.836

88.1% 6.5% 0.843

87.6% 6.7% 0.838

91.7% 6.9% 0.859

As you can see in Table 4, the proposed system achieves an improvement by using several HMM components on sequences with length 2. The performance of system but then is reduced when first approach was used to model sequences with length 3. However it is raised again when the second approach is used. According to our experiments the general trend of using this method is a considerable increase of 10 percent in detection rate of baseline classifier while F-measure has also improved from 0.825 to 0.859. Next Section is a comparison of the result of this study and other existing methods to show to what extent the application of HMMs is contributed to the improvement for Web spam detection.

248

5

A.A. Torabi, K. Taghipour, and S. Khadivi

Conclusion

For many applications like Web spam detection the i.i.d assumption would fail to exploit dependency patterns between data points. This study proposed a system to detect web spam according to the content and link based features and dependency between points in web host graph. To our knowledge this is the first attempt to boost the performance of web spam detection using Hidden Markov Models. Table 5 shows a comparison between the presented method and other systems according to the FMeasure value. Experimental results show that the proposed method is effective and yields better performance in comparison with other works on the same feature set. Geng et al in [26] boosted the performance of classification task using under sampling method and reached F-measure 0.759. Castillo et al [5] as one of the most significant studies on the web spam detection reports F-Measure 0.763 using stacked graphical learning. Benczúr et al [27] reported F-measure 0.738 following the same methodology as Castillo et al [5]. To compare the performance of the proposed system with the results of the participants in web spam challenge [28] The study also evaluated the proposed method on the test set provided by the organization. Table 5. Comparing performance of systems Web Spam Detection system Our Proposed system Castillo et al Geng et al Benczúr et al Filoche et al Abou et al Fetterly et al Cormack

F-Measure Test Set

Cross Validation

0.90 NA 0.87 0.91 0.88 0.81 0.79 0.67

0.85 0.76 0.75 0.73 NA NA NA NA

One disadvantage of the proposal system is the number of the needed HMMs in the second approach. For instance using the second approach needs to create 1000 HMM models for sequences of length 4; which is a proof that it is time consuming to estimate parameters of these HMMs. In near future, we plan to propose a HMM with parametric transition probabilities that can handle the weights of the edges. Moreover we intend to employ a new content based feature using language modeling techniques. Based on the ongoing researches and studies on the topic we strongly believe that it is possible to achieve better performance using these new features.

References 1. Henzinger, M.R., Motwani, R., Silverstein, C.: Challenges in web search engines. In: ACM SIGIR Forum, pp. 11–22. ACM (2002) 2. Bianchini, M., Gori, M., Scarselli, F.: Inside pagerank. ACM Transactions on Internet Technology (TOIT) 5, 92–128 (2005)

Web Spam Detection: New Approach with HMM

249

3. Gyongyi, Z., Garcia-Molina, H.: Web spam taxonomy. In: First International Workshop on Adversarial Information Retrieval on the Web, AIRWeb 2005 (2005) 4. Malaga, R.A.: Search Engine Optimization—Black and White Hat Approaches. Advances in Computers 78, 1–39 (2010) 5. Castillo, C., Donato, D., Gionis, A., Murdock, V., Silvestri, F.: Know your neighbors: Web spam detection using the web topology. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 423– 430. ACM (2007) 6. Ntoulas, A., Najork, M., Manasse, M., Fetterly, D.: Detecting spam web pages through content analysis. In: Proceedings of the 15th International Conference on World Wide Web, pp. 83–92. ACM (2006) 7. Mahmoudi, M., Yari, A., Khadivi, S.: Web spam detection based on discriminative content and link features. In: 2010 5th International Symposium on Telecommunications (IST), pp. 542–546. IEEE (2010) 8. Spirin, N., Han, J.: Survey on web spam detection: principles and algorithms. ACM SIGKDD Explorations Newsletter 13, 50–64 (2012) 9. Webb, G.I., Boughton, J.R., Wang, Z.: Not so naive Bayes: aggregating one-dependence estimators. Machine Learning 58, 5–24 (2005) 10. Fetterly, D., Manasse, M., Najork, M.: Spam, damn spam, and statistics: Using statistical analysis to locate spam web pages. In: Proceedings of the 7th International Workshop on the Web and Databases: Colocated with ACM SIGMOD/PODS 2004, pp. 1–6. ACM (2004) 11. Piskorski, J., Sydow, M., Weiss, D.: Exploring linguistic features for Web spam detection: A preliminary study. In: Proceedings of the 4th International Workshop on Adversarial Information Retrieval on the Web, pp. 25–28. ACM (2008) 12. Araujo, L., Martinez-Romo, J.: Web spam detection: new classification features based on qualified link analysis and language models. IEEE Transactions on Information Forensics and Security 5, 581–590 (2010) 13. Becchetti, L., Castillo, C., Donato, D., Leonardi, S., Baeza-Yates, R.: Link-based characterization and detection of web spam. In: 2nd Intl. Workshop on Adversarial Information Retrieval on the Web (AIRWeb), pp. 1–8 (2006) 14. Gyöngyi, Z., Garcia-Molina, H., Pedersen, J.: Combating web spam with trustrank. In: Proceedings of the Thirtieth international conference on Very Large Data Bases, vol. 30, pp. 576–587. VLDB Endowment (2004) 15. Becchetti, L., Castillo, C., Donato, D., Leonardi, S., Baeza-Yates, R.: Using rank propagation and probabilistic counting for link-based spam detection. In: Proc. of WebKDD (2006) 16. Davison, B.D.: Recognizing nepotistic links on the web. Artificial Intelligence for Web Search, 23–28 (2000) 17. Abernethy, J., Chapelle, O., Castillo, C.: Web spam identification through content and hyperlinks. In: Proceedings of the 4th International Workshop on Adversarial Information Retrieval on the Web, pp. 41–44. ACM (2008) 18. Yahoo! Research: Web Spam Collections, http://barcelona.research.yahoo.net/webspam/datasets/ Crawled by the Laboratory of Web Algorithmics, University of Milan, http://law.dsi.unimi.it/ (retrieved August 8, 2012) 19. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. Newsl. 11, 10–18 (2009)

250

A.A. Torabi, K. Taghipour, and S. Khadivi

20. Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artificial intelligence 97, 273–324 (1997) 21. Menczer, F.: Mapping the semantics of Web text and links. IEEE Internet Computing 9, 27–36 (2005) 22. Chakrabarti, S., Joshi, M.M., Punera, K., Pennock, D.M.: The structure of broad topics on the web. In: Proceedings of the 11th International Conference on World Wide Web, pp. 251–262. ACM (2002) 23. Chapelle, O., Schölkopf, B., Zien, A.: Semi-supervised learning. MIT Press (2006) 24. Rabiner, L., Juang, B.: An introduction to hidden Markov models. IEEE ASSP Magazine 3, 4–16 (1986) 25. Dorigo, M., Birattari, M., Stutzle, T.: Ant colony optimization. IEEE Computational Intelligence Magazine 1, 28–39 (2006) 26. Geng, G.-G., Wang, C.-H., Li, Q.-D., Xu, L., Jin, X.-B.: Boosting the performance of web spam detection with ensemble under-sampling classification. In: Fourth International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2007, pp. 583–587. IEEE (2007) 27. Benczúr, A., Bíró, I., Csalogány, K., Sarlós, T.: Web spam detection via commercial intent analysis. In: Proceedings of the 3rd International Workshop on Adversarial Information Retrieval on the Web, pp. 89–92. ACM (2007) 28. Web Spam Challenge (2007), http://webspam.lip6.fr/