Detecting Network Anomalies Using CUSUM and EM Clustering Wei Lu1,2 and Hengjian Tong3 1
Faculty of Computer Science, University of New Brunswick, Fredericton, NB E3B 5A3, Canada 2 Department of Electrical and Computer Engineering, University of Victoria, Victoria, BC V8W 3P6, Canada 3 Department of Computing Science and Technology, School of Computer Science China University of Geosciences, Wuhan, Hubei, 430074, P.R. China
[email protected]
Abstract. Intrusion detection has been extensively studied in the last two decades. However, most existing intrusion detection techniques detect limited number of attack types and report a huge number of false alarms. The hybrid approach has been proposed recently to improve the performance of intrusion detection systems (IDSs). A big challenge for constructing such a multi-sensor based IDS is how to make accurate inferences that minimize the number of false alerts and maximize the detection accuracy, thus releasing the security operator from the burden of high volume of conflicting event reports. We address this issue and propose a hybrid framework to achieve an optimal performance for detecting network traffic anomalies. In particular, we apply SNORT as the signature based intrusion detector and the other two anomaly detection methods, namely non-parametric CUmulative SUM (CUSUM) and EM based clustering, as the anomaly detector. The experimental evaluation with the 1999 DARPA intrusion detection evaluation dataset shows that our approach successfully detects a large portion of the attacks missed by SNORT while also reducing the false alarm rate. Keywords: Intrusion detection, clustering.
1 Introduction Intrusion detection has been extensively studied since the seminal work by Anderson [1]. Traditionally, intrusion detection techniques are classified into two categories: misuse (signature-based) detection and anomaly detection. Misuse detection is based on the assumption that most attacks leave a set of signatures in the stream of network packets or in audit trails, and thus attacks are detectable if these signatures can be identified by analyzing the audit trails or network traffic behaviors. However, misuse detection is strictly limited to the known attacks. Detecting new attacks is one of the biggest challenges faced by misuse detection. To address the weakness of misuse detection, the concept of anomaly detection was formalized in the seminal report of Denning [2]. Denning assumed that security Z. Cai et al. (Eds.): ISICA 2009, LNCS 5821, pp. 297–308, 2009. © Springer-Verlag Berlin Heidelberg 2009
298
W. Lu and H. Tong
violations could be detected by inspecting abnormal system usage patterns from the audit data. As a result, most anomaly detection techniques attempt to establish normal activity profiles by computing various metrics, and an intrusion is detected when the actual system behavior deviates from the normal profiles. According to Axelsson [3], the early network anomaly detection systems are selflearning, that is, they automatically form an opinion of what the subject’s normal behavior is. Self learning techniques combine the early statistical model based anomaly detection approaches, the AI based approaches, the biological models based approaches, and thus they still applied for current anomaly detection schemes. According to whether they are based on supervised or unsupervised learning techniques, anomaly detection schemes can be classified into supervised and unsupervised. Supervised anomaly detection establishes the normal profiles of systems or networks through training based on labeled datasets. In contrast, unsupervised anomaly detection attempts to detect intrusions without using any prior knowledge of attacks or normal instances. Although learning techniques achieve good results on network anomaly detection so far, they still face some challenges, such as "can machine learning be secure?" [4], "would behavioral non-similarity in training and testing data totally fail leaning algorithms on anomaly detection?" [5], and "what are the limitations for detecting previously unknown attacks due to large number of false alarms?" [6]. In order to overcome these challenges and keep the advantages of misuse detection, some researchers have proposed the idea of hybrid detection. There are currently two ways to achieve this goal, one is sequence based and the other is parallel based. Sequence based hybrid IDSs apply anomaly detection (or misuse detection) first and misuse detection (or anomaly detection) second [7-15]. Combing the advantages of both misuse and anomaly detection, hybrid IDSs achieve a better performance. However, the sequence based approaches might not provide a full coverage for the attack types due to the filtering of malicious (normal) traffic and also the sequence process will prolong the detection and make a real-time detection impossible. In contrast, parallel based hybrid IDSs apply multiple detectors in parallel and make an intrusion decision based on multiple output sources, which provide a wide coverage for intrusions and has the potential to detect previously unknown attacks [16-20]. One of the biggest challenges for parallel based IDSs is how to make accurate inferences that minimize the number of false alarms and maximize the detection accuracy. In order to address the aforementioned issues, we propose in this paper a hybrid framework, combining the well-known SNORT signature based IDS and the other two anomaly detectors, namely non-parametric CUmulative SUM (CUSUM) and EM based clustering, As illustrated in Figure 1, the general architecture of our detection scheme consists of three major components, namely SNORT, feature analysis and CUSUM-EM detector. During feature analysis, we define and generate fifteen features to characterize the network traffic behavior, in which we expect the more the number of features, the more accurate the entire network will be characterized. These proposed features are then input to the CUSUM-EM detector, in which CUSUM based detector and EM clustering based detector are fused according to a set of scoring coefficients. The final intrusion decision is given through a fuzzy attacking probability output by the inference model.
Detecting Network Anomalies Using CUSUM and EM Clustering Raw Packets
SNORT
Normal
299
Feature Analysis Features based on flows CUSUM
Alerts by SNORT
EM
CUSUM-EM based IDS Alerts generated based on attacking probabilities
Fig. 1. General architecture of the detection framework
The rest of the paper is organized as follows. Section 2 introduces related work, in which we summarize some existing works on hybrid IDSs. Section 3 introduces the fifteen flow-based features and explains the reasons to select them. Section 4 overview the two existing anomaly detection approaches, namely CUSUM algorithm and the Expectation-Maximization (EM) based clustering algorithm. Section 5 presents the complete network anomalies analysis for the 1999 DARPA intrusion detection evaluation dataset by using our intrusion inference model. Section 6 makes some concluding remarks and discusses future work.
2 Related Work Early research work on IDS suggest that intrusion detection capabilities can be improved through a hybrid approach consisting of both signature (misuse) detection as well as anomaly detection [8,9,10]. In such a hybrid system, the signature detection technique detects known attacks and the anomaly detection technique detects novel or unknown attacks. Typical recent research work for such a hybrid intrusion detection system are discussed in [7,11,12,13,14,15]. In [11], Tombini et al. applied an anomaly detection approach to create a list of suspicious items. Then a signature detection approach is employed to classify these suspicious items into three categories, namely false alarms, attacks, and unknown attacks. The approach is based on an assumption that a high detection rate can be achieved by the anomaly detection component because missed intrusions in the first step will not be found by the follow-up signature detection component. Zhang et al. proposed a hybrid IDS combining both misuse detection and anomaly detection components, in which a random forest algorithm was applied firstly in the misuse detection module to detect known intrusions [12]. The outlier detection provided by the random forest algorithm is then utilized to detect unknown intrusions. Evaluations with a part of the KDDCUP’99 dataset showed that their misuse detection module generated a high detection rate with a low false positive rate and at the same time the anomaly detection component has the potential to find novel intrusions. In [13], Peng et al. proposed a two-stage hybrid intrusion detection and visualization system that leverages the advantages of signature-based and anomaly detection methods. It was claimed that their hybrid system could identify both known and unknown attacks on system calls. However, evaluation results for their system were
300
W. Lu and H. Tong
missed in the paper. The work is more like an introduction on how to apply a multiple stage intrusion detection mechanism for improving the detection capability of an IDS. Similar with [12], Depren et al. proposed a novel hybrid IDS system consisting of an anomaly detection module, a misuse detection module and a decision support system [14]. The decision support system was used to combine the results of previous two detection modules. In the anomaly detection module, a Self-Organizing Map (SOM) structure was employed to model normal behavior and any deviation from the normal behavior would be classified as an attack. In the misuse detection module, a decision tree algorithm was used to classify various types of attacks. The final system was evaluated with the 1999 KDDCUP intrusion detection dataset and experimental results showed that the proposed hybrid approach gave better performance over individual approaches. Based on the idea of combining the advantages of low false positive rate of signature based IDS and the ability of anomaly detection system (ADS) for detecting new or unknown attacks, Hwang et al. proposed and reported a new experimental hybrid intrusion detection system (HIDS) in [18]. The ADS was built in HIDS, in which anomalies were detected beyond the capabilities of well known signature based SNORT or Bro systems through mining anomalous traffic episodes from Internet connections. A weighted signature generation scheme was developed to combine ADS with SNORT through modeling signatures from detected anomalies. The HIDS scheme was evaluated with real Internet trace data mixed with 10 days of the 1999 DARPA intrusion detection data set. The obtained results showed that HIDS achieves a 60% detection rate, compared with 30% and 22% detection rate acquired by the SNORT and Bro systems, respectively. Instead of combining signature detection techniques and anomaly detection techniques, some other hybrid systems fuse multiple anomaly detection systems according to some specific criteria considering that the detection capability for each anomaly detection technique is different. The main goal of such a hybrid system is to reduce the large number of false alerts generated by current anomaly detection approaches and at the same time keep an acceptable detection rate. Some examples of such research work are discussed in [16,17,18,19,20]. However, since such kinds of systems are just based on the idea of anomaly detection, they usually cannot outperform the other type of hybrid systems fusing misuse detection and anomaly detection.
3 Feature Analysis The major goal of feature analysis is to select and extract robust network features that have the potential to discriminate anomalous behaviors from normal network activities. Since most current network intrusion detection systems use network flow data (e.g. netflow, sflow, ipfix) as their information sources, we focus on features in terms of flows. The following five basic metrics are used to measure the entire network's behavior: FlowCount: A flow consists of a group of packets going from a specific source to a specific destination over a time period. There are various flow definitions so far, such as netflow, sflow, ipfix, to name a few. Basically, one network flow should at least include source IP/port, destination IP/port, protocol, number of bytes and number of packets. Flows are often considered as sessions between users and services. Since
Detecting Network Anomalies Using CUSUM and EM Clustering
301
attacking behaviors are usually different from normal user activities, they may be detected by observing flow characteristic. PacketCount: The average number of packets in a flow over a time interval. Most attacks happen with an increased packet count. For example, Distributed Denial of Service (DDoS) attacks often generate a large number of packets in a short time in order to consume the available resources quickly. ByteCount: The average number of bytes in a flow over a time interval. Through this metric, we can identify whether the network traffic consists of large size packets or not. Some previous Denial of Service (DoS) attacks use maximum packet size to consume the computation resources or to congest data paths, such as well known ping of death (pod) attack. PacketSize: The average number of bytes per packet over a time interval. FlowBehavior: The ratio between FlowCount and PacketSize. It measures the anomalousness of flow behavior. The higher the value of this ratio, the more anomalous the flows since most probing or surveillance attacks start a large number of connections with small packets in order to achieve the maximum probing performance. Based on the above five metrics, we define a set of features to describe entire traffic behavior on networks. Let F denote the feature space of network flows, a 15dimensional feature vector f∈F can be represented as { f1, f 2 ,..., f15} , where meaning of each feature is explained in Table 1. Table 1. List of features Features f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15
Description Number of TCP Flows per Minute Number of UDP Flows per Minute Number of ICMP Flows per Minute Average Number of TCP Packets per Flow over 1 Minute Average Number of UDP Packets per Flow over 1 Minute Average Number of ICMP Packets per Flow over 1 Minute Average Number of Bytes per TCP Flow over 1 Minute Average Number of Bytes per UDP Flow over 1 Minute Average Number of Bytes per ICMP Flow over 1 Minute Average Number of Bytes per TCP Packet over 1 Minute Average Number of Bytes per UDP Packet over 1 Minute Average Number of Bytes per ICMP Packet over 1 Minute Ratio of Number of flows to Bytes per Packet (TCP) over 1 Minute Ratio of Number of flows to Bytes per Packet (UDP) over 1 Minute Ratio of Number of flows to Bytes per Packet (ICMP) over 1 Minute
Empirical observations with the 1999 DARPA network traffic flow logs show that network traffic volumes can be characterized and discriminated through these features. For more information about the results of the empirical observation refer to [21].
4 Overview of Two Anomaly Detection Approaches In this section, we briefly introduce the two network intrusion detection techniques, namely non-parametric Cumulative SUM (CUSUM) algorithm and
302
W. Lu and H. Tong
Expectation-Maximization (EM) based clustering algorithm. More information about CUSUM and EM clustering algorithm can be found in [22] and [23], respectively. 4.1 Non-parametric CUSUM Algorithm The CUSUM algorithm is an approach to detect a change of the mean value of a stochastic process and it is based on the fact that if a change occurs, the probability distribution of the random sequence will also be changed. A basic assumption for the non-parametric CUSUM algorithm is that the mean value of the random sequence is negative during normal conditions, and becomes positive when a change occurs. Consequently, a transformation of {Xn} into a new sequence {Zn} is necessary, which is given by Zn = Xn − β, where β is a constant. The parameter β is set according to network normal conditions and it guarantees that the major part of values of the sequence Zn is negative during normal conditions and becomes positive when a change occurs. In practice, a recursive non-parametric CUSUM algorithm is used to detect anomalies online. The recursive version is presented in [22,24] and can be defined using a new sequence {Yn}:
⎧⎪Yn = (Yn−1 + Xn − β )+ ⎧ x, where x + = ⎨ ⎨ Y0 = 0 ⎪⎩ ⎩0,
x>0 otherwise
where β is set in a fashion that the values of Xn - β keep slightly negative during normal operations. As a result, increases in the metric are expected to be detected, once the values are bigger than β. A long time period of values larger than β will lead further increasing of the CUSUM function until a possible alarm level is reached. A large value of Yn is a strong indication of an attack. Based on this, we define an attacking probability p to measure the anomalous degree of initial sequence Xn: ⎧ Yn , Yn < α × β ⎪ p = ⎨α × β ⎪ 1.0, otherw ise ⎩ where p is the attacking probability for sequence Xn; α is an adjusting parameter, which is used to amplify the value of β and is set as such constant 1, 2,…; Yn is the CUSUM value of sequence Xn. 4.2 EM Based Clustering Algorithm
EM algorithm is widely used to estimate the parameters of Gaussian Mixture Model (GMM). GMM is based on the assumption that the data to be clustered are drawn from one of several Gaussian distributions. It is suggested that Gaussian mixture distributions can approximate any distribution up to an arbitrary accuracy, as long as a sufficient number of components are used. Consequently, the entire data collection is seen as a mixture of several Gaussian distributions, and their corresponding probability density functions can be expressed as a weighted finite sum of Gaussian components with different parameters and mixing proportions. The conditional probability in EM describes the likelihood that data points approximate a specified Gaussian component. The greater the value of conditional probability for a data point belonging to a
Detecting Network Anomalies Using CUSUM and EM Clustering
303
specified Gaussian component, the more accurate the approximation is. As a result, data points are assigned to the corresponding Gaussian components according to their conditional probabilities. However, in some cases, there exist some data points whose conditional probability of belonging to any component of a GMM is very low or close to zero. These data are naturally seen as the outliers or noisy data. All the outlier data will be deleted or considered as anomalies during anomaly detection, and their attacking probability is set to 1.0. Algorithm 1 illustrates a detailed EM based clustering algorithm in which Cm stands for the clustering results. Algorithm 1. EM based clustering algorithm Function EMCA (data) returns clusters Cm and posterior probability pr (i | xn ) Cm = φ , 1 ≤ m ≤ k , k is the number of clusters Call EM (data); For 1 ≤ m ≤ k , 1 ≤ n ≤ N If ( pr −1 (m | xn ) = max( pr −1 (m | xn )) ) Then assign xn to C m Return C m , m = 1,2..., k
In order to apply the EM based clustering technique for detecting network anomalies, we make two basic assumptions: (1) the input data points are composed of two clusters, namely anomalous cluster and normal cluster; (2) the size of the anomalous cluster is always smaller than the size of the normal cluster. Consequently, we can easily label the anomalous cluster according to the size of each cluster. The attack probability for each data point is equal to the conditional probability of the corresponding data point belonging to the anomalous cluster, which is defined as follows: p = pr −1 (Canomalous | xn )
where xn is the data point; Canomalous is the anomalous cluster; pr −1 (Canomalous | xn ) is the conditional probability of xn belonging to anomalous cluster Canomalous.
5 Performance Evaluation We evaluate our hybrid IDS with the full 1999 DARPA intrusion detection dataset. In particular, we conduct a comprehensive analysis for network traffic provided by the dataset and identify the intrusions based on each specific day. Since most current existing network intrusion detection systems use network flow data (e.g. netflow, sflow, ipfix, etc.) as their information sources, we covert all the raw TCPDUMP packet data into flow based traffic data by using the public network traffic analysis tools (e.g. editcap [25], tshark [26]), similarly with the 1999 KDDCUP dataset [27] in which the 1998 DAPRA intrusion detection dataset [28] has been converted into connection based dataset. Although the 1998 and 1999 DARPA dataset was criticized in [29,30] due to the methodology for simulating actual network environment, they are the widely used and acceptable benchmark for the current intrusion detection research.
304
W. Lu and H. Tong
During the evaluation, the results are summarized and analyzed in three different categories, namely how many attack instances are detected by each feature and all features correlation, how many attack types are detected by each feature and all features correlation and how many attack instances are detected for each attack type. We do not use the traditional Receiver Operating Characteristic (ROC) curve to evaluate our approach and analyze the tradeoff between the false positive rates and detection rates because ROC curves are often misleading and incomplete [31]. Compared to most, if not all, other evaluations with the 1999 DARPA dataset, our evaluation covers all types of attacks and all days' network traffic and thus, we consider our evaluation as a comprehensive analysis for network traffic in the 1999 DARPA dataset. Next, we will analyze and discuss the intrusion detection results we obtain. More information about the 1999 DAPRA/MIT Lincoln intrusion detection dataset and the method for converting the TCPDUMP packet logs into network flow based logs can be found in [32] and [33], respectively. The scoring coefficient is set up according to the detection rate (DR) and the false positive rate (FPR) for each detector over a long time history. The higher the DR, the better the performance of the detector; the lower the FPR, the better the detector performance. Therefore, the ratio of DR to FPR is used to measure the performance of each detector. We evaluate individually the two detectors with the 15 features and 9 days DARPA testing data on week 4 and week 5. The evaluation results are summarized and analyzed in three different categories described in above. For the detector using EM based clustering technique, Table 2 illustrates the average value of DR, FPR and the ratio of DR to FPR for each feature over those 9 days. For the detector using CUSUM algorithm, Table 3 illustrates the average value of DR, FPR and the ratio of DR to FPR for each feature. Based on the ratio of DR to FPR in Tables 2 and 3, we normalize them and use the normalized values as elements of the scoring coefficient matrix, which is given as follows: ⎡0.78 0.81 0.95 0.7 0.74 0.34 0.51 0.58 0.73 0.84 0.0 0.76 0.54 0.65 0.72⎤ ⎢0.22 0.19 0.05 0.3 0.26 0.66 0.49 0.42 0.27 0.16 0.0 0.24 0.46 0.35 0.28⎥ ⎣ ⎦
The coefficient matrix has 2 rows and 15 columns. Row 1 means the historical reputation weight for the detector using EM based clustering algorithm and row 2 stands for the historical reputation weight for CUSUM based detector. Columns 1 to 15 stand for the features F1 to F15. During our evaluation, we have known that there are 15 features and 2 detectors included in the detection system, and we denote n and m the number of features and the number of detectors, respectively. We define S1 as the detector using EM based clustering algorithm and S2 as the CUSUM based detector. CS j Fi stands for the scoring coefficient matrix, where i = 1,2,…,15 and j = 1,2. pFi S j is the attacking probability generated by feature Fi and detection agent Sj and pFi is the attacking probability of the hybrid detection system with a specific features Fi. Based on these, we have the attacking probability for the CUSUM-EM based IDS as follows: 2
pFi = ∑ pFi S j × CS j Fi j =1
i = 1, 2,...,15
Detecting Network Anomalies Using CUSUM and EM Clustering
305
Table 2. Performance of all 15 features over 9 days evaluation for EM detector Features
F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12 F13 F14 F15
Average DR (%) 39.83 52.22 32.25 12.0 51.8 32.25 3.2 49.26 32.25 6.81 0.0 32.25 8.57 52.41 32.25
Average FPR (%) 81.84 84.04 84.14 89.03 85.74 84.17 82.92 84.19 84.14 86.71 0.0 84.17 94.59 83.59 84.17
Ratio of Avg. DR to Avg. FPR 0.487 0.621 0.383 0.135 0.604 0.383 0.0386 0.585 0.383 0.0785 0.0 0.383 0.0906 0.627 0.383
Table 3. Performance of all 15 features over 9 days evaluation for CUSUM detector Features
F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12 F13 F14 F15
Average DR (%) 11.04 12.96 1.6325 4.9 17.84 7.23 2.94 33.57 11.5 1.4 0.7 10.8 6.015 27.73 12.87
Average FPR (%) 80.43 85.94 87.33 84.44 82.42 95.5 79.61 78.18 81.1 94.87 95.24 87.26 77.18 81.85 86.12
Ratio of Avg. DR to Avg. FPR 0.137 0.15 0.02 0.058 0.217 0.757 0.037 0.429 0.142 0.015 0.0074 0.124 0.078 0.339 0.15
We evaluated the detection system with one day's DARPA testing data (i.e. W4D1). There are 14 attack types on W4D1 and total 13 attack types are detected by our hybrid system. The DR in terms of attack types is 92.8%. The number of attack types detected by using the CUSUM technique only is 5, the number is 8 with the EM based clustering only and the number is 5 using the SNORT signature based IDS. More detailed detection results regarding the hybrid detection system see [20].
6 Conclusions and Future Work We propose in this paper a new hybrid model for detecting network anomalies. In order to characterize the behavior of the network flows, we present a 15-dimensional feature vector and the empirical observation results with the 1999 DARPA intrusion
306
W. Lu and H. Tong
detection dataset show that the proposed features have the potential to distinguish the anomalous activities from normal network behaviors. A complete traffic analysis for the 1999 DARPA intrusion detection dataset is conducted using the hybrid IDS with two well-known intrusion detection sensors. Based on the achieved evaluation results, we conclude that the proposed detection detect successfully the attacks missed by the signature based IDS, SNORT, and has the potential to find unknown intrusions through the attacking probability. The future work mainly consists of the following three parts: (1) We will include more detectors into our system in the near future. Currently we tested our model using only two existing detection techniques. We expect that the more the number of detection techniques involved with the system, the better the hybrid performance. (2) We will develop more evaluation metrics to judge the fusion performance. (3) Based on (1) and (2), we will improve the hybrid system through dynamic programming techniques. The final goal of this work is to answer such a theoretical question: how one plus one is bigger than two by exploring multi-detector techniques?
References 1. Anderson, J.P.: Computer Security Threat Monitoring and Surveillance. Technical Report, James P. Anderson Co., Fort Washington, Pennsylvania (1999) 2. Denning, D.E.: An Intrusion Detection Model. IEEE Transactions on Software Engineering 2, 222–232 (1987) 3. Axelsson, S.: The Base-Rate Fallacy and the Difficulty of Intrusion Detection. ACM Transactions on Information and System Security (TISSEC) 3(3), 186–201 (2000) 4. Barreno, M., Nelson, B., Sears, R., Joseph, A.D., Tygarcan, J.D.: Can Machine Learning be Secure? In: Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security, pp. 16–25 (2006) 5. Sabhnani, M., Serpen, G.: Analysis of a Computer Security Dataset: Why Machine Learning Algorithms Fail on KDD Dataset for Misuse Detection. Intelligent Data Analysis 8(4), 403–415 (2004) 6. Patcha, A., Park, J.M.: An Overview of Anomaly Detection Techniques: Existing Solutions and Latest Technologies Trends. Computer Networks: The International Journal of Computer and Telecommunications Networking 51(12), 3448–3470 (2007) 7. Barbarra, D., Couto, J., Jajodia, S., Popyack, L., Wu, N.: ADAM: Detecting Intrusions by Data Mining. In: Proceedings of the 2001 IEEE, Workshop on Information Assurance and Security, West Point, NY (June 2001) 8. Lunt, T.F., Tamaru, A., Gilham, F., Jagannathm, R., Jalali, C., Neumann, P.G., Javitz, H.S., Valdes, A., Garvey, T.D.: A Real-time Intrusion Detection Expert System (IDES). Technical Report, Computer Science Laboratory, SRI International, Menlo Park, USA (February 1992) 9. Anderson, D., Frivold, T., Tamaru, A., Valdes, A.: Next Generation Intrusion Detection Expert System (NIDES). Software Users Manual, Beta-Update release. Computer Science Laboratory, SRI International, Menlo Park, CA, USA, Technical Report SRI-CSL-95-0 (May 1994) 10. Porras, P., Neumann, P.: EMERALD: Event Monitoring Enabling Responses to Anomalous Live Disturbances. In: Proceedings of the 20th NIST-NCSC National Information Systems Security Conference, Baltimore, MD, USA, pp. 353–365 (1997)
Detecting Network Anomalies Using CUSUM and EM Clustering
307
11. Tombini, E., Debar, H., Mé, L., Ducassé, M.: A Serial Combination of Anomaly and Misuse IDSes Applied to HTTP traffic. In: Proceedings of the 20th Annual Computer Security Applications Conference, Tucson, AZ, USA (2004) 12. Zhang, J., Zulkernine, M.: A Hybrid Network Intrusion Detection Technique using Random Forests. In: Proceedings of the 1st International Conference on Availability, Reliability and Security, pp. 262–269. Vienna University of Technology (2006) 13. Peng, J., Feng, C., Rozenblit, J.W.: A Hybrid Intrusion Detection and Visualization System. In: Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems, pp. 505–506 (2006) 14. Depren, O., Topallar, M., Anarim, E., Ciliz, M.K.: An Intelligent Intrusion Detection System (IDS) for Anomaly and Misuse Detection in Computer Networks. Expert Systems with Applications 29(4), 713–722 15. Qin, M., Hwang, K., Cai, M., Chen, Y.: Hybrid Intrusion Detection with Weighted Signature Generation over Anomalous Internet Episodes. IEEE Transactions on Dependable and Secure Computing 4(1), 41–55 16. Xiang, C., Lim, S.M.: Design of Multiple-level Hybrid Classifier for Intrusion Detection System. In: Proceedings of the IEEE Workshop Machine Learning for Signal Processing, pp. 117–122 (2005) 17. Thames, J.L., Abler, R., Saad, A.: Hybrid Intelligent Systems for Network Security. In: Proceedings of the 44th ACM Annual Southeast Regional Conference, pp. 286–289 18. Peddabachigari, S., Abraham, A., Grosan, C., Thomas, J.: Modeling Intrusion Detection System using Hybrid Intelligent Systems. Special issue on Network and Information Security: A Computational Intelligence Approach. Journal of Network and Computer Applications 30(1), 114–132 (2007) 19. Shon, T., Moon, J.: A Hybrid Machine Learning Approach to Network Anomaly Detection. International Journal on Information Sciences 177(18), 3799–3821 (2007) 20. Sabhnani, M.R., Serpen, G.: Application of Machine Learning Algorithms to KDD Intrusion Detection Dataset within Misuse Detection Context. In: Proceedings of International Conference on Machine Learning: Models, Technologies, and Applications, pp. 209–215 (2003) 21. http://nsl.cs.unb.ca/wei/hybrid.htm 22. Wang, H.N., Zhang, D.L., Hin, K.G.: Detecting SYN flooding attacks. In: Proceedings of IEEE INFOCOM 2002 (June 2002) 23. Lu, W., Traore, I.: Unsupervised anomaly detection using an evolutionary extension of Kmeans algorithm. International Journal on Information and Computer Security 2(2), 107– 139 (2008) 24. Peng, T., Leckie, C., Ramamohanarao, K.: Detecting distributed denial of service attacks using source IP address monitoring. Draft (November 2002) 25. http://www.ethereal.com/docs/man-pages/editcap.1.html 26. http://www.wireshark.org/docs/man-pages/tshark.html 27. http://kdd.ics.uci.edu/databases/kddcup99/ kddcup99.html.kddcup 28. http://www.ll.mit.edu/IST/ideval/data/1998/ 1998_data_index.html 29. Mahoney, M.V., Chan, P.K.: An analysis of the 1999 DARPA/Lincoln Laboratory evaluation data for network anomaly detection. In: Proceedings of the 6th International Symposium on Recent Advances in Intrusion Detection, Pittsburgh, PA, USA, pp. 220–237 (2003)
308
W. Lu and H. Tong
30. McHugh, J.: Testing intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory. ACM Transactions on Information and System Security 3(4), 262–294 (2000) 31. Gaffney, J.E., Ulvila, J.W.: Evaluation of intrusion detectors: a decision theory Approach. In: Proceeding of IEEE Symposium on Security and Privacy, pp. 50–61 (2001) 32. http://www.ll.mit.edu/IST/ideval/data/1999/ 1999_data_index.html 33. Lu, W., Ghorbani, A.A.: Network anomaly detection based on wavelet analysis. EURASIP Journal on Advances in Signal Processing (2008) (in press)
