Real Time Network Anomaly Detection Using Relative Entropy Altyeb Altaher
Sureswaran Ramadass, Ammar Almomani
National Advanced IPv6 Center of Excellence
National Advanced IPv6 Center of Excellence
Universiti Sains Malaysia,Malaysia
Universiti Sains Malaysia,Malaysia
Abstract-As
Penang,Malaysia
Penang,Malaysia
[email protected]
Sures,
[email protected]
the computer networks continue to increase in size,
complexity and importance, the network security issue becomes more and more important. In this paper, we propose a real time anomaly detection system based on relative entropy. The proposed system captures the network traffic packets and then uses relative entropy and adaptive filter to dynamically determine the traffic changes and to examine anomaly.
The paper is organized as follows. Related work in section II . Section III
presents
the network traffic attributes and the
network entropy. Section IV presents the proposed anomaly detection method. Section V evaluates the effectiveness of our proposed scheme. Section VI concludes the paper.
whether the traffic change is normal or contains
II.
Our experimental results show that the proposed system is
RELATED WORK
efficient for on-line anomaly detection, using traffic trace collected in high-speed links.
The
Keywords- Network security ; anomaly detection; entropy
theory.
failure
of
traditional
signature-based
in
detecting
polymorphic and unseen malware, orients the research in network security to directions. The entropy of different packet attributes under normal and abnormal network conditions have been analyzed in [6-8,10]. The abnormal network traffic
I.
worms. Further work on As the computer networks continue to increase in size, complexity
and
importance,
the
network
is
affected by different attacks, such as DoS, port scanning and
INTRODUCTION
security
issue
becomes more and more important.
fmding out whether the entropy
values of different attributes are highly correlated is given in [6]. In our paper we also observe this phenomenon and verity the conclusion made in [6]. In [9], the authors make use of the concept of maximum entropy to build up a normal network
"Malware" is an abbreviation for 'malicious software' and is
distribution baseline and then use relative entropy to detect the
used to refer to any software designed to cause damage to a
anomalies. However, the baseline distribution in [9] is based on
single computer, server, or computer network[1].According to
the
Kaspersky labs in February 2011, 252,187,961 malicious
combination
TCP/IP
protocol of
field
different
they
only
attributes
which
attribute-values take
three
are
protocol
means very fields
the
large.
programs detected [2]. This worryingly high number is only
Subsequently,
likely to increase, especially as the malware author's incentives
consideration and their experiments showed that it would
into
for writing such software is now mainly a financial one.
generate a large
According to its propagation methods, malicious code is
should be ordered and labeled according to their features and
usually classified into the following categories
the complex preprocess will decrease the ability for detecting
[3][4][5]:
viruses, worms, Trojan horses, backdoors and spyware . Due to the
significant
loss
and
damages
induced
by
feature set. Moreover, the raw packet data
anomalies in real-time.
malicious
executables, the malware detection becomes one of the most
III.
critical issues in the field of computer security. Currently, most widely-used malware detection software uses signature-based method to recognize threats . Signatures are sequences of bytes in the machine code of the malware. The inability of traditional signature based malware detection approaches to catch polymorphic and new, previously unseen malwares has shifted the focus of malware detection research to find more generalized and scalable features that can identity malicious behavior as a process instead of a single static signature. In this paper, we propose a real time anomaly detection system
NETWORK TRAFFIC ATTRIBUTES AND NETWORK ENTROPY
In this section, we describe the important network traffic
attributes
for anomalies detection. We also describe the
computation on the entropy values of the network traffic attributes. A.
Network Traffic Attributes
After researching on customer service flows in an enterprise, we summarize the work flows of its customer service system are as follows:
based on relative entropy. The system uses the relative entropy to analyze the network traffic and detect the anomalies.
978-1-4577-1169-5/11/$26.00 ©2011 IEEE
258
Traffic attributes that are especially important (because their
adaptive filter to examine the traffic changes and determines
rapid change during typical attacks) and used during process of
whether the traffic is normal or contains anomaly.
anomaly detection are [11]: Source and destination IP address,
•
detect
anomalies, we
need
a
method
to
clearly
differentiate network anomalies from the normal behavior.
Number of bytes packets received by the local host,
•
Therefore we introduce an adaptive threshold to differentiate
TCP flags, especially SYN, RST and FIN flags
•
Adaptive Detection Threshold Setup To
Number of bytes and packets sent to the remote hosts,
•
•
B.
Source and destination port,
•
between steady
Duration of the connection
"normal"
network traffic behavior, and
non-steady network traffic behavior. We first compute entropy values of degree distributions in each time interval, and then
In our approach we take into consideration the following:
compute mean entropy in a particular time interval. We also
source/destination IP and port number,
use variance to reflect the deviation between normal and
Number of bytes sent/received.
abnormal behavior.
These attributes were selected because a significant number of worm attacks cause changes in the values of these attributes
Let us assume, the measured entropy Y be a random variable
and therefore could be recognized as an anomalous state [12].
with mean E( y)
=
!.I. and var( y)
=
cr2 . Then, the
Threshold is defined as follow Threshold
Network Entropy
B.
= Jl±3*0"
(2)
The entropy of normal network traffic behavior is less than
Entropy is a measurement of the disorder of a system. If the
equal the threshold
Beyond this normal region, the entropy
system tends to be in disorder, its entropy increases towards 1;
or
if the system tends to be in order, then its entropy decrease
represents traffic events as anomalous and assigns a severity
towards o.
level depending upon its deviation from the normal region.
We can view certain attributes of packets that we capture in a period of time as a set. The entropy of the packet attribute value can be defined as:
The processing engine in the proposed online detection system detect anomalies.
(1)
=
;=1
The anomaly detection system captures network traffic
Where in H(P), the P(x) is as follows:
(
XI
)
every time window (30 sec) and store the source and destination IP address of the all flows in the database, then it
Number· of . pkts· with· Xi as· certain· attribute •
=
Anomaly Detection Methodology
uses an efficient lightweight methodology based on entropy to
H(P) ip(xi)logP(xi)
p
C.
finds the number of flows sent to distinct destination IP
Total. number· of· pkts
address. Based on the number of total flows and the number of
It should be notes that the work in [6-8,10] takes this approach to check for the differences between the normal and abnormal network action.
flows sent to distinct destination the algorithm computes the entropy value for this time window. The algorithm calculates the detection threshold as entropy average value plus or minus the three standard deviations. If the entropy value of network traffic flow exceeds the threshold then the network traffic flow is considered as anomaly.
IV.
THE PROPOSED REAL TIME ANOMALY DETECTION
V.
SYSTEM In this section, we first give an overview of the proposed
anomaly detection System. Second, we describe our adaptive detection threshold setup. Then we present our methodology for computing the entropy and self-adjusting the threshold to raise an alert when abnormal traffic is detected.
A.
PERFORMANCE EVALUATION
The network topology for the experiment is shown in Fig.l. All the network traffic captured using the network interface card (NIC) on PC3 in Fig.3 and stored in database, the database design consists of one table and stores all information used and needed by our proposed system.
The proposed real time anomaly detection system description The proposed system captures the network traffic packets
and
then
uses
relative
entropy
and
adaptive
filter
to
dynamically determine the traffic changes. It then applies
259
entropy. To validate the efficiency of our online anomaly detection technique, we used real time network traffic from the National Advanced IPv6 Center of Excellence - University
Internet
Science Malaysia (USM). Next we injected the Witty Worm
AI nttwol'k tnf'Iicwu Cipwrtd using tht n.twort a«bpt.r on PC3and thin proc:ustd with the CAlDAdmnl bythl d.n�
dataset
of
CAIDA
into the
online network traffic.
Our
experimental results show that the proposed system is efficient
high spttd nttwo!' MlOClUIy d.ttction system
for on-line anomaly detection because it is based on the entropy which increases the sensitivity of the detection process to uncover well-known or unknown anomalies. Furthermore, the use of adaptive threshold results in lower false alarm rate. Our ongoing work further analyzes the traffic anomalous features, and extends the methodology proposed in this paper
pe3 High speed networt; anomaly detection system
to diagnose additional network-wide anomalies REFERENCES
Figure 1 : The network topology for the experiment [I]
P. Szor. The Art of Computer Virus Research and Defense. Addison Wesley for Symantec Press, New Jersey, 2005.
[2]
Zakorzhevsky, 2011. Monthly Malware S tatistics. Available from: http://www.securelist.comlen/analysis/2047921821M0nthly_Malware_St atistics_June_2011 [Accessed 2 July 2011).
[3]
Adleman, L.: An abstract theory of computer viruses (invitedtalk). In: CRYPTO '88: Proceedings on Advances in Cryptology, pp. 354-374, 1990.
[4]
Filiol, E: Computer Viruses: from Theory to Applications.Springer, Heidelberg ,2005.
[5]
McGraw, G., Morrisett, G.: Attacking malicious code: report to the infosec research council. IEEE Softw. 17(5, pp. 33-41), 2002.
[6]
G. NychisOV. Sekar, D. G. Anderson, etc. "An Empirical Evaluation of Entropy-based Anomaly Detection" Proceedings of the 8th ACM SIGCOMM conference on Internet measurement" ACM Press, 2008,pp151-156.
[7]
D.Brauckhoff, B. Tellenbach, A Wagner, etc. "Impact of traffic sampling on anomaly detection metrics." Proceedings of the 6th ACM SIGCOMM conference on Internet measurement. ACM Press, 2006, ppI59-164.
We used real time network traffic from National Advanced IPv6 Center of Excellence at the University Science Malaysia (USM).
Throughout
measurements remarkable
of
our
the
similarity
experimentation,
high
speed
except
for
network a
few
the
entropy
traffic peaks.
show These
exceptional entropy values represent the magnitude of traffic features'
distributional variations during the measurement
period. We picked sample snapshots of time where peaks are observed, and investigated the network traffic measurements. To validate the efficiency of our high speed network anomalies detection, we have injected anomaly network traffic which is Witty Worm dataset of CAIDA [15] in the online network traffic.
-[,..ropy
[8]
ALakhina, M. Crovella, and C. Diot. "Mining anomalies using traffic feature distributions". Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer.
-U�Ttorehold -lO'W�th,fthokt
communications.ACM Press, 2005, pp217-218 [9] Time
h«on
