Increasing overall network security by integrating ...

Increasing overall network security by integrating Signature-Based NIDS with Packet Filtering Firewall Hamed Salehi

Hossein Shirazi

Reza Askari Moghadam

Payam Noor University Information Services Corporation Tehran, Iran [email protected]

Malek-Ashtar University of Technology Tehran, Iran [email protected]

Payam Noor University Tehran, Iran [email protected]

Abstract— Today Network Intrusion Detection and Intrusion Prevention System (NIDS/IPS) are considered as one of the hottest topics in computer security. On the other side firewalls have optimized several times and different types have been introduced. Today by integrating NIDS and Firewall a new product comes to the market, which is called IPS. IPSs protect information systems from unauthorized access, damage or disruption. They are installed on network primary point and perform deep packet inspection (6 layers) so the hardware should be fast enough to sit almost invisibly within the network. This policy requires expensive hardware which is based on multiple server processor technology. It also needs appropriate changes in network design and policies. The cost may not be so reasonable for medium and small size companies. In this paper we are going to implement some kind of integration between signature-based NIDS and packet filtering firewalls which would increase the overall security by a reasonable cost in comparee with modern IPSs. We try to conclude this by optimizing snort as a famous open source NIDS with a sample firewall program in Linux which is implemented by means of IPTABLES commands. The data is transferred in standard XML format. We also test the model by standard DARPA99 data sets and the results are satisfied. Keywords-component; Security, NIDS, Firewall, IPS, Snort.

I.

INTRODUCTION

Intrusion Prevention Systems (IPS) are the latest advance in protecting networks from computer aided attack. The previous development of systems such as Firewalls and Intrusion Detection Systems (IDS) laid a strong foundation for IPS. Firewalls were first introduced in 1989 by Jeffery C. Mogul[1]. They were gone under different changes during these 19 years. Firewall is a useful tool for protecting networks from malicious and harmful attacks. They are usually installed in primary entry point of the network with a pre-defined policy and rules. The administrator will define appropriate rules to protect the network against malicious attacks. It can be a valid solution for preventing basic intrusion. But the needs for new tools which can detect attacks and notify the administrator for defining appropriate protection rules finally introduced NIDS. The goal of Intrusion Detection System (NIDS) is to monitor network assets in order to detect anomalous behaviors and misuse. Such a goal has been recognized as significant for nearly twenty years but only recently a dramatic rise has been found in popularity and incorporation into the overall information security infrastructure [17]. But the researches did not stop there and more proactive tools were needed to protect the new and complex set of attack. The result was IPS, and

integration of different security tools. IPS encompasses aspects of many well-known, existing security technologies including anti-virus, software, intrusion detection and firewalls. This enables organization to become self-protecting. In this case the tools will determine attack and take appropriate action with complete perfection. They are installed on network primary point and perform deep packet inspection (6 layers) so the hardware should be fast enough to sit almost invisibly within the network. This policy requires expensive hardware which is based on multiple server processor technology. It also needs appropriate changes in network design and policies. The cost may not be so reasonable for medium and small size companies. In this paper we are going to implement some kind of integration between signature-based NIDS and packet filtering firewalls which would increase the overall security by a reasonable cost in comparison with modern IPSs. In the following sections we will present some information about Firewalls especially packet filtering firewalls, IDS systems (Snort) and IPS(II).Section III is the beginning of the contribution and introduces the idea we are going to approve in this paper. Then we have a brief introduction to IPTABLES in Linux which will be used in our experiment (Section IV). The XML schema for data interchanges have been mentioned in section V. Finally we have an experimental implementation followed by conclusion. II.

FIREWALLS, IDS AND IPS

The first generation was introduced at 1985[1] which is called Packet Filtering firewalls. It uses filtering rules to determine the validity of the network traffics. The rules are defined on the basis of factors like source IP and port, destination IP and port [2]. The firewall allows direct connections among networks through address authorisation which could be a problem. In this project we used IPTABLES which is an administration tool for IPv4 packet filtering and NAT on Linux systems. Circuit Level Firewalls was the second generation which was introduced at about 1990. This kind of firewall is used for TCP connection. They examine each connection setup to ensure that it follows a legitimate “handshake” for the TCP is being used. The circuit level firewall then checks its records to make sure that the sender is allowed to send to the receiver and the receiver is allowed to receive from the sender. If the answer

is “yes” to both conditions, the connection and all associated packets are routed through with no more security checks [3]. The third one was the Application Layer Firewall which is an intermediary between the internal network and the Internet [4]. It can act as proxy server or proxy client. Firewall takes the intended communication from first computer and directs it to the second one [3]. Inbound connections are always made with the proxy client, while outbound connections are always made of the proxy server. There is no direct connection between the internal network and an insecure network. Because of the disadvantages with cost and performance of the application layer firewalls, and the lack of security surrounding packet filtering, a new method was developed which was called stateful inspection. Instead of examining the contents of each packet, the header information of the packet are compared to packets that are known to be trusted [5]. A new feature of the stateful inspection firewall was a state table [6]. This table kept a list of open connections. When a user accessed an outside service, the stateful inspection firewall remembered details about the original request such as port number, source and destination address. The next development of firewall technologies was the introduction of dynamic packet filtering firewalls. They were closely related to stateful inspection firewalls. [7].The current technology used for firewalls is classed as kernel proxy firewalls [7]. On the other side the goal of Network Intrusion Detection System (IDS) is to monitor network assets in order to detect anomalous behavior and misuse. Such a goal has been recognized as significant for nearly twenty years but recently a dramatic rise has been found in popularity and incorporation into the overall information security infrastructure. The first recognized IDS paper was published in 1980 by James Anderson, titled “Computer Security Threat Monitoring and Surveillance” [8]. They are two general types of available IDS today. Host Based IDS detect security violations from abnormal patterns of system usage [9]. Security violations include attempted break-in, masquerading or successful breakin, penetration by legitimate user and etc. Network Based IDS monitor suspicious behavior in computer networks. The cases are unauthorized use, anomalous behavior, and attempts to deny users, machines or access to services. Unlike host-based IDS, which detect malicious behavior outright, these systems deduce behavior based on the content and format of data packets on the network. Among other things, they analyze overt requests for sensitive information and repeated failed attempts to violate security policy. Network-based and host-based IDS would work best when coupled together to create a comprehensive security strategy. However this may be true, it is not an ideal situation. This situation started the development of Intrusion Prevention Systems (IPS).

purposes of this demonstration we use it as a NIDS1. Snort is capable of performing real-time traffic analysis and packet logging on IP networks and can perform protocol analysis, content searching/matching and can be used to detect a variety of attacks and probes, such as buffer overflows, stealth port scans, CGI attacks, SMB probes, OS fingerprinting attempts, and much more. Snort uses a flexible rules language to describe traffic that it should collect or pass, as well as a detection engine that utilizes modular plug-in architecture. Snort has a real-time alerting capability, incorporating alerting mechanisms for syslog, a user specified file, a Unix socket, or WinPopup messages to Windows clients using Samba's smbclient. The detail information is available in Snort website [16]. Finally IPS encompasses aspects of many well-known, existing security technologies including anti-virus, software, intrusion detection and firewalls. In the IPS model, instead of developing reactive security policies, security policy becomes a proactive tool to protect an organization. This enables the organization to become self-protecting. The end result would be only a limited need for NIDS or monitoring solutions since everything representing a threat is blocked [11]. Most IPS products are network-based and are deployed in the form of high-throughput appliances with hardened operating systems and firmware [12]. III.

INTEGRATION A NEW IDEA

In this section we are going to present a new schema for integrating NIDS and Packet filtering firewall. This is a kind of synchronization between these two important security tools which is used in most secure networks. As we now most rule-based NIDS stop at generating some alert based on a rule which has been fires. In This NIDS the administrator would install or define specific rule to protect the system against specific attacks. This rule are written based on the researched and analysis of the previous attacks. Some of these rules have False Positive effect meaning that there is no real attack by the format of a packet is just like the rule conditions so the NIDS will display an alert for it by mistake. The other may have false negative alert. This fault may be ignorable in NIDS system but in IPS it may lead to Denial Of Services (DOS) and it is very catastrophic. But as the researches are going to be more completed there are some rules proven to be no false positive for them or at least the percentage for false positive detection is too small. If we refer to rules signature documents in Snort we will find some useful information for each rule. For example the Snort alert generated for the previous DARPA99 dataset attack (ID: 44.080000) [14] is as follow: [1:2473:7] NETBIOS SMB ADMIN$ Unicode share access Classification: Generic Protocol Command Decode[Priority: 3] 04/01-08:16:01.899659 206.48.44.18:1070 -> 172.16.112.100:139

An example of widely used NIDS is Snort. Snort is marketed as a network intrusion detection system. For the 1

Network Based IDS

This means that the Rule ID 2473 catch the packet. By referring to this signature document we will found some information about this attack: False Positives: None Known. False Negatives: None Known. Corrective Action: Check the target host for signs of compromise. Apply any appropriate vendor supplied patches.

The document says that previously there was no False Positive and Negative for this signature. It also recommends a protection instruction for the administrator. The rule corresponding to this signature is as follow: alert tcp $EXTERNAL_NET any -> $HOME_NET 139 (msg:"NETBIOS SMB ADMIN$ unicode share access"; flow:established,to_server;… ; sid:2473; rev:7;)

This rule specify some condition for the packet which if met it means that the attack is going to happen and the specific alert in msg filed is shown. We are going to prove that why we can't define a corresponding protection rule and then implement the rule in our important network servers while we are sure that rule is confident?. This will lead to faster and better reaction to well known attacks. We prove if the NIDS could inform the servers about the attack and recommend them a protection schema then the network would be rapidly and automatically protected against these kind of attack without the interfere of the administration. The administrator may not have enough time and knowledge to update the security of the servers as soon as possible. Note that the place that NIDS reside could be the DMZ2 or inside the network but DMZ is recommended. The ideal structure we are looking would be as follow: Protection rule (XML) FTP Server + Firewall OS: Windows HTTP Server + Firewall

OS: Mac NIDS OS: Linux

Protection rule (XML)

SMTP Server + Firewall OS: UNIX

Protection rule (XML)

Figure 1. The new integration idea structure

To prove our claim we have used IPTABLES to design a basic Packet Filtering Firewall and Snort as an open source NIDS for our experimental purpose. We also provide an XML structure for transferring required data between NIDS and Firewall. In the following section we present some more information about IPTABLES and XML structure.

2

De Militarized Zone

IV.

IPTABLES

IPTABLES is a very practical tools in Linux and many open source firewalls rely on it [16,17]. IPTABLES are building blocks of a framework within the Linux 2.4.x and 2.6.x kernels. It is the successor of IPChains (2.2.x) and ipfwadm (2.0.x). This framework enables packet filtering, network address [and port] translation (NA[P]T). It also allows packet mangling. It works by creating a generic table structure of rule sets. As in IPChains each rule in an IPTABLES consists of a set of classifiers (IPTABLES matches) and a corresponding action (IPTABLES target). The format of the rule is as follow: IPTABLES –t (FILTER/RAW/NAT/MANGLE) -A/D/I/R (INPUT/ OUTPUT/FORWARD/ PREROUTING /POSTROUTING)-s (source address) -p (protocol) - d (destination) -j (DROP/REJECT/LOG/ACCEPT/ Userdefined chain) Table: there are 4 tables: Raw, Nat, Mangle and Filter tables. The filter table is mainly used for filtering packets. Chain: There are a minimum of 3 built-in chains: INPUT, OUTPUT and FORWARD. Other chains can be added to. The Jump: A jump instruction is written exactly the same as a target in IPTABLES, with the exception that instead of writing a target name, you write the name of another chain. If the rule matches, hence the packet will be sent to this second chain and be processed as usual in that chain. Detail information can be found in Linux reference and new incoming papers [13]. We will use the firewall option data to fill most of the rule parts according to the protection requested. As we want to protect server against incoming packet, The FILTER table and the INPUT chain is selected for most of the rules while this is also changeable according to XML data structure received from NIDS. The rule looks like as follow: IPTABLES -t Table -A Chain -p PRTCL -s SIP--sport SPORT -d DIP --dport D_PORT -j Jump In the following section we will present an XML schema which holds the required data in this rule. Note that we just provide brief information about IPTABLES while the reader should have deep knowledge to catch the concept within the rule. Linux reference may be a good place for this purpose. V.

XML SCHEMA FOR NIDS-FIREWALL COMMUNICATION

XML provides a platform and programming languageindependent scheme for sharing data among applications and corporations in an unambiguous, consistent, and extensible manner. Today most data interchange among disparate programs is done by using this magic tool [15]. In this paper we have proven that by integrating NIDS with Firewalls specially packet filtering one we reach to more confident security in the network. The idea is mostly based on the case that firewall and NIDS are installed on different platform. So they must have a kind of protocol to talk. Since XML is mostly

practical in this issue we have designed a structure for data being transferred between these two independent identities. In the following the XML Tree is presented:

FTP Server + Firewall Protection rule (XML)

NIDS OS: Linux

Figure 3. Schema for the experimental implementation

Figure 2. XML tree structure for the Protection rules

The root contains general information for IPTABLES rule including Table to apply the rule to, Protocol(PRTCL) , the Jump, the Chain and the destination port(D_Port). The leaves contain information about the source and the target and also the NIDS rule which has been fired in response to attack. This is an elementary structure and it may need further information about the attack, confidence rate and so on. Or include the kind of attack like FTP, SMTP or HTTP so that each server just applies the related recommended protection and discard the others. But the structure is sufficient for the basic standard communication. Considering Week 5 Thursday DARPA99 attack ID 44.080000[14].A sample XML presentation for the attack is as follow: 172.016.112.100 20 TCP 206.048.044.018 1041 TCP NETBIOS SMB ADMIN$ unicode share access 3 DROP INPUT filter TCP 1041

VI.

EXPERIMENTAL IMPLEMENTATION

A. Experimental Requirements For the experimental purpose we would implement the idea for and NIDS and an FTP server as follow:

In this schema we need to have a firewall on the FTP server which is listening on a special port number for the NIDS protection rule recommended by NIDS. We have designed this simple firewall using IPTABLES on Linux. The program will run the IPTABLES command as soon as it receives it from the NIDS. The code is presented in appendix A. On the other side we need to change the snort in such way that it can accept new option in his rules and send the value provided in this option to the dedicated servers. This needs Data and Functional changes in code that we have present the related information in appendix B and C. B. Initial States in Server Before running Snort we will clean the IPTABLES entries with the following command: IPTABLES - - flush After this the table will be entirely cleaned and there would be no rules. Then we set the default policy to ACCEPT meaning that we accept all the INPUT and OUTPUT packets if there is no rule matching them. IPTABLES -t filter –P INPUT ACCEPT IPTABLES -t filter –P OUTPUT ACCEPT For our FTP server a simple ruleset to allow an ftp connection would be: IPTABLES -A INPUT -p tcp --sport 21 -m state --state ESTABLISHED -j ACCEPT IPTABLES -A OUTPUT -p tcp --dport 21 -m state --state NEW,ESTABLISHED -j ACCEPT In active ftp the ftp client sends a port number over the ftp channel via a PORT command to the ftp server. The ftp server then connects to this port to send data such as a file, or the output from an ls command using port 20. The ftp-data connection is in the opposite of sense from the original ftp connection . In order to allow active ftp without knowing the port number that has been passed we need a general rule which allows connections from port 20 on remote ftp servers to high ports (port numbers > 1023) on ftp clients. The following rules will serve our purposes grandly:

IPTABLES -A INPUT -p tcp --sport 20 -m state --state ESTABLISHED,RELATED -j ACCEPT IPTABLES -A OUTPUT -p tcp --dport 20 -m state --state ESTABLISHED -j ACCEPT By applying these rules on the server we have the following IPTABLES entries: Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp spt:ftp-data state RELATED,ESTABLISHED

ACCEPT tcp -- anywhere anywhere

tcp spt:ftp

state ESTABLISHED

Chain FORWARD (policy DROP) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination ACCEPT tcp -- anywhere anywhere tcp dpt:ftp-data state ESTABLISHED ACCEPT tcp -- anywhere anywhere tcp dpt:ftp state NEW,ESTABLISHED a. IPTABLES entries after applying FTP server rules

We will see that the IPTABLES rule will be added here after applying Week 5 Thursday TCP dump dataset in our NIDS as input. VII. RUNNING SNORT WITH DARPA99 TEST DATASET We select the Outside TCP dump for Thursday of DARPA99 [14] data set for our experimental test. After running Snort by analyzing the output we have found that there were about 2 messages with the priority number 3. There maybe some iteration for same attack and we have discarded them all because once the first one is seen, the protection rule would be generated and notified to all servers. By filtering the same priority 3 alert the following unique messages would be extracted: (Attack ID: 44.080000) [**] [1:2467:7] NETBIOS SMB D$ unicode share access [Classification: Generic Protocol Command Decode] [Priority: 3] 04/01-08:16:01.918058 206.48.44.18:1070 -> 172.16.112.100:139 (Attack ID: 44.130700) [**] [1:1226:5] X11 xopen [**] [Classification: Unknown Traffic] [Priority: 3] 04/01-13:08:19.458736 209.12.13.144:1558 -> 172.16.114.168:6000 If we refer to Snort documentation about the first attack we will found that the rule number 2467 has been fired ([1:2467:7]). By looking in file 2467.txt in signatures document we will find the required information about the time this message is generated: Detailed Information: This event is generated when an attempt is made to use Samba to gain access to private or administrative shares on a host. This attack could lead to unauthorized administrative access to the server. It seems that we should disconnect such dangerous and famous connection by dropping the packets. So that we have defined the firewall command DROP in firewall option of the rule. As the result the rule will be changed to:

alert tcp $EXTERNAL_NET any -> $HOME_NET 139 (msg:"NETBIOS SMB D$ unicode share access";…;Firewall:"DROP";…; sid:2467; rev:7;) Also for the second one ([1:1226:5]) we have the following information: Detailed Information: Implementations of the X windows system from the X consortium may use weak authentication methods when allowing remote machines to connect to a host running X windows. So we change the corresponding rule to include a protection recommendation through firewall: alert tcp $EXTERNAL_NET any -> $HOME_NET 6000 (msg:"X11 xopen";…; Firewall:"DROP";… ; sid:1226; rev:4;) The details about the changes in Snort source code and data structure to accept the new option is described in Appendix B and C. After changing the rules and the Snort source codes and recompiling using gcc in Linux we have run the program with the same TCP DUMP dataset to see the effects on servers after detecting the priority 3 attacks. As soon as the first priority 3 attack is detected (04/0108:16:01) appropriate notification is send to the server and the delay is just the time between sending through NIDS server to the FTP server. Using this techniques the Firewall program which is installed in FTP server and has some kind of integration with the NIDS server will limit the harmful incoming connection by applying an appropriate IPTABLES command. The output of IPTABLES command after running Snort with DARPA99 data set will be as follow: Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:20 state RELATED,ESTABLISHED ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp spt:21 state ESTABLISHED DROP tcp -- 172.16.112.100 206.48.44.18 tcp spt:1070 dpt:139 DROP tcp -- 172.16.114.168 209.12.13.144 tcp spt:1558 dpt:6000

Chain FORWARD (policy DROP) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:20 state ESTABLISHED ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:21 state NEW,ESTABLISHED b. IPTABLES entries after firewall activity

Finally we can see two new protection rules are added in IPTABLES entries which protect the server against the two new detected attacks. The process is done without any interaction of human. The time between detecting the attack and applying the appropriate rule is just as much as the required time to send the packet from NIDS server to FTP server. These are the thing we have approved in this paper using a real experimental test with Standard TCP Dump data set.

Now the connection is disconnected either in server side or attacker. The main point is that the harmful connection is no longer alive. All of this process was successfully done because of well integration and interaction between NIDS and Firewall. This interaction could also be extended to include many other options but the primary schema shows such a high priority in successful reaction to new incoming attacks that we are sure there is no False positive detection for them.

REFERENCE [1]

[2] [3] [4] [5]

VIII. CONCLUSION Today NIDS and Firewall are two major tools for implementing security in many computer networks. By defining appropriate rules in firewalls not only we can limit the incoming and outgoing packets but also drop the suspicious ones. On the other sides, NIDSs monitor the traffic of network by capturing the suspicious packet and notifying through an appropriate alert message.

[6]

The security administrators analyze the NIDSs alerts and try to find it if the alert is generated for a high severity attack. If so they will define a protection policy and implement it using tools such as Firewalls. In the early days the NIDSs have so many False Positive rates while this rate is decreasing by analyzing the attack data using new different techniques.

[10]

Today the execution pattern of some basic attacks is known to all and there is almost no False Positive detection for them. Alert messages generated for this kind of attack in NIDSs usually have high priority. In this paper we have shown that by implementing a kind of coordination between NIDSs and Packet Filtering Firewalls we can simply detect these attacks and protect our server as soon as possible without the human based interaction which could be inaccurate and slow. In this technique as soon as detecting a high priority attack the NIDSs send some recommended protection information to the firewall which is installed on the servers using a specific format (XML structure). The firewalls which are listening under special port will receive the information and implement an appropriate protection rule in order to destroy the connection. This technique could be optionally enabled or disabled in firewall or NIDS sides but the overall process seems to be practical for networks respecting security issues. This schema does need any hardware reconfiguration, expensive cost (in comparison with other techniques) and human interaction and real-integration of tools such that NIDSFirewalls. Also by using xml it could be platform-independent and extensible too. The only thing is that NIDSs and Firewall generator companies should include this property in their products and there also should be a standard data transferring schema between these products.

[7] [8] [9]

[11]

[12] [13]

[14]

[15]

[16] [17]

Mogul, Jeffery C., 1989, Simple and Flexible Datagram Access Controls for Unix-based Gateways, USENIX Conference Proceedings, pp. 203221. PC Help, 1999, What is a Firewall, PC Help Website., http://www.pchelp.org/www.nwinternet.com/pchelp/security/firewalls.htm IT Security, 2002, Computer Security Dictionary, ITsecurity.com, http://www.itsecurity.com/dictionary/dictionary.htm Meyer, Helen, 1997, A History of Firewall Technology, Computers & Security, 16 (4), p. 331. Sheldon, Tom, 2002, General firewall Paper, WindowSecurity.com, http://www.secinf.net/firewalls_and_VPN/General_Firewall_White_Pap er.html Jackson, Jeromie, 1997, Making distinctions between firewall technologies, Computer Technology Review; Winter, pp. 38-40. Whitman, Michael E and Herbert J Mattord, 2003, Principles of Information Security, Thomson Publishing. Anderson, James P, 1980, Computer Security Threat Monitoring and Surveillance, http://csrc.nist.gov/publications/history/ande80.pdf. Denning, Dorothy, E. 1987, An Intrusion Detection Model, IEEE Transactions on Software Engineering, Number 2, February, p.222 Durst, Robert, Terrence Champion, Brian Witten, Eric Miller, and Luigi Spagnuolo, 1999, Testing and evaluating computer intrusion detection systems, Communications of the ACM, 42 (7), July, pp. 53- 61. Lindstrom, Pete, 2004, Intrusion Prevention Systems (IPS): Next Generation Firewalls, http://www.forumintrusion.com/Spire_IPS_Whitepaper.pdf Krull, Joseph E., 2003, What to expect from your IPS, Communications News, October, 40 (10), p. 19. Detecting and preventing peer-to-peer connections by Linux iptables Othman, Mohamed; Kermanian, Mostafa Nikpour; Information Technology, 2008. ITSim 2008. International Symposium on,Volume 4, 26-28 Aug. 2008 Page(s):1 - 6 DARPA Intrusion Detection Evaluation, MIT university, http://www.ll.mit.edu/mission/communications/ist/files/master_identific ations.list Peter Buneman , Susan B. Davidson , Mary F. Fernandez , Dan Suciu, Adding Structure to Unstructured Data, Proceedings of the 6th International Conference on Database Theory, p.336-350, January 0810, 1997. SNORT Website, an open source network intrusion prevention , http://www.snort.org/ Innella, Paul, 2001, The Evolution of Intrusion Detection Systems, Tetrad Digital Integrity, http://www.securityfocus.com/infocus/1514