This solution allows the IDS to monitor every different network segment attached to the ... 2.2.1 Host-based Intrusion Detection Systems . .... âStealthâ / advanced.
An Efficient Intrusion Detection System for Networks with Centralized Routing
Paulo Filipe Canha de Andrade
Dissertação para obtenção de Grau de Mestre em
Engenharia Informática e de Computadores
Júri Presidente:
Prof. Luis Eduardo Teixeira Rodrigues
Orientador:
Prof. Fernando Henrique Corte Real Mira da Silva
Co-orientador: Vogais:
Prof. Carlos Nuno da Cruz Ribeiro Prof. Rodrigo Seromenho Miragaia Rodrigues
Setembro de 2007
Abstract
As Internet becomes more and more ubiquitous, security is an increasingly important topic. Furthermore, private networks are expanding and security threats from within the network have to be cautioned. For these large networks, which are generally high-speed and with several segments, Intrusion Detection System (IDS) placement usually comes down to a compromise between investment and monitoring ability. One common solution in these cases, is to use more than one IDS scattered across the network, thus raising the amount invested and administrative power to operate. Another solution is to collect data through sensors and send it to one IDS via an Ethernet hub or switch. This option normally tends to overload the hub/switch port where the IDS is connected. This document presents a new solution, for networks with a star topology, where a single IDS is coupled to the network’s core router. This solution allows the IDS to monitor every different network segment attached to the router in a round-robin fashion. Practical implementation issues and operational implications of this solution are also analyzed and discussed. Keywords: Intrusion Detection Systems,Security Analysis, High-speed Networks, Switch-based Networks.
i
Resumo
À medida que a Internet se torna cada vez mais acessível, a segurança é cada vez mais um tópico muito importante. Para além disso, com o crescimento célere de redes privadas, as ameaças proveniente do interior da rede têm que ser acauteladas. Para estas redes de grande dimensão, que são geralmente de alta velocidade e com vários segmentos, o posicionamento de um Sistema de Detecção de Intrusões (SDI) acaba normalmente num compromisso entre o investimento e a capacidade de monitorização. Uma solução comum para estas redes, é o uso de vários SDIs espalhados pela rede. Elevando assim, o investimento necessário e os recursos administrativos necessários. Outra solução possível, é a de colher os dados da rede através de sensores e enviá-los para um SDI via um comutador de pacotes. Contudo, com esta solução, existe o risco de sobrecarregar a porta do comutador de pacotes onde está ligado o SDI. Este documento descreve uma nova solução para redes com uma topologia em estrela. Onde um único SDI está acoplado ao nó central da rede. Esta solução permite que o SDI monitorize todos os segmentos ligados a esse nó central de uma forma iterativa. Detalhes de implementação e implicações operacionais são também analisados. Palavras Chave: Sistemas de Detecção de Intrusos, Análise de Segurança, Redes de Alta Velocidade, Redes baseadas em Comutadores de Pacotes.
ii
Acknowledgements
I would like express my gratitude to the supervising professors Fernando Mira da Silva and Carlos Ribeiro for their support, critical judgment and expertise. And from whom I learnt a lot. Fellow coworkers at CIIST, Jorge Matias, Miguel Cabeça, Cláudio Martins and André Regateiro, for their pointers and overall good humor. They allowed for a great working environment. And finally my family and girlfriend, Diana Nunes, for being there when troubles emerged and for letting me spend so much time around my work.
iii
Table of Contents
Abstract
i
Resumo
ii
Acknowledgements
iii
Table of Contents
v
List of Figures
vi
List of Tables
vii
List of Acronyms
viii
1 Introduction
1
1.1 Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
1.3 Structure of this thesis
3
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 Context/Area
4
2.1 Intrusion Detection Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
2.1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
2.2 Different Types of Intrusion Detection Systems . . . . . . . . . . . . . . . . . . . . . . . . . .
5
2.2.1 Host-based Intrusion Detection Systems . . . . . . . . . . . . . . . . . . . . . . . . . .
7
2.2.2 Network-based Intrusion Detection Systems . . . . . . . . . . . . . . . . . . . . . . . .
9
3 Network Topologies 3.1 IDS Placement
13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
iv
3.1.1 Basic Network Setups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
3.1.2 Larger Network Setups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
4 Port hopping
20
4.1 Networks with Centralized Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
4.2 Port hopper approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
4.3 Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
4.4 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
5 Implementation
24
5.1 Software Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
5.1.1 IDS integration with the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
5.2 IDS Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
5.2.1 Signature Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28
6 Evaluation
32
6.1 Design Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
32
6.2 Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33
6.3 Monitoring Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33
7 Conclusion
35
7.1 Achieved Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
7.2 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
7.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
36
Bibliography
37
Appendix One
40
.1
Example configuration file for the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
.2
Example interaction with the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
42
Appendix Two .3
Packet information displayed by Basic Analysis and Security Engine (BASE) . . . . . . . . . .
44 44
v
List of Figures
1.1 Growth in number of incidents reported to the CERT/CC . . . . . . . . . . . . . . . . . . . . .
1
1.2 Attack sophistication versus intruder knowledge (reproduced from [CER06]). . . . . . . . . . .
2
3.1 A common network topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14
3.2 Using a hub/tap to copy traffic to the IDS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
3.3 Schematic representation of a network Tap. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
3.4 Using the switch port mirroring capabilities to copy traffic to the IDS. . . . . . . . . . . . . . . .
17
3.5 Consolidating the taps output with an application-switch which is, in turn, balancing the load to two IDSs.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
4.1 Example of a network with a star topology, where the IDS is connected directly to the central node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Coupling of the IDS with the central router.
21
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
5.1 Network topology of Instituto Superior Técnico (reproduced from [Mic06]). . . . . . . . . . . .
24
5.2 Structural view of the software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
vi
List of Tables
2.1 Summary of IDS properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
5.1 Top three signatures at the end of the first day. . . . . . . . . . . . . . . . . . . . . . . . . . .
29
5.2 Top four signatures at the end of the second day. . . . . . . . . . . . . . . . . . . . . . . . . .
30
vii
List of Acronyms IDS
Intrusion Detection System
CERT/CC Computer Emergency Response Team Coordination Center DoS
Denial-of-Service
TCP
Transmission Control Protocol
UDP
User Datagram Protocol
OS
Operating System
HIDS
Host-based Intrusion Detection System
NIDS
Network-based Intrusion Detection System
AIDS
Application-based Intrusion Detection System
SIDS
Stack-based Intrusion Detection System
AIDE
Advanced Intrusion Detection Environment
MAC
Media Access Control
SPAN
Switch Port Analyzer
CRC
Cyclic Redundancy Check
VLAN
Virtual Local Area Network
GPL
GNU General Public License
XML
Extensible Markup Language
IP
Internet Protocol
PDA
Personal Digital Assistant
SSH
Secure Shell
SNMP
Simple Network Management Protocol
CPU
Central Processing Unit
OID
Object Identifier
MIB
Management Information Base
IST
Instituto Superior Técnico
BASE
Basic Analysis and Security Engine
CIIST
Centro de Informática do Instituto Superior Técnico
ICMP
Internet Control Message Protocol
DNS
Domain Name System
P2P
Peer-to-Peer viii
Chapter 1
Introduction This document presents a new solution in the area of Intrusion Detection System placement, and discusses an actual implementation done at Instituto Superior Técnico (IST). This chapter intends to provide the motivation behind this solution, as well as describe its objectives.
1.1
Context
Over the past two decades, with the rapid growth of the Internet — which now counts with more than 100 million sites [Net06] — companies have been forced to change the way they do business. To keep up with new Internet-centric companies or simply to still be competitive, many companies have had to alter their business process to accommodate this new means of operation and communication. 160000 137529
140000
Number of Incidents
120000
100000 82094 80000 55100
60000
40000
31756
20000 1334
2340
2412
2573
2134
1993
1994
1995
1996
1997
3734
9852
0 1998
1999
2000
2001
2002
2003
Years
Figure 1.1: Growth in number of incidents reported to the CERT/CC
1
However, along with this growth, the number of attacks to Internet sites has also increased dramatically. As Fig. 1.1 [Las03] denotes, between 2000–2003, the number of incidents reported to the Computer Emergency Response Team Coordination Center (CERT/CC) grew around 400%. There are a few factors that contribute to this astonishing rate. First, there is the continuously publication of exploits and vulnerabilities on the Internet as they are discovered. Secondly, there is a profusion of intrusion tools and automated scripts available that duplicate known methods of attack. These two factors combined allow for practically anyone with little technical knowledge to be able to perform an attack. Consequently, the number of sophisticated attacks has been increasing. Fig. 1.2 illustrates this point [CER06].
HIGH Staged “Stealth” / advanced scanning techniques
Intruder Knowledge
Automated probes/scans Sniffers Sweepers Back doors Disabling audits
Tools
Auto Coordinated Cross site scripting Distributed attack tools
www attacks GUI Packet spoofing denial of service Network mgmt. diagnostics
Hijacking sessions Burglaries Exploiting known vulnerabilities Password cracking
Attack Sophistication
Self-replicating code Password guessing
LOW 1980
1985
Intruders 1990
1995
2000
Figure 1.2: Attack sophistication versus intruder knowledge (reproduced from [CER06]).
1.2
Objectives
The main goal of the proposal described in this document is to provide a cost-effective solution to IDS placement for networks with a star topology. The goals of this proposal are the following:
• Make use of existing technologies and freely available tools, as well as, require few hardware to implement, thus keeping costs at a minimum.
• To provide a flexible solution, such that it is feasible for all networks with a star topology and be adjustable for new and changing environments.
• To provide valuable information, namely statistics, in a format legible to any system administrator, preferably through a web interface.
2
1.3
Structure of this thesis
In chapter 2, an introduction to IDSs is presented and the state of the art is reviewed. In chapter 3, the problem of IDS placement is exposed. In this chapter several solutions to IDS placement are presented along with their benefits and drawbacks. Chapter 4 describes the proposed solution, and in chapter 5 its implementation at IST network backbone is detailed. Finally chapter 6 presents an evaluation of the proposed solution together with its implementation, chapter 7 concludes and presents future work.
3
Chapter 2
Context/Area This chapter makes a brief introduction to the intrusion detection field. It reviews the art of intrusion detection, from common terminology to the differences of the two main types of IDSs – Host-based Intrusion Detection System (HIDS) and Network-based Intrusion Detection System (NIDS).
2.1
Intrusion Detection Systems
Although the word "intrusion" might suggest a successful attack, Intrusion Detection Systems are used to detect anomalies, regardless of them being intentional or not. There are several definitions for "intrusion detection", but one widely accepted is presented in [Ric05]: "Intrusion detection is the methodology by which undesirable or aberrant activity is detected on a host or a network." Therefore, opposite to what one might assume, an "intrusion" is not just an attack but a much more general concept. All undesirable or abnormal activity might be considered an intrusion, being it planned or not. A Denial-of-Service (DoS) [PN98] attack, an user infected by a virus that is using the local mail server to send spam, or a buggy Transmission Control Protocol (TCP) stack in an Operating System (OS) that is incorrectly fragmenting packets are a few examples of possible intrusions. As the name denotes, an IDS is a system to detect intrusions such as those described above. These systems can generally be divided in three components:
Sensors Sensors are responsible for collecting data. Normally, the inputs of a sensor are network packets, log files or system call traces, but can be anything that could present evidence of an intrusion. The data collected is then passed on to the analyzer. Analyzer Analyzers, also referred to solely as "Engines", have the role of determining if there was an intrusion. To do that, the analyzer records the data from the sensors and based on a set of rules applied to the data infers whether or not there was an intrusion. The output of an analyzer is an alarm or action to countermeasure the intrusion.
4
User Interface The user interface enables the IDS administrator to monitor the analyzer output and control the behavior of the system, by adding or removing rules, tuning the sensors, etc.
2.1.1
Terminology
Throughout the literature in this field, there is some lack of consistency concerning the terms used and their meaning. The purpose of this section to clarify the concepts behind the terms used and to provide a clear terminology to be used in the remaining of this document.
• Vulnerability A vulnerability refers to a weakness in a system that may allow an attacker to perform unintended or unanticipated actions. Vulnerabilities may result from bugs or design flaws.
• Exploit An exploit is the process, usually by the means of a piece of software, that takes advantage of a vulnerability to perform an action that violates a security policy.
• Attack An attack is a set of actions perpetrated by the intruder/attacker to reach a specific goal. These actions may involve one or more security breaches.
• Intrusion As hinted in the previous section 2.1, an intrusion isn’t necessarily a deliberate attack; anything that prevents the normal functioning of a given service is considered an intrusion.
2.2
Different Types of Intrusion Detection Systems
Different IDSs are characterized by different sets of properties [Zúq06]. In this section, an overview of these properties is described. Table 2.1 is a summary of the most common ones. Because of it is importance to the field of intrusion detection in general and specifically to this document, the "source of events" property deserves a thorough explanation about the inherited characteristics of each of the possible values. That explanation is given in the next two sections.
Detection Method IDSs can detect intrusions based on two different methods: knowledge-based detection and behavior-based detection. With the first one, also called signature-based, the system analyses the data from the sensors and tries to match it to a known pattern, also known as a signature. These signatures have to be updated as new methods of attack are discovered.
5
Property
Value Knowledge-based
Detection Method Behavior-Based Host-based Event Source
Network-based Hybrid Real time
Detection Instance
Virtual real time A posteriori Active
Reaction Passive Solo Type of analysis Co-operative Table 2.1: Summary of IDS properties.
On the one hand, this method allows for a fast deployment as it is fairly easy to understand and implement, and it generates a low number of false positives. On the other hand, these systems depend solely on the signatures they know about, rendering them useless for unknown methods of attack. They are also more easily circumvented by obfuscation and evasion attempts by the attacker. With behavior-based detection, the system has to undergo a training period, with the purpose of establishing a baseline, that is, to establish what is the typical behavior. Later, when in production, the system uses that baseline as a reference to detect abnormal behavior. This class of methods relies on machine learning in order to generate traps or alarms. Event Source This property has to do with the role of the sensors. If the sensors are collecting data from a host, such as system processes, Central Processing Unit (CPU) usage, file access, etc, it is a hostbased system. If the sensors are collecting network data (typically by having its network interface in a promiscuous mode), it is a network-based system. This difference is very important and entails a very different set of abilities for each of these systems, so they deserve their own subsection. Subsections 2.2.1 and 2.2.2 explain in more detail the differences between host-based and network-based systems, respectively. Further to these, hybrid systems, as the name implies, means that the two types are mixed. For instance, host-based intrusion detection software that also scans network packets destined to the host where it is installed is a typical example of a hybrid system. Note, however, that the distinction between host-based and network-based is not unanimous in the field. Some authors, such as [Lai00], argue that network-based implies that the system has a dedicated network interface collecting network packets that pass through the attached segment. There are also those, such as [Sin05], that define a hybrid IDS as one that has both knowledge-based and behaviorbased detection methods. 6
It is clear that the field is still growing and the nomenclature is still not fully stable and mature. Detection Instance This property defines when the intrusion is discovered. Real time means that the intrusion is discovered as it happens. Although this would be the optimal scenario, it is very difficult to achieve due to the complexity of attacks. Virtual real time means that the intrusion is detected shortly after it took place, enabling the system administrator to try and isolate the affected systems to prevent more damage or even a second intrusion. A posteriori means that the attack is detected well after it took place. Normally, they are discovered based on log files. This is normally used for audit purposes. Reaction A system can be active or passive. Basically, passive systems only generate alerts, being it an e-mail, text message, a red dot on the screen or any other type of alarm. Active systems, on the other hand, can try to isolate the systems that are being/were attacked or even try to counter attack. For example, the system can try to block the attackers IP address with a firewall rule, or send RST packets to a system that is being SYN flooded. Although some systems might allow one to respond with other attacks, going beyond the defending techniques, like those mentioned in the previous paragraph, is not recommended. Counter attacking is not the best choice since it may be as illegal as the attack itself, and, secondly, because the system might be wrong about the origin of the attack. Type of analysis Solo analysis means that the system is analyzing its data in an isolated fashion. This doesn’t mean that there is only one IDS (it is very common to have more than one IDS), it only means that they work independently. Co-operative analysis means that the IDS is a distributed system that employs many nodes that work together in analyzing their data. These are very uncommon due to their complexity and cost.
2.2.1
Host-based Intrusion Detection Systems
As the name implies, host-based IDS run on the host that is providing one or more services. The sensors in this type of IDS collect data from various log files. These log files are maintained by the OS, and the various applications that are running to provide a given service. Therefore, HIDS depend on the applications ability to generate log files that are verbose enough to be useful. Log file verbosity, although necessary, leads to very large log files that difficult the task of analyzing their content. Log files might be checked continuously or periodically. However, since once the application writes to a log the corresponding fact has already occurred, the detection instance of host-based IDS is normally virtual real time. As already stated, the nomenclature for the field is not yet unanimous. Some authors, like [BT06] and [ACF+ 00], further divide the realm of host-based IDS to accommodate for Application-based Intrusion Detection System (AIDS). They define AIDSs as systems designed to monitor a specific application. They are a specialized kind of HIDS dedicated to monitor a single or small set of specific applications. 7
2.2.1.1
Strengths of HIDS
• Attack verification The tight relationship of HIDSs to log files enables these systems to accurately verify the successfulness of an attack (to that system). It may be difficult for a Network-based IDS to know whether or not an attack was successful.
• System specific activity Since HIDSs run on the host providing the services, they allow for better fine-tuning of what needs to be monitored in respect to the services running. Furthermore, they allow monitoring activity that would be impossible for network-based IDS to monitor. For example, adding or removing user accounts, improper file access or even log in or log out attempts from a person sitting at the keyboard.
• Encrypted and switched environments In encrypted environments, where the encryption is done inside the protocol stack, network-based IDSs are rendered almost useless. Host-based IDSs, however, are oblivious to this matter, being that their main source of events are log files. Another important aspect is that in many switched environments, HIDSs were adopted due to difficulty in getting the required coverage by a NIDS in such an environment. The numerous segments in a switched network makes it difficult and costly to deploy a solution based on NIDSs, even if the switches are able to do port mirroring. Note that this document proposes a solution for networks with a star topology that tries to avoid this limitation.
• No Additional Hardware Since host-based IDS software runs on the host, no additional machines are required, which helps to bring down the overall cost.
2.2.1.2
Weaknesses of HIDS
• Source Identification Many application’s log files, usually, do not contain information relevant to whom caused a given event. In such cases, identifying the attacker may be difficult.
• Operation Difficulties Log files tend to be huge in size, and often difficult to analyze. Thus the computational resources required for the host-based analyzer might hurt the performance of the services running on that host.
8
2.2.1.3
Publicly Available Software
There are several free software that perform host-based intrusion detection, including: Open Source Tripwire [Tri06], Advanced Intrusion Detection Environment [AID06], Tiger [Tig06], OSSEC [Cid06], LogSentry [Des06], HostSentry [Des06], etc. A brief description for two of these software is given to demonstrate different techniques for host-based intrusion detection.
• Advanced Intrusion Detection Environment Advanced Intrusion Detection Environment (AIDE), similar to the Open Source Tripwire, is a file integrity checker. AIDE constructs a database of the files specified in the AIDE’s configuration file. This database should be created even before the system is connected to the network, while it is known to be secure. Besides cryptographic checksums, AIDE’s database stores various file attributes including: permissions, inode number, user, group, file size, mtime and ctime, atime, growing size, number of links and link name. The database has to be kept in a secure place to stop the intruder from manipulating it, as well as AIDE’s binary executable. Although it is possible to manipulate attributes such as files dates, it is very difficult to manipulate cryptographic checksums like md5. These checksums can then be compared with subsequent recomputations of signatures for the same files. A difference in this comparisons indicates, with a high degree of confidence, which files have been changed.
• OSSEC Besides file integrity checking, OSSEC can also detect rootkits1 and do log analysis for several common services such as Bind, Apache, Squid and many others. For rootkit analysis, this package uses a list of rootkit signatures, which are then used while scanning the systems looking for matches. It can also check for the presence of unknown rootkits and kernel level ones using anomaly detection. Log files are analyzed based on a set of rules specified in Extensible Markup Language (XML) files. Each file concerns a specific log from a specific application (OSSEC web-site already contains a list of rule sets for many applications), and it contains which warnings in the application generated log file should be monitored. In addition, OSSEC can also be configured to run in a server-to-client environment. Hosts which are to be monitored are installed with an agent that can then forward events to a centralized server, which analyses the results and determines what action should be taken.
2.2.2
Network-based Intrusion Detection Systems
Although network-based Intrusion Detection Systems can monitor only network traffic destined to the host where the software is installed, they are typically machines dedicated to perform intrusion detection. Their 1
A rootkit is a set of software tools intended to conceal running processes, files or system data from the operating system.
9
main characteristic is that their source of events is the raw packet data flowing through the network segment where they are attached. They are able to collect this data by setting their network interfaces in a promiscuous mode, which means that all packet flow, whatever the source or destination, will be captured by the system sensor. Another important characteristic is the detection method used. On one end, using behavior-based detection, the software looks for activity that is different relative to a known standard behavior — the baseline. This means that the system must be trained for a period of time first. Although training such a system might be a difficult task, it is a very important step since the number of false positives in these systems is related to the quality of the training. The advantage of behavior-based detection is that the system isn’t trained to look for a specific set of attacks, allowing it to be more effective on new forms of attack. This form of detection is also known as statistical-based detection and relies on machine learning techniques. On the other end of detection methods is the knowledge-based detection. In this form the system compares the data collected with a set of signatures (patterns) of already known attacks. This is also referred to as pattern-matching and has an added benefit of having a low number of false positives. On the one hand, this form of detection is easy to understand and deploy, but on the other, it relies on the quality of the signature database and is also more easily fooled by attackers. It is also common for NIDSs to have their sensors scattered throughout the network, collecting data from various network segments — chapter 3 discusses this. Recently, a new breed of IDSs called Stack-based Intrusion Detection System (SIDS) [BT06] appeared, taking one step further to packet monitoring. These systems integrate with the OS network protocol stack monitoring the flow of packets as they traverse the layers of the stack, making it possible to take action (by removing packets, for example) before the packets reach the application. However, due to the immaturity of this technology, this type of IDS will not be discussed in this document.
2.2.2.1
Strengths of NIDS
• Cost of Ownership Although the initial investment might be larger, network-based IDS allow for strategic deployment where they can cover network traffic destined to numerous systems that need to be protected. This means that, instead of managing several intrusion detection software deployed on those systems (as one would have to, using HIDS), one only has to manage one system.
• Packet Analysis Because NIDSs analyze raw packets, they have access to the packet headers, allowing them to detect many of today DoS attacks, such as the LAND or the TearDrop attack [Jor02], even in an encrypted environment. On an unencrypted environment, NIDSs can also examine the payload of the packet and detect attempts of attack to systems that may not be vulnerable to that attack. This broader view of the network is a great advantage over HIDS.
10
• Evidence Removal Since these systems capture and analyze live network traffic, an attacker cannot remove his trail. With a HIDS an attacker might know which log files to tamper with, enabling him to remove his evidence. The data captured by the NIDS is also useful to help lead to attacker identification.
• Real Time Detection and Response As previously stated, NIDS analyze data in real time, which means they detect suspicious activity as it is occurring. As such, it can provide a faster response and/or notification. For example, a TCP based DoS attack might be efficiently stopped by having the IDS send TCP reset packets to the sender.
• Operating System Independence The network-based IDS is not dependent on the host Operating System for its data source, as is the host-based IDS. Because HIDS use log files, which are managed by the OS and applications running on that OS, they are highly dependent on these software.
2.2.2.2
Weaknesses of NIDS
• Maintenance Whether dealing with behavior-based NIDS or knowledge-based NIDS maintaining is crucial to the system’s effectiveness. The former needs periodic retraining to accompany changes in normal behavior; and the latter needs frequent signature updates to be aware of new types of attacks.
2.2.2.3
Publicly Available Software
There are a few freely available network-based intrusion detection such as Snort [Sou07] and PortSentry [Sen] and Scanlogd [Des06]. A brief description follows.
• Snort Snort is probably the most well known open-source tool for network-based intrusion detection. Snort is capable of performing real-time traffic analysis and packet logging on IP networks. It can perform protocol analysis, content searching/matching and can be used to detect a variety of attacks and probes, such as buffer overflows, stealth port scans, OS fingerprinting attempts, amongst other features. It can also be configured to have an active reaction. Snort uses a set of signatures as a data source for its pattern matching engine. The fact that these set of signatures can also contain regular expressions, and that the engine is designed with a plug-in architecture make this software very flexible. Snort was the chosen software to test the solution proposed in this document.
• Scanlogd
11
Scanlogd is a TCP port scan detection tool. It does its job by reading raw Internet Protocol (IP) packets that arrive in the host machine where it is running. If a source address sends multiple packets to different ports in a short time, it will log that event.
• PortSentry Besides being a port scan detection tool, it can also block these scans. PortSentry listens on a series of unused ports, and if a remote host attempts to connect, its IP address is added to a black list, and can not continue its scan or make any other TCP requests. Blocking is performed either through the appropriate firewall software (ipchains or iptables on Linux, ipfw on *BSD) or through the routing table (the route utility on most UNIX flavors).
12
Chapter 3
Network Topologies In this chapter, the problem of placing and Network-based Intrusion Detection System on a network is discussed. This chapter starts by describing the problems and limitation s of simple solutions for small networks. At the same time it introduces the basic issues that compose the problem of IDS placement. The chapter ends with a description of a solution for larger networks which serves as a comparative base to the proposal presented in chapter 4.
3.1
IDS Placement
IDS placement is one of the most important aspects in the design of a secure network infrastructure. The balance between monitoring coverage and allocated resources is not an easy subject. This is specially true when dealing with Network-based IDS. For Host-based IDSs, the decision is relatively easier. Host-based HIDS should, usually, be placed on the hosts that provide crucial services to the organization. The following sections discuss the rationale behind the design of the solution proposed in chapter 4. Starting from small and rather simplistic networks and building up to large and more complex networks, a description of a common solution about the placement of network-based IDS for each one is given, along with its pros and cons. Because the discussion about HIDS placement is not relevant to the description of this rationale, it will be left out. Furthermore, since this work is centered on NIDS, for the remaining of this document, the names IDS and NIDS will be used interchangeably.
3.1.1
Basic Network Setups
In this section, a simple network setup is used to introduce the main problems of NIDS placement. For now, the focus is on monitoring the traffic coming from the Internet, which is the main concern in small networks. Section 3.1.2 will delve into monitoring traffic inside larger networks.
13
Usually small to medium-sized networks follow the setup presented in Fig. 3.1. Possibly, instead of the switch hardware, there might be an ethernet hub. However, this would only be feasible for networks that generate small amounts of traffic, since using a shared-medium implies that only one device connected to the hub can successfully transmit at a time.
Gateway
Switch Segment A
Segment C
Segment B
Figure 3.1: A common network topology.
Using a shared-medium also implies that all devices connected to the hub will listen to all the networks traffic that flows through it. In this scenario an IDS may simply be connected to the hub to be able to monitor the entire network (assuming that there are no switches on either segments). This scenario is, however, rather simplistic. Assuming a 10-port 100 mbps hub, each device connected to the hub would have an average of 10 mbps throughput (100 mbps / 10 devices = 10 mbps), which is further degraded by collisions1 . Moreover, nowadays hubs are almost obsolete and seldom used in enterprise environments. When using a network switch, packets arriving in one port are sent to the port where the packet’s target might be found. Network switches work at the Data Link Layer of the OSI model. Packet switching is done based on the packets Media Access Control (MAC) addresses. If this target’s MAC address is not yet known, then the switch will broadcast the packet to all ports except the port from which the request originated. When the target’s response arrives, the switch makes an association between the port and the target’s MAC address. This information will be used to route all packets with that destination to that port only. In this scenario, simply connecting a NIDS to a port on the switch won’t suffice, since all traffic routed to the other ports would not be monitored, namely the traffic going to and from the gateway. There are mainly three solutions that allow one to use a NIDS to monitor traffic leaving and entering the network. Each of these solutions will be discussed in the following sections. Note that, in the following discussion, it is assumed that the network is trusted (which is generally the case for small to medium sized networks), and therefore monitoring traffic between network devices is not strictly required. 1
Collisions: A hub echoes packets arriving in one port to all the remaining ports. If two machines transmit at the same time, their
packets will collide, causing both of the machines to retransmit.
14
3.1.1.1
Hubs
Due to its properties, using a hub between the switch and the gateway, as shown in Fig. 3.2, allows all incoming and outgoing traffic to be copied to the IDS. This is a simple and inexpensive way to solve this problem. But, as stated before, hubs can easily degrade network performance, therefore this solution is presented merely as an example.
Gateway
IDS
HUB / TAP Switch
Figure 3.2: Using a hub/tap to copy traffic to the IDS.
This solution presents the following advantages:
• Hubs are usually limited to 10 mbps ports, therefore limiting behavior. • It is easily deployed. Deployment of the hub basically consists of connecting the network cables from the gateway, switch and IDS;
• The link between the IDS and the hub is bidirectional, allowing the IDS to be active; • No reconfigurations are needed on the remaining network devices, such as switches, routers or firewalls; • 4-port hubs are very economical. However this approach has the following drawbacks:
• Due to the properties of shared media, throughput is degraded with an increase of network traffic. Specially if in a full-duplex environment, collisions will severely damage performance;
• The previous point is aggravated if the management of the IDS is done via the hub. This would increase the number of collision and further degrade the performance of the traffic between the gateway and the switch;
• Low cost hubs tend to be prone to failure.
3.1.1.2
Taps
The tap solution [Lai00] is very similar to the hub (see Fig. 3.2). A network tap is a hardware device which provides a way to access the data flowing across a link. Taps have at least three ports; the A and B port are used to establish the connection between the two network segments; the remaining ports, also called
15
monitor ports, are used to connect the IDS. The main difference between taps and hubs is that the monitor ports are unidirectional, they are only used to copy off traffic that flows from port A to port B, or vice versa. One important aspect to note, is that if the network link is an 100 mbps full-duplex link, then the aggregate traffic comprising of the traffic in both directions would be 200 mbps. This is a problem if the tap has only one monitor port. In these cases, taps usually have two ports that monitor each direction of the traffic (Fig. 3.3). The IDS can use channel bonding2 if loss-less traffic monitoring is a strict system requirement. Power A Mirror 1
B Mirror 2
Figure 3.3: Schematic representation of a network Tap.
Furthermore, taps don’t interfere with the communication link between the two segments. Also, good taps are fault tolerant, since the connection between ports A and B is hardwired in, which means that in case of a power failure the communication link between the two connected segments will not be broken. In short, taps brings the following advantages:
• Taps can be fault tolerant; if power fails, the connection between the switch and the gateway remains functional;
• Taps have no impact on the flow of traffic; • Since taps don’t interfere with the network infrastructure, changes to the IDS infrastructure (section 3.1.2 will discuss this subject) can be made with no consequences to the former;
• Since the link between the tap and the IDS is unidirectional, no connections can be made to and from the IDS, which protects it from attacks.
• The IDS is allowed to monitor errors such as undersized and oversized packets, or packets with a bad Cyclic Redundancy Check (CRC). The following disadvantages are present:
• Taps can be expensive; • The connected IDS can’t have an active reaction. For example, it cannot terminate a suspicious TCP session by sending RST packets to the participants because, as stated, the link is unidirectional.
• To monitor traffic on both direction when using a full-duplex connection, extra modifications are needed. 2
Channel bonding is an arrangement in which two or more network interfaces on a host computer are combined for redundancy or
increased throughput.
16
3.1.1.3
Port Mirroring
This solution depends on the switch capabilities, namely if port mirroring is an available feature. It consists on replicating data from one or more ports onto a single port, referred to as the monitoring port. This feature is also known as: Monitoring Port, Spanning Port, Switch Port Analyzer (SPAN) port and Link Mode port. In Fig. 3.4 the switch can be configured to copy the traffic passing through the port where the gateway is connected, to the monitoring port. Both directions (TX and RX) of the traffic are copied, thus achieving the same monitoring capabilities of the previous solutions.
Gateway
IDS
Switch Monitoring Port
Figure 3.4: Using the switch port mirroring capabilities to copy traffic to the IDS.
The advantages of using this solution are:
• If the switch is able to do port mirroring, installation is straightforward. The core network infrastructure remains the same;
• No additional hardware is needed to install the IDS or to manage it; • Some switches allow for bidirectional traffic on the monitor port, thus allowing the IDS to have an active reaction. The disadvantages of using this solution are:
• Usually, a switch can only have one monitor port. This means that, to monitor more than one port, one must span a range of ports or an entire Virtual Local Area Network (VLAN);
• Following the previous point, spanning more than one port is not feasible as the mirrored port can quickly become overloaded;
• There could be an impact on the switch’s performance; • Usually, switches only allow the monitor port to be unidirectional.
3.1.2
Larger Network Setups
For larger networks, such as those created by universities for example, there is little control over who has access to the network. There are several hundred of installed workstations, various points where laptops can be connected to the network and, more recently, wireless access points for users to connect their laptops or Personal Digital Assistants (PDAs). 17
With this scenario, simply monitoring the Internet connection is not enough, since attacks perpetrated from inside the network would not be detected. This is an important issue, specially when users can bring their own machines to the network and system managers have no control over the running software and OS configuration. Figure 3.5 3 shows a simple example of a standard setup required to monitor more than one network segment. In this topology, two network segments are connected to a gateway through a central switch / router. In order to fully monitor this network, its segments are connected by a tap to an application-switch which is in turn connected to one or more IDSs.
IDS
IDS Gateway
Application-switch
Gateway Tap Switch
Segment 1 Tap
Segment 2 Tap
Figure 3.5: Consolidating the taps output with an application-switch which is, in turn, balancing the load to two IDSs.
Normal packet switching involves looking at a packet headers, whether these are layer-2 headers, in the case of switching based on the MAC address, or layer-3 headers, in case of switching based on the IP address. In either case, the forwarding decision can be made by looking at a single packet. An application-switch [Kes01] differs from conventional switches in the sense that it makes its decisions based on flows rather than single packets. Because IDS machines must monitor both directions of a flow, this switch awareness is crucial. The notion of flows allows the application-switch to send both TX and RX streams to one IDS — this is called flow mirroring. The application-switch can effectively balance the traffic load collected from the taps to various IDS machines, radically diminishing the possibilities of overloading the port where the IDS is connected. 3
Note: Actually there should be two lines connecting the taps and the application-switch, one for each traffic direction (RX and TX).
18
Of course, one could exclude the application-switch and use several NIDS, one for each tap. But this would be much more costly. The additional advantages of using an application-switch are:
• The load balancing capabilities of the application-switch are able to prevent overloading a single port when traffic increases;
• Consolidating various taps into one application-switch allows to reduce the number of IDSs, lowering the cost;
• As stated in the previous section, the IDS is protected from connections from the monitored operational network;
• Several IDSs can be connected to the application-switch to monitor all the traffic, thus improving scalability. On the negative side:
• Application-switches can be expensive and due to their differences to normal switches, they require specific expertise to operate;
• There must be a tap for each network segment that is to be monitored.
3.2
Discussion
As explained before, conventional IDS setups either are too limited to cope with large network requirements, or too complex to provide a manageable solution. In the following chapter, a different solution is presented, in respect to IDS placement, for networks with a star topology. This solution tries to attain a compromise between monitoring capabilities and manageability.
19
Chapter 4
Port hopping This chapter describes the proposed solution for networks with a star topology, also referred as networks with centralized routing. This chapter also introduces the necessary technologies and the requirement of the new solution. Implementation details are discussed in chapter 5.
4.1
Networks with Centralized Routing
Networks which have a star topology are one of the most commons found in medium sized enterprises. In this topology, a central node acts as a router for all the attached network segments. This node may be a switch, a router or even a computer. An example of these networks is illustrated in Fig. 4.1. Note that the clouds can represent any type of sub-network, including wireless networks. For these networks, if a decision was required regarding which network segments should be monitored by a network-based IDS, a common conclusion would be to monitor all segments that are connected to the central node, or a subset of these (only those that are considered relevant or vulnerable). One could place a tap on those segments and use the approach described in section 3.1.2. However this document suggests a different approach, one that dispenses the use of taps and an application-switch altogether.
4.2
Port hopper approach
Common routers and switches facilitate their management by either a Secure Shell (SSH) connection (older devices use telnet) and/or through Simple Network Management Protocol (SNMP) queries. The reasoning behind the solution is to let the IDS monitor all, or any subset, of the segments connected to the router, one by one, in a round robin fashion. If any suspicious activity is detected on the port being monitored, the IDS will stick to that port and emit appropriate warnings or take the appropriate actions. Once the suspicious activity
20
IDS Gateway
Switch / Router
Figure 4.1: Example of a network with a star topology, where the IDS is connected directly to the central node.
has terminated for a configurable amount of time or by an early administrator’s decision, the IDS will return to its normal cycle of operation. Another feature of many of modern routers is that they allow for mirroring more than one port at the same time. That is, one can aggregate the traffic from many source ports and send it to a single destination port – the mirroring port. This feature provides more flexibility to the system. It could be used to permanently monitor one port that is connected to a sensitive segment, whilst hopping between the other ports as mentioned previously. It can also be used when suspicious activity is detected. Instead of sticking to the port where that activity was detected, as mentioned above, blocking the system from monitoring the other ports, the system can keep monitoring that port and still continue normal operation.
4.3
Value
Attacks aren’t instantaneous. Port scanning, network topology discovery, OS fingerprinting, DoS attempts, running exploits, etc, are all time consuming activities. Attacks consist of several steps, many of which can take a considerable amount of time. Therefore, sampling the network traffic from each of the segments attached to the central node in a round-
21
robin fashion may be effective. Meaning that there is time for the IDS to rotate trough all the ports while the attack is in action; it is a matter of calibrating the time between hops with respect to the number of segments to monitor. However, fine tuning these parameters might prove to be a rather difficult task. There is no formula for calculating the average time an attack takes to occur, so calibrating this value must be based on testing and knowledge about current common attack times. The advantages of this solution are:
• No further hardware needed. Provided that the current central router/switch has port mirroring capabilities;
• Since the IDS is usually only monitoring one port at a time, there is a low risk of the monitoring port being overloaded.
• It is flexible. Many routers allow for mirroring of more than one port at a time, the system can be configured to keep monitoring the port where the gateway is connected, or keep monitoring the port where suspicious activity was detected and continue to monitor the other ports in a round-robin fashion. And the drawbacks are:
• Only segments attached to the central node can be monitored; • Can’t monitor all segments all the time, due to the risk of overloading the monitoring port.
4.4
Requirements
This approach requires developing the software that will control the router and integrate with the IDS; in this case, it will be Snort, which was referred in section 2.2.2.1. To be able to interact with the router, in case it doesn’t allow for the device connected to the monitoring port to transmit, the IDS must have two network cards, each connected to the router (Fig. 4.2). One for the traffic being mirrored and the other to allow the IDS to communicate with the router and control which port is being mirrored.
Switch / Router
IDS
Control link Monitoring link
Figure 4.2: Coupling of the IDS with the central router.
On the one hand, the software to be developed has to interact with the router, either trough SNMP queries or trough an SSH connection. In the latter case, the software may use Expect [Lib06] to automate the interaction.
22
On the other hand, the software must also take into account the intrusion detection software (in this case Snort) status, more precisely the alarms it generates.
23
Chapter 5
Implementation The implementation of the aforementioned proposal revolves around two main subsystems: the software to interact with the router and the IDS itself. The system was developed and tested at IST’s network backbone: Figure 5.1 [Mic06] illustrates its network topology. The system is coupled with the core router, which is clearly depicted in the center of the illustration below, all surrounding labels refer to other switches / routers.
GB-CLUSTERCI2 GB-CLUSTERCI
GB-RNL
GB-MecI
GB-TorreN
GB-OBRAS
GB-TM
GB-CIENT
GB-PGRAD
OmniStack 8008
GB-SECRET GB-BIBLIO
GB-Ciencia
CORE
GBcobra
GB-Civil2
CIIST-132_2
GB-MecanicaII
GB-Electro
CIIST-132_1
GB-Minas GB-Civil
GB-Backup2
GB-MecIII
GB-ASocial
GB-Cplx GB-BIBQUIM
HP-Electro enterasys HP-Minas_2P GB-AE GB-ADIST
GB-cplx2 HP Quim-Velha
Figure 5.1: Network topology of Instituto Superior Técnico (reproduced from [Mic06]).
This chapter details the steps taken to implement each of the subsystems referred earlier. It discusses the technologies used, the overall software architecture as well as required configurations details.
24
5.1
Software Architecture
The software was written in Python [LA03] due to the available frameworks and fast development time. The following is a description of the overall architecture and an explanation of the responsibilities of each of the architectural modules illustrated in Fig. 5.2.
Router Management
User Interaction MainLoop Controller
Port Management
Figure 5.2: Structural view of the software.
All of the modules listed below are configurable through a global configuration file — port_hopper.cfg. When describing each of the modules, the configuration options for that module will be referred. An example configuration file is depicted in appendix .1.
Router Management This module is responsible for the entire interaction with the router. The interaction is done via SNMPv3 [MS05]. All the communication is encrypted and authenticated. The Python SNMP framework called pysnmp [Eti02] was adopted to perform the SNMP queries and collect the answers from those queries. Through the configuration file the user can specify the router’s IP address as well as his username and password for the SNMP service. Note that this module is tailored to work with a specific router. However, it is relatively easy to create a new module, based on the existing one, adapted to any other router with the same management capabilities. It is a matter of changing the relevant Object Identifiers (OIDs) in the source code which can be found in the router’s Management Information Base (MIB) files. Port Management This module is responsible for deciding which ports are monitored and in which sequence. It keeps two sets of ports, one for dynamic ports and the other for static ports. It also maintains a sticky port used to support the integration with the IDS (section 5.1.1). Dynamic ports are those which the system should monitor iteratively, that is, one by one in a round robin fashion. Static ports are those ports that are always monitored, hopping operations do not affect this set. They are useful for monitoring critical segments. Every time the system determines it is time to hop, either through a timeout or a user command, this 25
module is asked for a new set of ports to monitor. The set of ports returned is a union of the static port set, the new dynamic port, and the sticky port. The configuration file allows for specifying both the dynamic set and the static set. The latter must be a subset of the first. At initialization time, the static port set is subtracted from the dynamic port set, creating two disjoint sets. This invariant exists for simplicity and is kept at the runtime, adding a port to the static set entails removing it from the dynamic set and vice versa. User Interaction The software provides a command line interface to facilitate monitoring and controlling several aspects of the system. From the command line the user is able to:
• Print the current static/dynamic port sets and the system status. Printing the system status displays the currently monitored ports and the name of the segment connected to each of those ports. This name is setup by the router’s administrator and is retrieved from the router also though SNMP.
• Add and remove ports from the static set. As mentioned before, adding a port to the static set removes that port from the dynamic set and vice versa. All the changes made to the dynamic and static port sets referred in the previous paragraph are volatile. Once the system is restarted the changes will revert back to the settings specified in the configuration file.
• Force the system to hop, ignoring the current timeout and recalculating a new timeout. • Stop the system from hopping. While the system is paused it continuously monitors the ports that were being monitored prior the pause command. Obviously, the user can undo the pause. An example of a user interaction with the system is presented in appendix .2, which illustrates the output of some of the commands explained here. MainLoop Controller This is the main module as it coordinates all the others modules. Briefly, it is a loop that listens to events which can either be: a timeout; some input from the user; or an alert from the IDS. Note that the latter case, which is relative to the integration with the IDS, is discussed in more detail in section 5.1.1. Namely, how these alerts are communicated to the software and what effect do they have. This module is also responsible for managing the time between hops. On the configuration file a minimum time and a maximum time are specified. This module then computes a random timeout for each hop between those two values for each hop. Randomly choosing the timeout introduces some entropy in the system, useful for making the task of an attacker trying to discover which ports are being monitored at a given time more difficult.
5.1.1
IDS integration with the software
In chapter 4, where the proposal was described, it was mentioned that the system should listen to alerts from the intrusion detection software and keep monitoring the port where that activity was detected. The idea behind this, is that if some suspicious activity is detected on a network segment, there is a high probability that it might continue. Therefore logging the activity post receiving the alert would be useful. The 26
following is a description of how this feature was implemented and why. Ideally the system would keep monitoring the port where the suspicious activity was detected. Determining this port while only monitoring one port is trivial. However to do so when monitoring more than one port can be complicated. In fact, with the router’s mirroring feature, the packets that are transmitted to the monitoring port are just a copy of the packets passing through the monitored ports. No information about source port the packet is added to it. Thus, determining a packets source port with certainty can be very difficult. The reason why it can be difficult and not just impossible, is because there are a few workarounds one might try to overcome this problem. In the next paragraphs these workarounds are discussed. The first workaround would be to use the packets VLAN tag to try and assess which port it came from. If there is a direct mapping between VLANs and ports this is trivial. However VLANs usually span more than one port (which is, in fact, the case). In such cases one could keep a mapping between VLANs and ports (or consult the router via SNMP). Then, when a packet triggers an alert, one can intersect that packet VLAN with the VLANs associated with the currently monitored ports. Of course, if the currently monitored ports belong to the same VLAN the intersection would yield more than one port. Therefore this workaround wouldn’t be perfect. Another similar workaround would be to use the source and destination IP addresses. This would involve knowing all address blocks connected to each port, which by itself would be difficult to maintain. Moreover, having private addresses, multicast addresses and the possibility of address spoofing, the efficiency of this solution would be unacceptable. Lastly, the best solution would be to use the packets source MAC address and use the router’s bridging table through SNMP to lookup which port is associated to that address. Since the MAC address would univocally determine the relevant port, it would be sufficient. However, Snort does not store the link-layer header information in its database. Having no access to the source MAC address means this workaround would not work either. All solutions are far from perfect and somewhat complex. Therefore, the adopted solution makes some compromises in exchange for simplicity. Has already mentioned, the Port Manager module maintains one sticky port. This port is set every time suspicious activity is detected, regardless there being any other static ports. The system will keep monitoring that port for a configurable number of hops (see appendix .1), and ignore any other alerts in the mean time.
5.2
IDS Deployment
As already referred in chapter 4, the intrusion detection software used was Snort (described in section 2.2.2.3). On the one hand, installing the software is straightforward. Snort was set-up to listen to traffic on the network interface connected to the mirroring port on the router, and to use a PostgreSQL [Pos07] database as a 27
back-end. It uses this database to log packets, warnings, keep signatures, etc. On the other hand, configuring Snort can be challenging. As already mentioned in section 2.2.2.3, Snort is a signature-based IDS, therefore, it requires maintenance not just to keep the signatures updated but also to create thresholds and suppression rules to adjust to the environment. Since this is a reasonably complex task, section 5.2.1 is dedicated to it. Having Snort simply generating data and storing it in a database is not very useful by itself. A front-end is an advisable tool for the user to view the data, analyze it and extract conclusions. Ultimately, the front-end is necessary for the sake of efficiency and ease of use. The installed front-end is the BASE [Sec07]. It is a web-based front-end that allows for querying and analyzing alerts that snort generates. It works by inspecting the database where snort stores its data. Using BASE the user can obtain various statistics on the alerts, such as the most frequent alerts or most frequent destination IP addresses. Thus allowing for a quick overview of what is happening on the network. The user can also do more complex queries such as time profiles of alerts based on attack signature, destination or source IP, etc. Because BASE is working directly with the snort database, it is even possible to view the packet’s payload, TCP and User Datagram Protocol (UDP) headers that Snort stores in the database. An example output from BASE is depicted in appendix .3.
5.2.1
Signature Thresholds
A common problem with IDS deployment is that they usually generate a profusion of warnings on a fresh install. The number of warnings can be so great and repetitive that they become annoying and tiring, leading to being ignored. Such setups nullify the use of an Intrusion Detection System. For this matter, Snort comes prepared to be configured for thresholding and suppression of rules. The configuration file — threshold.conf — exists solely for this purpose. By using this feature, one can significantly reduce the number of alerts. There are four types of suppression rules: 1. Suppress: Completely suppress a signature or just suppress those with a given destination or source address. 2. Limit: Alert on the first M events during a specified time interval, then ignore events for the rest of the time interval. 3. Threshold: Alert every M times this event occurs during the time interval. 4. Limit & Threshold: Alert once per time interval after seeing M occurrences of the event, then ignore any additional events during the time interval. To determine which rules needed to be suppressed or assigned a threshold, the system was left running for periods of 24 hours. At the end of each period, the most frequent alerts were analyzed and a decision about 28
them was made. A summary of this alarms is shown in table 5.1. Note that for some of these rules the appropriate action would be, not to suppress the rule, but to configure a firewall, eliminate unencrypted traffic, etc. However on a large network with several departments administering their own sub-networks, this is simply not possible. #
Signature Name
Signature Description
%
1
SNMP request udp
This event is generated when an SNMP request con-
48
nection over UDP to an SNMP daemon is made. 2
SNMP public access udp
This event is generated when an SNMP connection
41
over UDP using the default ’public’ community is made. 3
RPC portmap proxy attempt UDP
This event is generated when an attempt is made
5
to forward a Remote Procedure Call (RPC) request through the portmapper service. Table 5.1: Top three signatures at the end of the first day.
The following is a discussion of the analysis done to each of the alerts presented in table 5.1:
Rules 1 & 2 As it is clear from table 5.1, at the end of the first 24 hours the top two alerts accounted for 89% of the total alerts generated, which at this point were already in the order of hundreds of thousands. The SNMP traffic mentioned in these alerts descriptions is coherent to the monitoring tasks done by Centro de Informática do Instituto Superior Técnico (CIIST) to their servers. These tasks use the SNMP protocol to monitor various services on the network, they are used for service health checking and to log service usage. As such, they must be run frequently. All events generated by these two rules with a source address belonging to CIIST’s VLAN were completely suppressed. Rule 3 The third alert is a false positive, since no attempt to forward a Remote Procedure Call was made. Several departments (in this case there were three: Civil, Math and Lasef) use the port mapper service on their networks. The “False Positive” section of this rule on Snort’s website mentions this: “This rule also generates an event when legitimate hosts attempt to use the proxy feature.” Because all this traffic is broadcasted, it was easy to suppress it. All events with a destination address equal to the broadcast address of those department’s networks were suppressed. The rest of the most frequent alerts were evenly matched in terms of percentage. So it was time to begin another round. Snort’s database was reset and the system was left running for another 24 hours. Table 5.2 displays the result at the end of this iteration.
Rule 1 The first alert presented occurs when a server replies that it has an open port to a port scan. In a network this large there are many servers running many services for this rule to be of any use.
29
#
Signature Name
Signature Description
%
1
(portscan) Open Port
This event is generated when a server replies an open
45
port to a port scan. 2
ICMP PING NMAP
This event is generated when an ICMP ping typically
25
generated by nmap is detected. 3
SNMP trap udp
This event is generated when an SNMP-Trap connec-
15
tion over UDP to an SNMP daemon is made. 4
ICMP Destination Unreachable
This event is generated when a router was unable to
Communication
forward a packet due to filtering and used the Internet
Prohibited
Administratively
5
Control Message Protocol to alert involved hosts. Table 5.2: Top four signatures at the end of the second day.
Consequently this alert occurs with a high frequency. To lower this frequency a threshold was imposed so the alert only occurs once per minute per source address. Rule 2 The second alert is generated by an Internet Control Message Protocol (ICMP) ping generated by the Nmap tool [ins07] which characterized by having zero payload data size. Because Nmap is a very popular tool, usually used for network reconnaissance, there is a signature just for it. However, although this might indicate the beginning of an attack, the number of false positives is just too great. Software like Kontiki and avast! antivirus are know to generate this type of packets. Furthermore, even if it is generated by the Nmap tool nothing can be said about the intents of the person behind it. For these reasons a threshold was assign to this event, it will only generate an alert if it occurs more than fifty times over one minute and with the same source IP address. This way alerting legitimate pings can be avoided, and reconnaissance attempts can be detected. Rule 3 The third rule is related to the first two rules on the first day of running the system. SNMP traps are also used as a tool for collecting statistics. For example, Wireless access points report their status to a central central server via an SNMP trap. From Snort’s website the intent of this signature is clarified: “An attacker may attempt to send this request to determine if a device is using SNMP”. Having a network with so many SNMP-capable devices such as switches, routers, servers and so many of them using traps to communicate, this rule had to be ignored. Rule 4 The last event occurs when a router sends an ICMP notification when it can’t forward a packet, which usually happens when there is a firewall involved. This event is merely informative, and common in this network. There are many web pages and services that are only accessible from inside the campus network. Attempts to reach those from the outside will be filtered. However if this event keeps occurring at a high rate, it may be indicative of an attack by an automated tool. Thus, once again, a threshold was assigned to this rule stating that it should only emit an alert if this events occurs more than 20 times in 1 minute.
30
Before ending this section it must be noted that although an effort was made to decrease the amount of warnings generated by Snort, there still remains a considerable amount of low-priority (mostly informational) warnings. Snort logs these warnings to /var/log/snort/alert (a file also checked by the MainLoop Module) together with a priority tag between 1 and 4. Priorities are ordered by severity being 1 the most severe. Thus, to avoid setting the sticky port all the time, only the most severe alerts, those with priority 1, are taken into account. Concluding this section, it is worth noting that the number of alerts went from thousands per minute to about six per minute. Note that this must be a continue process. Every time the rules are updated or some other abnormal network activity is detected, there may be a need to update these thresholds.
31
Chapter 6
Evaluation This section intends to make an evaluation of the work done. It highlights some misconceptions during design as well as the problems faced at implementation time. It also tries to make an evaluation of the system’s monitoring window time.
6.1
Design Issues
The proposal described in this document, attempts to provide a cost effective solution to IDS placement. When designing this proposal, one of the issues to conquer was the danger of overloading the mirroring port. That is why the system, in regular operation, only monitors one port at a time. The bottleneck however, is far from being the mirroring port. With a gigabit connection, which is the case, the IDS machine starts ignoring packets long before the bandwidth limit is reached. Pattern matching is still a resource intensive task, and even with a powerful machine there is only so much one machine can process. In [SSMF03] there is an in depth discussion of performance issues with Network-based Intrusion Detection Systems. One conclusion it states is that “on a nearly saturated 100 Mbit per second network link, even the best-performing system can only support a maximum of 720 header rules without losing packets”. Snort currently comes bundled with about 1900 rules. This number clearly states the limitation of this approach. This means that even when only monitoring one port, there is a chance of overloading the intrusion detection software on the monitoring machine. Although on the test environment this never occurred, it is still a possibility. A solution to this could be to use one application-switch, mentioned in section 3.1.2, to balance traffic of the monitoring port to many intrusion detection machines. Once again it is a matter of cost versus monitoring abilities. Note however, that the design is still valid. The proposal is still cost-effective when compared to the others described in chapter 3. In the design point of view, monitoring in a round robin fashion is still as effective. The bottleneck just changed from the monitoring port to the IDS. 32
6.2
Implementation Issues
All the desirable features of the proposal described in chapter 4 were implemented. And even some new ones, such as the ability to pause the rotation of ports. There were also some improvements to the proposal that were made during implementation time. The randomization of the time between hops and the ability to dynamically change the which ports should be rotated and which should be permanently monitored. On the other hand, the complexity of integrating the software with the IDS was under estimated. As it is clear from section 5.1.1 the problem of determining the original port of a mirrored packet is very intricate. Furthermore, the benefits of monitoring the port where suspicious activity was detected for an extended amount of time, are difficult to measure. All workarounds in the mentioned section are far from perfect and somewhat too complex for the potential benefits this integration would bring. Therefore a compromise had to be made, and the adopted solution reflects it.
6.3
Monitoring Window
The monitoring window is the amount of time the system should keep monitoring one port before hopping to the next port. Determining a good value for this window is not trivial. Calibrating this window based on the number of attacks for different test times is not feasible nor accurate. Firstly the frequency of attacks is unpredictable,as they follow unknown attacker’s agendas and heuristics. The occurrence of attacks is, to put it simply, chaotic. Secondly, each port is connected to a different number of machines each of them running different services and Operating Systems. Meaning that each port may have a different attractiveness to attackers. A different approach would be to use common attack durations to determine good values for the maximum and minimum window times. There ate mainly two types of attacks: structured and non-structured. Structured attacks are outline by the following steps: 1. Find/discover target(s); 2. Determine the running OS (OS fingerprinting); 3. Discover open ports (Port scanning); 4. Exploit a vulnerability. Non-structured attacks skip one or more of these steps. The most common non-structured attack is the Denial-of-Service. Steps 2 and 3 of the attack outline are usually done at the same time and are usually detected by the IDS as a port scan attempt. Together with DoS attempts these are the most prevalent attacks detected by the IDS.
33
In [MVS01] experiments showed that, during a three week-long period, 50% of DoS attacks are less than 10 minutes in duration, 80% are less than 30 minutes and 90% are less than one hour. In [RSMS03] testing revealed that the majority of scanners (IPs performing port scans) scan for 30 seconds or less. Furthermore 48% of the total scanners detected scanned for more than one hour, and 39% of scanners spent more than one day scanning. It also states that the average “intraprobe” delay, that is the average time between probes, is about 5 seconds. The longer the delay the stealthier the attack. Having these values in mind, once can extrapolate good settings for the minimum and maximum window times. There should be a bare minimum of 5 seconds (the average “intraprobe” delay) to be able to detect port scan attempts. And rotating through all the ports shouldn’t take more than 10 minutes to try to detect most DoS attempts. IST’s core router is connected to 38 different segments.
10 × 60 = 15 38
(6.1)
Dividing the 10 minutes the system should use to complete one revolution, with the 38 connected ports, yields a value of 15 seconds. Therefore, statistically speaking, good settings for the minimum and maximum window times should be near 5 and 15 seconds respectively.
34
Chapter 7
Conclusion This chapter concludes this documents. It discusses the value achieved with this solution, presents some final remarks and introduces ideas for future work. Note that this section is not meant to be based solely on facts, but also on personal judgment. As such, some of the statements made here are of a subjective nature.
7.1
Achieved Value
The use of IDSs is commonly seen as the ability to detect network intruders. And although the system is still very useful for this purpose, it can equally be as useful for many others. For example, it can be used to detect and enforce company policies, such as not allowing unencrypted communication. Signatures can easily be created to detect common unencrypted protocols such as ftp and telnet. It can also be used to detect unauthorized use of network resources, such as the illegitimate use of Peerto-Peer (P2P) networks. Although creating signatures for this type of traffic can be much more difficult, the bleeding-edge [ber07] rule set already comes with a set of rules dedicated to do this. Another use for the system is to detect network problems. Having access to all the segments one could use the system to monitor a specific traffic pattern that is causing problems. The definition of intrusion detection presented in chapter 2 as become evident throughout the work done in developing the system. And it is clear by the previous paragraphs: “Intrusion detection is the methodology by which undesirable or aberrant activity is detected on a host or a network.”
7.2
Final Remarks
For networks with a star topology this solution can be a good compromise between the investment and monitoring coverage. The main costs of this solution are related to IDS maintenance.
35
Keeping the IDS functioning properly can be strenuous task. Maintaining the rules updated and constantly refining the thresholds to adapt to new rules and traffic patterns can be a demanding task. However, there are many benefits when implementing such a solution. Among them, attack awareness, enforcing company policies or detecting unauthorized traffic and network problems. The idea is simple and so is the software required to put it in practice. Using existing technologies such as Snort and SNMP and with some lines of code, it was possible to develop a flexible system that can be of many uses to system and network administrators. Overall, besides doing what the name implies, which is intrusion detection, the system can also provide a very useful overview of what is happening on the network at a very low cost.
7.3
Future Work
As mentioned above having and IDS monitoring the entire network can bring many benefits. But when dealing with a network with the size and characteristics of the IST’s network, compromises have to be made, specially when dealing with snort suppression rules. There are just too many servers / services with different administrative departments to allow for one IDS to be tailored for everyone. Thus, a future path to take, would be to have a dedicated IDS monitoring only the central services managed by CIIST. To do this, a separate instance of Snort may be ran with the only purpose of monitoring the VLAN where those servers are running. This way Snort could be configured considering only those services, which avoids making compromises like those made in section 5.2.1. When having complete control and knowledge over the servers and services being monitored, signatures and thresholds can be specifically tailored to cater for those services. For example, the “(portscan) Open Port” alert presented in table 5.2, which was then completely suppressed, could instead be assigned a threshold. Knowing all services that should be running a threshold could be maintained that prevented alerts for known services to be generated. That coupled with a firewall that filtered services only intended for internal use would be a better compromise than completely ignoring those alerts.
36
Bibliography [ACF+ 00] Julia Allen, Alan Christie, William Fithen, John McHugh, Jed Pickel, and Ed Stoner. State of the Practice of Intrusion Detection Technologies. Technical report, Carnegie Mellon, Software Engineering Institute, January 2000. [AID06] AIDE. AIDE - Advanced Intrusion Detection Environment, 2006.
http://sourceforge.net/projects/aide – Avail. January, 2007. [ber07] Bleeding Edge Threats, 2007.
http://www.bleedingsnort.com/ – Avail. September, 2007. [BT06] Vlasti Broucek and Paul Turner. Bridging the Divide: Rising Awareness of Forensic Issues amongst Systems Administrators. School of Information Systems, University of Tasmania, Australia, 2006. [CER06] CERT/CC. Vulnerabilty Discovery: Bridging the Gap Between Analysis and Engineering, 2006.
http://www.cert.org/archive/pdf/CERTCC_Vulnerability_Discovery.pdf – Avail. December, 2006. [Cid06] Daniel Cid. OSSEC HIDS - Open Source HIDS, 2006.
http://www.ossec.net/ – Avail. January, 2007. [Des06] Solar Designer. Scanlogd, 2006.
http://www.openwall.com/scanlogd/ – Avail. January, 2007. [Eti02] Ilya Etingof. Python SNMP framework, 2002.
http://pysnmp.sourceforge.net/ – Avail. May, 2007. [ins07] insecure.org. Nmap Security Scanner, 2007.
http://insecure.org/nmap/ – Avail. August, 2007. [Jor02] Katarina Jorzic. Tracking back DDoS attacks. PhD thesis, University of Stockholm - Royal Institute of Technology, April 2002. [Kes01] Gary C. Kessler. IDS-in-Depth: Top Layer’s AppSwitch filters a copy of traffic flows to downstream IDSeS. Information Security Magazine, August 2001. [LA03] Mark Lutz and David Ascher. Learning Python. O’Reilly, December 2003.
37
[Lai00] Brian Laing. How To Guide - Implementing a Network Based Intrusion Detection System. Internet Security Systems, 2000. [Las03] Barbara Laswell. Building Global CSIRT Capabilities. PhD thesis, Canegie Mellon, Software Engineering Institute, September 2003. [Lib06] Don Libes. Expect, 2006.
http://expect.nist.gov/ – Avail. April, 2007. [Mic06] Michaël Castro. Monitorização e Gestão de Tráfego em Redes IP. Master’s thesis, Instituto Superior Técnico, 2006. [MS05] Douglas R. Mauro and Kevin J. Schmidt. Essential SNMP. O’Reilly, second edition, September 2005. [MVS01] David Moore, Geoffrey M. Voelker, and Stefan Savage. Inferring Internet Denial-of-Service Activity. In USENIX Security Symposium, 2001. [Net06] Netcraft. November 2006 Web Server Survey, November 2006.
http://news.netcraft.com/archives/2006/11/01/november_2006_web_server_survey. html – Avail. November, 2006. [PN98] Thomas H. Ptacek and Timothy N. Newsham. Intursion, Evasion and Denial of Service: Eluding Intrusion Detection, January 1998. [Pos07] PostgreSQL Global Development Group. PostgreSQL, 2007.
http://www.postgresql.org/ – Avail. August, 2007. [Ric05] Andy Rich. Introduction to Intrusion Detection With Snort, Setember 2005.
http://www.sun.com/bigadmin/features/articles/intrusion_detection.html – Avail. December, 2006. [RSMS03] Seth Robertson, Eric V. Siegel, Marr Miller, and Salvatore J. Stolfo. Surveillance Detection in High Bandwidth Environments. In DARPA DISCEX III Conference, April 2003. [Sec07] SecureIdeas. Basic analysis and security engine, 2007.
http://base.secureideas.net/contact.php – Avail. August, 2007. [Sen] SentryTools. Portsentry.
http://sourceforge.net/projects/sentrytools/ – Avail. January, 2007. [Sin05] Kapil Kumar Singh. Intrusion Detection and Analysis. PhD thesis, University of British Columbia, 2005. [Sou07] Sourcefire. Snort, 2007.
http://www.snort.org/ – Avail. April, 2007.
38
[SSMF03] Lambert Schaelicke, Thomas Slabach, Branden Moore, and Curt Freeland. Characterizing the performance of network intrusion detection sensors. In Recent Advances in Intrusion Detection, 2003. [Tig06] Tiger. The Unix security audit and intrusion detection tool, 2006.
http://www.nongnu.org/tiger/ – Avail. January, 2007. [Tri06] Tripwire. Open Source Tripwire, 2006.
http://sourceforge.net/projects/tripwire/ – Avail. January, 2007. [Zúq06] André Zúquete. Segurança de Redes Informáticas, chapter Sistemas de Detecção de Intrusões. FCA, 2006.
39
Appendix One .1
Example configuration file for the software
# Configuration options for PortHopper script # NOTE: When dealing with slot/port replace the ’/’ with ’0’ #
example:
#
6/24 = 6024 1/1
= 1001
# Router’s _primary_ IP address ip = ’10.0.0.1’
# Username and pass used for SNMPv3 (SHA + DES) user = ’someuser’ pass = ’thepass’
# Port mirroring session number (this should be # the current session in the router) sessionNumber = 6
# Mirror destination port destinationPort = 6023
# Source ports to be monitored monitoredPorts = 1001-1011,2001-2007,2009-2011,5001-5009,6001,6003-6009
# optional ports to always monitor (must be a subset of the monitored ports) staticPorts = 6005,
# Min and Max time between each ’hop’ expressed in seconds # NOTE: the hop time used will be a random number between minTime and maxTime minTime = 180 40
maxTime = 900
# how many hops to wait before removing a sticky port #(ie, a port that suspicious activity was detected) # 0 disables stickyHopCount = 0
41
.2
Example interaction with the system
$ ./PortHopper.py 2007-09-06 18:28:22,742 INFO Opening config file: port_hopper.cfg... 2007-09-06 18:28:22,745 INFO ...OK 2007-09-06 18:28:23,167 INFO Using port mirroring session 6 2007-09-06 18:28:24,876 INFO Using port 6023 as mirror destination port 2007-09-06 18:28:24,876 INFO Removing all current source ports 2007-09-06 18:28:26,214 INFO Starting main loop Setting up ports:[6005, 5002]
> print status Session Number: 6 Mirroring to port: 6023 (Vishu-MirrorListener) Mirror Status: on Mirroring from ports: port name
status
direction
operation
------------------------------------------------------------------5002 Complexo
enabled
bidirectional
on
6005 CIIST-GAEL Link 1
enabled
bidirectional
on
Next hop in: 4 minutes (Thu Sep
6 18:33:35 2007)
> print static [6005]
> static del 6005
> hop Hopping! Setting up ports:[5003]
> print status Session Number: 6 Mirroring to port: 6023 (Vishu-MirrorListener) Mirror Status: on Mirroring from ports: port name
status
direction
operation
------------------------------------------------------------------5003 Electricidade
enabled
bidirectional
on
42
Next hop in: 14 minutes (Thu Sep
6 18:43:42 2007)
> print dynamic [5001, 5002, 5003, 5004, 5005, 5006, 5007, 5008, 5009, 1009, 1011, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2009, 2010, 2011, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 6001, 1010, 6003, 6004, 6005, 6006, 6007, 6008, 6009]
> static add 2001
> hop Hopping! Setting up ports:[2001, 5004]
> print status Session Number: 6 Mirroring to port: 6023 (Vishu-MirrorListener) Mirror Status: on Mirroring from ports: port name
status
direction
operation
------------------------------------------------------------------2001 gatekeeper2
enabled
bidirectional
on
5004 Torre Norte
enabled
bidirectional
on
Next hop in: 13 minutes (Thu Sep
6 18:43:46 2007)
> exit
43
Appendix Two .3
Packet information displayed by BASE
44
