An Experimental Intrusion Detection Prototype based on Cognitive ...

An Experimental Intrusion Detection Prototype based on Cognitive Architectures Catriona M. Kennedy School of Computer Science, University of Birmingham, UK [email protected] Abstract An important area of Artificial Intelligence is concerned with the integration of sensing, reasoning, selfmonitoring and action into a single cognitive architecture capable of surviving in a hostile environment. So far, this approach has mostly been applied to visio-spatial domains such as robotics and agents in an artificial world. We argue that the study of “complete” cognitive systems also has value for intrusion detection systems. We also present a distributed agent-based prototype that was first developed in a simulated network and is currently being developed further in a physical test network. We conclude that the cognitive systems methodology leads to an integrated high level view of both the IDS and the protected system. This can make it easier to identify limitations and weaknesses than is the case with typical IDS packages.

Key words: agent, cognition, policy, prototype, reflection, scenario.

1

Introduction

In recent developments in artificial intelligence, there has been a shift away from research into algorithms and representations and a move towards complete cognitive architectures. We believe that this approach has benefits for intrusion detection. Integrated models of perception, representation and reasoning are highly relevant. In this paper, we report on our experience of applying this paradigm to an intrusion detection system in both a simulated network and in a real physical network. The cognitive systems paradigm in AI is inspired by living systems that survive in an environment and act autonomously. The minimal foundation is a control loop with “sense”, “decide” and “act” components. (Note that these are summary terms and include perception, hypothetical reasoning, planning etc.) We use the term “agent” for this basic unit, as is consistent with [18]. It is also similar to the control loop of autonomic computing [22], except that autonomic computing is concerned more with the lower level regulation, while a cognitive system is concerned with high level reasoning and explanation. Additional capabilities may be built up on this basic “sense-decide-act” foundation. The cognitive functionality may be provided collectively by multiple interacting agents [15]. The system may apply the same capabilities to its own components as it does to its environment, a feature which is often called meta-management [4, 20] or “reflection” [22]. The development of “complete” cognitive systems requires the early integration of sensing, reasoning, meta-management and action components. The precise nature of these components evolves in the context of all other components in the architecture and in the context of the environment in which the system is embedded. Therefore the architecture is “broad and shallow” initially [3] and can be “deepened” gradually as we make progress. The development process is informal and complements a formal methodology based on CSP (communicating sequential processes), which is used to derive an intrusion detection automaton presented in [5]. One of the objectives of our work is to provide a high level representation of events in the network to such a CSP-derived automaton. The structure of this paper is as follows: section 2 introduces the architecture to be applied to intrusion detection, based on a distributed cognitive model. Section 3 describes its implementation in a simulated network using the example of protocol usage to represent network events. Section 4 describes the experience of implementing a test network for evaluating the agent based software. 1

2 Concept: Distributed Intrusion Detection Agents The concept that we have implemented is an abstract network of intrusion detection agents and sensors embedded in a physical network. The initial foundation for this work is a distributed model, based on minimal cognitive agents, each with its own “sense-decide-act” loop. Each minimal agent can monitor (reflectively) its own reasoning and reconfigure parts of itself dynamically. These same reflective operations are provided between agents [12]. Thus the meta-management of the whole system is distributed and mutual. The model has been demonstrated as a proof of concept in a simulated environment. We do not expect to reproduce the distributed cognitive model precisely in a physical network due to practical issues such as the limits of available monitoring tools that would be required for mutual meta-management (for example, it is difficult for agents to look inside each other). However, the aim is to maximise the transparency of all components in the IDS and focus monitoring resources on those components that are most critically depended on. The distributed IDS system should produce a single consistent sequence of actions that can be understood easily and trusted. The simplest solution is to ensure that only one agent (the “leader”) has response privileges at a particular time. A leader agent must give up its leadership to another randomly selected agent within a specified time period. This reduces damage if the leading agent becomes compromised and makes it easier to detect an agent that monopolises the leadership. The approach is similar to that in distributed systems [24] and the fault-tolerance concept of having a single primary and multiple backups (e.g. [16]). Our approach contrasts with a classic multi-agent system such as [11] which is based on a team of agents loosely cooperating on a task.

2.1

Internal structure of an agent

An agent has a representation of the physical network which contains “network objects” and relations between them (e.g. hosts, firewalls, agents, sensors). This representation also includes correct behaviour specifications for each object (e.g. adherence to protocol restrictions) and a recent history of its actual behaviour which is continually updated by sensors. Each agent uses the sensors to monitor all or part of the network it is embedded in. The part of the network that the agent “sees” is determined by the sensors it uses and their locations (i.e. what the sensors themselves can “see”). Each sensor sends regular reports containing sequences of events it detected since the previous report. The interval between reports is called the time window. The internal structure of an agent is shown in Figure 1. The agent accepts input from n sensors S1 , S2 , ..., Sn Sensors

S1

Sn

S2

redirect sensors

Agent Sense

Decide Has any object been compromised?

(summarise and filter)

(check events using network representation and topology)

Select and execute actions

User interface

other actions

receive reports

Other Agents

Act

Determine state of the world

send reports

A1

Am

Figure 1: Structure of an agent

2

and m agents A1 , A2 , ..., Am . It produces an output to a user interface and also sends heartbeat reports to all agents it receives reports from. The exchange of heartbeats between agents enables a simple form of mutual monitoring. Comparing the contents of other agent’s reports can also be used to detect substantial disagreement about observed events. The diagram also shows a possible re-direction of sensors, which is a form of dynamic attention focusing. The different components of an agent are as follows: • Sense: input alerts and regular reports from sensors and other agents. – determine events during the last time window. Produce an abstract summary from multiple overlapping sensor reports. • Decide: input summarised history of events; determine whether any object may have been compromised. • Act: input summarised history and any decision; if there is an alarm then report it immediately with explanation; send heartbeat to other agent(s) and user interface; if leader, send summarised history to higher level analyser or select an intrusion response;

2.2

Policy-based Anomaly Detection

Each agent makes a decision on whether a network object is acting according to its behaviour specification, which is based on the security policy. We assume that answers to the following questions are known: • What tasks are essential for the requirements of the business? • Given an infrastructure (network and software tools) what activities are required to do each task? (For example, e-mail, file sharing, file editing, web browsing). Different kinds of work (human roles) will have their own access requirements (e.g. a payroll department needs access to personal data and may have its own network partition separated by a firewall from partitions serving other departments or roles). • What are the necessary patterns of communication and interaction for each activity? (e.g. who can share files with whom?) What should be disallowed? This determines the security policy (what we have termed the “good behaviour” restrictions). • What events are generated by essential activities, assuming that all non-essential activities are disallowed? This defines the correct behaviour for each object. The approach is based loosely on the Domain-Based methodology of [17] for describing security requirements of a business. Related work in IDS includes [13] and [23]. Policy violation is a high level definition of “anomaly”. As with general anomaly detection, the policydirected approach assumes that an intrusion causes an observable deviation from the specified behaviour. The difference between “good” and “bad” behaviour may be observable immediately, or only after some model of “good” behaviour has been acquired by learning. Policy violation may miss subtle forms of anomaly that were not included in the specification. For example, an attacker may deliberately use protocols that do not violate a specification on correct use (such as which protocols are used to/from which hosts?) but may inadvertently cause some other details to be abnormal (e.g. the time spent in various states of the TCP/IP state machine or the frequency of certain state transitions). It is possible to combine policy with statistical forms of learning and anomaly-detection, such as is done by [19].

2.3

Assumptions about the physical network

The abstract multi-agent architecture cannot be considered in isolation from the underlying network. The topology should allow the following relations: • Agent/agent interface: allow mutual monitoring, exchange of reports and alerts; • Agent/sensor interface: allow for sensors to send data to more than one agent. If one or both of these assumptions is violated, the subversion resistance features could not be easily implemented. An example topology that allows these relationships is shown in Figure 2. It also shows a possible placement of agents and sensors. Variants of this example topology are possible; for example, two corporate networks 3

External net Attacker

Firewall/router 1

S3

S1 web server

DNS server

DMZ

S2

S4 A1

web proxy

mail scanner

A1 Analysis host 1

Firewall/router 2

A2

S5

S6 A2 internal host 1

S7

Analysis host 2

mail server

Corporate net

Optional management network

internal host 2

Figure 2: Example topology with placement of sensors and agents

separated from the DMZ by the same firewall. The diagram also shows an alternative option (shown with dotted lines and italics) where agents are placed in a dedicated management network containing analysis hosts. This takes IDS communications out of the business network, where they could cause excess traffic and allow the possibility of misusing the IDS communications as a way into the network. The management network may also have one-way connections to it, giving some protection against IDS subversion from outside the management network (although this would still be possible from inside). The one-way connections would limit the damage that an inside attacker from the management net can do to the businesss systems. Similarly an external attacker would find it difficult to probe the management net, since it returns no information. The disadvantage of a management net is the added complexity of the network to be protected, which also increases the complexity of the model that the agents use to reason about it. Simply leaving it out of the agents’ model is dangerous because an attacker may exploit an agent’s lack of knowledge of the network’s real topology. Furthermore, the system relies on obscurity. It is assumed that an attacker will first require information on the management net in order to attack it. This information should be difficult to obtain because the management net is “invisible” to an outsider. The invisibility can be implemented using one-way physical connections. If this system breaks down (e.g. a one-way connection fails), the attacker can gain access to the management net and then to the internal (business) net, since there are no firewalls. If the one-way connections remain intact, an “inside” attacker could still compromise the IDS system in the management net. In our current work we have assumed that there is no additional management net, but future work may take it into account. We will return to these issues later, since they are important when considering subversion resistance. Sensor types. We are assuming that sensors are software entities running on hosts. Although sensors may also be hardware objects placed between hosts, or at various points between firewalls and routers, we are assuming that this makes no major difference to the sensor/agent communication (except that the sensor may be more difficult to disrupt). A sensor either reports events on a single host (such as system calls) or events on the network, such as SYN packets. We are assuming that a packer logger can detect all packets in a subnet (e.g. it is a hub or a switched network with a switched port analyser (SPAN) port).

4

2.4

Agent/sensor connectivity

The topology of the agent/sensor connections is also important. Examples include the following: 1. All hosts in all subnets have sensors. Each sensor reports to a single agent, which is placed either in the DMZ or in the corporate net. 2. n agents with full connectivity: same as (1) except that each sensor reports to n agents where n > 1. Agents are spread over both subnets, with some in the DMZ and some in the corporate net. This produces n replicated views of the whole network. (This has maximum connectivity and maximum redundancy, but is clearly less efficient). 3. n agents with partial connectivity: All hosts have sensors, but they only report to the agent(s) in their subnet, thus producing at least one local view of each subnet. In the case where there are n replicated views of the network, conflicting views may be resolved by majority voting, with an odd value for n.

3 Case Study: Protocol Usage As an initial case study, we defined a set of minimal services in a business network and we specified the “policy” as the correct protocol usage for communications involved in each service. The agent concept in Figure 1 was implemented. For the initial version, we did not implement any reconfiguration response such as redirection of sensors. On startup, an agent is given a description of the network it is situated in, along with sensors and other agents. The ontology used to describe the network is encoded in XML and contains the following entities: • network (the root node - a network is a collection of subnets); • for each subnet: list of hosts; • for each host: list of interfaces (one or more); list of agents and sensors allocated to the host (zero or more in each case); • for each interface: list of protocols it can receive and send communications on (stated as port numbers). The restrictions may optionally specify which other machines the host is allowed to communicate with; • agent/sensor connectivity: – for each agent: list of sensors it receives input from; list of agents it communicates with; both lists correspond also to the IDS applications the agent is responsible for protecting; – for each sensor: list of agents it sends input to. The in- and out-restrictions for each interface in the network correspond to the high level policy derived from the business requirements. For example, if we are describing a web server in the DMZ, it must have one interface which is allowed to listen on port 80. (In a real network, there will be additional ports).

3.1

Limited fault tolerance and subversion resistance

To detect subversion against the IDS, the normal behaviour of agents and sensors is observed for a certain time after startup by each agent. We are assuming that no intrusions occur during the startup phase, which is like a training phase. The normal behaviour is the sending of regular heartbeat reports, which contain a summarised view of the part of the network being watched and may include warnings about suspicious activity. During the startup period, each agent monitors the pattern of communications it receives from other agents and sensors (e.g. what is the frequency of reporting? what is the tolerable level of inconsistency between the report contents?) Later, if the application deviates from that behaviour, it is classed as unreliable. For example, if the heartbeat times out, the source (agent or sensor) may have been disabled or slowed down; if the source is reporting large numbers of events that are not confirmed by other sources, it may be experiencing a noise attack (being overloaded with spurious information) or executing hostile code. An agent group is defined as one or more agents that receive input from the same sensors. If there is full agent/sensor connectivity, only one group exists. If there are multiple groups, each group monitors part of the 5

network only. It will also monitor the IDS applications in that part of the network, as well as receiving input from them. If there are multiple agents in a group, there are multiple views of that part of the network. Therefore, one of them must be selected as a leader. This is particularly important if the IDS is to deliver a response, or if some other component is depending on its decision. At startup, one randomly selected agent from each group is allocated the leadership role. Following a random time interval, this agent gives up its leadership to a randomly selected agent in its group. In a real world physical network, we expect to integrate the agent’s monitoring and reasoning with existing tools for fault-tolerant distributed communication and agreement.

3.2

Initial simulation

The first stage of the prototype development was to build a network simulation to support the different topologies above. A model of the network objects was developed, which was the basis of both the simulation and an agent’s representation of the network. The initial rapid prototyping was carried out in the following steps (not always in the order stated and with many iterations): 1. For each selected topology, explore different agent and multi-agent configurations and embed them in the network topology. This results in a “populated” network representation in XML (i.e. with agents and sensors, as well as physical network objects) 2. For each object in the network, generate some normal behaviour scenarios allowed by the security policy, which is in turn determined by the kind of jobs being done and the required actions for those jobs; 3. Identify important compromise scenarios and generate them in the simulation. A compromise scenario occurs when any object changes its behaviour from “good” behaviour to “bad” (where “bad” is any behaviour that is not permitted by the security policy). Since the security policy allows only essential activities to take place, then any non-essential activity is classed as “bad”. 4. Test the agent(s) in each topology and each compromise scenario. Modify the design as necessary. Simulating normal behaviour. The agents were tested with various scenarios. Each scenario is made up of two components: normal activity which is always happening and intrusion-related activity activated by a timetrigger. In the simulation, each network object has a “behaviour” associated with it. The simulation code is an infinite loop that activates the specified behaviour for each object for each loop iteration. Probabilities may be introduced. The principle is the same as that used in the SIM AGENT toolkit in [21]. With regard to behaviours there are two kinds of objects: client machines initiate actions by sending packets through the network; other objects such as servers and firewalls respond to packets sent to them. For a client, a behaviour is time-triggered. For a server or other object, its behaviour is executed if it has input (e.g. if a mail server has incoming packets). When running a scenario, the background activity is generated by typical work-related actions of a client machine in either the external or internal networks. Example actions are “send-mail”, “check-mail” or “webrequest”. They are activated at random intervals. This is then followed by responses from the appropriate servers. After a specified time delay, the bad behaviour is activated. Simulating bad behaviour. To test an anomaly-detection system, we cannot generate every possible form of “bad behaviour”. Instead we have to choose examples carefully so that the critical parts of the software are tested. Example bad behaviours include: • simple protocol violations (e.g. the mail scanner talks to the web server using the POP protocol). • an agent does not report within a timeout (assumes that network allows correctly behaving agents to act within a predictable timeout, e.g. no extremely fluctuating traffic levels) • sensor reports (falsely) masses of events that were not actually detected. • agent (falsely) reports events or problems that have not occurred (e.g. it could be a Trojan masquerading as an agent).

6

Examples of protocol violation only include conversations within a single subnet. The firewall should stop any invalid conversation between subnets unless they are abusing the protocol (which we cannot detect at present) or the firewall itself is compromised.

4

Experience with Test Network Implementation

The test network was set up according to the example topology in Figure 2. The current implementation comprises five PCs with VMWare installed on three of them. Figure 3 shows the relationship between the physical and virtual machines. Two PCs are used as firewall machines and contain two network interface cards, PC 1: External net VM 1a

VM 1b

Client 1 (Windows)

Attacker (Linux)

Crossover cable

PC 4:External Firewall Interface 1 Interface 2

PC 2: DMZ

VM 1c

VM 1d

VM 2a

VM 2b

Web server (Linux)

Mail server (Linux)

Client 1 (Windows)

Attacker (Linux)

VM 2c

VM 2d

Web server (Linux)

Mail server (Linux)

HUB PC 3: Corporate net VM 3a

VM 3b

Mail server (Linux)

Client 1 (Linux)

VM 3a

VM 3d

PC 5: Internal Firewall Interface 1

HUB Client 2 (Windows)

Interface 2

DNS server (Linux)

Figure 3: Test network allowing for both an external and internal facing Ethernet device with different IP addresses. Software was installed on the various machines being used as servers. We selected Apache’s web server, Squid web proxy, BIND (Berkeley Internet Name Domain) DNS Server and qmail for the Mail Server.

4.1

Sensor types

We used two types of sensor: one is a SYN packet logger which is a configuration of Snort. The other is a TCP/IP stack monitor based on the Linux Netstat. The stack monitor sends a regular report about the different port numbers the host is listening on, along with the state of its TCP connections. Both sensors have configurable report intervals. The report is a short history of events detected since the last report (for example, all SYN packets detected since then). The stack monitor report interval is also the frequency at which the TCP/IP stack is inspected. So, the higher the frequency, the more accurate will be its picture of changing states and the more likely it is to pick up transient states such as SYN SENT. On the other hand, a high frequency is extremely inefficient, since it overloads the agent with redundant information. We have experimented with different frequencies and have found that an interval of 2 seconds seems to be sufficient to detect most of the LISTEN and ESTABLISHED states. Future work should determine if these states provide sufficient information to indicate the port numbers and IP addresses that are actually being used for communications. (We have found that the ESTABLISHED states can be correlated with SYN packets so that it is not necessary to detect SYN SENT or SYN RCVD to determine which side started the connection). 7

4.2

Implemented agent/sensor architecture

The agent-based architecture was transferred from the Java network simulator to the test environment. The sensing and acting components of each agent were modified so that they handle the test network instead of the Java simulation. Figure 4 shows the implemented agent/sensor architecture in the situation where both sensor types run on one host along with one agent. This is just one possible configuration. Hosts will normally have one or two sensors and at most one agent. To avoid clutter, the agent’s responses or forwarding of information to a CSP-derived analyser are not shown. The controller process acts as a heartbeat and distribution process placed between the raw sensors and agents. We configured the test environment so that each sensor has exactly one controller process associated with it on the local host. Furthermore, we made the controller configurable so that it can do some filtering and summarisation of the raw data from the sensor. This significantly reduces network traffic when broadcasting reports to agents on other hosts. (If the test network had a management net, excess traffic would be less of a problem because the controllers and agents would be placed in that network, while the raw sensors would be in the business network). For the proof-of-concept integration with the CSP monitor, we are focusing on the following agent/sensor configurations: 1. The stack monitor runs on all Linux machines. SYN packet logging (modified Snort) runs on the Windows host. All sensors report to a single agent running on the Linux machine in the corporate net. This has already been tested successfully with different kinds of benign and hostile traffic. 2. As above except that each sensor reports to three different agents (one in the DMZ and two in the corporate net). This is to ensure that the multi-agent configuration can also interact with the CSP monitor. These correspond to two of the configurations that were tested in the simulation. In each case there is a single agent group. To/from agents on other hosts

To agents on other hosts (SYN packet reports)

CP1

PS

CP2

SM

Agent

SYN packet reports from other hosts

Stack monitor reports from other hosts

Key: SC1 = controller process (SYN packets) SC2 = controller process (stack monitor) PS = packet sensor SM = stack monitor

To agents on other hosts (Stack monitor reports)

Figure 4: Agent/sensor architecture

5

Extending the current prototype

The current prototype is a minimal proof-of-concept that is intended for further development. The following issues have been addressed but not yet implemented. 8

5.1

Estimating reliability

An agent has to evaluate the reliability of sensors and other agents to detect subversion against the IDS and to provide a reliable picture to a response component or a higher level analyser such as the CSP monitor. In the single agent configuration, there is just one view of recent events in the network, which causes no problems for a higher level analyser. In the n agent case, we can identify the following situations. In each case, guidelines are given for the leading agent’s action: 1. All agents agree (with only minor differences): leading agent sends its own report with highest degree of confidence; 2. Only a majority supports the leading agent’s view (the rest have either not reported within the timeout or they disagree substantially): agent sends its own report, but with a level of confidence proportional to the number of agents in the majority and inversely proportional to the number of disagreeing agents (timeouts need not contribute to estimated confidence levels). 3. There is no clear majority support for the agent’s view (e.g. they are all substantially different): agent still sends its own report but with lowest level of confidence. 4. There is a clear majority against its own view: the agent assumes that its view is incorrect and gives up its leadership to one of the majority. This agent will then send its report. In addition to exchanging their view of events in the network, agents also exchange their beliefs about untrusted objects, which are also reported to the user interface. The confidence level in the reliability of event reports should also be determined by the estimated reliability of sensor information. Different forms of sensor correlation include the following: Stack monitor: If host A says that it has an established connection with a remote host B, we expect the monitor on B to say that there is an established connection with A (assuming that both A and B are within the monitored network, i.e. not external). The timestamps on the reports can be correlated. If both monitors are in agreement, this indicates that the sensors are reliable and their reports can have a high confidence level. If no confirmation from the other host can be obtained, this does not necessarily mean that one sensor is faulty; it may just mean that the established state lasted only for a short time and was only picked by one monitor. The confidence level is lower in such a situation. It becomes lower, the longer one sensor reports an established state that is not confirmed by the other host. At a certain point, both sensor outputs will have to be regarded as unreliable, but this may change if they produce consistent outputs again. Correlation between SYN packets and stack monitor indicating established state: If there is a SYN packet with source or destination address within the monitored network, we expect the stack monitor on the host with that address to be in established state with the host indicated by the other address in the packet. If this not the case, it may indicate half-open connections, spoofing or unreliable sensors. Another possibility is that the stack monitor reports an established state consistent with the SYN packet but the local and/or remote ports returned by the stack monitor are not consistent with the source and destination ports in the SYN packet.

5.2

Extending the class of intrusions detected

The class of attacks that can be detected by the current agent prototype is very limited, since we are looking only at protocol violation and the behaviour of hosts. An attacker may avoid triggering a protocol violation. However, experience gained during agent development allowed us to identify additional domains in which it would be more difficult to avoid triggering an anomaly. Therefore we envisage the following stages in the further development of the prototype: • Extend monitored objects: For example, instead of just specifying what protocols a host can use, specify this also for applications. • Extend sensing to other domains: For example, once we include applications as monitored objects, we could monitor their system calls in addition to their protocol usage. • Introduce Learning: For some domains it may be difficult to specify the correct behaviour in advance because it may not be precisely known, or the process of specificaton may be error-prone. Introduction 9

of learning may combine policy-based IDS with more traditional anomaly-detection of the kind in [19]. The main difference is that we use controlled scenarios to generate the training data.

5.3

Enhancing subversion resistance

The distributed architecture has mechanisms for subversion resistance. However, it may also introduce new vulnerabilities. In the original concept, agents and sensors were to be included as monitored objects. This has partly been realised. The current prototype does not yet monitor applications but it monitors hosts on which agents and sensor processes are running. Incorrect behaviour of an agent’s host can be detected if it does not have the correct open port numbers (from the XML restrictions) and correct communications with other IDS components (as determined by the heartbeat). The disadvantage of this is that port numbers are visible for anyone who wants to disrupt the IDS or masquerade as an agent or sensor. A security net (as in figure 2) would overcome this problem. The IDS selfmonitoring would take place within the security network. It may be possible to specify a set of high level rules for IDS behaviour which would conflict with the interests of an attacker masquerading as an IDS agent and hence make it difficult to avoid triggering an anomaly. Our existing implementation of rotating leadership is a step towards this, but needs further development and experimentation.

5.4

Improved scenario generation

Scenarios that we have generated so far have limitations because they are only partially automated and therefore produce a limited volume of traffic, which tends to be homogeneous. In spite of this, we have found them to be valuable for providing a controlled environment in which to test the software. In future work, controlled scenarios may also be used to generate training data for learning. This may have advantages because the high level action that produces a particular pattern of normal activity is known and the agent can associate this action with the low-level patterns. Once the system has been trained in artificial scenarios, it is important to reconstruct a sample of live data in the test network so that differences between reality and hypotheses about normal use can be investigated. The data may be “reverse engineered” to identify the high level actions generating it.

5.5

Scaling up

For significantly larger networks, the configurations we have been using will be unrealistic. It is inefficient and probably unworkable to have all sensors providing input to each agent. Possible solutions include the following: 1. Partitioning the network so that multiple agent groups exist, each monitoring a partition. There are several possibilities: (a) Each group leader can send reports to a higher level group covering the whole network (or a larger part of it). Effectively the lower level group leaders act like sensors for each agent in the higher level group. The leader in the higher level group sends its reports to a CSP monitor overseeing the whole network. (b) The lower level group sends its reports directly to a local CSP monitor, which oversees the same network partition as the agent group. This monitor in turn sends its reports to a higher level monitor. (c) Probably a combination of the above may be best. At each level of abstraction, the advantages of the cognitive approach (e.g. learning, planning) may be exploited along with the advantages of formal methods. 2. Introducing other forms of partitioning (in addition to spatial locations in the network). For example, groups of agents may specialise in certain kinds of sensor information. The sensor information could be fused together by a higher level agent group summarising events in this partition. This in turn sends information to a higher group which brings together all the spatial partitions. 3. Exploring options for partial connectivity in agent/sensor communication within a group instead of maximal connectivity as is currently the case. For example, the sensors could be sampled sequentially in random order for random durations (instead of all sensors at once). This involves a tradeoff between reliability and efficiency. 10

The larger the network, the larger the number of agent groups and the longer it takes for information to be propagated from sensors to the highest levels. This introduces problems of speed as well as reliability of the information. The human users will normally only want to see the highest level of information. Therefore the following issues also arise: • Automated response becomes essential. Otherwise intrusions will have time to cause damage before an alert reaches a human user. For some types of intrusion a fast “dumb” reaction may be enough to avert damage (e.g. stopping a connection with RESET packets). Experience in prototype development has identified points in the architecture into which response capabilities can be added. For example, fast detection and response could be plugged into the sensor controllers. They would act as a reactive layer. Some existing commercial intrusion prevention system (IPS) may be exploited to do this. • Different kinds of responses are required at different levels. It is not enough just to have low level reactive responses because they may interfere with each other or have longer term adverse effects. To prevent this, more complex planned responses are required on the higher levels which involve coordination and resource management of the low-level responses (e.g. activating or inhibiting certain kinds of low level action at different stages in a planned sequence). Recent work on planned response includes [2]. • Automated responses should happen with minimal human intervention. In the largest networks humans may deal with exceptional situations only. Research in agent delegation and variable autonomy may be exploited (e.g. [8]). • If the interaction between groups is purely hierarchical, the fault-tolerance of the system may be less effective for the flow of information between groups than between agents in a group. It is important to explore non-hierarchical interactions between groups (either on the same level or from a higher level to a lower one). These interactions may be transient and continually reconfiguring (see discussion below on related work). Work on autonomic computing is particularly relevant for large scale networks and should be exploited. Example approaches include [14].

6 Comparison with Related Work Our multi-agent system is comparable to AAFID (“Autonomous Agents For Intrusion Detection”) [1] and MAIDS (“Mobile Agent Intrusion Detection System”) [10]. A more recent example is MMDS (“Multi-level Monitoring and Detection System”) [7]. In contrast to our system, these agent-based architectures do not consider the agents themselves as part of the system to be protected. Some limited subversion resistance may be provided if the agents cross-check each other’s outputs. Our current implementation is restricted only because we did not implement a wide range of sensors for mutual observation between agents. In principle, our architecture allows new kinds of sensing and reasoning components to be plugged in, which may be applied to agents, sensors or other network objects. Self-monitoring of IDS systems has been addressed in immunity-based systems. Recent work [27] uses an immunity-based diagnosis algorithm to detect corrupted IDS systems and is intended for mutually monitoring entities (such as hosts or mobile agents). The approach has been demonstrated in a simulation involving simplified rule-based IDS components. It is not yet known how it scales up to realistic IDS systems, or whether it could be applied to our architecture. Cognitive approach. MMDS has more similarity to a single cognitive architecture than most other agent-based systems. It is composed of monitoring, decision and action agents along with a management agent (containing the user interface) and is thus similar to our approach (based on “sense-decide-act”). The decision agent of MMDS uses fuzzy inference to detect abnormal activities. The architecture includes a genetic algorithm to evolve fuzzy rules that can correctly classify a set of normal and abnormal traffic. Components of this kind may be plugged into our architecture as part of an agent’s “decide” module. Agent roles. In existing agent-based systems, each agent typically has some specialist function, determined at design time. In our approach, multiple agents are replicas within a group, where groups are allocated different roles. For example, they may be assigned to different spatial partitions of the network or to different categories of sensor. Specialisation should emerge as a result of experience gained during rapid prototyping in 11

well-chosen scenarios and through gradual scaling up. The resulting system may be more effective than one that is specified tightly in the initial stages and does not allow evolution. Analysis levels. In its simplest form, our architecture assumes an analysis hierarchy when scaled up (with sensors at the lowest level, followed by layers of agent groups and CSP monitors). Its non-hierarchical aspects are concerned with fault-tolerance and are purely local (within an agent group). However, non-hierarchical relations between groups are not excluded as a possibility (as mentioned in Section 5.5 above). The “interest-driven agents” approach of [9] has an analysis hierarchy as its foundation but provides the option of non-hierarchical paths of information flow, which may be opened and closed dynamically. An agent on the lowest level of the hierarchy can announce its “interest” in a class of events outside of its specialisation domain (e.g. it is in a spatially different part of the network). The agent can then receive the information directly from other agents in the different specialisation without having to go through the stages of the analysis hierarchy. The agent’s interest may be temporary (meaning the connection would be closed down after a time interval). The principle of interest-driven agents could be applied to a scaled up version of our architecture containing several layers of agent groups. Fault tolerance. Much of the subversion resistance in our architecture uses a simplied form of fault-tolerance. However, we did not focus on implementing fault-tolerance in detail because it would duplicate existing work. For example, our prototype is very dependent on correct timing and clock synchronisation. Improving robustness in scenarios of uncertain timing is already an established area of research, much of which could be applied to our architecture. See for example, [26, 25]. MAFTIA’s intrusion-tolerant intrusion detection system [6] is intended to enhance the subversion resistance of an IDS, mostly using fault-tolerance methods. It has the concept of a “meta-IDS” which observes other analysers and determines whether they may be compromised. Our architecture does not have specialised “metaIDS-agents”. Instead all IDS agents can have “meta-IDS” capability, since they monitor each other. This may have an advantage because we are not adding new meta-levels that have to be monitored themselves. Instead the aim is to use the same software components to monitor different objects (whether or not they are agents), thus reducing additional complexity. In practice, it may be difficult to apply this principle in its pure form, since it requires IDS agents to be subject to the same policy restrictions that other network objects are subject to (so that the same code can be used). Additional meta-level functions may be required if we specify “friendliness” restrictions for IDS agents that do not apply to other network components (e.g. giving up control voluntarily) as mentioned in Section 5.3 above.

7 Summary and Conclusion The experience of building the prototype has confirmed that the cognitive approach has value in intrusion detection because we found that capabilities associated with cognitive systems were actually required in practice. In particular, the following applies: • Experience with agent/sensor communication has shown that we require filtering of irrelevant information on the lower levels and summarised reasoning and explanation on the higher levels. A layered cognitive architecture provides this. • Also consistent with cognitive systems is that the lower levels need to be fast and highly parallel, while the higher levels should be slower and sequential in order for humans to interact at this level. Many of the problems encountered in testing were due to a lack of difference in speed and parallelism between these levels, causing bottlenecks on the sensor controller level and too much output on the higher levels. • Cognitive research on attention focusing and interruption of high level analysis by urgent alarms from lower level sensory processing is also applicable to IDS (although not actually implemented) • Learning and adaptation is important (Section 5.2). • Scaling up requires autonomous response on different levels (Section 5.5) which is also an area of cognitive systens research.

12

• Meta-management (beyond fault-tolerance) has value for IDS: Our current implementation has limited (mutual) meta-management and does not provide more than conventional fault-tolerance. However, it is possible (and useful) to extend the system to include more diverse forms of introspection and selfreconfiguration than would typically be available to a fault-tolerant system (such as presented in MAFTIA), partly due to the conservative nature of fault-tolerant design. Developments in cognitive research should be exploited in this area (for example, on reasoning, explanation and learning from own mistakes). (Traditional fault-tolerance and cognitve systems will tend to merge together with advances in autonomic computing). The cognitive approach and its related methodology has also led to a high level unified view of the system to be protected as well as the IDS sensor/agent network protecting it. This has allowed quick identification of weaknesses and exploration of potential solutions. The simulation tool allowed the rapid testing and modification of potential agent designs and network representations.

Acknowledgements This research was supported by the Defence Science and Technology Laboratory (DSTL). I would like to thank Sue Haines and Helen Phillips from DSTL as well as Chris Cant (QinetiQ, Malvern) for providing valuable comments and advice.

References [1] Jai Sundar Balasubramaniyan, Jose Omar Farcia-Fernandez, David Isacoff, Eugene Spafford, and Diego Zamboni. An Architecture for Intrusion Detection using Autonomous Agents. Technical Report COAST TR 98-05, Department of Computer Sciences, Purdue University, 1998. [2] Ivan Balepin, Sergei Maltsev, Jeff Rowe, and Karl Levitt. Using specification-based intrusion detection for automated response. In Sixth International Symposium on Recent Advances in Intrusion Detection (RAID 2003), Pittsburgh, PA, USA, September 2003. [3] J. Bates, A. B. Loyall, and W. S. Reilly. Broad Agents. In AAAI Spring Symposium on Integrated Intelligent Architectures. American Association for Artificial Intelligence, 1991. (Repr. in SIGART BULLETIN, 2(4), Aug. 1991, pp. 38–40). [4] Luc P. Beaudoin. Goal Processing in Autonomous Agents. PhD thesis, University of Birmingham, 1994. [5] C.J. Cant, S.L. Haines, C.M. Kennedy, N.M. Moffat, C.M. O’Halloran, and H.L. Phillips. Methodology for instrumenting and protecting networked systems, 2003. Project deliverable DSTL/CR07508 V1.0. [6] Marc Dacier. Malicious- and Accidental-Fault Tolerance for Internet Applications (MAFTIA) – Design of an Intrusion-Tolerant Intrusion Detection System, 2002. Project deliverable - D10. [7] D. Dasgupta, J. Gomez, F. Gonzalez, M. Kaniganti, K. Yallapu, and R. Yarramsettii. MMDS: Multilevel Monitoring and Detection System. In Proceedings of the 15th Annual Computer Security Incident Handling Conference, Ottowa, Canada, 2003. [8] Gregory Dorais, R. Peter Bonasso, David Kortenkamp, Barney Pell, and Debra Schreckenghost. Adjustable autonomy for human-centered autonomous systems. In International Joint Conference on Artificial Intelligence (IJCAI’99) Workshop on Adjustable Autonomy, Stockholm, Sweden, July 1999. [9] R. Gopalakrishna and E. Spafford. A Framework for Distributed Intrusion Detection using Interest Driven Cooperating Agents. In Fourth International Symposium on Recent Advances in Intrusion Detection (RAID 2001), Davis, California, October 2001. [10] Guy Helmer, Johnny Wong, Vasant Honavar, and Les Miller. Intelligent agents for intrusion detection. In Proceedings, IEEE Information Technology Conference, pages 121–124, Syracuse, NY, September 1998. [11] N. R. Jennings. On agent-based software engineering. Artificial Intelligence, 117:277–296, 2000.

13

[12] C. M. Kennedy. Distributed Reflective Architectures for Anomaly Detection and Autonomous Recovery. PhD thesis, University Of Birmingham, Birmingham, UK, July 2003. Available online at: http://www.cs.bham.ac.uk/research/cogaff/0-INDEX03.html#03-03. [13] Calvin Ko, Paul Brutch, Jeff Rowe, Guy Tsafnat, and Karl Levitt. System health and intrusion monitoring using a hierarchy of constraints. In Recent Advances in Intrusion Detection (RAID 2001), LNCS, Davis, California, October 2001. Springer. [14] Scott M. Lewandowski, Daniel J. Van Hook, Gerald C. O’Leary, Joshua W. Haines, and Lee M. Rossey. SARA: Survivable Autonomic Response Architecture. In DARPA Information Survivability Conference and Exposition (DISCEX II’01) Volume 1, Anaheim, California, June 2001. [15] Marvin Minsky. The Socety of Mind. Simon and Schuster, New York, 1986. [16] D. Powell, G. Bonn, D. Seaton, P. Verissimo, and F. Waeselynck. The Delta-4 Approach to Dependability in Open Distributed Computing Systems. In Proceedings of the 18th International Symposium on FaultTolerant Computing Systems (FTCS-18), Tokyo, Japan, 1988. IEEE CS Press. [17] Clare L. Robinson. Security requirements models to support the accreditation process. In Second Annual Sunningdale Accreditor’s Conference, September 2001. [18] S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice-Hall International, Englewood Cliffs, NJ, USA, 1995. [19] R. Sekar, A. Gupta, J. Frullo, T.Shanbhag, A. Tiwari, H. Yang, and S. Zhou. Specification-based anomaly detection: a new approach for detecting network intrusions. In Proceedings of the 9th ACM conference on Computer and communications security(CCS’02), pages 265–274, 2002. [20] A. Sloman. Damasio, Descartes, alarms and meta-management. In Symposium on Cognitive Agents: Modeling Human Cognition, at IEEE International Conference on Systems, Man, and Cybernetics, pages 2652–7, San Diego, CA, October 1998. [21] A. Sloman and R. Poli. SIM AGENT: A toolkit for exploring agent designs. In Joerg Mueller Mike Wooldridge and Milind Tambe, editors, Intelligent Agents Vol II, Workshop on Agent Theories, Architectures, and Languages (ATAL-95) at IJCAI-95, pages 392–407. Springer-Verlag, 1995. [22] Daniel H. Steinberg. What you need to know now about autonomic computing, part 1: Introduction and overview, at: http://www-106.ibm.com/developerworks/ibm/library/ i-autonom1/. Last consulted March 2004, 2004. [23] Prem Uppuluri and R. Sekar. Experiences with specification-based intrusion detection. In Recent Advances in Intrusion Detection (RAID 2001), LNCS, Davis, California, October 2001. Springer. [24] Paulo Verissimo and Luis Rodrigues. Distributed Systems for System Architects. Kluwer Academic Publishers, Dordrecht, The Netherlands, 2001. [25] P. Verssimo. Uncertainty and predictability: Can they be reconciled? In Future Directions in Distributed Computing, Lecture Notes in Computer Science, Vol: 2584, Springer-Verlag, 2003. [26] P. Verssimo, A. Casimiro, and C. Fetzer. The Timely Computing Base: Timely Actions in the Presence of Uncertain Timeliness. In Proceedings of the International Conference on Dependable Systems and Networks, New York, USA, June 2000. [27] Yuji Watanabe and Yoshiteru Ishida. Immunity-based approaches for self-monitoring in distributed intrusion detection system. In Knowledge-Based Intelligent Information and Engineering Systems: 7th International Conference (KES 2003), pages 503–510, Oxford, UK, September 2003.

14