An Overview of SNIF: A Tool for Surveying Network ... - CiteSeerX

An Overview of SNIF: A Tool for Surveying Network Information Flow J. Alves-Foss Laboratory for Applied Logic Department of Computer Science University of Idaho Moscow, ID, 83844-1010

Abstract

Connection of a local area network to the Internet brings with it a certain amount of risk. Once connected, local computer systems are subject to attacks by other users on the Internet. To maintain system integrity, system administrators need tools to accurately display the status of communication between local hosts and remote hosts. This paper describes an ongoing research and development project for Surveying Network Information Flow (SNIF). The SNIF tool provides system administrators and security ocers an encapsulated view of network communication and information ow for a set of monitored subnetworks. The tool provides a real-time graphical interface, con gurable monitoring, alarms and signals, connection statistics, and an extensible interface to permit Links to External Systems (LES). This tool is one of many in a suite of security tools being developed at the University of Idaho.

1 Introduction

The 1980's saw computer networks reach a level of common acceptance. Technological improvements brought high speed, high reliability and low cost to computer users, enabling them to connect personal computers, minicomputers and mainframes over small and large local networks. In the 1990's we are seeing similar growth and acceptance of internetworking as local area networks are connecting to each other to provide even greater connectivity and access to users. Thousands of machines and millions of user's are continually being attached to the Internet making it one of the fastest growing systems in the world. This rapid expansion and interconnectivity provides the Internet community with increasing access to electronic resources. These include international databases, rapid communication between peers and access to remote computational facilities. Unfortunately, this growth has also increased the population of two groups of Internet users:  The roaming crackers. This group consists of users who, for some reason or another, wish to  This work was sponsored through a NASA/ASEE Summer Faculty Fellowship Program.

gain unauthorized access to other computer systems on the Internet. Their intent may be anything from browsing through system information to sabotage of computer systems.  The unwary user. This group consists of users who are unaware of the vulnerability of their computer systems to the actions of the previous group. This ignorance may be due to anything from a poorly placed trust on the security of their system software to lack of knowledge of the existence or the crackers. The growth of both of these groups has brought about an increase in the need to improve the security of systems connected to the Internet. This involves research into improving the security of the underlying system software, increasing the availability and usability of authentication and cryptographic mechanisms, addition of security mechanisms into communication protocols, and the development of tools to assist in the evaluation and analysis of system security and vulnerability. This paper describes the goals and current status of a tool for Surveying Network Information Flow (SNIF) that addresses this later research issue. In Section 2 of this paper we will discuss the exact problem domain that this tool is designed to address. We provide an overview of the design goals of the tool in Section 3, describe the architecture and operation of the tool in Section 4 and conclude with lessons learned, current status and future work in Section 5.

2 Problem Domain

In academic and research institutions, it is often the desire to provide freedom of access and action whenever possible. This applies to many disciplines and resources, including computer systems. Unfortunately, many system administrators and computer system owners have found that too much freedom often leads to compromise and misuse of their systems. Beyond increasing the security of system software and installing security mechanisms that may be overly restrictive, the system administrators often nd that they must implement an additional management system to increase site security. SNIF is a tool designed

to assist in the implementation of a security policy that allows users to maintain the connectivity to the Internet that they were used to and still enable system administrator's to maintain and manage system security. The policy that guided the development of SNIF can be summarized as: Users must be allowed the freedom to connect to the ever growing resources available on the Internet, be able to use current and existing tools and applications over the network, and be permitted to connect to and from remote sites across the country and world. It is important that the user's privacy be maintained while system administrators are able to obtain the information necessary to determine how our computer systems interact with the outside world. This information will be used to ensure the integrity of our systems and manage the ow of information o site.

This policy leaves out many options that other sites have taken to ensure the security of their systems. This includes rewalls, enhanced authentication protocols (until they become commonplace), and strict limitations on system resources, connectivity and tools. This policy exists for a large, ever-changing, and wide-spread user base which makes the implementation of a smart-card based system currently too costly and unwieldy. The large diversity in computer systems on the network (from DOS-based and MacIntosh PCs to large multiprocessors systems) makes the implementation of protocol wrappers, and operating system monitoring tools very dicult. The large diversity in systems prohibits the use of a tool that runs on each host, due both to lack of availability and technical diculties such as the lack of auditing capabilities in some operating systems. Unfortunately the large diversity of systems, several of which are managed by the users (PCs for example), increases the need to monitor information ow from these machines. The information needed, as described below, prohibits the use of existing network management tools. These tools are typically based on SNMP [1], and while extremely useful, are unable to collect the level of detail desired at the application level. This policy, and the environment under which it is to be implemented led to the decision that a tool was needed that would unobtrusively monitor the connections between local and remote hosts. This monitoring would not isolate individual users, but individual hosts and domains. Information collected would contain only information ow statistics down to the application level. In other words, the tool would tell administrators what application was communicating and the pattern of communication, but nothing about the contents of the communication. In addition, the tool would provide an alarm mechanism whenever particular actions occurred such as a connection to an untrusted remote host, occurrence of \intrusive behavior" such as attempted connection to successive network addresses, or connection to unauthorized local information services such as an FTP or WWW server.

3 SNIF Overview

SNIF is a tool designed to assist network system administrators with the monitoring and analysis of network information ow described in the previous section. Although the primary goal of SNIF is to monitor the ow of information to remote sites, it has been designed such that local interhost communication can also be monitored. Several speci c design goals have driven the development of SNIF: 1. Ease of use. The target users of SNIF are network system administrators and security of cers. These users do not have time to waste on complex con gurations, installation, and maintenance. Nor do they have time to pour over long, confusing textual data dumps. A system that provides a graphical user interface for straightforward system con guration and monitoring will be the most useful. 2. Distributed. Networks are inherently distributed and may consist of multiple subnetworks. A monitoring tool must be able to examine the information on each subnet and collate that information. 3. Real Time. Intrusion and anomaly detection tools are more eective if they can provide the system administrator information on the status of the network in real time. Although o-line, postprocessing systems can provide complex analysis, a real-time system can give the system administrator immediate notice of problems, possibly allowing them to direct more complex analysis using another tool. 4. User con gurable. Several components of SNIF are con gurable by the user. As well as certain default con gurations and options, SNIF also provides the user with the ability to de ne special functionality, such as lters, views and signals. 5. Scalable. Although only tested in limited domains, SNIF is designed to work on very large networks with hundreds of subnets. With distributed computing capabilities, and preprocessing, the central monitor should be able to handle very large systems. 6. Available. All eorts were made to use standard o-the-shelf development tools and libraries. The system was written in ANSI C and C++ using the SVR4 standard XView [3], DLPI [12] and NDBM [9] libraries. We avoided using speci c DLPI lter modules to permit easy installation, as many systems require that these lters run as part of the kernel. 7. Modern. It was determined that, for this tool to be useful, it would have to be developed understanding the latest trends in operating systems and data communication. With this in mind, the system was developed using UNIX SVR4 libraries and development tools. Every attempt was made

to avoid obsolete system calls and network communication interfaces. References to protocol osets, header sizes and network data structures use information in the protocol headers and system include les. Absolute values for this information was explicitly avoided (a problem we found in several publicly available network scanners). 8. Extensible. SNIF provides extensibility in several ways. First, the user con guration options are designed to permit user de ned ltering and data collection as well as user de ned views and signals. Secondly, SNIF has been designed with a modular approach, with the main monitor being an event driven system. Modi cations and additions to the code should be straight forward. Lastly, SNIF stores all information in an NDBM database that can be readily accessed by authorized user processes. Hooks to these processes are provided in SNIF, allowing them to trigger system signals and alarms. 9. Secure and Fault Tolerant. SNIF is designed to be managed and con gured by authorized users only. The database les can not be modi ed or examined without proper authorization. The lters can sit on any machine on a subnet and nonintrusively monitor the trac, making detection of the lter more dicult. Each lter/logger pair also maintaina local database that is used to store connection information when the central collator is unavailable, as well as to assist in the preprocessing of information, thus providing continuous information ow monitoring.

4 SNIF Speci cs

In the following sections we describe the some of the more interesting features and concepts of the SNIF system. We start by providing an overview of the system architecture, describing each of the major processes of the system. These components cooperate to provide the complete functionality of the system. We then describe the major operational concepts of the system. It is these concepts that the user manipulates to con gure the system. The section concludes with a description of the user interface employed by SNIF.

4.1 SNIF Architecture

The SNIF system is divided into several logical components as depicted in Figure 1. These components operate in a distributed manner, cooperating to provide the total SNIF functionality.  Filter. The lter process is the system's only direct interface with the network. The lter places the network interface card into promiscuous mode and then unobtrusively monitors all communication on the system's subnetwork. Each packet is run through a lter (con gured by the user) to determine if it is a packet of interest. If it is of interest, selected information is then passed on to the logger process, allowing the lter to handle the next network packet. This provides ecient processing of packets, reducing the likelihood that







packets will be lost. In addition, the interface is accessed using the DLPI library which also provides limited packet buering. A single lter is usually instantiated on one host per subnetwork. This process and the logger process are designed to minimize performance impact on the host machine. Logger. The logger process is the local system's SNIF manager. It acts as an intermediary between both the lter and the global system manager process. For eciency and security the logger is a child process spawned by the lter process, and communication occurs through a bidirectional UNIX pipe. The logger process also provides additional preprocessing of information from the lter. The logger collates information from the lter with information in a local NDBM database. This information is maintained locally until it is requested to send the information on to the collator process. Collator. This is the main system manager process. It communicates with all of the remote loggers, sending con guration information and collecting scanned information. The scanned information is then collated and combined with information in a global NDBM database. For space considerations and archival purposes, the NDBM database is divided into several subdatabases, based on the last time a packet was observed for a particular connection. This time division is speci ed by the user. Archival databases are stored in a compressed format, avoiding the \holes" typically found in NDBM database les which cause problems with some archiving utilities1. Monitor. This process is the system's only direct interface with the operator. It communicates with the collator when the user requests con guration changes, allowing the collator to send information on to the loggers. It also, independently, accesses the global database and displays the information in a graphical format. This access can be performed on a continual basis, providing a real-time interface. The monitor also provides the operator with signals and alarms for special events, and hooks into external user-provided analysis tools.

4.2 SNIF Concepts

When using SNIF, the user must be familiar with several dierent concepts. These concepts are what permit the user to con gure the ltering operation of the lters, the graphical representation of the scanned information, the triggering of alarms and the connection to external systems. Filters: One of the major concepts of the SNIF system is the lter; it is similar to lters seen in other communication tools. A lter consists of a speci c pattern that must be matched for a packet to be accepted

1 We have one le that contained 9MB of actual data, but the UNIX system reported that it contained 11GB.

Collator

Monitor Links to External Systems (SNIFLES)

Filter

Logger

Logger

Filter

Subnet 1

Subnet n

Figure 1: Overall architecture of SNIF system. and passed on from the lter to the logger. The user can de ne a speci c lter or use a prede ned lter. A lter is divided into several sub lters which each act on the dierent levels of the communication protocol hierarchy. The user can specify speci c Ethernet or IP host and network addresses, protocols (e.g., IP, ARP, TCP, UDP, etc.) or applications (e.g., telnet, ftp, www, etc.) The user may also specify a speci c pattern using byte osets into the protocol data unit. As of the writing of this paper no mechanism exists for specifying new protocols (speci cally header sizes) but this is planned for the near future. Data Collection: The system collects the following information for every connection seen:  Times: Connection start time and the time of last packet seen on the connection.  Information Flow: Number of packets and number of data bytes sent in each direction. This information is totaled for the whole connection as well as for 1-minute intervals (1 second intervals for the rst minute).  Protocol Information: Communication protocol in eect and the endpoint addresses and ports.  Status ags: Flags indicating various conditions of the connection such as if the protocol has terminated the connection, or if we have seen the full communication or started the lter too late.  User data: A section of user data that is updated with every packet. Views: A view is very similar to a lter, and can be speci ed in the same manner. The view de nes how the monitor lters information from the database. For example a user can de ne the view to see only those hosts using a speci c set of protocols, information about other hosts is not used to create the graphical display. To provide additional functionality, the

user can toggle certain options of the views, such as the display of a particular protocol. Alarms and Signals: An alarm or signal is also similar to a lter. In this situation, if a connection satis es a particular property (e.g., unauthorized ftp server on a local host) an event is triggered. The event is classi ed at one of four levels, Normal, Information, Warning, and Alarm. The rst three are signals and typically write a message to the system log le and window. At the user's discretion they may also popup an information window. The alarms are considered security critical situations and automatically pop-up an alarm window and a connection information window (see Section 4.3) associated with the triggering connection. External Processing: The SNIF Links to External Systems (SNIFLES) provide a mechanism where the user can attach security analysis tools for additional data processing and to receive alarm signals. As of the writing of this paper the system only permits the instantiation of the external systems and uses output of the system to trigger alarms and signals. The best way to make use of this mechanism is to write a wrapper that provides the interface between SNIF and the external system. We plan to provide closer functionality to existing IDS in the future.

4.3 SNIF User Interface

As the primary interface for the user, SNIF provides a graphical user interface to manage and monitor the system. A screen shot of the main screen can be seen in Figure 2. The screen is divided into several logical portions and is being continually modi ed based on user feedback: 

Command Menu: The command menu bar per-

mits the user to control several con guration options. The user can de ne the lters, views and alarms; indicate which hosts are running the remote loggers; de ne the time parameters for information update and connection expiration; and toggle speci c options for the graphical view.

Figure 2: An example main monitor screen. 





Host List: The host list is a scrolling list that

enumerates all of the hosts that have been seen communicating on the monitored system. Names from each host are provided using network name services. Upon selection of a host, the corresponding icon on the graphical view will be highlighted indicating the location of the host on the view. A user may double-click the host to obtain speci c connection information associated with that host (see Figure 3). Domain List: The domain list is a scrolling list that enumerates all of the domains (subnets) that have been seen communicating on the monitored system. This list is provided for future development which will enable the user to expand and condense all hosts of a speci c domain into a single icon on the graphical view. A double-click on the domain will also bring up an information window concerned with all communication to hosts within that domain. View Controls: The view controls permit the user to specify what information is made visible on the graphical view. This includes selecting which protocols are displayed using the check-boxes or which host-speci c connection to view using a pull-down menu. For example the user may wish





to see only FTP-DATA connection initiated at the selected hosts. Other commands are available to specify the con guration of the display and which hosts and subnets are displayed. Graphical View: The graphical view is one of the major components of the monitor. It provides the user with a con gurable real-time view of network activity. As mentioned previously, the user can select precisely what information is displayed and how much is displayed. There are various options for compressing the information displayed to provide a clear picture of the items of interest. The display also allows the user to select speci c hosts by selecting their icon. The selection of the icon is linked to selections in the host and domain lists discussed earlier. Selection in one will trigger selection in the others. Host Information Screen. Additional information about a particular connection or host can be found by selecting the speci c hosts of interest, using either the host list or the graphical representation of that host. The information associated with that host appears in a separate window as depicted in Figure 3. The user is provided speci c information about the host as well as for a each connection associated with that host. This infor-

Figure 3: An example connection information screen. mation includes the number of packets and bytes transmitted in each direction as well as times of observation. The data is presented in summary form as well as using a sliding bar graph showing the information collated in one minute intervals.

5.1 Lessons Learned

One of the concerns that came about during the development of this system is the security of the components. There are two main issues which need to be addressed: security of the data les, security of the communication links in the distributed system. Currently we provide the most rudimentary security in these areas. The data les are protected via the standard OS access control mechanisms (in other words not well protected), the communication is limited to very speci c ports and hosts. A logger process will not send information to any host except the one de ned at initialization. To further protect the system, the collator is run on a system that is con gured to allow minimal outside connections and communication protocols. Ideally we need a mechanism to provide mutual authentication between the loggers and collator, to secure the data les, and to secure the program components. We are currently investigating dierent methods for accomplishing this in conjunction with other projects at the University of Idaho.

Initial stages of this project attempted to use exiting network monitoring software such as \tcpdump" [10] and \netlogger" [11]. At that time, the inability of this software to run on certain architectures, or the use of hardcoded network protocol header sizes, made that code unusable. It was also decided that the development of all of the code in-house would permit better control over its dissemination. We also found, much to our frustration, that the amount of trac ow on the testbed network, and number of connections to remote hosts was much greater than the system support sta estimated. This required a complete reworking of the system architecture and user interface. An emphasis was placed on information compression and data summaries while still providing details when needed. Although we made every attempt to use standard libraries and interfaces to provide easy portability, we found that the industry had a long way to go in providing consistency. We spent tremendous time and eort in attempts to port the system from a SPARC station runing Solaris 2.3 to an HP 700 series machine running HPUX 9.x although both system are SVR4 compliant and provide DLPI and NDBM interfaces. We plan to attempt to port to other architectures in order to determine the lowest common denominator for the software to ensure portability.

In this paper we have described the motivations and goals of the SNIF project for surveying network information ow. It is our contention that existing system architectures and tools require that system administrators maintain awareness of the interaction between their site and hosts o-site. This tool is an attempt to provide that awareness in an easy-to-use fashion.

A prototype of the tool is currently in place and being used by system administrators to manage a local network consisting of a half dozen subnets with over 100 hosts. These hosts include UNIX workstations on 4 dierent architectures with even more variants of UNIX and also several personal computers. This prototype currently implements several of the features

4.4 Security

5 Conclusion and Future Work

5.2 Current Status

described above. Speci c advanced features, such as user de ned packet ltering, subnet compression, and enhanced graphical display are currently under development as of the time of the writing of this paper. We are continually working closely with the system administrators to improve this tool, adding new features and modifying the user interface. Given the availability of funding, we plan to employ a human factors researcher to assist in the user testing of this system and the development of the graphical interface.

5.3 Related Work

As mentioned previously, the computing environment at the site where this system was developed precluded the use of several of the existing network monitoring tools. This was due to either incompatability with existing systems or lack of support for desired analysis features. Current network monitoring tools can be classi ed into three categories:  Managers. Connection managers are designed to use existing management protocols (such as SNMP) to provide high-level statistics and system status information to the user. Information involving connections between hosts is not uncommon in these systems. However, these systems do not provide information at the protocol level, nor do they usually dierentiate between dierent types of connections but instead look at overall data ow. SunNet [8] is an example of such a system.  Dumps. This class of tools is grouped together because, although they usually provide the application protocol level monitoring, the output of the systems is typically presented to the user in a textual or binary trace of system activity. This dump of activity must be further processed by either the user or other programs. Typically, tools in the class are operationally similar to the SNIF lter process. Optionally they may have the ability to provide packet-decoding at the application level, providing a detailed analysis of the data

ow. Examples of such systems are tcpdump, and netlogger.  Monitors. This class of tools most closely matches the functionality desired of the SNIF system. These systems usually provide all of the functionality of the dump tools plus add additional analysis capabilities and an enhanced user interface. The tools that most closely resemble SNIF are Interman [6], and HP NetMetrix [4]. Both provide graphical views of the network (views which inspired our display format), user de ned application level ltering and alarms and signals. The netmon system is publically available and provides source code for a fee. Unfortunately the system does not work under SVR4. The HP NetMetrix system is commercially available, without source code. Neither system provides the hooks to external systems or the detailed view con guration that SNIF does. Another useful monitoring tool is NSM [2] which does not provide the

graphical user interface we feel is necessary for these tools. However, NSM is part of the DIDS system [7] which as a whole may provide a better user interface.

5.4 Future Work

There is much more work to be done on this tool and related research tools. We see SNIF as a growing system, developing into a collection of linked detection systems, managed by a central monitor such as the one that controls SNIF. SNIF is one of a suite of security tools being developed at the University of Idaho. Speci cally we will be focusing our near term research and development for SNIF in the following areas:  User Interface. The user interface still must undergo extensive user testing and improvements. Throughout eld operations we will be collecting comments from users to help improve the interface to provide the administrator with a much better \summary at a glance".  Evaluation of scalability. One of the goals of this project is to provide a tool that is scalable up to dozens or hundreds of subnetworks. More work is needed in simulation and eld testing of the scalability of this system.  Extension of monitored protocols. Currently the communication protocols speci cally detected by this system are the common TCP application protocols. Expansion of this list of recognized protocols to include more TCP applications, UDP applications and other non-TCP/UDP protocols is currently underway.  Archiving of collected data. Although much data compression occurs, we nd that this system still generates a large amount of information per connection. This information is directly proportional to the duration of each connection due to the 1minute data intervals. We are in the process of developing a set of tools to permit the archival (and retrieval) of outdated connection information.  External system tool-kit. Although we provide the SNIFLES interface we have yet to develop a suite of tools that takes advantage of the interface. Our vision is that each tool will perform a speci c analysis and that the total collection of linked detection systems, tied to SNIF with the SNIFLES interface, will provide the user with the type of analysis needed to ensure the security of their system.  Links to speci c intrusion detection systems (IDS). Including the suite of tools we will build, we also plan to develop a collection of interface tools to permit our system to interact with existing IDS, such as DIDS and NIDES [5], thus allowing us to utilize the expert systems and additional auditing capabilities of these systems.





Links to data-based monitoring. One problem with a connection-based monitoring system, such as SNIF, is that it does not provide information when a particular data item is copied or referenced, such as the password le. To include these capabilities, the system can monitor for speci c byte patterns or in the case of NIS, protocol packets, and allow additional links to speci c host's monitoring tools. Compatability with enhanced network protocols. There is much work being done on the development of secure network protocols. It is possible that in the near future a secure IP or TCP protocol may exist which will encrypt that data in the packets. If this happens much of the information this system currently collects may no longer be available. We are keeping abreast of these advancements and will make changes to this system as necessary. This may involve writing speci c wrappers for each host to record application level information, or to remove these details from the system. One other possibility is to apply some form of probabilistic analysis to estimate the protocols.

References

[1] J. Case, M. Fedor, M. Schostall, and J. Davin. Simple Network Management Protocol (SNMP). RFC 1157, SNMP Research, Performance Systems International, MIT Laboratory for Computer Science, May 1990. [2] L.T. Heberlein, G.V. Dias, K.N. Levitt, B. Mukherjee, J. Wood, and D. Wolber. A network security monitor. In Proc. IEEE Symposium on Security and Privacy, pages 296{304, 1990. [3] D. Heller. XView Programming Manual. O'Reilly & Associates, Inc., 1993. [4] Hewlett Packard. HP NetMetrix Applications Users Guide, 1994. [5] R. Jagannathan, T. Lunt, F. Gilham, A. Tamaru, C. Jalali, P. Neumann, D. Anderson, T. Garvey, and J. Lowrance. Next-generation intrusion detection expert systems (NIDES). Technical Report Project 3131, SRI International, September 1992. [6] M. Schulze, G. Benko, and C. Farrell. Homebrew network monitoring: A prelude to network management. Technical report, Curtin University of Technology, Department of Computer Science, March 1993. [7] S. Snapp, J. Brentano, et al. DIDS (Distributed Intrusion Detection System), motivation, architecture, and an early prototype. Proceedings of the 1991 National Computer Security Conference, 1991. [8] Sun Microsystems Inc. SunNet Manager 2.2 Reference Guide, 1993.

[9] Sun Microsystems UNIX Manual. NDBM Library functions. [10] Sun Microsystems UNIX Manual. Tcpdump. [11] Texas A&M University, Department of Computer Science. Netlogger manual pages. [12] UNIX International. Data Link Provider Interface Speci cation 2.0, August 1991.