discusses a NetFlow & sFlow collector and analyzing system using Web-based architecture as well as data mining approaches. The following sections give the.
The 9th Intemational Conference on Computer Supported Cooperative Work in Design Proceedings
A Flow-based Network Monitoring System Used for CSCW in Design Bo Yang'.*,Yi Li2, Yuehui Chen', Runzhang Yuan' 'State Key Lab. of Advanced Technologyfor Materials Synthesis and Processing, Wuhan Universi?y of Science and Technology, China 2Schoolof Information Science and Technology, Jinan University, China {yungbo, csmaster, yhchen,@ujn. edu.cn In recent years NetFlow and sFlow are becoming industry standard protocols used in this scenario. Among today's flow-based traffic monitoring technologies, Cisco NetFlow [I] efficiently provides the metering base for a key set of applications including network traffic accounting, usage-based network billing, network planning, as well as Denial of Services (DOS) monitoring capabilities, for both working groups and enterprise customers. NetFlow provides valuable information about who is using the network, what applications are used, when the network is utilized and where traffic is going on the network. The same mechanism includes sFlow developed by HP, InMon, Foundry and Juniper, which gives much flexibility to gather multi-layer network flows. The latest innovation in NetFlow is NetFlow version 9, a flexible and extensible method to record network performance data. Cisco is currently working with a number of partners to provide customers with comprehensive solutions for NetFlow-based billing, planning and network monitoring in high-speed campus networks. Although most of today's high-end network routers and switches support NetFlow or sFlow, there is still only few flows were gathered and analyzed due to the lack of real-time flow monitor systems. So, the development of a flexible, web-based net flow analyzing system is essential for the monitoring of today's cooperative environments. This paper is organized as follows. The background of network monitoring is introduced in this section, and some recent approaches or relative works are summarized in Section 2. Section 3 describes the structure of the embedded flow agent used to create the flow records, and section 4 gives the design of a webbased system for flow monitoring based on Cisco NetFlow version 5 . Finally, a brief conclusion is given in Section 5.
Abbstract Technology trendr in todoy 's cooperative design environments are making it more and more important to monitor the network performance ond ensure the network securi9. This paper describes the design and implementation of U distributed nehvork trofic monitoring system based on embedded NetFlow hardware and software engines. The system architecture and design principles were introduced in the paper, some discussions were also presented about the NetFlow-based network monitoring technologies. The Jystem had been successfully used to monitor highspeed campus networks ut f i l l wire speed without pocket sampling in scenarios where commercial NetFlow collectors could not be used due to their limitations. Results show that this is an effective mechanism to identify, diagnose, and determine controls for network activities in CSCW environments and other network-based applications. Keywords: NetFlow, Network Monitoring, CSCW
1. Introduction With the ever-increasing reliance on network services for cooperative design applications, there is a growing interest in an effective way to monitor network activity in order to get the network performance or security situation. The traditional method to gather the network information is packet sampling via probes such as W O N and capture TAPS. This worked very effectively in traditional shared-media networks where a single instrument can monitor all the traffic in a whole sub-network. However, with the trend towards switched, peer-to-peer networks, every port on a switch would have to be monitored to achieve the same visibility to network traffic. In addition, switches and routers make packet-forwarding decisions that affect the flow of traffic through a network. Understanding these traffic flows i s critical to maintaining visibility to network utility. So, implementing traffic monitoring within a switch or router is an effective way to monitor traffic on all ports and all VLANs.
2. Flow-based network monitoring and
relative works Traditional probes such as W O N and TAP in network devices gives only a partial view of traffic, where the basic frame and byte counters maintained by each network device can be polled via some traditional protocols such as SNMP. This gives an aggregate count
503
The 9th International Conference on Computer Supported Cooperative Work in Design Proceedings
.
’
of frames and bytes passing through an interface in network device, In this method, if a single host is attached to a switch port, the total volume of traffic to that host will be measured. If multiple services are hosted on that machine (e.g. email, web, telnet services in same server), then it is not possible to distinguish between traffic destined for the different users or services. Simitarly, it is not possible to distinguish between Internet web traffic and local file-sharing traffic and therefore bill differently for local and nonlocal traffic in a cooperative environment. In practice errors may also occur with this methodology, for example counters may wrap or line cards may be reset. In such a situation flow data may be lost. Flow-based sampling as En embedded network traffic monitoring technique is now compelling. In a switched environment, the most effective place to monitor traffic is within the switchhouter, where all the traffic of each VLAN can be seen. There were some technologies to monitoring today’s network activity using embedded packet or flow-based agents, and this type of traffic monitoring solution embedded within a switch or router must not impact forwarding performance. This solution provides detailed and quantitative traffic measurements, at gigabit speeds, gives insight into forwarding decisions, and does not impact forwarding or network performance. Among these technologies, Cisco NetFlow is a widespread standard for network traffic accounting [I]. Leading network manufacturers such as Cisco, Juniper Network and Extreme Networks provide NetFlow agents as part of their internetworking operating systems with Layer24 functions. The other competitive multi-vendor technology, sFlow (FSC 3176) [2], is a packet-based sampling technology consists of a packet sampling algorithm, typically performed by the switching/routing ASICs, and a embeded sFiow engine that runs as part of the network management software within the device. The sFlow engine combines flow samples (generated by the packet sampling fimction), interface counters, and the state of the forwarding/routing table entries associated with each sampled packet, into an sFlow datagram which is immediately forwarded to a central sFlow collector. This means that the sFlow agent does very little processing of the data and minimizes the CPU and memory utilization. Meanwhile, a central sFlow collector receives a continuous stream of sFlow datagrams from across the entire network and analyzes them to form a rich, real-time view of Layer2-7 traffic flows across the entire network. There are several NeFlow or sFlow collectors available in today’s software world. They range from simple “capture and store the flow into a database” collectors to complex applications such as cFIowd [3] and FlowScan [4]. Unfortunately most of the collectors have been designed for network experts so that very few network administrators really use NetFlow. The consequence of all this, is that most of the people still
monitor their networks using SNMP MIB-I1 interface counters [ 5 ] and MRTG [6]. In this paper, the author discusses a NetFlow & sFlow collector and analyzing system using Web-based architecture as well as data mining approaches. The following sections give the design and the implementation of the system and show how this system can be effectively used to monitor network performance.
3. Flow agents in routers and switches NetFlow & sFlow are effective sampling technologies embedded within core switches and routers. They provide the ability to continuously monitor multi-level traffrc flows at wire speed on all interfaces simultaneously. A Flow is a set of packets passing an observation point in the network during a certain time interval. A11 packets belonging to a particular flow have a set of common properties derived from the data contained in the packet and from the packet treatment at the observation point In today’s TCP/IP networks, the flow record was formed using some of the common field properties in an IP packet, such as Sourcehlestination IP-address, Protocol IDS, Data Bytes, and Port Numbers.
3.1. Overview of SHOW agents
Fig.1 Flow engine embedded in switchhouters
Fig.1 gives the structure of flow engine embedded in switcWrouters. The flow engine is a software process that runs as part of the network management software within a device. It combines interface counters and flow samples into flow datagram that are sent across the network to a Flow Collector (or DB Server). Packet sampling is typically performed by the switchinghouting ASICs, providing wire-speed performance. The state of the fonvardingkouting table entries associated with each sampled packet is also recorded. The flow engine does very little processing in the switch. It simply packages data into flow datagrams that are immediately sent on the network via UDP datagrams. Immediate fonvarding of data minimizes
504
The 9th IntemationaI Conference on Computer Supported Cooperative Work in Design Proceedings
memory and CPU requirements associated with the flow engines.
storage and the SQL-based database systems are adopted to store the flow data.
3.2. Aggregating traffic data on a switch
Web Server
NetFlow operates by accumulating traffic flow totals into an embeded onboard flow cache. For the most detailed data, totals are accumulated for unique flow of source IP address, source UDPflTP port, destination IP address, and destination UDPiTCP port. This method requires a variable, but significant amount of memory, especially under high load conditions. For example, during a denial of service attack when every packet i s a separate short-lived flow there may be 30,000 flows per second and the switch must export data rapidly to avoid flow cache overflow. In such a situation flow data may be lost or dropped. To solve the problem, flow aggregation was introduced to merge the flow records generated from different VLANs according to some predefined rules. Because flow aggregation is performed on the switch before being sent to a central data collector, a single measurement can represent a significant fraction of the overall traffic. If a packet containing this data is 1ost;the accuracy of the overall measurement will be impacted. The impact of these situations is impossible to quantify, and therefore the final accuracy of the measurement cannot be characterized,
SwitchlRouter
Browser
EEEJ 0
4
Data Mining ti App. Sewer
Fig.2 Components of the flow monitoring system Recently there has been much interest in applying data mining to computer network monitoring and intrusion detection. To ensure the security of the network, a hybrid intrusion detectian scheme is also used based on distributed flow monitoring and data mining [9]. Classification algorithm is used to assign flow data into pre-defined categories. Machine learning software modules perform this task by extracting or learning discrimination rules from examples of correctly classified data. Classification modules can be built using a wide variety of algorithms [ll], and each of them gives different accuracy and real-time performance. Because data mining procedures are very CPU and memory intensive, several web and data mining servers were adopted to improve the system activity.
4. Design of a Web-based flow analyzing system
.
000000 00
Switch/Router
The flow monitoring & analyzing system discussed here consists of three parts: The embeded Flow engines in router and switches, the DB server (collector), and several Web-based application servers for data processing and mining. NetFlow engines embeded in switchhouters throughout the network continuously send a stream of flow datagrams to a central NetFlow Collector (DB Server) where they are analyzed by some application servers to produce rich, real-time, networkwide view of traffic flows. The web-based architecture gives more friendly interface to the operators, and also more suitable for data integration under different operation systems. This also makes it possible for the network administrators to monitor their networks from a remote site by using navigator software such as Netscape or Microsoft Internet Explorer. Fig.2 shows the main elements of the flow monitoring system. There may be several routerdswitches spread all over the campus network or a CSCW-based design team. Real-time flow records are sent by the switcldrouters, collected by a central DB server and stored in disk arrays. Because the flow data may be very large, some compress method such as CaRT [lo] was used. To ensure the real-time performance of the system, both the indexed file-based
4.1. NetFlow formats and data structures The exported Neff low datagram consists of a header and a sequence of flow records. Every flow datagram was encapsulated into a UDP packet with up to nine data formats according to Cisco 1 0 s specifications. The Version I format was the original format supported in the initial Cisco 10s software releases containing NetFlow functionality. The Version 5 format is a later enhancement that adds Border Gateway Protocol (BGP) autonomous system information and flow sequence numbers. The Version 7 format is an enhancement that adds NetFlow support for Cisco Catalyst 450015000 series switches equipped with a NetFlow feature card (NFFC). Versions 2 through 4 and Version 6 were either not released or are not supported by some of ffow collectors. Version 8 is the NetFlow export format used when the router-based NetFlow aggregation feature is enabled on Cisco 10s router platforms. Most of the NetFlow versions are also supported by the popular Catalyst 6500 enterprise switches equipped with
505
The 9th Intemational Conference on Computer Supported Cooperative Work in Design Proceedings
different types of supervisor engine such as Sup I, Sup
application servers. Fig.3 shows the sofhvare structure of the flow monitoring system. The DB server is running Windows2003 and the application servers are running Windows or Linux systems. Apache and Tomcat was selected as the Web server.
11, and Sup 720.
The proposed NetFlow export version 7 header format and data format are shown below (using C language): struct Head-Record
I
UDP Flow Datagram
ushort version; /I Current version=7 /I No. of records in PDU ushort count; ulong SysUptime; I1 Current time in msecs dong unix-secs; // Current seconds since UTC 1970 dong mix-nsecs; //Residual nanoseconds since UTC 1970 ulong flow-sequence; /I Sequence number of total flows seen d o n g reserved;
Analvzer
UDP Decoder Get Flow Records
ScriptGerAets
.
1;
Socket API
Data Compression
cir
struct Data-Record:
. r
DB System
ipaddrtype srcaddr; ipaddriype dstaddr; ipaddrtype nexthop; ushort input; ushort output; ulong dPkts; ulong dOctets; ulong First; ulong Last; ushort srcport; ushort dstport; uchar flags; uchar tcp-flags; uchar prot; uchar tos;
dong src-as; ulong dst-as; uchar src-mask; uchar dst-mask; ush~rtpad; ipaddrtype router-sc;
Requests
/ I Source IP Address I/ Destination IP Addr. J l Next hop router // Input interface index // Output interface index //Packets sent in duration I1 Octets sent in duration I1 SysUptime at flow-start I/ SysUpTime at flow-end. // Source port or equivalent I/ Dest port or equivalent /I Shortcut mode flag // TCP flags
08 Sewer
Web Server
Fig.3 Software structure of the system
The flow-monitoring system is very usefui for the network administrators or designers to monitor and optimize their network performance. The other typical application of such a system is billing/accounting. In a typical billing'accounting application, the objective is to determine from all the packets crossing the network during the billing period (maybe 1 month), how many packets came from a particular source. For example, a common billing strategy is a flat monthly charge that includes an allowance (say 4 GB) with additional charges for each additional GB. In this scenario, the system must store enough records in the database system to meet these needs. To reduce the storage space needed, we merged some of the flow records according to some predefined conditions such as minimal flow size, certain source and destination IP addresses, specified VLANs, etc. Data compress technology was also used for this purpose.
/I Protocol, 6=TCP, I7=UDP. // IP Type-of-Service // Source AS number /I Destination AS number // Source subnet mask //Destination subnet mask //Padding
1; 4.2. Software design
5. Conclusion
When the network switch has been configured properly and the flow data has been exported from the embedded NetFlow engines using UDP datagram, a data processing and analyzing system is needed to give the network administrator some reasonable results under certain conditions. To do, a DB server is needed to collect and store the data, and one or more application server such as Web Server or Analyzing Server will also be needed. Both socket-API and SQL-engine are used for the communication between DB server and
Because flow-based sampling is an inexpensive monitoring technology that easily keeps up with today's high-speed switched networks, flow-based technology efficiently provides the metering base for a key set of applications including accountinghilling, network planning, network monitoring and outbound marketing for both service provider and enterprise customers. Many manufacturers also provide a set of NetFlow & sFlow management utilities to collect flow export data, perform data volume reduction, post-processing and storage and make flow detail records available to
506
The 9th International Conference on Computer Supported Cooperative Work in Design Proceedings
consumer applications in a convenient format. As part of the development of OUT network management system in the university campus network, we adopted a webbased architecture and made a prototype of NetFlow network monitoring system to monitor our campus network and inspect network activity for some certain groups under CSCW environment, Results show that it is very suitable for network monitoring and billing, rating and provisioning. The flow-based network monitoring scheme had been used in Jinan University for the development of several application systems such as distribute cement .material simulation, grid-computing, and campusnetwork management. Some future works are still under development to get detailed characterization of packet flows that supports itemized audit for different purpose and intrusion detection on many levels.
References [ I ] Cisco 10s NetFlow Technology Data Sheet, online at h~://www.cisco.comlao/netflow [Z] P. Phaal, S. Panchen, N. McKee, “A Method for Monitoring Traffic in Switched and Routed Networks”,IETF RFC 31 76, September 200 I. [3] “Cflowd‘’.htta:/~.cwarie,ca/canet4/mo~t~e/c8awdhcml [4] “FlowScan”. http:/hYww.caida.or~/tools/utilities/ffowscan/ [5] K. Mcaoghrie, M.T. Rose, “Management Information Base for Network management of TCPIIP-based Intemets: MIB-11”, RFC1213, March 1991. [SI T. Oetiker, “Multi Router Traffic Grapher (MRTG)”, online uzhttp://w.mrta.orn/. [7] L. Deri, R. Carbone, and S. Suin, “Monitoring Networks Using Ntop”, Proc. of MZOO1, Seattle, May 2001. [XI Network Research Group, “libpcap”, Lawrence Berkeley National Labs, online at httn://www.tcadurnv.org/ [ 9 ] Yang Bo, Li Han, et al. “A hybrid intrusion detection saategy used for web security”,Lecture Nofes in Arrificial Intelligence, 2 6 3 9 730-733.2003. [lo] L.Breiman,J.H. Friedman,R.A. et al. “Classification and Regression Trees”, Chapman & Hall, 1984 [ 113 Mnos Garofalakis,Rajeev Rastogi, “Network Data Mining and Ana1ysis:Tbe NEMESIS Project”, mite Paper, Bell Labs, Lucent Technologies.
Acknowledgement This work was supported by the National Natural Science Foundation of China under contract No.69902005; the National High Technology Development Program of China (863 Program) under contract No.2002AA423240; and the Science and Technology Development Project of Shandong under contract No.2001G09 and No.SDSP2004-0720-03.
507
