Efficient Cluster Security Gateway Architecture for per-packet load ...

Efficient Cluster Security Gateway Architecture for per-packet load balanced IP filtering on switched clusters V. Ramsurrun email: [email protected]

K. M. S. Soyjaudah email: [email protected]

Electrical and Electronics Engineering Department, University of Mauritius Mauritius

Abstract: This paper describes the design and implementation of a new firewall architecture by applying the concept of security gateway to switched clusters of computers. The idea is to build a cluster security gateway that performs per-packet filtering on IP traffic emanating from or going towards, the switched cluster. To this end, a combination of tools like the Linux firewall facility, IPTables, Ebtables, Bridge-utils, features like Cisco switch Access-Control Lists (ACLs) and Routing Information Protocol (RIP) per-packet-based load balancing on a Cisco router, are used.

I. INTRODUCTION Host-based firewalls have been developed to ensure the protection of individual hosts, whether it is a standalone PC or a network host. They also form the base upon which distributed firewalls have been created. However, the use and management of host-based firewalls is not always feasible from an economic, administrative or operational standpoint, especially when it comes to securing multiple hosts on a network. There are various problems associated with hostbased firewalls. First, there is a degradation of host performance. Since the packet filtering has to be done by the host itself, the load on hosts with limited resources is increased. In addition, with the implementation of other security tools at the host level like real-time host-based intrusion detection systems and the Psionic Sentry tools, as in the security model devised by M. Gangadharan and K. Hwang [1], the host will be heavily taxed. The micro-firewall module found on each end-user node in their system, made use of security tools like IPChains for packet filtering, Linux Intrusion Detection System (LIDS) for anomaly detection and LogCheck for access logging and auditing purposes. Mobile agents were also used to create a distributed IDS with the individual LIDS on each end-user node. All this added considerable strain on the nodes. Second, according to Wei Li [2], user tampering of firewall represents the biggest problem in distributed firewalls. Some users require administrator privileges to work. These users can modify the host-based firewall rules at will, or worse, completely remove the firewall and thereby expose those hosts to attacks. Hackers can, in turn, use those hosts as base for launching attacks from inside the network. Both internal hosts, and remote hosts in external networks, can be attacked. A. D. Keromytis and J. L. Wright [3] came up with an approach that solved these problems through the utilisation of

security gateways that enforce the relevant security policies and rules on behalf of the protected hosts. The security gateway can apply security policies devised specifically for that cluster. It caters for both Layer 2 and Layer 3 security, thus ensuring greater protection. Their design entailed a host that acted as a bridge/firewall with multiple nodes connected to it. The bridge/firewall host filtered traffic to and from the protected nodes. However, it had certain flaws that made it quite impractical for modern, real-life networks. First, the scalability of the system is poor because a host can contain a limited number of network interface cards (NICs), and hence, service a limited number of hosts. This limitation highly increases design complexity in networks having hundreds or thousands of hosts. Second, the use of a single host that acts both as a bridge and as a firewall can cause degradation in network performance for that cluster and can even prove to be a network chokepoint. The bridge/firewall host most definitely does not give as good performances as those of state-of-the-art switches that have now become key components of a modern-day network. Moreover, the use of computationally intensive security protocols like IPSec can tax the network even more [1]. The problem is exacerbated when the end-user utilises some distributed software. Third, applying such a security architecture in an already-in-place network requires significant re-design, which is timeconsuming and costly. In this work, the problems highlighted in the two security models described above are overcome while at the same time retaining their strong points. A load-balanced cluster of two hosts acts as the security gateway that is dedicated to Layer 3 traffic filtering. Load balancing is used to prevent the security gateway from becoming a chokepoint and it also removes the strain of packet filtering from the individual end-user nodes. The use of a dedicated hardware Layer 2 switch allows for scalability in terms of the number of connected end-user nodes, high performance and it also fits well in modern-day context, given the ubiquity of hardware switches. II. PACKET FILTERING ON A LAYER 2 SWITCH A switch is a networking device used for increasing available bandwidth and for reducing network congestion. It uses Layer 2 addressing for forwarding frames. A switch learns the MAC addresses of PCs or workstations that are connected to its ports by examining the source MAC

The financial support of the Tertiary Education Commission of Mauritius is gratefully acknowledged.

addresses of frames that are received on those ports. These learned source MAC addresses are then recorded in a switching table. When a switch receives a frame, it looks at the destination MAC address in the frame header and then forwards the frame to the appropriate switch port or interface based on the MAC address information stored in the switching table [4]. When two hosts that are connected to a switch want to communicate, the switch creates a virtual circuit. Upon creation of the virtual circuit, a dedicated communication path is established between the two devices [4]. Some switches like Cisco 2970 Catalyst switches provide ACLs for Layer 2 and Layer 3 traffic filtering [5]. However, these are not as elaborate and powerful as the full-fledged Layer 3 packet filtering provided by tools like IPTables. In the design presented by A. D. Keromytis and J. L. Wright [3], the bridge, which was serving as a firewall as well, acted directly upon the traffic flowing through it. The concept of security gateway held because the bridge/firewall host was performing the packet filtering on behalf of the end-user nodes. The bridge part provided network connectivity while the firewall part provided Layer 3 filtering. However, this approach is difficult to realise using a switch because the switch is a dedicated hardware device with its own software and it does not use IPTables.

Connection tracking is implemented by the ip_conntrack module in the Linux kernel. TCP and UDP connection states are maintained in /proc/net/ip_conntrack [9]. IPTables allows the use of connection tracking through its state option. For protocols like TCP, UDP and ICMP, connection tracking support is in-built in the conntrack framework. However, connection tracking does not play well with per-packet-based load balancing because of the in-built mechanism for handling TCP, UDP and ICMP traffic. For example, for a TCP connection to be considered ESTABLISHED, the SYN, SYN/ACK and the ACK packets must all pass through the same firewall node. If any one of these packets is not seen by the firewall node, then the status of the connection will never become ESTABLISHED. The Netfilter team is still working on ways to make stateful firewalling and per-packet load balancing compatible.

IV. DESIGN OF PROPOSED CLUSTER SECURITY GATEWAY ARCHITECTURE (CSG) Because direct IPTables packet filtering on a switch is not possible, the following architecture, as shown in Figure-1, was devised to cater for both Layer 2 and Layer 3 security. Load Balanced Stateless IPTables Firewall Nodes Host B

III. STATEFUL FIREWALLING & LOAD BALANCING IPTables is the interface to the networking subsystem called Netfilter found in the Linux kernel. Netfilter is used for stateful/stateless packet filtering, NAT and IP masquerading [6]. IPTables allows the creation of firewall rulesets against which packets are checked. The rules are used to determine whether to accept or block a packet. The rules associate an action, usually accepting, dropping, or rejecting packets, with a set of packet characteristics. A stateful firewall is one that keeps track of connections passing through it [6, 7, 8, 9]. This method of inspecting and restricting connections is known as connection tracking [8, 9]. Stateful firewalls do so by maintaining state information about a connection in a database known as a state table. To maintain this communication history for each connection, state information such as source & destination IP addresses and port number pairs (known as socket pairs), protocol types, connection state and timeouts are stored in the state table. The stateful firewall uses the state table to determine whether an incoming packet is part of an established authorised communication, is starting a new one or is an invalid packet. The stateless firewall does not keep any communication history for connections. Because the stateful firewall looks at the contents of packets and then tries to associate them with existing connections, it is more secure than its stateless counterpart [6, 7, 8, 9]. The state table defines four connection states: New - firewall has seen traffic in only one direction Established - two-way communication has taken place Related – a connection that is related to an already established connection Invalid - packet cannot be associated with any existing entries in the state table and thus cannot be identified [6, 8, 9]

The Internet

FW1

eth0

eth0

eth0 FW2

eth1

eth1

eth0

Gateway Firewall

Cisco Switch

Cisco Router

Host A

Fig. 1. The Cluster Security Gateway Architecture for IP packet filtering on switched clusters

The MAC addresses of the different NICs are as shown in Table 1: TABLE I M AC ADDRESSES ASSIGNED TO THE DIFFERENT NICS

End-user nodes Cluster Security Gateway

NIC Host A_eth0 Host B_eth0 FW1_eth0 FW1_eth1 FW2_eth0 FW2_eth1

MAC address E1 E2 E3 E4 E5 E6

All IP traffic from the end-user nodes, that is the hosts connected to the Cisco switch, are forced to pass through the two firewall nodes, FW1 and FW2, that form the CSG. The traffic is balanced onto FW1 and FW2 on a per-packet basis in a round robin fashion. It is worth noting that it is the enduser node that makes the load balancing decision whereas the firewall on FW1 and FW2 does the packet processing, that is, filters the traffic, and based on the destination IP address, sends the valid packets out of the appropriate interface.

V. TRAFFIC SCENARIOS Two possible traffic paths can be identified in this proposed system. They are intra-cluster and intercluster/remote communication. It is worth noting that for both types of communication, the traffic is forced to pass through the 2-node CSG. For intra-cluster communication, that is, between end-user nodes such as from Host A to Host B, Host A traffic is directed towards the eth0 interfaces of FW1 and FW2. If the traffic is valid, it is then sent back through the same eth0 interfaces to the switch, which then forwards it to Host B. The reply from Host B goes through the same procedure too, that is, from Host B onto FW1_eth0 and FW2_eth0 through the switch; then, upon acceptance by the firewall, the reply packets go through FW1_eth0 and FW2_eth0 onto the switch again before ultimately coming to Host A. For inter-cluster/remote communication, that is, between end-user nodes on different clusters, or between an end-user such as between Host A and some remote server on the Internet, the traffic from Host A goes through the two firewall nodes which, upon acceptance, will be forwarded from interface eth0 to interface eth1 on both FW1 and FW2, and then onto the router. The reply from the remote server goes in the reverse direction.

VI. IMPLEMENTATION The system was implemented and the major components of this CSG architecture and their configurations are given below: A. End-user Node Configuration All IP traffic emanating from any end-user node is forced to pass through the firewall hosts FW1 and FW2 using the MAC DNAT feature of Ebtables. To this end, two software packages, Ebtables and bridge-utils, had to be installed. These packages are user space administration utilities. The 2.6 kernels now provide kernel support for both of these packages [10]. Ethernet bridge tables (Ebtables) is analogous to IPTables and much of it was written by Bart De Schuymer [11]. It is a Layer 2 filtering utility where filtering is primarily focused on Ethernet frame fields. The Ebtables code is used together with bridging code in order to build transparent bridging firewalls and brouters. Because Ebtables work only with the bridging code, bridge-utils, which is used to build and administer bridges, was installed as well [12]. It was installed with the stp feature turned off. The MAC DNAT feature, which allows the modification of MAC addresses in Ethernet frames, was used to achieve load balancing. However, the dnat target of Ebtables supports only one destination MAC address [13]. Hence, the Ebtables code was enhanced in order to cater for two destination MAC addresses. A new target, lbdnat, was created. The destination MAC address of all the frames leaving the end-user host is changed to that of the firewall hosts FW1 and FW2, that is, E3 and E5, in a round robin manner on a per-packet basis. The rule looks as follows: root# ebtables –t nat –A OUTPUT –p ipv4 –o eth0 –j lbdnat –to-lbdst E3,E5

This makes per-packet load balancing possible, which prevents the CSG from becoming a chokepoint for this cluster. This not only boosts the efficiency of the firewall, but also the network performance in this particular cluster, and consequently, in the whole network. However, the destination IP address is not changed. Hence, the frames are forwarded to FW1 and FW2 even if the IP address is not that of the firewall hosts. This is because the Layer 2 switch only works with MAC addresses and uses Layer 2 addressing when forwarding traffic. It knows nothing about Layer 3 addressing. B. Cluster Security Gateway Configurations FW1 and FW2, the two firewall nodes used to build the CSG, have the task of enforcing security policies for the switched cluster. The role of the CSG is to filter not only any traffic on the switch coming from the connected end-user nodes but also traffic to and from the router. Both firewall nodes have two network interface cards (NICs). A restrictive [6] stateless IPTables firewall is used in order to permit load balancing of IP traffic to take place. Although a stateful firewall is more secure than its stateless counterpart, it does not allow for per-packet load balancing on a 2-node CSG since none of the firewall nodes will see all of the four packet types used in TCP connection establishment and termination, namely SYN, SYN/ACK, ACK and FIN, and also the TCP RESET packet that is used for abruptly terminating a connection. [14]. C. Switch Configuration The Cisco switch used is a Catalyst 2970 switch. ACLs are configured for filtering traffic passing through the switch. An ACL is basically a sequential list of permit or deny statements that apply to addresses or upper-layer protocols [4]. An Ethernet ACL is used for filtering Layer 2 traffic [5]. The ACLs are applied on all interfaces present for inbound directions, except on those to which the firewall hosts are connected. As packets enter the switch on an interface, ACLs associated with inbound traffic are checked. A named MAC extended ACL is created for filtering Layer 2 traffic using MAC addresses. This ACL will permit only frames that have the MAC addresses E3 or E5 as destination MAC address. Any other MAC address will cause the frames to be dropped. The part of the ACL responsible for allowing only those frames looks as follows: Switch (config)# mac access-list extended mac1 Switch (config-ext-mac1)# permit any host E3 Switch (config-ext-mac1)# permit any host E5 There is an implicit deny any statement at the end of the ACL by default, which prevents unmatched packets from being accepted [4]. D. Router configuration The Routing Information Protocol (RIP) is used to perform load balancing on the Cisco 2600 router. RIP is a distance vector routing protocol that uses hop count as metric for making routing decisions. If multiple paths to a destination exist, RIP selects the path with the least number of hop counts [4]. If two equal-cost routes to the same destination exist, RIP chooses any one of them. The load balancing functionality is a standard component of the Cisco IOS router

software and it is automatically activated when multiple paths to a particular destination are detected [4]. Two types of load balancing are possible:
Per-destination - where the router chooses the path of packets based on the destination address
Per-packet - where the router chooses the path to be taken for each individual packet The method chosen for load balancing on the router was per-packet-based because it allows a more equal usage of the links than per-destination-based load balancing. The perpacket-based load balancing is done in a round robin fashion. By default per-destination-based load balancing is enabled on most Cisco routers [15]. To enable per-packet-based load balancing on a router interface, the following command is issued: Router (config-if)# no ip route-cache

VII. DESIGN AND TESTING A. Design Effectiveness The Trusted Computer System Evaluation Criteria (commonly known as the Orange Book) highlights three essential qualities of security mechanisms. They are:
Correct implementation
Non-bypassability
Tamper-resistance [16] The Ebtables configuration on end-user hosts and the switch ACL, together, allow the above-mentioned desirable qualities to be achieved. The Ebtables configuration forces all IP traffic to pass through the firewall hosts FW1 and FW2. The switch ACL ensures that it remains so by permitting only frames that have the MAC addresses E3 or E5 as destination MAC address. If an end-user has got administrator privileges on one of the end-user nodes and decides to modify or delete the Ebtables configuration, this will result in the user being totally incapable of sending any IP traffic to the other enduser nodes or to remote nodes. Moreover, if the end-user tries to spoof source IP address to being that of the firewall hosts (as if acting like the firewall hosts), the destination MAC address of the frame will still have to be either E3 or E5 for it to be accepted and forwarded by the switch. Hence, when the spoofed packets arrive at the CSG, they will be dropped when they are verified by the IPTables firewall. Also, since the traffic will be filtered by the CSG first, the end-users can choose not to set up a firewall on their computer. Another advantage is the possibility of performing egress filtering

effectively and efficiently. Egress filtering basically is the task of filtering traffic that is being allowed out of one’s network, that is, outbound traffic. This aspect of traffic filtering, and in a broader sense, network security, is often neglected. Egress filtering is very important in the sense that it makes one’s network much less attractive to attackers who are in search of potential relay sites for launching their attacks [17]. In the proposed architecture, egress filtering is performed at cluster level. Cluster-level egress filtering allows for robust packet filtering without penalising end-user nodes with additional firewall processing while at the same time protecting clusters and their end-users from unscrupulous internal users. B. Performance Testing The test setup consisted of the systems shown in Fig. 1. with the following environment:
All systems had RedHat Linux 9.0 operating system.
The end-user hosts A & B used an upgraded kernel of 2.6.13 and had ebtables-v2.0.6 & bridge-utils-1.0.6 installed. The hardware configurations of the PCs were as follows:
Host A: Pentium3, 550Mhz, 256MB Ram
Host B: Pentium3, 1.0Ghz, 128MB Ram
FW 1: Pentium4, 2.40Ghz, 256MB Ram
FW 2: Pentium4, 2.40Ghz, 256MB Ram The tests were broken into 2 parts – the first part involved the CSG architecture and the second part involved the use of only host-based firewalls implemented on the end-user nodes themselves. The IPTables firewall on the 2-node CSG had to contain at least one rule, which was required for packet forwarding. The firewall rulesets used in both tests were restrictive, that is, allowed only authorised packets to enter or leave the firewall nodes [8]. They consisted of ~1 rule and ~2000 rules. The CSG architecture was tested for 1-node and 2-node CSGs to better appreciate the effect of the load balancing in preventing network chokepoints. Iperf [18] was used for testing the throughput. It generates TCP connections, which involve requests and replies and the test parameters were: Window size: 214KB Maximum Segment Size (MSS): the default for Iperf: 1460 Apart from throughput, CPU utilisation was also monitored and the effect of using the CSG in the place of the host-based firewalls became clear by comparing results in Tables 2 and 3. Performance tests for intra-cluster communication gave the following results:

TABLE II TEST RESULTS FOR THE CSG ARCHITECTURE

No. of firewall nodes 1 2

No. of rules 1 2000 1 2000

Bandwidth (Mbps) 89.7 89.4 93.4 93.1

Max CPU Usage (%) Host A Host B FW 1 FW 2 18.6 7.7 22.7 22.9 8.1 95.5 26.5 7.8 16.8 19.8 21.5 7.7 70.5 76.4

TABLE III TEST RESULTS FOR HOST-BASED FIREWALLS

Max CPU Usage (%) Host A Host B

No. of rules

Bandwidth (Mbps)

1

92.9

16.4

7.6

2000

58

90.9

10.1

Two salient points to be noted are:
The processing load on the end-user nodes decreases greatly.
The load-balanced firewall cluster allows better throughput to be achieved compared to host-based firewalls. VIII. CONCLUSION AND FUTURE WORKS In this paper, a new load-balanced firewall architecture is presented, which entailed a cluster of two hosts acting as the security gateway dedicated to Layer 3 traffic filtering. Load balancing allowed for high availability while at the same time removing the strain of packet filtering from the individual end-user nodes. Future works include: 1) Optimising the efficiency of the packet-based load balancing in conjunction with the stateless firewall by:



Using packet-out-of-order reduction algorithms Taking into account the size of packets sent to each member of the firewall cluster

2) Building a new distributed firewall architecture based on the CSG architecture.

REFERENCES [1] Kai Hwang, Muralidaran Gangadharan: “Micro-Firewalls for Dynamic Network Security with Distributed Intrusion Detection”, IEEE International Symposium on Network Computing and Applications (NCA'01), 2001 [2] Wei Li: “Distributed Firewall”, http://citeseer.ist.psu.edu/li00distributed.html, 2000 [3] Keromytis A. D., Wright J. L.: “Transparent Network Security Policy Enforcement”, Proceedings of the Annual USENIX Technical Conference, pages 215-226, June 2000 [4] Wendell Odom: “CCNA Self-Study CCNA INTRO Exam Certification Guide”, Cisco Press, 2004 [5] Cisco Systems: “Configuring Network Security with ACLs”, http://www.cisco.com/en/US/products/hw/switches/ps628/products_configur ation_guide_chapter09186a00800d84c8.html, 2005 [6] Bill McCarty: “Red Hat Linux Firewalls”, Wiley Publishing, Inc., 1st ed., 2003. [7] Jay Beale: “Linux Gets Stateful Firewalling…Introducing Netfilter (iptables!)”, http://www.bastille-linux.org/jay/introducing-netfilter.html, 2005 [8] Oskar Andreasson: “IPTables Tutorial 1.1.19”, http://www.faqs.org/docs/iptables/, 2003 [9] James C. Stephens: “ Connection tracking”, http://www.sns.ias.edu/~jns/security/iptables/iptables_conntrack.html, 2002 [10] Ebtables Website: “What is Ebtables?”, http://ebtables.sourceforge.net/, 2005 [11] Ebtables Website: “Who’s behind Ebtables?”, http://ebtables.sourceforge.net/about.html#behind, 2005 [12] Bridge website: “Bridge”, http://linux-net.osdl.org/index.php/Bridge, 2005

[13] Bart de Schuymer: “Ebtables hacking HOWTO”, http://ebtables.sourceforge.net/ebtables-hacking/ebtables-hackingHOWTO.html, 2003 [14] W. Richard Stevens, Bill Fenner, Andrew M. Rudoff: “Unix Network Programming: The Sockets Networking API Volume 1”, Addison-Wesley, 3rd Ed., 2004 [15] Cisco Systems: “Cisco: How does LB work?”, http://www.cisco.com/warp/public/105/46.html, 2005 [16] National Computer Security, “DoD 5200.28-STD, Trusted Computer System Evaluation Criteria”, http://csrc.nist.gov/publications/history/dod85.pdf, 1985 [17] Chris Brenton: “What is Egress Filtering and How Can I Implement It”, http://www.sans.org/rr/whitepapers/firewalls/1059.php, 2000 [18] Iperf version 1.7.0: “Iperf User Docs”, http://www.itl.ohiou.edu/iperfdoc/, 2003