Static and Dynamic Packet Filtering on Lightly Managed Systems James A. Lupo, Daniel Likarish Regis University, Denver, CO, USA
[email protected] [email protected] Abstract: Regis University operates servers that must be open to external access for a variety of educational purposes. These servers were managed by student and faculty, hence the experience level was generally low, and the time available for expert oversight was limited. In May of 2006 unauthorized attempts to gain access were noted and steps were taken to reduce risk of system compromise. The attempts increased in sophistication and severity with time. The pattern of access attempts seemed to follow a training curriculum from simple to more complex based on our responses to the attempts. The need to understand the threat profile resulted in the design of a HoneyNet to capture attack characteristics. When large numbers of login attempts were noted involving extremely long lists of pseudo user names and well known system account names, the following defensive measures were implemented. Password strength requirements were increased, IPTables was set up to block entire Class A network ranges, pinholes were defined to allow trusted host access, throttling was defined to slow down probes, and monitoring via swatch was implemented to block attempts from unexpected sources. The dynamic blocking process uses TCPWrappers and the hosts.deny file to immediately block the source address of the suspected activity. The addresses were then manually reviewed to determine both the location and associated network ranges. Depending on the circumstances, the host address was left in the hosts.deny file, or it was removed and the entire associated network range was permanently blocked by an IPTables filtering rule. Since the system was implemented, intrusion attempts have dropped from several thousand per day to approximately 1 per week. And after a year of operation, no legitimate access has been blocked. Keywords: intrusion detection protection filtering risk management 1. Introduction Regis University provides a wide range of on-line and class-room based courses which require access to a variety of computer based services. These services are provided under a graduate level program called the System Engineering and Application Development Practicum. This practicum is managed by graduate students and mentored by the computer and information technology faculty. The experience and knowledge base is very broad, but the amount of time that can be dedicated by the most experienced people tends to be rather limited. This presented issues with regards to monitoring the health of the systems and maintaining their integrity. The concerns peaked some 18 months ago when concerted attempts to illegally log onto the Linux server were detected. What follows describes the evolution of a protection system that minimizes exploit risks while maximizing system flexibility for student projects and service provisioning. 2. Threat Identification A manual review of the system logs began to show multiple occurrences of the following pattern: Dec 25 11:13:25 acadunix sshd[18280]: Connection from 147.83.7.82 port 53827 Dec 25 11:13:25 acadunix sshd[18280]: Did not receive identification string from 147.83.7.82 Dec 25 12:00:09 acadunix sshd[22010]: Connection from 147.83.7.82 port 51213 Dec 25 12:00:12 acadunix sshd[22010]: Invalid user admin from 147.83.7.82 Dec 25 12:00:17 acadunix sshd[22010]: (pam_unix) check pass; user unknown Dec 25 12:00:17 acadunix sshd[22010]: (pam_unix) authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=147.83.7.82 Dec 25 12:00:19 acadunix sshd[22010]: Failed password for invalid user admin from 147.83.7.82 port 51213 ssh2 This was often followed by up to several thousand additional illegal user names originating from the same IP address. To make the attempts more obvious, the Munin (Munin 2007) system monitoring package was installed. Failed attempts are displayed in the Munin log files in their own section with the following format:
Failed logins from these: 123/password from 218.232.104.221: 1 Time(s) 1234/password from 218.232.104.221: 1 Time(s) 12345/password from 218.232.104.221: 1 Time(s) 123456/password from 218.232.104.221: 1 Time(s) 12345678/password from 218.232.104.221: 1 Time(s) 123abc/password from 218.232.104.221: 1 Time(s) 123go/password from 218.232.104.221: 1 Time(s) 1q2w3e/password from 218.232.104.221: 1 Time(s) Aaliyah/password from 83.19.113.122: 1 Time(s) Aaron/password from 83.19.113.122: 1 Time(s) ... truncated for brevity ... 35 IP addresses were the source of 100 or more failed login attempts. These offenders with 200 or more attempts are listed in Table 1. IP Address
Login Attempts
Location
216.75.30.127
11796
CA, US
210.221.154.12
5460
QLD, AU
210.240.119.2
4176
QLD, AU
218.232.104.221
3370
QLD, AU
64.79.200.190
2894
WA, USA
202.60.82.26
1930
QLD, AU
59.124.43.106
1678
QLD, AU
222.234.2.154
1203
QLD AU
202.202.112.14
1056
QLD, AU
62.90.175.132
952
Amsterdam, NL
12.153.91.118
813
ATT WorldNet
83.19.113.122
594
Amsterdam, NL
200.228.66.158
394
Montevideo, UY
211.233.36.73
368
QLD, AU
67.176.55.247
364
Comcast
67.176.55.247
326
Comcast
125.240.213.67
306
QLD,AU
200.26.136.202
255
Montevideo, UY
Table 1. IP addresses with at least 200 undesired login attempts. It was clear that automated attempts to login were following a dictionary attack protocol. On further review, additional attempts were identified trying to gain access to well known services. The location data presented in column three was obtained by querying the WHOIS database using the www.whois.net online tool that provided access to ARIN (USA), RIPE (EU Russia and Middle East), and APNIC (Asia Pacific) IP location databases. Although the largest single attacks were launched from an ISP located in Bellevue WA USA, the largest total number of attacks originated from Milton Queensland AU. No attempt was made to trace the attacks to their originating source IP beyond the city location data that was provided by the WHOIS database. 3. Defense Mechanism Since a dictionary attack seem to be the primary method in use, the first defense implemented was the simplest. The password system uses PAM (pluggable authentication modules), so the password
strength requirements were increased to require no less than 8 letters with at least 1 for each of the four character classes (uppercase and lower case alpha, digits, and special). Since the password length is unlimited, users are encouraged to use pass phrases rather than complex character strings. Analysis of the IP addresses showed that the vast majority originated from outside the continental United States. Since the bulk of all students using the Regis systems do not originate connections from non-continental United States sites, a decision was made to block all networks controlled by non-US interests. The first approach used only the TCPWrappers (Venema 1992) facility provided by most Linux distributions. Though simple to use, it only protects services designed to support TCPWrappers. The preferred approach uses packet filtering. By using IPTables (Netfilter 200), all access from entire Class A networks can be blocked, while allowing trusted sites explicit access through pin-hole filtering. Since machines within trusted networks may be usurped, the static rules provided by IPTables can not provide absolute protection. Thus a dynamic monitoring system was also set up, using the SWatch (Atkins 2007) script package to prevent exploits from within the trusted address ranges. 3.1 TCPWrappers TCPWrappers provides a method of blocking network connections involving the TCP and UDP protocols. On UNIX-like systems, it typically involves setting up two files in the /etc directory: hosts.allow and hosts.deny. Entries in hosts.allow explicitly allow connections from specified IP addresses and address ranges. Similarly, entries in hosts.deny explicitly deny connects. The files allow specification of actions to take in addition to allowing or denying access, such as sending email alerts or making detailed log file entries. As the number of entries in hosts.deny increased, the Failed and Illegal connection attempts dropped to essentially 0, as seen in Figure 1. Incident Counts 100000 Failed Illegal Dropped 10000
Incidents
1000
100
10
1 0
50
100
150
200
250
300
350
400
450
Day Number (Start 11 Oct 2006; End 6 Dec 2007)
Figure 1. Illegal and failed login attempts, and dropped packets per day from undesired sources. The installation of the TCPWapper software was influenced by the initial and undocumented response of the system to what was hypothesized to be periodic attempts of increasing severity and complexity to attack the system. The periodic nature of the attacks indicated that the system may have been participating in a attack training curriculum. Review of Figure 1 indicates that there was no evidence of a periodic attempt to gain access to the system. Rather the data indicates that the TCPWapper method blocked attacking IP addresses upon the first attempt. This precluded use of the box as a
participant in a stealth training exercise. The existence of a training curriculum using lightly managed and monitored systems as a target has yet to be established and will require future study. 3.2 IPTables IPTables is a packet filtering facility often built into the Linux kernel and available for other operating systems. It is rule driven, allowing arbitrarily complex specifications for accepting or rejecting connections. In this application, a processing chain called LOG_DROP is first created which first generates a log message with details about a rejected packet, and then drops the packet without further action. The statements to accomplish this are: iptables -F LOG_DROP iptables -A LOG_DROP -j LOG --log-prefix "IPTABLES LOG_DROP : " iptables -A LOG_DROP -j DROP This causes the server to be essentially invisible to the system which sent the packet. Since the attacks are automated, a second set of rules are created to limit, or throttle, the number of attempts per second. The acceptance time is set small enough so as not to interfere with interactive users, but large enough to significantly impede automated scripts: iptables -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW -m recent \ --name SSH_THROTTLE --set iptables -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW -m recent \ --name SSH_THROTTLE --update --seconds 60 --hitcount 20 --rttl -j DROP Pin-hole rules are then established to allow trusted systems access: iptables -A INPUT -s 194.109.137.216/29 -j ACCEPT The last and largest set of rules are generated to explicitly block all untrusted networks. An example rule blocking a single network range looks like: iptables -A INPUT -s 202.0.0.0/7 -j LOG_DROP The full set of rules was deployed on 29 June 2007. A side affect is it allow the monitoring of incoming packets from undesired sources, as shown in Figure 1. 3.3 SWatch Swatch is a lightweight daemon which can be used to monitor system log files and take action based on the text patterns it sees. For this application, the auth.log file was selected and four patterns are defined for taking action: watchfor /Illegal user / exec "/usr/local/sbin/deny_host.sh $8 $9 $10" watchfor /Failed password for root from / exec "/usr/local/sbin/deny_host.sh $9 $10 $11" watchfor /Failed password for illegal user / exec "/usr/local/sbin/deny_host.sh $11 $12 $13" watchfor /Failed password for invalid user / exec "/usr/local/sbin/deny_host.sh $11 $12 $13" If any of the watchfor patterns are matched, a script is called passing fields from the offending record containing the user name and source address. The script processes this data to generate an alert email to the system administrator, and adds data to a file to tract the number of attempts from the given IP address. Once the number of attempts exceeds a set value, currently 5, the IP address is added to the hosts.deny file to block all further access to protected services. Periodically, the entries in hosts.deny are examined. If they are due to obvious user error, they are removed, and if they are found to be non-US originated, their associated network is identified and the network range is added to the IPTables rule set.
4. Results Since implementing, the list of blocked network ranges has grown to 84. Packets from 2391 distinct IP addresses have been dropped covering a total of 585 unique destination ports. The ports targeted by 100 or more packets are shown in Table 2. Port (Service)
Number of Packets
123 (ntp)
7711
1026
2010
443 (https)
1488
43165
979
1027
868
32785
741
5168
469
1443 (ms-sql-s)
340
22 (ssh)
283
3
165
2967
150
25 (smtp)
118
Table 2. Services with at least 100 packets dropped since 29 Jun 2007. An unanticipated side effect of reducing the number of undesired login attempts has been a noticeable improvement in system response for interactive users. This is considered anecdotal since no hard data supports it. Of more import, no legitimate user has been blocked incorrectly from the system. 5. Design and Architecture of a HoneyNet The system evolved over time as each subsequent attack resulted in installation of different software package to eliminate either attackers or the object of their attack. The system in its end stage became an unintended honeypot that represented an opportunity to characterize attacks. Analysis of the system logs were used to design a VMware-based (VMware 2007) HoneyNet shown in Figure 2. The simple HoneyNet design was derived from Spritzner (Spritzner 2003) as an external HoneyNet located on an exposed VLAN (virtual local area network) with public IP address. The firewall between the Internet and the HoneyNet passes all traffic to the isolated system. An additional system was introduced into the VLAN as a monitoring box that can be used to observe and record traffic.
Figure 2. HoneyNet constructed with honeypot and monitoring systems.
The present study provides the monitoring system with the most likely 100 attackable services. Figure 1 indicated the top IP subnets that were most likely to attack the exposed system. The implementation, monitoring and analysis of the data will be reported in follow-on studies. 6. Conclusions Several well known techniques were deployed to protect an academic system requiring open access to all legitimate users. By combining a static set of IPTables filtering rules, and a dynamic monitoring system, these goals were achieved without impacting legal users. Review of our illegal and failed login attempts did not demonstrate evidence of periodic attempts to access the systems. There was no evidence of staged attacks growing in sophistication and complexity associated with a training curriculum. Copies of all scripts and data files used to implement the methods described here are available by request from the authors. References Atkins, Todd. (2007) “Swatch – Simple Watcher”, [online] swatch.sourceforge.net. Honeynet.org (2007) “The Honeynet Project”, [online] www.honeynet.org. Munin (2007) [online] munin.projects.linpro.no. Netfilter. (2007) [online] www.netfilter.org/projects/iptables/index.html. SWatch (2007) [online] swatch.sourceforge.net. Spitzner, Lance (2003) “Honeypots, Tracking Hackers”, Boston, Addison-Wesley, Venema, Wietse (1992) “TCP WRAPPER: Network monitoring, access control, and booby traps”, Eindhoven University of Technology, The Netherlands, [online] www.vtcif.telstra.com.au/pub/docs/security/tcp_wrapper.txt. VMware (2007) [online] www.vmware.com. WHOIS (2007) [online] www.whois.net
