Signature-based Botnet Detection and Prevention - Semantic Scholar

Signature-based Botnet Detection and Prevention Sunny Behal1, Amanpreet Singh Brar2, Krishan Kumar3 1 Deptt of CSE, SBECET, Ferozepur, India 2 Deptt of CSE & IT, GNDEC, Ludhiana, India 3 Deptt of CSE, SBSCET, Ferozepur, India Abstract The Internet is used extensively for important services such as banking, business, medicine, education, research, stock trades, weather forecasting etc. Most of these services must be processed in a timely manner. However these services are delayed, degraded and sometimes completely disrupted because of unavailability of internet. The inherent vulnerabilities of the internet architecture provide opportunities for a lot of attacks on its infrastructure and services. Behind these attacks is a large pool of compromised hosts sitting in homes, school, business and governments around the world. These infected systems are called bots that communicates with a bot controller and other bots to form what is commonly referred to as a Zombie army or Botnet. For any organization, internal bot infections cause serious repercussions including loss of man hours and downtime. The average cost of such disasters runs into tens of thousands of dollars. So there is need to defend against such attacks. In this paper, we have analysed the feasibility of outbound traffic i.e. extrusions, to detect and prevent attacks caused because of botnets. As a part of the research work, a Network-based Detection and Prevention systems of botnets called N-EDPS has been proposed.

primary source of most of the threats used for scanning, (distributed) denial-of-service (DOS) activities and direct attacks, taking place across the Internet. At the center of these threats is a large pool of compromised hosts sitting in homes, schools, businesses, and governments around the world. Bot malware typically takes advantage of system vulnerabilities and software bugs or hacker-installed backdoors that allow malicious code to be installed on computers without the owners’ consent or knowledge. However, all bots distinguish themselves from the other malware forms by their ability to establish a command and control (C&C) channel through which they can be updated and directed. Once collectively under the control of a C&C server, bots form what is referred to as a botnet. The elements involved and the sequence of commands exchanged between different botnet elements is shown in figure 1.

Keywords: - Attacker, Bot, Botmaster, Extrusion, Intrusion, Pear to Pear, Zombie.

1. Introduction The Internet consists of hundreds of millions of computers distributed all around the world. Most of the companies, institutes, banks, businesses, and research heavily depend on a well-working and secure computer networks. Any incident could be critical to their routine work. The increasing usage of interactive internet applications in these areas has induced a rise in risks and possibilities of misuse of computer networks. The core objectives of information security have to be met in order to protect the network such as confidentiality, integrity, availability, authentication, non-repudiation. In order to meet all of these requirements, it is essential to protect a network against all possible threats. Over the last decade, malicious software or malware has risen to become a

Figure 1: Working of a typical IRC-based Botnet Firstly, a botmaster exploits the vulnerability on the victim. Then the victim downloads the actual bot binary and contacts the IRC server address in the executable, including resolving the DNS name. After that the bot joins an IRC communication channel to receive commands from the botmaster via communication channel. The proposed work uses the botnet life cycle given in [1] and is depicted by the State Transition Diagram shown in figure 2.

Figure 2: Life cycle of a Botnet The figure 2 is not intended to provide a strict ordering of events, but rather to capture a typical infection dialog. In the idealized sequence of a direct-exploit bot infection model, the bot infection begins with an external-tointernal communication flow that may encompass bot scanning (E1) or a direct inbound exploit (E2). When an internal host has been successfully compromised, the newly compromised host down- loads and instantiates a full malicious binary instance of the bot (E3). Once the full binary instance of the bot is retrieved and executed, this model accommodates two potential dialog paths, referred to as the bot Type I versus Type II split. Under Type II bots, the infected host proceeds to C&C server coordination (E4) before attempting self-propagation. Under a Type I bot, the infected host immediately moves to outbound scanning and attack propagation (E5), representing a classic worm infection. Botnets can serve both legitimate and illegitimate purposes as described in [6]. One legitimate purpose is to support the operations of IRC channels using administrative privileges on specific individuals. Nevertheless, such goals do not meet the vast number of bots that we have seen. The possibilities to use botnets for criminally motivated or for destructive goals has been categorized as DDoS Attacks [3, 15], Spamming and Spreading Malware [16, 17], Information Leakage [14, 15], Click Fraud [15], Identity Fraud [15], Hosting of Illegal Software [16], Political Activism [18]. Before discussing Botnet defense approaches, it is necessary to highlight here that traditional security technologies such as router access lists, firewalls, and Intrusion detection systems which are important components of overall security strategy, do not provide comprehensive botnet protection by themselves because of the reasons given in [19]. Since these security components are unable to restrict botnet attacks, some specific defense approaches are being deployed to combat botnet attacks. The stumbling barrier against these attacks is that it is almost impossible to differentiate between legitimate and attack packets. Therefore it has become a real challenge to defend against these attacks. The seriousness of botnet problem and growing sophistication of attackers have led to development of numerous defense mechanisms. These defense mechanisms are classified into three broad categories in Prevention [5, 6], Detection and Tracing [4,

5, 9, 11, 12] and Mitigation mechanisms [5, 8, 10]. Network-based intrusion detection systems (IDSs) and intrusion prevention systems (IPSs) may come to mind as the most appealing technology for detecting and mitigating botnet threats. Traditional IDSs, whether signature based [20, 21] or anomaly based [1, 8], typically focus on inbound packets flows for signs of malicious point-to-point intrusion attempts. Network IDSs have the capacity to detect initial incoming intrusion attempts, and the prolific frequency with which they produce such alarms in operational networks, However, distinguishing a successful local host infection from the daily myriad of scans and intrusion attempts is as critical and challenging a task as any facet of network defense [2]. Our primary contribution in this paper is (1) to introduce a new network monitoring strategy, which focuses on detecting and preventing malware infections (specifically bots/botnets) through monitoring outbound traffic i.e. extrusions only using the available signatures. (2) to reduce the rule set of a detection and prevention system so as to increase its efficiency under attack. (3) to utilizes the existing open source and freely available software to develop a network based detection and prevention system of botnet based attacks. The remainder of this paper is outlined as follows. The section-II focuses on the problem formulation and experimental setup of N-EDPS. The section-III discusses the various results obtained and the last section concludes the work by highlighting the scope for future work.

2. Related Work The proposed work utilizes the Botnet detection system called a BotHunter proposed in paper [1]. BotHunter is a passive network monitoring system driven by Snort. It correlates the inbound intrusion alarms with the outbound communication patterns that are highly indicative of successful local host infection. The experimental results using BotHunter are presented in a virtual and live testing environment. BotHunter focuses on botnet detection and its traffic. We focus on all outbound traffic generated by a malicious source. Paper [22] focuses on the outbound traffic with the intention to guarantee that the host will not be used as an attack launcher or intrusion relayer to compromise other systems. Therefore, the intended goal using the outbound traffic is different for the mentioned paper and the present work. Paper [22] focuses on the prevention of further propagation of malware. In this work, on the other hand, the outbound traffic is analysed to get a clear indication about a successful attack. The proposed work also utilizes the malware classifications given by C. Lussi in paper [2]. The author in [2] uses the concept of escalation rules with different weights. The applied rule with the highest weight determines the treatment of the corresponding alert. Apart from this work, other network-based automated botnet detection tools are in existent like Rishi [23], Strayer [24], and

BotMiner [13]. While there are few other detection tools available which are host based like Binder [25] and BotSwat [26]. The gaps in the existing work have been identified and efforts are made to address some of these gaps as part of current work

3. Methodology The conceptual methodology used for the development of N-EDPS has been shown in figure 3. The end product will first monitor the network traffic of the educational institute to capture the details of types of attacks occurring in the network and the output will be stored in some database or log file for future reference. Based on the available signatures, alerts will be generated corresponding to the various attacks occurring in the network.

The concept of signature-based detection and prevention system is used which searches for known malicious patterns in the payload whereas a behavior-based IDS, also known as an anomaly detection system, analyses in the first instance the traffic data. Most of the current used IDS focus on the intrusion from outside of the network into the monitored network. Such a detection of attacks creates a lot of false alarms. A new approach called extrusion detection is focusing on the traffic, whose source address is inside of the monitored network. The extrusion detection technique is a promising approach because the behaviour of an infected system and the generated traffic due to this infection is often conspicuous. An extrusion is a clear indication of occurred intrusion, because an extrusion only happens as a result of a successful attack. The differentiation between an attempted and a successful attack is the ultimate goal of this work. Therefore, we will analyse and test the efficiency of the extrusion detection approach. For the development of N-EDPS, a no. of Open Source Softwares and free Softwares has been used. This permits users to use, change, and improve the software, and to redistribute it in modified or unmodified forms Table 1: Classification of signatures for N-EDPS

File Name

Description

No. of Rules

E1.rules

contains all External to Internal

75

Inbound Scan related rules

Figure 3: Methodology to develop N-EDPS New rules/signatures will be developed and updated to the database of detection system as well as to the database of prevention system. The database of prevention system will contains only those rules for which we want to drop the incoming packets whereas the database of detection system will contains more no. of rules for which we want to generate alerts only. After the development of such rules, we would be able to filter malicious traffic from legitimate traffic resulting in Botnet free traffic. For the development of proposed N-EDPS, a prevention system, also known as active IDS has been used, which investigates the traffic inline. This means that the packets are analysed continuously and the reaction to an attack is in real-time. The IPS blocks traffic independently without human interaction. It aims not only at detecting, but also at preventing an attack. In contrast, a passive IDS does not act by itself but does only raise an alarm in case of a supposed attack. The Network-based deployment has been used instead of Host-based deployment. A Networkbased IPS monitors the network traffic of a particular network whereas a Host-based IPS monitors the operating system, applications, and the host specific network traffic.

E2. rules

contains all External to Internal

325

Inbound Exploit related rules E3.rules

contains all Internal to External

250

Binary Acquisition related rules E4.rules

contains all Internal to External C&C

370

Communication related rules E5.rules

contains all Internal to External

455

Outbound Infection Scanning rules

4. Experimental Setup The proposed N-EDPS consists of two components. One is the detection engine and the other one is the prevention engine. For the development of proposed system, we have used Bot Hunter as the detection engine and snort-inline as the prevention engine. The system topology for the NEDPS in the live environment in shown in figure 4. We placed the proposed N-EDPS between the network and the Internet Server to monitor all the outbound traffic. It is worth to be noted that the IDS engine works for two conditions;

•

•

Condition 1: Evidence of local host infection (E2), AND evidence of outward bot coordination or attack propagation (E3-E5); or Condition 2: At least two distinct signs of outward bot coordination or attack propagation (E3-E5).

As for the development of the proposed N-EDPS, we have focussed on only outbound traffic, so we will use condition 2 for the detection of botnets in a network. And will use only category E3, E4 and E5 rules\signatures for the above said purpose. The N-EDPS had been deployed in the SBSCET network for a period of three weeks and to run the N-EDPS, the rules\ signatures have been classified according the life cycle of the botnet shown in figure 2 and have been stored accordingly into five files as shown in table 1.

65 C & C servers, 32 Egg download servers and 24 IP addresses of outbound scanning servers which could be used for further infection. Apart from these results, the signatures that have been triggered in identifying the popular viruses/worms/spyware who were using botnets are shown in table 2. The name and type of viruses/worms/spyware found to use botnets inside the network, have been shown in table 3. Table 2: Top Triggered Signatures ET Known Russian Business Network Monitored Domain ET ShadowServer confirmed botnet control server ET TROJAN Down adup/Conficker A or B Worm reporting Detected intense malware port scanning of 30 IPs BotHunter MTC confirmed botnet control server ET SCAN Behavioral Unusual Port NETBIOS traffic, Potential Scan or Infection ET TROJAN Downadup/Conficker A Worm reporting ET TROJAN BOT - potential response BotHunter REPO confirmed botnet control server Detected moderate malware port scanning of 10 IPs ET TROJAN Sality Trojan User-Agent (KUKU v3.09 exp) ET VIRUS Sality Virus User Agent Detected (KUKU) ET MALWARE 180solutions Spyware Reporting ET MALWARE Hotbar Agent Partner Checkin ET TROJAN Likely Bot Nick in IRC (USA +..) ET MALWARE Hotbar Agent Reporting Information ET TROJAN Pakes/Cutwall/Kobcka Update URL Detected BACKDOOR Pushdo client communication attempt

Figure4:Experimental Topology of N-EDPS

BotHunter HTTP-based .exe Upload on backdoor port ET POLICY PE EXE or DLL Windows file download ET TROJAN Peed Report to Controller ET POLICY Outbound Multiple Non-SMTP Server Emails COMMUNITY BOT GTBot info command

A number of unusual ports have also been found (123, 137, 443, 445, 1036, 1170, 1199, 1863, 1900, 2711, 3670, 6667, 6697, 7000, 7920, 8000, 8067, 8448, 9006, 18384, 47221, 49158, 55273, 58670). It had been confirmed that the monitored network did not provide any service corresponding to these ports.

Figure5: As we concentrate only on outbound traffic, we use only E3, E4 and E5 type rules for botnet detection.

5. Results and Discussion The N-EDPS has been run for a period of three weeks in the live environment of SBSCET network. As shown in figure 5, we have been able to find 42 infected computers,

Table 3: Botnets found Conflicker A \ Downup \ Downadup \ Kido Conflicker B \ Downup \ Downadup \ Kido Sality trojan user agent (KUKU v3.09 exp) Sality Virus user agent (KUKU) 180 solutions spyware Hotbar Pakes Cutwall Kobcka Pushdo\ Pandex \ Cutwail Peed\ Storm variant of GT Bot

6. Conclusion and Scope for Future Work To better understand the botnet and to stop its attacks eventually, this research work focuses on the detection and prevention of successful botnet attacks based on the concept of analyzing outbound traffic i.e. Extrusions only. As a part of the work we have proposed a signature-based N-EDPS which examines only outbound traffic to detect Botnet related malicious traffic using various open source and freely available software. We run the N-EDPS for a period of three weeks in the live environment of SBSCET network. As a part of the run, we have been able to find a number of infected computers, C & C servers, Egg download servers and outbound scanning servers which could be used for further infection. The name and type of viruses/worms/spyware found to use botnets inside the network have been found. The proposed N-EDPS is better than an N-IDPS because N-EDPS requires a smaller database of rules \ signatures as compared to N-IDPS and delivers better results. But there are certain drawbacks of the proposed system. (1) As the case with any Antivirus software, the proposed signature-based N-EDPS also requires access to a current database of attack signatures. (2) The proposed N-EDPS is not capable of detecting encrypted C & C channels, if they exist. For this, anomaly based detection logic must be incorporated into N-EDPS. This work opens up a number of avenues for future work. (1) A perfect N-EDPS is one which can respond to attacks when they occur, i.e., the one which is able to provide real-time response to any kind of attack whether it is known or novice. For this we need to develop an N-EDPS which integrates the feature of both signature-based and behaviour-based detection system and dynamically develop new rules for novel attacks and drop the traffic in real time. (2) For detecting encrypted C & C channels, the anomaly based logic can be incorporated.

References [1] Gu, G., Porras, Ph., Yegneswaran, V., Fong, M., Lee, W. “BotHunter: Detecting malware infection through IDSdriven dialog correlation” , In 16th USENIX Security Symposium (Security’ 07), 2007. [2] Cecile Lussi. Master’s thesis on “Signature-based Extrusion detection” ETHZ (TIK), 2008.

[3] F. Freiling , T. Holz and G. Wicherski , “Botnet tracking: exploring a root-cause methodology to prevent distributed denial-of-service Attacks,” in Proceedings of the 10th European Symposium on Research in Computer Security (ESORICS ’05), vol. 3679 of Lecture Notes in Computer Science, pp. 319–335,Springer, Milan, Italy, 2005. [4] G. Carl, G. Kesidis, “Denial of Service Attack-Detection Techniques,” IEEE Internet Computing, Vol 10, No. 1, pp 82-89, 2006. [5] E. Cooke, F. Jahanian , and D. McPherson , “The zombie roundup: Understanding, detecting, and disrupting Botnets” in Proceedings of Workshop on Steps to Reducing Unwanted Traffic on the Internet (SRUTI’05), 2005. [6] McCarty B. (2003), “Botnets: big and bigger,” IEEE Security and Privacy, vol. 1, no. 4, pp. 87–90. [7] R. Villamarin-Salomon and J. , “Identifying botnets using anomaly detection techniques applied to DNS Traffic” in Proceedings of the 5th IEEE Consumer Communications and Networking Conference, pp. 476–481, Las Vegas, Nev, USA, 2008. [8] J. Liu, Y. Xiao, J. Zhang , ”Botnet: Classification, attacks, Detection, Tracing and Preventive measures” EURASIP journal of Wireless Communications and Networking, Vol. 2009, article ID 692654, 2009. [9] Y. Kugisaki , Y. Kasahara , Y. Hori , and K. Sakurai , “Bot detection based on Traffic Analysis,” in Proceedings of the International Conference on Intelligent Pervasive Computing (IPC ’07), pp. 303–306, Jeju Island, South Korea, 2007. [10] T. Holz , M. Steiner, F. Dahl, E. Biersack, Freiling , ”Measurements and Mitigation of Peer-to-Peer-based Botnets: A Case Study on StormWorm”, 2007 [11] X. Hu , M. Knyz and K. Shin , “RB-Seeker: auto-detection of redirection Botnets” in Proceedings of 16th Annual Network & Distributed System Security Symposium (NDSS ’09), 2009. [12] J. Grizzard , V. Sharma, C. Nunnery, B. Kang , and D. Dagon , “Peer-to-peer botnets: Overview and case study” in USENIX Workshop on Hot Topics in Understanding Botnets (HotBots’07), 2007. [13] G. Gu , R. Perdisci , J. Zhang and W. Lee , “Bot-Miner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection” in USENIX Security Symposium, 2008. [14] J. Govil, “Examining the criminology of bot zoo,” in Proceedings of the 6th International Conference on Information, Communications and Signal Processing (ICICS ’07), pp. 1–6, Singapore, December 2007. [15] P. Bacher, T. Holz, M. Kotter, and G. Wicherski, “Know your Enemy: Tracking Botnets,” http://www.honeynet.org/papers/ bots., Accessed: September 2009 [16] K. Pappas, “Back to basics to fight botnets,” Communications News, vol. 45, no. 5, p. 12, 2008. [17] P. Sroufe, S. Phithakkitnukoon, R. Dantu, and J. Cangussu, “Email shape analysis for spam botnet detection,” in Proceedings of the 6th IEEE Consumer Communications and Networking Conference (CCNC ’09), pp. 1–2, Las Vegas, Nev, USA, January 2009. [18] Info World Newsletter. http://www.infoworld.com/d/security-central/botnets-newpolitical-activism-392 Accessed November, 2009 [19] P. Ferguson, D. Senie, “Network ingress filtering: Defeating Denial of Service attacks which employ IP

[20]

[21]

[22] [23]

[24]

[25]

[26]

source address spoofing,” RFC 2267, the Internet Engineering Task Force (IETF), 1998 V. Paxson. BRO: A System for Detecting Network Intruders in Real Time. In Proceedings of the 7th USENIX Security Symposium, 1998. M. Roesch. Snort - lightweight intrusion detection for networks. In Proceedings of USENIX LISA’99, 1999, Accessed November, 2009 Mandujano, S., Galván, A. Ountbound Intrusion Detection, Center for Intelligent System Monterrey, Mexico, 2004 J. Goebel and T. Holz. Rishi: Identify bot contaminated hosts by irc nickname evaluation. In USENIX Workshop on Hot Topics in Understanding Botnets (HotBots’07), 2007. W. T. Strayer, R.Walsh, C. Livadas, and D. Lapsley, “Detecting botnets with tight command and control,” in Proceedings of the 31st Annual IEEE Conference on Local Computer Networks (LCN ’06), pp. 195–202, Tampa, Fla, USA, November 2006. W. Cui, R. H. Katz, W. Tan. Design and Implementation of an Extrusion-based Break-In Detector for Personal Computers in Annual Computer Security Applications Conf., Dec. 2005. E. Stinson, J. C. Mitchell. Characterizing Bots’ Remote Control Behaviour in Detection of Intrusions & Malware, and Vulnerability Assessment, July 2007.