Monitoring System's Network Activity for Rootkit ...

Monitoring System's Network A tivity for Rootkit Malware Dete tion

Mirosªaw Skrzewski Silesian University of Te hnology, Institute of Computer S ien e, Akademi ka 16, 44-100 Gliwi e, Poland

miroslaw.skrzewskipolsl.pl

Abstra t. Contemporary malware authors attempt many ways to make

its produ ts invisible for antymalware programs, and after infe tion deeply on eal its operation from users sight. The presen e of on ealed malware an be dete ted many ways. Most of them operate on demand and provides high s anning overload of the system, blo king the han es for normal users operation. The paper presents new method of rootkit operation dete tion, suitable for ontinuous operation, based on the analysis of network a tivity pi tures viewed from two sour es (internal and external to system), along with the results of method tests on virtual ma hines infe ted with the sele ted rootkits ode samples. Keywords: network ows monitoring, network auditing, rootkit tra

dete tion.

1

Introdu tion

Dete ting the presen e of ontemporary malware on user systems be omes over time more and more di ult task. Malware authors attempt many ways to make its program (through meta- and polymorphism modied ode, many levels of obfus ation, generaton of multiple short series) invisible for antymalware programs, and after infe tion deeply on eal its operation from users sight. Unix environment has known for many years rootkits  the methods of hiding mali ious programs on the system via modi ations of system tools, a tively ltering system state informations from users, masking the presen e of malware les, servi es and ommuni ation hannels. This te hnology was also transferred to the Windows environment, and from about 2000 begin to appear malware programs [1℄ that use it to hide its (and other downloaded programs) presen e in the system, and a

ording to Mi rosoft resear hes [2℄, in 2006 nearly 20 % of malware dete ted on Windows XP SP2 systems ontained elements of rootkits te hnology. As rootkits te hnology is known from many years, there exists also many anty-rootkit te hnologies [3℄, attempting to ountera t rootkit installation and to dete t and remove installed rootkit from systems. Most of them operate on demand and provides high s anning system overhead, ee tively ruling out the

2

M. Skrzewski

possibility of using the system by the user when sear hing for rootkits, and the results of their work are not always guaranteed to dete t threats. The presen e of on ealed malware an be dete ted from outside of the infe ted system by dete ting hidden (invisible to the system) elements of network

ommuni ation  su h as additional open ports or network onne tions, but this also requires the use of external spe ial tools (port s anners, network sniers) to manualy he k the state of system's network ommuni ation. The paper presents a new method of rootkit operation dete tion based on the analysis of network a tivity pi tures from the lo al a tivity monitoring program and from external monitoring system, whi h allows for ontinuous monitoring of the system behavior without ompromising the work ability of its users in

ontrary to the most of urrent rootkit dete tion methods. Remaining parts of the paper are organized as follows: Se t. 2 des ribes elements of rootkit and anty-rootkit te hnology, network a tivity monitoring tools are presented in Se t. 3, Se t. 4 presents details of the rootkit dete tion algorithm and its tests on resear h network on virtual ma hines and nal on lusions are presented in Se t. 5.

2

Rootkit Te hnology

Rootkits te hnology was established in order to allow legitimate appli ations or malware programs for permanent, undete table presen e and the ability to perform a tions on a omputer system. This task in ludes the ability to hide in the presen e of used resour es su h as pro esses, les, registry keys, and open

ommuni ation ports. These possibilities an be used with dierent intentions, both legal (prote tion of omplian e monitoring programs for DRM and opyright [4, 5℄, omplian e with se urity poli ies) or not ( on ealment of an intruder operation or malware program). Initial rootkits implementations [19℄ were based on ode modi ation of unix system tools (ls, ps, lsof, nd, . . . ), and ltration of the information provided system users. Modi ations to the ode were easy to dete t, so the next generation of rootkits implements ltration me hanisms deeper within the operating system, by modifying the data supplied to the unmodied system tools. Now hiding me hanisms may be found implemented at dierent levels and pla es of the operating system, from the appli ation level, through system libraries, kernel stru tures, virtualization platforms, to the hardware (rmware level), and may have the form of various modi ation of kernel obje ts, system tables, or dire t memory stru tures modi ation [6℄. There are emerging onstantly new ideas for possible new rootkits te hnologies [7, 8℄. Common feature of proposed solutions is tenden y to move deeper and deeper in system with the me hanisms of information ltering, loser to the system hardware, and modify at lower-levels information provided for tools operating at higher levels. Rootkit dete tion methods are aimed at dete tion of dieren es between the image of the system resour es (les, pro esses, servi es, open ports) presented

Monitoring System's Network A tivity for Rootkit Malware Dete tion

3

to the user and their real state. They are usually arried out by s anning with various methods the system resour es, and by omparing the information provided by the system tools to the user with the information about the state of system resour es provided by the low-level tools (su h as the kernel API fun tions, system drivers, et .), or by dete ting hanges in the ontent of key kernel obje ts. Another, rarely used alone method for dete ting the presen e of rootkits in the system is analysis of the image of network ommuni ation. Comparison of the image of network ommuni ation hannels visible in the system with the image visible from the outside requires the use of external port s anners like nmap. It allows for the dete tion of on ealed open ommuni ation ports, but like the other methods, is not very suitable for ontinuous monitoring of system status. Most of the methods of rootkits dete tion has the form of a single sear hing inside the system, usually performed without user a tivity due to the generated load. With new types of rootkits su h exploration does not always have a han e to dete t them. A more reliable method to dis lose the presen e of rootkits appears to be a ontinuous monitoring of the system network ommuni ations, both inside and outside, and the dete tion of on ealed a tivity before the system tools.

3

Network A tivity Monitoring

In Windows, as in other systems, there are tools to monitor network onne tions, open ports su h as netstat, PortReporter but they are not suitable for ontinuous operation. For ontinuous monitoring it is required the ability of a

urately re ording the moments of hanges in network onne tivity, the asso iated pro esses, and information on the amount of data transferred in ea h onne tion. For greater e ien y, the registration me hanism should not rely on y li polling of system interfa es, but on apturing of events generated by hanges in the state of system onne tions. Figure 1 shows the ar hite ture of the software interfa es and ommuni ation proto ol sta k of Windows. Of the system's kernel-level interfa es for re ording operation of the network best suited is NDIS interfa e through whi h passes all network tra . Events generated at this level, however, do not ontain information about the programs using them or addresses inside the upper layer proto ols to whi h they are dire ted. This information is available on the transport layer interfa e TDI (Transport Driver Interfa e), but does not in lude auxiliary proto ols su h as ICMP and ARP. A

ording to these requirements, a tool for logging on the interfa e TDI  tdilog was developed. The program re ords the laun h of pro esses, the moments of opening and losing ports, the establishment and ompletion of onne tions, with an indi ation of the dire tion to or from the system.

4

M. Skrzewski

Windows Sockets Application

Application

Windows Socket API Windows Sockets

Windows Socket SPI

Library Base Service Provider For TCP/IP

User Mode Kernel Mode

TDI TCP/IP

Kernel

NDIS NDIS Miniport Driver

NIC

Firmware

Fig. 1. Windows network proto ols sta k

From the point of view of rootkits dete tion it is important to assess the possibility of hiding information about events against the program. In Figure 1 indi ated the possible levels of pla ement of rootkits of dierent lasses. It seems that tdilog is likely to provide a

urate information on the appli ation layer rootkits a tivity, and perhaps also on rootkits modifying the system libraries, but rootkits a ting from the kernel and rmware an modify the data re orded by the program. For this reason, with simpler rootkits the expe ted dieren es in information about the network onne tions may not arise.

4

Rootkit Dete tion Algorithm

Proposed algorithm for dete tion signs of rootkit operation is based on the possibility of the omparison of the images of network ommuni ation in the form of ows list, oming from two dierent sour es  lo al re order tdilog and from external auditor  argus [20℄. Comparison overs the number of registered ows initiated in a ertain dire tion, the number of destination ports, and in the presen e of quantitative dieren es are ompared also destination addresses. Despite the same des ription of the ommuni ation model  the ow, des ribed by proto ol, sour e IP address and port, destination IP and port, there exists small dieren es in program re ords. In tdilog single ow is stored in the form of two re ords  the opening and ompletion of onne tion, the latter ontains information about the amount of transferred data. Argus internally writes to the database a lot of information, whi h manner of presentation diers depending on the lient program  ra (read argus), ramon, rasort, ra luster and

Monitoring System's Network A tivity for Rootkit Malware Dete tion

5

others. The primary lient  ra program presents a list of arranged hronologi ally subsequent onne tions. For fast dete tion of dieren es a more useful is an aggregate pi ture, grouping all the onne tions to the external system in one entry  a session, allowing to easily noti e the la k of ertain ports or IP addresses in one of the images. To obtain su h form of information, the text logs of tdilog were saved to the postgress database, and then using the SQL ommand group by sport, d_ip, dport order by time the list of sessions was reated. From argus the data were read by ra luster program, whi h groups in a similar way onne tion re ords. To simplify the analysis the in oming and outgoing sessions were ompared separately for TCP and UDP proto ols. To test the algorithm, network ommuni ation of some Windows XP systems were monitored during websites browsing and in online a tivities, and were obtained full omplian e in quantity and in addresses of registered session. There were slight dieren es of the session start time from 20 to 80 ms, presumably due to delays in the operation of the network infrastru ture. Below, few lines of session log from tdilog database: time 11:57:30.401 11:57:30.602 11:57:31.152 11:57:31.243 11:58:17.058

oper prot CONNECT TCP CONNECT TCP CONNECT TCP CONNECT TCP CONNECT TCP

dir OUT OUT OUT OUT OUT

sport 1035 1036 1037 1038 1040

dstIP dport 74.125.136.94 80 74.125.136.94 80 74.125.136.120 80 74.125.136.94 80 62.149.142.34 80

And for omparison, from argus using ra luster lient program: ra luster -M dsrs=time,flow,metri -n -p3 -r argus.out - t p and sr host 172.17.114.215 StartTime 11:57:30.455 11:57:30.662 11:57:31.207 11:57:31.307 11:58:17.085

Proto t p t p t p t p t p

Sport 1035 1036 1037 1038 1040

Dir DstAddr.Dport Pkts 74.125.136.94.80 108 74.125.136.94.80 55 74.125.136.120.80 16 74.125.136.94.80 8 62.149.142.34.80 11

Bytes State 89243 CON 40296 CON 8375 CON 1082 CON 1930 CON

4.1 Dete tion Algorithm Veri ation In order to verify the proposed method of dete ting the presen e of rootkits on the systems a series of experiments on the resear h network was planned,

overing the infe tion of the virtual ma hines Windows XP SP3 known opies of the malware programs. Obtaining opies of a tive programs that use rootkit me hanisms was quite embarrassing. On Internet there are websites posting the informations about domain names of malware infe ted systems [9, 10℄, monitoring the a tivity of systems that make up botnets [11℄, or oering a

ess to ar hived opies of the malware programs. The validity of this information, however, hanged qui kly over time, and often indi ated on the sites servi es were not a tive or systems were restored

6

M. Skrzewski

after the removal of malware. Part of the already obtained opies had been ina tive  program was looking for hard- oded addresses of C&C system, that have already been turned o, and did not takes of any other a tivity. Most opies of malware re orded by honeypot systems (e.g. Dionaea [12℄) showed no eviden e of masking of any information about the state of network

ommuni ation. After many attempts were olle ted, mainly from [13℄, and [11℄ and used to perform a number of tests a samples of programs lassied as rootkits. 4.2

Test Environment

All tests were run on separated resear h subnetwork. On main test system with installed xen virtualization platform and Centos 5.8 system as dom0, virtual Windows XP SP3 instan es were run. Another linux system was harged with task of monitoring the network ommuni ation. Both systems were onne ted via ethernet hub to the remaining part of unltered, internet onne ted, malware monitoring network. Laun hed malware samples were monitored by the installed on Windows XP tools  tdilog, pro monitor, pro explorer [14℄, apturebat [15℄ re orded the internal workings of malware on the system, hanges to the le system and

omputer's network ommuni ation. On the monitoring system runs programs argus monitor, t pdump ongured to register DNS queries and etherape [16℄, visualizing the urrent pi ture of network ommuni ation. Depending on the a tivity of the tested malware infe ted system operate from a few minutes (in ase of very a tive worms s anning the network) to a few hours for not showing greater network a tivity programs. Then virtual ma hine was turn o, external re ording was stopped and registered logs were exported for analysis. Ea h of the sele ted malware programs was tested independently. 4.3

Malware Samples

A well-known problem in the examination of malware samples are malware names. Parti ular opies of the malware o

ur in a variety of AV software manufa turers under a very dierent names. The tests used opies of programs des ribed as: MaxRootkit DarkMegi PhazeTDL ZeroA

ess Ramnit Rusto k-23 Zeus

Md5: Md5: Md5: Md5: Md5: Md5: Md5:

392ddf0d2ee5049da11afa4668e9 98f dd313b92f60bb66d3d613b 49 1ef35e 4a052246 5551e83d2d55f80e72f03eb 251a2 7eff890 58a9d9eda5b1391082 607b2219fb fbfe8e6a 9d7f3fb8d50e 1a713083a0b 21be19f1e 496df4e651 a7772183d2650d9d4f26ffa02fd41d64

On the web one an nd many dierent des riptions of the same malware, but do not by the end of them an gure out what type of rootkits given malware represents. Some o

ur under many dierent names used inter hangeably, and

Monitoring System's Network A tivity for Rootkit Malware Dete tion

7

often are lassied into dierent ategories (trojan, ba kdoor, downloader or virus). From sele ted for tests malware MaxRootkit, ZeroA

ess, Rusto k and Zeus are referred to as kernel mode rootkits, DarkMegi and Ramnit as user mode and PhazeTDL (TDL-4) as a boot mode rootkit. 4.4

Tests Results

Network a tivity of tested malware during the tests was varied. Some of them behaved like a tive Internet worms (Ramnit, MaxRootkit, ZeroA

ess) and strongly s anned the network environment, trying to make onne tions to a number of sele ted systems, others (e.g. Zeus) s anned the surroundings less intensively, at 200250 ms trying to onne t to the 810 systems, and others (Rusto k, PhazeTDL) did not show ex essive a tivity. For part of tested malware were no dieren es in the number of registered

onne tions via tdilog and the argus. This was the ase for rootkits Ramnit, DarkMegi, PhazeTDL and Rusto k. In other ases, in the logs re orded by argus there are onne tions that do not exist in the data of tdilog. The dieren es relate to individual ports to or from whi h ommuni ation takes pla e in hidden form. In ase of ZeroA

ess rootkit these were onne tion on port 16464, both TCP and UDP, in oming and outgoing. In the ourse of a few minutes of test argus registered a lot of udp s ans and 6 su

essful t p onne tions: StartTime Sr Addr.Sport Dir 11:00:10.587 96.239.109.92.1167 11:00:12.960 76.172.46.93.3932 11:04:01.975 75.109.6.239.1598 11:04:25.003 50.193.131.1.29578 10:58:51.741 172.17.114.215.1030 11:04:22.065 172.17.114.215.1032

DstAddr.Dport Pkts 172.17.114.215.16464 21 172.17.114.215.16464 3 172.17.114.215.16464 9 172.17.114.215.16464 2 68.50.148.254.16464 10 61.67.19.54.16464 10

State CON CON CON CON CON CON

Similarly, in ase of rootkit MaxRootkit they were hidden onne tion on port 13620, argus registered 11 additional TCP onne tions in about 5 minutes. For Zeus rootkit the situation was a little dierent, were hidden some onne tions to various IP and ports (13606, 14869 and 10333 in this ase), and on 10 TCP

onne tions registered during test by tdilog argus registered four more: StartTime 16:29:24.027 16:30:07.384 16:33:41.144 16:34:14.478

5

Sr Addr.Sport 172.17.114.193.1055 172.17.114.193.1057 172.17.114.193.1068 172.17.114.193.1077

Dir DstAddr.Dport Pkts 195.169.125.228.13606 6 99.148.69.143.14869 6 110.168.24.157.10333 6 110.168.24.157.10333 6

State CON CON CON CON

Con lusions

The proposed algorithm for rootkits dete tion on the basis of analysis of information about the network ommuni ation is working properly as intended, and indi ates the presen e of rootkits modifying the user reported information. In

8

M. Skrzewski

ondu ted tests the hiding of information was not always present and it is not

lear how this an be explained. After starting rootkit's ode in the system there was some program a tivity  there were hanges in the registry, were reated new les, often malware le were removed, but there was no expe ted ommuni ation in networks to the C&C servers, even after a few hours of the program operation. Perhaps the reason might be an in omplete rootkit ode, the dete tion of the virtual ma hine environment and the essation of a tivity, or another than the required version of the operating system. Adopted model of dete tion assumes that not in every ase may go register the expe ted dieren es in ommuni ation, e.g. for rootkits modifying the information in the appli ation layer the lo ally and externally re orded image of

ommuni ation an be the same. The proposed algorithm is aimed at the dete tion of symptoms of infe tion and on prevention of persisten e of its ee ts. The main advantage of the algorithm is the possibility of it ontinuous operation, without ae ting the normal working onditions of the system users. In

onjun tion with the threats dete tion algorithms based on the system's network a tivity monitoring it allows for dete tion of malware in the system, preventing its permanent presen e in the orporate network. Nowadays, in reasingly larger and more omplex systems are reated [17℄ requiring the se urity and ondentiality of information pro essed and stored, as well as high performan e and reliability. Su h systems require extensive prote tion against threats [18℄, in luding not only the level of preventing infe tion (the rst line of defense), as well as ontinuous monitoring and dete tion of signs of the possible presen e of a tive malware programs, represent a threat to the

ondentiality and integrity of information pro essed. A need to develop su h a se ond line of defense is be oming in reasingly obvious. Presented algorithm for rootkit's tra dete tion an be an important part of the su h solution.

Referen es 1. Shields, T.: Survey of Rootkit Te hnologies and Their Impa t on Digital Forensi s. http://www.donkeyonawaffle.org/mis /txs-rootkits_and_digital_ forensi s.pdf 2. Naraine, R.: Mi rosoft: Stealth Rootkits Are Bombarding XP SP2 Boxes. http://www.eweek. om/ /a/Se urity/Mi rosoft-Stealth-Rootkits-AreBombarding-XP-SP2-Boxes/ 3. Josse, S.: Rootkit dete tion from outside the Matrix. Journal in Computer Virology 3(2), 113123. Springer-Verlag Fran e (2007) 4. Geist, M.: Sony Rootkit Redux: Canadian Business Groups Lobby For Right To Install Spyware on Your Computer. http://www.mi haelgeist. a/ ontent/view/ 6777/125/ 5. Brown, B.: Sony BMG rootkit s andal: 5 years later. http://www.networkworld.

om/news/2010/110110-sonybmg-rootkit-fse ure-drm.html 6. Rozas, C., Khosravi, H., Sunder, D.K., Bulygin, Y.: Enhan ed dete tion of malware. Intel Te hnology Journal 13(2) (2009)

Monitoring System's Network A tivity for Rootkit Malware Dete tion

9

7. King, S.T., Chen, P.M.: SubVirt: implementing malware with virtual ma hines. In: Se urity and Priva y, 2006 IEEE Symposium on, pp. 315327 (May 2006) 8. Tsaur, W.-J.: Strengthening digital rights management using a new driver-hidden rootkit. Consumer Ele troni s, IEEE Transa tions on 58(2), 479483 (May 2012) 9. http://www.malwaredomainlist. om/mdl.php 10. https://se ure.mayhemi labs. om/malhosts/malhosts.txt 11. https://zeustra ker.abuse. h/monitor.php?browse=binaries 12. http://dionaea. arnivore.it/ 13. http:// ontagiodump.blogspot. om/sear h/label/rootkit 14. http://te hnet.mi rosoft. om/en-US/sysinternals 15. http://www.honeynet.org/proje t/CaptureBAT 16. http://etherape.sour eforge.net/ 17. Gorawski, M., Marks, P.: Towards Reliability and Fault-Toleran e of Distributed Stream Pro essing System, on. In: International Conferen e on Dependability of Computer Systems (DepCoS  RELCOMEX '07), pp: 246253. IEEE, Szklarska Poreba, Poland (2007) 18. Gorawski, M., Marks, P.: Che kpoint-based resumption in data warehouses. In: Sa ha, K. (ed.) IFIP International Federation for Information Pro essing. Software Engineering Te hniques, Design for Quality, vol. 227, pp. 313323. Boston: Springer 19. M Afee: Rootkits. Part 1 of 3: The growing threat. http://download.nai. om/ Produ ts/m afee-avert/whitepapers/akapoor_rootkits1.pdf 20. ARGUS  Auditing Network A tivity. http://www.qosient. om/argus