A Technique for Defending Against Buffer Overflow ... - Semantic Scholar

StackOFFence: A Technique for Defending Against Buffer Overflow Attacks∗ Bharat B. Madan IST Div., ARL Penn State Univ., State College, PA. 16801 [email protected]

Shashi Phoha Director, Information Tech. Lab NIST, Gaithersburg MD. 20899-8900 [email protected]

Abstract Software coding practices, in the interest of efficiency, often ignore to enforce strict bound checking on buffers, arrays and pointers. This results in software code that is more vulnerable to security intrusions exploiting buffer overflow vulnerabilities. Unfortunately, such attacks form the most common type of security threats to the computer and information systems, making it imperative to find efficient solutions for the buffer overflow vulnerabilities. Typically, an attacker is able to affect a successful intrusion by causing buffer overflow in the stack frame of a function call, thereby causing the valid return address to get overwritten by a malicious value. This allows the attacker to redirect the return from a function call to a malicious piece of code introduced by the attacker. Depending on the nature of the malicious code, the attacker is able to compromise availability, integrity, or confidentiality of a system. Researchers have suggested transforming the return address or even using an entirely separate stack for managing the return addresses. This paper describes a simple technique that ensures the integrity of the return address by pushing on the stack two copies of the return address, a transformed (or encrypted) return address value along with the original one. Before popping the return address, two return address values are compared to detect any malicious activity, thus preventing the exploitation of the stack based buffer overflow vulnerabilities. The proposed modification may be implemented at the CPU architecture level or by simple modification to the compiler’s prologue and epilogue code.

1 Introduction Rapid growth of the Internet and its use in high value and strategic applications has brought with it the un-welcome ∗ The

research of the first two authors was funded by the DARPA under the Emergent Surveillance Plexus MURI Award No. DAAD19-01-0504. The work of the third author was performed under the AFOSR MURI Award No. F49620-00-1-0327. Any opinions, findings and conclusions or recommendations expressed in this paper are those of the authors alone and these do not necessarily reflect the views of the sponsoring agencies.

Kishor S. Trivedi Pratt School of Engn. EC&E Dept., Duke Univ. Durham, NC. 27709 [email protected]

attention of cyber attackers. The motivation for an attacker may range from intellectual gratification, denying valid users access to some Internet sites, to stealing proprietary and confidential information. Attackers exploit a variety of vulnerabilities in the software systems driving the computer/information systems. One of the most popular vulnerability exploited by the attackers is the buffer overflow vulnerability. Buffer overflow typically results in some data or code area to be overwritten, thus destroying valid information. Since the system stack logically is also a buffer, a buffer overflow vulnerability centered around the stack may be exploited by corrupting the activation record of a subroutine function. Similarly, by overflowing the buffer set by the setjump(buffer) and used later by the longjump(buffer) system call [8] can cause the program to execute malicious code. Common ways of exploiting buffer overflow vulnerabilities include: 1. Defining a buffer (or a pointer to it) as a local variable in a function, procedure or a subroutine and subsequently placing code in this function that exceeds the normal size or the defined size of this buffer. 2. Using printf format strings to read and write protected memory areas, e.g., the stack area 3. Calling a function or more commonly submitting a URL to a web site with very long cgi argument string. In both these cases, the effect is to overwrite the activation frame created for the called function in such a way, that the return address or some other crucial piece of data gets overwritten. If the return address gets overwritten, subsequent return from the called function may cause the program control to get transferred to a malicious piece of code. Depending on the nature of this malicious code, its effect may be to wipe out certain crucial files, generate a traffic flood on the Internet, flood a web site with HTTP requests or even steal confidential information from files. Historically, the so called Morris Internet worm [9] was one of the first instances of exploiting the buffer overflow in a commonly used application code, e.g., UNIX f inger utility. Buffer overflow vulnerability has also been successfully ex-

Proceedings of the International Conference on Information Technology: Coding and Computing (ITCC’05) 0-7695-2315-3/05 $ 20.00 IEEE

ploited in software utilities, which were in fact designed to enhance security. ssh is replacing popular but insecure telnet and rsh utilities to allow secure remote access to systems. However, older versions of ssh had a vulnerability that could give the attacker an ability to create a shell with root level privileges by exploiting the buffer overflow vulnerability [1]. More recently, the CodeRed worm [5] gained a lot of notoriety by causing a distributed denial of service (DDoS) attacks on a specific web site (White House web site in this case). The CodeRed worm exploited a buffer overflow vulnerability in the IIS web server [15] code. The CodeRed worm was injected into a vulnerable IIS web server by submitting a URL with a long and well crafted argument string that caused the return address of the previous activation record in the stack to be overwritten with a malicious return address. Eventually, the worm code resulted in a flood of HTTP requests being generated for the White House web site, thereby creating DDoS attack on the targeted web site. Out of all the security alerts issued by the CERT/CC in the last five years, large portion of these alerts were due to the buffer overflow vulnerabilities [4]. As a result, tackling buffer overflow attacks has become a priority issues for the cyber security community. Cowan et al. [7] suggested the use of a canary value to protect a valid return address. This canary value is placed on the stack along with the return address, while calling a function. Any attempt to overwrite the stack area containing this return address would also result in overwriting the canary value as well. There is however a small probability that the attacker can guess this canary value and take appropriate measures to thwart this solution. Xu et al. [17] suggest the use an additional stack besides the system wide stack used to provide storage for local variables, function parameters and return addresses etc. This new stack has to be implemented either in hardware requiring changes to CPU architecture, or in software by modifying the compilers which generated the appropriate code for managing two stacks. StackShield presents a similar idea of splitting the stack and implementing the split stack using a assembly language code [3]. StackGhost [12] uses static system wide or process wide cookies to protect the return address by transforming the return address with cookies. In this paper, we suggest a new technique, called the StackOFFence1 that helps build a fence around the return address stored in the stack. The proposed technique involves transforming or encrypting the instruction pointer value, (i.e., the return address) residing in the EIP2 register. The return address value is encrypted when it is placed on the stack. The encryption is done efficiently by XORing EIP with a unique key value that can be generated in a fairly simple manner. When affecting return from a function call, the encrypted EIP value is again XORed with this key value

to yield true and correct value. If the attacker manages to overwrite this encrypted EIP value with a malicious return address, when XORed with the key value, the decrypted malicious return address will generate some random return address. This will cause the program control to be transferred to an arbitrary address instead of control being sent to the malicious code, as intended by the attacker. Sending the program control to a random address, though successful in defending against the intended attack, can still have unpredictable consequences. To address the issue of unpredictability, a second modification suggested in this paper detects any malicious changes made to the return address. If the valid return address is changed maliciously, the program control is transferred to a special function that can either halt the system or raise an alarm for the security administrator who can audit the situation using machine level debugging tools. The proposed modifications essentially simulate the dual stack functionality without actually requiring two physical stacks as in [17]. The modifications are easy to implement by making minor changes to the compiler. It is also possible to implement these changes in hardware by modifying the CPU architecture. The compiler-based solution adds only a very small number of machine level instructions to the overall object code it generates. As a result, performance overhead caused by the compiler-based solution is likely to so small that it will not be easy to justify CPU architecture changes. Consequently, much of our focus will be confined to the compiler based solution only. In order to be able to explain the proposed techniques clearly, rest of the discussion assumes Intel x386/Pentium type of CPU instruction set [2] and the public domain ’C’ compiler, gcc [11]. The remainder of the paper is organized in three sections. Section 2 describes in details the origin of the buffer overflow security problem and how attackers exploit such vulnerabilities. Section 3 describes the proposed solution and the modification required in the compiler. Further enhancements which allow the program control to be transferred to a suitable stack overflow monitoring function, in the event of a buffer overflow attack, are described in Section 4. Implementation issues related to the proposed scheme are discussed in Section 5 and the concluding remarks are provided in Section 6.

2 Anatomy of a Buffer Overflow Attack Consider the following example code in ’C’ language, in which the main program calls a function, sum, MAIN PROGRAM main() { int a,b,c; c = sum(a,b) }

SUBROUTINE sum(int x,y) { int z; z = x + y; return z; }

1 Stack

OverFlow Fence use Intel X86 register naming conventions and EIP denotes the 32-bit instruction pointer register 2 We

The corresponding assembly language code generated by a ’C’ compiler, such as the gcc compiler is,


1 2 3 4 5 6

MAIN ... PUSH PUSH CALL ADD

PROGRAM

1 2 3 4 5 6 7 8 9 10

b a adder ESP,4

(result is in AX) ...

SUBROUTINE adder:PUSH EBP;prologue start MOV EBP,ESP SUB ESP,4;prologue end MOV AX, [EBP+4] ADD AX, [EBP+8] MOV [EBP-4], AX MOV ESP,EBP;epilogue start POP EBP;epilogue end RET

Each function call and return requires the compiler to generate some extra code at the beginning (prologue) and at the end (epilogue) of the called function. The above assembly language listing of the code generated by the compiler shows the standard prologue (line 2-4) and the epilogue (lines 8-9), associated with a function call. For most CPU architectures and programming languages, the stack plays a central role in supporting nested subroutines calls and returns. The stack needs to accessed using two address registers. ESP CPU register is used to provide the usual last-in-first-out (LIFO) access, while EBP register is used to provide random access to the stack for accessing the local variables and the call parameters. Fig. 1 shows the stack contents for the above sample program. For proper

local variables (z) ebp eip

new stack frame for the called procedure (sum(x,y))

actual params (a, b)

stack growth local variables (a, b, c) eip ebp

actual params

previous stack frame of the calling procedure (main)

correct return addr

Figure 1: Activation record stack structure functioning of the system, it is important to maintain the integrity of the stack in general and the return address in particular. Majority of the buffer overflow security attacks work by overwriting the return address. Let us now look at a typical buffer overflow attack. Consider the anatomy of the well known CodeRed worm [5], which adversely affects the Microsoft’s IIS web server software. The attacker submits a URL with a long argument string and send it to the web server chosen for injecting the CodeRed attack. Part of the argument string of this URL consists of: HTTP GET ........ ...... %u9090%u6858 %ucbd3%u7801%u9090%u6858 %ucbd3%u7801 %u9090 %u6858%ucbd3%u7801%u9090%u9090

%u8190%u00c3%u0003 %u6858%ucbd3%u7801 %u9090 %u6858%ucbd3%u7801%u9090 ...... Buffer overflow causes the original EIP value to be overwritten with 0x7801cbd3, which is an address in the msvcrt.dll. The code contained in this DLL at this address is an indirect call instruction, call EBX. This indirect call address lies in the stack segment, and this area of the stack has been overwritten by the attacker to contain the actual malicious worm code. In the next section, we show that by using the proposed solution, the return address produced by the worm will get transformed to some arbitrary value which will ensure that the worm will code never get activated, thus preventing the attack from succeeding.

3 Protecting the Return Address The generic technique used by an attacker for exploiting a buffer overflow vulnerability in a software application consists of overwriting the correct return address stored in the stack. The attacker substitutes the valid return address with an address that points to the malicious code. To prevent the attacker from succeeding, it would therefore suffice to prevent the program control from being transferred to the malicious code. The solution proposed in [12] obfuscates the return address by XORing it with a constant but secret (unknown to the attacker) cookie value. The idea proposed in [7] places a canary value next to the return address. The canary value is expected to remain invariant on return from a function call. Attacker in the process of overwriting the return address value typically will also modify the canary value. Any change in the canary value is detected by the epilogue code. The solution proposed in this paper is based on encrypting the return address with a key. For the solution to workable, this key should have the following desirable properties: • Invariant: the key value should not change during the course of execution of a function call. This also implies that the attacker should not be able to alter the key value. • Dynamic: The key should not be a constant value. Instead it should keep changing in a way unknown to the attacker. Otherwise the attacker may be able to guess the key. Knowledge of the key and the encryption algorithm will make possible for the attacker to overwrite the actual (encrypted) return return address with his own encrypted malicious return address. • Simple: it should be possible to generate the key using simple operations to minimize processing overheads. For the same reasons, the encryption algorithm too should be simple. Moreover, despite simplicity yet the attacker should not be able to figure out the encryption process.


Keeping these criteria in mind, we propose to use the contents of an internal register (the EBP register), as a key for encrypting and decrypting the return address stored on the stack. Note the EBP is a supervisory mode (or Ring 0) register that is not normally accessible to a user from a program. Choice of EBP satisfies all the requirements of a good key as enumerated above. The proposed technique for preventing buffer overflow attacks from succeeding now only requires a single additional operation for both the prologue and the epilogue. This is explained by the code snippet shown in the listing below: 1 2 3 4

5 6 7 8 9 10

11 12 13

SUBROUTINE adder:PUSH EBP MOV EBP,ESP ;;;;;;; new prologue XOR [EBP+4], EBP ;[EBP+4] has EIP ; XOR encrypts ret addr (EIP) ;;;;;;; end of new prologue SUB ESP,4 MOV AX, [EBP+4] ADD AX, [EBP+8] MOV [EBP-4], AX ;;;;;; new epilogue XOR [EBP+4], EBP ;[EBP+4] has EIP ; XOR decrypts ret addr (EIP) ;;;;;; end of newer epilogue MOV ESP,EBP POP EBP RET

The prologue and the epilogue modifications are identical and both entail just one machine level operation, XOR [EBP+4], EBP. Since [EBP+4] fetches the return address value stored in the stack, this operation effectively encrypts the return address value with address contained in the EBP register. Because EBP value keeps changing arbitrarily, the attacker has no means to guess this value, thus ensuring the robustness of the proposed solution. Therefore, the proposed solution requires only a single additional assembly language instruction in the prologue and the epilogue and no extra variables, the overheads of the proposed solution are almost negligible. Note that we use the EBP contents as the key value used for encrypting the EIP (literature also refers to this register as the program counter or the PC) value. In actual fact, EBP is the address of a memory location at which the new activation frame starts. Being a real or virtual address, the attacker has no way of guessing the actual value of this address. Moreover, this value keeps changing and is different for each call to function or a procedure. These properties of EBP make the proposed scheme robust against inadvertent guessing of the key value by the attacker. It is also instructive to make another useful observation. Even though the EBP register has been stored on the stack, and hence it can be easily over written by a suitably crafted value by the attacker (using buffer overflow), such an operation will have no adverse effect on decryption of the return address value. This is due to the fact that while the modified EBP value may sit in the stack, the modified value does not actually get transferred to the EBP register. As a result, EBP will continue to hold the valid value (i.e., the value used earlier for encrypting the return address) for decrypting the return address value. This ensures that the attacker

will not be able to successfully transfer program control to a location of his choice which contains the malicious code. Revisiting the Code Red example, assume that the cur-

local variables z ebp eip=eip ebp actual params a b z1 local variables .. zk eip ebp

new stack frame for the called procedure sum(a,b) stack growth

previous stack frame of the calling procedure

actual params

Figure 2: Stack structure with encrypted EIP rent value stored in the EBP is 0x01234567. Therefore, the actual and correct return address will get overwritten by the worm crafted return address value of 0x7801cbd3. However, the epilogue will transform the return address to 0x79224eb3. This will cause the next instruction to be fetched from the random address 0x79224eb3 instead of from the 0x7801cbd3 as desired by this worm. This will cause the program control to be transferred to some arbitrary instruction stored at the address 0x79224eb3, thus defeating the CodeRed attack. Note that even though we have managed to defeat the CodeRed attack, the actual outcome may be quite unknown because of the arbitrary set of instructions found at address 0x79224eb3 onward. In the next section, we improve the proposed solution which ensures that in the event of buffer overflow attack, the program control does not get transferred to some arbitrary set of instructions.

4 Capturing and controlling a buffer overflow attack In the event of a buffer overflow attack, the technique discussed in the previous section is only capable of preventing the program control for being transferred to a malicious address desired by the attacker. The program control instead is sent to a random address. Execution of the instructions starting at this random address can have unpredictable consequences. A better strategy would be to transfer the program control to a well designed overflow monitoring function that can either halt the system, raise a security alarm for the system administrator or transfer the control to suitable debugging tool. To carry out such a modification, we first need to detect the occurrence of such an attack. In the proposed scheme, this is done by the function call’s modified


prologue code which redundantly stores the un-encrypted (plain) return address (along with the encrypted return address) on the stack, as shown in Fig. 3. The actual detection of malicious overwriting of the valid return address is carried out by the compiler generated epilogue code. The compiler’s epilogue generation part is further modified so as to compare the un-encrypted plain return address with

local variables (z)

new stack frame for the called procedure sum(a, b)

ebp eip=eip ebp eip actual params a b z1 local variables .. zk eip ebp

stack growth

previous stack frame of the calling procedure

actual params

Figure 3: Stack with encrypted and un-encrypted EIP decrypted return address. Any mismatch between the two is an indication of a buffer overflow attack. When such a mismatch is detected by the epilogue code, the program control is transferred to the stack overflow monitoring function. The working of the overflow monitor can be summarized by the program flow diagram shown in Fig. 4. called fn.

save EIP save and encrypt EIP

decrypt EIP and Compare with EIP mismatch No

Yes

call overflow monitor

Figure 4: Overflow monitor program flow The assembly language program generated by the compiler for the earlier example is shown in the listing below: 1 2 3 4 5 6

SUBROUTINE adder: POP EAX ; new prologue start PUSH EAX ; unencrypted EIP PUSH EAX ; duplicate EIP PUSH EBP ; standard prologue MOV EBP,ESP ;;;; new prologue 7 XOR [EBP+4], EBP; [EBP+4] has EIP ; XOR encrypts ret addr (EIP) ;;;;;;;;;;;; end of new prologue 8 SUB ESP,4 9 MOV EAX, [EBP+4] 10 ADD EAX, [EBP+8] 11 MOV [EBP-4], EAX ; local var ’z’ ;;;;;; new epilogue

12

13 14 15 16 17 18 19 20 21 22

XOR [EBP+4], EBP ; [EBP+4] has encrypted EIP ; XOR decrypts ret addr (EIP) XCHG EAX, [EBP+8] ; copy uncrypted EIP CMP EAX, [EBP+4] ; compare two EIP values XCHG EAX, [EBP+8] ; restore EAX CNZ OVF_MONITOR ; malicious ret ; address. Call monitor ; HLT or Halt the system ; end of new epilogoue MOV ESP,EBP POP EBP RET

The new prologue first retrieves the return address saved by the calling function on the stack into the register EAX. This return address is next pushed twice on to stack. This is followed by the standard function call prologue of setting up the boundary of the activation record for the called function. After this, we use the prologue modification described earlier in Section 2 which involves encrypting one of the return address values with the key value provided by the contents of the EBP register. Subsequently, after the called function has completed its actions, the modified function epilogue decrypts the encrypted return address value and compares it with the un-encrypted return address. Note that if the attacker is successful in causing a stack overflow attack, the encrypted and the un-encrypted return address values will both get overwritten by the values prescribed by the attacker. When this encrypted value is decrypted by XORing it with the EBP, an entirely different value will be generated. Comparison of the un-encrypted and the decrypted return address values will result in a mismatch. This mismatch is detected and the control will be transferred to the overflow monitor which can halt the system, raise an alarm or call a debugging tool. At this stage, the attack foot print is still retained in the stack and it may be possible to use a debugging tool which can roll back the state just prior to call to this function, as suggested in [13]. Rolling back the state and single step debugging will facilitate a more thorough forensic scrutiny of the buffer overflow attack. It is important to point out here that for debugging, a user level debugging tool (e.g., gdb) will not be useful. Instead, we will need to use a kernel level debugger [16, 6], such as the command line based i386kd or the GUI based windbg [14], etc. This is due to the fact that when a buffer overflow attack is detected and the control is transfered to the monitoring program, this monitoring has to carry out its debugging functions at the supervisory (or ring 0) level in order to be able to access the stack.

5 Implementation Issues The solution proposed in the previous section can be implemented either by slightly modifying the compiler or by augmenting the CPU architecture. To explain the compiler based solution, consider the freely available gcc compiler from the Free Software Foundation [10]. Typically,


a compiler, besides generating the actual code for a function, also has to produce certain housekeeping code just prior to the actual call to a function as well as just prior to returning from a function call. These operations are commonly referred to as the P rologue and the Epilogue, respectively. The gcc compiler documentation clearly documents these functions and the interested reader may refer to the online documentation [11] for further details about the internal working of this compiler. For the Intel x386 and Pentium class of CPUs [2], the prologue and the epilogue operations are carried out by calling the gcc’s internal functions ix86 expand prologue() and ix86 expand epilogue(), respectively. To implement the necessary modification required by the StackOF F ence, we need to modify just these two functions of the gcc compiler. These modification will additionally perform encryption and decryption of the return address. Note that the code for the prologue and the epilogue is identical and it consists of just a single assembly language instruction which performs XOR between return address stored on the stack and the EBP register contents. In addition, the epilogue function will have three additional assembly language instructions for comparing decrypted and un-encrypted return values available on the stack and finally a halt or call to a over flow monitor function assembly language instruction as discussed in Section 4. It is also feasible to implement the EIP encryption/decryption within the CPU hardware architecture itself. But this would require some modifications to the CPU architecture. These modifications will primarily change the semantics of the push EIP and pop EIP instructions such that the micro-operations for these two instructions would require the CPU to first perform the XOR operation between the EIP and the EBP registers, before pushing or popping the EIP from the stack. However, since the compiler based solution described earlier is fairly straight forward, entailing very small performance overhead, making the CPU architecture modification may not be cost effective.

6 Conclusions To deal with the buffer overflow security attacks which overwrite the return address of a function call, this paper describes a simple technique called the StackOF F ence, for defending against such attacks. The proposed technique consists of transforming or encrypting the actual return address and storing the encrypted return address back on the stack. The encryption is carried out by using a key value that has various desirable properties,e.g., invariance, dynamic and simple. For the Intel series of processors, contents of the EBP register are used as the key value. This choice not only satisfies the desired key attributes, the implementation of the proposed technique for the gcc compiler also turns out to be very simple. We also suggest a tech-

nique for capturing the occurrence of such an attack, before the attack does any damage to the system. The proposed technique also make it possible to subsequently perform a forensic analysis of the attack using a suitable kernel debugger. Additional processing overhead required for implementing the proposed schemes appears to be very small.

References [1] Buffer overflow in RSAREF2. http://www.securityfocus.com/ advisories/1892, 1999. [2] B. Brey. The Intel Microprocessors 8086, 80186, 80286, 80386, 80486, Pentium and Pentium Pro Processor Architecture, Programming and Interfacing. Prentice Hall, 2002. [3] Bulb and Kil3r. Bypassing StackGuard and StackShield. Phrack Magazine, 56(5), May 2000. [4] CERT/CC. Cert advisories. http://www.cert.org/advisories/. [5] CERT/CC. Cert advisory ca-2001-19 Code Red Exploiting Buffer Overflow in IIS Indexing Service DLL. http://www.cert.org/advisories/ CA2001-19.html, 2001. [6] M. Corporation. Microsoft Windows NT Workstation Resource Kit, Chapter 39-Windows NT Debugger. Microsoft Press, 1996. [7] C. Cowan, C. Pu, D. Maier, J. Walpole, P. Bakke, S. Beattie, A. Grier, P. Wagle, Q. Zhang, and H. Hinton. Stackguard: Automatic Adaptive Detection and Prevention of BufferOverflow Attacks. In Proc. 7th USENIX Security Conference, pages 63–78, Jan 1998. [8] C. Cowan, P. Wagle, C. Pu, S. Beattie, and J. Walpole. Buffer Overflows: Attacks and Defenses for the Vulnerability of the Decade. In Proc. of the DARPA Information Survivability Conference and Expo, pages 119–129, Jan. 1999. [9] M. Eichin and J. Rochlis. An Analysis of the Internet Virus of November 1988. Proc. IEEE Computer Society Symp. on Security and Privacy, 1989. [10] F. S. Foundation. The GNU C Compiler. http://gcc.gnu.org/. [11] F. S. Foundation. Internals of the GNU compilers. http://gcc.gnu.org/onlinedocs/gccint/. [12] M. Frantzen and M. Shuey. StackGhost: Hardware facilitated stack protection. In Proc. 10th USENIX Security Symposium, Washington, D.C., 2001. [13] S. Jajodia, P. Ammann, and C. McCollum. Surviving information warfare attacks. IEEE Computer, pages 57–63, April 1999. [14] R. Kozlowski. Using Microsoft’s x86 Kernel Debugger. http://www.winnwtmag.com/Article /ArticleID/5131/5131.html, 1999. [15] J. Scambray and M. Shema. Hacking Exposed (TM) Web Applications. McGraw-Hill Osborne Media, 2002. [16] D. A. Solomon. Inside Windows NT, Second Edition. Microsoft Press, 1998. [17] J. Xu, Z. Kalbarczyk, S. Patel, and R. K. Iyer. Architecture Support for Defending Against Buffer Overflow Attacks. In Proc. Dependable Systems and Networks (DSN), pages 163– 170, Aug. 2002.
