Mutation-Based Testing of Buffer Overflow ...

Annual IEEE International Computer Software and Applications Conference

Mutation-based Testing of Buffer Overflow Vulnerabilities Hossain Shahriar and Mohammad Zulkernine School of Computing Queen’s University, Kingston, Ontario, Canada {shahriar, mzulker}@cs.queensu.ca Traditional complementary approaches to test BOF vulnerabilities include source code auditing, static analysis [13], and runtime monitoring [8]. Source code auditing is an expensive and time consuming process. Static analysis suffers from numerous false positive warnings. The runtime monitoring approaches augment executable programs to prevent exploitations. However, BOF vulnerability testing helps fixing implementations early and decreasing losses incurred by the end users. Existing vulnerability testing approaches include mutating user supplied data [14], environment variables [11], and internal program states [15]. These approaches do not address the generation of adequate test data sets for testing BOF vulnerabilities. Here, an adequate test data set implies a collection of test cases that can expose BOF vulnerabilities. Moreover, the idea of source code mutation for BOF vulnerability testing has not been addressed adequately in the literature [18]. In this work, we apply mutation-based testing technique to perform BOF vulnerability testing. Mutation is a fault-based testing technique [4, 10], where the implementation is injected with faults to generate mutants. The rule of injecting fault is known as mutation operator. A mutant is said to be killed or distinguished, if at least one test case can produce different output between the mutant and the implementation. Otherwise, the mutant is live. If no test case can kill a mutant, then it is either equivalent to original implementation or new test case needs to be generated to kill the live mutant. Generating new test cases enhance the test data set. The adequacy of a test data set is measured by mutation score (MS), which is the ratio of the number of killed mutants to the total number of non-equivalent mutants. Similarly, we modify the source code to inject BOF vulnerabilities and force the generation of effective test cases that can expose the vulnerabilities. We apply mutation-based BOF vulnerability testing approach for ANSI C [3] language and its standard libraries [7] mainly for three reasons. First, they are the primary sources of BOF vulnerabilities according

Abstract Buffer overflow (BOF) is one of the major vulnerabilities that leads to non-secure software. Testing an implementation for BOF vulnerabilities is challenging as the underlying reasons of buffer overflow vary widely. Moreover, the existing vulnerability testing approaches do not address the issue of generating adequate test data sets for testing BOF vulnerabilities. In this work, we apply the idea of mutation-based testing technique to generate adequate test data set for BOF vulnerabilities. Our work addresses those BOF vulnerabilities, which are related to an implementation language and its associated libraries. We apply the concept for ANSI C language and its associated libraries. We propose 12 mutation operators to force the generation of adequate test data set for BOF vulnerabilities. The proposed operators are validated by using four open source programs. The results indicate that the proposed operators are effective for testing BOF vulnerabilities.

1. Introduction Buffer overflow (BOF) is one of the worst vulnerabilities [6]. The BOF vulnerabilities imply specific flaws in a piece of software, allowing attackers to overflow data buffers that might lead to the corruption of neighboring variables. If neighboring variables are sensitive (e.g., return address or frame pointer of a function), then BOF attacks might lead to program crashes (or segmentation violations) while trying to write illegal memory locations [19]. Therefore, testing software implementations for BOF vulnerabilities is important. However, it is challenging as the causes of BOF vulnerabilities include the limitations of implementation languages and their associated libraries (e.g., ANSI C standard libraries), logical errors in implementations (e.g., off by one in mod_rewrite.c file of apache-1.3.3) etc.

0730-3157/08 $25.00 © 2008 IEEE DOI 10.1109/COMPSAC.2008.123

979

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:29 from IEEE Xplore. Restrictions apply.

to vulnerability databases [16, 17]. Second, even though BOF vulnerabilities are known for many years for ANSI C and its libraries, they are widely used for developing many critical software such as ftp server (e.g., wu-ftpd), web server (e.g., apache) etc. Third, the existing mutation operators for ANSI C [1, 9, 12] are not designed for testing BOF vulnerabilities. We propose 12 mutation operators to support testing of BOF vulnerabilities along with three distinguishing criteria to kill the mutants. We demonstrate the effectiveness of the operators with four open source programs. Our experiments indicate that the operators are effective for BOF vulnerability testing. The paper is organized as follows: Section 2 describes the related work. In Section 3, the proposed operators along with mutants distinguishing criteria are described. Section 4 describes the evaluation of the operators. Section 5 draws the conclusion and discusses limitations and future work.

These data are sent back to the server, and a tester observes application responses (i.e., whether the protocol daemon crashes due to segmentation fault or not). In contrary, our work addresses BOF vulnerability testing by mutating source code. Ghosh et al. [15] mutate the internal states of program to detect vulnerabilities during runtime. They develop Fault Injection Security Tool (FIST) which injects various types of fault such as corruption of string variables, overwriting the return address of stack etc. In contrast, our work does not corrupt the return address directly to emulate BOF attack, rather our method forces the generation of test cases to expose BOF vulnerabilities. Du et al. [11] perform vulnerability testing of applications by perturbing environment variables during runtime. They consider vulnerabilities from indirect environment (e.g., programs using environment variables during initialization process), and direct environment (e.g., file system inputs, network packets etc.). In contrast, we address the generation of adequate test data set to test BOF vulnerabilities by mutating the source code of implementations. Agrawal et al. [10] propose a comprehensive set of mutation operators for ANSI C language which are applicable for program variables, constants, statements, and operators. However, their proposed operators are not designed to inject BOF vulnerabilities in source code. Nevertheless, we find that some of their operators are equivalent to our proposed operators. For example, SSDL operator removes each statement from the original implementation sequentially. In contrast, we propose to remove only those statements, which assign null characters at the end of buffers. Delamaro et al. [5] propose mutation operators for testing interfaces of C programs, which inject faults (i) inside the called functions and (ii) at the point of function calls (or interfaces). Some operators of the second group are similar to our proposed operators. For example, we propose to mutate the buffer size arguments of several ANSI C standard function calls (e.g., modifying the buffer size argument of strncpy function to emulate BOF vulnerabilities). However, we mutate function arguments to allow BOF vulnerabilities, whereas their approach replaces buffer sizes with all similar types of variables declared either globally or locally in the same program. Csaw is the most recent mutation-based testing tool for C developed by Ellims et al. [12]. The tool can distinguish mutants from an implementation based on CPU time usage differences, program crashes due to divide by zero etc. Csaw implements seven types of operators that include mutating operators and

2. Related work We first discuss the mutation-based testing of BOF vulnerabilities followed by some mutation-based testing works for C programs that motivated our work. Table 1 shows a summary of the related work in comparison to our work. Table 1. Summary of related works Tool name / Work Language Vilela et al. [18] ANSI C Tal et al. [14] PDU FIST [15] ANSI C Du et al. [11] Agrawal et al. [10] ANSI C Delamaro et al. [5] ANSI C Ellims et al. [12] ANSI C Our work ANSI C

BOF testing Adequate testing of BOF Yes Yes Yes No Yes No Yes No No No No No No No Yes Yes

Mutation-based testing has not been utilized to test BOF vulnerabilities through the generation of an adequate test data set except the work of Vilela et al. [18]. They propose two mutation operators which modifies static memory allocations (MSMA) and dynamic memory allocations (MDMA). Each operator replaces the allocated buffer size N with 1, N-1, N/2, and N*2 respectively. However, their approach does not consider BOF vulnerabilities related to the ANSI standard library functions (e.g., strcpy function does not check the destination buffer before copying) and other language specific features (e.g., absence of null character at the end of buffer) that might lead to BOF vulnerabilities. Tal et al. [14] perform vulnerability testing of framebased network protocol implementation by mutating protocol data units captured from client machines.

980

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:29 from IEEE Xplore. Restrictions apply.

variables, substituting constants (i.e., substituting each text with all the text that are not keywords) etc. However, these operators are not intended for testing BOF vulnerabilities.

respectively. N represents the allocation size of Buf. The valid locations for reading and writing the buffer are between Buf[0] and Buf[N-1]. Therefore, the neighboring locations of Buf start from Buf[N]. A test case kills M based on C1, when P does not crash and M does. We take advantage of the fact that the exit status of a crashed program is different than that of a program having normal termination. C2 distinguishes P from M, if a test case modifies the location of Buf[N] in M (i.e., overwriting the neighboring variable of the buffer by one byte in M). However, the location of Buf[N] in P remains unchanged. C3 distinguishes P from M, if the length of Buf in M exceeds the allocated size. However, the length of Buf in does not exceed in the original program P.

3. Proposed Operators We propose 12 operators that are divided into five categories. They are mutating ANSI C standard library functions, modifying buffer size arguments in ANSI C standard library functions, mutating format strings, increasing buffer variable sizes, and removing null character assignment statements. The first three categories consider the inherent vulnerabilities of ANSI C standard library functions, whereas the remaining two categories consider the limitations of the programming language. Table 2 summarizes the proposed operators that are described in Section 3.1 to 3.5.

Table 3. Mutants killing criteria Name Killing criteria C1 ESP ≠ ESM C2 BufP [N] ≠ BufM [N] C3 Len(BufP) < N && Len(BufM) > N P: The original implementation. M: The mutant version. ESP: The exit status of P. ESM: The exit status of M. N: Buffer size. BufP[N]: The Nth element of buffer Buf in P. BufM[N]: The Nth element of buffer Buf in M. Len(BufP): Length of Buf in P. Len(BufM): Length of Buf in M.

Table 2. Proposed mutation operators Category Mutating library functions

Mutating buffer size arguments

Operator

Brief description

S2UCP S2UCT S2UGT S2USN S2UVS RSSBO

Replace strncpy with strcpy. Replace strncat with strcat. Replace fgets with gets. Replace snprintf with sprintf. Replace vsnprintf with vsprintf. Replace buffer size with destination buffer size plus one. Replace “%ns” format string with “%s”. Replace “%ns” format string with “%ms”, where, m = size of the destination buffer plus one. Replace “%ns” format string with “%ms”, where, m is the size of the destination buffer plus ∆. Replace “%s” format string with “%ns”, where n is the size of the destination buffer. Increase buffer size by one byte.

RFSNS RFSBO Mutating format strings

RFSBD RFSIFS

Mutating buffer variable sizes Removing statements

MBSBO

RMNLS Remove null character assignment statement.

Killing criteria

3.1. Mutating library functions The proposed operators of this category replace safe ANSI library function calls with unsafe function calls. The safe functions check the size of the destination buffers before performing operations such as copy and concatenation. The unsafe functions do not check the lengths of destination buffers. There are five operators in this group: S2UCP, S2UCT, S2UGT, S2USN, and S2UVS. They replace strncpy, strncat, fgets, snprintf, and vsnprintf with strcpy, strcat, gets, sprintf, and vprintf respectively. The mutants are killed with test cases that satisfy either C1 or C2 criteria. Table 4 shows an example application of S2UCP operator.

C1 or C2

C1 or C3

We observe that writing beyond a buffer might crash a program, which makes it difficult to distinguish mutants from the original program by comparing the end output. It may also change the internal programs states without crashing immediately (e.g., one byte overflow). Moreover, a buffer having no null character at the end might lead a program to read from its neighboring locations. This might cause different output or program crashes. Therefore, we define three killing criteria for the mutants as shown in Table 3. Let us consider that P is the original implementation unit, and M is a mutant of P. ESP and ESM indicate the exit status of P and M respectively. BufP and BufM represent the buffer variable (Buf) in P and M

Table 4. Application of S2UCP operators Original statement Strncpy (dest, src, n);

Mutated statement strcpy (dest, src); //∆S2UCP

3.2. Mutating buffer size arguments The RSSBO operator modifies buffer size argument of safe ANSI C library functions such as strncpy, strncat, memcpy, memset, memmove, snprintf, fgets, and vsnprintf. The size is set to the length of destination buffer plus one. The mutants are killed with test cases satisfying either C1 or C2 criteria. Table 5 shows an application of RSSBO operator.

981

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:29 from IEEE Xplore. Restrictions apply.

mutants are killed with effective test cases that meet either C1 or C2 criteria. Table 8 shows an example application of the operator.

Table 5. Application of RSSBO operator Original statement char dest [32]; … strncpy (dest, src, 32);

Mutated statement char dest [32]; … strncpy (dest, src, 33); //∆ RSSBO

Table 8. Application of MBSBO operator Original statement char dest [32];

3.3 Mutating format strings

3.5. Removing statements

There are four operators in this category: RFSNS, RFSBO, RFSBD, and RFSIFS. The RFSNS, RFSBO, and RFSBD operators mutate format strings of scanf and printf family functions to emulate BOF vulnerabilities in destination buffers, which are passed as arguments. The RFSNS operator replaces “%xs” format specifiers with “%s”, where x represents the maximum number of bytes to be read from a source buffer. The RFSBO and RFSBD operators change the value of x with length(dest)+1 and length(dest)+∆ respectively. Here, length(dest) represents the length of the destination buffer. The ∆ value is the offset (in bytes) between the buffer starting location and the function’s return address in the stack.

The RMNLS operator removes statements that assign null character at the end of buffer. Since null character is used to mark the end of a buffer, removal of the statement allows reading to contiguous memory location beyond the buffer size. The mutants are killed by either C1 or C3 criteria. Table 9 shows an example application of the operator. Table 9. Application of RMNLS operator Original statement char dest [32]; … dest [i] = ‘\0’;

Table 6. Application of RFSNS, RFSBO, and RFSBD operators

Mutated statement char dest [32]; … //∆ RMNLS

4. Evaluation of the proposed operators

Original statement Mutated statement char dest [32]; char dest [32]; sscanf (src, “%32s”, dest); sscanf (src, “%s”, dest); //∆ RFSNS sscanf (src, “%33s”, dest); //∆RFSBO sscanf (src, “%40s”, dest); //∆RFSBD

We evaluate the effectiveness of the proposed operators with four open source programs. They have been reported to have BOF vulnerabilities in Open Source Vulnerabilities Database (OSVBD) [16] and Common Vulnerabilities and Exposures (CVE) [17] database. Table 10 shows different characteristics of the selected applications. The wu-ftpd, Edbrowse, and Rhapsody suffer from BOF vulnerabilities in SockPrintf, ftpls, and parse_input functions respectively. For each of the above three functions, the vulnerable buffers are located in the stack region. The cmdftp program allocates dynamic buffer inside store_line function, which affects the heap area. We call the vulnerable functions as bad programs and their corresponding fixed version as good programs. A fixed version is either a patch (Wu-ftpd and Rhapsody) or an upgraded version (Edbrowse and Cmdftp). We initially generate mutants for both good and bad programs. Table 11 shows the summary of mutation generation process. We find that the number of mutants generated for a good program is greater than that for a bad program. The operators are designed in such a way so that more mutants are generated for safe ANSI library functions than unsafe functions. For each of the good programs, we obtain an adequate test data set T that kills all the generated mutants. The T set is generated by the following two

Table 7. Application of RFSIFS operator Original statement char dest [32]; sscanf (src, “%s”, dest);

Mutated statement char dest [33]; //∆ MBSBO

Mutated statement char dest [32]; sscanf (src, “%33s”, dest); //∆ RFSIFS

Table 6 shows example applications of the three operators for sscanf function. The RFSNS operator replaces the format string “%32s” with “%s”. The RFSBO operator replaces the “%32s” with “%33s”. The RSSBD operator replaces the “%32s” with “%40s” as the offset from dest buffer to the last byte of return address is 8 (frame pointer and return address occupies 4 bytes) assuming no other variables are declared before dest. The size of the dest buffer is 32. Therefore, the ∆ value is 40. The RFSIFS operator replaces string specifiers “%s” with “%xs”, where the value of x is length(dest)+1. Table 7 shows an application of RFSIFS operator. The mutants are killed by test cases that satisfy either C1 or C2 criteria.

3.4. Mutating buffer variable sizes The MBSBO operator increases buffer variable sizes by one byte during buffer declarations. The generated

982

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:29 from IEEE Xplore. Restrictions apply.

steps: (i) Generate an initial test data set containing 10 test cases which are chosen randomly from a large test data pool (ii) If MS of the test data set does not reach 100%, then analyze the live mutants to generate additional test cases to kill them. Here, MS is the mutation score, which is the ratio of the total number of mutants killed to the total number of non-equivalent mutants. The large test pool is created using the Fuzz tool [2], which generates strings of random characters with variable lengths. Step (i) and (ii) are repeated 15 times to reduce any bias for analysis. We compute the average MS and average test data set sizes. Table 12 shows the summary of the mutation analysis for good programs. We notice that it is not possible to reach 100% MS. For each of the programs, we examine the live mutants manually and observe that the injected BOF vulnerabilities cannot be exposed with any test cases. Moreover, all the live mutants are generated from MBSBO operator.

bytes (Line 3-6). Line 7 places a null character at the end of buf to prevent BOF. The content of buf is written in the file variable sockfp at Line 8 through the SockWrite function call. Table 12: Summary of mutation analysis Name Wu-ftpd 2.6.2 Edbrowse-3.3.1 Rhapsody IRC 0.28b Cmdftp-0.9.5

Program

Stack

Wu-ftpd 2.6.2 Edbrowse2.2.10

Stack

App. type

Stack

Rhapsody IRC 0.28b

Heap

Cmdftp0.64

Bug ID

Source file, function name Ftp server CVE-2003ftpd.c, 1327 SockPrintf Command CVE- 2006- http.c, ftpls line editor 6909 browser main.c, Text-based CVE-20071502 parse_input IRC client console for Unix Command OSVDBcmdftp.c, line FTP 2522 store_line client

LOC 8 70

Table 13: Code snippet of SockPrintf

220

1. int SockPrintf (FILE *sockfp, char *format, ...){ 2. va_list ap; 3. char buf [32768]; 4. va_start (ap, format); 5. vsnprintf (buf, sizeof(buf), format, ap); 6. va_end (ap); 7. buf [sizeof(buf)] = '\0'; 8. return SockWrite (buf, 1, strlen(buf), sockfp); }

7

Table 11: Mutation generation summary Operator

Wu-ftpd Good Bad

Total

4

1

Edbrowse

Rhapsody

Good Bad Good 8

6

36

Cmdftp

Bad

Good

Bad

19

5

1

Avg. test data set size 12.53 12.86 20.40 12.93

Table 14 shows the four mutants generated by the operator named MBSBO, S2UVS, RSSBO, and RMNLS along with corresponding line and mutated statements. An initial test data set of size 10 is created by choosing 10 test cases from the test pool randomly. Each test case has at least three elements of the form . Here, t1, t2, and t3 represent a file pointer, a format string, and string input respectively. We set t2 as “%s” and keep the value of both t1 and t2 same for all test cases. Each of the t3 is string type data and has a length between one and 32768 bytes. The resultant test data set does not kill any of the four mutants and the MS is zero.

Table 10: Characteristics of bad programs Location

Avg. MS (%) 75 75 75 60

Table 14: Generated mutants of SockPrintf Line 3 5 5 7

We use the bad programs for generating mutants. For each of the bad programs, we apply the adequate test data sets obtained previously to kill all the mutants. We find that at least one test case of T is able to kill the mutants generated from a bad program. We observe similar findings for all the 15 test data sets generated for each of the programs. Therefore, our proposed operators help in obtaining adequate test data set capable of revealing BOF vulnerabilities. To illustrate how the proposed operators force the generation of adequate test data set for BOF vulnerabilities, we show the case of Wu-ftpd program. The code snippet of SockPrintf function (good version) is shown in Table 13. In general, the function receives format string (format) and variable arguments (by ‘…’ at Line 1), which are written in a locally declared buffer named buf having length of 32,768

Operator MBSBO S2UVS RSSBO RMNLS

Mutated statement char buf [32769]; vsprintf (buf, format, ap); vsnprintf (buf, 32769, format, ap); //statement removed

The test data set is enhanced with a new test case having t3 of length 32,769, while keeping t1 and t2 same as before. The test case will kill the mutants generated by RSSBO and RMNLS operator. Then a new test case having t3 of length 32,777 is added, which kills the mutant generated by S2UVS. The remaining live mutant generated by MBSBO cannot be killed with any test case as the rest of the code does not allow BOF. Therefore, the size of adequate test data set is 12. If we generate mutants for the bad version of SockPrintf and apply the adequate test data set, we find that all the mutants are killed.

983

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:29 from IEEE Xplore. Restrictions apply.

[5] Delamaro, M., Maldonado, J., Mathur, A., “Integration testing using interface mutations”, Proc. of the Seventh International Symposium on Software Reliability Engineering, IEEE Computer Society Press, White Plains, New York, October, 1996, pp. 112-121. [6] Common Weakness Enumeration (CWE), Vulnerabilities Type Distributions in CVE, Version 1.1 in http://cwe.mitre.org/documents/vuln-trends/index.html (Accessed Jan 2008) [7] C Standard Library in http://www.utas.edu.au/infosys/ info/documentation/C/CStdLib.html (Accessed Jan 2008) [8] Cowan, C., Pu, C., Maier, D., Walpole, J., Bakke, P., Beattie, S., Grier, A., Wagle, P., Zhang, Q., Hinton, H., “Stackguard: Automatic adaptive detection and prevention of buffer-overflow attacks”, In Pro. of the 7th USENIX Security Conference, Texas, Jan 1998, pp 63–78. [9] Delamaro, M.E., Maldonado, J.C., “Proteum - A Tool for the Assessment of Test Adequacy for C Programs”, Conf. on Performability in Computing Systems (PCS 96), July 1996, pp. 79–95. [10] DeMillo, R. A., Lipton, R. J., Sayward, F. G., “Hints on test data selection: Help for the practicing programmer”, IEEE Computer Magazine, Volume 11, Issue 4, 1978, pp. 34-41. [11] Du, W., Mathur, A., “Testing for software vulnerabilities using environment perturbation”, International conf. on Dependable Systems and Networks (DSN 2000), New York, Jun 2000, pp. 603-612. [12] Ellims, M., Ince, D.C., Petre, M., “The Csaw C Mutation Tool: Initial Results”, Third Workshop on Mutation Analysis (Mutation 2007), Windsor, UK, Sept 2007, pp. 185-192. [13] FlawFinder, http://www.dwheeler.com/flawfinder [14] Tal, O., Knight, S., Dean, T.R., "Syntax-based Vulnerabilities Testing of Frame-based Network Protocols", Proc. of the 2nd Annual Conference on Privacy, Security and Trust, Fredericton, Canada, Oct 2004, pp. 155-160. [15] Ghosh, A. K., O'Connor, T., McGraw, G., “An automated approach for identifying potential vulnerabilities in software”, IEEE Symposium on Security and Privacy, Los Alamitos, CA, USA, 1998, pp. 104-14. [16] Open Source Vulnerability Database (OSVDB) in http://osvdb.org (Accessed Jan 2008) [17] Common Vulnerabilities and Exposure (CVE) in http://cve.mitre.org (Accessed Jan 2008) [18] Vilela, P., Machado, M., Wong, E., “Testing for Security Vulnerabilities in Software”, Proceeding Software Engineering and Applications (SEA 2002), Cambridge, USA, November 2002. [19] Aleph One, “Smashing the Stack for Fun and Profit”, Phrack Magazine, Volume 7, Issue 49, Nov 1996. http://www.phrack.org/archives/49/P49-14.

5. Conclusions This work proposes mutation-based testing of BOF vulnerabilities that helps generating an adequate test data set, which has not been addressed properly in the area of vulnerability testing. We propose 12 mutation operators to support the mutation-based testing process. The operators inject BOF vulnerabilities in ANSI C standard libraries and programming elements such as buffer variables and statements. The mutants generated by the proposed operators can be killed in the presence of effective test cases exposing the vulnerabilities. The operators are found to be effective for four open source applications having known BOF vulnerabilities. The programs are chosen in such a way so that at least one argument is responsible for BOF attack. However, BOF vulnerabilities could be more complicated than those of the chosen programs. Our approach does address BOF due to pointers and aliases due to lack of runtime memory information during mutation generation process. Moreover, we consider the vulnerabilities inherent in ANSI C standard library functions as a fault model, which might be contradictory to other mutation-based testing works that use language syntaxes. However, these libraries are an inevitable part of many critical applications, and they are extensively used by programmers even though their limitations are known to the software engineering community. Our future work includes addressing other vulnerabilities and experimenting with large scale programs. Acknowledgement This work is partially supported by the National Sciences and Engineering Research Council (NSERC) of Canada.

6. References [1] Agrawal, H., DeMillo, R.A., Hataway, R., Hsu, W., Hsu, W., Krauser, E., Martin, R.J., Mathur, A.P., Spafford, E.H., “Design of Mutant Operators for C Programming Language”, Technical Report SERC-TR41-P, Revision 1.04, Software Eng. Research Center, Purdue University. [2] Miller, B., Fredriksen, L., and So, B. "An Empirical Study of the Reliability of UNIX Utilities", Communications of the ACM, Volume 33, Number 12, Dec 1990, pp. 32-44. [3] American National Standard for Information Systems Programming Language C, Technical Report ANSI X3.1591989, ANSI Inc., New York, USA 1990. [4] Budd, T., Lipton, R., DeMillo, R., Sayward, F., "Theoretical and Empirical Studies on Using Program Mutation to Test the Functional Correctness of Programs", Proc. of the Seventh Conf. on Principles of Programming Languages, Las Vegas, Jan 1980, pp. 220-233.

984

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:29 from IEEE Xplore. Restrictions apply.