Mutation-based SQL Injection Vulnerability Checking - CiteSeerX

The Eighth International Conference on Quality Software

MUSIC: Mutation-based SQL Injection Vulnerability Checking Hossain Shahriar and Mohammad Zulkernine School of Computing Queen’s University, Kingston, Ontario, Canada {shahriar, mzulker}@cs.queensu.ca said to have SQL injection vulnerabilities (SQLIV), when SQL queries are generated using an implementation language (e.g., Java Server Pages or JSP) and user supplied inputs become part of the query generation process without proper validation. As a result, the execution of these queries might cause unexpected results such as authentication bypassing, leaking of private information etc. SQLIV rank among the top three vulnerabilities over the past few years [1]. Moreover, successful exploitations of SQLIV have already caused significant financial loss to organizations [8]. Therefore, testing an application for SQLIV is important for ensuring software quality. In recent years, a number of techniques have been proposed to address SQLIV other than testing. These include input character filtering or input validation [12], static analysis [4], runtime monitoring [2], combination of static analysis and runtime monitoring [6], automatic fixing of source code [9], application level intrusion detection [11], randomization of SQL keywords [3] etc. However, an effective testing technique can ensure that applications are free from SQLIV and reduce related cost of performing static analysis, monitoring or post deployment of other defensive mechanisms. Unfortunately, very few works [5, 7] address the issues of testing applications for SQLIV. However, these approaches do not address the issue of assessing test data quality to detect SQLIV. Furthermore, classic white box testing techniques [14, 15] related to SQL oriented applications are not designed to test SQLIV. In this work, we apply the concept of mutationbased testing technique [17] for testing SQLIV. Mutation is a fault-based testing technique, where a program is injected with faults by applying syntactic changes in the source code (also known as mutants). The rules of injecting faults are known as mutation operators. A test case kills a mutant, if it causes either different end output (i.e., strong mutation testing [17]) or different intermediate state (i.e., weak mutation testing [18]) between the original program and a mutant. Otherwise, the mutant is said to be live. Additional test cases need to be generated to kill the

Abstract SQL injection is one of the most prominent vulnerabilities for web-based applications. Exploitation of SQL injection vulnerabilities (SQLIV) through successful attacks might result in severe consequences such as authentication bypassing, leaking of private information etc. Therefore, testing an application for SQLIV is an important step for ensuring its quality. However, it is challenging as the sources of SQLIV vary widely, which include the lack of effective input filters in applications, insecure coding by programmers, inappropriate usage of APIs for manipulating databases etc. Moreover, existing testing approaches do not address the issue of generating adequate test data sets that can detect SQLIV. In this work, we present a mutation-based testing approach for SQLIV testing. We propose nine mutation operators that inject SQLIV in application source code. The operators result in mutants, which can be killed only with test data containing SQL injection attacks. By this approach, we force the generation of an adequate test data set containing effective test cases capable of revealing SQLIV. We implement a MUtation-based SQL Injection vulnerabilities Checking (testing) tool (MUSIC) that automatically generates mutants for the applications written in Java Server Pages (JSP) and performs mutation analysis. We validate the proposed operators with five open source web-based applications written in JSP. We show that the proposed operators are effective for testing SQLIV.

1. Introduction Today’s organizations are highly dependent on applications that heavily interact with databases by generating and issuing queries to database engines to perform necessary tasks. These applications often contain vulnerabilities that may be exploited through SQL injection attacks (SQLIAs). An application is

1550-6002/08 $25.00 © 2008 IEEE DOI 10.1109/QSIC.2008.33

77

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:30 from IEEE Xplore. Restrictions apply.

is correct. Therefore, the attacker will be able to skip the authentication after the query is executed and assume the identity of the first user of the table members. It should be noted that tautology is not the only way of performing SQLIAs. Orso et al. ([19]) classify SQLIAs into seven categories. They are tautologies, union queries, illegal/logical incorrect queries, piggybacked queries, stored procedures, inference attacks, and alternate encodings (or Hex encoded queries). This work addresses SQLIAs of the above types except stored procedures and several inference attacks.

live mutants. In this way, the fault detection capability of a test data set increases. We propose nine mutation operators to inject SQLIV in the source code of applications. The first four operators inject faults in the generated SQL queries. The remaining five operators inject faults into the API method calls that are used to connect and manipulate database tables. The mutants generated based on the proposed operators can be killed by effective test cases having SQLIAs. We implement a prototype MUtation-based SQL Injection vulnerabilities Checking (testing) tool (MUSIC), which automatically generates mutants for the applications written in JSP and performs mutation analysis. We validate the proposed operators with five open source web-based applications, which are implemented in JSP. The proposed operators are found to be effective for detecting SQLIV and generating adequate test data set. The paper is organized as follows. Section 2 presents the related work. In Section 3, the proposed operators along with mutants killing criteria are described. Section 4 describes the implementation of MUSIC and experimental evaluation. Section 5 draws the conclusions and discusses the limitations and future work.

1. String LoginAction (HttpServletRequest request, ...) throws IOException { 2. String sLogin = getParam (request, "Login"); 3. String sPassword = getParam (request, "Password"); 4. java.sql.ResultSet rs = null; 5. String qry = "select member_id, member_level from members where "; 6. qry = qry + “member_login =’” + sLogin + “’ and member_password =’” + sPassword + “’”; 7. java.sql.ResultSet rs = stat.executeQuery (qry); 8. if (rs.next ()) { // Login and password passed 9. session.setAttribute ("UserID", rs.getString(1)); … }

Figure 2: JSP code snippet for authentication

2.2 Related work Table 3 shows a brief summary of prominent related works on the detection and testing of SQLIAs [2, 3, 4, 5, 6, 7, 9, 11]. It also includes some of the white box testing works [14, 15] that propose mutation-based testing applications having SQL queries.

2. SQL injection attacks and related work 2.1 SQL injection attacks We provide an example of an SQLIA by using the code snippet of server side application written in JSP as shown in Figure 2. Line 2 and 3 extract user supplied information from Login and Password field into sLogin and sPassword variables, respectively. The user input is not filtered and a dynamic SQL query is generated in Line 5 and 6. Let us assume that a user provides valid member_login and member_password, which are “guest” and “secret”, respectively. Then, the generated query at Line 6 becomes “select member_id, member_level from members where member_login =’guest’ and member_password = ’secret’”. The database engine executes the query at Line 7 and the user is authenticated with valid UserID at Line 9. A malicious user might supply input “’ or 1=1 -- ” in the first field and leave the second input field as blank. The resultant query will be “select member_id, member_level from members where member_login =’’ or 1=1 --’ and member_password =’’”. The resultant query is a tautology as the query portion after the symbol “--” will be ignored by the database engine (as it is comment symbol). The syntax of the altered query

Table 3: Summary of related work Works

Brief summary

SQLUnitGen [7] Unit testing of web applications against SQLIAs. Sania [5] Debugging and testing framework for SQLIAs using static analysis. SQLGuard [2] Runtime monitoring of SQLIAs. SQLrand [3] Randomizing SQL keywords to thwart SQLIAs. SQLCHECK [4] Parsing augmented SQL grammar to recognize syntactic deviation due to SQLIAs. AMNESIA [6] SQLIAs prevention by generating valid query models with static analysis and runtime monitoring. Thomas et al. [9] Retrofitting SQL statements with PerparedStatements in Java. Lin et al. [11] Deployment of an application gateway to filter SQLIAs. SQL-IDS [13] Specification-based intrusion detection system for SQLIAs. Chan et al. [14] Testing of database applications with mutation operators based on a conceptual data model. SQLMutation Mutation testing of SQL SELECT [15] type queries. MUSIC Mutation of SQL queries and database API method calls for adequate testing of SQLIV.

78

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:30 from IEEE Xplore. Restrictions apply.

Mutation SQLIV testing? testing? No Yes Yes

No

No No

No No

No

No

No

No

No

No

No

No

No

No

Yes

No

Yes

No

Yes

Yes

against the query model. If a generated query is not consumed by NDFA, then it is an attack. Thomas et al. [9] show an automatic fixing technique of Java applications that are vulnerable to SQLIAs. The basic idea is to convert a SQL statement into a PreparedStatement in Java that does not allow the semantic modification of queries during runtime. The solution only addresses Java specific implementations and might not be suitable for the applications implemented in other languages that have no support for PreparedStatement. Lin et al. [11] propose an application-level security gateway to prevent SQLIAs. All the possible entry points of an application are identified followed by employing meta-programs to filter out SQLIAs. However, it is a reactive approach, and effective testing can reduce the cost of such approach. Kemalis et al. [13] develop a specification-based SQL injection attack detection system (SQL-IDS) that uses information from software security specification. The intended SQL commands are specified using Extended Backus Naur Form (EBNF). An application is embedded with IDS architecture. The IDS monitors for SQLIAs through mismatches between actual query tokens (e.g., SQL keywords, column names) with EBNF specifications. However, the approach suffers from the limitation of correctly specifying all desired queries before the application is implemented. Chan et al. [14] apply fault-based testing of database applications. They propose seven mutation operators, which represent faults of entity relationship model of database driven applications (they named it as conceptual data model). The operators modify the cardinality of queries (e.g., replace “select count(column1)” with “select count(column2)”), replace attributes with similar types (e.g., change one column name with the others of similar data types), replace participation constraints (e.g., replace EXIST with NOT EXIST) etc. The approach is strong mutation-based testing. In contrast, we apply weak mutation-based testing of SQLIV by injecting faults in both the SQL queries and database API function calls of an application. Tuya et al. [15] develop SQLMutation tool that implements four categories of mutation operators for adequate testing of SQL SELECT queries. These include SQL clause operators (e.g., replacing SELECT with SELECT DISTINCT), operator replacement mutation operators (e.g., AND is replaced by OR), NULL mutation operators (e.g., replacing NULL with NOT NULL), and identifier replacement (e.g., replacing one column name with other of similar types). In contrast, our proposed operators test SQLIV and applicable for SELECT, UPDATE, DELETE, and INSERT type queries. Their approach is based on

Shin et al. [7] develop SQLUnitGen tool that combines static analysis with unit testing to detect the effectiveness of SQL injection filter in an application. Static analysis is used to track user inputs to the point of query generation. Initial test cases are generated by Jcrasher [16] tool. The test cases are modified so that they contain SQLIAs. Kosuga et al. [5] propose SQLIAs testing framework named Sania for application development and debugging phase. Their approach initially constructs parse trees of intended SQL queries written by developers. Terminal leafs of parse trees typically represent vulnerable spots, which are filled by possible attack strings. The difference between the initial parse tree and the modified parse tree generated from user supplied attack string results in warnings of SQLIAs. Both of the approaches do not inject SQLIV into source code and address the generation of adequate test data set for testing SQLIV. Buehrer et al. [2] develop SQLGuard tool that detects SQLIAs during application runtime by comparing the parse tree of an intended SQL query before and after the inclusion of user supplied input. However, the approach is ineffective, if the user supplied input does not appear at the leaf of the tree. In contrast, our approach does not rely on the generation of parse tree. Boyd et al. [3] propose the randomization of SQL keywords (SQLrand tool) to thwart injection attacks that contain SQL keywords. The main idea is that the injected SQL keywords from attackers do not have random numbers appended. Therefore, they are not interpreted as regular keywords. Their approach can be fooled by performing piggybacked query attacks, where additional statements have no SQL keywords. In contrast, our approach works for attacks having no SQL keywords. Su et al. [4] present grammar-based approach to detect and stop queries having SQLIAs by implementing SQLCHECK tool. They mark user supplied portions in queries with a special symbol and augment the standard SQL grammar with production rule. A parser is generated based on the augmented grammar. The parser successfully parses the generated query at runtime, if there are no SQLIAs in the generated queries after adding user inputs. In contrast, our approach is based on modifying intended queries and API method calls to inject SQLIV. Halfond et al. [6] develop AMNESIA (Analysis for Monitoring and Neutralizing SQL Injection Attack) tool to detect and prevent SQLIAs. The tool first identifies hotspots, where SQL queries are issued to database engines. At each hotspot, a query model is developed by using non deterministic finite automata (NDFA). The hotspot is instrumented with monitor code, which matches the dynamically generated query

79

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:30 from IEEE Xplore. Restrictions apply.

s2 represents the opposite state of s1 (i.e., M runs successfully and I generates syntax error message). The criterion C3 is used to distinguish between I and M based on the observation that s1 ≠ s2. Let the execution of I and M results in i1 and i2 number of record insertions; d1 and d2 number of record deletions; u1 and u2 number of records updates; o1 and o2 number of new object (e.g., tables, views etc.) creations. A test cases kill mutants, if any of the four conditions are satisfied: (i) i1 ≠ i2 (ii) d1 ≠ d2 (iii) u1 ≠ u2, and (iv) o1 ≠ o2. We denote the killing criterion as C4.

simple comparison of the end output generated by original and mutated queries. However, we distinguish mutants based on different intermediate database states and result sets returned by queries.

3. Proposed mutation operators We propose nine mutation operators divided into two categories. The first category consists of four operators that inject faults into where conditions (WC) of SQL queries. The second category consists of five operators that inject faults in database API method calls (AMC). A summary of the proposed mutation operators is provided in Table 4. Before discussing the operators (Section 3.2), we first discuss the proposed killing criteria for mutants in the following subsection.

Table 5: Mutant killing criteria Name Distinguishing criteria C0 (|n1 ∩ n2| = 0) || (|n1 ∩ n2| = N) || (|n1 U n2| > N) C1 |n1 ∩ n2| ≠ 0 C2 |n1 ∩ n2| > 0 C3 s1 ≠ s2 C4 (i1 ≠ i2) || (d1 ≠ d2) || (u1 ≠ u2) || (o1 ≠ o2) C5 p1 ≠ p2 C6 |n2| > |n1| C7 (t1 > T && t2 < T) || (t1 < T && t2 > T) I: Intended query; M: Mutated query. n1: The record set selected by I. n2: The record set selected by M. s1: State where I runs successfully and M generates error message. s2: State where M runs successfully and I generates error message. i1: # of records inserted by I. i2: # of records inserted by M. d1: # of records deleted by I. d2: # of records deleted by M. u1: # of records updated by I. u2: # of records updated by M. o1: # of database objects created by I. o2: # of database objects created by M. p1: # of external objects created by database by I. p2: # of external objects created by database by M. t1: Time elapsed to execute I. t2: Time elapsed to execute M. T: Default query execution timeout of application.

Table 4: The proposed operators Cat. Operators RMWH WC

NEGC FADP UNPR MQFT

AMC

OVCR SMRZ SQDZ OVEP

Description

Killing criteria C0

Remove WHERE keywords and conditions. Negate each of the unit expression C1 inside where conditions. Add parentheses in where conditions C2 and prepend “FALSE AND” after the WHERE keyword. Unbalance parentheses of where C3 condition expressions. Set multiple query execution flags to C4 || C5 || C6 true. Override commit and rollback C4 options. Set the maximum number of record C6 returned by a result set to infinite. Set query execution delay to infinite. C7 Override the escape character C5 processing flags.

Let after execution of I and M, the number of external objects created (e.g., files) outside a database are p1 and p2, respectively. The C5 criterion distinguishes between I and M, if p1 ≠ p2. The C6 criterion distinguishes between I and M, if the number of records selected in I is less than that of M. Let the time elapsed to execute I and M are t1 and t2, respectively. The default timeout set by an application is T. The C7 criterion distinguishes between I and M, if either t1 or t2 exceeds T, but not both.

3.1 Mutant killing criteria We propose eight distinguishing or killing criteria as shown in Table 5. A test case that satisfies any of the eight criteria will be able to kill a generated mutant. Let I and M be the intended and its corresponding mutated query, respectively. Let us assume that queries use tables having N number of records in total (N > 1). Let n1 and n2 be the set of records selected by executing I and M, respectively. The criterion C0 distinguishes I and M, if either (i) the cardinality of the intersection between n1 and n2 is zero or N; or (ii) the cardinality of the union of n1 and n2 is greater than N. C1 distinguishes between I and M, if the cardinality of the intersection between n1 and n2 is not zero. C2 distinguishes between I and M, if the cardinality of the intersection of n1 and n2 is greater than zero. Let s1 represents an application state, where query I runs successfully and query M results in syntax error.

3.2 Details of the mutation operators We show a sample database table named tlogin in Table 6. The table has three columns named id, uid, and pwd, which represents unique identification number of a user, login id, and login password, respectively. Let us assume that an intended query written by a programmer is “select id from tlogin where uid=’” + userid +”’”. Here, userid is a string variable that receives user supplied userid and

80

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:30 from IEEE Xplore. Restrictions apply.

examples of NEGC operator, where equal (=) operator in I is mutated to not equal (!=) in M. The first row shows that ’aaa’ (non SQLIA test case) cannot kill the mutant. The mutant is killed by a tautology (second row) and union attack test cases (third row), where the criterion C1 is satisfied.

becomes part of query generation process without filtering. We will use the tlogin table and the intended query to describe the proposed operators. Table 6: Records of the table tlogin id (numeric) 1 2 3

uid (varchar) aaa bbb ccc

pwd (varchar) aaa bbb ccc

Table 8: Example applications of NEGC T aaa

Intended query (I) Mutated query (M) n1 n2 St. select id from select id from tlogin {1} {2, L tlogin where where not uid!=’aaa’ 3} uid=’aaa’ //ΔNEGC {1, {1, K select id from tlogin ’ or select id from 1=1 -- tlogin where uid=’’ where not uid!=’’ or 1=1 2, 3} 2, 3} -- //ΔNEGC or 1=1 -’ union select id from select id from tlogin {20} {1, K select tlogin where uid=’’ where not uid!=’’ union 2, 3, 20 -- union select 20 -- select 20 -- //ΔNEGC 20} T: Test case. St.: Status of mutant. L: Live. K: Killed.

Remove SQL where conditions (RMWH). The RMWH operator removes where conditions of SQL queries, which results in selecting all rows form tables. The generated mutants are killed by test cases that satisfy the criterion C0. Test cases having tautology or union type attack will be able to kill the generated mutants. The operator is applicable for SELECT, UPDATE, and DELETE type queries.

Table 9: Example applications of FADP

Table 7: Example applications of RMWH T aaa

Intended query (I)

Mutated query (M)

T aaa

Intended query (I) Mutated query (M) select id from select id from tlogin tlogin where where false and uid=’aaa’ uid=’aaa’ //ΔFADP ’ or 1=1 -- select id from select id from tlogin tlogin where where false and uid =’’ uid=’’ or 1=1 -- or 1=1 -- //ΔFADP ’ union select id from select id from tlogin select 20 -- tlogin where where false and uid =’’ uid=’’ union union select 20 -select 20 -//ΔFADP T: Test case. St.: Status of mutant. L: Live. K: Killed.

n1

n2 St . select id from {1} {1, L tlogin //ΔRMWH 2, 3} select id from {1, 2, {1, K tlogin //ΔRMWH 3} 2, 3} select id from {20} {1, K tlogin //ΔRMWH 2, 3}

select id from tlogin where uid=’aaa’ ’ or select id from tlogin 1=1 -- where uid=’’ or 1=1 -’ union select id from tlogin select where uid=’’ union 20 -- select 20 -T: Test case. St.: Status of mutant. L: Live. K: Killed.

Table 7 shows four example applications of the operator. The first row shows intended queries (I) and mutated queries (M) with test cases ’aaa’. We notice that ’aaa’ is a valid uid of Table 6, which does not contain any SQLIA. Executions of I and M result in n1 and n2 set, which are {1} and {1, 2, 3}, respectively. The intersection and union of the two sets is {1} and {1, 2, 3}, respectively. The cardinalities of intersection and union are 1 and 3, respectively. Therefore, the test case does not satisfy the criterion C0, and the mutant remains live. However, the second and third test cases kill the mutant that contain tautology attack (“’ or 1=1 --”) and union attack (“’ union select 20 --”), respectively. Negation of expression in where conditions (NEGC). The NEGC operator negates unit expressions (e.g., uid=’aaa’ to uid!=’aaa’) present in where conditions of SQL queries. The operator is applicable for SELECT, UPDATE, and DELETE type queries. The intersection of two record sets (one selected by an arbitrary condition and the other selected by its negation) should be null, provided the semantic of the query does not change. We take advantage of this observation to force the generation of attack test cases that will violate the observation (i.e., satisfy the C1 criterion). Table 8 shows three

n1 n2 St. {1} {} L {1,2, {1, K 3} 2,3 } {20} {20 K }

Parenthesize where conditions and prepending ‘false and’ (FADP). The FADP operator parenthesizes the where condition of a SQL query and prepend ‘false and’ after the WHERE keyword. The objective is to nullify any output generated by the execution of an intended query in case of non attack test cases. Test cases having SQLIAs will kill the generated mutants based on C2 criteria. The operator is applicable for SELECT, UPDATE, and DELETE type queries. Table 9 shows three example applications of the operator, where the mutant is live with the non attack test case ’aaa’ (in the first row). The mutant is killed by a tautology (second row) and a union attack test case (third row), where the killing criterion C2 is satisfied by both test cases. Unbalance parentheses (UNPR). The UNPR operator makes SQL queries syntactically incorrect to the database engine by appending left parenthesis at the beginning of where condition. Test cases having right parenthesis can make the query syntactically correct and the execution of generated queries successful. However, non malicious test cases generate queries having unbalanced parenthesis, and the database driver will throw SQL syntax error. The operator can be applied for SELECT, INSERT,

81

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:30 from IEEE Xplore. Restrictions apply.

UPDATE, and DELETE type queries. The generated mutants are distinguished by C3 criterion in the presence of test cases containing SQLIAs of tautology, union or hexadecimal encoded query with right parenthesis. Table 10 shows three examples of the UNPR operator, where a left parenthesis is added after WHERE keyword in M. The mutant is live for the first test case ’aaa’ and killed for the last two attack test cases.

Overriding Commit and Rollback options (OVCR). The commit and rollback options are used to maintain the information consistency in database tables. The OVCR operator is used to override the commit and rollback flag to allow SQLIV, which can be exploited by creating the state of inconsistencies in databases with appropriate SQLIAs. The overriding can be performed either by (i) modifying the commit and rollback flag values or (ii) removing statements that set the flag value of commit and rollback. Since, any inconsistent information in tables of a database occurs through INSERT, UPDATE, and DELETE operations, the mutants are killed by the test cases that satisfy the C4 criteria.

Table 10: Example applications of UNPR T aaa

Intended query (I) Mutated query (M) s1 select id from select id from tlogin No tlogin where where (uid=’aaa’ error uid=’aaa’ //ΔUNPR in I ’) or select id from select id from tlogin Error 1=1 -- tlogin where where (uid=’’) or in I uid=’’) or 1=1 -- 1=1 -- //ΔUNPR ’) select id from select id from tlogin Error union tlogin where where (uid=’’) union in I select uid=’’) union select 20 -20 -select 20 -//ΔUNPR T: Test case. St.: Status of mutant. L: Live. K: Killed.

s2 St. Error L in M No K error in M No K error in M

Table 12: Example applications of OVCR Test case ’; update tlogin set pwd = ’thief’ where uid = ’admin’ -’;set autocommit=1; update tlogin set pwd = ’thief’ where uid = ’admin’ --

Setting multiple query execution flags true (MQFT). The MQFT operator injects SQLIV by setting multi query flag value to true. The database API methods are used to connect databases through URL string specifying database name and location, which also allows programmers to specify about executing multiple queries in a single connection. Allowing multiple queries makes it possible to perform piggybacked query attacks that consist of providing additional SQL queries. The operator either adds the flag value by appending the connection URL or modifying the existing flag value to true.

Original statement Mutated statement conn.setAutocom conn.setAutocommit mit (false); (true); //ΔOVCR …. conn.commit();

…. //Remove statement //ΔOVCR

Table 12 shows two examples of the operator. Here, conn is a Java Connection class for connecting with databases. The first row shows that the mutated statement sets the flag of setAutocommit method to true. This allows any changes due to INSERT, UPDATE, and DELETE operations become permanent and create inconsistency through an SQLIA test case (first test case). In the second example, the statement conn.commit() is removed. Therefore, autocommit mode is disabled. However, an SQLIA might exploit it by enabling autocommit flag to true and performing further operations to make information inconsistent (second test case). Set the maximum number of records returned by a result set to infinite (SMRZ). The database manipulation API often provides developers to specify the maximum number of records that can be returned by the result sets after executing SELECT type SQL queries. The SMRZ operator will set this value to zero of setMaxRows method call to allow maximum number of records in case of attack test cases. Therefore, the selected record set will be transferred to the result set used by the program without any truncation. If the code is written poorly, all the results might appear in output and information can be leaked. For example, when a user is logged on during authentication, only one record set related to his userid should be extracted from database table. Otherwise, it might extract more records and reveal information to an unauthorized user. The generated mutants are killed by the test cases that satisfy the criterion C6. Table 13

Table 11: Example application of MQFT Test case Original statement Mutated statement ’ ; show String dataCon = String dataCon = tables; -- "jdbc:mysql://localhost "jdbc:mysql://localhost/theDatabas /db1"; e?allowMultiQueries=true"; //ΔMQFT

The mutated queries can be killed by test cases that contain piggybacked query attacks. The mutants are killed by any of the criteria C4, C5, or C6. Table 11 shows an example application of MQFT operator. The original statement contains database URL through the variable dataCon, which does not have any multiple queries execution flag set. Therefore, the flag will assume the default value as false. In the mutated statement, the URL is modified by appending “allowMultiQueries=true”. An example test case that can kill it is “’; show tables; --”, as it satisfies the C6 criteria. Here, the “show tables” query provides all the tables of a database in MySQL.

82

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:30 from IEEE Xplore. Restrictions apply.

method’s argument is modified from false to true value, which means escape character processing is enable and “\\” character will be treated as “\”. In the second example, the opposite action is performed. Both attack test cases contain the sp_makewebtask command that runs the query “select * from tlogin” and saves output in the file users.html.

shows an example application of the operator. Here, Statement is a Java class that supports the execution of queries and the setting of related properties. The mutant is killed by a tautology attack test case, which will extract only one record in the original program and N (N >1) number of records in the mutated program.

Table 15: Example applications of OVEP

Table 13: Example application of SMRZ Test case Original statement Mutated statement ’ or 1=1; - Statement.setMaxRows (1); Statement.setMaxRows (0); //ΔSMRZ

Test case

Table 14: Example application of SQDZ Test case ’;SELECT BENCHMARK (10000000, ENCODE (’abc’,’123’)); ---

Original statement

Mutated statement

Statement.setEscap Statement.setEscapePr ’;EXEC master..sp_makewebtask eProcessing ocessing (true); "C:\\\\ users.html", select (false); //ΔOVEP * from tlogin; -’ Union EXEC Statement.setEscap Statement.setEscapePr master..sp_makewebtask eProcessing (true); ocessing (false); "C:\\ users.html", select //ΔOVEP * from tlogin; --

Original statement Mutated statement Statement.setQuery Statement.setQueryTime Timeout (30); out (0); //ΔSQDZ

4. Implementation and Evaluation

Set query execution delay to infinite (SQDZ). There are delays between issuing of queries by applications and receiving results from databases. Programmers can specify the maximum delay (or query timeout value) with supported database API method calls. The SQDZ operator makes the delay to infinite as a way of injecting SQLIV. The operator forces to generate test cases that exploit SQLIV by performing time-based attacks to infer execution paths of applications [10]. The mutants are killed, if a test case satisfies C6 criterion. Table 14 shows an example application of the operator, where setQueryTimeout method parameter is set with zero to allow infinite execution delay of a query. The attack test case that kills the mutant uses the BENCHMARK function supported by MySQL database, which repeatedly performs a given task. In this case, the task includes encrypting the string ’abc’ using password string ’123’ with the function ENCODE for 100 million times, which creates a significant delay, that might exceed default application query timeout value. Overriding the escape character processing flags (OVEP). The database manipulation API provides developers the option to allow escape character processing. The OVEP operator modifies the flag responsible for processing escape characters. It toggles the flag value and forces the generation of test cases incorporating both escape and non escape characters. The operator generates mutants that can be distinguished in the presence of piggybacked query or union type attacks. The criterion C5 is used to distinguish mutants. Table 15 shows two example applications of the operator with attack test cases, which kills the mutants. In the first row, setEscapeProcessing

We implement a prototype tool for MUtation-based SQL Injection vulnerabilities Checking (testing) (MUSIC) in Java. The tool accepts Java Server Pages (JSP) files and generates mutants based on the proposed operators. The original and mutated statements are instrumented to monitor internal result set returned after query executions and database states. A snapshot of the tool is shown in Figure 3. For each of the JSP files (e.g., Default.jsp), the corresponding Java and Class file are generated and kept into a real web application container (e.g., Apache web server) to perform mutation analysis with input test file (e.g., input.txt). Each test case consists of an appropriate URL having parameters and their corresponding values. The intermediate result set and database states are saved for both mutants and the original program for each test case. An oracle program is run to decide whether mutants are killed or not. At the end of mutation analysis, a report is generated indicating the current mutation score for the supplied test file and the list of live mutants. The test set can be enhanced to kill the remaining mutants and increase mutation scores. We use five web-based applications available from an open source web application repository [22] to evaluate the proposed operators. The same applications are used in several related works [4, 5, 6, 7, 10]. The applications are implemented in JSP and they use MySQL databases in the backend. The characteristics of the five applications are shown in Table 16, which include the total number of JSP files for each application, total lines of code (LOC), total number of files that might have SQLIV, total number of SELECT, INSERT, UPDATE, and DELETE queries generated by JSP code in vulnerable JSP files.

83

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:30 from IEEE Xplore. Restrictions apply.

Each of the applications has character and numeric filters to prevent SQLIA.

applications. We find that mutation score (MS) (i.e., ratio of the number of mutants killed to the total number of non-equivalent mutants generated) is zero for each of the applications with T1 test data set (column 5 of Table 17). We try to increase MS by enhancing T1. For each of the applications, we add attack test cases to kill the mutants. We denote the enhanced test data set as T2. We observe that the MS score is also zero for T2 (column 5). Therefore, the applications are not vulnerable to SQLIAs. Now, we want to investigate whether the proposed operators have any effect on the applications having SQLIV. We remove all the input filters from the vulnerable source files for each application. After removing the filters, we denote the resultant applications as bad applications, as they can be under SQLIAs with appropriate test input. Next, we apply T2 data sets to perform mutation analysis. We notice that the MS increases significantly for each of the applications as shown in column 6 of Table 17. The lowest MS is 64% for the Portal application, whereas the highest MS is 78% for the Bookstore application. The T2 set does not contain test cases to kill the mutants generated by UNPR operator. These live mutants are killed by generating appropriate test cases manually. After adding those test cases, the MS increases to 100% for each of the applications. Therefore, the proposed operator creates a difference between vulnerable and non-vulnerable applications in terms of MS.

Figure 3: Snapshot of MUSIC Table 16: Characteristics of the application Application Total LOC # of files with SEL INS UPD DEL files SQLIV Bookstore 10 16,959 7 17 2 2 0 Classifieds 19 10,949 6 7 2 1 1 Events 13 7,242 7 7 3 2 2 Employee 10 5,658 7 8 2 2 2 directory Portal 29 16,453 3 3 1 1 0 SEL: SELECT, INS: INSERT, UPD: UPDATE, DEL: DELETE.

Table 17: Mutation analysis with T2

For each of the applications, we generate an initial test data set that contains no SQLIAs. We use a benchmark test data set containing both attack test cases and non attack test cases from AMNESIA [6]. The initial test data set is denoted as T1 for each of the applications. We perform experiments primarily to answer the following questions. 1. Do the proposed operators help generating adequate test data set? (Section 4.1) 2. How to obtain selective mutation operators to save the cost of mutation analysis for testing SQLIV? (Section 4.2)

Application WC AMC

Total

Bookstore Classifieds

105 70

73 41

178 111

Initial MS (%) MS (%) of bad app. 0 0

77.53 70.27

Events Employee directory Portal

88 103

49 50

137 153

0 0

68.61 69.28

61

14

75

0

64.29

4.2 Obtaining selective mutation operators The objective of this experiment is to choose selective mutation operators from the proposed nine operators. The selection is helpful for reducing the cost of mutation analysis. We are motivated by several empirical studies conducted by Tuya et al. [15] and Baudry et al. [20] in this regard. We perform selective mutation for WC and AMC category separately. We use the notion of minimal test data suite (MTS) [24] for a set of mutants, which implies the test data set that kills all the mutants, and if any test case is removed from the test data set, then MS decreases. We generate an MTS for each operator. Then, the operators are ranked based on the subsume relationship. If the ranking of any two operators are equal, then the

4.1 Effectiveness of the operators We take the initial version of five web applications and use MUSIC tool to generate mutants. The second and third column of Table 17 shows the total number of mutants generated for the WC (where conditions) and AMC (API method calls) categories, respectively for each of the applications. We generate the Java files for the generated mutant JSP files and perform mutation analysis using T1 data set for each of the

84

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:30 from IEEE Xplore. Restrictions apply.

results. The second, third, fourth, and fifth column indicate the number of mutants generated from the operator RMWH, NEGC, FADP, and UNPR, respectively. We notice that test data set that is adequate for the mutants generated by UNPR operator, cannot kill any of the mutants generated by the other three operators. The underlying reason is that the killing criteria for a mutant generated by the UNPR operator is based on comparison of the syntax error in an intended query and no syntax error in the corresponding mutated query. A test case that kills the mutant must have a right parenthesis included. In contrast, the test case is not able to kill the mutants generated by other three operators. The last row of Table 18 shows the average number of mutants and mutation scores. We get the following ranking: MS!RMWH ≥ MS!FADP > MS!NEGC > MS!UNPR. The score of both MS!RMWH and MS!FADP are the highest. Therefore, we want to save the number of mutants (i.e., select only that operator that generates minimum number of mutants). Since the average number of mutants generated by RMWH and FADP are also same, we choose any of them to include in selective mutation operators list. We choose RMWH operator, which also subsumes NEGC operator. They are both designed to detect tautology and union type attacks. Therefore, we can exclude NEGC operator. However, the UNPR operator cannot be excluded, as the mutants generated by it are hard to kill (stubborn mutants). We keep them in the selective list. Therefore, the selective mutation operator set consists of RMWH and UNPR operators. We perform similar analysis for choosing selective mutation operators for the AMC category. A summary of analysis is shown in Table 19, and we select the MQFT and SMRZ operators.

operator that generates less number of mutants is considered in the selective list. We use the notion of subsume relationship among two test criteria based on Zhu et al. [21]. Let us assume that two test data T1 and T2 satisfy two test criteria C1 and C2, respectively. C1 will subsume C2 (or C1 > C2), if T1 satisfies both criteria C1 and C2. However, T2 does not satisfy C1. Therefore, in practice, a minimal test data set that is C1 adequate automatically satisfies C2 adequacy criteria. Moreover, test criteria C2 can be ignored, which helps reducing the cost of testing. Similarly, for two mutation operators m1 and m2, if the MTS for m1 also kills all the mutants generated from m2, then m1 > m2. In other words, the MTS of m1 will achieve 100% MS by killing all the mutants generated by applying operator m2. In practice, it might not be possible, and we chose the operator that achieves more MS than the other. If m1 = m2 (i.e., the score achieved by MTS of m1 and m2 are equal), then we keep the operator which generates minimum number of mutants. Table 18: Selective mutation analysis for WCM Application

RMWH NEGC FADP UNPR MS!RMWH MS!NEGC MS!FADP MS!UNPR

Bookstore

19

27

19

40

44

36

44

0

Classifieds Events Employee directory Portal Average

9 11 12

19 23 32

9 11 12

33 43 47

40 39 43

26 25 23

40 39 43

0 0 0

4 11

36 27

4 11

35 40

51 43

10 24

51 43

0 0

Table 19: Selective mutation analysis for AMC Application Bookstore Classifieds Events Employee directory Portal Average

MS!MQFT 38 41 43 42 42 41

MS!OVCR 10 15 14 14 16 14

MS!SMRZ 0 0 0 0 0 0

MS!SQDZ 10 15 14 14 16 14

MS!OVEP 10 15 14 14 16 14

5. Conclusions

For each category, we generate mutants and minimum adequate test data sets (i.e., MTS) for each of the applications separately. For example, the WC has four operators (RMWH, NEGC, FADP, and UNPR) and we generate four MTS separately denoted as T1, T2, T3, and T4. Therefore, T1 test data set kills all the mutants generated by RMWH operator, T2 kills all the mutants generated by NEGC, and so on. For each MTS, we combine all the mutants generated by all the operators and try to kill all those mutants. We obtain MS and note it for further analysis. For example, for the RMWH operator, we apply the T1 test data set to kill the mutants generated by four operators RMWH, NEGC, FADP, and UNPR. The obtained MS is denoted as MS!RMWH. Similarly, we obtain MS!NEGC, MS!FADP, and MS!UNPR. Table 18 shows the detail

This work applies mutation-based approach for testing SQLIV by proposing nine mutation operators and necessary killing criteria for the mutants. Our approach addresses the gap of testing software vulnerabilities and generation of adequate test data sets that can reveal vulnerabilities. The proposed operators inject SQLIV by mutating both programmerwritten queries and database API method calls implemented in applications. The unique feature of the operators is that the generated mutants can be killed only with test cases containing SQLIAs. Ordinary test cases having no SQLIAs do not kill the mutants. We implement a prototype MUtation-based SQL Injection vulnerabilities Checking (testing) tool named MUSIC to automate the generation of mutants and perform

85

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:30 from IEEE Xplore. Restrictions apply.

mutation analysis with input test case. The operators are found to be effective to detect SQLIV for five open source web-based applications written in JSP. The more vulnerable an application is, the higher MS score it will achieve. If vulnerabilities are fixed, the MS will be lower. We do not address the SQLIV of stored procedures, as attacks on stored procedures are very rare in practice. The proposed operators are effective for testing SQL queries having simple form of where conditions. However, we plan to propose operators for complex conditions (e.g., conditions having UNION, INNER JOIN etc.) in future. We will address the exploitation of SQLIV by tracing database error messages thrown by applications in future work. Moreover, we plan to apply the proposed distinguishing criteria for distributed applications performing database operations concurrently, where one application might insert rows in a table and the other might delete rows simultaneously. Our future work also includes extending the tool’s capability by generating mutants for other popular implementation languages (e.g., PHP), and enhancing the operators to address other web-based attacks related to injections (e.g., cross site scripting).

Conference on Automated Software Engineering (ASE 2005), Nov 2005, Long Beach, CA, USA, pp.174-183. [7] Y. Shin, L. Williams, and T. Xie, “SQLUnitGen: SQL Injection Testing Using Static and Dynamic Analysis”, In Proceedings of the International Symposium on Software Reliability Engineering (ISSRE), 2006, Raleigh, NC, ISBN 978-0-9671473-3-3-8. [8] L. A. Gordon, M. P. Loeb, W.Lucyshyn, and R. Richardson., “Ninth CSI/FBI computer crime and security survey”, Technical Report RL32331, C.S.I. Computer Security Institute, 2004. [9] S. Thomas and L. Williams, “Using Automated Fix Generation to Secure SQL Statements”, In Proceedings of Third International Workshop on Software Engineering for Secure Systems (SESS’07), Minneapolis, 2007, pp. 9-14. [10] SQL Injection Walkthrough, www.securiteam.com/ securityreviews/5DP0N1P76E.html (Accessed Feb 2008) [11] J. Lin and J. Chen, “The Automatic Defense Mechanism for Malicious Injection Attack”, In Proceedings of Seventh International Conference on Computer and Information Technology (CIT2007), Fukushima, Japan, Oct 2007, pp. 709-714. [12] Michael Howard and David C. LeBlanc, “Writing Secure Code”, Second Edition, Microsoft Press, 2002. [13] K. Kemalis and T. Tzouramanis, “SQL-IDS: A Specification-based Approach for SQL-Injection Detection”, In Proceedings of 23rd ACM Symposium on Applied Computing (SAC’08), Mar 2008, Fortaleza, pp. 2153- 2158. [14] W.K. Chan, S.C. Cheung, and T.H. Tse, “Fault-based Testing of Database Application Programs with Conceptual Data Model”, In Proceedings of the Fifth International Conference on Quality Software (QSIC 2005), Los Alamitos, California, 2005 [15] J. Tuya, M. Suárez-Cabal, and C. Riva, “Mutating Database Queries”, Information and Software Technology, 49(4) 398-417, April 2007. [16] Jcrasher, Accessed from www.cc.gatech.edu/jcrasher. [17] R. DeMillo, R. Lipton, and F. Sayward, “Hints on test data selection: Help for the practicing programmer”, IEEE Computer Magazine, Volume 11, Issue 4, 1978, pp. 34-41. [18] W. E. Howden, "Weak mutation testing and completeness of test sets," IEEE Transaction on Software Engineering, Volume 8, Number 4, July 1982, pp. 371-379. [19] W. G. Halfond, J. Viegas, and A. Orso, “A Classification of SQL-Injection Attacks and Countermeasures”, In Proceedings of the International Symposium on Secure Software Engineering (ISSSE 2006), Arlington, Virginia, Mar. 2006. [20] T. Mouelhi, Y. Le Traon, and B. Baudry. "Testing security policies: going beyond functional testing". In Proceedings of International Symposium on Software Reliability Engineering (ISSRE'07), Trollhättan, Sweden, November 2007. [21] H. Zhu, P. A. V. Hall, and J. H. R. May, “Software Unit Test Coverage and Adequacy”, ACM Computing Surveys (CSUR), Volume 29, Issue 4, Dec. 1997, pp. 366-427. [22] Open Source Web Applications with Source Code in ASP, JSP, PHP, Perl, ColdFusion, ASP.NET/C#, http://gotocode.com (Accessed Feb 2008).

Acknowledgement This work is partially supported by the Natural Sciences and Engineering Research Council of Canada (NSERC).

6. References [1] The Open Web Application Security Project (OWASP), www.owasp.org (Accessed Feb 2008) [2] G. Buehrer, B. W. Weide, P. A. G. Sivilotti, "Using parse tree validation to prevent SQL injection attacks", Proceedings of the 5th International Workshop on Software Engineering and Middleware (SEM '05), Lisbon, Portugal, 2005, pp.106-113. [3] S. W. Boyd and A. D. Keromytis, “SQLrand: Preventing SQL injection attacks”, In Proceedings of the 2nd Applied Cryptography and Network Security Conf. (ACNS 2004), Jun. 2004, pp. 292-302. [4] Z. Su and G. Wasserman, “The Essence of Command Injection Attacks in Web Applications”, In Proceedings of Symposium on Principles of Programming Languages POPL’06, Jan 2006, South Carolina, USA, pp. 372-382. [5] Y. Kosuga, K. Kono, M. Hanaoka, M. Hishiyama, Y. Takahama, “Sania: Syntactic and Semantic Analysis for Automated Testing against SQL Injection”, In Proceedings of 23rd Annual Computer Security Applications Conference, 2007 (ACSAC 2007), Miami, Dec 2007, pp. 107-117. [6] W. Halfond, and A. Orso, “AMNESIA: Analysis and Monitoring for NEutralizing SQL-Injection Attacks”, In Proceedings of the 20th IEEE/ACM International

86

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on January 27, 2009 at 01:30 from IEEE Xplore. Restrictions apply.