3000

2010 3rd

International Conference on Advanced Computer Theory and Engineering(ICACTE)

Software Security Testing Based on Typical SSD:A Case Study Zhanwei Hui

Song Huang

BinHu

Yi Yao

PLA Software Test and




Evaluation Centre for




Military Training

Military Training

Military Training

Military Training

PLA University of Science




and Technology

and Technology

and Technology

and Technology

Nanjing, jiangsu province




PRC

PRC

PRC

PRC

e-mail:

e-mail: [email protected]

e-mail: [email protected]

e-mail:

[email protected]

Abstract-Due

to

the

[email protected]

increasing

complexity

of

Web

applications, traditional function security testing ways, which only test and validate software security mechanisms, are becoming ineffective to detect latent software security defects (SSD). The number of reported web application vulnerabilities is increasing dramatically. However, the most of vulnerabilities result from some typical SSD. Based on SSD, this paper presents an effective software security testing (SST) model,

as well as the high degree of the quality required for their critical functioning. To guarantee such respects, we need to generate extensive and effective test suites to evaluate the typical defects. To reach this test completeness, we rely on this paper on Software Security Defects (SSD) -based methods. Surrounding SSD, the software security test would be more effective, and put an end to the typical security

which extends traditional security testing process to defects

defects. Data analyzing can test the intercourse of data, and

behavior analysis which incorporates advantages of traditional

guarantee the security of critical data information. We first

testing method and SSD-based security testing methodology.

describe the process and critical function

Primary applications show the effectiveness of our test model.

software security testing based on SSD. This testing process

Keywords- software security test; function test; vulnerability; software security defect; defect behavior I.

INTRODUCTION

modules of

is well-adapted to different Web systems features such as hyperlinks, sending and receiving data and browser-server communications etc. Second, we specify the system security requirements based on software security defects.

(HEADING 1)

The coming of the internet age should have led developers to collectively acknowledge that all applications need to be

9000

secure [1], but this has not been the case. As is depicted in

8000

the Figure 1,

"Z

lP

� 7000

CERT/CC[II] researches show that the

]

number security accident sharply up to 8078 under 2008. And in 2000, we can see that the number is only 1090.

4129

� 4000

�.

abundant, and the question no longer seems to be "if to test" for security, but rather "when and where to test" and "for what?" Few software engineering would argue against the

V>

value of performing testing of security software, or testing security mechanisms in general software. Software security

345 311 262 'f1.I ./' 0• * -, , __ , .-,

-:--

171 .

1995

1996

1997

1998

1999

2000

crucial for software which usually refer to the critical data information, and these show desires for higher demand to

•

/ / 3780

IO�/

2000

0

is one of the most important software quality properties. It is

-

0!'!ooo. 3784 .

2/4/

3000

1000

78

· 236

5990

6000

� 5000

Software security testing [2] (SST) researches are presently

�

8061

2��a/OO2

2003

2004

2005

2006

2007

2008

Fig.1. CERT/CC security accident statistics

security of software itself. But traditional security test pays more attention to function [3], but not security, and the even major problem is that the software test centers have no

The contributions of this ongoing work is that it applies the 3 new test model to a real software system, which is M TR.

guidance

And after testing, we compare the SSTBS with the SSFT,

standards

or

progresses

for

practical

web

applications security testing.

and the results show the advantages of our new testing

In practice, it is more and more difficult to ensure the

model.

respect of Web applications to their security requirements

This paper is organized as follows: section II presents

[4]. These difficulties are due to the complexity level of

background

such systems, their variety, and their increasing distribution

introduce the concept of SSD,

978-1-4244-6542-2/$26.00 © 2010 IEEE

V2-312

of

SSD-based

security

testing.

We

first

and then analysis the

2010 3rd

inspirations

of

our

International Conforence on Advanced Computer Theory and Engineering(1CACTE)

research.

Section

III

exposes

particular actions, and accountability for actions taken. So a

the

methodology to integrate SSD-based testing process in

SSD could result in software security mechanism incorrect.

software security testing model. In section IV, we briefly

We have tried to keep our use of the term "defect"

present the critical technology of our testing model. Section

intuitive without conflicting with standard terminology. The

V illustrates an ongoing case study: M3TR system security

IEEE

testing. And it also shows the test result of the experiment.

Terminology [5] includes definitions of

Finally, it presents the conclusion and introduces the future

Standard

A.

of

Software

Engineering

•

error: human action that produces an incorrect result (such as software containing a fault),

•

fault: an incorrect step, process, or data definition in

•

failure: the inability of a system or component to

work. II.

Glossary

BACKGROUND

a computer program, and

Different dejinations of SSD

perform its required functions performance requirements.

Generally, a security defect is a part of a program that

within

specified

can cause the system to violate its security requirements.

A failure may be produced when a fault is encountered. This

Finding security defects then, demands some knowledge of

glossary lists bug as a synonym for both error and fault. We

system

use defect as a synonym for bug, hence (in IEEE terms) as a

security

requirements.

These

requirements

vary

according to the software under test and the application, so

synonym for fault, except that we include flaws that have

we cannot address them in detail here. Usually, they concern

been

identification and authentication of users, authorization of

accidental ones.

)

r Software

)

Requirement

--

Soflware Security Defect

I Software . stmcture design

)

I

I

Detailed design

Coding

inserted

I

)

Testing

)

into

a

ImPlementation

system

)

intentionally,

Maintenance

� Software Vulnerability � Software Security Threat � Software Security Accident ----v ----v !------v

as well

as

Software Life Cycle

Software Security Life Cycle

Fig.2. The evolvement of SSD

G.McGraw uses a fairly broad definition of a SSD in the

CERT/CC [18] and researches of G.McGraw [4], most

report [7], which is that Software security defects are any

software vulnerabilities arise from common causes (such as

conditions or circumstances that can result in denial of

SSD), and the top 10 cause account for about 75percent of

service, unauthorized disclosure, unauthorized destruction

all software vulnerabilities; more than 90 percent are caused by known SSD types. Figure 3 shows that the typical

of data, or unauthorized modification of data. Our notion of SSD corresponds to that of a fault, with

vulnerabilities produced major security accidents in 2006.

the possibility that the fault may be introduced either

30.00%

accidentally or maliciously. Software security defect will be

25.00%

different during software life cycle.As is depicted in the

20.00%

Figure 2, the SSD may be induced during the phase of

15.00%

requirement, or structure design, or detailed design, or

10.00%

coding, or even testing. And the Life Cycle of the SSD is

5.00%

likely to Software Life Cycle.

0.00%

A software security defect is a security flaw in software

s� 'iji,i

which can result in a security policy violation. Software

��

u·

vulnerability is a kind of SSD representation as software function. The initial test bug report is a kind of software vulnerability.

As they

exist

in

software

system,

III � � �

i

" �� ij

I

��

Fig.3. 2006 OWASP Top-10 Vulnerability [6]

then

malicious users would make use of them to authorization •

operation. And software security accident will happen. B.

Top vulnerabilities list projects

Many organizations and research centers presents the

The feasibility of software seucurity test baased on SSD

Top vulnerability list every year. CWE has maintained a list

Typical SSD induce most of software vulnerabilities

of the top 25 dangerous programming errors [12]. OWASP

•

Security vulnerabilities result from defects that are

(Open Web Application Security Project) as an open

unintentionally introduced into software during design and

organization, it releases its Top-to projects every year.

development. According to a preliminary analysis by the

SANA (SysAdmin, Audit, and Security) institute put out its

V2-313

2010 3rd

Top_10

network


dangerous

vulnerabilities

from

2000,

The integration model is twofold, which is compatible with

together with NIPC (National Instructure Protection Center).

different SSD. On the left, it is the traditional function

And hereafter, it extends to Top_20, which is repaired by

security testing process, which tests software security

many security researchers every year.

mechanisms, and produces function test cases. Then on the

We summarize several top_10 lists, and give the Top 10

other side, it is the testing process based on SSD, which is

list from 2006 to 2009 in Table 1. Each of the errors or

also called adverse testing. At the end of the model, this

vulnerabilities on their list is described in detail, which has

integrated process will generate a new kind of test cases

enabled us to associate the listed errors with underlying

which not only validate software security policies, but also

software security defects.

provide the user with certain confidence that the SUT could resist typical security attacks.

Table 1. 2006-2009 Top_IO List of Software Vulnerabilities

------Al

A2

A3

2006

2007

Cross

Site

AS

Scripting

Scripting

(XSS)

(XSS)

Injection

Injection

Injection

Flaws

Flaws

Flaws

Insecure

Insecure

Insecure

Remote

File

Buffer Overflows

Direct Object

A8

Leakage

and

Improper

Remote

Injection Cross (XSS) Broken File

Direct Object

Direct Object

Reference

Reference

References

Cross

Management Insecure

Site

Request

Request

Forgery

Forgery

Forgery

(CSRF)

(CSRF)

In formation

Information

Leakage

Leakage

and

Broken

and

Authenticatio

Authenticatio

n and Session

n and Session

Management

Management

Failure

to

Restrict URL Access

Insecure

Insecure

Unvalidated

Cryptographi

Redirects and

c Storage

c Storage

Forwards

Insecure

Insecure

Insecure

Communicati

Communicati

Cryptographi

ons

ons

c Storage

to

Failure

Yes

Misconfigurat

Cryptographi

Failure

c

mappings;

Input

to

request

Restrict URL

Restrict URL

forgery

Access

Access

III.

dalaba

Security ion No direct

Un validated

Cross-site

Site

(CSRF)

Improper

Improper

Broken

session

Cross

Request

Error

and

n and Session

Direct Object

Handling

authentication

Authenticatio

Insecure

Error

Storage

Site

Scripting

Insecure

Handling

Insecure

also illustrate the critical technologies in details in Figure 4.

Include

Site

CTRITICAL TECHNOLOGY OF THE TEST MODEL

In order to specifY how our test model is executed, we

2009 Site

Include

Handling

management

AIO

File

Error

Broken

A9

Remote

Cross

Insecure

Infonnation

A7

Cross

Scripting

Reference

A6

2008 Site

(XSS)

Include

A4

Cross

IV.

Decompose

[)ecompose

Insufficient Transport Layer Protection

SOFTWARE SECURITY TESTING BASED ON SSD

In this section we propose a model to build an integrated

Fig. 4. Integrated software security test model based on SSD

process suitable for software security testing. As traditional security testing based on function test, such as white box testing or black testing, our model not only integrates

A.

security function testing methods, but also put out an effective security testing way, which is based on SSD. An overview of how our testing model is executed is presented in Figure 2.

Typical SSD classification module Typical

SSD

classification

includes

correct

and

appropriate definition of SSD, classification of SSD and typical SSD list. For the definition and classification of SSD are the preparation for typical SSD list and even database.

V2-314

Our

defects

list

is

based

on

the

opened

2010 3rd


vulnerabilities database. But the SSD is the underline cause

application,

of vulnerabilities. Although there are a great deal of

multimedia recourses. In our case study, we only consider

vulnerabilities, but the defects is numerable. We classifY the

function and typical SSD to generate test instances.

SSD not only form SSD induced vulnerabilities, but also

r--II

-

from their critical rank, which is important for most critical

provide

infon:l3tion

To reach this aim, in the attack points we specifY which

��:��arCh�ng

automatically

Scarch

the

methodology inspired from

paths

[3].

using

a

dedicated

____

and distributed character of web applications, the defects

Deleting/

list is usually very huge. Therefore, it is important to

!Cedi fying request

classifY the SSD. We bring forward with a PRI metric

[14].

list with executed PRJ is generated.

SSD test behavior formalization module description of adverse behavior and graphic

5.

A.

As we get the primary test requirements, we use a kind of Security Use Cases to describe function behavior, which is related to software security mechanisms. And then, a new Adversarial Security Use Case method is proposed for adverse behavior description, which is based on

[15].

generate test use case instance by hand, but tedious work. And it also needs experience. We provide with three to generate test instances, together with

coverage criteria, which not only cover with security mechanisms and latent security defects, but also traditional repair mechanisms. Security Test

CK I Up l oad 110

ACK

� r i'-' ��''''' f'--'!J.!LI!.:'""C·.ill----' "."Aro+CK i: "i'-----

---:-__-( Data management

____

t::=-

---'----'--

� :� I

o��n�g ....

-==-r---

______

MTR Analysises The

MTR

system is a typical B/S web application. Its

main function data flow graph can be seen in Figure

3.

An

identification module is also taken into account in the first step. Thus, a potential user can connect to the Web Then, he/she can

request for a visit to download the audio or video resource

For every piece of test behavior requirement, it is easy to

Behavior ofSoflware

'IIith 'altcd pass'll rd

�

application using a dedicated VRL.

D. Software security test cases generation module

[8]

Query for usern l:ne

Fig. 6. Level-O diagram of M3TR web application

This module includes description of security function description of test behavior, which is depicted in Figure

: t t 3·�' �h"�"'fud,�r,��

Sear�h reques I nfonna;l�i;O" scarch � ��ffi�� r; ion

Ollcdatc log rc�ucst

Uploading request

l ["",r",ooo"d,,,,,oo,,d,,>

After then, SSD testing

:::= = =1#�:::::f :;-1

=t'� CK ;;;;::==t� Uploading ....i.1.h... I.>loait � .j.!s

L

table with execute-tend paths. Considering the large-scale

userinfo

-�

request

Then according to the paths,

defects test requirements are generated, which is a SSD

algorithm, which is based on

l

Uscrinfo database .. "',,' "r,

path exist latent defects, we have to define algorithms to

algorithms

,

(user IdcntitY

:

of

,

:: i --\ V,[id,,'oo J+ Rot",=;: ,,! t-Y:-:'CK:-+

Y---1':-: ",-,t-''

kinds

:Qucry password !�alt· f�r llser

ldenti ty

the interaction path of SSD in web application under test.

detect

with

L1,o[-g �,-, t,-a� W-' �--�T � Return salt:

SSD detection and classification module

behavior,

users

Log in Dataintcrfacc II---'=-:'''-!-l�(�,UscrlD I �- -

After typical SSD list have generated, it should detects

C.

the

�

web applications. B.

which

during a specific period. Further, we generalized some typical latent SSD list to test the system security. These security defects are inspired from out security test model. Then the security mechanism and the SSD are formally specified using graphic and test behavior model. Their integration will be performed using our algorithms to lead to a secure and understandable specification. Our experiment is executed after traditional function

Reverse Behavior or Software SecurilyTest

test,

and

exposes

8

SSD

of

typical

Web

applications. B.

Experiments result analysis We compare our test model with traditional security

function test in 7 different aspects. For the number of security mechanisms, traditional SSFT StturityTcsl

To

prove

effectiveness

testifY

which

7

come

kinds

of

from

the

access

security

security

test

requirements. But SSTBS could overcome this restriction by typical SSD, which are the experience of different test engineering of security testing. So it produces

3 CASE STUDY: M TR SECURITY TESTING the

only

requirements. So it could not exceed the original security

Fig.5. SSD based software security test process

V.

could

mechanisms,

Function Behavior of Software

of our

security

framework, we carry out a case-study on a

19

kinds of

typical security mechanisms related to typical SSD for the testing

MTR [8]

web

software under test. And then, SSFT could not provide any guarantee for avoidance of SSD, but SSTBS could verifY

V2-315

2010 3rd


potential SSD, which would increase

tested. However, we believe the results we found is a good

the quality of the software. Most of all, SSTBS integrate

indication that Integrated SSD-based security testing model indeed can be effective process for critical software. By

the software with

19

security function requirements with adverse requirements and potential tolerance mechanisms. So the test requirement is more comprehensive than the security test requirement of SSFT. At last, our test model generate

19

security test cases,

much more than the test cases of traditional SSFT, which only have

8

security test cases. Figure 7 show the number

of security test cases of SSTBS and SSFT. As can be seen

running relatively few tests we managed to discover several bugs (related to SSD), and some potential bugs which were not investigated fully. The primary application displays the advantages of our new security testing model. And more practical testing applying is needed for improving of the details. And we believe that our new testing method will be effective.

from the graph, for every function module, our method produce more test cases. Although it means more time and testing work, the SSTBS could detect much more SSD than the SSFT, which is vividly depicted in the Figure

8.

Anymore, our test cases are also with test priority, which is important for test execution. Although more test cases means more test cost, for software security quality, I think it

ACKNOWLEDGMENT

This work was supported by National High Technology Research and Development Program of China (No:

2009AAO1Z402).

Resources of the PLA Software Test and

Evaluation Centre for Military Training were used in this research.

is worthy of the cost.

REFERENCES [I]

G. McGraw, "Software Security," IEEE Security & Privacy, vol. 2, no. 2, 2004, pp. 80-83.

[2]

B. Potter, G. McGraw, Software security testing, Security and Privacy Magazine, IEEE Volume 2,lssue 5, Sept.-Oct. 2004,pp.81-85.

[3]

Chris Wysopal. Lucas Nelson. Dino Dai Zovi. Elfriede Dustin. The Art of Software Security Testing. Symatec press..

[4]

N.Davis,W.Humphrey, Samuel.T, RedwineJr, GZibulski, G.McGrew. Processes for Producing Security Software. IEEE Security & Privacy. 2004.

[5]

IEEE COMPUTER SOCIETY 1990. Standard glossary of software engineering terminology. ANSIIIEEE Standard 610.12-1990. IEEE Press, New York..

[6]

http//www.OWASP.com

[7]

Noopur Davis. Watts Humphrey. Gerlinde Zibulski. Gray Mcgraw. Processes for producing secure software. IEEE Security& Privacy. 2004.

[8]

Zhanwei. Hui. Research on the techniques of software security testing based on software security defects. Master thesis. PLA University of Science and Technology. 2009.

[9]

Noopur Davis. Watts Humphrey. Gerlinde Zibulski. Gray Mcgraw. Processes for producing secure software. IEEE Security& Privacy. 2004.

4

[10] http//cwe.mitre.org/ [II] https//www.cert.org/. Log On

VI.

Registration

Search

Upload

[12] http://cwe.mitre.org/top25/

Fig. 8. Test result of SSTBS and SSFT

[13] http://www.sans.org/top20/

CONCLUSION AND FUTURE WORK

[14] Zhanwei Hui, Song Huang. A Taxonomy of Software Security Testing for SST. (To be appeared on ICISS 2010)

In this paper, we presented a framework for active security testing model integrated traditional security function testing and SSD-based security test process for security critical software. The test case studies we have been running

[15] G. Sindre and A.L. Opdahl, "Templates for Misuse Case Description", Proc. 7th Inti Workshop on Requirements Engineering, Foundation for Software Quality (REFSQ'200I), Interlaken, Switzerland, 4-5 June 2001.

are not comprehensive enough to give us a basis for making bold statements about the quality of the applications we have

V2-316