2010 3rd
International Conference on Advanced Computer Theory and Engineering(ICACTE)
Software Security Testing Based on Typical SSD:A Case Study Zhanwei Hui
Song Huang
BinHu
Yi Yao
PLA Software Test and
PLA Software Test and
PLA Software Test and
PLA Software Test and
Evaluation Centre for
Evaluation Centre for
Evaluation Centre for
Evaluation Centre for
Military Training
Military Training
Military Training
Military Training
PLA University of Science
PLA University of Science
PLA University of Science
PLA University of Science
and Technology
and Technology
and Technology
and Technology
Nanjing, jiangsu province
Nanjing, jiangsu province
Nanjing, jiangsu province
Nanjing, jiangsu province
PRC
PRC
PRC
PRC
e-mail:
e-mail:
[email protected]
e-mail:
[email protected]
e-mail:
[email protected]
Abstract-Due
to
the
[email protected]
increasing
complexity
of
Web
applications, traditional function security testing ways, which only test and validate software security mechanisms, are becoming ineffective to detect latent software security defects (SSD). The number of reported web application vulnerabilities is increasing dramatically. However, the most of vulnerabilities result from some typical SSD. Based on SSD, this paper presents an effective software security testing (SST) model,
as well as the high degree of the quality required for their critical functioning. To guarantee such respects, we need to generate extensive and effective test suites to evaluate the typical defects. To reach this test completeness, we rely on this paper on Software Security Defects (SSD) -based methods. Surrounding SSD, the software security test would be more effective, and put an end to the typical security
which extends traditional security testing process to defects
defects. Data analyzing can test the intercourse of data, and
behavior analysis which incorporates advantages of traditional
guarantee the security of critical data information. We first
testing method and SSD-based security testing methodology.
describe the process and critical function
Primary applications show the effectiveness of our test model.
software security testing based on SSD. This testing process
Keywords- software security test; function test; vulnerability; software security defect; defect behavior I.
INTRODUCTION
modules of
is well-adapted to different Web systems features such as hyperlinks, sending and receiving data and browser-server communications etc. Second, we specify the system security requirements based on software security defects.
(HEADING 1)
The coming of the internet age should have led developers to collectively acknowledge that all applications need to be
9000
secure [1], but this has not been the case. As is depicted in
8000
the Figure 1,
"Z
lP
� 7000
CERT/CC[II] researches show that the
]
number security accident sharply up to 8078 under 2008. And in 2000, we can see that the number is only 1090.
4129
� 4000
�.
abundant, and the question no longer seems to be "if to test" for security, but rather "when and where to test" and "for what?" Few software engineering would argue against the
V>
value of performing testing of security software, or testing security mechanisms in general software. Software security
345 311 262 'f1.I ./' 0• * -, , __ , .-,
-:--
171 .
1995
1996
1997
1998
1999
2000
crucial for software which usually refer to the critical data information, and these show desires for higher demand to
•
/ / 3780
IO�/
2000
0
is one of the most important software quality properties. It is
-
0!'!ooo. 3784 .
2/4/
3000
1000
78
· 236
5990
6000
� 5000
Software security testing [2] (SST) researches are presently
�
8061
2��a/OO2
2003
2004
2005
2006
2007
2008
Fig.1. CERT/CC security accident statistics
security of software itself. But traditional security test pays more attention to function [3], but not security, and the even major problem is that the software test centers have no
The contributions of this ongoing work is that it applies the 3 new test model to a real software system, which is M TR.
guidance
And after testing, we compare the SSTBS with the SSFT,
standards
or
progresses
for
practical
web
applications security testing.
and the results show the advantages of our new testing
In practice, it is more and more difficult to ensure the
model.
respect of Web applications to their security requirements
This paper is organized as follows: section II presents
[4]. These difficulties are due to the complexity level of
background
such systems, their variety, and their increasing distribution
introduce the concept of SSD,
978-1-4244-6542-2/$26.00 © 2010 IEEE
V2-312
of
SSD-based
security
testing.
We
first
and then analysis the
2010 3rd
inspirations
of
our
International Conforence on Advanced Computer Theory and Engineering(1CACTE)
research.
Section
III
exposes
particular actions, and accountability for actions taken. So a
the
methodology to integrate SSD-based testing process in
SSD could result in software security mechanism incorrect.
software security testing model. In section IV, we briefly
We have tried to keep our use of the term "defect"
present the critical technology of our testing model. Section
intuitive without conflicting with standard terminology. The
V illustrates an ongoing case study: M3TR system security
IEEE
testing. And it also shows the test result of the experiment.
Terminology [5] includes definitions of
Finally, it presents the conclusion and introduces the future
Standard
A.
of
Software
Engineering
•
error: human action that produces an incorrect result (such as software containing a fault),
•
fault: an incorrect step, process, or data definition in
•
failure: the inability of a system or component to
work. II.
Glossary
BACKGROUND
a computer program, and
Different dejinations of SSD
perform its required functions performance requirements.
Generally, a security defect is a part of a program that
within
specified
can cause the system to violate its security requirements.
A failure may be produced when a fault is encountered. This
Finding security defects then, demands some knowledge of
glossary lists bug as a synonym for both error and fault. We
system
use defect as a synonym for bug, hence (in IEEE terms) as a
security
requirements.
These
requirements
vary
according to the software under test and the application, so
synonym for fault, except that we include flaws that have
we cannot address them in detail here. Usually, they concern
been
identification and authentication of users, authorization of
accidental ones.
)
r Software
)
Requirement
--
Soflware Security Defect
I Software . stmcture design
)
I
I
Detailed design
Coding
inserted
I
)
Testing
)
into
a
ImPlementation
system
)
intentionally,
Maintenance
� Software Vulnerability � Software Security Threat � Software Security Accident ----v ----v !------v
as well
as
Software Life Cycle
Software Security Life Cycle
Fig.2. The evolvement of SSD
G.McGraw uses a fairly broad definition of a SSD in the
CERT/CC [18] and researches of G.McGraw [4], most
report [7], which is that Software security defects are any
software vulnerabilities arise from common causes (such as
conditions or circumstances that can result in denial of
SSD), and the top 10 cause account for about 75percent of
service, unauthorized disclosure, unauthorized destruction
all software vulnerabilities; more than 90 percent are caused by known SSD types. Figure 3 shows that the typical
of data, or unauthorized modification of data. Our notion of SSD corresponds to that of a fault, with
vulnerabilities produced major security accidents in 2006.
the possibility that the fault may be introduced either
30.00%
accidentally or maliciously. Software security defect will be
25.00%
different during software life cycle.As is depicted in the
20.00%
Figure 2, the SSD may be induced during the phase of
15.00%
requirement, or structure design, or detailed design, or
10.00%
coding, or even testing. And the Life Cycle of the SSD is
5.00%
likely to Software Life Cycle.
0.00%
A software security defect is a security flaw in software
s� 'iji,i
which can result in a security policy violation. Software
��
u·
vulnerability is a kind of SSD representation as software function. The initial test bug report is a kind of software vulnerability.
As they
exist
in
software
system,
III � � �
i
" �� ij
I
�� �
Fig.3. 2006 OWASP Top-10 Vulnerability [6]
then
malicious users would make use of them to authorization •
operation. And software security accident will happen. B.
Top vulnerabilities list projects
Many organizations and research centers presents the
The feasibility of software seucurity test baased on SSD
Top vulnerability list every year. CWE has maintained a list
Typical SSD induce most of software vulnerabilities
of the top 25 dangerous programming errors [12]. OWASP
•
Security vulnerabilities result from defects that are
(Open Web Application Security Project) as an open
unintentionally introduced into software during design and
organization, it releases its Top-to projects every year.
development. According to a preliminary analysis by the
SANA (SysAdmin, Audit, and Security) institute put out its
V2-313
2010 3rd
Top_10
network
International Conference on Advanced Computer Theory and Engineering(ICACTE)
dangerous
vulnerabilities
from
2000,
The integration model is twofold, which is compatible with
together with NIPC (National Instructure Protection Center).
different SSD. On the left, it is the traditional function
And hereafter, it extends to Top_20, which is repaired by
security testing process, which tests software security
many security researchers every year.
mechanisms, and produces function test cases. Then on the
We summarize several top_10 lists, and give the Top 10
other side, it is the testing process based on SSD, which is
list from 2006 to 2009 in Table 1. Each of the errors or
also called adverse testing. At the end of the model, this
vulnerabilities on their list is described in detail, which has
integrated process will generate a new kind of test cases
enabled us to associate the listed errors with underlying
which not only validate software security policies, but also
software security defects.
provide the user with certain confidence that the SUT could resist typical security attacks.
Table 1. 2006-2009 Top_IO List of Software Vulnerabilities
------Al
A2
A3
2006
2007
Cross
Site
AS
Scripting
Scripting
(XSS)
(XSS)
Injection
Injection
Injection
Flaws
Flaws
Flaws
Insecure
Insecure
Insecure
Remote
File
Buffer Overflows
Direct Object
A8
Leakage
and
Improper
Remote
Injection Cross (XSS) Broken File
Direct Object
Direct Object
Reference
Reference
References
Cross
Management Insecure
Site
Request
Request
Forgery
Forgery
Forgery
(CSRF)
(CSRF)
In formation
Information
Leakage
Leakage
and
Broken
and
Authenticatio
Authenticatio
n and Session
n and Session
Management
Management
Failure
to
Restrict URL Access
Insecure
Insecure
Unvalidated
Cryptographi
Redirects and
c Storage
c Storage
Forwards
Insecure
Insecure
Insecure
Communicati
Communicati
Cryptographi
ons
ons
c Storage
to
Failure
Yes
Misconfigurat
Cryptographi
Failure
c
mappings;
Input
to
request
Restrict URL
Restrict URL
forgery
Access
Access
III.
dalaba
Security ion No direct
Un validated
Cross-site
Site
(CSRF)
Improper
Improper
Broken
session
Cross
Request
Error
and
n and Session
Direct Object
Handling
authentication
Authenticatio
Insecure
Error
Storage
Site
Scripting
Insecure
Handling
Insecure
also illustrate the critical technologies in details in Figure 4.
Include
Site
CTRITICAL TECHNOLOGY OF THE TEST MODEL
In order to specifY how our test model is executed, we
2009 Site
Include
Handling
management
AIO
File
Error
Broken
A9
Remote
Cross
Insecure
Infonnation
A7
Cross
Scripting
Reference
A6
2008 Site
(XSS)
Include
A4
Cross
IV.
Decompose
[)ecompose
Insufficient Transport Layer Protection
SOFTWARE SECURITY TESTING BASED ON SSD
In this section we propose a model to build an integrated
Fig. 4. Integrated software security test model based on SSD
process suitable for software security testing. As traditional security testing based on function test, such as white box testing or black testing, our model not only integrates
A.
security function testing methods, but also put out an effective security testing way, which is based on SSD. An overview of how our testing model is executed is presented in Figure 2.
Typical SSD classification module Typical
SSD
classification
includes
correct
and
appropriate definition of SSD, classification of SSD and typical SSD list. For the definition and classification of SSD are the preparation for typical SSD list and even database.
V2-314
Our
defects
list
is
based
on
the
opened
2010 3rd
International Conference on Advanced Computer Theory and Engineering(ICACTE)
vulnerabilities database. But the SSD is the underline cause
application,
of vulnerabilities. Although there are a great deal of
multimedia recourses. In our case study, we only consider
vulnerabilities, but the defects is numerable. We classifY the
function and typical SSD to generate test instances.
SSD not only form SSD induced vulnerabilities, but also
r--II
-
from their critical rank, which is important for most critical
provide
infon:l3tion
To reach this aim, in the attack points we specifY which
���:������arCh�ng
automatically
Scarch
the
methodology inspired from
paths
[3].
using
a
dedicated
____
and distributed character of web applications, the defects
Deleting/
list is usually very huge. Therefore, it is important to
!Cedi fying request
classifY the SSD. We bring forward with a PRI metric
[14].
list with executed PRJ is generated.
SSD test behavior formalization module description of adverse behavior and graphic
5.
A.
As we get the primary test requirements, we use a kind of Security Use Cases to describe function behavior, which is related to software security mechanisms. And then, a new Adversarial Security Use Case method is proposed for adverse behavior description, which is based on
[15].
generate test use case instance by hand, but tedious work. And it also needs experience. We provide with three to generate test instances, together with
coverage criteria, which not only cover with security mechanisms and latent security defects, but also traditional repair mechanisms. Security Test
CK I Up l oad 110
ACK
� r i'-' ��''''' f'--'!J.!LI!.:'""C·.ill----' "."Aro+CK i: "i'-----
---:-__-( Data management
____
t::=-
---'----'--
� :� I
o������n�g ....
-==-r---
______
MTR Analysises The
MTR
system is a typical B/S web application. Its
main function data flow graph can be seen in Figure
3.
An
identification module is also taken into account in the first step. Thus, a potential user can connect to the Web Then, he/she can
request for a visit to download the audio or video resource
For every piece of test behavior requirement, it is easy to
Behavior ofSoflware
'IIith 'altcd pass'll rd
�
application using a dedicated VRL.
D. Software security test cases generation module
[8]
Query for usern l:ne
Fig. 6. Level-O diagram of M3TR web application
This module includes description of security function description of test behavior, which is depicted in Figure
: t t 3·�' �h"�"'fud,�r,���
Sear�h reques I nfonna;l�i;O" scarch � ���ffi�� �r; ion
Ollcdatc log rc�ucst
Uploading request
l ["",r",ooo"d,,,,,oo,,d,,>
After then, SSD testing
:::= = =1#�:::::f :;-1
=t'� CK ;;;;::==t� Uploading ....i.1.h... I.>loait � .j.!s
L
table with execute-tend paths. Considering the large-scale
userinfo
-�
request
Then according to the paths,
defects test requirements are generated, which is a SSD
algorithm, which is based on
l
Uscrinfo database .. "',,' "r,
path exist latent defects, we have to define algorithms to
algorithms
,
(user IdcntitY
:
of
,
:: i --\ V,[id,,'oo J+ Rot",=;: ,,! t-Y:-:'CK:-+
Y---1':-: ",-,t-''
kinds
:Qucry password !�alt· f�r llser
ldenti ty
the interaction path of SSD in web application under test.
detect
with
L1,o[-g �,-, t,-a� W-' �--�T � Return salt:
SSD detection and classification module
behavior,
users
Log in Dataintcrfacc II---'=-:'''-!-l�(�,UscrlD I �- -
After typical SSD list have generated, it should detects
C.
the
�
web applications. B.
which
during a specific period. Further, we generalized some typical latent SSD list to test the system security. These security defects are inspired from out security test model. Then the security mechanism and the SSD are formally specified using graphic and test behavior model. Their integration will be performed using our algorithms to lead to a secure and understandable specification. Our experiment is executed after traditional function
Reverse Behavior or Software SecurilyTest
test,
and
exposes
8
SSD
of
typical
Web
applications. B.
Experiments result analysis We compare our test model with traditional security
function test in 7 different aspects. For the number of security mechanisms, traditional SSFT StturityTcsl
To
prove
effectiveness
testifY
which
7
come
kinds
of
from
the
access
security
security
test
requirements. But SSTBS could overcome this restriction by typical SSD, which are the experience of different test engineering of security testing. So it produces
3 CASE STUDY: M TR SECURITY TESTING the
only
requirements. So it could not exceed the original security
Fig.5. SSD based software security test process
V.
could
mechanisms,
Function Behavior of Software
of our
security
framework, we carry out a case-study on a
19
kinds of
typical security mechanisms related to typical SSD for the testing
MTR [8]
web
software under test. And then, SSFT could not provide any guarantee for avoidance of SSD, but SSTBS could verifY
V2-315
2010 3rd
International Conference on Advanced Computer Theory and Engineering(ICACTE)
potential SSD, which would increase
tested. However, we believe the results we found is a good
the quality of the software. Most of all, SSTBS integrate
indication that Integrated SSD-based security testing model indeed can be effective process for critical software. By
the software with
19
security function requirements with adverse requirements and potential tolerance mechanisms. So the test requirement is more comprehensive than the security test requirement of SSFT. At last, our test model generate
19
security test cases,
much more than the test cases of traditional SSFT, which only have
8
security test cases. Figure 7 show the number
of security test cases of SSTBS and SSFT. As can be seen
running relatively few tests we managed to discover several bugs (related to SSD), and some potential bugs which were not investigated fully. The primary application displays the advantages of our new security testing model. And more practical testing applying is needed for improving of the details. And we believe that our new testing method will be effective.
from the graph, for every function module, our method produce more test cases. Although it means more time and testing work, the SSTBS could detect much more SSD than the SSFT, which is vividly depicted in the Figure
8.
Anymore, our test cases are also with test priority, which is important for test execution. Although more test cases means more test cost, for software security quality, I think it
ACKNOWLEDGMENT
This work was supported by National High Technology Research and Development Program of China (No:
2009AAO1Z402).
Resources of the PLA Software Test and
Evaluation Centre for Military Training were used in this research.
is worthy of the cost.
REFERENCES [I]
G. McGraw, "Software Security," IEEE Security & Privacy, vol. 2, no. 2, 2004, pp. 80-83.
[2]
B. Potter, G. McGraw, Software security testing, Security and Privacy Magazine, IEEE Volume 2,lssue 5, Sept.-Oct. 2004,pp.81-85.
[3]
Chris Wysopal. Lucas Nelson. Dino Dai Zovi. Elfriede Dustin. The Art of Software Security Testing. Symatec press..
[4]
N.Davis,W.Humphrey, Samuel.T, RedwineJr, GZibulski, G.McGrew. Processes for Producing Security Software. IEEE Security & Privacy. 2004.
[5]
IEEE COMPUTER SOCIETY 1990. Standard glossary of software engineering terminology. ANSIIIEEE Standard 610.12-1990. IEEE Press, New York..
[6]
http//www.OWASP.com
[7]
Noopur Davis. Watts Humphrey. Gerlinde Zibulski. Gray Mcgraw. Processes for producing secure software. IEEE Security& Privacy. 2004.
[8]
Zhanwei. Hui. Research on the techniques of software security testing based on software security defects. Master thesis. PLA University of Science and Technology. 2009.
[9]
Noopur Davis. Watts Humphrey. Gerlinde Zibulski. Gray Mcgraw. Processes for producing secure software. IEEE Security& Privacy. 2004.
4
[10] http//cwe.mitre.org/ [II] https//www.cert.org/. Log On
VI.
Registration
Search
Upload
[12] http://cwe.mitre.org/top25/
Fig. 8. Test result of SSTBS and SSFT
[13] http://www.sans.org/top20/
CONCLUSION AND FUTURE WORK
[14] Zhanwei Hui, Song Huang. A Taxonomy of Software Security Testing for SST. (To be appeared on ICISS 2010)
In this paper, we presented a framework for active security testing model integrated traditional security function testing and SSD-based security test process for security critical software. The test case studies we have been running
[15] G. Sindre and A.L. Opdahl, "Templates for Misuse Case Description", Proc. 7th Inti Workshop on Requirements Engineering, Foundation for Software Quality (REFSQ'200I), Interlaken, Switzerland, 4-5 June 2001.
are not comprehensive enough to give us a basis for making bold statements about the quality of the applications we have
V2-316