Towards Risk–Driven Security Testing of Service Centric Systems Philipp Zech Institute of Computer Science University of Innsbruck Innsbruck, Austria Email:
[email protected]
Michael Felderer Institute of Computer Science University of Innsbruck Innsbruck, Austria Email:
[email protected]
Abstract—The increased deployment of service centric systems in security critical application contexts poses new challenges to properly test such a system’s security. If taking a closer look at the inherent complexity of such applications, sophisticated approaches to testing security are indispensable. In our paper we propose a novel model–based methodology for the risk–driven security testing of service centric systems. Keywords-Service Centric Systems; Information Security; Risk Analysis; Model–Based Testing; Security Testing; Cloud;
I. I NTRODUCTION Ensuring securely running software systems is one of the most tedious and time–consuming tasks during the Software Development Life Cycle (SDLC). Due to the increased outsourcing of both, IT landscapes and applications, and the trend towards ubiquitous services computing, security is of major importance. Enabling such dynamic and distributed service centric systems (SCS), providing computing as a service, requires complex software systems intrinsically prone to security flaws due to their complexity and openness. Yet, security flaws are not always introduced into such complex systems due to improper coding, already at the modelling level, erroneous decisions can lead to crucial security flaws in the resulting system. Service tests are performed on the one hand side by service providers when the services or their infrastructure are developed and maintained. On the other hand, customers test their process integration when consuming services. Modelbased approaches are particularly promising since these tests involve different technologies and a high degree of evolution. However, so far, this potential has not been unlocked [1]. What makes security testing significantly different from traditional software testing is that it emphasizes what an application should not do rather than what it should do [2]. This fact was also pointed out by Alexander [3], where the author distinguishes between positive and negative requirements which express what a system should do and not do. For testing positive security requirements, classical model– based testing techniques can be applied [4]. For testing negative requirements, which is an infinite set on principle, risk–based testing techniques prioritizing tests based on assigned risks [5], [6] and penetration testing where one simulates hacker attacks incorporating in–depth knowledge on vulnerabilities [7], [8] have successfully been applied. We introduce a novel security testing method for service centric systems. Our approach is model–based to provide as much independence from frequently changing technologies
Ruth Breu Institute of Computer Science University of Innsbruck Innsbruck, Austria Email:
[email protected]
as possible, to support the integration of various service technologies, i.e., cloud platforms, and to support continuous testing. In addition, our approach is risk and knowledge based to solve the problem of negative testing. Based on a semi–automatic risk analysis and a vulnerability knowledge base, a misuse case model is generated. The misuse case model is interpreted by a security test engine equipped with a generating test data fuzzer by evaluating a system’s security by runtime model interpretation. Besides, our approach supports live tracing of the test execution allowing to feed back test execution information into the misuse case model at runtime. The remainder of our paper is structured as follows. Section II outlines our security testing framework. Sections III and IV fall into line by portraying our approach in detail. Section V concludes with a statement of work and future research tasks. II. A R ISK –D RIVEN M ETHODOLOGY FOR S ECURITY T ESTING System Model Design
Misuse Case Model Generation
SUT System Model Design
Risk Model
SUT Model
Risk Analysis
Misuse Case Generation
Misuse Case Model
Vulnerability Knowledge Database
Model Reviewing
Consistency Checking
Reviewed Misusecase Model
SUT Implemetation Validated Misusecase Model
Test Execution WSDL Files System Implementation Test Data Fuzzing
SUT Stub Generation
Test Execution
Testlog
Test Feedback Generation
Figure 1.
Risk–Driven Security Testing Methodology
To address the problem of defining a near complete security test suite, we define our security testing approach upon the results of a pivotal risk analysis. Figure 1 shows that, after the design of the system model of the SUT is done, we perform a risk analysis on the system model of the SUT.
After the successful generation of the risk model from the SUT’s system model, the misuse case model is generated out of the risk model. The idea of the misuse case generation (see Figure 1, center of upper right activity block) is to derive possible abuse cases out of the identified risks. The misuse case model contains abstract and formal descriptions of the test cases. A test designer has the possibility to manually refine the misuse case model. Allowing such a manual refinement gives a test designer the chance to intervene in the automated process and fine tune the model for custom testing scenarios (see Section III). Finally, after performing manual fine tuning of test cases in the misuse case model, the consistency of the model is checked, to assure that the contained test cases are sound and complete in respect to our underlying metamodel. Both of the above described model–to–model transformations use the vulnerability knowledge base during execution. The purpose of this knowledge base is to provide attack specific knowledge, necessary while analyzing the model and generating the misuse cases (see Section III-A). As a last step, the misuse case model is executed against the SUT. Thereby, the fuzzer provides the necessary test data for each test case. We directly execute the model, as this results in improved traceability and test feedback inside the models. III. S ECURITY M ODELING As explained in Section II, our testing methodology employs three different kinds of models, viz. • System Model A domain–specific language (DSL) based model, describing the service interfaces of the SUT. • Risk Model A tailored, automatically generated model, designed to contain descriptions of risks for a specific SUT according to a given system model. • Misuse Case Model A tailored, automatically generated test model, designed to contain formal descriptions of executable test (misuse) cases according to a given risk model. As the model of the SUT, also the risk and misuse case model will be based on a DSL. This is due to that textual modeling languages are more intuitive but at the same time, also more expressive than classic modeling languages, like i.e., UML. In the following, after a moment’s discussion of our Vulnerability Knowledge Base, each of the three models and its purpose will be explained in detail. A. Vulnerability Knowledge Base The vulnerability knowledge base (VKB) plays a central role in our approach, as it is the main driving force for the following test sequence and the overall test plan. The VKB is implemented using logic programming, more detailed, Answer Set Programming (ASP) [9]. ASP itself is based on Disjunctive Datalog [10], an extension of Datalog, allowing disjunctions in the head of a rule. Like any other logic programming language and system, ASP is based on the syntax of facts and rules. Listing 1 depicts sample content of
the VKB and a problem instance. The upper part of Listing 1 depicts the Extensional and Intensional Database (EDB and IDB), necessary for each ASP program. Put another way, the EDB states what is already known (facts, as well as rules), whereas the IDB states what can be derived and solely employs rules. The lower part of Listing 1 depicts a concrete problem instance (based on the System Model from Listing 2), as given as input to the depicted ASP program (lines 1–14). The VKB is maintained by experts in a first instance. The knowledge of the VKB is based both on expert knowledge itself and historic incidents, as i.e., available from the Common Vulnerability and Exposures database (CVE) [11]. For performing the risk analysis an a given input model, the model is transformed into a valid ASP problem specification by using a model–to–text transformation. The resulting stable model of the risk analysis then again is transformed into a textual model by using a model–to–model transformation. xmlinjection . sqlinjection . attack medium ( x m l i n j e c t i o n , message ) . attack variation ( xmlinjection , tag ) . attack variation ( xmlinjection , metacharachter ) . a t t a c k l a n g u a g e ( x m l i n j e c t i o n , xml ) . a f f e c t e d B y ( x m l i n j e c t i o n , O, I ) :− o p e r a t i o n (O, I ) . a f f e c t e d B y ( s q l i n j e c t i o n , O, I ) :− o p e r a t i o n (O, I ) , h a s S t r i n g P a r a m e t e r s (O ) . h a s S t r i n g P a r a m e t e r s (O) :− p a r a m e t e r (X, O) , p a r a m e t e r T y p e (X, s t r i n g ) . :− a f f e c t e d B y ( X M L I n j e c t i o n , O, ), −p a r a m e t e r ( , O ) . interface ( interface1 ). o p e r a t i o n ( op1 , i n t e r f a c e 1 ) . p a r a m e t e r ( a , op1 ) . p a r a m e t e r ( b , op1 ) . parameterType ( a , i n t e g e r ) . parameterType ( b , s t r i n g ) . r e t u r n T y p e ( op1 , v o i d ) . o p e r a t i o n ( op2 , i n t e r f a c e 1 ) . r e t u r n T y p e ( op2 , i n t e g e r ) . Listing 1. Sample Content of the Vulnerability Knowledge Base, depicting both, the Extensional and the Intentional Database (lines 1–14) and a concrete Problem Instance (lines 16–24)
B. System Modeling Listing 2 shows a sample system model. It is based on a custom DSL, allowing to describe SCS in a textual manner. Using familiar notions like Service, Operation, and Parameter the DSL allows to describe a service interface from a black box point of view. Besides providing concepts for modeling service operations, we also provide conceptual features for service related information like endpoint, package, implementation language or service engine. Figure 1 shows that the system model of the SUT acts as the main input for our testing methodology. At a first glance, referring to a system model containing interface description might sound poor. It is exactly this kind of information (in terms of WSDL descriptions) attackers employ to successfully perform their tailored attacks. C. Risk Modeling The risk analysis starts by scanning the system model and its interfaces, including the provided operations, for
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
suitable attack patterns. Upon an attack pattern may suit on an operation, the very operation is blacklisted for detailed inspection later on during the risk analysis. This blacklisting of operations is a very crucial step, as the resulting set is what our risk analysis effectively works on. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
S e r v i c e [ name Adder ] { e n d p o i n t h t t p : / / 1 9 2 . 1 5 6 . 3 4 5 . 2 3 : 8 0 8 0 / Adder package add mapped−t o i m p l . add language Java e n g i n e Jax−WS O p e r a t i o n [ name a d d I n t ] { r e t u r n s i n t mapped−t o java . lang . Integer Parameter [ name a ] { t y p e i n t mapped−t o java . lang . Integer } Parameter [ name b ] { t y p e i n t mapped−t o java . lang . Integer } } O p e r a t i o n [ name a d d F o a t ] { r e t u r n s f l o a t mapped−t o java . lang . Float Parameter [ name a ] { t y p e f l o a t mapped−t o java . lang . Float } Parameter [ name b ] { t y p e f l o a t mapped−t o java . lang . Float } } } Listing 2.
Sample System Model of a Basic Calculator Service
The computed risk consists of the factors impact, probability, and risk level. The basis for the determination of these factors is the calculation of a complexity factor c for each operation op(p1 : t1 , . . . , pn : tn ), depending on the input parameter types t1 , . . . , tn . The more complex the set of parameters types is, the higher is the overall complexity c. The automatically determined complexity of a parameter type ti considers whether a type is primitive or structured and is denoted by c(ti ), the common dependence factor between t1 , . . . , tn is denoted by d. The overall complexity c of an operation op with the operation signature op(p1 : t1 , . . . , pn : tn ) is then the sum of the parameter type complexities c(ti ) and the dependence factor d(t1 , . . . , tn ), i.e., c(op) =
n X
c(ti ) + d(t1 , . . . , tn )
i=1
Based on the number of parameters and their complexity factor, the impact is calculated, i.e., a higher number of parameters plus a higher complexity leads to a higher impact and vice–versa. This reasoning is also motivated by the circumstance that the complexity factor directly relates to the attack vector. The attack vector defines the means by which a malicious agent may gain access to a system. In the case of SCS, the attack vector is mostly based on the signature of service operations.
The probability of a risk is calculated by employing stored knowledge of the vulnerability knowledge base. Based on the number of exploits, we calculate the probability by again incorporating the complexity factor. However, in the case of the probability a higher complexity does not lead to a higher probability, instead it decreases the probability of a successful attack. Having calculated the impact of a risk and also its possibility, we calculate the risk level. This is done by combining, both the impact and the probability, to infer the risk level, based on a predefined scheme (see Table I).
Probability
LOW MEDIUM HIGH
Impact LOW LOW LOW HIGH
MEDIUM LOW MEDIUM HIGH
HIGH MEDIUM HIGH HIGH
Table I L OOK – UP TABLE USED TO CALCULATE THE Risk Level
Risk [ i d rTwD Adder addInt ] { impact MEDIUM p r o b a b i l i t y HIGH r i s k l e v e l HIGH Profile { d e s c r i p t i o n ”A m a l i c i o u s u s e r injects invalid tags into a request .” e x p l o i t XML I n j e c t i o n goal Tampering with Data } } Listing 3.
Sample Risk generated from the interface depicted in listing 2
As a last step during the risk analysis, for each identified risk a distinct risk profile is generated by the VKB. This risk profile contains, besides a textual description of the risk, information regarding the exploits, launched with the associated operation, and the destined goal of the attack. Listing 3 shows one of the risks, generated from the system model depicted in Listing 2. As can be seen from this profile, there is a possible threat of Tampering with Data if a malicious agent applies an XML Injection (the exploit) onto a message request. The affected operation is linked to the risk via its unique ID (rTwD Adder addInt). D. Misuse Case Modeling Realized as a model–to–model transformation from the risk model into the misuse case model, each concrete risk, contained in the risk model is processed apart. Listing 4 depicts one of the test cases, generated out of the risk pictured in Listing 3, i.e., a test case describing a possible attack with the intent of Tampering with Data abusing features of the XML language, i.e., injecting XML tags to corrupt the message. Lines 1–4 show basic features of the test case, viz. the objective to test, the priority of the test case, concurrency related information in case of the threaded feature and the number of executions. Starting from line 6, the Context of the test case is defined. The context holds a description of the necessary test data formats (fuzz language and fuzz feature) as well as the fuzz context. At the time
1 2 3 4 5 6 7 8 9 10 11 12
we support three different fuzz contexts, namely Data, if only simple data types need to be fuzzed, Message, if a message body needs to be fuzzed, and, Packet, if complete data packets to be sent via HTTP need to be fuzzed. Starting from line 11, each test case contains at least one Assertion. In the case of Listing 4 an assertion has been generated, where the expected result should be data. This would indicate, that the SUT processed an invalid message without any exceptions and hence, exhibits a vulnerability. 1 2 3 4 5 6 7 8 9 10 11 12 13 14
T e s t c a s e [ id tTwD Adder addInt ] { o b j e c t i v e rTwD Adder addInt priority Favorable threaded 1 executions 5 Context { f u z z c o n t e x t Message f u z z l a n g u a g e ”XML” f u z z f e a t u r e ” Tag ” } Assertion { r e s u l t := data ; } } Listing 4. Listing 3
Sample Misuse Case generated from the Risk depicted in
IV. S ECURITY T ESTING We skip the generation of executable test code but interpret the misuse case model directly. Doing so results in two major advantages, viz.: • Full traceability support inside the test (misuse case) model during test execution. • Interpreting the model and not generating test code, allows us to define target language and hence, also platform independent test suites. The text execution is quite straightforward. Upon a valid misuse case model is retrieved, each test case is executed either sequential or concurrent. In both cases, the order of operations remains the same, viz., request test data, invoke the operation on the SUT and evaluate the outcome. After completion of the execution, test feedback is annotated back into the model and used to improve the fuzzer and the VKB. V. C ONCLUSION In our paper we presented a novel methodology for risk–driven model–based security testing for service centric systems. We believe that the presented approach is seminal in various aspects, viz.: • It allows to define target platform and target language independent test suites. • It is both, expandable and generic by adapting or exchanging the contents of the VKB. • It addresses the concept of security testing from a negative perspective. It does not assure a system’s validity at any time but eagerly attempts to show its deficiency. At the time we are in the course of performing experiments to identify critical areas of improvement, whereas by now the most important among them are: • The VKB, both about representation of content and the actual content itself.
The risk analysis by extending its attack pattern matching capabilities and, as a result also its detection capabilities. • Enabling change–driven testing by attaching the necessary state machines to the models and incorporating change events as suggested by Breu [12]. Although we are still working with a prototypical implementation of our testing approach, it has already shown its potential in the case of error detection. We are convinced that our approach will be supreme to existing penetration and fuzz testing techniques, as in contradiction to existing work in that area, we effectively incorporate security and system related knowledge during the risk analysis and also during test case generation. •
ACKNOWLEDGMENT This research was funded by the Austrian Science Fund (FWF): P17380, MATE. R EFERENCES [1] G. Canfora and M. D. Penta, “Testing Services and ServiceCentric Systems: Challenges and Opportunities,” IT Professional, vol. 8, pp. 10–17, 2006. [2] M. C. C. and R. Will, “Risk–Based and Functional Security Testing,” Cigital, Tech. Rep., 2009, https://buildsecurityin.uscert.gov/bsi/articles/best-practices/testing/255-BSI.pdf [accessed: May 8, 2010]. [3] I. Alexander, “Misuse cases: Use cases with hostile intent,” Software, IEEE, vol. 20, no. 1, pp. 58–66, 2002. [4] M. Felderer, B. Agreiter, and R. Breu, “Security Testing by Telling TestStories,” in Modellierung 2010. Gesellschaft fuer Informatik, 2010, pp. 195–202. [5] S. Amland, “Risk-based testing: : Risk analysis fundamentals and metrics for software testing including a financial application case study,” Journal of Systems and Software, vol. 53, no. 3, pp. 287–295, 2000. [6] H. Stallbaum, A. Metzger, and K. Pohl, “An automated technique for risk-based test case generation and prioritization,” in Proceedings of the 3rd international workshop on Automation of software test. ACM, 2008, pp. 67–70. [7] M. Bishop, “About Penetration Testing,” IEEE Security & Privacy, vol. 5, no. 6, 2007. [8] M. Vieira, N. Antunes, and H. Madeira, “Using web security scanners to detect vulnerabilities in web services,” in DSN. IEEE, 2009, pp. 566–571. [9] G. Brewka, T. Eiter, and M. Truszczy´nski, “Answer set programming at a glance,” Communications of the ACM, vol. 54, no. 12, pp. 92–103, 2011. [10] T. Eiter, G. Gottlob, and H. Mannila, “Disjunctive datalog,” ACM Transactions on Database Systems (TODS), vol. 22, no. 3, pp. 364–418, 1997. [11] “Common Vulnerabilities https://cve.mitre.org/.
and
Exposures,”
[12] R. Breu, “Ten Principles for Living Models - A Manifesto of Change-Driven Software Engineering,” in CISIS. IEEE Computer Society, 2010, pp. 1–8.