A Specification-based Testing Framework for Web Service-based Software Chengying Mao School of Software, Jiangxi University of Finance and Economics, 330013 Nanchang, China
[email protected]
Abstract As a rapidly emerging technology, Web services offer a brand-new mechanism for program-to-program interactions over the Internet. Inevitably, this programming pattern brings great challenge to the improvement of the quality and reliability of Web services. In the paper, a two-level testing framework for WSBS is proposed through deeply analyzing the architecture of WSBS and black-box character of Web service unit. In service unit level, combinatorial testing method is used to ensure single service’s reliability through extracting interface information from WSDL file. In system level, BPEL specification is converted into state diagram at first, and then state transitionbased test cases generation algorithm is presented. In addition, the feasibility and effectiveness of our approach is validated by some examples. Keywords: Web services, test cases, state diagram, combinatorial testing, WSDL, BPEL
1. Introduction With the wide application of software technology in business activities, business organizations require software system can deliver flexibility by allowing business resources both inside and outside an organization to work together. One way to achieve integration throughout a business is to adopt serviceoriented architecture (SOA) [1]. SOA is the architectural strategy of building applications based on services. Web service is a particular set of standardized technologies that can be used to create an SOA. Although Web services technology [2,3] can bring quite a few benefits for constructing a complex business application, it also causes testing difficulty for Web service-based software (WSBS for short). Services are generally considered to have a component appearance from the perspective of a service consumer (requester). Therefore, He or she can’t acquire the
implementation details but only interface information. The phenomenon of code separation makes it quite difficult to test WSBS, especially for structural testing activities. While considering this reason, a rational way is to perform functional testing on such type software system. A service unit is one that implements a very specific function and obeys a series of standard specifications. Consequently, the specification-based testing method [4] should be a good choice. The paper considers the testing problem from two levels: service unit level and service interaction level. In Web service unit level, the interface information is extracted from specification such as WSDL or OWL-S file, and then the combinatorial testing (CT) [5] strategy is adopted to generate test cases. In service interaction level, dynamic behaviors described by BPEL are converted into a state diagram, and then test suite generation algorithm is addressed through an applied case. The remainder of this paper is organized as follows. In Section 2, we introduce the overall framework of specification-based testing. The implementation detail of adopting combinatorial testing to validate single Web service unit is described in Section 3. In Section 4, we discuss the state-based testing method for ensuring correct interactions between Web services in the system level. Section 5 concludes the paper.
2. Overall testing framework A Web service is defined by the W3C as “a software system designed to support interoperable machine-to-machine interaction over a network” [2]. It can also be treated as an internet-based application fulfilling a specific task or a set of tasks, which can be combined with other Web services to maintain workflow or business transactions [6]. In fact, Web service-based software is a heterogeneous, loose coupling and message-oriented component-based system. Service consumers compose Web services together according to their specific business workflow. Because the service units are from different providers,
the quality of the composed system is their primary concern. From the perspective of service providers, source code of service unit is available, so they can perform strict structural testing on each unit. However, it’s impossible to consider all kinds of application scenarios on the side of service provider. So it’s necessary to reconsider the testing problem for service requester. On the side of Web service users, they will hold dubitable attitude to the external provided component. Therefore, WSBS developers (i.e. Web service users) will re-test according to Web service’s specifications in the real application environment in unit level (shown in Figure 1). Because the source code is invisible for service user, testing methods on WSBS developer side are basically limited to black-box testing strategies, such as random testing, decision table, and boundary value analysis. In the paper, we attempt to adopt combinatorial testing into service unit testing (i.e. bottom level). In the system level, the whole WSBS is composed via several service units and works smoothly on the direct of workflow specifications. As expressed in the following figure, we will perform state-based testing for the system composed by services units.
construct combinatorial testing model for Web service unit based on such information. According to the abstract model, testers can generate test cases by some heuristic searching algorithms. Subsequently, they can treat test cases as input to execute the Web service unit under test and collect test results. According to testing experiences, 2-way (or pair-wise) testing method is a cost-effective and wide-used strategy in practice. In general, test results will be too large to be analyzed in manual style. Some existing rule mining algorithms (e.g. rough set-based reasoning [8]) can be utilized to discover some useful rules for latter debugging activity. Example 1. Suppose there is a Web service unit for querying computer price at present, and can be denoted as Web Service: PriceQuery(M,V,S). The first input parameter represents manufacturer who can produces computer, the second stands for the variety of computer product, and the last is the flag of screen size. The possible value can be assigned to input parameter of service unit as expressed in following table. Table 1. Possible values for input parameters of PriceQuery service unit No.
Factor
Levels
f1
manufacturer
Lenovo, TCL, Dell
f2
variety
home PC (HPC), business computer (BC), notebook (NB)
f3
size
15 inch, 14 inch, 12 inch
Although 27 test cases are needed for exhaustive combination testing of this service, some heuristic strategies can be adopted to reduce test suite size. In [7], three methods, i.e. orthogonal tables (arrays) [9], AETG-based method and “greedy + backtracking” method, are proposed to produce test cases. When considering the subject Web service, only 9 test cases as shown in Table 2 are generated via the above three methods. Table 2. Pair-wise test cases for the example service unit PriceQuery
Figure 1. Overall testing framework for the Web service users (i.e. WSBS developers)
3. Combinatorial testing in unit level In our previous work, the solution of utilizing combinatorial testing method to validate Web services’ quality has been proposed [7]. At first, testers can extract interface information about Web service according to WSDL or SOAP file. Then they can
No.
Manu.
Variety
Size
Results
tc1 tc2 tc3 tc4 tc5 tc6 tc7 tc8 tc9
Lenovo Lenovo Lenovo TCL TCL TCL Dell Dell Dell
HPC BC NB HPC BC NB HPC BC NB
15 inch 12 inch 14 inch 12 inch 14 inch 15 inch 14 inch 15 inch 12 inch
Succ. E1 Succ. E1 Succ. Succ. Succ. Succ. Succ.
Note: Succ. represents Web service can run successfully with corresponding test case, and E1 is some type of exception.
4. State-based testing in system level When software developers construct a Web servicebased application, they have to integrate several Web service units together. BPEL is the most popular one and becomes a de facto standard in the industrial field. A BPEL process specification is a kind of flow-chart, and its principal part is called activity. Example 2. The following example considers a simple loan approval web service that provides a port where customers can send their requests for loans [10,11]. Figure 2 is the schematic illustration of the example taken from the section on structured activities of the BPEL 1.1 specification. For the space consideration, the requirement specification is omitted here. As for the details, please refer to reference [10] and [11].
According to the above conversion rules, the BPEL specification can be converted into state diagram as shown in Figure 3. T1: null S1 T2: [request/amount =10000] / null
T4: [riskAssessment/ risk!='low'] / null
T5: [riskAssessment/ risk='low'] / assign
S3 T6: null
S4 T7: null
Figure 3. State diagram for the loan approval example In order to facilitate test cases generation, we transform state diagram into a spanning tree through a breadth-first search (BFS) process. For the example, the corresponding state transition tree (see Figure 4) can be deduced from the above state diagram.
Figure 2. Loan approval process described by BPEL In this section, we will convert BPEL process into state diagram and then perform state-based testing on system. The conversion can be implemented according to the following steps. Step 1: For a given BPEL process under test, construct a start and end state respectively. Step 2: For the diamond nodes of branch or joint, convert them into some state nodes in state diagram. Step 3: The service operation between two diamond nodes in BPEL specification can be converted into transition edge. The branch judgment is converted into the guarded condition in state diagram. Therefore, the transition edge in state chart can be denoted as “ [C ]/ Action ” , where C is the guarded condition (i.e. choose condition in BPEL specification), and Action is the corresponding Web service operation. Step 4: When there is no Web service operation between two diamond nodes, we can construct null transition edge only with guarded condition between these nodes.
Figure 4. State transition tree for the loan approval example After getting the state transition tree, test cases can be generated according to the following rule: Given a STT, each path from start node to leaf node constructs a test case which consists of a series of methods. While considering the loan approval example, three test cases expressed in Table 3 can be produced. In fact, each test case is a method invoking sequence in which methods are connected via an arrow symbol. Then, WSBS developers can build a test program which invokes the methods in test case orderly. If some test program fails with some specific test case, they can easily locate which Web services have the interoperation problem with their partners.
Table 3. Test cases for the subject Web service application at system level No. tc1 tc2 tc3
Test Case T1: null → T2: [request/amount
