TESSI: A Web Service Testing Tool Dessislava Petrova-Antonova, Sylvia Ilieva IICT, BAS Sofia, Bulgaria
[email protected],
[email protected] Abstract— Testing of Service-Oriented Architecture (SOA) implementations brings various challenges at infrastructure, service as well as orchestration level. In order to address these challenges the TASSA framework was developed for testing of web service orchestrations. The paper is focused on one of the TASSA framework tools, named TESSI (TESting of Service Implementations), providing test case generation, execution and management. The core functionality of the TESSI tool includes generation of SOAP request templates, definition of assertions at HTTP, SOAP and BPEL variable levels, support of data driven testing, test execution and management. Its first prototype is implemented as an open source plug-in for NetBeans IDE. Keywords— test case; web service testing; WSDL
I.
INTRODUCTION
Testing of Service-Oriented Systems (SOS) is considerably more challenging compared to the testing of the conventional software systems, which are not componentized and distributed. Due to dynamic and distributed nature of their building blocks, namely web services, it is difficult to simulate all possible configurations and loads during the testing process. A summary of the key testing challenges from web service perspective is presented by Morris et al. in [6]. Following the current research challenges, this paper proposes a tool for web services, called TESSI. The tool can be used for testing both single and composite web services, described with WS-BPEL. When the web service under test is a composite one, the TESSI tool has features for definition of assertions on BPEL variables. Thus, it facilitates testers in data flow analysis helping in location of possible faults and provides a grey-box testing technique of composite web services. The TESSI tool is implemented as open source solution within a research project, called TASSA [7]. The aim of the project is to provide a methodology and a corresponding framework for end-to-end testing of SOA implementations, described with WS-BPEL. Therefore, the proposed tool is a research prototype of the core TASSA framework’s functionality for generation, execution and management of test cases. The paper is organized as follows. Section II summarizes the related work. Section III outlines the TESSI tool. A feedback from sample usage of the TESSI tool is presented in Section IV. Section V concludes the paper. II.
RELATED WORK
The idea of web service testing through automatic generation of SOAP messages from WSDL descriptions is not new. However, most of the current software tools support only black-box testing providing only validation of SOAP response
Vera Stoyanova Sofia University “St. Kliment Ohridski”, FMI Sofia, Bulgaria
[email protected] returned from the web service under test. As web services become complex, including orchestration of other ones, more rigorous testing is needed. Therefore in order to be more effective, the functional testing of web services should use a kind of grey-box technique, which will provide additional information about the possible faults in the web service implementation. Due to space limitation, just a few widely used software tools for WSDL-based testing are mentioned, namely SoapUI [1], PushToTest [2], SOAtest [3] and Soap Sonar [4]. Our solution covers the idea behind them and provides additional functionality for monitoring data flow execution of the composite web services, described with WSBPEL. More about existing approaches about SOA testing can be found in an extensive survey [5]. III.
TESSI TOOL
TESSI tool is a solution for WSDL-based testing of both single and composite web services. The tasks that it carries out are (1) identification of web service operations as well BPEL variables in case of business process testing; (2) generation of SOAP request templates; (3) definition of assertions at HTTP, SOAP and BPEL variable level; (4) execution of test cases including SOAP messages sending and receiving; (5) collection of test outputs for result analysis. The TESSI tool is built from nine modules. The Test Model Definition module implements the main interfaces and classes of the domain model such as TestCase, TestSuitea and BpelOperation. The Test Definition Engine module manages test cases and test suites. The Persistence Engine module performs serialization and deserialization of test cases. The SOAP Request Generator module handles the generation of template SOAP requests based on a selected operation from WSDL file. The Settings module is responsible for the tool configuration related to connection timeout, request charset, request content type, proxy server address and port, address of OpenESB server and port, user name and password. The Test Execution Engine module performs execution of test cases. It provides functionality for subscription to events related to the test case execution. The Template Test Processor module provides support for generation data driven tests. The Assertion Manager module implements different types of assertions, described in the next section, and provides GUI for definition and modification of an assertion in a test case. The NetBeans plug-in packages the TESSI tool in a NetBeans plug-in. Each test case of TESSI tool is stored in a separate XML file. The root element of the test case has two attributes – “name”, defining the test case name, and “template”, specifying whether the test case is data driven. The element
“description” provides narrative description of the test case. The element “operation” specifies the WSDL operation under test including service name, port name and operation name. The element “testInput” stores the input data described with elements “body”, “header” and “authorization”. The “datasource” element defines the data source in case of datadriven testing. The last element “assertions” describes the test case assertions. The provided types of assertions are as follows. The HTTP Status Code verifies the status code of the HTTP response. The Response Time verifies that the response is received in a given time period. The SOAP Fault verifies that the HTTP response contains a valid SOAP fault message. The SOAP response valid verifies that the HTTP response contains a valid SOAP message. The XPath Equals provide validation according to a certain XPath expression applied to a SOAP body or a BPEL variable. It is possible to make it case insensitive and/or match it with regular expression. The XPath Exists verifies if a certain XPath expression applied to a SOAP body or a BPEL variable is not empty. The Contains verifies that a certain string exists within a SOAP body or a BPEL variable. In addition, Negated assertion applicable to all assertions described above is provided. The execution of test case includes two steps: (1) sending of SOAP request to the proper web service address defined in the WSDL file of the business process, and (2) receiving and assessment of obtained result according to the assertions defined in the test case. On the first step the information stored in the test case is used to retrieve the web service address, where the SOAP request should be sent and the message parts such as HTTP headers, user name and password if HTTP authentication is needed, and data of SOAP request body. The obtained result from test case execution consists of HTTP headers of the received response, data of SOAP response body, values of process variables and execution time. The actual data of the SOAP request body and the expected data of the SOAP response body are replaced with variables when data driven tests are executed. The possible test statuses are “success” (the test assertion is satisfied), “failure” (the test assertions is not satisfied), “error” (the assertion is not verified) and “unknown” (the assertion is not checked). IV.
A SAMPLE APPLICATION
To prove the feasibility of TESSI tool, we used it for testing a sample web service, providing operations for summation, multiplication, subtraction, division and square of integer and double numbers. Eight test cases are defined, showing all types of assertions presented in Section III. The first three test cases validate the DivideOfTwoDouble operation of the web service, where the first parameter takes a valid double value and the second parameter takes zero, invalid string and valid double value, respectively for each test case. The next two test cases validate the SquareOfAnInteger operation, when it receives as input parameter an invalid double and valid integer value. The last two test cases validate SubstractionOfTwoInteger operation, where the first parameter takes a valid integer value and the second parameter takes invalid string and valid integer value for the corresponding test cases.
The experiment results show the ability of the TESSI tool to facilitate testing of web services by supporting variety test case assertions. The first and the fifth test cases have failed tests. The first test case checks the behavior of the DivideOfTwoDouble operation, when division by zero occurred. The results from the tests show that this situation is handled in the operation, since SOAP fault doesn’t exist in the web service’s response message. That is why the second test using SOAP Fault assertion is failed. All tests in the fifth test case, except one validating the Response time, are failed. This is due to the fact that the operation SquareOfAnInteger for which the test case is intended, accept as input parameter both integer and double values. This is incorrect from web service implementation point of view, since the last has a special operation for square of double numbers. As was already mentioned, the TESSI tool can be applied also for testing composite web services, described with BPEL. In such case, it provides assertions for testing of BPEL variables – a feature, which is not presented in SoapUI. Additionally, the experimental results confirm the usage of the TESSI tool for testing of non-functional properties of web services, for example Response time. V.
CONCLUSION
The presented TESSI testing tool provides functionality for identification of web service operations as well as BPEL variables in case of business process testing, generation of SOAP request templates, data driven testing, definition of assertions at different levels, execution and management of test cases. As a part of TASSA framework, it contributes to achieve end-to-end testing of web service orchestrations. The further improvements of the TESSI tool include implementation of additional types of assertions and a GUI allowing using it as stand-alone application. In addition, the effectiveness of the proposed solution will be evaluated in comparison with other existing ones. ACKNOWLEDGMENT This work is supported by the National Scientific Fund of BMEY under Research Project, agreement n. DOO2-182. REFERENCES [1] [2] [3] [4]
[5] [6]
[7]
soapUI v.2.5.1, Project Documentation, eviware, 2009 http://www. whiteboxqa.com/ StudentMaterial/Books/D%20-%20soapui.pdf PushToTest, http://www.pushtotest.com/. “Parasoft SOAtest best practices,” http://www. jameslewiscoleman.info/ jlc_stuff/project_research/SOAtest_Best_Practices.pdf. D. Esposito, “SOAP Sonar measures the distance between your web services and the real world,” http://www.codeproject.com/Articles/ 15182/SOAPSonar-Measures-the-Distance-Between-Your-Web-S. G. Canfora and M. Di Penta. “Service-oriented architectures testing: a survey,” In Proc. of ISSSE, volume 5413 of LNCS, 2008, pp. 78-105. E. Morris, W. Anderson, S. Bala, D. Carney, J. Morley, P. Place and S. Simanta. Testing in service-oriented environments. Technical Report CMU/SEI-2010-TR-011, Carnegie Mellon University, 2010 V. Pavlov, B. Borisov, S. Ilieva and D. Petrova-Antonova, “Framework for testing service compositions,” Int. Symp. on Symbolic and Numeric Algorithms for Scientific Computing, Romania, September 2010, pp. 557-560.
