Turning web services descriptions into quickcheck ... - Semantic Scholar

Turning Web Services Descriptions into QuickCheck Models for Automatic Testing Miguel A. Francisco

Mac´ıas L´opez

Interoud S.L. (Spain) [email protected]

Henrique Ferreiro

University of A Coru˜na (Spain) macias.lopez [email protected] laura.castro

Abstract

There are multiple ways for describing these interactions, one of the most commonly used being WSDL (Web Services Description Language) [30], an XML-based language to specify the operations offered by a web service. The WSDL standard operates at the syntactic level and there is no way to represent the requirements or operational constraints of the web service just using WSDL information. Thus, in order to add semantic information to web service description, the WSDL description must be completed with semantic annotations expressing preconditions, postconditions, and the effect that a given request produces. A number of choices have been proposed to do this, such as WSDL-S (Web Services Semantics) [31], SWRL (Semantic Web Rule Language) [29], or OCL (Object Constraint Language) [25]. To ensure the quality of a web service [16], we need to guarantee that the operations work as their specification require, this is, that the semantic information is not violated. Based on previous work [19] which used UML descriptions together with OCL properties to perform automatic testing of software components, we propose to apply property-based testing (PBT) [15] to do perform automatic testing of web services. When using PBT, testers have to write properties that the system under test (SUT) needs to satisfy, rather than specific test cases. From the properties description, tools can later on produce the specific test cases automatically. Using this technique, we can derive a black-box model which describes the functional properties of the SUT and use it for testing purposes. In particular, given a WSDL description of a web service and its OCL semantic definition, we can automatically generate a QuickCheck model. The investment required to provide the semantics of a given web service in OCL, alongside with its syntactic WSDL description, is capitalised in the form of the effortless generation and execution of great amounts of test cases, automatically generated from such specification. Thus, we contribute:

In this work, we face the problem of generating good quality test suites and test cases for web services. We present a framework to test web services based on their formal description, following a black-box approach and using Property-Based Testing. Web services are a popular solution to integrate components when building a software system, or to allow communication between a system and third-party users, providing a flexible, reusable mechanism to access its functionalities. Testing of web services is a key activity: we need to verify their behaviour and ensure their quality as much as possible, as efficiently as possible. By automatically deriving QuickCheck models from its WSDL description and its OCL semantic constraints, we enable generation and execution of great amounts of automatically generated test cases. Thus, we avoid the usual compromise between effort and cost, which too often leads to smaller and less exhaustive test suites than desirable. To illustrate the advantages of our framework, we present an industrial case study: a distributed system which serves media contents customers’ TV screens. Categories and Subject Descriptors D.2.4 [Software Engineering]: Software/Program Verification; D.2.5 [Software Engineering]: Testing and Debugging General Terms

reliability; verification; web services

Keywords web services; model-based testing; Erlang; QuickCheck; WSDL; OCL

1.

Laura M. Castro

Introduction

As the Internet increases in importance in our society, the need to provide access to different kind of systems across the web has become more critical. The usual way to do this is through web services, which aim to provide a means for interaction between software systems, or between software systems and final users over the network.

• A methodology for the automatic generation of QuickCheck

models for testing web services that uses their WSDL syntactic description, their OCL semantic definition, and applies to both stateless and stateful web services. • An Erlang-based supporting framework for the proposed method-

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. Erlang ’13, September 28, 2013, Boston, MA, USA. c 2013 ACM 978-1-4503-2385-7/13/09. . . $15.00. Copyright http://dx.doi.org/10.1145/2505305.2505306

ology, which includes an implementation of OCL operators, QuickCheck generators for OCL types, convenience wrappers for web requests to the web service. In this paper we explain how we have developed our proposal into the implementing framework to test web services. We start by

79

coverage criteria and some heuristics. This tool is also used in [12], where information about how clients should interact with the web service is inferred. The closest previous research to our proposal is that of [22] where semantic information is provided in SWRL [29] instead of OCL. Finally, [20] presents one of the few fully automatic approaches, where a PBT tool is used, but no semantic information is provided, so the testing remains at the syntactic level.

describing the characteristics of WSDL descriptions and the different kind of testing required for different web services depending on their semantics in Section 2. Our methodology and the architecture of our framework is detailed in Section 3. As case study, we have used VoDKATV, an IPTV/OTT middleware to provide end-users an advanced media experience like accessing to video-on-demand services or browsing the Internet, which is described in Section 4. A discussion on the benefits and caveats of our proposal, as well as the analysis of other approaches, is presented in Section 5.

2.3

2.

Background

2.1

WSDL, WSDL-S and OCL

A web service is a well-defined and modular piece of software that offers one API over the network, typically, the Internet. Applications use web services by sending and receiving messages written in a standardised format, usually XML [32] or JSON [14]. Thanks to the use of standard formats and HTTP communications, web services enable the co-operation of software systems regardless of their technology specifics, as well as the integration of components in large systems. WSDL is an XML-based language for defining web services in terms of operations, inputs, outputs, messages, data types as well as specific protocols for accessing the service. With little effort we can transform WSDL specifications to an implementation in almost every programming language to make calls to a web service, and vice-versa. Some of the most popular transformation tools are Apache CXF [5] or Apache Axis2 [4] for Java, and we can find them even embedded in well-known IDEs. WSDL is thus considered widely accepted in industry as the standard for describing web services. Semantic web services are those that have been enriched with a semantic description of their behaviour. Semantic descriptions are expressed with ontologies, a formal conceptualisation of a particular domain. The description is created using components of that domain and expressing their relationships. WSDL-S aims to add this type of information to web services, extending the WSDL specification language and allowing automatic search, discovery, selection, composition and integration across heterogeneous users and domains. We use the modelReference attribute that WSDL-S adds for annotation purposes; this attribute defines the association between a WSDL entity and a (external) concept (describing its semantics). Using this, we maintain WSDL and OCL descriptions separately, so changes to one would not affect the other. We use OCL to express these concepts. OCL is a formal language used for representing expressions on UML models. These expressions can be invariant conditions that must hold for the system being modelled, or queries over objects described in a model. One important aspect of OCL constraints is that they do not have side effects: evaluating an OCL expression will not change the state of a model. In this work we use OCL to describe the preconditions and postconditions of the operations included in a web service WSDL specification. 2.2

Property-Based Testing with QuickCheck

As an alternative to manually producing tests from a high-level natural-language specification, or to writing a formal model to describe a system or component, PBT uses declarative statements to specify properties that the software needs to satisfy according to its specification. Using this approach, test cases can then be generated from those properties, a process that can be automated, allowing to run many tests for each written property. In our work, we have used QuickCheck, a PBT tool that automates generation, execution and evaluation of test cases. QuickCheck was developed for Haskell [13] by Claessen and Hughes and re-designed, improved, and commercialised by Quviq AB for Erlang [6]. Quviq QuickCheck1 provides random test case generation and execution, allowing the developer or tester to run a lot of tests with very little effort, checking whether the defined properties hold or not. For testing complex systems, however, isolated properties are not expressive nor powerful enough. For such scenarios, instead of sequences of independent test cases, we want to test sequences of calls which modify the state of the service, checking that some conditions hold before and after each interaction, and that the global state of the service remains coherent with its expected behaviour call after call. Cloud storage systems that are accessed through web services are examples of this type of behaviour.

3.

Test approach: from WSDL+OCL to properties

The requirements of a system represent the needs that it must fulfil, and they are usually specified in an abstract way, without technical details. As we want to use PBT to test the web services behind the WSDL specification, we need to have the appropriate properties which describe the requirements of the SUT. To do that, we get information both from the WSDL and the OCL constraints, and the combination of both allows us to build our test model, composed by properties. Depending on the requirements of the web service, the test model can differ: for stateless web services, universally quantified properties are generated; for stateful ones, the requirements are modelled into a state machine. Either from the properties or from the state machine modelling the web service, QuickCheck derives the specific test cases, and then, using an HTTP adaptor (generated from the WSDL specification), we feed the SUT. Thus, using our framework, the testers do not need to know any specific details about web services implementation languages. In addition, if the same API is preserved, different implementations of a web service can be tested with the same test properties. The general architecture of our framework is shown in Figure 1. First of all, we need to retrieve information from the WSDL file, so a WSDL parser has been developed. We decided to implement our own because we need to integrate the semantic information provided as OCL constraints, writing it as an easy-to-manage structure to transform into properties. Thus, we implemented a WSDL parser using Haskell and the Text.XML.Light library, returning a proplist-like structure suitable to our needs.

Testing web services

Most of the research carried out about web services testing focuses on methods to create good test suites, from a black-box perspective. Examples of this are [9], where authors use a technique called Orthogonal Array Testing (OAT), the pair-wise method defined in [23], a partition testing technique in [10]. Some proposals do build intermediate artefacts to help in the process, such as [34], who builds a finite automaton using BPEL [24], in [28] who uses UML+OCL. A complete framework to perform testing using a WSDL specification is explained in [11], where test cases are generated using a

1 From

80

now on, we refer to Quviq QuickCheck simply as ‘QuickCheck’.



SUT

WSDL file

OCL constraints

WSDL parser

OCL parser



we need to extract: Type information (data + services)

• the name of the web service from the service tag;

Semantic information (services)

• the name of each operation from the operation tag, contained

in interface; • the name of input and output tags for each operation; QuickCheck properties/model

• the types referred by each input and output elements; • the endpoint and the operation from the binding tag to get

the URL; • the modelReference attribute from WSDL-S pointing to the test cases

OCL file.

adapter

Then, we have to parse the referred OCL file and check if there is semantic information associated to any of the operations retrieved in the second item before. As in the previous case, we found several tools to parse OCL files, but in all cases the parsing functionality has to be executed associating a UML model to the OCL. This led us to develop our own OCL parser, taking advantage from the work made by the OCLNL project [2]: a labelled BNF grammar [18] for OCL. This grammar was fed to the BNF compiler (BNFC) [1] to produce the abstract syntax tree, lexer and parser which we used. For instance, in the WSDL above a powConstraints file was referred. Its content could be:

Figure 1. Proposed testing architecture For instance, from an extract of a WSDL file for a calculatorlike web service like this:

context MathUtils::pow(a: Integer, b: UnlimitedNatural): Integer post: result = Sequence{1..b}-> iterate(i : Integer; acc : Integer = 1 | acc*a)

...

Finally, when the required information from the WSDL and OCL file was retrieved, it is time to build the properties from which QuickCheck will generate test cases. 3.1

Stateless web services

Stateless services or systems do not have an internal state that affects the outcome of a sequence of calls to their API, so the response returned by a specific call is independent of the specific moment when it is executed. In this case, the name of the operation to be tested and the type of its result and arguments is parsed from the WSDL file; in turn, the test oracle is built out of the constraints specified in the OCL file. The WSDL in the previous section describes a MathUtils service offering a pow operation with a postcondition specified in an OCL file. With that information, the following property is generated:



prop_pow() -> begin ?FORALL({A, B}, {ocl_gen:int(), ocl_gen:nat()}, begin mathUtils:pow(A, B) ==

81

state transitions where, at each state, both pre- and postconditions are checked [7]. Our case study, explained in the next section (c.f. Sect. 4), falls in this category of stateful web services.

ocl_seq:iterate(fun (I, Acc) -> Acc * A end, 1, ocl_seq:new(1, B)) end) end.

QuickCheck can now run this property and produce specific test cases; this way, instead of specifying input data manually, data generators are used to generate data of the corresponding data type. The module ocl gen is a data generator library. This approach leads to a significant improvement over traditional tests [27] and most of the research mentioned in Sect. 2.2, since instead of specific values, we define types, ranges, and conditions that the input data has to meet, which are then produced automatically instead of manually listed. So, for each pair of an integer and a natural number that is generated, they are used as parameters for the mathUtils:pow/2 function, i.e., the HTTP adaptor responsible for making a call to the web service under test:

Case study: VoDKATV

4.1

VoDKATV architecture

VoDKATV is an IPTV/OTT middleware that provides end-users access to different services on a TV screen, tablet, smartphone, PC, etc., allowing an advanced multi-screen media experience. VoDKATV is a distributed system composed by several components, which are integrated through web services (cf. Fig. 2). The main part of the VoDKATV system is the core package, that contains the key components of the system. The core components use backend components to get information from external systems, for example the EPG of IPTV channels (provided by the EPGServer component) or multimedia contents to rent in a video on demand catalogue (provided by the AssetManager component). On the other hand, the core components are used by the clients, i.e., the applications that end-users use to access the system on a TV screen, smartphone or any other compatible device. Finally, the architecture also includes administration applications to manage and configure the system (admin package). We have used VoDKATV as a case study for our approach, in particular, we have used the web service provided by the VoDKATV-server component which returns data in XML format. As shown in the Fig. 2, this web service is used by administration components to configure the VoDKATV system.

pow(Arg1, Arg2) -> inets:start(), Req = httpc:request("http://localhost:8080/" ++ "mathUtils/pow?a=" ++ Arg1 ++ "&b=" ++ Arg2), case Req of {ok, {{_Version, 200, _Reason}, _Headers, Body}} -> list_to_float(Body); _ -> "Unknown error" end.

where the URL of the web service is retrieved from the WSDL file. The value returned by the web service is finally checked against the postcondition specified in the OCL. With QuickCheck we can not only generate a large amount of specific test cases derived from properties and executed against the real SUT. Another very interesting QuickCheck feature is that, when a failing test case is found, the tool automatically shrinks it to the smallest equivalent counterexample it can find, making it easier to understand the reason of the failing case [33], and thus improving also the debugging process. Following the example above, we run the prop pow property and we obtain this output:

4.2

Test approach in action

VoDKATV stores information about the users and devices that can access the system. Devices are identified by a MAC address, and they are associated to a household (room, in VoDKATV nomenclature). Thus, when a new user is registered in the VoDKATV system, a new household must be created and the devices of that user must be registered to that household. Hence, the web service chosen as a case study offers, among others, operations to create, modify, update and delete households and devices. For example, the operation used to create a new household is specified in WSDL as:

> Testing property: pow ..................................Failed! After 52 tests. {-13,16} Shrinking..(2 times) {13,15}



meaning tests execution reveals an inconsistency between property and web service real behaviour. In this case, the fault is the wellknown float rounding error, as explained in [8]. 3.2

4.

where createRoomParams specifies the parameters received by the web service (roomId and description):

Stateful web services

In opposition to stateless components, in which each action is independent of each other, many systems have a behaviour that depends on which actions were previously performed. In this case, isolated properties that test the operations provided by the web service are not useful, because the order in which the operations are executed may modify the result of the test results, i.e., the web service is composed by operation with side effects. In order to test these systems, the internal state has to be taken into account in the test process. QuickCheck has support for testing this kind of systems by using state machines. Instead of specifying general properties, the state machine behaviour is specified by defining an initial state and a state transition function. Additionally preconditions and postconditions are used to verify properties in each step. The generated tests cases consist in random sequences of



and createRoomResponse is the response returned by the web service:

82

Figure 2. VoDKATV Architecture



required Required code • if the specified household identifier (roomId) already ex-

ists in the VoDKATV system, the web service must return a duplicated error;



duplicated Duplicated



• otherwise, the household must be created, and its identifier

(roomId) and description (description) must be returned by the web service. ROOM_ID DESCRIPTION



This specification is written using OCL preconditions and postconditions. Unlike the MathUtils example (cf. Sect. 3), which is a stateless component (cf. Sect. 3.1), here we need to know which households were created in order to know (and test) if the VoDKATV system must return an error or the household should be created, To do that, we defined an auxiliary attribute, defined on the OCL specification, to store the households which are expected to

Our approach requires to specify the behaviour of the web service so that the test cases can be generated automatically. The specification of the CreateRoom operation is: • if the specified household identifier (roomId) is empty, the web

service must return a required error;

83

be created in the VoDKATV system. this is, we deal with a stateful web service (cf. Sect. 3.2).

postcondition(PreState, AfterState, {call, ?MODULE, createRoom, [RoomId, Description]}, R) -> case RoomId of "" -> Result = ocl_datatypes:get_property(errors, R), ocl_seq:eq(AfterState#ts.rooms, PreState#ts.rooms) andalso ocl_string:eq( ocl_datatypes:get_property(code, R))), "required") end; ...

context VoDKATVInterface def: state_rooms:Sequence(Tuple(roomId:String, description:String)) = Sequence{}

We write the OCL postconditions using state_rooms as a internal test state information. In addition, the complexType and element entities defined on the WSDL can be accessed as UML classes in the OCL code, where the element elements defined for each entity will be the attributes of those UML classes. With this approach, the specification for the CreateRoom operation can be written in OCL with the following code:

where ocl_seq, ocl_datatypes, etc. are ancillary Erlang modules that implement OCL operations, such as size, at, etc., respecting its syntax. In other works, they are our own Erlang implementations of the OCL operators. Therefore, as a result, we have a QuickCheck test model automatically generated by our tool from the WSDL and the OCL constraints. This test model checks that the web service described by the WSDL satisfies the constraints specified with OCL.

context VoDKATVInterface::CreateRoom( roomId:String, description:String): CreateRoomResponse post CreateRoom: if ((roomId = ’’) or (roomId = null)) then (self.state_rooms = self.state_rooms@pre and result.errors->size() = 1 and result.errors->at(0).code = ’required’) else if (self.state_rooms->select( room | room.roomId = roomId)->notEmpty()) then (self.state_rooms = self.state_rooms@pre and result.errors->size() = 1 and result.errors->at(0).code = ’duplicated’) else self.state_rooms = self.state_rooms@pre->including( Tuple { roomId:String = roomId, description:String = description }) and result.roomId = roomId and result.description = description endif endif

4.3

Error reports

QuickCheck generates specific test cases from the generated test model, i.e., random sequences of commands with random parameter values that satisfy the preconditions. As a second step, QuickCheck executes these commands, invoking the corresponding operations of the web service, and checks if the SUT fulfils the postconditions. To illustrate what happens when an error is found, consider the simple situation in which households with duplicated identifiers are created in the system and no errors are returned when we try to create a new household with an already existing household identifier. When we run the QuickCheck state machine to test the buggy web service, QuickCheck returns the following error (after shrinking):

This OCL specification, together with the WSDL, is used by our framework to generate QuickCheck properties. To do that, we use the same approach described in [19], but using WSDL and OCL to generate QuickCheck code. In this case, as we are using a state variable (state_rooms) and the @pre operator, a state machine representation will be generated. QuickCheck state machines require the specification of the following elements:

Shrinking..........(10 times) [{set,{var,3}, {call,test_sut,createRoom,["A",""]}}, {set,{var,4}, {call,test_sut,createRoom,["A",""]}}]

Thus, QuickCheck finds a smaller trace of operations that cause the same error in our model, i.e., create two households with the same identifier. The second create operation does not return a duplicated error as it is specified in the OCL. Therefore, the postcondition CreateRoom specified in the OCL is not satisfied.

• the commands to execute (i.e., the web service operations to

test); • the internal state, that stores the information used at some post-

5.

conditions with the @pre operator (in this case, state_rooms);

Discussion

This paper presents a black-box testing approach to check the correct behaviour of web services. This approach is based on the use of OCL constraints to specify how a web service, described through WSDL, must behave. The framework currently covers a wide set of the OCL features specified in the current OCL specification (2.3.1), which we have found most useful and used. We plan to extend our OCL parser to handle all of them, as a necessary step towards the release of the framework itself, alongside with other results of the PROWESS research project [3]. Thanks to the usage of the framework in our industrial case study, we consider its status to be relatively mature. Some important components have been reused, adapted and/or re-factored from our previous work in generating QuickCheck properties and test modules from UML models and OCL constraints [19]. Specifically, the OCL parser has been completely reused, enhanced with further coverage of the OCL specification. One could argue whether the

• the initial value for the internal state, which must be obtained

from the SUT; • the precondition for each operation, which are based on the

defined OCL preconditions; • the postcondition for each operation, which are generated from

the corresponding OCL postcondition and global invariant (if any); • the state update with the execution of each operation.

As we mentioned already, QuickCheck uses the HTTP adapter (cf. Sect. 3), which is generated by our tool from the WSDL. Thus, when an operation is executed, the corresponding web service operation will be invoked, and the result is analysed by the corresponding postcondition. For example, this is part of the code generated to check that a required error is returned when the household identifier is empty:

84

ful web services; we plan to extend our framework to adapt specifically to the intrinsic properties of these web services. Last but not least, we intend to reuse the framework to provide support for properties representing non-functional requirements (such as availability, responsiveness. . . ), usually described in EULA documents, which are specially relevant in the web services domain.

WSDL parser could have been replaced by a WSDL-to-UML conversor instead, since a UML parser was already in place. However, there were two main reasons we did not make that choice: on the one hand, UML is not a widely used standard to represent web services, so the WSDL-to-UML transformation would not have much value in itself. That being so, there was nothing to gain in adding one additional step to the transformation process, and so we chose to extract directly from the WSDL the information we needed, ready to be to merged with the output from our OCL parser. One of the main advantages of our approach is the use a PBT tool like QuickCheck to generate and run the test cases, because it automatically generates complex testing sequences which stresstest the real system in a more objective and efficient way than any human tester could [17, 21, 26]. Furthermore, QuickCheck allows the tester to work at a higher level of abstraction, closer to the specification level: there is no need to think of specific test cases, rather the general behavioural model. Another important aspect of QuickCheck is its shrinking and counterexample capabilities, reducing the sequence of test cases to the minimal expression which causes the same error, and providing such reduced counterexample is a very valuable asset to fault debugging. We are using a standard specification language, OCL, so testers do not only not have to learn a new specification language (Erlang) to write test cases, because properties are fully automatically generated from the OCL specification. This is a clear advantage when testers have a specialised role because a powerful test models can be built with little effort. Another advantage of using PBT and MBT instead of using hand-written tests is that the properties description is independent of the implementation details of the SUT. This means that an evolving code base does not force rewriting the test model, maintaining an intact, updated and executable specification of the SUT. Although we have not found any errors in the web service used as case study (which makes sense, considering the system has been in production for a number of years), we have introduced a number of non-trivial errors, inspired in the developers’ bug reports. All of them where found by the framework within few dozens or at most hundreds of test cases, with QuickCheck returning error reports that were indicative of the source of the problem. Developers acknowledged that, especially in those cases in which the bug reports have taken considerable effort to pin-point, the usefulness of the counterexamples was without question.

6.

Acknowledgments This research has been partially funded by the European Framework Program, FP7-ICT-2011-8 Ref. 317820, and Xunta de Galicia CN2011/007, partially supported by FEDER funds.

References [1] BNFC. http://bnfc.digitalgrammars.com/. [2] OCLNL. http://www.key-project.org/oclnl/. [3] PROWESS Project (Property-based Testing for Web Services). http://www.prowessproject.eu, 2013. [4] Apache. Apache Axis2. http://axis.apache.org/, 2013. [5] Apache. Apache CXF. http://cxf.apache.org/, 2013. [6] T. Arts and J. Hughes. Erlang QuickCheck. Ninth International Erlang/OTP User Conference, 2003. [7] T. Arts, J. Hughes, J. Johansson, and U. Wiger. Testing telecoms software with quviq quickcheck. In 5th ACM SIGPLAN Workshop on Erlang, ERLANG ’06, pages 2–10, 2006. [8] T. Arts, L. M. Castro, and J. Hughes. Testing Erlang data types with Quviq QuickCheck. In 7th ACM SIGPLAN Workshop on Erlang, ERLANG ’08, pages 1–8, 2008. [9] A. Askarunisa, A. Abirami, and S. Mohan. A test case reduction method for semantic based web services. In 2nd International Conference on Computing, Communication and Networking Technologies, ICCCNT 2010, pages 1–7, 2010. [10] X. Bai, S. Lee, W. Tsai, and Y. Chen. Ontology-based test modeling and partition testing of web services. In IEEE International Conference on Web Services, ICWS 2008, pages 465–472, 2008. [11] C. Bartolini, A. Bertolino, E. Marchetti, and A. Polini. WS-TAXI: A WSDL-based testing tool for web services. In 2nd International Conference on Software Testing, Verification, and Validation, ICST 2009, pages 326–335, 2009. [12] A. Bertolino, P. Inverardi, P. Pelliccione, and M. Tivoli. Automatic synthesis of behavior protocols for composable web-services. In Joint 12th European Software Engineering Conference and 17th ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC-FSE’09, pages 141–150, 2009.

Conclusions and future work

In this paper we have presented a test methodology, and supporting framework, to automatically build test models for web services from WSDL descriptions augmented with OCL constraints. The models we built are QuickCheck models and properties. Specifically, using this approach, properties are automatically generated from the web service WSDL specification, and specific test cases are automatically generated and executed using QuickCheck. Since the WSDL description includes mainly syntactic information, we require its semantics to be expressed as OCL constraints. Thanks to the inspection of the different types of OCL constraints, our framework can generate properties for both stateless and stateful web services, using declarative statements in the first case and state machines models in the second. In all cases, the test model produced by the framework can be used as an updated specification of the SUT with the shape of an executable model. In order to further improve the methodology and framework for applying it to other components of the VoDKATV system, we want be able to trace back the conditions that have failed when QuickCheck generates a counterexample, showing the specific piece of OCL code that has produced the error. Furthermore, nowadays a particular kind of web services is most popular: REST-

[13] K. Claessen and J. Hughes. Quickcheck: a lightweight tool for random testing of haskell programs. In Fifth ACM SIGPLAN International Conference on Functional programming, ICFP ’00, pages 268–279, 2000. [14] D. Crockford. The application/json media type for javascript object notation (JSON). http://www.ietf.org/rfc/rfc4627, 2006. [15] J. Derrick, N. Walkinshaw, T. Arts, C. B. Earle, F. Cesarini, L.-A. Fredlund, V. Gulias, J. Hughes, and S. Thompson. Property-based testing: the protest project. In 8th international conference on Formal methods for components and objects, FMCO’09, pages 250–271, 2010. [16] W. Emmerich. Managing web service quality. In 6th international workshop on Software engineering and middleware, SEM ’06, pages 1–1, 2006. [17] G. Fink and M. Bishop. Property-based testing: a new approach to testing for assurance. SIGSOFT Software Engineering Notes, 22(4): 74–80, July 1997. [18] M. Forsberg and A. Ranta. Labelled bnf: a highlevel formalism for defining well-behaved programming languages. In Estonian Academy of Sciences: Physics and Mathematics, 52:356–377, 2003.

85

[26] P. Farrell-Vinay. Managing Software Testing. Auerbach Publishers, 2008.

[19] M. A. Francisco and L. M. Castro. Automatic generation of test models and properties from UML models with OCL constraints. In 12th Workshop on OCL and Textual Modelling, OCL ’12, pages 49– 54, 2012.

[27] A. Petrenko. Why automata models are sexy for testers? (invited talk). Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2007.

[20] K. Lampropoulos, L. Sagonas. Automatic WSDL-guided test case generation for proper testing of web services. pages 3–16, 2012.

[28] J. Timm and G. Gannod. Specifying semantic web service compositions using uml and ocl. In IEEE International Conference on Web Services, ICWS 2007, pages 521–528, 2007.

[21] S. Mouchawrab, L. C. Briand, Y. Labiche, and M. Di Penta. Assessing, comparing, and combining state machine-based testing and structural testing: A series of experiments. IEEE Transactions on Software Engineering, 37(2):161–187, Mar. 2011. [22] S. Noikajana and T. Suwannasart. An approach for web service test case generation based on web service semantics. In International Conference on Semantic Web and Web Services, SWWS 2008, pages 171–177, 2008.

(SWRL).

[30] W3C. Web Services Description http://www.w3.org/TR/wsdl, 2013.

(WSDL).

Language

[31] W3C. Web Services Semantics http://www.w3.org/Submission/WSDL-S/, 2013.

[23] S. Noikajana and T. Suwannasart. An improved test case generation method for web service testing from WSDL-S and OCL with pair-wise testing technique. In International Computer Software and Applications Conference, volume 1, pages 115–123, 2009.

[32] W3C. eXtensible Markup http://www.w3.org/XML/, 2013.

Language

(WSDL-S). (XML).

[33] A. Zeller and R. Hildebrandt. Simplifying and isolating failureinducing input. IEEE Transactions on Software Engineering, 28(2): 183–200, Feb. 2002. ISSN 0098-5589.

[24] OASIS Project. Business Process Execution Language (BPEL). https://www.oasis-open.org/committees/tc home.php?wgabbrev=wsbpel. [25] OMG. Object Constraint Language http://www.omg.org/spec/OCL/2.3.1/, 2013.

[29] W3C. Semantic Web Rule Language http://www.w3.org/Submission/SWRL/, 2013.

[34] Y. Zheng, J. Zhou, and P. Krause. An automatic test case generation framework for web services. Journal of Software, 2(3):64–77, 2007.

(OCL).

86