XML Query Evaluation in Validation and Monitoring ...

1

XML Query Evaluation in Validation and Monitoring of Web Service Interface Contracts Sylvain Hallé University of California, Santa Barbara, USA Roger Villemaire Université du Québec à Montréal, Canada Fig. 1. Communications with a web service are carried through the exchange of XML messages in sequence.

I NTRODUCTION Service-oriented architecture (SOA) has become an important concept in systems development with the advent of web services. Because of their flexible nature, web services can be dynamically discovered and orchestrated together to form value-added, e-Business composite applications. However, this appealing modularity is also the source of one major issue: by dynamically combining heterogeneous, cross-business services, how can one ensure the interaction between each of them proceeds as was intended by their respective providers? Whether for specifying performance or interoperability constraints, business policies or legal guidelines, a good web service has to have a well defined and enforceable interface contract [42]. While not strictly an XML technology, web services depend heavily on XML for both its definition language, the Web Service Description Language (WSDL) [17] and its messaging protocol, the Simple Object Access Protocol (SOAP). Specification languages such as DTD, RELAX NG, Schematron and XML Schema (XSD) have become natural candidates for expressing so-called data contracts, specifying the shape of each message sent or received by a service. However, while data contracts are relatively straightforward to specify and verify, interface contracts go beyond such static requirements and also include a temporal aspect. For example, the online documentation for the popular Amazon E-Commerce Service [1] elicits constraints on possible sequences of operations that must be fulfilled to avoid error messages and transaction failures [31]. Compliance to interface contracts can be checked in two different and complementary ways. First, an interaction can be recorded by some observer into a trace (or log) file, which can then be analysed a posteriori to discover violations: this is called trace validation or log analysis. The same validation can also be performed on-the-fly, intercepting and analysing messages as they are received or sent and warning of violations in real time: this is called runtime monitoring. Although it is widely believed that such interface contracts should, in theory, be verified or monitored in various ways, these tasks are often dismissed in execution environments on the grounds that: 1) they will cause an overhead sacrificing performance; 2) they require the development of new algorithms and the use of new, possibly untrusted software components.

Indeed, we shall see that virtually all proposed technologies rely on custom-made algorithms that abstract away the XML nature of web services messages. The goal of this chapter is to challenge these two beliefs by showing how advanced XML query processing technologies can be leveraged to perform trace validation and runtime monitoring The key idea consists in representing message exchanges as the progressive construction of a global XML “trace” document. Since contracts impose restrictions on the content and sequence of messages, validating and enforcing them simply becomes appropriately querying this document —harnessing the available computing capabilities of readilyavailble XML query engines in web service environments. BACKGROUND In layman’s terms, a web service can be described as an independent software system providing its functionality through the exchange of messages. Generally, a service resides on a remote, public server accessible through the Internet using standard protocols. For example, when using the Simple Object Access Protocol (SOAP) over HTTP, communications to and from a web service are realized through the sending of XML documents to an endpoint URL. Among popular web service providers, we mention Amazon and Google. Figure 1 shows how, for example, on can send a request to the Google map service to obtain the geographical coordinates of a city. An Example: The Online Trading Company Although some web services, like the previous example, require stateless request/response patterns, many web services allow for long-running, stateful transactions spanning multiple messages. Examples of such services abound in the literature; the reader is referred to [24, 31, 39, 29, 2]. Let us consider the case of an online trading company, adapted from [32]. A trading company is responsible for selling and buying stocks for its customers; it exposes its functionalities through a web service interface. An external buyer (which can be a human interfacing through a web portal, or another web service acting on behalf of some customer) can communicate through a set of XML messages, each

2

representing a business operation performed by the trading company. A client service first logs into the system by providing a user name. The shop offers a discount if a user connects with the commitment to buy at least one product, which is signalled with the commitToBuy element. The shop responds to the login with a loginConfirmation, providing a unique ID for the session. Additionally, if the user’s commitment to buy a product has been accepted by the shop, a maximum delay in minutes before which the first transaction must take place is given in the expiration element. At first, a customer can ask for the list of available products. This is done through the exchange of a getAllStocks message, to which the company replies with a stockList message. The customer can then decide to get more information about each stock, such as its price and available quantity, using a getStockDetails message giving the list of stock names on which information is asked. The trading company replies with a stockDetails message listing the information of each stock. Finally, the customer can buy or sell stock products. In the case of a buy, this is done by first placing a placeBuyOrder message of the following form, listing the name and desired amount of each products to be bought. The trading company checks the availability of each product and returns a confirmation of the transaction giving a bill identifier. The last step is for the customer to complete the transaction by proceeding to a cash transfer. This is achieved by providing an account number. The transfer can be done for multiple buy orders at the same time. Alternatively, instead of a cash transfer, a cancelTransaction message listing some bill-ids can be sent to revoke these transactions before payment. A sell operation works similarly. All these operations can be intertwined. At any moment, we suppose that a customer can ask information about particular products, place concurrent buy/sell orders, complete or cancel any number of pending transactions. This general context is appropriate to represent the basic requirements of e-commerce applications.

Interface Contracts In order to interact with the trading company, a service consumer must be informed of how the “conversation” is expected to take place. Even though it is not explicitly documented, a web service always imposes an interface contract. The first obvious aspect of such a contract covers the structure and content of each possible message. To this end, a standardized language called the Web Service Description Language (WSDL) [17] allows a web service developer to ive the list of accepted operations (and hence possible message types) and the shape of each message, using either the DTD or the Schema syntax. For example, it might require that an order request be placed through a placeBuyOrder message following this pattern:

• • • • • •

No cash transfer can be initiated before a buy or sell confirmation has been issued. A user whose commitment to buy has been accepted by the shop will eventually send a placeBuyOrder message. Once a buy/sell order has been placed, no other buy/sell order can be placed until the first one is completed by a payment or a cancellation. Every buy/sell order is eventually completed by a cash transfer or a cancellation referring to the same bill ID. No cash transfer can occur for a buy order that has previously been cancelled. The same unique ID must be present in all confirmation messages.

TABLE I T EMPORAL CONSTRAINTS FOR THE ONLINE TRADING COMPANY ’ S INTERFACE CONTRACT

placeBuyOrder s1 a1 ... sn an The WSDL language, however, was designed for stateless services; the specification for the online trading company, hence, does not prevent messages from being called in any sequence. Yet, as was pointed out in [32], it is well-known that not all message exchanges represent valid transactions, i.e. interactions between services that lead to the successful completion of the cooperation. Conditions ensuring a safe and sound cooperation between services must also include specification of valid message traces. In the case of the trading company, possible contract properties are listed in Table I. It is important to note the varied nature of these properties. Someinvolve simple precedence relations; some state invariants that must be fulfilled at all times. Some go even further: by referring to “every buy and sell order”, constraint 5 requires that each bill ID appearing in a confirmation eventually appears inside a cancellation or a cash transfer. It implies an access to the data content of the message (the bill ID), and moreover correlates data values at two different moments along the message trace. These last choreography specifications are called “data-aware”, because the sequence of messages and their content are interdependent. The reader is referred to [29] for a deeper exposition of data-aware constraints. In many scenarios, additional constraints can complexify the monitoring process: asynchronous communications, lost, delayed or out-of-order messages can distort an otherwise valid interaction, without it being any of the services’ “fault”. In this chapter, a focus has been placed on providing means of detecting that an assumption on the communication has been violated for one of the collaborating services, and not on repairing an invalid transaction or determining the actual

3

cause of the violation. Enforcing Interface Contracts Although expressed in plain English, we shall see that the previous constraints can be formalized in a very large variety of specification languages. However, no matter the chosen language, a question remains: how can one verify that a completed transaction complied to the contract, and how does one enforce this contract at runtime? Several solutions have been proposed, which can be condensed into three categories. Static Analysis.: The first such category involves the static analysis of a model of the actual pair consumer-service before any interaction takes place; this is called model checking. This method provides the highest level of certainty by exhaustively exploring all possible execution paths with the aid of a tool called a model checker. Compliance to contracts by statically verifying the business process has been tried in various works [25, 27, 56, 22]. However, this exhaustiveness comes at a price: they require a specification of the client and service processes in order to check their possible interactions. Moreover, due to the sheer compelxity of the problem, these models abstract away the actual messages by representing them by single letters. Since they require specific tools to be realized, static analysis methods are not considered in this chapter. Trace Validation: A second possibility is to collect a trace of messages for later analysis. During a transaction between web services, several message traces can be obtained: • An external observer can record the global pattern of messages sent by each service and align them in the order in which they are intercepted. This global trace has been called in [14] a conversation. • A service can record the messages that are received or sent from its ports. There exists one such local message trace for each service cooperating in a transaction. This approach is called trace checking, trace validation or log analysis. The idea is roughly to mine such a trace to find violations of contracts [54]. One reason for doing so might to to check accountability: in the case a business contract has been violated, one might be interested to dig the circumstances of that violation, identify the guilty party and engage in compensation measures [? ]. Trace validation has been attempted in [30] where a trace of messages is converted into a transition system producing exactly that trace, contract properties using the SPIN model checker. With Specifications using XQuery on traces (SXQT) [55], a trace of XML messages is analyzed by means of formulæ converted into XQuery expressions. However, the approach covers only request-response patterns of messages, and not properties spanning an arbitrary number of messages. Runtime Monitoring: However, in trace validation, one has to wait for a trace to be complete for the corresponding XML structure to be generated; therefore, this method needs adaptations to be used in a context where the monitoring should occur in parallel with the execution of the workflow. This leads to a third solution called runtime monitoring, where the state of “validity” of a message exchanged is updated at every new message sent or received. Violations can then

be caught immediately, seconds after they have occured, as exemplified by the ASTRO and Dynamo projects [9]. Several arguments in favour of runtime monitoring approaches have been put forward [24]. First, the satisfaction of requirements sometimes depends on assumptions on the partners that cannot be verified prior to the actual implementation of the system. In the particular case of service-oriented architectures, partners can be discovered dynamically and can even change drastically during execution. In some occasions, a static, a priori model checking of the intended process is simply impossible or intractable when modelling channels of potentially infinite length or very large data domains [39]. Finally, there also exist situations at runtime which, although they do not constitute strict violations of a specification, must be addressed as soon as they are discovered: [8] gives the example of an online shop being refused a money transfer by its partner bank, or of a client repeatedly asking for products that are no longer in stock. Independent of these technical aspects, the runtime monitoring of a process is also sound business-wise. [49] remarks that monitoring can increase trust in an electronic marketplace by providing the consumer of a service the ability to check by itself the transaction that takes place. A number of approaches to the runtime monitoring of systems in general have been suggested over the years. Propositional runtime monitoring was used by [23], where patterns of messages exchanged by a web service are specified using UML 2.0 Sequence Diagrams, and then transformed into classical finite-state automata. [37] uses UML Message Sequence Charts with the same intent, using aspect-oriented programming to call monitoring methods with the use Java pointcuts. In [8] an elegant framework for the automatic synthesis of monitors is presented, using the Run-Time Monitor specification Language (RTML). [10] describes an algorithm for rule-based runtime monitoring, where the rules are temporal fixpoint functions that can include data arguments. [53] makes a similar use of data parameterization for a quantified variant of linear temporal logic. Input-Output State Transition Systems (IOSTS) can be used to specify and monitor properties [19]. [24] suggests a framework in which correlations between data in multiple messages are expressed and can be checked at runtime; to the best of our knowledge, the correlations imply a single request-response. Alternate approaches include storing execution traces into a database; contract properties are then converted into SQL queries into that database [6]. Inference on Event Calculus (EC) expressions can also be made to infer contract violations [40]. A C ASE

FOR

XML Q UERY P ROCESSING

Since the message traces considered are formed of XML documents, is is natural to think that properties on such traces can be validated using XML methods. Remark that a message trace σ = σ1 σ2 . . . can be seen as an XML document by itself. It suffices to encapsulate each message sequentially into a global trace “document” as follows:

4

σ1 σ2 ... The sequential ordering of the messages in σ is carried through the children nesting of the elements. This property can be used to map properties on sequences of messages in a trace into a form of query on an XML document using children-axis operands. For example, consider the following expression Q:

XML Q UERY P ROCESSING

FOR

T RACE VALIDATION

We now systematize the approach suggested above by rephrasing it into a formal model. It remains to choose such a formal model. In recent years, the importance of specifying patterns of interaction manifested itself in the development and use of numerous languages to define cooperations between services. Among them, we find Conversation Specifications [14], the Web Service Choreography Description Language (WS-CDL) developed by the W3C [33], the re-use of UML Sequence Diagrams [44] in a web service context [23], SSDL Message Exchange Patterns (MEP) [45], the Let’s Dance choreography description language [20], and the Web Service Choreography Interface (WSCI) [4]. Multi-agent web services are modelled in [56] using a custom protocol language called MAP; [34] models web service compositions by finite-state systems, akin to the conversation specifications of [14].

every $x in /trace/message/message[1] satisfies every $y in $x/message[1] satisfies (not($y/commitAccepted = “true”) or some $z in $y//message[1] satisfies $z/name = “buyOrder” An XQuery engine computing Q on the above document will return true if the second message of the trace does not contain an element commitAccepted with a value of “true”, or otherwise as soon as a buyOrder message is encountered; otherwise, the processor will return “false”. Therefore, the previous XQuery expression returns “true” exactly when a trace fulfills Choreography Specification 1. Based upon that observation, an XQuery processor can be used as a crude web service trace validator: 1) From a constraint, build an XQuery query Q such that Q returns false if and only if the constraint is violated 2) Compute the result of Q with an XQuery engine on a “document” formed by aggregating all exchanged messages with the service We list a few advantages of doing so. First, XML query processors are common. Virtually all existing solutions for trace validation and runtime monitoring require changes to the web service execution environments. In contrast, most (if not all) web service execution environments provide XML processing capabilities natively; hence, translating LTL into XML query evaluation allows to perform trace validation without any new technology. Using XML query engines to perform trace validation is a powerful way of leveraging existing resources available in production environments while minimizing the modifications required to a service. Second, XML query languages are standardized. Although, as we shall see later, there exists a very wide variety of specificationn languages, using XQuery as production environment equipped with an XML processing engin can process these constraints. Finally, XML query processors are efficient. Experimental results shown later in this section indicate that, in contexts where data domains of 100 elements and transactions formed of up to 1,500 messages, the validation of a trace requires less than 10 ms per message.

Formalizing Contract Specifications Although we do not deem realistic to propose XQuery as a specification language for interface contracts, the plethora of existing languages does not make realistic the prospect of providing XQuery translations for each one. To this end, we stop our choice on a logical formalism called Linear Temporal Logic (LTL). LTL has already been suggested for the expression of choreography constraints [43, 22, 48]. We do not advocate LTL as a “superior” language, but do not that most, if not all previous languages can be mapped into equivalent LTL expressions, or extensions thereof; this includes, among others, UML Sequence Diagrams [44] (or more specifically Message Sequence Charts) [15, 23], SSDL’s Message Exchange Patterns (MEP) and Rules protocol frameworks [45], and the Let’s Dance choreography description language [20]. Hence, any result valid for LTL applices to these notations as well. LTL has been introduced to express properties about states and sequences of states in systems called Kripke structures [18]. In the present case, the states to be considered are messages inside a conversation. Formally, let us denote by M the set of XML messages. A sequence of messages σ1 σ2 . . . , where σi ∈ M for every i ≥ 1, is called a message trace. We write σi to denote the i-th message of the trace σ, and σ i to denote the trace obtained from σ by starting at the i-th message. A domain function is used to fetch and compare values inside a message; it receives an argument π representing a path from the root to some element of the message. This path is defined using standard, XPath 1.0 notation. Formally, if we let D be a domain of values, and Π be the set of XPath expressions, the domain function Domm (π) is an application M × Π → 2D which, given a message m ∈ M and a path π ∈ Π, returns a subset of D. For example, if we let Π be the set of XPath formulæ, π ∈ Π be the particular formula “/message/stock-name”, and m ∈ M be the following message:

5

placeBuyOrder stock-1 stock-2 then Domm (π) = {stock-1, stock-2}. LTL’s syntax is based on classical propositional logic, using the connectives ¬ (“not”), ∨ (“or”), ∧ (“and”), → (“implies”), to which four temporal operators have been added. An LTL formula is a well-formed combination of these operators and connectives, according to the usual construction rules: Definition 1 (LTL syntax): Let π be a formula in some language, and S ⊆ D be a subset of D. Then we have the following: 1) π = S is an LTL formula 2) If ϕ and ψ are LTL formulæ, then ¬ϕ, ϕ ∧ ψ, ϕ ∨ ψ, ϕ → ψ, G ϕ, F ϕ, X ϕ, ϕ U ψ are LTL formulæ. Boolean connectives have their usual meaning. The temporal operator G means “globally”. For example, the formula G ϕ means that formula ϕ is true in every message of the trace, starting from the current message. The operator F means “eventually”; the formula F ϕ is true if ϕ holds for some future message of the trace. The operator X means “next”; it is true whenever ϕ holds in the next message of the trace. Finally, the U operator means “until”; the formula ϕ U ψ is true if ϕ holds for all messages until some message satisfies ψ. The semantics below formalizes these definitions: Definition 2 (LTL semantics): Let ϕ be a LTL formula, π and S be defined as previously. We say that a message trace σ satisfies ϕ, and write σ |= ϕ, if and only if it respects the following rules: σ |= π = S

⇔ Domσ1 (π) is equal to S

σ |= ¬ϕ σ |= ϕ ∨ ψ

⇔ σ 6|= ϕ ⇔ σ |= ϕ or σ |= ψ

Translation to XQuery Intuitively, the LTL formula on a message trace σ and the XQuery expression on a document are equivalent: they express the same thing, albeit using different notations. It seems reasonable to think that every LTL formula can be transformed, by hand, into an XQuery expression. Our goal is to devise a systematic way of transforming the former into the latter. To this end, we develop a recursive translation function ωρ , which takes as input an LTL formula and produces an equivalent XQuery expression. The function also carries a parameter ρ, which is a pointer to the current message of the trace. When starting the translation, ρ must point to the first message of the trace document. The XPath expression /trace/message[1] can be used to designate this first message. It suffices to define ωρ for each of the constructs given previously. The translation of XML path expressions is direct: for π a path in a message and d ∈ D a value at the end of that path, the following FLWOR expression gives an equivalent Boolean result: ωρ (π = d) ≡ for $x in ρ/π[1] return $x = “d” It is important to remark that the path π is relative to the current message of the trace; hence π must be appended to ρ. XQuery allows all logical connectors, therefore, the translation of ¬, ∨ and ∧ is also straightforward; it has, though, to be encapsulated within a FLWOR expression. ωρ (ϕ ∨ ψ) ≡

for $x in ρ return (ω$x (ϕ) or ω$x (ψ))

ωρ (ϕ ∧ ψ) ≡ ωρ (¬ϕ) ≡

for $x in ρ return (ω$x (ϕ) and ω$x (ψ)) for $x in ρ return not((ω$x (ϕ))

In this translation, $x is a fresh XQuery variable, bound to the every statement. By construction, there is only one candidate for $x; hence the FLWOR expression returns only one Boolean value, and not a sequence of Booleans. σ |= X ϕ ⇔ σ 2 |= ϕ It remains to translate the temporal operators into equivalent σ |= G ϕ ⇔ σ i |= ϕ for every i ≥ 1 XQuery code. We consider first the case of the G operator. i σ |= F ϕ ⇔ σ |= ϕ for some i ≥ 1 According to the semantics of LTL, a formula of the form σ |= ϕ U ψ ⇔ there exists k such that σ k |= ψ and σ i |= ϕ for i