the software engineering point of view, these applications resemble some features ... Service-oriented architecture, SOA, metrics, complexity, reliability, ...
Product Metrics for Service-Oriented Infrastructures
Product Metrics for Service-Oriented Infrastructures Dmytro Rud, Andreas Schmietendorf, Reiner R. Dumke Software Engineering Group, Institute for Distributed Systems, Faculty of Computer Science, Otto von Guericke University, Magdeburg, Germany {rud, schmiete, dumke}@ivs.cs.uni-magdeburg.de Abstract: Service-oriented architecture is nowadays widely adopted as modern approach for development of enterprise-wide and cross-enterprise distributed applications. From the software engineering point of view, these applications resemble some features of formerly known component-based and object-oriented software systems and web applications, but the differences are substantial enough to make it impossible to simply reuse existing metrics. In this paper we will try to analyse these differences and to formulate product metrics that consider all peculiarities of service-oriented software and assess its complexity, reliability and performance aspects. Keywords: Service-oriented architecture, SOA, metrics, complexity, reliability, performance
1
Introduction
Service-oriented architecture (SOA) [1] is a modern approach to building distributed applications which can cross organisational boundaries. “Building blocks” for these applications are services – independent software entities, whose functionalities and metadata are uniformly accessible through the network. Actually many technologies can be used to implement services – almost any type of distributed components, e.g. web services, CORBA, DCOM, EJB, etc. can build the basis for SOA. One of natural application fields of the service-oriented paradigm is business process management (BPM). SOA concepts and technologies not only belong to actual topics of academical research, but also are being broadly adopted by the “real” industry. This causes the need for development of formal quality assessment methods for SOA infrastructures. Therefore the objective of our paper is to propose a set of product metrics that can be used to estimate some qualitative properties of service-oriented systems. Note that service-oriented architecture is an approach/an ideology, not a product. A distributed software system built in accordance with SOA – i.e. an implementation of the SOA principle, “SOA as a thing” – will be referred in this paper to as “serviceoriented system” (or simply “system”). Other authors use the term “SOA IWSM/MetriKon 2006
D. Rud, A. Schmietendorf, R. R. Dumke infrastructure” in the same sense as well. A service-oriented system consists of a set of providers’ nodes. Each node is providing one or more non-mobile services, each service has one or more operations (business functions). Clients – i.e. end-user GUI applications, service choreography or orchestration engines, other services, etc. – communicate with services (i.e. invoke their operations) by means of message interchange. Both asynchronous and synchronous communication styles are possible, but the former one tends to be dominant. Services can be either atomic or compound, i.e. represent structured collaborations of other services. In the case of BPM, compound services correspond to orchestrated business processes (to a certain extent, they could be called “applications”, too). Due to their nature, compound services will be examined in this paper from both white-box and black-box views, i.e. as collaborations of multiple services and as single services, respectively. Metrics proposed in this paper are technology-agnostic. They base on no particular technology, protocol or standard and thus are applicable for various types of services and service compositions. In the next sections we examine available related work and inquire into the question of to what extent metrics from other domains are applicable to service-oriented systems. Later, in sections 4–7, we describe product metrics that we consider to be relevant in the context of service-oriented systems. Section 8 concludes the paper. 2
Related Work
As mentioned in the introduction, service-oriented architecture is an actual research topic. The question about what properties (first of all, granularity) a good service must have is well discussed in [2, 3] and other sources. There were also attempts to develop a formal web services quality model on the basis of the FCM (factors-criteria-metrics) model [4]. Presentation [5] discusses quality aspects of service assemblies and proposes some serviceoriented complexity metrics for them. Author of [6] asserts that the rate of valid transaction completion should be the key run-time performance metric for SOA environments. To the best of our knowledge, there exists currently no quality assessment model that considers specific character of service-oriented systems.
Software Measurement Conference
Product Metrics for Service-Oriented Infrastructures 3
Applicability of Metrics from the OOP, CBSE and Web Application Domains
Object-oriented [7], component-based [8, 9] and service-oriented [1] software engineering paradigms have many resembling features – modularity, encapsulation of functionality and data, separation of interface and implementation, and so on. Therefore a question arises to which extent the metrics developed in the context of the first two paradigms could be applied to the latter one as well. To answer this question, differences between objects, components and services must be analysed: 1. Services are distributed over the network and controlled by different parties. The communication between them is thus unreliable and “expensive” in the sense of time and computing resources. All components involved in an application reside on single node and belong to single owner, the communication between them is fast and reliable. 2. Services are usually stateless, because they represent functionalities. Components and objects are usually stateful and represent entities. 3. Services can take part in multiple applications simultaneously. Components and objects belong to exactly one application. 4. Services run as processes detached from each other, therefore partial failure of an application is possible. Elements of component-based applications collaborate in the context of a single process – if one of them falls out, the whole application aborts. 5. Services communicate using uniform data formats and protocols, while components commonly use proprietary ones. 6. In SOA there is no concept of service inheritance, because services’ implementation details are out of scope. Inheritance metrics from the OOP domain are thus not applicable. From the domain of web applications [10], the following metrics can be theoretically applied to service-oriented software: response time, throughput and network latency. But they become meaningless in the case of asynchronous document-oriented communication style of services. While presenting our metrics, we will mention their eventual “counterparts” from the OOP, CBSE and web application domains. 4
Quantities and Symbols
The following quantities and symbols will be used in this paper: IWSM/MetriKon 2006
D. Rud, A. Schmietendorf, R. R. Dumke • N – Set of service providers’ nodes. • S[n] – Set of services provided by the node n ∈ N. • S[∗] – Set of services from all nodes: [
S[∗] =
S[n].
(1)
n∈N
• C[n] ⊂ S[n] – Set of compound services provided by the node n ∈ N. • C[∗] ⊂ S[∗] – Set of compound services from all nodes: [
C[∗] =
C[n].
(2)
n∈N
• S[c] ⊆ S[∗] \ {c} – Set of services that directly take part in the compound service c ∈ C[∗]. • SR [c] ⊆ S[∗] \ {c} – Set of services that take part in the compound service c ∈ C[∗], both directly and indirectly (i.e. recursively): [
R
S [c] = S[c] ∪
SR [x].
(3)
x∈S[c]∩C[∗]
• M[s] – Set of operations provided by the service s ∈ S[n], n ∈ N. • M[n] – Set of operations provided by all services of the node n ∈ N: M[n] =
[
M[s].
(4)
s∈S[n]
• M[∗] – Set of operations of all services of all nodes: M[∗] =
[
M[n].
(5)
n∈N
• A – Set of tuples (relation) hInvokerN ode, Invoker, ServiceN ode, Service, Operationi representing possible (e.g. corresponding to the choreography/orchestration logic of configured business processes) interactions in the system. The meaning of each tuple is “Software component (i.e. a service, a composition engine or an end-user GUI application) Invoker Software Measurement Conference
Product Metrics for Service-Oriented Infrastructures that is located on the node InvokerNode invokes the operation Operation of the service Service that is located on the node ServiceNode”. Figure 1 shows an example of a service-oriented system that consists of six services located on four nodes. Invocations of services’ operations are represented by arrows, parameters and return types of the operations are omitted for simplicity.
BankNode (BN)
ClientNode (CN)
BankService (BS)
Client Application (CA)
+withdraw() +deposit() +debit()
SupplierNode (SN)
ManufacturerNode (MN)
SupplierService (SS)
ManufacturerService (MS)
+placeOrder() +recallOrder() +checkAvailability()
+placeOrder() +recallOrder() +getPriceList() WarehouseService1 (WS1)
WarehouseService2 (WS2)
+getStockInfo()
+getStockInfo()
Figure 1: Example of a service-oriented system Interactions in this system can be described by the following set of tuples: Aexample ={hCN, hCN, hM N, hM N, hM N, hM N, hSN,
CA, CA, M S, M S, M S, M S, SS,
BN, M N, M N, BN, SN, SN, SN,
BS, M S, W S1, BS, SS, SS, W S2,
BS.deposit() i, M S.placeOrder() i, W S1.getStockInf o() i, BS.debit() i, SS.placeOrder() i, SS.checkAvailability()i, W S2.getStockInf o() i}.
In general case, relation A can be constructed by means of the analysis of metadata and process descriptions (e.g. WS-BPEL [11] or IWSM/MetriKon 2006
D. Rud, A. Schmietendorf, R. R. Dumke WS-CDL [12] documents) of currently running compound services (business processes). Relational algebra [13, p. 114] can be efficiently applied to this data set for the calculation and extraction of various indicators. (Note that elements of the tuples are objects, not strings, and therefore cannot be compared lexicographically.) We will need in particular the following operations: – Selection σcondition,... (R) – selects tuples (rows) from the relation R, which conform to the given set of conditions. For the relation Aexample defined above, σService=BS (Aexample ) ={hCN, CA, BN, BS, BS.deposit()i, hM N, M S, BN, BS, BS.debit() i}. – Projection πattribute id,... (R) – selects attributes (columns) from the relation R, eliminating duplicates. Example: πInvokerN ode,Invoker (Aexample ) ={hCN, CA i, hM N, M Si, hSN, SS i}. – Renaming βnew attribute id←old attribute id,... (R) – renames attributes in the relation R. Example: πF ooBar (βF ooBar←Service (Aexample )) = {hBSi, hM Si,hW S1i, hSSi, hW S2i}. Besides the relational algebra, standard set operations (intersections, unions and complements) will be used in this paper as well. • B[m, n] – (Expected) invocation rate of the operation m ∈ M[∗] produced by clients located on the node n ∈ N. A comprehensive analysis of the infrastructure and the white-box-view may be necessary to determine these rates. Some initial ideas of such analysis were proposed in [14, 15]. 5 5.1
Complexity Metrics Network Cohesion in the System
This metric is determined as the count of direct (unidirectional) ties between the nodes in the system: N CY = |πInvokerN ode,ServiceN ode (σInvokerN ode6=ServiceN ode (A))|.
(6)
Software Measurement Conference
Product Metrics for Service-Oriented Infrastructures An unidirectional tie between nodes n1 and n2 exists when the composition logic supposes an invocation of a service provided by the node n2 from a client located on n1 . It were incorrect to simply use overall count of nodes instead of N CY as the overall system complexity metric. When nodes do not communicate with each other, they do not increase the system’s complexity, because standalone nodes are just not part of the system. Network cohesion of the example service-oriented infrastructure depicted on Figure 1 equals to 4. 5.2
Number of Services Involved in the Compound Service
N SIC[c] – Number of Services Involved in the Compound service c ∈ C[∗] (both directly and indirectly): N SIC[c] = |SR [c]|.
(7)
This metric is the simplest complexity indicator of a compound service. Other properties of service compositions will be investigated in the next section, where we will discuss criticality and reliability aspects. When we assume the Client Application from the Figure 1 to be actually a compound service, we can determine for it the following characterictics: S[CA] = {BS, M S}, SR [CA] = {BS, M S, SS, W S1, W S2}. Hence, N SIC[CA] = 5. 5.3
Services Interdependence in the System
SIY – Services Interdependence in the System is the count of pairs of services which depend on each other. This value can be determined as: SIY = |βService1←Invoker,Service2←Service (πInvoker,Service (A)) ∩ βService2←Invoker,Service1←Service (πInvoker,Service (A))|.
(8)
Service interdependence attests poor services’ design and therefore should be avoided – for example, by combining interdependent services to a coarsergrained one, if they belongs to the same provider [4, p. 73]. IWSM/MetriKon 2006
D. Rud, A. Schmietendorf, R. R. Dumke There is no service interdependence in the example system. 6 6.1
Criticality and Reliability Metrics Absolute Importance of the Service
AIS[s] – Absolute Importance of the Service s ∈ S[n], n ∈ N is defined as count of clients which depend on s, i.e. which invoke its operations. Note that we do not count here clients that are located on the node n, because: • Communication between pieces of software that are located on the same node does not produce additional network traffic (this is not very important in the context of system complexity and reliability, but plays a big role in the context of performance that will be discussed later in the section 7). • It does not seem to be very likely that one service will become unavailable while other services residing on the same node will not. The formula for AIS[s] is: AIS[s] = |πInvoker (σInvokerN ode6=n,Service=s (A))|.
(9)
For the services of the example system, AIS[BS ] = 2, AIS[M S ] = 1, AIS[SS ] = 1, AIS[W S1] = 0, AIS[W S2] = 0, AIS[CA ] = 0. 6.2
Absolute Dependence of the Service
ADS[s] – Absolute Dependence of the Service s ∈ S[n], n ∈ N is defined as count of other services this service depends on: ADS[s] = |πService (σInvoker=s,ServiceN ode6=n (A))|.
(10)
This metrics is inversely proportional to the self-sufficiency of the given service, and, consequently, to the level of its potential autonomy and reusability in another environment. Software Measurement Conference
Product Metrics for Service-Oriented Infrastructures For the services of the example system, ADS[BS ] = 0, ADS[M S ] = 2, ADS[SS ] = 0, ADS[W S1] = 0, ADS[W S2] = 0, ADS[CA ] = 2. 6.3
Absolute Criticality of the Service
ACS[s] – Absolute Criticality of the Service s ∈ S[n], n ∈ N is the product of its absolute importance and absolute dependence: ACS[s] = AIS[s] × ADS[s].
(11)
The rationale behind this metric is the following: neither an important service that does not depend on a plenty of other services nor an unimportant one that depends on many others are as critical for the system as a service that is both very important and highly dependent. To some extent, the ACS[s] metric represents the degree of attention that the designer of the service-oriented system should pay to the service s. The three service criticality metrics described above roughly correspond to class coupling metrics from the OOP-domain – “Coupling between object” (CBO) in [8, p. 400]1 and “Class coupling” in [7, p. 82]. Absolute criticality of the service M S in our example system equals to 2. For all other services of the system value of this metric is 0. 6.4
Overall Reliability of the Compound Service
RC[c] – Overall Reliability of the Compound service c ∈ C[∗] is inversely proportional to “the weakest link of a chain”, i.e. to the reliability of the most critical service that takes part in c: RC[c] = 1
1 . max({ACS[s] | s ∈ SR [c]})
The authors of this book seem to use the terms “class” and “object” interchangeably
IWSM/MetriKon 2006
(12)
D. Rud, A. Schmietendorf, R. R. Dumke This is the main reliability metric of compound services. Overall reliability of the only compound service of the example system shown on Figure 1 – RC[CA] – equals to 0.5. 7 7.1
Performance Metrics Sizes of Input and Output Messages
The number of arguments of a method (of a function, of an operation, etc.) is a classic design metric in the OOP and CBSE domains (“Average number of parameters per method” in [7, p. 76], “Argument per procedure” in [16], “Number of parameters on average” in [8, p. 400]). But it is not always applicable to service operations because under the document interaction style all values (e.g. name, address and phone number of a customer) will be “packed” into one coarse-grained data structure, which can also be weak, i.e. its some parts/elements may be optional. Therefore our metric must relate not to the number of parameters, but to their absolute size in bytes. In doing so, size overhead conditioned by the messages format (e.g. XML tags and auxiliary headers in SOAP) must be considered as well. The meaning of such metric in the case of SOA lies not only in design and interface complexity (as in the case of OOP and CBSE), but – due to the network-based nature of communication between services, as noted in section 3 – rather in performance. If an operation sends some data back to the caller (return value or output parameter), the size of this data must be taken into account as well. So we can define the following two metrics: • AIM SO[m] – Average Input Message Size of the Operation m ∈ M[∗]. • AOM SO[m] – Average Output Message Size of the Operation m ∈ M[∗]. This value equals to zero for one-way operations (which are used primarily under the document interaction style). To determine their values, knowledges of both business process’ domain and technical infrastructure are necessary. Average Message Size in the System can be calculated as P AM SY =
m∈M[∗] AIM SO[m] +
|M[∗]| +
P
m∈MRR [∗] AOM SO[m] , |MRR [∗]|
(13)
Software Measurement Conference
Product Metrics for Service-Oriented Infrastructures where MRR [∗] is a subset of M[∗] containing request-response operations (i.e. operations that require/imply both input and output messages). 7.2
Message Rates
An important metric of a node that is providing services is the total number of input and output messages this node has to handle during one unit of time – i.e. to parse messages when they arrive (unmarshalling) and to generate messages when some data must be sent (marshalling), respectively. Message processing consumes CPU time, memory and other resources of the node, therefore it could be expedient to assess it. We introduce the following metrics here: • IM RN [k] – Incoming Message Rate of the Node k ∈ N: IM RN [k] =
X X
B[m, n].
(14)
n∈N m∈M[k]
• OM RN [k] – Outgoing Message Rate of the Node k ∈ N: OM RN [k] =
X
B[m, k].
(15)
m∈M[∗]
• M RN [k] – Overall Message Rate of the Node k ∈ N. It can be supposed that the amounts of resources which are being consumed to handle an incoming message and an outgoing message are roughly equal. Therefore we can add their rates together and use this sum later on to estimate the total consumption of computing resources: M RN [k] = IM RN [k] + OM RN [k]. 7.3
(16)
Network Load
Based on the AIM SO[m] and AOM SO[m], the overall amount of data sent and received by a node during one unit of time can be estimated. The metrics for incoming and outgoing traffic base on expected invocation rates B[m, n] introduced in the previous subsection and are listed below: • IT N [k] – Incoming Traffic of the Node k ∈ N consists of: – input parameters of operations of services located on this node, called by services located on other nodes, and IWSM/MetriKon 2006
D. Rud, A. Schmietendorf, R. R. Dumke – return values of operations of services of other nodes, called by services located on this node:
IT N [k] =
X
X
(B[m, n] × AIM SO[m])
n∈N\{k} m∈M[k]
+
X
X
(B[m, k] × AOM SO[m]).
(17)
n∈N\{k} m∈M[n]
• OT N [k] – Outgoing Traffic of the Node k ∈ N consists of: – input parameters of operations of services located on other nodes, called by services located on this node, and – return values of operations of services of this node, called by services located on other nodes:
OT N [k] =
X
X
(B[m, k] × AIM SO[m])
n∈N\{k} m∈M[n]
+
X
X
(B[m, n] × AOM SO[m]).
(18)
n∈N\{k} m∈M[k]
Amounts of sent and received data characterise not only the network load, but also the consumption of other resources of the nodes (e.g. memory, CPU time, etc.) necessary to process this traffic. 7.4
Generalised Performance Metrics
The following metrics give a “bird’s-eye view” of the system’s performance: • Overall Message Rate in the System X X M RY = B[m, n].
(19)
n∈N m∈M[∗]
• Overall Network Traffic in the System during one unit of time: X X NT Y = (B[m, n] × (AIM SO[m] + AOM SO[m])). n∈N m∈M[∗]\M[n]
(20) Software Measurement Conference
Product Metrics for Service-Oriented Infrastructures 8
Conclusions and further work
In this paper we have proposed metrics, which can be used to assess complexity, performance and reliability of compound services and serviceoriented systems. We have described why these metrics matter and how one can calculate their values. One of properties of the presented model is its independence from particular technology, therefore it can be applied to any type of services and service compositions. The proposed set of metrics is definitely not exhaustive, but it constitutes a good basis for discussion and for subsequent work in this field. Other aspects of product quality should be taken into account as well. Furthermore, we plan to implement tools for the calculation of metrics on the basis of web service metadata and WS-BPEL process descriptions. This will give the possibility to automatically assess the quality of distributed business processes, both at design-time and at run-time. Another mayor improvement could be the consideration of the possibility of dynamic ties between services. Currently we assume that all services involved in a business process are known at this process’ design-time, but this is not always true. References [1] E. Newcomer and G. Lomow, Understanding SOA with web services. Independent Technology Guides, Addison Wesley, 2005. [2] D. J. N. Artus, “SOA realization: Service design principles.” IBM developerWorks, February 2006. [3] R. Schmelzer, “Solving the service granularity challenge.” ZapFlash200639, March 2006. [4] M. Thielen, “Qualit¨atssicherung von Webservices. Entwurf eines allgemeinen Qualit¨atsmodell f¨ ur eine Webservice-Zugriffsschicht,” Master’s thesis, Universit¨at Koblenz-Landau, 2004. (“Quality assurance of web services. Development of a generic quality model for a web service access layer”, in German). [5] “The agile enterprise begins with SOA.” Momentum SI and Cordys Presentation, July 2005. [6] J. Noel, “Transaction completion: The new performance metric for IWSM/MetriKon 2006
D. Rud, A. Schmietendorf, R. R. Dumke service oriented architecture environments,” tech. rep., Ptak, Noel & Associates, 2005. [7] M. Lorenz and J. Kidd, Object-oriented software metrics: a practical guide. PTR Prentice Hall, 1994. [8] J. Z. Gao, H.-S. J. Tsao, and Y. Wu, Testing and quality assurance for component-based software. Artech House, 2003. [9] C. Szyperski, Component software: Beyond object-oriented programming. Addison Wesley, 1998. [10] D. A. Menasc´e and V. A. F. Almeida, Capacity planning for web services: metrics, models, and methods. Prentice Hall, 2002. [11] T. Andrews, F. Curbera, H. Dholakia, Y. Goland, J. Klein, F. Leymann, K. Liu, D. Roller, D. Smith, S. Thatte, I. Trickovic, and S. Weerawarana, “Business process execution language for web services version 1.1,” May 2003. [12] N. Kavantzas, D. Burdett, G. Ritzinger, T. Fletcher, Y. Lafon, and C. Barreto, “Web services choreography description language version 1.0.” W3C Candidate Recommendation, November 2005. [13] A. Heuer and G. Saake, Datenbanken: Konzepte und Sprachen. mitpVerlag, 2 ed., 2000. (“Databases: concepts and languages”, in German). [14] D. Rud, A. Schmietendorf, and R. Dumke, “Performance modeling of WS-BPEL-based web service compositions,” in Proceedings of the Modeling, Design, and Analysis for Service-oriented Architecture Workshop 2006 (mda4soa’2006), (Chicago, USA), September 2006. [15] D. Rud, A. Schmietendorf, and R. Dumke, “Performance annotated orchestration models for service oriented architectures,” in Proceedings of the 22nd Annual UK Performance Engineering Workshop 2006 (UKPEW’06), Bournemouth University, Poole, Dorset, UK, July 2006. [16] M. Goul˜ao and F. B. e Abreu, “Formal definition of metrics upon the CORBA component model,” in Proceedings of the First International Conference on Software Architectures, QoSA 2005 and Second International Workshop on Software Quality, SOQUA 2005, (Erfurt, Germany), September 2005. Software Measurement Conference
