Enterprise Architecture Evaluation Using Utility Theory - IEEE Xplore

2013 17th IEEE International Enterprise Distributed Object Computing Conference Workshops

Enterprise Architecture Evaluation using Utility theory ¨ Magnus Osterlind, Pontus Johnson, Kiran Karnati, Robert Lagerstr¨om, Margus V¨alja Industrial Information and Control Systems KTH Royal Institute of Technology Stockholm, Sweden Email: {magnuso, pj101}@ics.kth.se For example for an ordinary person, the increase in utility could be greater when wealth changes from $0 to $100 and worth working for, but when wealth changes from $10.000 to $10.100 the extra workload might not be that attractive. The utility theory has the potential to aid decision makers in constructing enterprise architecture as-is and to-be scenarios and comparing them against each other. The utility theory can aid the decision maker in which architecture to implement and, as in the example of increased wealth, also provide insight to if a change, and the effort that comes with it, is needed or not. An enterprise architecture can incorporate many quality attributes. The decision maker usually has to balance these attributes against each other to obtain the best possible architecture. This article proposes a systematic and formal way of assessing the quality of an enterprise architecture based on a decision maker’s preferences. The proposed method in this article aims to provide support to the task of making trade-off decisions and balancing the enterprise architecture.

Abstract—With the increase in the number of quality attributes (e.g. cost, availability, reusability), that are being considered in the process of enterprise architecture analysis, the decision maker needs a systematic way to balance these attributes against each other to obtain the best possible architecture. Utility theory addresses this need by providing methods for numerical representation of preferences of a stakeholder involved in a decision-making process. In this paper utility theory key concepts are explained with examples. The process of calculating the utility metric, which reflects stakeholder’s set of preferences to select the most preferred architecture scenario is explained. The paper provides an explanation of how utility theory can be applied in enterprise architecture models which are meta-object facility compliant. This paper concludes by an example comparing two quality attributes on two architecture scenarios using utility theory and calculating the decision maker’s overall utility metric across both quality attributes is provided. This shows the applicability of utility theory on architecture scenario analysis with multiple quality attributes. Keywords-utility; enterprise architecture; multi-attribute decision making;

I. I NTRODUCTION

II. R ELATED

Enterprise architecture (EA) has become an established discipline for business and IT management. Architecture models constitute the core of the approach and serve the purpose of making the complexities of the real world understandable and manageable to humans. However, the models sometimes don’t show clearly what kind of improvements should have a higher priority. This can be especially true in situations with a large number of entities and requirements. The decision makers can become easily confused and the confusion can lead to decisions that bring little or no value to the organization. In this article utility is defined as the quality or condition of being useful. Utility theory has been used to construct models for better decision making[1]. The approach, based on extensive interviewing and calculations[1], allows to divide objectives into preferences and to turn the preferences into a quantifiable and comparable form. This can aid multi attribute trade-off situations and provide decision support. A change in utility between one scenario and another implies that one of the scenarios is better than the other. This can also be done not just to find out which scenario is the best but also to see if a change would be worth the effort. 978-0-7695-5085-5/13 $26.00 © 2013 IEEE DOI 10.1109/EDOCW.2013.45

WORK

There are several EA frameworks available today. Department of Defense Architecture Framework (DoDAF)[2] is a framework that focuses on architectural data and is suitable for modeling large complex systems. The Open Group Architecture Framework (TOGAF), developed by an industry consortium The Open Group, is an industry standard[3]. ArchiMate[4] modeling language is another Open Group standard for representing EA in graphical language. The focus of frameworks like DoDAF, TOGAT and ArchiMate is in describing various architecture domains. The frameworks are good for documenting and knowledge sharing. The frameworks lacks a little in formal analysis methods and metrics which could aid decision making. Giesen and V¨olker have used utility for measuring how much software projects meet stakeholders’ expectations[5]. They looked at functional and non-functional attributes in the context of conflicting requirements. They define an attribute as something that describes a system or a product, and a requirement as an attribute that must have a certain value. They use cost-value conflict as an example where functional features have to be traded against the cost of development. 347

are needed. ki is a scaling constant associated with attribute Xi , with range 0 < ki < 1. k is a non zero scaling constant k > −1 which is required in some cases to scale the utility function U (X) to keep it within its range 0 ≤ U (X) ≤ 1. A lottery is a gamble between two or more outcomes with a certain probability of getting one or the other outcome[1]. A certainty equivalent, x ˜, of lottery L is an amount of x˜i such that the decision maker is indifferent between L and the amount x˜i for certain[1].

Giesen and V¨olker focus on three roles: manager, technician, and user. They determine a utility function for each of those based on the subset of attributes for each role, and aggregate the individual utility functions in a single utility function. According to the paper, the aggregated utility functions can be used to reveal conflicts between the roles. The authors see the strength of the method in handling a large number of non-functional attributes. Enterprise architectures can also incorporate large numbers of non-functional attributes and the usage of utility theory combined with enterprise architecture models might be fruitful in the same way as identified by Giesen and V¨olker when combining utility theory and software requirements. Zayaraz and Thambidurai have looked into using nonfunctional requirements like cost, usability, and reliability for comparing software architectures[6]. Their framework allows to evaluate fitness of competing architectures based on an arbitrary process. The process includes classifying stakeholders, identifying their quality requirements, normalizing the quality attributes measurements, identifying strengths, deciding weights for the attributes and doing conversion and then calculations. The strength in the method lies in the scientific way of prioritizing quality attributes and the importance of the stakeholders preferences. This is also relevant to enterprise architecture decision making.

B. Independence To ease the workload when constructing the appropriate utility function for the decision maker a set of independence assumptions can be validate. Two possible independencies are presented below, for more independencies see [7], [1]. 1) Preferential independence: Xi ⊂ X is said to be preferentially independent of its complement, Xi , if the preference order for the outcome (xi , xi ), where the values xi are fixed, does not depend on the fixed value xi [1]. So a stakeholders preferences over an attribute Xi is not dependent on any other attribute. [1] The attributes in X are mutually preference independent if every subset of X1 , X2 , ..., Xn is preference independent of its complement. 2) Utility independent: Xi ⊂ X, is said to be utility independent of its complement, Xi , if the preference order over lotteries on Xi , (x˜i , xi ), with Xi fixed, does not depend on the fixed amount of xi . [1]. The attributes in X are mutually utility independent if every subset of X1 , X2 , ..., Xn is utility independent of its complement.

III. U TILITY T HEORY Utility theory has been developed to aid decision making with regards to multiple objectives. The general idea is to capture the decision makers preferences in a systematic way and evaluate decision alternatives based on the preferences, to select the most preferred alternative. This section explains parts of the utility theory, which are required in this paper.

C. The multi-attribute utility functions Keeney and Raiffa proposes three different multi-attribute utility functions, the additive, the multiplicative and the multi linear utility functions. The utility functions are explained in detail in[1]. This paper is due to page constraints limited to the multiplicative utility function. 1) Multiplicative utility: If each attribute in X is utility independent of its respective complement then the multiplicative utility function holds, equation 1. The multiplicative utility function multiplies the conditional utility functions with each other using both individual scaling constants ki and a final scaling constant k to ensure that 0 ≤ U (X) ≤ 1. When assessing the scaling constants, ki s, the stakeholder needs to rate the importance of each attribute. This can be done in a numerous of ways. In the end, the assessed weight for each attribute has to be normalized. One way of assessing the weights is to have the stakeholder assigning utility score pi of u(x∗i ; x0i ) = pi . The weight can also be evaluated by proposing a gamble, letting the stakeholder assign the probability pi , illustrated in Figure 1. The weight pi is then assigned to the scaling constant ki so that ki = pi .

A. Notation and requirements The following notation will be used throughout the paper. In this article an attribute is defined as a quality or characteristic inherent in or ascribed to someone or something. X = {X1 , X2 , ..., Xn } is a list of n attributes Xi . The complement of an attribute, Xi , is denoted Xi = {X1 , ..., Xi−1 , Xi+1 , ..., Xn }. The utility function is denoted U (X) and has a range of 0 ≤ U (X) ≤ 1. Each of the attributes Xi can be assigned a state or value xi which is within a state/value range containing two or more values. The stakeholder has to be able to tell if one value is preferred over another x1  x2  ...  xn , or if she is indifferent to them, this is called the preference order. The conditional utility of a specific attribute Xi , with the value xi is denoted ui (xi ). The conditional utility functions has a range of 0 ≤ ui (xi ) ≤ 1 where the least preferred consequence of the attribute ui (x0i ) = 0 and the most preferred outcome ui (x∗i ) = 1. To decision maker might not consider all attributes to be equally important, to adjust for these kind of preferences scaling constants to the attributes

1 + kU (X) =

n  i=1

348

(1 + kki ui (xi ))

(1)

( xi* ; xi0 )

pi

( x* )

1 p

( x0 )

i

Figure 1.

parameters. A Stakeholder may pose several ClassRequirements. The relative importance of each object is specified by the ClassRequirements’ scalingConstants. The aggregated utility for a given stakeholder is finally determined by the Stakeholder.utility operation, which, similarly to the ClassRequirements.utility operation, takes the conditional utilities and scaling constants of the ClassRequirements as parameters. Summarizing, a Stakeholder’s utility is a function of the satisfying object’s relative importance (ClassRequirement.scalingConstant) as well as of the extent to which the object satisfies its attribute requirements (ClassRequirement.utility) and the relative weights of those requirements.

The gamble approach on setting scaling constants.

To calculate k all attribute utility functions are set to 1, ui (xi ) = 1, and so U (X) = 1. Which gives: 1+k =

n 

(1 + kki )

(2)

i=1

Solving equation 2 gives the k value. Keep in mind that k = 0 and k > −1. If k = 1 the multiplicative utility function is reduced to the additive form. IV. E XTENDING EA METAMODELING WITH UTILITY THEORY

This section explains how the utility theory can be applied in EA metamodels which incorporates quality attributes. The metamodel is assumed to be meta-object facility compliant[8]. The utility theory extension is shown in Figure 2. The class Class is a normal class as specified in standard UML. The attribute in the utility theory Xi is modeled as a Property owned by a class. The decision maker’s conditional utility function ui (xi ) are specified using the AttributeRequirement class. The AttributeRequirement class includes a scalingConstant, ki , to specify the importance of the related attribute, relative to the other attributes of the owning class. The AttributeRequirement class also features an operation named utility, with the associated attribute value xi as parameter, and returning a real value between 0 and 1, ui (xi ), representing the associated conditional utility function. As an example, the utility operation of a certain AttributeRequirement might specify the stakeholder utility experienced by varying a non-functional property of a software service, such as the availability of the service. Each AttributeRequirement is owned by a ClassRequirement. ClassRequirements represent the stakeholder’s requirements on the existence of objects of certain classes. For instance, a class could represent a certain service of a software application. The association of a ClassRequirement to the service class indicates that there can be a requirement that the service is provided by the application. Owned AttributeRequirements may further specify the quality requirements of the service, such as availability, response time or cost. The joint utility of an object with multiple attributes is determined by the utility operation of the ClassRequirement, U (X), which takes the conditional utilities and scaling constants of the owned AttributeRequirements as

Figure 2. Utility theory extension to enterprise architecture metamodels.

V. E XAMPLE This sections goes through a small example with utility theory applied on two EA scenarios. The enterprise architecture is described using the ArchiMate modeling language[4]. The example covers the technology and application layers. The example considers the two attributes availability, A, and cost, C. The two attributes can be evaluated from the EA model using the approach detailed in [9], [10]. The evaluation is conducted by adding the cost and availability properties to the corresponding attributes in the classes in the architectural model. The cost is then aggregated into a total cost. The availability is calculated using a fault tree analysis approach[9]. At the fictitious company ACME they are facing problems with the process of billing their customers for the products they sell. They have decided to extend their application portfolio with the FinancE billing application. The decision maker is concerned about the availability and cost of the billing service. In a real case there are possibly more attributes which has to be taken into consideration, those attribute have been omitted to keep the example small. A higher availability can be obtained by having two generic servers executing the billing application. There are additional costs associated with the extra server. The architectural scenarios are evaluated using utility theory to find out if the decision maker prefers one or two generic servers in the architecture. ACME starts modeling the architecture for the new system, see Figure 3. The node, Generic Server [*], represents the amount of servers where the [*] serves as a

349

utility function proposed by [1] is used. The conditional utility curves for the two attributes are then elicited from the decision maker. The decision maker starts with expressing the least desirable level of availability, 85%, the level where the system is to be considered to be on a very good level and some points in between. The same procedure is done with the cost where the decision maker assigns the least desirable level according to how much money she can spend on the system, 500000. The full procedure on how to do this is described in [1]. The conditional utility functions are shown in Figure 5 and Figure 6.

placeholder for (1 or 2), servers ACME are considering to install and maintain.

Figure 5. Figure 3.

The decision maker’s availability utility curve, ua (xa ).

The Example Architectural Scenario.

The two properties, availability and cost, are added to the architectural model. For the decision maker the bill creation service is important. Figure 4 shows parts of the architectural model in combination with the proposed utility extension.

Decision maker Stakeholder -utility

Bill application requirement ClassRequirement

Bill creation

-scalingConstant, k -utility

Property: Cost

Figure 6.

AttributeRequirement -scalingConstant k_c: -conditionalUtility

The decision maker is then asked to assign the scaling constants for the two attributes. This is done using the gamble approach previously explained. The decision maker assigns the availability scaling constant to ka = 0.7 and the cost scaling constant to kc = 0.87. k is calculated using equation 2. 1 + k = (1 + 0.7k) ∗ (1 + 0.85k) ⇒ k = −0.92. The decision maker’s joint utility function is presented in equation 3 and visualized in Figure 7.

AttributeRequirement -scalingConstant, k_a -conditionalUtility

Figure 4.

The decision maker’s cost utility curve, uc (xc ).

Property: Availability

Utility theory combined with Architectural Scenario.

The decision maker is questioned about her independence preferences over the two attributes, availability and cost, this procedure is detailed in [1]. It turns out that the two attributes are mutually preference and utility independent but not additive independent. To spare the decision maker from assessing additional scaling constants, the multiplicative

1 − 0.92U (X) = (1 − 0.64ua (xa )) ∗ (1 − 0.8uc (xc )) (3)

350

modeling software that can aid and automate parts of this work. However, more empirical studies are needed in order to determine exactly how to employ the approach in practice. We believe that the approach can be of most use when planning for larger changes, for instance when future to-be scenarios contain many new entities and relationships with tough requirements on several non-functional requirements. It is of most use when one person or a team of people can manage to grasp all the complexities of the proposed architecture scenarios. VII. C ONCLUSIONS Figure 7.

This article has presented a way of evaluating an EA based on a decision makers preferences using utility theory. An explanation of how utility theory can be applied in EA models which are meta-object facility compliant is provided and the presented example in the paper show that it is feasible. The topic is not yet fully explored and requires future research.

The decision maker’s utility function.

The two attributes, availability and cost, are then evaluated with the approach presented in the beginning of this chapter, and the outcome is used as input to the decision maker’s utility function, equation 3. The result for the two scenarios using one or two generic server is shown in Table I.

R EFERENCES

Table I

[1] R. Keeney and H. Raiffa, Decisions with multiple objectives: preferences and value trade-offs. Cambridge University Press, 1993.

A RCHITECTURAL EVALUATION

1 Generic server 2 Generic servers

Availability 87.5% 94.7%

Cost 415840 416680

Decision maker’s utility 0.508 0.795

[2] Department of Defense Architecture Framework Working Group, “DoD Architecture Framework, version 1.5,” Department of Defense, USA, Tech. Rep., 2007.

Based on the utility score, it is shown that the decision maker would prefer to implement the scenario with two generic servers.

[3] The Open Group, The Open Group Architecture Framework (TOGAF) - version 9. The Open Group, 2009. [4] M. Lankhorst, Enterprise architecture at work: Modelling, communication and analysis. Springer-Verlag New York Inc, 2009.

VI. D ISCUSSION Approaches for modeling enterprise architecture are common in both research and industry. However, there is still a gap in methods supporting decision making with respect to different design alternatives for enterprise architecture. We believe that utility theory is a proper approach for filling this gap. However, a number of things are needed in order for it to work. 1) The modeling frameworks employed must be suited for the use of utility concepts. One solution is to be MOF compliant. This of course sets some requirements/limitations to the chosen approach. However, we believe that MOF is close to being academic and industry standard when it comes to complex system modeling making this a low risk assumption. 2) The approach suggested relies on input from decision makers on complex matters such as independencies and scaling constants. However, in an industry setting it is not required for a decision maker to know the specifics of these concepts. In the exploration phase we are in now the assessing of independencies (among other things) is led by a professional, in this case a researcher, with knowledge and experience on the topic. Although for future use we aim to provide tool support in an enterprise architecture

[5] J. Giesen and A. V¨olker, “Requirements interdependencies and stakeholders preferences,” in Requirements Engineering, 2002. Proceedings. IEEE Joint International Conference on. IEEE, 2002, pp. 206–209. [6] G. Zayaraz and P. Thambidurai, “Software architecture selection framework based on quality attributes,” in INDICON, 2005 Annual IEEE. IEEE, 2005, pp. 167–170. [7] P. Fishburn, “Utility theory for decision making,” DTIC Document, Tech. Rep., 1970. [8] R. Soley et al., “Model driven architecture,” OMG white paper, p. 308, 2000. [9] P. N¨arman, U. Franke, J. K¨onig, M. Buschle, and M. Ekstedt, “Enterprise architecture availability analysis using fault trees and stakeholder interviews,” Enterprise Information Systems, no. ahead-of-print, pp. 1–25, 2012. [10] P. N¨arman, T. Sommestad, S. Sandgren, and M. Ekstedt, “A framework for assessing the cost of it investments,” in Management of Engineering & Technology, 2009. PICMET 2009. Portland International Conference on. IEEE, 2009, pp. 3154–3166.

351