Simulation-based IT Process Governance

Simulation-based IT Process Governance 1,2

Vladimir Stantchev

3,4

, Gerrit Tamm

, and Konstantin Petruch

5

Berlin Institute of Technology, Berlin, Germany [email protected] Hochschule für Oekonomie und Management (FOM), Berlin, Germany 3 SRH University of Applied Sciences, Berlin, Germany 4 Humboldt-University at Berlin, Germany 5 Deutsche Telekom AG, Germany 1

2

IT departments within enterprises often utilize incident management and ticketing systems. While performance data from such systems is often stored in logles it is rarely evaluated extensively. In this work we assess the applicability of an approach for extended evaluation of logged data from such systems. Our evaluation is focused on Key Performance Indicators (KPIs) and aims to discover more in-depth insights as compared to typical data visualization and dashboard techniques. We also present a practical case study where we demonstrate the feasibility of the approach using real-life logles from an international telecommunication provider. Our modeling methodology is based on the approach of System Dynamics. Keywords: simulation, IT processes, IT governance, incident management Abstract.

1

Introduction

Modern enterprises and organizations conduct a wide range of operative processes electronically. Data gathered by such operative systems often hides nontrivial insights. Visualization is often a rst step towards a more detailed data analysis. This is the application domain of the business dashboards. One specic example is the usage of Google analytics6 to visualize web server logles. More complex application scenarios involve the aggregation of multiple data sources and the subsequent analytical processing of data within a data warehouse. Operative data can provide insights about two distinct types of measurements  key goal indicators (KGIs) which provide insights about the results of an operative task, and key performance indicators (KPIs) which dene the way these results were achieved (e.g., speed, transaction rate). When we dene such indicators with respect to specic business processes we can apply an approach known as process mining [1]. In this work we extend this approach with more complex simulation techniques. Our focus lies on KPIs and we address the question whether an extended 6

www.google.com/analytics/

data analysis using simulation methodologies can provide an additional value as compared to standard data visualization. The rest of this work is structured as follows: Section 2 presents the state of the art in the measurement of process indicators and the terminology we use. In Section 3 we give an overview of our assessment framework for such indicators in the area of IT operations. In Section 4 we describe a specic data transformation and simulation process that we have assessed within an industry case study with an international telecommunications provider. Section 5 contains a summary of our results and outlook on our future research activities.

2

Preliminaries

In this section we introduce the motivation for performance and output metrics. We also discuss the possible insights from existing datasets.

2.1

Concepts of Indicators

As stated above, indicators can be generally divided into two groups  key performance indicators (KPIs) and key goal indicators (KGIs). KPIs measure how well a process is performing and are expressed in precisely measurable terms. KGIs represent a description of the outcome of the process, often have a customer and nancial focus and can be typically measured after the fact has occurred [2]. While KGIs specify what should be achieved KPIs specify how it should be achieved.

2.2

Objectives of Data and Log File Analysis

Various algorithms [3,4] have been propose to discover dierent types of models based on a log le. A special issue of Computers in Industry on process mining [5] oers more insights. In the context of process model verication there are several notions for equivalence of process specications such as behavioral equivalence [6,7], trace equivalence, and bisimulation [8] that have been developed. Traditional equivalence notions like bisimulation or trace equivalence are dened as a verication property which yields a yes-or-no boolean value, but no insights on the degree of equivalence. When comparing a reference model with a process model, it is not realistic to assume that their granularities are the same. Therefore, the equivalence analysis with classical equivalence notions will most likely not be conclusive. In the context of process mining we should apply notions searching for behavioral similarity. Examples include causal footprint [6] and tness function [7]. In [6], the authors introduce an approach for determining the similarity between process models by comparing the footprint of such models. Thereby the footprint describes two relationships between activities  the soc. look-back and look-ahead links and returns the degree of the process similarity expressed in [0, 1]. This value is not conclusive and requires further explanation. It is not possible to trace the missing or diering activities.

Since traceability is an important requirement of the organization, the approach is not suitable in general. In [7], the authors introduce the behavioral and the structural precision and recall. The behavioral equivalence of the process models compares a process model with respect to some typical behavior recorded in log les. The structural precision and recall equate the term "`structure"' with all ring sequences of a Petri net that may occur in a process model. Other related works exist in the areas of pattern matching or semantic matching. Existing approaches [9] assume that the correspondence of activities can be established automatically. Since they suppose that the same label implies same function, they try to identify the content of an activity by using an automated semantic matching algorithm based on the label of activities. One specic approach for quality improvement in compliance is IT supported compliance evaluation [10]. The notion of compliance has also been discussed in the context of business alignment [11].

3 3.1

Assessment Framework for IT Governance IT Governance Frameworks

IT governance frameworks aim to dene standardized processes and control metrics for IT provision. Commonly applied frameworks in this area include the IT Infrastructure Library (ITIL) [12] and the Control Objectives for Information and Related Technology (CObIT) [13]. They typically provide best practices for measurement and control of IT-specic indicators.

3.2

IT-specic Indicators

IT indicators should demonstrate the added value of IT to the business side. A well accepted view of business objectives is Porter's distinction between operational eectiveness (eciency and eectiveness) and strategic positioning (reach and structure) [14]. This view can be translated directly into corresponding goals and indicators for IT [15]. Organizations require well designed business processes to achieve excellence in a competitive environment: Here, not one-time optimized business processes play the essential role, but rather the ability to quickly react to new developments and to exibly adapt respective business processes are decisive[16]. It is important that these processes are eectively supported through IT. These requirements have consequently been catalyzing increased interest in reference modeling for IT process management. Reference models such as ITIL and CObIT represent proven best practices and provide key indicators for the design and control of IT services [12]. On the one hand, utilization of reference models promises to enhance quality and facilitates better compliance according to statutes and contractual agreements. On the other hand, IT processes have to correspond to corporate strategy and its respective goals. Therefore, the question arises how best practices can be implemented in a particular corporate environment. Another challenge lurks in the checking of reference process execution

as well as in assuring compliance to IT procedure in respect to new or altered business processes [17].

3.3

Aligning IT and Business Indicators

One way towards IT and business alignment can be the application of approaches such as CObIT and ITIL for the optimization of IT organizations. We recently introduced an approach for the continuous quality improvement of IT processes based on such models [1] and process mining. An organization can also try to assure the continuous provision of service levels as demonstrated in our previous work with such reference models and our work in the area of service level assurance in SOA [18,19,20]. Furthermore, in order to coordinate and govern IT production, we can assess operative data and try to analyze it more deeply with the help of simulation models.

3.4

Cloud Governance Aspects

Governance of cloud computing should regard dierent deployment models. Abstracting services at the level of infrastructure (IaaS) allows comparatively easy virtualization  the user organization can congure and customize the platform and the services within the virtual image that is then being deployed and operated. This includes the denition of performance parameters for specic services (e.g., parameters of a Web Service Container), the security aspects of service access, and the integration of services within the platform. When using a standardized platform (the PaaS approach) the user organization deploys the services in a virtualized operating environment. This operating environment is typically provided as a service  the virtualization technology and the operating environment are managed by the provider. Integration capabilities are always provider-specic and there are currently no commonly accepted industry standards for integration between services operated in dierent PaaS environments.7 The usage of software services itself (the SaaS approach) precludes ne-grained control and enforcement of non-functional aspects (e.g., QoS, response time) and security parameters of the infrastructure and the platform by the user organization. These dierent levels of virtualization require dierent levels of security and abstraction. The grade of control and responsibility for security aspects declines with higher levels of abstraction  in IaaS the conguration is generally in the hand of the user organization, while in SaaS it is primarily a responsibility of the Cloud provider. There are several emerging patterns for cloud usage. The rst one is a natural consequence of the trend to outsource IT-Operations (aka. IT-RUN functions) 7

Two current standardization activities at the IEEE Standards Association are IEEE P2301, Draft Guide for Cloud Portability and Interoperability Proles, and IEEE P2302, Draft Standard for Intercloud Interoperability and Federation, see http://standards.ieee.org/news/2011/cloud.html

to external providers and results in demand for IaaS. IaaS is typically used for the implementation of test projects and as a way to overcome underprovisioning in on-premise infrastructures. The second one is coming from the SaaS area and focuses on the provision of Web 2.0 applications. Some well-known sites oer the user the chance to develop simple applications (a la PaaS) and oer them in a SaaS-like manner later on. This usage pattern could also be called extension facilities. PaaS is an optimal environment for users seeking testing and development capabilities, these are two new emerging use patterns which are gaining popularity. Probably, gaming will be one of the most remarkable usage patterns for Cloud technologies, due to an inherent scalability, endowing such applications with virtually unlimited graphical power and players. Also the rise of netbooks in the computer hardware industry triggered the development of Clouds. These slim devices depend on services being deployed in remote Cloud sites since their own capacity is limited. Behind this stand the idea of getting access to everything, from anywhere, at any time. A set of general Corporate Governance rules has to be specically rened and targeted for every operational area in an enterprise. The idea of manageability in Cloud Computing is closely related to the operationalization of Corporate Governance in the dierent phases of the use of a Cloud Computing oering. A specic manifestation of such operationalization can be the introduction of SLA-based Governance. This would mean that the organization has to incorporate specic governance requirements as part of a service level agreement for a Cloud Computing oering. Suitable examples include the so called "four-eyesprinciple" that can be part of the SLA for a SaaS oering, or data availability requirements that can also be part of the SLA for a SaaS oering. In order to introduce such transparent Cloud Governance mechanisms an organization has to consider all phases of the usage of a Cloud Computing oering. During the rst phase of requirements identication and elicitation (often called the Plan-Phase) these requirements need to be specied and formalized. This allows addressing them already within a rst assessment of the Cloud Computing market for the specic oering. Potential Cloud Computing providers can then be specically evaluated with respect to the requirements and specic SLAs can be negotiated with them during the second phase. The third phase can focus on the transparent communication of values and benets of the SLA during start of production for the specic business unit. The fourth phase would deal with performance monitoring and assessment of SLA fulllment and associated bonuses or penalties. These phases and their associated activities can be introduced as specic Cloud Computing extensions to more traditional IT-Governance approaches such as CObIT and ITIL. This introduction is typically non-trivial, as there are signicant dierences between the abstraction levels and the semantics of Cloud Computing and IT-Governance. In the specic area of SaaS a more straightforward approach can focus on the introduction of a more specic approach from the area of SOA Governance  the SOA LifeCycle [21]. It describes a governance approach for software func-

tionality as provided by web services which makes its paradigms and concepts more applicable to the aspects of SaaS Governance. On the other side, the SOA LifeCycle can be incorporated as part of a general IT-Governance strategy based on CObIT and ITIL.

4

Case Study: Transforming and Simulating Log Data

In this case study we describe our approach for extended log data evaluation using System Dynamics [22].

4.1

Operative Log Data

The operative log data is generated by software applications that provide service support as a set standardized IT functions. It describes request processing for three dierent IT services of an international telecommunications provider. The IT services are an e-mail service, an IP-based video-on-demand service, and a web-hosting service. Data is generated in a comma separated values (CSV) format and includes an incident number, as well as the following elds:

       4.2

priority, short description, aected service, start of incident, end of incident, range of impact, number of process steps needed.

The Concept of System Dynamics

System Dynamics allows to represent and analyze complex causality structures. It can often provide insights that are not easily derived from the original data and are sometimes even counterintuitive. Sometimes such analyzes can lead to the revision of already made decisions. Figure 1 shows an example for such model.

4.3

Selection of a Simulation Tool

There exist several tools for the denition and conduction of System Dynamics models. Our next objective was the selection of a suitable tool for the enterprise application scenario. Our assessment was based on cost benet analysis and the analytical hierarchy process (AHP) and included the following categories of requirements with their weightings.

 Technical (15%)  Functional (50%)  Environment (25%)

Fig. 1.

Fig. 2.

A Sample System Dynamics Model

An Excerpt from the Technical Assessment

 Supplier / Support (10%) Figure 2 shows exemplary an excerpt from the assessment of the technical requirements of the four alternatives. The nal decision was to use the simulation software Consideo Modeller. 8 8

http://www.consideo.de

4.4

Data Transformation

The use of the log data as input for the System Dynamics simulation required further transformation. Examples for two specic transformations that we had to conduct are:

 The original log data includes timestamps (start and end of an incident)

UNIX-type datetime data. It had to be transformed to the supported DD.MM.YYYY HH:MM:SS format.  We had to include an incident increment that we then facilitated to model coordinates between a time value and the number of incidents that are processed.  We used only an excerpt of the available log data that covered several years. Using data blocks per month allowed us to keep the execution time of the simulation short.

4.5

Sample Simulation Models and Results

Figure 3 shows an excerpt from our quantitative modeling approach.

Fig. 3.

An Excerpt from the Quantitative Model

Specic KPIs and KGIs that we can simulate from the log data include the incident processing rate (see Figure 4) and further indicators such as timing and responses dependencies.

Fig. 4.

5

Simulation of the Incident Processing Rate

Conclusion and Outlook

Objective of this work was to better evaluate enterprise indicators through facilitation of an extended analysis of operative log data. We used a System Dynamics model in order to map log data to established indicators from ITIL. The case study demonstrated the feasibility of our approach. Furthermore, it can be a suitable environment for the evaluation of knowledge and learning objects, processes, strategies, systems, and performance as dened in [23].

References 1. K. Gerke and G. Tamm. Continuous Quality Improvement of IT Processes based on Reference Models and Process Mining. AMCIS 2009 Proceedings, page 786, 2009. 2. Wim Van Grembergen, editor. Strategies for Information Technology Governance. IGI Publishing, Hershey, PA, USA, 2003. 3. W. M. P. van der Aalst, B. F. van Dongen, J. Herbst, L. Maruster, G. Schimm, and A. J. M. M. Weijters. Workow mining: a survey of issues and approaches. Data Knowl. Eng., 47:237267, November 2003. 4. A.K.A. de MEDEIROS, A.J.M.M. Weijters, and W.M.P. van der Aalst. Genetic process mining: an experimental evaluation. Data Mining and Knowledge Discovery, 14(2):245304, 2007. 5. W.M.P. van der Aalst and A. Weijters. Process mining: a research agenda. Computers in Industry, 53(3):231244, 2004. 6. B. van Dongen, R. Dijkman, and J. Mendling. Measuring similarity between business process models. In Advanced Information Systems Engineering, pages 450464. Springer, 2008.

7. W. Van der Aalst, A. de Medeiros, and A. Weijters. Process equivalence: Comparing two process models based on observed behavior. Business Process Management, pages 129144, 2006. 8. R.J. Van Glabbeek and W.P. Weijland. Branching time and abstraction in bisimulation semantics. Journal of the ACM (JACM), 43(3):555600, 1996. 9. M. Ehrig, A. Koschmider, and A. Oberweis. Measuring similarity between semantic business process models. In Proceedings of the fourth Asia-Pacic conference on Comceptual modelling-Volume 67, pages 7180. Australian Computer Society, Inc., 2007. 10. S. Sackmann and M. Kähmer. Expdt: A layer-based approach for automating compliance. Wirtschaftsinformatik, 50(5):366374, 2008. 11. W. M. P. van der Aalst. Business alignment: using process mining as a tool for delta analysis and conformance testing. Requir. Eng., 10:198211, November 2005. 12. J. Van Bon. Foundations of IT service management based on ITIL V3. Van Haren, 2008. 13. J.W. Lainhart IV. COBIT: A Methodology for Managing and Controlling Information and Information Technology Risks and Vulnerabilities. Journal of Information Systems, 14:21, 2000. 14. M.E. Porter. What is strategy? Harvard Business Review, 74(4134):6178, 1996. 15. Paul P. Tallon, Kenneth L. Kraemer, and Vijay Gurbaxani. Executives' perceptions of the business value of information technology: a process-oriented approach. J. Manage. Inf. Syst., 16:145173, March 2000. 16. J. Borzo. Business 2010 - Embracing the Challenge of Change. Technical report, 2005. 17. Vladimir Stantchev and Gerrit Tamm. Addressing non-functional properties of services in it service management. In Non-Functional Properties in Service Oriented Architecture: Requirements, Models and Methods, pages 324334, Hershey, PA, USA, 05 2011. IGI Global. 18. Vladimir Stantchev and Christian Schröpfer. Service level enforcement in webservices based systems. International Journal on Web and Grid Services, 5(2):130 154, 2009. 19. Vladimir Stantchev and Miroslaw Malek. Translucent replication for service level assurance. In High Assurance Services Computing, pages 118, Berlin, New York, 06 2009. Springer. 20. Vladimir Stantchev and Christian Schröpfer. Negotiating and enforcing qos and slas in grid and cloud computing. In GPC '09: Proceedings of the 4th International Conference on Advances in Grid and Pervasive Computing, pages 2535, Berlin, Heidelberg, 2009. Springer-Verlag. 21. Vladimir Stantchev and Miroslaw Malek. Addressing dependability throughout the soa life cycle. IEEE Transactions on Services Computing, 99(PrePrints), 2010. 22. George P. Richardson and Alexander L. Pugh. Introduction to System Dynamics Modeling with Dynamo. MIT Press, Cambridge, MA, USA, 1981. 23. M.D. Lytras and M.A. Sicilia. The Knowledge Society: a manifesto for knowledge and learning. International Journal of Knowledge and Learning, 1, 1(2):111, 2005.