group of tasks), to control if it respects the organization strategic goals, objectives or ... deduction process that derives the appropriate measures to answer a given business ... elements such as higher-level business goals, strategies, and assumptions [3]. ..... In section 6.1 in figure 3 we modeled the business strategy for.
Developing Business Process Monitoring Probes to Enhance Organization Control Fabio Mulazzani1, Barbara Russo1, Giancarlo Succi1 1
Free University of Bolzano, Faculty of Computer Science, Via della Mostra 4, 39100 Bolzano, Italy {fabio.mulazzani, barbara.russo, giancarlo.succi}@unibz.it
Abstract This work present business process monitoring agents we developed called Probes. Probes enable to control the process performance aligning it to the company’s strategic goals. Probes offer a real time monitoring of the strategic goals achievement, also increasing the understanding of the company activities. In this paper Probes are applied to a practical case of a bus company. Probes were developed and deployed into the company ERP system and determined a significant change in the strategy of the company and a corresponding enhancement of the performances of a critical business process.
Keywords: ERP Systems, Monitoring Agents, Probes, Business Strategy, Business Processes.
1
Introduction
Chief information officers and IT executives consider Strategic Alignment (SA) as a top priority in today’s competitive market [11]. In literature various definitions have been given to the concept of SA. Reich and Benbasth state that SA is “the degree to which the IT mission, objectives and plans support and are supported by business mission, objectives and plans” [9], while Henderson and Venkatraman in [6] provide a comprehensive framework to approach the alignment concept. In their framework, two distinct relationships are described: “strategic fit” and “functional integration”. Strategic fit is the external relationship concerned with the harmonization of business strategy choices (e.g. business scope, partnerships, alliances) and strategic choices concerning IS/IT deployment. Functional integration is the corresponding internal relationship concerned with organizational infrastructure and processes and IS/IT infrastructure and processes. As stated in [4] a crucial activity that helps managers to have an organization strategically aligned is the monitoring of the performances of their Business Process (BP). In fact, BP are designed and developed to satisfy a precise Business Strategy (BS), hence, the performance of a BP must conform or respect the related BS. It becomes clear that IT could play a fundamental role in the monitoring activity because it would provide managers with automated and ubiquitous instruments that
allow controlling the performance of the BP and its alignment with the BS expectations anytime they want. There are different approaches that can be used to identify what to measure of a BP, yet, unfortunately, they do not define how to relate the performance of the BP to its BS [1]. In this paper we address the problem of relating process performances to a BS presenting BP monitoring agents called Probes. Probes are monitoring agents deployed into the IT infrastructure that operate over a process, or part of it (e.g. a group of tasks), to control if it respects the organization strategic goals, objectives or constraints. We determined how to develop the Probes combining two engineering techniques, the Goal Question Metric (GQM) approach [2] and the Business Motivation Model (BMM) [8]. From an IT perspective, it is essential defining a method to identify these monitoring agents as it becomes easier to deploy them into the organization IT infrastructure, such as an Enterprise Resource Planning System (ERP) [5, 10]. Probes are included into the framework SAF [4] (Strategic Alignment Framework) that we developed in order to help IT Analyst to better understand the context of a company, thus helping them to: (i) align/make coherent organizations BP to organizations BS; (ii) align organizations IT infrastructure to organizations BP; (iii) create a strategically aligned IT infrastructure for those organizations newly founded. The paper exemplify (i) how Probes are developed and (ii) how Probes act into a real case study of a bus transportation company. The rest of the paper is structured as follows. In section 2 we introduce the concept of business process monitoring and we describe the state of the art in this field summarizing the most common techniques used by IT practitioners. In section 3 we describe the case study we considered. In section 4 we illustrate the approach we developed to create the Probes, and offer a practical example of Probes development. Finally we make our conclusions and identify the future works.
2
Theoretical Background
In the field of business process monitoring there are several techniques that aim to define the correct metrics that enables a high understanding of monitored process. In this section, we review the two top techniques used to monitor a process, namely the Balanced Scorecards and the Goal Question Metric approach. We also describe the Business Motivation Model, a formal technique developed to describe the strategic intents of a company, we used to relate the monitored performance of a process to its strategy. 2.1 The Balanced Scorecards Balanced Scorecards (BSc) were defined by their developers, Robert S. Kaplan and David P. Norton, as a multidimensional framework for describing, implementing and managing strategy at all level of enterprise by linking objectives, initiatives and measures to organization’s strategy [7]. The BSc provides a framework for studying a causal link analysis based on internal performance measurement through a set of
goals, drivers and indicators grouped into four different perspectives: (i) Financial; (ii) Customer; (iii) Internal processes; (iv) Learning and growth. The weakness of BSc is that its framework does not provide a constructive way to implement the strategy into the operational level. For example even though they use Key Performance Indicators (KPI) they do not discuss a method to derive and identify them. 2.2 The GQM and its evolution to the GQM+Strategy The Goal Question Metric approach [2] provides a top-down paradigm for an organization or a project to define goals, refine those goals down to specifications of data to be collected, and then analyze and interpret the resulting data with respect to the original goals. GQM goals are defined in terms of purpose, focus, object of study, viewpoint, and context. Such a goals are then refined into specific questions that must be answered in order to evaluate the achievement of the goal. The questions are then operationalized into specific quantitative measures. The GQM formalizes the deduction process that derives the appropriate measures to answer a given business goal. Traditionally, the GQM has been used in the measurement of the software development processes. Although it is a powerful instrument in measurement theory, the GQM lacks of an explicit support for integrating its measurement model with other organizational elements such as higher-level business goals, strategies, and assumptions [3]. That is why Basili et al. in [3] propose and describe a method that adds several extensions on top of the GQM model, thus developing the GQM+Strategy. The GQM+Strategy method makes the business goals, strategies, and corresponding software goals explicit. The GQM+Strategy method also makes the relationships between organization activities and measurement goals explicit. Sequence of activity necessary for accomplishing the goals are defined by the organization and embedded into scenarios in order to achieve some related goals. Links are established between each goal and the business-level strategy it supports. Attached to goals, strategies, and scenario at each level of the model is information about relationships between goals, relevant context factors, and assumptions. One of the weak point of the GQM+Strategy is that it is too software process specific and a refinement in its strategic overlayer is needed to use it in a more wide context. 2.3 The Business Motivation Model The Business Motivation Model [8], according to its developers, the Business Rules Group, is a meta-model of the concepts essential for business governance. The BMM provides: 1. A vocabulary for governance including such concepts as “Influence”, “Assessment”, “Business policy”, “Strategy”, “Tactic”, “Goal”, and fact type that relate them, such as “Business policy governs Course of Action”. 2. Implicit support for an end-to-end process that runs:
a.
From recognition that an influencer (regulation, competition, environment, etc.) has an impact on the business; b. To implementing the reaction to that impact in business processes, business rules and organization responsibilities. 3. The basis for logical design of a repository for storage of BMMs for individual businesses. There are two major components of the BMM. The first is the Ends and Means of business plans. Among the Ends are things the enterprise wishes to achieve - for example, Goals and Objectives. Among the Means are things the enterprise will employ to achieve those Ends - for example, Strategies, Tactics, Business Policies, and Business Rules. The second is the Influencer that shape the elements of the business plans, and the Assessments made about the impacts of such Influencers on Ends and Means (i.e. Strengths, Weaknesses, Opportunities, and Threats). In its specifications BMM define KPI as a metric particularly important, and suggest the use of KPI to monitor the performance of processes. Unfortunately, BMM does not suggest how to link the strategic level it defines to the operational level of the processes, furthermore BMM does not provide a way to develop such a KPI
3
The Case Study
Dolomitesbus is (a pseudonym for) a public transportation company operating in a province of Northern Italy. Dolomitesbus has a bus .fleet of 290 units, and serves 60 routes connection every day. Each route is covered by two buses, one that works from 6 a.m. to 2 p.m., the other from 2 p.m. to 10 p.m. The mission of the company is to offer to the customer a high quality transportation service over the province territory. The fleet is subjected to an extreme mechanical wear due to the mountain routes that serves, hence the main issue for the company is to concentrate a lot of resources on the maintenance process in order to have efficient buses. The mechanical workshop, that is part of Dolomitesbus, is in charge of any type of maintenance operation (i.e. planned maintenance or damage repair), and it also has to guarantee at least two fully operative buses every day for each route in accordance to the main quality constraint given by the Provincial Council - that is the supervisor for the company activities. The maintenance tactic imposed by the managers is the following: “Make a planned maintenance to every bus at most once per year - hence respecting the minimum requirement of the law - every other maintenance has to be made only when a problem occurs and needs to be repaired”. Furthermore the company has defined a feedback mechanism where the customers can report their complaints. Till the end of 2007 the company has never inspected and properly used the resulting complaints. For the last decade, the buses capacity reaches its critical load in the period of June/August. As such, the high number of passengers that are served in this period affect the company’s activities in two ways: (i) Increase the mechanical waste of the buses - due to the increase of the weight loaded; (ii) Increase the attention over the quality of the service offered - due to the increase of the customers transported.
Dolomitesbus has an efficient IT department that has always supported all the company’s software needs also adopting and developing Open Source Software. At the time the case study started, the company had its own in house ERP system that covered the activities of its departments but the mainte-nance process in the workshop. The operations of maintenance and repair done over a bus where only recorded manually with a paper schedule (containing fields such: bus number, type of operation, operation starting date, operation end date, details, etc.), not always properly filled, and then stored into an archive. As the first step of this phase we interviewed the main responsibles of Dolomitesbus in order to better understand the various aspects of the company context in the maintenance process. For this we developed the GQM to characterize the maintenance business process. 1. Measurement Goal: Analyze the bus maintenance process in the context of the company workshop to characterize it in terms of number of buses under maintenance. 1.1. Question: How many maintenance operations are performed periodically by type of operation? 1.1.1. Metric: Number of maintenance operations per day per type. 1.1.2. Metric: Yearly number of maintenance operations per type. 2. Measurement Goal: Analyze the bus maintenance process in the context of the company workshop to characterize it in terms of time spent to operate over a bus. 2.1. Question: How much does each type of operation last? 2.1.1. Metric: Time duration per type. The information needed to answer to the questions of the GQM were retrieved from the paper schedule. As mentioned, the schedule was designed with the following fields to be filled:(i) Bus number; (ii) Current Km; (iii) Operation Starting/Ending Date (hh.dd.mm.yyy); (iv) Total hours spent for the operation; (v) Type of operation checkbox chosen between planned maintenance and damage repair; (vi) Detailed description of the operations; (vii) Mechanic responsible for the operation. The paper schedule was used with the purpose of having an historical record of the operations done over the buses. During the conversion to the electronic format of the paper schedules we found out that some fields were not properly filled or not even considered at all. The fields left blank were: Current Km; Hours of starting and ending date; Total hours spent for the operation. The fields not properly filled were: Detailed description of the operations when filled it included a list of material used for the operation instead of a detailed description of the problem encountered; Mechanic responsible for the operation - it included an unreadable signature but not the cursive name and surname. The fact that the workshop mechanics have never filled the schedules in a proper way prove the fact that these have received a scarce importance or consideration. Namely, the schedules have never been used for any managerial analysis to establish the performance of the maintenance process. Furthermore, the bad compilation of the schedules has determined the permanent loss of important information that would have been useful for our analysis. The results obtained by the application of the GQM are summarized in Table 1. We then reported to the company managers our suggestions, as reported in detail in the following section.
Table 1. Metrics of the GQM collected.
Description (PM: Planned Maintenance; MfD: Maintenance
2006
2007
‘06/‘07
for Damages)
1.1 Question: How many maintenance operations are done each day divided by type of operations? Mean number of buses daily operated for PM 40 33 41 Median number of buses daily operated for PM 45 48 40 Mean number of buses daily operated for a MfD 90 91 101 Median number of buses daily operated for a MfD 115 111 100 1.2 Question: How many maintenance operations are done each year divided by type of operation? Total number of buses yearly operated for a PM 370 330 350 Total number of buses yearly operated for MfD 2360 2200 2280 2.1 Question: How much does each type of operation last? Mean duration in days for operations of type PM 39 69 52 Median duration in days for operations of type PM 7 16 9 Mean duration in days for operations of type MfD 13 20 16 Median duration in days for operations of type MfD 1 1 1
3.1
Data Analysis
The data collected for Metrics 1.1.1 shows that the median number of buses daily operated is 40 for those under a planned maintenance and 100 for those under a maintenance for damages. In figure 1 is represented a line chart of (i) the daily number of buses having planned maintenance, (ii) the daily number of buses having a maintenance for damages, and (iii) the sum of the previous two. In figure 1 we also show the threshold of the maximum number of buses that should be under maintenance every day without affecting the company mission. Dolomitesbus should have 120 buses (2 buses for each of the 60 routes) fully maintained every day in order to offer to its customers a high quality service, hence the maximum number of buses under maintenance every day is 170 (given by the 290 buses of the fleet minus the 120 fully maintained each day). From June until the end of August the threshold is of 150 buses, since the high number of customers in that period requires an increase to 140 buses fully maintained each day. If we consider these two levels for the thresholds as a quality constraint over which the quality of the service is not guaranteed, we counted that in 2006 the limit was exceeded for 63 times (55 during summer time and 8 during the rest of the year), while in 2007 the limit was exceeded for 151 times (85 during summer time and 66 during the rest of the year). That means that for 214 days during the biennium 2006-2007 the routes were served with buses not completely maintained and that could have negatively affected the customers’ quality perception. As confirmed by the managers feedbacks, during that particular days some of the 60 routes where covered by buses with a maintenance operation opened. In those cases the workshop manager
selected the buses with the less significant inefficiency in order not to affect the road and passengers security.
Number of Busses with an opened operation
300
250
200
150
100
50
1-Dec-2007
1-Oct-2007
1-Nov-2007
1-Sep-2007
1-Jul-2007
1-Aug-2007
1-Jun-2007
1-Apr-2007
1-May-2007
1-Mar-2007
1-Jan-2007
1-Feb-2007
1-Dec-2006
1-Oct-2006
1-Nov-2006
1-Sep-2006
1-Jul-2006
1-Aug-2006
1-Jun-2006
1-Apr-2006
1-May-2006
1-Mar-2006
1-Jan-2006
1-Feb-2006
0
Scheduled Operations for General Maintenance Total Operations Operations for Damages or Incovinients Maximum number of buses that should have a maintenance operation opened
Figure 1. Number of Maintenance Operations done in 2006 and 2007.
The data collected for Metrics 1.1.2 shows that in the years 2006 and 2007 the mean of total planned maintenance operations is 350, and the mean of total operation for maintenance for damages is 2280. Considering these numbers over the bus fleet we can notice that every year each bus is subjected to a mean of 1,2 operations for planned maintenance and 7,6 operations for maintenance for damages. The data collected for Metrics 2.1.1 shows that the mean duration for the planned maintenance operation is 52 days, while the mean duration for the operation of type maintenance for damages is 16 day. That means that every bus spend a mean of 69 days per year with an unsolved inefficiency - calculated multiplying the mean duration of the operation per the mean number of operation that a bus is subjected each year. Figure 1 clearly shows a different trend on the number of maintenance during 2006 and 2007. The managers of Dolomitesbus expected a difference between the two years, and they knew the causes. In fact, during an inspection at the company workshop at the end of 2006 they found out that the workshop managers did not spend much attention in filling the maintenance paper schedule. After the inspection, the top management imposed to the workshop manager more accuracy in filling the
schedule and wrote down and defined a first attempt of business strategy related to the maintenance process. In section 6.1 in figure 3 we modeled the business strategy for the year 2007.
4
Developing the Probes for our Case Study
In this section, we show the approach we defined to develop the monitoring Probes applied to the case study explained in section 3. After the managers defined a new business strategy for the maintenance process and after Dolomitesbus IT Department implemented the new functionalities into the ERP system we defined the type of Probes useful for the managerial control. To develop the Probe we first identified the business strategy that governs the maintenance process by modeling it with a standard notation called Business Motivation Model as described in Fig. 2. On the top of the figure is represented the mission of the company that can be achieved by means of some strategies. In the figure we included the specific strategy that refers to the maintenance process. On the bottom of the figure we then described the tactics, and the relative constraints, that the company needs to respect to fulfill the above strategy. On the left side we show the tactics used during the year 2007, while on the right side we show the tactic developed after our intervention at the end of 2007. For the year 2008 the managers decided to adopt three new tactics with the relative constraints related to the maintenance process. Here follows the list of the new tactics and constraints as reported in figure 2: • Tactic one (T1) imposes that of the 290 buses of the fleet only 240 are maintained by Dolomitesbus workshop, the remaining 50 needs to be maintained by a different workshop. The constraint for this tactic tells that the external workshop must guarantee at least 40 buses fully maintained every day, while the internal workshop must guarantee at least 90 buses fully maintained every day. • Tactic two (T2) imposes that in case one of the threshold given in the constraint for the tactic one is not respected then two or three routes must be aggregated in couples or triples. The aggregated routes must be served by a bus that works over its time limit. • Tactic three (T3) imposes that the manager must revise tactic one in case the company receives too many customers complaints. The constraint for this tactic is given by a threshold risk of 5%. We have adopted as additional overlayer of the GQM+Strategy the tactics that the managers defined for 2008 and we have then applied the GQM method to determine the metrics to be monitored. The logic of the threshold for the reporting function of the Probes has been derived from the business constraints that governs the relating tactics. Here it follows the structure of the GQM that determines the metrics to be measured by the Probes and the related threshold. -Probe 1 - Measurement Goal (MG) for T1: Analyze the maintenance process from the point of view of the managers in order to understand the level of maintained buses in the context of Dolomitesbus Workshop. Question 1.1: How many buses are under maintenance at Dolomitesbus workshop every day divided by typology of
maintenance? Metric 1.1.1:(Absolute) Number of Buses by day and typology. Threshold for this metric: According to CfT1 the max limit is 150. -Probe 2 - MG for T1: Analyze the maintenance process from the point of view of the managers in order to understand the level of maintained buses in the context of External Workshop. Question 2.1: How many buses are under maintenance at the external workshop every day divided by typology of maintenance? Metric 2.1.1: (Absolute) Number of Buses by day and typology. Threshold for this metric: According to CfT1 the max limit is 10.
Figure 2. Defining Probes
-Probe 3 - MG 1 for T2: Analyze the maintenance process from the point of view of the managers in order to understand the level of aggregation of bus routes caused by the unavailability of maintained buses (if there are not enough buses available the routes can be aggregated in couples or triples and served by a bus that works over it time limit). Question 3.1: How many bus routes are aggregated each day distributed by couples or triples? Metric 3.1.1: (Absolute) Number of bus routes. Threshold for this metric: The max limit can be defined after a simulation. -Probe 4 - MG 1 for T3: Analyze the maintenance process from the point of view of the managers in order to understand the level of customers complaints. Question
4.1: How many complaints there are every day? Metric 4.1.1: (Absolute) Number of Complaints. Threshold for this metric: Below a given threshold risk (i.e. 5%). Probes were developed using the model in figure 2. It is now under development a functionality that will send a warning mail to the managers if the indicators reach a below-threshold value.
5
The New Business & IT Set Up
In this phase we measured the performance of the maintenance process during the first three quarters of 2008. Table 2. Metrics collected by the probes in the first three quarters of 2008
Probe Metric Probe 1 Metric 1.1.1 Probe 2 Metric 1.1.2 P. 1 & 2 Metric 1.1.1+1.1.2
PM ‘08 20 6 26
MfD ‘08 PM+MfD ‘08 ‘06/‘07 50 70 N.A. 4 10 N.A. 54 80 140
Δ N.A. N.A. -43%
Table 2 shows the metrics values collected by the probes developed in section 6.1. The values collected in 2008 were compared, if possible, to the average value of 2006/2007. Probe 1 has measured that every day inside Dolomitesbus workshop there is an median of 70 buses that are subjected to a maintenance process. Probe 2 has measured that every day inside the external workshop there is a median of 10 buses that are subjected to a maintenance process. Summing the metric collected by the probes 1 and 2 we observe that the adoption of new business strategies defined by the company managers have determined a drastically decrease of 43% of the buses that are under any type of maintenance every day. That means that the choice of externalizing part of the maintenance process has determined a decrease not linear of buses present in the workshop every day for a maintenance. Furthermore, during 2008 Dolomitesbus have never exceeded the maximum threshold of buses that daily could be under maintenance. Probe 3, that is not represented in table 2, reported that no routes were aggregated neither in couples nor in triples. That is due to the fact that Dolomitesbus managed to constantly have all the efficient buses required. This metric is derived from the new strategy adopted for 2008, so it is not possible to compare this value to any previous year. Probe 4, that is not represented in table 2, reported that no complaints were presented from the customers. The quantity of complaints collected by this probe during 2008 cannot be compared to those of 2006/2007 since in these years the company has never stored the complaints received. The performances measured are the results of the combination of the following factors: 1. The analysis of the maintenance process taken in phase 1;
2. 3.
6
The adoption of significant changes (i) on the company business strategy concerning the main-tenance process, and (ii) on the relating business process; The development of new functionalities having the peculiar characteristic that included the probes for the existing ERP system.
Conclusion
This paper addresses the issue of monitoring business process performance in relation to strategic objectives. The solution we proposed is the development of monitoring agents called Probes. Probes are developed combining the technique of BMM and of the GQM. We present the development of Probes by means of a practical case study of a bus company. The company defined the Probes and integrated them into its ERP System in order to have a real time control of its strategic objectives. According to the company managers of our case study, Probes were able to enhance the overall organization strategy understanding, allowing the managers to take effective action to change inappropriate behaviors over the company processes. The integration of Probes into a company IT infrastructure require a customization of the ERP System, hence an investment of resources is required by the company. The benefit of developing Probes is that they help managers to better understand the achievement of the strategies they development, but Probes benefits are limited if not combined with proper managers actions and decisions. One of the future topic of research will be the automation of the Probes. The automation can be done by means of an appropriate ontology reasoner that first analyze the company business strategy written in a controlled natural language and then check if some metrics of the business processes respect the business strategy.
References 1.
2. 3.
4.
5. 6.
7.
Artigliano, F., Ceravolo, P., Fugazza, C., & Storelli, D. (2008). Business Metrics Discovery by Business Rules. Emerging Technologies and Information Systems for the Knowledge Society (p. 395-402). Athens, Greece: Springer Berlin / Heidelberg. Basili, V., Caldiera, G., & Rombach, D. Goal Question Metric Paradigm. Encyclopedia of Software Engineering. 1. John Wiley and Sons. Basili, V., Heindrich, J., Lindvall, M., Münch, J., Regardie, M., Rombach, D., et al. (2007). Bridging the Gap Between Business Strategy and Software Development. International Conference on Information Systems Management. Québec, Canada. Damiani, E., Mulazzani, F., Russo, B., & Succi, G. (2008). SAF: Strategic Alignment Framework for Monitoring Organizations. 11th International Conference on Business Information Systems. Insbruck, AT: Springer. Gross, H. G., Melideo, M., & Sillitti, A. (2005). Self Certification and Trust in Component Procurement. Journal of Science of Computer Programming , 56, 141 - 156. Henderson, J. C., & Venkatraman, N. (1993). Strategic alignment: Leveraging information technology for transforming organizations. IBM Systems Journal , 32 (1), 472--484. Kaplan, R. S., & Norton, D. P. (1996). The Balanced Scorecard: Translating Strategy into Action. Boston: Harvard Business School Press.
8. 9. 10.
11.
OMG. (2006). Business Motivation Model (BMM). Retrieved from http://www.omg.org/docs/dtc/06-08-03.pdf Reich, B. H., & Benbasat, I. (1996). Measuring the Linkage Between Business and Information Technology Objectives. MIS Quarterly , 20 (1), 55-81. Rossi, B., Russo, B., & Succi, G. (2007). Evaluation of a Migration to Open Source Software. In K. St.Amant, Handbook of Research on Open Source Software: Technological, Economic, and Social Perspectives (p. 728). IGI Global. Silvius, G. (2007). Business & IT Alignment in theory and practice. Proceedings of the 40th Hawaii International Conference on System Sciences, (p. 211b). Washington, DC, USA.
