On the Evaluation of Workflow Systems in Business Processes Sunil Choennia,b,*, Robin Bakkera and Walter Baetsa a Nyenrode University, Breukelen, The Netherlands b University of Twente, Enschede, The Netherlands
[email protected] [email protected] [email protected] Abstract. Although it is widely accepted that workflow systems add value to business processes, no substantial research has been reported in the literature that confirms this. Most of the efforts in the field of workflow systems are devoted to issues that are relevant to the design and implementation of this type of systems. Companies have a practical need to gain insight in to what extent workflow systems improve their business processes. In this paper, we present a model that measures the added value of workflow systems to business processes. We apply the model on two real-life cases in an IT company in the Netherlands. Although our model can be advanced with regard to several issues, the application of the model on the real-life cases resulted in valuable insight with regard to the impact of workflow systems for this company. From these cases we observed that value creation by a workflow system is not something to be taken for granted. Keywords: evaluation of IT, workflow management systems, performance objectives/metrics, business processes
1. Introduction Today, customers are increasingly demanding tailor-made products and services of the highest quality, delivered in a short time (Casati 1998). Additionally, the creation of a global market forces managers to continuously improve the performance of business processes. To meet this demand, companies put a lot of effort in integrating promising novel techniques and tools in their business processes. It is believed that workflow systems may considerably improve the efficiency and effectiveness of today’s business processes. The expectations that different type of organizations have regarding workflow systems are listed in (Lousa et al 2000) and some achievements of workflow management systems in the medical field are reported in (Graeber 1997). The goal of a workflow management system is the coordination of tasks and the routing of information (WfMC 1993, Grefen et al 1999). In order to introduce these systems successfully, it may be required for companies to reengineer their business activities. Issues concerning the re-engineering of business activities are studied extensively in the field of business process re-engineering (Hammer 1995). This paper is devoted to the impact of workflow management systems on business processes. Although it is widely accepted that wfms's add value to business processes of companies, no substantial research has been reported in the
www.ejise.com
literature that confirms this. Most of the efforts in the field of workflow systems are devoted to (technical) issues that are relevant to the design and implementation of this type of systems and/or formal modeling of workflows see e.g., (Aalst 1998, Eshuis 2002, Geppert et al 1998, Grefen & Vries 1998, Muth et al 1998, Proceedings of the Ninth International Workshop on Research Issues on Data Engineering 1999, Reichert & Dadam 1998) or to tools and methodologies (used by companies) to implement workflow applications (Doherty & Perry 1998, Berger & Ellmer 1998). Research that focuses on the measurement of the impact of wfms's in companies and how to integrate these systems in business processes successfully is still in its childhood. We are aware of the following efforts in this direction (Kueng 1998, Sarmento & Machado 2000). While (Sarmento & Machado 2000) focuses on the organizational changes due to the introduction of a wfms, (Kueng 1998) reports on the impact of wfms's more extensively and in a broader context. In (Kueng 1998), the results of a qualitative study on the impact of wfms's on organizations, especially on employees, tasks, and structures, are reported. Kueng (Kueng 1998) conducted his research in two steps. In the first step, he gathered data by interviewing eight persons from eight different companies. He used this data to formulate a set of hypotheses. Then in the second step, he presented the hypotheses to eight persons again --- the majority of these persons being different from the persons initially interviewed however --- and asked them to comment on each hypothesis. The
©Academic Conferences Limited
Electronic Journal of Information Systems Evaluation Volume 6 Issue 2 (2003) 33-44 overall conclusion of this research was that wfms’s indeed have an impact on the quality and productivity of an organization. Furthermore, it was concluded that the introduction of wfms's changed the business processes as well. In this paper, we introduce a quantitative model to measure the added value of workflow management systems. This model is built on a number of so-called performance objectives as introduced in (Slack 2001) (see Section 2). A performance objective may be regarded as an influential parameter of business processes. Optimizing these performance objectives leads to the increase of the performance of business processes. Therefore, we measure in our model which performance objectives are improved or deteriorated by the introduction of wfms’s. Second, we applied our model on two data processing cases at an IT company, NTNT, in The Netherlands. NTNT is a middle size company. Its core business is developing and maintaining network infrastructures for clients like financial and retail corporations, and governmental departments. The development and implementation of wfms's is a part of their core business. Each of the two business processes to which we apply our model is used to communicate and store data about human resources. The first business process concerns the request for vacation by employees. Prior to communicating a notification of acceptance/rejection to an employee, various activities should be performed. The second business process concerns the fulfilling of vacancies in the company. As in the first case, a number of activities should be performed here in order to take a final decision with regard to an applicant. While in the first business process we could clearly observe value creation by the introduction of the wfms, this was not the case for the second business process. In the second business process, we observe that some groups of people benefit from the introduction of the wfms, while other groups are disadvantaged by its introduction. Therefore, value creation by a wfms is not something to be taken for granted. Although it is in general very hard to evaluate IT systems, including wfms’s (Irani et al 2002), our model may be regarded as a first step to measure the added value of an operational wfms in a company. The model provides a good insight into the advantages and disadvantages of making a wfms operational for above-mentioned cases. Such an insight may help companies to make their business
www.ejise.com
34
processes more efficient and effective in the (near) future. The approach that we have taken in measuring the impact of a wfms is a combination of the so-called cross-sectional and retrospective approaches (Coupe & Onudu 1997). Our approach is cross-sectional, since the performance objectives are given beforehand, and cannot be determined by the interviewees. Furthermore, we compare the values of the performance objectives before and after the implementation of a wfms. Therefore, our approach can be regarded as retrospective as well. The remainder of this paper is organized as follows. In Section 2, we discuss our model and the performance objectives on which it is based in more detail. Then, in Section 3, we apply the model to two cases at NTNT, and we discuss the obtained results. Finally, Section 4 concludes the paper.
2. Modelling added value The field of workflow management is closely related to the field of business process reengineering (BPR) (Hammer 1995). The goal of both fields is to optimize the processes within organizations by exploiting ICT. While BPR focuses on a significant restructuring of the operational processes in a firm, the field of workflow systems is more focused on the automation of the movement and processing of information required to perform the operational processes (information transformation, distribution, etc.). A major component of a workflow system is a workflow management system (wfms), which is actually in charge of the control and coordination of workflows. A workflow can be regarded as an operational business process. It should be clear that a workflow system might be a tool in a set of restructured operational processes. In the following, we use the terms workflow systems and workflow management systems interchangeably. It is believed that the introduction of a wfms in an organization will lead to an increased availability of management information. A wfms can also serve as a monitoring tool resulting in an increase in the monitoring possibilities. Furthermore, it is also claimed that work satisfaction might be increased, since boring tasks can be taken over by wfms's. However, research has pointed out that the organizational structure and the culture in an organization play a vital role in the successful introduction of novel technologies, and there is always a resistance to changes concerning the
©Academic Conferences Limited
35 structure and culture. An organization may be regarded as a set of people that have a collection of rights, privileges, obligations, and responsibilities that are delicately balanced over a period of time through conflict and conflict resolution (Laudon & Laudon 1999). In many organizations, the introduction of a wfms has as a consequence, a disruptive effect on the balance of the organization. This is especially the case if the introduction of the wfms will rigorously change the achieved collection of rights, privileges, obligations, and responsibilities. Therefore, it is sensible to follow a gradual approach in introducing a wfms in organizations. Also for measuring the impact of wfms’s a gradual approach is preferred, since managers want to know the effect of the wfms in a relatively short period after the introduction of these systems. We note that a rigorous introduction of a wfms may disrupt the delicately balanced collection of rights, privileges, obligations, and responsibilities, and obtaining a new balanced situation generally takes a long time. This makes it impossible to measure the impact of a wfms in a relatively short period. It is expected that organizations, in which employees work according to a pre-defined and formalized set of rules and procedures are better suited to a gradual approach in introducing wfms’s. Therefore, the machine bureaucracy (Kreitner 1999) offers the best chances for implementing a wfms successfully.1 In the following, the model that we will develop is aimed at measuring the impact of a wfms that has been gradually introduced in an organization. A second assumption that we make with regard to our model is that a business process can be divided into a set of basic activities. A basic activity is in its turn an amount of work that is uninterruptible and that is performed in a certain amount of time (that is greater than zero). Our model is an application of a number of performance objectives described by (Slack 2001). These objectives might be used to determine the improvement of business processes. We have encoded a number of these performance objectives into a model. We use this model to measure the added value of a wfms. The performance objectives that we considered in setting up our model are: speed, quality, flexibility, reliability, and cost. 1
In more knowledge-intensive type of organizations, current wfm systems should be extended with a number of additional features in order to be introduced successfully (Zhuge 2003).
www.ejise.com
Sunil Choenni, Robin Bakker & Walter Baets According to (Slack 2001), speed concerns the time between the moment a customer orders a product/service and the delivery of the product/service, i.e. the throughput time. In general, the speed can be improved by reducing waiting queues in business processes. The notion of quality refers to the degree in which each product or service is in accordance with the product specifications. To achieve a high and constant quality a set of standard procedures is often followed in manufacturing a product. The more stringent these procedures are applied, the better the quality of the product is rated. Flexibility refers to the ability of a process to be adapted to the changing needs of the customers. The notion of reliability measures the extent to which processes perform as expected, without unscheduled maintenance. Finally, the notion of cost is obvious. Each organization attempts to minimize its overall cost. In the following, we will not consider cost as an explicit performance objective for the time being, but will determine it on the basis of the other performance objectives. We apply the performance objectives to measure the quantitative effect of wfms’s in an organization according to the model in Figure 1. We assume that each of the performance objectives can be related to a cost. Then, for each of these objectives, we compute the cost that is involved with regard to a business process before and after implementation of a wfms. The difference in cost is the cost savings due to a specific performance objective. To compute the cost related to the speed of a business process, the throughput time of a process is measured before and after the implementation of the wfms. Since flexibility is characterized by the ability to adapt to changing environments, the time to adjust a business process to a (changing) environment is measured. The added value of a wfms with respect to flexibility may be measured by asking the following question: “What is the amount of time needed to adapt a business process to a (changing) environment, before and after the implementation of a wfms?” As noted before, reliability is associated to keeping the promises made to customers. To meet the promises, we generally require stable business processes. Therefore, we use the maintenance time that is required to assure a stable business process to express the cost associated to reliability. To compute the added value of a wfms with respect to reliability we use the following question as a guideline:
©Academic Conferences Limited
Electronic Journal of Information Systems Evaluation Volume 6 Issue 2 (2003) 33-44 “What is the maintenance time to assure a stable business process, before and after the implementation of a wfms?” To measure the quality of a business process, we keep track of the number of (visible) errors within or due to a business process. An error means that it has to be corrected, which entails additional work. In
order to determine the added value of a wfms with respect to the quality of a business process, we use the following question as guideline: “What is the amount of additional work that has to be done due to errors in a business process before and after the implementation of a wfms?”
Before implementation of a wfms
Cost related to a business process w.r.t. performance objective x, in which x ∈ speed, flexibility, reliability quality
36
After implementation of a wfms
Difference
Cost related to a business process w.r.t. performance objective x, in which x ∈ speed, flexibility, reliability quality
Cost saving on the basis of performance objective x Figure 1: Model used to determine the quantitative effect of a wfms The quantitative impact of a wfms on a business process is then expressed by the total cost. The value of the total cost is composed of the speed, flexibility, reliability and quality. In the following, we determine the cost by adding the cost savings due to the performance objectives speed, flexibility, reliability, and quality. We note that such a model is a first step in measuring the impact of wfms's on organizations. The impact on more qualitative type of parameters, such as employee satisfaction, entailed by wfms's is evenly important as quantitative parameters. Therefore, gaining insight into the impact of wfms's requires combining both types of parameters.
3. Two cases at NTNT We tested the model on two business processes within NTNT. NTNT is a Dutch consultancy company that specializes in the development and maintenance of computer networks. The two business processes are the process of an employee requesting (a) day(s) off (request for vacation) and the process of applying for a job at NTNT (application). Both processes fall under the responsibility of the human resources department. The wfm applications that were developed for the two processes have been in use since May 2000.
www.ejise.com
In Section 3.1 we describe the way we have collected our data used in the case studies. The two sections thereafter deal with each of the two business processes respectively. In these sections, we describe the processes in more detail and present our results concerning the added value of a wfms in these cases. Furthermore, we comment on the case studies and the obtained results.
3.1
Data collection
The model specifies that we compare processes before and after the implementation of the wfms. The wfms itself gathers data that may help us to rate the processes on the performance objectives identified in Section 2. However, no such data was available for the performance of the process before the implementation of the wfms. Therefore we gathered this data by conducting questionnaires. The advantage of the data from the wfms is that it is precise. That might not be the case for the data collected from the questionnaires. The data from the questionnaires are estimations made by employees to their best knowledge, and therefore may not always be precise. Ideally we would like to have precise data before and after the implementation of a wfms, since now from a theoretically point of view there is always a risk that the result might be biased.
©Academic Conferences Limited
37
Sunil Choenni, Robin Bakker & Walter Baets
However, it is widely accepted to rely on estimations of human experts, see e.g., (Choenni & Blanken 2000). Another issue that we had to take care of during the presentation of the questionnaires to employees is that the same question is regarded from a different perspective by the different stakeholders of a process (see Section 3.2 and 3.3). For this reason, we have made a distinction between ‘clients’ and ‘owners’ of the process in case of the request for vacation. The clients of the process are the employees requesting the vacation. The managers, who evaluate the requests, are called the owners (see Section 3.2). Such a distinction is however not possible for the second case study, the application process, since the applicants that were rejected are not available for filling out questionnaires.
one of the performance objectives we have identified. In Table 1, we briefly summarized the typical questions that were asked. For a full description of the questionnaires, we refer to (Land & Hulsker 2001). On the Intranet of the company, a website was set up on which the questions were published. Owners and clients were given access to this site in order to answer the questions. This simplified the collection and processing of the answers. At the moment that this research was conducted, NTNT consisted of 43 managers and 416 employees. We decided to invite a subset of employees to participate in our research for several reasons. First, not all of the managers and employees have been with NTNT long enough to have worked with the process before implementation of the wfms. Second, many employees of the company were in detachment elsewhere for a long time. This implies that the actual populations of which the respondents are a sample, is slightly smaller than the before-mentioned 43 managers and 416 employees. At the end, 5 owners and 35 clients responded (correctly) to the questionnaires.
Both the clients and the owners of the processes were asked to fill out a questionnaire. The questions we asked in the questionnaires are mostly related to the time spent by a person (either employee or manager) on a specific activity of the business process. Each of these activities is linked to Table 1: Summary of the type of questions included in the questionnaires
Speed: With regard to this performance objective most of the questions concerned the time it took from the moment a request / application was sent in and the moment the requestor/applicant was notified about the results. Typical questions to a manager were e.g., How long does it take you to process a request? If the evaluation of a request is completed, will you notify the requestor/applicant immediately? If not, why and when will you notify her/him about your evaluation? How do you communicate the notification (e.g., email, conventional mail)? Typical questions to an employee were e.g., When did you sent in a request? When were you notified that your request was received? When were you notified about the results of your request? Quality: With regard to this performance objective most of the questions concerned the lost of requests/applications. Typical questions to managers and employees were e.g., What kind of mistakes are made with an application/request? How often is such a mistake made? Who is responsible in resolving the mistake? Who resolves actually the mistake? Reliability: With regard to this performance objective most of the questions were focused on finding out how much time the respondents spend on proposing improvements for a process. Typical questions that were included in the questionnaires: Can the process be improved according to you? If yes, how can it be improved? How often do you submit change proposals with regard to the process? Flexibility: With regard to this performance objective we focus on the time needed to implement a change in a process. Typical questions that were included in the questionnaires: Have you changed something in the process? If yes, what was the change and how long did it took to implement the change? Was it easy to create support of others for the change?
3.2
Request for vacation
An employee fills out a request form to ask for vacation. This request form is evaluated by his
www.ejise.com
/ her manager. At the same time, personnel administration checks whether all the formal rules have been satisfied (for instance, does the employee have enough vacation days left).
©Academic Conferences Limited
Electronic Journal of Information Systems Evaluation Volume 6 Issue 2 (2003) 33-44 When both the employee’s manager and personnel administration have finished their evaluation, a message is sent to the employee confirming the acceptance or rejection of the
38
request. The implementation of the wfms has not changed this process. The process is schematically depicted in Figure 2 below:
Employee
Leave
Message
Message
Evaluate Personnel administration
Manager
Figure 2: Schematic representation of the “request for vacation” process
3.2.1
Results for request for vacation process
The results from the questionnaires are presented in Table 2. As stated before, a client is someone requesting vacation, whereas an owner is someone evaluating the request for vacation. The speed of the process is determined by the time spent by an employee to fill out the form plus the time between sending the request to the wfms and receiving a notification of acceptance. The quality of the process is Table 2: Results for the request for vacation process
The speed of the process Average time between sending request and receiving notification of acceptance (work days) Average time spent to fill out/process request form (minutes) The quality of the process Average number of redone requests per employee per year Average number of requests per employee per year The reliability of the process Average time proposing improvements for the process per person (hours per month) The flexibility of the process Average time needed to adjust the process (days)
When it comes to the speed of the process a possible explanation of the disparity lies in the
www.ejise.com
related to the number of requests per employee that need to be redone in a given timeframe, relative to the total amount of requests per employee in the same timeframe. In this case, we measure the number of requests per employee in a timeframe of one year. The time spent on detecting errors in and proposing potential improvements to the process is an indication of the reliability of the process. The less time needed, the higher the reliability. The flexibility, finally, is determined by the time needed to change the process (average time needed to adjust the process).
Opinion client Before After wfms wfms
Opinion owner Before After wfms wfms
3,9
4,1
3,8
2,3
7,7
8,4
6,5
1,1
0,7
2,7
2,3
1,3
5,9
5,9
-
-
0,1
21,8
15,4
0,0
-
-
6,7
0,3
way the wfms handles the request. The request is stored, but the manager does not
©Academic Conferences Limited
39 receive a message that a request has been received by the system. It is up to the manager to check for requests. Before implementation of the wfms the request was put on the manager’s desk, where he / she would see it the same day. So, in general, the manager will have a delay in notifying that he received a request compared to the situation before the implementation of the wfms. However, the handling of the request may be faster once the manager sees the request, since in an automated system a lot of processing can be done by the system. According to the manager the processing of a request has been sped up. Therefore the throughput time (the time between receiving a request and notifying the employee about the results) has been decreased in the perception of the manager, while this is not what the client experiences. The clients observe that the average time is increased between sending and receiving a notification of acceptance with regard to their requests, which is probably due to the fact that a manager notes a request with a certain delay. Furthermore, we observe from Table 2 that the average time to fill out a form by an employee has been increased. We like to note that the decrease in speed in the perception of the clients, how small it may be, is remarkable, since the expectation was that the introduction of a wfms would make the process more efficient, i.e., an increase of the speed was expected. The increase in the average time to fill out a form can be explained by the fact that the number of requests that needs to be redone has been increased. We continue our discussion with explaining this increase. Before the implementation the biggest reason for having to redo a request was the loss of the request somewhere in the process. Managers were confronted with this problem more often than the employees, since they handle the requests for all employees. This may have led to their higher estimation on the number of redone requests. After the implementation of the wfms requests no longer get lost2. However the interface and questions of the wfms are not yet familiar to the clients and are in some cases multiply interpretable, leading to requests not being properly filled out. Before the implementation of the wfms multiply interpretability not necessarily led to a redo of a request, if an item was not properly filled in. If something was not clear for a manager, he/she tended to ask an employee what he/she 2
Except in the case of a system failure, but this is related to system reliability.
www.ejise.com
Sunil Choenni, Robin Bakker & Walter Baets exactly had meant. Since the process has been automated the wfms simply rejects a request if it does not meet the imposed demands. Consequently, the request has to be redone. The different perception on the issue of the number of redone requests is due to the fact that a manager is not aware of the requests by employees that are rejected by the wfms, e.g., if some fields are not filled in, without consulting the manager. Therefore, managers may think requests are redone less, whereas employees actually have to redo more. This may also be the explanation for the discrepancy in opinion on the reliability and flexibility of the process before and after the implementation of the wfms between owners and clients. From Table 2, we note a shift with regard to the reliability of the vacation process. Before the implementation of the wfms, it were the owners who detected and suggested improvements for the process, while after the introduction of the wfms it appears that the clients are fulfilling this task. The fact that employees are confronted with an increased number of rejections concerning their vacation requests without intervention of the managers since the introduction of the wfms, implies that there is a mismatch between the wfms and what clients expected from it. So, clients report about this mismatch to their supervisors, and, therefore, they are now more actively involved in the process. This may explain the shift with regard to the reliability issue. In (Leijen & Baets 2003), a framework is described that may be used in bridging the gap between what is expected from an automated business process by clients and what is delivered by a system. According to the managers adapting the process to allow for a change in the organization for instance has become much less time consuming. This can be declared by the fact that many changes in the process became now just a matter of changing some parameters, e.g., removing or adding some fields in a form. Hence, from the perception of the owner the flexibility has increased. We have seen that the results of the implementation of the wfms for this process are ambivalent. In the opinion of clients and owners, some performance objectives have improved, while others have deteriorated. The question as to whether the wfms has added any value to the process is therefore not yet answered. As we have mentioned above, this question can never truly be answered by relying solely on quantitative data. However we
©Academic Conferences Limited
Electronic Journal of Information Systems Evaluation Volume 6 Issue 2 (2003) 33-44 would like to draw a partial conclusion based on our data, which may then be used in an overall evaluation of the wfms. We can do this by calculating the costs of every hour (or minute, or day) spend longer on each activity after the implementation of the wfms, and the benefits of every hour spend less (1 We note that we actually compute the value of the implicit performance objective cost here, as defined in Section 2.)
40
For instance, the average employee now spends 21,7 hours more on the reliability of the process. However the average manager spends 153,8 hours less time on this activity. If we have 416 employees and 43 managers and suppose that the average salaries and related costs are 30 and 50 euros per hour respectively (fictitious numbers), the added value of the wfms for this activity of the process is:
((time won by manager) * (# of managers) * (average salary of manager per hour)) + ((time won by employee) * (# of employees) * (average salary of employee per hour)) or, (153,8*43*50) + (-21,7*416*30) = 59.854 euros.
We can do the same for each activity and come to a conclusion regarding the quantitative value added of the wfms for this process. Given the delicate nature of salary numbers we haven’t been able to perform above-mentioned computation, but it should be clear that the computation is straightforward. We may conclude that value creation depends on whom you ask, client or owner. Wfms’s seem to help the owners mostly. To come to a conclusion regarding the value creation of a wfms we therefore need to do a cost-benefit analysis. The costs are determined by the extra time spent by clients and owners. The benefits are determined by the time won by clients and owners.
3.3
Applicant process
The “applicant process” is the set of rules and procedures that is used to determine the
suitability of applicants for a vacancy within NTNT. In total there were 1608 applicants in the wfms. Of these, 251 were hired and 1357 were rejected. An applicant sends his or her curriculum vitae (CV) to the personnel administration that enters the CV into the wfms. The CV is assessed by an employee of personnel administration on the suitability for a vacancy. If a suitable vacancy is found one or two appointments are planned with a personnel manager and with one or more department managers. If the result of these appointments is positive an offer is made. The applicant can either reject or accept the offer. This description applies to the situation before and after implementation of the wfms. The schematic representation of the “applicant process” can be found in Figure 3.
Applicant
One or two interviews for a job are conducted.
CV’s
Personnel employee
Personnel manager
One or two appointments are made.
Figure 3: Schematic representation of the application process
www.ejise.com
©Academic Conferences Limited
41
3.3.1
Sunil Choenni, Robin Bakker & Walter Baets
Results for the applicant process
The results for this business process are summarized in Table 3 below. We note that the numbers for the situation after implementation of the wfms are mostly based on data from the wfms itself (reliability and flexibility are the exceptions). Within the wfms no data exists for instance on the number of applications that are lost and are never entered into the system. The speed of the process is determined by the time between an applicant sending in an application and an applicant receiving a notification of acceptance. Since we do not Table 3: Results for the applicant process
know exactly when an application was sent, or a notification received, we use the time between receiving the application and sending the notification. The relative amount of applications that are lost and the time spend to recover one lost application are an indication of the quality of the process. Different managers and employees of the personnel administration are involved in the process. Communication between them is a very important factor for the reliability of the process. The flexibility is determined by the amount of time needed to adjust the process.
The speed of the process Average time between receiving application and sending notification of acceptance (work days) The quality of the process Number of applications lost per 100 Time to recover one application (hours) The reliability of the process Average time spent to ensure smooth communications (hours per month) The flexibility of the process Time needed to change the process
The average time between receiving an application and notifying the applicant about the decision shows a significant increase after the implementation of the wfms. Part of the reason for this could be a change in the organization and its environment. Before the implementation of the wfms, NTNT was growing rapidly. This might have resulted in more pressure to hire people and thus a shorter process time. As explained above, no data is available for the number of applications lost before they are entered in to the wfms. Therefore the data for this performance objective can not be compared to the situation before the implementation of the wfms. This is also true for the time spent to recover the application, although we may assume that this would be the same for both situations. Both the flexibility and the reliability have increased since the implementation of the wfms. Here it seems to be easier to draw a conclusion than in the first case. It seems obvious that the wfms has created value in this case since all performance objectives, except for the speed, have improved. Furthermore the deterioration of speed objective might be
www.ejise.com
Before wfms
After wfms
10 work days
29 workdays
9 2
0 NA
2
0
2 work days-
5 minutes
explained by environmental factors. However, we know nothing about the opinions of applicants about this process. As we have seen above they could have a very different opinion on the value of the wfms. Still we feel that the wfms has added value in this case.
4. Conclusions and further research It is expected that the introduction of workflow systems will improve the efficiency and effectiveness of today’s business processes. Although there is a practical need to evaluate the impact of workflow systems on business processes, no substantial research has been reported on the evaluation of workflow systems. Most of the research in the field of workflow systems focuses on technical issues or on the development and implementation of workflows systems into an information system. In this paper, we have developed a quantitative model to provide insight into the added value of workflow systems. The model is based on four explicit performance objectives, namely speed, flexibility, reliability, and quality, and one implicit performance objective, namely cost. A notion of cost is associated with each of the explicit
©Academic Conferences Limited
Electronic Journal of Information Systems Evaluation Volume 6 Issue 2 (2003) 33-44 performance objectives. The costs associated with each performance objective before and after the implementation of a workflow system are measured. From these measurements, a total cost value is composed. This value provides, in a quantitative sense, insight into the impact of workflow systems on business processes. Organizations that have gradually introduced a workflow system can use the model to assess the added value of the system. We applied the model to two business processes at an IT company in the Netherlands. The two business processes to which we applied our model are both used to communicate and store data about human resources. The first business process concerns the request for vacation by employees. Prior to communicating a notification of acceptance/rejection to an employee, various activities should be performed. The second business process concerns the fulfilling of vacancies in the company. As in the first case, a number of activities should be performed in order to take a final decision with regard to an applicant. In both cases, our model provides insight into the quantitative effects of the introduction of a workflow system at NTNT. From the “request for vacation” case, we observe that the workflow system creates value for some groups of employees, but apparently diminishes value for other groups of employees. The decrease in value is primarily due to the restriction of the communication process between the different groups of employees; in this case between employees who are requesting vacation (clients) and managers who have to evaluate the request (owners). Especially issues/terms that may lead to multiple interpretations are a source of adverse effects on a number of performance objectives according to the clients. Another reason for the decrease in value is that although the time to process a vacation request is significantly decreased, the throughput time of a request has been increased in the perception of the clients. This is due to the fact that a manager notifies a request with a certain delay. Therefore, we conclude that a workflow system does not always offer an added value. For the second business process, the fulfilling of vacancies, it indeed appears that the introduction of the wfms adds value to this process according to the managers from different department at NTNT.
www.ejise.com
42
Topics for further research are an elaboration of quantitative models and the development of a model that is capable of assessing the qualitative impact that workflow systems have on organizations, such as employee satisfaction and increased availability of management information. The latter model is required to obtain a more complete overview of the impact of workflow systems in organizations.
5. Acknowledgement The authors would like to express their gratitude towards Johan van ‘t Land, AukeJan Hulsker and Leo Buytelaar for their work in conducting the interviews and setting up and distributing the questionnaires.
References Aalst, W., van der, The application of Petri nets to workflow management, The Journal of Circuits, Systems, and Computers, 8(1), 1998. Berger M., Ellmer, E., Closing the Gap between Business Modeling and Workflow Enactment, Proc. ECIS 98, European Con. On Information Systems, Aix-en-Provence, France. Casati, F., Models, Semantics, and Formal Methods for the design of Workflows and their exceptions, Ph.D. thesis, Politecnico de Milano, 1998. Choenni, R., Blanken, H., Modeling Uncertainty in Physical Database Design, KRDB 2000, Int. Workshop Knowledge Representation Meets Databases, Berlin, Germany. Coupe, R., Onudu, N., Evaluating the impact of CASE, an empirical comparison of retrospective and cross-sectional survey approaches, European Journal of Information Systems 6(1), 1997, pp. 15-24. Doherty N.F., Perry, I., The Uptake and Application of Workflow Management Systems in the UK Financial Sector, Proc. ECIS 98, European Con. On Information Systems, Aix-enProvence, France. Eshuis, R., Semantics and Verification of UML Activity Diagrams for Workflow Modeling, Ph.D. thesis, University of Twente, 2002. Geppert, A., Tombros, D., Dittrich, K., Defining the Semantics of Reactive Components in Event-driven Workflow Execution with Event Histories, Information Systems 23 (3/4), 1998.
©Academic Conferences Limited
43 Graeber, S., The Impact of Workflow Management Systems on the Design of Hospital Information Systems, Unversity of Saarland, Germany. Grefen, P., Pernice, B. and Sanchez, G. (eds) (1999) “Database Support for Workflow Management: The WIDE Project”, Kluwer Acadamic Publishers. Grefen, P., Vries, R. de Reference Architecture for Workflow Management Systems, Int. Journal of Data & Knowledge Engineering, 27(1), 1998. Hammer, M. , Reengineering the Corporation : A Manifesto for Business Revolution, Nicholas Breadley Publishing, 1995. Irani, Z., Themistocleaus, M., Love, P., Jones, Z., Information Systems Evaluations: Minitrack Introduction, in Proc. 8th Americas Conf. On Information Systems, 2002. Kreitner, R. , Organizational Behaviour, 1st edition, 1999. Kueng, P., Impact of Workflow Systems on People, Task, and Structure: a postimplementation evaluation, Proc. 5th European Conf. On the Evaluation of Information Technology, pp. 67-75. Land, J. van ‘t, Hulsker, A., The Added Value of Workflow Management Systems, M.Sc. thesis, Nyenrode University, Breukelen, 2001 (in Dutch). Leijen, H., van, Baets, W., A Cognitive Framework for Re-engineering Knowledge Intensive Processes, Proc. HICSS36, Hawaii Int. Conf. On System Sciences, 2003.
www.ejise.com
Sunil Choenni, Robin Bakker & Walter Baets Laudon, K.C., Laudon, J.P., Essentials of Management Information Systems, 3rd edition, New Jersey, 1999 Lousa, M., Sarmento, A., Machado, A., Expectations Towards the Adoption of WorkflowSystems: the Results of a Case Study, Proc. CRIWG 00, 6th Int . Workshop on Groupware, IEEE Press, Los Altimos, USA. Muth P., et al. From Centralized Workflow Specification to Distributed Workflow Execution, Int. Journal of Intelligent Information Systems 10(2), 1998. Proceedings of the Ninth International Workshop on Research Issues on Data Engineering: Information Technology for Virtual Enterprises, 23 - 24 March, 1999, Sydney, Australia, IEEE Computer Society, 1999 Reichert M., Dadam P., Adept-flex --Supporting Dynamic Changes of Workflows without Loosing Control, Int. Journal of Intelligent Information Systems 10(2), 1998. Sarmento, A., Machado, A., Impact Evaluation of Organisational Changes Enabled by Workflow Systems, Proc. CRIWG 00, 6th Int . Workshop on Groupware, IEEE Press, Los Altimos, USA. Slack, N., Operations Management. 3rd edition, London, 2001 WfMC, The workflow reference model, http://www.wfmc.org. Zhuge, H., Workflow- and Agent-based Cognitive Flow Management for Distributed Team Cooperation, Int. J. Information & Management 40(5), 2003.
©Academic Conferences Limited
Electronic Journal of Information Systems Evaluation Volume 6 Issue 2 (2003) 33-44
www.ejise.com
44
©Academic Conferences Limited
