and tools which accompany its application are rarely formalized. Therefore, this paper presents a method of diagnosis and three methods of data analysis in ...
Methods and Tools for First Five Steps of Benchmarking Process Jean-Luc Maire
Gülçin Büyüközkan
LLP / CESALP Université de Savoie ESIA - BP 806 - 41, avenue de la Plaine 74016 Annecy Cedex - FRANCE
ENSGI - INPG - IPI 46, avenue Félix Viallet 38031 Grenoble - FRANCE Galatasaray University Ciragan cad. No:102 80840 Istanbul - TURKEY
Abstract-Even though the idea of benchmarking is not a new one to improve the performance of enterprises, the methods and tools which accompany its application are rarely formalized. Therefore, this paper presents a method of diagnosis and three methods of data analysis in order to realize the first five steps of the benchmarking process.
Benchmarking process, as described in Balm [2] is divided in fifteen steps that we regroup in five phases : self-analysis, pre-benchmarking, benchmarking, post-benchmarking, observation and adjustment. Our work targets actually the first two phases of this process, that means, on the one hand, the measurement and the analysis of internal performance of the enterprise’s main processes, and on the other hand, the selection of subject and partners of a benchmarking study.
I. INTRODUCTION Today, many enterprises confront serious difficulties when they decide to undertake a continuous improvement of their performance. Among the approaches which may help an enterprise to improve its performance, benchmarking is required today as one of the most efficient and effective management tool. But, even though this technique is powerful and may introduce new paths, there are few European enterprises which utilize it. There exist certain misconceptions and obstructions against benchmarking approach. Many industrials think that their business processes are very company specific and different companies have nothing to learn from each other's practices. Then, it is not ethical to look into the technology and manufacturing methodology of other companies. Confidentiality of the technological know-how should be strictly respected by others. Finally, benchmarking methodology seems often, it is maybe the most important point, lacking formal modeling tools and theoretical foundations which make difficult the transposition and the reuse of realized experiments. For this reason, the aim of this paper is to formalize the benchmarking process and to propose the methods and tools accompanying the steps of that process.
A. Phase I : Self Analysis Three steps of this first phase measure and analyze the internal performance of the enterprise as a baseline to compare to others. 1) General description: a) Step 1: To define activities, customers and results Probably the single most common mistake in benchmarking is to begin studying someone else before understanding our own processes. We must first understand in great detail our own organization : how we do, what we do; what our problems and strengths are; our measures and our data. Utilization of a model appears indispensable for obtaining a representation of our organization from which we may determine specifically what we want to learn to others. Each enterprise actor manages at least one activity producing some results given to external or internal customers of the enterprise. To perform an improvement approach consists first in making sure that internal and external customers are satisfied. In fact, a necessary condition, but not sufficient, for external customers to be satisfied of the product or of the delivered service is that all internal customers distributed along the processing chain are satisfied. Therefore, this first step has to end, by realizing an organization modeling, to a precise definition of the main activities of the enterprise, their inputs and outputs, and suppliers and customers of these inputs and outputs. b) Step 2: To define good measure This step consists of defining a system of measures allowing the evaluation of our enterprise activities
II. BENCHMARKING PROCESS Benchmarking is a continuous process of evaluation of products, services and practices with respect to those of the strongest competitors or of the enterprises recognized as leaders [1]. Either the enterprise adopts these practices, or it adapts them with the aim of improving its performance. In a direct way, the benchmarking is a process of evaluation and improvement of performance.
1
results (especially through the satisfaction level of customers using these results) and allowing to compare them with other enterprises. These quantitative or qualitative measures have to be sufficiently general to share with benchmarking partners and sufficiently specific to produce a significant evaluation. Customer satisfaction, profit performance, freedom from deficiencies, product features, operations and equipment are some of the performance measures [3]. It seems important for us to complete the measurement of external customer satisfaction of the enterprise, very utilized in today's benchmarking applications, with the internal customer satisfaction. c) Step 3: To revise and improve the current enterprise performance On the basis of steps 1 and 2 results, an analysis of our current enterprise performance through graphic outputs has to be undertaken. This analysis leads to a diagnosis of principal enterprise malfunctions and to the proposal of a scheme of corrective actions for these malfunctions. In this step, we examine the factors that influence performance to learn which characteristics are most important and which are least important. The collected performance data create the baseline and structure for benchmarking comparisons [4].
input) and the satisfaction level (as output) and these two notions define the strategic aspects of the CSIS. The interview is realized according to the following structure : Activities What are the activity areas in which you take part ? For each activity area, what tasks are assigned to you ? Exchange relationships Utilization aspect For each function of the interviewed person : What resources do you need to execute this function ? For each resource : What is the specification of your need ? Do you get this resource ? If yes Then : What are the suppliers of this resource ? For each supplier : What is your satisfaction level (between 0 and 10) ? Realization aspect Among input resources which you receive : What is the input to be transformed ? What are the required resources ? What is the transformation procedure ? What are the transformed output ? For each transformed output : What is the level of the conformance quality ? Customer aspect For each provided output : Who is the customer ?
The diagnosis continues to classify the malfunctions into 15 different categories [7]. For each resource declared as necessary but not receiving or not giving enough satisfaction (satisfaction level is equal or inferior to 6 out of 10), the user has to give information about the improvement suggestions associated to this malfunction, the possible causes of dissatisfaction, the type of abnormality, the observed frequency of abnormality, the severity of abnormality, the detection of abnormality, and the gains expected from a corrective action on this abnormality (if they are known). At the end of the diagnosis, we have a great number of data about the enterprise which allow to display the strong points of the enterprise (with a list of resources and the activities associated with them, providing a high level of satisfaction), and the weak points of the organization leading into dissatisfaction (through a list of resources which is not received by a user who needs it, or which does not provide enough satisfaction).
2) Methods and tools for the first phase The first phase, based on a diagnosis, contains an analysis of the functioning of the enterprise so as to display its strong and weak points. The diagnosis method developed by the laboratory of LLP/CESALP at France, which relies on an organizational modeling by using the Olympios model, is utilized in order to get a representation of enterprise's organization [5] [6] [7]. In the Olympios model, the organization of the enterprise is seen through the « Customer-Supplier » relationships which exist among its employees. Each exchange relationship is established according to an objective that has to be reached even if this objective is not always explicit. This objective is conveyed by the customer into the form of a need expression transmitted to the supplier. The latter is in charge of supplying the output (product or service) requested by employing the required resources to satisfy that need. Besides, in order to assess this performance, it is essential to evaluate the realization of this objective. The model Olympios names these performance indicators as “satisfaction level”. Each exchange relationship is described in the Olympios model by a « Customer Supplier Information System » (CSIS). The CSIS are constructed by interviewing each of the main actors of the organization who intervene either as customer or as supplier. It means that complete CSIS can be constructed totally only after the end of all interviews. Decisions are communicated in CSIS through objectives (as
B. Phase 2 : Pre-Benchmarking The first two steps of this phase aim to target relevant elements that will be subject and partners of a benchmarking study. 1) General description: a) Step 4: To establish priorities and to select what has to be subject of a benchmarking
2
4. Domain : domain of activities of the user, [C(U)]. 5. User Section : nature of the activity of the user, [C(U)]. 6. Supplier : name of the supplier of the resource utilized by the user, [O(T)]. 7. Supplier Section : nature of the activity of the supplier, [C(U)]. 8. Resource : resource in entry of the activity of the user, [O(T)]. 9. Type : type of resource in entry of activity, [C(U)]. 10. Satisfaction : satisfaction level of the user about the input resource, [C(U)]. 11. Improvement issues : malfunction pointed out by the user, [O(T)]. 12. Cause : possible cause of malfunction (if known), [O(T)]. 13. Occurrence : observed frequency of malfunction, [C(S)]. 14. Severity : gravity of malfunction, [C(S)]. 15. Detection : detection frequency of malfunction, [C(S)]. 16. Abnormality : abnormality type of malfunction, [C(U)].
All process, activities, resources, products or services can be the subject of a benchmarking study. To decide what it has to benchmark, it is necessary to identify the critical success factors which should be prioritized. Criteria for choosing among them should include practices that influence customer satisfaction (external, but also internal), and those most urgently in need of improvement [8]. b) Step 5: To choose benchmarking partners The purpose for the enterprise is to become better (not necessarily the best) in a given area, this leads to the choice of the “good” benchmarking partners in area identified in step 4. The principal goal is to learn the practices used by competitors and other best practice organizations to achieve superior results. Identifying the organization you wish to benchmark may not always be obvious. Beside industry competitors, you will also want to look for organizations that are leaders in unrelated industries and that sets the standard in a functional activity.
The answers to this questionnaire are the data issued of diagnosis realized during the phase 1 of the benchmarking process. To treat and to analyze the data, three data analysis methods are proposed [10] [11] : lexical analysis, principal components analysis and common factor analysis. Each of these methods is directed to specific domains of analysis. a) Lexical analysis gives the possibility of analyzing the data corresponding to answers to open end questions. It consists in obtaining from the corpus (vocabulary obtained from all the answers) the lexicon (all of different words). Data interpretation is done by realizing operations on this lexicon. b) Principal Components Analysis (PCA) is a variable reduction factorial method which makes possible a geometric representation of individuals and characters. The specificity of PCA, as compared with other factorial methods, is that it treats only quantitative variables all playing the same role. c) Common Factor Analysis (CFA), another factorial method, aims of examining the dependency of two qualitative characters. With their mathematical properties and the richness of their interpretations, PCA and FCA have become the privileged methods of description of qualitative and quantitative data.
2) Methods and tools for the second phase The second phase of the benchmarking process involves data analysis of one or more enterprises. In order to learn the relations and/or the structures of quantitative or qualitative data, data analysis techniques are employed. Classical statistics focus on the notions of estimation and hypothesis testing problems mostly for univariate case. However, in practice, the observed individuals are frequently characterized by a multiple variables. The multivariate data analysis methods provide a global study of these variables, representing the relations, similarities or differences. The individuals and the variables are placed in geometric spaces and the data are transformed in order to be visualize on a plan or classify in homogeneous groups while loosing minimum information. This descriptive multidimensional approach is used widely in all domains where observation of complex phenomena is necessary : natural sciences, humanities, physics, etc. [9]. Our first task is to design a questionnaire grouping all the data obtained with the realization of the first phase described previously. The questionnaire for this benchmarking study contains three types of questions : closed end questions with unique answer [C(U)] (it is a question for which the respondent will select from a list of choices), closed end questions with scale answer [C(S)] (it is a unique answer question but for which the choices are classified according to a scale of values), and open end questions of text type [O(T)] (it is a question for which the answer is a text). The choices, namely possible answers to questions, are thus qualitative or quantitative. The simple form employed for data collection contains 16 different data fields :
III. BENCHMARKING APPLICATION Several benchmarking application results are obtained with the help of these three methods [12]. Some examples are explained in this part. A. Lexical Analysis Lexical analysis allows us to identify the points significative of the enterprise by their frequency and satisfaction level. In our case, the input resource of CSIS, the improvement suggestions for the abnormalities detected by the diagnosis, or the causes of these abnormalities may be the object of lexical analysis. Table I gives the results of the analysis with an extract from lexicon of the improvement issues of enterprise X, the number of occurrence of each
1. Company : name of the company, [C(U)]. 2. User : name of the person who is interviewed, [O(T)]. 3. Activity : activity of the user, [O(T)].
3
with four performance criteria reflected the occurrence frequency, criticality, detection frequency and dissatisfaction level. These criteria constitute the supplier performance which leads into customer satisfaction. Figure 1 gives a graphical representation of the evaluation of the Supplier performance of domains of activities limited to resources which don't give satisfaction generally and which arise from problems of informational resources type. The type of abnormality is the same for all the domains of activities (stratum utilized is types of resources Informational).
word of this lexicon, and the average of satisfaction levels of CSIS from which these words are extracted. TABLE I RESULTS OF THE LEXICAL ANALYSIS Words Number of Satisfaction observed occurrences levels Formation 25 2.40 Cycle time 19 3.26 Customer 18 3.89 Quality 13 3.00 Forecasting 10 2.80 Plan 7 3.86
With the help of this analysis realized on the key words, we have a quick understanding of weak points significant for the enterprise X. Then, the word formation, which is mentioned 25 times, appears to be a major weak point for this company with a satisfaction level average of 2.40 out of 10. Observation of the answers from which the words are extracted allows us to validate the results obtained by analysis, and to reveal extra information about each of the weak points identified. Lexical analysis provides us relevant results to assist selecting benchmarking topics and partners.
Occurrence Severity Detection Dissatisfaction 4,38
4,60
2,20
10,00
Human Resources Storage Quality management Production
B. Internal Benchmarking
Logistic Control Fabrication
Using the collected data, it is possible to realize an internal benchmarking study which aims to compare certain sections or domains of activities with other sections or domains of activities in the enterprise X. Table II provides us a first interesting start for our study by giving the average satisfaction level for domains of activities.
Purchasing 3,33
1,00
4,60
Fig. 1. Representation of Supplier performance
Then, we can conclude that the domains of activities Human resources, Quality management, Inspection and Production which are the least satisfying domains may collaborate a priority in closed way with the domains of activities Tools, Computer and Research and development which give the highest satisfaction level in order to improve their satisfaction levels. Thus, the results of the tables and the diagram allow us to display the subject and the partners of an internal benchmarking study. The internal benchmarking must be the first step. It is an excellent start for discovering differences between domains of activities and for better defining the essential elements of external benchmarking which can be undertaken later.
TABLE II AVERAGES SATISFACTION LEVEL for each choice of domains of activities for enterprise X
Domains of activities Satisfaction Level Marketing 6.33 Human Resources 5.14 Methods 6.66 Computer 8.53 Logistics 3.75 Storage 6.00 Inspection 3.00 Quality Management 5.71
3,20
Domains of activities Satisfaction Level Fabrication 4.61 Production 6.02 Research Development 7.90 Purchasing 5.16 Sales 7.28 Technical Management 7.23 Tools 9.00 Overall Average 6.23
C. External Benchmarking The dark values in the table indicate the extreme values of the analysis. With internal benchmarking study, our aim is to strengthen the cooperation among the domains of activities which provide different satisfaction levels and to ensure the information exchange about good practices. Thus, improving the satisfaction levels supplied by different domains would lead into an overall performance increase for the whole enterprise. For domains of activities which are least satisfying, we detail our analysis by passing to the table
In order to illustrate an external benchmarking study, we have added two other enterprises, Y and Z with have different sizes and different sector activities, into our analysis. Figure 2, obtained from an CFA on the variable Satisfaction level and the domain of activities Logistics, explains the differences of these three enterprises. This diagram shows the positions of the same domain of activities Logistics of three different enterprises. It is formed upon the
4
stratum of problems informational resource.
population
with
the
type
domain of activities "Logistics". So, enterprises X and Y can collaborate for this domain and they can learn the methods and practices in order to increase their satisfaction level.
of
Axis 2 (37.8%) Logistics-Y
IV. CONCLUSIONS
5 and more
Logistics-X
Axis 1 (62.2%)
The first three steps of benchmarking process may be realized using the diagnosis method relying upon the model Olympios. The following two steps may rely on an interpretation of the results obtained by lexical analysis, common factor analysis and principal components analysis. Among numerous information supplied by our diagnosis, we have chosen and analyzed the most pertinent ones. But it is not enough to have pertinent data, we need moreover good methods of analysis in order to communicate, argue and persuade better. This study allows us to compare different domains of activities of a company (internal benchmarking) and different domains of activities of different companies (external benchmarking) for their Supplier performances. These internal and external comparisons lead into steps four and five and define the subject and partners of a benchmarking study. Graphical displays are presented to strengthen the data analysis. There is still a substantial amount of research that has to be conducted for the formalization of the remaining steps of benchmarking process. These studies will be concretized by our future research and it will be experimented and validated on an industrial platform in the framework of an interregional research project (INTERREG II).
3 0
Logistics-Z
4
Fig. 2. CFA on the variable Satisfaction level for Logistics
The satisfaction level is defined by different choices and on the diagram, the areas of squares are proportional to the frequency of the choice it represents. The domains of activities Logistics-X, Logistics-Y and Logistics-Z are graphically distanced from each other since their average satisfactions are different (3.75, 8.40 and 7.53 respectively). Relying on Figure 2, we deduce that Logistics will be one of the domains of activities to be privileged for external benchmarking study, because they are different on the criterion satisfaction level. The evaluation tables and diagrams with balls allow us, for each domain of activities, to demonstrate and represent the profile of Supplier performance of the three analyzed enterprises. That helps us to take decisions about the choice of domains and partners of benchmarking. Figure 3 shows the diagram with balls for the domains of activities Logistics. 4,33
4,50
2,74
6,75
REFERENCES
less [1] R. Camp, Business Process Benchmarking- Finding and Implementing Best Practices, ASQC Quality Press, 1995. [2] G. Balm, Evaluer et améliorer ses performances - Le Benchmarking, Afnor Gestion Qualité, 1994. performance
[3] Benchmarking for world class leadership, Participant guide, Juran Institute, July 1994. [4] K. Bemowski, "The Benchmarking Bandwagon", Quality Progress, pp. 19-24, January 1991.
3,00
Logistics-X
3,00
1,00
Logistics-Y
more
0,00
[5] J-L. Maire, "Olympios : un modèle de conception du système d’information d’une entreprise manufacturière - Application à l’audit", Ph.D. Dissertation, Savoy University, December 1991.
Logistics-Z
[6] D. Beauchêne, "L’information industrielle: définition et spécification", Ph.D. Dissertation, Savoy University, 1993.
Fig. 3. Positioning of the domains of activities Logistics
[7] D. Beauchêne and J-L. Maire, "Diagnostic de l'organisation d'une entreprise avec OLYMPIOS", Revue Ingénierie des Systèmes d'Information, Vol. 3, No. 4, AFCET, 1995.
To interpret the results of these analyses, two types of profiles in the figures are particularly interesting to be mentioned : for the same domain of activities, the enterprises have profiles perfectly opposed to each other upon the four criteria, and for the same domain of activities, the enterprises have the same profile upon the four criteria. These two situations indicate the possibility of realizing a benchmarking study. In our case, the Figure 3 shows that X and Y have opposite profiles (for three criteria) for the
[8] A. Rao, L.P. Carr, I. Dambolena, R.J. Kapp, J. Martin, F. Rafii and P.F. Schlesinger, Total Quality Management: A Cross Functional Perspective, John Wiley & Sons, 1996. [9] J.F. Hair, R.E. Anderson and R.L. Tatham, Multivariate Data Analysis, second edition, MacMillan Publishing Company, New York, 1987.
5
[10] G. Büyüközkan, "Evaluation and improvement of the performance of enterprises by benchmarking", DEA Rapport, ENSGI/INPG, Grenoble, July 1996. [11] J-L. Maire and G. Büyüközkan, "Formalization of the benchmarking process", The 7th Australasian Conference on Information Systems, University of Tasmania, Australia, 11-13 December 1996. [12] G. Büyüközkan, "Benchmarking application based on internal customer satisfaction", MS. Thesis, Bogaziçi University, Istanbul, January 1997.
6
